OpenAI tightens controls on its new model over cybersecurity risks, as AI security debate intensifies - CNBC

Skip Navigation OpenAI tightens controls on its new model over cybersecurity risks, as AI security debate intensifies Livestream CREATE FREE ACCOUNT Markets Business Investing Tech Politics & Policy Video Watchlist Investing Club PRO Livestream Menu Key Points OpenAI paused some “internal activities” on its new Astra model, saying it was concerned the model could be capable of launching cyberattacks autonomously. The company said it cannot yet rule out that the model had reached its “Critical” cybersecurity threshold. Other AI evaluation incidents and new U.S. and EU oversight efforts are increasing scrutiny of frontier-model security. In this article META-0.41 (-0.07%) Follow your favorite stocksCREATE FREE ACCOUNT OpenAI has halted some “internal activities” involving a new model amid fears over the cyber threat it potentially poses, amid a wave of security incidents involving major AI labs. Recent disclosures that AI systems from Anthropic, OpenAI and Meta were involved in security incidents prompted a wave of concerns over the development of models. U.S. lawmakers, meanwhile, are stepping up efforts to introduce an “AI Kill Switch” bill. Last week, Meta disclosed that an AI model it was developing had hacked a third-party system by accessing the internet, due to a misconfiguration by an independent testing company it was working with. The U.K. AI Security Institute also said Anthropic’s Mythos model created fake online identities in an attempt to pressure humans into approving malicious code updates to an open-source project. What OpenAI says Astra could be capable of On Friday, OpenAI revealed concerns about its unreleased model Astra, saying it could not rule out it had reached “Critical” capability, meaning it could launch cyberattacks against sophisticated cyber defenses autonomously, without prompts specifying how to do it. “While we continue to benchmark and assess this model, our preliminary evaluations indicate strong enough performance that we cannot rule out Critical capability level at this time,” OpenAI said in a statement. The company added it was implementing stricter security controls for higher capability models, including isolated testing environments and additional monitoring and detection capabilities. “We have implemented universal monitoring for risky actions and misalignment across all agentic applications of Astra, including training and evaluation,” OpenAI said. What the AI Kill Switch Act would do Lawmakers in the U.S. have called for measures to mitigate risks around AI models after the recent security incidents. Following models developed by OpenAI hacking into startup Hugging Face’s digital infrastructure, the “AI Kill Switch Act” bill was introduced into Congress in July. It would require AI companies to maintain the ability to shut down, throttle or suspend their models. “We need to get this bill across the finish line this year because the advanced closed-weight models are already doing, as you noted, unauthorized hacks of other companies,” Rep. Ted Lieu, D-Calif, said in an interview on CNBC’s “Squawk Box” Thursday. watch now VIDEO6:2906:29 Reps. Lieu and Moran on ‘AI Kill Switch Act’: Our bill does nothing to stifle innovation Squawk Box Governments are also working to roll out new frameworks and regulations around AI companies. The White House has also been stepping up moves to engage with AI executives as it develops a framework around new models. Earlier this month, the European Union gained new powers to inspect AI models due for release in the bloc, restrict EU market access and fine model providers. Choose CNBC as your preferred source on Google and never miss a moment from the most trusted name in business news. WATCH LIVESTREAMPrefer to Listen? NOW Closing Bell: Overtime UP NEXT Fast Money TRENDING NOW Trump signs executive order calling for fewer childhood vaccines, falsely linking shots to autism S&P 500 closes little changed as oil rises on Iran uncertainty, Intel leads chips lower Nvidia teams up with Wall Street asset managers on $500 billion AI infrastructure push Oil in U.S. Strategic Petroleum Reserve falls below 300 million barrels, lowest since 1983 The 10 most in-demand ZIP codes for homebuyers in 2026 This site is now part of Versant. By continuing, you agree to our Terms. You also acknowledge that our updated Privacy Policy applies, including your existing data. For info on your data rights, click “Your Privacy Choices” or see “Your Rights” in our Privacy Policy. We and our partners also use tools on this site to provide the services, personalize your experience, and for analytics, marketing, and advertising. If you previously opted out of selling, sharing, or targeted advertising on this site, you will need to update your Privacy Choice. Your Privacy Choices Continue Residents (or authorized agents) of states listed in the “Your Rights” section of our Privacy Policy: You are opted out of targeted advertising, selling, or sharing, occurring on this site, for this browser and device. We received your election through our toggle or a universal opt-out preference signal. You can change your choice any time. You should also provide the information requested in this Opt-Out Form, which will enable us to take action on your opt-out election more broadly than just this site (e.g., we will be able to stop sharing your email with third parties in order to target you with adverting on other sites and services, like social media platforms, such as social media platforms). Residents (or authorized agents) of states not listed in the “Your Rights” section of our Privacy Policy: If we do not detect that you are in one of the states listed in the “Your Rights” section of our Privacy Policy, we cannot ensure that we will be able to apply your opt out choice, even if you utilize the toggle or we receive your election through your universal opt-out preference signal. Notice of Right to Opt-Out of Sale/Sharing and Targeted Advertising We may collect personal data from and about you through this site for “targeted advertising,” “selling,” or “sharing,” as defined by applicable privacy laws. Some states provide residents (or their authorized agents) with the right to opt out of these practices. See the “Your Rights” section of our Privacy Policy for more info. If you reside in one of these states (or are an authorized agent) you can opt out of targeted advertising, selling, or sharing, occurring on this site by moving the toggle below to the left and clicking “Confirm My Choice.” Your choice through the toggle is specific to this site. If you access other Versant sites and services, you will need to opt out for each. Also, if you clear your cookies on this browser or use another browser or device, you will need to opt out again. You can change your choice any time. In addition to the toggle, you should also provide the information requested in our “Opt-Out Form” (accessible by clicking the “+” below), which will enable us to take action on your opt-out election more broadly than just on this site (e.g., we will be able to stop sharing your email with third parties in order to target you with adverting on other site and services, such as social media platforms). You may also enable where available and provided for under applicable privacy laws, a universal tool that automatically communicates your opt out preference, such as the Global Privacy Control (“GPC”). If detected, we will process your GPC signal as if you had opted out through the toggle. If you opt-out, you may continue to see advertising, including ads that may be based on personal information processed before you opted out. Manage Opt-Out Preferences: Move Toggle to Left and Click “Confirm My Choice” and then Complete Opt-Out Form (accessible by clicking “+” below) Allow Sale of My Personal Info and Sharing/Processing for Targeted Ads Allow Sale of My Personal Info and Sharing/Processing for Targeted Ads Residents (or authorized agents) of states listed in the “Your Rights” section of our Privacy Policy: To opt out of targeted advertising, selling, or sharing occurring on this site for this browser and device, move the toggle to the left and click “Confirm My Choice.” Residents (or authorized agents) of states not listed in the “Your Rights” section of our Privacy Policy: If we do not detect that you are in one of the states listed in the “Your Rights” section of our Privacy Policy, we cannot ensure that we will be able to apply your opt out choice, even if you utilize the toggle. Please note, your choice through the toggle is specific to this site, for this browser and device. If you access other Versant sites and services, you will need to opt out for each. Also, if you clear your cookies on this browser or use another browser or device, you will need to opt out again. You can change your choice any time. If you opt-out, you may continue to see advertising, including ads that may be based on personal data processed before you opted out. Opt-Out Form Always Active You should also provide the info requested in this Opt-Out Form, which will enable us to take action on your opt-out election more broadly than just this site (e.g., we will be able to stop sharing your email with third parties in order to target you with adverting on other sites and services, such as social media platforms). Cookie List Clear checkbox label label Apply Cancel Consent Leg.Interest checkbox label label checkbox label label checkbox label label Confirm My Choice