(../../../../fm.cnbc.com/applications/cnbc.com/resources/img/CNBC_LOGO_FAVICON_1C_KO_RGB.ico)  OpenAI, Anthropic researchers ramp up calls for slowdown amid AI fears      (openai-anthropic-ai-safety-slowdown-extinction.html)                             (https://image.cnbcfm.com)  (https://static-redesign.cnbcfm.com)   (https://www.cnbc.com/amp/2026/09/10/openai-anthropic-ai-safety-slowdown-extinction.html)             (https://static-redesign.cnbcfm.com/dist/main-f384580fbf7b89fb95eb.js)  (https://static-redesign.cnbcfm.com/dist/54711-1b4b7f27485b536e2300.js)  (https://static-redesign.cnbcfm.com/dist/73154-f72e1b21698ca1449bb2.js)  (https://static-redesign.cnbcfm.com/dist/74430-af231b64f95f9e3d41ae.js)  (https://static-redesign.cnbcfm.com/dist/CNBCGlobalNav-1ef3d9f4f23da797ec6f.js)  (https://static-redesign.cnbcfm.com/dist/components-PcmModule-Ads-TopBanner-d40bf01647eaf56d45bf.js)  (https://static-redesign.cnbcfm.com/dist/components-PcmModule-ArticleHeader-439053a438abbbf01636.js)  (https://static-redesign.cnbcfm.com/dist/components-PcmModule-ArticleInlineMedia-ce4dc323bdf07999c782.js)  (https://static-redesign.cnbcfm.com/dist/89244-92cd1abce03aaa61c9fd.js)  (https://static-redesign.cnbcfm.com/dist/73965-491620d1cc04fe80de0b.js)  (https://static-redesign.cnbcfm.com/dist/components-PcmModule-KeyPoints-c55273c3b6aa04692701.js)  (https://static-redesign.cnbcfm.com/dist/71941-18b74b75c24a23391d7b.js)  (https://static-redesign.cnbcfm.com/dist/components-PcmModule-ArticleBody-c1ba49429e517765810a.js)  (https://static-redesign.cnbcfm.com/dist/93002-ae69b162db78f6308c82.js)  (https://static-redesign.cnbcfm.com/dist/components-shared-Article-ReadMore-cc390a4e41d3f8574678.js)  (https://static-redesign.cnbcfm.com/dist/components-PcmModule-ArticleRelatedVideo-2fb52a46c0576d7d9cfa.js)  (https://static-redesign.cnbcfm.com/dist/components-PcmModule-LiveEventModule-8a58dfc853cfadadf1fe.js)  (https://static-redesign.cnbcfm.com/dist/components-PcmModule-WatchLiveRightRail-090a6e7cb78c8cbead15.js)  (https://static-redesign.cnbcfm.com/dist/components-PcmModule-Ads-BoxRail-5b6a2c071c5d8bca4ef9.js)  (https://static-redesign.cnbcfm.com/dist/components-PcmModule-TransporterSection-c042b86ca9cdbf3fdab7.js)  (https://static-redesign.cnbcfm.com/dist/components-Footer-CNBCFooter-b4e61105c5acbfcef431.js)       (openai-anthropic-ai-safety-slowdown-extinction.html#MainContent) Skip Navigation    (https://www.cnbc.com/) (CNBC)       (https://www.cnbc.com/markets/)  Markets  (https://www.cnbc.com/pre-markets/) Pre-Markets  (https://www.cnbc.com/us-markets/) U.S. Markets  (https://www.cnbc.com/currencies/) Currencies  (https://www.cnbc.com/prediction-markets/) Prediction Markets  (https://www.cnbc.com/cryptocurrency/) Cryptocurrency  (https://www.cnbc.com/futures-and-commodities/) Futures & Commodities  (https://www.cnbc.com/bonds/) Bonds  (https://www.cnbc.com/funds-and-etfs/) Funds & ETFs     (https://www.cnbc.com/business/)  Business  (https://www.cnbc.com/economy/) Economy  (https://www.cnbc.com/finance/) Finance  (https://www.cnbc.com/health-and-science/) Health & Science  (https://www.cnbc.com/media/) Media  (https://www.cnbc.com/real-estate/) Real Estate  (https://www.cnbc.com/energy/) Energy  (https://www.cnbc.com/climate/) Climate  (https://www.cnbc.com/transportation/) Transportation  (https://www.cnbc.com/cnbc-investigations/) Investigations  (https://www.cnbc.com/industrials/) Industrials  (https://www.cnbc.com/retail/) Retail  (https://www.cnbc.com/wealth/) Wealth  (https://www.cnbc.com/sports/) Sports  (https://www.cnbc.com/life/) Life  (https://www.cnbc.com/small-business/) Small Business     (https://www.cnbc.com/investing/)  Investing  (https://www.cnbc.com/personal-finance/) Personal Finance  (https://www.cnbc.com/fintech/) Fintech  (https://www.cnbc.com/financial-advisors/) Financial Advisors  (https://www.cnbc.com/options-action/) Options Action  (https://www.cnbc.com/etf-street/) ETF Street  (https://buffett.cnbc.com) Buffett Archive  (https://www.cnbc.com/earnings/) Earnings  (https://www.cnbc.com/trader-talk/) Trader Talk     (https://www.cnbc.com/technology/)  Tech  (https://www.cnbc.com/cybersecurity/) Cybersecurity  (https://www.cnbc.com/ai-artificial-intelligence/) AI  (https://www.cnbc.com/enterprise/) Enterprise  (https://www.cnbc.com/internet/) Internet  (https://www.cnbc.com/media/) Media  (https://www.cnbc.com/mobile/) Mobile  (https://www.cnbc.com/social-media/) Social Media  (https://www.cnbc.com/cnbc-disruptors/) CNBC Disruptor 50  (https://www.cnbc.com/tech-guide/) Tech Guide     (https://www.cnbc.com/politics/)  Politics & Policy  (https://www.cnbc.com/white-house/) White House  (https://www.cnbc.com/policy/) Policy  (https://www.cnbc.com/defense/) Defense  (https://www.cnbc.com/congress/) Congress  (https://www.cnbc.com/expanding-opportunity/) Expanding Opportunity     (https://www.cnbc.com/tv/)  Video  (https://www.cnbc.com/latest-video/) Latest Video  (https://www.cnbc.com/live-tv/full-episodes/) Full Episodes  (https://www.cnbc.com/live-tv/) Livestream  (https://www.cnbc.com/live-audio/) Live Audio  (https://www.cnbc.com/live-tv/schedule/) Live TV Schedule  (https://www.cnbc.com/podcast/) CNBC Podcasts  (https://www.cnbc.com/video-ceo-interviews/) CEO Interviews  (https://www.cnbc.com/documentaries/) CNBC Documentaries  (https://www.cnbc.com/digital-original/) Digital Originals     (https://www.cnbc.com/watchlist/)  Watchlist   (https://www.cnbc.com/investingclub/subscribe?__source=investingclub|globalnav|join&tpcc=investingclub|globalnav|join)  Investing Club  (https://www.cnbc.com/investingclub/charitable-trust/) Trust Portfolio  (https://www.cnbc.com/investingclub/analysis/) Analysis  (https://www.cnbc.com/investingclub/trade-alerts/) Trade Alerts  (https://www.cnbc.com/investingclub/video/) Meeting Videos  (https://www.cnbc.com/investingclub/homestretch/) Homestretch  (https://www.cnbc.com/investingclub/jim-cramer-columns/) Jim's Columns  (https://www.cnbc.com/investingclub/education/) Education  (https://www.cnbc.com/investingclub/subscribe?__source=investingclub|dropdownnav&tpcc=investingclub|dropdownnav) Subscribe     (Join IC)   (https://www.cnbc.com/application/pro?__source=pro|globalnav|join&tpcc=pro|globalnav|join)  PRO  (https://www.cnbc.com/pro/news/) Pro News  (https://www.cnbc.com/josh-brown/) Josh Brown  (https://www.cnbc.com/mike-santoli-insight/) Mike Santoli  (https://www.cnbc.com/analyst-calls-of-the-day/) Calls of the Day  (https://www.cnbc.com/my-portfolio/) My Portfolio  (https://www.cnbc.com/live-tv/) Livestream  (https://www.cnbc.com/live-tv/full-episodes/) Full Episodes  (https://www.cnbc.com/pro-stock-screener/) Stock Screener  (https://www.cnbc.com/2023/05/11/market-strategist-survey-forecast.html) Market Forecast  (https://www.cnbc.com/pro/options-investing/) Options Investing  (https://www.cnbc.com/cnbc-pro-chart-investing/) Chart Investing  (https://www.cnbc.com/application/pro/?__source=pro|dropdownnav&tpcc=pro|dropdownnav) Subscribe     (Join Pro)       (https://www.cnbc.com/live-tv/) Livestream         Menu        (https://www.cnbc.com/make-it/) (makeit) Make It   select    (https://www.cnbc.com/?region=usa) USA  (https://www.cnbc.com/world/) INTL       (https://www.cnbc.com/live-tv/) Livestream        Search quotes, news & videos        (https://www.cnbc.com/live-tv/) Livestream       (https://www.cnbc.com/watchlist/) Watchlist   (Show Notifications)      (openai-anthropic-ai-safety-slowdown-extinction.html#) SIGN IN      (openai-anthropic-ai-safety-slowdown-extinction.html#) Create free account             (https://www.cnbc.com/markets/)  Markets   (https://www.cnbc.com/business/)  Business   (https://www.cnbc.com/investing/)  Investing   (https://www.cnbc.com/technology/)  Tech   (https://www.cnbc.com/politics/)  Politics & Policy   (https://www.cnbc.com/tv/)  Video   (https://www.cnbc.com/watchlist/)  Watchlist   (https://www.cnbc.com/investingclub/subscribe?__source=investingclub|globalnav|join&tpcc=investingclub|globalnav|join)  Investing Club   (Join IC)   (https://www.cnbc.com/application/pro?__source=pro|globalnav|join&tpcc=pro|globalnav|join)  PRO   (Join Pro)       (https://www.cnbc.com/live-tv/) Livestream         Menu             (https://www.cnbc.com/technology/) Tech 'Extinction' warnings ramp up as more OpenAI, Anthropic researchers join calls for an AI slowdown  Published Thu, Sep 10 2026 6:12 AM EDT  Updated 2 Hours Ago   (thumbnail)  (https://www.cnbc.com/kai-nicol-schwarz/) Kai Nicol-Schwarz  (https://linkedin.com/in/kains) @in/kains      (https://www.cnbc.com/live-tv/) WATCH LIVE        Key Points Researchers at OpenAI and Anthropic are publicly urging slower AI development, as debate grows over self-improving AI and its risks. Many concerns revolve around advanced models getting incredibly capable at improving their own performance, a process known as recursive self-improvement. Adding to the worries are numerous cyberattacks and security incidents in recent months by rogue AI models.        In this article  (https://www.cnbc.com/quotes/META) META      (https://www.cnbc.com/quotes/GOOGL) GOOGL        Follow your favorite stocks CREATE FREE ACCOUNT     OpenAI and Anthropic researchers are ramping up calls for an AI slowdown and warning of existential risks to humanity after the (https://www.cnbc.com/2026/09/09/anthropic-researcher-quits-ai-safety.html) resignation of a researcher at Anthropic fueled fresh scrutiny.  The concerns started after Anthropic researcher Jacob Coxon said Tuesday that he was quitting the company as he accused Anthropic and rival OpenAI of "gambling with our lives." He added that those building AI believed that it could "kill us all by the end of the decade." Evan Hubinger, Anthropic's alignment lead, responded that he expects there is a more than 10% chance of that happening.  Since then, several employees at both AI labs have come out in support of calls to slow the pace of AI development as they stressed the risks of the technology. The public warnings are the culmination of growing concern globally about the capability of AI, following numerous cyberattacks and security incidents in recent months by rogue models developed by both OpenAI and Anthropic. "In my personal capacity, I also think we need to slow down," Julie Steele, a member of OpenAI's technical staff who works on the safety team, said late on Wednesday in a post on X in response to Coxon's warnings.   (Anthropic AI researcher says company is 'gambling with our lives')   watch now      VIDEO 1:57 01:57  Anthropic AI researcher says company is 'gambling with our lives' (https://www.cnbc.com/techcheck/) TechCheck        Anthropic researcher Samuel Marks said that "AI developers believe their technology could cause human extinction (or similarly bad outcomes)," in an X post on Wednesday. "This could happen in the next few years. In general, the more senior the employee, the more concerned they are."  Anthropic was the first lab to publish a framework dedicated to mitigating "catastrophic risks from AI models," a spokesperson told CNBC when asked about the comments from employees on social media. "We have always been transparent that AI will bring both enormous benefits and unprecedented risks," an Anthropic spokesperson said, adding that the company was building models with "some of the strongest safeguards in the industry."   OpenAI declined to comment when approached by CNBC, noting recent blog posts on its site.   What is recursive self-improvement? Many of the biggest AI safety fears revolve around advanced models getting increasingly capable at improving their own performance, a technique known as recursive self-improvement, or RSI. "It's hard to overstate how dangerous speeding towards RSI is," (https://x.com/j_asminewang/status/2097840245786157432) said Jasmine Wang, an OpenAI researcher working on alignment, on Wednesday evening.  "There is not yet a viable scientific plan to solve risks from recursively self-improving AI. Please look up!" (https://x.com/a_nnawang/status/2097720574500102615) said Anna Wang, who works on AGI safety and alignment at Anthropic.  OpenAI's chief scientist Jakub Pachocki said Saturday that he has a "strong expectation" that the speed of progress in AI could be sustained into (https://openai.com/index/an-alien-mind/) recursive self-improvement . "If AI development continues along its current path, the systems we'll see in the next few years are likely to represent further capability jumps of equal or larger magnitude, and to increasingly drive their own development," he said in a company blog post. "This is a time that calls for extreme caution," Pachocki added. "I am concerned no one is prepared for the consequences of a continued rapid rise in machine intelligence." Paul Christiano, who was formerly head of safety at the U.S. Commerce Department's Center for AI Standards and Innovation (CAISI), (https://x.com/paulfchristiano/status/2097733214303645729) said recent development of AI capabilities led him to "believe there is a meaningful risk that rapid acceleration in AI capabilities leads to catastrophic and irreversible loss of control in the very near term." OpenAI announced Wednesday that Christiano is joining the board of OpenAI Foundation.   AI safety warnings reach Washington Concerns around the capability of AI models have ramped up in recent months. The announcement of Anthropic's Mythos model, which it touted as having advanced cyber capabilities, in April whipped up a frenzy of panic among financial institutions globally.  In July, OpenAI said its models were (https://www.cnbc.com/2026/07/22/open-ai-cyber-models-hack-hugging-face.html) responsible for a cyber incident on another company, while Anthropic's Claude models were also responsible for (https://www.cnbc.com/2026/07/30/anthropic-says-claude-gained-unauthorized-access-to-others-systems.html) cybersecurity incidents, including in one case where Mythos (https://www.cnbc.com/2026/08/05/anthropic-mythos-openai-security-breaches.html) created fake identities to fool humans. Roughly 1,400 AI researchers, from companies including OpenAI, Anthropic, (https://www.cnbc.com/quotes/META/) Meta      and (https://www.cnbc.com/quotes/GOOGL/) Google      DeepMind, published an open letter in July urging the U.S. government to develop the tools necessary to support an effort to "deliberately pace the frontier of automated AI development." While chiefs of AI labs have increasingly publicly (https://www.cnbc.com/2026/06/17/anthropic-amodei-google-hassabis-us-ai-coalition-g7.html) called for more rules and standards around the development of models, huge competition between companies developing the tech is spurring rapid advances.  Read more  (https://www.cnbc.com/2026/03/01/ai-artificial-intelligence-economy-business-risks.html) ‘Silent failure at scale’: The AI risk that can tip the business world into disorder    (https://www.cnbc.com/2026/09/08/meta-personal-ai-agents-public-reckoning-privacy-safety.html) Meta pushes into personal AI agents as company faces public reckoning over privacy and safety    (https://www.cnbc.com/2026/08/14/data-breaches-surge-2026-ai-cyberattacks.html) Data breach notices have already blown past last year’s total — and AI is playing a growing role      OpenAI and Anthropic are both racing towards public listings. Anthropic is expected to begin marketing its initial public offering in mid-October at the earliest and complete the listing days before the U.S. midterm elections in November, Reuters (https://www.cnbc.com/2026/09/05/anthropic-ipo-launch-shifts-toward-mid-october-reuters.html) reported on Friday, citing people familiar with the matter. Former AI czar of U.S. President Donald Trump, David Sacks appeared to suggest Anthropic's plans for an IPO should be halted. "Surely Anthropic's IPO must be paused until the claims of this "whistleblower" can be investigated," he (https://x.com/DavidSacks/status/2097861539491463331) said in a post on X. Anthropic declined to comment on the post.  Members of Congress have looked to introduce bills to address the rapid advance of AI in recent months, though there's little clear consensus about how the tech should be regulated. One, called the FRONTIER Act, aims to establish a framework for governing the deployment of advanced AI models. Another proposed legislation, dubbed the Ban Artificial Superintelligence Act, would temporarily pause advanced AI development until safety rules were established. "Safety researchers are resigning, powerful AI models are breaking out of their labs, and companies are racing ahead anyway," Rep. Lori Trahan, D-Mass., wrote in (https://x.com/RepLoriTrahan/status/2097641446983528943?s=20) a post on X on Wednesday. "It's past time for Congress to get off the sidelines and do its job."     (https://www.google.com/preferences/source?q=https://www.cnbc.com/) Choose CNBC as your preferred source on Google and never miss a moment from the most trusted name in business news.                      (https://www.cnbc.com/) (CNBC logo)    (https://www.facebook.com/cnbc/)   (https://www.twitter.com/cnbc/)   (https://www.linkedin.com/company/cnbc/)   (https://www.instagram.com/cnbc/)   (https://www.youtube.com/user/cnbc/)   (https://apple.news/T3OtoXcxtRkuHRkM7SpFP_Q)   (https://www.cnbc.com/rss-feeds/)     (https://www.cnbc.com/application/pro/?__source=pro|globalfooter) Subscribe to CNBC PRO  (https://www.cnbc.com/investingclub/subscribe?__source=investingclub|globalfooter) Subscribe to Investing Club  (https://www.cnbc.com/cnbc-reprints/) Licensing & Reprints  (https://www.cnbccouncils.com/) CNBC Councils  (https://www.cnbc.com/select/) Select Personal Finance  (https://cnbcrsh.qualtrics.com/jfe/form/SV_dgRx5X1nYC0YjNI?Origin=cnbc) Join the CNBC Panel  (https://www.cnbc.com/closed-captioning/) Closed Captioning  (https://www.cnbc.com/subscribe/all-access/) Digital Products  (https://www.cnbc.com/cnbc-news-releases/) News Releases  (https://www.cnbc.com/cnbc-internship-program/) Internships  (https://www.cnbc.com/corrections/) Corrections  (https://www.cnbc.com/about/) About CNBC  (https://www.cnbc.com/site-map/) Site Map  (https://www.cnbc.com/podcast/) Podcasts  (https://www.cnbc.com/cnbc-careers-and-employment/) Careers  (https://help.cnbc.com/) Help  (https://help.cnbc.com/contact/) Contact   News Tips Got a confidential news tip? We want to hear from you. (https://www.cnbc.com/news-tips/) Get In Touch   CNBC Newsletters Sign up for free newsletters and get more CNBC delivered to your inbox (https://www.cnbc.com/sign-up-for-cnbc-newsletters/) Sign Up Now Get this delivered to your inbox, and more info about our products and services.  Advertise With Us (https://together.nbcuni.com/advertise/?utm_source=cnbc&utm_medium=referral&utm_campaign=property_ad_pages) Please Contact Us   (https://www.versantprivacy.com/privacy/cookies#accordionheader3) Ad Choices  (https://www.versantprivacy.com/privacy) Privacy Policy  (California Consumer Privacy Act (CCPA) Opt-Out Icon)  Your Privacy Choices  (http://www.versantprivacy.com/privacy/california-consumer-privacy-act) CA Notice  (https://www.cnbc.com/terms/) Terms of Service   © 2026Versant Media, LLC. All Rights Reserved. A Versant Media Company. Data is a real-time snapshot *Data is delayed at least 15 minutes.
      Global Business and Financial News, Stock Quotes, and Market Data
      and Analysis.  (https://www.cnbc.com/market-data-terms-of-service/) Market Data Terms of Use and Disclaimers Data also provided by(https://www.cnbc.com/market-data-terms-of-service/) (Reuters logo)            (../../../../zephr-templates.cnbc.com/integration.css)