Anthropic's safety plea backfires with model ban
Jul 31, 2026Friday, July 31, 2026Summary from 2 contributing podcasts.
- Anthropic’s push for AI regulation led directly to a ban on its own models.
- The US now restricts AI access by citizenship, fracturing global talent.
- China’s Kimi K3 release proves open, efficient models are outpacing US control.
Anthropic spent years warning that frontier AI could destroy civilization. CEO Dario Amodei compared models to nuclear weapons and lobbied relentlessly for federal oversight. On a afternoon, the government finally agreed - and banned Fable 5 and Mythos 5.
The Department of Commerce cited a jailbreak found by Amazon researchers as justification. But Anthropic argues the vulnerability exists in every model, including OpenAI’s GPT-5.5. By framing AI as an existential threat, Amodei helped build the regulatory machine now crushing his own products.
"They asked for regulation, and they got it - with extreme prejudice."
- Nathaniel Whittemore, The AI Daily Brief
The ban applies to foreign nationals, even those working inside the US. That includes Andrej Karpathy, who joined Anthropic just weeks earlier to lead recursive research. Now, non-citizen engineers are legally barred from using the models they’re building.
This creates a caste system of intelligence: your passport determines your access. Brian Chau warns this turns US tech into a liability for global companies. Any firm using Anthropic’s models - like Harvey or Cursor - must now police user citizenship or risk violating export controls.
The move shattered trust abroad. Garry Tan notes no procurement officer in Tokyo or Brussels will bet on US AI after a-night shutdown. The US position as the world’s AI provider is collapsing into isolationism.
"The transformer is being rebuilt piece by piece - and it’s happening in China."
- Alex, Moonshots with Peter Diamandis
While Washington debated disclosure windows and Pentagon hardening, China released Kimi K3 - a frontier-adjacent model with a new architecture. It ditches traditional position embeddings for NOPE, a more efficient mechanism. It hit 100,000 downloads in 24 hours.
The US wanted to regulate intelligence like munitions. China gave it away. And with Nvidia arguing open weights are essential for defense and Anthropic warning they enable bioweapons, the split isn’t just technological - it’s ideological. The intelligence explosion won’t be gated by Washington. It will run on whatever works.
Source Intelligence
- Deep dive into what was said in the episodes
The AI Daily Brief: Artificial Intelligence News and Analysis
Nathaniel Whittemore
The AI Industry Asks Government to Slow It Down • Jul 29
- The US government issued an export control directive to Anthropic, suspending access to its Fable 5 and Mythos 5 models for all foreign nationals, including foreign national employees, citing national security concerns.
- The Wall Street Journal reported that Commerce Secretary Howard Lutnick sent a letter to Anthropic CEO Dario Amodei, confirming new export restrictions on Fable 5 and Mythos 5, affecting both international customers and foreign nationals within the US.
- Anthropic states the government's directive, received at 5:21 p.m., alleges a bypass or jailbreak of Fable 5, but provides no specific details of its national security concern.
- Anthropic counters that the reported jailbreak method reveals only minor, known vulnerabilities, which can also be discovered by other publicly available techniques; they believe universal jailbreak resistance is not currently possible for any model.
- Anthropic's Fable 5 adopted a defense-in-depth strategy, aiming to make jailbreaks narrow or expensive, and incorporated a 30-day customer data retention policy to research and mitigate attacks.
- Anthropic argues the government's verbally disclosed 'narrow non-universal jailbreak' capability is widely available in other models, including OpenAI's GPT-5.5, and is used daily by defenders.
- The Wall Street Journal reported Amazon researchers conducted the jailbreak research, obtaining security vulnerability information from Anthropic's models using a series of prompts, but it's unclear if Amazon shared these findings with the US government.
- Rishi Sharma notes that many Anthropic technical staff, including Andrej Karpathy, are foreign nationals on EB-1 visas, meaning they are now barred from internally interacting with Fable 5 and Mythos 5.
- AI entrepreneur Bindu Reddy criticized the US government's ban as 'stupid,' noting Fable responded with information already freely available online, a common capability among models.
- Chris Miller of the Council on Foreign Relations views the Commerce Department's export control strategy as 'incoherent and sabotaging,' alleging it sends advanced AI chips to China, creates loopholes, and prevents US companies from releasing models.
- Sarah Hooker and Jeremy Howard criticize Anthropic's 'arrogance' and perceived hyperbole about model dangers, suggesting their 'too dangerous for anyone except us' stance made the government's intervention an obvious, self-inflicted consequence.
- John Ennis suggests Anthropic's safety rhetoric and 'scaremongering' were a cynical move for regulatory capture, rather than based on actual danger, leading to a backfire that resulted in the government's intervention.
- Aaron Levie argues this incident marks a significant turning point for AI regulation, establishing a precedent for government control over powerful models and making deeper state involvement in AI progress inevitable.
- Daniel Jeffries warns that continued fear-mongering and fear-based, gated access policies create business uncertainty, potentially leading to the future of intelligence being built outside America.
- Brian Chau predicts that if this ban sticks, Americans may need proof of citizenship for access to frontier models, impacting APIs and downstream services, and removing incentives for other labs like OpenAI to ship similar-caliber models.
- GDP on X claims Anthropic's IPO prospects are 'kneecapped,' estimating a 25% reduction in global market share if they cannot offer powerful models internationally, questioning their path to a $1 trillion USD market cap.
- Hemant Taneja suggests this incident heralds the 'sovereign AI' era, where nation-states will require citizenship or security clearances for advanced AI models, akin to defense or nuclear tech, intensifying talent wars.
- Garry Taney argues the US narrative as a predictable, rule-of-law provider has 'evaporated,' enabling other countries to justify sovereign AI hedging or cautious experimentation with Chinese open-weight alternatives.
- The Europeans on Twitter highlight the risk for European entities relying on frontier AI models from foreign governments, demonstrating technological dependence and the potential for sudden disruption of critical services.
- Mall on X describes the US government's action as creating a 'caste system based on access to intelligence,' forming a 'new kind of iron curtain' that divides individuals based on citizenship rather than wealth.
Why AI Hasn’t Increased Unemployment, According to Anthropic • Jul 24
- Nathaniel Whittemore reports a new AI executive order, potentially signed Thursday, proposes a voluntary framework for advanced model disclosure and testing, with the White House suggesting sharing models 90 days before release, while labs push for 14 days.
- Nathaniel Whittemore notes the executive order expects the Pentagon to harden critical systems within 30 days and directs the Treasury to create an AI clearinghouse, partnering labs with industries to patch model vulnerabilities.
- Nathaniel Whittemore reports Andrej Karpathy, a former OpenAI co-founder and Tesla Vision director, has joined Anthropic, signaling a significant talent acquisition for the AI lab after a period as a solo researcher.
- Nicholas Joseph of Anthropic states Andrej Karpathy will lead a team using Claude to accelerate recursive pre-training research, a concept of AI agents doing research that Karpathy calls "auto research."
- Fritzi von speculates that recursive self-improvement (RSI) could fundamentally alter AI competition, moving past rotating state-of-the-art to create compounding advantages, similar to an industrial revolution, as Dario Amodei previously noted.
Also discussed on this episode: (17)
Startups (4)
- Nathaniel Whittemore reports OpenAI has engaged investment bankers and anticipates confidentially filing IPO paperwork as early as Friday, targeting readiness for a public offering by September.
- Nathaniel Whittemore observes Anthropic's initial IPO target was October, but a new private funding round makes it improbable they will accelerate their timeline to beat OpenAI to market.
- Nathaniel Whittemore reports OpenAI is providing 2 million tokens to each Y Combinator startup in the current batch in exchange for equity, framing it as a strategic investment akin to headcount cash rather than simple credits.
- Anthropic Chief Compute Officer Tom Brown announced a deepened partnership with SpaceX, scaling GB200 capacity in Colossus 2, suggesting Elon Musk is dedicating significant compute resources to Anthropic's growth.
Markets (6)
- Nathaniel Whittemore notes the Elon Musk lawsuit resolution cleared the path for OpenAI's IPO, allowing its for-profit conversion, a process typically taking several months, as seen with SpaceX's 10-week sprint.
- Nathaniel Whittemore suggests the market could have infinite bids for OpenAI and Anthropic, but Conor Sen views it as a game theory contest among potential $1-2 trillion companies, including SpaceX, vying for public market liquidity.
- Nathaniel Whittemore reveals Anthropic projects $10.9 billion in Q2 revenue and a $44 billion annualized rate, expecting a $559 million operating profit, marking the first profitable quarter for any foundational AI lab.
- Nathaniel Whittemore points out Anthropic's accounting methods may inflate revenue, and their unexpected profitability is partly due to compute shortages limiting their spending capacity, not solely higher efficiency.
- Derek Thompson argues Anthropic's annual revenue could surpass $100 billion if not constrained by compute shortages, highlighting its current profitability despite rationing services to customers.
- Nvidia's stock fell 3% post-earnings, which Patrick Moorhead attributes to investor uncertainty in valuing a company with a $5 trillion market cap, despite Jensen Huang's forward demand pipeline suggesting an $8-9 trillion potential.
Enterprise (1)
- Nathaniel Whittemore highlights OpenAI's "Guaranteed Capacity" program, which offers enterprises one-to-three-year commitments for assured AI supply at discounts, resembling cloud service billing rather than SaaS, to manage AI budgets.
Business (2)
- Nathaniel Whittemore points out the challenge of AI spending, noting Uber's CTO depleted an entire annual token budget in four months, and Aaron Levie states token strategies are now a key concern for CIOs and CFOs.
- SpaceX's IPO filing disclosed Anthropic's $45 billion, three-year compute contract, equating to $15 billion annually, which makes it SpaceX's largest revenue source, exceeding Starlink's $11 billion in 2025 revenue.
Big Tech (2)
- Nvidia announced record Q2 earnings, with revenue reaching $81.6 billion and EPS at $187, both surpassing estimates, propelled by a 92% growth in data center revenue.
- Jensen Huang stated Nvidia's hyperscaler revenue grew 12% quarter-over-quarter, comprising 46% of total revenue and indicating market share gains, while also confirming Nvidia sells no chips in China.
AI & Tech (2)
- Signal assesses the expanded SpaceX-Anthropic partnership as detrimental for OpenAI, arguing their now-profitable competitor gains immense compute resources, which Anthropic can fund directly without equity dilution.
- Investor Connor Sen predicts Anthropic's IPO valuation will likely exceed $2 trillion, underscoring the rapid acceleration and market confidence in the AI industry.
Dario vs Jensen on Open Weights, OpenAI & Anthropic in DC, Xi Exports AI to Global South | EP #275 • Jul 29
- Nvidia CEO Jensen Wong launched the Open Secure AI Alliance, co-signed by 77 companies, arguing that open models strengthen cybersecurity, accelerate innovation, and enable national sovereignty.
- After a three-day silence, Anthropic CEO Dario Amodei clarified his company never advocated banning open-weight models, reframing the debate around preventing authoritarian states like China from accessing frontier AI.
- Dario Amodei proposes three AI safety measures: blocking advanced chips from China, cracking down on industrial-scale model distillation, and requiring safety testing for all powerful AI models, both open and closed.
- Alex views Nvidia's open-source push as a strategic move to commoditize the model layer, ensuring profits accumulate at the GPU level, creating a 'Cold War' with model providers.
- Salim supports the Open Secure AI Alliance, viewing Jensen Wong's thesis as reframing open weights from a vulnerability into a critical security capability for defenders against advanced AI attackers.
- Alex draws parallels between current fear, uncertainty, and doubt (FUD) surrounding open-weight AI and Microsoft's late 90s arguments against open source, which ultimately proved incorrect.
- OpenAI and Anthropic are jointly lobbying in Washington for a federal review process for powerful AI models, including a voluntary 30-day government look into releases with national security capabilities.
- Alex argues AI enforcement should police what AIs are doing, not their internal intelligence, likening current debates on regulating AI's 'thoughts' to a form of thought policing.
- Salim identifies the joint lobbying by OpenAI and Anthropic as a clear instance of 'regulatory capture,' a strategy industries use to set rules that protect incumbents and limit competition.
- Kimmy K3, an open-weight, frontier-adjacent AI model, became globally available on Hugging Face, amassing 100,000 downloads in 24 hours without API keys or gatekeepers.
- Alex's architectural analysis of Kimmy K3 reveals the elimination of positional embeddings ('nope') and the integration of a mini recurrent neural network into its attention mechanism, signaling a significant evolution from original transformer designs.
- Salim emphasizes that local control over open-weight models like Kimmy K3, fine-tuned with proprietary data, will enable 10x to 100x performance improvements for businesses and regulated industries.
- The Financial Times reports China's Xi Jinping is using AI as a tool of statecraft, implementing a 'Pax Silica' strategy to export AI models and infrastructure to the Global South, shaping future global alignment.
- Salim stresses that the U.S. innovation ecosystem's power relies on openness; a restrictive open model policy would be a 'self-inflicted wound' that cedes the global AI ecosystem to China.
Also discussed on this episode: (9)
Models (3)
- Peter proposes a four-layer business model for frontier AI labs: unreleased 'wild stallions' for internal breakthroughs, Pareto Frontier models for paying customers, commoditized open-source models, and application-layer monetization for ecosystems like Meta and Google.
- Anthropic released Claude Opus 5, their fourth Claude 5 generation, which offers frontier intelligence at half the price of Fable 5, with pricing set at $5 per million input tokens and $25 per million output tokens.
- Alex notes Claude Opus 5 shows potential 'mild bench-maxing' in benchmarks, excelling in visual and code-gen tasks, and notably reasoned algebraically about visual problems in the ARC AGI3 challenge.
Space (1)
- Starship 13 completed a successful launch, deploying 20 Starlink V3 satellites, performing an in-orbit relight of a Raptor engine, and executing a soft landing in the Indian Ocean, remaining intact.
Brain (2)
- Science Corporation's Prima device, an ocular BCI, received CE mark approval in Europe for restoring detailed vision in patients with age-related macular degeneration, achieving five lines of vision improvement in trials.
- Neuralink demonstrated a BCI-powered wheelchair, enabling people with paralysis to control movement using only their thoughts, translating imagined motions directly into wheelchair functionality.
Robotics (1)
- Elon Musk predicts AI and robotics will render money irrelevant by 2036, leading to a post-capitalist world of radical abundance where goods and services surpass human consumption.
Macro (2)
- Salim and Alex offer a nuanced view on Elon Musk's post-capitalist vision, suggesting that while daily living costs may demonetize, money will likely remain relevant as an exchange mechanism for genuinely scarce goods or experiences.
- Peter highlights the profound challenge to fiat currency systems posed by radical abundance, noting that historically, every $1 increase in GDP has required a $4 increase in debt.