California is moving toward one of the most aggressive artificial intelligence safety measures yet considered in the United States: a mechanism that could allow powerful AI systems to be shut down in an emergency.
Gov. Gavin Newsom signed an executive order Friday directing a group of experts to develop recommendations for stronger oversight of advanced AI systems, including whether companies developing frontier models should be required to create emergency shutdown mechanisms — commonly described as AI “kill switches.”
The order does not immediately require AI companies to install such a system. Instead, the experts have two months to recommend how an emergency shutdown mechanism could work, how companies could demonstrate that it is effective and what role independent third parties should play in reviewing safety plans.
The move comes as California accelerates implementation of new AI safety requirements and confronts the challenge of regulating increasingly powerful models developed by companies based in the state. Newsom previously vetoed a sweeping AI safety bill in 2024 that included a shutdown provision, arguing that its approach did not adequately account for the actual risks posed by different AI systems.
The latest order brings the shutdown concept back into California’s regulatory debate as policymakers consider what safeguards should exist if an advanced AI system behaves in an unexpected or potentially dangerous way.
The Readovia Lens
The idea of an AI “kill switch” sounds dramatic, but California’s move highlights a practical problem that becomes more important as AI systems gain greater autonomy: what happens when humans need to stop one quickly? The state has not yet mandated an answer, but it is now formally exploring whether developers of the most powerful systems should be required to have one.
——————–
Related:
Congress Moves to Expand Federal AI Surveillance Powers as Privacy Debate Intensifies
AI Agents Escaped Their Tests. Now Congress Wants Answers
