Fletcher faculty weigh in on safety and regulation of artificial intelligence
Discussions on ethics and safety with artificial intelligence (AI) have stirred into high gear since an Anthropic whistleblower warned that there is at least a 10% chance that AI will cause an extinction of the human race within the next decade. Sam Altman, Elon Musk, and Dario Amodei, the CEOs of OpenAI, xAI, and Anthropic, joined together in a call to slow down the technology’s development.
“For quite some time, there have been people raising alarm bells about the harms of AI,” said Alnoor Ebrahim, Thomas Schmidheiny Professor of International Business. “But in the past two years, there was a shift away from any serious discussion of safety towards much more AI boosterism, the potential benefits that it will bring, and the competitive race to be at the top. That all changed this past week.”
Jacob Coxon, the former Anthropic researcher whose social media posts about the threats AI poses to humanity went viral, was not the first one to sound the alarm, Ebrahim noted, but the confluence of protests against data centers in the United States and the alarming power of new models set the stage for industry insiders to speak out, and for bipartisan pushback to the technology’s rapid and unfettered development.
“The Hugging Face event showed that agents can act collectively and deceptively, and that they can cover their tracks,” said Ebrahim. “OpenAI has since revealed at least six more incidents of AI agents getting out of control. And those are just the instances that we know about. It’s possible that it’s happened even more frequently but has not been observed.”
Altman, Amodei, and Musk echoed concerns about safety. However, as Ebrahim previously wrote, OpenAI quietly deleted the term “safely” from its mission statement late last year, around the same time it converted from a nonprofit to a public benefit corporation, raising concerns about its commitment. And xAI’s Grok has been slow to adopt fairly basic guardrails.
“There is momentum now on safety. AI agents have shown they can act collectively. But can humans do so too?” said Ebrahim. “Some U.S. legislators are paying close attention. But there’s a short window of opportunity for legislators and diplomats to step up. This problem cannot be addressed by companies acting alone. We need governments and diplomats to be entrepreneurial and seize the opportunity.”
“Industries can agree on some code of conduct, but it’s hard to get consistent, rigorous, and enforceable societal protections,” said Ebrahim. “Markets operate on competition and the protection of intellectual property, whereas safety with AI would require some degree of opening up the black box of how these companies operate in order to identify what good guardrails and standards look like and to enforce them. They are concerned about putting their intellectual property at risk. That’s where an opportunity for the government to step in can be very productive, in order to put public interests ahead of private interests.”
Successful efforts to regulate require not only the cooperation of governments and companies, but also third-party evaluators who can monitor and enforce the guardrails impartially. The terms of such oversight cannot be set by the companies. The problem is further complicated by the fact that it transcends national borders. Chinese President Xi Jinping will visit President Trump next week, and it is expected that he will be accompanied by executives from Chinese technology companies.
“The timing of Xi Jinping’s visit couldn’t be better,” said Ebrahim. “China is already regulating aspects of AI and is increasingly concerned about losing control. The two most powerful nations in the world could join forces to address this challenge facing society. If Trump and Xi were to say this together, it would be a moment for the history books, and it would not make either of them look weak. It would make them look strong together.”
Fletcher faculty weighed in on additional issues at play.
For interview opportunities with Fletcher faculty, please contact Katie Coleman at katie.coleman@tufts.edu.
Regulatory Models
By Thomas Cao, assistant professor of technology policy
(September 16) Anthropic's Dario Amodei and OpenAI's Sam Altman seem to have agreed upon “pacing the frontier” of AI model release, yet President Trump rejects this idea, insisting that American AI innovation must be unleashed to win the fierce technological rivalry with China. Their positions are not inherently contradictory: AI developments could lead to both scientific progress and serious risks, so oversight is necessary, but the role of the government is a separate question. Evaluating frontier models relies on not only the kind of technological knowledge that formal education provides, but also tacit knowledge gained only by building the systems, and it is the latter that is moving at accelerating rates and has become increasingly crucial. The government can hire the former but would face much higher challenges keeping pace with the latter. “Peer review" of models across AI firms, an idea Elon Musk has proposed, is aimed at addressing the tacit knowledge gap, but is unlikely to be incentive-compatible, as each frontier lab may wish to undermine its rivals.
A more effective approach can involve partnership between the AI firms and academia. Universities such as Stanford and Berkeley are probably the best place outside the frontier labs themselves that possess both technological and some tacit knowledge of model development. Developing innovative institutional arrangements where a cross-university workgroup audits the frontier AI models can avoid the rivalry problem while training the next generation of AI researchers with hands-on experience, talents that AI firms will need in the future. Moreover, because universities lack frontier-scale computing resources, academic research on AI has gradually found its own comparative advantage and focused more on safety and ethics, which makes the arrangement an efficient division of labor. Anthropic and OpenAI have also already begun working with academic researchers on various issues, and the experience accumulated in the process can help facilitate a more systematic framework covering data transfer, privacy, intellectual property, and other concerns.
On the other hand, a role that the federal government should play involves foreign policy. Amodei argues the U.S. will benefit both economically and geopolitically from leading AI research and provision in the free world. The challenge with China is substantially more severe, as Beijing is also determined to win the fight over AI supremacy. But the open-weights models many Chinese firms release, which anyone can download and modify without corporate gating, especially invite misuse by states and non-state actors, from cyberattacks to the production of biological and chemical weapons. These risks conflict with Xi Jinping's own doctrine of "holistic national security," giving the Chinese government incentives to regulate its companies. If the September 24 meeting opens this conversation, a deal that benefits all of humanity is not completely out of the question.
The Role of Semiconductors
By Chris Miller, professor of international history
(September 16) Key to regulating AI is recognizing that AI ultimately depends on physical hardware, above all advanced semiconductors. It is easy to lose sight of this amidst the current fight over AI safety regulations, with President Trump dismissing warnings as a “hoax” while executives and researchers at leading AI firms have increasingly raised concerns. Advanced chips come from a highly concentrated global supply chain, giving the U.S. and allies a tangible source of leverage. This is why chip export controls have become central to U.S.-China tech policy.
The challenge is that Washington’s use of that policy lever has been inconsistent, oscillating between tightening restrictions for national security reasons and loosening them for commercial ones. President Xi’s upcoming visit to the White House will test this, as AI and chips will likely be on the agenda alongside trade and tariffs.
Any signal about export licensing will have far-reaching consequences. U.S. allies like the Netherlands, Japan, and South Korea will be watching to see whether Washington actually wants coordinated controls or would rather cut bilateral deals. Taiwan, a critical link in the semiconductor supply chain, will be watching the summit to determine whether Washington’s longstanding ambiguity over defending the island is weakening. This is not simply a U.S.-China contest, and the broader geopolitical question is whether the U.S. can preserve its technological leadership while maintaining the international coalition that makes this leadership possible.
Architectural Power: Why the AI Contest Is Really About Access
By Monica Duffy Toft, Shelby Cullom Davis Professor of International Security Studies
(September 16) The current debate over AI is being framed too narrowly as a choice between AI safety and American technological leadership. That misses where the contest is actually being decided.
Advanced chips, cloud computing, data centers, energy, model access, technical standards, critical minerals, and the manufacturing networks beneath them form a single system. Capability at any one point becomes influence only when it is converted through the others. In my forthcoming book, Spheres of Power (Princeton), I argue that technological capability does not automatically become geopolitical power. The states and firms that control the system determine who gains access, on what terms, and with what degree of dependence. The book names this architectural power.
This is what makes the coming summit between Presidents Trump and Xi important. The immediate bargaining may concern chips, export controls, rare earths, or regulation and safeguards. The deeper question is whether the U.S. and China are constructing rival technological spheres, each with its own rules of access.
Both countries face a choice about how openly or coercively to manage those spheres. The great American advantage is not that the U.S. controls every chokepoint. It does not. Its advantage is that it sits at the center of a coalition whose members collectively occupy critical positions across the AI system. American firms lead in chip design, cloud computing, and frontier models. Taiwan dominates advanced fabrication. The Netherlands controls essential lithography. Japan and South Korea hold crucial positions in equipment, materials, and memory. These allied capabilities are not merely support for American power. They are part of the power the U.S. can mobilize. It cannot reproduce them alone. Washington therefore makes a strategic mistake when it treats allies chiefly as governments to be pressured into compliance.
If Washington asks allies to bear economic costs for common security while changing the rules to serve immediate domestic commercial interests, its geopolitical, geoeconomic, and geotechnological policies begin to pull apart. Spheres of Power names this domain incoherence. China does not need to surpass the U.S. at every layer. It benefits whenever American unpredictability leads allies and firms to hedge, develop alternatives, or adopt Chinese infrastructure and open-weight models.
The American interest is a technological sphere that allies want to remain inside. Denial may protect a chokepoint. Durable influence comes from setting terms of access that allies consider predictable and legitimate. The summit tests whether Washington understands that distinction.