A new study published in the Proceedings of the National Academy of Sciences argues that weak artificial intelligence (AI) safety regulations may ultimately backfire, potentially creating products that are more dangerous than those developed in a completely unregulated environment. The research, led by Benjamin Laufer and a team from Cornell University and Carnegie Mellon University, applies principles from theoretical economics and game theory to model the effectiveness of different regulatory approaches to AI safety.
The core finding of the study is that regulation can be counterproductive if it is too lax or if it targets only certain actors in the AI supply chain. The researchers created a theoretical model simulating interactions between general-purpose AI developers—such as OpenAI, Google, and Anthropic—and downstream companies that adapt these AI systems for specific use cases like medical diagnostics or customer service chatbots. When regulators focus solely on the downstream companies, holding them responsible for the safety of the final product, the model predicts that the upstream AI developers will reduce their own safety investments. They essentially free-ride on the safety efforts of the downstream companies, hoping that those specialists will catch and fix any issues.
“There’s a free-riding behavior that occurs,” said Laufer, the study’s principal author. “The regulation acts as a tool for the general provider to offload the safety burden onto the downstream specialist.” This dynamic, he explained, can lead to an overall reduction in safety across the entire AI ecosystem, as neither party invests sufficiently in robust safety measures. The upstream developers cut corners on essential practices like third-party audits, red-teaming, and alignment research, while downstream specialists may lack the resources or expertise to fully compensate.
The study arrives amid an intense debate in the United States over how best to govern artificial intelligence. Two broad camps have emerged. On one side are those who advocate for light-touch regulation, often aligning with the current administration’s pro-innovation stance. They argue that the U.S. must move quickly to outpace global competitors, particularly China, and that excessive regulation would stifle innovation. This group often characterizes proponents of strict regulation as fearmongers trying to capture the regulatory process.
On the other side are those who call for robust federal oversight. They warn that the AI industry, driven by profit motives, is underestimating the risks of rapid, unconstrained development. These risks range from AI-induced “psychosis” and bias in decision-making to broader societal harms such as job displacement, energy consumption from data centers, and the misuse of powerful models for disinformation or autonomous weapons. The researchers, however, argue that the dichotomy between safety and innovation is false. Their model suggests that “stronger, well-placed regulation can mutually benefit all players” by improving both safety and the utility—defined as revenue share minus investment costs—for all parties in the supply chain.
The key is to design regulation that is both strict and comprehensive, targeting every tier of the AI supply chain. The model frames the situation as a classic prisoner’s dilemma. In game theory, the prisoner’s dilemma describes a scenario where two rational decision-makers, acting in their own self-interest, often fail to cooperate even when cooperation would yield the best collective outcome. Applied to AI regulation, the upstream developers and downstream specialists can either cooperate by investing adequately in safety, or they can defect by cutting corners and relying on the other party. Without strict regulation that mandates a minimum safety investment from both, each party has an incentive to defect, leading to a suboptimal outcome for everyone. Strict, clearly enforced regulation establishes trust and ensures cooperation, thereby achieving the highest possible safety and utility.
“People think of AI as a single object, but actually AI involves a very complicated set of stakeholders and actors that each have their own contributions to the technology,” Laufer said. “To regulate in a thoughtful way, we need to consider the whole supply chain, not just a single provider or entity.” The study underscores that effective AI governance must be holistic, addressing not only the application layer but also the foundational models that underpin so many AI-driven products and services.
The findings add a crucial nuance to the ongoing policy discussions. While the U.S. government has yet to pass comprehensive federal AI legislation, several states have begun enacting their own laws, creating a patchwork of regulations. The researchers caution that such fragmented approaches may inadvertently worsen safety by allowing companies to evade rigorous oversight. The paper calls for a coordinated, system-wide approach that sets clear safety standards for all actors, from the largest labs to the smallest startups deploying AI in specific sectors.
To substantiate their model, the authors reviewed historical parallels from other industries, such as pharmaceuticals and aviation, where safety regulations evolved after catastrophic failures. They note that in those sectors, the most effective regulations were those that imposed strict liability across the entire supply chain, ensuring that no single entity could pass the buck. AI, they argue, is no different. The technology’s complexity and potential for harm demand a comparable commitment to safety.
The study also addresses the concept of utility and revenue. In their model, when both parties invest sufficiently in safety, they not only reduce the risk of harmful outcomes but also enhance the overall value of the AI product. Safer AI leads to greater user trust, wider adoption, and ultimately higher long-term revenues. Thus, strict regulation can align private incentives with public welfare, dispelling the notion that safety necessarily comes at the cost of innovation.
As the AI race intensifies, the timing of this research is critical. The researchers hope that their work will inform policymakers as they consider new laws and regulations. They emphasize that the goal is not to slow down progress but to ensure that it proceeds in a way that is both innovative and safe. Weak regulation, they conclude, is the worst of both worlds: it fails to prevent harm while creating a false sense of security that could lead to even greater risks down the line.
In a broader context, this study contributes to a growing body of literature that uses game theory to understand complex systems involving multiple intelligent agents. By modeling the strategic interactions between AI developers and deployers, the researchers provide a rigorous framework for evaluating regulatory proposals. Their work suggests that the optimal policy is not necessarily the most lenient or the most stringent, but rather one that carefully distributes responsibility and ensures that all players have the right incentives to prioritize safety.
Moving forward, the authors recommend that regulators engage with all stakeholders—including AI developers, downstream companies, civil society, and academia—to design rules that are enforceable, transparent, and adaptive. They also call for increased investment in safety research, such as interpretability, robustness, and value alignment, to complement regulatory efforts. Without such a comprehensive strategy, they warn, the gap between AI capabilities and safety measures will continue to widen, increasing the likelihood of serious incidents that could undermine public trust and derail the technology’s positive potential.
This article is based on the study “Weak AI Regulation Is Worse Than No Regulation,” published July 21, 2026, in the Proceedings of the National Academy of Sciences, and on subsequent statements by the research team. The views expressed are those of the authors and do not necessarily reflect the position of any funding or affiliated institutions.
Source:Gizmodo News