The parallels between industrial safety and artificial intelligence governance run deeper than most realize. In 1865, a brewery explosion in Germany killed one person and injured several others—a disaster that could have been prevented through routine inspection. The tragedy spurred more than 20 business owners to establish an association dedicated to steam boiler inspection and insurance, laying the groundwork for what eventually became TÜV, the globally recognized independent safety certification system. From vehicle testing to medical device approval, this model has proven its worth across industries for over a century.

The stakes surrounding general-purpose artificial intelligence dwarf those early safety concerns. Unlike a faulty boiler, which poses localized risk, advanced AI systems threaten systemic disruption and national security implications. Waiting for catastrophe to force action on safety oversight would be reckless. Instead, the TÜV precedent suggests a different path: establishing independent assessment institutions now, before disaster makes the case for them undeniably.

The AI industry has already begun moving in this direction. Major frontier AI companies have published safety frameworks and signed the Frontier AI Safety Commitments unveiled at the 2024 AI Seoul Summit, pledging support for external red-teaming and independent model evaluations. Academic research and expert surveys consistently identify third-party assessments as among the most effective tools for managing AI risks. Regulators have taken notice: the EU's Code of Practice under the AI Act mandates independent assessments in specific scenarios, while the US AI Action Plan emphasizes rigorous evaluation as essential for measuring AI system reliability.

The current assessment landscape is unsustainable

Yet the funding model supporting these assessments remains fragile and problematic. Many assessment organizations operate at modest scale, making them vulnerable to financial dependence on the very AI companies whose systems they are meant to scrutinize objectively. This creates obvious conflicts of interest that threaten assessment credibility.

The risks extend beyond direct funding relationships. Assessors may become overly reliant on recurring contracts from a single firm. Some may hold equity stakes or receive performance bonuses tied to an AI company's financial success—a particular concern when technical staff come from those same companies and cannot liquidate their holdings immediately. Assessors may later move into employment at companies they previously evaluated, mirroring the revolving-door dynamics seen in financial regulation. Each of these dynamics can erode critical scrutiny without requiring outright corruption. Research on financial auditing demonstrates that such incentive structures systematically skew findings in favor of management.

History provides a sobering cautionary tale. During the early 2000s, Arthur Andersen, Enron's auditor, collected millions in consulting fees from the same client it was auditing. When fraud signals emerged, Andersen chose to overlook them rather than jeopardize a lucrative relationship. The result was one of history's largest corporate collapses, destroying thousands of jobs and retirement savings. The lesson is unmistakable: financial entanglement corrodes oversight.

Demand for independent AI assessment is mounting. Governments and civil society increasingly expect external oversight of frontier AI models. As these systems grow more capable and proliferate across markets, the need for credible third-party evaluation intensifies. Philanthropic funding has partially filled the void, but such support remains unpredictable and insufficient to sustain a robust, scalable assessment infrastructure. A durable, independent funding mechanism is essential to preserve the integrity of third-party oversight.

Pool the funding

The solution lies in a pooled funding model. Frontier AI companies should establish a shared AI Assessment Fund, with the Frontier Model Forum positioned to launch this initiative by recruiting founding members and providing initial capital—potentially by expanding its existing AI Safety Fund.

Contributions would flow primarily from leading AI developers and could be calibrated to their investment intensity. One approach: companies contribute 0.05% of total capital expenditures devoted to general-purpose AI development. For top-tier firms spending between $19 billion and $100 billion on such development, this would generate roughly $10 to $50 million annually—comparable to average audit fees paid by S&P 500 companies. Philanthropic organizations could supplement this base, following the collaborative model already established by the AI Safety Fund.

The critical innovation is governance: the fund itself, not individual companies, would select and compensate assessors. This "pooled pot" structure eliminates the direct financial ties between companies and evaluators that have historically undermined oversight elsewhere, as the Enron case illustrates. By breaking these commercial links, the fund preserves the credibility on which effective oversight depends.

Governance and implementation

Effective operation requires several structural safeguards. An independent non-profit organization should administer the fund, actively incorporating perspectives from industry participants and civil society. This governance model addresses potential antitrust concerns that can surface when competitors collaborate on shared infrastructure. The AI Safety Fund's initial management by Meridian Institute provides a workable precedent.

The fund must establish clear criteria for assessor qualification. Baseline requirements should include demonstrated expertise in red-teaming and robust internal security practices. The fund should also codify approved evaluation methodologies, with flexibility to evolve as technology advances. These standards should be developed collaboratively with industry, academia, and non-profit organizations to ensure credibility and balance.

Funding mechanisms alone are insufficient. Profession-wide standards, rigorous governance, and structural checks—including the four-eyes principle—are equally vital. Assessors require meaningful access to models while maintaining intellectual property protections and security for model weights. Encrypted analysis techniques and secure sandbox environments can facilitate this balance. A protected reporting channel for assessors to flag interference attempts, coupled with sanctions for retaliatory conduct, must be established and maintained.

Extended partnerships between assessors and companies can improve efficiency but risk fostering unhealthy dependence. To counter this, assessors assigned to a single company should rotate after a defined period, mirroring requirements for listed companies in the United Kingdom. Similar rotation practices already exist in other sectors, including TÜV organizations serving the automotive industry.

Long-term sustainability: standards, insurance and liability

Building a durable, trustworthy assessment ecosystem ultimately requires three elements: clear standards, insurance mechanisms, and effective liability frameworks.

Standards must articulate how independent assessments should be conducted and what constitutes sufficient evidence. They should specify assessment methodologies, transparency obligations, and safeguards for impartiality and accuracy. A formal accreditation process, overseen by recognized accreditation bodies, should follow, assuring regulators, companies, and the public that accredited assessors maintain high standards of quality and independence.

The insurance sector can serve as a powerful incentive mechanism. If insurers offer substantial premium reductions to companies undergoing credible third-party evaluations, firms would face clear economic pressure to support comprehensive and frequent assessments. However, the systemic and global character of potential AI risks—far more diffuse than localized hazards in automotive or other industries—complicates this approach. Robust liability frameworks become necessary to align incentives properly. Liability laws should clearly assign responsibility for harm caused by advanced AI systems, creating pressure on developers to invest in rigorous independent evaluations.

The technical infrastructure for independent AI assessment is already emerging, but institutional commitment from three sets of actors remains essential: AI companies, industry bodies such as the Frontier Model Forum, governments, and insurers. The window to build this thoughtfully remains open—but only if action begins now, before the first disaster forces a reactive response.

Source: Tech Policy Press