Six of the world’s most influential artificial-intelligence firms recently signed an agreement with the White House promising to allow independent external audits of their most advanced models. The pact, announced on September 29, includes Nvidia, OpenAI, Anthropic, Google, Meta and xAI, and was described by the U.S. president as a "morally binding" commitment. While the move is welcomed as a step toward greater accountability, the agreement provides no guidance on who will conduct the audits, what aspects of the technology will be examined, or whether audit results will be made public. Moreover, it contains no enforcement mechanisms for non-compliance.
Background of the AI Audit Accord
The accord follows a series of voluntary commitments made by leading AI developers to the U.S. government in 2023. Those earlier pledges called for outside experts to test systems before release, but a subsequent evaluation by researchers at three major universities found that the average compliance rate was just over half of the promised criteria. On the specific promise to assist external parties in identifying and reporting problems, half of the examined companies received a score of zero, indicating a lack of substantive action.
Gaps in Auditor Selection and Transparency
Under the current framework, each company retains the authority to pick its own auditor and to set the scope of the review. This mirrors the pre-Sarbanes-Oxley environment where corporations could select accounting firms that also provided lucrative consulting services, creating an incentive to overlook red flags. The absence of a public registry of auditors or a requirement to disclose audit coverage means that stakeholders cannot assess the independence or thoroughness of the examinations.
Historical Parallel: Enron and the Sarbanes-Oxley Response
The Enron scandal highlighted how auditors paid large fees by a single client could become compromised. After Enron’s collapse, Congress enacted the Sarbanes-Oxley Act, which removed the ability of company management to choose auditors, prohibited auditors from offering most other services to audit clients, and established an independent board to oversee auditors themselves. The act created an “inspector of inspectors” to ensure audit integrity, a mechanism absent from the AI accord.
Emerging Double-Blind Testing Methods
A recent experiment by Google DeepMind, conducted with Singapore’s AI Safety Institute and three other groups, demonstrated a double-blind testing protocol akin to clinical drug trials. In this setup, the AI model is run on sealed hardware, preventing the testing team from accessing the model while the developer cannot see the test inputs. Both parties can mathematically verify that their respective data remained confidential. While earlier research showed that sealed hardware could be breached with modest equipment, the approach reduces the need for auditors to obtain a copy of the model, addressing concerns about trade-secret leakage.
Recommendations for Effective Oversight
To make the audit pledge meaningful, experts suggest that auditors be allowed to define the scope of their investigations, leveraging double-blind methods to avoid exposing proprietary code. Companies should publicly name their auditors and outline the areas examined, even if detailed findings stay confidential. Additionally, an external oversight body—potentially mandated by Congress—could review audit reports and enforce corrective actions, replicating the inspector-of-inspectors model from financial regulation.
Why it matters
The credibility of AI systems increasingly influences public policy, economic activity, and societal trust. Without clear standards for auditor independence, scope, and transparency, the current accord risks becoming a symbolic gesture rather than a robust safeguard. Implementing mechanisms that balance trade-secret protection with rigorous, independent scrutiny will be essential to prevent repeat scenarios akin to historic corporate failures and to ensure that AI development proceeds responsibly.
