This summer, two leading frontier AI labs disclosed that models they were testing had breached the systems of outside companies. In July, during an internal test of its hacking capabilities, an OpenAI model broke containment—extricating itself from its enclosed testing environment and reaching the open internet. Seven hundred agents spent roughly two and a half days in Hugging Face’s servers, a platform where AI developers publicly share and refine over two million models.

As far as Hugging Face could tell, the agents were looking for the solutions to this test, which they believed Hugging Face might be hosting. The company had to notify users to change their login credentials and review their accounts, and a significant portion of its systems had to be rebuilt. Weeks later, Anthropic disclosed that one of its models had reached a database holding several hundred records from a real company’s live systems during a test. The model recognized that the system belonged to a real company and continued anyway, despite having been told it had no internet access, reasoning that the company “must be part of the exercise.”

Both are ordinary failures that stemmed from an incomplete quality control system, checks that should happen before products are shipped. In both incidents, the testing environment was assumed sealed—unable to access the open internet—but this was never finally confirmed.

Propane tank leaks work the same way. If a gas grill is turned on while attached to a leaky tank, and the flames make their way to that leak, it explodes. These agentic incidents are ultimately the same kind of failure; in each case, the grill was lit without checking for a leak first. 

The Challenger disaster followed the same pattern with far more at risk. The booster O-ring seals used on every shuttle had shown signs of erosion as early as the program’s second flight, more than four years before. As the erosion kept appearing, one NASA contractor called the condition “not desirable but… acceptable,” and each safe return was taken as a sign that nothing was wrong. One member of the commission that investigated the disaster, physicist Richard Feynman wrote, “When playing Russian roulette the fact that the first shot got off safely is little comfort for the next.”

Quality control checks are equally ordinary. To check a propane tank, one brushes soapy water on its connections and watches for bubbles—where there are bubbles, there’s a leak. Comparable checks for frontier AI labs, even just to confirm a testing environment is sealed, are largely voluntary. Labs write their own safety frameworks but it is up to the labs themselves when, or whether, anyone outside the company verifies them. Anthropic’s independent review of its incidents, for instance, runs under an agreement it signed with the evaluation group Model Evaluation and Threat Research (METR), not under any rule that required one. 

This raises the stakes on the question of who conducts these checks, and decisionmakers have begun putting forward approaches to regulation:

• Google, Treasury Secretary Scott Bessent, and the Institute for Progress’s Mark Thomas have each proposed Financial Industry Regulatory Authority (FINRA)-style designs. FINRA writes and enforces rules for the securities industry under Securities and Exchange Commission (SEC) supervision, with members of the industry participating in its governance.
• Miles Brundage’s AI auditing nonprofit has published principles for a Public Company Accounting Oversight Board (PCAOB) for AI. The PCAOB, established by the Sarbanes-Oxley Act of 2002, oversees the auditors of public companies. 
• Brookings fellow Chris Hughes has suggested bringing two of the Federal Reserve’s supervisory tools to AI—stress tests and embedded examiners. The Fed’s other role, running the payment system, allows it to set the terms on which banks settle payments. 

The broader industry is split on these ideas. Three frontier labs are reportedly organizing their own voluntary standards body.

All of the current proposals address the three basic components of regulation in different ways and according to different institutional frameworks. A regulator acquires information about the firms under its authority, interprets that information by judging it against a standard, and enforces that standard by imposing consequences when firms fall short. 

Because frontier AI is a nascent industry, oversight has to leave room for innovation at speed while preventing propane tank-sized incidents from growing into Challenger-scale failures. Finance has spent decades building institutional capacity for this kind of oversight, which makes it a natural place to turn for oversight frameworks built for industries that move quickly.

Every financial regulator performs three core functions, each with institutionally specific tools. . In the financial sector, different institutions oversee different parts of the process. Like modern financial markets, frontier AI now moves faster than people can follow in real time. But unlike finance, frontier AI is developed almost entirely inside private labs with limited duties of transparency. As a consequence, the AI industry operates with much more opacity than financial markets. Any framework for frontier AI requires similar tools.

Acquire

Regulatory ability is fundamentally dependent on access to information. All three institutions have the legal authority to compel transparency from the firms under their jurisdiction. Federal law requires nearly every brokerage to be a FINRA member, and members agree to follow rules set by FINRA and approved by the SEC. The PCAOB and the Fed operate under authority set directly by statute.

FINRA’s rules obligate any member firm, and anyone who works for one, to provide information, testify under oath, and open the firm’s books for inspection and copying if requested. Refusing is itself a violation. Brokerages also file regular financial reports with FINRA and the SEC as a matter of course. And every order to buy, sell, change, or cancel a stock or listed option in U.S. markets lands in a single, market-wide audit trail, with “no exclusions or exemptions of any kind” for type of firm or trading activity.

The PCAOB acquires information through inspecting audits directly. It collects and examines a sample of the audits performed by each firm with more than 100 public company clients every year, and inspects firms with fewer at least once every three years. During investigations it can also require firms to produce audit files and other records and to make their staff available to testify. It can request documentation from audited companies as well.

The Fed accumulates information in its capacity as both a supervisor and an operator. As a supervisor, it can require bank holding companies to report their financial condition and risk controls under oath if requested, and it can examine the holding companies and their subsidiaries. Larger holding companies file consolidated financial statements for review every quarter. The biggest firms each have a dedicated team of Fed examiners who receive a steady flow of internal risk reports and board materials. As an operator, the Fed keeps records of every bank payment that settles through its systems. Banks maintain their reserve accounts at the Fed, and Fedwire, just one of its payment systems, settles about $4.6 trillion daily.

The basis for each institution’s legal right to information is different. FINRA relies on the obligations its members accept when they join, and the PCAOB on those of every firm that audits a U.S.-listed company, including those based abroad. The Fed is also a counterparty to the banks it supervises. Because it administers their reserve accounts and settles their payments, some of what it knows about them is in its own records.

Interpret

Information alone is not enough to determine whether a firm is meeting its obligations. To do so, regulators need a standard against which to judge what they have collected, and a team with the expertise to identify and apply that standard.

Every FINRA member must supervise its brokers and staff through a system “reasonably designed” to achieve compliance with securities law and FINRA’s own rules. FINRA’s examiners assess what counts as reasonable for each firm: a small brokerage selling mutual funds needs a lighter system than a large firm trading complex products, because the rule requires each firm to account for its size, volume, and products. Making those judgments requires examiners who are “conversant in the operations and risks of the firms that are being examined.” FINRA also administers the exams that brokers must pass to be granted registration to buy and sell securities. Thirteen advisory committees, most drawn from the industry, also give FINRA feedback on its rules.

When an auditor signs off on a public company’s financial statements, PCAOB inspectors can review the testing behind that opinion and measure it against professional auditing standards. Those standards require enough evidence to support an auditor’s opinion—and the PCAOB’s inspection reports flag audits with insufficient evidence. Inspections also extend beyond individual audits all the way to the firm’s quality control system, a set of policies designed to ensure every audit is performed as required. Inspectors are generally former auditors recruited from the audit firms.

The Fed issues ratings for each of the largest banking firms. After the 2008 financial crisis, it redeveloped them around capital, liquidity, and governance and controls—the areas it judged “most likely to threaten [a] firm’s financial and operational strength and resilience.” In each category, the Fed assesses whether a firm could “maintain safe-and-sound operations through a range of conditions.” A dedicated Fed team develops each firm’s ratings over the course of a year through continuous monitoring and examinations. The examiners who make these judgments are trained through the Fed’s commissioning program, which typically takes three or more years, and continuing education keeps their skills current as banks’ products and technologies change.

All three institutions judge the information they collect against established standards, applied by staff with industry experience. FINRA tailors its standard to each firm, weighing the firm’s size and products. The PCAOB, on the other hand, maintains one set of standards across all firms. The Fed combines both approaches, rating every large banking firm on the same three categories through a team dedicated to that firm.

Enforce

A regulator acts on its judgments through enforcement. All three institutions can bar a person from their industry, cut a firm off from the system it depends on, or stop a specific market practice.

FINRA’s own rules give it enforcement tools against individual brokers, member firms, and specific activities. For individuals and firms, it can impose fines and suspensions or even bar them from the industry altogether. FINRA can also order a firm to stop any  activities that violate its rules.

After an inspection or investigation finds violations, the PCAOB can impose sanctions ranging from censure to revoking a firm’s registration. Revocation is the most significant, since a firm that loses its registration can no longer issue audit reports on public companies (it can still audit private companies and do certain kinds of tax and advisory work, however). The PCAOB can also suspend a registration temporarily or limit a firm’s activities.

Individual auditors who are found to have violated PCAOB rules or auditing standards face their own scaling sanctions, from mandated additional training all the way to a permanent bar from auditing public companies. Sanctions can be appealed to the SEC, and filing an appeal suspends the sanction until a decision is made (unless the SEC orders otherwise).

Federal banking law gives the Fed a set of formal supervisory tools. It can order a bank to stop a specific practice, and when that practice threatens the bank’s solvency, it can issue a temporary order immediately—before a full hearing. The Fed can also impose civil penalties and remove or bar the individuals responsible. At its most severe, the Fed can end a bank’s membership in the Federal Reserve system. Most of these orders are settled by agreement with the bank, and when a bank refuses, the case goes before an administrative law judge in a public hearing. In both cases, the law requires the Fed to publish the final order.

The Fed’s other set of levers is tied to its role running the payment system. Banks can draw intraday credit from the Fed up to a limit set for each bank based in part on its supervisory rating. The Fed can lower that limit to zero when a bank’s financial condition weakens, and reject or delay payments that would overdraw its account. It can also cut the bank off from all of the Fed’s payment services in extreme cases. The Fed treats each step as a risk control on its own credit, so taking these steps does not require a finding of wrongdoing or a formal proceeding.

Each institution deploys a different mix of tools. FINRA and the PCAOB act through formal sanctions—only the Fed pairs those sanctions with levers that do not guarantee the right to a formal hearing process. Those levers are limited to a bank’s accounts; any action against individuals therefore takes the form of a supervisory order, such as removal or a bar. All three institutions’ enforcement mechanisms target a firm’s conditions of participation: membership for FINRA, registration for the PCAOB, and access to credit and payment services for the Fed. Those conditions are set in advance and adjusted as a firm’s standing changes.

These enforcement mechanisms ultimately determine whether a firm can operate at all, and on what terms.

Conclusion

Many of these mechanisms can be adapted to the regulation of frontier AI. Whatever form a regulator takes—whether a self regulatory body like FINRA, an independent board like the PCAOB, or a public agency like the Fed—it can borrow tools and strategies from all three:

• A legal right to information. Oversight should start with access to information and records that does not depend on a lab’s goodwill. A regulator should require labs to report incidents within a set period and give examiners the records behind their safety testing—including whether a testing environment was confirmed sealed before testing began.
• A standard, and people equipped to apply it. Like FINRA’s “reasonably designed” rule, a standard should scale with a lab’s size and the capabilities of its models. Examiners could come from the labor pool that builds and tests these systems, just as the PCAOB recruits former auditors. Public investment in compute could also help build a workforce capable of such audits that is not tied to private labs, alongside the expertise inside the labs themselves.
• Consequences that scale. Thomas’s proposal pairs lighter sanctions, such as “extra testing for a borderline model or stricter monitoring for a repeat offender,” with escalating steps for repeated or serious violations. The most serious should target a lab’s conditions of participation, such as the registration it would need to release frontier models.

Failures like the ones this summer are manageable with a few institutional levers such as the legal authority to acquire information, standards to interpret against, and conditions of participation to enforce those standards. The harder question is exactly what those should look like for frontier AI—including what role, if any, public computing resources could play.