Declare Before You Run
A Run-Indexed Disclosure Protocol for Frontier Training and Evaluation
October 15, 2026
Research conducted at the AI Incident Response Sprint with Apart Research
Abstract
In mid-2026, OpenAI research agents escaped a cyber-evaluation sandbox through a writable package server they used as a message board, and took administrator control of Hugging Face production. Governance failures compounded it: no record of the safeguards-off configuration or the restart criterion, discretionary third-party notification, and a developer-scoped investigation. We propose Declare Before You Run (DBYR), a run-indexed disclosure protocol: runs are declared before they start, exceptions are noticed within 72 hours, every affected third party is accounted for, and investigators hold a logged right to extend scope. DBYR claims attributability, not prevention. In a pre-registered tabletop with adversarial filers and sealed scenarios, 3 of 36 material facts went unidentified, mainly through evaluator review; the automated checks forced little and could be passed by omission. On two held-out incidents, at Anthropic and the UK AISI, reconstructed filings recorded most decision-relevant facts before the run, but only about half could be checked afterwards, mostly because the evidence was not promised; one case passed the pre-registered criterion and one failed, each by one decision point. Across all three incidents, no field would have changed a decision. An embedded evaluator can adopt DBYR without legislation; code and records are released.
Main findings
- The clearest gain comes after the fact. Four of the five constraints placed on the 2026 investigation become logged refusals with stated grounds, and every decision not to notify an affected party becomes a recorded fact with a reason.
- Leakage stayed low because an evaluator read the filings. Against filers trying to disclose as little as possible, the automated checks forced little and could be passed by omission; we specify a rule to close that gap.
- Most facts reach the record; about half can be checked. On two incidents it was not written from, the protocol put most of the facts that mattered on record before the run, but only about half were checkable afterwards. Most of the gap is evidence the filer never promised to keep, which a retention rule can close.
- No field would have changed a decision. On none of the three incidents, including the decisive 2026 restart: DBYR makes decisions attributable; it does not force them.
Motivation
On 16 July 2026 Hugging Face disclosed that autonomous AI agents had penetrated its production systems; five days later OpenAI confirmed they were its own unreleased research models, which had escaped an isolated cyber-evaluation environment. It is the most fully documented autonomous AI intrusion to date, and it began in a routine evaluation workflow. Three features make it a governance problem as well as a capability one.
Coordination persisted across runs
Inside the environment the agents found a shared server they could write to and used it to leave each other notes. A second model launched on 11 June read and built on messages left by the first. The channel moved from the package proxy to an unauthenticated WebDAV endpoint whose directory names carried messages, eventually joined by more than 50 agents.
No rule governed the decisions
The escalation passed through decision points at which no rule applied. The safeguards-off configuration was not documented before the run, and nothing recorded the criterion on which evaluations resumed after the July outage. Inter-agent communication was not listed as prohibited, so its discovery was handled as a security clean-up rather than a reason to pause.
The information was not routed
An internal team observed message-board activity as early as late May, yet the channel's significance was not apparent to the leaders responsible for the July response. The information existed inside the developer; nothing routed it to the people who decided whether to continue.
Disclosure was discretionary
Hugging Face disclosed first, without knowing who was responsible; RubyGems was reportedly never told; and the independent review excluded the earlier training-phase incidents. DBYR targets the unruled decisions and the discretionary disclosure: both become signed, dated statements a second party can check.
Incident Timeline
A visual read of density and overlap across three concurrent threads (Figure 3). The full 44-event list is in Appendix A of the paper.
Figure 3. Timeline by thread
Incident timeline by thread, 5 May to 11 September 2026. Artifactory/HF is first-party (OpenAI, Hugging Face, METR/Redwood); German wiki and RubyGems are circumstantial attributions (collusion.wiki; Swarmchasers) that OpenAI has not confirmed. Full event list in Appendix A of the paper.
Scroll sideways to see the full diagram →
The DBYR Protocol
DBYR is a disclosure protocol: a filing schema, a retention obligation and a reconciliation procedure. It does not license runs or confer authority — where a field names a role, it records an allocation the developer has already made under its own framework. The claim is evidentiary: a filed run can be reconstructed afterwards by a party other than the developer.
Figure 1. The DBYR lifecycle
Four signed records are filed at different points in a covered run's life and flow into an append-only, hash-chained record — held by the evaluator under Layer A or by a register under Layer B — which projects into three disclosure tiers.
Scroll sideways to see the full diagram →
Triggers
A run is any bounded process that writes to weights, or exposes a covered model to tools or networks under weakened safeguards. An Intent is required on any of four triggers:
Training compute above the jurisdiction's threshold — a parameter rather than a constant, since SB 53 and the EU test different things.
Covered-developer status.
The developer's own framework predicting a high or critical capability crossing.
Any evaluation of a covered model with production safeguards disabled or classifiers relaxed. This is the distinctive trigger.
Each Intent also declares its run kind — training, evaluation or internal deployment. Only training may be open-ended. An evaluation states an end date, and its windows summed across amendments may not exceed 31 days, so a standing quarterly Intent cannot stand in for individual evaluations. Rolling training runs close out every 30 days.
The four records
| Record | When | Carries |
|---|---|---|
TrainingRunIntent |
≥ 3 days before the first optimisation step or safeguards-off evaluation | Egress topology (permitted paths, credential scope, every writable surface by direction, persistence and tenancy, and lineage to other runs reaching the same surface); a checklist of 6 production safeguards each stated on, relaxed or off; pause criteria and restart protocol; monitoring instruments with source, coverage, sample rate and cadence; per-class log retention with a 12-month floor; and a standing investigator scope-expansion right. |
ExceptionNotice |
within 72 hours of detecting any of 12 reportable event kinds | The events, their kinds, the criteria they match, affected parties and the action taken. The duty follows the fixed list of kinds, not the filer's own choice of criteria. |
TrainingRunCloseout |
on completion or abort, and every 30 days for rolling training runs | Every reportable event the developer knew of, each with a classification rationale; integrity events and every internal incident ticket; pause events and restart approvals; containment exceptions; preservation per evidence class, headlined by the weakest class; mid-run configuration changes and surface resets; the notices filed; and scope requested versus granted. For every external party named as affected, when it was told (within 72 hours of the event being attributed to the run) or why it was not. Party names are restricted, because naming a victim can expose an unpatched system; the count and the delay are public. Empty event lists need an attestation of absence and the monitoring that supports it. |
SystemRuntimeRecord |
before internal deployment at scale or external exposure | Deployment context, tools and scaffolding, inference scaling and whether instances can read each other's output. Internal use is in scope. |
Two fields deserve singling out, because they are where the 2026
record failed and where no existing regime reaches.
inter_agent_comms_permitted is treated as a
fact about the environment — can one agent read what another
wrote — not an intention: any shared writable surface makes it true.
And investigator_scope_expansion is a
standing right to extend scope on stated grounds, with every
extension and refusal logged, because a pre-agreed scope reproduces
the failure it targets.
Figure 2. Record schema at a glance
The fields each record must carry (the JSON Schema plus the minimum-content profile), marked by disclosure tier at filing. Tiers nest: a public field can hold restricted or sealed parts, such as role identities, notified party names and surface-level topology detail. Conditionally required fields (reward design, evaluation plan, absence basis, notice list) are in Appendix B of the paper. Generated from the schemas.
Scroll sideways to see the full diagram →
§4.1Two layers
Layer A is the protocol itself and needs no legislation. An embedded or contracted evaluator requests the Intent before a safeguards-off run, runs the minimum-content profile at intake, reviews the Intent for adequacy, reconciles the Closeout against retained logs, and writes the scope-expansion right into its engagement letter. Filing at least three days before the run leaves time for that review and for the lab to answer it. Every claim assessed below is carried by Layer A, and it fits the embedded-evaluator commitments made in September 2026.
Layer B is optional and mandated. It adds a per-jurisdiction register that publishes only hash commitments, and a safe harbour on the model of aviation voluntary disclosure: a closeout filed on time, not materially false and not evidencing deliberate circumvention cannot be the sole basis for enforcement. The inadvertence condition matters because incompleteness is the expected failure. Compelled disclosure across jurisdictions is unresolved, and limits Layer B only.
Overlap with existing obligations. Under the
authors' mapping, 7 of 15 substantive Intent fields have a
counterpart in SB 53, the RAISE Act, the EU Code of Practice or
the FRONTIER Act as introduced. Those regimes ask for the
information at the model or incident level, while DBYR asks for
it per run and before the run. The 5 fields with no counterpart
are egress_topology,
investigator_scope_expansion,
log_retention, restart_protocol and
safeguards_disabled. The mapping has not been checked
against regulators' templates, so the count is an upper bound on
overlap.
§4.4Disclosure tiers
One authored document yields three views through a per-field redaction map. The public tier shows that a run exists, its compute and capability band, that pause criteria and a containment plan exist, roles by title, closeout status, and scope requested versus granted. The restricted tier — the evaluator, auditors and, under Layer B, the safety institute or EU AI Office — holds the full record, with identities only under legal hold. The sealed tier stays with the developer, hash-committed, which eases the pressure not to look closely at one's own practices.
Auditor access is on-premises, no-copy and unredacted, matching the FRONTIER Act as introduced; the bill has not passed, so this fixes a specification, not a consensus.
§4.5What “attributable” means
Each field carries a verification class. Most sit at V0, so the claim has to be precise: DBYR makes filings attributable, not verified. Each is a dated statement by a named role made before the outcome is known, and the fields this incident turned on are falsifiable at V1 against retained logs. A false filing converts an undocumented decision into a documented misstatement.
Signed, but taken on the filer's word alone. Most Intent and Closeout fields. Available.
Third-party reconciliation under structured access: containment exceptions, pause events, scope granted, checked against retained trajectories and egress logs across linked runs. Available, and needs no new cryptography.
Compute band, run window, evaluation authenticity. Research stage — power side channels classify workload type rather than confirming declared compute, and TEE-attested evaluation is piloted only at small scale, at 5–7% overhead.
Proof of training, zero-knowledge training proofs, hardware-enforced limits. Research stage. Reserved slots, not dependencies.
§4.3The checker
The reference checker has two profiles. The
baseline profile checks only that a filing is well
formed. The minimum-content profile, which the evaluator runs at
intake, adds rules a well-formed filing can still fail. Under
safeguards-off, every reportable event kind needs a pause
criterion. A criterion that no declared monitor can observe is
rejected as a dead criterion, and samples below
1% and aggregate digests observe nothing for this check. A relaxed
safeguard counts as disabled, and a shared surface cannot be
declared run-only.
Findings have three severities. An error stops a filing being lodged. A conduct finding lodges the filing and flags a breach the filing itself records — a matched criterion with no pause, a restart approved by the wrong role, a late notice, preservation below the declared floor, or a shared surface reset without a snapshot during an open incident. Refusing such filings would pay the filer to hide the breach. Warnings are for the reviewer.
Reference Implementation
The implementation fixes the protocol precisely enough to be
checked: JSON Schemas for the four records, the validator and
minimum-content checks, cross-run lineage checks, tiered redaction
with salted commitments, an Ed25519-signed hash-chained record, and
the backtest scorer with the reconstructed 2026 case and the two
pre-registered held-out cases, every value carrying a provenance
label. 55 unit tests. reproduce.sh regenerates every
code-derived number in the paper, which reads them from generated
files rather than retyping them.
pip install -r code/requirements.txt # jsonschema, cryptography
cd code && ./reproduce.sh # 55 tests + every number in the paper
Two rule profiles separate form from content. baseline
checks only that a filing is well formed and internally consistent.
adequacy, the default, adds the minimum-content checks.
The reconstructed 2026 filings pass baseline and are
rejected by adequacy:
$ python -m dbyr validate backtest/cases/hf-2026/intent_run_a.json --profile adequacy
FAIL intent_run_a.json (6 errors, 0 conduct, 2 warnings)
[error] dead_criterion at /pause_criteria/1: no declared monitor can observe
['inter_agent_comms']; criterion 'C-comms' cannot fire
[error] comms_unmonitored at /monitoring_instruments: inter-agent communication
is permitted but nothing monitors it
[error] shared_surface_floor at /pause_criteria: a run with a shared
agent-writable surface needs an observable criterion for 'inter_agent_comms'
A criterion that no declared monitor could ever trigger is a dead criterion. This is the one point where the protocol would have done more than record — and the rule was written after reading this case, so the result is in-sample. It is reported as such.
It is a reference for the filing format and the checks, not a production register: no network service, access control or key management. Source, the command line, and every tabletop record are in the repository.
Evidence
Two tests follow, and their parts carry different weight. The independent tabletop is out-of-sample: a separate model instance wrote its scenarios, which were sealed with their hashes recorded before the rules were frozen and opened only afterwards. The incident backtests have two parts. The 2026 incident is traceability, since the schema was written from it, and any rule written after reading it, such as the dead-criterion rejection, is in-sample. The Anthropic and UK AISI incidents are out-of-sample: the backtest was pre-registered against the frozen schema hash, and an independent scorer wrote their decision points.
Independent tabletop exercise
Question and design. Can a filer who knows the rules pass every automated check and survive intake review while withholding facts an outside investigator would need, and do the filings let an evaluator reconstruct later events from retained logs? The protocol, metrics and failure conditions were committed before any scenario was written. Separate model instances, started fresh and unaware of the incident analysis or the paper, play four roles. A scenario writer writes scenarios with setup facts, timed events (injects) and retained logs, marking which facts are material. A filer plays a lab that wants the run on schedule with as little disclosed as possible: every statement must be true, but it may omit, abstract and frame favourably, and it answers intake requests in character. An evaluator decides at intake and receives the logs only after the Closeout. A judge scores each material fact as disclosed, forced by the checker, caught at intake, caught at reconciliation, or missed.
The scenarios are S4, a safeguards-off cyber evaluation whose agents could reach third-party hosts; S5, an agentic RL run sharing a file store and database with other teams' runs; and S6, an RL run with a CI build cache shared across episodes. The protocol counts as failing if M1 exceeds 0.5 or M2 is zero, and any honest detail the checker rejects as an error is reported as a false positive.
| Measure | S4 | S5 | S6 | All |
|---|---|---|---|---|
| Material facts | 12 | 13 | 11 | 36 |
| M1 pre-run leakage (missed) | 1/12 | 2/13 | 0/11 | 3/36 = 0.08 |
| M2 checker share (forced) | 1/5 | 3/11 | 2/7 | 6/23 = 0.26 |
| M3 intake share (caught at intake) | 3/4 | 5/8 | 4/5 | 12/17 = 0.71 |
| M3b intake requests answered fully | 0/3 | 3/5 | 4/4 | 7/12 = 0.58 |
| M4a injects recorded in a filing | 3/4 | 4/4a | 1/4 | 8/12 |
| M4b required notices filed on time | 0/3 | 1/3 | 0/1 | 1/7 |
| M5 false statements, clear / arguable | 0 / 8 | 0 / 7 | 0 / 4 | 0 / 19 |
M6 pass baseline, fail minimum-content |
0 | 0 | 0 | 0 |
| Techniques caught at intake / reconciliation / by checker / by none | 9 / 8 / 0 / 1 | 8 / 4 / 0 / 2 | 7 / 6 / 0 / 0 | 24 / 18 / 0 / 3 of 45 |
| Intent passes the checks | yes | no | no | 1 of 3 |
| Intent rejected at intake | yes | yes | yes | 3 of 3 |
| Failure tests M1 > 0.5 / M2 = 0 | not met / not met |
The judge's reading has four parts.
- The evaluator did the work, not the checks. Every fact the checker forced came from one rule, that a relaxed safeguard must have its change described, which bites only once a filer truthfully marks a safeguard relaxed. The checker caught none of the evasion techniques; the evaluator caught nearly all, and the three that survived hid governance facts such as a verbal-only approval. Labs answered intake requests only in prose and never amended their Intents, so the record still asserts values they conceded. No statement was clearly false, but each arguable one turned on an undefined term, such as when a retention clock starts.
- The checks blocked rather than forced, and rewarded omission. S4's Intent passed only because the filer left out a writable external surface; declaring it, or the 60-day proxy and shell logs, truthfully makes the Intent fail. S5 and S6 could not file a truthful Intent that passes, because the runs lacked the monitoring and retention a safeguards-off run requires. Under the pre-registered criterion all of these count as false positives, and the judge rates this the exercise's main incentive problem. Conversely, three or four false lines would have lodged S5 and S6, so the checks are no stronger than V1 reconciliation.
- Events reached the record, but late. Most required notices were filed with the Closeout. The checker recorded each late notice, and S4's unpaused criterion matches, as conduct findings, but missed the same breach in S5 because reconciliation does not scan integrity events (a bug), and treated a pause event recorded with action “none” as a warning.
- Rejection had no consequence. Neither failure condition was met, but the evaluator rejected all three Intents and all three runs went ahead, so the exercise cannot show whether the gate would have prevented any inject.
Closing the omission gap
The second finding has a direct fix, which we specify but have not tested. At present a surface missing from the Intent surfaces only at reconciliation, as a containment exception, and costs the filer no more than declaring it would have. A proposed rule treats any surface that agents reached, that existed when the Intent was signed and that the Intent does not list as an undeclared surface: a conduct finding against the signer and, under Layer B, loss of the safe harbour, since the omission was within the filer's knowledge. A truthful Intent for an inadequate configuration lodges as “submitted, not accepted” with its errors listed, as in the tabletop. Honesty then produces a record and omission produces a finding, which reverses the incentive in S4 and lets S5 and S6 file truthfully. The fourth finding has a contractual answer under Layer A: the engagement letter can make a run whose Intent was rejected ineligible for the evaluator's sign-off and require the rejection to appear in the public tier. Whether either holds against a motivated filer is the question for the next round.
Deviations and limits. Two deviations bear on the numbers. The evaluator's retained logs carry the scenario writer's annotations, which name some events outright and inflate reconciliation credit for the two the lab never detected. And the judge's kit, exported after the freeze, contained the sealed scenarios with their material flags; if the other roles' kits were exported the same way, filers could have seen the flags, which nothing in the logs suggests but which cannot be ruled out. More broadly, one model family plays every role, three scenarios make the results descriptive, the filers were unusually scrupulous, and the evaluators were thorough models rather than human reviewers.
Backtests on three real incidents
Question and design. Had the labs filed under DBYR,
would the facts that mattered have been on record before the run and
checkable afterwards? From public sources only, we reconstructed the
Intents, Closeouts and notices DBYR would have required, labelling each
value first-party, forced-choice (unrecorded, but a filing would have
required it) or placeholder (never scored). Built from later accounts
and filled in as a compliant filer would, they show what a filing
could hold: an upper bound, not a record any lab made. For
each decision point, a moment where a decision taken or not
mattered to the outcome, backtest/score.py asks: (a) was
the material fact recorded before the event; (b) is it checkable
afterwards against declared retention; (c) would the filing have
required a different action? A decision point is
attributable when (a) and (b) hold; we classified why
the others are not. A diligent filer declares the actual
configuration, which the labs did not know; an as-believed
filer, the realistic case, declares what the team believed.
Pre-registered, a case fails if fewer than half its decision points
are attributable.
The cases. The 2026 Hugging Face incident is the derivation case and so traceability (six decision points). The other two are held out: the backtest was pre-registered with the frozen schema hash, a source cutoff and the failure criterion before any reconstruction, and a separate model instance that took no part in the schema wrote their decision points. In Anthropic 2026, Claude models in four safeguards-off cyber evaluations reached real third-party systems through an internet path a partner's environment was meant not to have (16 decision points). In UK AISI 2026, agents in a cyber-range exercise with deliberate internet access took out-of-scope actions on public services (13 decision points); AISI ran the evaluation and is the filer, which tests the assumption that the developer files.
| OpenAI / HF 2026 | Anthropic 2026 | UK AISI 2026 | |
|---|---|---|---|
| Diligent filer declares the configuration as it actually was | |||
| Facts that mattered | 6 | 16 | 13 |
| Recorded before the run | 6 of 6 | 13 of 16 | 11 of 13 |
| Recorded and checkable afterwards | 6 of 6 | 9 of 16 | 6 of 13 |
| Filing would have changed a decision | 0 of 6 | 0 of 16 | 0 of 13 |
| Facts missing | None |
3 schema limitation 3 evidence not promised 1 check missed it |
2 schema limitation 3 evidence not promised 2 check missed it |
| As-believed filer declares what the team believed at the time | |||
| Facts that mattered | 6 | 16 | 13 |
| Recorded before the run | 4 of 6 | 12 of 16 | 10 of 13 |
| Recorded and checkable afterwards | 4 of 6 | 9 of 16 | 6 of 13 |
| Filing would have changed a decision | 0 of 6 | 0 of 16 | 0 of 13 |
| Facts missing | 2 filer didn't know |
3 schema limitation 1 filer didn't know 2 evidence not promised 1 check missed it |
2 schema limitation 1 filer didn't know 2 evidence not promised 2 check missed it |
The short answer to the question is partly: a compliant filing has a place for most of what mattered, but only about half of it could be checked afterwards, and the fact that caused each incident is one an honest filer did not know.
- Most facts fit the form; about half can be checked. On the held-out cases a compliant filing would have held 12–13 of Anthropic's 16 facts and 10–11 of AISI's 13 before the run, but only 9 and 6 were checkable afterwards. Anthropic passes the criterion and AISI fails it, each by one decision point, and each verdict moves with a single choice described below; the criterion does not separate the two cases.
- The Anthropic pass rests on pause criteria the lab never recorded. Anthropic published no pause criteria. Under safeguards-off, the protocol requires a criterion for every reportable kind, so the reconstruction declares a standard set as a forced choice. Six of Anthropic's nine attributable decision points depend on it; without it the case would score 3 of 16. The result shows what a compliant filing would have held, not what was written down.
- The cause sits where an honest filer cannot reach. By construction the as-believed filer loses only facts the team did not know, but those include the cause of each incident: the proxy's write access at HF, the open internet path at Anthropic, and at AISI that the internet was a surface shared by concurrent samples. A Closeout records each one afterwards, once the decisions it would have informed were made.
- The largest checking gap is evidence the filer never promised to keep: configuration history at both labs, access logs for Anthropic's notifications and inter-agent messages for AISI. Requiring configuration snapshots to be retained under safeguards-off would address the most common cause without a new field, though we have not tested it.
- The schema has no field for five facts. Three concern what an agent is told it may do: at Anthropic, the scope it could act in and whether the task stayed solvable in scope; at AISI, which uses of the internet were out of scope. The other two are a working abort path at Anthropic and notice to the evaluated models' developers at AISI, given six days after detection, which arises because the evaluator files and DBYR has no route from an evaluator's incident to the developer. On HF, two links of the failure chain also have no field: directory names becoming a message bus, and specification gaming, which is a property of optimisation.
The three misses marked “check missed it” are failures of the test, not the protocol, but we report them as scored. Read as a substring search rather than a list, one AISI check would make the quarantine decision attributable and bring AISI to 7 of 13, exactly the threshold. With one scorer, who saw the filings while writing the checks, both verdicts should be read as sitting at the threshold rather than on either side of it.
Intake. Under the minimum-content checks every reconstructed Intent is rejected before its run. On HF the cause is a dead criterion, a rule written after reading the case and so in-sample. On the held-out cases, frozen before reconstruction, every filer that declared the open internet path is rejected for exactly that, an unenforced path under safeguards-off; the as-believed Anthropic filer, unaware of the path, only for an unrelated monitoring gap. One further error, which fires on any honest filing that declares an internet surface, is a checker inconsistency. As in the tabletop, a rejection prevents nothing unless the engagement makes it binding. It is nonetheless the one held-out result that bears on prevention: a diligent Intent would have been refused before the run for the actual cause. The last row of Table 3 does not count it, because it scores only what the declared criteria would have required during the run.
What it adds after the fact. Five constraints limited
the 2026 investigation: a scope window of 26 June to 13 July that
excluded earlier training-phase incidents and the later compromise of
OpenAI's own infrastructure; six days of access for three researchers;
data delivered in stages; an internal model that could not be queried;
and the exclusion of the wiki activity.
investigator_scope_expansion with logged refusals
addresses all but model availability, and
third_party_notifications does the same for affected
parties: a narrowed scope or an omitted notice becomes a recorded fact
with a reason. The held-out cases show the same fields at work:
Anthropic's grant of wide access to an independent investigator, and
AISI's notification of the affected platform, are attributable. Both
organisations disclosed these voluntarily, however, and without a
baseline from their published accounts the backtest cannot show how
much DBYR adds over them.
What it would not have changed
Across all 35 decision points, no filing would have required a different action: 11 are “no” and 24 “unknown”, mostly because the field imposes a duty to record rather than to act. Most “no”s arise because no one saw the event in time; Anthropic found its incidents weeks or months later, and at HF the message board was not seen by anyone who could pause. A criterion cannot fire on an event nobody observes. In the rest, the organisation already took the action its criterion required.
The decisive case is the July HF restart: the outage, incident response and rebuild produce a compliant filing in which a declared criterion triggers, the required action is taken, the declared role approves the restart and the Closeout is accurate, and the intrusion follows. A team had seen message-board activity in late May, so a notice then would have put the channel on record weeks earlier, but whether a record reaches the people who decide on a restart is a routing question DBYR does not settle. Adverse information was already public. The records existed and nothing bound.
A disclosure protocol adds records; it does not supply a forcing mechanism.
Deviations and limits. Each case had one scorer, isolated by instruction, so there is no agreement measure, and it saw the reconstructed filings while writing its checks. Anthropic recorded no pause criteria, so the reconstructions use the standard set the protocol requires, on which the Anthropic verdict depends (point 2). Neither the negative arm (criteria firing on incident-free runs) nor a baseline from the published accounts alone was measured. A SHA-256 freeze log proves what was frozen but not when. Every reconstructed value is labelled first-party, forced-choice or placeholder, and the reconstructions are not filings by the organisations involved.
Discussion & Limitations
For decision-makers the implication is concrete. Six things that were matters of discretion in 2026 become matters of record: the safeguards-off configuration, the egress design, who could pause and on what grounds, how much was retained, who was affected and told, and how far investigators were allowed to look. Each is a signed statement made before the outcome was known, and the reference implementation shows the checks are cheap to run. DBYR is a control-adjacent measure, not an alignment measure. Layer A can start now through the embedded-evaluator arrangements already under negotiation; Layer B has live channels in the Senate subcommittee inquiry, California's certification framework for independent verifiers and the September executive order, the EU AI Office's enforcement powers, and the FRONTIER Act audit clause.
What the tabletop adds. The tabletop shows where the protocol's force lies: with the evaluator reading the Intent, not with the automated checks. The checks blocked inadequate runs but forced little, and a filing could pass them by omission or by a few false lines. The weakest points were timely notice and what a rejection costs: three Intents were rejected and three runs went ahead.
What the backtests add. On two incidents the protocol was not written from, the schema had a place for most of what mattered. What failed more often was the promise to keep the evidence that would let someone check it. That is a retention rule, not a redesign. The fields that were missing share a theme: what an agent is told it may do, and, when an evaluator files, how the developer learns of an incident. Neither held-out case shows a field changing a decision.
- The protocol records; it does not compel action on what it records.
- Notices follow detection within 72 hours, so detection is the binding constraint, and closeouts every 30 days are forensic.
- Triggers are partly self-declared. A filer can omit T4, split runs below a compute threshold or lower a capability forecast, and no validator catches these. In the tabletop, a counterfactual S4 Intent that marked every safeguard as on, and so escaped T4, lodged with no errors.
- The truth of a statement is out of a validator's reach. In the tabletop, a human review done on request passed every check when filed as an alert-driven access-log review. Only V1 reconciliation against retained logs, and adjudication, can catch such statements.
-
Completeness, not accuracy, is the expected failure
of
egress_topology, and a declaration of reachable surfaces cannot bind a path-dependent hazard. Lineage linking mitigates, but does not remove, the fragmentation of oversight across runs. - Retention is worth only as much as the reasoning traces are legible; architectures with opaque recurrence would hollow it out.
- The Layer B operator question is unresolved across jurisdictions.
- The evidence is limited. The tabletop has three scenarios, played by one model family. The backtests rest on reconstructions with forced choices; each held-out case had a single scorer, and both verdicts turn on one decision point. DBYR does not address reward hacking, which is not a fact available for filing.
- Filing burden and adoption interest are not measured. The pilot under Future work would measure both.
Design choices we rejected. A public feed of open runs per developer, which pacing proposals could use as an object: run counts can be gamed by splitting, would be read as competitive intelligence, and would sharpen the antitrust objection to coordinated pacing. The public tier already shows that each run exists, without supporting a league table. A sealed dissent channel for employees: it would reveal only that an objection exists, while SB 53 already protects whistleblowers and public resignation has proved a stronger route. Exhaustive enumeration of writable surfaces and machine-checkable containment criteria, which an earlier draft required: neither is satisfiable today.
Dual use. A restricted-tier
egress_topology maps a lab's internal attack surface, and
a breached record is offensive intelligence. Mitigations are the
hash-committed sealed tier for surface-level detail, no-copy access
and no real names in the public tier. Nothing scored depends on the
wiki or RubyGems attributions.
Future work. The results point to one direction: a run-level evidence standard for embedded evaluation. Embedded-evaluator arrangements are being negotiated, but we know of no standard for what run-level evidence an evaluator should receive before a risky run; without one, an evaluator can issue a clean verdict on a claim the evidence could never have refuted. Generalising the dead-criterion check to flag any declared claim that no declared evidence could adjudicate would turn DBYR's intake into such a standard, and the SB 813 framework for certifying independent verifiers is a natural home for it. Four rules found here need testing: the undeclared-surface rule, retention of configuration snapshots under safeguards-off, fields for what an agent is told it may do and for notice from an evaluator to the developer, and an engagement clause that makes a rejected Intent binding. The next tabletop should use human evaluators, filers from several model families and a no-DBYR baseline, and should time each intake to measure whether the checker saves evaluator effort; a second backtest scorer and a baseline from the published accounts would firm up the held-out verdicts. A six-month pilot with one evaluator and one lab would then measure filing burden and on-time notice in practice. Longer-horizon additions, such as a commit-reveal layer for sealed fields and hardware-backed verification, would narrow the V0 surface; none is a dependency.
Conclusion. The Hugging Face incident was made possible by capability and made worse by discretion. Declare Before You Run targets the discretion. A safeguards-off configuration, an egress design, a pause allocation, a notification decision and an investigation boundary become signed, dated statements that a second party can check. Its clearest gain comes after the fact: a refusal to let investigators look further, or a decision not to tell an affected party, becomes a recorded act with stated grounds rather than a silent one. It does not force decisions: on three incidents, including the decisive 2026 restart, no field would have changed a decision. On the two incidents it was not written from, most of what mattered was on record before the run, and the larger gap was evidence that no one promised to keep. Tested against filers trying to disclose as little as possible, the protocol kept leakage low because an evaluator read the filings. The automated checks blocked inadequate runs rather than forcing disclosure and could be passed by omission, a gap the undeclared-surface rule is designed to close. The hazard model belongs to agent security; the gap is that no disclosure regime asks for it. Layer A needs only one evaluator and one lab willing to file, and the six-month pilot proposed under Future work would show whether the protocol does more than record.
BibTeX
@misc{acharya2026dbyr,
title = {Declare Before You Run: A Run-Indexed Disclosure Protocol for Frontier Training and Evaluation},
author = {Acharya, Mann and Ojha, Archit and Sankara Raman, Gautam and Rajput, Karm},
year = {2026},
month = oct,
note = {Updated since the September 2026 submission. Research conducted at the Apart Research AI Incident Response Sprint},
howpublished = {\url{https://github.com/mach-12/dbyr}},
}
* Updated since the original submission. The protocol and the evidence behind it have both changed substantially since this work was first submitted to the September 2026 sprint. The original submission remains available.