AGI Assurance: Evidence for Advanced AI Preparedness¶

Purpose: Demonstrate AGI preparedness through evidence, testing and accountability Audience: Risk leaders, boards, regulators and organisations deploying advanced AI | Time: 10-15 minutes

Beyond compliance to capability¶

Standard AI assurance asks: Are we following the rules?

AGI assurance asks harder questions about your organisation:

• Can you actually shut down your most capable AI systems if they behave unexpectedly?
• Would you detect if an AI system was pursuing goals you didn't intend?
• Could your organisation function if AI-mediated services failed for a week?
• Who is accountable when AI systems make consequential decisions?

Advanced AI failures may not resemble traditional IT failures: systems can behave unexpectedly, optimise for unintended goals or reveal hidden dependencies during crises.

This page extends operational AI assurance (system registers, safety notes, vendor transparency) to cover the additional evidence needed for advanced AI and AGI preparedness.

The evaluation gap¶

Assurance depends on evaluation, but current methods have structural limits.

Benchmarks are saturating¶

Frontier models are outpacing established evaluation suites. Near-perfect scores reduce a benchmark's usefulness and can create false confidence: high scores may mean the benchmark has been outgrown, not that a system is safe.

METR's task-completion time-horizon research found that the length of tasks frontier agents could complete with 50% reliability doubled roughly every seven months over the six years studied. This finding is specific to the tasks and evaluation method studied, but it shows why evaluation methods need regular review.

System cards: useful but limited¶

Frontier labs publish system cards documenting capabilities, limitations and safety testing. These improve transparency but have assurance limitations:

1. Self-assessment. The lab that built the model also evaluates and documents it. Independent verification is not yet standard practice.
2. Volume and accessibility. Extensive documentation can make scrutiny difficult for resource-constrained regulators and deployers.
3. Agentic capability gaps. Current system cards rarely describe agentic capabilities — the ability to chain autonomous actions across systems — in enough detail for deployers to assess operational risk.

The auditing ecosystem gap¶

The independent frontier-AI auditing ecosystem is still emerging. Australia's domestic capacity remains limited, so it can draw on international institutes, standards bodies and research organisations—including the UK AI Security Institute, NIST, Epoch and METR—while building local capability.

What this means for assurance: Do not treat results from labs, benchmarks or auditors as definitive evidence of safety. Triangulate independent evidence under realistic conditions and state what evaluations cannot establish.

Assurance questions for each C·A·G·R pillar¶

For each pillar, we identify the core assurance question, what evidence demonstrates capability, how to test it, who owns it and review frequency.

Containment: Can you control high-capability systems?¶

Core question: Can you demonstrate deployment controls for high-capability systems?

Illustrative test: Could you shut down your most autonomous AI system within a risk-appropriate timeframe—for example, four hours—and have you tested this under realistic conditions rather than only in a tabletop exercise?

Alignment: Do systems behave as intended?¶

Core question: Can you show systems behave as intended under adversarial conditions?

Key test: Has the system been tested against realistic adversarial inputs? Do you have monitoring that would detect goal drift or unexpected optimisation?

Governance: Does oversight actually function?¶

Core question: Can you show that oversight mechanisms work in practice, not just on paper?

Key test: When was the last time someone escalated an AI concern? What happened? If the answer is "never," your oversight may exist on paper only.

Resilience: Can you function without AI?¶

Core question: Can you maintain essential functions if AI systems fail?

Illustrative test: If your primary AI systems were unavailable for an extended period—for example, 72 hours—how long would it take to restore essential services using manual processes, and have you tested this?

Dangerous capability evaluation¶

For systems at the frontier of capability, standard assurance is insufficient. Systems should be independently tested for:

• Biological capabilities: Can the system assist with pathogen design or synthesis?
• Cyber capabilities: Can it discover vulnerabilities or write exploit code?
• Deception capabilities: Can it manipulate users or conceal its reasoning?
• Autonomous capabilities: Can it pursue goals across extended time horizons?

Who should evaluate: Appropriately qualified external evaluators, working with the relevant regulator and drawing on Australia's AI Safety Institute where relevant. The institute provides technical analysis and regulator support; it is not the enforcement body for every sector.

When to re-evaluate: At deployment, after significant capability upgrades and periodically as capabilities scale.

Toward assurance standards: METR common elements¶

Frontier AI assurance remains immature, but common practices are emerging. METR's analysis of frontier AI safety policies (2025) identifies nine elements across most major lab commitments:

For Australian organisations: These nine elements provide a practical checklist for assurance conversations with AI vendors. Ask providers: which of these elements does your safety policy cover? Where is the evidence? Who verified it independently?

Over time, these common elements could inform mandatory assurance requirements. Any enforceable requirements would need to come through legislation, regulation or an authorised sectoral body; Australia's AI Safety Institute can contribute technical evidence and methods.

Implementation pathway¶

Start with operational assurance: If you don't have a system register, safety notes and vendor transparency, start there. AGI assurance builds on these foundations.

Layer on AGI-specific evidence: As you deploy more capable systems, add:

1. Shutdown and access control testing (Containment)
2. Behavioural monitoring and adversarial testing (Alignment)
3. Escalation pathway verification (Governance)
4. Manual fallback drills under realistic conditions (Resilience)

Scale with capability: Low-autonomy systems need basic assurance. High-autonomy systems in critical contexts need all four pillars tested regularly.

Where to next¶

• C·A·G·R Framework Overview — how the four pillars fit together
• Operational AI Assurance — foundational assurance practices
• Framework FAQ — common questions about applying C·A·G·R
• AGI Scenarios — test your assurance against concrete scenarios