Labs · Early
Measuring the oversight gap.
Public talk about AI swings between alarm and excitement, often in the same week. Neither helps you decide what to do. The Observatory tracks two curves side by side: what AI systems can do on their own, and how much of what they do people can still see, check, and steer.
Why two curves matter
When a new AI model appears, most headlines ask a single question: how smart is it? For a person deciding whether to trust an assistant, or a city deciding what software to rely on, that is only half the picture.
The question that matters for everyday life is whether AI capability is growing faster than our ability to oversee it and share its benefits. To answer that with evidence rather than mood, we track two curves side by side.
The first curve tracks how long a task an AI system can finish on its own. Researchers at METR time how long a skilled person needs for a software or research task, then test whether an AI can complete it successfully half the time.1 Their 2025 study found that the length of task an AI can finish on its own has doubled roughly every seven months since 2019.1
The second curve tracks how much of what AI does people can still inspect, verify, and steer. It asks whether capable systems are tested independently before release, whether they leave clear records when they act, and whether access is spread out rather than held by a handful of firms.2
The oversight gap
The space between these two curves is the oversight gap. A capable system inside clear boundaries, leaving signed receipts and open to outside checks, creates a safe world. An equally capable system acting in the dark does not.
Four stages of agent capability and oversight
As the length of tasks an AI can handle grows from minutes to days, the kind of human oversight that works has to change with it. Watching every line of text on a screen works for a chatbot. It fails when software works all afternoon on its own.
We break this progression into four plain stages. Each stage brings new usefulness, and each requires stronger, simpler checks so people keep the final say.
1. Short tasks: a few minutes
At the first stage, an assistant answers a question, summarizes a meeting, or drafts a short reply while you wait. Up through early 2025, METR found that even leading models reached a 50 percent success rate only on tasks taking a skilled person under two hours, with most everyday uses lasting only minutes.1
What oversight looks like: You can read the whole answer yourself before sending or using it. You need to know where a fact came from, like a postmarked letter rather than an anonymous note, and your assistant should work only for you rather than an advertiser.
2. Multi-hour work: a morning or a workday
At the second stage, coding and research agents read dozens of files, write and test programs, or compare long records for several hours without stopping. A person can no longer watch every step in real time.
Controlled benchmarks show where this autonomy succeeds and where it hits a wall. In multi-hour machine learning engineering tests timed against human experts, the best AI agents scored four times higher than human experts when both had two hours.6 With eight hours, humans narrowly beat the top agent, and with 32 hours they scored twice as high. Current agents struggle with multi-step planning and recovering from unexpected errors once tasks stretch beyond a few hours.6
What oversight looks like: The assistant needs to run inside a strict boundary on hardware you control, touching only the files and tools you explicitly hand it. Every action that reaches the outside world needs a receipt showing who the agent acted for, what it was allowed to do, and what it actually did.
3. Multi-day workflows: days to weeks
At the third stage, autonomous workflows carry a project across days or weeks: auditing records, scheduling repairs, or coordinating supplies across several tools. Once a person clicks "allow" on a permanent permission, it is easy to forget what software still has access to.
What oversight looks like: Standing permission is no longer safe enough. Agents need power on loan, like a library card that expires unless you choose to renew it. Major models also need independent safety testing before release and honest public reporting when something goes wrong.
4. Multi-agent coordination: weeks to months
At the fourth stage, many agents deal directly with other agents at machine speed across organizations and over long horizons. Even when each agent follows its own instructions, traffic between thousands of fast programs can jam, collude, or drift away from what people intended.
What oversight looks like: Cities keep freight trains off the footpaths. At this stage, agents need a separate network of their own, where every message carries identity and scope and agents never impersonate people. Anyone should be able to check a sample of actions on a shared public record that nobody can erase.
How we track signals and score our forecasts
To watch both curves in the open, we organize our indicators into four families. One family feeds the capability line, while the other three feed the line for human oversight and shared power.
- What AI can do: autonomous task duration from METR evaluations, signs of systems helping build newer models, and compute trends tracked by Epoch AI.1,2
- Who checks it: the share of major releases tested by independent outsiders before launch, whether agents run in isolated boundaries, and whether incidents are reported promptly.
- Who holds the power: how few organizations control frontier computing power, how close open models stay to closed ones, and what ordinary hardware can run at home.2
- How society holds up: whether people can take their data with them, tell when they are dealing with a machine, and keep a real voice in public decisions.
It is easy to build a dashboard that only celebrates progress. We give warning signs (rising concentration, releases that skip outside testing, or tools locked to a single provider) at least as much space as good news. A dramatic headline never moves an indicator on its own; values change only when their pre-agreed sources update.
Charts also invite hindsight, while a forecast written in advance does not. Every milestone on our public timeline is a dated prediction with a stated probability. A clear rule decides from a named source whether it came true.
When a date arrives, we score the prediction with the Brier score, a standard measure of forecast accuracy.3 We publish our misses alongside our hits, and check whether events we called at 60 percent happened about 60 percent of the time. Benchmarks like the Forecasting Research Institute's ForecastBench show how public predictions can be tested cleanly against simple baselines.4
What is measured today, and our honest limits
We want to be clear about what is already established by others and what is our own early work. We do not run the capability or compute benchmarks ourselves, and citing external researchers never implies a partnership.
Measured today by others
Capability, compute, and forecasting benchmarks
Independent groups already measure key parts of the capability curve. METR evaluates how long autonomous software and research tasks AI agents can complete at a 50 percent success rate. It calibrates these task suites against human expert working times on open runners.1,6,28 Epoch AI tracks training compute, algorithmic efficiency gains, and composite capability trajectories across dozens of standardized benchmarks. It also models physical memory limits on concurrent agent populations.2,7,30,31
The Forecasting Research Institute runs ForecastBench to test AI and human forecasters on unresolved future questions. Meanwhile, prediction-market benchmarks measure whether models know the limits of their own knowledge.4,19,26 Diagnostic backtesting benchmarks like FinForesight enforce strict point-in-time retrieval controls. This ensures research models forecast future developments without hindsight lookahead data leakage.29
Meanwhile, the AI Incident Database, MIT AI Risk Repository, and Agent Incident Registry catalog real-world failures to classify risks and link recurring vulnerabilities. They track whether identifying responsible developers actually triggers corrective action.5,8,12,20
Our early framework
The oversight curve and stage map
Pairing capability horizons with a matching curve for human oversight and shared power is our own proposal. Our four-stage map outlines what oversight tools are needed at each task horizon. Along with our candidate list of oversight and concentration indicators, it is an early framework we are testing and refining.
Rather than trusting an assistant's self-reported score, our framework checks whether long work leaves verifiable checkpoints. It tests whether agents run inside hardware-enforced boundaries and undergo independent read-only auditing.
Every public milestone we register carries an explicit probability and a pre-agreed resolution source. Brier calibration scoring keeps our misses just as visible as our hits.3
Where it stands
Our public scoring dashboard is still early. We are writing down the exact definitions, sources, update schedules, and known biases for our initial set of indicators before displaying a combined score. Read our introduction to dual-dial tracking in Why speedometers don't steer, or follow our updates on the blog.
We also know how an effort like this can fail. Any measure that matters can be gamed if organizations optimize for the test rather than real safety. And while capability has clear benchmarks, human oversight is much harder to measure from the outside, because much of what happens inside labs and private deployments is not disclosed.
If our forecasts score no better than simple baselines over a fair run, we will redesign or stop. We will do the same if reviewers find these indicators fail to help people make better decisions.
We measure in order to notice where human oversight is slipping while people still have time to act. If you work on AI evaluation, forecasting, or public indicators, we would love your critique.
1. Thomas Kwa, Ben West, Lawrence Chan et al. (METR), Measuring AI Ability to Complete Long Software Tasks (2025). Evaluates frontier models on software tasks timed against skilled human baselines, finding that the 50% task-completion time horizon has doubled roughly every seven months since 2019. arxiv.org/abs/2503.14499 · metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks
2. Epoch AI, Trends in AI and AI Data (continually updated), epoch.ai/trends; Ben Cottier, Tamay Besiroglu and David Owen, Who Is Leading in AI? An Analysis of Industry AI Research (2023), arxiv.org/abs/2312.00043; Anson Ho et al., Algorithmic Progress in Language Models (2024), arxiv.org/abs/2403.05812. Datasets tracking training compute, industry concentration, and algorithmic efficiency gains that halve the compute needed for a given result roughly every eight months.
3. Glenn W. Brier, "Verification of Forecasts Expressed in Terms of Probability," Monthly Weather Review 78, no. 1 (1950): 1–3. Introduces the Brier score for evaluating probabilistic predictions. doi.org/10.1175/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2
4. Ezra Karger, Houtan Bastani, Chen Yueh-Han, Zachary Jacobs, Danny Halawi, Fred Zhang, and Philip E. Tetlock (Forecasting Research Institute), ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilities (2024). arxiv.org/abs/2409.19839
5. Kevin Paeth, Daniel Atherton, Nikiforos Pittaras, Heather Frase, and Sean McGregor, Lessons for Editors of AI Incidents from the AI Incident Database (2024). Examines reporting challenges and taxonomy methods across hundreds of documented real-world AI failures. arxiv.org/abs/2409.16425 · incidentdatabase.ai/research
6. METR, RE-Bench: Evaluating Frontier AI R&D Capabilities of Language Model Agents (2024). Tests frontier agents across 7 machine learning engineering environments against 61 human experts across 71 8-hour sessions, finding agents excel in short 2-hour bursts but trail human scaling over 8 to 32 hours. arxiv.org/abs/2411.15114
7. Hans Gundlach, Alex Fogelson, Jayson Lynch, Ana Trišović, Jonathan S. Rosenfeld, Anmol Sandhu, and Neil Thompson, On the Origin of Algorithmic Progress in AI (2025). Shows that algorithmic efficiency gains are scale-dependent, with architectural shifts accounting for the majority of frontier efficiency leaps. arxiv.org/abs/2511.21622
8. Peter Slattery, Alexander K. Saeri, Stephen Casper, Neil Thompson et al. (MIT FutureTech), The AI Risk Repository: A Comprehensive Meta-Review, Database, and Taxonomy of Risks From Artificial Intelligence (2024). Unifies 74 risk frameworks across 1,725 cataloged risks into standardized causal and domain taxonomies. arxiv.org/abs/2408.12622 · airisk.mit.edu/risks
9. Ben Johnson and Hayden Helm, Task-Efficient Agent Evaluation Using Cached Traces (2026). Shows that structured execution trace geometries predict agent benchmark performance on SWE-bench Verified and Terminal-Bench 2.0 with far fewer full task executions. arxiv.org/abs/2611.06803
10. Avinash Agarwal and Manisha J. Nene, Standardised schema and taxonomy for AI incident databases in critical digital infrastructure (2025). Proposes unified classification schemas and taxonomies for AI failures in critical infrastructure, standardizing severity metrics, root-cause attribution, and propagation pathways. arxiv.org/abs/2501.17037
11. Lukas Bieringer, Kathrin Grosse, et al., Practice-Informed, Practice-Ready: An AI Security Incident Taxonomy (2024). Develops a foundational taxonomy with cross-sector practitioners, demonstrating that automated LLM incident classification remains unreliable compared to structured human schemas. arxiv.org/abs/2412.14855
12. Isabel Richards, Claire Benn, and Miri Zilka, From Incidents to Insights: Patterns of Responsibility following AI Harms (2025). Analyzes 962 incidents and 4,743 reports in the AI Incident Database, showing that identifiable developers do not guarantee accountability without public records and structured response tracking. arxiv.org/abs/2505.04291 · incidentdatabase.ai/research
13. Diego Russo, Gian Marco Orlando, Valerio La Gatta, and Vincenzo Moscato, Automating AI Failure Tracking: Semantic Association of Reports in AI Incident Database (2025). Formalizes failure report association as a semantic ranking task, showing that transformer sentence embeddings and semantic similarity connect incoming reports to cataloged AI incidents across critical sectors without manual triage delays. arxiv.org/abs/2507.23669
14. Mingguang Chen, Licheng Wang, and Bo Qu, The Horizon Gap: Planning, Memory, Execution, Training, and Evaluation for Long-Horizon LLM Agents (2026). Surveys over 1,500 studies to formalize why outcome-only evaluation fails over multi-hour tasks, demonstrating that long-horizon reliability requires dense trajectory diagnostics and structured execution state checks. arxiv.org/abs/2608.06663
15. Drew T. Nguyen and William Fithian (Department of Statistics, UC Berkeley), On the estimation and validity of AI time horizons—a statistical look at the METR plot (2026). Recomputes autonomous time horizons across 228 tasks and 26 AI models using splines and item-response theory, demonstrating that AI task difficulty is nearly flat from 2 to 30 minutes but steepens significantly beyond 30 minutes. arxiv.org/abs/2610.12466
16. Yuxuan Liu, Zhaochen Su, Hangyu Guo et al., SkillHorizon: Constructing Long-Horizon Agent Tasks through Skill Composition (2026). Evaluates five frontier language models across 1,050 tasks in seven business environments, finding that mean local-goal completion falls from 90.4% to 46.8% as workloads scale up to 1M tokens, with agents frequently declaring premature completion while substantial work remains unfinished. arxiv.org/abs/2610.14966
17. Rishi Desai, Jesse Hu, Joan Cabezas, Neel Harsola et al., SWE-Marathon: Can Agents Autonomously Complete Ultra-Long-Horizon Software Work? (2026). Evaluates frontier coding agents across 20 project-scale tasks averaging 27.2 million tokens per rollout, finding agents solve under 30% of projects due to flawed self-verification and premature termination. arxiv.org/abs/2606.07682
18. Harleen Kaur Sidhu, Rebecca Scholefield, Nour Annan, Kevin Hernandez, Isabel Nieh Hou, Abdulrahman Alshaikhi, Ze Shen Chin, and Rokas Gipiškis, Open Problems in AI Incident Governance (2026). Audits monitoring and incident disclosure policies across 11 frontier AI developers, proposing standardized reporting templates and shared near-miss definitions for public incident databases. arxiv.org/abs/2607.05163
19. Jaeho Lee, Nick Merrill, and Ezra Karger (Forecasting Research Institute), ForecastBench-Sim: A Simulated-World Forecasting Benchmark (2026). Evaluates probabilistic reasoning and dynamic world forecasting across simulated environments to test arbitrary horizons, counterfactuals, and tail-risk calibration under strictly proper scoring rules. arxiv.org/abs/2606.18686
20. Divyanshu Kumar, Rohith HN, Nitin Aravind Birur, Sahil Agarwal, and Prashanth Harshangi, The Agent Incident Registry: Toward Preventing Repeated AI Agent Failures (2026). Establishes a source-grounded catalog of verified AI agent failures with missingness-aware labels for causal roles, mechanisms, and outcomes, distinguishing realized harms from capability demonstrations. arxiv.org/abs/2609.11030
21. Canwen Wang, Qing Xiao, Haiyi Zhu, and Hong Shen, When AI Agents Act on Our Behalf: A Taxonomy of Human-Agent Interaction Failure (2026). Analyzes 695 real-world AI incidents from public repositories to develop a 12-mode failure taxonomy across three interaction stages, demonstrating how failure severity escalates with autonomy and establishing the necessity of intermediate runtime controls. arxiv.org/abs/2611.00291 · arxiv.org/html/2611.00291
22. Ziyu Ma, Hailang Huang, Shun Zou, Yong Wang, Shidong Yang, Yiming Hu, Fei Wei, and Xiangxiang Chu, LongHorizon-Harness: Advancing Long-Horizon Agents for Real-World Tasks (2026). Demonstrates that maintaining task execution, state tracking, and completion assessment within a single expanding context allows error cascades to compound, showing that an external Manage-Execute-Audit (MEA) loop with independent read-only auditing substantially boosts benchmark completion rates. arxiv.org/abs/2608.01964
23. Salman Rahman, Yubin Kim, Mihir Parmar et al. (UCLA, NYU, Google Research, and Google DeepMind), Locating Hidden Failures Makes Long-Horizon Agents More Reliable (2026). Evaluates 2,518 agent trajectories and 6,967 errors across 78 failure types in the Traverse benchmark, demonstrating that outcome-only evaluation conceals intermediate damages and that specialized test-time verifiers pinpoint hidden failure points missed by standard frontier judges in over two-thirds of runs. arxiv.org/abs/2609.17930
24. Xinyu Jessica Wang, Haoyue Bai, Yiyou Sun et al., The Long-Horizon Task Mirage? Diagnosing Where and Why Agentic Systems Break (2026). Introduces the HORIZON benchmark across 3,100+ trajectories in web, database, operating systems, and embodied environments, revealing that long-horizon agent reliability collapses sharply past a threshold of chained subgoals due to compounding assumption errors and history drift rather than scaling smoothly. arxiv.org/abs/2604.11978
25. METR, Independent Assessment of AI R&D Acceleration at Anthropic in October 2026 (November 2026). Evaluates whether frontier language models accelerate the training and development of subsequent models, finding model capabilities on well-specified tasks improved at roughly twice the historical rate while demonstrating the operational viability of high-access independent nonprofit audits under contractual disclosure rights. metr.org/blog/2026-11-05-anthropic-ai-rd-acceleration-assessment · metr.org/blog/2026-11-05-rd-section-anthropic-risk-report-august-2026-review
26. Lukas Nel (Lotus AI), Do Large Language Models Know What They Don't Know? KalshiBench: A New Benchmark for Evaluating Epistemic Calibration via Prediction Markets (2025). Evaluates frontier language models across 300 prediction market questions resolved after training cutoffs, finding systematic overconfidence, negative Brier skill scores against historical baselines, and higher calibration errors in reasoning models. arxiv.org/abs/2512.16030
27. E. Kelly Buchanan, Crystal Zhou, Scott W. Linderman, and Christopher Ré (Stanford University), Benchmarks as Software: Terminal-Bench and the Case for Continuous Benchmarks (2026). Identifies benchmark maintenance debt as static test verifiers fail to accommodate valid alternative agent solutions or drift across execution environments, demonstrating that reliable capability evaluation requires continuous trace validation, automated environment oracles, and human-gated test maintenance. arxiv.org/abs/2611.08417
28. METR, Task-completion time horizons of frontier AI models (Time Horizon 1.1, 2026). Expands human-calibrated software task evaluations to 228 benchmarks across the Human-Calibrated Autonomy Software Tasks (HCAST) suite, standardizing autonomous time-horizon measurement on the open-source Inspect evaluation framework. metr.org/time-horizons
29. Fengyuan Liu, Mukai Li, Jieshun You, Yilin Zheng, Chenjia Min, Yizhou Zhang, Harper Chen, and Qi Liu, FinForesight: A Diagnostic Backtesting Benchmark for Deep Research Systems in Financial Forecasting (2026). Establishes temporal point-in-time backtesting methodologies across valuation, return ranking, event direction, and fundamental metrics to evaluate autonomous research agents on prospective reasoning without hindsight lookahead data leakage. arxiv.org/abs/2611.01621
30. Epoch AI, The Epoch Capabilities Index (2026). Stitches over 50 distinct benchmarks onto a unified continuous difficulty scale, preventing single-benchmark saturation from distorting empirical capability trends. epoch.ai/eci
31. Epoch AI, Estimating the Agent Population: How many AI agents could we run? (2026). Models physical manufacturing bounds and compute constraints, finding high-bandwidth memory (HBM) supply sets hard limits on concurrent frontier-model agents. epoch.ai/publications/estimating-the-agent-population
32. Zongxia Li, Zhongzhi Li, Yucheng Shi, Ruhan Wang et al., Long-Horizon-Terminal-Bench 2.0: Automating Diverse Task Construction and Measuring Sustained Agent Progress (2026). Evaluates sustained agent execution across 77 multi-hour terminal tasks and 13 domains in containerized environments, introducing dense 30-minute verifier checkpoints to evaluate partial progress and detect drift over extended horizons. arxiv.org/abs/2610.14146 · arxiv.org/abs/2607.08964
33. Yumou Liu, Mingju Chen, Zhoufutu Wen, Qianyu Yang, Zijie Wang et al., DeliverBench: Benchmarking AI Agents on Delivering Realistic Multi-Stage Professional Projects (2026). Evaluates autonomous agents across 11 multi-stage professional projects and 150 subtasks developed with domain specialists, finding top systems score only 65 out of 100 on final delivery due to cumulative error inheritance across iterative feedback rounds. arxiv.org/abs/2611.03412