Last week, I showed you evidence from Stanford that AI progress isn’t slowing down.
Today, I want to look at the same story from a different angle.
Benchmark scores can tell us a lot about artificial intelligence. They show whether a new model can solve harder math problems, write better code or outperform other AI systems on increasingly difficult tests.
And we should be paying attention to those scores. They help us understand whether AI is getting smarter.
But if you’re trying to understand where AI technology could be heading, I think there’s an even better question to ask:
How long can it keep working before it starts making mistakes?
Five years ago, most AI systems could only reliably handle tasks that took humans a few seconds to accomplish. Today, some can stay on track for nearly an hour.
That’s a very different kind of progress.
And it changes what AI can realistically be trusted to do.
AI’s Growing Attention Span
The chart below comes from METR, a nonprofit research group that studies how quickly artificial intelligence is advancing.
Instead of measuring benchmark scores, researchers tracked how long different AI systems can work through real-world tasks like software engineering, cybersecurity, machine learning and reasoning problems before making mistakes.
And the pace of change here is hard to ignore.
Image: METR
Back in 2019, the most advanced AI systems could handle tasks lasting just a few seconds. By 2023, they’d reached several minutes. And by 2025, some frontier systems were approaching or surpassing the one-hour mark.
In other words, AI has been doubling the amount of time it can stay on task roughly every seven months.
That lines up with Stanford’s evidence that AI progress is still accelerating across major benchmarks. It’s just telling that story through a different lens.
Instead of asking how smart AI systems are, METR asks how long they can stay productive.
And that’s a crucial measure. Because once AI can stay on task longer, it can take on work that was previously assigned to interns, junior employees or contractors.
We’re already seeing it happen.
Ken Griffin, founder and CEO of Citadel and one of the most successful hedge fund managers in history, has long been one of AI’s more vocal skeptics.
Image by Paul Elledge – Citadel Enterprise Americas LLC
Earlier this year, he dismissed much of the excitement around AI as overblown.
But after watching AI agents inside Citadel tackle increasingly complex work, he recently said he went home on a Friday feeling “fairly depressed.”
According to Griffin, work that once required teams of finance professionals with master’s degrees and PhDs working for weeks or even months was suddenly being completed by AI agents in hours or days.
And he made clear these weren’t entry-level jobs. He was talking about highly specialized research and analytical work.
METR’s chart helps explain why stories like this are starting to emerge.
It’s not that AI is suddenly becoming smarter overnight. It’s simply staying useful long enough to handle larger and more complicated tasks.
And as it continues to improve in this regard, it will undoubtedly have profound implications for the future of work.
Here’s My Take
Maybe the leap from AI that can stay on task for thirty seconds to AI that can work for nearly an hour doesn’t sound like a big deal to you.
That’s because our brains tend to think in straight lines.
But exponential change has a way of looking slow…
Until it doesn’t.
METR’s research suggests AI has been roughly doubling the amount of time it can stay on task every seven months. If that trend continues, an hour will eventually become a day. Then two days. Then a week.
Extend that trend out far enough and you move beyond work that can be done by junior employees.
You’re talking about software that can take on projects that once required teams of highly trained people working for weeks, months or even years.
Ken Griffin has already gotten a glimpse of that future.
But I suspect he won’t be the last.
Regards,
Ian KingChief Strategist, Banyan Hill Publishing
Editor’s Note: We’d love to hear from you!
If you want to share your thoughts or suggestions about the Daily Disruptor, or if there are any specific topics you’d like us to cover, just send an email to [email protected].
Don’t worry, we won’t reveal your full name in the event we publish a response. So feel free to comment away!
