Industry Commentary →

Karpathy Showed Opus 5 Working for Two Hours Straight. Now Figure Out When to Let It.

Andrej Karpathy's Opus 5 experiment is a real capability demonstration. The harder question for enterprise teams is not whether the AI can run for two hours — it's what authorization, scope, and review look like before and after it does.

Andrej Karpathy posted an experiment last week that is worth slowing down to understand. He gave Opus 5 the opening paragraph of The Lord of the Rings, a one-million-token context budget worth about ten dollars, and asked it to produce a Three.js rendering. The model worked autonomously for roughly two hours and wrote 5,500 lines of procedural code. His framing: LLMs now have “the stamina and patience to do custom work that no one would ever do manually.”

That observation is accurate. The capability is real. What Karpathy did not address — because it was not his point — is the operational governance question that follows for anyone running production systems: when exactly should an enterprise team greenlight a multi-hour, multi-dollar autonomous run, and what does the authorization model look like before it starts and after it finishes?

quadrantChart
title Agentic Run Authorization Matrix
x-axis Low Scope Clarity --> High Scope Clarity
y-axis Low Cost / Risk --> High Cost / Risk
quadrant-1 Checkpoint mid-run
quadrant-2 Human-led only
quadrant-3 Pilot first
quadrant-4 Greenlight and review
LotR render (Karpathy): [0.88, 0.18]
Code refactor scoped: [0.78, 0.38]
Document summarization: [0.84, 0.22]
Open-ended research: [0.22, 0.44]
PII data analysis: [0.52, 0.88]
Multi-system migration: [0.38, 0.84]

The rundown: what Karpathy’s Opus 5 experiment shows

Karpathy’s post from August 2 is short but the demonstration is significant. Opus 5 handled a creative, open-ended technical task over a two-hour window, self-directing its work through thousands of lines of output without human intervention between steps. The cost was approximately ten dollars for the full run. The result was a complete Three.js rendering derived from a literary description.

His point is about what LLMs are now capable of sustaining — not just generating a burst of output and stopping, but maintaining coherent progress on a complex task through a long autonomous session. The “stamina and patience” framing is apt. Most people’s intuition about AI tools is still shaped by prompt-and-response interactions measured in seconds. Multi-hour autonomous execution is a different operational category, and it requires different controls.

For engineers: scope definition is the design decision

Karpathy’s experiment worked because the task had a specific, unambiguous goal: render this description in Three.js. There was no uncertainty about what success looked like. The model had one well-defined output to drive toward.

That kind of task clarity is not a given in enterprise work. Most real agentic run proposals begin with something closer to “analyze the customer support data and identify patterns” or “refactor the billing module” — tasks where the scope boundary is implicit rather than explicit. An agent working on a vague task with a large budget will produce output. Whether that output is what the team actually needed is a different question.

Before authorizing an agentic run, the engineers’ job is to define two things explicitly: what done looks like, and what the agent is not permitted to do. The second constraint is often skipped. It is also often the more consequential one — an agentic system that knows its boundaries will reach them and stop; one that does not will continue in ways that are expensive and sometimes irreversible.

Cost thresholds belong in code, not policy. If a run should not exceed a particular token budget, set a hard limit in the framework. If it should not write to production systems, enforce that through tool access configuration — not through a rule someone is supposed to remember. Relying on the model to self-limit on cost or scope is exactly where enterprise teams get surprised.

For business owners: authorization is a process, not a setting

Karpathy’s ability to run a ten-dollar two-hour experiment without formal planning is appropriate for a personal demonstration. Enterprise contexts are different, and the difference is organizational more than technical.

Authorizing an agentic run in a business context means answering a few questions before the run starts: Who approved this? What is the maximum allowed cost? Which systems can it touch? When does it require a human checkpoint? What does review of the output look like, and who owns that step?

Most teams do not have consistent answers to these questions yet, which means authorization decisions are being made ad hoc, one run at a time, based on whoever happens to be in the room. That works until it does not — a run that exceeds its expected budget, touches data it should not have accessed, or produces output that gets acted on before it has been reviewed.

This is a workflow governance problem that agentic AI has made more expensive to defer. Building the authorization model now, while the use cases are narrow and the stakes are manageable, is the right time to do it.

My take: the workflow architecture question comes first

I worked with a legal services company on a mail-returns processing system built on top of the United States Postal Service’s API infrastructure. The workflow was automated, high-volume, and completely dependent on getting the architecture right before anything ran at scale. The lesson I took from that engagement was that automated workflows have an outsized impact on project outcomes long before the team fully appreciates it. Once a batch process is running, changing the underlying architecture is expensive. The decisions made at the outset — about what the system can initiate, what it can modify, and where it is required to hand off to a human — become structural. Getting those decisions wrong means rebuilding from a point of pain.

Agentic AI runs belong in the same category. “Let it run and see what happens” produces results in a demo context. In production, it shifts the governance decisions to after the fact — after the budget was spent, after the system touched something it should not have, after the output was used. The discipline of defining the authorization model before the first autonomous run is not about limiting what AI can do. It is about being able to trust the output when it is done.

Karpathy’s experiment is a meaningful capability benchmark. The follow-on work for enterprise teams is governance design: understanding which tasks belong in which quadrant of the authorization matrix, and building the checkpoints, cost limits, and review steps that let capable systems do useful work without creating liability in the process.

At G4S Justice Services, we built a GPS monitoring application that tracked participants within geo-fenced coordinate boundaries. The system acted autonomously on real-time location data and triggered alerts based on rule violations. What made it operationally reliable was that every action was bounded at the architecture level — the fence was enforced in code, not described in a policy document someone was supposed to follow. That constraint was not a limitation on what the system could do. It was the reason the system could be trusted to run.

That is the design principle that needs to carry over into enterprise agentic workflows. The scope boundary makes the autonomy possible.

Frequently Asked Questions

What is an agentic AI run and how is it different from a regular AI prompt?

A regular AI prompt sends a question or request and receives a response in seconds. An agentic AI run gives the model a goal and a budget — tokens, tools, time — and lets it work through a sequence of actions autonomously to reach that goal. The model self-directs its next steps rather than waiting for human input between each one. An agentic run for a complex task can involve dozens of intermediate steps, file operations, API calls, or sub-tasks before producing a final output. The defining characteristic is that meaningful work happens without a human in the loop.

How should enterprise teams decide when to greenlight an autonomous agentic run?

The two factors that matter most are scope clarity and cost or risk exposure. If the task is precisely defined and the cost or system-access risk is low, you can greenlight the run and review the output when it completes. If the task has ambiguous boundaries or touches sensitive systems, you need mid-run checkpoints — defined pause points where a human reviews intermediate state before the agent continues. Tasks with genuinely unclear scope and high cost or risk should be decomposed into smaller, well-defined sub-tasks before any autonomous execution begins.

What does AI cost governance look like for agentic runs?

Cost governance starts with setting explicit token and time budgets before the run begins — not after. Most production agentic frameworks support a maximum context window spend and a timeout. Beyond that, good governance tracks cost per task type so teams build accurate intuition about what autonomous work actually costs at scale, includes a human review step for any run above a defined cost threshold, and logs every agentic run with its input, output, and spend so patterns of overreach can be identified. The instinct to just let it run and see what happens is what produces surprise bills and unintended actions.

Is the Karpathy Opus 5 experiment representative of what enterprise AI can do today?

It represents the capability ceiling, not what most enterprise teams have built to use. Karpathy's experiment used a one-million-token context window, a ten-dollar budget, and a creative coding task with a completely unambiguous goal. Most enterprise use cases involve data security constraints, system integration requirements, and compliance boundaries that require more bounded contexts and tighter authorization controls. The experiment demonstrates that the raw capability is real. Whether your infrastructure, authorization model, and review processes are ready to use it well is a separate question that every enterprise team needs to answer on its own terms.

Shawn Livermore — Fractional CTO & Chief AI Officer
About the Author

Shawn Livermore

Fractional CTO and Chief AI Officer with nearly 3 decades of enterprise architecture experience. Clients include Kelley Blue Book, LERETA ($18B property tax processor), First American Financial, Carvana, WellPoint/Anthem, and PacifiCare. 92 client reviews, 5-star average.

View full background →

Need a fractional CTO or CAIO?

Technology leadership without the full-time headcount. Engagements start with a conversation.

Man writing a flowchart diagram on a whiteboard with a blue marker.