When Andrej Karpathy posted in early 2026 that he had shifted from 80% manual coding to 80% agent coding over the course of a few weeks, most of the responses treated it as commentary on individual developer productivity. The more important implication was organizational: if a skilled engineer’s workflow has structurally shifted, the skills required to lead an engineering team have shifted with it.
Most hiring processes for fractional CTOs — and for CTOs in general — are built around the previous model. The evaluation criteria are weighted toward code review, architecture design, vendor evaluation, and engineering team management. Those skills still matter. But they are no longer sufficient, and in some cases they are no longer the primary determinant of whether a technical leader is effective in the current environment.
stateDiagram-v2 direction TB state "Evaluate for code skills only" as CodeOnly state "Add agent-orchestration criteria" as Full state "Mis-hire: wrong mental model" as Fail state "Effective fractional CTO" as Win [*] --> CodeOnly CodeOnly --> Full: recognizes workflow shift CodeOnly --> Fail: ignores agent-first context Full --> Win: aligns with team workflow Fail --> Full: corrects after costly lesson Win --> [*]
What the Shift Actually Changes
An engineering team building primarily with agents is doing different work than an engineering team writing code manually. The primary skill is no longer implementation — it is specification, evaluation, and integration. The team writes prompts, evaluates agent outputs, defines what correct looks like, and integrates agent results into larger workflows. The code is increasingly the output of agents, not the output of developers.
The fractional CTO’s role in this environment requires competencies that were secondary in the prior era:
Eval loop design. An eval loop is the mechanism that determines whether an agent’s output is correct before it goes to production. Designing one requires knowing what correct means in measurable terms — not just conceptually, but in a way that can be checked programmatically. Most engineering teams building with agents are running without structured eval loops. The fractional CTO who can design and implement these is adding value at the highest-leverage point in the workflow, because catching quality problems before production is orders of magnitude cheaper than addressing them after.
Agent boundary definition. In any agentic workflow, the most important architectural decisions are about what the agent is and is not authorized to do. These are not purely technical decisions — they are policy decisions with business risk implications. A fractional CTO needs to define these boundaries in terms that engineering, legal, and operations can all align on. This requires a different skill set than API design or database schema, but it is equally critical.
Cost and quality tradeoffs at the model layer. An engineering team building agent workflows makes ongoing decisions about which AI models to use for which tasks, at what cost, with what quality tradeoffs. These decisions compound. A fractional CTO who understands the model landscape — what different models cost, where they fail, how to structure evaluation to catch quality drift — can establish policy that produces consistent outcomes. One who does not will find that model selection is happening at the individual engineer level with no consistency or accountability.
What Still Matters From the Prior Era
The shift to agent-first engineering does not make traditional technical leadership competencies irrelevant. The underlying principles — architecture should drive decisions, not follow them; key-person dependency is always a risk; technical debt accumulates if nobody owns the cleanup — apply in an agentic engineering context as much as in a traditional one.
At Digital Business Services, where I served as Chief Architect over 18 developers, one of the highest-impact decisions had nothing to do with the application itself: a 3-tier DNS infrastructure build that saved $500,000 over nine months. The application work got the attention. The infrastructure decision drove most of the value. That pattern repeats in agentic environments: the invisible architectural decisions — how agents are evaluated, how failures are routed, how model access is governed — drive more business impact than the visible application decisions. A fractional CTO who only looks at the application layer is missing the same category of decision in a different form.
What to Evaluate Differently
When assessing a fractional CTO candidate for an organization where engineering is shifting toward agent-first workflows, the criteria that matter most are:
Has the candidate designed an agentic workflow end to end — including the eval loop, the failure modes, and the handoff to operations? Not described one. Designed one, for a real deployment.
Can the candidate describe a technical decision that was primarily about defining constraints — what the system was not authorized to do — rather than expanding capabilities? This is the agent boundary question. Most candidates with only pre-agentic experience struggle to answer it specifically.
Does the candidate have a concrete metric for measuring quality in an AI-generated output workflow? A specific metric they have used, and a specific case where it caught something. Not a framework description.
The fractional CTO who can answer those questions with specificity is already operating in the environment where your engineering team is heading. One who cannot is still operating in the environment from which it came. Given the pace of the shift Karpathy described, the gap between those two positions is widening faster than most hiring processes recognize.
