AI changed what's possible. Knowing whats' worth doing did not get easier

Core Thesis

The bottleneck for AI leverage has shifted from capability to what we might call the inner operating system: the structure of attention, the pattern of self-production, the stability of vision. Many experienced engineers sense that they bring something essential to AI-augmented work, something intangible that resists articulation. This essay argues that what they sense is real. It is the accumulated history of the observer: the sediment of countless interactions with code, systems, organisations, and purposes. This history cannot be transferred to agents. It cannot be compressed or accelerated. And as AI changes the economics of what can be built, this history becomes even more important ~ the substrate. The path forward is not to abandon craft for abstraction, nor to cling to making while leverage moves elsewhere, but to understand that the charioteer’s value lies precisely in knowing where the chariot must go and why.

I. What the Experienced Engineer Knows (But Cannot Quite Say)

A senior engineer I know paused while verifying agent-generated code. “This is fine,” he said. “But it’s not right. Something is off.”

The code worked. The logic was sound. The tests passed. Yet something in his accumulated experience registered a dissonance that had no obvious vocabulary.

Press experienced engineers to articulate what exactly is off, and they often struggle. Many share a conviction: AI cannot figure it all out, not because of current limitations but because of something deeper about the nature of complex systems. They know that the generate-check loop (agents that produce, verify, iterate until correct) works remarkably well for bounded problems where correctness can be specified and checked. But they also sense that their presence in the loop is not merely quality assurance. It is something more fundamental.

This essay attempts to give that intuition a theoretical foundation.

What the experienced engineer brings is not a skill that can be listed or a heuristic that can be encoded. It is a history. Humberto Maturana’s biology of cognition, combined with insights from Vedic philosophy and W. Edwards Deming’s systems thinking, together illuminate what this means and why it matters for the future of AI-augmented work.

The question is: what is this recognition, exactly? And why can it not be delegated?

II. The Charioteer’s Ladder

The Bhagavad Gita, a dialogue between the warrior Arjuna and his charioteer Krishna, provides the governing image for this essay. The charioteer does not pull the chariot. The charioteer steers. The horses supply the force; the charioteer supplies the direction. But steering requires contact with the horses: feeling when they drift, sensing when the terrain changes, knowing when to give rein and when to gather. And steering requires knowing the destination, holding the map in mind even when the immediate path is obscured.

This is the situation of the AI-augmented engineer. The agents are the horses. They are powerful, fast, capable of sustained effort over distances that would exhaust any human. But they do not know where to go. They do not know why the journey matters. They cannot recognise when the destination itself needs to change.

Bret Victor, in his interactive essay “Up and Down the Ladder of Abstraction,” articulates how the charioteer must move. The deepest insights emerge not at any one level of abstraction but in the transitions between them. Down into the concrete (this line of code, this specific failure, this moment in time) to fix what is broken at the level where it is broken. Up to the system level (the algorithm’s behaviour across parameters, the emergent dynamics, the architectural pattern) to see what caused the breakdown.

The charioteer who never descends from high altitude, who orchestrates agents without ever inhabiting the machinery, creates what we might call experiential debt: systems that function but that no one understands, code that future developers must excavate archaeologically because the foundations were never inhabited. Peter Naur cautioned long ago that programming is a form of theory building. You create deep mental models of the systems you build, curate, and maintain. These models may not form if all you do is verify behaviour.

But there is an equal and opposite failure. The charioteer who insists on pulling alongside the horses has misunderstood their role. The craftsperson who cannot stop building, who keeps descending into implementation because that is where the old identity regenerates, confuses the pleasure of making with the creation of value.

The charioteer who never touches the reins is not steering; they are hoping. The charioteer who insists on being a horse has abandoned their post.

III. The Observer Who Cannot Be Eliminated

For Maturana, there is no cognition without an observer, and the observer cannot be separated from what is observed. There is no view from nowhere, no objective standpoint outside the system. Every observation is made by an observer with a particular history of structural coupling: a particular accumulation of past interactions that shapes what can be perceived and how.

This is not relativism in the weak sense (everyone’s opinion is equally valid). It is a structural claim about the nature of cognition itself. What you can see depends on what you have become through your history of seeing.

Applied to AI-augmented work, this suggests what cannot be delegated.

The AI can generate patterns endlessly. It can produce code, prose, strategies, and designs in quantities that dwarf human output. But it cannot have your history of structural coupling. When you recognise that something “feels right,” when quality registers in your perception, you are not applying a rule. You are resonating with your own accumulated history.

This is what “taste” actually is: the sediment of an observer’s accumulated couplings. It cannot be specified, only enacted. It cannot be transferred, only cultivated through experience that cannot be compressed.

The engineer who was present when the product team fought for six months over the notification system, who watched the compromise emerge and saw which stakeholders had to be satisfied, does not need documentation to know why the architecture looks the way it does. They see the politics embedded in the code. They know that what looks like technical debt is actually organisational compromise made material. They know which constraints are load-bearing and which were workarounds for people who have since left.

This is what the experienced engineer senses but cannot articulate. When they look at agent-generated code and say “this will cause problems later,” they are often not identifying a technical flaw. They are recognising a mismatch between what the code assumes and what the organisation actually needs: a feature that duplicates something another team owns, an abstraction that cuts across a political boundary, a design that will require sign-off from a stakeholder who has historically blocked similar changes.

This is fine. But it’s not right.

The recognition lives in the history.

IV. The Verification Paradox

The generate-check loop appears robust: agents produce, verification catches errors, iteration converges on correctness. But this appearance conceals a structural problem.

You can only verify at the layers you know to check. And you cannot know which layers you are failing to check.

Consider what happens when agents build a reporting dashboard. The code works. The visualisations render. The data flows correctly from source to display. But the engineer who has sat through quarterly business reviews, who has watched executives misinterpret metrics and make bad decisions, who knows that “active users” means something different to the product team than to the sales team, sees something the verification process cannot capture. The dashboard will generate confusion because it uses terminology that means different things to different stakeholders. No test can catch this. It is only visible to someone who has been in the rooms where the terminology was contested.

Or consider a team using agents to automate a workflow. Each step is correctly implemented. The integration tests pass. But someone who has been at the company for three years knows that the workflow exists because of a compliance audit that is no longer relevant, that everyone routes around it anyway, that automating it will simply make an obsolete process more efficient. The agents optimised the wrong thing because they could not know that the process itself was the problem.

Or consider agents generating a migration plan for moving services between cloud providers. The technical steps are sound. But the engineer who was present during the vendor negotiation knows that the current contract includes credits that expire in six months, that the CFO has a relationship with the account manager that affects pricing discussions, that the timeline everyone agreed to was based on assumptions about hiring that have not materialised. None of this lives in any document the agent could access.

The verification paradox is this: under-specification is invisible from the inside. You need the observer with the right history to even see that there is a layer missing from your verification.

This is fine. But it’s not right. Something is off.

The charioteer who lacks the history cannot steer around obstacles they cannot perceive.

V. The Complexity Objection

A reasonable objection presents itself. Coding agents are demonstrating that they can develop fairly large systems. Cursor’s experiments show agents running for a week to create a reasonable web browser rendering engine. Agents can handle complexity. And if code becomes cheap enough to regenerate rather than understand, if context windows can hold entire codebases and git histories, what remains for the human observer?

The objection deserves a serious response.

Technical observer history is indeed becoming more compressible. The scar tissue of implementation (recognising antipatterns, sensing architectural decay, knowing common failure modes) is exactly what agents absorb through scale. If the argument rests only on technical pattern recognition, it may have an expiration date.

But the objection conflates technical complexity with purposive coherence.

A browser rendering engine is a specification-rich problem. The W3C standard is the purpose. Success is measurable against an external criterion. This is where agents excel precisely because the “why” is fully specified. Render HTML and CSS according to the specification. The purpose requires no history to understand.

Most software is not like this. Most software serves human purposes that shift, in contexts that evolve, for users whose needs remain partially tacit and are discovered through interaction.

Consider what git logs do not contain: the meeting where the VP insisted on a feature that everyone knew was wrong but no one could refuse. The Slack conversation where two engineers agreed to a workaround that was never documented. The customer call that revealed a need the product team still has not prioritised. The reorg that left a module orphaned, owned by no one, changed by everyone. The informal agreement between teams about who handles edge cases at the boundary.

An agent can reconstruct a plausible history from artifacts. Plausible is not actual. The gap between them is exactly where costly mistakes live.

As implementation becomes cheap, the human role moves upstream. The observer history that cannot be delegated is increasingly history with humans and contexts: the accumulated sense of what this team needs, what these users struggle with, what “working” means in this specific situation.

The charioteer’s job was never to understand equine musculature. It was to know where the chariot must go. As the horses grow stronger, the steering becomes more consequential, not less.

VI. The System Behind the Code

W. Edwards Deming, the management theorist whose work transformed Japanese manufacturing, observed that “the bulk of the causes of low quality and low productivity belong to the system and thus lie beyond the power of the work force.”

The dysfunction is not in the component but in the organisational system that produced it.

Conway’s Law is the technical expression: code mirrors the communication structures that created it. The awkward API boundary exists because two teams did not coordinate. The duplicated logic exists because incentives discouraged shared ownership. The brittle architecture exists because deadlines made proper design unjustifiable.

When an agent regenerates a component, it learns from artifacts that embed this dysfunction. The pathology becomes the pattern. Regeneration reproduces the disease.

This is why “just regenerate” fails as a strategy. You cannot regenerate your way out of a system-level problem. The component is fine. The system is not right.

The component is fine. The system is not right.

Here is where AI becomes genuinely transformative: it changes the cost of addressing systemic problems.

What was previously unjustifiable (the six-month rewrite, the cross-team refactor, the infrastructure rebuild) now becomes tractable. The ROI calculation that kept dysfunction in place no longer holds. Organisations have lived with embedded dysfunction because the cost of addressing it exceeded the visible cost of tolerating it. Technical debt accumulated not from ignorance but from rational calculation: fixing this properly would take six months; we ship in two.

AI changes the arithmetic. The six-month rewrite becomes a two-week effort. The “impossible” refactor becomes merely expensive. The constraint relaxes.

But knowing what to change requires exactly the observer history we have been discussing. Only now the relevant history is with the organisational system, not just the technical one.

The agent can rewrite the code. It cannot see that the code’s awkwardness traces to a turf war between teams that ended three years ago, that the constraint no longer applies, that the “requirement” everyone treats as fixed was actually a workaround for a VP’s pet project that was cancelled. It cannot know that the engineer who blocked the last attempt to fix this has since left, that the political obstacle has been removed, that what was impossible is now merely difficult.

This history lives in no artifact. It lives in the observer who was present, who remembers, who can say: this is the part that was always wrong, that we could never fix, that we can fix now.

VII. The Generative Turn

The experienced engineer’s value is not merely defensive (steering agents away from mistakes) but generative (identifying the improvements that only became possible when AI changed the cost structure).

This reframes the human role entirely:

AI provides cheap implementation; the observer provides recognition of what needs implementing

AI provides capacity to change; the observer provides judgment about what change serves

AI provides power to regenerate; the observer provides understanding of what regeneration perpetuates

AI provides ability to refactor at scale; the observer provides memory of why things are the way they are

The dysfunction everyone learned to live with. The workaround that became load-bearing. The “that’s just how it is” that no longer needs to be. These are visible only to someone with history in the system.

The charioteer’s destination is not just “working software.” It is software that no longer carries the scar tissue of organisational dysfunction that was previously too expensive to heal.

This is the positive vision: AI enables systemic improvement that was previously impossible, but only if someone can see what needs improving. The observer’s history becomes more valuable as the economics shift, because that history identifies which changes to pursue.

VIII. Flow, Reflection, and the Two Modes of Steering

The charioteer cannot steer continuously. There are two distinct modes of operation, and both are necessary.

Mihaly Csikszentmihalyi’s research on flow states describes the first mode. Flow occurs when skill level matches challenge level. Self-consciousness dissolves. Action and awareness merge. The charioteer directing multiple agents, switching context fluidly, watching outputs cascade, can achieve flow states that were previously inaccessible. The skill level is not in the making but in the directing, and when it matches the challenge, the same dissolution of self-consciousness occurs.

But scale requires more than flow. It requires stepping back.

The Yoga Sutras offer a precise framework for this dual requirement. The text identifies two complementary disciplines: abhyasa (persistent practice) and vairagya (non-attachment). Abhyasa is the commitment to show up, to engage, to act. Vairagya is the capacity to step back, to not be captured by the momentum of action, to see clearly without the distortion of desire.

Flow is abhyasa at full power: the practitioner absorbed in practice. Reflection is vairagya made active: the observer disentangling from the observed to see the pattern.

Without stepping back, without pausing to review how the agents performed (not just what they produced) you cannot improve the system. You increase velocity but not vector. Speed without direction.

The experienced engineer knows this intuitively. The best decisions about what to build next come not from rapid iteration but from stepping back far enough to see the whole: remembering what customers actually complained about last quarter, recalling which features were built because of promises made in sales calls that should never have been made, knowing which technical investments will unlock future capabilities and which are rabbit holes.

The charioteer must drive and must also stop to consult the map. Both modes serve the journey.

IX. Clarity as the Charioteer’s Discipline

The Gita’s psychology recognises three fundamental qualities that characterise all action. I am using the original Sanskrit terms (moving on in the rest of this essay) as they capture the essence better and carry a rich history that is worth reflecting on deeply.

Tamas is inertia, avoidance. The tamasic response to the capability shift is denial: pretending the leverage has not moved, refusing to update, clinging to obsolete patterns out of comfort.

Rajas is passion, restlessness, the drive toward outcomes. The rajasic response seizes AI as an amplifier of desire: generate faster, ship more, accumulate achievements, feed the hunger for results without examining what results serve.

Sattva is clarity, discernment. The sattvic response uses increased capability as an opportunity for increased clarity: asking not only “what can I build?” but “what is worth building?”

Here is the danger that experienced engineers recognise intuitively: AI provides rapid feedback loops. The dopamine hit of generation (watching something come into existence through your direction) is immediate and compelling. Ship a feature. See it work. Ship another. The rhythm is seductive.

This is rajasic, passion driven, velocity masquerading as productivity.

Without sattvic clarity about what matters, you can be enormously productive at things that do not matter. The chariot moves very fast. But toward what?

The rajasic charioteer is fused with outcomes. When results disappoint, they have failed. When results succeed, they have achieved. The identity depends on the horses’ speed.

The sattvic (discerning) charioteer rests in the quality of attention, not the quantity of output. When outcomes disappoint, there is information. When they succeed, there is gratitude. The identity is stable because it is grounded in clarity about purpose, not attachment to results.

The difference is not in what gets done but in what it costs to do it, and in how sustainable the doing is.

X. How Scar Tissue and Clarity Reinforce Each Other

Accumulated experience and sattvic clarity are not merely parallel virtues. They are mutually reinforcing capacities.

Clarity makes learning from experience more effective. The rajasic (pure passion driven) engineer who watches a product launch fail is already planning their explanation, already managing the politics, already feeling the sting to their ego. Their attention is divided. The sattvic (discerning and clear thinking) engineer, not fused with outcomes, can attend fully to what actually happened: which assumptions were wrong, which stakeholders were not consulted, which signals were ignored. The learning is deeper because the seeing is clearer.

Organisational scar tissue enables sattvic clear perception. The engineer without history in the organisation cannot see what there is to be clear about. They lack the context that would make the system’s state legible. The accumulated history of being present (in meetings, in launches, in failures, in recoveries) provides the categories through which clarity operates. You cannot be clear about something you cannot perceive.

This creates a virtuous cycle. Clarity enables better learning from experience. Richer experience enables clearer perception. Clearer perception enables better learning.

It also explains why the rajasic “vibe coder”; despite generating enormous output, may develop slowly as a charioteer. The fusion with outcomes interferes with learning. The avoidance of engaging with the organisational reality (the stakeholders, the history, the politics) prevents the relevant scar tissue from forming. The speed is real, but the growth is stunted.

Velocity without vector.

The recognition requires both: the history to perceive the wrongness, and the clarity to attend to what the perception reveals.

XI. Recovery: When You Realise You Have Drifted

Everyone drifts. The question is whether you notice and what you do about it.

Signs that you have drifted into rajasic velocity:

You feel anxious when agents are idle, compelled to generate even without clear purpose. You have stopped reviewing what you shipped last week; only next week matters. Conversations about “why” feel like interruptions to the real work of “what”. You are building features without checking whether anyone asked for them. You avoid meetings with stakeholders because they slow you down. The work feels increasingly frenetic but less meaningful.

Signs that you have drifted into tamasic avoidance:

You are doing extensive “preparation” before engaging agents, organising and structuring so that you feel like you contributed something tangible. You have convinced yourself that certain tasks “really need a human touch” when they do not. You feel resentment toward colleagues who have adapted faster. You are clinging to workflows that feel comfortable but are not creating value. You avoid learning what agents can now do because you do not want to know.

The path back:

Recognise that drift is normal. The pull of rajas (the seduction of velocity) and tamas (the comfort of the familiar) is constant. Staying sattvic is not a state you achieve but a practice you maintain.

Reconnect with context. Sit in on a customer call. Attend the planning meeting you have been skipping. Have coffee with someone from a team you do not usually work with. The goal is not efficiency but rebuilding the organisational awareness that makes steering possible.

Schedule reflection. Not as a luxury but as infrastructure. Weekly reviews of what the system produced, not just whether it met requirements but whether it cohered with what the organisation actually needs. Monthly examination of your own patterns: where are you getting stuck? What relationships are you neglecting?

Articulate what matters. The stable destination requires a vision clear enough to recognise deviation. If you cannot say what the work is for (beyond metrics, beyond features, beyond the next milestone) you cannot steer toward it.

XII. A Concrete Example: Laddering in Practice

The situation: A charioteer is directing agents to build an internal tool for the operations team. The agents produce code. Tests pass. The system works.

The recognition: Something feels off. The charioteer cannot immediately name it, but there is a dissonance: a pattern-match against organisational history that has not surfaced to conscious articulation yet.

It meets the spec. It misses the point.

To frame it, I am pulling in the ideas of Brett Victor’s Ladders of Abstraction to descend and ascend; as well as the mechanism to manage drift (reflect and synthesis).

Descending: Instead of overriding the unease and moving on, the charioteer descends. Not to find a bug (there is no bug) but to inhabit the system. Walking through the workflow. Looking at the terminology used in the interface. Checking the permissions model.

The descent surfaces the issue: the tool assumes a workflow that the operations team officially follows but actually routes around. The charioteer was present six months ago when the team lead complained that the official process was too slow, when an informal workaround emerged, when everyone quietly agreed to pretend the official process was still in use. The agents built to the specification. The specification does not match reality.

Ascending: The charioteer now ascends. Not just “fix this specific tool” but “what produced this gap?” The requirements came from documentation that no longer reflects practice. The agents had no way to know that the documentation was organisational fiction.

The insight: the real requirement is not “automate the official workflow” but “formalise the workaround that everyone actually uses.” This requires a conversation with the operations team lead, possibly a discussion with compliance about whether the workaround can become official, maybe a negotiation about changing the process rather than automating the existing one.

Reflecting - The charioteer steps back further. Not “how do I fix this tool” but “what does this tell me about how I am working?” The reflection surfaces a pattern: trusting documentation without checking whether it reflects current practice. Treating specifications as ground truth rather than hypotheses to be validated.

Stabilising - The vision clarifies. The goal is not “tools that match specifications” but “tools that serve how people actually work.” The destination is not “features shipped” but “systems that people will actually use.”

The whole sequence might take a day or two, including the conversation with the operations team. The rajasic response would have been to ship the working tool and move on: velocity preserved, a tool that will be quietly abandoned within a month, effort wasted on automating a fiction.

XIII. The Reins

The charioteer does not pull. The charioteer steers. The horses (the agents, the capabilities, the generative velocity) supply the force. The charioteer supplies the direction.

The technological moment we occupy offers extraordinary leverage. The horses have never been stronger, the chariot never faster.

The question is not capability but character: who is holding the reins, and from what centre do they steer?

Deming saw that most dysfunction lives in the system, beyond the power of the workforce to address. AI changes this calculus. What was impossible to justify now becomes tractable. The six-month rewrite, the cross-team refactor, the deep architectural repair: these are now within reach.

But reaching requires seeing. Seeing requires history. Not history with the code (agents can read code) but history with the humans who wrote the code, the organisations that shaped its constraints, the purposes it was meant to serve, the compromises that became calcified into requirements.

The engineers who scale successfully will not be those who abandon craft for abstraction, nor those who cling to making while leverage moves elsewhere. They will be those who learn to move: up and down the ladder, between flow and reflection, from stable centres that hold vision without grasping at outcomes.

What they bring cannot be transferred to agents. It is the history of the observer, the sediment of organisational coupling, the taste that recognises when specifications have drifted from reality. It is the memory of conversations that were never documented, decisions that were never recorded, compromises that became invisible. It is the sattvic clarity that resists the rajasic pull of velocity for its own sake. It is the knowledge of why things are the way they are, which is also the capacity to see what can finally change.

The chariot moves. The destination drifts. The charioteer knows where to go.

References

Csikszentmihalyi, M. (1990). Flow: The Psychology of Optimal Experience. Harper & Row.

Deming, W. E. (1986). Out of the Crisis. MIT Press.

Maturana, H. R., & Varela, F. J. (1980). Autopoiesis and Cognition: The Realization of the Living. D. Reidel Publishing.

Naur, P. (1985). “Programming as Theory Building.” Microprocessing and Microprogramming, 15(5), 253–261.

Patanjali. (c. 400 CE). Yoga Sutras. (Edwin Bryant’s 2009 translation recommended for depth; Chip Hartranft’s 2003 translation recommended for accessibility.)

The Bhagavad Gita. (c. 200 BCE–200 CE). (Eknath Easwaran’s translation recommended for accessibility.)

Victor, B. (2011). “Up and Down the Ladder of Abstraction.” worrydream.com/LadderOfAbstraction/