CONTROLLED AGENCY | Issue 10: The Hidden Cost of AI
The Apprenticeship We Are About to Lose
We spend a lot of time asking whether AI will replace experts.
I wonder if the bigger story is that AI is quietly changing how experts are formed in the first place.
Over the past few months, I have found myself returning to the same idea from different angles. AI is moving from “help me do this” to “I’ll do this for you.” Models are no longer just helping us write, code, research, or analyze, they are increasingly expected to perform those tasks on our behalf.
That shift feels like a natural progression in capability, but delegation changes both sides of the relationship. As AI learns to do more of the work, humans inevitably spend less time doing it themselves.
That may have consequences beyond productivity.
For generations, expertise emerged through repetition. Writers found their voice by producing hundreds of thousands of imperfect words, engineers learned by debugging broken systems, security analysts developed intuition by triaging endless false positives and chasing dead ends. Doctors, researchers, product managers, and lawyers all went through similar apprenticeships, built not on brilliance, but on exposure to uncertainty.
Most of that work feels inefficient.
AI’s promise is to remove exactly that inefficiency.
But what if the inefficiency was doing something important?
The Hidden Function of Repetition
In cybersecurity, junior analysts do not become good because they immediately encounter sophisticated attacks, they become good because they see thousands of ordinary events until they internalize what “normal” looks like.
They investigate alerts that turn out to be nothing, they follow leads that go nowhere; they make mistakes, correct them, and slowly develop the intuition that experienced practitioners seem to possess almost effortlessly.
The apprenticeship is not a detour on the road to expertise, it is the road.
The same is true for programmers learning to debug their own code, researchers learning to question their assumptions, or writers learning why a sentence does not quite work. The bad drafts, failed hypotheses, wrong assumptions, and false alarms are not merely the cost of learning. They are the mechanism by which judgment is built.
AI increasingly offers to bypass that mechanism.
We May Be Optimizing Away Judgment
There is a subtle but important difference between producing an answer and learning how to recognize a good answer.
If AI rewrites every rough draft, summarizes every log, explains every unfamiliar codebase, prioritizes every alert, and suggests every architectural pattern, then humans may become more productive while simultaneously spending less time wrestling with ambiguity.
The irony is that ambiguity may have been where expertise was actually formed.
Perhaps expertise is not accumulated knowledge.
Perhaps expertise is accumulated error.
A senior engineer often knows what to do because they have already done the wrong thing dozens of times before. A seasoned writer develops a voice not by reading polished prose, but by writing enough bad sentences to recognize when one rings false. An experienced analyst does not simply know the textbook answer; they know what previous failures looked like.
AI can help produce the output.
It cannot easily transfer the intuition those mistakes created.
Cybersecurity Has Seen This Before
Security engineering has spent decades designing systems around an uncomfortable truth: humans are fallible.
We assume passwords will be reused, we assume users will click the phishing link, and we assume analysts will miss alerts. The goal was never to eliminate mistakes; it was to build systems that remain resilient when mistakes inevitably happen.
The solution was not to expect perfect humans, but to build technical controls that reduce the consequences of human imperfection.
I wonder whether AI presents the inverse problem.
What happens when we remove so much friction from the learning process that the next generation never accumulates enough mistakes to develop intuition in the first place?
What happens when the junior analyst never has to investigate the false positive because the AI already filtered it? When the junior developer never has to untangle the broken logic because the AI already suggested the fix? When the writer never sits with the uncomfortable first draft because the model already produced a polished second one?
Perhaps we are not just automating work.
Perhaps we are automating parts of the apprenticeship itself.
Controlled Agency
Most discussions about AI focus on what machines will become capable of doing.
I think the more interesting question is what humans may gradually stop doing.
Controlled Agency has largely been an exploration of delegated authority: what happens when we hand more decision-making and action-taking to systems that cannot themselves be accountable. We often frame this as a governance challenge, asking how humans can remain in control as AI becomes more autonomous.
But perhaps there is another side to that equation.
As machines become more capable of acting, humans may spend less time experiencing the repetition, uncertainty, and failure that once shaped expertise. The more work we delegate, the fewer opportunities we have to build the judgment required to supervise what we have delegated.
Maybe AI will not replace experts.
Maybe it will replace the long, imperfect apprenticeship that created them.
And if that happens, the hardest problem for AI governance may not be controlling the machine, it may be preserving the conditions under which humans learn to exercise judgment in the first place.