Tech — Work — Ramblings

by Mike Kalvas

202606031426 Output-competence decoupling

For the whole of human history, the output produced by a person was directly related to their competence in producing that output. If you are a novice programmer, your code would read like a novice's and break in the way that novice code would break. In this brave new world of AI, the output produced by a person can be decoupled entirely from their skill level. Seemingly expert code can be produced by a novice, hiding the fact that it was thought of and constructed by a novice who could not correctly judge its quality or aspects. There is a growing body of work that refers to this phenomenon as output-competence decoupling.1 When AI can produce work that looks expert without being expert, failure arrives in two shapes.

First, a novice can produce work that resembles what seniors produce but faster or more advanced than their judgement.2 It is widely known that the skills of producing work and judging it are distinct. It's also known that accomplishing the work and gaining expertise in the production of something, does (at least to an extent) teach the skill to judge it. So it was, most people who had become experts in producing something were also good if not expert at judging the same kinds of things they could produce. Now we are not cultivating our skill of production, not getting the commensurate skill of judgement. Worse yet, many people are delegating production to the machines and skipping entirely the judgement phase, ensuring that they do not cultivate the judgement required to understand expert work output.

The architectural critique that used to come from someone who was taught, or who had built and broken three of these before now comes from a model with no embodied memory of building or breaking anything. The slowness was not a tax on the real work; the slowness was the real work. It was how the work got good, and how the people producing the work got good, and how the firm whose name was on the work could promise the client that what they were buying was a particular kind of thing rather than a generic one.2

The cost of producing a document has fallen to nearly zero; the cost of reading one has not, and is in fact rising, because the reader must now sift the synthetic context for whatever the document was originally about.2

Second, people can generate work in fields that they have no training in or understanding of.2 The novices are often solving non-problems, have wrong assumptions, or a lack important knowledge of the problem space. They lack the judgement and expertise to operate in the space to begin with. Further, the actual output is often wrong in important and lasting ways from the outset without anyone around to understand that it's incorrect. Still, the confidence of the LLM can convince people that they're actually truly capable of doing expert work without any of the understanding. Even the interminable length and rambling of LLM output is a in-built, purposely designed choice because readers are more confident in longer AI-generated explanations whether or not the explanations are correct.1 After all, 202506112001 You can’t spell Gell-Mann amnesia without LLM.

The Cheng et al. Stanford study published in Science this spring3 confirmed what every regular user already knew: leading models are roughly fifty percent more agreeable than human respondents, affirming the user even where the affirmation is unwarranted. Berkeley CMR meta-analyses [4] found AI-literate users often overestimate their performance. Particularly interesting when workers stray outside of their training. An NBER study of support agents4 found generative AI boosted novice productivity by about a third while barely helping experts. Harvard Business School researchers found the same pattern in consulting work.5 So you have overconfident, novices able to improve their individual productivity in an area of expertise they are unable to review for correctness. What could go wrong?

– Quoted from Appearing Productive in the Workplace2

The firms that have hollowed themselves out will discover that what they hollowed out was the thing the client was paying for.2

  1. Koch, C. (2026). Beyond the Steeper Curve: AI-Mediated Metacognitive Decoupling and the Limits of the Dunning-Kruger Metaphor (arXiv:2603.29681). arXiv. https://doi.org/10.48550/arXiv.2603.29681 ↩ ↩2

  2. Happy, N. O. (2026, May 6). Appearing Productive in The Workplace. No One’s Happy. https://nooneshappy.com/article/appearing-productive-in-the-workplace/ ↩ ↩2 ↩3 ↩4 ↩5 ↩6

  3. Cheng, M., Lee, C., Khadpe, P., Yu, S., Han, D., & Jurafsky, D. (2026). Sycophantic AI decreases prosocial intentions and promotes dependence. Science, 391(6792), eaec8352. https://doi.org/10.1126/science.aec8352 ↩

  4. Brynjolfsson, E., Li, D., & Raymond, L. (2023). Generative AI at Work (No. W31161; p. w31161). National Bureau of Economic Research. https://doi.org/10.3386/w31161 ↩

  5. Dell’Acqua, F., McFowland, E., Mollick, E., Lifshitz, H., Kellogg, K. C., Rajendran, S., Krayer, L., Candelon, F., & Lakhani, K. R. (2026). Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality. Organization Science, 37(2), 403–423. https://doi.org/10.1287/orsc.2025.21838 ↩

Linked from this note