AI changed what I need to hire for and what I owe a new employee

When I think about hiring someone into McQueen Analytics now, I am less worried about whether that person can produce a professional-looking first answer because AI can help almost anyone reach that point faster than it took me to learn the same work. I am much more interested in what happens five minutes later, when the answer is clean, confident and wrong in a way that only becomes obvious if someone understands the work underneath it.

I had more than fifteen years of professional judgment before I began working this closely with agents, and much of what I ask them to do is work I had already done slowly, manually and sometimes badly enough to remember why a certain answer misses the mark. A person beginning a career in 2026 should not need to wait fifteen years before using the best tools available, although I also cannot pretend that producing what looks like senior work gives someone the experience to judge it.

That changes what I would hire for, what I would test during an interview and what I would owe someone after they joined the company.

the calculator argument was never really about calculators

I do not want to sound like the teachers who told us to memorize arithmetic because we would not always have a calculator, since all of us now carry one and refusing to use it would usually make the work slower without making it better. The useful part of knowing arithmetic was never proving that I could suffer through long division after the tool arrived; it was learning enough about numbers to know whether the calculator's answer belonged anywhere near the truth.

If I can estimate the scale of a percentage, understand the relationship between the inputs and notice that a bill is off by a zero, I can use the calculator faster and with more confidence while also being harder to fool. Knowing the answer for myself does not always mean knowing the exact number before I press the button, but it does mean having enough of a model in my head to recognize an impossible result, ask which input was wrong and understand who benefits if I accept it anyway.

AI requires the same kind of independent understanding across work that is much harder to reduce to one number. A candidate may produce a polished analysis, a working application or a persuasive strategy in an hour, while the skill I need is knowing what evidence would change the answer, which assumption carries the most risk and where the output became more confident than the source allowed.

faster early work can be a real advantage

There is good evidence that AI can help newer workers become productive faster. Erik Brynjolfsson, Danielle Li and Lindsey Raymond studied the introduction of a generative AI assistant among 5,172 customer-support agents and found a 15 percent average productivity increase, including a 30 percent increase in issues resolved per hour for less skilled and less experienced workers. The researchers also found evidence that the system helped newer workers move through the experience curve more quickly, which is exactly the part of AI-assisted work I would want a new employee to have.¹

I do not want to build a company where every new person has to rediscover the same answer through years of avoidable mistakes. If an agent can show them a strong example, explain a term, adapt a lesson to the question in front of them and let them practice without waiting for me to become available, the tool can make training more personal rather than less.

A randomized study in an introductory Harvard physics course offers a version of what that can look like when it is designed for learning. Students using a structured AI tutor had more than twice the median learning gains of students in an in-person active-learning lesson while spending less time on the material, but the tutor was not an open chatbot asked to improvise a class. Experienced instructors supplied the learning sequence, detailed solutions, feedback design and boundaries around when the system should guide rather than answer.²

That distinction should follow AI from the classroom into a company because access to an agent is not a training program. Someone still has to decide what the employee should learn, what a good answer contains, which mistakes are instructive and when the tool should stop completing the work for them.

producing the answer can hide whether learning happened

The harder evidence comes from a 2026 randomized study by Judy Hanwen Shen and Alex Tamkin, who asked participants to learn a new asynchronous programming library with or without AI assistance. AI use hurt conceptual understanding, code-reading and debugging performance on average, while the people who fully delegated the coding sometimes gained productivity at the cost of learning the library they were using. The preprint also found AI interaction patterns that preserved learning when participants stayed cognitively engaged, which makes the study more useful than a simple warning to turn the tool off.³

The danger is not that a new employee used AI; it is that the company rewarded a finished artifact without checking whether the person gained the ability to read it, challenge it or repair it after the agent left. If I measure only how quickly the work appears, full delegation can look like the best employee behavior right up until the first unfamiliar failure.

Microsoft researchers surveying 319 knowledge workers found a similar issue in how people described their own work: greater confidence in AI was associated with less critical-thinking effort, while greater confidence in their own task knowledge was associated with more. The study relies on self-reported examples rather than an experiment that proves cause, but its description of critical thinking shifting toward verification, integration and stewardship feels close to the job I now do with agents every day.⁴

Verification is not a lesser skill left over after AI does the important work because it requires someone to understand the source, the system, the audience and the cost of being wrong well enough to decide whether the output belongs, which remains an act of senior judgment even when the first draft arrives from a machine.

interviews should test the work after the first answer

I would let a candidate use AI during an interview because banning the tool would create a performance that has little to do with how I expect the person to work after being hired. I would also stop treating the first answer as the main evidence of skill.

I want to see the candidate state what a reasonable answer should roughly look like before asking the tool, identify the assumptions that could change it, inspect the sources instead of citing the agent, and explain which parts they would be willing to defend under their own name. I would give them an output with a planted mistake and ask how they found it, then change one fact and watch whether they understand the consequence or simply ask for another complete answer.

For analytical work, that may mean estimating the direction and scale before running the model, tracing a surprising result back to the data and explaining when a statistically correct answer would still mislead the client. For writing, it means noticing when every factual sentence is supported and the article still says nothing I believe. For software, it means reading an unfamiliar change, finding the decision that does not fit the system and describing how the failure would appear outside a passing test.

The candidate does not need to know every answer without help, although I need evidence that they can build an answer rather than only receive one.

training should expose the judgment that experience used to hide

Experienced people often say that juniors need judgment without being honest about how much of our own judgment came from seeing the work fail, fixing it and having someone explain why. If AI removes some of those slow repetitions, the company has to make the reasoning visible on purpose instead of hoping experience will eventually leave the right scars.

I would begin with strong worked examples that show the decision points instead of presenting only the finished product, then have the employee explain why each choice was made, compare a good answer with a plausible bad one and repair the bad one before generating something new. As the person develops a model of the work, I would remove some of that support, introduce less familiar cases and ask them to make the call before seeing what the agent recommends.

That resembles cognitive apprenticeship, an older training idea built around making an expert's normally hidden thinking visible through modeling, coaching, scaffolding, reflection and gradually greater independence.⁵ AI can make each part more available because the employee can ask more questions, practice more variations and get quicker feedback, while a human still owns the examples, reviews the consequential work and decides when the support should fade.

Some tasks should still include bounded manual work, although the purpose is diagnosis rather than nostalgia. I do not need someone to spend a day formatting a table by hand, but I may need them to calculate a small case, read the raw rows, draft the claim before the prose agent runs or debug one path without asking for a replacement system, because those exercises reveal whether the person knows what the tool is doing and can recover when it fails.

the company has to earn the judgment it expects

Hiring for judgment cannot become an excuse for companies to demand ten years of experience from every entry-level applicant while refusing to train anyone. If the old path to expertise is being compressed, employers have to build a better path rather than blaming new workers for using the tools we gave them.

That means experienced employees have to leave behind more than approvals because a junior person cannot learn much from seeing that the final answer passed. We have to preserve the rejected version, name the assumption that failed, explain why one source was stronger than another and let new employees watch how responsibility changes the decision when the easy answer carries a real cost.

I expect the people entering work now will become capable in ways my generation could not because they can reach examples, explanations and practice almost immediately. I also expect some of them will learn to produce the appearance of expertise before they have built the understanding underneath it, especially when employers reward volume and never examine what happened after the agent answered.

The next generation does not need to repeat my fifteen-plus years without agents, but they do need enough independent knowledge, guided practice and responsibility to know when the answer is wrong before a client, an employee or the company pays for it.

Source notes

Erik Brynjolfsson, Danielle Li and Lindsey Raymond's “Generative AI at Work”, published in the Quarterly Journal of Economics in 2025, studied the staggered introduction of an AI assistant among 5,172 customer-support agents. The authors report a 15 percent average productivity increase and a 30 percent increase in issues resolved per hour among less skilled and less experienced workers, along with evidence that AI assistance helped newer workers move more quickly through the experience curve.
Greg Kestin and colleagues' 2025 randomized crossover trial compared two lessons delivered by a structured AI tutor with in-person active learning in a Harvard undergraduate physics course. The AI condition produced more than twice the median learning gain with a median 49 minutes on task, compared with a 60-minute class. The authors caution against generalizing the result to every context and describe substantial expert work in the sequence, prompts, step-by-step solutions and platform.
Judy Hanwen Shen and Alex Tamkin's January 2026 preprint “How AI Impacts Skill Formation” reports randomized experiments in which participants learned a new asynchronous programming library. The paper finds weaker conceptual understanding, code-reading and debugging performance with AI use on average, while identifying cognitively engaged interaction patterns that preserved learning. It is a recent preprint and should be read as one bounded experiment rather than a verdict on every job or tool.
Hao-Ping Lee and colleagues' CHI 2025 paper “The Impact of Generative AI on Critical Thinking” surveyed 319 knowledge workers who supplied 936 first-hand examples. Higher reported confidence in AI was associated with less critical-thinking effort, while task-specific self-confidence was associated with more. The study describes associations in self-reported work rather than a causal experiment.
Allan Collins, John Seely Brown and Susan Newman's “Cognitive Apprenticeship: Teaching the Crafts of Reading, Writing, and Mathematics” proposed making expert thinking visible through methods including modeling, coaching, scaffolding, articulation, reflection and exploration. The 1989 framework predates generative AI; its application here is my interpretation of how a company can preserve learning while changing the tool.

Earlier read: story and workflow reality: ADHD, parallel tracks, and finishing more