How AI Coding Tools Boost Productivity While Impeding Novice Developer Learning
AI coding assistants provide substantial productivity gains to novice software developers. But do these gains come at the cost of genuine skill development? This is one of the most consequential open questions in computing education and workforce development.
Shen et al. (2026) document that junior developers receive disproportionately large productivity boosts from AI tools, yet explicitly identify the effect on skill formation as unknown. Our work addresses this gap through computational cognitive modeling.
Skills consolidate through active recall and application. AI tools that provide ready-made solutions may bypass this retrieval process (Bjork & Bjork, 1992).
Moderate challenge during practice enhances long-term retention, even at the cost of immediate performance. AI reconfigures this trade-off (Bjork, 1994).
Declarative knowledge becomes procedural through practice. If AI handles the procedural step, the compilation process is interrupted (Anderson, 1982).
We simulate a 12-month, three-arm randomized trial with 80 novice developers per condition (240 total). Each developer encounters 5 coding tasks per day over 252 working days, with monthly tool-removed skill assessments.
Developers work without any AI assistance. Full cognitive engagement on every task. This is the baseline for skill development.
Full access to AI coding assistant with passive acceptance behavior. Difficulty is reduced; cognitive processing depth drops to ~15%.
AI access with mandatory engagement: developers must read, modify, and explain AI output before proceeding. Processing depth maintained at ~70%.
| Dimension | Description | AI Weight |
|---|---|---|
| Syntactic Fluency | Writing correct code from specifications | 0.80 |
| Algorithmic Reasoning | Solving novel computational problems | 0.50 |
| Debugging | Locating and fixing defects | 0.35 |
| Code Comprehension | Reading and predicting code behavior | 0.25 |
| Architectural Judgment | System-level design evaluation | 0.15 |
| Autonomous Learning | Learning new frameworks independently | 0.10 |
| Condition | Initial Skill | Final Skill | Growth | Cohen's d vs Control |
|---|---|---|---|---|
| Control (No AI) | 0.238 | 0.643 | +0.404 | |
| Unrestricted AI | 0.228 | 0.562 | +0.334 | -1.04 |
| Scaffolded AI | 0.236 | 0.641 | +0.405 | -0.04 |
The impact of AI is not uniform across skill dimensions. Skills that AI automates most effectively suffer the greatest impairment under unrestricted use.
| Dimension | AI Wt | Control | Unrestricted | Scaffolded | d (Unr.) | d (Scaf.) |
|---|---|---|---|---|---|---|
| Syntactic Fluency | 0.80 | 0.651 | 0.390 | 0.650 | -5.10 | -0.02 |
| Algorithmic Reasoning | 0.50 | 0.648 | 0.566 | 0.660 | -2.07 | +0.34 |
| Debugging | 0.35 | 0.666 | 0.615 | 0.647 | -1.28 | -0.59 |
| Code Comprehension | 0.25 | 0.662 | 0.620 | 0.649 | -1.21 | -0.42 |
| Architectural Judgment | 0.15 | 0.664 | 0.648 | 0.656 | -0.44 | -0.22 |
| Autonomous Learning | 0.10 | 0.566 | 0.535 | 0.582 | -0.72 | +0.30 |
The Spearman correlation between AI automation weight and the unrestricted AI effect size is -0.94, confirming that AI most impairs the very skills where it provides the most help.
The central finding is a dissociation between observed productivity and underlying skill. Unrestricted AI users appear more productive in daily work, yet possess weaker skills when assessed without AI tools. This creates a "dependency trap" invisible under continued AI access.
By systematically varying the cognitive processing depth parameter, we identify the threshold at which AI assistance transitions from skill-harming to skill-enhancing.
At processing depth 0.15: AI HARMS skill formation. The skill delta is negative.
The crossover occurs at ~0.75. Below this, AI reduces net skill development. Above it, the benefits of reduced difficulty and increased success rate outweigh the cost of reduced cognitive effort.
Incorporate scaffolding features that promote active engagement: explain-before-accept prompts, modification requirements, and progressive withdrawal of assistance as skills develop.
Supplement AI-assisted productivity metrics with periodic tool-removed skill assessments. The gap between measured productivity and genuine skill is a hidden organizational risk.
Integrate AI tools into curricula with explicit scaffolding protocols rather than unrestricted access. Teach students to evaluate and modify rather than merely accept AI output.
Prioritize empirical studies that disentangle productivity from skill, measure multiple skill dimensions, and test engagement-mode interventions. We recommend a Randomized Longitudinal Skill Assessment (RLSA) design as the most direct path to validating these predictions.
AI-induced skill deficits are largest for syntactic/algorithmic skills and smallest for architectural/meta-cognitive skills.
Active engagement protocols substantially reduce or eliminate the skill deficit across all dimensions.
Tool-removed assessments reveal skill gaps invisible in AI-assisted performance metrics.
Interventions pushing processing depth above ~0.75 flip the AI effect from negative to positive.