Retention Curves Don't Lie: What 18 Months of AI Coding Tool Data Actually Shows

In February 2025, METR published a study that should have been a wake-up call. Experienced open-source developers — people with years of contribution history to the specific repositories they were working on — were given tasks with and without access to AI coding assistants including Cursor Pro and Claude 3.5 Sonnet.

The result: developers using AI completed tasks 19% slower than those working without it. Not faster. Slower.

The truly striking finding wasn't the speed result. It was that the same developers predicted beforehand that AI would make them 20% faster. That's a 39 percentage point gap between perception and reality. Developers didn't just fail to get faster — they fundamentally misperceived their own productivity while using these tools.

This single data point reframes the entire AI coding tools market. Not because the tools are useless — they clearly aren't, given adoption rates — but because the most common measure of their value (developer self-report) is unreliable. And if you can't trust the main signal, you need better data.

Eighteen months of retention curves, code quality metrics, and financial data provide that better data. Here's what it actually shows.

The Adoption Numbers Everyone Cites

Let's start with the topline metrics because they're genuinely impressive and they're not wrong — they're just incomplete.

GitHub Copilot remains the most widely adopted AI coding tool, with over 15 million developers using it as of late 2025. GitHub reports a roughly 30% acceptance rate on suggestions — meaning developers accept about one in three completions offered.

Cursor has been the breakout story. The AI-native code editor reportedly scaled from $100M to $2 billion in ARR in approximately 15 months, raised at a $10 billion valuation, and became the default editor for a generation of developers who started coding with AI assistance as the baseline.

Other entrants have found traction in narrower lanes. Codeium (now Windsurf) focuses on enterprise deployments. Amazon CodeWhisperer is bundled with AWS. Tabnine targets regulated industries that need on-premises AI. Sourcegraph Cody focuses on codebase-aware AI assistance.

The total addressable market for AI coding tools is estimated at $45 billion by 2028, growing at 35%+ annually. By any standard metric — adoption rate, revenue growth, market expansion — this is a healthy and rapidly scaling category.

But adoption and retention are different things. And retention and value creation are different things again.

The Retention Divergence

The most important chart in AI coding tools isn't revenue growth — it's the retention curve split between individual and enterprise customers.

Individual developer subscriptions to AI coding tools show approximately 16% monthly churn. That means roughly one in six paying individuals cancel each month. Over a year, that's a retention rate around 14% — for every 100 developers who sign up in January, only about 14 are still paying in January of the following year.

Enterprise accounts tell a different story entirely: roughly 1% monthly churn, which translates to about 89% annual retention. That's in line with best-in-class SaaS benchmarks.

The 16x gap between individual and enterprise churn rates is the single most revealing data point in this market. It tells you several things:

Individual developers are experimenting, not committing. The low friction of a $20/month subscription means developers try the tool for a project, hit its limitations, cancel, and possibly return later when the tool improves. This creates a "revolving door" pattern rather than a true adoption curve.

Enterprise adoption is sticky for non-product reasons. When a company rolls out Copilot or Cursor to its engineering team, the procurement process, IT setup, and workflow integration create switching costs that don't exist for individual users. A developer who chose Copilot personally can switch to Cursor in five minutes. An enterprise that deployed Copilot across 500 seats has a six-month migration project.

The product's value proposition is stronger for teams than individuals. This is counterintuitive — you'd expect a tool that helps you write code to be equally valuable regardless of context. But the data suggests that AI coding tools provide compounding value in team settings: shared context, consistent code patterns, accelerated code review, and reduced onboarding time for new team members.

Cursor appears to be the exception with the lowest individual churn in the category, likely because its AI-native editor approach creates a form of lock-in that plugins to existing editors (like Copilot in VS Code) don't. When the AI is the editor rather than an add-on to the editor, switching means changing your entire development environment rather than just toggling an extension.

The Code Quality Crisis

While retention data tells you about perceived value, code quality data tells you about actual value. And the signals here are concerning.

GitClear analyzed millions of lines of code across thousands of repositories and found that code churn — code that is rewritten or reverted within two weeks of being committed — increased from 3.1% to 5.7% as AI coding tool adoption grew. That near-doubling of churn suggests that developers are committing AI-generated code that doesn't survive contact with production, testing, or code review.

The GitClear analysis also found that the ratio of "moved" code (copy-paste-style duplication) increased significantly, while the ratio of new, original code decreased. In plain terms: AI tools are generating more duplicated code and less novel code. That's a codebase health concern that compounds over time through increased maintenance burden.

Security data paints a similar picture. Apiiro's research identified a roughly 10x increase in vulnerability introduction rates in codebases with heavy AI code generation. A Stanford study found that developers using AI assistants produced more security vulnerabilities while simultaneously rating their code as more secure than developers working without AI. The confidence-competence inversion is particularly dangerous in security contexts.

DORA metrics — the standard framework for measuring software delivery performance — showed a 7.2% decline in delivery stability across teams that adopted AI coding tools in 2024. The decline was driven not by deployment frequency (which increased) but by change failure rate and mean time to recovery. Teams were shipping faster but breaking more things.

These findings don't mean AI coding tools are net-negative for code quality. They mean the default mode of adoption — let developers accept suggestions without additional quality gates — produces measurable quality degradation. The teams that pair AI tools with enhanced code review, automated testing, and AI-specific linting rules report neutral-to-positive quality outcomes. But that requires deliberate process investment, not just tool adoption.

What the Perception Gap Means for Product Builders

The METR study's 39-point perception gap — developers think AI makes them 20% faster when it actually makes them 19% slower — deserves deeper analysis because it affects how every AI product company should think about measuring value.

The gap likely exists because AI coding tools provide intense psychological satisfaction even when they don't improve objective performance:

Reduced cognitive effort feels like increased speed. When an AI writes a boilerplate function that you would have typed from memory, it feels like saved time. But you already knew the code. The writing wasn't the bottleneck — the thinking was. Studies in cognitive load theory show that reducing effort and increasing output are perceived similarly even when they're not the same thing.

Context-switching masquerades as productivity. AI tools make it easy to jump between tasks — "write this function, now write those tests, now draft that PR description." The fluid task-switching feels productive. But research on attention residue shows that rapid task-switching reduces quality on each individual task. The developer feels like they did more; the commit history shows they revisited and rewrote more.

The acceptance rate illusion. GitHub reports Copilot has a 30% acceptance rate. But accepting a suggestion isn't the same as that suggestion being valuable. Developers often accept a suggestion, modify it, and move on. The modification might be trivial (changing a variable name) or significant (rewriting the logic). The acceptance rate counts both as "accepted," overstating the tool's contribution.

For product builders, the implication is: stop relying on user sentiment surveys to measure AI tool value. Instrument your product to measure objective outcomes — time to task completion, code that survives code review without changes, code that doesn't generate bugs within 30 days, and time between commit and deploy. If the objective metrics tell a different story than the NPS survey, trust the metrics.

The Financial Underpinnings

Developer tools are a uniquely attractive market for AI companies because developers have high willingness-to-pay, low price sensitivity relative to value delivered, and organizational influence that can drive bottom-up adoption to top-down contracts.

But the financial data reveals an increasingly bifurcated market:

Category leaders are posting extraordinary numbers. Cursor's $2B ARR at $10B valuation implies a 5x revenue multiple — modest by SaaS standards but extraordinary for a company that was at $100M ARR just 15 months prior. GitHub Copilot contributes an estimated $2B+ in ARR to GitHub's parent Microsoft. The top two players alone command roughly $4B in recurring revenue.

Everyone else is fighting for scraps. The combined ARR of all other AI coding tools — Codeium/Windsurf, Tabnine, Cody, CodeWhisperer, Replit's Ghostwriter — is estimated at under $500M. In a winner-take-most market, being in third place with 5% market share is a fundamentally different business than being in first place with 40%.

The retention data explains the financial bifurcation. Category leaders benefit from a flywheel: more users generate more code context data, which improves suggestion quality, which improves retention, which generates more users. This flywheel has a minimum scale threshold — you need enough users in enough codebases to train meaningfully better models. Once a leader clears that threshold, followers face a structural data disadvantage.

Enterprise contracts amplify the gap. When a Fortune 500 company evaluates AI coding tools, it typically pilots two or three and selects one for standardization. The winner gets a multi-year contract covering thousands of seats. The losers get nothing. Enterprise sales in developer tools are not "we'll use a bit of everything" — they're "we pick one and roll it out." This creates a power law where the top two vendors capture 80%+ of enterprise revenue.

The Honest Assessment: What AI Coding Tools Are Good and Bad At

Eighteen months of data points to a nuanced picture that neither the enthusiasts nor the skeptics get right.

What AI coding tools are genuinely good at:

• Boilerplate generation — writing CRUD operations, API endpoints, data models, and repetitive patterns where the logic is well-known and the implementation is rote
• Code translation — converting between languages, frameworks, or API versions where the semantic mapping is well-defined
• Test generation — writing unit tests for existing code, where the function signature and expected behavior provide clear constraints
• Documentation — generating docstrings, README sections, and inline comments from code context
• Code review assistance — identifying potential issues, suggesting improvements, and explaining unfamiliar code

What AI coding tools are genuinely bad at:

• Architecture decisions — choosing between design patterns, structuring module boundaries, or designing data models for novel domains
• Complex debugging — tracing issues that span multiple services, involve race conditions, or require understanding production behavior
• Performance optimization — identifying bottlenecks and implementing fixes that require understanding of memory models, caching behavior, or database query planning
• Security-sensitive code — authentication flows, cryptographic implementations, authorization logic, and input validation where errors are high-consequence
• Novel algorithm development — implementing approaches that don't have close analogues in training data

The pattern is that AI tools excel at high-frequency, well-defined, previously-solved tasks and struggle with low-frequency, ambiguous, novel tasks. This maps directly to Dreyfus's model of skill acquisition: AI tools can automate the "novice" and "advanced beginner" levels of coding work but cannot yet perform at the "competent," "proficient," or "expert" levels.

The retention implication is that developers who primarily do work in the "good at" category — junior developers, full-stack generalists, developers in agencies — will see sustained value and retain well. Developers who primarily do work in the "bad at" category — senior backend engineers, infrastructure specialists, security engineers — will see diminishing returns and churn faster.

What 2026 Will Reveal

The next twelve months will determine whether AI coding tools mature from a productivity feature into a platform shift. Three indicators to watch:

Enterprise renewal rates from the first wave. Companies that signed initial Copilot or Cursor enterprise contracts in 2024 will face renewals in 2025-2026. If renewal rates exceed 90%, the enterprise value proposition is real. If they drop below 80%, it signals that initial enthusiasm didn't survive measured evaluation. Early signals from GitHub's enterprise metrics suggest renewal rates above 90%, but the sample is still small.

Code quality metrics at scale. As more companies instrument their CI/CD pipelines to measure AI's impact on code quality, we'll get the large-sample data that METR's small study hinted at. If code churn and vulnerability rates stabilize as teams develop AI-specific workflows, the quality concerns are a process problem. If they continue rising, it's a technology problem.

The Cursor-Copilot convergence. Cursor's strength is the AI-native editor; Copilot's strength is the GitHub ecosystem integration. Both are moving toward each other's turf — Cursor is building collaboration features, and GitHub is making Copilot more deeply integrated into the editor experience. Whether the market sustains two category leaders or converges to one will tell us whether AI coding tools are a feature or a product.

The retention curves will tell the story before the revenue numbers do. In SaaS, retention is a leading indicator of everything — revenue growth, expansion potential, competitive defensibility, and long-term unit economics. The companies whose retention curves flatten into a stable horizontal line at 12+ months have found product-market fit. The ones whose curves keep declining have found product-market interest, which is a different and much less valuable thing.

Eighteen months of data doesn't tell us whether AI coding tools are good or bad. It tells us something more useful: exactly how good, for whom, under what conditions, and at what cost. The companies and teams that read the data clearly — rather than the press releases — will make better adoption decisions. The rest will keep believing they're 20% faster while the git log tells a different story.