Why AI Gets Your Own Codebase Wrong
Your assistant is excellent at code it has never seen and unreliable about code you wrote yourself. The cause is structural: the distance between the way you asked and the form the tool can compute over.
Your AI assistant is excellent at code it has never seen and unreliable about code you wrote yourself. That inversion feels wrong, and it is the single most common complaint engineers have about AI tools in 2026. The model can explain a sorting algorithm perfectly and then tell you, with total confidence, that changing a field in your billing code is safe when it is not.
The usual explanations — the context window is too small, the model needs a better prompt, retrieval needs tuning — all describe symptoms. The cause is simpler and more structural: there is a distance between the way you asked the question and the form the tool can compute over, and everything that goes wrong happens while crossing it.
The journey nobody talks about
Consider a request that arrives in a real team, from a real person:
"What happens if we let customers pause a subscription instead of cancelling?"
That sentence contains no filenames. It names no functions, no classes, no services. It is expressed entirely in the vocabulary of the product — customers, subscriptions, pausing, cancelling — because that is the vocabulary the person thinks in.
Now watch what a code-search-shaped tool has to do with it. It cannot compute over "pause a subscription". It computes over symbols. So it translates: find files matching subscription, retrieve the top-scoring chunks, pull in what they import, assemble the pile, hand it to a model, and ask the model to reconstruct — from those fragments — what the product actually does when a subscription is paused.
Every step in that translation is lossy, and the loss is not random. It systematically drops the thing you asked about: the behaviour that spans files. Renewal lives in one place, invoicing in another, access checks in a third, and the emails that fire on a state change somewhere else entirely. Retrieval scores each fragment on its resemblance to your words. Nothing scores the relationship between them, which is the entire content of your question.
Distance is the variable
It is useful to give this a name, because once you can name it you can measure it. Call it semantic distance: how far the information a system works from sits from the intent the question was asked in.
Ranked from furthest to nearest:
- Raw files. Everything, unsorted. The model reads implementation and infers intent, which is a guess dressed up as an answer.
- Symbols and call graphs. Precise about the code, silent about the product. A call graph will tell you that
applyPausecallsrecalculateBilling. It will not tell you that a paused subscription must not send a renewal notice. - Retrieved snippets. Fragments that resemble the question. Meaning is reconstructed from scratch on every request, differently each time, privately, with nothing checking the reconstruction.
- Established product knowledge. What the product does and how it behaves, worked out once and reused. Almost no distance left to travel.
The prediction that falls out of this is specific and testable: the further a system starts, the more reasoning it spends reconstructing rather than answering, and the more often it gets the answer wrong. Not slower — wrong, because reconstruction is where the errors are introduced.
What the measurement showed
We ran that test. One repository, fifty questions written the way work is actually assigned, the model and its settings held constant, answers scored blind against verified repository evidence. The only variable was what each system supplied to the model.
| What the model was given | Answer quality | Context used | Cost per question |
|---|---|---|---|
| Established product knowledge | 92% | 8.4K tokens | $0.07 |
| Retrieval, best in test | 78% | 27.4K tokens | $0.19 |
| Retrieval, second | 74% | 31.6K tokens | $0.22 |
| Code-graph tooling | 69% | 38.1K tokens | $0.26 |
| The raw repository | 58% | 61.3K tokens | $0.41 |
Read the middle column first. The systems that started furthest away used the most context and scored the least. That is the shape you would expect if the extra tokens were being spent on reconstruction rather than on the answer — and it is the opposite of the "just give the model more context" instinct.
The gap is also not uniform across question types. Broken out by what the question demanded, the lead over the best alternative was widest on questions about how the product behaves and narrowest on reasoning about failures:
- Product understanding — 94% against 75%
- Multi-step behaviour — 93% against 73%
- Change impact — 91% against 74%
- Failure reasoning — 88% against 76%
Those middle two are exactly the questions whose answers live between files. Which is the point.
Why more context stopped helping
There is a version of this problem every team has already hit. You gave the agent a bigger window. You pasted in more files. The answers did not get better, and the bill did.
That is what a distance problem looks like from the inside. Adding fragments does not shorten the journey from intent to computable form — it lengthens it, because now there is more to sift. The model spends its reasoning budget deciding what is relevant before it can start on what was asked. Past a point, additional context is not additional information; it is additional noise with a price tag.
The fix is not more. It is nearer.
What this means practically
If you are evaluating tools, the useful test is not "can it find this function". Every tool in this market can find the function, and a benchmark built on lookups plays on the vendor's home ground. Ask instead:
- What could break if we change how refunds work after shipment?
- Why can a customer be charged and still not get access?
- What does a new engineer need to know before touching availability?
Then check two things about the answer. Is it about the product, or is it a summary of files? And can you follow it back to the code it came from, or are you being asked to take it on trust?
If you are building, the lever is the same one: reduce what the model has to reconstruct. Every fact about your system that is established once and reused is a fact the model never has to re-derive, and re-derivation is where both your token bill and your error rate live.
The model was never the limit. What it knows about your product is.
Put a number on your own workflow
Bring us the recurring workflow where AI still needs expensive people to supervise, review and correct. We baseline what it costs and put the target in writing before we build. Or connect a repository and see it on your own code first — 1,000 credits free, no card.
Build My Business Case → Start free →Frequently asked questions
Why does AI give wrong answers about my own codebase but good answers about general programming?
General programming knowledge is in the model's weights, so it is available directly. Knowledge about your product is not — it has to be reconstructed on every request from whatever fragments the tool retrieved. Reconstruction is lossy, and it loses exactly the thing most questions are about: behaviour that spans several files. The model is not weaker on your code; it is working from a much worse starting position.
Will a bigger context window fix AI's understanding of my codebase?
Not on its own. In a controlled test the systems that used the most context scored the lowest — 61.3K tokens for 58% correct answers, against 8.4K tokens for 92%. Extra fragments lengthen the work of deciding what is relevant before the model can begin answering. What helps is starting nearer to the question, not supplying more raw material.
What is semantic distance in AI code tools?
The gap between the way a person asks a question — customers, plans, money, consequences — and the form a tool can actually compute over, such as files, symbols or retrieved text. The larger that gap, the more reasoning is spent translating rather than answering, and the more often the answer is confidently wrong.
How do I test whether an AI tool really understands my product?
Ask questions in the words work arrives in, not in the names of classes. "What breaks if we allow subscriptions to be paused" is a real test; "find the definition of SubscriptionService" is not, because every tool in the market passes it. Then check whether the answer is about the product rather than about files, and whether every claim traces back to code you can open.
Related articles
MCP, Explained for Engineering Teams
What the Model Context Protocol actually is, why it turns an N×M integration problem into N+M, and the question to ask a vendor that 'we support MCP' does not answer.
Ask Questions, Not Queries
Most engineers talk to AI the way they talk to a search box, and lose answers they could be getting today. Six question patterns worth copying, and how to tell a good answer from a fluent one.
The Comprehension Economy
Developers spend 58% to 70% of their time understanding code, not writing it. The industry spent three years optimising the smaller half — and increased the load on the constraint.