Knowing What Breaks Before You Write the Code

Impact analysis fails because reference lists answer at the wrong altitude. What a useful answer contains, why it must run in both directions, and how to improve this without new tooling.

Written by Vladimir Miroshnichenko·Published August 31, 2026·Updated September 4, 2026·5 min read

The most expensive moment in software is not when a change is written. It is when somebody discovers, two weeks later, what else that change touched.

Every team has a ritual for avoiding this. A senior engineer is asked "is this risky?" and answers from memory. A working group is convened. Someone greps for the field name and reads what comes back. These rituals work in proportion to how much of the system is still in somebody's head, which is a resource that leaves the company at some point.

Why "find all references" is the wrong altitude

Ask a code intelligence tool what depends on a field and it will tell you, accurately: these functions read it, these files import that module, these tests exercise this path. That answer is correct and, for the question you were actually asking, close to useless.

Because the question was never "which symbols touch this". It was:

If we change how refunds work after shipment, what will customers see differently, and what will we have to verify?

Those are different altitudes. One is about the code's structure. The other is about the product's behaviour. A reference list answers the first and leaves you to infer the second, which is exactly the inference that goes wrong — because the behaviour you are worried about is usually spread across several files that a reference list has no reason to connect.

Concretely: a refund change might touch a payment handler, an order state check, an access gate, a notification template and a report. A call graph will show you edges between some of those. It will not tell you that the notification fires on a transition that the state check no longer permits, which is the actual bug you are going to ship.

The two directions that matter

Useful impact analysis runs both ways, and most tooling only runs one.

Downstream — what does this reach? What behaviour changes for a customer, what other parts of the product are involved, what has to be re-verified. This is the direction people ask about.

Upstream — what already depends on the thing I am about to change, including things owned by other teams? This is the direction that produces the surprises, because the dependencies you do not know about are by definition the ones you did not go looking for.

In a product split across squads, upstream is where the failures live. Every squad is genuinely expert in its own area; the coupling between areas is not owned by anybody, and it is the coupling that breaks.

What a good answer contains

If you are evaluating tools on this, here is a workable standard. A useful impact answer states:

What changes for a person using the product, in product terms — not "these six files".
Which parts of the system are involved, including parts owned by other teams.
What must be verified, specifically enough to become a test plan rather than a feeling.
The evidence for each claim, traceable to code you can open.

That last one carries more weight than it looks. An impact analysis you cannot check is a second opinion from a source with no accountability. Once every claim names the file it came from, a senior engineer can audit the analysis in minutes instead of redoing it — and that shift, from producing the analysis to checking it, is where the time is actually recovered.

Where the measurement lands

Change impact is the question type where the difference between tools is easiest to see, because it is the one that most requires connecting things that live apart.

In a controlled comparison — one repository, fifty questions written the way work arrives, the model held constant, blind scoring — change-impact questions scored 91% with established product knowledge against 74% for the best alternative. The widest margin in that run was on questions about how the product behaves (94% against 75%), which is the same underlying capability seen from a different angle.

Both numbers describe the same thing: whether the answer had to be reconstructed from fragments or was already established. Reconstruction is where dependencies get dropped, and a dropped dependency is precisely what a late discovery is.

Doing this without new tooling

If you want to improve this next quarter without buying anything, three things help.

Write the impact analysis before the branch, not in the pull request. By review time the change exists and the analysis is a rationalisation of it. Before the branch, it is a scoping decision, and scoping decisions are cheap to change.

Make it a document with citations. "This touches the renewal path (billing/renew.ts:88) and the access check (auth/gate.ts:140)" is auditable. "This touches billing" is a feeling.

Record the misses. When something surfaces that the analysis did not predict, write down what it was and why it was missed. After a dozen entries you will have a map of where your system's coupling actually is — which is the map nobody has, and the reason this problem persists.

The economics

The argument for doing any of this is not tidiness. A dependency found during scoping costs a conversation. The same dependency found at integration costs a rebase and a re-review. Found in staging, it costs a release slot. Found by a customer, it costs the release slot plus an incident plus the trust.

The cost of discovery rises by roughly an order of magnitude at each of those boundaries, and every step of that curve is paid by the same team that would have been fine if the question had been answerable at the start.

The question is not hard. It is just not answerable from files, and files are what most tools have.

Put a number on your own workflow

Bring us the recurring workflow where AI still needs expensive people to supervise, review and correct. We baseline what it costs and put the target in writing before we build. Or connect a repository and see it on your own code first — 1,000 credits free, no card.

Build My Business Case → Start free →

Frequently asked questions

What is change impact analysis in software?

Working out, before a change is made, what else it will affect — what behaviour changes for users, which parts of the system are involved, and what has to be re-verified. Done well it runs in two directions: downstream, what this change reaches, and upstream, what already depends on the thing being changed.

Why isn't "find all references" enough for impact analysis?

Because it answers at the wrong altitude. A reference list tells you which symbols touch a field; the question was what customers will see differently and what must be verified. The behaviour you are worried about usually spans several files that a reference list has no reason to connect, and that gap is where late discoveries come from.

When should impact analysis happen?

Before the branch is cut, not in the pull request. At review time the change already exists and the analysis becomes a justification of it. Before the work starts, it is a scoping decision — and scoping decisions are cheap to change, while shipped ones are not.

How much does a missed dependency actually cost?

The cost rises steeply with the point of discovery. Found while scoping, it costs a conversation. At integration, a rebase and re-review. In staging, a release slot. In production, the release slot plus an incident plus the trust — roughly an order of magnitude at each boundary.

Vladimir Miroshnichenko

Founder, GitMir

Founder of GitMir, the intelligence layer that gives AI real context about a company's software. I write about AI agents, context engineering, workflow economics and keeping AI-generated work under control.

LinkedIn →

← More articles