Co-change is not proof, but it is evidence / Bob's Site

I wanted a tool to tell me where P6X4's boundaries should be. I ended up building one that was useful only after it refused to tell me.

The initial idea was appealing: inspect repository history, find files that often changed together, cluster them, and compare the clusters with the current service map. If catalogue code and rendering code co-changed in most features, perhaps they belonged together. If delivery code evolved alone, perhaps its separation was justified.

The first visualization produced confident nonsense.

One enormous cluster contained package manifests, generated clients, formatting configuration, documentation indexes, and deployment files. A second joined unrelated features because I had made several broad cleanup commits. The graph accurately represented commits and badly represented design.

History contained evidence. It did not contain an architecture waiting to be extracted.

A commit is a container, not a reason

Two files can appear in one commit for many reasons:

One feature required both.
A public contract changed and a consumer adapted.
A formatter touched the whole repository.
A dependency update regenerated lockfiles and clients.
Two unrelated repairs happened to be convenient to commit together.
A large migration deliberately bridged old and new structures.
A developer habit grouped work by day rather than intent.

My analyzer initially treated every shared commit as the same kind of relationship. That made the graph precise in the least useful sense.

I changed the unit of analysis from “files in a commit” to “candidate evidence for a change relationship.” The distinction made the tool less automatic and more honest.

I started with explicit exclusions.

Generated files, lockfiles, compiled assets, snapshots produced mechanically from one source, and vendored code were removed from the coupling graph. They mattered to releases but usually did not reveal domain cohesion. Repository-wide formatting and dependency sweeps were tagged and excluded from feature clustering.

I did not exclude tests. Tests often revealed the actual behavior a change crossed. A feature edit touching a document rule and a delivery scenario was more informative than an edit touching two implementation files in the same folder.

I mapped remaining files to several overlapping dimensions:

Repository and deployable unit.
Package or module.
Domain capability.
runtime role such as web, worker, or indexer.
persistence authority or projection.
contract surface.

That let me ask different questions of the same change. “Which services changed?” was not equivalent to “Which user concepts changed?”

I also limited pair expansion. A commit touching 200 files creates 19,900 unordered file pairs, overwhelming many small, repeated relationships. Large changes received lower weight, and changes beyond a threshold were reported separately instead of becoming graph edges.

The goal was not to hide inconvenient history. It was to stop one mechanical sweep from outweighing twelve focused features.

Time changed the meaning of a relationship

Raw co-change counts favored old, stable relationships and ignored direction.

I divided the history into rolling windows and recorded when a pair first and last appeared. Catalogue and renderer code had changed together heavily during the first P6X4 prototype. After I introduced an immutable render request, renderer internals changed several times without catalogue edits. The overall count still looked coupled; the recent trajectory looked more independent.

That difference mattered. Architecture decisions concern the system becoming, not merely the sum of everything it has been.

I used time decay in one view, but never hid the unweighted count. A half-life parameter can manufacture a trend if chosen carelessly. The interface therefore showed the recent and lifetime relationships side by side and listed the actual changes beneath every edge.

The graph was an index into history, not the conclusion.

Direction was often more useful than proximity

Symmetric co-change says that A and B changed in the same unit of work. It does not say which one drove the change.

Commit messages and issue labels in the synthetic project carried a feature category. I used them cautiously to construct a directional view:

A document feature changed the authoritative model and required search to adapt.
An adapter policy changed delivery without altering the document model.
A search ranking experiment changed only projection code.
A rendering optimization changed worker internals but not the request contract.

This did not require natural-language classification pretending to understand intent. I assigned the category while creating each scenario and could inspect the underlying patch. For older unlabelled changes, the tool displayed “unknown” rather than guessing.

The directional view exposed a common pattern. A projection may co-change with its source because it responds to new source fields, while the source never changes because of the projection. That is dependency, not necessarily failed cohesion.

Conversely, two services that repeatedly negotiate one shared rule may be peers around a missing authority. Their symmetric coordination is more troubling.

I measured surprise at the boundary

The service map made a prediction: changes inside a boundary should be more common than changes across it, after accounting for broad work.

I calculated a simple boundary surprise ratio for each capability pair:

observed focused changes crossing boundary
------------------------------------------
expected crossing changes from base rates

The denominator prevented large modules from looking coupled merely because they changed often. I did not promote the ratio as a universal metric. It helped rank boundaries for inspection.

A high score opened a page showing:

The focused changes contributing to it.
Files and tests involved.
Contract versions changed.
Whether releases were coordinated.
Whether the change introduced temporary compatibility.
The user intent behind the scenario, when known.

The page also showed counterexamples: recent changes that stayed cleanly within each side. Without those, I could tell an overly simple consolidation story.

Co-change had several innocent explanations

The renderer and document module initially crossed boundaries often. Consolidation looked obvious until I separated causes.

Some changes reflected a genuinely shared invariant: a document revision and template version needed an atomic relationship. Some reflected an immature contract: callers passed a broad mutable document object instead of an immutable render request. Others reflected the deliberate migration from one template engine to another, a temporary period when both sides had to evolve together.

Only the first category strongly argued for shared authority. The second argued for a better interface. The third argued for finishing the migration.

The same visual edge could therefore produce three different actions.

I added a review worksheet to each suspicious boundary:

Name three representative co-changes.
Classify the source: invariant, contract evolution, migration, mechanical work, or unknown.
Identify whether one side can change independently today.
Check scale, authority, recovery, and consistency needs.
Propose the smallest intervention.
Define evidence that would reverse the decision.

This kept “the graph says merge” out of architecture discussions with myself.

Lack of co-change was also ambiguous

An edge with little activity could mean healthy independence. It could also mean abandonment.

P6X4's notification service rarely changed because the first implementation was simplistic and no scenario exercised its edge cases. Its low coupling score looked excellent. Failure injection revealed that it silently lost messages after one class of restart.

Another service changed rarely because its contract was so awkward that I avoided features involving it. Historical stability was organizational friction encoded as absence.

I added activity context to every boundary: change frequency, open failure scenarios, ignored tests, time since the last contract exercise, and whether a feature had deliberately bypassed it. A quiet component was not presumed healthy.

This was one reason the analyzer could never be authoritative. Repository history records performed work, not work avoided, missing requirements, or incidents never represented in code.

My synthetic history was intentionally tidy enough to study, but even it contained fixups and mixed work. Real histories would be messier.

I tested three ways of grouping changes:

Individual commits.
Pull-request or branch groups reconstructed from merge metadata.
Manually tagged feature episodes spanning several branches and releases.

Individual commits fragmented changes when I committed incrementally. Squashed merge groups sometimes combined refactoring and behavior. Feature episodes best matched intent but required deliberate metadata.

The interface allowed switching between them. If a conclusion existed only under one grouping strategy, I treated it as weak.

This produced a useful secondary lesson: version-control history is an input to later reasoning. Small, coherent changes and clear descriptions make that evidence better. I did not contort current work to satisfy a future graph, but I stopped treating commit messages as disposable narration.

The most valuable screen was not the cluster map. It was a timeline for one boundary.

For document and rendering, it showed:

Early shared schema edits.
A broad request object added for convenience.
Repeated coordinated deployments when fields changed.
An incident scenario caused by mismatched template revisions.
Introduction of an immutable request contract.
Several renderer-only optimizations.
Consolidation of the authoritative state while preserving a worker process.

The story revealed that “merge” and “separate” were not stable opposites. The authority moved together, the contract narrowed, and execution retained resource isolation. A file cluster alone could not express that result.

For search, the timeline showed source-driven co-change followed by projection rebuilds and ranking-only changes. The boundary stayed because lag, alternative storage, and replaceability were part of its design.

The analyzer made both decisions explainable without making them identical.

I tested the analyzer with planted counterexamples

Because P6X4 was a laboratory, I could construct histories that should fool simplistic rules.

I created:

Two independent modules changed together by a formatting sweep.
One tangled module split into files that had not changed recently.
A stable contract with clients adapting to new optional capabilities.
A migration that temporarily touched every layer.
A projection that followed source changes directionally.
Two services sharing one invariant through compensating events.

The initial tool misclassified all six. The revised tool did not “classify” them at all. It surfaced different evidence and required the boundary worksheet.

That felt like a reduction in ambition. It was actually the useful breakthrough. Architecture tools become dangerous when uncertainty is erased from the output.

The output was a queue of questions

At the end of the audit, the analyzer produced no proposed module map. It produced ranked questions:

Why did these two capabilities coordinate nine focused changes?
Why has this supposedly independent service never released alone?
Why does this projection change only when its source changes?
Why is this quiet boundary absent from failure scenarios?
Why did coordinated changes decline after contract version two?
Why does one feature repeatedly cross authorization rules?

Some questions led to consolidation. Others led to narrower contracts, better tests, a completed migration, or no change.

I preserved the tool in P6X4 as a periodic architecture review, not a continuous score developers had to optimize. Scores invite gaming and imply a target. The questions were useful only when a real maintenance or reliability cost made the boundary worth revisiting.

Co-change is attractive because it is countable. Architecture contains many important things that are not present in a repository: regulatory authority, human ownership, failure containment, latency budgets, data sensitivity, and anticipated product direction.

I did not abandon the graph because it was incomplete. I stopped asking it to be complete.

Repeated, focused co-change can challenge a claim of independence. Direction can distinguish a source from its projection. Declining coordination can show that a new contract is working. Silence can suggest independence or neglect. Every signal needs the changes beneath it and the constraints around it.

The analyzer became valuable when it left room for those interpretations.

History did not draw P6X4's architecture. It supplied evidence that made several confident lines harder to defend.

Visual distance could manufacture a conclusion

Graph layout created its own hazards. A force-directed view placed dense clusters close together and sparse nodes at the edge. It looked like a map of natural modules even when small parameter changes moved nodes dramatically.

I kept the view for exploration but removed it from the primary review. The main interface used a matrix with capabilities on both axes. Each cell showed focused co-changes, directional source changes, recent trend, and the number of underlying episodes. Sorting and thresholds were explicit. Clicking a cell always opened the evidence.

I added a comparison mode that held the same history constant while varying exclusions, grouping strategy, and time window. If a boundary changed from “strong” to “weak” because generated clients were included, the visual made that sensitivity obvious.

Colors used discrete, labelled ranges rather than an impressive continuous heat map. The page worked without color, and exact counts remained available. A small relationship supported by two changes did not receive the same visual confidence as one supported by twenty.

The goal was to make uncertainty inspectable. A beautiful cluster diagram had made one interpretation feel discovered instead of chosen.

File paths were not stable architectural identities. During P6X4's consolidation, code moved from service repositories into packages. A path-based analyzer interpreted the move as one component disappearing and an unrelated component appearing.

I introduced lineage records for significant moves. Git's similarity detection provided a useful starting point, but splits and rewrites needed manual mapping at the capability level. The analyzer tracked a logical unit such as document/render-contract across its old repository and new package location.

This let the timeline distinguish two questions:

Did the code move?
Did the responsibility or public contract change?

A mechanical relocation should not look like sudden decoupling. Conversely, leaving code in the same folder should not hide a meaningful change in authority.

I resisted building a universal code-identity engine. For the limited P6X4 history, a checked-in mapping file made important lineage explicit and reviewable. Unknown moves stayed unknown. The tool displayed gaps rather than joining files based on weak similarity.

That choice followed the same principle as the rest of the analyzer: assisted inspection was valuable; silent certainty was not.

I compared change relationships with traces from deterministic scenarios. The sources answered different questions.

Repository history showed what I had edited together. Scenario traces showed which capabilities participated in one user intent and where latency or failure propagated. A document subtitle feature co-changed across catalogue and renderer, while the runtime also touched search and audit. Search participated in the flow but remained a lag-tolerant projection. Runtime proximity alone would have merged too much.

Failure injection made the comparison more useful. If disabling one component caused an otherwise unrelated intent to fail, the trace exposed operational coupling. Sometimes the cause was legitimate authority. Sometimes it was a synchronous call added for convenience, or a shared connection pool that contradicted the deployment diagram.

I never combined the sources into one architecture score. Their disagreements were the interesting output:

High co-change and low runtime dependency suggested contract evolution or build-time coupling.
Low co-change and high failure propagation suggested hidden shared infrastructure or a neglected path.
High values in both deserved an explicit explanation of why the boundary remained.
Low values in both supported independence but did not prove value.

The cross-view prevented code history from becoming the only form of evidence simply because it was easy to count.

A review ended with a reversible decision

Each boundary review produced a short decision record. It named the observed cost, evidence, intervention, owner, review date, and reversal condition. The intervention could be smaller than moving code: narrow a contract, stop a coordinated release, isolate a pool, make authority explicit, or add a failure scenario.

For consolidation candidates, I tracked whether the predicted improvement appeared. Did a representative feature touch fewer contracts? Could one transaction replace a compensation path? Did local setup lose a dependency? Did failure remain contained where intended?

If those outcomes did not materialize, “the graph suggested it” was not a defence. The decision could be reversed before the old path was removed.

This made the analyzer part of a learning loop rather than an architecture linter. Its success was not the number of boxes deleted. It was the number of vague boundary claims converted into testable reasons.

The method remained deliberately modest. A history can reveal patterns of work, and tools can make those patterns hard to ignore. The person reviewing them still has to understand the domain, the future being considered, and the costs the repository never recorded.