The error stream should tune the machine / Bob's Site

The most interesting reliability result I read this week came from a machine whose errors cannot be inspected in the ordinary way.

In Reinforcement learning control of quantum error correction, Google Quantum AI reports a control framework that reuses quantum error-detection events as a learning signal. A decoder still uses those events to infer corrections to logical quantum information. In parallel, a reinforcement-learning agent uses the stream to adjust the analogue controls that help cause future errors.

One signal now supports two different jobs: correct the present and improve the conditions of the future.

That separation is the part I want to keep.

I am not treating a superconducting quantum processor as a metaphor for a web service. The physics, observability limits, timescales, and error models are different. But the control shape exposes a weakness in many software reliability systems. They recover from symptoms repeatedly while leaving the machine that produced them unchanged.

Correction and adaptation are different loops

Quantum error correction uses redundancy and parity checks to turn analogue noise into discrete error-detection events without directly measuring the protected logical state. Those events do not simply name an exact broken qubit. A decoder interprets their pattern and determines a likely correction.

Correction preserves the logical computation. It does not answer why the error occurred.

Some errors arise from unavoidable interaction with the environment. Others are influenced by imperfect calibration and drift in the frequencies, amplitudes, and phases of the control signals driving the processor. Traditional recalibration interrupts computation. That is an awkward bargain if a useful algorithm eventually needs to run continuously for days or months.

The new work closes another loop. Error-detection events feed an RL controller that steers more than a thousand control parameters while quantum error correction continues. The published experiment on the Willow processor reports a 2.4-fold improvement in logical-error-rate stability against injected drift, rising to 3.5-fold when decoder steering is included. After extensive conventional and human calibration, RL fine-tuning produced roughly 20 percent further suppression of the logical error rate.

The distinction matters: the decoder repairs the logical account of what happened; the controller changes the conditions under which the next cycle happens.

Correction

Preserve the current computation

Interpret error events, infer the likely fault, and restore the logical state without claiming that the physical source has been removed.

Adaptation

Reduce future error pressure

Treat the same event stream as evidence of drift, then tune bounded controls while the computation continues.

Most production systems implement the first loop. Retries, replicas, checksums, circuit breakers, reconciliations, and compensating work preserve service or repair state. They are essential. They can also normalize a machine that is continuously degrading.

If a retry succeeds, the request is counted as healthy. If a replica answers, the user sees no outage. If reconciliation repairs the row, the invariant passes. The correction layer did its job, but it may have hidden a growing tax in latency, capacity, or risk.

The error stream should sometimes teach the machine, not merely disappear into recovery metrics.

The detector must remain independent

An adaptive controller creates a dangerous incentive: optimize the signal instead of the condition.

A service could reduce reported errors by loosening a timeout, dropping hard requests, sampling fewer checks, or changing a classifier threshold. The dashboard improves while user outcomes deteriorate. A learned controller is especially good at finding these gaps because it searches the policy rather than following the engineer's intended story.

The quantum paper addresses the analogous concern experimentally. The controller's surrogate objective uses error-detection probabilities, but the researchers evaluate logical performance separately. They report lower logical error rates, not only quieter detectors. The independent outcome check is what turns signal suppression into evidence of improved control.

For a software controller, I would keep three layers distinct:

The detector emits a stable, versioned account of symptoms and outcomes.
The controller proposes bounded parameter changes using those signals.
The verifier measures user or system outcomes through a path the controller cannot redefine.

If the controller can change the detector, the objective, and the verifier together, the system has no external standard left.

One stream can have two consumers

The design breakthrough for me is not “put RL in operations.” That is too broad to be useful. It is this narrower pattern: preserve an event stream with enough semantic quality that a fast correction loop and a slower adaptation loop can consume it independently.

The correction loop is latency sensitive. It works with current state and must remain predictable. The adaptation loop accumulates evidence, estimates drift, compares candidate controls, and changes only a constrained surface. It may operate in shadow, canary, or scheduled activation. Its output is another proposal until policy and verification allow it into the live controller.

The two loops should not share every clock. Correction may happen in milliseconds. Drift estimation may require minutes or days. Promotion may wait for representative traffic. Rollback needs a known-good control revision, not a model's memory of what it changed.

This is where a trace like N17Q becomes part of control rather than an observability accessory. Every proposed adjustment needs input evidence, previous parameter revision, eligible range, activation scope, verifier result, and rollback receipt. Without that lineage, an adaptive machine can improve and nobody can explain why, then regress and nobody can reconstruct how.

Drift is a state, not a series of incidents

Traditional incident systems are organized around thresholds and episodes. Something crosses a line, an alert opens, someone restores service, and the incident closes. Slow drift often lives below each threshold while consuming margin.

An adaptive loop needs a state estimate. It asks whether the relationship between control and outcome is changing, not only whether today's value is bad. It should separate a step change from gradual movement, local faults from shared environmental changes, and genuine drift from a new workload mix.

That requires context the error counter does not carry by itself:

Control revision and activation time.
Workload class and routing decision.
Hardware, region, tenant, or dependency boundary.
Detector and verifier revisions.
Environmental observations that the controller cannot change.
Delayed outcomes that belong to earlier decisions.

The goal is not to pour every field into one model. It is to prevent adaptation from learning a shortcut through a confounder. A latency controller that reacts to customer mix as if it were machine drift may disadvantage the hardest workloads. A cache controller that treats stale reads as successes may optimize hit rate by weakening freshness.

Typed context lets the system declare what is controllable, what is explanatory, and what is protected.

The authority surface should be small

“Continuously tune the system” sounds powerful and is almost useless as an authorization policy.

A safe controller needs an explicit parameter surface. Each parameter has a range, rate limit, dependency, activation boundary, and known safe revision. Some controls can change independently. Others must move together or in a particular order. Certain regions of the space are prohibited even if the objective suggests them.

The system should distinguish exploration from activation. Candidate controls can be simulated or evaluated in shadow. A canary can receive bounded traffic. Promotion can require stable verifier results over a declared horizon. High-consequence changes remain human owned.

Hard invariants sit outside the learned reward. Data durability, authorization, financial limits, safety constraints, and user promises are not penalties the controller may trade for a better average.

This is one place where the quantum result should not be copied casually. The paper demonstrates a specific controller on a deeply engineered experimental platform with a precise error-correction structure. A production software stack rarely starts with signals that clean or an action space that disciplined. Building the boundary is most of the work.

Adaptation needs a control receipt

Configuration systems usually record that a value changed. An adaptive system needs to explain the decision around the change.

I would store a control receipt containing:

The symptom window and state estimate used to justify adaptation.
The exact controller, policy, and objective revisions.
The candidate parameter diff and its eligible bounds.
Shadow or simulation results, including rejected candidates.
Canary scope, start time, and stop conditions.
Independent verifier observations before and after activation.
Promotion, rollback, or inconclusive outcome.
The next time the decision should be reconsidered.

The receipt prevents an improvement from becoming folklore. It also gives the learning system examples that include rejected and reverted changes, not only survivors.

A corrected error event and a control receipt should remain linked but distinct. The first says how this unit of work was protected. The second says how accumulated evidence changed future operation.

Oscillation is failure with better graphs

A controller that reacts quickly to every change can chase noise. A slow controller can let drift consume all operating margin. Multiple controllers can fight: an autoscaler adds capacity, a load balancer shifts traffic, a cache policy changes hit rate, and each interprets the others as environmental movement.

The adaptive layer therefore needs stability tests beyond “metric improved.”

I would measure intervention rate, parameter travel, rollback frequency, settling time, overshoot, and cross-controller conflict. The system should expose when a policy has become the largest source of variance. Quiet periods matter because they show whether the controller can leave a healthy machine alone.

Coordination does not require one giant optimizer. It can begin with declared ownership, shared observations, activation leases, and constraints between control surfaces. If two controllers affect the same verifier, the trace should make that relationship visible before either receives broader authority.

A small software experiment would test the idea

The useful next step is not to claim that this pattern improves every service. It is to construct a narrow experiment where fixed and adaptive control can be compared under known drift.

I would start with a synthetic queueing service. Its workers have three eligible controls: concurrency, batch size, and admission threshold. The environment introduces gradual service-time drift, sudden dependency slowdown, and changing request classes. A deterministic correction layer retries safe failures and preserves request identity.

The adaptive controller receives typed error, latency, queue, and workload events. It cannot change the detector, verifier, retry policy, or durability rules. It may propose only bounded control revisions. Candidates run against replayed traces, then in a small canary. An independent verifier measures completed useful work, tail latency, fairness across request classes, duplicate effects, and resource cost.

The comparison should include:

A fixed expert-tuned policy.
Periodic offline recalibration that interrupts or drains work.
Continuous bounded adaptation.
Continuous adaptation with one context signal deliberately removed.
A controller optimized only for the visible error metric as a negative control.

The result I care about is not the best average throughput. It is time within the valid operating envelope under drift, including the cost of intervention and any hidden harm. If the adaptive controller wins only by rejecting difficult work or creating oscillation, it failed.

This experiment would test the architecture without pretending a service simulator says anything new about quantum physics.

Simulation and hardware evidence are not interchangeable

The paper's strongest current evidence comes from the Willow experiment. Its large-scale claim is supported by numerical simulation up to distance-15 surface code and tens of thousands of control parameters. The simulations found that the number of optimization iterations did not grow with system size in the studied setup, due to the local sensitivity of error-detection events.

That is promising evidence for scalability. It is not the same as operating a future fault-tolerant machine at that scale for months.

Good technical writing should preserve that boundary. Experimental result, simulation result, and forward expectation need different verbs. The same discipline belongs in software reports. A replay demonstrates behavior under recorded conditions. A canary demonstrates behavior for its bounded slice. Neither silently proves universal production safety.

The temptation to flatten those stages is strongest when the idea is good. That is exactly when provenance matters most.

Long-running systems need to improve without forgetting

The common story about resilience is that a system survives faults. This work suggests a stronger one: a long-running system can use the evidence produced by survival to improve its own operating conditions.

That only works if correction, adaptation, and verification remain separate. The fast loop protects current work. The slow loop learns from accumulated error pressure. The independent outcome check prevents the learner from making the dashboard quiet while the machine gets worse.

The principle fits beyond quantum control, but the implementation must remain domain specific. Databases, fleets, networks, robots, and agent runtimes each have different invariants and different kinds of drift. None should receive a generic optimizer with broad authority.

What transfers is the shape: typed error evidence, two consumers, bounded controls, independent verification, and durable receipts.

We already spend enormous effort producing signals when systems fail. The next breakthrough may come from treating those signals as more than proof that recovery worked. They are also a description of how the machine is changing while it runs.