Reconnect is a data operation

The Socket.IO connect callback felt like a finish line. The browser had lost its connection, the library retried, and the callback fired. I removed the warning and resumed rendering incoming messages.

The first message after one reconnect was revision 418. The browser had last applied revision 411.

Nothing in the open socket explained what happened to 412 through 417. They might have contained independent notices that no longer mattered, or the only changes to values still visible on the screen. Treating 418 as a return to normal meant accepting an unknown local projection as current.

The network had recovered. The data had not.

Delivery resumed in the middle of a story

T04P kept a current projection in the browser: counts by entry point, equipment state, and an ordered set of operator notes. Updates were patches against that projection. A counter message could say “increment north entrance by 4.” An equipment message could change one unit from available to degraded.

Those patches were efficient because they assumed a shared starting state. A gap broke the assumption.

If the missed events were absolute replacements, applying the newest one might be enough for that field. If they were increments, applying the newest one alone was definitely wrong. If a missed event removed a note and the next event edited it, the behavior depended on an object the browser still believed existed.

The socket library could preserve ordering among messages it actually delivered. It could not decide whether the application state remained a valid base after an interruption. That was a product protocol question.

I stopped treating reconnect as a transport notification and defined it as the beginning of reconciliation.

The client reported what it knew

Every accepted feed event received a monotonically increasing service revision. The browser stored the last contiguous revision it had applied, not merely the highest revision it had seen.

On reconnect, the client sent a small request:

type ResumeRequest = {
  feed: string
  lastContiguousRevision: number | null
  projectionVersion: number
}

The service compared that claim with the history it still retained. There were three useful outcomes:

• The revisions matched, so no data recovery was required.
• The service retained every missing event and could send a contiguous delta.
• The gap was unknown or too old, so the service sent a complete snapshot.

The client did not get to request “the latest event” as a shortcut. Latest was not synonymous with complete. It asked for a path from its named state to a confirmed state.

This made the browser's local knowledge explicit. null meant it had no usable projection, such as after a hard reload with empty memory. A revision meant it could defend continuity through that point and no further.

Highest seen was not contiguous

The first revision tracker stored only lastRevision. That failed when messages arrived out of order during a deliberately hostile simulator run.

Suppose the browser held revision 20 and received 22 before 21. If it set lastRevision to 22, its next resume request implied that everything through 22 had been applied. Revision 21 might contain a necessary change.

I split the concepts:

last contiguous: 20
buffered: [22]
gap begins: 21

When 21 arrived, the client could apply 21 and then 22, advancing the contiguous revision to 22. If the gap remained past a short bound, it requested recovery rather than buffering indefinitely.

Duplicates were harmless when their revision was at or below the contiguous point. They were counted for diagnostics and ignored for projection changes. A duplicate could still signal an adapter problem, but it did not deserve a second user-visible effect.

The tracker stayed per feed. A global revision would have created a total order across unrelated sources and made one noisy feed force unnecessary recovery elsewhere. T04P needed completeness within each projection contract, not an invented universal chronology.

Delta was an optimization with a proof obligation

Sending only missed events was attractive. It used less bandwidth and preserved the exact transition history. It was valid only if the service held a complete range beginning immediately after the client's revision.

The retained event window was bounded. T04P was a personal project running on modest hardware, not a permanent event archive. The service kept enough recent revisions to cover common interruptions, then discarded older detail after folding it into the current snapshot.

The resume response named its range:

type DeltaResponse = {
  kind: 'delta'
  fromExclusive: number
  toInclusive: number
  events: FeedEvent[]
}

The client checked that the first event was exactly fromExclusive + 1, that every next revision advanced by one, and that the final revision matched toInclusive. A response labeled delta was not trusted merely because it came from the server.

If any check failed, the client discarded the partial delta and requested a snapshot. It did not attempt to guess around the hole.

That fallback made the delta protocol easier to keep simple. Optimization could fail closed into a complete replacement. It did not have to solve every restart, retention, or version mismatch.

A snapshot needed an identity

The earliest snapshot was just a JSON object. The browser could replace its projection, but it could not tell which stream position the object represented. An event arriving during the snapshot request might be applied before or after replacement unpredictably.

The service created the snapshot at a named revision:

type SnapshotResponse<T> = {
  kind: 'snapshot'
  throughRevision: number
  projectionVersion: number
  observedAt: string | null
  value: T
}

Events after throughRevision could then be applied in order. Events at or below it were already represented and ignored. While the snapshot was in flight, the browser buffered a small bounded set of newer events. If the buffer exceeded its limit, it restarted recovery rather than risking unbounded memory.

The snapshot was produced from one coherent service projection. Serializing several maps at unrelated moments could create an object that had never actually existed. I avoided asynchronous work inside the snapshot read and captured the revision with the data under the same update boundary.

This was not a database transaction in a grand architecture. It was an application-level guarantee that the snapshot label described the contents beside it.

I initially kept the old screen visible and cleared the transport warning as soon as the socket reopened. Reconciliation happened silently. For a few hundred milliseconds in development that seemed acceptable. Under a delayed snapshot it became another period of false confidence.

The display learned a recovering state. The last known values remained visible when useful, but their labels said they belonged to before the interruption. Controls that depended on current state were disabled. The status line named the work: “Checking missed counter updates” or “Loading a current equipment snapshot.”

This was deliberately different from a loading screen. The browser had information; it just could not yet prove that the information was complete. Erasing everything would have hidden useful context. Showing it without qualification would have repeated the stale-dashboard failure.

When recovery finished, the state transition used one commit. The snapshot and any buffered tail became visible together. The interface did not animate through every missed increment as if the event were happening again in real time.

The distinction mattered for attention. Recovery was reconstruction, not live theatre.

Reconnect could happen twice

Unreliable networks did not wait for the first recovery to finish. A socket could open, request a snapshot, close, and open again before the response arrived. The first callback could still complete after the second attempt began.

I gave each recovery attempt an identifier. State changes were accepted only from the active attempt:

type RecoveryState = {
  attemptId: number
  phase: 'requesting' | 'applying' | 'complete' | 'failed'
}

Starting a new attempt invalidated the old one. Late responses were ignored. This prevented an older snapshot from replacing a newer recovered projection.

The same rule applied when the user changed which feed was displayed. A response belonged to both a recovery attempt and a feed identity. Checking only that “a request completed” was insufficient.

I avoided elaborate cancellation machinery. The underlying request could finish; it simply lost authority to change state. That was easier to test in the browser environments I cared about at the time.

This pattern later appeared in search, preview builds, and generated answers. Asynchronous completion does not grant relevance. The consumer needs an identity that says which intent the result belongs to.

Restarts broke the revision assumption

The first service revision lived in memory and returned to zero when the Node.js process restarted. A browser left open across the restart might report revision 418 to a server whose newest revision was 12. Comparing only numbers made the client look impossibly far ahead.

I paired the revision with a stream instance identifier generated when the service projection was initialized:

stream: 7f2c...
revision: 418

If the instance changed, delta recovery was unavailable. The client requested a complete snapshot and adopted the new identity. Diagnostics recorded the discontinuity.

Persisting the revision and event window would have survived process restarts, but it introduced storage and recovery complexity that T04P did not yet need. A snapshot after restart was an honest and bounded tradeoff.

The stream identity also protected against a restored old server process or a development tab pointing at a different environment. Revision 42 from one stream had no ordering relationship with revision 42 from another.

This was a reminder that monotonic values are monotonic only within a named scope. Removing the scope turns a useful sequence into a misleading integer.

Side effects did not belong in replay

Most T04P messages updated a read-only projection. Operator notes introduced actions that could have side effects, such as acknowledging a condition. I considered replaying all missed messages through the same event handlers after reconnect.

That was unsafe. A handler written for a newly received command might show a notification, play a sound, or submit an acknowledgement. Reconstructing state should not repeat those effects.

I separated projection events from ephemeral presentation effects. Durable events described facts required to rebuild the current view. Sounds and transient notifications were triggered only for events received in live mode after continuity was established.

The recovery engine applied missed events with an explicit context:

mode: recovering | live

Projection reducers behaved the same in both modes. Effect handlers did not. A snapshot bypassed event-by-event effects entirely.

This stopped recovery from becoming a burst of old alerts. It also made the state logic more testable because rebuilding the projection did not depend on browser side effects.

Every recovery mechanism can become a resource problem. A browser offline for hours might request a huge delta. A malfunctioning client might repeatedly restart recovery. An event burst could fill the buffer faster than a snapshot could be applied.

T04P set ordinary, explicit bounds:

• Maximum retained delta length per feed.
• Maximum buffered events during snapshot application.
• Timeout for one recovery attempt.
• Backoff before retrying after repeated failure.
• Maximum number of automatic attempts before asking for a reload or showing a persistent degraded state.

The values reflected the project's scale rather than pretending to be universal. The important part was the fallback behavior. Exceeding a delta bound selected a snapshot. Exceeding the snapshot buffer restarted from a newer snapshot. Repeated failure left the screen visibly unavailable instead of looping forever behind a spinner.

I logged each fallback as a named transition. “Snapshot selected because client revision predates retained window” was actionable. “Reconnect failed” compressed too much.

The limits turned pathological behavior into designed states. They also made simulator scenarios terminate, which is a surprisingly important property of a recovery system.

A small state machine replaced callback optimism

The first implementation spread behavior across socket callbacks: connect, disconnect, message, and error. Each callback changed UI flags and sometimes data. Interleavings produced combinations nobody had intended, such as connected: true beside loading: false while a snapshot was still pending.

I wrote the recovery flow as a state machine:

disconnected
  -> transport-open
  -> reconciling
      -> applying-delta
      -> applying-snapshot
  -> current
 
any recovery state
  -> disconnected
  -> failed

Events that did not make sense in a state were ignored or raised a diagnostic error. A live patch could not enter the current projection while the client was waiting to establish a base. An old attempt could not complete a newer transition.

The machine was plain code, not a framework. Naming the states mattered more than the implementation technique. It gave tests a vocabulary and kept transport state from directly deciding product state.

The visible status derived from this machine plus feed freshness. It was not independently toggled by callbacks. That removed a class of contradictions where the data said recovering and the banner said live.

The simulator made gaps ordinary

I added deterministic scenarios for the sequences that were difficult to produce by hand:

disconnect after 411
drop 412–417
reconnect
deliver 418 before resume response
expire delta window
return snapshot through 420
deliver buffered 421

The expected result named both the projection and the transitions. The browser should never call itself current at revision 418. It should choose a snapshot when the delta window was gone, ignore the early live event until it had a base, then finish at 421.

Other scenarios restarted the service, duplicated the final delta event, disconnected during snapshot fetch, and returned a malformed revision range. These were not randomized chaos tests. Each fixture isolated one protocol promise and produced the same sequence every time.

That determinism made failures discussable. Instead of “reconnect sometimes loses a count,” I could say “a second connection invalidates the first recovery attempt, but the late snapshot still commits.” The sentence pointed to one ownership rule.

Automatic reconciliation could fail because the service was repeatedly restarting, an adapter was producing invalid revisions, or the browser had a bug the protocol did not anticipate. A perpetual automatic loop would turn that uncertainty into motion without progress.

After the bounded retries, the interface stopped and exposed a deliberate action: “Try a fresh snapshot.” The action discarded the local projection only after the user confirmed it, then requested a complete base with no resume revision. The previous value remained visible as last known until replacement succeeded.

A diagnostic copy button captured stream identity, contiguous revision, active attempt, last response kind, and failure reason. It did not copy raw operator notes or every payload. The goal was enough protocol state to reproduce the transition without leaking unrelated content.

There was also a plain page reload as a final path, but it was not presented as a magical fix. A reload cleared browser memory and initiated the same snapshot contract. If the service could not produce a coherent snapshot, reloading would not make the feed current.

Designing this escape hatch improved the automatic path. Every stopped state needed an explanation, every retry needed a bound, and every destructive reset needed to say what information it would discard. Recovery became something a person could supervise rather than an invisible callback loop.

Reconnection was never just a green socket

The final recovery path was not enormous, but it was larger than calling connect() again. It named what the client knew, proved or replaced the missing interval, bounded its buffers, separated reconstruction from live effects, and represented uncertainty in the interface.

The key invariant was concise:

The browser may present a feed as current only when it holds a complete projection through a named revision and the underlying source evidence is fresh enough for that feed.

Transport recovery satisfied none of that invariant by itself. It merely made the next data operation possible.

I still value libraries that reconnect automatically. They solve socket establishment, retry timing, and message delivery details I do not want to reproduce. I no longer let their success callback define application recovery.

A connection can return in an instant. Trust has to be rebuilt from a known state.