Reading the run, not just the answer

OpenAI introduced the Responses API on March 11, 2025 alongside built-in web search, file search, computer use, an Agents SDK, and integrated tracing tools.

The release was interesting to N17Q for a reason larger than a new endpoint. The platform treated multi-step tool use and observability as normal building blocks instead of application glue every developer should rediscover alone.

It also made a tempting shortcut more plausible: let the provider's response and trace become the product's run record.

I built a synthetic adapter experiment to learn from the new primitives without transferring that ownership.

The launch matched the industry's direction

Agent experiments had been moving from one prompt and one answer toward model turns interleaved with retrieval, tools, and application state. The March release named that shape directly.

The Responses API used an item-oriented design and streaming events, while the associated tools and SDK addressed search, files, computer interaction, orchestration, handoffs, guardrails, and trace inspection. Computer use was explicitly a research preview, an important boundary for any evaluation.

The announcement did not make every workflow reliable. It made the workflow itself harder to ignore.

That was exactly the pressure behind N17Q.

Provider APIs evolve. A future reader using newer documentation could accidentally backport features and guarantees into this evaluation.

I recorded the March 11 announcement, initial item and event shapes needed by the experiment, available built-in tool categories, SDK concepts, and stated preview boundaries as a versioned research fixture. Later changes would create new adapter revisions.

The public article describes this launch-day surface, not every capability the platform may gain.

Historical accuracy became part of the integration contract, just as a run pinned the tool registry it actually saw.

A single assistant message had always been a poor container for reasoning output, tool requests, tool results, and final prose. An item stream made those units more explicit.

The N17Q adapter mapped provider items into candidate semantic events: model output observed, tool requested, built-in tool observation available, and response completed. Provider identity and sequence remained attached.

It did not copy items directly into the domain stream. N17Q still assigned its own run, turn, request, attempt, and artifact identities, validated payloads, and linked consequences.

The provider representation improved parsing. The product representation preserved meaning across providers.

Streaming events were evidence, not state

Provider events could arrive incrementally, repeat after reconnection, or stop before a terminal response. N17Q stored bounded raw events for diagnostics and ingested them through an adapter sequence.

Only complete, validated semantic observations changed run projections. Text deltas did not become final account. Tool-call fragments did not become executable requests. Usage updates did not overwrite earlier observations without preserving source.

If a stream broke, the attempt state reflected the strongest evidence available. A provider response identity could support recovery, but a new local attempt remained distinct when reconstruction was necessary.

The event stream made activity visible. It did not become the authority to act.

Web search and file search could be selected within the same API family as model generation. That reduced custom transport and orchestration for common retrieval work.

N17Q still wrapped them as product capabilities with source scope, data policy, output limits, citations, request identity, budget, and replay strategy. A built-in label did not prove a result authoritative or make private files eligible for a task.

Provider-returned citations and file references entered the evidence adapter as observations. Claims still needed source identity, revision, scope, and review.

Convenient retrieval was valuable. Evidence lineage remained application work.

A live search result depended on query, time, region, ranking, accessible pages, and provider behavior. Replaying it later could return a different world.

The experiment captured query, provider request identity, returned source links, selected passages where allowed, retrieval time, and response digest. Record mode used an approved read-only connection. Replay mode used a redacted fixture and never called live search.

The evaluator could assess whether the run cited authoritative sources and represented coverage honestly. It did not treat the provider's top result as ground truth.

Built-in search made retrieval easy enough that fixture discipline became more important.

File search needed resource authority

File search worked over provider-managed stores and returned relevant material. N17Q had to decide which corpus a run could access, what files could leave the workspace, how deletion and retention worked, and how a result mapped to a durable source revision.

The adapter accepted an opaque approved-corpus handle rather than letting the model select arbitrary store identifiers. Retrieval observations linked provider file and result identities to the local fixture manifest.

A replay could reconstruct permitted results without depending on the hosted store. Removing the provider did not erase the run's safe evidence record.

Search quality and data authority stayed separate concerns.

Computer use expanded the world-state problem

The computer-use tool translated model output into mouse and keyboard actions in an environment. The launch described it as a research preview and recommended human oversight, which matched the caution N17Q needed.

An action trace alone could not establish outcome. A click might miss, a page might change, a dialog might appear, or an external effect might occur before the next screenshot.

I did not connect it to a real desktop or account. A deterministic browser fixture exposed synthetic pages, controlled actions, and a separate world log. Policy limited coordinates and allowed actions by scenario.

The experiment evaluated observation, proposal, execution, and world change as distinct events.

A screenshot showed rendered pixels at one moment. It did not expose hidden state, complete accessibility semantics, network effects, or whether a click committed remotely.

N17Q stored image digest, viewport, fixture revision, capture time, and related action. A structured synthetic accessibility view accompanied it where the scenario allowed. The agent could use both and still needed receipts for consequential effects.

Replay returned the recorded observation sequence. It did not infer that visually similar pixels represented identical application state.

Computer interaction made the difference between seeing and knowing particularly concrete.

Guardrails were one policy layer

The Agents SDK introduced guardrail concepts around agent input and output. Those could reject unsafe or invalid material early and improve orchestration.

N17Q did not treat a model-facing guardrail as the final execution gate. Deterministic policy ran immediately before every capability attempt with current arguments, world state, authority, prior effects, approval, and budget.

A guardrail could say a proposed plan violated a content rule. It could not grant filesystem scope, revive an expired approval, or decide that an unknown external effect was safe to repeat.

Layered controls were complementary when their authority stayed clear.

The SDK's handoff model made multi-agent delegation an explicit orchestration concept. N17Q represented a handoff as a transition between configured decision roles within one run.

The receiving role saw a bounded checkpoint, allowed tools, remaining budget, and relevant evidence. It did not inherit every capability of the sender or an opaque conversation history by default.

The trace recorded who initiated the handoff, why, which state crossed, and which configuration became active. Policy applied again.

A handoff could improve specialization. It did not make responsibility disappear between agents.

Provider tracing was valuable diagnostic evidence

Integrated tracing could reveal model calls, tool use, handoffs, and timing without each application rebuilding capture. I wanted that visibility.

N17Q linked provider trace and span identities to its run events. It did not make them primary keys. The provider trace might omit local policy denials, sandbox state, user approval detail, simulated-world effects, or commands performed outside its SDK.

Conversely, N17Q did not reproduce every internal provider timing field in its semantic timeline. Diagnostic detail remained inspectable through a bounded adapter.

Two traces described overlapping systems. Their disagreement could be useful evidence.

One green trace was not completion

A provider span could show a successful tool return while the product failed to commit the observation. A local effect could succeed while the response stream ended unexpectedly. A final response could complete while the scenario retained an unknown external outcome.

N17Q completion required product invariants: intended artifacts, allowed world state, no prohibited effects, resolved required actions, and a final account consistent with evidence.

Provider completion became one event in that decision.

Observability was most useful when it stopped a technical success label from becoming a product success label.

The launch emphasized the ability to store data for evaluation and tracing. That could simplify continuation and analysis. It also affected retention, privacy, deletion, access, and portability.

N17Q's provider profile recorded whether remote storage was used for a run and which data classes were eligible. Synthetic experiments used only controlled content. Sensitive or prohibited scenarios selected another adapter or disabled storage.

Local checkpoints never assumed the remote response object would exist forever. Deleting a provider record could reduce native diagnostics without breaking the run's semantic history.

Convenience around storage remained an explicit data decision.

Usage observations fed product budgets

The adapter captured provider-reported usage when available and preserved the dimensions and source. N17Q mapped those observations into its versioned budget model without pretending estimates and billing were identical.

Built-in tool calls, model tokens, wall time, and local fixture work remained separate categories. A run could exhaust one budget while retaining room for manual review or safe status query.

The provider could report what its platform consumed. The product decided what this scenario was allowed to consume.

Trace visibility made cumulative cost easier to see and no less necessary to govern.

Errors retained platform detail

The adapter mapped invalid input, refusal, rate limit, unavailability, stream interruption, and malformed output into N17Q outcomes only when provider evidence supported the classification.

Original safe codes, request identity, response status, and retry advice remained linked. Free-form error text did not control recovery. A tool interruption after possible effect still entered outcome unknown according to the product contract.

This preserved the value of platform diagnostics without letting provider vocabulary flatten consequence.

An API error and a failed run were never assumed to be the same event.

I implemented only the launch-day surface needed by three synthetic scenarios: item ingestion, function-style candidate requests, bounded web-search observations, usage, and provider trace linkage. Computer use stayed in the deterministic imitation rather than a live preview connection.

Unsupported item kinds failed visibly and retained safe diagnostic shape. The adapter did not claim feature parity or become the default for every N17Q case.

This kept the evaluation about boundaries rather than integration breadth.

A new platform is easiest to misunderstand when the first goal is to use all of it.

Fixtures supplied recorded provider item streams and expected semantic run events. Cases covered duplicate events, interrupted streams, tool requests denied by local policy, citations with missing source identity, rate limits, and a final response before required world reconciliation.

Assertions checked the provider trace link, N17Q event graph, simulated world, and final status. A passing item parser could not hide a forbidden effect. A correct run could tolerate harmless provider-event duplication.

The adapter contract focused on preserving meaning between trace systems.

Built-in did not mean automatically allowed

A platform could make web search, file search, or computer use available to the response. N17Q still selected the offered set per scenario after data, effect, and budget policy.

A repository-edit task could receive function capabilities for a synthetic sandbox and no live web access. A historical research task could use recorded search and have no computer tool. A request crafted outside the UI met the same gate before any provider call.

The provider knew which tools were available inside its request. The product knew why that set was eligible for this run. Both appeared in the trace.

Tool convenience changed integration effort, not least-privilege design.

One response could contain several tool requests, and one external effect could require attempts across more than one response. Using the response ID as an idempotency key would bind consequence to the provider's turn structure.

N17Q assigned semantic effect identity after normalizing a candidate request. Retries and recovery preserved it even if another response or provider continued the plan. Provider response and item IDs remained causal evidence.

This also kept a replay from treating a repeated recorded response as permission to repeat its tools. Response content proposed; effect state authorized and reconciled.

The platform object and product object overlapped without becoming interchangeable.

Provider tracing offered a useful inspection surface, while N17Q compared several adapters and local tools. I defined a stable semantic export containing configuration segments, events, artifact references, policy decisions, tool effects, budgets, checkpoints, and evaluator evidence.

Provider spans could be linked or included under retention policy. They were not required for another evaluator to understand the run. Conversely, platform-native evaluation could consume a carefully prepared view without receiving every private artifact.

The export made comparison possible without forcing all providers into one raw trace schema.

Interoperable evaluation began with preserving the distinctions that mattered.

Deletion was part of the adapter test

I created a synthetic run, deleted its remotely stored response and trace where the experiment allowed, and reopened N17Q. The local run retained its semantic evidence and labelled provider diagnostics unavailable.

Deleting the N17Q run followed a separate workflow across semantic events, artifacts, fixtures, indexes, and any remote references under its control. One delete action did not make universal claims about caches or third-party sources.

This exercise tested data ownership more concretely than a diagram. A convenient stored response could remain optional only if removal did not make the run incoherent.

Stop remained a valid outcome

The release emphasized systems accomplishing multi-step tasks. N17Q added the complementary state: a run should stop when policy, uncertainty, budget, or missing evidence makes safe progress unavailable.

The adapter could cancel or abandon provider work according to its actual guarantees. The semantic run recorded what had already been observed and which external effects remained unresolved. A harness-generated account supported handoff even if no final response arrived.

Tracing made stopped work explainable. It did not make continued execution automatically desirable.

The provider remained replaceable

After storing several experiment runs, I disabled the adapter. Their configurations, semantic events, artifacts, tool receipts, world assertions, and evaluations stayed readable. Provider trace links became unavailable where remote retention had ended and were labelled accordingly.

A comparison run could use another adapter from the same checkpoint fixture. It received a new configuration segment and did not pretend to continue the original provider response.

Replaceability was not an argument against using platform tracing. It was the condition that let me use it without giving up run ownership.

The release changed the default expectation

The most durable part of the announcement was cultural as much as technical. Agent builders were being offered tool use, orchestration, and trace inspection as one platform-shaped problem.

That raised the minimum expectation for applications. A multi-step system should be inspectable. A tool call should not disappear between prompt and final text. A handoff should have identity. Evaluation should see more than an answer.

N17Q agreed and pushed the boundary further toward policy, world state, replay, and recovery.

Platforms can normalize instrumentation without defining the product's truth.

A first-class concern, not a surrendered record

The Responses API and associated agent tooling made the trace easier to obtain and the agent trajectory more explicit. Those were real improvements.

N17Q treated provider items and traces as high-quality observations inside a larger run. Local events supplied authority, approval, sandbox state, fixtures, effects, checkpoints, and final world assertions. Neither layer was flattened into the other.

The experiment left me with a stronger adapter and a clearer standard: every provider integration should preserve enough event structure to explain what it observed, while the product retains the durable account of what was allowed and what changed.

The March release made traces a first-class platform concern. Making them a trustworthy product record remained architecture.