Inside the sandbox, still unsafe

The first N17Q sandbox had a writable directory and a reassuring label.

Commands ran inside a temporary workspace. The agent could not see files above its repository root through the file tool. When the run ended, I deleted the directory.

The process still inherited most of my environment, could start child processes, could reach the network, and shared the host kernel. “Sandboxed” described one filesystem convention and implied a security guarantee I had not built.

I stopped using the word as a verdict. A sandbox is a set of enforced constraints with explicit limits.

The threat model came before the mechanism

N17Q evaluated synthetic repositories, not arbitrary malware. The main risks were accidental destructive commands, prompt injection in project files, unexpected scripts, data leakage, runaway resource use, network side effects, and tests modifying state outside the task.

The harness did not claim resistance to a hostile kernel exploit, advanced side channel, or deliberately weaponized native dependency. Scenarios used known fixture code and controlled toolchains.

Writing that boundary down prevented container, virtual machine, or permission vocabulary from expanding into “safe to execute anything.”

Security claims followed the adversary and environment actually tested.

The immutable base reduced ambiguity

Every repository scenario began from a content-addressed read-only base plus a writable overlay. The base included file metadata and an explicit initial-change layer.

Commands could modify only the overlay. The harness calculated final tree and diff independently. Resetting a run discarded writes without mutating the source fixture.

This made cleanup reliable and changes inspectable. It did not stop a process from reading other available resources or using the network by itself.

One constraint solved state ownership and no more.

Filesystem scope needed operating-system enforcement

The file tool normalized paths, rejected traversal, resolved symlinks, and allowed only configured roots. A command could bypass that tool unless the process environment enforced similar boundaries.

N17Q ran commands in a constrained environment exposing the workspace, required runtime files, and narrow temporary storage. Host home directories and unrelated project roots were absent. Sensitive mounts were read-only or omitted.

Tests created symlink escapes, absolute paths, parent traversal, device paths, and temporary-file tricks supported by the environment. Denials became trace events.

Path parsing and process isolation reinforced each other.

A process often needs temporary files, package caches, sockets, or compiler output. Making the whole filesystem writable for convenience weakened review and cleanup.

The environment provided bounded writable workspace, temporary, and tool-cache locations. Their size and lifetime were explicit. Cache keys included trusted dependency manifests and did not accept arbitrary executable output from another run.

Writes elsewhere failed. The trace summarized writable-byte growth and produced artifacts only from approved roots.

The agent could see which paths were intended for generated state without receiving a general host filesystem.

Process capability was narrower than shell capability

N17Q did not expose a generic interactive shell in ordinary scenarios. Its command tool accepted executable, arguments, working directory, environment subset, time limit, and resource limits.

Policy allowed known command families and denied privilege changes, host process inspection, background daemons, and arbitrary interpreters unless the scenario explicitly studied them. The environment also constrained process count and signals.

A repository script remained untrusted even when launched through an allowed package command. Its subprocesses inherited the sandbox constraints.

Application policy and operating-system limits protected different paths to execution.

Environment variables started empty

Developer shells accumulate tokens, paths, sockets, cloud configuration, package credentials, and debugging flags. Inheriting them into an agent process would turn ambient convenience into invisible authority.

N17Q constructed a small environment from declared runtime variables. Scenarios needing authorization received synthetic scoped values. Real credentials were absent.

The manifest recorded variable names and safe classifications, not secret values. Command output passed through redaction before trace storage and model context.

An allowed executable did not inherit the identity of the person who launched the harness.

Network was denied outside named tools

Replay had no ambient outbound network. Approved source observations came from fixture adapters. Record-capable experiments used explicit HTTP capabilities with destination, method, data, redirect, size, and budget policy.

Commands could not open arbitrary sockets to recreate a denied tool. Known tests attempted DNS, direct addresses, redirects, and child processes. Environment support determined the strength of enforcement and the manifest named it.

A package install that needed a registry either used a prepared offline cache or stopped for a separate approved workflow.

Network access became traceable product state instead of an inherited property of the machine.

Localhost was not automatically safe

Services on the host could expose development databases, metadata, browser automation, or privileged control endpoints. A rule blocking only public internet would leave a useful escape path.

N17Q denied loopback and private network ranges by default in sandboxed command execution. Fixture services lived on an isolated channel exposed only through reviewed adapters or a dedicated test network.

Redirect and DNS-rebinding-style fixtures checked that destination policy applied after resolution as well as before request construction.

“No internet” was not allowed to mean “everything nearby is trusted.”

Resource budgets constrained polite denial of service

Code did not need malicious intent to consume all CPU, memory, disk, processes, or output. A test loop or generated file could do it accidentally.

The sandbox enforced per-command and cumulative run limits. N17Q recorded wall time, CPU where available, peak memory, process count, writable bytes, and output bytes. Exceeding a limit stopped the command and checkpointed the run.

Truncated output remained an artifact with explicit omission. The agent could not raise limits through repository configuration.

Availability was part of the boundary even in a personal experiment.

Output was another untrusted input

Commands could emit escape sequences, enormous lines, fake status markers, instructions to the model, or data that resembled structured tool events.

N17Q captured raw bounded bytes separately, sanitized terminal presentation, and converted known command results through an adapter. Output included in model context was quoted observation with truncation and redaction metadata.

A string saying “policy allowed” could not create a policy event. A fake receipt could not enter the effect registry.

The boundary continued after execution returned.

A fixture file told the agent to edit the sandbox policy and enable network. The file was valid project content and untrusted instruction.

Harness configuration lived outside writable mounts. The model could propose a capability request, but no tool exposed policy mutation. The run recorded the denied attempt and safe alternatives.

Project instructions could govern style and local commands within their scoped authority. They could not grant resources the environment withheld.

Instruction hierarchy improved behavior; isolation preserved control when behavior failed.

Dependency execution widened the boundary

Tests and builds execute dependencies, compiler plugins, package lifecycle scripts, and native binaries. Reading a repository is different from running it.

N17Q pinned an offline dependency snapshot for each scenario and recorded runtime versions. Unknown install scripts and native extensions were excluded from ordinary cases. New dependency execution required fixture review.

The sandbox constrained known code without claiming the supply chain was harmless. Artifact hashes and manifests made what ran inspectable.

A green test result did not certify every dependency in the environment.

Replay supplied scenario time and seeded fixture randomness. Commands still observed process scheduling and runtime behavior that could vary.

The environment fixed locale, timezone, common encoding, and declared random seeds where tools supported them. Tests relying on uncontrolled clocks or host paths were flagged.

Deterministic surroundings improved replay and reduced covert dependence. They were not presented as proof that every binary execution was reproducible.

The manifest separated controlled inputs from residual variability.

Where the environment supported syscall filtering, namespaces, or stronger virtualization, N17Q used a reviewed profile and recorded its revision. Missing support reduced which scenarios were eligible.

I avoided describing a specific mechanism as universal containment. Kernel bugs, device exposure, runtime configuration mistakes, and platform differences remained possible.

Contract tests exercised prohibited operations, and environment health checks verified the expected profile before a run.

The mechanism supplied evidence about constraints. The threat model defined what conclusions that evidence supported.

Sandboxes needed identities and versions

A run recorded base image or environment digest, policy profile, runtime manifest, mounted artifacts, network mode, and resource limits. Changing any of them created a new configuration segment or run.

This made evaluation comparisons honest. A model did not receive credit for efficiency if another run had a warmer cache or broader executable set. A security fixture did not pass under a profile different from the one claimed.

“Same prompt” was not enough when the world around it changed.

Workspace snapshots were not backups of the host

The harness exported only base reference, initial overlay, final overlay, relevant artifacts, and manifest. It did not archive the whole machine to guarantee reproducibility.

This kept sensitive and irrelevant host state outside the run. A missing undeclared dependency caused replay failure and revealed the accidental coupling.

Portable fixtures could be reconstructed in another compatible environment and verified by digest.

The smaller explicit package was more trustworthy than a complete opaque machine image.

Cleanup had a receipt

Deleting the process and temporary directory was not enough if background children, mounts, caches, or prepared artifacts remained.

N17Q's teardown reached a verified state: processes terminated under known guarantees, writable overlays removed, ephemeral credentials revoked, network fixtures closed, and retained artifacts moved under policy. Failures produced an operations event instead of being ignored after test completion.

Unknown teardown state blocked reuse of the environment.

Isolation included ending the run cleanly.

Review happened before anything left

The agent could generate a patch inside the sandbox. Pushing a branch, opening a request, uploading an artifact, or sending a report crossed into another capability boundary.

N17Q prepared immutable output, ran scope and secret checks, displayed the diff and validation, and required the appropriate approval. External delivery used stable effect identity and receipts.

Sandbox completion never implied publication authority.

Containment protected local work; product policy governed its release.

The final diff could still be harmful

A perfectly isolated run could produce insecure, incorrect, or malicious code. The sandbox limited what happened during generation. It did not certify the artifact later executed elsewhere.

Independent tests, static checks, dependency review, human diff inspection, and deployment policy remained necessary. N17Q evaluation checked prohibited changes and world state but could not prove absence of every vulnerability.

This was the most important language correction: safe execution environment and safe output were different claims.

The sandbox constrained the process, not the future meaning of its work.

N17Q recorded commands, files, processes exposed by the adapter, policy decisions, resource counters, and detected network attempts. It did not claim to observe every kernel event or microarchitectural side channel.

The run manifest listed enabled telemetry and known gaps. Sensitive raw traces followed retention and redaction. Missing visibility could make a scenario ineligible for a strong assertion.

An audit trail helped investigate constraints. It was not proof that unobserved behavior did not occur.

Precise uncertainty made the sandbox more credible.

A graphical browser or desktop fixture could expose clipboard, downloads, drag-and-drop, and operating-system dialogs outside ordinary page content. N17Q modelled those channels explicitly rather than treating pixels as the whole sandbox.

Synthetic computer-use scenarios received an isolated clipboard and download directory. Opening external applications, host file pickers, and notification controls was unavailable. Screenshots and accessibility observations entered the trace with viewport and world revision.

An action could remain inside the visible browser and still cause a remote effect, so tool receipts and world assertions stayed necessary.

The interface boundary did not end at the process window.

Data leaving the sandbox had a gate

Generated patches, logs, archives, and screenshots could contain source text or synthetic secrets. Moving them into durable artifact storage or an external review destination was not automatic cleanup.

N17Q classified outputs, scanned known secret forms, applied size and retention policy, and prepared immutable artifacts. External delivery required a separate approved capability with destination scope and receipts.

The sandbox could be isolated during computation and still leak through careless export. Egress policy followed every artifact that crossed out.

Runtime images and toolchains required security updates. Historical replay required knowing what an older run used. Replacing every old environment digest with the newest image would improve patch level and destroy reproducibility.

N17Q retained manifests and safe fixture packages, while execution policy could prohibit running an obsolete profile. A migrated scenario created a new environment revision and comparison run. The original trace remained inspectable.

Current eligibility and historical identity stayed separate, as they did for tools and policies.

Host failure remained possible

Disk exhaustion, kernel panic, power loss, and harness bugs could interrupt the environment outside its modeled command states. N17Q persisted intent before execution and checkpointed after meaningful observations, but it could not make the host infallible.

After restart, recovery inspected sandbox lease, overlay integrity, running-process evidence, and external-effect receipts. Uncertain teardown or command outcome paused rather than assuming clean failure.

The sandbox limited consequences during normal and tested fault paths. Durable workflow state handled the fact that the container around it could still disappear.

Fixtures attempted path traversal, symlink escapes, environment reads, process spawning, long loops, disk growth, localhost access, external DNS, output injection, policy-file editing, and live credential lookup.

The expected outcome combined operating-system denial, application policy, and clean world state. A test passing on one platform did not automatically certify another profile.

Canaries outside the workspace detected writes where environment support allowed. Replay also asserted no live client construction.

Negative tests turned the constraint list into evidence rather than documentation alone.

The interface described what was enforced

Instead of one green Sandbox badge, N17Q showed workspace root, writable areas, network mode, credential mode, process profile, resource budget, environment revision, and last verification status.

Normal users saw a short summary such as “synthetic workspace, offline fixtures, no live credentials.” Technical detail remained inspectable. Unsupported guarantees were stated plainly.

An environment with weaker controls did not borrow the same label. Policy could restrict it to read-only or pure transformation scenarios.

The product vocabulary followed evidence.

With ambient paths, network, and credentials removed, every meaningful observation came through a known channel. The trace could explain what informed a plan. Replay could provide the same fixture world. Prohibited attempts became clear.

This did more than reduce risk. It improved scientific usefulness. A surprising result was less likely to depend on an invisible host file or current web response.

The sandbox and evaluation harness strengthened each other because both valued explicit boundaries.

When a platform could not enforce network denial, process isolation, or mounted-path scope with enough evidence, N17Q did not display the same sandbox profile and continue normally.

Policy limited that environment to pure transformations over non-sensitive synthetic inputs, or made it unavailable. The run manifest explained which constraint was missing. A developer override created a separately labelled experiment and could not satisfy stricter benchmark claims.

Capability followed the environment actually present. Portability never meant pretending every host supplied equivalent isolation.

The name stayed, with qualifications

I kept calling the component a sandbox because the term was useful shorthand. The interface and documentation always followed it with the profile and scope.

N17Q could say that commands ran in an ephemeral, bounded synthetic workspace with controlled files, processes, network, credentials, and budgets. It could say which escape tests passed under one environment revision.

It could not say arbitrary code was safe, secrets could never leak, or generated changes were trustworthy.

A sandbox is not one magic wall. It is a collection of constraints that must be enforced, observed, versioned, tested, and kept smaller than the claims made about it.