Host requirements
Every environment needs the same grants from the host, and no environment needs
--privileged. One entry in the list is a denial rather than a grant:
--network=none.
-
--cap-add=SYS_PTRACE— Load-bearing. Crash analysis attaches to the target process to record where it failed. -
--cap-add=SYS_ADMIN— Load-bearing. The environment puts each run of the target program in its own mount, network and IPC namespaces, and creating those namespaces needs this capability. Without it the environment fails to start a run at all, rather than degrading. -
--cap-add=NET_ADMIN— Best effort. Brings the loopback interface up inside the run's private network namespace. Without it the target still runs; only targets that talk to themselves over the network are affected. -
--security-opt seccomp=unconfined— The default seccomp profile blocks the namespace and tracing calls above. -
--security-opt apparmor=unconfined— As above, for hosts that enforce AppArmor. On a host using SELinux, add--security-opt label=disableas well. -
--network=none— Run the environment with no outbound network. Nothing inside an environment needs egress: the server speaks the protocol over standard input and output rather than over a port, git is a localfile://transport, and builds resolve from caches already in the image. The container keeps its own loopback, so anything talking to127.0.0.1inside it is unaffected.
Two of the grants are load-bearing: without SYS_ADMIN the environment cannot
start a run of the target program at all, and the failure is an error rather
than a silent loss of function. If your platform cannot grant these, an
environment will not work, and it is better to discover that before a scheduled
evaluation. --network=none is the opposite case — leaving it off breaks
nothing you can see, which is why it has its own section below.
Why the network is off
A task is scored on the model solving a defect from the source in front of it. A model that can reach the internet may fetch the upstream project, the advisory, or a public proof-of-vulnerability for that exact defect, and that lowers the hardness of every type — exploit, detect, patch and audit alike. The score then measures retrieval rather than capability, and it does so without any sign in the transcript that it did.
Isolating the runtime costs nothing, because no environment needs a network to
run. If you substitute another runtime for Docker, use whatever it offers in
place of --network=none; what matters is that a graded episode has no egress.
Container runtimes
The environments are built and validated under the default runtime, runc.
They have also been validated under gVisor (runsc) and Kata Containers, and
part of our own fleet runs on gVisor, so both are safe choices; Kata gives the
strongest isolation of the three. None of them need flags different from those
above.
Firecracker-backed platforms deserve a check rather than avoidance: a migration to one once broke the environment's process-demotion step, which is the kind of fault that shows up as every agent shell failing to start. Launch one environment and confirm a shell command returns before relying on a fleet.
Sizing
Each environment declares the resources it needs, in the form
4vcpu+16gib. Patch environments are much the larger of the two shapes,
because they build the program repeatedly; the others are modest.
Treat the declared figure as a floor, not a suggestion. Pass it through as
--cpus and --memory. An environment given less will not fail cleanly — it
will time out during a build or a test run, which reads like a broken task
rather than an under-resourced one.
