The tool surface

The server can expose ten operations. Which of them your agent should see is not a matter of taste — it is part of the task definition, and the startup command that came with the environment already encodes the right answer.

Tool	Reachable by
setup_problem	Orchestrator, every type
reproduce	Agent, exploit and detect
submit	Agent, audit
read_file	Available, not in the shipped tool set
edit_file	Available, not in the shipped tool set
bash	Agent, every type
verify	Agent, patch
grade_problem	Orchestrator, every type
triage	Never exposed
validate	Never exposed

In practice a shipped environment exposes bash plus exactly one action operation, with setup_problem and grade_problem reserved to your orchestrator. Do not forward those two to the agent: an agent that can call its own grader is not being measured.

read_file and edit_file exist and work, but are not part of the shipped tool set. The agent edits through bash instead, which runs inside the environment's isolation; a file editor driven from outside it can read parts of the container the task is meant to hide. If your harness advertises an editor under the name str_replace_editor, map it to bash or leave it out.

triage and validate are ours. They exist so we can check an environment is sound before shipping it, and exposing either to a model invalidates the episode.

Signatures

setup_problem(problem_id: str, use_hinted_problem: bool = False, extra_fields: dict[str, Any] | None = None) -> str

reproduce(filename: str) -> str

submit(pov: str, summary: str) -> str

read_file(file_path: str, line_start: int | None = None, line_end: int | None = None) -> str

edit_file(file_path: str, old_text: str, new_text: str, replace_all: bool = False) -> str

bash(command: str, cwd: str | None = None) -> str

verify(final: bool = False) -> str

grade_problem(problem_id: str, transcript: str, output_directory: str | None = None, extra_fields: dict[str, Any] | None = None) -> Grade

bash runs a command inside the environment as an unprivileged user and returns its combined output. It is a persistent session: a directory change or an exported variable in one call is still in effect in the next.