BugcrowdBugcrowdSavant Environments

Option reference

Every option the server accepts is below, grouped by what it affects.

The startup command delivered with each environment already contains the right values. Read this section to understand a run, and to change the things that are yours to change — the artifacts directory, the log level, the tool timeout. The weights, budgets and thresholds are part of the task definition: an environment run with different ones is measuring something different, and its score cannot be compared with anyone else's, including your own from last month.

Every environment

  • --tools (default: empty) — Comma-separated list of the tools to expose. Omitting it exposes every tool, which is a misconfigured launch rather than a supported mode.

  • --output-directory (default: not set) — Directory the grader fills with per-episode artifacts for a frontend to collect after grading; when omitted, no artifacts are written

  • --tool-timeout-seconds (default: 60) — Seconds a single bash tool call may run before its session is killed and restarted

  • --log-level (default: INFO) — Log level for rlenv-mcp messages One of DEBUG, INFO, WARNING, ERROR.

  • --call (default: --) — Call a single tool, print its result, and exit. Useful for checking an environment responds before wiring up an agent.

Bug report detail (exploit, patch)

  • --context-level (default: adaptive) — Context level for the bug report in patch and exploit problems (0=CWE only, 1=source file, 2=full backtrace, 3=with line numbers). Use 'adaptive' to infer the level from same-CWE defect density.

Exploit and detect scoring

  • --grade-dimension (default: 4) — Dimensionality of the grade for exploit/detect problems (1 or 4). One of 1, 4.

  • --grade-temperature (default: 0.1) — Temperature for 1D grading logistic function. Lower values create steeper curves.

  • --weight-defect-matching (default: 0.6) — Weight for defect_matching factor

  • --weight-pov-quality (default: 0.2) — Weight for pov_quality factor

  • --weight-summary-quality (default: 0.1) — Weight for summary_quality factor

  • --weight-target-calls (default: 0.1) — Weight for target_calls factor

  • --failure-threshold (default: not set) — Minimum score threshold. Scores below this are set to 0. Optional, no default.

  • --max-attempts (default: 20) — Maximum number of exploit/detect attempts allowed

Patch scoring

  • --weight-patch-verified (default: 0.25) — Weight for patch_verified factor

  • --weight-number-of-attempts (default: 0.25) — Weight for number_of_attempts factor

  • --weight-size-of-patch (default: 0.25) — Weight for size_of_patch factor

  • --weight-minimization (default: 0.25) — Weight for minimization factor

  • --max-patch-attempts (default: 3) — Maximum number of patch attempts allowed

  • --min-patch-size (default: 2) — Patch size (lines) at or below which full size credit is awarded

  • --max-patch-size (default: 8) — Patch size (lines) at or above which zero size credit is awarded

  • --max-line-length (default: 100) — Added lines longer than this zero the patch-size score

  • --minimization-half-multiple (default: 10.0) — Final patch size, as a multiple of max-patch-size, at which minimization credit is halved; the tail is asymptotic and never reaches 0

  • --functionality-loss-threshold (default: 0.75) — Coverage feature-loss fraction at which a libFuzzer patch's credit reaches zero; credit decreases linearly from full at zero loss

  • --max-regression-tests (default: 100) — Maximum number of passing (regression) testcases sampled during validate/verify/grade. The sample is seeded, so all three select the identical subset for a given value

  • --max-defect-tests (default: 100) — Maximum number of the target defect's own PoVs replayed during verify. Every input in the directory triggers the same defect, so a seeded sample is enough to decide whether it still reproduces

  • --max-adjacent-tests (default: 30) — Maximum number of adjacent-defect PoVs replayed during verify. The sweep is advisory — it only annotates a PASS — so it is capped and may be cut short without failing the attempt

Audit scoring

  • -k, --submissions-per-defect (default: 3) — Maximum submissions per known defect for audit environments