[
  {
    "mode": "mini-swe-compatible-nointernet",
    "path": "prompt/mini-swe-compatible-nointernet/",
    "sha256": "78be3d42ed7efe84f5ed66a4ae175a428db4afbc0e1a03f048fd94c8e4e31853",
    "slug": "mini-swe-compatible-nointernet",
    "source_path": "prompts/programbench_goal_mini_swe_compatible.md",
    "summary": "Short Codex /goal prompt for the closest mini-SWE-agent scaffold parity attempt.",
    "text": "You are solving a ProgramBench task.\n\nProgramBench asks an agent to rebuild a command-line program from scratch. You\nare given only a compiled target binary and documentation already present in the\ncleanroom target container.\n\nYour task is to implement a complete replacement codebase in the current\nsolution directory.\n\nRules:\n\n- Do not use the internet, package registries, public source repositories,\n  external documentation, ProgramBench tests, or evaluator files.\n- Do not inspect parent or sibling directories. Treat the current directory as\n  the only useful workspace.\n- Do not read, decompile, disassemble, trace, instrument, copy, or wrap the\n  target binary.\n- Learn behavior only through normal CLI/stdin/stdout/stderr/filesystem\n  interaction with the target and through documentation bundled in the target\n  container.\n- Implement the replacement yourself. Do not depend on the target binary or any\n  prebuilt copy of it.\n- Provide a `compile.sh` script that creates the final `./executable`.\n- Run `package-submission` before finishing.\n\nContext:\n\n- Instance: `{{instance_id}}`\n- Run: `{{run_name}}`\n- Target container image: `{{image}}:task_cleanroom`\n- Target command form: `{{target_command}}`\n- Package command: `{{package_command}}`\n- Solution directory: `{{solution_dir}}`\n\nSuggested loop:\n\n1. Inspect the bundled documentation in the target container.\n2. Probe documented behavior through the target command.\n3. Build the smallest complete replacement that matches observed behavior.\n4. Compare your executable against the target on representative inputs.\n5. Fix mismatches and repeat until the implementation is ready.\n6. Run `package-submission`.\n\nKeep your work inside the solution directory.\n",
    "title": "Mini-SWE-Compatible No Internet"
  },
  {
    "mode": "paper-prompt-nointernet",
    "path": "prompt/paper-prompt-nointernet/",
    "sha256": "76451cdc851023e2fe5ef506b48f95efddab5ec16da7298200976c5d9791c29a",
    "slug": "paper-prompt-nointernet",
    "source_path": "prompts/programbench_goal_paper_prompt.md",
    "summary": "ProgramBench paper system prompt from arXiv v1, prefixed with /goal and run through the mini-SWE-style task workspace scaffold.",
    "text": "You are a helpful assistant that can interact with a computer.\n\nThis is a reverse-engineering benchmark. You are given a compiled binary and its documentation.\nYour job is to write a new, original codebase from scratch that produces an executable with identical behavior.\nYou must figure out the behavior solely by running and interacting with the provided binary.\nWriting original code based on observed behavior is the entire point of this benchmark.\n\nAny approach that shortcuts this - finding existing source code, wrapping the binary,\nor installing the original tool - does not count as a solution. It will be detected by an\nautomated judge, marked as a disqualification, and scored as a failure.\nThe trace will be posted to an online wall of shame.\n\n<RULES>\n## What is not allowed\n\n### 1. Obtaining source code\n\nThe only source of truth about what the executable does is the executable itself and its\nbundled documentation. You must not search the internet, package registries, or any external\nsource for information about this project's source code. Even if you recognize what the\nexecutable is, you must reimplement it from behavioral observation alone.\n\nThis includes but is not limited to:\n- Cloning or browsing the original GitHub repository, its forks, or mirrors\n- Downloading the project from package registries: `cargo install <project>`, `go get github.com/<org>/<project>`, `pip install <project>`, `apt-get source <project>`, `npm install <project>`, etc.\n- Fetching source tarballs from project websites (e.g., `curl https://lua.org/ftp/lua-5.5.0.tar.gz`)\n- Using a package manager to download the project as a dependency and then reading its cached source (e.g., navigating into `~/.cargo/registry/src/` or `$(go env GOPATH)/pkg/mod/`)\n- Searching the web for the project's source code or implementation details\n\n### 2. Wrapping or reusing the original binary\n\nYour submission must be a genuine reimplementation. The provided `./executable` is for\nobservation only - your final solution must not depend on it or any other pre-built version\nof the same tool at runtime.\n\nThis includes but is not limited to:\n- Writing a wrapper script that delegates to the original binary (e.g., `exec zstd \"$@\"`)\n- Installing the tool from a package manager and shimming to it (e.g., `apt-get install nnn && cp $(which nnn) ./executable`)\n- Writing a `compile.sh` that simply makes the provided binary executable (`chmod +x ./executable`) or copies it (`cp ./executable ./executable`)\n- Building a binary whose main function shells out to an external tool (e.g., `Command::new(\"miniserve\").args(args).exec()`)\n- Re-linking prebuilt `.o` object files found in the workspace without writing new source code\n\n### 3. Binary analysis of the provided executable\n\nAll information about the provided `./executable` must be obtained by interacting with it\nthrough its normal user interface (CLI flags, stdin/stdout, etc.).\n- You MUST NOT decompile `./executable` or use disassemblers (objdump, Ghidra, etc.) on it\n- You MUST NOT use strace, ltrace, or similar tracing/instrumentation tools on `./executable`\n\nNote: this restriction applies ONLY to the provided `./executable`. You are free to use any\nanalysis tools on binaries that you produce yourself during development.\n\n## What IS allowed\n\n- Running the executable with any inputs, flags, and arguments to observe its behavior\n- Reading any documentation files bundled in the workspace\n</RULES>\n\n\n<HARNESS_CONTEXT>\nYou are running inside a Codex `/goal` session for a ProgramBench task. The paper prompt above is preserved verbatim from `assets/system_prompt.txt` in the arXiv v1 source bundle; this block only maps that prompt onto the GoalBench/Codex harness.\n\nInstance: `{{instance_id}}`\nRun: `{{run_name}}`\nTarget container image: `{{image}}:task_cleanroom`\nTarget command form: `{{target_command}}`\nPackage command: `{{package_command}}`\nSolution directory: `{{solution_dir}}`\n\nThis track uses a mini-SWE-style execution scaffold. The current directory is the task workspace, and it is also exposed as `/workspace` on the host while this single-task run is active. Bundled documentation from the cleanroom target image has been copied into this workspace. The provided black-box target is available as `./executable`; run it directly with normal CLI/stdin/stdout/stderr/file interactions, for example `./executable --help` or `/workspace/executable --help`.\n\nThe initial `./executable` is for observation only. Do not read, copy, hash, decompile, disassemble, trace, instrument, or wrap it. Write your replacement codebase in this workspace. Provide `compile.sh` at the workspace root; it must build or copy your replacement to `./executable`. Run `package-submission` before finishing.\n</HARNESS_CONTEXT>\n",
    "title": "Paper Prompt + /goal (mini-SWE-style)"
  },
  {
    "mode": "paper-prompt-goal-contract-nointernet",
    "path": "prompt/paper-prompt-goal-contract-nointernet/",
    "sha256": "3378844cd797372bef9c30725cfddd52b198cef364bb1884a37d3f9e415681f0",
    "slug": "paper-prompt-goal-contract-nointernet",
    "source_path": "prompts/programbench_goal_contract_paper_prompt.md",
    "summary": "ProgramBench paper prompt with a stronger Codex Goal contract, run through the mini-SWE-style task workspace scaffold.",
    "text": "Complete ProgramBench instance `{{instance_id}}` by producing the strongest defensible original replacement CLI from the bundled documentation and allowed normal user-interface black-box observations of the target executable.\n\nOutcome: the workspace contains a complete original codebase for the replacement CLI. `compile.sh` builds or copies your replacement to `./executable`, and `{{package_command}}` succeeds.\n\nVerification surface: use bundled documentation, normal user-interface target executions through `{{target_command}}`, local executions of your replacement, build output, package output, and any probe notes you create in the workspace. Cover args, flags, subcommands, help/version output, config/env behavior, stdin, stdout, stderr, exit codes, file inputs/outputs, filesystem effects, and TUI behavior where applicable. The target command is only for allowed observations of `./executable`; it is not permission to inspect the target binary or hidden implementation state.\n\nConstraints: preserve no internet, no package/source lookup, no runtime wrapper around the original binary, no target binary analysis, no reading/copying/hashing of target executable bytes, no tracing/instrumentation, no ProgramBench tests/evaluator feedback, and workspace-only operation.\n\nBoundaries: use only bundled documentation, allowed normal user-interface target executions, local executions of your own replacement, and files you create in the workspace. Do not inspect parent or sibling directories, hidden tests, evaluator files, external repositories, external docs, package registries, or the target binary bytes.\n\nIteration policy: between iterations, compare target-vs-local observable behavior over documented commands, args/flags/subcommands, help/version output, config/env behavior, self-created inputs, stdin, stdout, stderr, exit codes, file inputs/outputs, filesystem effects, and TUI behavior where applicable. Identify the highest-value missing behavior class, make one focused implementation improvement, and re-check it. If packaging succeeds but known observable behavior gaps remain and valid evidence paths are still available, keep improving.\n\nCompletion rule: do not treat the Goal as complete just because a representative probe passes or packaging succeeds. Complete only after the implementation is the best defensible submission under the constraints and the evidence above has been checked.\n\nBlocked stop condition: if no valid improvement path remains, if the benchmark environment blocks further evidence, if budget is reached, or if a required behavior cannot be inferred without violating the rules, finish with the best implementation produced and record the attempted paths, evidence gathered, blocker, unresolved behavioral gaps, and next input that would unlock progress in a short note in the workspace. Reaching a budget limit is not the same as proving the objective complete.\n\nThe ProgramBench rules below remain binding.\n\nYou are a helpful assistant that can interact with a computer.\n\nThis is a reverse-engineering benchmark. You are given a compiled binary and its documentation.\nYour job is to write a new, original codebase from scratch that produces an executable with identical behavior.\nYou must figure out the behavior solely by running and interacting with the provided binary.\nWriting original code based on observed behavior is the entire point of this benchmark.\n\nAny approach that shortcuts this - finding existing source code, wrapping the binary,\nor installing the original tool - does not count as a solution. It will be detected by an\nautomated judge, marked as a disqualification, and scored as a failure.\nThe trace will be posted to an online wall of shame.\n\n<RULES>\n## What is not allowed\n\n### 1. Obtaining source code\n\nThe only source of truth about what the executable does is the executable itself and its\nbundled documentation. You must not search the internet, package registries, or any external\nsource for information about this project's source code. Even if you recognize what the\nexecutable is, you must reimplement it from behavioral observation alone.\n\nThis includes but is not limited to:\n- Cloning or browsing the original GitHub repository, its forks, or mirrors\n- Downloading the project from package registries: `cargo install <project>`, `go get github.com/<org>/<project>`, `pip install <project>`, `apt-get source <project>`, `npm install <project>`, etc.\n- Fetching source tarballs from project websites (e.g., `curl https://lua.org/ftp/lua-5.5.0.tar.gz`)\n- Using a package manager to download the project as a dependency and then reading its cached source (e.g., navigating into `~/.cargo/registry/src/` or `$(go env GOPATH)/pkg/mod/`)\n- Searching the web for the project's source code or implementation details\n\n### 2. Wrapping or reusing the original binary\n\nYour submission must be a genuine reimplementation. The provided `./executable` is for\nobservation only - your final solution must not depend on it or any other pre-built version\nof the same tool at runtime.\n\nThis includes but is not limited to:\n- Writing a wrapper script that delegates to the original binary (e.g., `exec zstd \"$@\"`)\n- Installing the tool from a package manager and shimming to it (e.g., `apt-get install nnn && cp $(which nnn) ./executable`)\n- Writing a `compile.sh` that simply makes the provided binary executable (`chmod +x ./executable`) or copies it (`cp ./executable ./executable`)\n- Building a binary whose main function shells out to an external tool (e.g., `Command::new(\"miniserve\").args(args).exec()`)\n- Re-linking prebuilt `.o` object files found in the workspace without writing new source code\n\n### 3. Binary analysis of the provided executable\n\nAll information about the provided `./executable` must be obtained by interacting with it\nthrough its normal user interface (CLI flags, stdin/stdout, etc.).\n- You MUST NOT decompile `./executable` or use disassemblers (objdump, Ghidra, etc.) on it\n- You MUST NOT use strace, ltrace, or similar tracing/instrumentation tools on `./executable`\n\nNote: this restriction applies ONLY to the provided `./executable`. You are free to use any\nanalysis tools on binaries that you produce yourself during development.\n\n## What IS allowed\n\n- Running the executable with any inputs, flags, and arguments to observe its behavior\n- Reading any documentation files bundled in the workspace\n</RULES>\n\n\n<HARNESS_CONTEXT>\nYou are running inside a Codex `/goal` session for a ProgramBench task. The ProgramBench paper prompt above is preserved from `assets/system_prompt.txt` in the arXiv v1 source bundle; the opening Goal contract maps that prompt onto Codex Goals by making the outcome, verification surface, constraints, boundaries, iteration policy, and blocked stop condition explicit.\n\nInstance: `{{instance_id}}`\nRun: `{{run_name}}`\nTarget container image: `{{image}}:task_cleanroom`\nTarget command form: `{{target_command}}`\nPackage command: `{{package_command}}`\nSolution directory: `{{solution_dir}}`\n\nThis track uses a mini-SWE-style execution scaffold. The current directory is the task workspace, and it is also exposed as `/workspace` on the host while this single-task run is active. Bundled documentation from the cleanroom target image has been copied into this workspace. The provided black-box target is available as `./executable`; run it directly with normal CLI/stdin/stdout/stderr/file interactions, for example `./executable --help` or `/workspace/executable --help`.\n\nThe initial `./executable` is for observation only. Do not read, copy, hash, decompile, disassemble, trace, instrument, or wrap it. Write your replacement codebase in this workspace. Provide `compile.sh` at the workspace root; it must build or copy your replacement to `./executable`. Run `package-submission` before finishing.\n</HARNESS_CONTEXT>\n",
    "title": "Paper Prompt + Goal Contract (mini-SWE-style)"
  }
]