Codex Current Results
| # | Model | Run | Agent | Resolved | Almost | Avg. est. cost | Avg. calls | Avg /goal | Max /goal |
|---|---|---|---|---|---|---|---|---|---|
| 1 | GPT 5.5 (xhigh) | 20260517T094700Z |
Codex /goal | 0.5% (1/200) | 6.0% (12/200) | $4.89 | 80.2 | 14.7m | 27.2m |
| 2 | GPT 5.5 (xhigh) | 20260518T232006Z |
Codex /goal | 0.0% (0/200) | 0.5% (1/200) | $3.54* | 59.5 | 11.4m | 25.6m |
| 3 | GPT 5.5 (xhigh) | 20260519T215046Z |
Codex /goal | 0.0% (0/197) | 0.0% (0/197) | $3.95* | 67.7 | 13.2m | 22.5m |
Goal Session Duration
Wall-clock time is measured per Codex /goal session from launch to packaged submission. ProgramBench's public mini-SWE-agent runs use a much larger timeout, so this block makes latency differences explicit.
Longest task: hairyhenderson__gomplate.05eb3aa at 27.2 min.
Official Baseline Context
For orientation only. ProgramBench's public extended table reports mini-SWE-agent over 200 tasks, sorted by resolved, almost-resolved, then average pass rate.
| # | Model | Agent | Resolved | Almost | Avg. cost | Avg. calls |
|---|---|---|---|---|---|---|
| 1 | GPT 5.5 (xhigh) | mini-SWE-agent | 0.5% | 13.5% | $8.85 | 82 |
| 2 | GPT 5.5 (high) | mini-SWE-agent | 0.5% | 5.0% | $3.65 | 41 |
| 3 | Claude Opus 4.7 (xhigh) | mini-SWE-agent | 0.0% | 4.5% | $10.96 | 159 |
| 4 | Claude Opus 4.7 | mini-SWE-agent | 0.0% | 3.0% | $3.81 | 93 |
| 5 | Claude Opus 4.6 | mini-SWE-agent | 0.0% | 2.5% | $11.38 | 260 |
| 6 | GPT 5.5 | mini-SWE-agent | 0.0% | 1.5% | $1.21 | 19 |
| 7 | Claude Sonnet 4.6 | mini-SWE-agent | 0.0% | 1.0% | $26.73 | 472 |
| 8 | GPT 5.4 | mini-SWE-agent | 0.0% | 0.0% | $0.33 | 16 |
| 9 | Gemini 3.1 Pro | mini-SWE-agent | 0.0% | 0.0% | $1.51 | 94 |
| 10 | Gemini 3 Flash | mini-SWE-agent | 0.0% | 0.0% | $0.30 | 85 |
| 11 | Claude Haiku 4.5 | mini-SWE-agent | 0.0% | 0.0% | $0.80 | 124 |
| 12 | GPT 5.4 mini | mini-SWE-agent | 0.0% | 0.0% | $0.04 | 18 |
| 13 | GPT 5 mini | mini-SWE-agent | 0.0% | 0.0% | $0.03 | 15 |
GPT-5.5 baseline rows link to ProgramBench's official run-detail pages for total cost, total calls, distribution plots, and all 200 per-instance results.
Method Notes
GoalBench reports separate Codex /goal runs on ProgramBench tasks; these are not official mini-SWE-agent leaderboard submissions. Resolved means ProgramBench's filtered behavioral pass rate is exactly 100%, and almost resolved means at least 95%.
GoalBench currently keeps three public prompt tracks separate: mini-swe-compatible-nointernet for the already-published mini-SWE-compatible result, paper-prompt-nointernet for the ProgramBench paper prompt with /goal and mini-SWE-style execution, and paper-prompt-goal-contract-nointernet for the same scaffold with an explicit Codex Goal contract.
GoalBench uses a host-side wrapper to transport allowed black-box CLI interactions into the target container. This differs from mini-SWE-agent's in-container execution, but the wrapper is restricted to normal user-interface observations of the target executable and forbids source lookup, binary reading, disassembly, tracing, instrumentation, and evaluator/test access.
The public table is scoped to the latest published result set. Cost is estimated from Codex token logs, not billing; rows marked n/a or aggregate costs marked * have incomplete local token-log coverage, so calls are the more complete compute proxy for those tracks. See Task Details and the runbook for scoring, evidence, egress, and setup details. Sources: ProgramBench extended results and GPT 5.5 xhigh run detail.