GPT 5.5 (xhigh) · Mini-SWE-compatible no internet · 20260517T094700Z
200instances
0.5%resolved
6.0%almost
63.6%avg pass
$977est. from token logs
16,038total calls
$4.89est. cost / costed task
80.2calls / task
14.7mavg /goal session
27.2mmax /goal session
GPT 5.5 (xhigh) · Paper prompt + /goal no internet · 20260518T232006Z
200instances
0.0%resolved
0.5%almost
53.4%avg pass
$142*token logs for 40/200 tasks
11,900total calls
$3.54*est. cost / costed task
59.5calls / task
11.4mavg /goal session
25.6mmax /goal session
GPT 5.5 (xhigh) · Paper prompt + Goal contract no internet · 20260519T215046Z
197instances
0.0%resolved
0.0%almost
52.5%avg pass
$158*token logs for 40/197 tasks
13,337total calls
$3.95*est. cost / costed task
67.7calls / task
13.2mavg /goal session
22.5mmax /goal session
Latency
Goal Session Duration
Wall-clock time is measured per Codex /goal session from launch to packaged submission. ProgramBench's public mini-SWE-agent runs use a much larger timeout, so this block makes latency differences explicit.
Average0.244h14.7 min
Median0.234h14.0 min
P750.281h16.9 min
P900.329h19.8 min
P950.363h21.8 min
Max0.454h27.2 min
Over 6h0 / 2000.0%
<15 min
121 / 200
60.5%
15-30 min
79 / 200
39.5%
>30 min
0 / 200
0.0%
Longest task: hairyhenderson__gomplate.05eb3aa at 27.2 min.
Score by Model × Task
Behavioral Test Pass Rate Distribution
These plots mirror ProgramBench's run-detail views: a score histogram and a cumulative count by behavioral test pass-rate bucket.
Histogram
Cumulative
Efficiency Plots
Each point is one evaluated task. Compute is shown as Codex calls for every row; cost plots include only rows with available local token logs.
Pass vs. Est. Cost
Pass vs. Calls
Pass vs. Latency
Extended Results
| # | Model | Run | Agent | Resolved | Almost | Avg. est. cost | Avg. calls | Avg /goal | Max /goal |
|---|---|---|---|---|---|---|---|---|---|
| 1 | GPT 5.5 (xhigh) | 20260517T094700Z |
Codex /goal | 0.5% (1/200) | 6.0% (12/200) | $4.89 | 80.2 | 14.7m | 27.2m |
| 2 | GPT 5.5 (xhigh) | 20260518T232006Z |
Codex /goal | 0.0% (0/200) | 0.5% (1/200) | $3.54* | 59.5 | 11.4m | 25.6m |
| 3 | GPT 5.5 (xhigh) | 20260519T215046Z |
Codex /goal | 0.0% (0/197) | 0.0% (0/197) | $3.95* | 67.7 | 13.2m | 22.5m |
Columns mirror ProgramBench's extended leaderboard shape while adding the latency fields needed for Codex /goal: average and max wall-clock session duration from launch to packaged submission. Run versions keep repeated same-config sweeps separate instead of silently merging attempts.
Run Disclosures
| # | Model | Run | Mode | Compliance | Host profile | Tasks | Avg. pass | Prompt | Avg /goal | Max /goal | Total wall |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | GPT 5.5 (xhigh) | 20260517T094700Z |
Mini-SWE-compatible no internet | Codex /goal mini-SWE-compatible no-internet | smaller VM: 16 CPU / 30g | 200/200 | 63.6% | Mini-SWE-Compatible No Internet | 14.7m | 27.2m | 48.89h |
| 2 | GPT 5.5 (xhigh) | 20260518T232006Z |
Paper prompt + /goal no internet | Codex /goal paper prompt no-internet, mini-SWE-style execution | smaller VM: 16 CPU / 30g | 200/200 | 53.4% | Paper Prompt + /goal (mini-SWE-style) | 11.4m | 25.6m | 38.11h |
| 3 | GPT 5.5 (xhigh) | 20260519T215046Z |
Paper prompt + Goal contract no internet | Codex /goal contract no-internet, mini-SWE-style execution | smaller VM: 16 CPU / 30g | 197/200 | 52.5% | Paper Prompt + Goal Contract (mini-SWE-style) | 13.2m | 22.5m | 43.45h |
These disclosure fields make scaffold differences explicit: prompt, compliance label, host size, per-session latency, and total wall-clock sum. Rows labeled smaller VM are Codex /goal scaffold experiments on the disclosed runner size.
Model Comparison
| Y model | X model | Resolved Δ | Almost Δ | Avg. pass Δ |
|---|---|---|---|---|
| GPT 5.5 (xhigh) | GPT 5.5 (xhigh) | 0.5% | 5.5% | 10.3% |
| GPT 5.5 (xhigh) | GPT 5.5 (xhigh) | 0.5% | 6.0% | 11.2% |
| GPT 5.5 (xhigh) | GPT 5.5 (xhigh) | -0.5% | -5.5% | -10.3% |
| GPT 5.5 (xhigh) | GPT 5.5 (xhigh) | 0.0% | 0.5% | 0.9% |
| GPT 5.5 (xhigh) | GPT 5.5 (xhigh) | -0.5% | -6.0% | -11.2% |
| GPT 5.5 (xhigh) | GPT 5.5 (xhigh) | 0.0% | -0.5% | -0.9% |
Task Details
Per-task pages show official ProgramBench context, GoalBench rows by model/mode, failed-test evidence, and baseline links. See how to read task pages.
| # | Task | Generated tests | Official best | Codex best | Codex model | Codex rows | Official rows | ProgramBench |
|---|---|---|---|---|---|---|---|---|
| 1 | abishekvashok__cmatrix.5c082c6 |
507 | 100.0% | 100.0% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 2 | wfxr__csview.8ac4de0 |
335 | 99.7% | 98.8% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 3 | jarun__nnn.cb2c535 |
477 | 98.1% | 98.1% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 4 | xorg62__tty-clock.f2f847c |
281 | 86.5% | 97.2% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 5 | wintermute-cell__ngrrram.8ea13c3 |
303 | 97.0% | 97.0% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 6 | blake3-team__blake3.15e83a5 |
647 | 97.8% | 96.3% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 7 | htop-dev__htop.523600b |
693 | 98.1% | 96.1% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 8 | sheepla__pingu.926d475 |
383 | 98.7% | 95.8% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 9 | sitkevij__hex.61ae69b |
823 | 99.6% | 95.5% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 10 | sstadick__hck.b66c751 |
855 | 97.2% | 95.1% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 11 | miserlou__loop.209927c |
710 | 97.3% | 95.1% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 12 | mgdm__htmlq.6e31bc8 |
1455 | 98.0% | 95.1% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 13 | ogham__dog.721440b |
1300 | 93.9% | 94.7% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 14 | raviqqe__muffet.a882908 |
293 | 89.4% | 94.5% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 15 | agourlay__zip-password-finder.704700d |
680 | 98.7% | 94.1% | GPT 5.5 (xhigh) | 2 | 13 | official |
| 16 | lymphatus__caesium-clt.a529b2e |
575 | 96.2% | 94.1% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 17 | tarka__xcp.5e5b448 |
1184 | 95.8% | 93.8% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 18 | clog-tool__clog-cli.7066cba |
575 | 98.4% | 93.6% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 19 | riquito__tuc.16fb471 |
1196 | 96.1% | 93.3% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 20 | wfxr__code-minimap.0ddeea5 |
313 | 95.8% | 93.3% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 21 | madler__pigz.fe4894f |
831 | 94.9% | 93.1% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 22 | sharkdp__hexyl.2e26437 |
906 | 95.4% | 92.9% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 23 | rust-ethereum__ethabi.b1710ad |
997 | 93.3% | 92.8% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 24 | nikolassv__bartib.6b9b5ce |
722 | 96.0% | 92.4% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 25 | rhysd__kiro-editor.4157485 |
595 | 95.3% | 92.3% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 26 | altdesktop__i3-style.f93821b |
539 | 93.3% | 92.0% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 27 | guumaster__hostctl.d6d9699 |
1051 | 82.8% | 91.8% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 28 | shashwatah__jot.a92aad8 |
752 | 97.1% | 91.8% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 29 | simeg__eureka.df3796c |
344 | 97.4% | 91.6% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 30 | astro__deadnix.d590041 |
602 | 91.7% | 91.5% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 31 | chmln__sd.87d1ba5 |
810 | 97.4% | 91.5% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 32 | ismaelgv__rnr.fc0733b |
683 | 94.0% | 90.8% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 33 | sirwart__ripsecrets.34c9e03 |
611 | 91.2% | 90.5% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 34 | svenstaro__miniserve.8449e8b |
304 | 92.1% | 90.5% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 35 | sharkdp__pastel.b60e899 |
1114 | 93.1% | 89.9% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 36 | anordal__shellharden.6a6ffd4 |
1095 | 88.2% | 89.6% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 37 | oppiliappan__eva.41ae245 |
913 | 98.4% | 88.9% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 38 | tomnomnom__gron.88a6234 |
224 | 90.2% | 88.8% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 39 | burntsushi__xsv.f430466 |
1182 | 90.7% | 88.8% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 40 | rs__jplot.2a54bcc |
583 | 91.3% | 88.5% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 41 | yaa110__nomino.f892499 |
313 | 94.2% | 88.5% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 42 | pemistahl__grex.fa3e8ed |
1312 | 97.2% | 88.5% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 43 | kyoheiu__felix.95df390 |
502 | 88.2% | 88.4% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 44 | arthursonzogni__json-tui.17a22b6 |
755 | 90.2% | 88.2% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 45 | ekzhang__bore.8e059cd |
394 | 93.1% | 87.8% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 46 | byron__dua-cli.8570c15 |
709 | 93.1% | 87.6% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 47 | mibk__dupl.1bf052b |
373 | 92.8% | 87.4% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 48 | johanneskaufmann__html-to-markdown.3006818 |
885 | 92.8% | 87.3% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 49 | rs__curlie.5dfcbb1 |
701 | 92.9% | 87.0% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 50 | brocode__fblog.3b54330 |
978 | 87.8% | 86.7% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 51 | mookid__diffr.2152742 |
606 | 90.6% | 86.6% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 52 | kyoh86__richgo.313114f |
546 | 88.1% | 86.6% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 53 | halitechallenge__halite.822cfb6 |
275 | 95.6% | 85.5% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 54 | zevv__duc.a58fa4e |
874 | 93.0% | 85.4% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 55 | hooklift__gowsdl.2a06cec |
391 | 90.8% | 85.2% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 56 | eradman__entr.8e2e8b4 |
586 | 91.8% | 85.2% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 57 | jonas__tig.8334123 |
1586 | 85.2% | 85.1% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 58 | chmln__handlr.90e78ba |
722 | 91.4% | 84.5% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 59 | mfridman__tparse.2416b4b |
425 | 82.4% | 84.2% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 60 | cmatsuoka__figlet.202a0a8 |
872 | 77.5% | 84.1% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 61 | hatoo__oha.8dc6349 |
899 | 78.6% | 83.9% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 62 | psampaz__go-mod-outdated.bb79367 |
285 | 98.2% | 83.9% | GPT 5.5 (xhigh) | 2 | 13 | official |
| 63 | sibprogrammer__xq.b89f681 |
792 | 86.5% | 83.2% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 64 | eudoxia0__hashcards.48aa136 |
1019 | 86.8% | 82.9% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 65 | alexpovel__srgn.89f943b |
1852 | 80.1% | 82.3% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 66 | esubaalew__run.0fb9dec |
1212 | 88.0% | 82.2% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 67 | junegunn__fzf.b56d614 |
1858 | 82.4% | 82.1% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 68 | lz4__lz4.1519f46 |
1496 | 87.9% | 82.0% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 69 | burntsushi__ripgrep.3b7fd44 |
1994 | 84.1% | 81.9% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 70 | sharkdp__fd.40d8eb3 |
1235 | 82.8% | 81.6% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 71 | incu6us__goimports-reviser.81bd549 |
513 | 89.7% | 81.1% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 72 | ksxgithub__parallel-disk-usage.96978ed |
531 | 95.7% | 81.0% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 73 | bensadeh__tailspin.6278437 |
615 | 88.0% | 81.0% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 74 | pier-cli__pier.5e1bde9 |
692 | 94.2% | 80.1% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 75 | astaxie__bat.17d1080 |
1091 | 79.5% | 79.7% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 76 | blacknon__hwatch.edfcb62 |
1016 | 85.3% | 79.7% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 77 | eliukblau__pixterm.1a93fd5 |
430 | 85.8% | 79.3% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 78 | nuta__nsh.bdd0702 |
1963 | 83.7% | 79.3% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 79 | facebookresearch__fasttext.1142dc4 |
312 | 80.4% | 79.2% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 80 | noborus__ov.b96c2ba |
1854 | 87.6% | 79.0% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 81 | o2sh__onefetch.e5958ce |
1166 | 85.1% | 78.9% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 82 | segmentio__chamber.5f93f5f |
1748 | 88.0% | 78.9% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 83 | codesnap-rs__codesnap.f81e4f3 |
571 | 82.7% | 78.5% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 84 | skeema__skeema.6a76243 |
1708 | 79.7% | 78.3% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 85 | gabotechs__dep-tree.60a95a2 |
865 | 77.8% | 77.8% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 86 | lua__lua.c6b4848 |
1338 | 76.2% | 75.9% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 87 | sayanarijit__xplr.1751065 |
419 | 78.5% | 75.9% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 88 | ajeetdsouza__zoxide.67ca1bc |
531 | 87.2% | 75.9% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 89 | rbakbashev__elfcat.52f8cc7 |
564 | 98.2% | 75.9% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 90 | nachoparker__dutree.44e877d |
641 | 96.1% | 75.5% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 91 | bootandy__dust.62bf1e1 |
584 | 87.3% | 75.3% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 92 | konradsz__igrep.aa75630 |
385 | 86.2% | 75.3% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 93 | isona__dirble.e2dea9f |
718 | 81.5% | 75.2% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 94 | antonmedv__walk.bf802ef |
470 | 89.6% | 75.1% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 95 | ggreer__the_silver_searcher.a61f178 |
1004 | 85.7% | 75.0% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 96 | git-bahn__git-graph.87b4473 |
568 | 93.7% | 75.0% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 97 | rust-embedded__svd2rust.1760b5e |
920 | 82.0% | 74.9% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 98 | canop__rhit.ae90bcb |
817 | 73.2% | 74.7% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 99 | multiprocessio__dsq.c3ae0ba |
542 | 80.3% | 74.2% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 100 | yoav-lavi__melody.f4af9b4 |
1205 | 87.4% | 74.1% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 101 | jqlang__jq.b33a763 |
6072 | 89.9% | 74.0% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 102 | lfos__calcurse.49180d5 |
666 | 75.8% | 73.4% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 103 | drew-alleman__datasurgeon.d257cee |
502 | 80.7% | 72.9% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 104 | dundee__gdu.ede21d2 |
1161 | 78.9% | 72.9% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 105 | ninja-build__ninja.cc60300 |
1438 | 82.0% | 72.7% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 106 | dalance__amber.69a0f52 |
565 | 74.3% | 72.2% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 107 | unhappychoice__gittype.34b72d0 |
741 | 91.3% | 71.9% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 108 | lh3__seqtk.94e7070 |
429 | 74.6% | 71.6% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 109 | naggie__dstask.ff57396 |
1278 | 76.9% | 71.4% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 110 | foriequal0__git-trim.07c2f50 |
509 | 79.0% | 71.3% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 111 | jrnxf__thokr.09375ef |
445 | 93.0% | 71.2% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 112 | luajit__luajit.a553b3d |
2967 | 78.0% | 71.2% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 113 | orf__gping.26eb5b9 |
339 | 88.2% | 71.1% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 114 | noborus__trdsql.d8c5ff6 |
1312 | 76.4% | 70.9% | GPT 5.5 (xhigh) | 2 | 13 | official |
| 115 | xampprocky__tokei.505d648 |
732 | 69.5% | 70.4% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 116 | direnv__direnv.02040c7 |
849 | 80.9% | 69.8% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 117 | facebook__zstd.1168da0 |
2038 | 68.8% | 69.6% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 118 | hairyhenderson__gomplate.05eb3aa |
2926 | 74.7% | 69.5% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 119 | cheat__cheat.b8098dc |
297 | 71.7% | 69.4% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 120 | peco__peco.4e58dad |
1224 | 76.7% | 69.0% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 121 | pls-rs__pls.4e1ae50 |
332 | 76.5% | 69.0% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 122 | cordx56__rustowl.655bc5c |
589 | 88.5% | 68.4% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 123 | jhspetersson__fselect.c3559ca |
3115 | 76.4% | 68.3% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 124 | oppiliappan__statix.e9df54c |
815 | 69.0% | 67.6% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 125 | tinycc__tinycc.9b8765d |
1978 | 72.5% | 67.4% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 126 | thezoraiz__ascii-image-converter.d05a757 |
465 | 80.9% | 67.3% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 127 | kaushiksrini__parqeye.8072121 |
479 | 81.6% | 67.2% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 128 | rust-lang__mdbook.37273ba |
1114 | 72.5% | 67.1% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 129 | ammarabouzor__tui-journal.2b4540d |
1402 | 71.0% | 66.9% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 130 | ariga__atlas.6d81150 |
1318 | 72.5% | 66.8% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 131 | rochacbruno__marmite.7d4bc2d |
668 | 68.4% | 65.0% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 132 | ecumene__rust-sloth.051c559 |
380 | 66.8% | 64.7% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 133 | cweill__gotests.2a672c5 |
603 | 71.0% | 64.2% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 134 | y2z__monolith.8702e66 |
713 | 67.7% | 63.8% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 135 | quinn-rs__quinn.bb359cc |
522 | 84.5% | 63.2% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 136 | rcoh__angle-grinder.9c2fc88 |
1130 | 73.3% | 62.4% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 137 | canop__broot.d6c798e |
539 | 67.0% | 61.8% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 138 | axodotdev__oranda.27d60c7 |
767 | 64.9% | 61.1% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 139 | sharkdp__hyperfine.327d5f4 |
291 | 77.0% | 60.8% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 140 | robertdavidgraham__masscan.b99d433 |
2549 | 65.8% | 60.0% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 141 | jesseduffield__lazygit.1d0db51 |
855 | 63.4% | 59.9% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 142 | crowdagger__crowbook.ea214d7 |
807 | 60.3% | 59.9% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 143 | epistates__treemd.825c6dd |
1569 | 58.9% | 59.4% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 144 | kisielk__errcheck.dacab89 |
341 | 83.3% | 58.9% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 145 | yassinebridi__serpl.c48a9d7 |
446 | 65.0% | 58.7% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 146 | sharkdp__bat.f822bd0 |
799 | 61.5% | 58.6% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 147 | ys-l__flamelens.0b4dc33 |
224 | 62.1% | 58.0% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 148 | trasta298__keifu.3331426 |
262 | 87.0% | 57.3% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 149 | wgunderwood__tex-fmt.3f1aef6 |
455 | 80.7% | 56.9% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 150 | stranger6667__jsonschema.d52e881 |
2933 | 57.1% | 55.7% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 151 | cslarsen__jp2a.61d205f |
631 | 61.3% | 55.2% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 152 | nikoladucak__caps-log.2cf2d1e |
551 | 75.3% | 55.0% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 153 | stacked-git__stgit.430027d |
1488 | 48.7% | 53.2% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 154 | parcel-bundler__lightningcss.aa2ed1e |
2828 | 55.0% | 52.9% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 155 | ducaale__xh.4a6e44f |
1171 | 50.0% | 52.2% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 156 | svenstaro__genact.16f96e3 |
232 | 71.1% | 52.2% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 157 | elkowar__pipr.fae0b17 |
525 | 67.0% | 51.4% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 158 | chirlu__sox.42b3557 |
1202 | 38.9% | 50.3% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 159 | filosottile__age.706dfc1 |
676 | 63.5% | 50.3% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 160 | mkj__dropbear.75f699b |
678 | 65.5% | 48.5% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 161 | rvben__rumdl.2d75c4d |
3322 | 48.6% | 46.3% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 162 | paradigmxyz__solar.5190d0e |
1978 | 47.4% | 46.1% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 163 | ip7z__7zip.839151e |
1043 | 46.8% | 45.8% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 164 | arq5x__bedtools2.dd57059 |
1053 | 46.1% | 44.3% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 165 | tukaani-project__xz.1007bf0 |
1410 | 84.7% | 43.8% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 166 | go-critic__go-critic.9aea378 |
493 | 56.8% | 43.8% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 167 | boyter__scc.515f91c |
464 | 47.8% | 43.3% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 168 | tomarrell__wrapcheck.c058da1 |
480 | 81.9% | 42.1% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 169 | hpjansson__chafa.dd4d4c1 |
1931 | 58.4% | 42.0% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 170 | tree-sitter__tree-sitter.5e23cca |
1232 | 46.1% | 41.9% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 171 | typst__typst.88356d0 |
1724 | 32.7% | 41.5% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 172 | ivanceras__svgbob.6d00ad9 |
472 | 47.7% | 40.3% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 173 | stathissideris__ditaa.f2286c4 |
609 | 38.4% | 37.6% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 174 | mgechev__revive.201451e |
727 | 48.3% | 37.1% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 175 | dandavison__delta.acd758f |
950 | 38.9% | 35.9% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 176 | mikefarah__yq.602586d |
2000 | 39.5% | 34.5% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 177 | doxygen__doxygen.966d98e |
229 | 34.5% | 34.1% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 178 | zk-org__zk.10d93d5 |
1108 | 43.1% | 34.0% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 179 | samtools__samtools.aa823b5 |
1425 | 22.6% | 32.1% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 180 | johnkerl__miller.8d85b46 |
14637 | 24.1% | 32.1% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 181 | osgeo__gdal.0847f12 |
657 | 25.4% | 26.5% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 182 | google__brotli.b3dc9cc |
441 | 95.5% | 24.9% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 183 | ast-grep__ast-grep.dde0fe0 |
882 | 23.6% | 24.6% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 184 | osgeo__proj.75d455c |
5319 | 73.8% | 21.5% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 185 | nukesor__pueue.8b9d6fe |
638 | 16.1% | 17.1% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 186 | jgm__pandoc.5caad90 |
5228 | 14.7% | 17.0% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 187 | danmar__cppcheck.0a5b103 |
2126 | 14.6% | 16.3% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 188 | duckdb__duckdb.bdb65ec |
5650 | 16.8% | 16.3% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 189 | bellard__quickjs.d7ae12a |
3034 | 31.0% | 14.5% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 190 | tstack__lnav.ee34494 |
990 | 21.7% | 14.0% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 191 | gromacs__gromacs.665ea4c |
1245 | 9.3% | 14.0% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 192 | hush-shell__hush.560c33a |
1201 | 83.3% | 12.0% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 193 | php__php-src.c891263 |
14288 | 13.2% | 11.9% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 194 | ffmpeg__ffmpeg.360a402 |
3050 | 7.0% | 9.9% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 195 | antonmedv__fx.86d0d34 |
2047 | 75.7% | 8.6% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 196 | universal-ctags__ctags.243595e |
2258 | 13.3% | 7.8% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 197 | sclevine__yj.8016400 |
767 | 81.7% | 2.1% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 198 | sqlite__sqlite.839433d |
13514 | 71.6% | 1.3% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 199 | sigoden__argc.04a08f1 |
995 | 44.1% | 0.4% | GPT 5.5 (xhigh) | 3 | 13 | official |
| 200 | alecthomas__chroma.8d04def |
503 | 41.7% | 0.0% | GPT 5.5 (xhigh) | 3 | 13 | official |
Per-Instance Results
| # | Instance | Run | Mode | Model | Compliance | Status | Eval | Score | Tests | Est. cost | Calls | Wall | Host | Docker | Evidence |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | abishekvashok__cmatrix.5c082c6 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | resolved | ok | 100.0% | 507/507 | $3.54 | 55 | 0.17h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 2 | wfxr__csview.8ac4de0 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | almost | ok | 98.8% | 331/335 | $4.74 | 86 | 0.27h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 3 | jarun__nnn.cb2c535 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | almost | ok | 98.1% | 468/477 | $3.71 | 73 | 0.18h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 4 | abishekvashok__cmatrix.5c082c6 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | almost | ok | 97.4% | 494/507 | $1.81 | 41 | 0.10h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 5 | xorg62__tty-clock.f2f847c |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | almost | ok | 97.2% | 273/281 | $4.11 | 61 | 0.20h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 6 | wintermute-cell__ngrrram.8ea13c3 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | almost | ok | 97.0% | 294/303 | $4.21 | 84 | 0.20h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 7 | blake3-team__blake3.15e83a5 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | almost | ok | 96.3% | 623/647 | $5.75 | 97 | 0.25h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 8 | htop-dev__htop.523600b |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | almost | ok | 96.1% | 666/693 | $2.81 | 49 | 0.15h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 9 | sheepla__pingu.926d475 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | almost | ok | 95.8% | 367/383 | $2.58 | 49 | 0.20h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 10 | sitkevij__hex.61ae69b |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | almost | ok | 95.5% | 786/823 | $2.48 | 46 | 0.14h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 11 | sstadick__hck.b66c751 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | almost | ok | 95.1% | 813/855 | $2.94 | 58 | 0.24h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 12 | miserlou__loop.209927c |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | almost | ok | 95.1% | 675/710 | $5.20 | 90 | 0.28h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 13 | mgdm__htmlq.6e31bc8 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | almost | ok | 95.1% | 1383/1455 | $5.11 | 93 | 0.24h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 14 | ogham__dog.721440b |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 94.7% | 1231/1300 | $8.56 | 127 | 0.44h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 15 | raviqqe__muffet.a882908 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 94.5% | 277/293 | $5.35 | 102 | 0.26h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 16 | mgdm__htmlq.6e31bc8 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 94.4% | 1374/1455 | $3.37 | 61 | 0.20h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 17 | agourlay__zip-password-finder.704700d |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 94.1% | 640/680 | n/a | 48 | 0.14h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 18 | lymphatus__caesium-clt.a529b2e |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 94.1% | 541/575 | n/a | 61 | 0.21h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 19 | lymphatus__caesium-clt.a529b2e |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 93.9% | 540/575 | $4.44 | 77 | 0.24h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 20 | tarka__xcp.5e5b448 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 93.8% | 1111/1184 | $5.28 | 102 | 0.24h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 21 | mgdm__htmlq.6e31bc8 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 93.7% | 1364/1455 | $4.47 | 78 | 0.25h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 22 | tarka__xcp.5e5b448 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 93.7% | 1109/1184 | $3.59 | 65 | 0.21h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 23 | sstadick__hck.b66c751 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 93.6% | 800/855 | n/a | 52 | 0.15h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 24 | clog-tool__clog-cli.7066cba |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 93.6% | 538/575 | n/a | 71 | 0.21h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 25 | jarun__nnn.cb2c535 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 93.5% | 446/477 | n/a | 58 | 0.14h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 26 | riquito__tuc.16fb471 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 93.3% | 1116/1196 | n/a | 44 | 0.19h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 27 | wfxr__code-minimap.0ddeea5 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 93.3% | 292/313 | $5.75 | 92 | 0.34h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 28 | madler__pigz.fe4894f |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 93.1% | 774/831 | $4.74 | 91 | 0.22h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 29 | sharkdp__hexyl.2e26437 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 92.9% | 842/906 | $6.73 | 100 | 0.27h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 30 | rust-ethereum__ethabi.b1710ad |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 92.8% | 925/997 | $3.14 | 55 | 0.21h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 31 | riquito__tuc.16fb471 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 92.6% | 1108/1196 | $5.68 | 86 | 0.26h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 32 | agourlay__zip-password-finder.704700d |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 92.5% | 629/680 | $1.86 | 37 | 0.22h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 33 | nikolassv__bartib.6b9b5ce |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 92.4% | 667/722 | $4.29 | 70 | 0.22h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 34 | rhysd__kiro-editor.4157485 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 92.3% | 549/595 | $4.24 | 75 | 0.23h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 35 | clog-tool__clog-cli.7066cba |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 92.2% | 530/575 | $5.01 | 83 | 0.27h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 36 | altdesktop__i3-style.f93821b |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 92.0% | 496/539 | $3.31 | 47 | 0.21h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 37 | sheepla__pingu.926d475 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 91.9% | 352/383 | n/a | 59 | 0.20h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 38 | guumaster__hostctl.d6d9699 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 91.8% | 965/1051 | $7.51 | 121 | 0.29h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 39 | shashwatah__jot.a92aad8 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 91.8% | 690/752 | $4.60 | 90 | 0.22h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 40 | simeg__eureka.df3796c |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 91.6% | 315/344 | $3.73 | 63 | 0.18h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 41 | astro__deadnix.d590041 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 91.5% | 551/602 | $3.63 | 49 | 0.21h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 42 | chmln__sd.87d1ba5 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 91.5% | 741/810 | $3.26 | 62 | 0.20h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 43 | sheepla__pingu.926d475 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 91.4% | 350/383 | n/a | 43 | 0.17h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 44 | ismaelgv__rnr.fc0733b |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 90.8% | 620/683 | n/a | 85 | 0.19h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 45 | sirwart__ripsecrets.34c9e03 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 90.5% | 553/611 | $4.55 | 98 | 0.24h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 46 | svenstaro__miniserve.8449e8b |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 90.5% | 275/304 | $4.29 | 56 | 0.24h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 47 | sharkdp__pastel.b60e899 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 89.9% | 1002/1114 | $8.41 | 122 | 0.37h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 48 | lymphatus__caesium-clt.a529b2e |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 89.9% | 517/575 | n/a | 72 | 0.23h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 49 | ismaelgv__rnr.fc0733b |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 89.9% | 614/683 | $5.91 | 112 | 0.27h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 50 | miserlou__loop.209927c |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 89.9% | 638/710 | n/a | 51 | 0.14h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 51 | anordal__shellharden.6a6ffd4 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 89.6% | 981/1095 | $4.99 | 80 | 0.26h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 52 | oppiliappan__eva.41ae245 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 88.9% | 812/913 | n/a | 72 | 0.23h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 53 | tomnomnom__gron.88a6234 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 88.8% | 199/224 | $4.05 | 84 | 0.21h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 54 | burntsushi__xsv.f430466 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 88.8% | 1050/1182 | $4.00 | 85 | 0.25h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 55 | svenstaro__miniserve.8449e8b |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 88.8% | 270/304 | n/a | 71 | 0.20h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 56 | sharkdp__pastel.b60e899 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 88.6% | 987/1114 | n/a | 121 | 0.38h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 57 | rs__jplot.2a54bcc |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 88.5% | 516/583 | $3.71 | 71 | 0.21h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 58 | yaa110__nomino.f892499 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 88.5% | 277/313 | $3.41 | 65 | 0.19h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 59 | pemistahl__grex.fa3e8ed |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 88.5% | 1161/1312 | $7.01 | 115 | 0.35h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 60 | kyoheiu__felix.95df390 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 88.4% | 444/502 | $3.30 | 67 | 0.18h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 61 | sirwart__ripsecrets.34c9e03 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 88.2% | 539/611 | $3.67 | 74 | 0.23h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 62 | arthursonzogni__json-tui.17a22b6 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 88.2% | 666/755 | $4.10 | 77 | 0.17h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 63 | yaa110__nomino.f892499 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 88.2% | 276/313 | n/a | 59 | 0.16h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 64 | astro__deadnix.d590041 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 88.0% | 530/602 | $2.93 | 51 | 0.18h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 65 | sharkdp__hexyl.2e26437 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 87.9% | 796/906 | n/a | 66 | 0.22h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 66 | ekzhang__bore.8e059cd |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 87.8% | 346/394 | $3.72 | 77 | 0.22h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 67 | byron__dua-cli.8570c15 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 87.6% | 621/709 | $3.66 | 59 | 0.23h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 68 | byron__dua-cli.8570c15 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 87.4% | 620/709 | n/a | 59 | 0.20h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 69 | mibk__dupl.1bf052b |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 87.4% | 326/373 | $4.82 | 85 | 0.23h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 70 | johanneskaufmann__html-to-markdown.3006818 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 87.3% | 773/885 | $3.76 | 62 | 0.22h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 71 | rust-ethereum__ethabi.b1710ad |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 87.3% | 870/997 | n/a | 66 | 0.20h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 72 | chmln__sd.87d1ba5 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 87.2% | 706/810 | n/a | 47 | 0.17h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 73 | mibk__dupl.1bf052b |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 87.1% | 325/373 | n/a | 69 | 0.21h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 74 | rs__curlie.5dfcbb1 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 87.0% | 610/701 | $4.58 | 83 | 0.21h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 75 | sharkdp__pastel.b60e899 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 86.9% | 968/1114 | n/a | 66 | 0.24h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 76 | brocode__fblog.3b54330 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 86.7% | 848/978 | $3.26 | 49 | 0.19h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 77 | mookid__diffr.2152742 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 86.6% | 525/606 | $5.83 | 84 | 0.31h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 78 | kyoh86__richgo.313114f |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 86.6% | 473/546 | $3.50 | 83 | 0.17h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 79 | riquito__tuc.16fb471 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 86.5% | 1035/1196 | n/a | 40 | 0.16h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 80 | ismaelgv__rnr.fc0733b |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 86.4% | 590/683 | n/a | 62 | 0.19h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 81 | pemistahl__grex.fa3e8ed |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 86.4% | 1133/1312 | n/a | 58 | 0.25h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 82 | oppiliappan__eva.41ae245 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 86.3% | 788/913 | $5.51 | 112 | 0.28h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 83 | guumaster__hostctl.d6d9699 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 86.2% | 906/1051 | n/a | 78 | 0.24h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 84 | nikolassv__bartib.6b9b5ce |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 86.1% | 622/722 | n/a | 84 | 0.20h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 85 | sirwart__ripsecrets.34c9e03 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 85.9% | 525/611 | $2.80 | 55 | 0.17h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 86 | kyoheiu__felix.95df390 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 85.7% | 430/502 | n/a | 54 | 0.16h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 87 | halitechallenge__halite.822cfb6 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 85.5% | 235/275 | $3.17 | 69 | 0.19h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 88 | zevv__duc.a58fa4e |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 85.4% | 746/874 | $4.01 | 59 | 0.25h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 89 | hooklift__gowsdl.2a06cec |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 85.2% | 333/391 | n/a | 79 | 0.26h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 90 | eradman__entr.8e2e8b4 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 85.2% | 499/586 | $5.54 | 93 | 0.31h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 91 | zevv__duc.a58fa4e |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 85.1% | 744/874 | n/a | 50 | 0.17h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 92 | jonas__tig.8334123 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 85.1% | 1350/1586 | n/a | 60 | 0.16h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 93 | kyoheiu__felix.95df390 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 85.1% | 427/502 | n/a | 37 | 0.12h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 94 | rs__curlie.5dfcbb1 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 85.0% | 596/701 | $3.33 | 52 | 0.18h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 95 | brocode__fblog.3b54330 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 84.6% | 827/978 | n/a | 44 | 0.16h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 96 | chmln__handlr.90e78ba |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 84.5% | 610/722 | $6.89 | 125 | 0.39h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 97 | abishekvashok__cmatrix.5c082c6 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 84.4% | 428/507 | $3.03 | 73 | 0.17h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 98 | mfridman__tparse.2416b4b |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 84.2% | 358/425 | n/a | 94 | 0.28h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 99 | jonas__tig.8334123 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 84.1% | 1334/1586 | $3.08 | 62 | 0.17h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 100 | cmatsuoka__figlet.202a0a8 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 84.1% | 733/872 | $7.34 | 89 | 0.31h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 101 | clog-tool__clog-cli.7066cba |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 84.0% | 483/575 | n/a | 68 | 0.19h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 102 | hatoo__oha.8dc6349 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 83.9% | 754/899 | $6.21 | 107 | 0.28h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 103 | psampaz__go-mod-outdated.bb79367 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 83.9% | 239/285 | $2.13 | 45 | 0.14h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 104 | jonas__tig.8334123 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 83.9% | 1330/1586 | n/a | 45 | 0.12h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 105 | rs__curlie.5dfcbb1 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 83.7% | 587/701 | $3.24 | 55 | 0.21h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 106 | oppiliappan__eva.41ae245 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 83.7% | 764/913 | n/a | 80 | 0.21h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 107 | mfridman__tparse.2416b4b |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 83.3% | 354/425 | $5.64 | 107 | 0.22h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 108 | sibprogrammer__xq.b89f681 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 83.2% | 659/792 | $5.63 | 99 | 0.26h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 109 | eudoxia0__hashcards.48aa136 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 82.9% | 845/1019 | $7.12 | 128 | 0.30h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 110 | hooklift__gowsdl.2a06cec |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 82.6% | 323/391 | n/a | 64 | 0.20h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 111 | alexpovel__srgn.89f943b |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 82.3% | 1525/1852 | $5.89 | 84 | 0.33h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 112 | esubaalew__run.0fb9dec |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 82.2% | 996/1212 | $3.77 | 75 | 0.21h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 113 | junegunn__fzf.b56d614 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 82.1% | 1526/1858 | $6.12 | 90 | 0.27h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 114 | lz4__lz4.1519f46 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 82.0% | 1227/1496 | $1.95 | 34 | 0.16h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 115 | byron__dua-cli.8570c15 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 81.9% | 581/709 | n/a | 50 | 0.15h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 116 | burntsushi__ripgrep.3b7fd44 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 81.9% | 1634/1994 | $8.00 | 108 | 0.29h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 117 | burntsushi__xsv.f430466 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 81.9% | 968/1182 | $2.55 | 48 | 0.17h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 118 | sharkdp__fd.40d8eb3 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 81.6% | 1008/1235 | $8.74 | 157 | 0.41h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 119 | zevv__duc.a58fa4e |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 81.5% | 712/874 | n/a | 53 | 0.17h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 120 | mibk__dupl.1bf052b |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 81.2% | 303/373 | n/a | 60 | 0.21h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 121 | incu6us__goimports-reviser.81bd549 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 81.1% | 416/513 | $4.46 | 83 | 0.24h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 122 | ksxgithub__parallel-disk-usage.96978ed |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 81.0% | 430/531 | $5.72 | 82 | 0.29h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 123 | bensadeh__tailspin.6278437 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 81.0% | 498/615 | $7.03 | 84 | 0.31h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 124 | yaa110__nomino.f892499 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 80.8% | 253/313 | n/a | 59 | 0.20h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 125 | junegunn__fzf.b56d614 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 80.4% | 1494/1858 | $4.31 | 50 | 0.22h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 126 | shashwatah__jot.a92aad8 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 80.3% | 604/752 | $2.81 | 50 | 0.18h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 127 | esubaalew__run.0fb9dec |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 80.3% | 973/1212 | $3.44 | 62 | 0.19h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 128 | pier-cli__pier.5e1bde9 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 80.1% | 554/692 | $2.93 | 49 | 0.16h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 129 | eradman__entr.8e2e8b4 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 80.0% | 469/586 | n/a | 36 | 0.15h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 130 | hooklift__gowsdl.2a06cec |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 79.8% | 312/391 | $5.09 | 64 | 0.28h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 131 | astaxie__bat.17d1080 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 79.7% | 870/1091 | $2.76 | 43 | 0.15h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 132 | blacknon__hwatch.edfcb62 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 79.7% | 810/1016 | $3.87 | 82 | 0.18h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 133 | burntsushi__ripgrep.3b7fd44 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 79.7% | 1589/1994 | n/a | 89 | 0.26h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 134 | eliukblau__pixterm.1a93fd5 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 79.3% | 341/430 | $4.63 | 72 | 0.25h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 135 | ksxgithub__parallel-disk-usage.96978ed |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 79.3% | 421/531 | n/a | 77 | 0.30h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 136 | nuta__nsh.bdd0702 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 79.3% | 1556/1963 | $5.45 | 82 | 0.31h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 137 | junegunn__fzf.b56d614 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 79.2% | 1472/1858 | $3.67 | 49 | 0.21h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 138 | esubaalew__run.0fb9dec |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 79.2% | 960/1212 | $3.83 | 72 | 0.20h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 139 | facebookresearch__fasttext.1142dc4 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 79.2% | 247/312 | $4.30 | 88 | 0.26h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 140 | sharkdp__fd.40d8eb3 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 79.1% | 977/1235 | n/a | 84 | 0.26h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 141 | rust-ethereum__ethabi.b1710ad |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 79.0% | 788/997 | n/a | 49 | 0.17h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 142 | noborus__ov.b96c2ba |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 79.0% | 1465/1854 | $5.41 | 87 | 0.23h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 143 | ksxgithub__parallel-disk-usage.96978ed |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 78.9% | 419/531 | n/a | 49 | 0.17h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 144 | o2sh__onefetch.e5958ce |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 78.9% | 920/1166 | $4.28 | 60 | 0.22h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 145 | segmentio__chamber.5f93f5f |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 78.9% | 1379/1748 | n/a | 61 | 0.20h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 146 | eradman__entr.8e2e8b4 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 78.7% | 461/586 | n/a | 28 | 0.10h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 147 | codesnap-rs__codesnap.f81e4f3 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 78.5% | 448/571 | $4.88 | 55 | 0.22h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 148 | skeema__skeema.6a76243 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 78.3% | 1338/1708 | $7.71 | 104 | 0.30h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 149 | segmentio__chamber.5f93f5f |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 78.3% | 1369/1748 | $5.75 | 90 | 0.27h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 150 | eudoxia0__hashcards.48aa136 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 78.2% | 797/1019 | n/a | 107 | 0.27h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 151 | chmln__handlr.90e78ba |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 77.8% | 562/722 | n/a | 61 | 0.17h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 152 | gabotechs__dep-tree.60a95a2 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 77.8% | 673/865 | n/a | 62 | 0.22h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 153 | blacknon__hwatch.edfcb62 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 77.8% | 790/1016 | n/a | 62 | 0.16h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 154 | segmentio__chamber.5f93f5f |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 77.3% | 1351/1748 | n/a | 82 | 0.27h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 155 | ekzhang__bore.8e059cd |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 77.2% | 304/394 | $3.69 | 81 | 0.19h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 156 | codesnap-rs__codesnap.f81e4f3 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 76.9% | 439/571 | $3.32 | 36 | 0.16h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 157 | alexpovel__srgn.89f943b |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 76.6% | 1419/1852 | n/a | 71 | 0.20h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 158 | codesnap-rs__codesnap.f81e4f3 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 76.4% | 436/571 | $5.10 | 59 | 0.18h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 159 | tomnomnom__gron.88a6234 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 76.3% | 171/224 | n/a | 80 | 0.20h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 160 | blacknon__hwatch.edfcb62 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 76.3% | 775/1016 | n/a | 79 | 0.15h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 161 | miserlou__loop.209927c |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 76.1% | 540/710 | n/a | 61 | 0.19h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 162 | lua__lua.c6b4848 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 75.9% | 1016/1338 | $6.49 | 108 | 0.28h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 163 | sayanarijit__xplr.1751065 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 75.9% | 318/419 | $4.75 | 82 | 0.20h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 164 | ajeetdsouza__zoxide.67ca1bc |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 75.9% | 403/531 | $3.69 | 61 | 0.22h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 165 | rbakbashev__elfcat.52f8cc7 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 75.9% | 428/564 | $4.24 | 54 | 0.19h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 166 | rhysd__kiro-editor.4157485 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 75.6% | 450/595 | n/a | 35 | 0.13h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 167 | rhysd__kiro-editor.4157485 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 75.6% | 450/595 | n/a | 96 | 0.21h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 168 | nachoparker__dutree.44e877d |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 75.5% | 484/641 | $3.20 | 56 | 0.19h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 169 | bootandy__dust.62bf1e1 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 75.3% | 440/584 | $6.85 | 107 | 0.35h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 170 | konradsz__igrep.aa75630 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 75.3% | 290/385 | $2.51 | 55 | 0.15h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 171 | isona__dirble.e2dea9f |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 75.2% | 540/718 | $3.74 | 77 | 0.22h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 172 | astaxie__bat.17d1080 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 75.2% | 820/1091 | n/a | 36 | 0.16h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 173 | antonmedv__walk.bf802ef |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 75.1% | 353/470 | n/a | 52 | 0.15h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 174 | raviqqe__muffet.a882908 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 75.1% | 220/293 | n/a | 46 | 0.14h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 175 | nikolassv__bartib.6b9b5ce |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 75.1% | 542/722 | n/a | 49 | 0.17h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 176 | ggreer__the_silver_searcher.a61f178 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 75.0% | 753/1004 | n/a | 81 | 0.26h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 177 | git-bahn__git-graph.87b4473 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 75.0% | 426/568 | n/a | 64 | 0.24h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 178 | rust-embedded__svd2rust.1760b5e |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 74.9% | 689/920 | n/a | 53 | 0.22h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 179 | burntsushi__ripgrep.3b7fd44 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 74.8% | 1491/1994 | n/a | 117 | 0.29h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 180 | o2sh__onefetch.e5958ce |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 74.7% | 871/1166 | n/a | 48 | 0.20h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 181 | canop__rhit.ae90bcb |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 74.7% | 610/817 | $5.40 | 81 | 0.22h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 182 | burntsushi__xsv.f430466 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 74.5% | 881/1182 | $3.35 | 57 | 0.19h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 183 | nachoparker__dutree.44e877d |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 74.3% | 476/641 | n/a | 73 | 0.23h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 184 | rust-embedded__svd2rust.1760b5e |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 74.2% | 683/920 | $4.59 | 58 | 0.19h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 185 | multiprocessio__dsq.c3ae0ba |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 74.2% | 402/542 | $4.72 | 111 | 0.26h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 186 | yoav-lavi__melody.f4af9b4 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 74.1% | 893/1205 | $4.67 | 79 | 0.24h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 187 | konradsz__igrep.aa75630 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 74.0% | 285/385 | n/a | 47 | 0.12h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 188 | jqlang__jq.b33a763 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 74.0% | 4493/6072 | $7.61 | 132 | 0.36h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 189 | mookid__diffr.2152742 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 73.9% | 448/606 | n/a | 44 | 0.16h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 190 | mfridman__tparse.2416b4b |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 73.9% | 314/425 | n/a | 47 | 0.20h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 191 | astaxie__bat.17d1080 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 73.9% | 806/1091 | n/a | 44 | 0.12h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 192 | lfos__calcurse.49180d5 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 73.4% | 489/666 | $5.43 | 62 | 0.20h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 193 | lfos__calcurse.49180d5 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 73.4% | 489/666 | n/a | 77 | 0.23h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 194 | lua__lua.c6b4848 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 73.4% | 982/1338 | n/a | 46 | 0.20h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 195 | chmln__handlr.90e78ba |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 73.3% | 529/722 | n/a | 73 | 0.21h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 196 | nachoparker__dutree.44e877d |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 73.2% | 469/641 | n/a | 52 | 0.19h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 197 | drew-alleman__datasurgeon.d257cee |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 72.9% | 366/502 | $4.61 | 79 | 0.21h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 198 | dundee__gdu.ede21d2 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 72.9% | 846/1161 | $4.50 | 83 | 0.25h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 199 | sitkevij__hex.61ae69b |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 72.8% | 599/823 | n/a | 66 | 0.19h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 200 | konradsz__igrep.aa75630 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 72.7% | 280/385 | n/a | 32 | 0.12h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 201 | ninja-build__ninja.cc60300 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 72.7% | 1045/1438 | $7.28 | 129 | 0.35h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 202 | dundee__gdu.ede21d2 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 72.5% | 842/1161 | n/a | 57 | 0.23h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 203 | lz4__lz4.1519f46 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 72.3% | 1081/1496 | n/a | 33 | 0.15h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 204 | dalance__amber.69a0f52 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 72.2% | 408/565 | $1.96 | 38 | 0.12h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 205 | dalance__amber.69a0f52 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 72.0% | 407/565 | $2.36 | 42 | 0.15h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 206 | wintermute-cell__ngrrram.8ea13c3 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 71.9% | 218/303 | $1.98 | 37 | 0.15h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 207 | wintermute-cell__ngrrram.8ea13c3 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 71.9% | 218/303 | $1.40 | 31 | 0.11h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 208 | unhappychoice__gittype.34b72d0 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 71.9% | 533/741 | $3.08 | 61 | 0.17h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 209 | wfxr__code-minimap.0ddeea5 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 71.6% | 224/313 | n/a | 89 | 0.31h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 210 | lh3__seqtk.94e7070 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 71.6% | 307/429 | n/a | 82 | 0.25h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 211 | rbakbashev__elfcat.52f8cc7 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 71.5% | 403/564 | n/a | 57 | 0.20h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 212 | naggie__dstask.ff57396 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 71.4% | 913/1278 | $3.61 | 53 | 0.20h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 213 | ajeetdsouza__zoxide.67ca1bc |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 71.4% | 379/531 | n/a | 87 | 0.21h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 214 | wfxr__csview.8ac4de0 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 71.3% | 239/335 | n/a | 74 | 0.23h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 215 | foriequal0__git-trim.07c2f50 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 71.3% | 363/509 | $4.09 | 71 | 0.22h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 216 | jrnxf__thokr.09375ef |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 71.2% | 317/445 | n/a | 47 | 0.15h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 217 | luajit__luajit.a553b3d |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 71.2% | 2113/2967 | $7.69 | 135 | 0.34h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 218 | orf__gping.26eb5b9 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 71.1% | 241/339 | n/a | 91 | 0.23h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 219 | wfxr__csview.8ac4de0 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 71.0% | 238/335 | n/a | 82 | 0.43h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 220 | multiprocessio__dsq.c3ae0ba |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 71.0% | 385/542 | n/a | 66 | 0.23h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 221 | incu6us__goimports-reviser.81bd549 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 71.0% | 364/513 | n/a | 32 | 0.12h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 222 | noborus__trdsql.d8c5ff6 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 70.9% | 930/1312 | $2.96 | 55 | 0.16h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 223 | nuta__nsh.bdd0702 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 70.8% | 1389/1963 | n/a | 107 | 0.25h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 224 | noborus__trdsql.d8c5ff6 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 70.7% | 928/1312 | n/a | 54 | 0.18h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 225 | eudoxia0__hashcards.48aa136 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 70.7% | 720/1019 | n/a | 106 | 0.28h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 226 | xampprocky__tokei.505d648 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 70.4% | 515/732 | n/a | 41 | 0.17h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 227 | psampaz__go-mod-outdated.bb79367 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 70.2% | 200/285 | n/a | 26 | 0.11h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 228 | jrnxf__thokr.09375ef |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 70.1% | 312/445 | $2.56 | 54 | 0.15h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 229 | lh3__seqtk.94e7070 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 69.9% | 300/429 | $3.88 | 71 | 0.22h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 230 | orf__gping.26eb5b9 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 69.9% | 237/339 | $1.84 | 35 | 0.12h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 231 | direnv__direnv.02040c7 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 69.8% | 593/849 | n/a | 59 | 0.15h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 232 | jrnxf__thokr.09375ef |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 69.7% | 310/445 | n/a | 46 | 0.13h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 233 | facebook__zstd.1168da0 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 69.6% | 1419/2038 | $4.54 | 77 | 0.23h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 234 | hairyhenderson__gomplate.05eb3aa |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 69.5% | 2033/2926 | $9.60 | 131 | 0.45h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 235 | lz4__lz4.1519f46 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 69.4% | 1038/1496 | n/a | 70 | 0.21h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 236 | cheat__cheat.b8098dc |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 69.4% | 206/297 | $6.81 | 116 | 0.30h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 237 | facebookresearch__fasttext.1142dc4 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 69.2% | 216/312 | n/a | 55 | 0.17h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 238 | incu6us__goimports-reviser.81bd549 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 69.2% | 355/513 | n/a | 72 | 0.20h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 239 | ninja-build__ninja.cc60300 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 69.2% | 995/1438 | n/a | 62 | 0.20h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 240 | nuta__nsh.bdd0702 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 69.1% | 1357/1963 | n/a | 67 | 0.24h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 241 | dundee__gdu.ede21d2 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 69.1% | 802/1161 | n/a | 54 | 0.17h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 242 | peco__peco.4e58dad |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 69.0% | 845/1224 | n/a | 46 | 0.16h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 243 | pls-rs__pls.4e1ae50 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 69.0% | 229/332 | $3.61 | 75 | 0.19h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 244 | hatoo__oha.8dc6349 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 69.0% | 620/899 | n/a | 51 | 0.14h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 245 | raviqqe__muffet.a882908 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 68.9% | 202/293 | n/a | 35 | 0.14h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 246 | rs__jplot.2a54bcc |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 68.8% | 401/583 | n/a | 41 | 0.12h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 247 | ajeetdsouza__zoxide.67ca1bc |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 68.7% | 365/531 | n/a | 74 | 0.21h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 248 | guumaster__hostctl.d6d9699 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 68.7% | 722/1051 | n/a | 79 | 0.24h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 249 | bootandy__dust.62bf1e1 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 68.5% | 400/584 | n/a | 62 | 0.21h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 250 | rs__jplot.2a54bcc |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 68.4% | 399/583 | n/a | 32 | 0.10h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 251 | orf__gping.26eb5b9 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 68.4% | 232/339 | n/a | 56 | 0.14h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 252 | cordx56__rustowl.655bc5c |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 68.4% | 403/589 | n/a | 41 | 0.13h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 253 | ninja-build__ninja.cc60300 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 68.4% | 983/1438 | n/a | 87 | 0.24h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 254 | jhspetersson__fselect.c3559ca |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 68.3% | 2129/3115 | n/a | 70 | 0.27h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 255 | halitechallenge__halite.822cfb6 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 68.0% | 187/275 | $4.31 | 75 | 0.22h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 256 | gabotechs__dep-tree.60a95a2 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 68.0% | 588/865 | $3.80 | 67 | 0.22h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 257 | pls-rs__pls.4e1ae50 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 67.8% | 225/332 | $4.68 | 87 | 0.23h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 258 | oppiliappan__statix.e9df54c |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 67.6% | 551/815 | $5.82 | 101 | 0.30h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 259 | canop__rhit.ae90bcb |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 67.6% | 552/817 | n/a | 83 | 0.26h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 260 | jhspetersson__fselect.c3559ca |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 67.5% | 2103/3115 | $6.60 | 95 | 0.29h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 261 | tinycc__tinycc.9b8765d |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 67.4% | 1334/1978 | n/a | 50 | 0.17h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 262 | arthursonzogni__json-tui.17a22b6 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 67.4% | 509/755 | n/a | 91 | 0.22h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 263 | thezoraiz__ascii-image-converter.d05a757 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 67.3% | 313/465 | n/a | 81 | 0.31h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 264 | cordx56__rustowl.655bc5c |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 67.2% | 396/589 | n/a | 39 | 0.11h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 265 | kaushiksrini__parqeye.8072121 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 67.2% | 322/479 | $3.97 | 60 | 0.23h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 266 | rust-lang__mdbook.37273ba |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 67.1% | 748/1114 | n/a | 60 | 0.19h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 267 | naggie__dstask.ff57396 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 67.0% | 856/1278 | n/a | 73 | 0.21h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 268 | ammarabouzor__tui-journal.2b4540d |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 66.9% | 938/1402 | n/a | 53 | 0.15h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 269 | ariga__atlas.6d81150 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 66.8% | 881/1318 | $5.36 | 94 | 0.25h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 270 | unhappychoice__gittype.34b72d0 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 66.8% | 495/741 | $3.97 | 66 | 0.18h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 271 | wfxr__code-minimap.0ddeea5 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 66.5% | 208/313 | n/a | 76 | 0.33h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 272 | skeema__skeema.6a76243 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 66.3% | 1133/1708 | n/a | 54 | 0.22h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 273 | direnv__direnv.02040c7 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 66.1% | 561/849 | $5.68 | 71 | 0.31h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 274 | tinycc__tinycc.9b8765d |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 66.0% | 1305/1978 | n/a | 45 | 0.16h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 275 | xampprocky__tokei.505d648 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 65.8% | 482/732 | $3.29 | 53 | 0.24h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 276 | halitechallenge__halite.822cfb6 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 65.8% | 181/275 | $2.64 | 45 | 0.14h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 277 | ariga__atlas.6d81150 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 65.7% | 866/1318 | $6.61 | 116 | 0.35h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 278 | blake3-team__blake3.15e83a5 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 65.4% | 423/647 | $3.43 | 58 | 0.18h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 279 | cordx56__rustowl.655bc5c |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 65.4% | 385/589 | $3.89 | 54 | 0.20h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 280 | lfos__calcurse.49180d5 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 65.3% | 435/666 | n/a | 98 | 0.34h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 281 | direnv__direnv.02040c7 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 65.1% | 553/849 | n/a | 48 | 0.17h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 282 | rochacbruno__marmite.7d4bc2d |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 65.0% | 434/668 | $6.14 | 94 | 0.31h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 283 | skeema__skeema.6a76243 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 64.8% | 1107/1708 | n/a | 77 | 0.19h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 284 | madler__pigz.fe4894f |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 64.7% | 538/831 | n/a | 85 | 0.24h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 285 | madler__pigz.fe4894f |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 64.7% | 538/831 | n/a | 110 | 0.29h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 286 | ecumene__rust-sloth.051c559 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 64.7% | 246/380 | n/a | 55 | 0.22h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 287 | lua__lua.c6b4848 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 64.7% | 866/1338 | n/a | 70 | 0.26h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 288 | ogham__dog.721440b |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 64.6% | 840/1300 | n/a | 63 | 0.21h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 289 | blake3-team__blake3.15e83a5 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 64.6% | 418/647 | $2.26 | 37 | 0.14h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 290 | ecumene__rust-sloth.051c559 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 64.5% | 245/380 | $5.93 | 82 | 0.30h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 291 | multiprocessio__dsq.c3ae0ba |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 64.4% | 349/542 | n/a | 96 | 0.31h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 292 | rust-embedded__svd2rust.1760b5e |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 64.3% | 592/920 | n/a | 53 | 0.20h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 293 | cweill__gotests.2a672c5 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 64.2% | 387/603 | n/a | 71 | 0.30h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 294 | rust-lang__mdbook.37273ba |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 64.0% | 713/1114 | n/a | 79 | 0.23h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 295 | y2z__monolith.8702e66 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 63.8% | 455/713 | $4.25 | 77 | 0.21h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 296 | facebook__zstd.1168da0 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 63.7% | 1299/2038 | n/a | 50 | 0.17h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 297 | naggie__dstask.ff57396 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 63.5% | 812/1278 | n/a | 63 | 0.19h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 298 | quinn-rs__quinn.bb359cc |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 63.2% | 330/522 | n/a | 67 | 0.18h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 299 | cweill__gotests.2a672c5 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 62.9% | 379/603 | $4.71 | 89 | 0.22h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 300 | pls-rs__pls.4e1ae50 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 62.7% | 208/332 | $3.65 | 58 | 0.20h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 301 | ammarabouzor__tui-journal.2b4540d |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 62.5% | 876/1402 | n/a | 89 | 0.26h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 302 | rochacbruno__marmite.7d4bc2d |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 62.4% | 417/668 | n/a | 66 | 0.20h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 303 | rcoh__angle-grinder.9c2fc88 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 62.4% | 705/1130 | $7.08 | 120 | 0.38h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 304 | altdesktop__i3-style.f93821b |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 62.3% | 336/539 | $2.17 | 39 | 0.14h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 305 | lh3__seqtk.94e7070 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 62.2% | 267/429 | n/a | 44 | 0.16h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 306 | ecumene__rust-sloth.051c559 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 62.1% | 236/380 | n/a | 61 | 0.17h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 307 | jarun__nnn.cb2c535 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 62.1% | 296/477 | n/a | 77 | 0.20h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 308 | canop__broot.d6c798e |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 61.8% | 333/539 | $4.80 | 52 | 0.17h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 309 | altdesktop__i3-style.f93821b |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 61.6% | 332/539 | $3.63 | 55 | 0.22h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 310 | facebook__zstd.1168da0 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 61.3% | 1250/2038 | n/a | 45 | 0.17h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 311 | axodotdev__oranda.27d60c7 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 61.1% | 469/767 | n/a | 81 | 0.17h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 312 | git-bahn__git-graph.87b4473 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 60.9% | 346/568 | n/a | 93 | 0.30h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 313 | canop__broot.d6c798e |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 60.9% | 328/539 | n/a | 42 | 0.15h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 314 | sharkdp__hyperfine.327d5f4 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 60.8% | 177/291 | n/a | 109 | 0.30h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 315 | pemistahl__grex.fa3e8ed |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 60.7% | 796/1312 | n/a | 77 | 0.27h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 316 | robertdavidgraham__masscan.b99d433 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 60.0% | 1530/2549 | $4.86 | 80 | 0.23h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 317 | jesseduffield__lazygit.1d0db51 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 59.9% | 512/855 | n/a | 47 | 0.14h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 318 | crowdagger__crowbook.ea214d7 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 59.9% | 483/807 | $4.62 | 68 | 0.25h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 319 | robertdavidgraham__masscan.b99d433 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 59.8% | 1524/2549 | n/a | 53 | 0.18h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 320 | sharkdp__fd.40d8eb3 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 59.4% | 734/1235 | n/a | 44 | 0.22h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 321 | epistates__treemd.825c6dd |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 59.4% | 932/1569 | $4.96 | 92 | 0.23h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 322 | kisielk__errcheck.dacab89 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 58.9% | 201/341 | $4.90 | 69 | 0.28h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 323 | yassinebridi__serpl.c48a9d7 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 58.7% | 262/446 | $1.68 | 38 | 0.11h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 324 | sharkdp__bat.f822bd0 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 58.6% | 468/799 | $3.99 | 54 | 0.20h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 325 | thezoraiz__ascii-image-converter.d05a757 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 58.3% | 271/465 | $3.49 | 61 | 0.20h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 326 | ys-l__flamelens.0b4dc33 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 58.0% | 130/224 | $3.31 | 56 | 0.32h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 327 | xorg62__tty-clock.f2f847c |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 58.0% | 163/281 | n/a | 44 | 0.17h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 328 | hatoo__oha.8dc6349 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 57.8% | 520/899 | n/a | 53 | 0.19h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 329 | rochacbruno__marmite.7d4bc2d |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 57.8% | 386/668 | n/a | 57 | 0.22h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 330 | luajit__luajit.a553b3d |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 57.8% | 1714/2967 | $4.35 | 61 | 0.23h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 331 | thezoraiz__ascii-image-converter.d05a757 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 57.6% | 268/465 | n/a | 64 | 0.23h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 332 | trasta298__keifu.3331426 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 57.3% | 150/262 | $2.59 | 49 | 0.13h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 333 | luajit__luajit.a553b3d |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 57.2% | 1698/2967 | $3.72 | 47 | 0.21h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 334 | kaushiksrini__parqeye.8072121 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 57.0% | 273/479 | n/a | 46 | 0.18h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 335 | yassinebridi__serpl.c48a9d7 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 57.0% | 254/446 | $3.20 | 56 | 0.15h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 336 | wgunderwood__tex-fmt.3f1aef6 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 56.9% | 259/455 | n/a | 75 | 0.28h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 337 | xorg62__tty-clock.f2f847c |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 56.6% | 159/281 | n/a | 49 | 0.17h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 338 | xampprocky__tokei.505d648 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 56.6% | 414/732 | n/a | 55 | 0.19h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 339 | svenstaro__miniserve.8449e8b |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 56.2% | 171/304 | n/a | 60 | 0.21h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 340 | foriequal0__git-trim.07c2f50 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 56.0% | 285/509 | n/a | 64 | 0.25h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 341 | ogham__dog.721440b |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 55.8% | 726/1300 | n/a | 66 | 0.25h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 342 | rcoh__angle-grinder.9c2fc88 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 55.8% | 630/1130 | $5.22 | 90 | 0.26h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 343 | stranger6667__jsonschema.d52e881 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 55.7% | 1634/2933 | $8.19 | 149 | 0.37h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 344 | kaushiksrini__parqeye.8072121 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 55.5% | 266/479 | n/a | 61 | 0.18h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 345 | cweill__gotests.2a672c5 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 55.4% | 334/603 | n/a | 78 | 0.17h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 346 | antonmedv__walk.bf802ef |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 55.3% | 260/470 | $3.34 | 56 | 0.20h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 347 | peco__peco.4e58dad |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 55.3% | 677/1224 | $4.35 | 63 | 0.21h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 348 | quinn-rs__quinn.bb359cc |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 55.2% | 288/522 | $2.64 | 54 | 0.16h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 349 | cslarsen__jp2a.61d205f |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 55.2% | 348/631 | n/a | 89 | 0.29h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 350 | crowdagger__crowbook.ea214d7 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 55.1% | 445/807 | n/a | 37 | 0.15h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 351 | nikoladucak__caps-log.2cf2d1e |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 55.0% | 303/551 | $7.39 | 86 | 0.29h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 352 | canop__rhit.ae90bcb |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 55.0% | 449/817 | n/a | 43 | 0.15h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 353 | o2sh__onefetch.e5958ce |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 54.2% | 632/1166 | n/a | 57 | 0.23h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 354 | sayanarijit__xplr.1751065 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 54.2% | 227/419 | n/a | 56 | 0.16h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 355 | jesseduffield__lazygit.1d0db51 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 54.0% | 462/855 | $6.58 | 88 | 0.28h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 356 | robertdavidgraham__masscan.b99d433 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 53.6% | 1367/2549 | n/a | 56 | 0.17h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 357 | isona__dirble.e2dea9f |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 53.5% | 384/718 | n/a | 78 | 0.27h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 358 | stacked-git__stgit.430027d |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | warn 1 | 53.2% | 821/1542 | $3.88 | 58 | 0.20h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 359 | oppiliappan__statix.e9df54c |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 53.0% | 432/815 | $4.21 | 63 | 0.22h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 360 | ggreer__the_silver_searcher.a61f178 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 53.0% | 532/1004 | $5.18 | 85 | 0.22h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 361 | oppiliappan__statix.e9df54c |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 52.9% | 431/815 | $3.05 | 59 | 0.17h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 362 | parcel-bundler__lightningcss.aa2ed1e |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 52.9% | 1495/2828 | $4.59 | 80 | 0.27h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 363 | stranger6667__jsonschema.d52e881 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 52.7% | 1547/2933 | n/a | 95 | 0.26h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 364 | wgunderwood__tex-fmt.3f1aef6 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 52.5% | 239/455 | n/a | 67 | 0.22h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 365 | shashwatah__jot.a92aad8 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 52.3% | 393/752 | $3.27 | 70 | 0.21h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 366 | stacked-git__stgit.430027d |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | warn 1 | 52.2% | 802/1537 | $3.84 | 58 | 0.20h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 367 | ducaale__xh.4a6e44f |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 52.2% | 611/1171 | n/a | 54 | 0.21h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 368 | jesseduffield__lazygit.1d0db51 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 52.2% | 446/855 | n/a | 71 | 0.18h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 369 | svenstaro__genact.16f96e3 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 52.2% | 121/232 | n/a | 55 | 0.18h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 370 | ducaale__xh.4a6e44f |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 52.1% | 610/1171 | n/a | 79 | 0.27h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 371 | isona__dirble.e2dea9f |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 51.7% | 371/718 | n/a | 76 | 0.20h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 372 | elkowar__pipr.fae0b17 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 51.4% | 270/525 | n/a | 45 | 0.11h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 373 | crowdagger__crowbook.ea214d7 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 51.4% | 415/807 | n/a | 49 | 0.22h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 374 | stacked-git__stgit.430027d |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 51.3% | 763/1488 | $4.65 | 72 | 0.23h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 375 | hairyhenderson__gomplate.05eb3aa |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 51.2% | 1499/2926 | n/a | 75 | 0.26h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 376 | quinn-rs__quinn.bb359cc |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 51.1% | 267/522 | n/a | 97 | 0.23h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 377 | trasta298__keifu.3331426 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 50.8% | 133/262 | $3.02 | 59 | 0.15h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 378 | chirlu__sox.42b3557 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 50.3% | 605/1202 | $5.04 | 65 | 0.27h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 379 | filosottile__age.706dfc1 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 50.3% | 340/676 | $4.87 | 62 | 0.23h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 380 | ducaale__xh.4a6e44f |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 49.4% | 579/1171 | $8.63 | 139 | 0.32h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 381 | anordal__shellharden.6a6ffd4 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 49.4% | 541/1095 | n/a | 69 | 0.21h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 382 | chirlu__sox.42b3557 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 49.3% | 592/1202 | $3.82 | 44 | 0.19h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 383 | trasta298__keifu.3331426 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 49.2% | 129/262 | $1.91 | 37 | 0.14h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 384 | y2z__monolith.8702e66 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 49.1% | 350/713 | n/a | 77 | 0.27h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 385 | anordal__shellharden.6a6ffd4 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 49.0% | 537/1095 | n/a | 64 | 0.20h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 386 | kisielk__errcheck.dacab89 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 49.0% | 167/341 | n/a | 63 | 0.22h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 387 | y2z__monolith.8702e66 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 48.9% | 349/713 | n/a | 69 | 0.24h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 388 | svenstaro__genact.16f96e3 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 48.7% | 113/232 | $3.17 | 35 | 0.15h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 389 | epistates__treemd.825c6dd |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 48.7% | 764/1569 | n/a | 68 | 0.21h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 390 | mkj__dropbear.75f699b |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 48.5% | 329/678 | $3.02 | 54 | 0.16h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 391 | mkj__dropbear.75f699b |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 48.5% | 329/678 | $2.44 | 53 | 0.17h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 392 | elkowar__pipr.fae0b17 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 48.0% | 252/525 | n/a | 53 | 0.15h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 393 | arthursonzogni__json-tui.17a22b6 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 47.7% | 360/755 | n/a | 88 | 0.14h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 394 | cslarsen__jp2a.61d205f |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 47.1% | 297/631 | $5.17 | 82 | 0.28h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 395 | sharkdp__hexyl.2e26437 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 46.8% | 424/906 | n/a | 72 | 0.25h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 396 | mkj__dropbear.75f699b |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 46.8% | 317/678 | $2.77 | 52 | 0.15h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 397 | rvben__rumdl.2d75c4d |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 46.3% | 1538/3322 | $6.81 | 67 | 0.27h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 398 | yassinebridi__serpl.c48a9d7 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 46.2% | 206/446 | $2.85 | 61 | 0.15h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 399 | sitkevij__hex.61ae69b |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 46.2% | 380/823 | n/a | 62 | 0.17h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 400 | nikoladucak__caps-log.2cf2d1e |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 46.1% | 254/551 | $8.60 | 100 | 0.24h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 401 | sharkdp__bat.f822bd0 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 46.1% | 368/799 | $3.79 | 61 | 0.16h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 402 | paradigmxyz__solar.5190d0e |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 46.1% | 911/1978 | $4.28 | 82 | 0.21h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 403 | ip7z__7zip.839151e |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 45.8% | 478/1043 | n/a | 30 | 0.13h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 404 | rvben__rumdl.2d75c4d |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 45.5% | 1510/3322 | $5.98 | 76 | 0.24h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 405 | epistates__treemd.825c6dd |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 45.1% | 708/1569 | n/a | 54 | 0.18h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 406 | filosottile__age.706dfc1 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 45.1% | 305/676 | $9.83 | 113 | 0.40h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 407 | sharkdp__bat.f822bd0 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 45.1% | 360/799 | $4.62 | 71 | 0.24h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 408 | sibprogrammer__xq.b89f681 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 44.9% | 356/792 | n/a | 50 | 0.20h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 409 | sibprogrammer__xq.b89f681 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 44.6% | 353/792 | n/a | 42 | 0.16h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 410 | elkowar__pipr.fae0b17 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 44.4% | 233/525 | $3.55 | 78 | 0.20h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 411 | arq5x__bedtools2.dd57059 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 44.3% | 467/1053 | n/a | 63 | 0.22h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 412 | tukaani-project__xz.1007bf0 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 43.8% | 618/1410 | n/a | 72 | 0.21h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 413 | ggreer__the_silver_searcher.a61f178 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 43.8% | 440/1004 | n/a | 40 | 0.14h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 414 | go-critic__go-critic.9aea378 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 43.8% | 216/493 | $5.76 | 91 | 0.29h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 415 | facebookresearch__fasttext.1142dc4 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 43.6% | 136/312 | n/a | 81 | 0.22h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 416 | boyter__scc.515f91c |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 43.3% | 201/464 | n/a | 65 | 0.22h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 417 | cslarsen__jp2a.61d205f |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 43.3% | 273/631 | n/a | 83 | 0.29h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 418 | gabotechs__dep-tree.60a95a2 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 43.2% | 374/865 | n/a | 63 | 0.20h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 419 | arq5x__bedtools2.dd57059 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 43.2% | 455/1053 | n/a | 41 | 0.17h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 420 | sstadick__hck.b66c751 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 43.0% | 368/855 | n/a | 48 | 0.18h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 421 | noborus__ov.b96c2ba |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 42.4% | 786/1854 | n/a | 50 | 0.17h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 422 | noborus__ov.b96c2ba |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 42.3% | 785/1854 | n/a | 44 | 0.16h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 423 | ip7z__7zip.839151e |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 42.3% | 441/1043 | $4.94 | 75 | 0.20h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 424 | tomarrell__wrapcheck.c058da1 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 42.1% | 202/480 | $4.20 | 85 | 0.22h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 425 | chirlu__sox.42b3557 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 42.0% | 505/1202 | $3.22 | 38 | 0.19h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 426 | hpjansson__chafa.dd4d4c1 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 42.0% | 811/1931 | n/a | 49 | 0.17h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 427 | go-critic__go-critic.9aea378 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 42.0% | 207/493 | $4.72 | 64 | 0.28h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 428 | tree-sitter__tree-sitter.5e23cca |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 41.9% | 516/1232 | $5.51 | 106 | 0.29h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 429 | typst__typst.88356d0 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 41.5% | 716/1724 | $3.90 | 54 | 0.20h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 430 | tukaani-project__xz.1007bf0 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 41.4% | 584/1410 | n/a | 57 | 0.20h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 431 | rbakbashev__elfcat.52f8cc7 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 41.3% | 233/564 | n/a | 46 | 0.16h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 432 | cmatsuoka__figlet.202a0a8 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 41.1% | 358/872 | n/a | 118 | 0.35h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 433 | wgunderwood__tex-fmt.3f1aef6 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 40.9% | 186/455 | $3.64 | 82 | 0.21h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 434 | tukaani-project__xz.1007bf0 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 40.9% | 576/1410 | $5.48 | 118 | 0.25h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 435 | cheat__cheat.b8098dc |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 40.7% | 121/297 | n/a | 44 | 0.14h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 436 | typst__typst.88356d0 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 40.7% | 701/1724 | n/a | 45 | 0.17h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 437 | nikoladucak__caps-log.2cf2d1e |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 40.5% | 223/551 | $4.51 | 68 | 0.15h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 438 | axodotdev__oranda.27d60c7 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 40.4% | 310/767 | $3.83 | 63 | 0.24h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 439 | arq5x__bedtools2.dd57059 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 40.3% | 424/1053 | $4.91 | 55 | 0.24h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 440 | ivanceras__svgbob.6d00ad9 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 40.3% | 190/472 | $3.47 | 72 | 0.19h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 441 | cheat__cheat.b8098dc |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 39.4% | 117/297 | n/a | 68 | 0.19h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 442 | sayanarijit__xplr.1751065 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 39.4% | 165/419 | n/a | 69 | 0.20h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 443 | typst__typst.88356d0 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 39.3% | 677/1724 | n/a | 51 | 0.23h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 444 | canop__broot.d6c798e |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 39.1% | 211/539 | n/a | 26 | 0.14h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 445 | cmatsuoka__figlet.202a0a8 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 39.1% | 341/872 | n/a | 111 | 0.34h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 446 | go-critic__go-critic.9aea378 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 38.9% | 192/493 | $3.83 | 67 | 0.19h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 447 | jqlang__jq.b33a763 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 38.0% | 2309/6072 | n/a | 129 | 0.36h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 448 | ys-l__flamelens.0b4dc33 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 37.9% | 85/224 | n/a | 68 | 0.18h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 449 | boyter__scc.515f91c |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 37.9% | 176/464 | $5.82 | 87 | 0.22h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 450 | foriequal0__git-trim.07c2f50 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 37.7% | 192/509 | n/a | 79 | 0.23h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 451 | stathissideris__ditaa.f2286c4 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 37.6% | 229/609 | $5.23 | 91 | 0.24h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 452 | mgechev__revive.201451e |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 37.1% | 270/727 | n/a | 88 | 0.21h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 453 | axodotdev__oranda.27d60c7 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 37.0% | 284/767 | n/a | 52 | 0.17h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 454 | brocode__fblog.3b54330 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 36.9% | 361/978 | n/a | 39 | 0.18h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 455 | paradigmxyz__solar.5190d0e |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 36.9% | 729/1978 | n/a | 84 | 0.23h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 456 | bootandy__dust.62bf1e1 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 36.5% | 213/584 | n/a | 92 | 0.31h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 457 | drew-alleman__datasurgeon.d257cee |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 36.3% | 182/502 | $4.01 | 71 | 0.20h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 458 | antonmedv__walk.bf802ef |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 36.0% | 169/470 | n/a | 40 | 0.11h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 459 | dandavison__delta.acd758f |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 35.9% | 341/950 | $4.56 | 46 | 0.20h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 460 | tomarrell__wrapcheck.c058da1 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 35.8% | 172/480 | n/a | 91 | 0.24h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 461 | svenstaro__genact.16f96e3 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 35.8% | 83/232 | n/a | 41 | 0.21h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 462 | bensadeh__tailspin.6278437 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 35.8% | 220/615 | n/a | 63 | 0.19h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 463 | boyter__scc.515f91c |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 35.3% | 164/464 | n/a | 96 | 0.32h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 464 | simeg__eureka.df3796c |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 35.2% | 121/344 | n/a | 94 | 0.17h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 465 | stranger6667__jsonschema.d52e881 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 35.0% | 1027/2933 | n/a | 104 | 0.24h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 466 | tree-sitter__tree-sitter.5e23cca |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 34.9% | 430/1232 | n/a | 84 | 0.25h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 467 | simeg__eureka.df3796c |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 34.9% | 120/344 | n/a | 54 | 0.13h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 468 | ivanceras__svgbob.6d00ad9 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 34.7% | 164/472 | $4.26 | 62 | 0.21h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 469 | kisielk__errcheck.dacab89 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 34.6% | 118/341 | n/a | 38 | 0.21h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 470 | mikefarah__yq.602586d |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 34.5% | 690/2000 | $4.29 | 82 | 0.22h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 471 | doxygen__doxygen.966d98e |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 34.1% | 78/229 | $3.11 | 50 | 0.15h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 472 | zk-org__zk.10d93d5 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 34.0% | 377/1108 | n/a | 56 | 0.21h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 473 | paradigmxyz__solar.5190d0e |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 33.9% | 670/1978 | n/a | 65 | 0.25h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 474 | mgechev__revive.201451e |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 33.8% | 246/727 | $5.35 | 100 | 0.26h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 475 | tomarrell__wrapcheck.c058da1 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 33.8% | 162/480 | n/a | 49 | 0.18h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 476 | bensadeh__tailspin.6278437 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 33.2% | 204/615 | n/a | 48 | 0.16h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 477 | tree-sitter__tree-sitter.5e23cca |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 32.7% | 403/1232 | n/a | 45 | 0.18h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 478 | peco__peco.4e58dad |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 32.7% | 400/1224 | n/a | 83 | 0.31h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 479 | samtools__samtools.aa823b5 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 32.1% | 458/1425 | $4.10 | 42 | 0.23h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 480 | eliukblau__pixterm.1a93fd5 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 32.1% | 138/430 | n/a | 62 | 0.23h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 481 | eliukblau__pixterm.1a93fd5 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 32.1% | 138/430 | n/a | 67 | 0.29h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 482 | johnkerl__miller.8d85b46 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 32.1% | 4694/14637 | $7.10 | 97 | 0.36h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 483 | doxygen__doxygen.966d98e |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 31.9% | 73/229 | n/a | 108 | 0.27h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 484 | mgechev__revive.201451e |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 31.4% | 228/727 | n/a | 67 | 0.26h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 485 | drew-alleman__datasurgeon.d257cee |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 30.9% | 155/502 | $2.89 | 56 | 0.17h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 486 | stathissideris__ditaa.f2286c4 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 30.9% | 188/609 | n/a | 97 | 0.28h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 487 | ekzhang__bore.8e059cd |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 30.5% | 120/394 | $3.44 | 75 | 0.17h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 488 | stathissideris__ditaa.f2286c4 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 29.4% | 179/609 | n/a | 57 | 0.19h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 489 | johnkerl__miller.8d85b46 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 28.3% | 4147/14637 | n/a | 41 | 0.29h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 490 | chmln__sd.87d1ba5 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 28.3% | 229/810 | n/a | 56 | 0.22h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 491 | mookid__diffr.2152742 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 28.1% | 170/606 | n/a | 52 | 0.13h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 492 | johnkerl__miller.8d85b46 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 27.6% | 4044/14637 | n/a | 63 | 0.27h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 493 | zk-org__zk.10d93d5 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 26.8% | 297/1108 | n/a | 44 | 0.14h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 494 | osgeo__gdal.0847f12 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 26.5% | 174/657 | n/a | 51 | 0.22h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 495 | mikefarah__yq.602586d |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 25.6% | 511/2000 | n/a | 44 | 0.17h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 496 | ys-l__flamelens.0b4dc33 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 25.4% | 57/224 | n/a | 46 | 0.12h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 497 | ivanceras__svgbob.6d00ad9 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 25.4% | 120/472 | $4.53 | 71 | 0.20h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 498 | google__brotli.b3dc9cc |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 24.9% | 110/441 | $4.01 | 64 | 0.20h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 499 | ast-grep__ast-grep.dde0fe0 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 24.6% | 217/882 | $4.71 | 77 | 0.23h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 500 | yoav-lavi__melody.f4af9b4 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 23.8% | 287/1205 | n/a | 124 | 0.37h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 501 | hpjansson__chafa.dd4d4c1 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 23.5% | 454/1931 | n/a | 118 | 0.37h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 502 | johanneskaufmann__html-to-markdown.3006818 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 23.5% | 208/885 | $4.09 | 57 | 0.23h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 503 | samtools__samtools.aa823b5 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 22.5% | 321/1425 | n/a | 40 | 0.21h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 504 | alexpovel__srgn.89f943b |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 21.5% | 399/1852 | n/a | 81 | 0.25h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 505 | osgeo__proj.75d455c |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 21.5% | 1143/5319 | n/a | 78 | 0.26h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 506 | parcel-bundler__lightningcss.aa2ed1e |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 21.4% | 604/2828 | $3.65 | 57 | 0.20h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 507 | ast-grep__ast-grep.dde0fe0 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 21.2% | 187/882 | n/a | 57 | 0.22h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 508 | osgeo__gdal.0847f12 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 21.2% | 139/657 | $5.66 | 67 | 0.27h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 509 | ast-grep__ast-grep.dde0fe0 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 20.5% | 181/882 | n/a | 55 | 0.18h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 510 | kyoh86__richgo.313114f |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 20.0% | 109/546 | $3.48 | 60 | 0.21h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 511 | yoav-lavi__melody.f4af9b4 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 19.9% | 240/1205 | n/a | 155 | 0.23h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 512 | google__brotli.b3dc9cc |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 19.5% | 86/441 | n/a | 96 | 0.28h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 513 | tinycc__tinycc.9b8765d |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 19.5% | 385/1978 | $4.23 | 69 | 0.24h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 514 | jhspetersson__fselect.c3559ca |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 19.4% | 603/3115 | n/a | 68 | 0.25h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 515 | dalance__amber.69a0f52 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 19.3% | 109/565 | $2.68 | 47 | 0.15h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 516 | tarka__xcp.5e5b448 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 18.9% | 224/1184 | $3.24 | 54 | 0.21h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 517 | kyoh86__richgo.313114f |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 17.8% | 97/546 | $2.38 | 43 | 0.14h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 518 | nukesor__pueue.8b9d6fe |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 17.1% | 109/638 | $3.08 | 48 | 0.20h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 519 | jgm__pandoc.5caad90 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 17.0% | 889/5228 | $6.38 | 91 | 0.36h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 520 | tomnomnom__gron.88a6234 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 17.0% | 38/224 | n/a | 61 | 0.20h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 521 | hpjansson__chafa.dd4d4c1 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 16.4% | 317/1931 | $5.85 | 94 | 0.29h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 522 | danmar__cppcheck.0a5b103 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 16.3% | 347/2126 | n/a | 28 | 0.12h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 523 | duckdb__duckdb.bdb65ec |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 16.3% | 919/5650 | $6.77 | 121 | 0.29h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 524 | dandavison__delta.acd758f |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 16.2% | 154/950 | n/a | 80 | 0.26h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 525 | osgeo__proj.75d455c |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 15.8% | 840/5319 | $6.06 | 80 | 0.33h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 526 | jgm__pandoc.5caad90 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 15.7% | 819/5228 | n/a | 41 | 0.21h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 527 | jgm__pandoc.5caad90 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 14.9% | 779/5228 | n/a | 66 | 0.22h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 528 | nukesor__pueue.8b9d6fe |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 14.9% | 95/638 | $2.50 | 44 | 0.15h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 529 | bellard__quickjs.d7ae12a |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 14.5% | 440/3034 | n/a | 90 | 0.35h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 530 | tstack__lnav.ee34494 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 14.0% | 139/990 | n/a | 51 | 0.15h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 531 | osgeo__gdal.0847f12 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 14.0% | 92/657 | n/a | 56 | 0.16h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 532 | gromacs__gromacs.665ea4c |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 14.0% | 174/1245 | $3.13 | 43 | 0.15h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 533 | tstack__lnav.ee34494 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 13.6% | 135/990 | n/a | 49 | 0.16h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 534 | ip7z__7zip.839151e |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 13.6% | 142/1043 | n/a | 56 | 0.24h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 535 | samtools__samtools.aa823b5 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 13.5% | 192/1425 | n/a | 45 | 0.21h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 536 | nukesor__pueue.8b9d6fe |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 13.0% | 83/638 | $5.66 | 69 | 0.24h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 537 | rcoh__angle-grinder.9c2fc88 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 12.6% | 142/1130 | $5.07 | 74 | 0.25h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 538 | hush-shell__hush.560c33a |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 12.0% | 144/1201 | $6.39 | 112 | 0.32h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 539 | parcel-bundler__lightningcss.aa2ed1e |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 12.0% | 339/2828 | $3.96 | 74 | 0.22h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 540 | php__php-src.c891263 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 11.9% | 1696/14288 | $8.04 | 106 | 0.39h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 541 | tstack__lnav.ee34494 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 11.6% | 115/990 | $4.81 | 85 | 0.24h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 542 | zk-org__zk.10d93d5 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 10.6% | 117/1108 | $5.66 | 99 | 0.26h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 543 | gromacs__gromacs.665ea4c |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 10.4% | 129/1245 | n/a | 49 | 0.17h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 544 | pier-cli__pier.5e1bde9 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 10.0% | 69/692 | n/a | 82 | 0.19h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 545 | pier-cli__pier.5e1bde9 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 10.0% | 69/692 | n/a | 89 | 0.24h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 546 | htop-dev__htop.523600b |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 10.0% | 69/693 | n/a | 72 | 0.22h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 547 | ffmpeg__ffmpeg.360a402 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 9.9% | 302/3050 | $5.34 | 83 | 0.22h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 548 | htop-dev__htop.523600b |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 9.7% | 67/693 | n/a | 57 | 0.15h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 549 | ariga__atlas.6d81150 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 9.2% | 121/1318 | $4.47 | 71 | 0.23h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 550 | ffmpeg__ffmpeg.360a402 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 8.8% | 269/3050 | n/a | 54 | 0.19h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 551 | php__php-src.c891263 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 8.8% | 1254/14288 | n/a | 64 | 0.24h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 552 | ammarabouzor__tui-journal.2b4540d |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | warn 1 | 8.6% | 121/1403 | $4.65 | 83 | 0.22h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 553 | antonmedv__fx.86d0d34 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 8.6% | 176/2047 | n/a | 47 | 0.13h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 554 | php__php-src.c891263 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 8.5% | 1218/14288 | n/a | 82 | 0.32h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 555 | hush-shell__hush.560c33a |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 8.3% | 100/1201 | $6.14 | 83 | 0.30h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 556 | duckdb__duckdb.bdb65ec |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 8.2% | 466/5650 | n/a | 40 | 0.14h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 557 | ffmpeg__ffmpeg.360a402 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 8.2% | 250/3050 | n/a | 78 | 0.26h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 558 | duckdb__duckdb.bdb65ec |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 8.1% | 457/5650 | n/a | 88 | 0.26h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 559 | universal-ctags__ctags.243595e |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 7.8% | 175/2258 | n/a | 78 | 0.33h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 560 | hush-shell__hush.560c33a |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 7.2% | 87/1201 | $4.04 | 56 | 0.24h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 561 | astro__deadnix.d590041 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 6.5% | 39/602 | $4.15 | 62 | 0.26h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 562 | jqlang__jq.b33a763 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 5.5% | 333/6072 | n/a | 115 | 0.37h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 563 | danmar__cppcheck.0a5b103 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 5.3% | 112/2126 | $4.04 | 84 | 0.20h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 564 | danmar__cppcheck.0a5b103 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 4.5% | 96/2126 | n/a | 64 | 0.18h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 565 | rust-lang__mdbook.37273ba |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 4.1% | 46/1114 | $3.72 | 43 | 0.16h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 566 | antonmedv__fx.86d0d34 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 4.0% | 82/2047 | $6.90 | 118 | 0.31h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 567 | antonmedv__fx.86d0d34 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 4.0% | 82/2047 | n/a | 53 | 0.15h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 568 | gromacs__gromacs.665ea4c |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 3.9% | 49/1245 | n/a | 49 | 0.17h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 569 | johanneskaufmann__html-to-markdown.3006818 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 3.8% | 34/885 | $4.54 | 90 | 0.23h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 570 | unhappychoice__gittype.34b72d0 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | warn 1 | 3.6% | 27/742 | $3.00 | 59 | 0.16h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 571 | rvben__rumdl.2d75c4d |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | warn 1 | 2.7% | 91/3323 | $4.96 | 56 | 0.25h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 572 | sclevine__yj.8016400 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 2.1% | 16/767 | $5.98 | 109 | 0.30h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 573 | sclevine__yj.8016400 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 2.1% | 16/767 | n/a | 34 | 0.12h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 574 | sclevine__yj.8016400 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 2.1% | 16/767 | n/a | 53 | 0.18h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 575 | hairyhenderson__gomplate.05eb3aa |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 2.1% | 60/2926 | n/a | 58 | 0.22h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 576 | google__brotli.b3dc9cc |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 1.4% | 6/441 | n/a | 72 | 0.24h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 577 | sqlite__sqlite.839433d |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 1.3% | 182/13514 | $8.14 | 140 | 0.36h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 578 | doxygen__doxygen.966d98e |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | warn 1 | 1.3% | 3/230 | n/a | 61 | 0.17h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 579 | mikefarah__yq.602586d |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 1.1% | 23/2000 | n/a | 43 | 0.15h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 580 | osgeo__proj.75d455c |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 1.0% | 52/5319 | n/a | 46 | 0.20h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 581 | dandavison__delta.acd758f |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 0.7% | 7/950 | n/a | 81 | 0.19h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 582 | sqlite__sqlite.839433d |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 0.5% | 62/13514 | n/a | 75 | 0.22h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 583 | sigoden__argc.04a08f1 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 0.4% | 4/995 | $7.46 | 110 | 0.32h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 584 | sigoden__argc.04a08f1 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 0.4% | 4/995 | n/a | 76 | 0.27h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 585 | sigoden__argc.04a08f1 |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 0.4% | 4/995 | n/a | 84 | 0.30h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 586 | universal-ctags__ctags.243595e |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 0.1% | 3/2258 | $6.11 | 86 | 0.30h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 587 | universal-ctags__ctags.243595e |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 0.1% | 3/2258 | n/a | 62 | 0.29h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 588 | sqlite__sqlite.839433d |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 0.0% | 5/13514 | n/a | 97 | 0.32h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 589 | alecthomas__chroma.8d04def |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 0.0% | 0/503 | $4.76 | 74 | 0.18h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 590 | bellard__quickjs.d7ae12a |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 0.0% | 0/3034 | $6.76 | 118 | 0.27h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 591 | git-bahn__git-graph.87b4473 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | error | 0.0% | 0/568 | $6.07 | 96 | 0.31h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 592 | sharkdp__hyperfine.327d5f4 |
20260517T094700Z |
Mini-SWE-compatible no internet | GPT 5.5 (xhigh) | Codex /goal mini-SWE-compatible no-internet | open | ok | 0.0% | 0/291 | $4.75 | 76 | 0.25h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · usage audit |
| 593 | filosottile__age.706dfc1 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | error | 0.0% | 0/676 | $5.20 | 80 | 0.24h | Linux/x86_64 | 16 CPU / 30g | manifest · eval json · eval summary · agent summary · usage audit |
| 594 | bellard__quickjs.d7ae12a |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 0.0% | 0/3034 | n/a | 76 | 0.22h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 595 | alecthomas__chroma.8d04def |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 0.0% | 0/503 | n/a | 57 | 0.21h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 596 | sharkdp__hyperfine.327d5f4 |
20260518T232006Z |
Paper prompt + /goal no internet | GPT 5.5 (xhigh) | Codex /goal paper prompt no-internet, mini-SWE-style execution | open | ok | 0.0% | 0/291 | n/a | 42 | 0.16h | Linux/x86_64 | 16 CPU / 30g | not exported |
| 597 | alecthomas__chroma.8d04def |
20260519T215046Z |
Paper prompt + Goal contract no internet | GPT 5.5 (xhigh) | Codex /goal contract no-internet, mini-SWE-style execution | open | ok | 0.0% | 0/503 | n/a | 41 | 0.19h | Linux/x86_64 | 16 CPU / 30g | not exported |
Official Baseline Context
For orientation only. ProgramBench's public extended table reports mini-SWE-agent over 200 tasks, sorted by resolved, almost-resolved, then average pass rate.
| # | Model | Agent | Resolved | Almost | Avg. cost | Avg. calls |
|---|---|---|---|---|---|---|
| 1 | GPT 5.5 (xhigh) | mini-SWE-agent | 0.5% | 13.5% | $8.85 | 82 |
| 2 | GPT 5.5 (high) | mini-SWE-agent | 0.5% | 5.0% | $3.65 | 41 |
| 3 | Claude Opus 4.7 (xhigh) | mini-SWE-agent | 0.0% | 4.5% | $10.96 | 159 |
| 4 | Claude Opus 4.7 | mini-SWE-agent | 0.0% | 3.0% | $3.81 | 93 |
| 5 | Claude Opus 4.6 | mini-SWE-agent | 0.0% | 2.5% | $11.38 | 260 |
| 6 | GPT 5.5 | mini-SWE-agent | 0.0% | 1.5% | $1.21 | 19 |
| 7 | Claude Sonnet 4.6 | mini-SWE-agent | 0.0% | 1.0% | $26.73 | 472 |
| 8 | GPT 5.4 | mini-SWE-agent | 0.0% | 0.0% | $0.33 | 16 |
| 9 | Gemini 3.1 Pro | mini-SWE-agent | 0.0% | 0.0% | $1.51 | 94 |
| 10 | Gemini 3 Flash | mini-SWE-agent | 0.0% | 0.0% | $0.30 | 85 |
| 11 | Claude Haiku 4.5 | mini-SWE-agent | 0.0% | 0.0% | $0.80 | 124 |
| 12 | GPT 5.4 mini | mini-SWE-agent | 0.0% | 0.0% | $0.04 | 18 |
| 13 | GPT 5 mini | mini-SWE-agent | 0.0% | 0.0% | $0.03 | 15 |
GPT-5.5 baseline rows link to ProgramBench's official run-detail pages for total cost, total calls, distribution plots, and all 200 per-instance results.
Method Notes
GoalBench reports separate Codex /goal runs on ProgramBench tasks; these are not official mini-SWE-agent leaderboard submissions. Resolved means ProgramBench's filtered behavioral pass rate is exactly 100%, and almost resolved means at least 95%.
GoalBench currently keeps three public prompt tracks separate: mini-swe-compatible-nointernet for the already-published mini-SWE-compatible result, paper-prompt-nointernet for the ProgramBench paper prompt with /goal and mini-SWE-style execution, and paper-prompt-goal-contract-nointernet for the same scaffold with an explicit Codex Goal contract.
GoalBench uses a host-side wrapper to transport allowed black-box CLI interactions into the target container. This differs from mini-SWE-agent's in-container execution, but the wrapper is restricted to normal user-interface observations of the target executable and forbids source lookup, binary reading, disassembly, tracing, instrumentation, and evaluator/test access.
The public table is scoped to the latest published result set. Cost is estimated from Codex token logs, not billing; rows marked n/a or aggregate costs marked * have incomplete local token-log coverage, so calls are the more complete compute proxy for those tracks. See Task Details and the runbook for scoring, evidence, egress, and setup details. Sources: ProgramBench extended results and GPT 5.5 xhigh run detail.