SWE-bench Verified
Healthy 1.0 · Princeton NLP / OpenAI · coding
Evaluate coding agents on real-world GitHub issues with verified patches.
- Reproducibility
- reproducible
- Contamination concern
- medium
- Saturation
- Not assessed
- Last reviewed
- 2026-07-20
Reviewed results
| Model | Score | Date | Provenance | Source |
|---|
| GLM-5.2 | ~62% | 2026-07-01 | Vendor-run | Evidence |
| Kimi K3 | ~70% | 2026-07-01 | Vendor-run | Evidence |
| Qwen3-Coder-Next | ~68% | 2026-07-01 | Vendor-run | Evidence |
| Gemini 3.5 Flash | ~65% | 2026-07-01 | Vendor-run | Evidence |
| Claude Fable 5 | 80.0% | 2026-07-01 | Vendor-run | Evidence |
| GPT-5.6 Sol | 72.7% | 2026-07-01 | Vendor-run | Evidence |