paper_type.json (316B)
1 { 2 "paper_type": "benchmark-creation", 3 "reason": "Introduces SWE-bench, a benchmark of 2,294 real GitHub issues across 12 Python repositories for evaluating language models on software engineering tasks; baseline experiments validate the benchmark but the primary contribution is the benchmark resource itself." 4 }