paper_type.json (334B)
1 { 2 "paper_type": "benchmark-creation", 3 "reason": "Introduces SWE-rebench, an automated pipeline that extracts 21,336 verifiable tasks from GitHub to create a new decontaminated benchmark for evaluating software engineering agents; the primary contribution is the benchmark and pipeline infrastructure, not experimental findings." 4 }