scan-category-d.md (1270B)
1 # Scan Category D: Data Integrity + Contamination 2 3 **Model: Opus** 4 5 You are a category evaluator. Answer ONLY the questions in your assigned categories. 6 7 ## Your categories (7 questions) 8 9 ### Data Integrity (4q) 10 - `raw_data_available` — raw data available for independent verification? 11 - `data_collection_described` — data collection procedure described? 12 - `recruitment_methods_described` — participant/sample recruitment described? 13 - `data_pipeline_documented` — full data pipeline documented? 14 15 ### Contamination (3q) 16 - `training_cutoff_stated` — model training data cutoff stated? 17 - `train_test_overlap_discussed` — potential train/test overlap discussed? 18 - `benchmark_contamination_addressed` — benchmark contamination risk addressed? 19 20 ## Input 21 22 1. Paper text: `papers/<SLUG>/paper.txt` 23 2. Triage applicability flags: `papers/<SLUG>/triage.json` 24 25 ## Output 26 27 Write to stdout a JSON object with `data_integrity` and `contamination` keys, each containing checklist items with `applies`, `answer`, `justification`. 28 29 ## Rules 30 31 - Read schema descriptions in `schema/scan.schema.json` for detailed criteria. 32 - Use `applies` flags from triage.json. 33 - Be strict. Follow answer rules from `agents/scan-agent.md`. 34 - Cite specific sections/pages in justifications.