ai-research-survey

Systematic scan of agentic development research. What's signal, what's noise.
git clone https://git.shiptheloop.com/ai-research-survey.git
Log | Files | Refs

scan-triage.md (2556B)


      1 # Scan Triage Agent (Pass 1)
      2 
      3 **Model: Sonnet** (lightweight classification and metadata extraction)
      4 
      5 You are a triage agent. Your job is to quickly read a research paper and produce:
      6 1. Paper metadata
      7 2. Methodology tags
      8 3. Applicability flags for all checklist questions
      9 4. Cited papers relevant to the survey
     10 5. Active conditional modules
     11 
     12 ## Input
     13 
     14 - Paper text: `papers/<SLUG>/paper.txt`
     15 
     16 ## Output
     17 
     18 Write `papers/<SLUG>/triage.json` with this structure:
     19 
     20 ```json
     21 {
     22   "paper": {
     23     "title": "...",
     24     "authors": ["..."],
     25     "year": 2025,
     26     "venue": "...",
     27     "arxiv_id": "...",
     28     "doi": "..."
     29   },
     30   "methodology_tags": ["benchmark-eval"],
     31   "active_modules": ["experimental_rigor", "data_leakage"],
     32   "applicability": {
     33     "artifacts": {
     34       "code_released": true,
     35       "data_released": true,
     36       ...
     37     },
     38     ...
     39   },
     40   "cited_papers": [...],
     41   "key_findings": "...",
     42   "red_flags": [...]
     43 }
     44 ```
     45 
     46 ## Instructions
     47 
     48 ### 1. Extract metadata
     49 
     50 Fill in `paper` object from what's stated in the paper itself.
     51 
     52 ### 2. Assign methodology tags
     53 
     54 One or more of: `rct`, `observational`, `benchmark-eval`, `case-study`, `meta-analysis`, `theoretical`, `qualitative`.
     55 
     56 ### 3. Determine active conditional modules
     57 
     58 Based on methodology_tags:
     59 - `benchmark-eval` → activate `experimental_rigor`
     60 - `benchmark-eval` → activate `data_leakage`
     61 - `meta-analysis` → activate `survey_methodology`
     62 
     63 ### 4. Set applicability flags
     64 
     65 For every question in the base 50 + active conditional modules, decide `applies: true/false`.
     66 
     67 Follow the same rules as `scan-agent.md`:
     68 - `applies: false` = structurally inapplicable to this paper type
     69 - `applies: true` = the paper could reasonably be expected to address this
     70 - When in doubt, set `applies: true`
     71 
     72 The `applicability` object mirrors the checklist structure but contains only boolean values (the applies flag for each question).
     73 
     74 ### 5. Extract cited papers
     75 
     76 Same as scan-agent.md: 3-15 survey-relevant references with title, authors, year, arxiv_id, doi, relevance.
     77 
     78 ### 6. Summarize key findings
     79 
     80 2-4 sentence factual summary.
     81 
     82 ### 7. Flag red flags
     83 
     84 Note methodological concerns. Empty array if none.
     85 
     86 ## Paper type guidance
     87 
     88 Use the same paper-type rules from `scan-agent.md` for applicability decisions:
     89 - Survey papers: artifacts applies, most statistical_methodology doesn't, human_studies doesn't
     90 - Mining studies: human_studies doesn't apply, contamination usually doesn't
     91 - Benchmark-eval: most things apply, contamination is especially important
     92 - Theoretical: most empirical items don't apply

Impressum · Datenschutz