commit 1c6a723f27a3efff840ec524ce2f155237b0429a
parent aaa7097d653f63eb5a6a63611954e3c1ec4c4887
Author: Brian Graham <brian@buildingbetterteams.de>
Date: Fri, 27 Feb 2026 21:00:54 +0100
Document target scope (~1000 papers) and model assignments
- Target: ~1000 papers scanned, subset for deep eval
- Harvester: Sonnet (structured metadata, no deep reasoning)
- Scan agent: Opus (methodology quality judgment)
- Deep-eval agent: Opus
- Add harvester to agent tier design in requirements
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Diffstat:
2 files changed, 13 insertions(+), 1 deletion(-)
diff --git a/agents/harvester-agent.md b/agents/harvester-agent.md
@@ -1,6 +1,8 @@
# Harvester Agent
-You are a paper discovery agent. Your job is to find research papers relevant to the survey and add them to the registry. You do NOT download papers.
+**Model: Sonnet** (fast, cheap -- this agent does structured metadata extraction, not deep reasoning)
+
+You are a paper discovery agent. Your job is to find research papers relevant to the survey and add them to the registry. You do NOT download papers. Target: ~1,000 papers total in the registry.
## Input
diff --git a/context/requirements.md b/context/requirements.md
@@ -6,6 +6,8 @@ Research papers in the agentic AI / LLM programming space are proliferating rapi
This project is a **systematic review** evaluating the methodological quality of research papers in this space. The goal is not to determine which AI tool is best, but to assess how rigorously the claims are supported and help readers calibrate their confidence in reported findings.
+**Target scope**: ~1,000 papers. The scan agent is cheap enough to run on every paper. Deep eval is reserved for a smaller subset selected after scan results are in.
+
## Pipeline Architecture
```
@@ -27,18 +29,26 @@ discover -> download -> scan -> deep eval (optional) -> aggregate
## Agent Tier Design
+### Harvester Agent (Discovery)
+- Bulk discovery from arXiv, Semantic Scholar, HuggingFace, conferences
+- Fills registry entries with metadata only
+- **Model: Sonnet** (fast, cheap, structured metadata extraction)
+
### Scan Agent (Tier 1)
- Reads paper text
- Fills out `scan.json` per schema
- Scores six rubric dimensions
- Extracts claims with supporting evidence
+- Extracts cited papers for citation-chasing pipeline
- Fast, cheap, runs on every paper
+- **Model: Opus** (requires judgment on methodology quality)
### Deep-Eval Agent (Tier 2)
- Attempts to run released code
- Tries to reproduce key results
- Checks for benchmark contamination
- Expensive, slow, runs selectively
+- **Model: Opus**
## Tag Taxonomy
