commit 9aa129f9efbb8bf248c15cdd99a3c0205c7295b7
parent fde1ea0a6da1bd0c26c61b0bc3cde325a3551fe1
Author: Brian Graham <brian@buildingbetterteams.de>
Date: Fri, 27 Feb 2026 21:32:03 +0100
Harvester: write to registry immediately after each search
Prevent data loss from session timeouts by requiring the agent to
append new entries to registry.jsonl after each API call or web search,
not accumulate in memory.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Diffstat:
1 file changed, 1 insertion(+), 0 deletions(-)
diff --git a/agents/harvester-agent.md b/agents/harvester-agent.md
@@ -173,6 +173,7 @@ You don't run these steps. Just know that your output feeds them, so complete an
- Use Semantic Scholar API as your primary discovery engine. Do not rely only on web search.
- Always include `arxiv_id` when the paper is on arXiv.
+- **Write immediately**: After each search (API call or web search), append any new papers found to `registry.jsonl` right away. Do NOT accumulate results in memory and write them later. This ensures no work is lost if the session ends unexpectedly.
- Log your search queries and results for reproducibility.
- Work in batches. Report count after each batch.
- Keep going until you hit 1000 or exhaust sources.
