Claude Research Agent Architecture for Treekipedia

Version: 4.0 Created: January 5, 2026 Updated: January 6, 2026 Status: Implemented and Tested

Executive Summary

Claude Code CLI-native research system for enriching Treekipedia's 67,743 species. Uses a unified research agent (consolidated from 4 agents) with auto-approved WebSearch for seamless operation. Queue-based workflow from web UI to CLI research.

Key Files:

• orchestrator/unified_research_prompt.py - NEW Single unified prompt for all 35 fields
• orchestrator/research_prompts.py - Legacy 4-agent prompts (deprecated)
• orchestrator/research_orchestrator.py - Queue and insights API (port 5003)
• .claude/skills/species-research/SKILL.md - NEW Claude Code skill definition
• .claude/hooks/approve-research-websearch.py - NEW Auto-approve hook for WebSearch
• .claude/settings.json - NEW Hook configuration

Recent Changes (v4.0):

1. Consolidated 4 agents into 1 unified research session per species
2. Added Claude Code skill for standardized research workflow
3. Added PreToolUse hook for auto-approving research WebSearches
4. Improved context building by researching all 35 fields in one session

1. Research Workflow Overview

┌─────────────────────────────────────────────────────────────────────────┐
│                    UNIFIED RESEARCH WORKFLOW (v4.0)                     │
├─────────────────────────────────────────────────────────────────────────┤
│                                                                          │
│  WEB UI                                                                  │
│    │                                                                     │
│    ▼                                                                     │
│  [Click Research Button] ─────► research_queue table                    │
│                                       │                                  │
│  CLI (with auto-approve hook)         │                                  │
│    │                                  ▼                                  │
│    ├── /research (skill) or python scripts/research_species.py --next  │
│    │                                  │                                  │
│    ▼                                  ▼                                  │
│  UNIFIED AGENT (1 per species):                                         │
│    └── ALL 35 fields in one session ─► WebSearch (auto-approved)       │
│        • Identity (4)     │            └── approve-research-websearch.py│
│        • Ecological (10)  │                                              │
│        • Morphological (10)│           Builds context as research       │
│        • Stewardship (11) │           progresses for better quality     │
│                                                                          │
│    ▼                                                                     │
│  POST /research/{taxon_id}/save ─────► insights table                   │
│                                              │                           │
│    ▼                                         ▼                           │
│  POST /queue/{id}/complete          species.*_ai columns synced         │
│                                                                          │
└─────────────────────────────────────────────────────────────────────────┘

Why Unified Agent (v4.0)?

Before (4 agents):

• Each agent ran independently
• No context sharing between agents
• 4 separate sessions per species
• Redundant searches for same data

After (1 unified agent):

• Single session builds cumulative context
• Information from taxonomy informs ecology searches
• Conservation status informs stewardship recommendations
• Better cross-referencing and consistency
• Fewer redundant searches

1.5 Auto-Approve Hook Configuration

The research system includes an automatic approval hook for WebSearch tool calls that match botanical research patterns. This eliminates manual approval prompts during research sessions.

Files

How It Works

1. PreToolUse Hook: Before any WebSearch call, the hook script runs
2. Pattern Matching: Query is checked against research keywords (species, IUCN, habitat, etc.)
3. Security: Blocked patterns prevent non-research searches (passwords, exploits, etc.)
4. Decision: Research queries are auto-approved; others fall through to manual approval

Approved Query Patterns

The hook auto-approves queries containing:

• Species/taxonomy terms: species, tree, plant, genus, family, flora
• Database names: IUCN, GBIF, POWO, WCVP, Kew, FAO, USDA
• Ecological terms: habitat, ecosystem, distribution, conservation
• Morphology terms: height, bark, leaf, flower, fruit
• Stewardship terms: propagation, cultivation, agroforestry, timber
• Common genus names: Acacia, Quercus, Pinus, Eucalyptus, etc.

Verification

# Test that research queries are approved
echo '{"tool_name": "WebSearch", "tool_input": {"query": "Acacia acuminata IUCN"}}' | \
  python3 .claude/hooks/approve-research-websearch.py
# Should output: {"hookSpecificOutput": {"permissionDecision": "allow", ...}}

# Test that non-research queries fall through
echo '{"tool_name": "WebSearch", "tool_input": {"query": "weather today"}}' | \
  python3 .claude/hooks/approve-research-websearch.py
# Should output nothing (exit 0, no auto-approval)

2. Research Fields (35 Total)

Identity Agent (4 fields)

popular_common_name, etymology, synonyms, identification_features

Ecological Agent (10 fields)

general_description, habitat, elevation_ranges, ecological_function,
native_adapted_habitats, conservation_status, compatible_soil_types,
climate_tolerance, tolerances, associated_species

Morphological Agent (10 fields)

growth_form, leaf_type, deciduous_evergreen, flower_color, fruit_type,
bark_characteristics, maximum_height, maximum_diameter, lifespan, maximum_tree_age

Stewardship Agent (11 fields)

stewardship_best_practices, planting_recipes, pruning_maintenance,
disease_pest_management, fire_management, propagation_methods,
cultural_significance, agroforestry_use_cases, timber_value,
non_timber_products, nutritional_caloric_value

3. Agent Architecture

Each agent has an exact prompt defined in orchestrator/research_prompts.py.

┌─────────────────────────────────────────────────────────────────────────┐
│                         4 SPECIALIZED AGENTS                             │
├─────────────────────────────────────────────────────────────────────────┤
│                                                                          │
│  IDENTITY AGENT (4 fields)                                               │
│    └── Names, etymology, synonyms, visual ID features                   │
│    └── Sources: POWO, WCVP, IPNI, regional floras                       │
│                                                                          │
│  ECOLOGICAL AGENT (10 fields)                                            │
│    └── Habitat, conservation, distribution, ecology                     │
│    └── Sources: IUCN, GBIF, FAO Ecocrop, ecological studies            │
│                                                                          │
│  MORPHOLOGICAL AGENT (10 fields)                                         │
│    └── Physical traits, dimensions, appearance                          │
│    └── Sources: Flora databases, botanical descriptions                 │
│                                                                          │
│  STEWARDSHIP AGENT (11 fields)                                           │
│    └── Cultivation, uses, cultural significance                         │
│    └── Sources: FAO, ICRAF, USDA, ethnobotanical databases             │
│                                                                          │
└─────────────────────────────────────────────────────────────────────────┘

Design Principles:

1. Exact prompts - Same prompt structure every time for reproducibility
2. Structured output - Each field has a specific JSON schema
3. Source citations - Every insight must cite sources with credibility scores
4. Confidence tracking - 0.0-1.0 confidence per insight

3. Research Prompt

RESEARCH_PROMPT = """
Research the tree species "{scientific_name}" (Family: {family}, Native to: {wcvp_native}).

Return a JSON object with these fields. Use null for unavailable data. Be concise but accurate.

IDENTITY:
- popular_common_name: Most widely-used English common name
- etymology: Origin/meaning of scientific name (brief)
- identification_features: 2-3 key visual ID features

ECOLOGICAL:
- general_description: 2-3 sentence botanical description
- habitat: Natural habitat types
- elevation_ranges: "min-max" in meters
- ecological_function: Key ecological roles
- native_adapted_habitats: Native range and climate zones
- conservation_status: IUCN status if known
- compatible_soil_types: Soil preferences
- climate_tolerance: Temperature/precipitation preferences
- tolerances: Drought/flood/salt tolerance

MORPHOLOGICAL:
- growth_form: tree/shrub/palm/etc
- leaf_type: simple/compound/needle/etc
- deciduous_evergreen: deciduous/evergreen/semi-deciduous
- flower_color: Primary colors
- fruit_type: Fruit/seed type
- bark_characteristics: Texture and color
- maximum_height: Number only (meters)
- maximum_diameter: Number only (meters DBH)
- lifespan: Typical lifespan
- maximum_tree_age: Number only (years)

STEWARDSHIP:
- stewardship_best_practices: Care guidelines
- planting_recipes: Planting conditions
- pruning_maintenance: Pruning needs
- disease_pest_management: Common issues
- fire_management: Fire tolerance
- propagation_methods: Seed/cutting/grafting

USES:
- cultural_significance: Traditional importance
- agroforestry_use_cases: Agroforestry applications
- timber_value: Wood quality/uses
- non_timber_products: Fruits, resins, medicines
- nutritional_caloric_value: Edible parts (if any)

Return ONLY valid JSON. No markdown, no explanation.
"""

3.5 Insight-Based Output Format (FAIR-Compliant)

Updated January 6, 2026 - Research agents now output insights with provenance.

Instead of flat field values, agents return insights that include:

• claim_value: The actual data (text or structured)
• confidence: 0.0-1.0 certainty score
• sources: Array of citations with URLs and metadata

Insight Output Schema

{
  "taxon_id": "AngMaFaFgCx14888-20",
  "species_name": "Quercus robur",
  "research_session_id": "session_2026-01-06_001",
  "model_version": "claude-3-haiku-20240307",
  "agent_type": "ecological",
  "insights": [
    {
      "claim_type": "maximum_height",
      "claim_value": {"value": 40, "unit": "meters", "note": "exceptional specimens to 45m"},
      "confidence": 0.90,
      "methodology": "extraction",
      "sources": [
        {
          "url": "https://www.conifers.org/cu/Quercus_robur.php",
          "title": "Quercus robur - The Gymnosperm Database",
          "type": "database",
          "accessed_date": "2026-01-06",
          "credibility": 0.85
        },
        {
          "url": "https://www.iucnredlist.org/species/194843/167525622",
          "title": "IUCN Red List - Quercus robur",
          "type": "database",
          "accessed_date": "2026-01-06",
          "credibility": 0.95
        }
      ]
    },
    {
      "claim_type": "habitat",
      "claim_value": {"text": "Temperate broadleaf and mixed forests, particularly on clay-rich soils in lowland areas"},
      "confidence": 0.85,
      "methodology": "extraction",
      "sources": [
        {
          "url": "https://www.worldagroforestry.org/treedb2/speciesprofile.php?Ession=0&Spession=0&Id=1559",
          "title": "World Agroforestry - Quercus robur",
          "type": "database",
          "accessed_date": "2026-01-06",
          "credibility": 0.80
        }
      ]
    },
    {
      "claim_type": "conservation_status",
      "claim_value": {"iucn_status": "LC", "full_name": "Least Concern", "year_assessed": 2017},
      "confidence": 0.95,
      "methodology": "extraction",
      "sources": [
        {
          "url": "https://www.iucnredlist.org/species/194843/167525622",
          "title": "IUCN Red List - Quercus robur",
          "type": "database",
          "doi": "10.2305/IUCN.UK.2018-1.RLTS.T194843A167525622.en",
          "accessed_date": "2026-01-06",
          "credibility": 0.98
        }
      ]
    }
  ],
  "token_usage": {
    "input_tokens": 1200,
    "output_tokens": 800,
    "cost_usd": 0.005
  }
}

Source Types & Credibility Scores

Updated Research Prompt (with sources)

INSIGHT_RESEARCH_PROMPT = """
Research the tree species "{scientific_name}" (Family: {family}, Native to: {wcvp_native}).

Return a JSON object with "insights" array. Each insight must include:
- claim_type: field name (e.g., "maximum_height", "habitat")
- claim_value: the data (use structured objects where appropriate)
- confidence: 0.0-1.0 (how certain are you?)
- sources: array of sources used (with url, title, type, credibility)

For sources, prefer authoritative databases:
- IUCN Red List (conservation)
- GBIF, POWO, WCVP (taxonomy, distribution)
- FAO, USDA (uses, cultivation)
- Published papers with DOIs

If you cannot find a reliable source, set confidence lower and note "model_inference" as methodology.

Fields to research:
{field_list}

Return ONLY valid JSON. Structure:
{{
  "insights": [
    {{
      "claim_type": "...",
      "claim_value": {{...}},
      "confidence": 0.85,
      "sources": [...]
    }}
  ]
}}
"""

Backward Compatibility

Insights are stored in the insights table. A sync process copies current values to species.*_ai columns for frontend compatibility:

-- Sync insights to flat species columns
UPDATE species s
SET general_description_ai = (
    SELECT i.claim_value->>'text'
    FROM insights i
    WHERE i.taxon_id = s.taxon_id
    AND i.claim_type = 'general_description'
    AND i.is_current = TRUE
    LIMIT 1
)
WHERE EXISTS (
    SELECT 1 FROM insights WHERE taxon_id = s.taxon_id AND is_current = TRUE
);

4. Tracking Schema

SQLite Database (research_tracking.db)

-- Track each research attempt
CREATE TABLE research_log (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  taxon_id TEXT NOT NULL,
  species_name TEXT NOT NULL,
  started_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
  completed_at TIMESTAMP,
  status TEXT DEFAULT 'in_progress',  -- in_progress, completed, failed
  fields_populated INTEGER,
  fields_total INTEGER DEFAULT 35,
  error_message TEXT,
  session_id TEXT  -- Group by CLI session
);

-- Track what was written to database
CREATE TABLE field_updates (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  research_log_id INTEGER NOT NULL,
  field_name TEXT NOT NULL,
  old_value TEXT,
  new_value TEXT,
  confidence REAL,
  FOREIGN KEY (research_log_id) REFERENCES research_log(id)
);

-- Session summaries
CREATE TABLE sessions (
  id TEXT PRIMARY KEY,  -- UUID or timestamp
  started_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
  completed_at TIMESTAMP,
  species_attempted INTEGER DEFAULT 0,
  species_completed INTEGER DEFAULT 0,
  species_failed INTEGER DEFAULT 0,
  notes TEXT
);

-- Aggregation views
CREATE VIEW research_progress AS
SELECT
  COUNT(*) as total_attempts,
  COUNT(*) FILTER (WHERE status = 'completed') as completed,
  COUNT(*) FILTER (WHERE status = 'failed') as failed,
  AVG(fields_populated) as avg_fields_populated,
  DATE(started_at) as date
FROM research_log
GROUP BY DATE(started_at);

5. Database Versioning

Migration file already created: treekipedia/database/07_research_versioning.sql

Adds to species table:

• research_version (INTEGER, default 0)
• research_date (TIMESTAMP)
• research_agent (TEXT)
• research_confidence (REAL)
• research_sources (JSONB)
• research_flags (JSONB)
• research_token_cost (REAL)

Plus research_history table for audit trail.

6. Implementation

Step 1: Run Database Migration

psql treekipedia < treekipedia/database/07_research_versioning.sql

Step 2: Create Research Script

File: scripts/research_species.py

#!/usr/bin/env python3
"""
Treekipedia Species Research Script
Run via Claude Code CLI to research species using subscription quota.

Usage:
  python scripts/research_species.py --taxon-id "AngQuRo1234-00"
  python scripts/research_species.py --batch 10
  python scripts/research_species.py --list-pending
"""

import argparse
import json
import sqlite3
import psycopg2
from datetime import datetime
from uuid import uuid4

# Database connections
PG_CONN = "dbname=treekipedia"
SQLITE_DB = "research_tracking.db"

def init_sqlite():
    """Initialize SQLite tracking database."""
    conn = sqlite3.connect(SQLITE_DB)
    conn.executescript("""
        CREATE TABLE IF NOT EXISTS research_log (
            id INTEGER PRIMARY KEY AUTOINCREMENT,
            taxon_id TEXT NOT NULL,
            species_name TEXT NOT NULL,
            started_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
            completed_at TIMESTAMP,
            status TEXT DEFAULT 'in_progress',
            fields_populated INTEGER,
            fields_total INTEGER DEFAULT 35,
            error_message TEXT,
            session_id TEXT
        );

        CREATE TABLE IF NOT EXISTS sessions (
            id TEXT PRIMARY KEY,
            started_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
            completed_at TIMESTAMP,
            species_attempted INTEGER DEFAULT 0,
            species_completed INTEGER DEFAULT 0,
            species_failed INTEGER DEFAULT 0,
            notes TEXT
        );
    """)
    conn.commit()
    return conn

def get_species_to_research(limit=10):
    """Get species that haven't been researched yet."""
    conn = psycopg2.connect(PG_CONN)
    cur = conn.cursor()
    cur.execute("""
        SELECT taxon_id, species_scientific_name, family, wcvp_native
        FROM species
        WHERE research_version = 0 OR research_version IS NULL
        ORDER BY total_occurrences DESC NULLS LAST
        LIMIT %s
    """, (limit,))
    species = cur.fetchall()
    conn.close()
    return species

def get_species_by_id(taxon_id):
    """Get a specific species by taxon_id."""
    conn = psycopg2.connect(PG_CONN)
    cur = conn.cursor()
    cur.execute("""
        SELECT taxon_id, species_scientific_name, family, wcvp_native
        FROM species
        WHERE taxon_id = %s
    """, (taxon_id,))
    species = cur.fetchone()
    conn.close()
    return species

def update_species_research(taxon_id, research_data):
    """Write research results to PostgreSQL."""
    conn = psycopg2.connect(PG_CONN)
    cur = conn.cursor()

    # Build UPDATE query for AI fields
    updates = []
    values = []

    field_mapping = {
        'popular_common_name': 'common_name',  # Map to existing column
        'general_description': 'general_description_ai',
        'habitat': 'habitat_ai',
        'elevation_ranges': 'elevation_ranges_ai',
        'ecological_function': 'ecological_function_ai',
        'native_adapted_habitats': 'native_adapted_habitats_ai',
        'conservation_status': 'conservation_status_ai',
        'compatible_soil_types': 'compatible_soil_types_ai',
        'growth_form': 'growth_form_ai',
        'leaf_type': 'leaf_type_ai',
        'deciduous_evergreen': 'deciduous_evergreen_ai',
        'flower_color': 'flower_color_ai',
        'fruit_type': 'fruit_type_ai',
        'bark_characteristics': 'bark_characteristics_ai',
        'maximum_height': 'maximum_height_ai',
        'maximum_diameter': 'maximum_diameter_ai',
        'lifespan': 'lifespan_ai',
        'maximum_tree_age': 'maximum_tree_age_ai',
        'stewardship_best_practices': 'stewardship_best_practices_ai',
        'planting_recipes': 'planting_recipes_ai',
        'pruning_maintenance': 'pruning_maintenance_ai',
        'disease_pest_management': 'disease_pest_management_ai',
        'fire_management': 'fire_management_ai',
        'cultural_significance': 'cultural_significance_ai',
        'agroforestry_use_cases': 'agroforestry_use_cases_ai',
    }

    for field, value in research_data.items():
        if value is not None and field in field_mapping:
            db_column = field_mapping[field]
            updates.append(f"{db_column} = %s")
            values.append(str(value) if not isinstance(value, str) else value)

    # Add versioning fields
    updates.extend([
        "research_version = COALESCE(research_version, 0) + 1",
        "research_date = %s",
        "research_agent = %s"
    ])
    values.extend([datetime.now(), 'claude-code-cli'])

    values.append(taxon_id)

    query = f"""
        UPDATE species
        SET {', '.join(updates)}
        WHERE taxon_id = %s
    """

    cur.execute(query, values)
    conn.commit()
    updated = cur.rowcount
    conn.close()

    return updated

def list_pending():
    """Show species pending research."""
    species = get_species_to_research(limit=20)
    print(f"\n{'='*60}")
    print(f"Top 20 Species Pending Research (by occurrence count)")
    print(f"{'='*60}\n")
    for s in species:
        print(f"  {s[0]}: {s[1]} ({s[2]})")
    print(f"\nTotal: {len(species)} shown")

def main():
    parser = argparse.ArgumentParser(description='Research species via Claude Code CLI')
    parser.add_argument('--taxon-id', help='Research specific species by taxon_id')
    parser.add_argument('--batch', type=int, help='Research N species from queue')
    parser.add_argument('--list-pending', action='store_true', help='List pending species')

    args = parser.parse_args()

    if args.list_pending:
        list_pending()
        return

    if args.taxon_id:
        species = get_species_by_id(args.taxon_id)
        if species:
            print(f"\nResearch target: {species[1]} ({species[0]})")
            print(f"Family: {species[2]}")
            print(f"Native to: {species[3]}")
            print("\n[Ready for Claude Code to research this species]")
        else:
            print(f"Species {args.taxon_id} not found")
        return

    if args.batch:
        species = get_species_to_research(limit=args.batch)
        print(f"\nBatch of {len(species)} species ready for research:")
        for s in species:
            print(f"  - {s[1]} ({s[0]})")
        return

    parser.print_help()

if __name__ == '__main__':
    main()

Step 3: Test Single Species

# List what's pending
python scripts/research_species.py --list-pending

# Research one species (Claude Code will execute the research)
python scripts/research_species.py --taxon-id "AngQuRo1234-00"

7. Usage Workflow

Manual Research (On-Demand)

1. User clicks Research button on species page
2. Frontend calls backend endpoint
3. Backend spawns CLI task or queues for next session
4. Claude Code researches species, writes to DB
5. Frontend polls for completion

Batch Research (Quota Maximization)

# Start a research session
cd /path/to/silvi-open
claude

# In Claude Code CLI:
> Research the next 10 unresearched species. For each:
> 1. Get species from database using scripts/research_species.py --batch 10
> 2. Research each species using the research prompt
> 3. Update database with results
> 4. Log to research_tracking.db

Weekly Schedule (Suggested)

Actual throughput depends on quota usage and will be learned empirically.

8. Quality Tracking

After Each Session

-- Check today's progress
SELECT
  COUNT(*) as researched_today,
  AVG(fields_populated) as avg_fields
FROM research_log
WHERE DATE(started_at) = DATE('now');

-- Overall progress
SELECT
  COUNT(*) FILTER (WHERE research_version >= 1) as researched,
  COUNT(*) as total,
  ROUND(100.0 * COUNT(*) FILTER (WHERE research_version >= 1) / COUNT(*), 2) as pct
FROM species;

Quality Checks

• Field completion: How many of 35 fields populated?
• Null rate: Which fields frequently return null?
• Consistency: Do similar species get similar results?

9. Files Created

10. Next Steps

1. Run migration: psql treekipedia < treekipedia/database/07_research_versioning.sql
2. Create research script: scripts/research_species.py
3. Initialize SQLite: Run script once to create tracking DB
4. Test on 1 species: Manually research, verify database update
5. Test on 5 species: Small batch, check quality
6. Iterate: Adjust prompts based on results

Appendix: Sample Research Output

{
  "popular_common_name": "English Oak",
  "etymology": "Quercus from Latin for oak; robur meaning strength",
  "identification_features": "Deeply lobed leaves with 4-5 pairs; acorns on long stalks",

  "general_description": "Large deciduous tree reaching 20-40m with broad spreading crown. Iconic European forest species.",
  "habitat": "Temperate broadleaf forests, particularly on clay-rich soils",
  "elevation_ranges": "0-1000",
  "ecological_function": "Keystone species supporting 280+ insect species",
  "conservation_status": "Least Concern (IUCN)",

  "growth_form": "Large deciduous tree",
  "maximum_height": 40,
  "maximum_diameter": 4,
  "deciduous_evergreen": "deciduous",
  "leaf_type": "Simple, deeply lobed",

  "stewardship_best_practices": "Plant in full sun to partial shade. Tolerates periodic flooding.",
  "propagation_methods": "Seed (acorns) - plant fresh in autumn"
}

CLI-native architecture for Claude Code Max 5x subscription