Deep Research Core Purpose Deliver citation-tracked research reports through a structured pipeline with evidence persistence, source identity management, claim-level verification, and progressive context management. Autonomy Principle: Operate independently. Infer assumptions from context. Only stop for critical errors or incomprehensible queries. Surface high-materiality assumptions explicitly in the Introduction and Methodology rather than silently defaulting. --- Decision Tree Default assumptions: Technical query = technical audience. Comparison = balanced perspective. Trend = recent 1-2 y…

,\n r'\u003cdiv class=\"section\">\u003ch2 class=\"section-title\">\\1\u003c/h2>',\n html,\n flags=re.MULTILINE\n )\n\n # ### Subsection → \u003ch3 class=\"subsection-title\">Subsection\u003c/h3>\n html = re.sub(\n r'^### (.+)

Deep Research Core Purpose Deliver citation-tracked research reports through a structured pipeline with evidence persistence, source identity management, claim-level verification, and progressive context management. Autonomy Principle: Operate independently. Infer assumptions from context. Only stop for critical errors or incomprehensible queries. Surface high-materiality assumptions explicitly in the Introduction and Methodology rather than silently defaulting. --- Decision Tree Default assumptions: Technical query = technical audience. Comparison = balanced perspective. Trend = recent 1-2 y…

,\n r'\u003ch3 class=\"subsection-title\">\\1\u003c/h3>',\n html,\n flags=re.MULTILINE\n )\n\n # #### Subsubsection → \u003ch4 class=\"subsubsection-title\">Title\u003c/h4>\n html = re.sub(\n r'^#### (.+)

Deep Research Core Purpose Deliver citation-tracked research reports through a structured pipeline with evidence persistence, source identity management, claim-level verification, and progressive context management. Autonomy Principle: Operate independently. Infer assumptions from context. Only stop for critical errors or incomprehensible queries. Surface high-materiality assumptions explicitly in the Introduction and Methodology rather than silently defaulting. --- Decision Tree Default assumptions: Technical query = technical audience. Comparison = balanced perspective. Trend = recent 1-2 y…

,\n r'\u003ch4 class=\"subsubsection-title\">\\1\u003c/h4>',\n html,\n flags=re.MULTILINE\n )\n\n # Convert **bold** text\n html = re.sub(r'\\*\\*(.+?)\\*\\*', r'\u003cstrong>\\1\u003c/strong>', html)\n\n # Convert *italic* text\n html = re.sub(r'\\*(.+?)\\*', r'\u003cem>\\1\u003c/em>', html)\n\n # Convert inline code `code`\n html = re.sub(r'`(.+?)`', r'\u003ccode>\\1\u003c/code>', html)\n\n # Convert unordered lists\n html = _convert_lists(html)\n\n # Convert tables\n html = _convert_tables(html)\n\n # Convert paragraphs (wrap non-HTML lines in \u003cp> tags)\n html = _convert_paragraphs(html)\n\n # Close all open sections\n html = _close_sections(html)\n\n # Wrap executive summary if present\n html = html.replace(\n '\u003ch2 class=\"section-title\">Executive Summary\u003c/h2>',\n '\u003cdiv class=\"executive-summary\">\u003ch2 class=\"section-title\">Executive Summary\u003c/h2>'\n )\n if '\u003cdiv class=\"executive-summary\">' in html:\n # Close executive summary at the next section\n html = html.replace(\n '\u003c/h2>\\n\u003cdiv class=\"section\">',\n '\u003c/h2>\u003c/div>\\n\u003cdiv class=\"section\">',\n 1\n )\n\n return html\n\n\ndef _convert_bibliography_section(markdown: str) -> str:\n \"\"\"Convert bibliography section to HTML\"\"\"\n if not markdown.strip():\n return \"\"\n\n html = markdown\n\n # Convert each [N] citation to a proper bibliography entry\n # Look for patterns like [1] Title - URL\n html = re.sub(\n r'\\[(\\d+)\\]\\s*(.+?)\\s*-\\s*(https?://[^\\s\\)]+)',\n r'\u003cdiv class=\"bib-entry\">\u003cspan class=\"bib-number\">[\\1]\u003c/span> \u003ca href=\"\\3\" target=\"_blank\">\\2\u003c/a>\u003c/div>',\n html\n )\n\n # Convert any remaining **bold** sections\n html = re.sub(r'\\*\\*(.+?)\\*\\*', r'\u003cstrong>\\1\u003c/strong>', html)\n\n # Wrap in bibliography content div\n html = f'\u003cdiv class=\"bibliography-content\">{html}\u003c/div>'\n\n return html\n\n\ndef _convert_lists(html: str) -> str:\n \"\"\"Convert markdown lists to HTML lists\"\"\"\n lines = html.split('\\n')\n result = []\n in_list = False\n list_level = 0\n\n for i, line in enumerate(lines):\n stripped = line.strip()\n\n # Check for unordered list item\n if stripped.startswith('- ') or stripped.startswith('* '):\n if not in_list:\n result.append('\u003cul>')\n in_list = True\n list_level = len(line) - len(line.lstrip())\n\n # Get the content after the marker\n content = stripped[2:]\n result.append(f'\u003cli>{content}\u003c/li>')\n\n # Check for ordered list item\n elif re.match(r'^\\d+\\.\\s', stripped):\n if not in_list:\n result.append('\u003col>')\n in_list = True\n list_level = len(line) - len(line.lstrip())\n\n # Get the content after the number and period\n content = re.sub(r'^\\d+\\.\\s', '', stripped)\n result.append(f'\u003cli>{content}\u003c/li>')\n\n else:\n # Not a list item\n if in_list:\n # Check if we're still in the list (indented continuation)\n current_level = len(line) - len(line.lstrip())\n if current_level > list_level and stripped:\n # Continuation of previous list item\n if result[-1].endswith('\u003c/li>'):\n result[-1] = result[-1][:-5] + ' ' + stripped + '\u003c/li>'\n continue\n else:\n # End of list\n result.append('\u003c/ul>' if '\u003cul>' in '\\n'.join(result[-10:]) else '\u003c/ol>')\n in_list = False\n list_level = 0\n\n result.append(line)\n\n # Close any remaining open list\n if in_list:\n result.append('\u003c/ul>' if '\u003cul>' in '\\n'.join(result[-10:]) else '\u003c/ol>')\n\n return '\\n'.join(result)\n\n\ndef _convert_tables(html: str) -> str:\n \"\"\"Convert markdown tables to HTML tables\"\"\"\n lines = html.split('\\n')\n result = []\n in_table = False\n\n for i, line in enumerate(lines):\n if '|' in line and line.strip().startswith('|'):\n if not in_table:\n result.append('\u003ctable>')\n in_table = True\n # This is the header row\n cells = [cell.strip() for cell in line.split('|')[1:-1]]\n result.append('\u003cthead>\u003ctr>')\n for cell in cells:\n result.append(f'\u003cth>{cell}\u003c/th>')\n result.append('\u003c/tr>\u003c/thead>')\n result.append('\u003ctbody>')\n elif '---' in line:\n # Skip separator row\n continue\n else:\n # Data row\n cells = [cell.strip() for cell in line.split('|')[1:-1]]\n result.append('\u003ctr>')\n for cell in cells:\n result.append(f'\u003ctd>{cell}\u003c/td>')\n result.append('\u003c/tr>')\n else:\n if in_table:\n result.append('\u003c/tbody>\u003c/table>')\n in_table = False\n result.append(line)\n\n if in_table:\n result.append('\u003c/tbody>\u003c/table>')\n\n return '\\n'.join(result)\n\n\ndef _convert_paragraphs(html: str) -> str:\n \"\"\"Wrap non-HTML lines in paragraph tags\"\"\"\n lines = html.split('\\n')\n result = []\n in_paragraph = False\n\n for line in lines:\n stripped = line.strip()\n\n # Skip empty lines\n if not stripped:\n if in_paragraph:\n result.append('\u003c/p>')\n in_paragraph = False\n result.append(line)\n continue\n\n # Skip lines that are already HTML tags\n if (stripped.startswith('\u003c') and stripped.endswith('>')) or \\\n stripped.startswith('\u003c/') or \\\n '\u003ch' in stripped or '\u003cdiv' in stripped or '\u003cul' in stripped or \\\n '\u003col' in stripped or '\u003cli' in stripped or '\u003ctable' in stripped or \\\n '\u003c/div>' in stripped or '\u003c/ul>' in stripped or '\u003c/ol>' in stripped:\n if in_paragraph:\n result.append('\u003c/p>')\n in_paragraph = False\n result.append(line)\n continue\n\n # Regular text line - wrap in paragraph\n if not in_paragraph:\n result.append('\u003cp>' + line)\n in_paragraph = True\n else:\n result.append(line)\n\n if in_paragraph:\n result.append('\u003c/p>')\n\n return '\\n'.join(result)\n\n\ndef _close_sections(html: str) -> str:\n \"\"\"Close all open section divs\"\"\"\n # Count open and closed divs\n open_divs = html.count('\u003cdiv class=\"section\">')\n closed_divs = html.count('\u003c/div>')\n\n # Add closing divs for sections\n # Each section should be closed before the next section starts\n lines = html.split('\\n')\n result = []\n section_open = False\n\n for i, line in enumerate(lines):\n if '\u003cdiv class=\"section\">' in line:\n if section_open:\n result.append('\u003c/div>') # Close previous section\n section_open = True\n result.append(line)\n\n # Close final section if still open\n if section_open:\n result.append('\u003c/div>')\n\n return '\\n'.join(result)\n\n\ndef main():\n \"\"\"Test the converter with a sample markdown file\"\"\"\n import sys\n\n if len(sys.argv) \u003c 2:\n print(\"Usage: python md_to_html.py \u003cmarkdown_file>\")\n sys.exit(1)\n\n md_file = Path(sys.argv[1])\n if not md_file.exists():\n print(f\"Error: File {md_file} not found\")\n sys.exit(1)\n\n markdown_text = md_file.read_text()\n content_html, bib_html = convert_markdown_to_html(markdown_text)\n\n print(\"=== CONTENT HTML ===\")\n print(content_html[:1000])\n print(\"\\n=== BIBLIOGRAPHY HTML ===\")\n print(bib_html[:500])\n\n\nif __name__ == \"__main__\":\n main()\n","content_type":"text/x-python; charset=utf-8","language":"python","size":9965,"content_sha256":"506c38771d32fbccf931a27b7b64cf20c0df5113b2914b051c17e9d178fc1b17"},{"filename":"scripts/research_engine.py","content":"#!/usr/bin/env python3\n\"\"\"\nDeep Research Engine — STATE SCAFFOLD (not a runtime orchestrator)\n\nThis file provides phase instruction templates and research state persistence.\nIt does NOT drive Claude Code — Claude is the orchestrator; this file provides\ndata structures and CLI utilities for state management.\n\nFor the actual research workflow, see reference/methodology.md.\nFor the evidence substrate, see scripts/citation_manager.py and scripts/evidence_store.py.\n\"\"\"\n\nimport argparse\nimport json\nimport sys\nimport time\nfrom datetime import datetime\nfrom pathlib import Path\nfrom typing import Dict, List, Optional, Any\nfrom dataclasses import dataclass, asdict\nfrom enum import Enum\n\n\nclass ResearchPhase(Enum):\n \"\"\"Research pipeline phases\"\"\"\n SCOPE = \"scope\"\n PLAN = \"plan\"\n RETRIEVE = \"retrieve\"\n TRIANGULATE = \"triangulate\"\n SYNTHESIZE = \"synthesize\"\n CRITIQUE = \"critique\"\n REFINE = \"refine\"\n PACKAGE = \"package\"\n\n\nclass ResearchMode(Enum):\n \"\"\"Research depth modes\"\"\"\n QUICK = \"quick\" # 3 phases: scope, retrieve, package\n STANDARD = \"standard\" # 6 phases: skip refine and critique\n DEEP = \"deep\" # Full 8 phases\n ULTRADEEP = \"ultradeep\" # 8 phases + extended iterations\n\n\n@dataclass\nclass Source:\n \"\"\"Represents a research source\"\"\"\n url: str\n title: str\n snippet: str\n retrieved_at: str\n credibility_score: float = 0.0\n source_type: str = \"web\" # web, academic, documentation, code\n verification_status: str = \"unverified\" # unverified, verified, conflicted\n\n def to_citation(self, index: int) -> str:\n \"\"\"Generate citation string\"\"\"\n return f\"[{index}] {self.title} - {self.url} (Retrieved: {self.retrieved_at})\"\n\n\n@dataclass\nclass ResearchState:\n \"\"\"Maintains research state across phases\"\"\"\n query: str\n mode: ResearchMode\n phase: ResearchPhase\n scope: Dict[str, Any]\n plan: Dict[str, Any]\n sources: List[Source]\n findings: List[Dict[str, Any]]\n synthesis: Dict[str, Any]\n critique: Dict[str, Any]\n report: str\n metadata: Dict[str, Any]\n\n def save(self, filepath: Path):\n \"\"\"Save research state to file with retry logic\"\"\"\n max_retries = 3\n for attempt in range(max_retries):\n try:\n with open(filepath, 'w') as f:\n json.dump(self._serialize(), f, indent=2)\n return # Success\n except (IOError, OSError) as e:\n if attempt == max_retries - 1:\n # Final attempt failed\n raise IOError(f\"Failed to save state after {max_retries} attempts: {e}\")\n # Wait with exponential backoff before retry\n wait_time = (attempt + 1) * 0.5 # 0.5s, 1s, 1.5s\n time.sleep(wait_time)\n\n def _serialize(self) -> dict:\n \"\"\"Convert to serializable dict\"\"\"\n return {\n 'query': self.query,\n 'mode': self.mode.value,\n 'phase': self.phase.value,\n 'scope': self.scope,\n 'plan': self.plan,\n 'sources': [asdict(s) for s in self.sources],\n 'findings': self.findings,\n 'synthesis': self.synthesis,\n 'critique': self.critique,\n 'report': self.report,\n 'metadata': self.metadata\n }\n\n @classmethod\n def load(cls, filepath: Path) -> 'ResearchState':\n \"\"\"Load research state from file\"\"\"\n with open(filepath, 'r') as f:\n data = json.load(f)\n\n return cls(\n query=data['query'],\n mode=ResearchMode(data['mode']),\n phase=ResearchPhase(data['phase']),\n scope=data['scope'],\n plan=data['plan'],\n sources=[Source(**s) for s in data['sources']],\n findings=data['findings'],\n synthesis=data['synthesis'],\n critique=data['critique'],\n report=data['report'],\n metadata=data['metadata']\n )\n\n\nclass ResearchEngine:\n \"\"\"Main research orchestration engine\"\"\"\n\n def __init__(self, mode: ResearchMode = ResearchMode.STANDARD):\n self.mode = mode\n self.state: Optional[ResearchState] = None\n self.output_dir = Path.home() / \".claude\" / \"research_output\"\n self.output_dir.mkdir(parents=True, exist_ok=True)\n\n def initialize_research(self, query: str) -> ResearchState:\n \"\"\"Initialize new research session\"\"\"\n self.state = ResearchState(\n query=query,\n mode=self.mode,\n phase=ResearchPhase.SCOPE,\n scope={},\n plan={},\n sources=[],\n findings=[],\n synthesis={},\n critique={},\n report=\"\",\n metadata={\n 'started_at': datetime.now().isoformat(),\n 'version': '1.0'\n }\n )\n return self.state\n\n def get_phase_instructions(self, phase: ResearchPhase) -> str:\n \"\"\"Get instructions for current phase\"\"\"\n instructions = {\n ResearchPhase.SCOPE: \"\"\"\n# Phase 1: SCOPE\n\nYour task: Define research boundaries and success criteria\n\n## Execute:\n1. Decompose the question into 3-5 core components\n2. Identify 2-4 key stakeholder perspectives\n3. Define what's IN scope and what's OUT of scope\n4. List 3-5 success criteria for this research\n5. Document 3-5 assumptions that need validation\n\n## Output Format:\n```json\n{\n \"core_components\": [\"component1\", \"component2\", ...],\n \"stakeholder_perspectives\": [\"perspective1\", \"perspective2\", ...],\n \"in_scope\": [\"item1\", \"item2\", ...],\n \"out_of_scope\": [\"item1\", \"item2\", ...],\n \"success_criteria\": [\"criteria1\", \"criteria2\", ...],\n \"assumptions\": [\"assumption1\", \"assumption2\", ...]\n}\n```\n\nUse extended reasoning to explore multiple framings before finalizing scope.\n\"\"\",\n ResearchPhase.PLAN: \"\"\"\n# Phase 2: PLAN\n\nYour task: Create intelligent research roadmap\n\n## Execute:\n1. Identify 5-10 primary sources to investigate\n2. List 5-10 secondary/backup sources\n3. Map knowledge dependencies (what must be understood first)\n4. Create 10-15 search query variations\n5. Plan triangulation approach (how to verify claims)\n6. Define 3-5 quality gates\n\n## Output Format:\n```json\n{\n \"primary_sources\": [\"source_type1\", \"source_type2\", ...],\n \"secondary_sources\": [\"source_type1\", \"source_type2\", ...],\n \"knowledge_dependencies\": {\"concept1\": [\"prerequisite1\", \"prerequisite2\"], ...},\n \"search_queries\": [\"query1\", \"query2\", ...],\n \"triangulation_strategy\": \"description of verification approach\",\n \"quality_gates\": [\"gate1\", \"gate2\", ...]\n}\n```\n\nUse Graph-of-Thoughts: branch into 3-4 potential research paths, evaluate, then converge on optimal strategy.\n\"\"\",\n ResearchPhase.RETRIEVE: \"\"\"\n# Phase 3: RETRIEVE\n\nYour task: Systematically collect information from multiple sources\n\n## Execute:\n1. Use WebSearch with iterative query refinement (minimum 10 searches)\n2. Use WebFetch to deep-dive into 5-10 most promising sources\n3. Extract key passages with metadata\n4. Track information gaps\n5. Follow 2-3 promising tangents\n6. Ensure source diversity (different domains, perspectives)\n\n## Tools to Use:\n- WebSearch: For current information and broad coverage\n- WebFetch: For detailed extraction from specific URLs\n- Grep/Read: For local documentation if relevant\n- Task: Spawn 2-3 parallel retrieval agents for efficiency\n\n## Output:\nStore all sources with metadata. Each source should include:\n- URL/location\n- Title\n- Key excerpts\n- Relevance score\n- Source type\n- Retrieved timestamp\n\nAim for 15-30 distinct sources minimum.\n\"\"\",\n ResearchPhase.TRIANGULATE: \"\"\"\n# Phase 4: TRIANGULATE\n\nYour task: Validate information across multiple independent sources\n\n## Execute:\n1. List all major claims from retrieved information\n2. For each claim, find 3+ independent confirmatory sources\n3. Flag any contradictions or uncertainties\n4. Assess source credibility (domain expertise, recency, bias)\n5. Document consensus areas vs. debate areas\n6. Mark verification status for each claim\n\n## Quality Standards:\n- Core claims MUST have 3+ independent sources\n- Flag any single-source claims as \"unverified\"\n- Note information recency\n- Identify potential biases\n\n## Output Format:\n```json\n{\n \"verified_claims\": [\n {\n \"claim\": \"statement\",\n \"sources\": [\"source1\", \"source2\", \"source3\"],\n \"confidence\": \"high|medium|low\"\n }\n ],\n \"unverified_claims\": [...],\n \"contradictions\": [\n {\n \"topic\": \"what's contradicted\",\n \"viewpoint1\": {\"claim\": \"...\", \"sources\": [...]},\n \"viewpoint2\": {\"claim\": \"...\", \"sources\": [...]}\n }\n ]\n}\n```\n\"\"\",\n ResearchPhase.SYNTHESIZE: \"\"\"\n# Phase 5: SYNTHESIZE\n\nYour task: Connect insights and generate novel understanding\n\n## Execute:\n1. Identify 5-10 key patterns across sources\n2. Map relationships between concepts\n3. Generate 3-5 insights that go beyond source material\n4. Create conceptual frameworks or mental models\n5. Build argument structures\n6. Develop evidence hierarchies\n\n## Use Extended Reasoning:\n- Explore non-obvious connections\n- Consider second-order implications\n- Think about what sources might be missing\n- Generate novel hypotheses\n\n## Output Format:\n```json\n{\n \"patterns\": [\"pattern1\", \"pattern2\", ...],\n \"concept_relationships\": {\"concept1\": [\"related_to1\", \"related_to2\"], ...},\n \"novel_insights\": [\"insight1\", \"insight2\", ...],\n \"frameworks\": [\"framework_description1\", ...],\n \"key_arguments\": [\n {\n \"argument\": \"main claim\",\n \"supporting_evidence\": [\"evidence1\", \"evidence2\"],\n \"strength\": \"strong|moderate|weak\"\n }\n ]\n}\n```\n\"\"\",\n ResearchPhase.CRITIQUE: \"\"\"\n# Phase 6: CRITIQUE\n\nYour task: Rigorously evaluate research quality\n\n## Execute Red Team Analysis:\n1. Check logical consistency\n2. Verify citation completeness\n3. Identify gaps or weaknesses\n4. Assess balance and objectivity\n5. Test alternative interpretations\n6. Challenge assumptions\n\n## Red Team Questions:\n- What's missing from this research?\n- What could be wrong?\n- What alternative explanations exist?\n- What biases might be present?\n- What counterfactuals should be considered?\n- What would a skeptic say?\n\n## Output Format:\n```json\n{\n \"strengths\": [\"strength1\", \"strength2\", ...],\n \"weaknesses\": [\"weakness1\", \"weakness2\", ...],\n \"gaps\": [\"gap1\", \"gap2\", ...],\n \"biases\": [\"bias1\", \"bias2\", ...],\n \"improvements_needed\": [\n {\n \"issue\": \"description\",\n \"recommendation\": \"how to fix\",\n \"priority\": \"high|medium|low\"\n }\n ]\n}\n```\n\"\"\",\n ResearchPhase.REFINE: \"\"\"\n# Phase 7: REFINE\n\nYour task: Address gaps and strengthen weak areas\n\n## Execute:\n1. Conduct additional research for identified gaps\n2. Strengthen weak arguments with more evidence\n3. Add missing perspectives\n4. Resolve contradictions where possible\n5. Enhance clarity and structure\n6. Verify all revised content\n\n## Focus On:\n- High priority improvements from critique\n- Missing stakeholder perspectives\n- Weak evidence chains\n- Unclear explanations\n\n## Output:\nUpdated findings, sources, and synthesis with improvements documented.\n\"\"\",\n ResearchPhase.PACKAGE: \"\"\"\n# Phase 8: PACKAGE\n\nYour task: Deliver professional, actionable research report\n\n## Generate Complete Report:\n\n```markdown\n# Research Report: [Topic]\n\n## Executive Summary\n[3-5 key findings bullets]\n[Primary recommendation]\n[Confidence level: High/Medium/Low]\n\n## Introduction\n### Research Question\n[Original question]\n\n### Scope & Methodology\n[What was investigated and how]\n\n### Key Assumptions\n[Important assumptions made]\n\n## Main Analysis\n\n### Finding 1: [Title]\n[Detailed explanation with evidence]\n[Citations: [1], [2], [3]]\n\n### Finding 2: [Title]\n[Detailed explanation with evidence]\n[Citations: [4], [5], [6]]\n\n[Continue for all findings...]\n\n## Synthesis & Insights\n[Patterns and connections]\n[Novel insights]\n[Implications]\n\n## Limitations & Caveats\n[Known gaps]\n[Assumptions]\n[Areas of uncertainty]\n\n## Recommendations\n[Action items]\n[Next steps]\n[Further research needs]\n\n## Bibliography\n[1] Source 1 full citation\n[2] Source 2 full citation\n...\n\n## Appendix: Methodology\n[Research process]\n[Sources consulted]\n[Verification approach]\n```\n\nSave report to file with timestamp.\n\"\"\"\n }\n\n return instructions.get(phase, \"No instructions available for this phase\")\n\n def execute_phase(self, phase: ResearchPhase) -> Dict[str, Any]:\n \"\"\"Execute a research phase\"\"\"\n print(f\"\\n{'='*80}\")\n print(f\"PHASE {phase.value.upper()}: Starting...\")\n print(f\"{'='*80}\\n\")\n\n instructions = self.get_phase_instructions(phase)\n print(instructions)\n\n # In real usage, Claude will execute these instructions\n # This returns a structured result that Claude should populate\n result = {\n 'phase': phase.value,\n 'status': 'instructions_displayed',\n 'timestamp': datetime.now().isoformat()\n }\n\n return result\n\n def run_pipeline(self, query: str) -> str:\n \"\"\"Run complete research pipeline\"\"\"\n print(f\"\\n{'#'*80}\")\n print(f\"# DEEP RESEARCH ENGINE\")\n print(f\"# Query: {query}\")\n print(f\"# Mode: {self.mode.value}\")\n print(f\"{'#'*80}\\n\")\n\n # Initialize research\n self.initialize_research(query)\n\n # Determine phases based on mode\n phases = self._get_phases_for_mode()\n\n # Execute each phase\n for phase in phases:\n self.state.phase = phase\n result = self.execute_phase(phase)\n\n # Save state after each phase\n state_file = self.output_dir / f\"research_state_{datetime.now().strftime('%Y%m%d_%H%M%S')}.json\"\n self.state.save(state_file)\n print(f\"\\n✓ Phase {phase.value} complete. State saved to: {state_file}\\n\")\n\n # Generate report path\n report_file = self.output_dir / f\"research_report_{datetime.now().strftime('%Y%m%d_%H%M%S')}.md\"\n\n print(f\"\\n{'='*80}\")\n print(f\"RESEARCH PIPELINE COMPLETE\")\n print(f\"Report will be saved to: {report_file}\")\n print(f\"{'='*80}\\n\")\n\n return str(report_file)\n\n def _get_phases_for_mode(self) -> List[ResearchPhase]:\n \"\"\"Get phases based on research mode\"\"\"\n if self.mode == ResearchMode.QUICK:\n return [\n ResearchPhase.SCOPE,\n ResearchPhase.RETRIEVE,\n ResearchPhase.PACKAGE\n ]\n elif self.mode == ResearchMode.STANDARD:\n return [\n ResearchPhase.SCOPE,\n ResearchPhase.PLAN,\n ResearchPhase.RETRIEVE,\n ResearchPhase.TRIANGULATE,\n ResearchPhase.SYNTHESIZE,\n ResearchPhase.PACKAGE\n ]\n elif self.mode == ResearchMode.DEEP:\n return list(ResearchPhase)\n elif self.mode == ResearchMode.ULTRADEEP:\n # In ultradeep, we might iterate some phases\n return list(ResearchPhase)\n\n return list(ResearchPhase)\n\n\ndef main():\n \"\"\"CLI entry point\"\"\"\n parser = argparse.ArgumentParser(\n description=\"Deep Research Engine for Claude Code\",\n formatter_class=argparse.RawDescriptionHelpFormatter,\n epilog=\"\"\"\nExamples:\n python research_engine.py --query \"state of quantum computing 2025\" --mode deep\n python research_engine.py --query \"PostgreSQL vs Supabase comparison\" --mode standard\n python research_engine.py -q \"longevity biotech funding trends\" -m ultradeep\n \"\"\"\n )\n\n parser.add_argument(\n '--query', '-q',\n type=str,\n required=True,\n help='Research question or topic'\n )\n\n parser.add_argument(\n '--mode', '-m',\n type=str,\n choices=['quick', 'standard', 'deep', 'ultradeep'],\n default='standard',\n help='Research depth mode (default: standard)'\n )\n\n parser.add_argument(\n '--resume',\n type=str,\n help='Resume from saved state file'\n )\n\n args = parser.parse_args()\n\n # Initialize engine\n mode = ResearchMode(args.mode)\n engine = ResearchEngine(mode=mode)\n\n if args.resume:\n # Load previous state\n state_file = Path(args.resume)\n if not state_file.exists():\n print(f\"Error: State file not found: {state_file}\", file=sys.stderr)\n sys.exit(1)\n engine.state = ResearchState.load(state_file)\n print(f\"Resumed research from: {state_file}\")\n\n # Run pipeline\n report_path = engine.run_pipeline(args.query)\n\n print(f\"\\nResearch complete! Report path: {report_path}\")\n print(f\"\\nNow Claude should execute each phase using the displayed instructions.\")\n\n\nif __name__ == '__main__':\n main()\n","content_type":"text/x-python; charset=utf-8","language":"python","size":16765,"content_sha256":"5eb7ef01642e8158d7e7d8eb044a71aca3334493f4b2c6d7b4e1299e074cbfd6"},{"filename":"scripts/source_evaluator.py","content":"#!/usr/bin/env python3\n\"\"\"\nSource Credibility Evaluator\nAssesses source quality, credibility, and potential biases\n\"\"\"\n\nfrom dataclasses import dataclass\nfrom typing import List, Dict, Optional\nfrom urllib.parse import urlparse\nfrom datetime import datetime, timedelta\nimport re\n\n\n@dataclass\nclass CredibilityScore:\n \"\"\"Represents source credibility assessment\"\"\"\n overall_score: float # 0-100\n domain_authority: float # 0-100\n recency: float # 0-100\n expertise: float # 0-100\n bias_score: float # 0-100 (higher = more neutral)\n factors: Dict[str, str]\n recommendation: str # \"high_trust\", \"moderate_trust\", \"low_trust\", \"verify\"\n\n\nclass SourceEvaluator:\n \"\"\"Evaluates source credibility and quality\"\"\"\n\n # Domain reputation tiers\n HIGH_AUTHORITY_DOMAINS = {\n # Academic & Research\n 'arxiv.org', 'nature.com', 'science.org', 'cell.com', 'nejm.org',\n 'thelancet.com', 'springer.com', 'sciencedirect.com', 'plos.org',\n 'ieee.org', 'acm.org', 'pubmed.ncbi.nlm.nih.gov',\n\n # Government & International Organizations\n 'nih.gov', 'cdc.gov', 'who.int', 'fda.gov', 'nasa.gov',\n 'gov.uk', 'europa.eu', 'un.org',\n\n # Established Tech Documentation\n 'docs.python.org', 'developer.mozilla.org', 'docs.microsoft.com',\n 'cloud.google.com', 'aws.amazon.com', 'kubernetes.io',\n\n # Reputable News (Fact-check verified)\n 'reuters.com', 'apnews.com', 'bbc.com', 'economist.com',\n 'nature.com/news', 'scientificamerican.com'\n }\n\n MODERATE_AUTHORITY_DOMAINS = {\n # Tech News & Analysis\n 'techcrunch.com', 'theverge.com', 'arstechnica.com', 'wired.com',\n 'zdnet.com', 'cnet.com',\n\n # Industry Publications\n 'forbes.com', 'bloomberg.com', 'wsj.com', 'ft.com',\n\n # Educational\n 'wikipedia.org', 'britannica.com', 'khanacademy.org',\n\n # Tech Blogs (established)\n 'medium.com', 'dev.to', 'stackoverflow.com', 'github.com'\n }\n\n LOW_AUTHORITY_INDICATORS = [\n 'blogspot.com', 'wordpress.com', 'wix.com', 'substack.com'\n ]\n\n def __init__(self):\n pass\n\n def evaluate_source(\n self,\n url: str,\n title: str,\n content: Optional[str] = None,\n publication_date: Optional[str] = None,\n author: Optional[str] = None\n ) -> CredibilityScore:\n \"\"\"Evaluate source credibility\"\"\"\n\n domain = self._extract_domain(url)\n\n # Calculate component scores\n domain_score = self._evaluate_domain_authority(domain)\n recency_score = self._evaluate_recency(publication_date)\n expertise_score = self._evaluate_expertise(domain, title, author)\n bias_score = self._evaluate_bias(domain, title, content)\n\n # Calculate overall score (weighted average)\n overall = (\n domain_score * 0.35 +\n recency_score * 0.20 +\n expertise_score * 0.25 +\n bias_score * 0.20\n )\n\n # Determine factors\n factors = self._identify_factors(\n domain, domain_score, recency_score, expertise_score, bias_score\n )\n\n # Generate recommendation\n recommendation = self._generate_recommendation(overall)\n\n return CredibilityScore(\n overall_score=round(overall, 2),\n domain_authority=round(domain_score, 2),\n recency=round(recency_score, 2),\n expertise=round(expertise_score, 2),\n bias_score=round(bias_score, 2),\n factors=factors,\n recommendation=recommendation\n )\n\n def _extract_domain(self, url: str) -> str:\n \"\"\"Extract domain from URL\"\"\"\n parsed = urlparse(url)\n domain = parsed.netloc.lower()\n # Remove www prefix\n domain = domain.replace('www.', '')\n return domain\n\n def _evaluate_domain_authority(self, domain: str) -> float:\n \"\"\"Evaluate domain authority (0-100)\"\"\"\n if domain in self.HIGH_AUTHORITY_DOMAINS:\n return 90.0\n elif domain in self.MODERATE_AUTHORITY_DOMAINS:\n return 70.0\n elif any(indicator in domain for indicator in self.LOW_AUTHORITY_INDICATORS):\n return 40.0\n else:\n # Unknown domain - moderate skepticism\n return 55.0\n\n def _evaluate_recency(self, publication_date: Optional[str]) -> float:\n \"\"\"Evaluate information recency (0-100)\"\"\"\n if not publication_date:\n return 50.0 # Unknown date\n\n try:\n pub_date = datetime.fromisoformat(publication_date.replace('Z', '+00:00'))\n age = datetime.now() - pub_date\n\n # Recency scoring\n if age \u003c timedelta(days=90): # \u003c 3 months\n return 100.0\n elif age \u003c timedelta(days=365): # \u003c 1 year\n return 85.0\n elif age \u003c timedelta(days=730): # \u003c 2 years\n return 70.0\n elif age \u003c timedelta(days=1825): # \u003c 5 years\n return 50.0\n else:\n return 30.0\n\n except Exception:\n return 50.0\n\n def _evaluate_expertise(\n self,\n domain: str,\n title: str,\n author: Optional[str]\n ) -> float:\n \"\"\"Evaluate source expertise (0-100)\"\"\"\n score = 50.0\n\n # Academic/research domains get high expertise\n if any(d in domain for d in ['arxiv', 'nature', 'science', 'ieee', 'acm']):\n score += 30\n\n # Government/official sources\n if '.gov' in domain or 'who.int' in domain:\n score += 25\n\n # Technical documentation\n if 'docs.' in domain or 'documentation' in title.lower():\n score += 20\n\n # Author credentials (if available)\n if author:\n if any(title in author.lower() for title in ['dr.', 'phd', 'professor']):\n score += 15\n\n return min(score, 100.0)\n\n def _evaluate_bias(\n self,\n domain: str,\n title: str,\n content: Optional[str]\n ) -> float:\n \"\"\"Evaluate potential bias (0-100, higher = more neutral)\"\"\"\n score = 70.0 # Start neutral\n\n # Check for sensationalism in title\n sensational_indicators = [\n '!', 'shocking', 'unbelievable', 'you won\\'t believe',\n 'secret', 'they don\\'t want you to know'\n ]\n title_lower = title.lower()\n if any(indicator in title_lower for indicator in sensational_indicators):\n score -= 20\n\n # Academic sources are typically less biased\n if any(d in domain for d in ['arxiv', 'nature', 'science', 'ieee']):\n score += 20\n\n # Check for balance in content (if available)\n if content:\n # Look for balanced language\n balanced_indicators = ['however', 'although', 'on the other hand', 'critics argue']\n if any(indicator in content.lower() for indicator in balanced_indicators):\n score += 10\n\n return min(max(score, 0), 100.0)\n\n def _identify_factors(\n self,\n domain: str,\n domain_score: float,\n recency_score: float,\n expertise_score: float,\n bias_score: float\n ) -> Dict[str, str]:\n \"\"\"Identify key credibility factors\"\"\"\n factors = {}\n\n if domain_score >= 85:\n factors['domain'] = \"High authority domain\"\n elif domain_score \u003c= 45:\n factors['domain'] = \"Low authority domain - verify claims\"\n\n if recency_score >= 85:\n factors['recency'] = \"Recent information\"\n elif recency_score \u003c= 40:\n factors['recency'] = \"Outdated information - verify currency\"\n\n if expertise_score >= 80:\n factors['expertise'] = \"Expert source\"\n elif expertise_score \u003c= 45:\n factors['expertise'] = \"Limited expertise indicators\"\n\n if bias_score >= 80:\n factors['bias'] = \"Balanced perspective\"\n elif bias_score \u003c= 50:\n factors['bias'] = \"Potential bias detected\"\n\n return factors\n\n def _generate_recommendation(self, overall_score: float) -> str:\n \"\"\"Generate trust recommendation\"\"\"\n if overall_score >= 80:\n return \"high_trust\"\n elif overall_score >= 60:\n return \"moderate_trust\"\n elif overall_score >= 40:\n return \"low_trust\"\n else:\n return \"verify\"\n\n\n# Example usage\nif __name__ == '__main__':\n evaluator = SourceEvaluator()\n\n # Test sources\n test_sources = [\n {\n 'url': 'https://www.nature.com/articles/s41586-2025-12345',\n 'title': 'Breakthrough in Quantum Computing',\n 'publication_date': '2025-10-15'\n },\n {\n 'url': 'https://someblog.wordpress.com/shocking-discovery',\n 'title': 'SHOCKING! You Won\\'t Believe This Discovery!',\n 'publication_date': '2020-01-01'\n },\n {\n 'url': 'https://docs.python.org/3/library/asyncio.html',\n 'title': 'asyncio — Asynchronous I/O',\n 'publication_date': '2025-11-01'\n }\n ]\n\n for source in test_sources:\n score = evaluator.evaluate_source(**source)\n print(f\"\\nSource: {source['title']}\")\n print(f\"URL: {source['url']}\")\n print(f\"Overall Score: {score.overall_score}/100\")\n print(f\"Recommendation: {score.recommendation}\")\n print(f\"Factors: {score.factors}\")\n","content_type":"text/x-python; charset=utf-8","language":"python","size":9492,"content_sha256":"8421efa7b862bb139108e6da40b285706188cb272843bf39082c8b9888d5050e"},{"filename":"scripts/validate_report.py","content":"#!/usr/bin/env python3\n\"\"\"\nReport Validation Script\nEnsures research reports meet quality standards before delivery\n\"\"\"\n\nimport argparse\nimport re\nimport sys\nfrom pathlib import Path\nfrom typing import List, Tuple, Dict\n\n\nclass ReportValidator:\n \"\"\"Validates research report quality\"\"\"\n\n def __init__(self, report_path: Path):\n self.report_path = report_path\n self.content = self._read_report()\n self.errors: List[str] = []\n self.warnings: List[str] = []\n\n def _read_report(self) -> str:\n \"\"\"Read report file\"\"\"\n try:\n with open(self.report_path, 'r', encoding='utf-8') as f:\n return f.read()\n except Exception as e:\n print(f\"❌ ERROR: Cannot read report: {e}\")\n sys.exit(1)\n\n def validate(self) -> bool:\n \"\"\"Run all validation checks\"\"\"\n print(f\"\\n{'='*60}\")\n print(f\"VALIDATING REPORT: {self.report_path.name}\")\n print(f\"{'='*60}\\n\")\n\n checks = [\n (\"Executive Summary\", self._check_executive_summary),\n (\"Required Sections\", self._check_required_sections),\n (\"Citations\", self._check_citations),\n (\"Bibliography\", self._check_bibliography),\n (\"Placeholder Text\", self._check_placeholders),\n (\"Content Truncation\", self._check_content_truncation),\n (\"Word Count\", self._check_word_count),\n (\"Source Count\", self._check_source_count),\n (\"Broken Links\", self._check_broken_references),\n ]\n\n for check_name, check_func in checks:\n print(f\"⏳ Checking: {check_name}...\", end=\" \")\n passed = check_func()\n if passed:\n print(\"✅ PASS\")\n else:\n print(\"❌ FAIL\")\n\n self._print_summary()\n\n return len(self.errors) == 0\n\n def _check_executive_summary(self) -> bool:\n \"\"\"Check executive summary exists and is 200-400 words\"\"\"\n pattern = r'## Executive Summary(.*?)(?=##|\\Z)'\n match = re.search(pattern, self.content, re.DOTALL | re.IGNORECASE)\n\n if not match:\n self.errors.append(\"Missing 'Executive Summary' section\")\n return False\n\n summary = match.group(1).strip()\n word_count = len(summary.split())\n\n if word_count > 400:\n self.warnings.append(f\"Executive summary too long: {word_count} words (should be ≤400)\")\n\n if word_count \u003c 50:\n self.warnings.append(f\"Executive summary too short: {word_count} words (should be ≥50)\")\n\n return True\n\n def _check_required_sections(self) -> bool:\n \"\"\"Check all required sections are present\"\"\"\n required = [\n \"Executive Summary\",\n \"Introduction\",\n \"Main Analysis\",\n \"Synthesis\",\n \"Limitations\",\n \"Recommendations\",\n \"Bibliography\",\n \"Methodology\"\n ]\n\n # Recommended sections (warnings if missing, not errors)\n recommended = [\n \"Counterevidence Register\",\n \"Claims-Evidence Table\"\n ]\n\n missing = []\n for section in required:\n if not re.search(rf'##.*{section}', self.content, re.IGNORECASE):\n missing.append(section)\n\n if missing:\n self.errors.append(f\"Missing sections: {', '.join(missing)}\")\n return False\n\n # Check recommended sections (warnings only)\n missing_recommended = []\n for section in recommended:\n if not re.search(rf'##.*{section}', self.content, re.IGNORECASE):\n missing_recommended.append(section)\n\n if missing_recommended:\n self.warnings.append(f\"Missing recommended sections (for academic rigor): {', '.join(missing_recommended)}\")\n\n return True\n\n def _check_citations(self) -> bool:\n \"\"\"Check citation format and presence\"\"\"\n # Find all citation references [1], [2], etc.\n citations = re.findall(r'\\[(\\d+)\\]', self.content)\n\n if not citations:\n self.errors.append(\"No citations found in report\")\n return False\n\n unique_citations = set(citations)\n\n if len(unique_citations) \u003c 10:\n self.warnings.append(f\"Only {len(unique_citations)} unique sources cited (recommended: ≥10)\")\n\n # Check for consecutive citation numbers\n citation_nums = sorted([int(c) for c in unique_citations])\n if citation_nums:\n max_citation = max(citation_nums)\n expected = set(range(1, max_citation + 1))\n missing = expected - set(citation_nums)\n\n if missing:\n self.warnings.append(f\"Non-consecutive citation numbers, missing: {sorted(missing)}\")\n\n return True\n\n def _check_bibliography(self) -> bool:\n \"\"\"Check bibliography exists, matches citations, and has no truncation placeholders\"\"\"\n pattern = r'## Bibliography(.*?)(?=##|\\Z)'\n match = re.search(pattern, self.content, re.DOTALL | re.IGNORECASE)\n\n if not match:\n self.errors.append(\"Missing 'Bibliography' section\")\n return False\n\n bib_section = match.group(1)\n\n # CRITICAL: Check for truncation placeholders (2025 CiteGuard enhancement)\n truncation_patterns = [\n (r'\\[\\d+-\\d+\\]', 'Citation range (e.g., [8-75])'),\n (r'Additional.*citations', 'Phrase \"Additional citations\"'),\n (r'would be included', 'Phrase \"would be included\"'),\n (r'\\[\\.\\.\\.continue', 'Pattern \"[...continue\"'),\n (r'\\[Continue with', 'Pattern \"[Continue with\"'),\n (r'etc\\.(?!\\w)', 'Standalone \"etc.\"'),\n (r'and so on', 'Phrase \"and so on\"'),\n ]\n\n for pattern_re, description in truncation_patterns:\n if re.search(pattern_re, bib_section, re.IGNORECASE):\n self.errors.append(f\"⚠️ CRITICAL: Bibliography contains truncation placeholder: {description}\")\n self.errors.append(f\" This makes the report UNUSABLE - complete bibliography required\")\n return False\n\n # Count bibliography entries [1], [2], etc.\n bib_entries = re.findall(r'^\\[(\\d+)\\]', bib_section, re.MULTILINE)\n\n if not bib_entries:\n self.errors.append(\"Bibliography has no entries\")\n return False\n\n # Check citation number continuity (no gaps)\n bib_nums = sorted([int(n) for n in bib_entries])\n if bib_nums:\n expected = list(range(1, bib_nums[-1] + 1))\n actual = bib_nums\n missing = [n for n in expected if n not in actual]\n if missing:\n self.errors.append(f\"Bibliography has gaps in numbering: missing {missing}\")\n return False\n\n # Find citations in text\n text_citations = set(re.findall(r'\\[(\\d+)\\]', self.content))\n bib_citations = set(bib_entries)\n\n # Check all citations have bibliography entries\n missing_in_bib = text_citations - bib_citations\n if missing_in_bib:\n self.errors.append(f\"Citations missing from bibliography: {sorted(missing_in_bib)}\")\n return False\n\n # Check for unused bibliography entries\n unused = bib_citations - text_citations\n if unused:\n self.warnings.append(f\"Unused bibliography entries: {sorted(unused)}\")\n\n return True\n\n def _check_placeholders(self) -> bool:\n \"\"\"Check for placeholder text that shouldn't be in final report\"\"\"\n placeholders = [\n 'TBD', 'TODO', 'FIXME', 'XXX',\n '[citation needed]', '[needs citation]',\n '[placeholder]', '[TODO]', '[TBD]'\n ]\n\n found_placeholders = []\n for placeholder in placeholders:\n if placeholder in self.content:\n found_placeholders.append(placeholder)\n\n if found_placeholders:\n self.errors.append(f\"Found placeholder text: {', '.join(found_placeholders)}\")\n return False\n\n return True\n\n def _check_content_truncation(self) -> bool:\n \"\"\"Check for content truncation patterns (2025 Progressive Assembly enhancement)\"\"\"\n truncation_patterns = [\n (r'Content continues', 'Phrase \"Content continues\"'),\n (r'Due to length', 'Phrase \"Due to length\"'),\n (r'would continue', 'Phrase \"would continue\"'),\n (r'\\[Sections \\d+-\\d+', 'Pattern \"[Sections X-Y\"'),\n (r'Additional sections', 'Phrase \"Additional sections\"'),\n (r'comprehensive.*word document that continues', 'Pattern \"comprehensive...document that continues\"'),\n ]\n\n for pattern_re, description in truncation_patterns:\n if re.search(pattern_re, self.content, re.IGNORECASE):\n self.errors.append(f\"⚠️ CRITICAL: Content truncation detected: {description}\")\n self.errors.append(f\" Report is INCOMPLETE and UNUSABLE - regenerate with progressive assembly\")\n return False\n\n return True\n\n def _check_word_count(self) -> bool:\n \"\"\"Check overall report length\"\"\"\n word_count = len(self.content.split())\n\n if word_count \u003c 500:\n self.warnings.append(f\"Report is very short: {word_count} words (consider expanding)\")\n # No upper limit warning - progressive assembly supports unlimited lengths\n\n return True\n\n def _check_source_count(self) -> bool:\n \"\"\"Check minimum source count\"\"\"\n pattern = r'## Bibliography(.*?)(?=##|\\Z)'\n match = re.search(pattern, self.content, re.DOTALL | re.IGNORECASE)\n\n if not match:\n return True # Already caught in bibliography check\n\n bib_section = match.group(1)\n bib_entries = re.findall(r'^\\[(\\d+)\\]', bib_section, re.MULTILINE)\n\n source_count = len(set(bib_entries))\n\n if source_count \u003c 10:\n self.warnings.append(f\"Only {source_count} sources (recommended: ≥10)\")\n\n return True\n\n def _check_broken_references(self) -> bool:\n \"\"\"Check for broken internal references\"\"\"\n # Find all markdown links [text](./path)\n internal_links = re.findall(r'\\[.*?\\]\\((\\.\\/.*?)\\)', self.content)\n\n broken = []\n for link in internal_links:\n # Remove anchor if present\n link_path = link.split('#')[0]\n full_path = self.report_path.parent / link_path\n\n if not full_path.exists():\n broken.append(link)\n\n if broken:\n self.errors.append(f\"Broken internal links: {', '.join(broken)}\")\n return False\n\n return True\n\n def _print_summary(self):\n \"\"\"Print validation summary\"\"\"\n print(f\"\\n{'='*60}\")\n print(f\"VALIDATION SUMMARY\")\n print(f\"{'='*60}\\n\")\n\n if self.errors:\n print(f\"❌ ERRORS ({len(self.errors)}):\")\n for error in self.errors:\n print(f\" • {error}\")\n print()\n\n if self.warnings:\n print(f\"⚠️ WARNINGS ({len(self.warnings)}):\")\n for warning in self.warnings:\n print(f\" • {warning}\")\n print()\n\n if not self.errors and not self.warnings:\n print(\"✅ ALL CHECKS PASSED - Report meets quality standards!\\n\")\n elif not self.errors:\n print(\"✅ VALIDATION PASSED (with warnings)\\n\")\n else:\n print(\"❌ VALIDATION FAILED - Please fix errors before delivery\\n\")\n\n\ndef main():\n parser = argparse.ArgumentParser(\n description=\"Validate research report quality\",\n formatter_class=argparse.RawDescriptionHelpFormatter,\n epilog=\"\"\"\nExamples:\n python validate_report.py --report report.md\n python validate_report.py -r ~/.claude/research_output/research_report_20251104_153045.md\n \"\"\"\n )\n\n parser.add_argument(\n '--report', '-r',\n type=str,\n required=True,\n help='Path to research report markdown file'\n )\n\n args = parser.parse_args()\n\n report_path = Path(args.report)\n\n if not report_path.exists():\n print(f\"❌ ERROR: Report file not found: {report_path}\")\n sys.exit(1)\n\n validator = ReportValidator(report_path)\n passed = validator.validate()\n\n sys.exit(0 if passed else 1)\n\n\nif __name__ == '__main__':\n main()\n","content_type":"text/x-python; charset=utf-8","language":"python","size":12406,"content_sha256":"851c1de275c039b8e5858785010181500960b05a9be463cdd6cecd5bbf101aa3"},{"filename":"scripts/verify_citations.py","content":"#!/usr/bin/env python3\n\"\"\"\nCitation Verification Script\n\nCatches fabricated citations by checking:\n1. DOI resolution (via doi.org)\n2. Basic metadata matching (title similarity, year match)\n3. URL accessibility verification\n4. Hallucination pattern detection (generic titles, suspicious patterns)\n5. Flags suspicious entries for manual review\n\nUsage:\n python verify_citations.py --report [path]\n python verify_citations.py --report [path] --strict # Fail on any unverified\n\nDoes NOT require API keys - uses free DOI resolver and heuristics.\n\"\"\"\n\nimport sys\nimport argparse\nimport re\nfrom pathlib import Path\nfrom typing import List, Dict, Tuple\nfrom urllib import request, error\nfrom urllib.parse import quote\nimport json\nimport time\nfrom datetime import datetime\n\nclass CitationVerifier:\n \"\"\"Verify citations in research report\"\"\"\n\n def __init__(self, report_path: Path, strict_mode: bool = False):\n self.report_path = report_path\n self.strict_mode = strict_mode\n self.content = self._read_report()\n self.suspicious = []\n self.verified = []\n self.errors = []\n\n # Hallucination detection patterns (2025 CiteGuard enhancement)\n self.suspicious_patterns = [\n # Generic academic-sounding but fake patterns\n (r'^(A |An |The )?(Study|Analysis|Review|Survey|Investigation) (of|on|into)',\n \"Generic academic title pattern\"),\n (r'^(Recent|Current|Modern|Contemporary) (Advances|Developments|Trends) in',\n \"Generic 'advances' title pattern\"),\n # Too perfect, templated titles\n (r'^[A-Z][a-z]+ [A-Z][a-z]+: A (Comprehensive|Complete|Systematic) (Review|Analysis|Guide)

Deep Research Core Purpose Deliver citation-tracked research reports through a structured pipeline with evidence persistence, source identity management, claim-level verification, and progressive context management. Autonomy Principle: Operate independently. Infer assumptions from context. Only stop for critical errors or incomprehensible queries. Surface high-materiality assumptions explicitly in the Introduction and Methodology rather than silently defaulting. --- Decision Tree Default assumptions: Technical query = technical audience. Comparison = balanced perspective. Trend = recent 1-2 y…

,\n \"Too perfect, templated structure\"),\n ]\n\n def _read_report(self) -> str:\n \"\"\"Read report file\"\"\"\n try:\n with open(self.report_path, 'r', encoding='utf-8') as f:\n return f.read()\n except Exception as e:\n print(f\"L ERROR: Cannot read report: {e}\")\n sys.exit(1)\n\n def extract_bibliography(self) -> List[Dict]:\n \"\"\"Extract bibliography entries from report\"\"\"\n pattern = r'## Bibliography(.*?)(?=##|\\Z)'\n match = re.search(pattern, self.content, re.DOTALL | re.IGNORECASE)\n\n if not match:\n self.errors.append(\"No Bibliography section found\")\n return []\n\n bib_section = match.group(1)\n\n # Parse entries: [N] Author (Year). \"Title\". Venue. URL\n entries = []\n lines = bib_section.strip().split('\\n')\n\n current_entry = None\n for line in lines:\n line = line.strip()\n if not line:\n continue\n\n # Check if starts with citation number [N]\n match_num = re.match(r'^\\[(\\d+)\\]\\s+(.+)

Deep Research Core Purpose Deliver citation-tracked research reports through a structured pipeline with evidence persistence, source identity management, claim-level verification, and progressive context management. Autonomy Principle: Operate independently. Infer assumptions from context. Only stop for critical errors or incomprehensible queries. Surface high-materiality assumptions explicitly in the Introduction and Methodology rather than silently defaulting. --- Decision Tree Default assumptions: Technical query = technical audience. Comparison = balanced perspective. Trend = recent 1-2 y…

, line)\n if match_num:\n if current_entry:\n entries.append(current_entry)\n\n num = match_num.group(1)\n rest = match_num.group(2)\n\n # Try to parse: Author (Year). \"Title\". Venue. URL\n year_match = re.search(r'\\((\\d{4})\\)', rest)\n title_match = re.search(r'\"([^\"]+)\"', rest)\n doi_match = re.search(r'doi\\.org/(10\\.\\S+)', rest)\n url_match = re.search(r'https?://[^\\s\\)]+', rest)\n\n current_entry = {\n 'num': num,\n 'raw': rest,\n 'year': year_match.group(1) if year_match else None,\n 'title': title_match.group(1) if title_match else None,\n 'doi': doi_match.group(1) if doi_match else None,\n 'url': url_match.group(0) if url_match else None\n }\n elif current_entry:\n # Multi-line entry, append to raw\n current_entry['raw'] += ' ' + line\n\n if current_entry:\n entries.append(current_entry)\n\n return entries\n\n def verify_doi(self, doi: str) -> Tuple[bool, Dict]:\n \"\"\"\n Verify DOI exists and get metadata.\n Returns (success, metadata_dict)\n \"\"\"\n if not doi:\n return False, {}\n\n try:\n # Use content negotiation to get JSON metadata\n url = f\"https://doi.org/{quote(doi)}\"\n req = request.Request(url)\n req.add_header('Accept', 'application/vnd.citationstyles.csl+json')\n\n with request.urlopen(req, timeout=10) as response:\n data = json.loads(response.read().decode('utf-8'))\n\n return True, {\n 'title': data.get('title', ''),\n 'year': data.get('issued', {}).get('date-parts', [[None]])[0][0],\n 'authors': [\n f\"{a.get('family', '')} {a.get('given', '')}\"\n for a in data.get('author', [])\n ],\n 'venue': data.get('container-title', '')\n }\n except error.HTTPError as e:\n if e.code == 404:\n return False, {'error': 'DOI not found (404)'}\n return False, {'error': f'HTTP {e.code}'}\n except Exception as e:\n return False, {'error': str(e)}\n\n def verify_url(self, url: str) -> Tuple[bool, str]:\n \"\"\"\n Verify URL is accessible (2025 CiteGuard enhancement).\n Returns (accessible, status_message)\n \"\"\"\n if not url:\n return False, \"No URL\"\n\n try:\n # HEAD request to check accessibility without downloading\n req = request.Request(url, method='HEAD')\n req.add_header('User-Agent', 'Mozilla/5.0 (Research Citation Verifier)')\n\n with request.urlopen(req, timeout=10) as response:\n if response.status == 200:\n return True, \"URL accessible\"\n else:\n return False, f\"HTTP {response.status}\"\n except error.HTTPError as e:\n return False, f\"HTTP {e.code}\"\n except error.URLError as e:\n return False, f\"URL error: {e.reason}\"\n except Exception as e:\n return False, f\"Connection error: {str(e)[:50]}\"\n\n def detect_hallucination_patterns(self, entry: Dict) -> List[str]:\n \"\"\"\n Detect common LLM hallucination patterns in citations (2025 CiteGuard).\n Returns list of detected issues.\n \"\"\"\n issues = []\n title = entry.get('title', '')\n\n if not title:\n return issues\n\n # Check against suspicious patterns\n for pattern, description in self.suspicious_patterns:\n if re.match(pattern, title, re.IGNORECASE):\n issues.append(f\"Suspicious title pattern: {description}\")\n\n # Check for overly generic titles\n generic_words = ['overview', 'introduction', 'guide', 'handbook', 'manual']\n if any(word in title.lower() for word in generic_words) and len(title.split()) \u003c 5:\n issues.append(\"Very generic short title\")\n\n # Check for placeholder-like titles\n if any(x in title.lower() for x in ['tbd', 'todo', 'placeholder', 'example']):\n issues.append(\"Placeholder text in title\")\n\n # Check for inconsistent metadata\n if entry.get('year'):\n year = int(entry['year'])\n current_year = datetime.now().year\n # Very recent without DOI or URL is suspicious\n if year >= current_year - 1 and not entry.get('doi') and not entry.get('url'):\n issues.append(f\"Recent year ({year}) with no verification method\")\n # Future year is definitely wrong\n if year > current_year:\n issues.append(f\"Future year: {year} (current: {current_year})\")\n # Very old with modern phrasing is suspicious\n if year \u003c 2000 and any(word in title.lower() for word in ['ai', 'llm', 'gpt', 'transformer']):\n issues.append(f\"Anachronistic: pre-2000 ({year}) citation mentioning modern AI terms\")\n\n return issues\n\n def check_title_similarity(self, title1: str, title2: str) -> float:\n \"\"\"\n Simple title similarity check (word overlap).\n Returns score 0.0-1.0\n \"\"\"\n if not title1 or not title2:\n return 0.0\n\n # Normalize: lowercase, remove punctuation, split\n def normalize(s):\n s = s.lower()\n s = re.sub(r'[^\\w\\s]', ' ', s)\n return set(s.split())\n\n words1 = normalize(title1)\n words2 = normalize(title2)\n\n if not words1 or not words2:\n return 0.0\n\n overlap = len(words1 & words2)\n total = len(words1 | words2)\n\n return overlap / total if total > 0 else 0.0\n\n def verify_entry(self, entry: Dict) -> Dict:\n \"\"\"Verify a single bibliography entry (Enhanced 2025 with CiteGuard)\"\"\"\n result = {\n 'num': entry['num'],\n 'status': 'unknown',\n 'issues': [],\n 'metadata': {},\n 'verification_methods': []\n }\n\n # STEP 1: Run hallucination detection (CiteGuard 2025)\n hallucination_issues = self.detect_hallucination_patterns(entry)\n if hallucination_issues:\n result['issues'].extend(hallucination_issues)\n result['status'] = 'suspicious'\n\n # STEP 2: Has DOI?\n if entry['doi']:\n print(f\" [{entry['num']}] Checking DOI {entry['doi']}...\", end=' ')\n success, metadata = self.verify_doi(entry['doi'])\n\n if success:\n result['metadata'] = metadata\n result['status'] = 'verified'\n print(\"\")\n\n # Check title similarity if we have both\n if entry['title'] and metadata.get('title'):\n similarity = self.check_title_similarity(\n entry['title'],\n metadata['title']\n )\n\n if similarity \u003c 0.5:\n result['issues'].append(\n f\"Title mismatch (similarity: {similarity:.1%})\"\n )\n result['status'] = 'suspicious'\n\n # Check year match\n if entry['year'] and metadata.get('year'):\n if int(entry['year']) != int(metadata['year']):\n result['issues'].append(\n f\"Year mismatch: report says {entry['year']}, DOI says {metadata['year']}\"\n )\n result['status'] = 'suspicious'\n\n else:\n print(f\"✗ {metadata.get('error', 'Failed')}\")\n result['status'] = 'unverified'\n result['issues'].append(f\"DOI resolution failed: {metadata.get('error', 'unknown')}\")\n\n # STEP 3: Check URL accessibility (if no DOI or DOI failed)\n if entry['url'] and result['status'] != 'verified':\n url_ok, url_status = self.verify_url(entry['url'])\n if url_ok:\n result['verification_methods'].append('URL')\n # Upgrade status if URL verifies\n if result['status'] in ['unknown', 'no_doi', 'unverified']:\n result['status'] = 'url_verified'\n print(f\" [{entry['num']}] URL accessible ✓\")\n else:\n result['issues'].append(f\"URL check failed: {url_status}\")\n\n # STEP 4: Final fallback - no verification method\n if not entry['doi'] and not entry['url']:\n if 'No DOI provided' not in ' '.join(result['issues']):\n result['issues'].append(\"No DOI or URL - cannot verify\")\n result['status'] = 'suspicious'\n\n return result\n\n def verify_all(self):\n \"\"\"Verify all bibliography entries\"\"\"\n print(f\"\\n{'='*60}\")\n print(f\"CITATION VERIFICATION: {self.report_path.name}\")\n print(f\"{'='*60}\\n\")\n\n entries = self.extract_bibliography()\n\n if not entries:\n print(\"L No bibliography entries found\\n\")\n return False\n\n print(f\"Found {len(entries)} citations\\n\")\n\n results = []\n for entry in entries:\n result = self.verify_entry(entry)\n results.append(result)\n\n # Rate limiting\n time.sleep(0.5)\n\n # Summarize\n print(f\"\\n{'='*60}\")\n print(f\"VERIFICATION SUMMARY\")\n print(f\"{'='*60}\\n\")\n\n verified = [r for r in results if r['status'] == 'verified']\n url_verified = [r for r in results if r['status'] == 'url_verified']\n suspicious = [r for r in results if r['status'] == 'suspicious']\n unverified = [r for r in results if r['status'] in ['unverified', 'no_doi', 'unknown']]\n\n print(f'DOI Verified: {len(verified)}/{len(results)}')\n print(f'URL Verified: {len(url_verified)}/{len(results)}')\n print(f'Suspicious: {len(suspicious)}/{len(results)}')\n print(f'Unverified: {len(unverified)}/{len(results)}')\n print()\n\n if suspicious:\n print('SUSPICIOUS CITATIONS (Manual Review Needed):')\n for r in suspicious:\n print(f\"\\n [{r['num']}]\")\n for issue in r['issues']:\n print(f\" - {issue}\")\n print()\n\n if unverified and len(unverified) > 0:\n print('UNVERIFIED CITATIONS (Could not check):')\n for r in unverified:\n print(f\" [{r['num']}] {r['issues'][0] if r['issues'] else 'Unknown'}\")\n print()\n\n # Decision (Enhanced 2025 - includes URL-verified as acceptable)\n total_verified = len(verified) + len(url_verified)\n\n if suspicious:\n print('WARNING: Suspicious citations detected')\n if self.strict_mode:\n print(' STRICT MODE: Failing due to suspicious citations')\n return False\n else:\n print(' (Continuing in non-strict mode)')\n\n if self.strict_mode and unverified:\n print('STRICT MODE: Unverified citations found')\n return False\n\n if total_verified / len(results) \u003c 0.5:\n print('WARNING: Less than 50% citations verified')\n return True # Pass with warning\n else:\n print('CITATION VERIFICATION PASSED')\n return True\n\n\ndef main():\n parser = argparse.ArgumentParser(\n description=\"Verify citations in research report\",\n formatter_class=argparse.RawDescriptionHelpFormatter,\n epilog=\"\"\"\nExamples:\n python verify_citations.py --report report.md\n\nNote: Requires internet connection to check DOIs.\nUses free DOI resolver - no API key needed.\n \"\"\"\n )\n\n parser.add_argument(\n '--report', '-r',\n type=str,\n required=True,\n help='Path to research report markdown file'\n )\n\n parser.add_argument(\n '--strict',\n action='store_true',\n help='Strict mode: fail on any unverified or suspicious citations'\n )\n\n args = parser.parse_args()\n report_path = Path(args.report)\n\n if not report_path.exists():\n print(f\"ERROR: Report file not found: {report_path}\")\n sys.exit(1)\n\n verifier = CitationVerifier(report_path, strict_mode=args.strict)\n passed = verifier.verify_all()\n\n sys.exit(0 if passed else 1)\n\n\nif __name__ == '__main__':\n main()\n","content_type":"text/x-python; charset=utf-8","language":"python","size":15467,"content_sha256":"83b3eab573e7b30eb8200bb74e834f454762150ed3d4f13196d5b85a459553f2"},{"filename":"scripts/verify_claim_support.py","content":"#!/usr/bin/env python3\n\"\"\"\nClaim-Support Verification — checks whether evidence supports claims.\n\nCLI subcommands:\n verify Check all claims against evidence, update support_status\n report Generate a support verification summary\n\nVersion 1 is deterministic and cheap: entity, number, date, and\nlexical-overlap checks over stored evidence. No LLM calls.\n\nOnly factual claims hard-fail on unsupported status.\nSynthesis/recommendation need traceability but softer thresholds.\n\"\"\"\n\nimport argparse\nimport json\nimport os\nimport re\nimport sys\nfrom collections import Counter\nfrom datetime import datetime, timezone\n\n\n# ---------------------------------------------------------------------------\n# JSONL helpers\n# ---------------------------------------------------------------------------\n\ndef read_jsonl(path: str) -> list[dict]:\n rows = []\n if not os.path.exists(path):\n return rows\n with open(path) as f:\n for line in f:\n line = line.strip()\n if line:\n rows.append(json.loads(line))\n return rows\n\n\ndef write_jsonl(path: str, rows: list[dict]) -> None:\n with open(path, 'w') as f:\n for row in rows:\n f.write(json.dumps(row, ensure_ascii=False) + '\\n')\n\n\n# ---------------------------------------------------------------------------\n# Support verification logic\n# ---------------------------------------------------------------------------\n\n# Extract numbers (integers and decimals)\nNUMBER_RE = re.compile(r'\\b\\d+(?:\\.\\d+)?(?:%|x|X)?\\b')\n\n# Extract year-like numbers\nYEAR_RE = re.compile(r'\\b(19|20)\\d{2}\\b')\n\n# Extract capitalized entities (naive NER)\nENTITY_RE = re.compile(r'\\b[A-Z][a-z]+(?:\\s+[A-Z][a-z]+)*\\b')\n\n# Common stop entities to ignore\nSTOP_ENTITIES = frozenset([\n 'The', 'This', 'That', 'These', 'However', 'Furthermore',\n 'Moreover', 'Additionally', 'Therefore', 'Nevertheless',\n])\n\n\ndef extract_tokens(text: str) -> set[str]:\n \"\"\"Extract significant lowercase tokens (>3 chars).\"\"\"\n words = re.findall(r'\\b[a-z]{4,}\\b', text.lower())\n return set(words)\n\n\ndef extract_numbers(text: str) -> set[str]:\n \"\"\"Extract numeric values.\"\"\"\n return set(NUMBER_RE.findall(text))\n\n\ndef extract_years(text: str) -> set[str]:\n \"\"\"Extract year mentions.\"\"\"\n return set(YEAR_RE.findall(text))\n\n\ndef extract_entities(text: str) -> set[str]:\n \"\"\"Extract capitalized entity mentions.\"\"\"\n ents = set(ENTITY_RE.findall(text))\n return ents - STOP_ENTITIES\n\n\ndef compute_support_score(claim_text: str, evidence_quotes: list[str]) -> tuple[str, float, str]:\n \"\"\"\n Compute support status for a claim given its linked evidence quotes.\n\n Returns (status, score, notes).\n Score range: 0.0 (no overlap) to 1.0 (strong support).\n \"\"\"\n if not evidence_quotes:\n return ('unsupported', 0.0, 'no evidence linked')\n\n claim_tokens = extract_tokens(claim_text)\n claim_numbers = extract_numbers(claim_text)\n claim_years = extract_years(claim_text)\n claim_entities = extract_entities(claim_text)\n\n best_score = 0.0\n best_notes = []\n\n for quote in evidence_quotes:\n ev_tokens = extract_tokens(quote)\n ev_numbers = extract_numbers(quote)\n ev_years = extract_years(quote)\n ev_entities = extract_entities(quote)\n\n # Token overlap (Jaccard-like)\n if claim_tokens:\n token_overlap = len(claim_tokens & ev_tokens) / len(claim_tokens)\n else:\n token_overlap = 0.0\n\n # Number match\n if claim_numbers:\n number_match = len(claim_numbers & ev_numbers) / len(claim_numbers)\n else:\n number_match = 1.0 # No numbers to check\n\n # Year match\n if claim_years:\n year_match = len(claim_years & ev_years) / len(claim_years)\n else:\n year_match = 1.0\n\n # Entity match\n if claim_entities:\n entity_match = len(claim_entities & ev_entities) / len(claim_entities)\n else:\n entity_match = 1.0\n\n # Weighted composite\n score = (\n 0.4 * token_overlap +\n 0.25 * number_match +\n 0.15 * year_match +\n 0.2 * entity_match\n )\n\n if score > best_score:\n best_score = score\n best_notes = []\n if token_overlap \u003c 0.3:\n best_notes.append('low lexical overlap')\n if claim_numbers and number_match \u003c 0.5:\n best_notes.append('number mismatch')\n if claim_years and year_match \u003c 1.0:\n best_notes.append('year mismatch')\n if claim_entities and entity_match \u003c 0.3:\n best_notes.append('entity mismatch')\n\n # Threshold decision\n if best_score >= 0.6:\n status = 'supported'\n elif best_score >= 0.35:\n status = 'partial'\n else:\n status = 'needs_review'\n\n notes = '; '.join(best_notes) if best_notes else 'adequate overlap'\n return (status, round(best_score, 3), notes)\n\n\n# ---------------------------------------------------------------------------\n# Subcommands\n# ---------------------------------------------------------------------------\n\ndef cmd_verify(args: argparse.Namespace) -> None:\n \"\"\"Verify all claims against evidence, update claims.jsonl.\"\"\"\n claims_path = os.path.join(args.dir, 'claims.jsonl')\n evidence_path = os.path.join(args.dir, 'evidence.jsonl')\n sources_path = os.path.join(args.dir, 'sources.jsonl')\n\n claims = read_jsonl(claims_path)\n evidence = read_jsonl(evidence_path)\n sources = read_jsonl(sources_path)\n\n # Build evidence index by source_id\n ev_by_source: dict[str, list[str]] = {}\n ev_by_id: dict[str, dict] = {}\n for ev in evidence:\n sid = ev.get('source_id', '')\n eid = ev.get('evidence_id', '')\n ev_by_source.setdefault(sid, []).append(ev.get('quote', ''))\n ev_by_id[eid] = ev\n\n # Deduplicate claims\n seen = set()\n unique_claims = []\n for c in claims:\n cid = c.get('claim_id')\n if cid not in seen:\n seen.add(cid)\n unique_claims.append(c)\n\n verified = 0\n updated_claims = []\n\n for claim in unique_claims:\n claim_type = claim.get('claim_type', 'factual')\n\n # Gather evidence for this claim\n cited_ids = claim.get('cited_source_ids', [])\n evidence_ids = claim.get('evidence_ids', [])\n\n # Collect evidence quotes from linked evidence_ids\n quotes = []\n for eid in evidence_ids:\n if eid in ev_by_id:\n quotes.append(ev_by_id[eid].get('quote', ''))\n\n # Also gather from cited sources\n for sid in cited_ids:\n if sid in ev_by_source:\n quotes.extend(ev_by_source[sid])\n\n if not quotes and not cited_ids and not evidence_ids:\n # No links at all\n if claim_type == 'speculation':\n claim['support_status'] = 'supported' # Speculation doesn't need evidence\n else:\n claim['support_status'] = 'unsupported'\n elif not quotes:\n # Has cited sources but no evidence captured yet\n claim['support_status'] = 'needs_review'\n else:\n status, score, notes = compute_support_score(claim['text'], quotes)\n claim['support_status'] = status\n claim['_support_score'] = score\n claim['_support_notes'] = notes\n\n verified += 1\n updated_claims.append(claim)\n\n # Rewrite claims.jsonl with updated statuses\n write_jsonl(claims_path, updated_claims)\n\n # Compute summary\n status_counts = Counter(c.get('support_status') for c in updated_claims)\n factual_unsupported = sum(\n 1 for c in updated_claims\n if c.get('claim_type') == 'factual' and c.get('support_status') == 'unsupported'\n )\n total_factual = sum(1 for c in updated_claims if c.get('claim_type') == 'factual')\n\n # Strict mode: fail if any factual claim is unsupported\n passed = True\n if args.strict and factual_unsupported > 0:\n passed = False\n\n print(json.dumps({\n 'status': 'pass' if passed else 'fail',\n 'verified': verified,\n 'support_status_counts': dict(status_counts),\n 'factual_unsupported': factual_unsupported,\n 'total_factual': total_factual,\n 'unsupported_rate': round(factual_unsupported / max(total_factual, 1), 3),\n }, indent=2))\n\n if not passed:\n sys.exit(1)\n\n\ndef cmd_report(args: argparse.Namespace) -> None:\n \"\"\"Generate human-readable support verification report.\"\"\"\n claims_path = os.path.join(args.dir, 'claims.jsonl')\n claims = read_jsonl(claims_path)\n\n # Deduplicate\n seen = set()\n unique = []\n for c in claims:\n cid = c.get('claim_id')\n if cid not in seen:\n seen.add(cid)\n unique.append(c)\n\n lines = ['# Claim Support Verification Report', '']\n\n # Summary\n status_counts = Counter(c.get('support_status') for c in unique)\n type_counts = Counter(c.get('claim_type') for c in unique)\n lines.append(f'**Total claims:** {len(unique)}')\n lines.append(f'**By type:** {dict(type_counts)}')\n lines.append(f'**By status:** {dict(status_counts)}')\n lines.append('')\n\n # Unsupported factual claims (the failures)\n unsupported_factual = [\n c for c in unique\n if c.get('claim_type') == 'factual' and c.get('support_status') in ('unsupported', 'needs_review')\n ]\n if unsupported_factual:\n lines.append('## Unsupported/Review-needed Factual Claims')\n lines.append('')\n for c in unsupported_factual:\n lines.append(f'- [{c[\"support_status\"]}] `{c[\"section_id\"]}`: {c[\"text\"][:100]}...')\n if c.get('_support_notes'):\n lines.append(f' Notes: {c[\"_support_notes\"]}')\n lines.append('')\n\n # All clear\n if not unsupported_factual:\n lines.append('## All factual claims have adequate support.')\n lines.append('')\n\n print('\\n'.join(lines))\n\n\n# ---------------------------------------------------------------------------\n# CLI entry point\n# ---------------------------------------------------------------------------\n\ndef main() -> None:\n parser = argparse.ArgumentParser(\n prog='verify_claim_support',\n description='Claim-support verification for deep-research v3.0',\n )\n sub = parser.add_subparsers(dest='command', required=True)\n\n # verify\n p_ver = sub.add_parser('verify', help='Verify claims against evidence')\n p_ver.add_argument('--dir', required=True, help='Run directory')\n p_ver.add_argument('--strict', action='store_true', help='Exit 1 if any factual claim unsupported')\n\n # report\n p_rep = sub.add_parser('report', help='Generate verification report')\n p_rep.add_argument('--dir', required=True, help='Run directory')\n\n args = parser.parse_args()\n dispatch = {\n 'verify': cmd_verify,\n 'report': cmd_report,\n }\n dispatch[args.command](args)\n\n\nif __name__ == '__main__':\n main()\n","content_type":"text/x-python; charset=utf-8","language":"python","size":11049,"content_sha256":"b172e88a3844e1ba590a7fc9a9d141ef310f5e628231e77448e9c45a7d881bc8"},{"filename":"scripts/verify_html.py","content":"#!/usr/bin/env python3\n\"\"\"\nHTML Report Verification Script\nValidates that HTML reports are properly generated with all sections from MD\n\"\"\"\n\nimport argparse\nimport re\nfrom pathlib import Path\nfrom typing import List, Tuple\n\n\nclass HTMLVerifier:\n \"\"\"Verify HTML research reports\"\"\"\n\n def __init__(self, html_path: Path, md_path: Path):\n self.html_path = html_path\n self.md_path = md_path\n self.errors = []\n self.warnings = []\n\n def verify(self) -> bool:\n \"\"\"\n Run all verification checks\n\n Returns:\n True if all checks pass, False otherwise\n \"\"\"\n print(f\"\\n{'='*60}\")\n print(f\"HTML REPORT VERIFICATION\")\n print(f\"{'='*60}\\n\")\n\n print(f\"HTML File: {self.html_path}\")\n print(f\"MD File: {self.md_path}\\n\")\n\n # Read files\n try:\n html_content = self.html_path.read_text()\n md_content = self.md_path.read_text()\n except Exception as e:\n self.errors.append(f\"Failed to read files: {e}\")\n return False\n\n # Run checks\n self._check_sections(html_content, md_content)\n self._check_no_placeholders(html_content)\n self._check_no_emojis(html_content)\n self._check_structure(html_content)\n self._check_citations(html_content, md_content)\n self._check_bibliography(html_content, md_content)\n\n # Report results\n self._print_results()\n\n return len(self.errors) == 0\n\n def _check_sections(self, html: str, md: str):\n \"\"\"Verify all markdown sections are present in HTML\"\"\"\n # Extract section headings from markdown\n md_sections = re.findall(r'^## (.+)

Deep Research Core Purpose Deliver citation-tracked research reports through a structured pipeline with evidence persistence, source identity management, claim-level verification, and progressive context management. Autonomy Principle: Operate independently. Infer assumptions from context. Only stop for critical errors or incomprehensible queries. Surface high-materiality assumptions explicitly in the Introduction and Methodology rather than silently defaulting. --- Decision Tree Default assumptions: Technical query = technical audience. Comparison = balanced perspective. Trend = recent 1-2 y…

, md, re.MULTILINE)\n\n # Extract sections from HTML\n html_sections = re.findall(r'\u003ch2 class=\"section-title\">(.+?)\u003c/h2>', html)\n\n # Check if we have placeholder sections like \u003cdiv class=\"section\">#\u003c/div>\n placeholder_sections = re.findall(r'\u003cdiv class=\"section\">#\u003c/div>', html)\n\n if placeholder_sections:\n self.errors.append(\n f\"Found {len(placeholder_sections)} placeholder sections (empty '#' divs) - content not converted properly\"\n )\n\n # Compare section counts\n if len(md_sections) > len(html_sections) + 1: # +1 for bibliography which is separate\n self.errors.append(\n f\"Section count mismatch: MD has {len(md_sections)} sections, HTML has only {len(html_sections)} + bibliography\"\n )\n missing = set(md_sections) - set(html_sections)\n if missing:\n self.errors.append(f\"Missing sections in HTML: {missing}\")\n\n # Verify Executive Summary is present\n if \"Executive Summary\" in md and \"Executive Summary\" not in html:\n self.errors.append(\"Executive Summary missing from HTML\")\n\n def _check_no_placeholders(self, html: str):\n \"\"\"Check for common placeholders that shouldn't be in final report\"\"\"\n placeholders = [\n '{{TITLE}}', '{{DATE}}', '{{CONTENT}}', '{{BIBLIOGRAPHY}}',\n '{{METRICS_DASHBOARD}}', '{{SOURCE_COUNT}}', 'TODO', 'TBD',\n 'PLACEHOLDER', 'FIXME'\n ]\n\n found = []\n for placeholder in placeholders:\n if placeholder in html:\n found.append(placeholder)\n\n if found:\n self.errors.append(f\"Found unreplaced placeholders: {', '.join(found)}\")\n\n def _check_no_emojis(self, html: str):\n \"\"\"Verify no emojis are present in HTML\"\"\"\n # Common emoji patterns\n emoji_pattern = re.compile(\n \"[\"\n \"\\U0001F600-\\U0001F64F\" # emoticons\n \"\\U0001F300-\\U0001F5FF\" # symbols & pictographs\n \"\\U0001F680-\\U0001F6FF\" # transport & map symbols\n \"\\U0001F1E0-\\U0001F1FF\" # flags\n \"\\U00002702-\\U000027B0\"\n \"\\U000024C2-\\U0001F251\"\n \"]+\",\n flags=re.UNICODE\n )\n\n emojis = emoji_pattern.findall(html)\n if emojis:\n unique_emojis = set(emojis)\n self.errors.append(f\"Found {len(emojis)} emojis in HTML (should be none): {unique_emojis}\")\n\n def _check_structure(self, html: str):\n \"\"\"Verify HTML has proper structure\"\"\"\n required_elements = [\n ('\u003chtml', 'HTML tag'),\n ('\u003chead', 'head tag'),\n ('\u003cbody', 'body tag'),\n ('\u003ctitle>', 'title tag'),\n ('class=\"header\"', 'header section'),\n ('class=\"content\"', 'content section'),\n ('class=\"bibliography\"', 'bibliography section'),\n ]\n\n for element, name in required_elements:\n if element not in html:\n self.errors.append(f\"Missing {name} in HTML\")\n\n # Check for unclosed tags (basic check)\n open_divs = html.count('\u003cdiv')\n close_divs = html.count('\u003c/div>')\n\n if abs(open_divs - close_divs) > 2: # Allow small discrepancy\n self.warnings.append(\n f\"Possible unclosed divs: {open_divs} opening tags, {close_divs} closing tags\"\n )\n\n def _check_citations(self, html: str, md: str):\n \"\"\"Verify citations are present\"\"\"\n # Extract citations from markdown\n md_citations = set(re.findall(r'\\[(\\d+)\\]', md))\n\n # Extract citations from HTML (excluding bibliography)\n html_content = html.split('class=\"bibliography\"')[0] if 'class=\"bibliography\"' in html else html\n html_citations = set(re.findall(r'\\[(\\d+)\\]', html_content))\n\n if len(md_citations) > 0 and len(html_citations) == 0:\n self.errors.append(\"No citations found in HTML content (but present in MD)\")\n\n if len(md_citations) > len(html_citations) * 1.5: # Allow some variation\n self.warnings.append(\n f\"Fewer citations in HTML ({len(html_citations)}) than MD ({len(md_citations)})\"\n )\n\n def _check_bibliography(self, html: str, md: str):\n \"\"\"Verify bibliography is present and formatted\"\"\"\n if '## Bibliography' in md:\n if 'class=\"bibliography\"' not in html:\n self.errors.append(\"Bibliography section missing from HTML\")\n elif 'class=\"bib-entry\"' not in html:\n self.warnings.append(\"Bibliography present but entries not properly formatted\")\n\n def _print_results(self):\n \"\"\"Print verification results\"\"\"\n print(f\"\\n{'-'*60}\")\n print(\"VERIFICATION RESULTS\")\n print(f\"{'-'*60}\\n\")\n\n if self.errors:\n print(f\"❌ ERRORS ({len(self.errors)}):\")\n for i, error in enumerate(self.errors, 1):\n print(f\" {i}. {error}\")\n print()\n\n if self.warnings:\n print(f\"⚠️ WARNINGS ({len(self.warnings)}):\")\n for i, warning in enumerate(self.warnings, 1):\n print(f\" {i}. {warning}\")\n print()\n\n if not self.errors and not self.warnings:\n print(\"✅ All checks passed! HTML report is valid.\")\n print()\n\n print(f\"{'-'*60}\\n\")\n\n\ndef main():\n \"\"\"Main entry point\"\"\"\n parser = argparse.ArgumentParser(description='Verify HTML research report')\n parser.add_argument('--html', type=Path, required=True, help='Path to HTML report')\n parser.add_argument('--md', type=Path, required=True, help='Path to markdown report')\n\n args = parser.parse_args()\n\n if not args.html.exists():\n print(f\"Error: HTML file not found: {args.html}\")\n return 1\n\n if not args.md.exists():\n print(f\"Error: Markdown file not found: {args.md}\")\n return 1\n\n verifier = HTMLVerifier(args.html, args.md)\n success = verifier.verify()\n\n return 0 if success else 1\n\n\nif __name__ == \"__main__\":\n exit(main())\n","content_type":"text/x-python; charset=utf-8","language":"python","size":7800,"content_sha256":"28bbba06906fa2f9bae87ea20b68390ce4f4bc76f00343991a8ad2119e21b36c"},{"filename":"templates/mckinsey_report_template.html","content":"\u003c!DOCTYPE html>\n\u003chtml lang=\"en\">\n\u003chead>\n \u003cmeta charset=\"UTF-8\">\n \u003cmeta name=\"viewport\" content=\"width=device-width, initial-scale=1.0\">\n \u003ctitle>{{TITLE}} - Deep Research Report\u003c/title>\n \u003cstyle>\n * {\n margin: 0;\n padding: 0;\n box-sizing: border-box;\n }\n\n body {\n font-family: \"Helvetica Neue\", Helvetica, Arial, sans-serif;\n font-size: 14px;\n line-height: 1.5;\n color: #1a1a1a;\n background: #ffffff;\n }\n\n .container {\n max-width: 1400px;\n margin: 0 auto;\n background: white;\n }\n\n .header {\n background: #003d5c;\n color: white;\n padding: 25px 40px;\n border-bottom: 3px solid #002840;\n }\n\n .header h1 {\n font-size: 26px;\n font-weight: 600;\n margin-bottom: 8px;\n letter-spacing: -0.5px;\n }\n\n .header-meta {\n font-size: 13px;\n color: #b8d4e6;\n display: flex;\n gap: 25px;\n }\n\n .metrics-dashboard {\n display: grid;\n grid-template-columns: repeat(auto-fit, minmax(200px, 1fr));\n gap: 0;\n border-bottom: 2px solid #003d5c;\n }\n\n .metric {\n padding: 20px 30px;\n background: #f8f9fa;\n border-right: 1px solid #d1d5db;\n text-align: center;\n }\n\n .metric:last-child {\n border-right: none;\n }\n\n .metric-number {\n font-size: 32px;\n font-weight: 700;\n color: #003d5c;\n display: block;\n margin-bottom: 6px;\n }\n\n .metric-label {\n font-size: 12px;\n color: #4a5568;\n text-transform: uppercase;\n letter-spacing: 0.5px;\n font-weight: 500;\n }\n\n .content {\n padding: 30px 40px;\n }\n\n .section {\n margin-bottom: 30px;\n }\n\n .section-title {\n font-size: 20px;\n font-weight: 700;\n color: #003d5c;\n margin: 32px 0 16px 0;\n padding-bottom: 8px;\n border-bottom: 2px solid #003d5c;\n text-transform: uppercase;\n letter-spacing: 0.8px;\n line-height: 1.3;\n }\n\n .subsection-title {\n font-size: 16px;\n font-weight: 700;\n color: #1a1a1a;\n margin: 24px 0 12px 0;\n line-height: 1.4;\n }\n\n .executive-summary {\n background: #f8f9fa;\n padding: 20px;\n margin-bottom: 30px;\n border-left: 4px solid #003d5c;\n }\n\n .executive-summary p {\n margin-bottom: 12px;\n font-size: 14px;\n line-height: 1.6;\n }\n\n p {\n margin-bottom: 14px;\n line-height: 1.7;\n text-align: left;\n }\n\n /* Better paragraph spacing in content sections */\n .content > p,\n .section p {\n margin-bottom: 16px;\n }\n\n .findings-grid {\n display: grid;\n grid-template-columns: repeat(2, 1fr);\n gap: 20px;\n margin-bottom: 30px;\n }\n\n .finding-card {\n background: #f8f9fa;\n padding: 18px;\n border-left: 3px solid #003d5c;\n }\n\n .finding-card h3 {\n font-size: 14px;\n font-weight: 700;\n color: #003d5c;\n margin-bottom: 10px;\n }\n\n .finding-card p {\n font-size: 13px;\n line-height: 1.5;\n margin-bottom: 8px;\n }\n\n .data-table {\n width: 100%;\n border-collapse: collapse;\n margin: 15px 0;\n font-size: 13px;\n }\n\n .data-table th {\n background: #003d5c;\n color: white;\n padding: 10px 15px;\n text-align: left;\n font-weight: 600;\n font-size: 12px;\n text-transform: uppercase;\n letter-spacing: 0.5px;\n }\n\n .data-table td {\n padding: 10px 15px;\n border-bottom: 1px solid #e5e7eb;\n }\n\n .data-table tr:hover {\n background: #f8f9fa;\n }\n\n ul, ol {\n margin: 16px 0 16px 28px;\n padding-left: 0;\n }\n\n li {\n margin-bottom: 10px;\n font-size: 14px;\n line-height: 1.6;\n padding-left: 8px;\n }\n\n /* Nested lists */\n li ul, li ol {\n margin-top: 10px;\n margin-bottom: 10px;\n }\n\n /* Better bullet/number spacing */\n ol {\n list-style-position: outside;\n padding-left: 0;\n }\n\n ul {\n list-style-position: outside;\n padding-left: 0;\n }\n\n .key-insight {\n background: white;\n border: 1px solid #d1d5db;\n border-left: 3px solid #003d5c;\n padding: 15px;\n margin: 15px 0;\n }\n\n .key-insight strong {\n color: #003d5c;\n font-weight: 600;\n }\n\n .citation {\n color: #003d5c;\n font-weight: 600;\n text-decoration: none;\n cursor: pointer;\n position: relative;\n padding: 2px 4px;\n background: #f0f7fc;\n border-radius: 2px;\n transition: all 0.2s ease;\n }\n\n .citation:hover {\n background: #003d5c;\n color: white;\n }\n\n /* Attribution Gradients (2025 Enhancement) */\n .citation-tooltip {\n display: none;\n position: absolute;\n bottom: 100%;\n left: 50%;\n transform: translateX(-50%);\n margin-bottom: 8px;\n background: white;\n border: 2px solid #003d5c;\n box-shadow: 0 4px 12px rgba(0, 61, 92, 0.15);\n padding: 12px;\n min-width: 300px;\n max-width: 500px;\n z-index: 1000;\n font-size: 12px;\n line-height: 1.5;\n }\n\n .citation:hover .citation-tooltip {\n display: block;\n }\n\n .tooltip-title {\n font-weight: 700;\n color: #003d5c;\n margin-bottom: 8px;\n font-size: 13px;\n border-bottom: 1px solid #d1d5db;\n padding-bottom: 6px;\n }\n\n .tooltip-source {\n color: #4a5568;\n margin-bottom: 8px;\n font-style: italic;\n }\n\n .tooltip-claim {\n background: #f8f9fa;\n padding: 8px;\n margin-top: 8px;\n border-left: 3px solid #003d5c;\n font-size: 11px;\n }\n\n .tooltip-claim-label {\n font-weight: 600;\n color: #003d5c;\n text-transform: uppercase;\n font-size: 10px;\n letter-spacing: 0.5px;\n margin-bottom: 4px;\n }\n\n .evidence-chain {\n margin-top: 10px;\n padding-top: 10px;\n border-top: 1px solid #d1d5db;\n }\n\n .evidence-chain-label {\n font-weight: 600;\n color: #003d5c;\n font-size: 11px;\n margin-bottom: 6px;\n text-transform: uppercase;\n letter-spacing: 0.5px;\n }\n\n .evidence-step {\n padding: 6px;\n background: #f8f9fa;\n margin-bottom: 4px;\n font-size: 11px;\n border-left: 2px solid #d1d5db;\n padding-left: 8px;\n }\n\n .bibliography {\n background: #f8f9fa;\n padding: 30px;\n margin-top: 40px;\n border-left: 4px solid #003d5c;\n }\n\n .bibliography-content {\n background: #f8f9fa;\n padding: 20px 0;\n }\n\n .bib-entry {\n margin-bottom: 18px;\n padding-left: 50px;\n text-indent: -50px;\n line-height: 1.6;\n font-size: 13px;\n }\n\n .bib-number {\n color: #003d5c;\n font-weight: 700;\n margin-right: 8px;\n }\n\n .bib-entry a {\n color: #003d5c;\n word-wrap: break-word;\n text-decoration: none;\n }\n\n .bib-entry a:hover {\n text-decoration: underline;\n }\n\n .compact-list {\n display: grid;\n grid-template-columns: repeat(2, 1fr);\n gap: 15px;\n margin: 15px 0;\n }\n\n .compact-list li {\n margin-bottom: 8px;\n font-size: 13px;\n }\n\n .info-box {\n background: white;\n border: 1px solid #d1d5db;\n padding: 15px;\n margin: 15px 0;\n }\n\n .info-box h4 {\n font-size: 13px;\n font-weight: 700;\n color: #003d5c;\n margin-bottom: 8px;\n text-transform: uppercase;\n }\n\n .highlight-stat {\n font-weight: 700;\n color: #003d5c;\n }\n\n strong {\n font-weight: 600;\n color: #1a1a1a;\n }\n\n @media print {\n .container {\n max-width: 100%;\n }\n }\n\n @media (max-width: 768px) {\n .metrics-dashboard {\n grid-template-columns: repeat(2, 1fr);\n }\n .findings-grid {\n grid-template-columns: 1fr;\n }\n .compact-list {\n grid-template-columns: 1fr;\n }\n }\n \u003c/style>\n\u003c/head>\n\u003cbody>\n \u003cdiv class=\"container\">\n \u003cdiv class=\"header\">\n \u003ch1>{{TITLE}}\u003c/h1>\n \u003cdiv class=\"header-meta\">\n \u003cspan>{{DATE}}\u003c/span>\n \u003cspan>•\u003c/span>\n \u003cspan>{{SOURCE_COUNT}} Sources\u003c/span>\n \u003c/div>\n \u003c/div>\n\n {{METRICS_DASHBOARD}}\n\n \u003cdiv class=\"content\">\n {{CONTENT}}\n\n \u003cdiv class=\"bibliography\">\n \u003cdiv class=\"section-title\">Bibliography\u003c/div>\n {{BIBLIOGRAPHY}}\n \u003c/div>\n\n \u003c/div>\n \u003c/div>\n\u003c/body>\n\u003c/html>\n","content_type":"text/html; charset=utf-8","language":"markup","size":10394,"content_sha256":"6b2d742f5279e78df71543bfb6c4fd88d7ee1c726b54fa945616307b2157c1f1"},{"filename":"templates/report_template.md","content":"# Research Report: [Topic]\n\n\u003c!-- =============================================================================\nPROGRESSIVE FILE ASSEMBLY STRATEGY (2025 - Unlimited Length):\n\nThis report is generated section-by-section using progressive file assembly.\nEach section is generated to APPROPRIATE depth (however many words needed) and\nwritten to file immediately using Write/Edit tools.\n\nWHY: Manages output token limits while maintaining quality throughout\nRESULT: Large reports (up to 20,000 words per skill run) - sections sized naturally by content\n\nCLAUDE CODE LIMIT: 32,000 output tokens (≈20,000 words max per run)\nFor reports >20,000 words: Run skill multiple times for different parts\n\nGENERATION WORKFLOW:\n1. Generate Executive Summary → Write to file\n (As long as needed for comprehensive summary)\n\n2. Generate Introduction → Edit/append to file\n (As long as needed to establish context)\n\n3. Generate Finding 1 → Edit/append to file\n (As long as needed to fully present evidence and analysis)\n\n4. Generate Finding 2 → Edit/append to file\n (Each finding sized appropriately - some may need 300 words, others 1,500)\n\n5. Continue for ALL findings (no limit on number OR length per finding!)\n\n6. Generate Synthesis → Edit/append to file\n (As long as needed for deep synthesis)\n\n7. Generate Limitations → Edit/append to file\n8. Generate Recommendations → Edit/append to file\n9. Generate Bibliography (ALL citations) → Edit/append to file\n10. Generate Methodology → Edit/append to file\n\nSIZING PRINCIPLE:\n- Each section should be as long as IT NEEDS TO BE\n- Simple finding? Maybe 400 words is enough\n- Complex multi-faceted finding? Could be 1,200 words\n- Let evidence and analysis determine length, not arbitrary targets\n- Only constraint: Keep each INDIVIDUAL generation under ~2,000 words to avoid output limits\n- If a section needs >2,000 words, break it into subsections and generate progressively\n\nCITATION TRACKING (CRITICAL):\n- Maintain running list in working memory: citations_used = [1, 2, 3, ...]\n- After each section: Add new citations to list\n- In Bibliography: Generate entry for EVERY citation in final list\n- NO gaps, NO ranges, NO placeholders\n\n============================================================================= -->\n\n\u003c!-- WRITING STANDARDS (Apply to EACH section): -->\n\u003c!-- - PRECISION: Each word deliberately chosen, carries intention -->\n\u003c!-- - ECONOMY: No fluff, eliminate fancy grammar, unnecessary adjectives -->\n\u003c!-- - CLARITY: Use exact numbers, specific data, precise technical terms -->\n\u003c!-- - DIRECTNESS: State findings without embellishment -->\n\u003c!-- - HIGH SIGNAL-TO-NOISE: Respect reader's time, dense information -->\n\u003c!-- Examples: \"reduced mortality 23%\" not \"significantly improved outcomes\" -->\n\u003c!-- Examples: \"5 RCTs (n=1,847)\" not \"several studies suggest\" -->\n\n\u003c!-- SOURCE ATTRIBUTION (CRITICAL - PREVENTS FABRICATION): -->\n\u003c!-- EVERY factual claim MUST be followed by [N] citation in same sentence -->\n\u003c!-- Use \"According to [1]...\" or \"[1] reports...\" for factual statements -->\n\u003c!-- DISTINGUISH fact from synthesis: -->\n\u003c!-- ✅ GOOD: \"Mortality decreased 23% (p\u003c0.01) in treatment group [1].\" -->\n\u003c!-- ❌ BAD: \"Studies show mortality improved significantly.\" -->\n\u003c!-- NO vague attributions like \"research suggests\" or \"experts believe\" -->\n\u003c!-- ADMIT uncertainty: \"No sources found for X\" not fabricated citations -->\n\u003c!-- LABEL speculation: \"This suggests...\" not \"Research shows...\" -->\n\n\u003c!-- ANTI-TRUNCATION (CRITICAL - Each Section Must Be COMPLETE): -->\n\u003c!-- ❌ FORBIDDEN: \"Content continues...\", \"Due to length...\", \"[Sections X-Y...]\" -->\n\u003c!-- ✅ REQUIRED: Generate current section COMPLETELY (you're only writing 500 words!) -->\n\u003c!-- ✅ REQUIRED: Write to file immediately, then move to next section -->\n\u003c!-- Progressive assembly handles unlimited length - you handle quality per section -->\n\n## Executive Summary\n\n[Write 3-5 bullet points, 200-400 words total]\n- **Key Finding 1:** [Major discovery with specific data/metrics]\n- **Key Finding 2:** [Important insight with evidence]\n- **Key Finding 3:** [Critical conclusion with implications]\n- [Additional findings as needed]\n\n**Primary Recommendation:** [One clear sentence stating the main recommendation]\n\n**Confidence Level:** [High/Medium/Low with brief justification]\n\n---\n\n## Introduction\n\n### Research Question\n[State the original question clearly and completely]\n\n[Add 1-2 sentences providing context for why this question matters]\n\n### Scope & Methodology\n[2-3 paragraphs explaining:]\n- What specific aspects were investigated\n- What was included vs excluded from scope\n- What research methods were used (web search, academic sources, industry reports, etc.)\n- How many sources were consulted\n- Time period covered\n\n### Key Assumptions\n[List 3-5 important assumptions made during research]\n- Assumption 1: [Description and why it matters]\n- Assumption 2: [Description and why it matters]\n- [Continue...]\n\n---\n\n## Main Analysis\n\n\u003c!-- CRITICAL: Write 4-8 detailed findings, each 600-2,000 words -->\n\u003c!-- Each finding should have multiple paragraphs with evidence -->\n\u003c!-- Include specific data, quotes, statistics, not vague statements -->\n\u003c!-- PRECISION: Use exact numbers, specific metrics, no fluff words -->\n\u003c!-- \"mortality reduced 23%\" not \"significantly improved\" -->\n\u003c!-- \"5 trials (n=1,847)\" not \"several studies\" -->\n\n### Finding 1: [Descriptive Title That Captures the Key Point]\n\n[Opening paragraph: State the finding clearly and why it matters]\n\n[Body paragraphs:\n- Present detailed evidence\n- Include specific data, statistics, dates, numbers\n- Explain mechanisms, causes, or relationships\n- Discuss implications\n- Address nuances or exceptions\n]\n\n**Key Evidence:**\n- Data point 1 from Source A [1]\n- Data point 2 from Source B [2]\n- Conflicting view from Source C [3] and how it was resolved\n\n**Implications:**\n[1-2 paragraphs on what this finding means for the user's decision/understanding]\n\n**Sources:** [1], [2], [3], [4]\n\n---\n\n### Finding 2: [Descriptive Title]\n\n[Follow same detailed structure as Finding 1]\n[Minimum 300 words per finding]\n[Include multiple paragraphs with evidence]\n\n**Sources:** [5], [6], [7], [8]\n\n---\n\n### Finding 3: [Descriptive Title]\n\n[Continue with same detail level]\n\n**Sources:** [9], [10], [11]\n\n---\n\n### Finding 4: [Descriptive Title]\n\n[And so on... Include 4-8 major findings minimum]\n\n**Sources:** [12], [13], [14]\n\n---\n\n[Continue with additional findings as needed]\n\n---\n\n## Synthesis & Insights\n\n\u003c!-- This section should be 500-1000 words -->\n\u003c!-- Go beyond just summarizing - generate NEW insights -->\n\n### Patterns Identified\n\n[2-3 paragraphs identifying key patterns across findings]\n\n**Pattern 1: [Name]**\n[Explain the pattern in detail, cite which findings support it]\n\n**Pattern 2: [Name]**\n[Continue...]\n\n### Novel Insights\n\n[2-3 paragraphs of insights that go BEYOND what sources explicitly stated]\n\n**Insight 1: [Name]**\n[What you discovered by connecting information across sources]\n[Why this matters even though no single source said it explicitly]\n\n**Insight 2: [Name]**\n[Continue...]\n\n### Implications\n\n[2-3 paragraphs on what all this means]\n\n**For [User Context]:**\n[Specific implications for the user's situation/decision]\n\n**Broader Implications:**\n[Wider significance of these findings]\n\n**Second-Order Effects:**\n[What might happen as consequences of these findings]\n\n---\n\n## Limitations & Caveats\n\n\u003c!-- Be honest and comprehensive about what's uncertain -->\n\n### Counterevidence Register\n\n\u003c!-- Document findings that contradict or challenge main conclusions -->\n\n[2-3 paragraphs explaining contradictory evidence found during research]\n\n**Contradictory Finding 1:** [Description]\n- Source: [Citation]\n- Why it contradicts: [Explanation]\n- How resolved/interpreted: [Your analysis]\n- Impact on conclusions: [Minimal/Moderate/Significant]\n\n**Contradictory Finding 2:** [Continue...]\n\n### Known Gaps\n\n[2-3 paragraphs explaining:]\n- What information was not available\n- What questions remain unanswered\n- What would strengthen this research\n\n**Gap 1:** [Description]\n- Why it's missing\n- How it affects conclusions\n- How to address it in future research\n\n**Gap 2:** [Continue...]\n\n### Assumptions\n\n[Revisit key assumptions from intro, now with more detail on their validity]\n\n**Assumption 1:** [Restate]\n- Evidence supporting it: [...]\n- Evidence challenging it: [...]\n- Overall validity: [...]\n\n### Areas of Uncertainty\n\n[2-3 paragraphs on:]\n- Where sources disagree\n- Where evidence is thin\n- Where extrapolation was necessary\n- What could change conclusions\n\n**Uncertainty 1:** [Topic]\n[Detailed explanation of what's uncertain and why]\n\n**Uncertainty 2:** [Continue...]\n\n---\n\n## Recommendations\n\n\u003c!-- Make this actionable and specific -->\n\n### Immediate Actions\n\n[3-5 specific actions the user should take NOW]\n\n1. **[Action Title]**\n - What: [Specific action]\n - Why: [Rationale based on findings]\n - How: [Implementation steps]\n - Timeline: [When to do this]\n\n2. **[Continue with similar detail...]**\n\n### Next Steps\n\n[3-5 actions for the near-term future (1-3 months)]\n\n1. **[Step Title]**\n - [Similar detailed structure]\n\n### Further Research Needs\n\n[3-5 areas where additional research would be valuable]\n\n1. **[Research Topic]**\n - What to investigate: [Specific question]\n - Why it matters: [Connection to current findings]\n - Suggested approach: [How to research it]\n\n---\n\n## Bibliography\n\n\u003c!-- ============================================================================ -->\n\u003c!-- CRITICAL: Generate COMPLETE bibliography with ALL sources cited in report -->\n\u003c!-- DO NOT use placeholders like \"[8-75] Additional citations\" or \"etc.\" -->\n\u003c!-- DO NOT use \"...continue...\" or \"[Continue with all sources...]\" -->\n\u003c!-- EVERY citation [N] in report body MUST have corresponding entry here -->\n\u003c!-- If report cites [1]-[25], bibliography MUST contain all 25 complete entries -->\n\u003c!-- Format: [N] Author/Organization (Year). \"Title\". Publication. URL -->\n\u003c!-- ============================================================================ -->\n\n[1] Author Name or Organization ([YEAR]). \"Full Title of Article or Paper\". Publication Name or Website. https://full-url.com (Retrieved: [CURRENT_DATE])\n\n[2] Second Author ([YEAR]). \"Second Article Title\". Journal Name, Volume(Issue), pages. https://doi-or-url.com (Retrieved: [CURRENT_DATE])\n\n\u003c!-- Add ALL remaining citations [3] through [N] here -->\n\u003c!-- Standard reports: 15-30 sources | Deep/UltraDeep: 30-50 sources -->\n\u003c!-- Write each entry completely - NO ranges, NO \"etc.\", NO placeholders -->\n\n---\n\n## Appendix: Methodology\n\n### Research Process\n\n[2-3 paragraphs describing the research process in detail]\n\n**Phase Execution:**\n- Phase 1 (SCOPE): [What was done]\n- Phase 2 (PLAN): [What was done]\n- Phase 3 (RETRIEVE): [What was done]\n- [Continue for all phases executed]\n\n### Sources Consulted\n\n**Total Sources:** [Number]\n\n**Source Types:**\n- Academic journals: [Number]\n- Industry reports: [Number]\n- News articles: [Number]\n- Government/regulatory: [Number]\n- Documentation: [Number]\n- [Other categories]\n\n**Geographic Coverage:**\n[If relevant, note geographic distribution of sources]\n\n**Temporal Coverage:**\n[Date range of sources, recency distribution]\n\n### Verification Approach\n\n[2-3 paragraphs explaining:]\n\n**Triangulation:**\n- How claims were verified across multiple sources\n- Minimum sources required per major claim: 3\n- How contradictions were handled\n\n**Credibility Assessment:**\n- How source quality was evaluated\n- Scoring system used (0-100)\n- Average credibility score: [Number]/100\n- Distribution: [High/medium/low source counts]\n\n**Quality Control:**\n- Validation checks performed\n- Issues found and corrected\n- Final quality metrics\n\n### Claims-Evidence Table\n\n\u003c!-- Explicit mapping of major claims to supporting sources -->\n\n| Claim ID | Major Claim | Evidence Type | Supporting Sources | Confidence |\n|----------|-------------|---------------|-------------------|------------|\n| C1 | [First major claim from findings] | [Primary data / Meta-analysis / Expert opinion] | [1], [2], [3] | High / Medium / Low |\n| C2 | [Second major claim] | [Evidence type] | [4], [5], [6] | High / Medium / Low |\n| C3 | [Third major claim] | [Evidence type] | [7], [8] | High / Medium / Low |\n| ... | [Continue for all major claims] | ... | ... | ... |\n\n**Confidence Levels:**\n- **High**: 3+ independent sources, consistent findings, strong methodology\n- **Medium**: 2 sources OR single high-quality source with minor contradictions\n- **Low**: Single source OR significant contradictions in evidence\n\n---\n\n## Report Metadata\n\n**Research Mode:** [Quick/Standard/Deep/UltraDeep]\n**Total Sources:** [Number]\n**Word Count:** [Approximate count]\n**Research Duration:** [Time taken]\n**Generated:** [Date and time]\n**Validation Status:** [Passed with X warnings / Passed without warnings]\n\n---\n\n\u003c!-- END OF TEMPLATE -->\n\u003c!-- Remember: Write COMPREHENSIVE, DETAILED reports -->\n\u003c!-- Target 2,000-5,000 words minimum, more for deep modes -->\n\u003c!-- Include specific data, evidence, and analysis throughout -->\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":13144,"content_sha256":"50788e213c78d42a2fcedbfe153e4c34bbea5b7b978f8646f1b4b5c83ba04115"}],"content_json":{"type":"doc","content":[{"type":"heading","attrs":{"level":1},"content":[{"text":"Deep Research","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Core Purpose","type":"text"}]},{"type":"paragraph","content":[{"text":"Deliver citation-tracked research reports through a structured pipeline with evidence persistence, source identity management, claim-level verification, and progressive context management.","type":"text"}]},{"type":"paragraph","content":[{"text":"Autonomy Principle:","type":"text","marks":[{"type":"strong"}]},{"text":" Operate independently. Infer assumptions from context. Only stop for critical errors or incomprehensible queries. Surface high-materiality assumptions explicitly in the Introduction and Methodology rather than silently defaulting.","type":"text"}]},{"type":"hr","attrs":{"markup":"---"}},{"type":"heading","attrs":{"level":2},"content":[{"text":"Decision Tree","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":""},"content":[{"text":"Request Analysis\n+-- Simple lookup? --> STOP: Use WebSearch\n+-- Debugging? --> STOP: Use standard tools\n+-- Complex analysis needed? --> CONTINUE\n\nMode Selection\n+-- Initial exploration --> quick (3 phases, 2-5 min)\n+-- Standard research --> standard (6 phases, 5-10 min) [DEFAULT]\n+-- Critical decision --> deep (8 phases, 10-20 min)\n+-- Comprehensive review --> ultradeep (8+ phases, 20-45 min)","type":"text"}]},{"type":"paragraph","content":[{"text":"Default assumptions:","type":"text","marks":[{"type":"strong"}]},{"text":" Technical query = technical audience. Comparison = balanced perspective. Trend = recent 1-2 years.","type":"text"}]},{"type":"hr","attrs":{"markup":"---"}},{"type":"heading","attrs":{"level":2},"content":[{"text":"Workflow Overview","type":"text"}]},{"type":"table","attrs":{"layout":null},"content":[{"type":"tr","content":[{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Phase","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Name","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Quick","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Std","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Deep","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Ultra","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"1","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"SCOPE","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Y","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Y","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Y","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Y","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"2","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"PLAN","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"-","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Y","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Y","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Y","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"3","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"RETRIEVE","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Y","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Y","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Y","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Y","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"4","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"TRIANGULATE","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"-","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Y","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Y","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Y","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"4.5","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"OUTLINE REFINEMENT","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"-","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Y","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Y","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Y","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"5","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"SYNTHESIZE","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"-","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Y","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Y","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Y","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"6","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"CRITIQUE","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"-","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"-","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Y","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Y","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"7","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"REFINE","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"-","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"-","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Y","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Y","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"8","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"PACKAGE","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Y","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Y","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Y","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Y","type":"text"}]}]}]}]},{"type":"paragraph","content":[{"text":"Note:","type":"text","marks":[{"type":"strong"}]},{"text":" Phases 3-5 operate as an evidence loop per section (retrieve → evidence store → refine outline → draft → verify claims → delta-retrieve if needed), not as strict sequential gates.","type":"text"}]},{"type":"hr","attrs":{"markup":"---"}},{"type":"heading","attrs":{"level":2},"content":[{"text":"Execution","type":"text"}]},{"type":"paragraph","content":[{"text":"On invocation, load relevant reference files:","type":"text","marks":[{"type":"strong"}]}]},{"type":"ordered_list","attrs":{"order":1,"listStyle":"number"},"content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Phase 1-7:","type":"text","marks":[{"type":"strong"}]},{"text":" Load ","type":"text"},{"text":"methodology.md","type":"text","marks":[{"type":"link","attrs":{"href":"./reference/methodology.md","title":null}}]},{"text":" for detailed phase instructions","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Phase 8 (Report):","type":"text","marks":[{"type":"strong"}]},{"text":" Load ","type":"text"},{"text":"report-assembly.md","type":"text","marks":[{"type":"link","attrs":{"href":"./reference/report-assembly.md","title":null}}]},{"text":" for progressive generation","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"HTML/PDF output:","type":"text","marks":[{"type":"strong"}]},{"text":" Load ","type":"text"},{"text":"html-generation.md","type":"text","marks":[{"type":"link","attrs":{"href":"./reference/html-generation.md","title":null}}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Quality checks:","type":"text","marks":[{"type":"strong"}]},{"text":" Load ","type":"text"},{"text":"quality-gates.md","type":"text","marks":[{"type":"link","attrs":{"href":"./reference/quality-gates.md","title":null}}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Long reports (>18K words):","type":"text","marks":[{"type":"strong"}]},{"text":" Load ","type":"text"},{"text":"continuation.md","type":"text","marks":[{"type":"link","attrs":{"href":"./reference/continuation.md","title":null}}]}]}]}]},{"type":"paragraph","content":[{"text":"Templates:","type":"text","marks":[{"type":"strong"}]}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Report structure: ","type":"text"},{"text":"report_template.md","type":"text","marks":[{"type":"link","attrs":{"href":"./templates/report_template.md","title":null}}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"HTML styling: ","type":"text"},{"text":"mckinsey_report_template.html","type":"text","marks":[{"type":"link","attrs":{"href":"./templates/mckinsey_report_template.html","title":null}}]}]}]}]},{"type":"paragraph","content":[{"text":"Scripts:","type":"text","marks":[{"type":"strong"}]}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"python scripts/validate_report.py --report [path]","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"python scripts/verify_citations.py --report [path]","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"python scripts/md_to_html.py [markdown_path]","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"hr","attrs":{"markup":"---"}},{"type":"heading","attrs":{"level":2},"content":[{"text":"Output Contract","type":"text"}]},{"type":"paragraph","content":[{"text":"Required sections:","type":"text","marks":[{"type":"strong"}]}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Executive Summary (200-400 words)","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Introduction (scope, methodology, assumptions)","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Main Analysis (4-8 findings, 600-2,000 words each, cited)","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Synthesis & Insights (patterns, implications)","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Limitations & Caveats","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Recommendations","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Bibliography (COMPLETE - every citation, no placeholders)","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Methodology Appendix","type":"text"}]}]}]},{"type":"paragraph","content":[{"text":"Output files (all to ","type":"text","marks":[{"type":"strong"}]},{"text":"~/Documents/[Topic]_Research_[YYYYMMDD]/","type":"text","marks":[{"type":"code_inline"},{"type":"strong"}]},{"text":"):","type":"text","marks":[{"type":"strong"}]}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Markdown (primary source of truth)","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"sources.jsonl","type":"text","marks":[{"type":"code_inline"}]},{"text":" — stable source registry with canonical IDs","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"evidence.jsonl","type":"text","marks":[{"type":"code_inline"}]},{"text":" — append-only evidence store with quotes and locators","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"claims.jsonl","type":"text","marks":[{"type":"code_inline"}]},{"text":" — atomic claim ledger with support status","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"run_manifest.json","type":"text","marks":[{"type":"code_inline"}]},{"text":" — query, mode, assumptions, provider config","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"HTML (McKinsey style, auto-opened)","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"PDF (professional print, auto-opened)","type":"text"}]}]}]},{"type":"paragraph","content":[{"text":"Quality standards:","type":"text","marks":[{"type":"strong"}]}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"10+ sources, 3+ per major claim (cluster-independent, not just count)","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"All factual claims cited immediately [N] with evidence backing in ","type":"text"},{"text":"evidence.jsonl","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Claim-support verification mandatory: no unsupported factual claims pass delivery","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"No placeholders, no fabricated citations","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Prose-first (>=80%), bullets sparingly","type":"text"}]}]}]},{"type":"hr","attrs":{"markup":"---"}},{"type":"heading","attrs":{"level":2},"content":[{"text":"When to Use / NOT Use","type":"text"}]},{"type":"paragraph","content":[{"text":"Use:","type":"text","marks":[{"type":"strong"}]},{"text":" Comprehensive analysis, technology comparisons, state-of-the-art reviews, multi-perspective investigation, market analysis.","type":"text"}]},{"type":"paragraph","content":[{"text":"Do NOT use:","type":"text","marks":[{"type":"strong"}]},{"text":" Simple lookups, debugging, 1-2 search answers, quick time-sensitive queries.","type":"text"}]}]},"metadata":{"date":"2026-11-02","name":"deep-research","author":"@skillopedia","source":{"stars":1205,"repo_name":"claude-deep-research-skill","origin_url":"https://github.com/199-biotechnologies/claude-deep-research-skill/blob/HEAD/SKILL.md","repo_owner":"199-biotechnologies","body_sha256":"0c8e777eb13d37b4d5130f59c9237a82528224ba31b58b774cd490a742df8620","cluster_key":"b0a4aece764d8b89c53073820ccb0d8dac2dba5dd62fa1787b757a3072fac640","clean_bundle":{"format":"clean-skill-bundle-v1","source":"199-biotechnologies/claude-deep-research-skill/SKILL.md","text_bytes":175135,"source_commit":"f2f2c0fa4e7617ca84c86b63f4bb40f77a746933","skills_sh_rank":2387,"attachment_count":18,"text_attachments":18,"attachment_storage":"skill_attachments","binary_attachments":0},"cluster_size":1,"skill_md_path":"SKILL.md","import_metadata":{"date":"2026-11-02","author":"@skillopedia","version":"v1","category":"education-research","category_label":"Education"},"exact_dupes_collapsed_into_this":0},"version":"v1","category":"education-research","import_tag":"skills-sh-top20k-2026-11","description":"Use when the user needs multi-source research with citation tracking, evidence persistence, and structured report generation. Triggers on \"deep research\", \"comprehensive analysis\", \"research report\", \"compare X vs Y\", \"analyze trends\", or \"state of the art\". Not for simple lookups, debugging, or questions answerable with 1-2 searches."}},"renderedAt":1793671945912}

Deep Research Core Purpose Deliver citation-tracked research reports through a structured pipeline with evidence persistence, source identity management, claim-level verification, and progressive context management. Autonomy Principle: Operate independently. Infer assumptions from context. Only stop for critical errors or incomprehensible queries. Surface high-materiality assumptions explicitly in the Introduction and Methodology rather than silently defaulting. --- Decision Tree Default assumptions: Technical query = technical audience. Comparison = balanced perspective. Trend = recent 1-2 y…