memory-pipeline — Skillopedia

Memory Pipeline + Performance Routine A complete memory and performance system for AI agents. Two subsystems, one package: - Memory Pipeline (Python scripts) — Extracts facts, builds knowledge graphs, generates daily briefings - Performance Routine (TypeScript hooks) — Pre-game briefing injection, tool discipline, output compression, after-action review What This Does Memory Pipeline (Between Sessions) A three-stage system that helps AI agents maintain long-term memory: 1. Extract — Pulls structured facts (decisions, preferences, learnings, commitments) from daily notes and session transcript…

, '', content)\n \n facts = json.loads(content)\n return facts if isinstance(facts, list) else []\n except Exception as e:\n print(f\"OpenAI extraction error: {e}\")\n return []\n\n\ndef extract_with_anthropic(text: str, api_key: str) -> List[Dict]:\n \"\"\"Extract facts using Anthropic API (claude-haiku).\"\"\"\n url = \"https://api.anthropic.com/v1/messages\"\n headers = {\n \"Content-Type\": \"application/json\",\n \"x-api-key\": api_key,\n \"anthropic-version\": \"2023-06-01\"\n }\n \n prompt = f\"\"\"Extract structured facts from the following conversation/notes. For each significant fact, decision, preference, learning, or commitment, output a JSON object with these fields:\n- type: one of \"decision\", \"preference\", \"learning\", \"commitment\", \"fact\"\n- content: the actual fact/decision/etc (concise but complete)\n- subject: what/who this is about (infer from context - projects, people, technologies, etc.)\n- confidence: 0.0-1.0 (how certain is this fact)\n\nReturn ONLY a JSON array of fact objects. No other text.\n\nText to analyze:\n{text[:8000]}\"\"\"\n\n payload = {\n \"model\": \"claude-haiku-4-20250514\",\n \"max_tokens\": 2000,\n \"messages\": [{\"role\": \"user\", \"content\": prompt}]\n }\n \n try:\n response = requests.post(url, headers=headers, json=payload, timeout=30)\n response.raise_for_status()\n result = response.json()\n content = result[\"content\"][0][\"text\"]\n \n # Clean up markdown if present\n content = content.strip()\n if content.startswith(\"```\"):\n content = re.sub(r'^```(?:json)?\\n', '', content)\n content = re.sub(r'\\n```

Important: agents should read /llm.txt, /llms.txt, or /.well-known/skills.json to discover the public Skillopedia API.

, '', content)\n \n facts = json.loads(content)\n return facts if isinstance(facts, list) else []\n except Exception as e:\n print(f\"Anthropic extraction error: {e}\")\n return []\n\n\ndef extract_with_gemini(text: str, api_key: str) -> List[Dict]:\n \"\"\"Extract facts using Gemini API (gemini-flash).\"\"\"\n url = f\"https://generativelanguage.googleapis.com/v1beta/models/gemini-2.0-flash-lite:generateContent?key={api_key}\"\n headers = {\"Content-Type\": \"application/json\"}\n \n prompt = f\"\"\"Extract structured facts from the following conversation/notes. For each significant fact, decision, preference, learning, or commitment, output a JSON object with these fields:\n- type: one of \"decision\", \"preference\", \"learning\", \"commitment\", \"fact\"\n- content: the actual fact/decision/etc (concise but complete)\n- subject: what/who this is about (infer from context - projects, people, technologies, etc.)\n- confidence: 0.0-1.0 (how certain is this fact)\n\nReturn ONLY a JSON array of fact objects. No other text.\n\nText to analyze:\n{text[:8000]}\"\"\"\n\n payload = {\n \"contents\": [{\n \"parts\": [{\"text\": prompt}]\n }],\n \"generationConfig\": {\n \"temperature\": 0.3,\n \"maxOutputTokens\": 2000\n }\n }\n \n try:\n response = requests.post(url, headers=headers, json=payload, timeout=30)\n response.raise_for_status()\n result = response.json()\n content = result[\"candidates\"][0][\"content\"][\"parts\"][0][\"text\"]\n \n # Clean up markdown if present\n content = content.strip()\n if content.startswith(\"```\"):\n content = re.sub(r'^```(?:json)?\\n', '', content)\n content = re.sub(r'\\n```

Important: agents should read /llm.txt, /llms.txt, or /.well-known/skills.json to discover the public Skillopedia API.

, '', content)\n \n facts = json.loads(content)\n return facts if isinstance(facts, list) else []\n except Exception as e:\n print(f\"Gemini extraction error: {e}\")\n return []\n\n\ndef extract_facts(text: str) -> List[Dict]:\n \"\"\"Extract facts using available LLM API.\"\"\"\n provider, api_key = find_api_key()\n \n if not provider:\n print(\"ERROR: No API key found. Set OPENAI_API_KEY, ANTHROPIC_API_KEY, or GEMINI_API_KEY\")\n print(\"Or create a file at ~/.config/openai/api_key (or anthropic/gemini)\")\n return []\n \n print(f\"Using {provider} for extraction...\")\n \n if provider == \"openai\":\n return extract_with_openai(text, api_key)\n elif provider == \"anthropic\":\n return extract_with_anthropic(text, api_key)\n elif provider == \"gemini\":\n return extract_with_gemini(text, api_key)\n \n return []\n\n\ndef read_daily_memory(date: str) -> Optional[str]:\n \"\"\"Read daily memory file for a given date (YYYY-MM-DD).\"\"\"\n file_path = MEMORY_DIR / f\"{date}.md\"\n if file_path.exists():\n return file_path.read_text()\n return None\n\n\ndef read_transcripts() -> Optional[str]:\n \"\"\"Read session transcripts. Only reads recent messages to avoid OOM on large files.\"\"\"\n MAX_CHARS = 15000 # Limit total text to avoid huge LLM calls\n \n if not TRANSCRIPTS_DIR.exists():\n return None\n \n # Find all JSONL files, sorted by modification time (most recent first)\n jsonl_files = sorted(TRANSCRIPTS_DIR.glob(\"*.jsonl\"), key=lambda f: f.stat().st_mtime, reverse=True)\n \n if not jsonl_files:\n return None\n \n text_parts = []\n total_chars = 0\n \n for file in jsonl_files[:3]: # Check up to 3 most recent session files\n # Only read the tail of large files\n file_size = file.stat().st_size\n read_offset = max(0, file_size - 500_000) # Last 500KB max\n \n with open(file) as f:\n if read_offset > 0:\n f.seek(read_offset)\n f.readline() # Skip partial line\n \n for line in f:\n try:\n entry = json.loads(line)\n msg = entry.get(\"message\", {})\n role = msg.get(\"role\", \"\")\n if role not in (\"user\", \"assistant\"):\n continue\n content = msg.get(\"content\", \"\")\n if isinstance(content, list):\n # Extract text from content blocks\n content = \" \".join(\n b.get(\"text\", \"\") for b in content \n if isinstance(b, dict) and b.get(\"type\") == \"text\"\n )\n if not isinstance(content, str) or not content.strip():\n continue\n label = \"User\" if role == \"user\" else \"Assistant\"\n text_parts.append(f\"{label}: {content[:500]}\")\n total_chars += len(text_parts[-1])\n except (json.JSONDecodeError, KeyError):\n pass\n \n if total_chars >= MAX_CHARS:\n break\n \n if total_chars >= MAX_CHARS:\n break\n \n if text_parts:\n return \"\\n\".join(text_parts[-100:]) # Last 100 messages max\n \n return None\n\n\ndef load_existing_facts() -> List[Dict]:\n \"\"\"Load existing facts from extracted.jsonl.\"\"\"\n if not EXTRACTED_FILE.exists():\n return []\n \n facts = []\n with open(EXTRACTED_FILE) as f:\n for line in f:\n try:\n facts.append(json.loads(line))\n except:\n pass\n return facts\n\n\ndef normalize_for_comparison(text: str) -> str:\n \"\"\"Normalize text for dedup comparison.\"\"\"\n t = text.lower().strip()\n t = re.sub(r'[^\\w\\s]', '', t) # Remove punctuation\n t = re.sub(r'\\s+', ' ', t) # Collapse whitespace\n return t\n\ndef word_overlap_ratio(a: str, b: str) -> float:\n \"\"\"Calculate Jaccard similarity between word sets.\"\"\"\n words_a = set(normalize_for_comparison(a).split())\n words_b = set(normalize_for_comparison(b).split())\n if not words_a or not words_b:\n return 0.0\n intersection = words_a & words_b\n union = words_a | words_b\n return len(intersection) / len(union)\n\ndef is_duplicate(new_fact: Dict, existing_facts: List[Dict]) -> bool:\n \"\"\"Check if a fact is a duplicate using normalized text + word overlap.\"\"\"\n new_content = new_fact.get(\"content\", \"\")\n new_norm = normalize_for_comparison(new_content)\n \n for existing in existing_facts:\n existing_content = existing.get(\"content\", \"\")\n existing_norm = normalize_for_comparison(existing_content)\n \n # Exact normalized match\n if new_norm == existing_norm:\n return True\n \n # Substring containment\n if len(new_norm) > 20 and new_norm in existing_norm:\n return True\n if len(existing_norm) > 20 and existing_norm in new_norm:\n return True\n \n # High word overlap (>75% Jaccard similarity) with same subject\n if (new_fact.get(\"subject\", \"\").lower() == existing.get(\"subject\", \"\").lower()\n and word_overlap_ratio(new_content, existing_content) > 0.75):\n return True\n \n return False\n\n\ndef main():\n \"\"\"Main extraction pipeline.\"\"\"\n today = datetime.now().strftime(\"%Y-%m-%d\")\n \n print(f\"Memory Extraction for {today}\")\n print(f\"Workspace: {WORKSPACE_DIR}\")\n print(\"=\" * 50)\n \n # Try to read source text — prefer daily notes (curated), fall back to transcripts\n source = None\n source_type = None\n \n # Try daily memory first (cleaner, smaller, curated)\n print(\"Checking for daily memory file...\")\n daily_text = read_daily_memory(today)\n if daily_text:\n source = daily_text\n source_type = \"daily-note\"\n print(f\"✓ Found today's memory file ({len(source)} chars)\")\n else:\n # Try yesterday's file\n yesterday = (datetime.now() - timedelta(days=1)).strftime(\"%Y-%m-%d\")\n daily_text = read_daily_memory(yesterday)\n if daily_text:\n source = daily_text\n source_type = \"daily-note\"\n today = yesterday # Set date for extracted facts\n print(f\"✓ Found yesterday's memory file ({len(source)} chars)\")\n else:\n print(\"✗ No daily memory files found\")\n \n # Try transcripts as fallback or supplement\n if not source:\n print(\"Checking for session transcripts...\")\n transcript_text = read_transcripts()\n if transcript_text:\n source = transcript_text\n source_type = \"session\"\n print(f\"✓ Found session transcripts ({len(source)} chars)\")\n else:\n print(\"✗ No session transcripts found\")\n \n if not source:\n print(\"\\nERROR: No source text found to extract from\")\n print(f\" Memory dir: {MEMORY_DIR}\")\n print(f\" Transcripts dir: {TRANSCRIPTS_DIR}\")\n return\n \n # Extract facts\n print(\"\\nExtracting facts...\")\n facts = extract_facts(source)\n \n if not facts:\n print(\"No facts extracted\")\n return\n \n print(f\"✓ Extracted {len(facts)} facts\")\n \n # Load existing facts for deduplication\n existing_facts = load_existing_facts()\n print(f\"Loaded {len(existing_facts)} existing facts\")\n \n # Deduplicate and save\n new_facts = []\n duplicates = 0\n \n MEMORY_DIR.mkdir(exist_ok=True)\n \n with open(EXTRACTED_FILE, \"a\") as f:\n for fact in facts:\n # Add metadata\n fact[\"date\"] = today\n fact[\"source\"] = source_type\n \n if is_duplicate(fact, existing_facts):\n duplicates += 1\n continue\n \n # Write to file\n f.write(json.dumps(fact) + \"\\n\")\n new_facts.append(fact)\n existing_facts.append(fact) # Add to existing for subsequent checks\n \n print(f\"\\n✓ Saved {len(new_facts)} new facts\")\n print(f\"✗ Skipped {duplicates} duplicates\")\n \n # Show sample facts\n if new_facts:\n print(\"\\nSample extracted facts:\")\n for fact in new_facts[:3]:\n print(f\" [{fact['type']}] {fact['content'][:80]}...\")\n \n print(f\"\\nExtracted facts saved to: {EXTRACTED_FILE}\")\n\n\nif __name__ == \"__main__\":\n main()\n","content_type":"text/x-python; charset=utf-8","language":"python","size":15618,"content_sha256":"4aeae520f569bec240b7a6ed1b0d3c68c2e37d77efcdbb46900969b97da0dc01"},{"filename":"scripts/memory-link.py","content":"#!/usr/bin/env python3\n\"\"\"\nMemory Linking Script (A-Mem / Zettelkasten Layer)\nCreates a knowledge graph with embeddings and bidirectional links between facts.\n\"\"\"\n\nimport json\nimport os\nimport requests\nfrom datetime import datetime\nfrom pathlib import Path\nfrom typing import List, Dict, Optional\nfrom collections import defaultdict, Counter\nimport math\n\n# Configuration - Auto-detect workspace\ndef get_workspace_dir() -> Path:\n \"\"\"Auto-detect workspace directory from environment or standard locations.\"\"\"\n # 1. Check CLAWDBOT_WORKSPACE env var\n if env_workspace := os.getenv(\"CLAWDBOT_WORKSPACE\"):\n return Path(env_workspace)\n \n # 2. Check current working directory\n cwd = Path.cwd()\n if (cwd / \"SOUL.md\").exists() or (cwd / \"AGENTS.md\").exists():\n return cwd\n \n # 3. Fall back to ~/.clawdbot/workspace\n default_workspace = Path.home() / \".clawdbot\" / \"workspace\"\n if not default_workspace.exists():\n default_workspace.mkdir(parents=True, exist_ok=True)\n \n return default_workspace\n\nWORKSPACE_DIR = get_workspace_dir()\nMEMORY_DIR = WORKSPACE_DIR / \"memory\"\nEXTRACTED_FILE = MEMORY_DIR / \"extracted.jsonl\"\nKNOWLEDGE_GRAPH_FILE = MEMORY_DIR / \"knowledge-graph.json\"\nKNOWLEDGE_SUMMARY_FILE = MEMORY_DIR / \"knowledge-summary.md\"\n\n# API Configuration\nOPENAI_API_KEY = os.getenv(\"OPENAI_API_KEY\")\n\n\ndef find_api_key() -> Optional[str]:\n \"\"\"Find OpenAI API key for embeddings.\"\"\"\n if OPENAI_API_KEY:\n return OPENAI_API_KEY\n \n key_path = Path.home() / \".config\" / \"openai\" / \"api_key\"\n if key_path.exists():\n return key_path.read_text().strip()\n \n return None\n\n\ndef get_embedding(text: str, api_key: str) -> Optional[List[float]]:\n \"\"\"Get embedding vector from OpenAI.\"\"\"\n url = \"https://api.openai.com/v1/embeddings\"\n headers = {\n \"Content-Type\": \"application/json\",\n \"Authorization\": f\"Bearer {api_key}\"\n }\n \n payload = {\n \"model\": \"text-embedding-3-small\",\n \"input\": text\n }\n \n try:\n response = requests.post(url, headers=headers, json=payload, timeout=30)\n response.raise_for_status()\n result = response.json()\n return result[\"data\"][0][\"embedding\"]\n except Exception as e:\n print(f\"Embedding error: {e}\")\n return None\n\n\ndef cosine_similarity(vec1: List[float], vec2: List[float]) -> float:\n \"\"\"Calculate cosine similarity between two vectors.\"\"\"\n dot_product = sum(a * b for a, b in zip(vec1, vec2))\n mag1 = math.sqrt(sum(a * a for a in vec1))\n mag2 = math.sqrt(sum(b * b for b in vec2))\n \n if mag1 == 0 or mag2 == 0:\n return 0.0\n \n return dot_product / (mag1 * mag2)\n\n\ndef extract_keywords(content: str, subject: str) -> List[str]:\n \"\"\"Extract keywords from content (simple approach).\"\"\"\n # Combine content and subject\n text = f\"{subject} {content}\".lower()\n \n # Remove common words\n stop_words = {\n \"the\", \"a\", \"an\", \"and\", \"or\", \"but\", \"in\", \"on\", \"at\", \"to\", \"for\",\n \"of\", \"with\", \"by\", \"from\", \"as\", \"is\", \"was\", \"are\", \"were\", \"be\",\n \"been\", \"being\", \"have\", \"has\", \"had\", \"do\", \"does\", \"did\", \"will\",\n \"would\", \"should\", \"could\", \"can\", \"may\", \"might\", \"must\", \"this\",\n \"that\", \"these\", \"those\", \"i\", \"you\", \"he\", \"she\", \"it\", \"we\", \"they\"\n }\n \n # Simple tokenization\n words = text.replace(\".\", \" \").replace(\",\", \" \").replace(\"!\", \" \").replace(\"?\", \" \").split()\n keywords = [w for w in words if len(w) > 3 and w not in stop_words]\n \n # Get unique keywords, limit to top 10 by frequency\n word_counts = Counter(keywords)\n return [word for word, _ in word_counts.most_common(10)]\n\n\ndef auto_detect_domain_tags(content: str, subject: str) -> List[str]:\n \"\"\"Auto-detect domain tags from content without hardcoded lists.\"\"\"\n tags = []\n content_lower = content.lower()\n subject_lower = subject.lower()\n combined = f\"{content_lower} {subject_lower}\"\n \n # Common technical domains (auto-detected)\n domain_keywords = {\n \"blockchain\": [\"blockchain\", \"web3\", \"crypto\", \"smart contract\", \"wallet\", \"nft\"],\n \"ai\": [\"ai\", \"llm\", \"machine learning\", \"neural\", \"model\", \"gpt\", \"claude\"],\n \"development\": [\"code\", \"programming\", \"development\", \"git\", \"github\", \"api\"],\n \"mobile\": [\"mobile\", \"android\", \"ios\", \"flutter\", \"react native\", \"app\"],\n \"web\": [\"web\", \"website\", \"frontend\", \"backend\", \"server\", \"http\"],\n \"database\": [\"database\", \"sql\", \"postgres\", \"mongodb\", \"query\"],\n \"cloud\": [\"cloud\", \"aws\", \"gcp\", \"azure\", \"deploy\", \"kubernetes\"],\n }\n \n for domain, keywords in domain_keywords.items():\n if any(keyword in combined for keyword in keywords):\n tags.append(domain)\n \n return tags\n\n\ndef generate_tags(fact: Dict) -> List[str]:\n \"\"\"Generate tags based on fact type, subject, and content.\"\"\"\n tags = [fact[\"type\"]]\n \n subject = fact.get(\"subject\", \"\").lower()\n if subject:\n tags.append(subject)\n \n # Auto-detect domain tags from content\n content = fact.get(\"content\", \"\")\n domain_tags = auto_detect_domain_tags(content, subject)\n tags.extend(domain_tags)\n \n return list(set(tags)) # Remove duplicates\n\n\ndef check_contradiction(fact1: Dict, fact2: Dict) -> bool:\n \"\"\"Check if two facts might contradict each other (simple heuristic).\"\"\"\n # Same subject but different content\n if fact1.get(\"subject\") != fact2.get(\"subject\"):\n return False\n \n content1 = fact1.get(\"content\", \"\").lower()\n content2 = fact2.get(\"content\", \"\").lower()\n \n # Check for negation patterns\n negation_pairs = [\n (\"will\", \"won't\"),\n (\"is\", \"isn't\"),\n (\"do\", \"don't\"),\n (\"yes\", \"no\"),\n (\"enable\", \"disable\"),\n (\"active\", \"inactive\"),\n ]\n \n for pos, neg in negation_pairs:\n if (pos in content1 and neg in content2) or (neg in content1 and pos in content2):\n return True\n \n return False\n\n\ndef load_facts() -> List[Dict]:\n \"\"\"Load facts from extracted.jsonl.\"\"\"\n if not EXTRACTED_FILE.exists():\n print(f\"ERROR: {EXTRACTED_FILE} not found. Run memory-extract.py first.\")\n return []\n \n facts = []\n with open(EXTRACTED_FILE) as f:\n for i, line in enumerate(f):\n try:\n fact = json.loads(line)\n fact[\"id\"] = i # Assign unique ID\n facts.append(fact)\n except:\n pass\n \n return facts\n\n\ndef build_knowledge_graph(facts: List[Dict], api_key: Optional[str]) -> Dict:\n \"\"\"Build knowledge graph with embeddings and links.\"\"\"\n print(\"Building knowledge graph...\")\n \n graph = {\n \"nodes\": [],\n \"links\": [],\n \"metadata\": {\n \"created\": datetime.now().isoformat(),\n \"total_facts\": len(facts),\n \"has_embeddings\": api_key is not None\n }\n }\n \n # Process each fact\n for fact in facts:\n # Generate keywords and tags\n keywords = extract_keywords(fact.get(\"content\", \"\"), fact.get(\"subject\", \"\"))\n tags = generate_tags(fact)\n \n # Get embedding if API key available\n embedding = None\n if api_key:\n embed_text = f\"{fact.get('subject', '')} {fact.get('content', '')}\"\n embedding = get_embedding(embed_text, api_key)\n \n # Create enhanced node\n node = {\n \"id\": fact[\"id\"],\n \"type\": fact.get(\"type\"),\n \"content\": fact.get(\"content\"),\n \"subject\": fact.get(\"subject\"),\n \"date\": fact.get(\"date\"),\n \"source\": fact.get(\"source\"),\n \"confidence\": fact.get(\"confidence\", 0.8),\n \"keywords\": keywords,\n \"tags\": tags,\n \"embedding\": embedding,\n \"valid_from\": fact.get(\"date\"),\n \"valid_to\": None, # Updated if contradicted\n \"links\": [] # Will be filled in next step\n }\n \n graph[\"nodes\"].append(node)\n \n # Find related facts and create links\n print(\"Finding related facts...\")\n \n for i, node1 in enumerate(graph[\"nodes\"]):\n if i % 10 == 0:\n print(f\" Processing {i}/{len(graph['nodes'])}...\")\n \n for j, node2 in enumerate(graph[\"nodes\"]):\n if i >= j: # Skip self and already processed pairs\n continue\n \n # Calculate similarity\n similarity = 0.0\n \n # Embedding-based similarity\n if node1[\"embedding\"] and node2[\"embedding\"]:\n similarity = cosine_similarity(node1[\"embedding\"], node2[\"embedding\"])\n else:\n # Fallback: keyword overlap\n keywords1 = set(node1[\"keywords\"])\n keywords2 = set(node2[\"keywords\"])\n if keywords1 and keywords2:\n overlap = len(keywords1 & keywords2)\n total = len(keywords1 | keywords2)\n similarity = overlap / total if total > 0 else 0.0\n \n # Create link if similarity above threshold\n if similarity > 0.3:\n link = {\n \"source\": node1[\"id\"],\n \"target\": node2[\"id\"],\n \"strength\": similarity,\n \"type\": \"related\"\n }\n \n # Check for contradiction\n fact1 = facts[i]\n fact2 = facts[j]\n if check_contradiction(fact1, fact2):\n link[\"type\"] = \"contradicts\"\n \n # Mark older fact as superseded\n date1 = node1[\"date\"]\n date2 = node2[\"date\"]\n if date1 \u003c date2:\n node1[\"valid_to\"] = date2\n else:\n node2[\"valid_to\"] = date1\n \n graph[\"links\"].append(link)\n node1[\"links\"].append(node2[\"id\"])\n node2[\"links\"].append(node1[\"id\"])\n \n return graph\n\n\ndef generate_summary(graph: Dict) -> str:\n \"\"\"Generate human-readable summary of the knowledge graph.\"\"\"\n summary = [\"# Knowledge Graph Summary\\n\"]\n summary.append(f\"Generated: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}\\n\")\n summary.append(f\"Total Facts: {graph['metadata']['total_facts']}\\n\")\n summary.append(f\"Total Links: {len(graph['links'])}\\n\\n\")\n \n # Group facts by subject\n by_subject = defaultdict(list)\n for node in graph[\"nodes\"]:\n subject = node.get(\"subject\", \"Unknown\")\n by_subject[subject].append(node)\n \n # Sort subjects by number of facts\n sorted_subjects = sorted(by_subject.items(), key=lambda x: len(x[1]), reverse=True)\n \n summary.append(\"## Facts by Subject\\n\\n\")\n for subject, nodes in sorted_subjects[:20]: # Top 20 subjects\n summary.append(f\"### {subject} ({len(nodes)} facts)\\n\\n\")\n for node in nodes[:5]: # Top 5 facts per subject\n valid_status = \"\"\n if node.get(\"valid_to\"):\n valid_status = f\" [SUPERSEDED on {node['valid_to']}]\"\n \n summary.append(f\"- **[{node['type']}]** {node['content'][:100]}...{valid_status}\\n\")\n summary.append(f\" - Date: {node['date']} | Confidence: {node['confidence']:.2f}\\n\")\n summary.append(f\" - Tags: {', '.join(node['tags'][:5])}\\n\")\n \n if node.get(\"links\"):\n summary.append(f\" - Related to: {len(node['links'])} other facts\\n\")\n summary.append(\"\\n\")\n \n # Recent decisions\n summary.append(\"## Recent Decisions\\n\\n\")\n decisions = [n for n in graph[\"nodes\"] if n[\"type\"] == \"decision\"]\n decisions.sort(key=lambda x: x[\"date\"], reverse=True)\n for decision in decisions[:10]:\n summary.append(f\"- **{decision['date']}**: {decision['content']}\\n\")\n \n summary.append(\"\\n## Recent Learnings\\n\\n\")\n learnings = [n for n in graph[\"nodes\"] if n[\"type\"] == \"learning\"]\n learnings.sort(key=lambda x: x[\"date\"], reverse=True)\n for learning in learnings[:10]:\n summary.append(f\"- **{learning['date']}**: {learning['content']}\\n\")\n \n return \"\".join(summary)\n\n\ndef main():\n \"\"\"Main linking pipeline.\"\"\"\n print(\"Memory Linking (A-Mem)\")\n print(f\"Workspace: {WORKSPACE_DIR}\")\n print(\"=\" * 50)\n \n # Load facts\n facts = load_facts()\n if not facts:\n return\n \n print(f\"Loaded {len(facts)} facts\")\n \n # Check for API key\n api_key = find_api_key()\n if api_key:\n print(\"✓ OpenAI API key found (embeddings enabled)\")\n else:\n print(\"✗ No OpenAI API key (using keyword-based similarity)\")\n \n # Build knowledge graph\n graph = build_knowledge_graph(facts, api_key)\n \n # Save knowledge graph\n MEMORY_DIR.mkdir(exist_ok=True)\n with open(KNOWLEDGE_GRAPH_FILE, \"w\") as f:\n json.dump(graph, f, indent=2)\n \n print(f\"\\n✓ Knowledge graph saved to: {KNOWLEDGE_GRAPH_FILE}\")\n \n # Generate summary\n summary = generate_summary(graph)\n with open(KNOWLEDGE_SUMMARY_FILE, \"w\") as f:\n f.write(summary)\n \n print(f\"✓ Summary saved to: {KNOWLEDGE_SUMMARY_FILE}\")\n \n # Stats\n contradictions = sum(1 for link in graph[\"links\"] if link[\"type\"] == \"contradicts\")\n avg_links = sum(len(node[\"links\"]) for node in graph[\"nodes\"]) / len(graph[\"nodes\"]) if graph[\"nodes\"] else 0\n \n print(f\"\\nStats:\")\n print(f\" - Total facts: {len(facts)}\")\n print(f\" - Total links: {len(graph['links'])}\")\n print(f\" - Contradictions found: {contradictions}\")\n print(f\" - Avg links per fact: {avg_links:.1f}\")\n\n\nif __name__ == \"__main__\":\n main()\n","content_type":"text/x-python; charset=utf-8","language":"python","size":13804,"content_sha256":"2a6a0974fc29c79fb75265ce57e56fb2e1c2aa05c22e64431fc23151dad55500"},{"filename":"src/briefing.ts","content":"export function buildBriefingPacket(opts: {\n checklist: string[];\n memoryText: string;\n maxChars: number;\n taskHint: string;\n}) {\n const header =\n `# Pre-Game Routine\\n` +\n `Task hint: ${truncate(opts.taskHint, 240)}\\n\\n` +\n `## Checklist\\n` +\n opts.checklist.map((x) => `- ${x}`).join(\"\\n\") +\n `\\n\\n## Retrieved Memory (bounded)\\n`;\n const body = truncate(opts.memoryText, Math.max(0, opts.maxChars - header.length));\n return truncate(header + body, opts.maxChars);\n}\n\nfunction truncate(s: string, n: number) {\n if (!s) return \"\";\n if (s.length \u003c= n) return s;\n return s.slice(0, Math.max(0, n - 20)) + \"\\n[truncated]\\n\";\n}\n","content_type":"text/typescript; charset=utf-8","language":"typescript","size":650,"content_sha256":"fdddb2adbfa960b8310263a6692114da629fd01077bd3c29e3840d4c895da320"},{"filename":"src/compress.ts","content":"export function compressToolResult(result: any, maxChars: number) {\n const s = typeof result === \"string\" ? result : safeJson(result);\n if (s.length \u003c= maxChars) return result;\n const head = s.slice(0, Math.floor(maxChars * 0.6));\n const tail = s.slice(-Math.floor(maxChars * 0.3));\n const out = `${head}\\n[tool output truncated]\\n${tail}`;\n return typeof result === \"string\" ? out : { truncated: true, preview: out };\n}\n\nfunction safeJson(x: any) {\n try {\n return JSON.stringify(x, null, 2);\n } catch {\n return String(x);\n }\n}\n","content_type":"text/typescript; charset=utf-8","language":"typescript","size":543,"content_sha256":"4bf925a9b2475be488f9d922655929a0bb3a93daa998147dabe81ce7425d7c4c"},{"filename":"src/index.ts","content":"import path from \"node:path\";\nimport { buildBriefingPacket } from \"./briefing\";\nimport { loadMemoryFiles, appendAfterAction } from \"./memory\";\nimport { compressToolResult } from \"./compress\";\n\nexport default function register(api: any) {\n const cfg = api.config?.plugins?.entries?.[\"memory-pipeline\"] ?? {};\n if (cfg.enabled === false) return;\n\n const briefingCfg = cfg.briefing ?? {};\n const toolsCfg = cfg.tools ?? {};\n const afterActionCfg = cfg.afterAction ?? {};\n\n // 1) Purposeful Thinking: build a briefing packet before each run.\n api.hooks.on(\"before_agent_start\", async (ctx: any) => {\n const workspaceRoot = ctx.workspaceRoot ?? api.config?.agents?.defaults?.workspace;\n const files = (briefingCfg.memoryFiles ?? []).map((p: string) =>\n path.isAbsolute(p) ? p : path.join(workspaceRoot, p)\n );\n const memoryText = await loadMemoryFiles(files);\n const packet = buildBriefingPacket({\n checklist: briefingCfg.checklist ?? [],\n memoryText,\n maxChars: briefingCfg.maxChars ?? 6000,\n taskHint: ctx.input?.message ?? \"\",\n });\n ctx.systemPromptAppend?.(`\\n${packet}\\n`);\n ctx.systemPromptParts?.push(packet);\n return ctx;\n });\n\n // 2) Reactive Execution: enforce discipline right before tool calls.\n api.hooks.on(\"before_tool_call\", async (ctx: any) => {\n const deny: string[] = toolsCfg.deny ?? [];\n const toolName = ctx.tool?.name;\n if (toolName && deny.includes(toolName)) {\n throw new Error(`Tool denied by performance-routine policy: ${toolName}`);\n }\n return ctx;\n });\n\n // 3) Prevent tool bloat: compress tool results before persisting.\n api.hooks.on(\"tool_result_persist\", async (ctx: any) => {\n const limit = toolsCfg.maxToolResultChars ?? 12000;\n ctx.toolResult = compressToolResult(ctx.toolResult, limit);\n return ctx;\n });\n\n // 4) After Action Review: write durable notes at the end of the run.\n api.hooks.on(\"agent_end\", async (ctx: any) => {\n const workspaceRoot = ctx.workspaceRoot ?? api.config?.agents?.defaults?.workspace;\n const outFile = path.isAbsolute(afterActionCfg.writeMemoryFile)\n ? afterActionCfg.writeMemoryFile\n : path.join(workspaceRoot, afterActionCfg.writeMemoryFile ?? \"memory/AFTER_ACTION.md\");\n const maxBullets = afterActionCfg.maxBullets ?? 8;\n await appendAfterAction({\n filePath: outFile,\n sessionId: ctx.sessionId,\n runId: ctx.runId,\n maxBullets,\n finalAnswer: ctx.finalAnswerText ?? \"\",\n toolCalls: ctx.toolCalls ?? [],\n });\n return ctx;\n });\n}\n","content_type":"text/typescript; charset=utf-8","language":"typescript","size":2545,"content_sha256":"1e5ebf76fe1761547c6ee12e5d9e34b61083896814b406f583f5b6185a26b4f5"},{"filename":"src/memory.ts","content":"import fs from \"node:fs/promises\";\nimport nodePath from \"node:path\";\n\nexport async function loadMemoryFiles(files: string[]) {\n const parts: string[] = [];\n for (const f of files) {\n try {\n const txt = await fs.readFile(f, \"utf8\");\n parts.push(`## ${f}\\n${txt}\\n`);\n } catch {\n // Missing files are fine. Keep routine resilient.\n }\n }\n return parts.join(\"\\n\");\n}\n\nexport async function appendAfterAction(opts: {\n filePath: string;\n sessionId: string;\n runId: string;\n maxBullets: number;\n finalAnswer: string;\n toolCalls: Array\u003c{ name: string; ok?: boolean }>;\n}) {\n const toolLine = (opts.toolCalls || [])\n .slice(0, 12)\n .map((t) => `- ${t.name}${t.ok === false ? \" (failed)\" : \"\"}`)\n .join(\"\\n\");\n\n const bullets = deriveBullets(opts.finalAnswer, opts.maxBullets)\n .map((b) => `- ${b}`)\n .join(\"\\n\");\n\n const entry =\n `\\n## After Action Review\\n` +\n `session: ${opts.sessionId}\\n` +\n `run: ${opts.runId}\\n\\n` +\n `### What happened\\n${bullets}\\n\\n` +\n `### Tools used\\n${toolLine || \"- none\"}\\n`;\n\n await fs.mkdir(nodePath.dirname(opts.filePath), { recursive: true });\n await fs.appendFile(opts.filePath, entry, \"utf8\");\n}\n\nfunction deriveBullets(finalAnswer: string, maxBullets: number) {\n const lines = (finalAnswer || \"\")\n .split(\"\\n\")\n .map((l) => l.trim())\n .filter(Boolean);\n const bullets: string[] = [];\n for (const l of lines) {\n if (bullets.length >= maxBullets) break;\n if (l.length \u003c 8) continue;\n bullets.push(l.slice(0, 180));\n }\n if (bullets.length === 0) bullets.push(\"Completed run. No durable notes extracted.\");\n return bullets;\n}\n","content_type":"text/typescript; charset=utf-8","language":"typescript","size":1645,"content_sha256":"13b2a35de1c6f7494361f20a90949582a23afb118d21dace8f11359ff8a30c43"}],"content_json":{"type":"doc","content":[{"type":"heading","attrs":{"level":1},"content":[{"text":"Memory Pipeline + Performance Routine","type":"text"}]},{"type":"paragraph","content":[{"text":"A complete memory and performance system for AI agents. Two subsystems, one package:","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Memory Pipeline","type":"text","marks":[{"type":"strong"}]},{"text":" (Python scripts) — Extracts facts, builds knowledge graphs, generates daily briefings","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Performance Routine","type":"text","marks":[{"type":"strong"}]},{"text":" (TypeScript hooks) — Pre-game briefing injection, tool discipline, output compression, after-action review","type":"text"}]}]}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"What This Does","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Memory Pipeline (Between Sessions)","type":"text"}]},{"type":"paragraph","content":[{"text":"A three-stage system that helps AI agents maintain long-term memory:","type":"text"}]},{"type":"ordered_list","attrs":{"order":1,"listStyle":"number"},"content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Extract","type":"text","marks":[{"type":"strong"}]},{"text":" — Pulls structured facts (decisions, preferences, learnings, commitments) from daily notes and session transcripts using LLM extraction","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Link","type":"text","marks":[{"type":"strong"}]},{"text":" — Builds a knowledge graph with embeddings and bidirectional links between related facts, identifies contradictions","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Briefing","type":"text","marks":[{"type":"strong"}]},{"text":" — Generates a compact BRIEFING.md file loaded at session start with personality reminders, active projects, recent decisions, and key context","type":"text"}]}]}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Performance Routine (Within Sessions)","type":"text"}]},{"type":"paragraph","content":[{"text":"Four lifecycle hooks that enforce consistency during agent runs:","type":"text"}]},{"type":"ordered_list","attrs":{"order":1,"listStyle":"number"},"content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Pre-Game Routine","type":"text","marks":[{"type":"strong"}]},{"text":" (","type":"text"},{"text":"before_agent_start","type":"text","marks":[{"type":"code_inline"}]},{"text":") — Assembles a bounded briefing packet from memory files + checklist, injects into system prompt","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Tool Discipline","type":"text","marks":[{"type":"strong"}]},{"text":" (","type":"text"},{"text":"before_tool_call","type":"text","marks":[{"type":"code_inline"}]},{"text":") — Enforces deny lists, normalizes params, prevents unsafe tool calls","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Output Compression","type":"text","marks":[{"type":"strong"}]},{"text":" (","type":"text"},{"text":"tool_result_persist","type":"text","marks":[{"type":"code_inline"}]},{"text":") — Head+tail compression of large tool results to prevent context bloat","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"After-Action Review","type":"text","marks":[{"type":"strong"}]},{"text":" (","type":"text"},{"text":"agent_end","type":"text","marks":[{"type":"code_inline"}]},{"text":") — Writes durable notes about what happened, tools used, and outcomes","type":"text"}]}]}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Quick Start","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Installation","type":"text"}]},{"type":"paragraph","content":[{"text":"The skill includes three Python scripts in ","type":"text"},{"text":"scripts/","type":"text","marks":[{"type":"code_inline"}]},{"text":":","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"memory-extract.py","type":"text","marks":[{"type":"code_inline"}]},{"text":" — Fact extraction","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"memory-link.py","type":"text","marks":[{"type":"code_inline"}]},{"text":" — Knowledge graph building","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"memory-briefing.py","type":"text","marks":[{"type":"code_inline"}]},{"text":" — Daily briefing generation","type":"text"}]}]}]},{"type":"paragraph","content":[{"text":"All scripts auto-detect your workspace from:","type":"text"}]},{"type":"ordered_list","attrs":{"order":1,"listStyle":"number"},"content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"CLAWDBOT_WORKSPACE","type":"text","marks":[{"type":"code_inline"}]},{"text":" environment variable","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Current working directory (if contains SOUL.md or AGENTS.md)","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"~/.clawdbot/workspace","type":"text","marks":[{"type":"code_inline"}]},{"text":" (default fallback)","type":"text"}]}]}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Requirements","type":"text"}]},{"type":"paragraph","content":[{"text":"At least one LLM API key","type":"text","marks":[{"type":"strong"}]},{"text":" is required:","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"OpenAI API key (for GPT-4o-mini + embeddings)","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Anthropic API key (for Claude Haiku)","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Gemini API key (for Gemini Flash)","type":"text"}]}]}]},{"type":"paragraph","content":[{"text":"Set via environment variable or config file:","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"bash"},"content":[{"text":"# Environment variable\nexport OPENAI_API_KEY=\"sk-...\"\n\n# OR config file\necho \"sk-...\" > ~/.config/openai/api_key","type":"text"}]},{"type":"paragraph","content":[{"text":"The scripts will auto-detect and use whichever API key is available.","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Basic Usage","type":"text"}]},{"type":"paragraph","content":[{"text":"Run the full pipeline:","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"bash"},"content":[{"text":"python3 scripts/memory-extract.py\npython3 scripts/memory-link.py\npython3 scripts/memory-briefing.py","type":"text"}]},{"type":"paragraph","content":[{"text":"Or run individual steps as needed.","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Pipeline Stages","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Stage 1: Extract Facts","type":"text"}]},{"type":"paragraph","content":[{"text":"Script:","type":"text","marks":[{"type":"strong"}]},{"text":" ","type":"text"},{"text":"memory-extract.py","type":"text","marks":[{"type":"code_inline"}]}]},{"type":"paragraph","content":[{"text":"Reads from (in priority order):","type":"text"}]},{"type":"ordered_list","attrs":{"order":1,"listStyle":"number"},"content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Daily memory files (","type":"text"},{"text":"{workspace}/memory/YYYY-MM-DD.md","type":"text","marks":[{"type":"code_inline"}]},{"text":") — today or yesterday","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Session transcripts (","type":"text"},{"text":"~/.clawdbot/agents/main/sessions/*.jsonl","type":"text","marks":[{"type":"code_inline"}]},{"text":")","type":"text"}]}]}]},{"type":"paragraph","content":[{"text":"Extracts structured facts:","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Type","type":"text","marks":[{"type":"strong"}]},{"text":": decision, preference, learning, commitment, fact","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Content","type":"text","marks":[{"type":"strong"}]},{"text":": The actual information","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Subject","type":"text","marks":[{"type":"strong"}]},{"text":": What it's about (auto-detected from context)","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Confidence","type":"text","marks":[{"type":"strong"}]},{"text":": 0.0-1.0 reliability score","type":"text"}]}]}]},{"type":"paragraph","content":[{"text":"Output:","type":"text","marks":[{"type":"strong"}]},{"text":" ","type":"text"},{"text":"{workspace}/memory/extracted.jsonl","type":"text","marks":[{"type":"code_inline"}]},{"text":" — One JSON fact per line, deduplicated","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Stage 2: Build Knowledge Graph","type":"text"}]},{"type":"paragraph","content":[{"text":"Script:","type":"text","marks":[{"type":"strong"}]},{"text":" ","type":"text"},{"text":"memory-link.py","type":"text","marks":[{"type":"code_inline"}]}]},{"type":"paragraph","content":[{"text":"Takes extracted facts and:","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Generates embeddings (if OpenAI key available, else uses keyword similarity)","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Creates bidirectional links between related facts","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Detects contradictions and marks superseded facts","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Auto-generates domain tags from content","type":"text"}]}]}]},{"type":"paragraph","content":[{"text":"Output:","type":"text","marks":[{"type":"strong"}]}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"{workspace}/memory/knowledge-graph.json","type":"text","marks":[{"type":"code_inline"}]},{"text":" — Full graph with nodes and links","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"{workspace}/memory/knowledge-summary.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" — Human-readable summary","type":"text"}]}]}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Stage 3: Generate Briefing","type":"text"}]},{"type":"paragraph","content":[{"text":"Script:","type":"text","marks":[{"type":"strong"}]},{"text":" ","type":"text"},{"text":"memory-briefing.py","type":"text","marks":[{"type":"code_inline"}]}]},{"type":"paragraph","content":[{"text":"Creates a compact daily briefing loaded at session start.","type":"text"}]},{"type":"paragraph","content":[{"text":"Combines:","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Personality traits (from SOUL.md if exists)","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"User context (from USER.md if exists)","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Active projects (top subjects from recent facts)","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Recent decisions and preferences","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Active todos (from any todos*.md files)","type":"text"}]}]}]},{"type":"paragraph","content":[{"text":"Output:","type":"text","marks":[{"type":"strong"}]},{"text":" ","type":"text"},{"text":"{workspace}/BRIEFING.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" — Under 2000 chars, LLM-generated or template-based","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Wiring Into HEARTBEAT.md","type":"text"}]},{"type":"paragraph","content":[{"text":"To run automatically, add to your workspace's ","type":"text"},{"text":"HEARTBEAT.md","type":"text","marks":[{"type":"code_inline"}]},{"text":":","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"markdown"},"content":[{"text":"# Heartbeat Tasks\n\n## Daily (once per day, morning)\n- Run memory extraction: `cd {workspace} && python3 skills/memory-pipeline/scripts/memory-extract.py`\n- Build knowledge graph: `cd {workspace} && python3 skills/memory-pipeline/scripts/memory-link.py`\n- Generate briefing: `cd {workspace} && python3 skills/memory-pipeline/scripts/memory-briefing.py`\n\n## Weekly (Sunday evening)\n- Review `memory/knowledge-summary.md` for insights\n- Clean up old daily notes (optional)","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Loading BRIEFING.md","type":"text"}]},{"type":"paragraph","content":[{"text":"Important:","type":"text","marks":[{"type":"strong"}]},{"text":" BRIEFING.md needs to be loaded as workspace context at session start. This requires the OpenClaw context loading feature (currently in development).","type":"text"}]},{"type":"paragraph","content":[{"text":"Once available, configure your agent to load BRIEFING.md along with SOUL.md, USER.md, and AGENTS.md at the start of each session.","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Output Files","type":"text"}]},{"type":"paragraph","content":[{"text":"All files are created in ","type":"text"},{"text":"{workspace}/memory/","type":"text","marks":[{"type":"code_inline"}]},{"text":":","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"extracted.jsonl","type":"text","marks":[{"type":"strong"}]},{"text":" — All extracted facts (append-only)","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"knowledge-graph.json","type":"text","marks":[{"type":"strong"}]},{"text":" — Full knowledge graph with embeddings and links","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"knowledge-summary.md","type":"text","marks":[{"type":"strong"}]},{"text":" — Human-readable summary of the graph","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"BRIEFING.md","type":"text","marks":[{"type":"strong"}]},{"text":" (in workspace root) — Daily context cheat sheet","type":"text"}]}]}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Customization","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Changing Models","type":"text"}]},{"type":"paragraph","content":[{"text":"Edit the model names in each script:","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"memory-extract.py","type":"text","marks":[{"type":"code_inline"}]},{"text":": Lines with ","type":"text"},{"text":"\"model\": \"gpt-4o-mini\"","type":"text","marks":[{"type":"code_inline"}]},{"text":" (or claude/gemini equivalents)","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"memory-link.py","type":"text","marks":[{"type":"code_inline"}]},{"text":": Line with ","type":"text"},{"text":"\"model\": \"text-embedding-3-small\"","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"memory-briefing.py","type":"text","marks":[{"type":"code_inline"}]},{"text":": Lines with ","type":"text"},{"text":"\"model\": \"gpt-4o-mini\"","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Adjusting Extraction","type":"text"}]},{"type":"paragraph","content":[{"text":"In ","type":"text"},{"text":"memory-extract.py","type":"text","marks":[{"type":"code_inline"}]},{"text":", modify the extraction prompt (lines ~75-85) to focus on different types of information or change the output format.","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Link Threshold","type":"text"}]},{"type":"paragraph","content":[{"text":"In ","type":"text"},{"text":"memory-link.py","type":"text","marks":[{"type":"code_inline"}]},{"text":", change the similarity threshold for creating links (currently 0.3 at line ~195).","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Troubleshooting","type":"text"}]},{"type":"paragraph","content":[{"text":"No facts extracted:","type":"text","marks":[{"type":"strong"}]}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Check that daily notes or transcripts exist","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Verify API key is set correctly","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Check script output for LLM errors","type":"text"}]}]}]},{"type":"paragraph","content":[{"text":"Low-quality links:","type":"text","marks":[{"type":"strong"}]}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Add OpenAI API key for embedding-based similarity (more accurate than keyword matching)","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Adjust similarity threshold in ","type":"text"},{"text":"memory-link.py","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"paragraph","content":[{"text":"Briefing too long:","type":"text","marks":[{"type":"strong"}]}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Reduce number of facts included in template (edit ","type":"text"},{"text":"generate_fallback_briefing","type":"text","marks":[{"type":"code_inline"}]},{"text":")","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"LLM-generated briefings are automatically constrained to 2000 chars","type":"text"}]}]}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Performance Routine (Hook System)","type":"text"}]},{"type":"paragraph","content":[{"text":"The performance routine is implemented as OpenClaw lifecycle hooks in ","type":"text"},{"text":"src/","type":"text","marks":[{"type":"code_inline"}]},{"text":". It applies a core principle from performance psychology: ","type":"text"},{"text":"separate thinking from doing","type":"text","marks":[{"type":"strong"}]},{"text":". Athletes don't redesign their technique mid-game — they prepare (purposeful thinking), then execute trained sequences (reactive execution). The only exception is genuine error handling.","type":"text"}]},{"type":"paragraph","content":[{"text":"For agents, this means: front-load all context, constraints, and memory retrieval into a briefing packet ","type":"text"},{"text":"before","type":"text","marks":[{"type":"em"}]},{"text":" inference starts. Keep execution clean. Write the after-action review ","type":"text"},{"text":"after","type":"text","marks":[{"type":"em"}]},{"text":". Never inject corrections mid-run.","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Architecture","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":""},"content":[{"text":"User Message → Gateway → Agent Loop\n ├── before_agent_start → Briefing Packet (checklist + memory + constraints)\n ├── LLM Inference (clean context, no mid-run corrections)\n ├── before_tool_call → Policy enforcement (deny list)\n ├── Tool Execution → Result\n ├── tool_result_persist → Compression (head+tail, bounded)\n └── agent_end → After-Action Review → durable memory for next run","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"The Core Idea: No Mid-Swing Coaching","type":"text"}]},{"type":"paragraph","content":[{"text":"Constant correction during execution degrades output. Mid-run prompt patches create instruction collision — two competing directives the agent must reconcile instead of executing. The alternative:","type":"text"}]},{"type":"ordered_list","attrs":{"order":1,"listStyle":"number"},"content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Capture corrections","type":"text","marks":[{"type":"strong"}]},{"text":" — don't inject them into the current run","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Condense into deltas","type":"text","marks":[{"type":"strong"}]},{"text":" — merge all corrections into a clean update","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Inject next run","type":"text","marks":[{"type":"strong"}]},{"text":" — the next briefing packet includes the corrected instructions","type":"text"}]}]}]},{"type":"paragraph","content":[{"text":"The after-action review (","type":"text"},{"text":"agent_end","type":"text","marks":[{"type":"code_inline"}]},{"text":") feeds back into the next briefing (","type":"text"},{"text":"before_agent_start","type":"text","marks":[{"type":"code_inline"}]},{"text":"). The loop is closed — just not during execution.","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Configuration","type":"text"}]},{"type":"paragraph","content":[{"text":"Configure via ","type":"text"},{"text":"openclaw.plugin.json","type":"text","marks":[{"type":"code_inline"}]},{"text":" or your agent config:","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"json"},"content":[{"text":"{\n \"enabled\": true,\n \"briefing\": {\n \"maxChars\": 6000,\n \"checklist\": [\n \"Restate the task in one sentence.\",\n \"List constraints and success criteria.\",\n \"Retrieve only the minimum relevant memory.\",\n \"Prefer tools over guessing when facts matter.\"\n ],\n \"memoryFiles\": [\"memory/IDENTITY.md\", \"memory/PROJECTS.md\"]\n },\n \"tools\": {\n \"deny\": [\"dangerous_tool\"],\n \"maxToolResultChars\": 12000\n },\n \"afterAction\": {\n \"writeMemoryFile\": \"memory/AFTER_ACTION.md\",\n \"maxBullets\": 8\n }\n}","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Hook Details","type":"text"}]},{"type":"paragraph","content":[{"text":"before_agent_start","type":"text","marks":[{"type":"code_inline"},{"type":"strong"}]},{"text":" — Briefing Packet","type":"text","marks":[{"type":"strong"}]}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Loads configured memory files from workspace","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Builds bounded packet: task hint + checklist + retrieved memory","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Injects into system prompt (respects ","type":"text"},{"text":"maxChars","type":"text","marks":[{"type":"code_inline"}]},{"text":" limit)","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Missing memory files are silently skipped","type":"text"}]}]}]},{"type":"paragraph","content":[{"text":"before_tool_call","type":"text","marks":[{"type":"code_inline"},{"type":"strong"}]},{"text":" — Tool Discipline","type":"text","marks":[{"type":"strong"}]}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Checks tool name against ","type":"text"},{"text":"deny","type":"text","marks":[{"type":"code_inline"}]},{"text":" list","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Throws error if denied (prevents execution)","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Extensible for param normalization","type":"text"}]}]}]},{"type":"paragraph","content":[{"text":"tool_result_persist","type":"text","marks":[{"type":"code_inline"},{"type":"strong"}]},{"text":" — Output Compression","type":"text","marks":[{"type":"strong"}]}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Keeps results under ","type":"text"},{"text":"maxToolResultChars","type":"text","marks":[{"type":"code_inline"}]},{"text":" (default 12K)","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Uses head (60%) + tail (30%) strategy","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Preserves structure for JSON results","type":"text"}]}]}]},{"type":"paragraph","content":[{"text":"after_action_review","type":"text","marks":[{"type":"code_inline"},{"type":"strong"}]},{"text":" — Durable Notes","type":"text","marks":[{"type":"strong"}]}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Appends session summary to configured memory file","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Extracts key bullets from final answer","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Logs tools used (with failure flags)","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Creates directories automatically","type":"text"}]}]}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Source Files","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"src/index.ts","type":"text","marks":[{"type":"code_inline"}]},{"text":" — Hook registration and wiring","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"src/briefing.ts","type":"text","marks":[{"type":"code_inline"}]},{"text":" — Briefing packet builder","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"src/compress.ts","type":"text","marks":[{"type":"code_inline"}]},{"text":" — Tool result compressor","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"src/memory.ts","type":"text","marks":[{"type":"code_inline"}]},{"text":" — Memory file loader + after-action writer","type":"text"}]}]}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"See Also","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Setup Guide","type":"text","marks":[{"type":"link","attrs":{"href":"references/setup.md","title":null}}]},{"text":" — Detailed installation and configuration","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Blog Post Draft","type":"text","marks":[{"type":"link","attrs":{"href":"../../drafts/blog-pregame-routine.md","title":null}}]},{"text":" — Full writeup of the performance routine concept","type":"text"}]}]}]},{"type":"hr","attrs":{"markup":"---"}}]},"metadata":{"date":"2026-06-05","name":"memory-pipeline","author":"@skillopedia","source":{"stars":0,"repo_name":"openclaw-workspace","origin_url":"https://github.com/boomsystel-code/openclaw-workspace/blob/HEAD/skills/memory-pipeline-0-1-0/SKILL.md","repo_owner":"boomsystel-code","body_sha256":"4c82166a87a1cc751352e50a82926db437ee5126142c5431f90b43abb8f08d6e","cluster_key":"c3e8592a678f6b191254268bef1b725b89b342c61710d872b4f878ea487199b6","clean_bundle":{"format":"clean-skill-bundle-v1","source":"boomsystel-code/openclaw-workspace/skills/memory-pipeline-0-1-0/SKILL.md","attachments":[{"id":"99a63363-c575-5546-a1b9-eab5168fc885","key":"uploads/10433ee7-ad12-4ae0-b34e-97553e46c6c8/99a63363-c575-5546-a1b9-eab5168fc885/attachment.json","path":".clawhub/origin.json","size":153,"sha256":"c3194b9bb208620c0e4e4f737b571844adab092b12ebae37bad6fed176403de3","contentType":"application/json; charset=utf-8"},{"id":"9f474284-3309-55dc-a0fe-ded22ab9aaf9","key":"uploads/10433ee7-ad12-4ae0-b34e-97553e46c6c8/9f474284-3309-55dc-a0fe-ded22ab9aaf9/attachment.json","path":"_meta.json","size":140,"sha256":"4658a079d573219d6de24ae04497f62a7608fee25cc202903eb3cc127a2be053","contentType":"application/json; charset=utf-8"},{"id":"4dcdda74-41cd-5331-ac7e-0c745ed13d34","key":"uploads/10433ee7-ad12-4ae0-b34e-97553e46c6c8/4dcdda74-41cd-5331-ac7e-0c745ed13d34/attachment.json","path":"openclaw.plugin.json","size":1857,"sha256":"41259f6ebe4b450cca1342f2d2827753f6be3c359682aed5e41ffd88a43a505d","contentType":"application/json; charset=utf-8"},{"id":"41c30bd2-d55f-55cb-875f-3f61f8d00c93","key":"uploads/10433ee7-ad12-4ae0-b34e-97553e46c6c8/41c30bd2-d55f-55cb-875f-3f61f8d00c93/attachment.md","path":"references/setup.md","size":10945,"sha256":"f58cb8775625a36d6d66fca6837f95fe7935d6ef09b94074e6384f75059b1b6d","contentType":"text/markdown; charset=utf-8"},{"id":"9bd1e3ed-9869-59c4-972d-23abfedf787d","key":"uploads/10433ee7-ad12-4ae0-b34e-97553e46c6c8/9bd1e3ed-9869-59c4-972d-23abfedf787d/attachment.py","path":"scripts/memory-briefing.py","size":12951,"sha256":"a86cb1f56049462a38077b18fa0aa467039bd920bf50d993f6d7d2ef920e786b","contentType":"text/x-python; charset=utf-8"},{"id":"009a8b67-694d-52bf-b576-4d48753a741a","key":"uploads/10433ee7-ad12-4ae0-b34e-97553e46c6c8/009a8b67-694d-52bf-b576-4d48753a741a/attachment.py","path":"scripts/memory-extract.py","size":15618,"sha256":"4aeae520f569bec240b7a6ed1b0d3c68c2e37d77efcdbb46900969b97da0dc01","contentType":"text/x-python; charset=utf-8"},{"id":"fe1a3baf-1884-59ac-8f3f-cfe9fbc918e0","key":"uploads/10433ee7-ad12-4ae0-b34e-97553e46c6c8/fe1a3baf-1884-59ac-8f3f-cfe9fbc918e0/attachment.py","path":"scripts/memory-link.py","size":13804,"sha256":"2a6a0974fc29c79fb75265ce57e56fb2e1c2aa05c22e64431fc23151dad55500","contentType":"text/x-python; charset=utf-8"},{"id":"e815b30c-d643-5266-b38c-663fa434e515","key":"uploads/10433ee7-ad12-4ae0-b34e-97553e46c6c8/e815b30c-d643-5266-b38c-663fa434e515/attachment.ts","path":"src/briefing.ts","size":650,"sha256":"fdddb2adbfa960b8310263a6692114da629fd01077bd3c29e3840d4c895da320","contentType":"text/typescript; charset=utf-8"},{"id":"5f14c0e5-88d8-585a-ab60-d67c603a41d2","key":"uploads/10433ee7-ad12-4ae0-b34e-97553e46c6c8/5f14c0e5-88d8-585a-ab60-d67c603a41d2/attachment.ts","path":"src/compress.ts","size":543,"sha256":"4bf925a9b2475be488f9d922655929a0bb3a93daa998147dabe81ce7425d7c4c","contentType":"text/typescript; charset=utf-8"},{"id":"0cab2190-2186-5e0d-ba85-802fbabb210b","key":"uploads/10433ee7-ad12-4ae0-b34e-97553e46c6c8/0cab2190-2186-5e0d-ba85-802fbabb210b/attachment.ts","path":"src/index.ts","size":2545,"sha256":"1e5ebf76fe1761547c6ee12e5d9e34b61083896814b406f583f5b6185a26b4f5","contentType":"text/typescript; charset=utf-8"},{"id":"fabf7686-daec-5b24-a58c-ac9e34066a3d","key":"uploads/10433ee7-ad12-4ae0-b34e-97553e46c6c8/fabf7686-daec-5b24-a58c-ac9e34066a3d/attachment.ts","path":"src/memory.ts","size":1645,"sha256":"13b2a35de1c6f7494361f20a90949582a23afb118d21dace8f11359ff8a30c43","contentType":"text/typescript; charset=utf-8"}],"bundle_sha256":"6b4c6f35ffd712f57401a2337b3a8a6cf7b125ddfc285977a20f43d3c9b9f4cb","attachment_count":11,"text_attachments":11,"attachment_storage":"skillopedia-attachments-v1","binary_attachments":0,"excluded_attachments":[]},"cluster_size":1,"skill_md_path":"skills/memory-pipeline-0-1-0/SKILL.md","import_metadata":{"date":"2026-06-05","author":"@skillopedia","version":"v1","category":"web-development","category_label":"Web"},"exact_dupes_collapsed_into_this":0},"version":"v1","category":"web-development","import_tag":"clean-skills-v1","description":"Complete agent memory + performance system. Extracts structured facts, builds knowledge graphs, generates briefings, and enforces execution discipline via pre-game routines, tool policies, result compression, and after-action reviews. Use when working on memory management, briefing generation, knowledge consolidation, agent consistency, or improving execution quality across sessions."}},"renderedAt":1782980421918}

Important: agents should read /llm.txt, /llms.txt, or /.well-known/skills.json to discover the public Skillopedia API.