detecting-network-anomalies-with-zeek

Detecting Network Anomalies with Zeek When to Use - Deploying passive network security monitoring at key network choke points for continuous visibility - Generating structured connection, DNS, HTTP, SSL, and file transfer logs for SIEM ingestion and threat hunting - Writing custom Zeek scripts to detect organization-specific threats, policy violations, or beaconing behavior - Performing retrospective analysis on network metadata to investigate security incidents - Complementing IDS solutions with protocol-level metadata analysis that signature-based tools may miss Do not use as a replacement…

\\t' -k5 -rn | head -20\n\n# Find connections with high data transfer volumes\ncat /opt/zeek/logs/current/conn.log | zeek-cut ts id.orig_h id.resp_h orig_bytes resp_bytes | \\\n awk '$4 > 100000000 || $5 > 100000000 {print $0}'\n\n# Identify rare user agents (potential malware)\ncat /opt/zeek/logs/current/http.log | zeek-cut user_agent | sort | uniq -c | sort -n | head -20\n\n# Find self-signed or expired certificates\ncat /opt/zeek/logs/current/ssl.log | zeek-cut ts id.orig_h id.resp_h server_name validation_status | \\\n grep -v \"ok\"\n\n# Detect DNS queries to newly registered domains (DGA patterns)\ncat /opt/zeek/logs/current/dns.log | zeek-cut ts id.orig_h query | \\\n awk -F'\\t' '{n=split($3,a,\".\"); if(length(a[n-1]) > 10) print $0}'\n\n# Find SSH brute force attempts\ncat /opt/zeek/logs/current/ssh.log | zeek-cut ts id.orig_h id.resp_h auth_success | \\\n grep \"F\" | awk '{print $2}' | sort | uniq -c | sort -rn | head -10\n\n# Identify unusual port usage\ncat /opt/zeek/logs/current/conn.log | zeek-cut id.resp_p proto service | \\\n sort | uniq -c | sort -rn | head -50\n```\n\n### Step 6: Integrate with SIEM and Set Up Alerting\n\n```bash\n# Configure JSON log output for SIEM ingestion\nsudo tee /opt/zeek/share/zeek/site/json-logs.zeek \u003c\u003c 'EOF'\n@load policy/tuning/json-logs.zeek\nredef LogAscii::use_json = T;\nEOF\n\n# Forward logs to Elastic via Filebeat\n# /etc/filebeat/filebeat.yml\nsudo tee /etc/filebeat/filebeat.yml \u003c\u003c 'EOF'\nfilebeat.inputs:\n - type: log\n enabled: true\n paths:\n - /opt/zeek/logs/current/*.log\n json.keys_under_root: true\n json.add_error_key: true\n fields:\n source: zeek\n fields_under_root: true\n\noutput.elasticsearch:\n hosts: [\"https://elastic-siem:9200\"]\n index: \"zeek-%{+yyyy.MM.dd}\"\n username: \"elastic\"\n password: \"${ES_PASSWORD}\"\nEOF\n\nsudo systemctl enable --now filebeat\n\n# Set up log rotation\nsudo tee /etc/cron.d/zeek-logrotate \u003c\u003c 'EOF'\n0 0 * * * root /opt/zeek/bin/zeekctl cron\nEOF\n\n# Monitor Zeek health\nsudo zeekctl status\nsudo zeekctl netstats\n```\n\n## Key Concepts\n\n| Term | Definition |\n|------|------------|\n| **Network Security Monitor** | Passive analysis tool that observes network traffic and generates structured metadata logs without altering or blocking traffic flow |\n| **Zeek Script** | Event-driven scripts written in Zeek's domain-specific language that process network events and generate notices, logs, and metrics |\n| **Connection Log (conn.log)** | Core Zeek log recording every observed connection with source/destination IPs, ports, protocol, duration, and byte counts |\n| **Notice Framework** | Zeek subsystem for generating alerts when detection scripts identify suspicious activity, outputting to notice.log |\n| **SumStats Framework** | Statistical analysis framework in Zeek for tracking metrics over time windows, enabling threshold-based detection of anomalies |\n| **Intel Framework** | Zeek module for matching observed network indicators against threat intelligence feeds and generating alerts on matches |\n\n## Tools & Systems\n\n- **Zeek 6.0+**: Open-source network security monitor generating comprehensive protocol-level logs from passive traffic analysis\n- **zeek-cut**: Zeek utility for extracting specific columns from tab-separated Zeek log files for quick analysis\n- **zeekctl**: Zeek management tool for deploying, monitoring, and managing Zeek instances across single or clustered deployments\n- **RITA (Real Intelligence Threat Analytics)**: Open-source tool that analyzes Zeek logs for beaconing, DNS tunneling, and other threat indicators\n- **Filebeat**: Elastic agent for shipping Zeek JSON logs to Elasticsearch for centralized analysis and visualization\n\n## Common Scenarios\n\n### Scenario: Detecting Command-and-Control Beaconing in Enterprise Traffic\n\n**Context**: A threat intelligence report indicates that a specific threat actor uses HTTPS beaconing with 60-second intervals to compromised hosts. The SOC team needs to analyze Zeek logs to identify any hosts exhibiting this pattern across the enterprise network carrying 2 Gbps of traffic.\n\n**Approach**:\n1. Deploy Zeek on a network tap at the internet egress point with AF_PACKET for high-throughput capture\n2. Enable the custom beacon detection script with thresholds tuned for 60-second intervals over 1-hour observation windows\n3. Query conn.log for connections to external IPs with consistent duration and inter-connection timing: filter connections where the standard deviation of inter-arrival times is less than 5 seconds\n4. Cross-reference suspicious destination IPs against threat intelligence feeds loaded into Zeek's Intel framework\n5. Examine ssl.log for the associated TLS certificates -- check for self-signed certificates, unusual issuer names, or certificates with short validity periods\n6. Generate a notice for each identified beaconing source and feed into the SIEM for SOC triage\n\n**Pitfalls**:\n- Not tuning beacon detection thresholds for the environment, resulting in false positives from legitimate update services (Windows Update, AV updates)\n- Failing to exclude CDN and cloud service provider IP ranges that naturally receive many repeat connections\n- Running Zeek without sufficient CPU cores, causing packet drops on high-throughput links\n- Not enabling JSON log output, making SIEM integration unnecessarily complex with custom parsers\n\n## Output Format\n\n```\n## Zeek Network Anomaly Detection Report\n\n**Sensor**: zeek-sensor-01 (10.10.1.250)\n**Monitoring Interface**: eth1 (span port from Core-SW1)\n**Analysis Period**: 2024-03-15 00:00 to 2024-03-16 00:00 UTC\n**Total Connections Logged**: 2,847,392\n\n### Anomalies Detected\n\n| Notice Type | Source | Destination | Details |\n|-------------|--------|-------------|---------|\n| DNS_Tunneling_Detected | 10.10.3.45 | 8.8.8.8 | 847 queries to suspect-domain.xyz in 5 min |\n| Possible_Beaconing | 10.10.5.12 | 203.0.113.50:443 | 62 connections with 59.8s avg interval |\n| SSL::Invalid_Server_Cert | 10.10.8.22 | 198.51.100.33:443 | Self-signed cert, CN=localhost |\n| SSH::Password_Guessing | 45.33.32.156 | 10.10.20.11:22 | 487 failed attempts in 30 min |\n\n### Recommendations\n1. Isolate 10.10.3.45 and investigate for DNS tunneling malware\n2. Block 203.0.113.50 at firewall and forensically image 10.10.5.12\n3. Investigate self-signed TLS certificate on 198.51.100.33\n4. Block 45.33.32.156 and enforce SSH key-only authentication\n```\n---","attachment_filenames":["references/api-reference.md","scripts/agent.py"],"attachments":[{"filename":"references/api-reference.md","content":"# Zeek Network Anomaly Detection API Reference\n\n## Zeek CLI\n\n```bash\n# Process PCAP file\nzeek -r capture.pcap -C\n\n# Run on live interface\nzeek -i eth1\n\n# Run with custom script\nzeek -r capture.pcap local.zeek\n\n# ZeekControl\nzeekctl deploy # Deploy and start\nzeekctl status # Check status\nzeekctl stop # Stop all workers\nzeekctl diag # Diagnostics\n```\n\n## Zeek Log Files\n\n| Log | Content | Key Fields |\n|-----|---------|------------|\n| `conn.log` | All connections | ts, id.orig_h, id.resp_h, service, duration |\n| `dns.log` | DNS queries/responses | query, qtype_name, rcode_name |\n| `ssl.log` | TLS handshakes | server_name, ja3, validation_status |\n| `http.log` | HTTP requests | method, host, uri, user_agent |\n| `files.log` | File transfers | md5, sha1, mime_type, filename |\n| `notice.log` | Zeek notices/alerts | note, msg, src, dst |\n| `weird.log` | Protocol anomalies | name, addl |\n| `x509.log` | Certificate details | san.dns, certificate.not_valid_after |\n\n## Zeek Scripting - Custom Detection\n\n```zeek\n# Detect DNS tunneling (long queries)\nevent dns_request(c: connection, msg: dns_msg, query: string, qtype: count, qclass: count) {\n if (|query| > 60) {\n NOTICE([$note=DNS::Tunneling,\n $conn=c,\n $msg=fmt(\"Long DNS query (%d chars): %s\", |query|, query),\n $identifier=cat(c$id$orig_h)]);\n }\n}\n\n# Detect C2 beaconing\n@load base/frameworks/sumstats\nevent connection_established(c: connection) {\n if (Site::is_local_addr(c$id$orig_h) && !Site::is_local_addr(c$id$resp_h)) {\n SumStats::observe(\"ext_conns\",\n SumStats::Key($str=cat(c$id$orig_h, \"->\", c$id$resp_h)),\n SumStats::Observation($num=1));\n }\n}\n```\n\n## Zeek Log Parsing (Python)\n\n```python\n# Parse tab-separated Zeek logs\nwith open(\"conn.log\") as f:\n for line in f:\n if line.startswith(\"#\"):\n continue\n fields = line.strip().split(\"\\t\")\n ts, uid, orig_h, orig_p, resp_h, resp_p = fields[:6]\n```\n\n## zeek-cut (CLI field extraction)\n\n```bash\n# Extract specific fields\ncat conn.log | zeek-cut id.orig_h id.resp_h id.resp_p service\n\n# DNS queries sorted by count\ncat dns.log | zeek-cut query | sort | uniq -c | sort -rn | head -20\n\n# JA3 fingerprints\ncat ssl.log | zeek-cut ja3 server_name | sort | uniq -c | sort -rn\n```\n\n## Beaconing Detection Formula\n\n```\ninterval_avg = mean(connection_intervals)\njitter = mean(|interval - interval_avg|) / interval_avg\nif jitter \u003c 0.15 and connections > 10:\n flag as potential C2 beacon\n```\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":2550,"content_sha256":"6f09a8c7c7e39f49a0ba90d1981d227de32a8a4063e577ff90f20f8b12b86e2d"},{"filename":"scripts/agent.py","content":"#!/usr/bin/env python3\n\"\"\"Zeek network anomaly detection agent for log analysis and threat hunting.\"\"\"\n\nimport json\nimport os\nimport subprocess\nimport sys\nfrom collections import Counter, defaultdict\nfrom datetime import datetime\nfrom pathlib import Path\n\n\nZEEK_BIN = os.environ.get(\"ZEEK_BIN\", \"/opt/zeek/bin/zeek\")\nZEEK_LOG_DIR = os.environ.get(\"ZEEK_LOG_DIR\", \"/opt/zeek/logs/current\")\n\n\ndef check_zeek_status():\n \"\"\"Check Zeek installation and running status.\"\"\"\n version = {\"installed\": False}\n try:\n result = subprocess.run([ZEEK_BIN, \"--version\"], capture_output=True, text=True, timeout=10)\n version = {\"installed\": True, \"version\": result.stdout.strip() or result.stderr.strip()}\n except FileNotFoundError:\n try:\n result = subprocess.run([\"zeek\", \"--version\"], capture_output=True, text=True, timeout=10)\n version = {\"installed\": True, \"version\": result.stdout.strip()}\n except FileNotFoundError:\n version = {\"installed\": False}\n\n running = False\n try:\n r = subprocess.run([\"zeekctl\", \"status\"], capture_output=True, text=True, timeout=10)\n running = \"running\" in r.stdout.lower()\n except FileNotFoundError:\n pass\n\n return {**version, \"running\": running}\n\n\ndef parse_conn_log(log_path=None):\n \"\"\"Parse Zeek conn.log for connection statistics and anomalies.\"\"\"\n log_path = log_path or os.path.join(ZEEK_LOG_DIR, \"conn.log\")\n if not os.path.exists(log_path):\n return {\"error\": f\"conn.log not found: {log_path}\"}\n\n total = 0\n protocols = Counter()\n services = Counter()\n top_talkers = Counter()\n top_destinations = Counter()\n long_connections = []\n\n with open(log_path, \"r\") as f:\n header = {}\n for line in f:\n if line.startswith(\"#fields\"):\n fields_list = line.strip().split(\"\\t\")[1:]\n header = {name: i for i, name in enumerate(fields_list)}\n continue\n if line.startswith(\"#\"):\n continue\n parts = line.strip().split(\"\\t\")\n total += 1\n if not header:\n continue\n\n src = parts[header.get(\"id.orig_h\", 2)] if len(parts) > header.get(\"id.orig_h\", 2) else \"\"\n dst = parts[header.get(\"id.resp_h\", 4)] if len(parts) > header.get(\"id.resp_h\", 4) else \"\"\n proto = parts[header.get(\"proto\", 6)] if len(parts) > header.get(\"proto\", 6) else \"\"\n service = parts[header.get(\"service\", 7)] if len(parts) > header.get(\"service\", 7) else \"-\"\n duration = parts[header.get(\"duration\", 8)] if len(parts) > header.get(\"duration\", 8) else \"-\"\n\n protocols[proto] += 1\n if service != \"-\":\n services[service] += 1\n top_talkers[src] += 1\n top_destinations[dst] += 1\n\n if duration != \"-\":\n try:\n dur = float(duration)\n if dur > 3600:\n long_connections.append({\"src\": src, \"dst\": dst, \"duration_sec\": dur, \"service\": service})\n except ValueError:\n pass\n\n return {\n \"total_connections\": total,\n \"protocols\": dict(protocols),\n \"top_services\": services.most_common(15),\n \"top_sources\": top_talkers.most_common(15),\n \"top_destinations\": top_destinations.most_common(15),\n \"long_connections\": sorted(long_connections, key=lambda x: x[\"duration_sec\"], reverse=True)[:20],\n }\n\n\ndef parse_dns_log(log_path=None):\n \"\"\"Parse Zeek dns.log for DNS anomaly detection.\"\"\"\n log_path = log_path or os.path.join(ZEEK_LOG_DIR, \"dns.log\")\n if not os.path.exists(log_path):\n return {\"error\": f\"dns.log not found: {log_path}\"}\n\n queries = Counter()\n query_types = Counter()\n long_queries = []\n nxdomain = []\n\n with open(log_path, \"r\") as f:\n header = {}\n for line in f:\n if line.startswith(\"#fields\"):\n fields_list = line.strip().split(\"\\t\")[1:]\n header = {name: i for i, name in enumerate(fields_list)}\n continue\n if line.startswith(\"#\"):\n continue\n parts = line.strip().split(\"\\t\")\n if not header:\n continue\n\n query = parts[header.get(\"query\", 9)] if len(parts) > header.get(\"query\", 9) else \"\"\n qtype = parts[header.get(\"qtype_name\", 13)] if len(parts) > header.get(\"qtype_name\", 13) else \"\"\n rcode = parts[header.get(\"rcode_name\", 15)] if len(parts) > header.get(\"rcode_name\", 15) else \"\"\n src = parts[header.get(\"id.orig_h\", 2)] if len(parts) > header.get(\"id.orig_h\", 2) else \"\"\n\n queries[query] += 1\n query_types[qtype] += 1\n\n if len(query) > 60:\n long_queries.append({\"source\": src, \"query\": query, \"length\": len(query)})\n if rcode == \"NXDOMAIN\":\n nxdomain.append({\"source\": src, \"query\": query})\n\n return {\n \"unique_queries\": len(queries),\n \"top_queries\": queries.most_common(20),\n \"query_types\": dict(query_types),\n \"long_queries_tunneling\": long_queries[:20],\n \"nxdomain_count\": len(nxdomain),\n \"nxdomain_samples\": nxdomain[:20],\n }\n\n\ndef parse_ssl_log(log_path=None):\n \"\"\"Parse Zeek ssl.log for TLS anomalies and certificate issues.\"\"\"\n log_path = log_path or os.path.join(ZEEK_LOG_DIR, \"ssl.log\")\n if not os.path.exists(log_path):\n return {\"error\": f\"ssl.log not found: {log_path}\"}\n\n ja3_hashes = Counter()\n server_names = Counter()\n expired_certs = []\n\n with open(log_path, \"r\") as f:\n header = {}\n for line in f:\n if line.startswith(\"#fields\"):\n fields_list = line.strip().split(\"\\t\")[1:]\n header = {name: i for i, name in enumerate(fields_list)}\n continue\n if line.startswith(\"#\"):\n continue\n parts = line.strip().split(\"\\t\")\n if not header:\n continue\n\n ja3 = parts[header.get(\"ja3\", -1)] if header.get(\"ja3\") and len(parts) > header[\"ja3\"] else \"-\"\n sni = parts[header.get(\"server_name\", -1)] if header.get(\"server_name\") and len(parts) > header[\"server_name\"] else \"-\"\n valid = parts[header.get(\"validation_status\", -1)] if header.get(\"validation_status\") and len(parts) > header[\"validation_status\"] else \"-\"\n\n if ja3 != \"-\":\n ja3_hashes[ja3] += 1\n if sni != \"-\":\n server_names[sni] += 1\n if \"expired\" in valid.lower() if valid != \"-\" else False:\n expired_certs.append({\"sni\": sni, \"validation\": valid})\n\n return {\n \"unique_ja3\": len(ja3_hashes),\n \"top_ja3\": ja3_hashes.most_common(20),\n \"top_sni\": server_names.most_common(20),\n \"expired_certs\": expired_certs[:20],\n }\n\n\ndef detect_beaconing(log_path=None, interval_tolerance=0.15):\n \"\"\"Detect C2 beaconing patterns from Zeek conn.log.\"\"\"\n log_path = log_path or os.path.join(ZEEK_LOG_DIR, \"conn.log\")\n if not os.path.exists(log_path):\n return {\"error\": f\"conn.log not found: {log_path}\"}\n\n pair_times = defaultdict(list)\n with open(log_path, \"r\") as f:\n header = {}\n for line in f:\n if line.startswith(\"#fields\"):\n fields_list = line.strip().split(\"\\t\")[1:]\n header = {name: i for i, name in enumerate(fields_list)}\n continue\n if line.startswith(\"#\"):\n continue\n parts = line.strip().split(\"\\t\")\n if not header:\n continue\n ts = parts[header.get(\"ts\", 0)] if len(parts) > header.get(\"ts\", 0) else \"\"\n src = parts[header.get(\"id.orig_h\", 2)] if len(parts) > header.get(\"id.orig_h\", 2) else \"\"\n dst = parts[header.get(\"id.resp_h\", 4)] if len(parts) > header.get(\"id.resp_h\", 4) else \"\"\n try:\n pair_times[f\"{src}->{dst}\"].append(float(ts))\n except ValueError:\n pass\n\n beacons = []\n for pair, times in pair_times.items():\n if len(times) \u003c 10:\n continue\n times.sort()\n intervals = [times[i+1] - times[i] for i in range(len(times)-1)]\n if not intervals:\n continue\n avg = sum(intervals) / len(intervals)\n if avg \u003c 1:\n continue\n jitter = sum(abs(i - avg) for i in intervals) / len(intervals) / avg if avg > 0 else 1\n if jitter \u003c interval_tolerance:\n src, dst = pair.split(\"->\")\n beacons.append({\n \"source\": src, \"destination\": dst,\n \"connections\": len(times),\n \"avg_interval_sec\": round(avg, 1),\n \"jitter_pct\": round(jitter * 100, 1),\n })\n\n beacons.sort(key=lambda x: x[\"connections\"], reverse=True)\n return {\"beacons_detected\": len(beacons), \"beacons\": beacons[:20]}\n\n\ndef analyze_pcap(pcap_path):\n \"\"\"Analyze a PCAP file with Zeek to generate logs.\"\"\"\n if not os.path.exists(pcap_path):\n return {\"error\": f\"PCAP not found: {pcap_path}\"}\n\n import tempfile\n output_dir = os.path.join(\n os.environ.get(\"ZEEK_OUTPUT_DIR\", tempfile.gettempdir()),\n f\"zeek_analysis_{datetime.now().strftime('%Y%m%d_%H%M%S')}\"\n )\n os.makedirs(output_dir, exist_ok=True)\n try:\n result = subprocess.run(\n [\"zeek\", \"-r\", pcap_path, \"-C\"],\n capture_output=True, text=True, timeout=120, cwd=output_dir\n )\n logs = list(Path(output_dir).glob(\"*.log\"))\n return {\n \"pcap\": pcap_path,\n \"output_dir\": output_dir,\n \"logs_generated\": [l.name for l in logs],\n \"exit_code\": result.returncode,\n }\n except Exception as e:\n return {\"error\": str(e)}\n\n\ndef generate_report(log_dir=None):\n \"\"\"Generate comprehensive Zeek network analysis report.\"\"\"\n log_dir = log_dir or ZEEK_LOG_DIR\n return {\n \"timestamp\": datetime.utcnow().isoformat() + \"Z\",\n \"status\": check_zeek_status(),\n \"connections\": parse_conn_log(os.path.join(log_dir, \"conn.log\")),\n \"dns\": parse_dns_log(os.path.join(log_dir, \"dns.log\")),\n \"tls\": parse_ssl_log(os.path.join(log_dir, \"ssl.log\")),\n \"beaconing\": detect_beaconing(os.path.join(log_dir, \"conn.log\")),\n }\n\n\nif __name__ == \"__main__\":\n action = sys.argv[1] if len(sys.argv) > 1 else \"report\"\n log_dir = sys.argv[2] if len(sys.argv) > 2 else ZEEK_LOG_DIR\n if action == \"report\":\n print(json.dumps(generate_report(log_dir), indent=2, default=str))\n elif action == \"connections\":\n print(json.dumps(parse_conn_log(os.path.join(log_dir, \"conn.log\")), indent=2))\n elif action == \"dns\":\n print(json.dumps(parse_dns_log(os.path.join(log_dir, \"dns.log\")), indent=2))\n elif action == \"tls\":\n print(json.dumps(parse_ssl_log(os.path.join(log_dir, \"ssl.log\")), indent=2))\n elif action == \"beaconing\":\n print(json.dumps(detect_beaconing(os.path.join(log_dir, \"conn.log\")), indent=2))\n elif action == \"pcap\" and len(sys.argv) > 2:\n print(json.dumps(analyze_pcap(sys.argv[2]), indent=2))\n else:\n print(\"Usage: agent.py [report|connections|dns|tls|beaconing|pcap \u003cfile>] [log_dir]\")\n","content_type":"text/x-python; charset=utf-8","language":"python","size":11389,"content_sha256":"3d2099f65640c04f2d5be36139ba95a9455fa1e705cbf8e118f86cb698e09785"}],"content_json":{"type":"doc","content":[{"type":"heading","attrs":{"level":1},"content":[{"text":"Detecting Network Anomalies with Zeek","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"When to Use","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Deploying passive network security monitoring at key network choke points for continuous visibility","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Generating structured connection, DNS, HTTP, SSL, and file transfer logs for SIEM ingestion and threat hunting","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Writing custom Zeek scripts to detect organization-specific threats, policy violations, or beaconing behavior","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Performing retrospective analysis on network metadata to investigate security incidents","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Complementing IDS solutions with protocol-level metadata analysis that signature-based tools may miss","type":"text"}]}]}]},{"type":"paragraph","content":[{"text":"Do not use","type":"text","marks":[{"type":"strong"}]},{"text":" as a replacement for inline IDS/IPS that can actively block traffic, for monitoring encrypted payloads without TLS inspection, or on endpoints where host-based agents are more appropriate.","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Prerequisites","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Zeek 6.0+ installed from source or package manager (","type":"text"},{"text":"zeek --version","type":"text","marks":[{"type":"code_inline"}]},{"text":")","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Network interface configured on a span port, network tap, or virtual switch mirror for passive capture","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Sufficient disk storage for log files (estimate 1-5 GB/day per 100 Mbps of monitored traffic)","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Familiarity with Zeek's scripting language for writing custom detections","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Log aggregation system (Splunk, Elastic, Graylog) for centralized analysis","type":"text"}]}]}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Workflow","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Step 1: Install and Configure Zeek","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"bash"},"content":[{"text":"# Install Zeek on Ubuntu/Debian\nsudo apt install -y zeek\n\n# Or install from source for latest version\ngit clone --recursive https://github.com/zeek/zeek\ncd zeek && ./configure --prefix=/opt/zeek && make -j$(nproc) && sudo make install\nexport PATH=/opt/zeek/bin:$PATH\n\n# Configure the monitoring interface\nsudo vi /opt/zeek/etc/node.cfg","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"ini"},"content":[{"text":"# /opt/zeek/etc/node.cfg\n[zeek]\ntype=standalone\nhost=localhost\ninterface=eth1","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"bash"},"content":[{"text":"# Configure local network definitions\nsudo vi /opt/zeek/etc/networks.cfg","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":""},"content":[{"text":"# /opt/zeek/etc/networks.cfg\n10.0.0.0/8 Internal\n172.16.0.0/12 Internal\n192.168.0.0/16 Internal","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"bash"},"content":[{"text":"# Disable NIC offloading for accurate packet capture\nsudo ethtool -K eth1 rx off tx off gro off lro off tso off gso off\n\n# Deploy Zeek\nsudo zeekctl deploy\n\n# Verify Zeek is running\nsudo zeekctl status","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Step 2: Understand and Navigate Zeek Logs","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"bash"},"content":[{"text":"# Zeek generates structured log files in /opt/zeek/logs/current/\nls /opt/zeek/logs/current/\n\n# Key log files:\n# conn.log - All network connections (TCP, UDP, ICMP)\n# dns.log - DNS queries and responses\n# http.log - HTTP requests and responses\n# ssl.log - SSL/TLS handshake details\n# files.log - File transfers observed on the network\n# notice.log - Alerts from Zeek detection scripts\n# weird.log - Protocol anomalies and errors\n# x509.log - X.509 certificate details\n# smtp.log - SMTP email transactions\n# ssh.log - SSH connection details\n\n# View connection log with zeek-cut for column selection\ncat /opt/zeek/logs/current/conn.log | zeek-cut ts id.orig_h id.orig_p id.resp_h id.resp_p proto service duration orig_bytes resp_bytes\n\n# View DNS log\ncat /opt/zeek/logs/current/dns.log | zeek-cut ts id.orig_h query qtype_name answers\n\n# View HTTP log\ncat /opt/zeek/logs/current/http.log | zeek-cut ts id.orig_h host uri method status_code user_agent","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Step 3: Write Custom Detection Scripts","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"bash"},"content":[{"text":"# Create a custom detection script directory\nsudo mkdir -p /opt/zeek/share/zeek/site/custom-detections","type":"text"}]},{"type":"paragraph","content":[{"text":"Create a script for detecting DNS tunneling:","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"zeek"},"content":[{"text":"# /opt/zeek/share/zeek/site/custom-detections/dns-tunneling.zeek\n@load base/frameworks/notice\n\nmodule DNSTunneling;\n\nexport {\n redef enum Notice::Type += {\n DNS_Tunneling_Detected,\n DNS_Long_Query\n };\n\n # Threshold: number of unique queries per source in time window\n const query_threshold: count = 200 &redef;\n const time_window: interval = 5min &redef;\n const max_query_length: count = 50 &redef;\n}\n\n# Track query counts per source IP\nglobal dns_query_counts: table[addr] of count &create_expire=5min &default=0;\n\nevent dns_request(c: connection, msg: dns_msg, query: string, qtype: count, qclass: count)\n{\n local src = c$id$orig_h;\n\n # Check for unusually long domain queries (base64-encoded data)\n if ( |query| > max_query_length )\n {\n NOTICE([\n $note=DNS_Long_Query,\n $msg=fmt(\"Unusually long DNS query from %s: %s (%d chars)\", src, query, |query|),\n $src=src,\n $identifier=cat(src, query)\n ]);\n }\n\n # Track query volume per source\n dns_query_counts[src] += 1;\n\n if ( dns_query_counts[src] == query_threshold )\n {\n NOTICE([\n $note=DNS_Tunneling_Detected,\n $msg=fmt(\"Possible DNS tunneling: %s sent %d queries in %s\", src, query_threshold, time_window),\n $src=src,\n $identifier=cat(src)\n ]);\n }\n}","type":"text"}]},{"type":"paragraph","content":[{"text":"Create a script for detecting beaconing:","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"zeek"},"content":[{"text":"# /opt/zeek/share/zeek/site/custom-detections/beacon-detection.zeek\n@load base/frameworks/notice\n@load base/frameworks/sumstats\n\nmodule BeaconDetection;\n\nexport {\n redef enum Notice::Type += {\n Possible_Beaconing\n };\n\n const beacon_threshold: count = 50 &redef;\n const observation_window: interval = 1hr &redef;\n}\n\nevent zeek_init()\n{\n local r1 = SumStats::Reducer(\n $stream=\"beacon.connections\",\n $apply=set(SumStats::SUM)\n );\n\n SumStats::create([\n $name=\"detect-beaconing\",\n $epoch=observation_window,\n $reducers=set(r1),\n $threshold_val(key: SumStats::Key, result: SumStats::Result) = {\n return result[\"beacon.connections\"]$sum;\n },\n $threshold=beacon_threshold + 0.0,\n $threshold_crossed(key: SumStats::Key, result: SumStats::Result) = {\n NOTICE([\n $note=Possible_Beaconing,\n $msg=fmt(\"Possible beaconing: %s made %d connections in %s\",\n key$str, result[\"beacon.connections\"]$sum, observation_window),\n $identifier=key$str\n ]);\n }\n ]);\n}\n\nevent connection_state_remove(c: connection)\n{\n if ( c$id$resp_h !in Site::local_nets )\n {\n local key = fmt(\"%s->%s:%d\", c$id$orig_h, c$id$resp_h, c$id$resp_p);\n SumStats::observe(\"beacon.connections\", [$str=key], [$num=1]);\n }\n}","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Step 4: Load Custom Scripts and Deploy","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"bash"},"content":[{"text":"# Add custom scripts to local.zeek\nsudo tee -a /opt/zeek/share/zeek/site/local.zeek \u003c\u003c 'EOF'\n\n# Custom detection scripts\n@load custom-detections/dns-tunneling.zeek\n@load custom-detections/beacon-detection.zeek\n\n# Enable additional protocol analyzers\n@load protocols/ftp/software\n@load protocols/http/software\n@load protocols/smtp/software\n@load protocols/ssh/detect-bruteforcing\n@load protocols/ssl/validate-certs\n@load protocols/ssl/log-hostcerts-only\n@load protocols/dns/detect-external-names\n\n# Enable file extraction\n@load frameworks/files/extract-all-files\n\n# Enable Intel framework for threat intelligence\n@load frameworks/intel/seen\n@load frameworks/intel/do_notice\nEOF\n\n# Reload Zeek configuration\nsudo zeekctl deploy\n\n# Verify scripts loaded without errors\nsudo zeekctl diag","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Step 5: Threat Hunting Queries on Zeek Logs","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"bash"},"content":[{"text":"# Find long-duration connections (possible C2)\ncat /opt/zeek/logs/current/conn.log | zeek-cut ts id.orig_h id.resp_h id.resp_p duration | \\\n awk '$5 > 3600 {print $0}' | sort -t

Important: agents should read /llm.txt, /llms.txt, or /.well-known/skills.json to discover the public Skillopedia API.