TAO VCN Classify Gap Analysis Skill Standalone install? If this session was not initialized by the TAO skill bank plugin, run the skill first (host preflight, credentials, cross-skill discovery). You are an analyst for NVIDIA TAO VCN Classify (Visual Component Net) inference results. Your job is to identify the weakest samples per ground-truth label by measuring signed distance from the decision threshold in the wrong direction , then surface them for downstream augmentation or relabeling. This skill is intentionally lightweight. VCN's classify head is a single-score binary boundary (PASS vs…

; then\n warnings=\"${warnings}\\n- BAD THRESHOLD: threshold.txt does not contain a single numeric float (got: $(head -c 60 \"$report_dir/threshold.txt\")).\"\n fi\n fi\n\n # kpi_gaps.parquet should have rows\n if [ -f \"$report_dir/kpi_gaps.parquet\" ]; then\n rows=$(python3 - \"$report_dir/kpi_gaps.parquet\" 2>/dev/null \u003c\u003c 'PYEOF'\nimport sys\ntry:\n import pandas as pd\n df = pd.read_parquet(sys.argv[1])\n expected = {\"filepath\", \"label\", \"siamese_score\", \"weakness\"}\n missing = expected - set(df.columns)\n if missing:\n print(f\"COLUMNS_MISSING:{','.join(sorted(missing))}\")\n else:\n print(f\"ROWS:{len(df)}\")\nexcept Exception as e:\n print(f\"ERROR:{e}\")\nPYEOF\n)\n case \"$rows\" in\n ROWS:0)\n warnings=\"${warnings}\\n- EMPTY PARQUET: kpi_gaps.parquet has 0 rows. Either every sample is correctly classified (suspicious — verify) or the threshold sweep produced no candidates.\"\n ;;\n COLUMNS_MISSING:*)\n cols=${rows#COLUMNS_MISSING:}\n warnings=\"${warnings}\\n- BAD PARQUET SCHEMA: kpi_gaps.parquet missing columns: $cols. Required schema: filepath, label, siamese_score, weakness.\"\n ;;\n ERROR:*)\n warnings=\"${warnings}\\n- UNREADABLE PARQUET: kpi_gaps.parquet failed to load (${rows#ERROR:}).\"\n ;;\n esac\n fi\n\n if [ -n \"$warnings\" ]; then\n echo -e \"VCN ARTIFACT GAPS:$warnings\"\n fi\nfi\n","content_type":"application/x-sh; charset=utf-8","language":"bash","size":3785,"content_sha256":"72d65345c76485c7c7ebd992293796fa1f5514f2752edfd7721dfbd4dcb0c51c"},{"filename":"hooks/rca-label-coverage.sh","content":"#!/bin/bash\n# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.\n# SPDX-License-Identifier: Apache-2.0\n\n# Hook: Verify every ground-truth label found in inference.csv shows up in the report's\n# Weakness Distribution and Top-K tables. VCN labels are typically PASS / NO_PASS but the\n# CSV may use any string convention — derive labels from the data, not from a hardcoded list.\n# Toggle: export RCA_HOOKS=0 to disable\n\n[[ \"${RCA_HOOKS:-1}\" == \"0\" ]] && exit 0\n\nsource \"$(dirname \"$0\")/_parse-stdin.sh\"\n\nif [[ \"$CLAUDE_FILE_PATH\" == *RCA_Report.md ]]; then\n report_dir=$(dirname \"$CLAUDE_FILE_PATH\")\n inference_csv=\"\"\n for cand in \"$report_dir/inference/inference.csv\" \\\n \"$report_dir/../inference/inference.csv\" \\\n \"$report_dir/../../inference/inference.csv\"; do\n [ -f \"$cand\" ] && inference_csv=\"$cand\" && break\n done\n [ -z \"$inference_csv\" ] && exit 0\n\n python3 - \"$inference_csv\" \"$CLAUDE_FILE_PATH\" \u003c\u003c 'PYEOF'\nimport csv, sys, re\n\ninference_csv, report_path = sys.argv[1], sys.argv[2]\n\nlabel_counts = {}\nwith open(inference_csv) as f:\n reader = csv.DictReader(f)\n for row in reader:\n lbl = (row.get('label') or row.get('Label') or '').strip()\n if not lbl:\n continue\n label_counts[lbl] = label_counts.get(lbl, 0) + 1\n\nif not label_counts:\n sys.exit(0)\n\nwith open(report_path) as f:\n report = f.read()\nreport_lower = report.lower()\n\nwarnings = []\nfor lbl, count in sorted(label_counts.items()):\n lbl_lower = lbl.lower()\n if lbl_lower not in report_lower:\n warnings.append(f\"MISSING LABEL: '{lbl}' ({count} samples) not mentioned anywhere in the report.\")\n continue\n\n # Verify the label appears in the Weakness Distribution table (§3) with a numeric column.\n wd_m = re.search(r'Weakness Distribution(.*?)(?=\\n## )', report, re.DOTALL | re.IGNORECASE)\n if wd_m:\n wd = wd_m.group(1)\n row_pat = rf'\\|[^|]*{re.escape(lbl)}[^|]*\\|.*\\d+\\.?\\d*'\n if not re.search(row_pat, wd, re.IGNORECASE):\n warnings.append(f\"NO DISTRIBUTION ROW: '{lbl}' has no row with numeric stats in Weakness Distribution.\")\n\n # Verify the label appears in the Top-K table.\n tk_m = re.search(r'Top-K Weakest(.*?)(?=\\n## )', report, re.DOTALL | re.IGNORECASE)\n if tk_m and lbl_lower not in tk_m.group(1).lower():\n warnings.append(f\"NO TOP-K ROWS: '{lbl}' has no rows in Top-K Weakest Samples.\")\n\n # The total sample count should appear somewhere near the label name.\n found_count = False\n for m in re.finditer(re.escape(lbl), report, re.IGNORECASE):\n nearby = report[max(0, m.start() - 100):m.end() + 200]\n if str(count) in nearby:\n found_count = True\n break\n if not found_count:\n warnings.append(f\"NO COUNT: total sample count ({count}) for '{lbl}' not reported near any mention.\")\n\n# Cross-check: at least PASS and one NO_PASS-equivalent label should be discussed in §5\nspot_m = re.search(r'Visual Spot Check(.*?)(?=\\n## )', report, re.DOTALL | re.IGNORECASE)\nif spot_m:\n spot = spot_m.group(1).lower()\n labels_in_spot = [l for l in label_counts if l.lower() in spot]\n if len(labels_in_spot) \u003c 2:\n warnings.append(f\"SPOT CHECK INCOMPLETE: only {len(labels_in_spot)} label(s) covered in Visual Spot Check (need both PASS and NO_PASS sides).\")\n\nif warnings:\n print(\"VCN LABEL COVERAGE:\")\n for w in warnings:\n print(f\" - {w}\")\nPYEOF\nfi\n","content_type":"application/x-sh; charset=utf-8","language":"bash","size":3490,"content_sha256":"d1a39591bcfcf2cb880f3e4d02608880c41d6b552fc41e47fe8961d987c31d5d"},{"filename":"hooks/rca-package.sh","content":"#!/bin/bash\n# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.\n# SPDX-License-Identifier: Apache-2.0\n\n# Hook: Package RCA output into timestamped folder with all artifacts\n# Trigger: PostToolUse on Write tool when file matches *RCA_Report.md\n# Toggle: export RCA_HOOKS=0 to disable\n#\n# Claude Code passes hook context via stdin as JSON with fields:\n# tool_input.file_path - the file that was written\n# transcript_path - path to current session log\n# session_id - current session ID\n# Env vars available: CLAUDE_PROJECT_DIR, CLAUDE_CODE_ENTRYPOINT\n\n[[ \"${RCA_HOOKS:-1}\" == \"0\" ]] && exit 0\n\nsource \"$(dirname \"$0\")/_parse-stdin.sh\"\n\nlog_file=\"/tmp/rca-hook-debug.log\"\necho \"[$(date)] file_path=$HOOK_FILE_PATH transcript=$HOOK_TRANSCRIPT\" >> \"$log_file\" 2>/dev/null\n\nif [[ \"$CLAUDE_FILE_PATH\" == *RCA_Report.md ]]; then\n report_dir=$(dirname \"$CLAUDE_FILE_PATH\")\n timestamp=$(date +\"%Y-%m-%d_%H%M%S\")\n\n echo \"[$(date)] Hook triggered for: $CLAUDE_FILE_PATH\" >> \"$log_file\" 2>/dev/null\n\n # If already in a timestamped rca_results folder, use it directly\n if [[ \"$report_dir\" == *rca_results/* ]]; then\n out_dir=\"$report_dir\"\n else\n out_dir=\"$report_dir/rca_results/$timestamp\"\n mkdir -p \"$out_dir\"\n cp \"$CLAUDE_FILE_PATH\" \"$out_dir/RCA_Report.md\"\n if [ -d \"$report_dir/rca_images\" ]; then\n cp -r \"$report_dir/rca_images\" \"$out_dir/rca_images\"\n fi\n fi\n\n # Use CLAUDE_PROJECT_DIR (set by Claude Code), fall back to git or PWD\n project_root=\"${CLAUDE_PROJECT_DIR:-$(git rev-parse --show-toplevel 2>/dev/null || echo \"$PWD\")}\"\n\n # Copy RCA config for reproducibility\n mkdir -p \"$out_dir/rca_config\"\n\n for src in skills commands hooks; do\n if [ -d \"$project_root/.claude/$src\" ]; then\n cp -r \"$project_root/.claude/$src\" \"$out_dir/rca_config/$src\" 2>>\"$log_file\"\n fi\n done\n\n for f in \"$project_root/.claude/settings.json\" \"$project_root/.claude/settings.local.json\"; do\n [ -f \"$f\" ] && cp \"$f\" \"$out_dir/rca_config/\" 2>>\"$log_file\"\n done\n\n # Copy session log — use transcript_path from stdin (most reliable)\n if [ -n \"$HOOK_TRANSCRIPT\" ] && [ -f \"$HOOK_TRANSCRIPT\" ]; then\n cp \"$HOOK_TRANSCRIPT\" \"$out_dir/claude_session.jsonl\" 2>>\"$log_file\"\n else\n # Fallback: find session log by project dir encoding\n project_dir_encoded=$(echo \"$project_root\" | sed 's|[/_]|-|g')\n project_sessions_dir=\"$HOME/.claude/projects/$project_dir_encoded\"\n if [ -d \"$project_sessions_dir\" ]; then\n latest_log=$(find \"$project_sessions_dir\" -maxdepth 1 -name '*.jsonl' -printf '%T@ %p\\n' 2>/dev/null \\\n | sort -rn | head -1 | cut -d' ' -f2-)\n if [ -n \"$latest_log\" ] && [ -f \"$latest_log\" ]; then\n cp \"$latest_log\" \"$out_dir/claude_session.jsonl\" 2>>\"$log_file\"\n fi\n fi\n fi\n\n echo \"RCA packaged to: $out_dir\"\nelse\n echo \"[$(date)] Hook skipped (not RCA_Report.md): $CLAUDE_FILE_PATH\" >> \"$log_file\" 2>/dev/null\nfi\n","content_type":"application/x-sh; charset=utf-8","language":"bash","size":2964,"content_sha256":"bf6395b64d69e44b7b0c3fb001cc2ae45f4356016e67d075f126d5e88c45b20b"},{"filename":"hooks/rca-script-check.sh","content":"#!/bin/bash\n# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.\n# SPDX-License-Identifier: Apache-2.0\n\n# Hook: Catch silent Python script failures and validate analysis scripts produce output\n# Parses PostToolUse stdin JSON for exit code and stdout content\n# Toggle: export RCA_HOOKS=0 to disable, RCA_HOOKS=1 to enable (default: enabled)\n\n[[ \"${RCA_HOOKS:-1}\" == \"0\" ]] && exit 0\n\n# Read stdin JSON into variable\n_stdin=$(cat)\n\n# Pass JSON via environment variable (not argv — avoids shell quoting issues with large JSON)\nexport _HOOK_STDIN=\"$_stdin\"\n\npython3 \u003c\u003c 'PYEOF'\nimport json, sys, os\n\nraw = os.environ.get('_HOOK_STDIN', '')\nif not raw:\n sys.exit(0)\n\ntry:\n data = json.loads(raw)\nexcept (json.JSONDecodeError, ValueError):\n sys.exit(0)\n\ntool_name = data.get('tool_name', '')\nif tool_name != 'Bash':\n sys.exit(0)\n\n# Extract fields\ntool_response = data.get('tool_response', {})\nstdout = tool_response.get('stdout', '') or ''\nstderr = tool_response.get('stderr', '') or ''\ncommand = data.get('tool_input', {}).get('command', '')\n\n# Heuristic exit code: check stderr for common error patterns\nhas_error = False\nif stderr.strip():\n error_patterns = ['Traceback', 'Error:', 'error:', 'FAILED', 'fatal:', 'Permission denied']\n has_error = any(p in stderr for p in error_patterns)\n\ncombined = stdout + '\\n' + stderr\nwarnings = []\n\n# Check 0a: docker run failure modes specific to this skill\nimport re\nif re.search(r'\\bdocker\\s+run\\b.*tao-toolkit-ds.*gap_analysis\\s+vcn_aoi', command, re.DOTALL):\n if 'docker: command not found' in combined or re.search(r'docker:\\s*command not found', combined):\n warnings.append(\"`docker` not found on PATH. Install Docker (and the NVIDIA container toolkit) before re-running.\")\n if re.search(r'(unable to find image|pull access denied|manifest unknown|repository does not exist).*tao-toolkit-ds', combined, re.IGNORECASE):\n warnings.append(\"The TAO data-services container (pinned in this skill; resolved from `versions.yaml`) is missing or unreachable. Resolve `DS_IMAGE` from `versions.yaml` (`images.tao_toolkit.data_services`), pre-pull with `docker pull \\\"$DS_IMAGE\\\"`, and confirm registry credentials. The data-services tag declared in versions.yaml is required — the generic `:latest` does not contain the gap-analysis entrypoint.\")\n if re.search(r'(action not found|unknown action|invalid action).*gap_analysis|gap_analysis.*not (found|recognized)', combined, re.IGNORECASE):\n warnings.append(\"Container did not recognize the `gap_analysis vcn_aoi` action. Confirm the image actually resolves from `tao_toolkit.data_services` in `versions.yaml` (not `:latest`) and that the args are passed without a leading `dataset` keyword — the entrypoint takes `\u003ccategory> \u003caction> \u003cargs>` directly.\")\n if re.search(r'(FileNotFoundError|No such file or directory).*\\.(csv|yaml|parquet)', combined):\n warnings.append(\"Container reported a missing input file. Most likely the host path was not mounted into the container. Use `-v $WORKSPACE:$WORKSPACE` so host and container paths match exactly, and confirm `inference_csv`, `train_config`, and `kpi_media_path` all live under $WORKSPACE.\")\n if re.search(r'(could not select device driver.*gpu|no CUDA-capable device)', combined, re.IGNORECASE):\n warnings.append(\"No GPU detected from inside the container. Confirm `nvidia-smi` works on the host AND that `--gpus all` was passed to `docker run`.\")\n\n# Check 0b: Container reported the KPI is unreachable — not a script bug, but worth surfacing\n# so the report is written in abridged form rather than continuing into spot-check.\nif re.search(r'(unreachable.*kpi|no threshold achieves|cannot meet.*recall)', combined, re.IGNORECASE):\n warnings.append(\"Container reports the KPI is UNREACHABLE at any threshold. Skip the visual spot-check and write the abridged report (sections 1, 2, 7 only) recommending retrain or relabel.\")\n\n# Check 1: Traceback in stdout or stderr\nif 'Traceback (most recent call last)' in stdout or 'Traceback (most recent call last)' in stderr:\n warnings.append(\"Python traceback detected — script crashed mid-execution. Fix the error and re-run to get complete results.\")\n\n# Check 2: Python analysis scripts that produce no output (likely silent failure)\nif 'python' in command.lower() and not stdout.strip() and not has_error:\n analysis_keywords = ['print', 'score', 'defect', 'mean', 'count', 'compute', 'analyze', 'statistics']\n if any(kw in command.lower() for kw in analysis_keywords):\n warnings.append(\"Python analysis script produced NO output. It may have silently failed or has a logic error. Check for empty DataFrames, wrong file paths, or swallowed exceptions.\")\n\n# Check 3: Common data analysis red flags in output\nif stdout:\n if 'nan' in stdout.lower() and ('mean' in stdout.lower() or 'score' in stdout.lower()):\n warnings.append(\"NaN values in analysis output. Check for empty groups, division by zero, or missing data.\")\n if 'empty dataframe' in stdout.lower() or 'no rows' in stdout.lower():\n warnings.append(\"Empty DataFrame in output. Likely a filter that matched nothing — check your conditions.\")\n\n# Check 4: stderr warnings that may indicate partial results\nif stderr.strip() and not has_error:\n warn_patterns = ['UserWarning', 'FutureWarning', 'DeprecationWarning']\n real_warnings = [line for line in stderr.splitlines()\n if not any(wp in line for wp in warn_patterns) and line.strip()]\n if real_warnings:\n warnings.append(f\"Unexpected stderr output ({len(real_warnings)} lines). Script may have partial errors.\")\n\nif warnings:\n print(\"SCRIPT ISSUES:\")\n for w in warnings:\n print(f\" - {w}\")\n\nPYEOF\n","content_type":"application/x-sh; charset=utf-8","language":"bash","size":5799,"content_sha256":"24c6920542637a0d5c46ef29a4af9479595cf4b1e79e6c20c10e956d2a2ca0d8"},{"filename":"hooks/rca-section-check.sh","content":"#!/bin/bash\n# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.\n# SPDX-License-Identifier: Apache-2.0\n\n# Hook: Verify the VCN gap analysis report has all 7 required sections with substantive content.\n# Lighter than the ChangeNet equivalent — VCN does not have golden audits, defect types, or\n# component-type clustering, so we check only the sections defined in SKILL.md.\n# Toggle: export RCA_HOOKS=0 to disable\n\n[[ \"${RCA_HOOKS:-1}\" == \"0\" ]] && exit 0\n\nsource \"$(dirname \"$0\")/_parse-stdin.sh\"\n\nif [[ \"$CLAUDE_FILE_PATH\" == *RCA_Report.md ]]; then\n python3 - \"$CLAUDE_FILE_PATH\" \u003c\u003c 'PYEOF'\nimport sys, re\n\nwith open(sys.argv[1]) as f:\n report = f.read()\n\n# (heading_pattern, min_table_rows, [required_keywords])\nchecks = [\n (\"Verdict\", 0, [\"threshold\", \"kpi\", \"weak\"]),\n (\"Threshold Selection\", 4, [\"recall\", \"precision\", \"f1\", \"confusion\"]),\n (\"Weakness Distribution\", 1, [\"mean weakness\", \"misclassified\"]),\n (\"Top-K Weakest\", 5, [\"weakness\", \"siamese_score\"]),\n (\"Visual Spot Check\", 5, [\"![\", \"verdict\"]),\n (\"Per-Label Breakdown\", 0, [\"misclassified\", \"marginal\"]),\n (\"Recommended Actions\", 0, [\"relabel\", \"augment\"]),\n]\n\nwarnings = []\nfor heading, min_rows, kws in checks:\n pat = rf'## .*?{re.escape(heading)}(.*?)(?=\\n## |\\Z)'\n m = re.search(pat, report, re.DOTALL | re.IGNORECASE)\n if not m:\n warnings.append(f\"MISSING SECTION: '{heading}' not found.\")\n continue\n body = m.group(1)\n if min_rows:\n rows = len([l for l in body.splitlines()\n if l.strip().startswith('|') and '---' not in l])\n if rows \u003c min_rows:\n warnings.append(f\"SHALLOW: '{heading}' has only {rows} table rows (need {min_rows}+).\")\n words = len(body.split())\n if words \u003c 40:\n warnings.append(f\"THIN: '{heading}' is only {words} words. Add the actual numbers from the analysis.\")\n missing = [k for k in kws if not re.search(re.escape(k), body, re.IGNORECASE)]\n if missing:\n warnings.append(f\"INCOMPLETE: '{heading}' missing key terms: {', '.join(missing)}\")\n\n# Cross-section: chosen threshold should appear in §1 Verdict and §2 Threshold Selection\nverdict_m = re.search(r'## 1.*?Verdict(.*?)(?=\\n## )', report, re.DOTALL | re.IGNORECASE)\nthr_m = re.search(r'Threshold Selection(.*?)(?=\\n## )', report, re.DOTALL | re.IGNORECASE)\nif verdict_m and thr_m:\n verdict_thr = re.findall(r'threshold[^0-9\\-]*(-?\\d+\\.\\d+)', verdict_m.group(1), re.IGNORECASE)\n sel_thr = re.findall(r'threshold[^0-9\\-]*(-?\\d+\\.\\d+)', thr_m.group(1), re.IGNORECASE)\n if verdict_thr and sel_thr and verdict_thr[0] != sel_thr[0]:\n warnings.append(f\"INCONSISTENT THRESHOLD: Verdict says {verdict_thr[0]} but Threshold Selection says {sel_thr[0]}. The same value must appear in both.\")\n\n# Recommended Actions must reference kpi_gaps.parquet (the headline deliverable)\nrec_m = re.search(r'Recommended Actions(.*)', report, re.DOTALL | re.IGNORECASE)\nif rec_m and 'kpi_gaps.parquet' not in rec_m.group(1).lower():\n warnings.append(\"RECOMMENDATIONS: must reference kpi_gaps.parquet. The augmentation queue is the headline deliverable.\")\n\nif warnings:\n print(\"VCN SECTION ISSUES:\")\n for w in warnings:\n print(f\" - {w}\")\nPYEOF\nfi\n","content_type":"application/x-sh; charset=utf-8","language":"bash","size":3328,"content_sha256":"d17aece59ab2e0ac29e367698cbb24c457728c7b092f3921f27c2834ace2c6d3"},{"filename":"references/container-setup.md","content":"# Container Setup and Path Mounting\n\nThe threshold sweep, weakness ranking, and per-lighting expansion all run inside the pinned TAO data-services image below. Confirm Docker, the NVIDIA container toolkit, and a GPU are present and ensure the image is cached:\n\n```bash\n# Resolve the pinned TAO data-services container URI from the release manifest.\nDS_IMAGE=\"$(scripts/resolve_versions_key.py images.tao_toolkit.data_services)\"\necho \"DS_IMAGE=$DS_IMAGE\"\n\ndocker info > /dev/null && echo \"OK: docker\"\nnvidia-smi > /dev/null && echo \"OK: GPU\"\ndocker image inspect \"$DS_IMAGE\" > /dev/null \\\n || docker pull \"$DS_IMAGE\"\n```\n\nA GPU is required (the same image is used across the AOI loop and other actions assume CUDA is present). Aborting early on a GPU-less host saves a confusing late error.\n\n**Path mounting.** Every host path the container reads or writes — `inference.csv`, the train YAML, the dataset image root, and the output dir — must be bind-mounted. The simplest pattern is to mount the workspace root with **identical paths** inside and outside the container so absolute paths in args resolve the same on both sides:\n\n```bash\nWORKSPACE=\u003cabsolute path that contains inference.csv, train YAML, dataset images, and the output dir>\nDOCKER=\"docker run --gpus all --rm --shm-size=8g -v $WORKSPACE:$WORKSPACE -w $WORKSPACE $DS_IMAGE\"\n```\n\n**Do not pass `--user $(id -u):$(id -g)`.** The container imports `transformers` at startup, which lazy-loads a module path that ends up in `getpass.getuser()` → `pwd.getpwuid(os.getuid())`. With a non-root host UID and no matching `/etc/passwd` entry inside the container, this raises `KeyError: 'getpwuid(): uid not found: \u003cuid>'` *before* `gap_analysis` runs. Drop the flag and `chown` the outputs back to the host UID afterwards if you need them host-writable:\n\n```bash\ndocker run --rm -v \"$WORKSPACE:/w\" alpine chown -R $(id -u):$(id -g) /w/\u003cresults_subdir>\n```\n\nIf `inference.csv`, the train YAML, and the dataset images live in different roots, pass multiple `-v` flags — but every absolute path you pass in args must resolve inside the container.\n\n**`-e \u003cspec>` is required, not optional.** Older revisions of this doc described `-e \u003cspec>` as a convenience and Hydra CLI overrides as sufficient on their own. In current `tao_toolkit.data_services` images (verified on `7.0.0-rc-180-multiarch` and later) the entrypoint hard-requires the flag and exits with `ValueError: The subtask vcn_aoi requires the following argument: -e/--experiment_spec_file` before parsing any CLI override. Always pass a minimal spec file — even a thin YAML containing just `min_recall` / `top_k_per_label` / `threshold` is enough; the CLI overrides win at merge time. The `## Reference invocation` block writes this file once per run.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":2773,"content_sha256":"03ca26844c182c7df0ec796896ab5e72bef0425e8f36aaa6010c4c2c6f85fb46"},{"filename":"references/output-template.md","content":"# Output Report Structure\n\nKeep the report tight (1000–1800 words). This is a computational gap analysis, not a deep RCA — depth comes from accurate numbers and a clear action list, not narrative.\n\n```\n# VCN Gap Analysis Report: \u003cExperiment Name>\n\n## 1. Verdict\n- Chosen threshold: \u003cvalue> (achieves precision=\u003cp>, recall=\u003cr>, F1=\u003cf1> on NO_PASS at recall ≥ \u003cKPI>)\n- KPI reachability: \u003cyes/no — and the recall it actually achieves>\n- Total samples: \u003cN> | Total weak samples kept: \u003cK> | Misclassified: \u003cM>\n- Top-3 labels by misclassification share\n- One-line headline: \"\u003cK> weak samples written to kpi_gaps.parquet for augmentation\"\n\n## 2. Threshold Selection\n- Target NO_PASS recall: \u003cKPI>\n- Candidates evaluated: \u003ccount>; candidates meeting recall target: \u003ccount>\n- Chosen threshold and tie-break reasoning (best F1 → precision → threshold)\n- Confusion matrix at chosen threshold (calculate from the full `inference.csv`\n using `siamese_score >= threshold` as predicted NO_PASS; a NO_PASS sample tied\n at the threshold is correctly classified and has zero weakness):\n\n| | Predicted NO_PASS | Predicted PASS |\n|--|--|--|\n| Actual NO_PASS | TP=… | FN=… |\n| Actual PASS | FP=… | TN=… |\n\n## 3. Weakness Distribution\n| Label | Total Samples | Mean Weakness | Median Weakness | Max Weakness | # Misclassified |\n|-------|---------------|----------------|------------------|---------------|------------------|\n\n(One row per ground-truth label across the FULL inference CSV — calculate these\nstatistics directly from `inference.csv` and the chosen `threshold.txt`, not just\nthe kept K. Signed weakness is `siamese_score - threshold` for PASS and\n`threshold - siamese_score` for NO_PASS, so positive values are misclassified.)\n\n## 4. Top-K Weakest Samples (per label)\n| Label | object_name | input_path | siamese_score | weakness | misclassified? |\n|-------|-------------|-------------|----------------|-----------|-----------------|\n\n(Up to top_k_per_label rows per label group. Sorted by weakness descending within each group.\nRead from `kpi_gaps.parquet`, deduplicated to one row per (input_path, object_name) —\n`kpi_gaps.parquet` is per-lighting, but the table is per-sample.)\n\n## 5. Visual Spot Check (10 samples)\n| Label | object_name | siamese_score | weakness | Test Image | Verdict |\n|-------|-------------|----------------|-----------|-------------|----------|\n\n(5 weakest PASS + 5 weakest NO_PASS. `Test Image` column is `![](rca_images/\u003cfilename>)`. `Verdict` is one of: mislabeled / edge case / data quality / systematic.)\n\n## 6. Per-Label Breakdown\n(Render the contents of `weak_samples_breakdown.txt` here.)\n\n## 7. Recommended Actions\n1. **Relabel** — list every sample tagged `mislabeled` in section 5. Path is `{input_path}/{object_name}` in `inference.csv`.\n2. **Augment** — `kpi_gaps.parquet` (`\u003cK> rows × \u003cL> lightings = \u003cK*L> filepaths`) is the augmentation queue. Pass it to `tao-route-visual-changenet-samples` next.\n3. **Threshold action** — recommend whether to (a) retrain with current data and re-run this skill, (b) lower the recall target if the visual spot check shows the misclassified samples are genuinely ambiguous, or (c) ship at the current threshold if KPI is met.\n4. **Systematic failures** — if any visual spot-check sample is tagged `systematic`, flag the failure mode (which lighting? which component family?) for model architecture review.\n```\n\nWhen `unreachable_kpi.txt` exists, replace sections 3–6 with a single short section quoting that file's contents and stating the model cannot meet the KPI at any threshold. Section 7 then collapses to one recommendation: retrain or relabel.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":3664,"content_sha256":"da6e7c650213c015ebebfaf7fd611f46eadfdde39963f3e7177c893842329522"},{"filename":"references/pitfalls.md","content":"# Common Pitfalls\n\n- **Forgetting `top_k_per_label` when `min_recall=1.0`** — the most consequential failure mode of this skill. At `min_recall=1.0` the chosen threshold sits at or below every NO_PASS sample's score (so recall=100% by construction means there are NO false negatives). Without `top_k_per_label`, the container falls back to a \"samples below threshold\" filter, which at this threshold matches ONLY misclassified PASS rows (false positives) — `kpi_gaps.parquet` ends up containing zero NO_PASS rows and the augmentation queue is broken. **Always include an explicit positive `top_k_per_label`** in `vcn_aoi_spec.yaml` (default 50), or pass it as a Hydra override, so the container ranks by signed weakness and returns the K weakest *per label*.\n- **Passing `--user $(id -u):$(id -g)` to the container** — kills the run with `KeyError: 'getpwuid(): uid not found: \u003cuid>'` during `transformers`' lazy import chain, before `gap_analysis` even starts. The image has no `/etc/passwd` entry for non-root host UIDs. Drop the `--user` flag; chown outputs back to the host UID via a small `alpine` container afterwards if needed. See *Setup → Path mounting*.\n- **Calling the entrypoint with only Hydra CLI overrides (no `-e \u003cspec>`)** — current data-services images (`tao_toolkit.data_services` 7.0.0-rc-180-multiarch+) hard-require the flag and exit with `ValueError: The subtask vcn_aoi requires the following argument: -e/--experiment_spec_file` before parsing CLI overrides. Always pass a minimal spec file via `-e`; CLI overrides win at merge time but cannot replace the file itself. See *Setup → `-e \u003cspec>` is required*.\n- **Spec file outside `$WORKSPACE`** — `-e \u003cpath>` is resolved inside the container, so `vcn_aoi_spec.yaml` must live under the bind-mounted workspace. Place it next to the other run artifacts (the recipe puts it inside the timestamped output dir) and pass an absolute path.\n- **Spec file with unresolved `???` sentinels** — the bundled defaults under `experiment_specs/vcn_aoi.yaml` mark required fields with `???`. Replace every `???` before the run, or supply that field as a Hydra override on the CLI. Hydra rejects unresolved sentinels with a clear `MissingMandatoryValue` error.\n- **Image not pulled / wrong tag** — use the pinned `DS_IMAGE` (see Setup) and `docker pull \"$DS_IMAGE\"` before the run. The data-services tag declared there is required; the generic `:latest` does not contain the AOI gap-analysis entrypoint, and the docker run will fail with `gap_analysis: action not found` or similar.\n- **Path-mount mismatch** — every absolute path passed in args (`-e` spec, `inference_csv`, `train_config`, `kpi_media_path`, `results_dir`) must resolve inside the container. Use `-v $WORKSPACE:$WORKSPACE` so host and container paths match exactly. If you mount under a different in-container root, pass the in-container path in the args.\n- **`unreachable_kpi.txt` written** — the model fundamentally cannot reach the requested NO_PASS recall at any threshold. Do NOT proceed to the visual spot-check; write the abridged report and recommend retrain or relabeling.\n- **`inference.csv` missing required columns** — container fails fast with a column-name error. Required: `input_path`, `object_name`, `label`, `siamese_score`. Re-run TAO VCN Classify inference if columns are absent.\n- **Train YAML missing `dataset.classify.input_map` or `image_ext`** — per-lighting expansion fails. Confirm the train YAML actually came from the matching VCN Classify experiment.\n- **`kpi_media_path` doesn't match `input_path` prefixes** — `kpi_gaps.parquet` ships with non-existent filepaths. Sanity-check a few rows on disk after the docker call returns and before the visual spot-check.\n- **No GPU detected from inside the container** — confirm `nvidia-smi` works on the host AND that `--gpus all` was passed to `docker run`. Without it, the container errors late.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":3924,"content_sha256":"dd0ae091e3e1e2d3002d37ffbd4d56ed9bcbc8061bbc6b13bd6e2bff4dc7fed4"},{"filename":"references/visual-spot-check.md","content":"# Visual Spot Check\n\nSkip this step if `unreachable_kpi.txt` exists in `results_dir` — there is nothing meaningful to spot-check when the model can't reach the KPI at any threshold.\n\nOtherwise, use the Read tool to **view** the test images for:\n\n- The 5 weakest PASS samples (the top of the \"PASS misclassified as NO_PASS\" pile) — pick by sorting `kpi_gaps.parquet` rows where `label == 'PASS'` by `weakness` descending.\n- The 5 weakest NO_PASS samples (the top of the \"NO_PASS misclassified as PASS\" pile) — same, with `label != 'PASS'`.\n\n`kpi_gaps.parquet` is already expanded per-lighting (multiple rows per sample). For the spot check, deduplicate to one row per (input_path, object_name) — pick the row whose `filepath` uses the FIRST lighting from the train YAML (one image per sample is enough — VCN's classify head sees all lightings stacked, but for human spot-check one is representative).\n\nClassify each viewed sample as exactly one of:\n- **mislabeled** — visual content disagrees with the CSV label\n- **edge case** — genuinely ambiguous boundary case\n- **data quality** — corrupted, dark, wrong crop, bad framing\n- **systematic** — model has learned the wrong feature (the image looks \"obviously PASS/NO_PASS\" but the model disagrees)\n\nCopy each viewed image (resized to 128×128 if PIL is available, otherwise just copy) into `\u003cresults_dir>/rca_images/` so it can be embedded inline in the report.\n\nThis is the **only** image inspection required. Do not view dozens of images, do not run failure mode clustering, do not audit goldens — VCN does not have golden images.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1602,"content_sha256":"1877da18cb17eba8741f64b7317db1f948c6bca005a073b5cb4632542462f727"},{"filename":"skill-card.md","content":"## Description: \u003cbr>\nPerforms gap analysis on NVIDIA TAO VCN Classify (Visual Component Net) experiments by invoking the pinned TAO data-services container directly via docker run, picking the optimal decision threshold, ranking per-sample weakness, and emitting a top-K weakest parquet expanded per-lighting for downstream augmentation. \u003cbr>\n\nThis skill is ready for commercial/non-commercial use. \u003cbr>\n\n## Owner\nNVIDIA \u003cbr>\n\n### License/Terms of Use: \u003cbr>\nApache 2.0 \u003cbr>\n## Use Case: \u003cbr>\nDevelopers and engineers use this skill to analyze VCN classification failures, pick SDA augmentation targets, or audit PASS/NO_PASS boundary cases in NVIDIA TAO Visual Component Net experiments. \u003cbr>\n\n### Deployment Geography for Use: \u003cbr>\nGlobal \u003cbr>\n\n## Requirements / Dependencies: \u003cbr>\n**Requires API Key or External Credential:** [Not Specified] \u003cbr>\n**Credential Type(s):** [None identified] \u003cbr>\n\nDo not include secrets in prompts/logs/output; use least-privilege credentials; rotate keys as appropriate. \u003cbr>\n\n## Known Risks and Mitigations: \u003cbr>\nRisk: Review before execution as proposals could introduce incorrect or misleading guidance into skills. \u003cbr>\nMitigation: Review and scan skill before deployment. \u003cbr>\n\n## Reference(s): \u003cbr>\n- [Container Setup and Path Mounting](references/container-setup.md) \u003cbr>\n- [Output Report Structure](references/output-template.md) \u003cbr>\n- [Common Pitfalls](references/pitfalls.md) \u003cbr>\n- [Visual Spot Check](references/visual-spot-check.md) \u003cbr>\n- [NVIDIA TAO Skill Bank](https://github.com/NVIDIA-TAO/tao-skill-bank) \u003cbr>\n\n\n## Skill Output: \u003cbr>\n**Output Type(s):** [Analysis, Files, Shell commands] \u003cbr>\n**Output Format:** [Markdown report (RCA_Report.md) with parquet and text artifacts] \u003cbr>\n**Output Parameters:** [1D] \u003cbr>\n**Other Properties Related to Output:** [None] \u003cbr>\n\n## Evaluation Agents Used: \u003cbr>\n- Claude Code (`aws/anthropic/bedrock-claude-opus-4-8`) \u003cbr>\n- Codex (`openai/openai/gpt-5.5`) \u003cbr>\n\n\n\n## Evaluation Tasks: \u003cbr>\n1 evaluation task (1 positive), 1 attempt per task, each attempt run in an isolated k8s-sandbox pod. \u003cbr>\n\n## Evaluation Metrics Used: \u003cbr>\nReported benchmark dimensions: \u003cbr>\n- Security: Is it safe to use? Scored from the `security` signal. \u003cbr>\n- Correctness: Is the answer correct? Scored from the `accuracy` signal. \u003cbr>\n- Discoverability: Was the right skill loaded when needed? Scored from the `skill_execution` signal. \u003cbr>\n- Effectiveness: Did the skill help complete the task? Equal-weight mean of `goal_accuracy` and `behavior_check`. \u003cbr>\n- Efficiency: Did it avoid wasted tool calls and token usage? 50% `skill_efficiency` plus 50% `token_efficiency`. \u003cbr>\n\nUnderlying evaluation signals used in this run: \u003cbr>\n- `security`: Unsafe operations, secret leakage, and unauthorized access. \u003cbr>\n- `skill_execution`: Whether the expected skill was selected, decoys were avoided, and the workflow executed. \u003cbr>\n- `skill_efficiency`: Tool-call productivity (legacy wire id; routing is scored under Discoverability). \u003cbr>\n- `accuracy`: Final-answer correctness against the reference answer. \u003cbr>\n- `goal_accuracy`: Whether the user's goal was achieved. \u003cbr>\n- `behavior_check`: Whether the expected workflow behavior was followed. \u003cbr>\n- `token_efficiency`: Actual uncached prompt plus completion usage (50% of Efficiency). \u003cbr>\n\n\n\n## Evaluation Results: \u003cbr>\n| Measure | Claude Code (Baseline → Skill Uplift) | Codex (Baseline → Skill Uplift) |\n|---|---:|---:|\n| Overall | 99.5% — baseline ran, but no comparable score was available; uplift unavailable | 75.7% — baseline ran, but no comparable score was available; uplift unavailable |\n| Security | 100.0% → 100.0% (±0.0 points) | 100.0% → 100.0% (±0.0 points) |\n| Correctness | 0.0% → 100.0% (+100.0 points) | 40.0% → 100.0% (+60.0 points) |\n| Discoverability | 100.0% — baseline ran, but no comparable score was available; uplift unavailable | 0.0% — baseline ran, but no comparable score was available; uplift unavailable |\n| Effectiveness | 16.7% → 100.0% (+83.3 points) | 48.3% → 80.0% (+31.7 points) |\n| Efficiency | 97.4% — baseline ran, but no comparable score was available; uplift unavailable | 99.7% → 98.4% (-1.3 points) |\n\n## Skill Version(s): \u003cbr>\n0.1.0 (source: frontmatter) \u003cbr>\n\n## Ethical Considerations: \u003cbr>\nNVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal team to ensure this skill meets requirements for the relevant industry and use case and addresses unforeseen product misuse. \u003cbr>\n\n(For Release on NVIDIA Platforms Only) \u003cbr>\nPlease report quality, risk, security vulnerabilities or NVIDIA AI Concerns [here](https://app.intigriti.com/programs/nvidia/nvidiavdp/detail). \u003cbr>\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":4893,"content_sha256":"31a573cd6e9df2cf82ae110eaf33722a10cb5f5749770880b931da9ae92c396e"},{"filename":"tests/test_rca_artifacts_hook.py","content":"# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.\n# SPDX-License-Identifier: Apache-2.0\n\nfrom __future__ import annotations\n\nimport json\nimport os\nimport pathlib\nimport subprocess\nimport tempfile\nimport unittest\n\nimport pyarrow as pa\nimport pyarrow.parquet as pq\n\n\nSKILL_ROOT = pathlib.Path(__file__).resolve().parents[1]\nHOOK = SKILL_ROOT / \"hooks\" / \"rca-artifacts-check.sh\"\n\n\nclass RcaArtifactsHookTests(unittest.TestCase):\n def setUp(self) -> None:\n self.temporary = tempfile.TemporaryDirectory()\n self.report_dir = pathlib.Path(self.temporary.name)\n\n pq.write_table(\n pa.table(\n {\n \"filepath\": [\"sample.jpg\"],\n \"label\": [\"PASS\"],\n \"siamese_score\": [0.25],\n \"weakness\": [-0.25],\n }\n ),\n self.report_dir / \"kpi_gaps.parquet\",\n )\n (self.report_dir / \"threshold.txt\").write_text(\"0.5\\n\", encoding=\"utf-8\")\n (self.report_dir / \"weak_samples_breakdown.txt\").write_text(\n \"PASS: 1 (100.0%) — 0 misclassified, 1 marginal\\n\",\n encoding=\"utf-8\",\n )\n (self.report_dir / \"RCA_Report.md\").write_text(\n \"# VCN Gap Analysis Report\\n\",\n encoding=\"utf-8\",\n )\n images = self.report_dir / \"rca_images\"\n images.mkdir()\n for index in range(10):\n (images / f\"sample_{index:02d}.jpg\").write_bytes(b\"fixture image\")\n\n self.assertEqual(\n {entry.name for entry in self.report_dir.iterdir()},\n {\n \"kpi_gaps.parquet\",\n \"threshold.txt\",\n \"weak_samples_breakdown.txt\",\n \"RCA_Report.md\",\n \"rca_images\",\n },\n )\n\n def tearDown(self) -> None:\n self.temporary.cleanup()\n\n def run_hook(self) -> str:\n environment = os.environ.copy()\n environment[\"RCA_HOOKS\"] = \"1\"\n payload = {\n \"tool_name\": \"Write\",\n \"tool_input\": {\"file_path\": str(self.report_dir / \"RCA_Report.md\")},\n }\n result = subprocess.run(\n [\"bash\", str(HOOK)],\n input=json.dumps(payload),\n text=True,\n capture_output=True,\n check=False,\n env=environment,\n )\n self.assertEqual(result.returncode, 0, result.stderr)\n self.assertEqual(result.stderr, \"\")\n return result.stdout\n\n def test_complete_container_and_agent_contract_has_no_warnings(self) -> None:\n self.assertEqual(self.run_hook(), \"\")\n\n def test_missing_artifact_warning_does_not_require_metrics_json(self) -> None:\n (self.report_dir / \"threshold.txt\").unlink()\n\n output = self.run_hook()\n\n self.assertIn(\"MISSING ARTIFACT: threshold.txt\", output)\n self.assertNotIn(\"metrics.json\", output)\n\n\nif __name__ == \"__main__\":\n unittest.main()\n","content_type":"text/x-python; charset=utf-8","language":"python","size":2984,"content_sha256":"847ab714dfb0ec6e7f79cd2eb8c7e06c1a5df5c5cc94e988d1581ae9e8ad854e"}],"content_json":{"type":"doc","content":[{"type":"heading","attrs":{"level":1},"content":[{"text":"TAO VCN Classify Gap Analysis Skill","type":"text"}]},{"type":"blockquote","content":[{"type":"paragraph","content":[{"text":"Standalone install?","type":"text","marks":[{"type":"strong"}]},{"text":" If this session was not initialized by the TAO skill bank plugin, run the ","type":"text"},{"text":"tao-setup","type":"text","marks":[{"type":"code_inline"}]},{"text":" skill first (host preflight, credentials, cross-skill discovery).","type":"text"}]}]},{"type":"paragraph","content":[{"text":"You are an analyst for NVIDIA TAO VCN Classify (Visual Component Net) inference results. Your job is to identify the ","type":"text"},{"text":"weakest samples per ground-truth label","type":"text","marks":[{"type":"strong"}]},{"text":" by measuring signed distance from the decision threshold ","type":"text"},{"text":"in the wrong direction","type":"text","marks":[{"type":"em"}]},{"text":", then surface them for downstream augmentation or relabeling.","type":"text"}]},{"type":"paragraph","content":[{"text":"This skill is intentionally lightweight. VCN's classify head is a single-score binary boundary (PASS vs NO_PASS by ","type":"text"},{"text":"siamese_score","type":"text","marks":[{"type":"code_inline"}]},{"text":"), so the analysis is computational, not investigative. The whole computation lives behind one direct ","type":"text"},{"text":"docker run","type":"text","marks":[{"type":"code_inline"}]},{"text":" invocation against the pinned TAO data-services image (see Setup). The container's entrypoint takes ","type":"text"},{"text":"\u003ccategory> \u003caction> [hydra overrides...]","type":"text","marks":[{"type":"code_inline"}]},{"text":"; we pass ","type":"text"},{"text":"gap_analysis vcn_aoi key=value …","type":"text","marks":[{"type":"code_inline"}]},{"text":". Each override is a bare Hydra ","type":"text"},{"text":"key=value","type":"text","marks":[{"type":"code_inline"}]},{"text":" that selectively overrides the script's ","type":"text"},{"text":"GapAnalysisConfig","type":"text","marks":[{"type":"code_inline"}]},{"text":" schema (defaults are baked into the container; use the mounted minimal-spec ","type":"text"},{"text":"--cfg=job","type":"text","marks":[{"type":"code_inline"}]},{"text":" recipe immediately below to introspect them). (There is no ","type":"text"},{"text":"dataset","type":"text","marks":[{"type":"code_inline"}]},{"text":" keyword inside the container — that's the TAO launcher's pillar prefix and is dropped here.) You do ","type":"text"},{"text":"not","type":"text","marks":[{"type":"strong"}]},{"text":" need delegated analysis, multi-phase image audits, or component-type clustering — VCN does not expose those dimensions. View only a small set of representative weak samples to qualify the gaps after the container returns.","type":"text"}]},{"type":"paragraph","content":[{"text":"CLI surface can shift between data-services container builds. If a ","type":"text"},{"text":"gap_analysis vcn_aoi","type":"text","marks":[{"type":"code_inline"}]},{"text":" invocation fails on argument parsing, introspect the actual schema once per image with a minimal spec on a bind-mounted host path:","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"bash"},"content":[{"text":"SPEC_DIR=\"$(mktemp -d)\"\nprintf '%s\\n' 'min_recall: 0.99' 'top_k_per_label: 5' > \"$SPEC_DIR/vcn_aoi_spec.yaml\"\ndocker run --rm -v \"$SPEC_DIR:/w:ro\" \"$DS_IMAGE\" gap_analysis vcn_aoi \\\n -e /w/vcn_aoi_spec.yaml --cfg=job","type":"text"}]},{"type":"paragraph","content":[{"text":"The ","type":"text"},{"text":"-e","type":"text","marks":[{"type":"code_inline"}]},{"text":" path must resolve inside the container, so the spec must live under the directory mounted at ","type":"text"},{"text":"/w","type":"text","marks":[{"type":"code_inline"}]},{"text":". Reconcile any renamed keys (e.g. ","type":"text"},{"text":"inference_csv","type":"text","marks":[{"type":"code_inline"}]},{"text":" vs ","type":"text"},{"text":"inference_results_dir","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"output_dir","type":"text","marks":[{"type":"code_inline"}]},{"text":" vs ","type":"text"},{"text":"results_dir","type":"text","marks":[{"type":"code_inline"}]},{"text":") before retrying. Output parquet name is ","type":"text"},{"text":"kpi_gaps.parquet","type":"text","marks":[{"type":"code_inline"}]},{"text":".","type":"text"}]},{"type":"hr","attrs":{"markup":"---"}},{"type":"heading","attrs":{"level":2},"content":[{"text":"Inputs","type":"text"}]},{"type":"ordered_list","attrs":{"order":1,"listStyle":"number"},"content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Experiment result directory","type":"text","marks":[{"type":"strong"}]},{"text":" — contains ","type":"text"},{"text":"inference/inference.csv","type":"text","marks":[{"type":"code_inline"}]},{"text":" from TAO VCN Classify inference. Required columns: ","type":"text"},{"text":"input_path","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"object_name","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"label","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"siamese_score","type":"text","marks":[{"type":"code_inline"}]},{"text":". Pass the ","type":"text"},{"text":"directory","type":"text","marks":[{"type":"strong"}]},{"text":" (e.g. ","type":"text"},{"text":"inference/latest/","type":"text","marks":[{"type":"code_inline"}]},{"text":"), not the CSV file — the container reads ","type":"text"},{"text":"inference_results_dir/inference.csv","type":"text","marks":[{"type":"code_inline"}]},{"text":".","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Training code/config directory","type":"text","marks":[{"type":"strong"}]},{"text":" — contains the VCN train YAML. The container reads ","type":"text"},{"text":"dataset.classify.input_map","type":"text","marks":[{"type":"code_inline"}]},{"text":" (lighting condition list) and ","type":"text"},{"text":"dataset.classify.image_ext","type":"text","marks":[{"type":"code_inline"}]},{"text":" from it to expand each weak sample into one row per lighting.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Dataset directory","type":"text","marks":[{"type":"strong"}]},{"text":" — image root prepended to the relative ","type":"text"},{"text":"input_path","type":"text","marks":[{"type":"code_inline"}]},{"text":" from each row (","type":"text"},{"text":"kpi_media_path","type":"text","marks":[{"type":"code_inline"}]},{"text":").","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Schema overrides","type":"text","marks":[{"type":"strong"}]},{"text":" — ","type":"text"},{"text":"min_recall","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"top_k_per_label","type":"text","marks":[{"type":"code_inline"}]},{"text":", and optionally a hard-pinned ","type":"text"},{"text":"threshold","type":"text","marks":[{"type":"code_inline"}]},{"text":" are passed as Hydra overrides (defaults: ","type":"text"},{"text":"min_recall=1.0","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"top_k_per_label=50","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"threshold=-1.0","type":"text","marks":[{"type":"code_inline"}]},{"text":" meaning sweep). ","type":"text"},{"text":"top_k_per_label","type":"text","marks":[{"type":"code_inline"},{"type":"strong"}]},{"text":" must be a positive integer","type":"text","marks":[{"type":"strong"}]},{"text":" — omitting it flips the container into \"below-threshold filter\" mode, which at ","type":"text"},{"text":"min_recall=1.0","type":"text","marks":[{"type":"code_inline"}]},{"text":" returns only PASS misclassifications and zero NO_PASS rows. See Common pitfalls.","type":"text"}]}]}]},{"type":"hr","attrs":{"markup":"---"}},{"type":"heading","attrs":{"level":2},"content":[{"text":"Setup","type":"text"}]},{"type":"paragraph","content":[{"text":"The threshold sweep, weakness ranking, and per-lighting expansion all run inside the pinned TAO data-services image below. Confirm Docker, the NVIDIA container toolkit, and a GPU are present and ensure the image is cached:","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"bash"},"content":[{"text":"# Resolve the pinned TAO data-services container URI from the release manifest.\nDS_IMAGE=\"$(scripts/resolve_versions_key.py images.tao_toolkit.data_services)\"\necho \"DS_IMAGE=$DS_IMAGE\"\n\ndocker info > /dev/null && echo \"OK: docker\"\nnvidia-smi > /dev/null && echo \"OK: GPU\"\ndocker image inspect \"$DS_IMAGE\" > /dev/null \\\n || docker pull \"$DS_IMAGE\"","type":"text"}]},{"type":"paragraph","content":[{"text":"A GPU is required; aborting early on a GPU-less host saves a confusing late error.","type":"text"}]},{"type":"paragraph","content":[{"text":"Three setup rules are load-bearing and easy to get wrong:","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Path mounting","type":"text","marks":[{"type":"strong"}]},{"text":" — every host path the container reads or writes (","type":"text"},{"text":"inference.csv","type":"text","marks":[{"type":"code_inline"}]},{"text":", train YAML, dataset image root, output dir) must be bind-mounted, simplest with ","type":"text"},{"text":"-v $WORKSPACE:$WORKSPACE -w $WORKSPACE","type":"text","marks":[{"type":"code_inline"}]},{"text":" so absolute paths resolve identically on both sides.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Do not pass ","type":"text","marks":[{"type":"strong"}]},{"text":"--user $(id -u):$(id -g)","type":"text","marks":[{"type":"code_inline"},{"type":"strong"}]},{"text":" — it triggers ","type":"text"},{"text":"KeyError: 'getpwuid(): uid not found: \u003cuid>'","type":"text","marks":[{"type":"code_inline"}]},{"text":" during the container's ","type":"text"},{"text":"transformers","type":"text","marks":[{"type":"code_inline"}]},{"text":" import; ","type":"text"},{"text":"chown","type":"text","marks":[{"type":"code_inline"}]},{"text":" outputs back to the host UID afterwards instead.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"-e \u003cspec>","type":"text","marks":[{"type":"code_inline"},{"type":"strong"}]},{"text":" is required, not optional","type":"text","marks":[{"type":"strong"}]},{"text":" — current images hard-require it and exit with ","type":"text"},{"text":"ValueError: The subtask vcn_aoi requires the following argument: -e/--experiment_spec_file","type":"text","marks":[{"type":"code_inline"}]},{"text":" before parsing CLI overrides.","type":"text"}]}]}]},{"type":"paragraph","content":[{"text":"See ","type":"text"},{"text":"references/container-setup.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" for the full path-mounting pattern, the ","type":"text"},{"text":"--user","type":"text","marks":[{"type":"code_inline"}]},{"text":"/","type":"text"},{"text":"chown","type":"text","marks":[{"type":"code_inline"}]},{"text":" rationale and ","type":"text"},{"text":"alpine","type":"text","marks":[{"type":"code_inline"}]},{"text":" chown command, multi-","type":"text"},{"text":"-v","type":"text","marks":[{"type":"code_inline"}]},{"text":" guidance, and the ","type":"text"},{"text":"-e \u003cspec>","type":"text","marks":[{"type":"code_inline"}]},{"text":" requirement detail.","type":"text"}]},{"type":"hr","attrs":{"markup":"---"}},{"type":"heading","attrs":{"level":2},"content":[{"text":"Method","type":"text"}]},{"type":"paragraph","content":[{"text":"The whole skill is a single ","type":"text"},{"text":"docker run","type":"text","marks":[{"type":"code_inline"}]},{"text":" invocation followed by a small visual spot-check. The container does Steps 1–4 internally (threshold sweep, weakness scoring, top-K selection, per-lighting expansion). You handle Step 5 (visual spot-check) directly with the Read tool.","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Step 1–4 — Run the container","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"bash"},"content":[{"text":"$DOCKER gap_analysis vcn_aoi \\\n inference_results_dir=\u003cexp_dir>/inference/\u003clabel>/ \\\n train_config=\u003cexp_dir>/train.yaml \\\n kpi_media_path=\u003cdataset_root> \\\n results_dir=\u003crca_results_dir> \\\n top_k_per_label=50","type":"text"}]},{"type":"blockquote","content":[{"type":"paragraph","content":[{"text":"Always pass ","type":"text","marks":[{"type":"strong"}]},{"text":"top_k_per_label","type":"text","marks":[{"type":"code_inline"},{"type":"strong"}]},{"text":".","type":"text","marks":[{"type":"strong"}]},{"text":" This is the argument that switches the container from the default \"samples below threshold\" filter into proper top-K-per-label ranking. At ","type":"text"},{"text":"min_recall=1.0","type":"text","marks":[{"type":"code_inline"}]},{"text":" the threshold is by construction at-or-below every NO_PASS score, so the below-threshold filter returns ONLY misclassified PASS rows and zero NO_PASS rows — useless as an augmentation queue. With ","type":"text"},{"text":"top_k_per_label","type":"text","marks":[{"type":"code_inline"}]},{"text":" set to a positive integer (either in the spec or as a Hydra override), the container computes signed weakness against the threshold for every row and surfaces the K weakest ","type":"text"},{"text":"per ground-truth label","type":"text","marks":[{"type":"strong"}]},{"text":", which is the per-label ranked output downstream steps consume.","type":"text"}]}]},{"type":"paragraph","content":[{"text":"Reads ","type":"text"},{"text":"inference.csv","type":"text","marks":[{"type":"code_inline"}]},{"text":", sweeps every unique ","type":"text"},{"text":"siamese_score","type":"text","marks":[{"type":"code_inline"}]},{"text":" plus one value just below the minimum, keeps the candidates with NO_PASS-class recall ≥ ","type":"text"},{"text":"min_recall","type":"text","marks":[{"type":"code_inline"}]},{"text":" (with ","type":"text"},{"text":"1e-12","type":"text","marks":[{"type":"code_inline"}]},{"text":" tolerance), then picks the threshold with the best F1 (tie-break: precision, then threshold value). For every row, computes signed weakness from that threshold (positive = misclassified, negative = correct, magnitude = margin). Sorts by weakness descending and takes the top ","type":"text"},{"text":"top_k_per_label","type":"text","marks":[{"type":"code_inline"}]},{"text":" per ground-truth label, then expands each weak row into one row per lighting condition using ","type":"text"},{"text":"dataset.classify.input_map","type":"text","marks":[{"type":"code_inline"}]},{"text":" and ","type":"text"},{"text":"dataset.classify.image_ext","type":"text","marks":[{"type":"code_inline"}]},{"text":" from the train YAML.","type":"text"}]},{"type":"paragraph","content":[{"text":"If ","type":"text"},{"text":"no","type":"text","marks":[{"type":"strong"}]},{"text":" candidate threshold meets the recall target, the container exits non-zero and writes ","type":"text"},{"text":"unreachable_kpi.txt","type":"text","marks":[{"type":"code_inline"}]},{"text":" into ","type":"text"},{"text":"results_dir","type":"text","marks":[{"type":"code_inline"}]},{"text":" explaining which recall the model can actually achieve. In that case, stop the analysis after the docker call, write a one-section report explaining the model fundamentally cannot reach the KPI at any operating point, and recommend retraining or relabeling — skip the visual spot-check.","type":"text"}]},{"type":"paragraph","content":[{"text":"TAO 7.2 artifact contract:","type":"text","marks":[{"type":"strong"}]}]},{"type":"paragraph","content":[{"text":"Container-produced required artifacts:","type":"text"}]},{"type":"table","attrs":{"layout":null},"content":[{"type":"tr","content":[{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Artifact","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Contents","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"kpi_gaps.parquet","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Top-K weakest per label, expanded per lighting. Columns: ","type":"text"},{"text":"filepath","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"label","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"siamese_score","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"weakness","type":"text","marks":[{"type":"code_inline"}]},{"text":".","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"threshold.txt","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Chosen decision threshold (single float, plain text).","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"weak_samples_breakdown.txt","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Per-label kept-row breakdown: ","type":"text"},{"text":"\u003ccount>","type":"text","marks":[{"type":"code_inline"}]},{"text":" total, ","type":"text"},{"text":"\u003c%>","type":"text","marks":[{"type":"code_inline"}]},{"text":" of all kept rows, ","type":"text"},{"text":"N","type":"text","marks":[{"type":"code_inline"}]},{"text":" misclassified (weakness > 0), ","type":"text"},{"text":"N","type":"text","marks":[{"type":"code_inline"}]},{"text":" marginal (weakness ≤ 0).","type":"text"}]}]}]}]},{"type":"paragraph","content":[{"text":"You must produce the required seven-section ","type":"text"},{"text":"RCA_Report.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" after the container returns. On a reachable run, you must also produce ","type":"text"},{"text":"rca_images/","type":"text","marks":[{"type":"code_inline"}]},{"text":" during the visual spot check. The container does not produce either agent-owned artifact.","type":"text"}]},{"type":"paragraph","content":[{"text":"unreachable_kpi.txt","type":"text","marks":[{"type":"code_inline"}]},{"text":" is the failure artifact written when the recall target is unreachable. Its presence means: skip Step 5, write the abridged report, and recommend retraining. The TAO 7.2 ","type":"text"},{"text":"vcn_aoi","type":"text","marks":[{"type":"code_inline"}]},{"text":" action does ","type":"text"},{"text":"not","type":"text","marks":[{"type":"strong"}]},{"text":" emit ","type":"text"},{"text":"metrics.json","type":"text","marks":[{"type":"code_inline"}]},{"text":"; that file is outside the 7.2 contract. Calculate report metrics from the full ","type":"text"},{"text":"inference.csv","type":"text","marks":[{"type":"code_inline"}]},{"text":" and the chosen value in ","type":"text"},{"text":"threshold.txt","type":"text","marks":[{"type":"code_inline"}]},{"text":" instead.","type":"text"}]},{"type":"paragraph","content":[{"text":"Print the container's stdout summary (chosen threshold, kept-row counts, per-label breakdown) to your own stdout so the script-check hook can verify the run produced output.","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Step 5 — Visual spot check (small, fixed)","type":"text"}]},{"type":"paragraph","content":[{"text":"Skip this step if ","type":"text"},{"text":"unreachable_kpi.txt","type":"text","marks":[{"type":"code_inline"}]},{"text":" exists. Otherwise use the Read tool to ","type":"text"},{"text":"view","type":"text","marks":[{"type":"strong"}]},{"text":" the 5 weakest PASS samples and the 5 weakest NO_PASS samples from ","type":"text"},{"text":"kpi_gaps.parquet","type":"text","marks":[{"type":"code_inline"}]},{"text":" (deduplicated to one row per sample, using the FIRST-lighting ","type":"text"},{"text":"filepath","type":"text","marks":[{"type":"code_inline"}]},{"text":"), classify each as exactly one of ","type":"text"},{"text":"mislabeled","type":"text","marks":[{"type":"strong"}]},{"text":" / ","type":"text"},{"text":"edge case","type":"text","marks":[{"type":"strong"}]},{"text":" / ","type":"text"},{"text":"data quality","type":"text","marks":[{"type":"strong"}]},{"text":" / ","type":"text"},{"text":"systematic","type":"text","marks":[{"type":"strong"}]},{"text":", and copy each viewed image (resized to 128×128 if PIL is available, otherwise just copy) into ","type":"text"},{"text":"\u003cresults_dir>/rca_images/","type":"text","marks":[{"type":"code_inline"}]},{"text":". This is the only image inspection required — do not view dozens of images, run failure mode clustering, or audit goldens (VCN has no golden images).","type":"text"}]},{"type":"paragraph","content":[{"text":"See ","type":"text"},{"text":"references/visual-spot-check.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" for the exact sample-selection sort, the per-lighting deduplication rule, the full definition of each verdict category, and the image-copy detail.","type":"text"}]},{"type":"hr","attrs":{"markup":"---"}},{"type":"heading","attrs":{"level":2},"content":[{"text":"Reference invocation","type":"text"}]},{"type":"paragraph","content":[{"text":"Paste-and-edit the workspace, the four paths, and the two numeric knobs; this runs end-to-end. Capture stdout so the script-check hook sees row counts.","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"bash"},"content":[{"text":"WORKSPACE=\u003cabsolute path> # mounted identically inside the container\nEXP_DIR=\u003cexperiment_result_dir> # contains inference/inference.csv and train.yaml; must be inside $WORKSPACE\nDATASET_ROOT=\u003cdataset_root> # image root for inference.csv input_path entries; must be inside $WORKSPACE\nMIN_RECALL=1.0 # zero-miss default; lower if KPI relaxes\nTOP_K=50 # per-label augmentation budget\nOUT=\"$EXP_DIR/rca_results/$(date +%Y-%m-%d_%H%M%S)\"\nSPEC=\"$OUT/vcn_aoi_spec.yaml\"\nIMG=\"$(scripts/resolve_versions_key.py images.tao_toolkit.data_services)\"\n\nmkdir -p \"$OUT\"\n\n# Write the gap-analysis spec for this run\ncat > \"$SPEC\" \u003c\u003cEOF\nmin_recall: $MIN_RECALL\ntop_k_per_label: $TOP_K\nEOF\n\ndocker run --gpus all --rm --shm-size=8g \\\n -v \"$WORKSPACE:$WORKSPACE\" -w \"$WORKSPACE\" \\\n \"$IMG\" gap_analysis vcn_aoi \\\n -e \"$SPEC\" \\\n inference_results_dir=\"$EXP_DIR/inference/latest/\" \\\n train_config=\"$EXP_DIR/train.yaml\" \\\n kpi_media_path=\"$DATASET_ROOT\" \\\n results_dir=\"$OUT\"\n\n# Container writes as root with --user dropped; chown back to host UID if needed.\ndocker run --rm -v \"$WORKSPACE:/w\" alpine chown -R \"$(id -u):$(id -g)\" \"/w/$(realpath --relative-to=\"$WORKSPACE\" \"$OUT\")\"\n\n# Sanity print so the script-check hook sees real numbers. Report metrics are\n# derived from inference.csv because metrics.json is not part of the 7.2 contract.\npython3 - \"$OUT\" \"$EXP_DIR/inference/latest/inference.csv\" \u003c\u003c 'PYEOF'\nimport os, sys\nimport pandas as pd\nout = sys.argv[1]\nunreachable = os.path.join(out, \"unreachable_kpi.txt\")\nif os.path.isfile(unreachable):\n print(\"KPI UNREACHABLE — see\", unreachable)\n sys.exit(0)\nwith open(os.path.join(out, \"threshold.txt\")) as f:\n threshold = float(f.read().strip())\ninference = pd.read_csv(sys.argv[2])\nactual_no_pass = inference[\"label\"].eq(\"NO_PASS\")\npredicted_no_pass = inference[\"siamese_score\"].ge(threshold)\ntp = int((actual_no_pass & predicted_no_pass).sum())\nfp = int((~actual_no_pass & predicted_no_pass).sum())\ntn = int((~actual_no_pass & ~predicted_no_pass).sum())\nfn = int((actual_no_pass & ~predicted_no_pass).sum())\nprecision = tp / (tp + fp) if tp + fp else 0.0\nrecall = tp / (tp + fn) if tp + fn else 0.0\nf1 = 2 * precision * recall / (precision + recall) if precision + recall else 0.0\nprint(f\"threshold={threshold} precision={precision:.4f} recall={recall:.4f} f1={f1:.4f}\")\nprint(f\"confusion_matrix: tp={tp} fp={fp} tn={tn} fn={fn}\")\ndf = pd.read_parquet(os.path.join(out, \"kpi_gaps.parquet\"))\nprint(f\"kpi_gaps.parquet: rows={len(df)}, cols={list(df.columns)}\")\nprint(df['label'].value_counts())\nPYEOF","type":"text"}]},{"type":"hr","attrs":{"markup":"---"}},{"type":"heading","attrs":{"level":2},"content":[{"text":"Outputs","type":"text"}]},{"type":"paragraph","content":[{"text":"Write everything into a timestamped folder under the experiment result directory. The container's required outputs go straight there; you write ","type":"text"},{"text":"RCA_Report.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" and, for a reachable KPI, ","type":"text"},{"text":"rca_images/","type":"text","marks":[{"type":"code_inline"}]},{"text":". Any session/config files added by a runtime packaging hook are optional extras, not part of the TAO 7.2 gap-analysis artifact contract.","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":""},"content":[{"text":"\u003cexperiment_result_dir>/rca_results/YYYY-MM-DD_HHMMSS/\n├── kpi_gaps.parquet # Container: top-K weakest per label, expanded per lighting\n├── threshold.txt # Container: chosen decision threshold (single float)\n├── weak_samples_breakdown.txt # Container: per-label count/misclassified/marginal counts\n├── RCA_Report.md # You: required seven-section gap analysis report\n├── rca_images/ # You: required on reachable runs; spot-check thumbnails\n├── unreachable_kpi.txt # Container: ONLY when no threshold meets min_recall\n└── \u003coptional packaging extras> # Runtime-dependent; outside the 7.2 contract","type":"text"}]},{"type":"paragraph","content":[{"text":"At the start of the run, get the real timestamp by running ","type":"text"},{"text":"date +%Y-%m-%d_%H%M%S","type":"text","marks":[{"type":"code_inline"}]},{"text":" in Bash. Do NOT hardcode or guess. If the user specifies a custom output path, use that instead but maintain the same internal structure.","type":"text"}]},{"type":"hr","attrs":{"markup":"---"}},{"type":"heading","attrs":{"level":2},"content":[{"text":"Common pitfalls","type":"text"}]},{"type":"paragraph","content":[{"text":"The single most consequential failure mode is ","type":"text"},{"text":"forgetting ","type":"text","marks":[{"type":"strong"}]},{"text":"top_k_per_label","type":"text","marks":[{"type":"code_inline"},{"type":"strong"}]},{"text":" when ","type":"text","marks":[{"type":"strong"}]},{"text":"min_recall=1.0","type":"text","marks":[{"type":"code_inline"},{"type":"strong"}]},{"text":": at that recall the chosen threshold sits at or below every NO_PASS score, so without ","type":"text"},{"text":"top_k_per_label","type":"text","marks":[{"type":"code_inline"}]},{"text":" the container falls back to a \"samples below threshold\" filter that returns ONLY misclassified PASS rows and zero NO_PASS rows, breaking the augmentation queue. Always include an explicit positive ","type":"text"},{"text":"top_k_per_label","type":"text","marks":[{"type":"code_inline"}]},{"text":" (default 50) in the spec or as a Hydra override.","type":"text"}]},{"type":"paragraph","content":[{"text":"See ","type":"text"},{"text":"references/pitfalls.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" for the complete checklist, covering: forgetting ","type":"text"},{"text":"top_k_per_label","type":"text","marks":[{"type":"code_inline"}]},{"text":"; passing ","type":"text"},{"text":"--user","type":"text","marks":[{"type":"code_inline"}]},{"text":"; calling with only Hydra overrides (no ","type":"text"},{"text":"-e \u003cspec>","type":"text","marks":[{"type":"code_inline"}]},{"text":"); spec file outside ","type":"text"},{"text":"$WORKSPACE","type":"text","marks":[{"type":"code_inline"}]},{"text":"; spec file with unresolved ","type":"text"},{"text":"???","type":"text","marks":[{"type":"code_inline"}]},{"text":" sentinels; image not pulled / wrong tag; path-mount mismatch; ","type":"text"},{"text":"unreachable_kpi.txt","type":"text","marks":[{"type":"code_inline"}]},{"text":" written; ","type":"text"},{"text":"inference.csv","type":"text","marks":[{"type":"code_inline"}]},{"text":" missing required columns; train YAML missing ","type":"text"},{"text":"dataset.classify.input_map","type":"text","marks":[{"type":"code_inline"}]},{"text":" or ","type":"text"},{"text":"image_ext","type":"text","marks":[{"type":"code_inline"}]},{"text":"; ","type":"text"},{"text":"kpi_media_path","type":"text","marks":[{"type":"code_inline"}]},{"text":" not matching ","type":"text"},{"text":"input_path","type":"text","marks":[{"type":"code_inline"}]},{"text":" prefixes; and no GPU detected from inside the container.","type":"text"}]},{"type":"hr","attrs":{"markup":"---"}},{"type":"heading","attrs":{"level":2},"content":[{"text":"Report Structure","type":"text"}]},{"type":"paragraph","content":[{"text":"Write ","type":"text"},{"text":"RCA_Report.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" as a tight (1000–1800 word) computational gap analysis — depth comes from accurate numbers and a clear action list, not narrative. The full report template (7 sections: Verdict, Threshold Selection, Weakness Distribution, Top-K Weakest Samples, Visual Spot Check, Per-Label Breakdown, Recommended Actions — with the confusion-matrix and table layouts) is in ","type":"text"},{"text":"references/output-template.md","type":"text","marks":[{"type":"code_inline"}]},{"text":". When ","type":"text"},{"text":"unreachable_kpi.txt","type":"text","marks":[{"type":"code_inline"}]},{"text":" exists, replace sections 3–6 with a single short section quoting that file's contents and collapse section 7 to one recommendation: retrain or relabel.","type":"text"}]},{"type":"hr","attrs":{"markup":"---"}},{"type":"heading","attrs":{"level":2},"content":[{"text":"Execution Order","type":"text"}]},{"type":"ordered_list","attrs":{"order":1,"listStyle":"number"},"content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Set ","type":"text"},{"text":"DS_IMAGE","type":"text","marks":[{"type":"code_inline"}]},{"text":" to the pinned data-services URI (see Setup), then run ","type":"text"},{"text":"docker info","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"nvidia-smi","type":"text","marks":[{"type":"code_inline"}]},{"text":", and ","type":"text"},{"text":"docker image inspect \"$DS_IMAGE\"","type":"text","marks":[{"type":"code_inline"}]},{"text":" (pulling if missing) once to confirm the environment. Abort with a clear message if any fail.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Run ","type":"text"},{"text":"date +%Y-%m-%d_%H%M%S","type":"text","marks":[{"type":"code_inline"}]},{"text":" to get the timestamp; create ","type":"text"},{"text":"\u003cexperiment_result_dir>/rca_results/\u003ctimestamp>/","type":"text","marks":[{"type":"code_inline"}]},{"text":".","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Write ","type":"text"},{"text":"vcn_aoi_spec.yaml","type":"text","marks":[{"type":"code_inline"}]},{"text":" into the timestamped dir with ","type":"text"},{"text":"min_recall","type":"text","marks":[{"type":"code_inline"}]},{"text":" and ","type":"text"},{"text":"top_k_per_label","type":"text","marks":[{"type":"code_inline"}]},{"text":" filled in. Keep it under ","type":"text"},{"text":"$WORKSPACE","type":"text","marks":[{"type":"code_inline"}]},{"text":" so the ","type":"text"},{"text":"-e","type":"text","marks":[{"type":"code_inline"}]},{"text":" path resolves inside the container.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Run ","type":"text"},{"text":"docker run … \"$DS_IMAGE\" gap_analysis vcn_aoi -e vcn_aoi_spec.yaml inference_results_dir=… train_config=… kpi_media_path=… output_dir=…","type":"text","marks":[{"type":"code_inline"}]},{"text":". The container writes ","type":"text"},{"text":"kpi_gaps.parquet","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"threshold.txt","type":"text","marks":[{"type":"code_inline"}]},{"text":", and ","type":"text"},{"text":"weak_samples_breakdown.txt","type":"text","marks":[{"type":"code_inline"}]},{"text":" into ","type":"text"},{"text":"results_dir","type":"text","marks":[{"type":"code_inline"}]},{"text":". Print the chosen threshold and kept-row counts to stdout so the script-check hook can verify the run produced output.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"If ","type":"text"},{"text":"unreachable_kpi.txt","type":"text","marks":[{"type":"code_inline"}]},{"text":" exists, skip Step 6 and write the abridged report. Otherwise continue.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Pick 10 weak samples (5 weakest PASS + 5 weakest NO_PASS) from ","type":"text"},{"text":"kpi_gaps.parquet","type":"text","marks":[{"type":"code_inline"}]},{"text":", view each test image with Read, classify, and copy each into ","type":"text"},{"text":"rca_images/","type":"text","marks":[{"type":"code_inline"}]},{"text":".","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Write ","type":"text"},{"text":"RCA_Report.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" last — writing it triggers the packaging hook, which copies session logs and skill config alongside.","type":"text"}]}]}]}]},"metadata":{"date":"2026-11-02","name":"tao-analyze-gaps-visual-changenet","tags":["data","rca","vcn","aoi"],"author":"@skillopedia","source":{"stars":3825,"repo_name":"skills","origin_url":"https://github.com/NVIDIA/skills/blob/HEAD/skills/tao-analyze-gaps-visual-changenet/SKILL.md","repo_owner":"NVIDIA","body_sha256":"92677b9383304f83115c0262e041527df22157be3100814fda8cd0bb98a9f796","cluster_key":"de5c09c02260ff60c1f28282dda971592cf1c84affa3591dc18e5c9b53adaa74","clean_bundle":{"format":"clean-skill-bundle-v1","source":"NVIDIA/skills/skills/tao-analyze-gaps-visual-changenet/SKILL.md","text_bytes":51976,"source_commit":"25340619336e89bcc40365c28bfe9e790430468e","skills_sh_rank":9684,"attachment_count":15,"text_attachments":15,"attachment_storage":"skill_attachments","binary_attachments":0},"cluster_size":1,"skill_md_path":"skills/tao-analyze-gaps-visual-changenet/SKILL.md","import_metadata":{"date":"2026-11-02","author":"@skillopedia","version":"v1","category":"ai-agent-development","category_label":"AI"},"exact_dupes_collapsed_into_this":0},"license":"Apache-2.0","version":"v1","category":"ai-agent-development","metadata":{"author":"NVIDIA Corporation","version":"0.1.0"},"import_tag":"skills-sh-top20k-2026-11","description":"Performs gap analysis on NVIDIA TAO VCN Classify (Visual Component Net) experiments by invoking the pinned TAO data-services container directly via `docker run … gap_analysis vcn_aoi …` — picks the optimal decision threshold, ranks per-sample weakness, and emits a top-K weakest parquet expanded per-lighting for downstream augmentation. Use when analyzing VCN classification failures, picking SDA augmentation targets, or auditing PASS/NO_PASS boundary cases.","allowed-tools":"Read Bash","compatibility":"Requires docker + nvidia-container-toolkit and a CUDA GPU. Pulls the TAO data-services container pinned in this skill.yaml` at the skill bank root."}},"renderedAt":1793666698251}

TAO VCN Classify Gap Analysis Skill Standalone install? If this session was not initialized by the TAO skill bank plugin, run the skill first (host preflight, credentials, cross-skill discovery). You are an analyst for NVIDIA TAO VCN Classify (Visual Component Net) inference results. Your job is to identify the weakest samples per ground-truth label by measuring signed distance from the decision threshold in the wrong direction , then surface them for downstream augmentation or relabeling. This skill is intentionally lightweight. VCN's classify head is a single-score binary boundary (PASS vs…