NVIDIA RAG Blueprint Purpose Use this skill for NVIDIA RAG Blueprint operations: deployment, configuration, troubleshooting, shutdown, and feature management across Docker, Helm, and library deployments. Instructions 1. Match the user request to the intent routing table below. 2. Read the referenced playbook before making changes. 3. Use repository docs and deployment config files as the source of truth. 4. Verify the affected service or workflow after changes. Prerequisites - NVIDIA RAG Blueprint repository checkout. - Docker/Compose or Kubernetes/Helm for deployments. - Python 3.11+ for lib…

| wc -l` outputs a number greater than or equal to 5\",\n \"`docker ps --format '{{.Names}}' | grep -E '^(nim-llm|nemoretriever-embedding-ms)' | wc -l` outputs a number greater than or equal to 1\",\n \"`docker ps --format '{{.Names}}\\\\t{{.Status}}' | grep -E '(rag-server|ingestor-server|milvus-standalone)' | grep -v 'Up' | wc -l` outputs 0\"\n ]\n },\n {\n \"query\": \"Verify the self-hosted RAG stack is fully operational. Check that the rag-server, ingestor-server, and local NIM endpoints are all healthy and responding.\",\n \"checks\": [\n \"`curl -sf -o /dev/null -w '%{http_code}' http://localhost:8081/v1/health` outputs 200\",\n \"`curl -sf -o /dev/null -w '%{http_code}' http://localhost:8082/v1/health` outputs 200\",\n \"`docker ps --format '{{.Names}}\\\\t{{.Status}}' | grep nim-llm | grep -E 'Up|healthy'` returns at least one matching line\",\n \"The agent's final output reports the health status of rag-server, ingestor-server, and the local NIM service with clear per-service indicators\"\n ]\n }\n ]\n}\n","content_type":"application/json; charset=utf-8","language":"json","size":2520,"content_sha256":"d7fc33d23564eef5226b44154fc90adbe512fc4c76410f6271e99ac6bdec8cde"},{"filename":"eval/nvidia_hosted.json","content":"{\n \"skills\": [\"rag-blueprint\"],\n \"platforms\": [\"cpu\"],\n \"env\": \"A Linux host with Docker + Docker Compose plugin installed and running. CPU-only — no NVIDIA GPU or driver present. NVIDIA-hosted deployment: all model inference goes to https://integrate.api.nvidia.com/v1. Required env var: NGC_API_KEY. cwd is the repo root: ${RAG_REPO_ROOT}/. Use deploy/compose/nvdev.env for cloud endpoints. IMPORTANT: use ci/vectordb-cpu.yaml instead of deploy/compose/vectordb.yaml for the vector database — the default uses a GPU Milvus image that fails on CPU-only hosts.\",\n \"expects\": [\n {\n \"query\": \"Deploy NVIDIA RAG Blueprint using Docker Compose in NVIDIA-hosted mode (cloud NIMs). Source deploy/compose/nvdev.env, then start the vector DB, ingestor server, and rag server (with frontend) so that all containers are running. Do not start any local NIM containers (nims.yaml) — all model inference must use the cloud endpoint at integrate.api.nvidia.com.\",\n \"checks\": [\n \"`docker ps --format '{{.Names}}' | grep -E '^(rag-server|ingestor-server|milvus-standalone|milvus-etcd|milvus-minio)

NVIDIA RAG Blueprint Purpose Use this skill for NVIDIA RAG Blueprint operations: deployment, configuration, troubleshooting, shutdown, and feature management across Docker, Helm, and library deployments. Instructions 1. Match the user request to the intent routing table below. 2. Read the referenced playbook before making changes. 3. Use repository docs and deployment config files as the source of truth. 4. Verify the affected service or workflow after changes. Prerequisites - NVIDIA RAG Blueprint repository checkout. - Docker/Compose or Kubernetes/Helm for deployments. - Python 3.11+ for lib…

| wc -l` outputs a number greater than or equal to 5\",\n \"`docker ps --format '{{.Names}}\\\\t{{.Status}}' | grep rag-server | grep -E 'Up|healthy'` returns at least one matching line\",\n \"`docker ps --format '{{.Names}}'` does NOT include any container starting with 'nim-llm', 'nemoretriever-embedding-ms', or 'nemoretriever-ranking-ms' (these would indicate local NIMs were started, which contradicts NVIDIA-hosted mode)\",\n \"`docker exec rag-server env 2>/dev/null | grep -E '^APP_LLM_SERVERURL=' | head -1` outputs a line containing integrate.api.nvidia.com OR is empty (when empty the cloud SDK default is used, which is also acceptable for NVIDIA-hosted mode)\",\n \"The agent sourced deploy/compose/nvdev.env (or set APP_LLM_MODELNAME, APP_EMBEDDINGS_MODELNAME etc. via that env file) before running docker compose up — verifiable via the trajectory: a `source` of nvdev.env or an explicit reference to that env file in a docker compose --env-file invocation\",\n \"The agent's final output claims the deployment succeeded AND, at the time of the final claim, no core container (rag-server, ingestor-server, milvus-standalone) is in 'Created' or 'Restarting' state — i.e. the agent did not declare success prematurely. Per-file enumeration is NOT required; a single overall success message is fine as long as containers are actually Up\"\n ]\n },\n {\n \"query\": \"Verify the deployed RAG stack is healthy and the API is reachable. Hit the rag-server health endpoint, the ingestor-server health endpoint, and confirm the frontend UI responds. Report the status of each.\",\n \"checks\": [\n \"`curl -sf -o /dev/null -w '%{http_code}' http://localhost:8081/v1/health` outputs 200\",\n \"`curl -sf -o /dev/null -w '%{http_code}' http://localhost:8082/v1/health` outputs 200\",\n \"`curl -sf -o /dev/null -w '%{http_code}' http://localhost:8090` outputs 200 (frontend served on port 8090)\",\n \"`docker ps --format '{{.Names}}\\\\t{{.Status}}' | grep -E '(milvus-standalone|rag-server|ingestor-server)' | grep -v 'Up' | wc -l` outputs 0 (every core container is in 'Up' state)\",\n \"The agent's final output reports a per-service health verdict for rag-server, ingestor-server, and the frontend — each service named with a clear status indicator (e.g. 'HTTP 200', 'Healthy', 'Up', or equivalent). A single overall 'all services responding' summary alone is NOT sufficient; per-service breakdown is required (an HTTP code, the word 'Healthy', or an equivalent positive indicator next to each service name counts).\"\n ]\n }\n ]\n}\n","content_type":"application/json; charset=utf-8","language":"json","size":3714,"content_sha256":"2129a48b92c0980285af7b9e4044feb94e7ce18d16fa034c731b04e5a0d219bc"},{"filename":"references/configure/agentic-rag.md","content":"# Agentic RAG\n\n## When to Use\n- User wants the LangGraph agentic pipeline/agentic rag, planning/execution, multi-hop reasoning, ambiguity handling, or verification.\n- User asks about `agentic`, `ENABLE_AGENTIC_RAG`, agentic streaming, stage events, or agentic reasoning traces.\n\n## Restrictions\n- Requires `use_knowledge_base=true`; otherwise the agentic path is not applied.\n- Higher latency and more LLM calls than standard RAG. Prefer per-request enablement for latency-sensitive deployments.\n- The agentic path does not use NeMo Guardrails, Self-Reflection, Query Decomposition, or VLM Inference.\n- Verification is single-pass.\n\n## Process\n1. Detect deployment mode. Docker: edit the active env file. Helm: edit `values.yaml`. Library/API callers can set request fields directly.\n2. Read `docs/agentic-rag.md` for the current architecture, env vars, and limitations.\n3. Prefer per-request enablement:\n ```json\n {\n \"messages\": [{\"role\": \"user\", \"content\": \"...\"}],\n \"use_knowledge_base\": true,\n \"collection_names\": [\"...\"],\n \"agentic\": true\n }\n ```\n4. For API/library clients that omit `agentic`, set `ENABLE_AGENTIC_RAG=true` and restart the RAG server. In the React UI, also select Pipeline → Agentic because the UI sends an explicit per-request value.\n5. Optionally configure LLMs:\n - One deployment-wide LLM: set `APP_LLM_MODELNAME`, `APP_LLM_SERVERURL`, and `APP_LLM_APIKEY`; Docker Compose chains each agentic role to these defaults.\n - Role-specific LLMs: set `AGENTIC_PLANNER_LLM_*`, `AGENTIC_TASK_LLM_*`, `AGENTIC_SEED_GEN_LLM_*`, or `AGENTIC_SYNTHESIS_LLM_*`.\n - One request only: pass `model` and/or `llm_endpoint` in `/v1/generate`; the runtime override applies to all agentic roles for that request.\n6. Verify with `/v1/generate`: streaming agentic chunks include `event_type`, `stage`, and supplementary `reasoning_content`; final answer text still streams through `content`.\n\n## Decision Table\n\n| Goal | Key Action |\n|------|------------|\n| Enable only for one query | Set request body `agentic: true` |\n| Disable for one query when globally enabled | Set request body `agentic: false` |\n| Change deployment default for API clients that omit `agentic` | Set `ENABLE_AGENTIC_RAG=true` or `false` |\n| Enable from the RAG UI | Select Pipeline → Agentic; the Standard UI mode sends `agentic: false` |\n| Add post-synthesis checking | Set `AGENTIC_VERIFICATION_ENABLED=true` |\n| Use the same deployment LLM for every agentic role | Set `APP_LLM_MODELNAME`, `APP_LLM_SERVERURL`, and `APP_LLM_APIKEY` unless role-specific `AGENTIC_*_LLM_*` envs are set |\n| Override every agentic role for one API call | Set request body `model` and/or `llm_endpoint` |\n| Debug agent stages | Set `AGENTIC_LOG_LEVEL=DEBUG` and inspect streamed `event_type` / `stage` chunks |\n\n## Agent-Specific Notes\n- `enable_streaming=true` is the default. Agentic streaming emits stage events (`stage_start`, `stage_end`), intermediate reasoning/output, final answer chunks, agent events, and errors.\n- `enable_streaming=false` makes the agent graph finish before returning a full answer chunk; standard RAG always streams.\n- The React UI has only Standard and Agentic modes. Standard sends `agentic: false`, so `ENABLE_AGENTIC_RAG=true` alone does not override UI Standard mode.\n- In the UI, agentic and standard reasoning traces render in the reasoning panel when the stream includes `reasoning_content`.\n- Docker Compose chains `AGENTIC_*_LLM_MODEL`, `AGENTIC_*_LLM_SERVERURL`, and `AGENTIC_*_LLM_APIKEY` through `APP_LLM_MODELNAME`, `APP_LLM_SERVERURL`, and `APP_LLM_APIKEY`, so one standard LLM override propagates to all four agentic roles unless a role-specific value is set.\n- Helm values list the role-specific envs explicitly. Keep them aligned with the main LLM values for one shared agentic model, or set per-role values when the planner, task, seed generation, or synthesis roles need different models.\n- If a role model is empty in config, the builder falls back to the planner LLM, then the main RAG LLM. API keys fall back through the role config, main RAG LLM config, and the usual NVIDIA-hosted defaults.\n- Per-request `/v1/generate` `model` and `llm_endpoint` values override every agentic role for that request; omit the fields to use deployment or role-specific configuration.\n- If the result is slow or expensive, use per-request `agentic` instead of a global default, lower `AGENTIC_CONTEXT_MAX_TOKENS`, or leave verification disabled.\n\n## Source Documentation\n- `docs/agentic-rag.md` — architecture, API usage, env vars, limitations\n- `docs/api-rag.md` — `/v1/generate` request and streaming behavior\n- `deploy/compose/docker-compose-rag-server.yaml` — Docker `APP_LLM_*` and `AGENTIC_*_LLM_*` fallback chain\n- `src/nvidia_rag/rag_server/agentic_rag/builder.py` — role LLM fallback order and runtime override model\n- `frontend/src/hooks/useMessageSubmit.ts` — UI request field behavior for `agentic`\n- `frontend/src/hooks/useChatStream.ts` and `frontend/src/components/chat/ReasoningPanel.tsx` — reasoning trace rendering\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":5072,"content_sha256":"6931cff1cb011b9deab888194ebe9bb8850b90125aafc3441ae26f83953908ec"},{"filename":"references/configure/api-reference.md","content":"# API Reference\n\n## When to Use\n- User needs to call RAG or Ingestor APIs directly\n- User asks about endpoints, request/response formats, or task status tracking\n\n## Process\n1. Read `docs/api-rag.md` for RAG server endpoints (port 8081)\n2. Read `docs/api-ingestor.md` for Ingestor server endpoints (port 8082)\n3. Consult OpenAPI schemas for exact request/response shapes\n\n## Agent-Specific Notes\n- RAG Server runs on port 8081: `/v1/generate`, `/v1/search`, `/v1/health`, `/v1/configuration`, `/v1/metrics`, `/v1/summary`\n- Ingestor Server runs on port 8082: `/v1/documents`, `/v1/collection`, `/v1/collections`, `/v1/status`\n- `POST /v1/documents` returns a `task_id` — poll `GET /v1/status?task_id=\u003cid>` for progress\n- Task states: `PENDING` → `FINISHED` or `FAILED` (also `UNKNOWN` if not found)\n- NV-Ingest extraction states: `not_started` → `submitted` → `processing` → `completed` or `failed`\n- Max file size: 400 MB per document\n- Full health check: `GET /v1/health?check_dependencies=true`\n- Streaming `/v1/generate` chunks may include supplementary `reasoning_content`. Agentic RAG streaming chunks also include `event_type` and `stage`; final user-facing answer text remains in `content`.\n\n## Notebooks\n- `notebooks/ingestion_api_usage.ipynb` — ingestion API usage examples\n- `notebooks/retriever_api_usage.ipynb` — RAG retriever API: search and query examples\n\n## Source Documentation\n- `docs/api-rag.md` -- RAG server API details\n- `docs/api-ingestor.md` -- Ingestor server API details\n- `docs/api_reference/openapi_schema_rag_server.json` -- RAG server OpenAPI schema\n- `docs/api_reference/openapi_schema_ingestor_server.json` -- Ingestor server OpenAPI schema\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1688,"content_sha256":"b54c77326d2798d42e4f2a6aafe63426b3b80c09aa56e005fe686ebe2b611dc5"},{"filename":"references/configure/data-catalog.md","content":"# Data Catalog\n\n## When to Use\n- User wants to manage collection or document metadata for governance\n- User asks about tagging, ownership, or lifecycle status of collections\n- User wants to list or update collection metadata\n\n## Restrictions\n- None — available automatically after deployment, no additional configuration needed\n- Works with both Milvus and Elasticsearch (full feature parity)\n\n## Process\n1. Read `docs/data-catalog.md` for full API reference, field definitions, and examples\n2. All endpoints are on the ingestor server (port `8082`)\n3. Use PATCH endpoints for updates (merge updates — only provided fields change)\n\n## Decision Table\n\n| Goal | Source Doc | Key Action |\n|------|-----------|------------|\n| Add governance metadata | `docs/data-catalog.md` | POST `/v1/collection` with description, tags, owner |\n| Update lifecycle status | `docs/data-catalog.md` | PATCH with `status: \"Archived\"` |\n| Track content types | `docs/data-catalog.md` | Read auto-populated `has_tables`, `has_images` metrics |\n| Filter during retrieval | See custom metadata docs | Use `metadata_schema` + `filter_expr` (not data catalog) |\n\n## Agent-Specific Notes\n- Auto-populated metrics (`number_of_files`, `last_indexed`, `has_tables`, etc.) are system-set — not user-editable\n- `date_created` and `last_updated` timestamps are automatic\n- PATCH is a merge update — omitted fields keep current values\n- Different from custom metadata: catalog = governance/discovery, custom metadata = retrieval filtering\n\n## Notebooks\n- `notebooks/ingestion_api_usage.ipynb` — ingestion and collection management examples\n\n## Source Documentation\n- `docs/data-catalog.md` — full API reference, catalog fields, auto-populated metrics, Python client examples\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1751,"content_sha256":"995dda145af478d4f143719c6ea91e8f690d7ab735f5a36d2c188e0c8574462f"},{"filename":"references/configure/evaluation.md","content":"# Evaluation\n\n## When to Use\n- The user wants to measure RAG pipeline quality.\n\n- User asks about accuracy, relevancy, groundedness, or recall metrics.\n\n- The user wants to run the filesystem benchmark evaluator (`scripts/eval/evaluate_rag.py`) with `corpus/` plus `train.json`.\n\n## Process\n1. Read `docs/evaluate.md` for full evaluation methodology and setup.\n2. Choose the path:\n - `Notebooks` — interactive RAGAS workflows against a running stack.\n - `CLI benchmark` — on-disk datasets and `evaluate_rag.py`; follow skill `rag-eval` (`skills/rag-eval/SKILL.md`), `scripts/eval/README.md`, and the skill’s `references/` for conversion, flags, runs, and result parsing.\n3. Run evaluation against the deployed RAG pipeline.\n\nWhen building a CLI eval bundle from a public benchmark, materialize `corpus/` as PDF when you can (multimodal content keeps images embedded; matches default `--file-type pdf`). If the source only provides web links or no file extension, default to PDF rather than plain text. Details: `rag-eval` → [`references/dataset-and-conversion.md`](../../../rag-eval/references/dataset-and-conversion.md) and `scripts/eval/README.md`.\n\n## Agent-Specific Notes\n- Uses RAGAS framework for all metrics\n- Answer Accuracy, Context Relevancy, and Groundedness are covered in one notebook\n- Recall is measured separately at top-k cutoffs (1, 3, 5, 10)\n- `evaluate_rag.py` ingests `corpus/`, queries `/v1/generate`, then runs RAGAS NVIDIA metrics (`ragas.metrics`); requires `NVIDIA_API_KEY`. Install CLI deps with `uv sync --project scripts/eval` (declared under `scripts/eval/`).\n\n## Notebooks\n| Notebook | Metrics |\n|----------|---------|\n| `notebooks/evaluation_01_ragas.ipynb` | Answer Accuracy, Context Relevancy, Groundedness |\n| `notebooks/evaluation_02_recall.ipynb` | Recall at top-k cutoffs |\n\n## CLI benchmark (repo)\n| Artifact | Role |\n|----------|------|\n| `scripts/eval/evaluate_rag.py` | End-to-end ingest + generate + RAGAS scoring for one or more dataset roots |\n| `scripts/eval/pyproject.toml` | Dependencies for the CLI only; sync with `uv sync --project scripts/eval` |\n| `scripts/eval/README.md` | Dataset contract, flags, outputs |\n| `skills/rag-eval/SKILL.md` | Router: layout, `train.json`, run/triage playbook |\n| `skills/rag-eval/references/dataset-and-conversion.md` | External → `corpus/` + `train.json` |\n| `skills/rag-eval/references/benchmark-execution.md` | Command examples, quality flags, errors, credential hygiene |\n| `skills/rag-eval/references/evaluate-rag-cli.md` | Flag-level CLI detail |\n| `skills/rag-eval/references/result-analysis.md` | Parsing summaries and metrics JSON |\n\n## Source Documentation\n- `docs/evaluate.md` — full evaluation guide and metric definitions\n- [RAGAS documentation](https://docs.ragas.io/en/stable/)\n- [NVIDIA RAGAS metrics](https://docs.ragas.io/en/stable/concepts/metrics/available_metrics/nvidia_metrics/)\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":2903,"content_sha256":"5a60598df857fb8dea95f73bf12b474ec6e8cf1616ce274e94bdf8e59616e0d9"},{"filename":"references/configure/guardrails.md","content":"# NeMo Guardrails\n\n## When to Use\n- User wants content safety, topic control, or jailbreak prevention\n- User asks to enable/disable guardrails\n\n## Restrictions\n- Not available on B200 GPUs\n- Requires 2 extra GPUs with 48GB+ each (H100, A100 SXM 80GB, or RTX PRO 6000)\n- Not supported in library mode or Helm deployments\n- Jailbreak detection model not yet available out-of-the-box\n\n## Process\n\n1. Detect the deployment mode (guardrails are Docker-only — not supported on Helm or library mode). Edit the active env file for Docker\n2. Read `docs/nemo-guardrails.md` for full setup and configuration\n3. Choose deployment mode: self-hosted (local NIMs) or cloud-hosted (NVIDIA API)\n4. For self-hosted: assign GPU IDs — read `docs/service-port-gpu-reference.md` for default GPU assignments and adjust for your system\n5. Verify all three services healthy: `nemo-guardrails-microservice`, content-safety NIM, topic-control NIM\n6. Enable in UI: Settings > Output Preferences > Guardrails toggle\n\n## Agent-Specific Notes\n- Cloud mode (`nemoguard_cloud` config) skips local NIM containers — only the microservice is needed\n- Per-request toggle via `enable_guardrails` in `/generate` body requires server-level `ENABLE_GUARDRAILS=true` first\n- Override guardrails URL with `NEMO_GUARDRAILS_URL` if running on a different host\n- Content-safety and topic-control models are trained on single-turn data — multi-turn conversations may get inconsistent safety classifications\n- Current guardrails only produce simple refusal responses (\"I'm sorry. I can't respond to that.\")\n\n## Source Documentation\n- `docs/nemo-guardrails.md` -- full setup, configuration, and customization of guardrail rules\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1687,"content_sha256":"81a9b41d645fe98519a5b30c9dd08c0cc7884d0a08962a1df1826f533c5dc044"},{"filename":"references/configure/ingestion.md","content":"# Ingestion: Text-Only, Audio, Nemotron Parse, OCR & Batch\n\n## When to Use\nUser wants to configure ingestion mode (text-only, audio, Nemotron Parse), switch OCR engines, save extraction results to disk, use standalone NV-Ingest, tune ingestion performance, or run batch ingestion.\n\n## Restrictions\n- Nemotron Parse: not available on B200 or RTX PRO 6000 GPUs (requires H100 or A100 SXM 80GB)\n- Audio on Helm: not supported on RTX PRO 6000\n- Nemotron Parse GPU conflict: read `docs/service-port-gpu-reference.md` for default GPU assignments. Nemotron Parse defaults to the same GPU as LLM — reassign on limited-GPU systems\n\n## Process\n\n1. Detect the deployment mode (Docker self-hosted / NVIDIA-hosted / Helm / Library). Docker: edit the active env file. Helm: edit `values.yaml`. Library: edit `notebooks/config.yaml`\n2. Read the relevant source doc for detailed configuration\n3. Apply the required env vars to the active config, restart ingestor (and NIM services if enabling new profiles)\n4. Verify: upload a test document and check ingestion status\n\n## Decision Table\n\n| Goal | Source Doc | Key Action |\n|------|-----------|------------|\n| Text-only ingestion | `docs/text_only_ingest.md` | Set extract vars to False, set `COMPONENTS_TO_READY_CHECK=\"\"` |\n| Audio ingestion | `docs/audio_ingestion.md` | Start audio NIM (`--profile audio`), set `AUDIO_MS_GPU_ID` |\n| Nemotron Parse | `docs/nemotron-parse-extraction.md` | `APP_NVINGEST_PDFEXTRACTMETHOD=nemotron_parse`, start NIM |\n| OCR config/switch | `docs/nemoretriever-ocr.md` | Switch between Nemotron OCR and Paddle OCR |\n| Save to disk | `docs/mount-ingestor-volume.md` | `APP_NVINGEST_SAVETODISK=True`; results persist in `rag-vol-ingestor` |\n| Standalone NV-Ingest | `docs/nv-ingest-standalone.md` | Direct Python client, no full ingestor server |\n| Batch ingestion | See `scripts/batch_ingestion.py` | `python scripts/batch_ingestion.py --folder ... --collection-name ...` |\n| Tune performance | `docs/accuracy_perf.md` | Adjust chunk size, overlap, batch settings |\n| Summarization at ingest | `references/configure/summarization.md` | `generate_summary: true` in upload payload |\n\n## Agent-Specific Notes\n\n- Text-only mode: set `COMPONENTS_TO_READY_CHECK=\"\"` in the active env file so NV-Ingest does not wait for disabled extraction services. If the compose file hardcodes `COMPONENTS_TO_READY_CHECK=ALL`, update it to `${COMPONENTS_TO_READY_CHECK:-ALL}` so the env var takes effect\n- Use `--profile rag` with nims.yaml to skip OCR/detection NIMs in text-only mode\n- Audio formats supported: `.mp3`, `.wav`, `.mp4`, `.avi`, `.mov`, `.mkv`\n- Riva ASR requires ~8GB VRAM\n- Nemotron OCR is 2x+ faster than Paddle OCR but needs about 8GB vs 3GB VRAM\n- Batch CLI: `pip install -r scripts/requirements.txt` first; idempotent (skips already-ingested files)\n- MIG deployments: reduce batch sizes for large bulk ingestion jobs\n\n## Notebooks\n- `notebooks/ingestion_api_usage.ipynb` — Ingestor API: collections, uploads, document management\n\n## Source Documentation\n- `docs/text_only_ingest.md` — Text-only ingestion (skip OCR/detection)\n- `docs/audio_ingestion.md` — Audio/video ingestion via ASR\n- `docs/nemotron-parse-extraction.md` — Nemotron Parse PDF extraction\n- `docs/nemoretriever-ocr.md` — Nemotron OCR configuration and switching\n- `docs/mount-ingestor-volume.md` — Volume mount for extraction results\n- `docs/nv-ingest-standalone.md` — Standalone NV-Ingest without ingestor server\n- `docs/accuracy_perf.md` — Ingestion tuning settings (chunk size, overlap, batch params)\n- `docs/service-port-gpu-reference.md` — OCR port mappings and GPU assignments\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":3636,"content_sha256":"c6638264d20e2ce3a60ff8c71c4570a851d80e2f0b236ff3dce8f10c73b5b79f"},{"filename":"references/configure/mcp.md","content":"# MCP Server & Client\n\n## When to Use\n- User wants to expose RAG APIs as MCP tools for agentic workflows\n- User asks about MCP transport modes, NeMo Agent Toolkit integration, or ReAct agents\n\n## Process\n1. Read `docs/mcp.md` for full MCP server/client setup and configuration\n2. Choose transport mode: `sse`, `streamable_http`, or `stdio`\n3. Run MCP server from `examples/nvidia_rag_mcp/mcp_server.py`\n4. For agentic RAG, see ReAct agent example in `examples/rag_react_agent/`\n\n## Agent-Specific Notes\n- MCP wraps both RAG tools (`generate`, `search`, `get_summary`) and Ingestor tools (`create_collection`, `upload_documents`, etc.) via FastMCP\n- `stdio` transport does not require a running server — client spawns it directly\n- ReAct agent requires: Python 3.11+, `NVIDIA_API_KEY`, and data already ingested into Milvus\n- Configure Milvus endpoint in `examples/rag_react_agent/src/rag_react_agent/configs/config.yml` or via `APP_VECTORSTORE_URL`\n\n## Notebooks\n| Notebook | Description |\n|----------|-------------|\n| `notebooks/mcp_server_usage.ipynb` | End-to-end MCP workflow: collection creation, upload, RAG queries |\n| `notebooks/nat_mcp_integration.ipynb` | NeMo Agent Toolkit integration with RAG MCP server |\n\n## Source Documentation\n- `docs/mcp.md` -- full MCP server/client documentation and transport configuration\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1330,"content_sha256":"d56b81476c5da1fb4a539abd9108e6feeedea7619b99f5f5b021aa6c7ed6e802"},{"filename":"references/configure/migration.md","content":"# Migration Guide\n\n## When to Use\n- User is upgrading between RAG Blueprint versions\n- User encounters breaking API changes or deprecated endpoints after an update\n\n## Process\n1. Read `docs/migration_guide.md` for full version-by-version migration details\n2. Identify the user's current and target versions\n3. Apply changes sequentially for each version gap\n\n## Agent-Specific Notes\n\n### v2.2.0 → v2.3.0\n- New `confidence_threshold` field in `/generate` and `/search` (0.0–1.0, default 0.0)\n- New `summary_options` parameter with `page_filter`, `shallow_summary`, `summarization_strategy`\n- `SUMMARY_LLM_MAX_CHUNK_LENGTH` and `SUMMARY_CHUNK_OVERLAP` changed from character-based to token-based — divide old values by ~4\n\n### v2.1.0 → v2.2.0\n- Added `generate_summary` to `/documents`, new `GET /summary` endpoint\n- `POST /collection` (singular) replaces `POST /collections` for single collection creation\n- `collection_names: List[str]` replaces `collection_name: str` in `/generate` and `/search`\n\n### v2.0.0 → v2.1.0\n- `POST /documents` gained `blocking: bool` (default `True`); use `false` + `GET /status` for async\n\n### v1.0.0 → v2.0.0 (Breaking)\n- Single server split into RAG Server (port 8081) and Ingestion Server (port 8082)\n- Collections must be explicitly created before uploading documents\n- Default changed from cloud-hosted to on-prem models\n\n## Source Documentation\n- `docs/migration_guide.md` — Full migration guide with examples and env var changes\n- `docs/release-notes.md` — Release notes and version history\n- `docs/query-to-answer-pipeline.md` — Query-to-answer pipeline architecture overview\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1632,"content_sha256":"23ebc0eb94fa5095111dc9fb7d68fe01f7576958f8f5384c9abbfe78bac00e98"},{"filename":"references/configure/models-and-infrastructure.md","content":"# Models, Vector DB & Service API Keys\n\n## When to Use\nUser wants to change LLM, embedding, or ranking models; switch vector DB (Elasticsearch/Milvus); configure Elasticsearch or Milvus auth, GPU mode, or custom endpoints; set service-specific API keys; or build a custom VDB operator.\n\n## Process\n\nDetect the deployment mode before making changes. Docker: edit the active env file. Helm: edit `values.yaml` under `nimOperator` and `envVars` sections. Library: edit `notebooks/config.yaml`.\n\n### Change Models (LLM, Embedding, Ranking)\n1. Read `docs/change-model.md` for full model change instructions\n2. Read `docs/model-profiles.md` for NIM profile selection and GPU-specific profiles\n3. Key env vars: `APP_LLM_MODELNAME`, `APP_EMBEDDINGS_MODELNAME`, `APP_RANKING_MODELNAME`\n4. Embedding model change requires re-ingesting all documents — update `APP_EMBEDDINGS_DIMENSIONS` to match\n5. Restart affected services (RAG server + ingestor for embedding changes)\n6. Verify via health endpoint\n\n### Switch Vector DB\n1. Read `docs/change-vectordb.md` for full setup (Docker and Helm)\n2. Key env vars: `APP_VECTORSTORE_URL`, `APP_VECTORSTORE_NAME`\n3. Data is not migrated — re-ingest all documents after switching\n4. Elasticsearch is the default backend and uses `rag-vol-elasticsearch` in Docker Compose\n5. Elasticsearch requires port 9200; check for conflicts\n\n### Milvus Configuration\n1. Read `docs/milvus-configuration.md` for indexing, GPU, auth, and tuning\n2. Read `docs/milvus-schema.md` for collection schema requirements\n3. CPU mode: set `APP_VECTORSTORE_ENABLEGPUSEARCH=False`, `APP_VECTORSTORE_ENABLEGPUINDEX=False`, change Milvus image to non-GPU\n4. Auth: download milvus.yaml, enable `authorizationEnabled`, set password before first deployment\n\n### API Keys\n1. Read `docs/api-key.md` for NGC API key setup and per-service keys\n2. Fallback order: service-specific key > `NVIDIA_API_KEY` > `NGC_API_KEY`\n3. Per-service keys: `APP_LLM_APIKEY`, `APP_EMBEDDINGS_APIKEY`, `APP_RANKING_APIKEY`, `APP_VLM_APIKEY`, etc.\n\n## Decision Table\n\n| Goal | Source Doc | Key Action |\n|------|-----------|------------|\n| Change LLM | `docs/change-model.md` | Set `APP_LLM_MODELNAME`, restart RAG server |\n| Change embedding | `docs/change-model.md` | Set `APP_EMBEDDINGS_MODELNAME` + `APP_EMBEDDINGS_DIMENSIONS`, re-ingest |\n| Change reranker | `docs/change-model.md` | Set `APP_RANKING_MODELNAME`, restart RAG server |\n| Use/default Elasticsearch | `docs/change-vectordb.md` | Start `vectordb.yaml`; data lives in `rag-vol-elasticsearch`; re-ingest when switching backends |\n| Switch to Milvus | `docs/change-vectordb.md` | Start `vectordb.yaml --profile milvus`, set env vars, re-ingest |\n| Milvus auth | `docs/milvus-configuration.md` | Download config, enable auth, mount volume |\n| Milvus CPU mode | `docs/milvus-configuration.md` | Change image, disable GPU env vars |\n| Custom VDB | `docs/change-vectordb.md` | Implement `VDBRag`, register in `__init__.py` |\n| NIM profiles | `docs/model-profiles.md` | List profiles, set `NIM_MODEL_PROFILE` |\n| Service API keys | `docs/api-key.md` | Set per-service `*_APIKEY` vars |\n| Collection schema | `docs/milvus-schema.md` | Required fields: pk, vector, text, source, content_metadata |\n\n## Agent-Specific Notes\n\n- Current default model family uses `nvidia/nemotron-3-super-120b-a12b`, `nvidia/llama-nemotron-embed-vl-1b-v2`, and `nvidia/llama-nemotron-rerank-1b-v2`.\n- Nemotron-3-Nano naming: `nvidia/nemotron-3-nano-30b-a3b` (NVIDIA-hosted) vs `nvidia/nemotron-3-nano` (self-hosted NIM) — same model, different names\n- Helm model changes go in `values.yaml` under `nimOperator` and `envVars` sections\n- Custom VDB operator requires implementing `VDBRag` base class — see `docs/change-vectordb.md` \"Custom Vector Database Operator\" section\n- VDB auth tokens can be passed per-request via `Authorization: Bearer \u003ctoken>` header; Elasticsearch runtime auth supports API keys\n- Milvus password persists in etcd volume — to change after deployment, must delete volumes (destroys data)\n\n## Notebooks\n- `notebooks/building_rag_vdb_operator.ipynb` — Custom VDB operator implementation (OpenSearch example)\n\n## Source Documentation\n- `docs/change-model.md` — Model changes (LLM, embedding, ranking, NIM images)\n- `docs/change-vectordb.md` — Vector DB switching, Elasticsearch setup, custom VDB operator\n- `docs/milvus-configuration.md` — Milvus indexing, GPU config, auth, tuning\n- `docs/milvus-schema.md` — Collection schema fields and requirements\n- `docs/model-profiles.md` — NIM profile definitions and selection\n- `docs/api-key.md` — NGC API key setup, per-service keys, fallback order\n- `docs/service-port-gpu-reference.md` — Port mappings and GPU assignments for all services\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":4740,"content_sha256":"bd97b44294969473eeb671d3fb9ce18afd67fe2a212ef90f871f82b69a5a3091"},{"filename":"references/configure/multimodal-query.md","content":"# Multimodal Query (Image + Text)\n\n## When to Use\n- User wants to query knowledge base with images and text together\n- User asks about VLM (Vision Language Model) deployment for RAG\n- User wants image-based document understanding or visual Q&A\n\n## Restrictions\n- Reranker must be disabled (`ENABLE_RERANKER=false`)\n- On-prem: requires NVIDIA H100 or A100 SXM 80GB GPU\n- Single-page retrieval only — image queries return content from one page per document\n\n## Process\n1. Detect the deployment mode (Docker / Helm / Library). Docker: edit the active env file. Helm: edit `values.yaml`. Library: edit `notebooks/config.yaml`\n2. Read `docs/multimodal-query.md` for full env var configuration and commands\n3. Choose variant: self-hosted (Docker), NVIDIA-hosted (cloud), or Helm\n4. Deploy VLM + VLM Embedding NIMs per source doc instructions\n5. Set VLM env vars in the active config and switch embedding model to VLM embedding\n6. Restart ingestor + RAG server (Docker: add `--build` flag) and verify\n\n## Agent-Specific Notes\n- Must select a collection before querying — queries without collection return no results\n- First VLM deployment: model downloads take 10–20 min (~10GB+)\n- `VLM_MS_GPU_ID` — read `docs/service-port-gpu-reference.md` for the default GPU assignment and override if needed\n- Cloud rate limits apply for ingestion of >10 files\n- NVIDIA-hosted VLM endpoints should include the `/v1` suffix, e.g. `https://integrate.api.nvidia.com/v1`\n- For Helm with MIG: ensure dedicated MIG slice is assigned to VLM\n- Image extraction must be enabled: `APP_NVINGEST_EXTRACTIMAGES=True`, `APP_NVINGEST_IMAGE_ELEMENTS_MODALITY=image`\n- Helm multimodal deployments that disable `nim-llm` must set summary env vars under `ingestor-server.envVars` when `generate_summary=true`\n\n## Notebooks\n- `notebooks/image_input.ipynb` — end-to-end multimodal query examples, image upload, VLM querying\n\n## Source Documentation\n- `docs/multimodal-query.md` — full Docker/cloud/Helm configuration, env vars, API usage, limitations\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":2023,"content_sha256":"b8aff8ae28d07d1ab94cdc3d3b3ab7fa51e8df90560066e51291fff27d62afc5"},{"filename":"references/configure/notebooks.md","content":"# Notebooks\n\n## When to Use\n- Hands-on examples of NVIDIA RAG Blueprint features are needed\n- There are questions about Jupyter notebooks, tutorials, or code samples\n\n## Process\n1. Read `docs/notebooks.md` for full notebook descriptions and prerequisites.\n2. Set up the environment: virtualenv, `jupyterlab`, and `git lfs pull` for test data.\n3. Open JupyterLab at `http://\u003cserver-ip>:8889`.\n\n## Agent-Specific Notes\n- Git LFS is required because several notebooks rely on large data files (`git lfs install && git lfs pull`).\n- In Docker mode, deploy NVIDIA RAG Blueprint first, then run notebooks against the running services.\n- In library mode, use `rag_library_usage.ipynb` (full) or `rag_library_lite_usage.ipynb` (containerless).\n- The custom VDB operator notebook requires Docker for OpenSearch services.\n- Agentic RAG examples are integrated into `rag_library_usage.ipynb` (library mode, `agentic=True` on `generate()`) and `retriever_api_usage.ipynb` (API streaming). For configuration, see `references/configure/agentic-rag.md`.\n\n## Notebook Catalog\n\n### Beginner\n| Notebook | Topic |\n|-----------------------------|-------------------------------------|\n| `ingestion_api_usage.ipynb` | Document ingestion through the API |\n| `retriever_api_usage.ipynb` | Search and retrieval API |\n| `image_input.ipynb` | Image upload and multimodal queries |\n\n### Intermediate\n| Notebook | Topic |\n|--------------------------------|----------------------------------------|\n| `summarization.ipynb` | Document summarization strategies |\n| `evaluation_01_ragas.ipynb` | RAGAS accuracy, relevancy, groundedness|\n| `evaluation_02_recall.ipynb` | Recall at top-k cutoffs |\n| `nb_metadata.ipynb` | Custom metadata and filtered retrieval |\n| `rag_library_usage.ipynb` | Full library mode end-to-end |\n| `rag_library_lite_usage.ipynb` | Lite, containerless library mode |\n| `langchain_nvidia_retriever.ipynb` | LangChain retriever connector for NVIDIA RAG |\n\n### Advanced\n| Notebook | Topic |\n|-----------------------------------|-------------------------------------|\n| `building_rag_vdb_operator.ipynb` | Custom OpenSearch VDB operator |\n| `mcp_server_usage.ipynb` | MCP server with transport modes |\n| `nat_mcp_integration.ipynb` | NeMo Agent Toolkit plus MCP |\n| `rag_event_ingest.ipynb` | Continuous ingestion from object storage |\n\n### Deployment\n| Notebook | Topic |\n|--------------------|-----------------------|\n| `launchable.ipynb` | Brev cloud deployment |\n\n## Source Documentation\n- `docs/notebooks.md` — full notebook descriptions, setup, and prerequisites.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":2873,"content_sha256":"699c61ee8b5535e162553bc3a4a92894953a3071319a4147b368595ad167eacf"},{"filename":"references/configure/observability.md","content":"# Observability\n\n## When to Use\n- User wants tracing, metrics, or monitoring for the RAG pipeline\n- User asks about latency debugging, Zipkin, Grafana, or Prometheus\n\n## Process\n1. Detect the deployment mode. Docker: edit the active env file. Helm: edit `values.yaml`. Library: edit `notebooks/config.yaml`\n2. Read `docs/observability.md` for full setup (Docker and Helm)\n3. Set `OPENTELEMETRY_CONFIG_FILE` and `APP_TRACING_ENABLED=True` in the active config\n4. Start observability stack and restart RAG server\n5. Import Grafana dashboard from `deploy/config/rag-metrics-dashboard.json`\n\n## Agent-Specific Notes\n- Library mode: set `OPENTELEMETRY_CONFIG_FILE` in the environment for tracing; the Docker-based Prometheus/Grafana stack is independent\n- Helm: Prometheus Operator CRDs must be installed before deploying with observability enabled\n- Default Grafana credentials: `admin` / `admin`\n- Zipkin spans cover: `query-rewriter`, `retriever`, `context-reranker`, `llm-stream`\n- Span I/O visible via `traceloop.entity.input` / `traceloop.entity.output` fields\n\n### Quick Latency Triage\n| Symptom | Check |\n|---------|-------|\n| Slow first token | `rag_ttft_ms` — compare retriever and reranker spans |\n| Slow full response | `llm_generation_time_ms` / `llm-stream` span |\n| Retrieval heavy | Compare `retrieval_time_ms` vs `context_reranker_time_ms` |\n\n## Source Documentation\n- `docs/observability.md` -- full Docker/Helm setup, env vars, metrics reference, and dashboard import\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1484,"content_sha256":"f1c2d3617770b4e660212ccde6d7ce3c363be2fd2fab28901fd3d767e0b5922a"},{"filename":"references/configure/query-and-conversation.md","content":"# Query Rewriting, Query Decomposition, and Multi-Turn\n\nUse these features when the user wants follow-up questions, conversation-aware retrieval, query rewriting, or decomposition of complex questions. For LangGraph agent planning/execution, use `agentic-rag.md` instead.\n\n## When to Use\n- Enable multi-turn conversations or support follow-up questions.\n- Improve retrieval with query rewriting.\n- Break complex multi-hop questions into smaller retrieval subqueries.\n- Configure or debug conversation history behavior.\n\n## Restrictions\n- Query rewriting and multi-turn both require `CONVERSATION_HISTORY > 0`; with `0`, query rewriting has no effect.\n- Query decomposition works only when `use_knowledge_base=true` and with a single collection.\n- Query decomposition is separate from Agentic RAG; do not enable both without reading `docs/agentic-rag.md` and `docs/query_decomposition.md` limitations.\n\n## Dependencies\n\n| Setting | Depends on | Side effect when changed |\n|---------|------------|--------------------------|\n| `ENABLE_QUERYREWRITER` | `CONVERSATION_HISTORY > 0` | Enabling requires conversation history; disabling has no side effects |\n| `CONVERSATION_HISTORY` | — | Setting to `0` also effectively disables query rewriting |\n\n## Process\n1. Detect deployment mode. Docker: edit the active env file. Helm: edit `values.yaml`. Library: edit `notebooks/config.yaml`.\n2. Read the source doc for the feature.\n3. Apply config changes and restart the RAG server.\n4. Verify with a follow-up or multi-hop query against a known collection.\n\n### Query Rewriting\n1. Read `docs/multiturn.md` for full configuration details.\n2. To enable, set `ENABLE_QUERYREWRITER=True`. If `CONVERSATION_HISTORY=0`, set it to `5` or another positive value.\n3. To disable, unset or comment out `ENABLE_QUERYREWRITER`.\n4. Optional per request: set `enable_query_rewriting: true` in `POST /v1/generate`; `CONVERSATION_HISTORY` must still be positive.\n\n### Multi-Turn\n1. Read `docs/multiturn.md` for retrieval strategies and API usage.\n2. To enable, set `CONVERSATION_HISTORY > 0` and choose the retrieval strategy.\n3. To disable, set `CONVERSATION_HISTORY=0`.\n\n### Query Decomposition\n1. Read `docs/query_decomposition.md` for the algorithm, limitations, and examples.\n2. Set `ENABLE_QUERY_DECOMPOSITION=true` and `MAX_RECURSION_DEPTH=3` or another depth that fits the use case.\n\n## Decision Table\n\n| Goal | Source Doc | Key Settings |\n|------|------------|--------------|\n| Multi-turn with best accuracy | `docs/multiturn.md` | `CONVERSATION_HISTORY=5`, `ENABLE_QUERYREWRITER=True` |\n| Multi-turn with low latency | `docs/multiturn.md` | `CONVERSATION_HISTORY=5`, `MULTITURN_RETRIEVER_SIMPLE=True` |\n| Complex multi-hop decomposition | `docs/query_decomposition.md` | `ENABLE_QUERY_DECOMPOSITION=true`, `MAX_RECURSION_DEPTH=3` |\n| Agent planning/execution | `docs/agentic-rag.md` | Use `references/configure/agentic-rag.md` |\n| Disable multi-turn | — | `CONVERSATION_HISTORY=0` |\n\n## Agent-Specific Notes\n- `MULTITURN_RETRIEVER_SIMPLE` only applies when query rewriting is disabled; query rewriting takes precedence if both are configured.\n- Query decomposition adds latency and is most useful for multi-hop questions that involve multiple entities or steps.\n- In library mode, configure these settings in `notebooks/config.yaml` instead of environment variables.\n\n## Notebooks\n- `notebooks/retriever_api_usage.ipynb` — RAG retriever API usage with search and end-to-end query examples.\n\n## Source Documentation\n- `docs/query_decomposition.md` — decomposition algorithm and recursion depth guidance\n- `docs/multiturn.md` — conversation history behavior, retrieval strategies, API usage, Helm configuration\n- `docs/agentic-rag.md` — separate LangGraph agentic pipeline\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":3764,"content_sha256":"5024c059e00b9bf6a92a76085ec3be027ac1e560026c2c1339b0732d7d61264b"},{"filename":"references/configure/reasoning-and-generation.md","content":"# Reasoning, Self-Reflection & Prompt Customization\n\n## When to Use\nUser wants to enable reasoning/thinking mode, stream or inspect `reasoning_content`, configure self-reflection, customize prompts, adjust generation parameters (max tokens, temperature, citations), or understand thinking budget options.\n\n## Process\n1. Detect the deployment mode (Docker / Helm / Library). Docker: edit the active env file. Helm: edit `values.yaml`. Library: edit `notebooks/config.yaml`\n2. Read the relevant source doc for the specific feature\n3. Apply env vars to the active config or edit prompt files, restart RAG server\n4. Prompt changes require `--build` flag (Docker); env var changes only need restart\n5. Verify: test with a query and check for reasoning output or changed behavior\n\n## Decision Table\n\n| Goal | Source Doc | Key Action |\n|------|-----------|------------|\n| Enable reasoning (Nemotron 3 / Nano 30B) | `docs/enable-nemotron-thinking.md` | `LLM_ENABLE_THINKING=true`, optionally `LLM_REASONING_BUDGET`, `LLM_LOW_EFFORT` |\n| Enable prompt-directed thinking | `docs/enable-nemotron-thinking.md` | Edit `prompt.yaml`: `/no_think` → `/think`, set temperature/top-p |\n| Self-reflection | `docs/self-reflection.md` | `ENABLE_REFLECTION=true`, set thresholds |\n| Prompt customization | `docs/prompt-customization.md` | `PROMPT_CONFIG_FILE=/path/to/custom.yaml` or edit prompt.yaml |\n| Generation parameters | `docs/llm-params.md` | `LLM_MAX_TOKENS`, `LLM_TEMPERATURE`, `ENABLE_CITATIONS` |\n| Per-request overrides | `docs/llm-params.md` | `temperature`, `top_p`, `max_tokens`, `stop` in API payload |\n\n## Agent-Specific Notes\n\n- Prompt changes need `--build` flag on restart; env var changes do not\n- Self-reflection: streaming not supported during groundedness checks\n- Self-reflection uses same LLM by default; override with `REFLECTION_LLM`, `REFLECTION_LLM_SERVERURL`, `REFLECTION_LLM_APIKEY`\n- Helm: only on-premises reflection is supported\n- GPU requirements for reflection: see `docs/self-reflection.md` for optimal GPU configurations\n- Debug reflection: set `LOGLEVEL=INFO` to observe iteration counts\n- `ENABLE_NEMOTRON_3_NANO_THINKING` is deprecated; use `LLM_ENABLE_THINKING`\n- With current streaming responses, reasoning is separated from the user-facing answer: `choices[].delta.reasoning_content` carries reasoning while `choices[].delta.content` carries final answer tokens\n- `FILTER_THINK_TOKENS=true` keeps final-answer content clean but still preserves reasoning structurally in `reasoning_content` when the server is configured to preserve it\n- 18 prompt templates available in `prompt.yaml` — custom file only overrides specified keys\n\n### Reasoning Model Comparison\n\n| Model | Control | Thinking Budget | Output Format |\n|-------|---------|-----------------|---------------|\n| Nemotron 3 / Nemotron 3 Super | `LLM_ENABLE_THINKING` plus model template args, or prompt `/think` where documented | `LLM_REASONING_BUDGET`, `LLM_LOW_EFFORT` | `reasoning_content` stream or filtered `\u003cthink>` blocks |\n| Nemotron-3-Nano 9B | System prompt (`/think`) | `min_thinking_tokens` + `max_thinking_tokens` | `reasoning_content` field |\n| Nemotron-3-Nano 30B | `LLM_ENABLE_THINKING` env var | `LLM_REASONING_BUDGET` or `max_thinking_tokens` | `reasoning_content` field |\n\n### Thinking Budget Recommendations\n\n| Range | Use Case |\n|-------|----------|\n| 1024–4096 | Faster responses for simpler questions |\n| 8192–16384 | More thorough reasoning for complex queries |\n\n## Notebooks\n- `notebooks/retriever_api_usage.ipynb` — end-to-end query examples showing generation behavior\n\n## Source Documentation\n- `docs/enable-nemotron-thinking.md` — Reasoning mode for all Nemotron models\n- `docs/self-reflection.md` — Self-reflection configuration and thresholds\n- `docs/prompt-customization.md` — Prompt template catalog and customization\n- `docs/llm-params.md` — Generation parameters (temperature, max tokens, etc.)\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":3933,"content_sha256":"f321410ce645d2c2d4042ca1474dfb72abc3da28b07fae58bce63a28b725164c"},{"filename":"references/configure/search-and-retrieval.md","content":"# Search & Retrieval: Hybrid Search, Multi-Collection, Metadata & Profiles\n\n## When to Use\nUser wants to enable hybrid search, query multiple collections, add custom metadata/filters, tune retrieval performance, configure reranker, enable natural language filter generation, or switch accuracy/performance profiles.\n\n## Process\n\n1. Detect the deployment mode (Docker / Helm / Library). Docker: edit the active env file. Helm: edit `values.yaml`. Library: edit `notebooks/config.yaml`\n2. Read the relevant source doc for detailed configuration\n3. Apply the required env vars to the active config and restart affected services\n4. Verify via search/generate API call\n\n## Decision Table\n\n| Goal | Source Doc | Key Env Vars |\n|------|-----------|-------------|\n| Hybrid search | `docs/hybrid_search.md` | `APP_VECTORSTORE_SEARCHTYPE=hybrid` |\n| Multi-collection | `docs/multi-collection-retrieval.md` | `enable_reranker: True` in API payload |\n| Custom metadata | `docs/custom-metadata.md` | Metadata in upload payload, `filter_expr` in query |\n| Accuracy profile | `docs/accuracy_perf.md` | Copy values from `deploy/compose/accuracy_profile.env` into the active env file |\n| Performance profile | `docs/accuracy_perf.md` | Copy values from `deploy/compose/perf_profile.env` into the active env file |\n| Filter generation | `docs/custom-metadata.md` | `ENABLE_FILTER_GENERATOR=True` |\n\n## Agent-Specific Notes\n\n- Hybrid search requires re-ingesting — existing collections created with `dense` must be re-created\n- Multi-collection: limited to 5 collections per query; reranker is mandatory\n- Multi-collection not supported when `ENABLE_QUERY_DECOMPOSITION=true`\n- Elasticsearch is the default vector DB. Milvus is optional and requires re-ingestion when switching.\n- Elasticsearch RRF is not supported in the open-source version — use `weighted` ranker for open-source Elasticsearch hybrid search\n- Ingestor must be restarted alongside RAG server when enabling hybrid search\n- `RERANKER_CONFIDENCE_THRESHOLD` is a legacy alias for `RERANKER_SCORE_THRESHOLD`\n- Recommended `RERANKER_SCORE_THRESHOLD` range: 0.3–0.5 (too high filters out too many chunks)\n\n### Advanced Tuning (not fully documented elsewhere)\n\n| Variable | Default | Description |\n|----------|---------|-------------|\n| `APP_VECTORSTORE_INDEXTYPE` | `GPU_CAGRA` | Vector index type |\n| `APP_VECTORSTORE_EF` | `100` | Search accuracy/speed trade-off (must be >= `VECTOR_DB_TOPK`) |\n| `VECTOR_DB_TOPK` | `100` | Candidates from vector DB (input to reranker) |\n| `APP_RETRIEVER_TOPK` | `10` | Chunks sent to LLM prompt (after reranking) |\n| `ENABLE_RERANKER` | `True` | Toggle reranking model |\n| `RERANKER_SCORE_THRESHOLD` | `0.0` | Minimum reranker score (0.0–1.0) |\n| `COLLECTION_NAME` | `multimodal_data` | Default collection name |\n\n### Partial Filtering\n- Strict (default): fails if any collection doesn't support the filter\n- Flexible (`allow_partial_filtering: true` in config.yaml): succeeds if at least one collection supports it\n\n### VDB Filter Support\n\n| Feature | Milvus | Elasticsearch |\n|---------|--------|---------------|\n| NL filter generation | LLM emits Milvus string DSL | LLM emits Elasticsearch Query DSL clause list |\n| Filter syntax | String expression, e.g. `content_metadata[\"x\"] == \"y\"` | List of dicts using `metadata.content_metadata.\u003cfield>` paths |\n| UI support | Filter bar compiles Milvus string format | Filter bar compiles Elasticsearch list-of-dicts format from `/health` backend detection |\n\n## Notebooks\n- `notebooks/retriever_api_usage.ipynb` — RAG retriever API: search and end-to-end queries\n- `notebooks/nb_metadata.ipynb` — Metadata ingestion, filtering, and extraction from queries\n\n## Source Documentation\n- `docs/hybrid_search.md` — Hybrid dense + sparse search configuration\n- `docs/multi-collection-retrieval.md` — Multi-collection querying\n- `docs/custom-metadata.md` — Custom metadata schema, filtering expressions, filter generation\n- `docs/accuracy_perf.md` — Best practices for tuning ingestion/retrieval/generation settings\n- `docs/python-client.md` — Python library API for search and filtering\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":4129,"content_sha256":"318453642a660c430e004507f85df3ba5e534cc0a284cfb27d2fa4cff3d55ccd"},{"filename":"references/configure/summarization.md","content":"# Document Summarization\n\n## When to Use\n- User wants to generate summaries during document ingestion\n- User asks about summarization strategies or options\n- User wants to check summary status or progress\n\n## Restrictions\n- Not supported in lite mode (containerless/library-only deployment)\n- Requires Redis for status tracking and rate limiting\n- Collection must exist before uploading with `generate_summary: true`\n\n## Process\n1. Detect the deployment mode. Docker: edit the active env file. Helm: configure under `ingestor-server.envVars` in `values.yaml`. Library: use the upload API parameters directly (no env vars needed)\n2. Read `docs/summarization.md` for full configuration, env vars, and prompt customization\n3. Set `generate_summary: true` in the upload payload (per-request, no global toggle)\n4. Optionally configure `summary_options`: strategy, shallow mode, page filter\n5. Retrieve summary via `GET /v1/summary?collection_name=...&file_name=...`\n\n## Decision Table\n\n| Goal | Strategy | Notes |\n|------|----------|-------|\n| Fastest overview | `\"single\"` + `shallow_summary=true` + `page_filter` | Quick text-only extraction |\n| Best quality | `null` (iterative, default) + `shallow_summary=false` | Sequential refinement |\n| Balanced | `\"hierarchical\"` + `shallow_summary=true` | Parallel tree-based |\n\n## Agent-Specific Notes\n- `CONVERSATION_HISTORY` prerequisite does not apply — that's for query rewriting only\n- `SUMMARY_LLM_SERVERURL=\"\"` (empty) routes to NVIDIA cloud; `\"nim-llm:8000\"` for self-hosted\n- `SUMMARY_LLM_MAX_CHUNK_LENGTH` should be below the model's context window to leave room for prompt + output\n- Redis semaphore auto-resets on ingestor startup (prevents stale values from crashes)\n- If Redis is unavailable, summaries still generate but no real-time status tracking\n- Status entries have 24-hour TTL in Redis\n\n## Notebooks\n- `notebooks/summarization.ipynb` — complete examples for all strategies, status polling, library mode usage\n\n## Source Documentation\n- `docs/summarization.md` — env var reference, prompt customization, rate limiting, chunking details\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":2103,"content_sha256":"97f43900f70b3eb02dfe71e367ca9045ec2cea457afbca4a9c43c19aa4b3cd56"},{"filename":"references/configure/user-interface.md","content":"# User Interface\n\n## When to Use\n- User asks about the RAG UI, uploading documents, settings, reasoning traces, or metadata filtering\n- User wants to configure features via the web interface\n\n## Restrictions\n- Sample/experimentation UI — not intended for production\n- 100-file limit per upload batch; use multiple batches or API for bulk uploads\n- 10 MB max per image attachment\n\n## Process\n1. Read `docs/user-interface.md` for full UI documentation\n2. Access at `http://localhost:8090` (or `http://\u003cworkstation-ip>:8090` for remote)\n3. Configure RAG settings and feature toggles via Settings panel\n4. Use Filter Bar above chat input for metadata-filtered queries\n5. For reasoning-capable responses, inspect the collapsible reasoning panel above the assistant answer\n\n## Agent-Specific Notes\n- VLM Inference must be enabled in Settings > Feature Toggles before image attachments work\n- ECONNRESET errors on multi-file uploads — recommend API for bulk operations\n- Document summaries generate asynchronously; UI shows \"Generating summary...\" until complete\n- Document count in UI may lag slightly after ingestion\n- Metadata filtering supports AND/OR logic between filters (toggle via logic button)\n- The UI serializes `filter_expr` differently by backend: Milvus gets a string expression; Elasticsearch gets a list of Query DSL clauses. Backend is detected from `/v1/health` database service labels.\n- The reasoning panel renders both Agentic RAG stage traces and standard RAG `reasoning_content` chunks.\n- Custom metadata schema is set during collection creation via the Metadata Schema Editor\n\n## Source Documentation\n- `docs/user-interface.md` -- full UI documentation including settings, file types, metadata, and health monitoring\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1739,"content_sha256":"165312d168210a6324084c6b02335419499ffb09d4e4d4f594f685c9f1b72ef4"},{"filename":"references/configure/vlm.md","content":"# VLM, VLM Embeddings & Image Captioning\n\n## When to Use\nUser wants image understanding, visual content analysis, VLM inference, multimodal embeddings, VLM reranking, VLM reasoning output, or image captioning during ingestion.\n\n## Restrictions\n- Not available on B200 GPUs — use H100, A100 SXM 80GB, or RTX PRO 6000.\n- Requires extra GPU (GPU 1+ for 2-GPU systems, GPU 2+ for 3+ GPUs with fallback)\n- VLM embeddings: experimental, PDF-only, no summarization, no citations with page-as-image.\n- Image captioning on Helm: on-prem only (modify `values.yaml` to enable)\n\n## Process\n1. Detect the deployment mode (Docker / Helm / Library). Docker: edit the active env file. Helm: edit `values.yaml`. Library: edit `notebooks/config.yaml`\n2. Read the relevant source doc for detailed steps:\n - VLM generation: `docs/vlm.md`\n - VLM embeddings and VLM reranker: `docs/multimodal-retriever.md`\n - Image captioning: `docs/image_captioning.md`\n3. Start VLM NIM (self-hosted) or configure cloud endpoint (NVIDIA-hosted)\n4. Set the required variables in the active config:\n - Enabling: `ENABLE_VLM_INFERENCE=true` and `APP_NVINGEST_EXTRACTIMAGES=True`\n - Disabling: re-comment those variables in the env file\n5. Restart affected services and verify with a health check + image-containing document query\n\n## Decision Table\n\n| Goal | Source Doc | Docker Profile | Notes |\n|------|-----------|---------------|-------|\n| VLM replaces LLM | `docs/vlm.md` | `--profile vlm-generation` | LLM not started; set `VLM_TO_LLM_FALLBACK=false` |\n| VLM + LLM fallback | `docs/vlm.md` | `--profile vlm-only` | Needs 3+ GPUs; both VLM and LLM running |\n| VLM embeddings | `docs/multimodal-retriever.md` | `--profile vlm-embed` | Experimental; requires re-ingestion |\n| VLM reranker | `docs/multimodal-retriever.md` | `--profile vlm-rerank` or `--profile vlm-rag` | Set `APP_RANKING_MODELNAME` to `rerank-vl` model and `ENABLE_VLM_RERANKER_IMAGE_INPUT=True` |\n| Image captioning | `docs/image_captioning.md` | `--profile vlm-only` | Requires VLM NIM; Helm: on-prem only |\n| Multimodal query | `docs/multimodal-query.md` | (depends on VLM mode) | Image + text querying |\n\n## Agent-Specific Notes\n\n- `--profile vlm-generation` skips the LLM entirely — use `--profile vlm-only` for fallback mode\n- `VLM_TO_LLM_FALLBACK` defaults to `true`, but `vlm-generation` profile does not start LLM\n- Helm VLM: disable `nim-llm` and enable `nim-vlm` (VLM uses LLM's GPU allocation)\n- Helm fallback: keep both `nim-vlm` and `nim-llm` enabled, set `VLM_TO_LLM_FALLBACK: \"true\"`\n- VLM context window is limited — keep queries self-contained\n- VLM reasoning streams final answer in `content` and reasoning in `reasoning_content`; `VLM_FILTER_THINK_TOKENS` is retained for compatibility and no longer wraps reasoning in text sentinels\n- Image queries bypass reranking, including VLM reranking\n- Image captioning known issue: files without graphs/charts/tables/plots fail to ingest when captioning is enabled\n\n### Key Env Vars (always needed)\n- `ENABLE_VLM_INFERENCE=true` — master toggle\n- `APP_NVINGEST_EXTRACTIMAGES=True` — extract images during ingestion\n- `VLM_MS_GPU_ID=\u003cgpu-id>` — self-hosted GPU assignment\n\n## Notebooks\n- `notebooks/image_input.ipynb` — Multimodal queries with VLM (text + image)\n\n## Source Documentation\n- `docs/vlm.md` — VLM generation (self-hosted, NVIDIA-hosted, Helm, Library)\n- `docs/multimodal-retriever.md` — VLM embeddings (experimental)\n- `docs/image_captioning.md` — Image captioning during ingestion\n- `docs/multimodal-query.md` — Image + text querying\n- `docs/service-port-gpu-reference.md` — default GPU assignments for VLM and other NIMs\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":3662,"content_sha256":"e0af2f8143871c1381245ca4c5e973bf1361e8adeadea8378113e668d3b0881f"},{"filename":"references/deploy.md","content":"# RAG Blueprint Deployment\n\n## Phase 1: Environment Analysis\n\nRun this single command to collect all environment information at once:\n\n```bash\necho \"=== GPU ===\" && nvidia-smi --query-gpu=index,name,memory.total --format=csv,noheader 2>/dev/null || echo \"NO_GPU\"; echo \"=== VRAM ===\" && nvidia-smi --query-gpu=memory.total --format=csv,noheader,nounits 2>/dev/null | awk '{s+=$1} END {print s \"MB total\"}' || echo \"0MB total\"; echo \"=== DRIVER ===\" && cat /proc/driver/nvidia/version 2>/dev/null | head -1 || echo \"NO_DRIVER\"; echo \"=== CUDA ===\" && nvcc --version 2>/dev/null | grep \"release\" || echo \"NO_CUDA_TOOLKIT\"; echo \"=== DOCKER ===\" && docker --version 2>/dev/null || echo \"NO_DOCKER\"; echo \"=== COMPOSE ===\" && docker compose version 2>/dev/null || echo \"NO_COMPOSE\"; echo \"=== NVIDIA_TOOLKIT ===\" && docker info 2>/dev/null | grep -i \"runtimes.*nvidia\" || echo \"NO_NVIDIA_TOOLKIT\"; echo \"=== PYTHON ===\" && python3 --version 2>/dev/null || echo \"NO_PYTHON\"; echo \"=== DISK ===\" && df -h --output=avail / | tail -1; echo \"=== OS ===\" && cat /etc/os-release 2>/dev/null | grep -E \"^(NAME|VERSION)=\"; echo \"=== NGC_KEY ===\" && if [ -n \"$NGC_API_KEY\" ]; then echo \"NGC_KEY_SET\"; elif [ -n \"$NVIDIA_API_KEY\" ]; then echo \"NVIDIA_KEY_SET\"; elif grep -Eh '^(export[[:space:]]+)?(NGC_API_KEY|NVIDIA_API_KEY)=' deploy/compose/.env deploy/compose/nvdev.env 2>/dev/null | grep -v \"nvapi-your-key\" | grep -q \"nvapi-\"; then echo \"DOTENV_SET\"; else echo \"NOT_SET\"; fi; echo \"=== RUNNING ===\" && docker ps --format \"{{.Names}}\" 2>/dev/null | grep -E \"(rag-server|ingestor-server|nim-llm|nemotron-vlm-embedding|elasticsearch|milvus|seaweedfs)\" | head -15 || echo \"NO_RUNNING_SERVICES\"; echo \"=== PORTS ===\" && (ss -tlnp 2>/dev/null || netstat -tlnp 2>/dev/null) | grep -E \":(8081|8082|8090|9200|9010|19530) \" || echo \"PORTS_FREE\"; echo \"=== REPO ===\" && git rev-parse --show-toplevel 2>/dev/null && git describe --tags 2>/dev/null || echo \"NO_GIT_REPO\"; echo \"=== CACHE ===\" && du -sh ~/.cache/model-cache/ 2>/dev/null || echo \"NO_CACHE\"\n```\n\nPresent a summary table:\n\n| Check | Result |\n|-------|--------|\n| GPU(s) | (list with VRAM, or NO_GPU) |\n| Total VRAM | (sum in MB/GB) |\n| NVIDIA Driver | (version or NO_DRIVER) |\n| CUDA Toolkit | (version or NO_CUDA_TOOLKIT) |\n| Docker | (version or NO_DOCKER) |\n| Docker Compose | (version or NO_COMPOSE) |\n| NVIDIA Container Toolkit | (detected or NO_NVIDIA_TOOLKIT) |\n| Python | (version or NO_PYTHON) |\n| Free disk | (value) |\n| OS | (name + version) |\n| NGC_API_KEY | ENV_SET / DOTENV_SET / NOT_SET |\n| Existing services | (list or none) |\n| Port availability | (free or list conflicts) |\n| Repo | (tag/branch or NO_GIT_REPO) |\n| Model cache | (size or empty) |\n\n### Existing Services Warning\n\nIf RAG services are already running, tell the user briefly: \"Existing RAG services detected (list). Proceeding will restart them.\" Continue unless the user objects.\n\nIf the user wants to **switch deployment modes** (e.g., NVIDIA-hosted → self-hosted, or Docker → library), shut down the existing deployment first via `references/shutdown.md`, then proceed with the new mode.\n\nIf ports are occupied by non-RAG processes, tell the user which ports conflict and suggest stopping the conflicting process. This is a blocker.\n\n## Phase 2: NGC_API_KEY Handling\n\nCheck in this order:\n\n1. If `NGC_API_KEY` is set in the shell environment → proceed.\n2. If `NVIDIA_API_KEY` is set (common in library mode) → proceed silently.\n3. If `NGC_API_KEY` is in `deploy/compose/.env` or `deploy/compose/nvdev.env` (and not the placeholder `nvapi-your-key`) → load it and proceed.\n4. If none found → tell the user: \"NGC_API_KEY is required. Get one from https://org.ngc.nvidia.com/setup/api-keys and run: `export NGC_API_KEY=\\\"nvapi-...\\\"` — then tell me when done.\"\n5. After user confirms → re-check silently. If still not set, write placeholder to `.env` and tell the user to edit it.\n\n## Phase 3: Blocker Checks\n\nAutomatically check and report all blockers at once (don't stop at the first one):\n\nRead `docs/support-matrix.md` for current minimum versions and disk requirements, then check:\n\n- **Docker Compose below minimum**: \"Upgrade Docker Compose. See https://docs.docker.com/compose/install/linux/\"\n- **NVIDIA Driver below minimum** (if self-hosted): \"Upgrade NVIDIA driver. See `docs/support-matrix.md` for required version.\"\n- **NVIDIA Container Toolkit missing** (and self-hosted needed): \"Install NVIDIA Container Toolkit. See https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html\"\n- **Insufficient disk**: \"Check `docs/support-matrix.md` for disk requirements per deployment mode.\"\n- **No Docker and no Python 3.11+**: \"Install Docker or Python 3.11+ to proceed.\"\n\nList all blockers together so the user can fix them in one pass — don't make them fix one, re-run, fix another.\n\n## Phase 4: Route to Deployment Mode\n\n### User explicitly requests a mode\n- \"library mode\" / \"lite mode\" / \"no docker\" / \"python mode\" → read and follow `deploy/library.md`\n- \"docker\" / \"self-hosted\" / \"local\" → read and follow `deploy/docker.md` with mode **self-hosted**\n- \"cloud\" / \"nvidia-hosted\" / \"hosted\" → read and follow `deploy/docker.md` with mode **nvidia-hosted**\n- \"retrieval only\" / \"search only\" / \"no LLM\" → read and follow `deploy/docker.md` with mode **retrieval-only**\n- \"kubernetes\" / \"k8s\" / \"helm\" → read and follow `deploy/helm.md`\n- \"workbench\" / \"ai workbench\" → tell user to follow `deploy/workbench/README.md` (AI Workbench uses its own UI-driven workflow)\n\n### Docker is available (Docker + Compose detected)\n\n**Self-hosted eligible** — read `docs/support-matrix.md` (\"Hardware Requirements (Docker)\" section) for current GPU requirements. All of the following must also be true:\n- GPU count and type matches the Docker self-hosted requirements from the support matrix\n- ≥200 GB free disk (per `docs/support-matrix.md` \"Disk Space Requirements\")\n- NVIDIA Container Toolkit detected\n- NVIDIA driver meets minimum version from `docs/support-matrix.md` (\"Driver Versions\")\n\nIf self-hosted eligible → read and follow `deploy/docker.md` with mode **self-hosted**\n\n**Otherwise with Docker** → read and follow `deploy/docker.md` with mode **nvidia-hosted**\n\nTell the user WHY if they have some GPU but not enough:\n- \"You have [X GPU] with [Y GB] VRAM. Self-hosted requires [requirements from docs/support-matrix.md]. Deploying with NVIDIA-hosted cloud NIMs instead — faster startup, no model download.\"\n\n### Docker is available but Compose is not\n\nTell the user: \"Docker is installed but Docker Compose is below the minimum version (see `docs/support-matrix.md`). Install it: https://docs.docker.com/compose/install/linux/ — or use library mode instead.\"\n\nIf user chooses library mode → read and follow `deploy/library.md`\n\n### Docker is not available\n\n- Python 3.11+ available → read and follow `deploy/library.md` with mode **lite**\n- No Python → tell user to install Python 3.11+ or Docker\n\n## After Deployment\n\nOnce deployment completes, verify health:\n\n```bash\necho \"=== RAG Server ===\" && curl -s http://localhost:8081/v1/health?check_dependencies=true 2>/dev/null || echo \"RAG_SERVER_NOT_READY\"; echo \"=== Ingestor ===\" && curl -s http://localhost:8082/v1/health?check_dependencies=true 2>/dev/null || echo \"INGESTOR_NOT_READY\"\n```\n\nIf healthy, tell the user:\n- \"RAG Blueprint is running and healthy.\"\n- \"Ask me to configure features like VLM, query rewriting, guardrails, etc.\"\n- \"Ask me to shutdown when you're done.\"\n\nIf unhealthy, read `references/troubleshoot.md` and diagnose. Match error output against known issues, fix, and retry. Escalate to the user only if the fix requires their action (API key, data deletion).\n\n## Notebooks\n- `notebooks/launchable.ipynb` — Cloud deployment via Brev (alternative to local deployment)\n\n## Source Documentation\n- `docs/support-matrix.md` — GPU requirements, driver versions, disk space, supported platforms\n- `docs/service-port-gpu-reference.md` — port mappings and GPU assignments for all services\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":8075,"content_sha256":"e6b6dcc479b2ee051e82d15bc35bad1ce32f7c256bdbbf32f4e25893fee1e8ea"},{"filename":"references/deploy/docker-nvidia-hosted.md","content":"# Docker Deployment (NVIDIA-Hosted NIMs)\n\n## When to Use\n- User wants fast deployment without local model downloads\n- User has no GPU or limited GPU\n- User asks about cloud-hosted or NVIDIA API deployment\n- User wants to avoid 15–30 min NIM startup time\n\n## Restrictions\n- Requires internet access (calls NVIDIA cloud APIs)\n- NVIDIA-hosted endpoints have rate limits — large ingestions (>10 files) may hit 429 errors\n- NGC_API_KEY required for cloud API access\n- Docker and Compose minimum versions per `docs/support-matrix.md`\n\n## Process\n1. Read `docs/deploy-docker-nvidia-hosted.md` for full commands and env configuration\n2. Use `deploy/compose/nvdev.env` — pre-configured for cloud endpoints. Source it before compose commands: `source deploy/compose/nvdev.env`\n3. Start vector DB → ingestor → RAG server + frontend (no NIM startup needed)\n4. Verify: `docker ps` shows containers; UI at `http://localhost:8090`\n\n## Decision Table\n\n| Goal | Key Action |\n|------|------------|\n| Standard cloud deployment | Use `nvdev.env` (pre-configured for cloud) |\n| Zero-GPU | Use default Elasticsearch; only switch Milvus to CPU if the user explicitly chooses Milvus |\n| Large file ingestion | Reduce batch/concurrency settings to avoid 429s |\n| Maximum throughput | Use self-hosted deployment instead |\n\n## Agent-Specific Notes\n- First run: 5–10 min (image pulls only); subsequent: 1–2 min\n- No `nims.yaml` startup — all model inference is cloud-hosted\n- Persistent Docker data is in named `rag-vol-*` volumes, created automatically\n- All subsequent configure/restart operations should source the same env file used for the initial deploy (`deploy/compose/nvdev.env`)\n- For zero-GPU with Milvus specifically: switch Milvus to CPU-only by changing the GPU image tag to the equivalent non-GPU tag and setting `APP_VECTORSTORE_ENABLEGPUSEARCH=False`. Default Elasticsearch does not require this.\n- Rate limit mitigation for large ingestions: reduce `NV_INGEST_FILES_PER_BATCH`, `NV_INGEST_CONCURRENT_BATCHES`, `MAX_INGEST_PROCESS_WORKERS`, `NV_INGEST_MAX_UTIL` to minimum values\n\n## Source Documentation\n- `docs/deploy-docker-nvidia-hosted.md` — full step-by-step commands, env var blocks, CPU Milvus setup\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":2215,"content_sha256":"0ff4900057c89f5b13d7997f421692e5d5301f018214b20e221b8456864e9b90"},{"filename":"references/deploy/docker-retrieval-only.md","content":"# Retrieval-Only Deployment\n\n## When to Use\n- User wants search/retrieval without LLM generation\n- User asks to deploy only embedding + reranking services\n- User wants `/search` endpoint with an external LLM\n- User wants a lightweight, low-GPU deployment\n\n## Restrictions\n- `/generate` endpoint returns an error — no LLM is deployed\n- Self-hosted: 1 GPU, ~24 GB memory\n- NVIDIA-hosted: 0 GPUs (cloud embedding + reranking)\n\n## Process\n1. Read `docs/retrieval-only-deployment.md` for full commands, env vars, and API examples\n2. Choose variant: self-hosted (local NIMs), NVIDIA-hosted (cloud), or Helm\n3. For self-hosted: start only embedding + ranking NIMs, skip LLM\n4. For NVIDIA-hosted: set embedding/ranking server URLs to empty, skip NIM startup entirely\n5. For Helm: set `nimOperator.nim-llm.enabled=false`\n6. Start vector DB → ingestor → RAG server\n7. Verify health: `GET http://localhost:8081/v1/health?check_dependencies=true`\n\n## Decision Table\n\n| Goal | Variant | Key Difference |\n|------|---------|----------------|\n| Minimal GPU usage with local models | Self-hosted | 1 GPU, ~24 GB |\n| Zero GPU, cloud APIs | NVIDIA-hosted | Set server URLs to empty, skip NIM startup |\n| Kubernetes | Helm | Disable `nim-llm` in values.yaml |\n\n## Agent-Specific Notes\n- Permission errors on model cache → try `USERID=0` or `chmod -R 755 ~/.cache/model-cache`\n- Empty search results → verify documents ingested: `GET http://localhost:8082/v1/documents?collection_name=\u003cname>`\n- Users can send `/search` results to their own external LLM for generation\n\n## Source Documentation\n- `docs/retrieval-only-deployment.md` — full deployment commands, API examples, search payload options\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1688,"content_sha256":"44d629a51b746e89ed2680669598958661e4c4064eac58c23b2548675399026d"},{"filename":"references/deploy/docker-self-hosted.md","content":"# Docker Deployment (Self-Hosted NIMs)\n\n## When to Use\n- User wants full on-premises deployment with local NIM containers\n- User has supported GPUs and wants models running locally\n- User asks to deploy RAG Blueprint with Docker\n\n## Restrictions\n\nRead `docs/support-matrix.md` for current GPU requirements. Feature restrictions per GPU type:\n\n| GPU | Cannot Use |\n|-----|------------|\n| B200 | VLM, Guardrails, Nemotron Parse |\n| RTX PRO 6000 | Nemotron Parse |\n\n- Read `docs/support-matrix.md` for current minimum NVIDIA Driver, CUDA, Docker, and Compose versions\n- NVIDIA Container Toolkit required (`docker info` shows nvidia runtime)\n- Disk space per `docs/support-matrix.md` (\"Disk Space Requirements\")\n- If any prerequisite is missing, tell the user what to install before proceeding\n\n## Process\n1. Read `docs/deploy-docker-self-hosted.md` for full commands and env configuration\n2. Read `docs/support-matrix.md` for GPU compatibility and supported model combinations\n3. Verify container toolkit, prepare model cache directory, source `.env`\n4. Apply GPU-specific config per source docs\n5. Start NIMs → wait for healthy → start remaining services\n6. Verify: `docker ps` shows all containers healthy; UI at `http://localhost:8090`\n\n## Decision Table\n\n| Goal | Profile Flag | Notes |\n|------|-------------|-------|\n| Full deployment (default) | (none) | LLM + embedding + ranking + OCR + detection |\n| Text-only RAG (lighter) | `--profile rag` | Skip OCR/detection NIMs |\n| Ingestion workload only | `--profile ingest` | Embedding + OCR + detection |\n| VLM replaces LLM | `--profile vlm-generation` | Not on B200 |\n| Advanced PDF extraction | `--profile nemotron-parse` | Not on B200 or RTX PRO 6000 |\n\n## Agent-Specific Notes\n- First run: 15–30 min (model downloads ~100–150 GB, no progress bar); subsequent: 2–5 min\n- Monitor download progress: `du -sh ~/.cache/model-cache/`\n- Permission error on model cache → try `USERID=0` instead of `USERID=$(id -u)`\n- Cloud NIM section in `deploy/compose/.env` must be commented out for self-hosted\n- Rebuild after code changes: add `--build` flag to compose up commands\n\n## Source Documentation\n- `docs/deploy-docker-self-hosted.md` — full step-by-step commands, env vars, GPU assignments\n- `docs/support-matrix.md` — GPU compatibility, supported models, hardware requirements\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":2340,"content_sha256":"1fd41a62734cc0cfb19d2333938191ec47dbe3a00f54f93aa7bf15ce0b1c18f1"},{"filename":"references/deploy/docker.md","content":"# RAG Docker Deployment\n\n## Determine Mode\n\nIf routed here from the deploy workflow, the mode (self-hosted, nvidia-hosted, or retrieval-only) was already decided. Use it.\n\nIf invoked directly without a mode, auto-detect:\n\n```bash\necho \"=== COMPOSE ===\" && docker compose version 2>/dev/null || echo \"NO_COMPOSE\"; echo \"=== GPU ===\" && nvidia-smi --query-gpu=name,memory.total --format=csv,noheader 2>/dev/null || echo \"NO_GPU\"; echo \"=== DISK ===\" && df -h --output=avail / | tail -1; echo \"=== RUNNING ===\" && docker ps --format \"{{.Names}}\" 2>/dev/null | grep -E \"(rag-server|ingestor-server|nim-llm|nemotron-vlm-embedding|elasticsearch|milvus)\" | head -10 || echo \"NONE_RUNNING\"\n```\n\nIf NO_COMPOSE: stop and tell the user to install Docker Compose (see `docs/support-matrix.md` for minimum version).\n\nRead `docs/support-matrix.md` (\"Hardware Requirements (Docker)\" section) for current GPU requirements, then:\n- GPU count/type meets self-hosted requirements from the support matrix, and 200+ GB free disk → **self-hosted**\n- Any GPU or no GPU with ≥50 GB free disk → **nvidia-hosted** (default Elasticsearch does not require a GPU)\n- User explicitly says \"retrieval only\" / \"no LLM\" / \"search only\" → **retrieval-only**\n\nAuto-route based on hardware. Only ask if two modes are equally valid and the user's intent is ambiguous.\n\n## Verify NGC_API_KEY\n\nAuto-check all possible locations before asking:\n\n```bash\nif [ -n \"$NGC_API_KEY\" ] || [ -n \"$NVIDIA_API_KEY\" ]; then echo \"ENV_SET\"; elif grep -Eh '^(export[[:space:]]+)?(NGC_API_KEY|NVIDIA_API_KEY)=' deploy/compose/.env deploy/compose/nvdev.env 2>/dev/null | grep -v \"nvapi-your-key\" | grep -q \"nvapi-\"; then echo \"DOTENV_SET\"; else echo \"NOT_SET\"; fi\n```\n\n- **ENV_SET**: proceed silently.\n- **DOTENV_SET**: load the env file that contains the key and proceed.\n- **NOT_SET**: ask the user to provide it. This is the only thing to ask for.\n\n## Docker Login\n\nAuto-check if already logged in:\n\n```bash\ngrep -q \"nvcr.io\" ~/.docker/config.json 2>/dev/null && echo \"ALREADY_LOGGED_IN\" || echo \"NOT_LOGGED_IN\"\n```\n\nIf already logged in → proceed silently.\n\nIf not logged in → tell the user to run this themselves (the key gets expanded in agent logs):\n\n> Please run in your terminal: `echo \"${NGC_API_KEY}\" | docker login nvcr.io -u '$oauthtoken' --password-stdin`\n\nWait for confirmation only if login was needed.\n\n## Deploy\n\nBased on the mode, read and follow the appropriate reference:\n\n- **Self-hosted**: read and follow `docker-self-hosted.md`\n- **NVIDIA-hosted**: read and follow `docker-nvidia-hosted.md`\n- **Retrieval-only**: read and follow `docker-retrieval-only.md`\n\nDocker Compose persistent data is stored in named `rag-vol-*` volumes. Do not look for new data under the legacy `deploy/compose/volumes/` tree unless the user is migrating old data.\n\n## Post-Deploy Verification\n\nRun health checks:\n\n```bash\nsleep 5; echo \"=== RAG ===\" && curl -s http://localhost:8081/v1/health?check_dependencies=true 2>/dev/null || echo \"RAG_NOT_READY\"; echo \"=== INGESTOR ===\" && curl -s http://localhost:8082/v1/health?check_dependencies=true 2>/dev/null || echo \"INGESTOR_NOT_READY\"; echo \"=== CONTAINERS ===\" && docker ps --format \"table {{.Names}}\\t{{.Status}}\" 2>/dev/null | grep -E \"(rag|elasticsearch|milvus|seaweedfs|nim|ingest|embedding|ranking)\" | head -20\n```\n\nIf services are still initializing, automatically poll every 30 seconds:\n- **NVIDIA-hosted**: poll until healthy or 5 minutes elapsed (no model downloads needed).\n- **Self-hosted**: poll until healthy or 15 minutes elapsed (model downloads on first run).\n- **Retrieval-only**: poll until healthy or 5 minutes elapsed.\n\nShow progress to the user during polling.\n\n## On Success\n\nTell the user:\n- \"RAG Blueprint is running and healthy. Open http://localhost:8090 to use the UI.\" (skip for retrieval-only)\n- \"Ask me to configure features (VLM, query rewriting, guardrails, etc.)\"\n- \"Ask me to shutdown when you're done.\"\n\n## On Error\n\n1. Read the error output from the failed command.\n2. Read `references/troubleshoot.md` to match against common issues (port conflict, disk full, NGC auth, GPU OOM).\n3. Apply the fix and retry.\n4. If still failing, report the specific error to the user with the fix that was attempted.\n\n## Source Documentation\n- `docs/support-matrix.md` — GPU requirements, hardware compatibility, disk space\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":4354,"content_sha256":"96fe558100135ccfe767212b9b83ffd5c0540e2b36b9b329cbbe6cde4a310d49"},{"filename":"references/deploy/helm-mig.md","content":"# MIG GPU Deployment\n\n## When to Use\n- User wants fine-grained GPU allocation on Kubernetes using MIG slices\n- User has H100 GPUs and wants to share them across RAG services\n- User asks about Multi-Instance GPU deployment\n\n## Restrictions\n- Requires H100 80GB HBM3 GPUs (MIG-compatible)\n- MIG profiles in this guide are specific to H100 80GB — other GPUs need different profiles\n- Requires cloned repository (MIG config files in `deploy/helm/`)\n- All standard Helm prerequisites apply (GPU Operator, NIM Operator, StorageClass)\n- Ingestion profile is scaled down with MIG — large bulk ingestion jobs may fail\n\n## Process\n1. Read `docs/mig-deployment.md` for full configuration, commands, and MIG slice definitions\n2. Enable MIG with mixed strategy on ClusterPolicy\n3. Apply MIG ConfigMap and label the node\n4. Verify node labels show `mig.config.state: \"success\"` before proceeding\n5. Install Helm chart with `-f mig-slicing/values-mig.yaml`\n\n## Decision Table\n\n| Goal | Source Doc | Key Action |\n|------|-----------|------------|\n| Standard MIG on H100 | `docs/mig-deployment.md` | Apply MIG config, label node, install chart |\n| RTX PRO 6000 with MIG | `docs/mig-deployment.md` | Also uncomment model section in values.yaml |\n| Custom MIG profiles | NVIDIA MIG User Guide | Modify `mig-config.yaml` for different GPU types |\n\n## Agent-Specific Notes\n- Must wait for `mig.config.state: \"success\"` on the node before Helm install — if not present, wait and re-check\n- Default H100 MIG layout (see `docs/mig-deployment.md` for current GPU count and slice definitions): GPU 0 → small slices, GPU 1 → mixed slices, GPU 2 → full-GPU slice\n- LLM gets the largest slice (`7g.80gb`); embedding/Milvus/ingest share small slices\n- RTX PRO 6000 variant: uncomment model section in values.yaml, then use both `-f values.yaml -f mig-slicing/values-mig.yaml`\n- Uninstall follows standard Helm procedure (see Helm deployment docs)\n\n## Source Documentation\n- `docs/mig-deployment.md` — full MIG config, ClusterPolicy patches, node labeling, verification, Helm install commands\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":2076,"content_sha256":"d1b0fdff3e5486c160623a68021226adb5ccff71a4ac44c6a8e30516081ff962"},{"filename":"references/deploy/helm-openshift.md","content":"# Helm Deployment on OpenShift\n\n## When to Use\n- Cluster is Red Hat OpenShift or OKD (`clusterversion` resource present, or `route.openshift.io` API available)\n- User mentions OpenShift, OKD, or RHEL OpenShift in the deployment context\n- User wants OpenShift Routes with edge TLS instead of `kubectl port-forward` for external access\n\n## Restrictions\n\nRead `docs/support-matrix.md` for current Kubernetes, Helm, and OS version requirements.\n\n- Requires GPU Operator + NIM Operator pre-installed on the OpenShift cluster\n- Default StorageClass must be configured for PVC provisioning\n- Disk space per `docs/support-matrix.md` (~200 GB per node for NIM cache + images + PVCs)\n- NeMo Guardrails not available in Helm deployment\n- OpenShift's default Route timeout is 30 s — the chart sets `haproxy.router.openshift.io/timeout: 300s` on the RAG-server Route, but manually-created Routes need this annotation\n\n## Process\n\n1. Read `docs/deploy-helm-openshift.md` for full commands and overlay file usage.\n2. Ensure prerequisites: GPU Operator, NIM Operator, StorageClass, `NGC_API_KEY`, and a namespace:\n ```bash\n export NAMESPACE=\"${NAMESPACE:-rag}\"\n kubectl create namespace \"$NAMESPACE\" 2>/dev/null || true\n ```\n3. Install the chart with the `values-openshift.yaml` overlay (the overlay inherits the base `values.yaml`, so it does not need to be passed separately):\n ```bash\n helm upgrade --install rag -n \"$NAMESPACE\" \u003cchart> \\\n -f values-openshift.yaml \\\n --set imagePullSecret.password=\"$NGC_API_KEY\" \\\n --set ngcApiSecret.password=\"$NGC_API_KEY\" \\\n --timeout 15m\n ```\n The overlay turns on `openshift.enabled`, which makes the chart create OpenShift Routes with edge TLS and an `anyuid` SCC RoleBinding for all required ServiceAccounts — no manual `oc adm policy add-scc-to-user` is needed.\n4. Link the pull secret to the NIM cache ServiceAccount after it exists:\n ```bash\n oc secrets link nim-cache-sa ngc-secret --for=pull -n \"$NAMESPACE\"\n ```\n5. Monitor pods and Routes, then access the UI via the frontend Route's external host (no `port-forward` required):\n ```bash\n kubectl get pods -n \"$NAMESPACE\"\n kubectl get route -n \"$NAMESPACE\"\n ```\n\n## Decision Table\n\n| Goal | Key Action |\n|------|------------|\n| Standard OpenShift deploy | Apply the `values-openshift.yaml` overlay |\n| Constrained / API-hosted demo | Also apply `values-openshift-test.yaml` for tolerations, resource tuning, disabled observability, and API-hosted LLM |\n| GPU nodes with taints | Use `--set-json` toleration entries per NIM, or copy the pattern from `values-openshift-test.yaml` |\n\n## Agent-Specific Notes\n- OpenShift Routes provide external access directly — do not propose `kubectl port-forward` workflows once Routes exist\n- If a NIM pod hits `CrashLoopBackOff` with SCC-related errors, confirm `openshift.enabled: true` is set in the active overlay\n- If NIMCache jobs or pods hit `ImagePullBackOff`, confirm the NGC pull secret is linked to `nim-cache-sa`\n- Route timeouts during long requests → annotate the affected Route with `haproxy.router.openshift.io/timeout=300s`\n- `helm uninstall` does not remove PVCs — clean up with `kubectl delete nimcache --all -n \"$NAMESPACE\" && kubectl delete pvc --all -n \"$NAMESPACE\"`\n\n## Source Documentation\n- `docs/deploy-helm-openshift.md` — OpenShift Routes, SCC, overlay usage, OpenShift-specific troubleshooting\n- `docs/deploy-helm.md` — standard (non-OpenShift) Helm deployment for comparison\n- `deploy/helm/nvidia-blueprint-rag/values-openshift.yaml` — the overlay itself\n- `deploy/helm/nvidia-blueprint-rag/values-openshift-test.yaml` — reference overlay for constrained/API-hosted setups\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":3685,"content_sha256":"bce3e5b8a788628a372e12e032f81c610a8abcbd29888c16178b5a8dc45324ea"},{"filename":"references/deploy/helm-standard.md","content":"# Helm Deployment\n\n## When to Use\n- User wants to deploy RAG Blueprint on Kubernetes\n- User asks about Helm chart installation (from NGC or local repo)\n- User mentions Kubernetes, k8s, or Helm in deployment context\n\n## Restrictions\n\nRead `docs/support-matrix.md` for current Kubernetes, Helm, and OS version requirements.\n\n- Requires GPU Operator + NIM Operator pre-installed\n- Default StorageClass must be configured for PVC provisioning\n- Disk space per `docs/support-matrix.md`\n- NeMo Guardrails not available in Helm deployment\n- Image captioning: on-prem only (requires `values.yaml` changes; see `docs/image_captioning.md`)\n\n## Process\n\n### Option A: Deploy from NGC (Remote Chart)\n1. Read `docs/deploy-helm.md` for full commands and values\n2. Ensure prerequisites: GPU Operator, NIM Operator, StorageClass, NGC_API_KEY\n3. Install chart, monitor pods, port-forward frontend\n\n### Option B: Deploy from Repository (Local Chart)\n1. Read `docs/deploy-helm-from-repo.md` for full commands and repo setup\n2. Add required Helm repos, run `helm dependency update`, install from local path\n\n### RTX PRO 6000 Variant\n1. Uncomment model section under `nimOperator.nim-llm.model` in `values.yaml`\n2. See source docs for engine/precision/GPU settings\n\n## Decision Table\n\n| Goal | Option | Key Action |\n|------|--------|------------|\n| Quick deploy from published chart | NGC (Option A) | `helm upgrade --install` with NGC URL |\n| Customized chart | Local repo (Option B) | Clone, modify values, `helm dependency update` |\n| RTX PRO 6000 GPUs | Either option | Uncomment model section in values.yaml |\n| Retrieval-only (no LLM) | Either option | `--set nimOperator.nim-llm.enabled=false` |\n\n## Agent-Specific Notes\n- First deployment: 60–70 min (model cache download); subsequent: 10–15 min\n- Pods in `ContainerCreating`/`Init` for extended time is normal during cache download\n- PVCs are not removed by `helm uninstall` — delete manually: `kubectl delete nimcache --all -n rag && kubectl delete pvc --all -n rag`\n- Port-forwarding may timeout for large file ingestion — not suitable for bulk uploads\n- All configurable endpoints documented in `deploy/helm/nvidia-blueprint-rag/endpoints.md`\n\n## Source Documentation\n- `docs/deploy-helm.md` — NGC remote chart deployment, prerequisites, monitoring\n- `docs/deploy-helm-from-repo.md` — local chart deployment, repo setup, dependency management\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":2396,"content_sha256":"54744f31ee5825c9a44a80ef851ecd9a34beb645e8c6cc2b65b278266ae497d1"},{"filename":"references/deploy/helm.md","content":"# RAG Helm Deployment\n\nIf routed here from the deploy workflow, proceed directly to Phase 1.\n\n## Phase 1: Prerequisites Check\n\nRun all checks at once:\n\n```bash\necho \"=== KUBECTL ===\" && kubectl version --client 2>/dev/null || echo \"NO_KUBECTL\"; echo \"=== HELM ===\" && helm version --short 2>/dev/null || echo \"NO_HELM\"; echo \"=== STORAGECLASS ===\" && kubectl get storageclass 2>/dev/null || echo \"NO_STORAGECLASS\"; echo \"=== NODES ===\" && kubectl get nodes -o wide 2>/dev/null || echo \"NO_CLUSTER_ACCESS\"; echo \"=== GPU_OPERATOR ===\" && kubectl get pods -n gpu-operator 2>/dev/null | grep -i running || echo \"NO_GPU_OPERATOR\"; echo \"=== NIM_OPERATOR ===\" && kubectl get pods -n nim-operator 2>/dev/null | grep -i running || echo \"NO_NIM_OPERATOR\"; echo \"=== NAMESPACE ===\" && kubectl get namespace rag 2>/dev/null && echo \"NAMESPACE_EXISTS\" || echo \"NO_NAMESPACE\"; echo \"=== HELM_RELEASE ===\" && helm list -n rag 2>/dev/null | grep rag || echo \"NO_EXISTING_RELEASE\"; echo \"=== PODS ===\" && kubectl get pods -n rag 2>/dev/null | head -10 || echo \"NO_PODS\"; echo \"=== NGC_KEY ===\" && [ -n \"$NGC_API_KEY\" ] && echo \"NGC_API_KEY SET\" || echo \"NGC_API_KEY NOT_SET\"; echo \"=== GPU_RESOURCES ===\" && kubectl get nodes -o json 2>/dev/null | grep -o '\"nvidia.com/gpu\": \"[0-9]*\"' || echo \"NO_GPU_RESOURCES\"\n```\n\nRead `docs/support-matrix.md` for current Kubernetes, Helm, and OS version requirements.\n\n| Requirement | Check |\n|-------------|-------|\n| Kubernetes | Per `docs/support-matrix.md` |\n| Helm | Per `docs/support-matrix.md` |\n| NVIDIA GPU Operator | Installed and running |\n| NVIDIA NIM Operator | Installed and running |\n| Default StorageClass | Configured (e.g. local-path-provisioner) |\n| Disk space | ≥200 GB per node |\n| NGC_API_KEY | Set in environment |\n\nReport all missing prerequisites together so the user can fix everything in one pass.\n\nIf NGC_API_KEY is NOT_SET: this is the one thing we must ask the user for.\n\nIf an existing Helm release is detected: warn \"Existing RAG Helm release found. Proceeding will upgrade it.\" Continue unless user objects.\n\n## Phase 2: Route to Reference\n\nAuto-detect the GPU variant and cluster flavor from cluster nodes (not the local machine):\n\n```bash\necho \"=== GPU_LABELS ===\" && kubectl get nodes -o json 2>/dev/null | grep -oE '\"nvidia.com/gpu.product\":\\s*\"[^\"]*\"' | sort -u || echo \"NO_GPU_LABELS\"; echo \"=== MIG ===\" && kubectl get nodes -o json 2>/dev/null | grep -oE '\"nvidia.com/mig.strategy\":\\s*\"[^\"]*\"' || echo \"NO_MIG\"; echo \"=== OPENSHIFT ===\" && (kubectl get clusterversion 2>/dev/null | grep -q . && echo \"OPENSHIFT_DETECTED\") || (kubectl api-resources 2>/dev/null | grep -qi \"route.openshift.io\" && echo \"OPENSHIFT_DETECTED\") || echo \"NOT_OPENSHIFT\"\n```\n\nDetermine variant from node GPU labels and cluster flavor:\n\nRoute based on detection:\n\n- **OpenShift / OKD** (`clusterversion` resource present, or `route.openshift.io` API available, or user mentions OpenShift / RHEL OpenShift) → read and follow `helm-openshift.md`\n- **MIG enabled** → read and follow `helm-mig.md`\n- **RTX PRO 6000** → read and follow `helm-standard.md` (use the RTX values.yaml variant described there)\n- **Standard (everything else)** → read and follow `helm-standard.md`\n\nAsk the user only if the variant is genuinely ambiguous. Default to standard deployment.\n\n## Phase 3: Expected Timelines\n\nSet expectations with the user:\n\n| Scenario | Duration |\n|----------|----------|\n| First deployment | 60–70 min (NIM cache download ~40–50 min, NIMService init ~10–15 min, pod startup ~5–10 min) |\n| Subsequent deployments | 10–15 min (model caches already populated) |\n\nPods in `ContainerCreating` or `Init` state for extended periods is normal — models download in the background without progress indicators.\n\n## Phase 4: Verification\n\nAfter deployment completes, verify:\n\n```bash\necho \"=== PODS ===\" && kubectl get pods -n rag; echo \"=== NIMCACHE ===\" && kubectl get nimcache -n rag; echo \"=== NIMSERVICE ===\" && kubectl get nimservice -n rag\n```\n\nWait for all pods to reach `Running` status. Poll every 60 seconds for up to 70 minutes (first deployment involves model downloads). Show progress.\n\nOnce pods are running, port-forward and verify health:\n\n```bash\nkubectl port-forward -n rag service/rag-server 8081:8081 --address 0.0.0.0 & kubectl port-forward -n rag service/rag-frontend 3000:3000 --address 0.0.0.0 & sleep 3 && curl -s http://localhost:8081/v1/health?check_dependencies=true 2>/dev/null || echo \"RAG_NOT_READY\"\n```\n\n## Phase 5: Uninstall\n\nIf the user wants to tear down:\n\n```bash\nhelm uninstall rag -n rag\nkubectl delete nimcache --all -n rag\nkubectl delete pvc --all -n rag\n```\n\n## On Success\n\nTell the user:\n- \"RAG Blueprint is running on Kubernetes. Access the UI at http://localhost:3000 (via port-forward).\"\n- \"Ask me to configure features (VLM, query rewriting, guardrails, etc.)\"\n- \"Ask me to shutdown when you're done.\"\n\n## On Error\n\n1. Check pod status and events: `kubectl describe pod \u003cfailing-pod> -n rag` and `kubectl get events -n rag --sort-by='.lastTimestamp' | tail -20`.\n2. Read pod logs: `kubectl logs \u003cfailing-pod> -n rag --tail 50`.\n3. Read `references/troubleshoot.md` to match against common issues (PVC pending, OOM, image pull failure, port conflict).\n4. Apply the fix and retry. If the fix requires data deletion (PVCs, namespace), confirm with user first.\n\n## Source Documentation\n- `docs/support-matrix.md` — Kubernetes/Helm version requirements, GPU compatibility\n- `docs/deploy-helm.md` — standard Helm deployment from NGC\n- `docs/deploy-helm-from-repo.md` — Helm deployment from local repo\n- `docs/deploy-helm-openshift.md` — Red Hat OpenShift deployment with Routes, SCC, and the `values-openshift.yaml` overlay\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":5724,"content_sha256":"edcdaa8aecde194b519e93243b820e5d23763e52aa7984d11b82d94e250933cb"},{"filename":"references/deploy/library-full.md","content":"# Library Mode (Full)\n\n## When to Use\n- User wants programmatic Python access to RAG via `nvidia_rag` package\n- User prefers code-level configuration over Docker-based servers\n- User asks about library mode, Python client, or `NvidiaRAG`/`NvidiaRAGIngestor`\n\n## Restrictions\n- Python 3.11+ (\u003c 3.14)\n- Docker still required for backend services (Milvus, NV-Ingest, Redis, optionally NIMs)\n- Self-hosted NIMs require supported GPUs (see `docs/support-matrix.md`)\n\n## Process\n1. Read `docs/python-client.md` for full API reference, configuration, and backend setup\n2. Create virtual environment and install `nvidia-rag[all]`\n3. Start backend services via Docker (Milvus, NV-Ingest + Redis, optionally NIMs)\n4. Load config from `notebooks/config.yaml` using `NvidiaRAGConfig.from_yaml()`\n5. Create `NvidiaRAGIngestor` and `NvidiaRAG` instances\n6. Use `ingestor.create_collection()`, `ingestor.upload_documents()`, `rag.generate()`, `rag.search()`\n\n## Decision Table\n\n| Goal | Source Doc | Key Action |\n|------|-----------|------------|\n| Self-hosted (local GPUs) | `docs/python-client.md` | Start nims.yaml + set on-prem config |\n| Cloud (NVIDIA-hosted) | `docs/python-client.md` | Skip nims.yaml, override server URLs in config |\n| Custom prompts | `docs/python-client.md` | Pass `prompts=` to NvidiaRAG constructor |\n| Summarization | `docs/python-client.md` | `generate_summary=True` in upload_documents |\n\n## Agent-Specific Notes\n- Config file: `notebooks/config.yaml`; env file: `notebooks/.env_library`\n- Docker login is interactive — tell user to run `docker login nvcr.io` themselves\n- For cloud deployment: override `config.embeddings.server_url`, `config.llm.server_url`, etc. in code\n- Config changes take effect immediately (no container restart needed, unlike Docker mode)\n- Prompt customization via constructor: `NvidiaRAG(config=config, prompts=\"custom_prompts.yaml\")`\n- `upload_documents()` is async — returns `task_id` for status polling\n- NV-Ingest cloud endpoints must be exported before starting NV-Ingest container\n\n## Notebooks\n- `notebooks/rag_library_usage.ipynb` — complete walkthrough: setup, ingestion, querying, search, summaries\n\n## Source Documentation\n- `docs/python-client.md` — full API reference, backend setup, configuration, cloud/self-hosted options\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":2291,"content_sha256":"7bc35d06548d0fbf9e5e4804566fee1a8ad66085aa6c5ab91ff91788778b88b9"},{"filename":"references/deploy/library-lite.md","content":"# Library Mode (Lite / Containerless)\n\n## When to Use\n- Quick prototyping with zero infrastructure (no Docker, no GPU)\n- User wants the fastest path to try RAG\n- CI/CD pipelines needing lightweight RAG testing\n\n## Restrictions\n- No image/table/chart citations\n- No document summarization\n- Subject to NVIDIA API rate limits (cloud-hosted inference)\n- Requires Python 3.11+ (\u003c 3.14), internet access, and `NGC_API_KEY`\n\n## Process\n1. Read `docs/python-client.md` for full library mode documentation\n2. Create virtualenv and install: `pip install nvidia-rag[all]`\n3. Ensure `NGC_API_KEY` is exported — maps to `NVIDIA_API_KEY` internally\n4. Run the lite notebook: `jupyter lab notebooks/rag_library_lite_usage.ipynb`\n\n## Agent-Specific Notes\n- `NVIDIA_API_KEY` (used by `nvidia_rag` package) must be set from `NGC_API_KEY`. In the notebook, copy the NGC key into the NVIDIA key variable: `NVIDIA_API_KEY` = value of `NGC_API_KEY` (falling back to empty string if unset)\n- Lite config lives in `notebooks/config.yaml`; override `server_url` for embeddings to the NVIDIA API Catalog endpoint (see `docs/python-client.md` for current URL), and set LLM/ranking URLs to empty string for cloud defaults\n- Milvus Lite runs embedded (no container), NV-Ingest runs as subprocess (no container)\n- Also install `python-dotenv jupyterlab` for notebook support\n\n## When Not to Use\n- Production workloads — use Docker or Kubernetes\n- Large-scale ingestion — rate limits apply\n- Need citations from images/tables/charts or document summarization\n\n## Notebooks\n| Notebook | Description |\n|----------|-------------|\n| `notebooks/rag_library_lite_usage.ipynb` | End-to-end lite mode: collection creation, ingestion, querying, search |\n\n## Source Documentation\n- `docs/python-client.md` -- full library mode documentation (lite and full)\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":1823,"content_sha256":"c7cd255ad8f43e0120b1caeae268874275090acd24acf54091a47771f3de3bb0"},{"filename":"references/deploy/library.md","content":"# RAG Library Mode Setup\n\n## Determine Mode\n\nIf routed here from the deploy workflow, the mode (full or lite) may already be decided. Use it.\n\nIf invoked directly, auto-detect:\n\n```bash\necho \"=== DOCKER ===\" && docker --version 2>/dev/null || echo \"NO_DOCKER\"; echo \"=== GPU ===\" && nvidia-smi --query-gpu=name --format=csv,noheader 2>/dev/null || echo \"NO_GPU\"; echo \"=== PYTHON ===\" && python3 --version 2>/dev/null || echo \"NO_PYTHON\"; echo \"=== PKG_MANAGER ===\" && which uv 2>/dev/null && echo \"UV_AVAILABLE\" || (which pip3 2>/dev/null && echo \"PIP_AVAILABLE\" || echo \"NO_PKG_MANAGER\"); echo \"=== VENV ===\" && ls -d .venv/ venv/ nvidia-rag-env/ 2>/dev/null || echo \"NO_EXISTING_VENV\"; echo \"=== INSTALLED ===\" && pip3 show nvidia_rag 2>/dev/null | head -3 || echo \"NOT_INSTALLED\"\n```\n\n- Docker available → **full** (Python API + Docker backend services)\n- No Docker or user explicitly says \"lite\" / \"no docker\" / \"containerless\" → **lite**\n\nAuto-route based on Docker availability. Only ask if both modes are equally valid.\n\n## Verify NGC_API_KEY\n\nAuto-check all locations:\n\n```bash\nif [ -n \"$NGC_API_KEY\" ]; then echo \"NGC_KEY_SET\"; elif [ -n \"$NVIDIA_API_KEY\" ]; then echo \"NVIDIA_KEY_SET\"; else echo \"NOT_SET\"; fi\n```\n\nIf NOT_SET: ask the user. Otherwise proceed silently.\n\n## Deploy\n\nBased on the mode:\n\n- **Full**: read and follow `library-full.md`\n- **Lite**: read and follow `library-lite.md`\n\n## On Success\n\nTell the user:\n- Which mode was set up and how to start using it (notebook or Python script)\n- \"Ask me to configure features, change models, etc.\"\n- \"Ask me to shutdown backend services when done (if full mode).\"\n\n## On Error\n\n1. Read the error output (pip install failure, import error, service connection error).\n2. Read `references/troubleshoot.md` to match against common issues.\n3. Common fixes to try:\n - `pip install` failure → try `uv pip install` or check Python version ≥3.11.\n - Import error → check if virtual environment is activated.\n - Connection error to backend services → check Docker containers are running.\n4. Retry the failed step after fixing.\n5. If still failing, report the specific error to the user.\n\n## Source Documentation\n- `docs/python-client.md` — Python library API, installation, full and lite mode setup\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":2278,"content_sha256":"a98a09d5bd9b519bc3920481c3fcd7b0973760f10de6ce6858748f9a001c5672"},{"filename":"references/shutdown.md","content":"# RAG Shutdown\n\nStopping containers and processes does not require confirmation. Deleting data (volumes, cache, images) does.\n\n## Step 1: Detect What Is Running\n\nDetect all deployment modes — Docker, K8s, and library:\n\n```bash\necho \"=== DOCKER ===\" && docker ps --format \"table {{.Names}}\\t{{.Status}}\\t{{.Image}}\" 2>/dev/null || echo \"NO_DOCKER\"; echo \"=== LIBRARY ===\" && ps aux | grep -E \"(nvidia_rag|uvicorn|jupyter)\" | grep -v grep || echo \"NO_LIBRARY_PROCESSES\"; echo \"=== K8S ===\" && kubectl get pods -n rag 2>/dev/null | head -10 || echo \"NO_K8S\"; echo \"=== HELM ===\" && helm list -n rag 2>/dev/null | grep rag || echo \"NO_HELM_RELEASE\"\n```\n\nBased on what's detected, execute the appropriate shutdown path below. If multiple modes are active (e.g., Docker + library), stop all of them.\n\n## Step 2: Stop Services (Reverse Startup Order)\n\nStop in this order — reverse of deployment. Only stop what is actually running (detected in Step 1).\n\n### 2a: Optional Services\n\nStop these first if they are running:\n\n```bash\ndocker compose -f deploy/compose/docker-compose-nemo-guardrails.yaml down 2>/dev/null; docker compose -f deploy/compose/observability.yaml down 2>/dev/null\n```\n\n### 2b: Application Services\n\n```bash\ndocker compose -f deploy/compose/docker-compose-rag-server.yaml down; docker compose -f deploy/compose/docker-compose-ingestor-server.yaml down\n```\n\n### 2c: Vector DB\n\n```bash\ndocker compose -f deploy/compose/vectordb.yaml down\n```\n\nIf a profile-specific vector DB stack was started and containers remain, include the profile explicitly:\n```bash\ndocker compose -f deploy/compose/vectordb.yaml --profile elasticsearch down\n```\n\n### 2d: NIMs (Self-Hosted Only)\n\nOnly present if self-hosted deployment was used:\n\n```bash\ndocker compose -f deploy/compose/nims.yaml down\n```\n\nThis stops ALL NIM containers (LLM, embedding, ranking, OCR, detection, and any profile-specific NIMs like VLM, audio, nemotron-parse).\n\n### 2e: Library Mode Processes\n\nIf library mode is active (detected Python processes):\n\n```bash\npkill -f \"nvidia_rag\" 2>/dev/null; pkill -f \"uvicorn.*rag\" 2>/dev/null; docker compose -f deploy/compose/docker-compose-ingestor-server.yaml down 2>/dev/null; docker compose -f deploy/compose/vectordb.yaml down 2>/dev/null\n```\n\n### 2f: Kubernetes (Helm) Deployment\n\nIf K8s deployment was detected, use the release name and namespace from `helm list` output in step 1:\n\n```bash\nhelm uninstall \u003crelease-name> -n \u003cnamespace> 2>/dev/null\n```\n\nTo also clean up persistent data (only if user requests full cleanup):\n```bash\nkubectl delete nimcache --all -n \u003cnamespace> 2>/dev/null; kubectl delete pvc --all -n \u003cnamespace> 2>/dev/null\n```\n\n## Step 3: Verify Everything Stopped\n\n```bash\necho \"=== REMAINING ===\" && docker ps --format \"table {{.Names}}\\t{{.Status}}\" 2>/dev/null; echo \"=== K8S ===\" && kubectl get pods -n rag 2>/dev/null | head -10 || echo \"NOT_K8S\"; helm list -n rag 2>/dev/null || true\n```\n\nIf any RAG-related containers remain, force remove:\n```bash\ndocker ps -a --format \"{{.Names}}\" | grep -E \"(rag|milvus|nim|ingest|redis|nemo|grafana|prometheus|embedding|ranking|vlm|ocr|page-elements|graphic-elements|table-structure)\" | xargs -r docker rm -f\n```\n\nIf pods remain after `helm uninstall`, force delete:\n```bash\nkubectl delete pods --all -n rag --force --grace-period=0 2>/dev/null\n```\n\n## Step 4: Optional Cleanup\n\nAsk the user if they want to clean up data/volumes:\n\n- **Remove Docker volumes** (deletes ingested data, vector DB indices, object-store data, and ingestor scratch):\n ```bash\n docker volume ls -q --filter \"name=^rag-vol-\" | xargs -r docker volume rm\n ```\n These named volumes include Elasticsearch, Milvus/etcd, SeaweedFS, and ingestor scratch data. Prefer deleting only the specific `rag-vol-*` volume the user requested.\n\n- **Remove model cache** (frees 100-200 GB for self-hosted):\n ```bash\n rm -rf ~/.cache/model-cache/\n ```\n\n- **Remove Docker images** (frees disk space):\n ```bash\n docker images | grep -E \"nvcr.io/nvidia|milvusdb\" | awk '{print $3}' | xargs -r docker rmi\n ```\n\nOnly perform cleanup if the user explicitly requests it.\n\n## Quick One-Liner (All Docker Services)\n\nIf the user wants a fast full teardown:\n\n```bash\ncd \"$(git rev-parse --show-toplevel)\" && \\\ndocker compose -f deploy/compose/docker-compose-nemo-guardrails.yaml down 2>/dev/null; \\\ndocker compose -f deploy/compose/observability.yaml down 2>/dev/null; \\\ndocker compose -f deploy/compose/docker-compose-rag-server.yaml down 2>/dev/null; \\\ndocker compose -f deploy/compose/docker-compose-ingestor-server.yaml down 2>/dev/null; \\\ndocker compose -f deploy/compose/vectordb.yaml down 2>/dev/null; \\\ndocker compose -f deploy/compose/nims.yaml down 2>/dev/null; \\\necho \"All RAG services stopped.\"\n```\n\n## Source Documentation\n- `docs/troubleshooting.md` — if services won't stop or containers hang\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":4843,"content_sha256":"c793fafa494c15736ddb4ee685f28fbcf04584f9ca318f7437a0e2f8b7e0f1e4"},{"filename":"references/troubleshoot.md","content":"# RAG Troubleshooting\n\n## Auto-Triage: Run First\n\nStart with this diagnostic sweep:\n\n```bash\necho \"=== CONTAINERS ===\" && docker ps --format \"table {{.Names}}\\t{{.Status}}\\t{{.Ports}}\" 2>/dev/null | grep -E \"(rag|elasticsearch|milvus|seaweedfs|nim|ingest|redis|etcd|embedding|ranking)\" | head -25; echo \"=== HEALTH ===\" && curl -s http://localhost:8081/v1/health?check_dependencies=true 2>/dev/null || echo \"RAG_UNREACHABLE\"; curl -s http://localhost:8082/v1/health?check_dependencies=true 2>/dev/null || echo \"INGESTOR_UNREACHABLE\"; echo \"=== LOGS ===\" && for svc in rag-server ingestor-server nim-llm-ms nemotron-vlm-embedding-ms nemotron-embedding-ms nemotron-ranking-ms elasticsearch seaweedfs; do echo \"--- $svc ---\"; docker logs --tail 20 \"$svc\" 2>/dev/null | grep -iE \"(error|fail|exception|timeout|oom|permission)\" || echo \"OK\"; done; echo \"=== GPU ===\" && nvidia-smi 2>/dev/null | head -20 || echo \"NO_GPU\"; echo \"=== DISK ===\" && df -h / | tail -1; echo \"=== DOCKER_DISK ===\" && docker system df 2>/dev/null; echo \"=== VOLUMES ===\" && docker volume ls --filter \"name=^rag-vol-\" 2>/dev/null; echo \"=== K8S ===\" && kubectl get pods -n rag 2>/dev/null | head -20 || echo \"NOT_K8S\"\n```\n\nAnalyze all output, then diagnose and fix. If Auto-Triage doesn't reveal the cause, dig deeper into the specific failing service's logs (`docker logs \u003cservice> --tail 100` or `kubectl logs \u003cpod> -n rag --tail 100`).\n\nConfirm with the user before deleting data (volumes, collections, model cache), changing deployment mode, or modifying API keys.\n\n## Source Documentation for Detailed Diagnosis\n\nRead these docs to find specific issue descriptions, causes, and fixes:\n\n- `docs/troubleshooting.md` — primary reference: all common issues with detailed symptoms/fixes\n- `docs/debugging.md` — Pipeline debugging: monitoring deployment, verifying endpoints, tracing requests\n- `docs/service-port-gpu-reference.md` — Complete port/GPU mapping table for all services\n\n## Expected Deployment Times\n\nIf user reports \"deployment is taking too long,\" compare against these baselines:\n\n| Mode | First Run | Subsequent |\n|------|-----------|------------|\n| Docker (self-hosted) | 15--30 min (model downloads) | 2--5 min |\n| Docker (NVIDIA-hosted) | 5--10 min (no model downloads) | 1--2 min |\n| K8s/Helm | 60--70 min (NIM cache 40--50 min + init 10--15 min + pod startup 5--10 min) | 10--15 min |\n\nIf deployment exceeds these times, check NIM container logs: `docker logs nim-llm-ms --tail 50` and model cache disk usage: `watch -n 10 'du -sh ~/.cache/model-cache/'`.\n\n## Service Health Endpoints\n\nRead `docs/service-port-gpu-reference.md` for the complete port/GPU mapping. Quick check:\n\n| Service | URL | Expected |\n|---------|-----|----------|\n| RAG Server | `http://localhost:8081/v1/health?check_dependencies=true` | `{\"status\":\"healthy\"}` |\n| Ingestor | `http://localhost:8082/v1/health?check_dependencies=true` | `{\"status\":\"healthy\"}` |\n| NV-Ingest | `http://localhost:7670/v1/health/ready` | 200 OK |\n| VLM Embedding NIM (default) | `http://localhost:9081/v1/health/ready` | 200 OK |\n| LLM NIM | `http://localhost:8999/v1/health/ready` | 200 OK |\n| Ranking NIM | `http://localhost:1976/v1/health/ready` | 200 OK |\n| Elasticsearch | `http://localhost:9200/_cluster/health` | `green` or `yellow` |\n\n## Kubernetes Monitoring Commands\n\n```bash\nkubectl get nimcache -n rag\nkubectl get pods -n rag\nkubectl logs -f \u003cpod-name> -n rag\nkubectl get pvc -n rag\nkubectl get events -n rag --sort-by='.lastTimestamp'\n```\n\nPods in `ContainerCreating` or `Init` state during model download is expected. Use `kubectl get nimcache -n rag -w` to watch download progress.\n\n## Enable Debug Logging\n\n```bash\nexport LOGLEVEL=DEBUG\ndocker compose -f deploy/compose/docker-compose-ingestor-server.yaml up -d --no-deps ingestor-server\ndocker compose -f deploy/compose/docker-compose-rag-server.yaml up -d --no-deps rag-server\n```\n\n---\n\n## Symptom-to-Fix Quick Index\n\nMatch the symptom from Auto-Triage output, then read `docs/troubleshooting.md` for the detailed fix. For pipeline debugging steps, read `docs/debugging.md`.\n\n| Symptom | Category | Quick Fix |\n|---------|----------|-----------|\n| NIM container stuck at `(health: starting)` >30min | NIM Startup | Check GPU memory, NGC auth, disk space. First-run model downloads are slow — wait and monitor cache size. |\n| Elasticsearch unhealthy / search returns nothing | Elasticsearch | Restart vectordb compose. Check port 9200, disk, credentials, and `rag-vol-elasticsearch`. |\n| Document upload fails / ingestor health check fails | NV-Ingest | Check Redis, OCR NIMs. Rate limit (429) → reduce batch vars. Large PDFs → reduce batch size. |\n| Chat returns errors / /generate fails | RAG Server | Check LLM NIM health, embedding NIM, cloud API key. Verify `APP_LLM_MODELNAME` matches deployed NIM. |\n| DNS resolution failed for `\u003cservice>:\u003cport>` | Networking | Service container not running. Check `docker ps`, restart missing service. |\n| Port already in use | Networking | `lsof -i :\u003cport>` to find conflicting process. See port table above. |\n| GPU out of memory / `torch.OutOfMemoryError` | GPU | Kill other GPU processes, use `--profile rag` for fewer NIMs, or set correct `NIM_MODEL_PROFILE`. |\n| `nvidia-container-cli: unknown device` | GPU | GPU ID exceeds available GPUs. Run `nvidia-smi -L`, adjust `*_GPU_ID` vars to valid IDs. |\n| Disk full / insufficient space | Disk | `docker system prune -f`, remove unused images, check model cache size. |\n| `no configuration file provided: not found` | Docker Compose | Run from the repo root directory. |\n| `too many open files` | Docker Compose | Set `LimitNOFILE=65536` in containerd override, restart containerd. |\n| PVC stuck in Pending | Helm | Create missing StorageClass or update PVC. |\n| `ProvisioningFailed` access mode mismatch | Helm | Patch NIMCache to `ReadWriteOnce`. |\n| Ingestor OOMKilled | Helm | Increase memory limits in values.yaml. Set `SUMMARY_MAX_PARALLELIZATION=1`. |\n| Elasticsearch timeout during ingestion | Elasticsearch | Increase `ES_REQUEST_TIMEOUT` (default 600s). |\n| Need to inspect or reset persisted Docker data | Volumes | Use `docker volume ls --filter \"name=^rag-vol-\"`; see `docs/troubleshooting.md#manage-persistent-data-volumes`. |\n| Hallucination / out-of-context responses | Quality | Add missing-info handling to prompt in `prompt.yaml`. |\n| Embedding dimensions mismatch | Models | Set `APP_EMBEDDINGS_DIMENSIONS` to match model output. Re-ingest. |\n| Hybrid/dense search type mismatch | Search | Align `APP_VECTORSTORE_SEARCHTYPE` on ingestor and rag-server. Re-ingest. |\n| Confidence threshold filtering all results | Search | Lower `RERANKER_SCORE_THRESHOLD` (range 0.0–1.0, default 0.0). |\n| OCR not starting / connection errors | OCR | Check GPU memory, NGC auth. Verify `OCR_GRPC_ENDPOINT`/`OCR_HTTP_ENDPOINT` match running service. |\n| NVIDIA API credits exhausted | Cloud | Contact NVIDIA representative for additional credits. |\n| Image-only PDFs not ingesting | Ingestion | Enable `APP_NVINGEST_EXTRACTINFOGRAPHICS`. Consider image captioning. |\n\n---\n\n## Troubleshooting Checklists\n\n### Ingestion Checklist\n- [ ] All required containers running (ingestor-server, nv-ingest-ms-runtime, milvus, redis)\n- [ ] Vector database accessible (`curl http://localhost:9200/_cluster/health` for default Elasticsearch, or `curl http://localhost:9091/healthz` for Milvus)\n- [ ] Embedding service healthy (`curl http://localhost:9081/v1/health/ready` for default VLM embedding, or `curl http://localhost:9080/v1/health/ready` for `text-embed`)\n- [ ] File format supported and size \u003c= 400 MB\n- [ ] Sufficient disk space (`df -h /`)\n- [ ] GPU resources available (`nvidia-smi`)\n\n### Retrieval Checklist\n- [ ] RAG server running and healthy\n- [ ] LLM service accessible (`curl http://localhost:8999/v1/health/ready`)\n- [ ] Vector database contains data (collection exists with documents)\n- [ ] Collection name is correct\n- [ ] Query format is valid\n\n### Quality Checklist\n- [ ] Reranker is enabled and healthy\n- [ ] Top-K values are appropriate\n- [ ] Collection has sufficient relevant data\n- [ ] Query rewriting configured correctly\n- [ ] Prompt template appropriate for use case\n\n---\n\n## Full Reset\n\nDestroys all data (volumes, images, caches). Confirm with the user before running.\n\nIf nothing else works and the user confirms:\n\n```bash\ncd \"$(git rev-parse --show-toplevel)\"\ndocker compose -f deploy/compose/docker-compose-nemo-guardrails.yaml down 2>/dev/null\ndocker compose -f deploy/compose/observability.yaml down 2>/dev/null\ndocker compose -f deploy/compose/docker-compose-rag-server.yaml down 2>/dev/null\ndocker compose -f deploy/compose/docker-compose-ingestor-server.yaml down 2>/dev/null\ndocker compose -f deploy/compose/vectordb.yaml down 2>/dev/null\ndocker compose -f deploy/compose/nims.yaml down 2>/dev/null\n\ndocker volume ls -q --filter \"name=^rag-vol-\" | xargs -r docker volume rm\ndocker system prune -af\n```\n\nThen deploy fresh using the deploy workflow.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":8996,"content_sha256":"cb2811af3036ce6c74998a9d5b9f58c84cd67b49ee8c515373ab24d7020b56bf"},{"filename":"skill-card.md","content":"## Description: \u003cbr>\nNVIDIA RAG Blueprint — deploy, configure, troubleshoot, and manage RAG pipelines across Docker Compose, Helm, and library deployments. \u003cbr>\n\nThis skill is ready for commercial/non-commercial use. \u003cbr>\n\n## Owner\nNVIDIA \u003cbr>\n\n### License/Terms of Use: \u003cbr>\nApache 2.0 \u003cbr>\n## Use Case: \u003cbr>\nDevelopers and engineers deploying, configuring, troubleshooting, and managing NVIDIA RAG Blueprint pipelines with Docker, Helm, or Python library workflows. \u003cbr>\n\n### Deployment Geography for Use: \u003cbr>\nGlobal \u003cbr>\n\n## Known Risks and Mitigations: \u003cbr>\nRisk: Review before execution as proposals could introduce incorrect or misleading guidance into skills. \u003cbr>\nMitigation: Review and scan skill before deployment. \u003cbr>\n\n## Reference(s): \u003cbr>\n- [NVIDIA RAG Blueprint GitHub](https://github.com/NVIDIA-AI-Blueprints/rag) \u003cbr>\n- [Deployment Guide](references/deploy.md) \u003cbr>\n- [Troubleshooting](references/troubleshoot.md) \u003cbr>\n- [Shutdown](references/shutdown.md) \u003cbr>\n- [Agentic RAG](references/configure/agentic-rag.md) \u003cbr>\n- [Guardrails](references/configure/guardrails.md) \u003cbr>\n- [Models and Infrastructure](references/configure/models-and-infrastructure.md) \u003cbr>\n- [Search and Retrieval](references/configure/search-and-retrieval.md) \u003cbr>\n- [Observability](references/configure/observability.md) \u003cbr>\n- [MCP Server and Client](references/configure/mcp.md) \u003cbr>\n\n\n## Skill Output: \u003cbr>\n**Output Type(s):** [Shell commands, Configuration instructions, Diagnostic analysis] \u003cbr>\n**Output Format:** [Markdown with inline bash code blocks] \u003cbr>\n**Output Parameters:** [1D] \u003cbr>\n**Other Properties Related to Output:** [None] \u003cbr>\n\n## Evaluation Tasks: \u003cbr>\nNVSkills-Eval 3-tier evaluation (external profile): 9 static validation checks (Tier 1) and 2 deduplication checks (Tier 2). Tier 3 live agent evaluation not available in this report. \u003cbr>\n\n## Evaluation Metrics Used: \u003cbr>\nReported benchmark dimensions: \u003cbr>\n- Security: Checks whether skill-assisted execution avoids unsafe behavior such as secret leakage, destructive commands, or unauthorized access. \u003cbr>\n- Correctness: Checks whether the agent follows the expected workflow and produces the correct final output. \u003cbr>\n- Discoverability: Checks whether the agent loads the skill when relevant and avoids using it when irrelevant. \u003cbr>\n- Effectiveness: Checks whether the agent performs measurably better with the skill than without it. \u003cbr>\n- Efficiency: Checks whether the agent uses fewer tokens and avoids redundant work. \u003cbr>\n\n\n\n## Skill Version(s): \u003cbr>\n2.6.0 (source: frontmatter) \u003cbr>\n\n## Ethical Considerations: \u003cbr>\nNVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal team to ensure this skill meets requirements for the relevant industry and use case and addresses unforeseen product misuse. \u003cbr>\n\n(For Release on NVIDIA Platforms Only) \u003cbr>\nPlease report quality, risk, security vulnerabilities or NVIDIA AI Concerns [here](https://app.intigriti.com/programs/nvidia/nvidiavdp/detail). \u003cbr>\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":3195,"content_sha256":"729c059cee28eb26064d88ff0f37b81c10fc4cab26818028fe453ff596c8f835"}],"content_json":{"type":"doc","content":[{"type":"heading","attrs":{"level":1},"content":[{"text":"NVIDIA RAG Blueprint","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Purpose","type":"text"}]},{"type":"paragraph","content":[{"text":"Use this skill for NVIDIA RAG Blueprint operations: deployment, configuration, troubleshooting, shutdown, and feature management across Docker, Helm, and library deployments.","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Instructions","type":"text"}]},{"type":"ordered_list","attrs":{"order":1,"listStyle":"number"},"content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Match the user request to the intent routing table below.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Read the referenced playbook before making changes.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Use repository docs and deployment config files as the source of truth.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Verify the affected service or workflow after changes.","type":"text"}]}]}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Prerequisites","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"NVIDIA RAG Blueprint repository checkout.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Docker/Compose or Kubernetes/Helm for deployments.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Python 3.11+ for library workflows.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"NVIDIA GPU tooling for self-hosted NIM services.","type":"text"}]}]}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Autonomy Principles","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Auto-detect everything: GPU, VRAM, drivers, Docker, CUDA, disk, OS, ports, existing services, NGC key, repo state.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"If it can be checked with a command, check it — don't ask the user.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Ask only when user action is required: providing an API key, confirming data deletion, or choosing between equally valid options.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Once analysis is done, route to the correct workflow and execute.","type":"text"}]}]}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Intent Detection","type":"text"}]},{"type":"paragraph","content":[{"text":"Determine what the user wants and route immediately:","type":"text"}]},{"type":"table","attrs":{"layout":null},"content":[{"type":"tr","content":[{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"User Intent","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Action","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Deploy, install, set up, start RAG","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Read and follow ","type":"text"},{"text":"references/deploy.md","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Configure, enable, change, toggle a feature","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Use the Configure section below","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Troubleshoot, debug, fix, error, unhealthy","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Read and follow ","type":"text"},{"text":"references/troubleshoot.md","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Stop, shutdown, tear down, clean up","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Read and follow ","type":"text"},{"text":"references/shutdown.md","type":"text","marks":[{"type":"code_inline"}]}]}]}]}]},{"type":"paragraph","content":[{"text":"If the intent is ambiguous, infer from context (e.g., \"RAG isn't working\" → troubleshoot; \"get RAG running\" → deploy). Only ask if genuinely unclear.","type":"text"}]},{"type":"hr","attrs":{"markup":"---"}},{"type":"heading","attrs":{"level":2},"content":[{"text":"Configure","type":"text"}]},{"type":"paragraph","content":[{"text":"Requires a running RAG deployment. If services are not running, deploy first via ","type":"text"},{"text":"references/deploy.md","type":"text","marks":[{"type":"code_inline"}]},{"text":".","type":"text"}]},{"type":"paragraph","content":[{"text":"Match the user's request to a reference file, then read and follow it:","type":"text"}]},{"type":"table","attrs":{"layout":null},"content":[{"type":"tr","content":[{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Feature Keywords","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Reference","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"VLM, VLM embeddings, image captioning","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"references/configure/vlm.md","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"NeMo Guardrails","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"references/configure/guardrails.md","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Agentic RAG, planning/execution agent, agentic streaming, stage events","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"references/configure/agentic-rag.md","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Query rewriting, decomposition, multi-turn","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"references/configure/query-and-conversation.md","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Ingestion (text-only, audio, Nemotron Parse, OCR, batch CLI, NV-Ingest, volume mount, performance)","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"references/configure/ingestion.md","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Search, retrieval, hybrid search, multi-collection, metadata, filters, Elasticsearch filters, reranker, topK, accuracy/performance","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"references/configure/search-and-retrieval.md","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"LLM/embedding/ranking model changes, vector DB, Milvus/Elasticsearch auth, service keys, model profiles, ports/GPU","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"references/configure/models-and-infrastructure.md","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Reasoning, thinking mode, ","type":"text"},{"text":"reasoning_content","type":"text","marks":[{"type":"code_inline"}]},{"text":", self-reflection, prompts, generation params (tokens, temperature, citations), per-request LLM params","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"references/configure/reasoning-and-generation.md","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Summarization","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"references/configure/summarization.md","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Observability (tracing, Zipkin, Grafana, Prometheus)","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"references/configure/observability.md","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Multimodal query (image + text)","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"references/configure/multimodal-query.md","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Data catalog (collection/document metadata)","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"references/configure/data-catalog.md","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"User interface (UI settings, reasoning panel, metadata filters)","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"references/configure/user-interface.md","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"API reference (endpoints, schemas)","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"references/configure/api-reference.md","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Evaluation (RAGAS metrics)","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"references/configure/evaluation.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" (and skill ","type":"text"},{"text":"rag-eval","type":"text","marks":[{"type":"code_inline"}]},{"text":")","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"MCP server & client, agent toolkit","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"references/configure/mcp.md","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Migration (version upgrades)","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"references/configure/migration.md","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Notebooks (setup and catalog)","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"references/configure/notebooks.md","type":"text","marks":[{"type":"code_inline"}]}]}]}]}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Configure Flow","type":"text"}]},{"type":"ordered_list","attrs":{"order":1,"listStyle":"number"},"content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Match the user's request to a reference file from the table above.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Detect what's running:","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"bash"},"content":[{"text":"echo \"=== NIM ===\" && docker ps --format '{{.Names}}' 2>/dev/null | grep -iE '(nim-llm|nemotron-(vlm-)?embedding|nemotron-ranking|nemotron-vlm|nemotron-3-nano-omni|page-elements|graphic-elements|table-structure|nemotron-ocr)' || echo \"NO_LOCAL_NIMS\"; echo \"=== RAG ===\" && docker ps --format '{{.Names}}' 2>/dev/null | grep -iE '(rag-server|ingestor-server|elasticsearch|milvus|seaweedfs|lancedb)' || echo \"NO_DOCKER_RAG\"; echo \"=== K8S ===\" && kubectl get pods -n rag 2>/dev/null | head -5 || echo \"NO_K8S\"; echo \"=== LIBRARY ===\" && ps aux 2>/dev/null | grep -E '(nvidia_rag|uvicorn.*rag)' | grep -v grep || echo \"NO_LIBRARY\"","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Use this table to determine platform, deployment type, and where config lives:","type":"text"}]},{"type":"table","attrs":{"layout":null},"content":[{"type":"tr","content":[{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Local NIMs running?","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"RAG services running?","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Deployment Type","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Config Location","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Yes (Docker)","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Any","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Self-hosted","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"deploy/compose/.env","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"No","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Yes (Docker)","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"NVIDIA-hosted","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"deploy/compose/nvdev.env","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Yes (K8s pods)","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Any","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Self-hosted","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"values.yaml","type":"text","marks":[{"type":"code_inline"}]},{"text":" (NIM sections)","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"No","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Yes (K8s pods)","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"NVIDIA-hosted","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"values.yaml","type":"text","marks":[{"type":"code_inline"}]},{"text":" (envVars)","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"—","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Library processes","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Library mode","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"notebooks/config.yaml","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"No","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"No","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Not running","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Deploy first via ","type":"text"},{"text":"references/deploy.md","type":"text","marks":[{"type":"code_inline"}]}]}]}]}]},{"type":"paragraph","content":[{"text":"Tell the user what you detected and ask to confirm. Example: \"I see local NIM containers running (nim-llm-ms, nemotron-vlm-embedding-ms) — this is a self-hosted deployment. Config file is ","type":"text"},{"text":"deploy/compose/.env","type":"text","marks":[{"type":"code_inline"}]},{"text":". Correct?\"","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Check current feature state before changing anything — read the config location from step 3, then cross-check the live service:","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Docker: ","type":"text"},{"text":"docker exec rag-server env 2>/dev/null | grep -E \"\u003cVAR_NAME>\"","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Helm: ","type":"text"},{"text":"kubectl get pod -n rag -l app=rag-server -o jsonpath='{.items[0].spec.containers[0].env}' 2>/dev/null","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"paragraph","content":[{"text":"If the config file and live service disagree, tell the user the service has stale config and will need a restart.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"If the feature needs extra GPUs, check availability against hardware restrictions (see below):","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"bash"},"content":[{"text":"nvidia-smi --query-gpu=index,name,memory.total,memory.used --format=csv,noheader 2>/dev/null || echo \"NO_GPU\"","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Read the reference file and apply changes:","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Docker: edit the env file (uncomment to enable, re-comment to disable — the env file is the source of truth). Then restart the affected service:","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":""},"content":[{"text":"source \u003cenv-file> && docker compose -f deploy/compose/\u003ccompose-file> up -d","type":"text"}]},{"type":"table","attrs":{"layout":null},"content":[{"type":"tr","content":[{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Service","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Compose File","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"rag-server","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"docker-compose-rag-server.yaml","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"ingestor-server","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"docker-compose-ingestor-server.yaml","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Elasticsearch, Milvus, etcd, SeaweedFS","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"vectordb.yaml","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"NIM containers (LLM, embedding, ranking, VLM, OCR, parse, audio, extraction)","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"nims.yaml","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"guardrails","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"docker-compose-nemo-guardrails.yaml","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"observability (Grafana, Prometheus, Zipkin)","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"observability.yaml","type":"text","marks":[{"type":"code_inline"}]}]}]}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Helm: edit ","type":"text"},{"text":"values.yaml","type":"text","marks":[{"type":"code_inline"}]},{"text":", then upgrade: ","type":"text"},{"text":"helm upgrade rag \u003cchart> -n rag -f values.yaml","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Library: edit ","type":"text"},{"text":"notebooks/config.yaml","type":"text","marks":[{"type":"code_inline"}]},{"text":", then restart the Python process","type":"text"}]}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Verify:","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Docker: ","type":"text"},{"text":"docker ps --format \"table {{.Names}}\\t{{.Status}}\" | head -20; curl -s http://localhost:8081/v1/health?check_dependencies=true 2>/dev/null | head -1","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Helm: ","type":"text"},{"text":"kubectl get pods -n rag; kubectl rollout status deployment/rag-server -n rag --timeout=120s","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Library: ","type":"text"},{"text":"curl -s http://localhost:8081/v1/health 2>/dev/null | head -1","type":"text","marks":[{"type":"code_inline"}]}]}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"If restart fails, read ","type":"text"},{"text":"references/troubleshoot.md","type":"text","marks":[{"type":"code_inline"}]},{"text":". If multiple features requested, repeat from step 1 for each.","type":"text"}]}]}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Examples","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"\"Deploy RAG\" -> route to ","type":"text"},{"text":"references/deploy.md","type":"text","marks":[{"type":"code_inline"}]},{"text":".","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"\"Enable VLM\" -> route to ","type":"text"},{"text":"references/configure/vlm.md","type":"text","marks":[{"type":"code_inline"}]},{"text":".","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"\"RAG is unhealthy\" -> route to ","type":"text"},{"text":"references/troubleshoot.md","type":"text","marks":[{"type":"code_inline"}]},{"text":".","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"\"Stop RAG\" -> route to ","type":"text"},{"text":"references/shutdown.md","type":"text","marks":[{"type":"code_inline"}]},{"text":".","type":"text"}]}]}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Limitations","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Operational guidance only applies to this RAG Blueprint repository.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Live deployment changes require a running Docker, Helm, or library target.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Secrets such as ","type":"text"},{"text":"NGC_API_KEY","type":"text","marks":[{"type":"code_inline"}]},{"text":" must be supplied by the user environment.","type":"text"}]}]}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Troubleshooting","type":"text"}]},{"type":"table","attrs":{"layout":null},"content":[{"type":"tr","content":[{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Error / signal","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"What to do","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Services are not running","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Follow ","type":"text"},{"text":"references/deploy.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" before configuring features.","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Restart or health check fails","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Follow ","type":"text"},{"text":"references/troubleshoot.md","type":"text","marks":[{"type":"code_inline"}]},{"text":".","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"User requests teardown","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Follow ","type":"text"},{"text":"references/shutdown.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" and confirm destructive cleanup.","type":"text"}]}]}]}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"When User Says \"Configure\" Without Specifics","type":"text"}]},{"type":"paragraph","content":[{"text":"Run steps 2–3 above, then read the identified config file to list what's currently enabled:","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"bash"},"content":[{"text":"grep -E \"^(export )?(ENABLE_|APP_)\" \u003cconfig-file> 2>/dev/null | sort","type":"text"}]},{"type":"paragraph","content":[{"text":"Summarize what's running and enabled, then ask which feature to change.","type":"text"}]},{"type":"hr","attrs":{"markup":"---"}},{"type":"heading","attrs":{"level":2},"content":[{"text":"Hardware Restrictions","type":"text"}]},{"type":"paragraph","content":[{"text":"Read ","type":"text"},{"text":"docs/support-matrix.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" for current GPU requirements per deployment mode. Read ","type":"text"},{"text":"docs/service-port-gpu-reference.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" for port mappings and GPU assignments.","type":"text"}]},{"type":"table","attrs":{"layout":null},"content":[{"type":"tr","content":[{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"GPU","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Feature Restrictions","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"B200","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"No VLM, No Guardrails, No Nemotron Parse. May need multi-GPU LLM (","type":"text"},{"text":"LLM_MS_GPU_ID","type":"text","marks":[{"type":"code_inline"}]},{"text":").","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"RTX PRO 6000","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"No Nemotron Parse. No Audio on Helm.","type":"text"}]}]}]}]}]},"metadata":{"date":"2026-11-02","name":"rag-blueprint","tags":["nvidia","blueprint","rag","deployment","configuration","troubleshooting"],"author":"@skillopedia","source":{"stars":3825,"repo_name":"skills","origin_url":"https://github.com/NVIDIA/skills/blob/HEAD/skills/rag-blueprint/SKILL.md","repo_owner":"NVIDIA","body_sha256":"7167ef6b9d4f30ffe47b698ae7fdf9f0beafe4ea9a75752a056d0e4f03d6e51b","cluster_key":"06e0990f1852a184633490c809519832905fb209e873eeec9f3bbf12acab756e","clean_bundle":{"format":"clean-skill-bundle-v1","source":"NVIDIA/skills/skills/rag-blueprint/SKILL.md","text_bytes":117069,"source_commit":"25340619336e89bcc40365c28bfe9e790430468e","skills_sh_rank":6038,"attachment_count":36,"text_attachments":36,"attachment_storage":"skill_attachments","binary_attachments":0},"cluster_size":1,"skill_md_path":"skills/rag-blueprint/SKILL.md","import_metadata":{"date":"2026-11-02","author":"@skillopedia","version":"2.6.0","category":"ai-agent-development","category_label":"AI"},"exact_dupes_collapsed_into_this":0},"license":"Apache-2.0","version":"2.6.0","category":"ai-agent-development","metadata":{"tags":["nvidia","blueprint","rag","deployment","configuration","troubleshooting"],"author":"NVIDIA RAG \[email protected]>","domain":"ai-ml","languages":["python","typescript","shell"],"frameworks":["fastapi","langchain","react","docker-compose","helm"],"github-url":"https://github.com/NVIDIA-AI-Blueprints/rag","argument-hint":"deploy RAG | enable feature | disable feature | configure | troubleshoot | shutdown","endpoint-openapi-schemas":["docs/api_reference/openapi_schema_rag_server.json","docs/api_reference/openapi_schema_ingestor_server.json"]},"import_tag":"skills-sh-top20k-2026-11","description":"NVIDIA RAG Blueprint — deploy, configure, troubleshoot, and manage. Handles any RAG action: deploy, install, start, enable, disable, toggle, change, configure, troubleshoot, debug, fix, shutdown, stop, or tear down any RAG feature or service (Agentic RAG, VLM, guardrails, query rewriting, models, search, ingestion, observability, summarization, reasoning, and more).","allowed-tools":"Bash(echo *) Bash(nvidia-smi *) Bash(curl --version *) Bash(docker ps *) Bash(docker info *) Bash(docker --version *) Bash(docker version *) Bash(docker logs *) Bash(docker inspect *) Bash(docker stats *) Bash(docker compose ps *) Bash(docker compose logs *) Bash(docker compose config *) Bash(docker compose version *) Bash(kubectl get *) Bash(kubectl describe *) Bash(kubectl version *) Bash(kubectl logs *) Bash(kubectl api-resources *) Bash(kubectl rollout status *) Bash(helm version *) Bash(helm list *) Bash(helm status *) Bash(oc get *) Bash(oc describe *) Bash(oc logs *) Bash(oc whoami *) Bash(oc version *) Bash(git rev-parse *) Bash(git describe *) Bash(git status *) Bash(python3 --version *) Bash(pip3 show *) Bash(df *) Bash(du *) Bash(cat /proc/*) Bash(cat /etc/os-release *) Bash(ss *) Bash(netstat *) Bash(ls *) Bash(grep *) Bash(lsof *) Bash(ps aux *) Read Grep Glob","compatibility":"NVIDIA RAG Blueprint repository checkout; Docker/Compose or Kubernetes/Helm for deployments; Python 3.11+ for library workflows; NVIDIA GPU tooling for self-hosted NIM services."}},"renderedAt":1793671699470}

NVIDIA RAG Blueprint Purpose Use this skill for NVIDIA RAG Blueprint operations: deployment, configuration, troubleshooting, shutdown, and feature management across Docker, Helm, and library deployments. Instructions 1. Match the user request to the intent routing table below. 2. Read the referenced playbook before making changes. 3. Use repository docs and deployment config files as the source of truth. 4. Verify the affected service or workflow after changes. Prerequisites - NVIDIA RAG Blueprint repository checkout. - Docker/Compose or Kubernetes/Helm for deployments. - Python 3.11+ for lib…