Local AI App Integration (Embeddable Lemonade) Add a local AI mode to an existing app that already talks to a cloud AI API (OpenAI, Anthropic, or Ollama-compatible). The app launches , the Embeddable Lemonade binary, as a private subprocess and the existing client talks to it on . The user gets local, private, hardware-optimized inference (CPU, AMD iGPU/dGPU, XDNA2 NPU) with no separate install. What you'll end up with: one new launcher module ( 30 lines), three mandatory changes to the existing HTTP client ( , , and a 120-second HTTP timeout), one vendored binary under . When this skill is t…

,'' # unpacked dir = asset name without .zip\nCopy-Item -Recurse \"$env:TEMP\\lemond-unpack\\$folder\\*\" vendor\\lemonade\\\n# Sanity check: resources/ must be nested under vendor\\lemonade\\ (not flattened)\nif (-not (Test-Path vendor\\lemonade\\resources\\*.json)) { throw \"resources/ missing — re-extract and copy again\" }\n```\n\nOn Linux (bash):\n\n```bash\nURL=$(curl -s https://api.github.com/repos/lemonade-sdk/lemonade/releases/latest \\\n | grep browser_download_url | grep ubuntu-x64.tar.gz | cut -d'\"' -f4)\ncurl -L \"$URL\" | tar -xz --strip-components=1 -C vendor/lemonade\n```\n\n> **Copy the full package, not just the binary.** The archive contains\n> `lemond[.exe]`, `lemonade[.exe]`, `LICENSE`, and `resources/`. The\n> `resources/` directory is required — without it lemond starts and passes the\n> health check but fails on every model and backend request. Copying only the\n> binary produces a server that looks healthy but cannot function.\n\n> **`lemond` vs `lemonade` CLI:** `lemond` is the embedded server binary that\n> ships with the app. The `lemonade` CLI is a separate packaging tool used\n> only during development/build time to install backends. The same embeddable\n> archive unpacked above already contains a matching `lemonade[.exe]` next to\n> `lemond[.exe]`, so its version aligns with the bundled `lemond`. Do **not**\n> `pip install lemonade-sdk` to get it: the PyPI package is a separate, older\n> release line whose ports, model names, and install API do not match the\n> `lemond` bundled here, and mixing the two is a known source of silent\n> version mismatches. Keep the `lemonade` CLI, `lemond`, and the backends all\n> from the one release downloaded in this step so their versions stay aligned.\n\nThe expected layout **after setup** (first run + backend install). A freshly\nunzipped package contains only `lemond[.exe]`, `lemonade[.exe]`, `LICENSE`, and\n`resources/` — the items below are created later, as their comments note:\n\n```\nvendor/lemonade/\n lemond[.exe] # the only binary the app ships\n LICENSE\n config.json # generated on first run; commit a seed copy\n resources/\n server_models.json # do not edit; use GET /api/v1/models at runtime\n backend_versions.json\n bin/ # backends bundled at packaging time\n llamacpp/vulkan/llama-server[.exe]\n models/ # pre-bundled model weights (optional)\n models--unsloth--Qwen3-4B-GGUF/\n```\n\n> **`server_models.json`:** Do not edit or rely on this file. It can be stale.\n> The only authoritative model list is `GET /api/v1/models` on a running\n> `lemond` instance with the backend already installed.\n\n**Bundle decisions: pick deliberately**\n\n- **Backends:** Bundle `llamacpp:vulkan` at packaging time (works on every\n GPU). Install `llamacpp:rocm` at first run on supported AMD systems via\n `POST /api/v1/install` after probing `GET /api/v1/system-info`. Never ship\n every backend, or the artifact balloons.\n- **Models:** Either bundle the default model under `models/` (offline\n install, larger installer) **or** pull on first run with\n `POST /api/v1/pull` (smaller installer, needs network). Pick one and\n document it.\n- **`models_dir`:** Set to `./models` in `config.json` to keep weights\n private to the app. Leave as `auto` only if the user explicitly wants to\n share weights with other apps.\n\n**Backend install timing — two distinct paths:**\n\n> **Packaging time** (developer machine, before bundling). Use the lemonade\n> CLI that shipped inside `vendor/lemonade/` so it matches the bundled\n> `lemond` version (prefix with `./` or the full path):\n> ```\n> vendor/lemonade/lemonade backends install llamacpp:vulkan\n> vendor/lemonade/lemonade backends install flm:npu # Windows NPU path only\n> ```\n> This bakes the backend binaries into `vendor/lemonade/bin/` before the app\n> ships. `lemond` does not need to be running. Use a modern `lemonade` CLI\n> whose version matches the bundled `lemond` (the copy in the archive you\n> unpacked works); do not `pip install lemonade-sdk` for it.\n>\n> **First-run / runtime** (user's machine, after `lemond` is running):\n> ```http\n> POST /api/v1/install\n> {\"recipe\": \"llamacpp\", \"backend\": \"rocm\"}\n> ```\n> Use this for hardware-specific backends (e.g. `llamacpp:rocm`) that cannot\n> be bundled universally. `lemond` must already be running (Step 4 complete).\n\n## Step 4: Add a `lemond` launcher\n\nWrite the launcher as a new module named **`lemond_launcher.py`** (or\n`lemond_launcher.\u003cext>` for the app's language). It is a thin process\nsupervisor. Its only jobs:\n\n1. Generate a fresh random API key: `key = secrets.token_urlsafe(32)`\n2. Pick a free localhost port: bind a `socket` to port 0, read back the assigned port, close it.\n3. Spawn lemond as a `subprocess`: `subprocess.Popen([LEMOND_BIN, LEMOND_DIR, \"--port\", str(port)], env={**os.environ, \"LEMONADE_API_KEY\": key})`\n4. Poll `GET /api/v1/health` with `Authorization: Bearer {key}` in a loop until HTTP 200 — this is the only correct readiness check.\n5. Expose the chosen `port` and `key` to the rest of the app.\n\n> **Log one line per lifecycle stage.** Build the logging in from the start —\n> not as an afterthought when something breaks. Each silent transition needs a\n> visible marker so a failure points at the exact stage. Aim for:\n>\n> ```\n> [lemond] Starting on port \u003cport>\n> [lemond] Healthy on port \u003cport>\n> [lemond] \u003crecipe>:\u003cbackend> installed (or: already installed / install failed)\n> [lemond] Pulling model \u003cname>... then: Model \u003cname> ready (or: pull returned \u003cstatus>)\n> [local] \u003cmodality> result: \u003cvalue> (first inference output — empty string here = unpulled model)\n> ```\n>\n> Logging the **first inference result verbatim** is what turns the\n> silent-empty failure (Step 6) from a multi-hour mystery into a one-line\n> diagnosis. Route these through the app's normal logging so they can be quieted\n> for release.\n\n> **Dev-mode file watchers:** If the app runs with a file watcher (Tauri,\n> Electron, Next.js, Vite, etc.) that watches the source tree, ensure\n> `vendor/lemonade/` is excluded from the watched paths. Lemond writes config\n> and cache files at runtime; a watcher that picks these up will restart the\n> app, kill the lemond subprocess, and spawn a new one on a new port —\n> silently breaking any in-flight transcription. Add `vendor/` (or the\n> equivalent) to the watcher's ignore list before testing.\n\n**Use the reference implementation from [reference.md § Reference launchers](reference.md#reference-launchers) directly** — copy it verbatim and adapt only the `LEMOND_DIR` path. Do not write a launcher from scratch. The reference Python launcher uses `secrets` (for the API key), `socket` (for the free-port probe), and `subprocess` (to spawn lemond); the Node.js launcher uses the equivalent stdlib modules. Both handle port-race retries and health polling correctly.\n\nReadiness is always determined by polling the exact endpoint\n`GET http://127.0.0.1:\u003cport>/api/v1/health` and checking for HTTP 200 — never\nby reading `lemond`'s stdout or stderr. Any health-check helper you write must\nhit that `/api/v1/health` path.\n\n## Step 5: Re-point the existing client at `lemond`\n\nMake **three** changes to the app's existing client construction — all three\nare required, not optional:\n\n1. Set `base_url` to `http://127.0.0.1:{port}/api/v1`\n2. Set `api_key` to the launcher key\n3. **Set the HTTP timeout to 120 seconds** — this is mandatory, not optional\n\nThe 120-second timeout is not a tuning suggestion. The default on most HTTP\nclients is 30s, which is shorter than lemond's first-run model load time on\nreal hardware. Without it the request silently times out and the UI shows\nnothing, which is indistinguishable from a broken integration.\n\n**Python (openai) — the exact change to make:**\n\n```python\nimport httpx\nfrom openai import OpenAI\n\nproc, key, port = start_lemond()\nclient = OpenAI(\n base_url=f\"http://127.0.0.1:{port}/api/v1\",\n api_key=key,\n http_client=httpx.Client(timeout=120), # required: 120s for first-run model load\n)\n```\n\nFor other clients:\n\n| Existing client | New `base_url` | New auth | Timeout |\n|---|---|---|---|\n| `openai-python` | `http://127.0.0.1:{port}/api/v1` | `api_key=key` | `httpx.Client(timeout=120)` |\n| `openai-node` | `http://127.0.0.1:{port}/api/v1` | `apiKey: key` | `timeout: 120000` |\n| `@anthropic-ai/sdk` | `http://127.0.0.1:{port}/api/v1` | `apiKey: key` | `timeout: 120000` |\n| Raw `fetch` / `requests` | same | `Authorization: Bearer {key}` | set per-request |\n| Ollama-compatible code | `http://127.0.0.1:{port}/api/v0` | pass key anyway | 120s |\n\nThe model identifier on requests stays a Lemonade model name (e.g.\n`Qwen3-4B-GGUF`), not the cloud name.\n\n**Local mode needs no cloud API key — at all.** This is a defining property of\nlocal mode, not an edge case: there is no cloud service to authenticate to, so\nnothing should ever ask the user for a key. Any onboarding wall, validator, or\nstartup check that demands one must not block local-mode users. Concretely:\n\n- Skip or auto-satisfy the key-entry screen in local mode.\n- Treat local mode as already-authorized in every validation path — an\n empty-key check must short-circuit to \"valid\" when the active mode is local,\n never throw \"API key not configured\".\n- Re-enable the gate **only** for cloud mode.\n\nThe `lemond` key from Step 4 is generated internally by the launcher and used\nonly for the local loopback connection, so the user never sees or enters one;\nany UI placeholder (e.g. `\"local\"`) is fine. Flipping into local mode should\nnever strand the user on a key-entry wall.\n\n## Step 6: Health, backend, then pull the model — *before* first inference\n\n`GET /api/v1/health` returning 200 means the **server** is up. It does **not**\nmean inference will work. Before the first real request succeeds, three more\nthings must be true: the backend for your modality is installed, the model's\nweights are **downloaded to disk**, and (on the first call) the model is loaded\ninto memory. Treating health=200 as \"ready\" is the single biggest cause of a\nbroken-looking integration.\n\n**Do not call `POST /api/v1/load` at startup.** Lemond lazy-loads the model\ninto memory on the first inference request and handles that step on its own.\nPre-loading is unreliable across lemond versions (the `/load` request body\nshape has changed between releases) and a malformed call can crash or\ndestabilise the server before the user takes any action. Loading is the one\nstep you let lemond do lazily — pulling is not.\n\n### Pull the model so it exists on disk\n\nLazy-load only loads weights that are **already downloaded**. If the model was\nnever pulled, the first inference does not error — lemond returns an empty /\nblank result with HTTP 200. So after health passes and the backend is\ninstalled, proactively pull the model:\n\n```http\nPOST /api/v1/pull\n{\"model\": \"Whisper-Large-v3-Turbo\"}\n```\n\nThis is **idempotent** — a no-op if the weights are already present, a download\nif they are not. Run it once during setup (after backend install, before the\nfirst user-triggered inference) and log the result.\n\n- **Default model** (the one you chose in Step 2): pull it by name as above.\n- **Custom / user-overridden model:** do not assume it exists. Confirm it is a\n real Lemonade model first via `GET /api/v1/models` (the **only** trusted\n catalog — see [reference.md](reference.md)), then pull it the same way. A\n model appearing in the catalog is **not** proof its weights are downloaded;\n a successful pull is.\n\n> **Silent-empty is almost always an unpulled model.** If inference returns an\n> empty string / blank output with no HTTP error, the model was not downloaded.\n> Check your pull step before debugging anything else — this is the failure mode\n> that wastes the most time. Log the pull result and the first inference result\n> (see Step 4) so this is diagnosable from the console, not by guesswork.\n\n### Surface the *whole* setup, not just model load\n\nFirst-run cold start is more than a model load. The full sequence is:\n\n```\nserver spawn → health 200 → backend install → model download → model load → first result\n```\n\nOn a fresh machine, backend install and model download can each take from tens\nof seconds to several **minutes** (multi-GB weights over the network). Model\nload alone is 10–30s. An app that shows nothing during this will look frozen.\n\nMinimum: show a loading indicator or status message (\"Setting up local AI…\")\nfrom the moment setup begins until the first response arrives — covering the\n*entire* sequence above, not just the final load. The simplest implementation\nis a flag set when setup/first-request starts and cleared when the first\nresponse arrives. Once the model is pulled and loaded once, subsequent runs are\nfast; the long wait is first-run only.\n\n## Step 7: Lifecycle and recovery\n\nThese are the only failure modes worth handling. Do not over-engineer.\n\n| Symptom | Cause | Recovery |\n|---|---|---|\n| **Inference returns empty / blank with HTTP 200, no error** | Model never pulled: backend is installed but weights are absent, so lazy-load has nothing to load | `POST /api/v1/pull` with `{\"model\":\"...\"}`, wait for success, retry. Log the pulled result and the first inference result. This is the most common silent failure — see [Step 6](#step-6-health-backend-then-pull-the-model--before-first-inference) |\n| `POST /api/v1/load` returns 404 / model not found | Model not pulled yet (same root cause as the empty-result row above) | `POST /api/v1/pull` with `{\"model\": \"...\"}` then retry `/api/v1/load` |\n| `POST /api/v1/load` returns 500 with backend error | Backend not installed for this hardware | `GET /api/v1/system-info`, pick a supported backend, `POST /api/v1/install` with `{\"recipe\": \"...\", \"backend\": \"...\"}`, retry |\n| Subprocess exits immediately | Port race: another process grabbed the port between `freePort()` and lemond binding | The reference launcher retries with a fresh port automatically (3 attempts) |\n| `/api/v1/health` never returns 200 | First-run backend extraction is slow on cold disk | Extend timeout to 90s on first launch, 30s after |\n| HTTP 401 on every request | Forgot the `Authorization: Bearer` header | Audit the client config because Lemonade rejects unauth'd calls when `LEMONADE_API_KEY` is set |\n\n**Shutdown:** On app exit, `proc.terminate()` (Unix) or\n`proc.kill()` (Windows). `lemond` flushes config and exits cleanly within a\ncouple of seconds. Always wait on the process; never orphan it.\n\n**Do not** parse `lemond` stdout to detect readiness; use the HTTP\n`/api/v1/health` probe. Stdout format is not a stable contract.\n\n---\n\n## Verification checklist\n\nThe integration is done when **all** of these are true:\n\n- [ ] `vendor/lemonade/` contains the full package: `lemond[.exe]`,\n `lemonade[.exe]`, `LICENSE`, and `resources/` — not just the binary.\n- [ ] `lemond` starts as a subprocess with a fresh API key per launch.\n- [ ] `GET /api/v1/health` returns 200 within the timeout.\n- [ ] The default model is pulled (or bundled) before the first inference; a\n custom/overridden model is confirmed via `GET /api/v1/models` and then\n pulled. A blank result with no error means this step was skipped.\n- [ ] Each lifecycle stage logs a clear line (spawn, health, backend install,\n model pull, first result) so a failure is diagnosable from the console.\n- [ ] The existing client's chat / image / speech call returns a valid\n response with the base URL and key swapped, with no other code changed.\n- [ ] First-run latency is surfaced: the interface shows a loading state from the\n moment the first inference request is sent until the response arrives.\n- [ ] The HTTP client timeout is set to 120 seconds.\n- [ ] In local mode the app requires **no** cloud API key: no onboarding wall,\n validator, or startup check blocks the user, and no code path throws\n \"API key not configured\" when the active mode is local.\n- [ ] If the app uses a dev-mode file watcher, `vendor/lemonade/` is excluded\n from the watched paths so runtime writes by lemond do not trigger restarts.\n- [ ] Killing the parent process leaves no `lemond` subprocess behind.\n- [ ] On a fresh machine without the optimal backend, the app still works\n via the Vulkan fallback bundled in `bin/`.\n\nIf any box is unchecked, do not declare the task complete.\n\n---\n\n## Reference\n\nFor detailed model catalog, backend selection matrix, full endpoint reference,\nconfig keys, and per-model `recipe_options.json` tuning, see\n[reference.md](reference.md).","attachment_filenames":["evals/evals.json","evals/files/apikey-guard/main.py","evals/files/openai-stub/main.py","reference.md","skill-card.md"],"attachments":[{"filename":"evals/evals.json","content":"{\n \"evaluations\": [\n {\n \"id\": \"vendor-lemonade-launcher\",\n \"skill_should_trigger\": true,\n \"prompt\": \"Help me add local AI to this Python app so it replaces the OpenAI API with a local backend. Do not download or install anything, just write the launcher module.\",\n \"workspace\": \"evals/files/openai-stub\",\n \"unexpected_behavior\": [\n \"Hardcode a fixed port instead of binding one dynamically\"\n ],\n \"logs_contain\": [\n \"secrets\",\n \"socket\",\n \"subprocess\"\n ]\n },\n {\n \"id\": \"lemond-health-check\",\n \"skill_should_trigger\": true,\n \"prompt\": \"Modify my local cloud app to use Lemonade. For now, simply write a lemond launcher for my app that waits for the server to be ready. Do not download or install anything, just write the code.\",\n \"workspace\": \"evals/files/openai-stub\",\n \"unexpected_behavior\": [\n \"Read or parse the local server's stdout or stderr to detect readiness\",\n \"Call POST /api/v1/load to pre-load the model at startup\"\n ],\n \"logs_contain\": [\n \"/api/v1/health\"\n ]\n },\n {\n \"id\": \"local-mode-without-api-key\",\n \"skill_should_trigger\": true,\n \"prompt\": \"Edit main.py so my app works with local llms (without a cloud OPENAI_API_KEY). Do not download or install anything, just edit the file.\",\n \"workspace\": \"evals/files/apikey-guard\",\n \"expected_behavior\": [\n \"Remove or bypass the API-key guard so the app starts in local mode without requiring OPENAI_API_KEY to be set\"\n ],\n \"files_exist\": [\n \"main.py\"\n ]\n },\n {\n \"id\": \"electron-offline-mode\",\n \"skill_should_trigger\": true,\n \"prompt\": \"My Electron app calls the OpenAI chat completions endpoint. I want to ship an offline mode that bundles the inference engine into the installer so it still works with no internet.\"\n },\n {\n \"id\": \"swap-anthropic-backend\",\n \"skill_should_trigger\": true,\n \"prompt\": \"My Python desktop tool talks to the Anthropic Messages API today. Give it a mode where the model runs on the customer's own PC so their data never leaves the machine.\"\n },\n {\n \"id\": \"vendor-binary\",\n \"skill_should_trigger\": true,\n \"prompt\": \"Can you vendor an inference binary into my app and repoint my existing HTTP client at it instead of the Ollama server I'm hitting now?\"\n },\n {\n \"id\": \"in-app-embeddings\",\n \"skill_should_trigger\": true,\n \"prompt\": \"I want my app to compute embeddings on the end user's hardware rather than calling a hosted embeddings API. How do I wire that up?\"\n },\n {\n \"id\": \"rate-limit-openai-calls\",\n \"skill_should_trigger\": false,\n \"prompt\": \"Add rate limiting and retry-with-backoff to the OpenAI API calls in my Express server.\"\n },\n {\n \"id\": \"pyinstaller-packaging\",\n \"skill_should_trigger\": false,\n \"prompt\": \"Package my Python app as a standalone Windows installer with PyInstaller so users don't need Python.\"\n }\n ]\n}\n","content_type":"application/json; charset=utf-8","language":"json","size":3040,"content_sha256":"6250314828ee9956bf9ac472278df991af00397024d93fbfe2e1ecd11abd0f04"},{"filename":"evals/files/apikey-guard/main.py","content":"import os\n\nfrom openai import OpenAI\n\napi_key = os.environ.get(\"OPENAI_API_KEY\", \"\")\nif not api_key:\n raise SystemExit(\"No API key set. Please enter your OpenAI key.\")\n\nclient = OpenAI(api_key=api_key)\n","content_type":"text/x-python; charset=utf-8","language":"python","size":205,"content_sha256":"35fe2402f4e5051a8bedbca4801206bb0a7f44feb97f84ab54faa86130448762"},{"filename":"evals/files/openai-stub/main.py","content":"from openai import OpenAI\n\nclient = OpenAI()\n","content_type":"text/x-python; charset=utf-8","language":"python","size":45,"content_sha256":"bb9f95bfde591f1d62d66f0e3e73baf1a7f5d63a2b45582e07feb6ac94be9c16"},{"filename":"reference.md","content":"# Local AI App Integration: Reference\n\nDetailed reference material for the `local-ai-app-integration` skill. Read\nthis only when the main `SKILL.md` flow needs a decision that isn't covered\nby the default-path tables.\n\n## Contents\n\n- [Backend selection matrix](#backend-selection-matrix)\n- [Model picker by use case](#model-picker-by-use-case)\n- [Hardware probing with /v1/system-info](#hardware-probing-with-v1system-info)\n- [Endpoint reference](#endpoint-reference)\n- [Config keys you may need to set](#config-keys-you-may-need-to-set)\n- [Per-model tuning via recipe_options.json](#per-model-tuning-via-recipe_optionsjson)\n- [Linux packaging notes](#linux-packaging-notes)\n\n---\n\n## Backend selection matrix\n\n`lemond` supports multiple inference backends per modality. Bundle the\nbroadest-compatibility one at packaging time and install the\nhardware-optimized one at first run after a system probe.\n\n### Text generation (`llamacpp` recipe)\n\n| Backend | Hardware | OS | Bundle strategy |\n|---|---|---|---|\n| `vulkan` | x86_64 CPU, AMD iGPU/dGPU, most others | Windows, Linux | **Bundle at packaging time.** Universal fallback. |\n| `rocm` | gfx1151 (Strix Halo), gfx120X (RDNA4), gfx110X (RDNA3) | Windows, Linux | **Install at first run** if `/api/v1/system-info` shows `state: installable`. Cannot be packaging-time bundled. |\n| `cpu` | x86_64 CPU | Windows, Linux | Install only if you need a non-Vulkan CPU path. |\n| `metal` | Apple Silicon | macOS (beta) | macOS-only path. |\n\n### Text generation (NPU recipes, Windows only)\n\n| Recipe | Backend | Hardware | Notes |\n|---|---|---|---|\n| `flm` | `npu` | XDNA2 NPU | Cannot be packaging-time bundled on Linux. |\n| `ryzenai-llm` | `npu` | XDNA2 NPU | Windows only. Best for the Hybrid model family. |\n\n### Speech-to-text\n\n| Recipe | Backend | Model | Hardware | OS |\n|---|---|---|---|---|\n| `whispercpp` | `vulkan` | `Whisper-Large-v3-Turbo` | AMD iGPU / dGPU | Windows, Linux |\n| `whispercpp` | `cpu` | `Whisper-Large-v3-Turbo` | x86_64 CPU | Windows, Linux |\n| `whispercpp` | `npu` | `Whisper-Large-v3-Turbo` | XDNA2 NPU | Windows |\n| `flm` | `npu` | `whisper-v3-turbo-FLM` | XDNA2 NPU | Linux (runtime-install only) |\n\n### Text-to-speech\n\n| Recipe | Backend | Hardware |\n|---|---|---|\n| `kokoro` | `cpu` | x86_64 CPU |\n\n### Image generation (`sd-cpp`)\n\n| Backend | Hardware | OS |\n|---|---|---|\n| `rocm` | Supported AMD ROCm iGPU/dGPU | Windows, Linux |\n| `cpu` | x86_64 CPU | Windows, Linux |\n\n---\n\n## Model picker by use case\n\nPick **one** model as the app default. Do not list options to the user;\nship a default and document how to override.\n\n| Use case | Recommended model | Approx size | Recipe |\n|---|---|---|---|\n| Smallest viable chat | `Qwen3-0.6B-GGUF` | 0.5 GB | `llamacpp` |\n| General chat (default) | `Qwen3-4B-GGUF` | 2.5 GB | `llamacpp` |\n| Tool calling / agents | `Qwen3-4B-GGUF` or `OmniCoder-9B-GGUF` | 2.5 / 5.7 GB | `llamacpp` |\n| Coding | `Qwen2.5-Coder-7B-Instruct-GGUF` | 4.5 GB | `llamacpp` |\n| Multimodal (vision) chat | `Gemma-4-E2B-it-GGUF` | 2.0 GB | `llamacpp` |\n| Hybrid NPU chat (Ryzen AI) | `Llama-3.2-3B-Instruct-Hybrid` | 2.0 GB | `ryzenai-llm` |\n| Speech-to-text | `Whisper-Large-v3-Turbo` | 1.6 GB | `whispercpp` |\n| NPU speech-to-text (Ryzen AI) | `whisper-v3-turbo-FLM` | 0.6 GB | `flm` |\n| Text-to-speech | `kokoro-v1` | 0.3 GB | `kokoro` |\n| Image generation | `SDXL-Turbo` | 6.9 GB | `sd-cpp` |\n\nFor a catalog with more models, fetch `GET /api/v1/models` after starting `lemond`.\nThis is the **only** trusted source of available models. Never read or trust\n`vendor/lemonade/resources/server_models.json` (or any other static file) as a\nmodel catalog; it can be stale or incomplete. A model only appears in\n`GET /v1/models` once its backend is installed (see Step 3), so install the\nbackend first or the list will look empty/incomplete.\n\n**Catalogued ≠ downloaded.** A model listed by `GET /v1/models` is *available\nto use*, not necessarily present on disk. It must be **pulled**\n(`POST /api/v1/pull {\"model\":\"...\"}`) before it can serve — until then,\ninference returns an empty result with HTTP 200, not an error. The surest\nsignal that a model is ready is a successful pull, not its presence in the\ncatalog. See SKILL.md\n[Step 6](SKILL.md#step-6-health-backend-then-pull-the-model--before-first-inference).\n\n---\n\n## Hardware probing with /v1/system-info\n\nCall this **once at app first-run**, cache the result, and use it to decide\nwhich optional backend to install.\n\n```http\nGET /api/v1/system-info\nAuthorization: Bearer {key}\n```\n\nResponse shape (truncated):\n\n```json\n{\n \"recipes\": {\n \"llamacpp\": {\n \"backends\": {\n \"rocm\": { \"devices\": [\"amd_igpu\"], \"state\": \"installable\" },\n \"vulkan\": { \"devices\": [\"amd_igpu\", \"cpu\"], \"state\": \"installed\" },\n \"cpu\": { \"devices\": [\"cpu\"], \"state\": \"installed\" }\n }\n },\n \"ryzenai-llm\": {\n \"backends\": { \"npu\": { \"devices\": [\"xdna2\"], \"state\": \"installable\" } }\n }\n }\n}\n```\n\nThe same pattern applies to **every** recipe: read the per-backend `state`,\ninstall the best one that is `installable`, use it if already `installed`, and\nfall back down the priority list otherwise. Apply it to whichever recipe matches\nthe app's modality.\n\nDecision rules in priority order, for the default `llamacpp` recipe (text gen):\n\n1. If `recipes.llamacpp.backends.rocm.state == \"installable\"` →\n `POST /api/v1/install {\"recipe\":\"llamacpp\",\"backend\":\"rocm\"}`.\n2. Else if `state == \"installed\"` for `vulkan` → use it as-is.\n3. Else fall back to `cpu`.\n\nDecision rules for the `whispercpp` recipe (speech-to-text), NPU-first:\n\n1. If `recipes.whispercpp.backends.npu.state == \"installed\"` → use NPU as-is.\n2. Else if `npu.state == \"installable\"` →\n `POST /v1/install {\"recipe\":\"whispercpp\",\"backend\":\"npu\"}`, then use NPU.\n3. Else if `vulkan` is `installed`/`installable` → use the iGPU/dGPU path.\n4. Else fall back to `cpu`.\n\nProbe **once**, cache the chosen backend for the session (the result does not\nchange while the app runs), and log which backend was selected. This is the\nmechanism that lets one build run on an NPU machine and a CPU-only machine\nwithout any user configuration.\n\nFor Ryzen AI Hybrid models on Windows, additionally check\n`ryzenai-llm.backends.npu.state` and install if `installable`.\n\n---\n\n## Endpoint reference\n\nAll endpoints require `Authorization: Bearer {key}` when\n`LEMONADE_API_KEY` is set (it always should be in an embedded deployment).\n\n### App-facing (use these from the app's existing client)\n\n| Endpoint | Purpose |\n|---|---|\n| `GET /api/v1/health` | Readiness probe and loaded-model list |\n| `GET /api/v1/models` | List available models |\n| `POST /api/v1/chat/completions` | OpenAI Chat Completions (text + vision + tool calls) |\n| `POST /api/v1/embeddings` | OpenAI Embeddings |\n| `POST /api/v1/audio/transcriptions` | OpenAI Whisper-style transcription |\n| `POST /api/v1/audio/speech` | OpenAI TTS |\n| `POST /api/v1/images/generations` | OpenAI image generation |\n| `POST /api/v1/messages` | Anthropic Messages API |\n\n### Lifecycle (use these from the launcher / supervisor)\n\n| Endpoint | Purpose |\n|---|---|\n| `POST /api/v1/pull` | Download a model |\n| `POST /api/v1/load` | Load a model into memory |\n| `POST /api/v1/unload` | Free a model |\n| `POST /api/v1/delete` | Remove a downloaded model |\n| `POST /api/v1/install` | Install a backend (`{\"recipe\",\"backend\"}`) |\n| `POST /api/v1/uninstall` | Remove a backend |\n| `GET /api/v1/system-info` | Probe supported backends and devices |\n\n### Internal config (use sparingly)\n\n| Endpoint | Purpose |\n|---|---|\n| `GET /internal/config` | Full runtime config snapshot |\n| `POST /internal/set` | Update one or more config keys atomically |\n\n---\n\n## Config keys you may need to set\n\nSet these via the `lemonade` CLI's `config set` at packaging time, by\nhand-editing `config.json`, or at runtime via `POST /internal/set`.\n\n### Server-level (immediate effect)\n\n| Key | Type | Notes |\n|---|---|---|\n| `port` | int | Bind port. Override at launch with `--port` instead. |\n| `host` | string | Default `127.0.0.1`. **Do not** expose on `0.0.0.0` from an embedded app. |\n| `log_level` | enum | `trace`/`debug`/`info`/`warning`/`error`/`fatal`/`none` |\n| `global_timeout` | int seconds | HTTP client timeout for backend installs and pulls |\n| `no_broadcast` | bool | **Set `true` for embedded apps**, disables UDP discovery beacon |\n| `extra_models_dir` | string | Search path for arbitrary GGUFs (see below) |\n\n### Deferred (apply on next load)\n\n| Key | Type | Notes |\n|---|---|---|\n| `max_loaded_models` | int (-1 or positive) | Cap concurrent loaded models |\n| `ctx_size` | int | LLM context window |\n| `llamacpp_backend` | string | Pin to `rocm` / `vulkan` / `cpu` / `metal`; leave unset for auto |\n| `llamacpp_args` | string | Raw args appended to `llama-server` |\n| `sdcpp_backend` | string | `rocm` / `cpu` |\n| `whispercpp_backend` | string | `npu`/`cpu` (Windows), `cpu`/`vulkan` (Linux). For NPU prefer the `flm` recipe instead |\n| `whispercpp_args` | string | Raw whisper.cpp args |\n| `flm_args` | string | Raw FastFlowLM args |\n| `steps` | int | SD step count |\n| `cfg_scale` | number | SD CFG scale |\n| `width`, `height` | int | SD output size |\n\n### Recommended embedded defaults\n\n```json\n{\n \"host\": \"127.0.0.1\",\n \"no_broadcast\": true,\n \"log_level\": \"warning\",\n \"models_dir\": \"./models\",\n \"max_loaded_models\": 2,\n \"ctx_size\": 8192\n}\n```\n\n---\n\n## Per-model tuning via recipe_options.json\n\nFor per-model overrides (custom `llama-server` args, alternate context size\nfor one model only, alternate prompt template), drop a `recipe_options.json`\nnext to `config.json`. Example:\n\n```json\n{\n \"Qwen3-4B-GGUF\": {\n \"llamacpp_args\": \"--threads 8 --batch-size 512\",\n \"ctx_size\": 16384\n }\n}\n```\n\nThis file is consulted on every model load. No restart required.\n---\n\n## Linux packaging notes\n\nTwo backend limitations on Linux as of this writing:\n\n- `flm` (FastFlowLM, NPU) cannot be bundled at packaging time on Linux.\n Install at runtime only.\n- `llamacpp:rocm` cannot be bundled at packaging time on **any** OS. Always\n install at runtime via `/api/v1/install`.\n\nWhen building from source for an unusual Linux distro, see the upstream\n`docs/embeddable/building.md` in the lemonade-sdk/lemonade repo.\n\n---\n\n## Reference launchers\n\nFull implementations for Step 4. Adapt to the app's language; the key\nconstraints are: retry with a fresh port on spawn failure (the socket is\nreleased before lemond binds), poll `/api/v1/health` with the Bearer key,\nand kill the process on app exit.\n\n**Python:**\n\n```python\nimport os, secrets, socket, subprocess, sys, time, urllib.request\nfrom pathlib import Path\n\nLEMOND_DIR = Path(__file__).parent / \"vendor\" / \"lemonade\"\nLEMOND_BIN = LEMOND_DIR / (\"lemond.exe\" if sys.platform == \"win32\" else \"lemond\")\n\ndef _free_port() -> int:\n with socket.socket() as s:\n s.bind((\"127.0.0.1\", 0))\n return s.getsockname()[1]\n\ndef start_lemond(retries: int = 3) -> tuple[subprocess.Popen, str, int]:\n last_err: Exception | None = None\n for _ in range(retries):\n port = _free_port()\n key = secrets.token_urlsafe(32)\n env = {**os.environ, \"LEMONADE_API_KEY\": key}\n proc = subprocess.Popen(\n [str(LEMOND_BIN), str(LEMOND_DIR), \"--port\", str(port)],\n env=env,\n stdout=subprocess.PIPE, stderr=subprocess.STDOUT,\n )\n try:\n _wait_for_health(port, key, timeout_s=30)\n return proc, key, port\n except RuntimeError as e:\n proc.kill()\n proc.wait()\n last_err = e\n raise RuntimeError(f\"lemond failed to start after {retries} attempts\") from last_err\n\ndef _wait_for_health(port: int, key: str, timeout_s: int) -> None:\n url = f\"http://127.0.0.1:{port}/api/v1/health\"\n req = urllib.request.Request(url, headers={\"Authorization\": f\"Bearer {key}\"})\n deadline = time.monotonic() + timeout_s\n while time.monotonic() \u003c deadline:\n try:\n with urllib.request.urlopen(req, timeout=1) as r:\n if r.status == 200:\n return\n except Exception:\n time.sleep(0.25)\n raise RuntimeError(f\"lemond on port {port} did not become healthy within {timeout_s}s\")\n```\n\n**Node.js:**\n\n```js\nimport { spawn } from \"node:child_process\";\nimport { randomBytes } from \"node:crypto\";\nimport { createServer } from \"node:net\";\nimport path from \"node:path\";\n\nconst LEMOND_DIR = path.join(import.meta.dirname, \"vendor\", \"lemonade\");\nconst LEMOND_BIN = path.join(LEMOND_DIR, process.platform === \"win32\" ? \"lemond.exe\" : \"lemond\");\n\nconst freePort = () => new Promise((res) => {\n const s = createServer().listen(0, \"127.0.0.1\", () => {\n const { port } = s.address(); s.close(() => res(port));\n });\n});\n\nexport async function startLemond(retries = 3) {\n let lastErr;\n for (let i = 0; i \u003c retries; i++) {\n const port = await freePort();\n const key = randomBytes(32).toString(\"base64url\");\n const proc = spawn(LEMOND_BIN, [LEMOND_DIR, \"--port\", String(port)], {\n env: { ...process.env, LEMONADE_API_KEY: key },\n stdio: [\"ignore\", \"pipe\", \"pipe\"],\n });\n try {\n await waitForHealth(port, key, 30_000);\n return { proc, key, port };\n } catch (e) {\n proc.kill();\n lastErr = e;\n }\n }\n throw new Error(`lemond failed to start after ${retries} attempts: ${lastErr?.message}`);\n}\n\nasync function waitForHealth(port, key, timeoutMs) {\n const url = `http://127.0.0.1:${port}/api/v1/health`;\n const headers = { Authorization: `Bearer ${key}` };\n const deadline = Date.now() + timeoutMs;\n while (Date.now() \u003c deadline) {\n try {\n const r = await fetch(url, { headers });\n if (r.ok) return;\n } catch {}\n await new Promise((r) => setTimeout(r, 250));\n }\n throw new Error(`lemond on port ${port} did not become healthy within ${timeoutMs}ms`);\n}\n```\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":13905,"content_sha256":"f4e30715ada0c4753e8aed30dc7f7f9d7cfdeab73331736fa6459d1fa732e0b7"},{"filename":"skill-card.md","content":"# Skill Card\n\n## Description\n\nIntegrate local AI into cloud LLM apps for offline support, better privacy, and lower API costs.\n\n## Owner\n\nAMD\n\n## License\n\nMIT\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":159,"content_sha256":"a90a1c2855d6323d0607c87015bea612fede58063ea6e698658a8ef143c9cc1d"}],"content_json":{"type":"doc","content":[{"type":"heading","attrs":{"level":1},"content":[{"text":"Local AI App Integration (Embeddable Lemonade)","type":"text"}]},{"type":"paragraph","content":[{"text":"Add a local AI mode to an existing app that already talks to a cloud AI API (OpenAI, Anthropic, or Ollama-compatible). The app launches ","type":"text"},{"text":"lemond","type":"text","marks":[{"type":"code_inline"}]},{"text":", the Embeddable Lemonade binary, as a private subprocess and the existing client talks to it on ","type":"text"},{"text":"http://localhost:PORT/api/v1","type":"text","marks":[{"type":"code_inline"}]},{"text":". The user gets local, private, hardware-optimized inference (CPU, AMD iGPU/dGPU, XDNA2 NPU) with no separate install.","type":"text"}]},{"type":"paragraph","content":[{"text":"What you'll end up with:","type":"text","marks":[{"type":"strong"}]},{"text":" one new launcher module (~30 lines), three mandatory changes to the existing HTTP client (","type":"text"},{"text":"base_url","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"api_key","type":"text","marks":[{"type":"code_inline"}]},{"text":", and a 120-second HTTP timeout), one vendored binary under ","type":"text"},{"text":"vendor/lemonade/","type":"text","marks":[{"type":"code_inline"}]},{"text":".","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"When this skill is the right tool","type":"text"}]},{"type":"paragraph","content":[{"text":"Use this skill when ","type":"text"},{"text":"all","type":"text","marks":[{"type":"strong"}]},{"text":" of the following are true:","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"The app already calls a cloud AI service over HTTP (OpenAI Chat Completions, Anthropic Messages, or Ollama).","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"The user wants that AI to run on the end-user's PC, with the AI engine bundled into the app, not as a separate user install.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"The target platform is Windows x64 or Linux x64 (macOS embeddable is in beta).","type":"text"}]}]}]},{"type":"paragraph","content":[{"text":"If the user instead wants a ","type":"text"},{"text":"system-wide","type":"text","marks":[{"type":"strong"}]},{"text":" Lemonade Server (one install, shared across apps), do not use this skill; point them at ","type":"text"},{"text":"https://lemonade-server.ai/install_options.html","type":"text","marks":[{"type":"code_inline"}]},{"text":" and the standard OpenAI base URL ","type":"text"},{"text":"http://localhost:13305/api/v1","type":"text","marks":[{"type":"code_inline"}]},{"text":".","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"The opinionated path","type":"text"}]},{"type":"paragraph","content":[{"text":"This skill follows one fixed sequence. Do not deviate without a stated reason.","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":""},"content":[{"text":"[ ] 1. Survey the app's current AI integration\n[ ] 2. Pick a model + backend profile\n[ ] 3. Place Embeddable Lemonade in the app's tree (full package, not just the binary)\n[ ] 4. Add a `lemond` launcher (subprocess + API key + port + per-stage logging)\n[ ] 5. Re-point the existing client at lemond (base_url, api_key, 120s timeout — all three required)\n[ ] 6. Wait for /api/v1/health, install backend, then PULL the model before first use\n[ ] 7. Wire shutdown and error recovery","type":"text"}]},{"type":"paragraph","content":[{"text":"Track progress against this checklist. Move on only when each step verifies.","type":"text"}]},{"type":"blockquote","content":[{"type":"paragraph","content":[{"text":"Log every stage.","type":"text","marks":[{"type":"strong"}]},{"text":" A local integration has many silent failure points — spawn, health, backend install, model download, first inference. Without a log line at each transition, \"nothing happened\" is indistinguishable from \"broke at stage 3.\" Emit one clear line per stage as you build (see ","type":"text"},{"text":"Step 4","type":"text","marks":[{"type":"link","attrs":{"href":"#step-4-add-a-lemond-launcher","title":null}}]},{"text":"); the most common dead-end in this integration — a blank result with no error — is invisible without them.","type":"text"}]}]},{"type":"hr","attrs":{"markup":"---"}},{"type":"heading","attrs":{"level":2},"content":[{"text":"Step 1: Survey the app","type":"text"}]},{"type":"paragraph","content":[{"text":"Find every place the app currently calls a cloud AI API. Search the repo for:","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"openai","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"OpenAI(","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"chat.completions","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"responses.create","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"anthropic","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"Anthropic(","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"messages.create","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"api.openai.com","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"api.anthropic.com","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"localhost:11434","type":"text","marks":[{"type":"code_inline"}]},{"text":" (Ollama)","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"OPENAI_API_KEY","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"ANTHROPIC_API_KEY","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"paragraph","content":[{"text":"Record three things before continuing:","type":"text"}]},{"type":"ordered_list","attrs":{"order":1,"listStyle":"number"},"content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Client library and language","type":"text","marks":[{"type":"strong"}]},{"text":" (e.g., ","type":"text"},{"text":"openai-python","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"openai-node","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"@anthropic-ai/sdk","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"go-openai","type":"text","marks":[{"type":"code_inline"}]},{"text":", raw ","type":"text"},{"text":"fetch","type":"text","marks":[{"type":"code_inline"}]},{"text":").","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Modalities used:","type":"text","marks":[{"type":"strong"}]},{"text":" text chat, tool calling, embeddings, image gen, transcription, TTS. This drives the model + backend choice in Step 2.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"One single place","type":"text","marks":[{"type":"strong"}]},{"text":" where the base URL and API key are constructed. If there isn't one, refactor to one before going further. Local-mode toggling must flip exactly one config object.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Any API-key gating","type":"text","marks":[{"type":"strong"}]},{"text":" that blocks the app before a key is entered (onboarding walls, validators that reject empty keys, startup checks that disable AI until a key exists). Note each one — Step 5 bypasses them in local mode.","type":"text"}]}]}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Step 2: Pick a model + backend profile","type":"text"}]},{"type":"paragraph","content":[{"text":"Choose ","type":"text"},{"text":"one","type":"text","marks":[{"type":"strong"}]},{"text":" default profile based on the app's primary modality. Do not ship a buffet. Ship one good default and document how the user can override it.","type":"text"}]},{"type":"table","attrs":{"layout":null},"content":[{"type":"tr","content":[{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"App's primary need","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Default model","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Recipe","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Why","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"General chat / assistant","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Qwen3-4B-GGUF","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"llamacpp","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Small, fast, good tool calling, fits 8GB systems","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Coding assistant","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Qwen2.5-Coder-7B-Instruct-GGUF","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"llamacpp","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Strong code, runs on iGPU","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Vision / multimodal chat","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Gemma-4-E2B-it-GGUF","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"llamacpp","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Small multimodal default","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"NPU-first on Ryzen AI","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Llama-3.2-3B-Instruct-Hybrid","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"ryzenai-llm","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"XDNA2 NPU on Windows","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Speech-to-text (Windows)","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Whisper-Large-v3-Turbo","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"whispercpp","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"One model; probe picks NPU → iGPU/dGPU → CPU automatically","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Speech-to-text (Linux NPU)","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"whisper-v3-turbo-FLM","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"flm","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Linux NPU path; falls back to ","type":"text"},{"text":"whispercpp","type":"text","marks":[{"type":"code_inline"}]},{"text":" iGPU/CPU off-NPU","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Text-to-speech","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"kokoro-v1","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"kokoro","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"CPU-only, low latency","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Image generation","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"SDXL-Turbo","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"sd-cpp","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Single-step generation","type":"text"}]}]}]}]},{"type":"paragraph","content":[{"text":"For the LLM backend, default to ","type":"text"},{"text":"llamacpp","type":"text","marks":[{"type":"code_inline"}]},{"text":" and let ","type":"text"},{"text":"lemond","type":"text","marks":[{"type":"code_inline"}]},{"text":" pick ","type":"text"},{"text":"rocm","type":"text","marks":[{"type":"code_inline"}]},{"text":" → ","type":"text"},{"text":"vulkan","type":"text","marks":[{"type":"code_inline"}]},{"text":" → ","type":"text"},{"text":"cpu","type":"text","marks":[{"type":"code_inline"}]},{"text":" automatically by leaving ","type":"text"},{"text":"llamacpp_backend","type":"text","marks":[{"type":"code_inline"}]},{"text":" unset. Override only if the app has hard hardware requirements.","type":"text"}]},{"type":"paragraph","content":[{"text":"Scope: this skill selects a backend once at integration time on the developer's machine.","type":"text","marks":[{"type":"strong"}]},{"text":" Runtime fallback based on the end user's hardware is out of scope. Bundle ","type":"text"},{"text":"vulkan","type":"text","marks":[{"type":"code_inline"}]},{"text":" as the universal fallback so the app works on any machine. If the dev machine has an NPU and the chosen recipe supports it, the skill will use the NPU backend — otherwise it falls back to ","type":"text"},{"text":"vulkan","type":"text","marks":[{"type":"code_inline"}]},{"text":".","type":"text"}]},{"type":"blockquote","content":[{"type":"paragraph","content":[{"text":"Note:","type":"text","marks":[{"type":"strong"}]},{"text":" having an NPU does not mean every recipe supports NPU. Confirm the recipe/backend pair is ","type":"text"},{"text":"installed","type":"text","marks":[{"type":"code_inline"}]},{"text":" or ","type":"text"},{"text":"installable","type":"text","marks":[{"type":"code_inline"}]},{"text":" via ","type":"text"},{"text":"GET /api/v1/system-info","type":"text","marks":[{"type":"code_inline"}]},{"text":" before committing to it. See ","type":"text"},{"text":"reference.md","type":"text","marks":[{"type":"link","attrs":{"href":"reference.md#hardware-probing-with-v1system-info","title":null}}]},{"text":" for per-recipe decision rules.","type":"text"}]}]},{"type":"paragraph","content":[{"text":"For more options and tradeoffs, see ","type":"text"},{"text":"reference.md","type":"text","marks":[{"type":"link","attrs":{"href":"reference.md","title":null}}]},{"text":".","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Step 3: Place Embeddable Lemonade in the app's tree and install backends","type":"text"}]},{"type":"paragraph","content":[{"text":"Get the embeddable artifact","type":"text","marks":[{"type":"strong"}]},{"text":" from the latest Lemonade release:","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":""},"content":[{"text":"https://github.com/lemonade-sdk/lemonade/releases/latest","type":"text"}]},{"type":"paragraph","content":[{"text":"Download the file matching your target OS:","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Windows: ","type":"text"},{"text":"lemonade-embeddable-{VERSION}-windows-x64.zip","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Linux: ","type":"text"},{"text":"lemonade-embeddable-{VERSION}-ubuntu-x64.tar.gz","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"blockquote","content":[{"type":"paragraph","content":[{"text":"Don't hand-build the download URL from the tag.","type":"text","marks":[{"type":"strong"}]},{"text":" The git tag carries a leading ","type":"text"},{"text":"v","type":"text","marks":[{"type":"code_inline"}]},{"text":" (e.g. ","type":"text"},{"text":"v10.8.0","type":"text","marks":[{"type":"code_inline"}]},{"text":") but the asset filename strips it (","type":"text"},{"text":"lemonade-embeddable-10.8.0-...","type":"text","marks":[{"type":"code_inline"}]},{"text":"), so using the tag verbatim 404s. Ask the GitHub API for the asset by its stable name pattern and use the URL it returns, as below — this stays correct across version and naming changes.","type":"text"}]}]},{"type":"paragraph","content":[{"text":"First, create the target directory","type":"text","marks":[{"type":"strong"}]},{"text":" — it does not exist in a fresh repo:","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"powershell"},"content":[{"text":"# Windows\nNew-Item -ItemType Directory -Force vendor\\lemonade","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"bash"},"content":[{"text":"# Linux\nmkdir -p vendor/lemonade","type":"text"}]},{"type":"paragraph","content":[{"text":"Then download and unpack on Windows (PowerShell):","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"powershell"},"content":[{"text":"$rel = Invoke-RestMethod https://api.github.com/repos/lemonade-sdk/lemonade/releases/latest\n$asset = $rel.assets | Where-Object { $_.name -like \"lemonade-embeddable-*-windows-x64.zip\" } | Select-Object -First 1\nInvoke-WebRequest $asset.browser_download_url -OutFile lemond.zip\nExpand-Archive lemond.zip -DestinationPath \"$env:TEMP\\lemond-unpack\"\n$folder = $asset.name -replace '\\.zip

Local AI App Integration (Embeddable Lemonade) Add a local AI mode to an existing app that already talks to a cloud AI API (OpenAI, Anthropic, or Ollama-compatible). The app launches , the Embeddable Lemonade binary, as a private subprocess and the existing client talks to it on . The user gets local, private, hardware-optimized inference (CPU, AMD iGPU/dGPU, XDNA2 NPU) with no separate install. What you'll end up with: one new launcher module ( 30 lines), three mandatory changes to the existing HTTP client ( , , and a 120-second HTTP timeout), one vendored binary under . When this skill is t…

,'' # unpacked dir = asset name without .zip\nCopy-Item -Recurse \"$env:TEMP\\lemond-unpack\\$folder\\*\" vendor\\lemonade\\\n# Sanity check: resources/ must be nested under vendor\\lemonade\\ (not flattened)\nif (-not (Test-Path vendor\\lemonade\\resources\\*.json)) { throw \"resources/ missing — re-extract and copy again\" }","type":"text"}]},{"type":"paragraph","content":[{"text":"On Linux (bash):","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"bash"},"content":[{"text":"URL=$(curl -s https://api.github.com/repos/lemonade-sdk/lemonade/releases/latest \\\n | grep browser_download_url | grep ubuntu-x64.tar.gz | cut -d'\"' -f4)\ncurl -L \"$URL\" | tar -xz --strip-components=1 -C vendor/lemonade","type":"text"}]},{"type":"blockquote","content":[{"type":"paragraph","content":[{"text":"Copy the full package, not just the binary.","type":"text","marks":[{"type":"strong"}]},{"text":" The archive contains ","type":"text"},{"text":"lemond[.exe]","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"lemonade[.exe]","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"LICENSE","type":"text","marks":[{"type":"code_inline"}]},{"text":", and ","type":"text"},{"text":"resources/","type":"text","marks":[{"type":"code_inline"}]},{"text":". The ","type":"text"},{"text":"resources/","type":"text","marks":[{"type":"code_inline"}]},{"text":" directory is required — without it lemond starts and passes the health check but fails on every model and backend request. Copying only the binary produces a server that looks healthy but cannot function.","type":"text"}]}]},{"type":"blockquote","content":[{"type":"paragraph","content":[{"text":"lemond","type":"text","marks":[{"type":"code_inline"},{"type":"strong"}]},{"text":" vs ","type":"text","marks":[{"type":"strong"}]},{"text":"lemonade","type":"text","marks":[{"type":"code_inline"},{"type":"strong"}]},{"text":" CLI:","type":"text","marks":[{"type":"strong"}]},{"text":" ","type":"text"},{"text":"lemond","type":"text","marks":[{"type":"code_inline"}]},{"text":" is the embedded server binary that ships with the app. The ","type":"text"},{"text":"lemonade","type":"text","marks":[{"type":"code_inline"}]},{"text":" CLI is a separate packaging tool used only during development/build time to install backends. The same embeddable archive unpacked above already contains a matching ","type":"text"},{"text":"lemonade[.exe]","type":"text","marks":[{"type":"code_inline"}]},{"text":" next to ","type":"text"},{"text":"lemond[.exe]","type":"text","marks":[{"type":"code_inline"}]},{"text":", so its version aligns with the bundled ","type":"text"},{"text":"lemond","type":"text","marks":[{"type":"code_inline"}]},{"text":". Do ","type":"text"},{"text":"not","type":"text","marks":[{"type":"strong"}]},{"text":" ","type":"text"},{"text":"pip install lemonade-sdk","type":"text","marks":[{"type":"code_inline"}]},{"text":" to get it: the PyPI package is a separate, older release line whose ports, model names, and install API do not match the ","type":"text"},{"text":"lemond","type":"text","marks":[{"type":"code_inline"}]},{"text":" bundled here, and mixing the two is a known source of silent version mismatches. Keep the ","type":"text"},{"text":"lemonade","type":"text","marks":[{"type":"code_inline"}]},{"text":" CLI, ","type":"text"},{"text":"lemond","type":"text","marks":[{"type":"code_inline"}]},{"text":", and the backends all from the one release downloaded in this step so their versions stay aligned.","type":"text"}]}]},{"type":"paragraph","content":[{"text":"The expected layout ","type":"text"},{"text":"after setup","type":"text","marks":[{"type":"strong"}]},{"text":" (first run + backend install). A freshly unzipped package contains only ","type":"text"},{"text":"lemond[.exe]","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"lemonade[.exe]","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"LICENSE","type":"text","marks":[{"type":"code_inline"}]},{"text":", and ","type":"text"},{"text":"resources/","type":"text","marks":[{"type":"code_inline"}]},{"text":" — the items below are created later, as their comments note:","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":""},"content":[{"text":"vendor/lemonade/\n lemond[.exe] # the only binary the app ships\n LICENSE\n config.json # generated on first run; commit a seed copy\n resources/\n server_models.json # do not edit; use GET /api/v1/models at runtime\n backend_versions.json\n bin/ # backends bundled at packaging time\n llamacpp/vulkan/llama-server[.exe]\n models/ # pre-bundled model weights (optional)\n models--unsloth--Qwen3-4B-GGUF/","type":"text"}]},{"type":"blockquote","content":[{"type":"paragraph","content":[{"text":"server_models.json","type":"text","marks":[{"type":"code_inline"},{"type":"strong"}]},{"text":":","type":"text","marks":[{"type":"strong"}]},{"text":" Do not edit or rely on this file. It can be stale. The only authoritative model list is ","type":"text"},{"text":"GET /api/v1/models","type":"text","marks":[{"type":"code_inline"}]},{"text":" on a running ","type":"text"},{"text":"lemond","type":"text","marks":[{"type":"code_inline"}]},{"text":" instance with the backend already installed.","type":"text"}]}]},{"type":"paragraph","content":[{"text":"Bundle decisions: pick deliberately","type":"text","marks":[{"type":"strong"}]}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Backends:","type":"text","marks":[{"type":"strong"}]},{"text":" Bundle ","type":"text"},{"text":"llamacpp:vulkan","type":"text","marks":[{"type":"code_inline"}]},{"text":" at packaging time (works on every GPU). Install ","type":"text"},{"text":"llamacpp:rocm","type":"text","marks":[{"type":"code_inline"}]},{"text":" at first run on supported AMD systems via ","type":"text"},{"text":"POST /api/v1/install","type":"text","marks":[{"type":"code_inline"}]},{"text":" after probing ","type":"text"},{"text":"GET /api/v1/system-info","type":"text","marks":[{"type":"code_inline"}]},{"text":". Never ship every backend, or the artifact balloons.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Models:","type":"text","marks":[{"type":"strong"}]},{"text":" Either bundle the default model under ","type":"text"},{"text":"models/","type":"text","marks":[{"type":"code_inline"}]},{"text":" (offline install, larger installer) ","type":"text"},{"text":"or","type":"text","marks":[{"type":"strong"}]},{"text":" pull on first run with ","type":"text"},{"text":"POST /api/v1/pull","type":"text","marks":[{"type":"code_inline"}]},{"text":" (smaller installer, needs network). Pick one and document it.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"models_dir","type":"text","marks":[{"type":"code_inline"},{"type":"strong"}]},{"text":":","type":"text","marks":[{"type":"strong"}]},{"text":" Set to ","type":"text"},{"text":"./models","type":"text","marks":[{"type":"code_inline"}]},{"text":" in ","type":"text"},{"text":"config.json","type":"text","marks":[{"type":"code_inline"}]},{"text":" to keep weights private to the app. Leave as ","type":"text"},{"text":"auto","type":"text","marks":[{"type":"code_inline"}]},{"text":" only if the user explicitly wants to share weights with other apps.","type":"text"}]}]}]},{"type":"paragraph","content":[{"text":"Backend install timing — two distinct paths:","type":"text","marks":[{"type":"strong"}]}]},{"type":"blockquote","content":[{"type":"paragraph","content":[{"text":"Packaging time","type":"text","marks":[{"type":"strong"}]},{"text":" (developer machine, before bundling). Use the lemonade CLI that shipped inside ","type":"text"},{"text":"vendor/lemonade/","type":"text","marks":[{"type":"code_inline"}]},{"text":" so it matches the bundled ","type":"text"},{"text":"lemond","type":"text","marks":[{"type":"code_inline"}]},{"text":" version (prefix with ","type":"text"},{"text":"./","type":"text","marks":[{"type":"code_inline"}]},{"text":" or the full path):","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":""},"content":[{"text":"vendor/lemonade/lemonade backends install llamacpp:vulkan\nvendor/lemonade/lemonade backends install flm:npu # Windows NPU path only","type":"text"}]},{"type":"paragraph","content":[{"text":"This bakes the backend binaries into ","type":"text"},{"text":"vendor/lemonade/bin/","type":"text","marks":[{"type":"code_inline"}]},{"text":" before the app ships. ","type":"text"},{"text":"lemond","type":"text","marks":[{"type":"code_inline"}]},{"text":" does not need to be running. Use a modern ","type":"text"},{"text":"lemonade","type":"text","marks":[{"type":"code_inline"}]},{"text":" CLI whose version matches the bundled ","type":"text"},{"text":"lemond","type":"text","marks":[{"type":"code_inline"}]},{"text":" (the copy in the archive you unpacked works); do not ","type":"text"},{"text":"pip install lemonade-sdk","type":"text","marks":[{"type":"code_inline"}]},{"text":" for it.","type":"text"}]},{"type":"paragraph","content":[{"text":"First-run / runtime","type":"text","marks":[{"type":"strong"}]},{"text":" (user's machine, after ","type":"text"},{"text":"lemond","type":"text","marks":[{"type":"code_inline"}]},{"text":" is running):","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"http"},"content":[{"text":"POST /api/v1/install\n{\"recipe\": \"llamacpp\", \"backend\": \"rocm\"}","type":"text"}]},{"type":"paragraph","content":[{"text":"Use this for hardware-specific backends (e.g. ","type":"text"},{"text":"llamacpp:rocm","type":"text","marks":[{"type":"code_inline"}]},{"text":") that cannot be bundled universally. ","type":"text"},{"text":"lemond","type":"text","marks":[{"type":"code_inline"}]},{"text":" must already be running (Step 4 complete).","type":"text"}]}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Step 4: Add a ","type":"text"},{"text":"lemond","type":"text","marks":[{"type":"code_inline"}]},{"text":" launcher","type":"text"}]},{"type":"paragraph","content":[{"text":"Write the launcher as a new module named ","type":"text"},{"text":"lemond_launcher.py","type":"text","marks":[{"type":"code_inline"},{"type":"strong"}]},{"text":" (or ","type":"text"},{"text":"lemond_launcher.\u003cext>","type":"text","marks":[{"type":"code_inline"}]},{"text":" for the app's language). It is a thin process supervisor. Its only jobs:","type":"text"}]},{"type":"ordered_list","attrs":{"order":1,"listStyle":"number"},"content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Generate a fresh random API key: ","type":"text"},{"text":"key = secrets.token_urlsafe(32)","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Pick a free localhost port: bind a ","type":"text"},{"text":"socket","type":"text","marks":[{"type":"code_inline"}]},{"text":" to port 0, read back the assigned port, close it.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Spawn lemond as a ","type":"text"},{"text":"subprocess","type":"text","marks":[{"type":"code_inline"}]},{"text":": ","type":"text"},{"text":"subprocess.Popen([LEMOND_BIN, LEMOND_DIR, \"--port\", str(port)], env={**os.environ, \"LEMONADE_API_KEY\": key})","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Poll ","type":"text"},{"text":"GET /api/v1/health","type":"text","marks":[{"type":"code_inline"}]},{"text":" with ","type":"text"},{"text":"Authorization: Bearer {key}","type":"text","marks":[{"type":"code_inline"}]},{"text":" in a loop until HTTP 200 — this is the only correct readiness check.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Expose the chosen ","type":"text"},{"text":"port","type":"text","marks":[{"type":"code_inline"}]},{"text":" and ","type":"text"},{"text":"key","type":"text","marks":[{"type":"code_inline"}]},{"text":" to the rest of the app.","type":"text"}]}]}]},{"type":"blockquote","content":[{"type":"paragraph","content":[{"text":"Log one line per lifecycle stage.","type":"text","marks":[{"type":"strong"}]},{"text":" Build the logging in from the start — not as an afterthought when something breaks. Each silent transition needs a visible marker so a failure points at the exact stage. Aim for:","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":""},"content":[{"text":"[lemond] Starting on port \u003cport>\n[lemond] Healthy on port \u003cport>\n[lemond] \u003crecipe>:\u003cbackend> installed (or: already installed / install failed)\n[lemond] Pulling model \u003cname>... then: Model \u003cname> ready (or: pull returned \u003cstatus>)\n[local] \u003cmodality> result: \u003cvalue> (first inference output — empty string here = unpulled model)","type":"text"}]},{"type":"paragraph","content":[{"text":"Logging the ","type":"text"},{"text":"first inference result verbatim","type":"text","marks":[{"type":"strong"}]},{"text":" is what turns the silent-empty failure (Step 6) from a multi-hour mystery into a one-line diagnosis. Route these through the app's normal logging so they can be quieted for release.","type":"text"}]}]},{"type":"blockquote","content":[{"type":"paragraph","content":[{"text":"Dev-mode file watchers:","type":"text","marks":[{"type":"strong"}]},{"text":" If the app runs with a file watcher (Tauri, Electron, Next.js, Vite, etc.) that watches the source tree, ensure ","type":"text"},{"text":"vendor/lemonade/","type":"text","marks":[{"type":"code_inline"}]},{"text":" is excluded from the watched paths. Lemond writes config and cache files at runtime; a watcher that picks these up will restart the app, kill the lemond subprocess, and spawn a new one on a new port — silently breaking any in-flight transcription. Add ","type":"text"},{"text":"vendor/","type":"text","marks":[{"type":"code_inline"}]},{"text":" (or the equivalent) to the watcher's ignore list before testing.","type":"text"}]}]},{"type":"paragraph","content":[{"text":"Use the reference implementation from ","type":"text","marks":[{"type":"strong"}]},{"text":"reference.md § Reference launchers","type":"text","marks":[{"type":"link","attrs":{"href":"reference.md#reference-launchers","title":null}},{"type":"strong"}]},{"text":" directly","type":"text","marks":[{"type":"strong"}]},{"text":" — copy it verbatim and adapt only the ","type":"text"},{"text":"LEMOND_DIR","type":"text","marks":[{"type":"code_inline"}]},{"text":" path. Do not write a launcher from scratch. The reference Python launcher uses ","type":"text"},{"text":"secrets","type":"text","marks":[{"type":"code_inline"}]},{"text":" (for the API key), ","type":"text"},{"text":"socket","type":"text","marks":[{"type":"code_inline"}]},{"text":" (for the free-port probe), and ","type":"text"},{"text":"subprocess","type":"text","marks":[{"type":"code_inline"}]},{"text":" (to spawn lemond); the Node.js launcher uses the equivalent stdlib modules. Both handle port-race retries and health polling correctly.","type":"text"}]},{"type":"paragraph","content":[{"text":"Readiness is always determined by polling the exact endpoint ","type":"text"},{"text":"GET http://127.0.0.1:\u003cport>/api/v1/health","type":"text","marks":[{"type":"code_inline"}]},{"text":" and checking for HTTP 200 — never by reading ","type":"text"},{"text":"lemond","type":"text","marks":[{"type":"code_inline"}]},{"text":"'s stdout or stderr. Any health-check helper you write must hit that ","type":"text"},{"text":"/api/v1/health","type":"text","marks":[{"type":"code_inline"}]},{"text":" path.","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Step 5: Re-point the existing client at ","type":"text"},{"text":"lemond","type":"text","marks":[{"type":"code_inline"}]}]},{"type":"paragraph","content":[{"text":"Make ","type":"text"},{"text":"three","type":"text","marks":[{"type":"strong"}]},{"text":" changes to the app's existing client construction — all three are required, not optional:","type":"text"}]},{"type":"ordered_list","attrs":{"order":1,"listStyle":"number"},"content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Set ","type":"text"},{"text":"base_url","type":"text","marks":[{"type":"code_inline"}]},{"text":" to ","type":"text"},{"text":"http://127.0.0.1:{port}/api/v1","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Set ","type":"text"},{"text":"api_key","type":"text","marks":[{"type":"code_inline"}]},{"text":" to the launcher key","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Set the HTTP timeout to 120 seconds","type":"text","marks":[{"type":"strong"}]},{"text":" — this is mandatory, not optional","type":"text"}]}]}]},{"type":"paragraph","content":[{"text":"The 120-second timeout is not a tuning suggestion. The default on most HTTP clients is 30s, which is shorter than lemond's first-run model load time on real hardware. Without it the request silently times out and the UI shows nothing, which is indistinguishable from a broken integration.","type":"text"}]},{"type":"paragraph","content":[{"text":"Python (openai) — the exact change to make:","type":"text","marks":[{"type":"strong"}]}]},{"type":"code_block","attrs":{"wrap":false,"language":"python"},"content":[{"text":"import httpx\nfrom openai import OpenAI\n\nproc, key, port = start_lemond()\nclient = OpenAI(\n base_url=f\"http://127.0.0.1:{port}/api/v1\",\n api_key=key,\n http_client=httpx.Client(timeout=120), # required: 120s for first-run model load\n)","type":"text"}]},{"type":"paragraph","content":[{"text":"For other clients:","type":"text"}]},{"type":"table","attrs":{"layout":null},"content":[{"type":"tr","content":[{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Existing client","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"New ","type":"text"},{"text":"base_url","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"New auth","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Timeout","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"openai-python","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"http://127.0.0.1:{port}/api/v1","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"api_key=key","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"httpx.Client(timeout=120)","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"openai-node","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"http://127.0.0.1:{port}/api/v1","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"apiKey: key","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"timeout: 120000","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"@anthropic-ai/sdk","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"http://127.0.0.1:{port}/api/v1","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"apiKey: key","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"timeout: 120000","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Raw ","type":"text"},{"text":"fetch","type":"text","marks":[{"type":"code_inline"}]},{"text":" / ","type":"text"},{"text":"requests","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"same","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Authorization: Bearer {key}","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"set per-request","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Ollama-compatible code","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"http://127.0.0.1:{port}/api/v0","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"pass key anyway","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"120s","type":"text"}]}]}]}]},{"type":"paragraph","content":[{"text":"The model identifier on requests stays a Lemonade model name (e.g. ","type":"text"},{"text":"Qwen3-4B-GGUF","type":"text","marks":[{"type":"code_inline"}]},{"text":"), not the cloud name.","type":"text"}]},{"type":"paragraph","content":[{"text":"Local mode needs no cloud API key — at all.","type":"text","marks":[{"type":"strong"}]},{"text":" This is a defining property of local mode, not an edge case: there is no cloud service to authenticate to, so nothing should ever ask the user for a key. Any onboarding wall, validator, or startup check that demands one must not block local-mode users. Concretely:","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Skip or auto-satisfy the key-entry screen in local mode.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Treat local mode as already-authorized in every validation path — an empty-key check must short-circuit to \"valid\" when the active mode is local, never throw \"API key not configured\".","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Re-enable the gate ","type":"text"},{"text":"only","type":"text","marks":[{"type":"strong"}]},{"text":" for cloud mode.","type":"text"}]}]}]},{"type":"paragraph","content":[{"text":"The ","type":"text"},{"text":"lemond","type":"text","marks":[{"type":"code_inline"}]},{"text":" key from Step 4 is generated internally by the launcher and used only for the local loopback connection, so the user never sees or enters one; any UI placeholder (e.g. ","type":"text"},{"text":"\"local\"","type":"text","marks":[{"type":"code_inline"}]},{"text":") is fine. Flipping into local mode should never strand the user on a key-entry wall.","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Step 6: Health, backend, then pull the model — ","type":"text"},{"text":"before","type":"text","marks":[{"type":"em"}]},{"text":" first inference","type":"text"}]},{"type":"paragraph","content":[{"text":"GET /api/v1/health","type":"text","marks":[{"type":"code_inline"}]},{"text":" returning 200 means the ","type":"text"},{"text":"server","type":"text","marks":[{"type":"strong"}]},{"text":" is up. It does ","type":"text"},{"text":"not","type":"text","marks":[{"type":"strong"}]},{"text":" mean inference will work. Before the first real request succeeds, three more things must be true: the backend for your modality is installed, the model's weights are ","type":"text"},{"text":"downloaded to disk","type":"text","marks":[{"type":"strong"}]},{"text":", and (on the first call) the model is loaded into memory. Treating health=200 as \"ready\" is the single biggest cause of a broken-looking integration.","type":"text"}]},{"type":"paragraph","content":[{"text":"Do not call ","type":"text","marks":[{"type":"strong"}]},{"text":"POST /api/v1/load","type":"text","marks":[{"type":"code_inline"},{"type":"strong"}]},{"text":" at startup.","type":"text","marks":[{"type":"strong"}]},{"text":" Lemond lazy-loads the model into memory on the first inference request and handles that step on its own. Pre-loading is unreliable across lemond versions (the ","type":"text"},{"text":"/load","type":"text","marks":[{"type":"code_inline"}]},{"text":" request body shape has changed between releases) and a malformed call can crash or destabilise the server before the user takes any action. Loading is the one step you let lemond do lazily — pulling is not.","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Pull the model so it exists on disk","type":"text"}]},{"type":"paragraph","content":[{"text":"Lazy-load only loads weights that are ","type":"text"},{"text":"already downloaded","type":"text","marks":[{"type":"strong"}]},{"text":". If the model was never pulled, the first inference does not error — lemond returns an empty / blank result with HTTP 200. So after health passes and the backend is installed, proactively pull the model:","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"http"},"content":[{"text":"POST /api/v1/pull\n{\"model\": \"Whisper-Large-v3-Turbo\"}","type":"text"}]},{"type":"paragraph","content":[{"text":"This is ","type":"text"},{"text":"idempotent","type":"text","marks":[{"type":"strong"}]},{"text":" — a no-op if the weights are already present, a download if they are not. Run it once during setup (after backend install, before the first user-triggered inference) and log the result.","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Default model","type":"text","marks":[{"type":"strong"}]},{"text":" (the one you chose in Step 2): pull it by name as above.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Custom / user-overridden model:","type":"text","marks":[{"type":"strong"}]},{"text":" do not assume it exists. Confirm it is a real Lemonade model first via ","type":"text"},{"text":"GET /api/v1/models","type":"text","marks":[{"type":"code_inline"}]},{"text":" (the ","type":"text"},{"text":"only","type":"text","marks":[{"type":"strong"}]},{"text":" trusted catalog — see ","type":"text"},{"text":"reference.md","type":"text","marks":[{"type":"link","attrs":{"href":"reference.md","title":null}}]},{"text":"), then pull it the same way. A model appearing in the catalog is ","type":"text"},{"text":"not","type":"text","marks":[{"type":"strong"}]},{"text":" proof its weights are downloaded; a successful pull is.","type":"text"}]}]}]},{"type":"blockquote","content":[{"type":"paragraph","content":[{"text":"Silent-empty is almost always an unpulled model.","type":"text","marks":[{"type":"strong"}]},{"text":" If inference returns an empty string / blank output with no HTTP error, the model was not downloaded. Check your pull step before debugging anything else — this is the failure mode that wastes the most time. Log the pull result and the first inference result (see Step 4) so this is diagnosable from the console, not by guesswork.","type":"text"}]}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Surface the ","type":"text"},{"text":"whole","type":"text","marks":[{"type":"em"}]},{"text":" setup, not just model load","type":"text"}]},{"type":"paragraph","content":[{"text":"First-run cold start is more than a model load. The full sequence is:","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":""},"content":[{"text":"server spawn → health 200 → backend install → model download → model load → first result","type":"text"}]},{"type":"paragraph","content":[{"text":"On a fresh machine, backend install and model download can each take from tens of seconds to several ","type":"text"},{"text":"minutes","type":"text","marks":[{"type":"strong"}]},{"text":" (multi-GB weights over the network). Model load alone is 10–30s. An app that shows nothing during this will look frozen.","type":"text"}]},{"type":"paragraph","content":[{"text":"Minimum: show a loading indicator or status message (\"Setting up local AI…\") from the moment setup begins until the first response arrives — covering the ","type":"text"},{"text":"entire","type":"text","marks":[{"type":"em"}]},{"text":" sequence above, not just the final load. The simplest implementation is a flag set when setup/first-request starts and cleared when the first response arrives. Once the model is pulled and loaded once, subsequent runs are fast; the long wait is first-run only.","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Step 7: Lifecycle and recovery","type":"text"}]},{"type":"paragraph","content":[{"text":"These are the only failure modes worth handling. Do not over-engineer.","type":"text"}]},{"type":"table","attrs":{"layout":null},"content":[{"type":"tr","content":[{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Symptom","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Cause","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Recovery","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Inference returns empty / blank with HTTP 200, no error","type":"text","marks":[{"type":"strong"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Model never pulled: backend is installed but weights are absent, so lazy-load has nothing to load","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"POST /api/v1/pull","type":"text","marks":[{"type":"code_inline"}]},{"text":" with ","type":"text"},{"text":"{\"model\":\"...\"}","type":"text","marks":[{"type":"code_inline"}]},{"text":", wait for success, retry. Log the pulled result and the first inference result. This is the most common silent failure — see ","type":"text"},{"text":"Step 6","type":"text","marks":[{"type":"link","attrs":{"href":"#step-6-health-backend-then-pull-the-model--before-first-inference","title":null}}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"POST /api/v1/load","type":"text","marks":[{"type":"code_inline"}]},{"text":" returns 404 / model not found","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Model not pulled yet (same root cause as the empty-result row above)","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"POST /api/v1/pull","type":"text","marks":[{"type":"code_inline"}]},{"text":" with ","type":"text"},{"text":"{\"model\": \"...\"}","type":"text","marks":[{"type":"code_inline"}]},{"text":" then retry ","type":"text"},{"text":"/api/v1/load","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"POST /api/v1/load","type":"text","marks":[{"type":"code_inline"}]},{"text":" returns 500 with backend error","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Backend not installed for this hardware","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"GET /api/v1/system-info","type":"text","marks":[{"type":"code_inline"}]},{"text":", pick a supported backend, ","type":"text"},{"text":"POST /api/v1/install","type":"text","marks":[{"type":"code_inline"}]},{"text":" with ","type":"text"},{"text":"{\"recipe\": \"...\", \"backend\": \"...\"}","type":"text","marks":[{"type":"code_inline"}]},{"text":", retry","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Subprocess exits immediately","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Port race: another process grabbed the port between ","type":"text"},{"text":"freePort()","type":"text","marks":[{"type":"code_inline"}]},{"text":" and lemond binding","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"The reference launcher retries with a fresh port automatically (3 attempts)","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"/api/v1/health","type":"text","marks":[{"type":"code_inline"}]},{"text":" never returns 200","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"First-run backend extraction is slow on cold disk","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Extend timeout to 90s on first launch, 30s after","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"HTTP 401 on every request","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Forgot the ","type":"text"},{"text":"Authorization: Bearer","type":"text","marks":[{"type":"code_inline"}]},{"text":" header","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Audit the client config because Lemonade rejects unauth'd calls when ","type":"text"},{"text":"LEMONADE_API_KEY","type":"text","marks":[{"type":"code_inline"}]},{"text":" is set","type":"text"}]}]}]}]},{"type":"paragraph","content":[{"text":"Shutdown:","type":"text","marks":[{"type":"strong"}]},{"text":" On app exit, ","type":"text"},{"text":"proc.terminate()","type":"text","marks":[{"type":"code_inline"}]},{"text":" (Unix) or ","type":"text"},{"text":"proc.kill()","type":"text","marks":[{"type":"code_inline"}]},{"text":" (Windows). ","type":"text"},{"text":"lemond","type":"text","marks":[{"type":"code_inline"}]},{"text":" flushes config and exits cleanly within a couple of seconds. Always wait on the process; never orphan it.","type":"text"}]},{"type":"paragraph","content":[{"text":"Do not","type":"text","marks":[{"type":"strong"}]},{"text":" parse ","type":"text"},{"text":"lemond","type":"text","marks":[{"type":"code_inline"}]},{"text":" stdout to detect readiness; use the HTTP ","type":"text"},{"text":"/api/v1/health","type":"text","marks":[{"type":"code_inline"}]},{"text":" probe. Stdout format is not a stable contract.","type":"text"}]},{"type":"hr","attrs":{"markup":"---"}},{"type":"heading","attrs":{"level":2},"content":[{"text":"Verification checklist","type":"text"}]},{"type":"paragraph","content":[{"text":"The integration is done when ","type":"text"},{"text":"all","type":"text","marks":[{"type":"strong"}]},{"text":" of these are true:","type":"text"}]},{"type":"checkbox_list","attrs":{"id":null},"content":[{"type":"checkbox_item","attrs":{"checked":false},"content":[{"type":"paragraph","content":[{"text":"vendor/lemonade/","type":"text","marks":[{"type":"code_inline"}]},{"text":" contains the full package: ","type":"text"},{"text":"lemond[.exe]","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"lemonade[.exe]","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"LICENSE","type":"text","marks":[{"type":"code_inline"}]},{"text":", and ","type":"text"},{"text":"resources/","type":"text","marks":[{"type":"code_inline"}]},{"text":" — not just the binary.","type":"text"}]}]},{"type":"checkbox_item","attrs":{"checked":false},"content":[{"type":"paragraph","content":[{"text":"lemond","type":"text","marks":[{"type":"code_inline"}]},{"text":" starts as a subprocess with a fresh API key per launch.","type":"text"}]}]},{"type":"checkbox_item","attrs":{"checked":false},"content":[{"type":"paragraph","content":[{"text":"GET /api/v1/health","type":"text","marks":[{"type":"code_inline"}]},{"text":" returns 200 within the timeout.","type":"text"}]}]},{"type":"checkbox_item","attrs":{"checked":false},"content":[{"type":"paragraph","content":[{"text":"The default model is pulled (or bundled) before the first inference; a custom/overridden model is confirmed via ","type":"text"},{"text":"GET /api/v1/models","type":"text","marks":[{"type":"code_inline"}]},{"text":" and then pulled. A blank result with no error means this step was skipped.","type":"text"}]}]},{"type":"checkbox_item","attrs":{"checked":false},"content":[{"type":"paragraph","content":[{"text":"Each lifecycle stage logs a clear line (spawn, health, backend install, model pull, first result) so a failure is diagnosable from the console.","type":"text"}]}]},{"type":"checkbox_item","attrs":{"checked":false},"content":[{"type":"paragraph","content":[{"text":"The existing client's chat / image / speech call returns a valid response with the base URL and key swapped, with no other code changed.","type":"text"}]}]},{"type":"checkbox_item","attrs":{"checked":false},"content":[{"type":"paragraph","content":[{"text":"First-run latency is surfaced: the interface shows a loading state from the moment the first inference request is sent until the response arrives.","type":"text"}]}]},{"type":"checkbox_item","attrs":{"checked":false},"content":[{"type":"paragraph","content":[{"text":"The HTTP client timeout is set to 120 seconds.","type":"text"}]}]},{"type":"checkbox_item","attrs":{"checked":false},"content":[{"type":"paragraph","content":[{"text":"In local mode the app requires ","type":"text"},{"text":"no","type":"text","marks":[{"type":"strong"}]},{"text":" cloud API key: no onboarding wall, validator, or startup check blocks the user, and no code path throws \"API key not configured\" when the active mode is local.","type":"text"}]}]},{"type":"checkbox_item","attrs":{"checked":false},"content":[{"type":"paragraph","content":[{"text":"If the app uses a dev-mode file watcher, ","type":"text"},{"text":"vendor/lemonade/","type":"text","marks":[{"type":"code_inline"}]},{"text":" is excluded from the watched paths so runtime writes by lemond do not trigger restarts.","type":"text"}]}]},{"type":"checkbox_item","attrs":{"checked":false},"content":[{"type":"paragraph","content":[{"text":"Killing the parent process leaves no ","type":"text"},{"text":"lemond","type":"text","marks":[{"type":"code_inline"}]},{"text":" subprocess behind.","type":"text"}]}]},{"type":"checkbox_item","attrs":{"checked":false},"content":[{"type":"paragraph","content":[{"text":"On a fresh machine without the optimal backend, the app still works via the Vulkan fallback bundled in ","type":"text"},{"text":"bin/","type":"text","marks":[{"type":"code_inline"}]},{"text":".","type":"text"}]}]}]},{"type":"paragraph","content":[{"text":"If any box is unchecked, do not declare the task complete.","type":"text"}]},{"type":"hr","attrs":{"markup":"---"}},{"type":"heading","attrs":{"level":2},"content":[{"text":"Reference","type":"text"}]},{"type":"paragraph","content":[{"text":"For detailed model catalog, backend selection matrix, full endpoint reference, config keys, and per-model ","type":"text"},{"text":"recipe_options.json","type":"text","marks":[{"type":"code_inline"}]},{"text":" tuning, see ","type":"text"},{"text":"reference.md","type":"text","marks":[{"type":"link","attrs":{"href":"reference.md","title":null}}]},{"text":".","type":"text"}]}]},"metadata":{"date":"2026-11-02","name":"local-ai-app-integration","author":"@skillopedia","source":{"stars":473,"repo_name":"skills","origin_url":"https://github.com/amd/skills/blob/HEAD/skills/local-ai-app-integration/SKILL.md","repo_owner":"amd","body_sha256":"e66d30bcb8969ed98679fa2332b65002d5b94c299b843916d026a1aa6e2718e3","cluster_key":"8e9fef3bdd38f578e40af77b3cf818878ae4411b6653dafd9af154830020c609","clean_bundle":{"format":"clean-skill-bundle-v1","source":"amd/skills/skills/local-ai-app-integration/SKILL.md","text_bytes":17354,"source_commit":"c67366c8d1ab85bbe5f5e9000f40617a1bfb9edb","skills_sh_rank":12892,"attachment_count":5,"text_attachments":5,"attachment_storage":"skill_attachments","binary_attachments":0},"cluster_size":1,"skill_md_path":"skills/local-ai-app-integration/SKILL.md","import_metadata":{"date":"2026-11-02","author":"@skillopedia","version":"v1","category":"ai-agent-development","category_label":"AI"},"exact_dupes_collapsed_into_this":0},"version":"v1","category":"ai-agent-development","import_tag":"skills-sh-top20k-2026-11","description":"Integrates local AI capabilities into applications using Embeddable Lemonade. Use when the user wants to add local AI, offline AI, private AI, on-device AI, a local LLM, local chat, embeddings, image generation, speech-to-text, or text-to-speech to an existing app; replace or supplement OpenAI, Anthropic, Ollama, or other cloud AI APIs with a local backend; only use to convert user apps. Do not use when the user just wants the agent itself to generate images, transcribe, or speak locally in the current workspace, even to cut their own API bill."}},"renderedAt":1793666307048}

Local AI App Integration (Embeddable Lemonade) Add a local AI mode to an existing app that already talks to a cloud AI API (OpenAI, Anthropic, or Ollama-compatible). The app launches , the Embeddable Lemonade binary, as a private subprocess and the existing client talks to it on . The user gets local, private, hardware-optimized inference (CPU, AMD iGPU/dGPU, XDNA2 NPU) with no separate install. What you'll end up with: one new launcher module ( 30 lines), three mandatory changes to the existing HTTP client ( , , and a 120-second HTTP timeout), one vendored binary under . When this skill is t…