geo-sleuth — Skillopedia

geo-sleuth (v2) Goal: find a location that holds up under checking . Every conclusion must point to the clue it used, the command it ran and the file it produced. in commands means the directory containing this SKILL.md (the same for commands in references). Claude Code substitutes it automatically; in other agents, first before running (if the shell doesn't keep variables between commands, prefix every command with it), or replace it with that path directly. The work is split into three layers. Remember this first, then read the flow: | Layer | Who | Tools | |---|---|---| | Decision: which c…

\\t' read -r qid url; do\n [ -s \"thumbs/$qid.jpg\" ] || curl -sfL -A \"$UA\" \"$url?width=480\" -o \"thumbs/$qid.jpg\"; sleep 1\n done\n# 3. Rank them against the building in the photo\nuv run ${CLAUDE_SKILL_DIR}/scripts/match.py rank --query photo.jpg --query-box \u003cbuilding part> --images thumbs/ \\\n --method both --refine sift --top 24 --out pool_rank.json --sheet pool_rank.jpg\n```\n\n- Throttled Wikimedia requests get an HTML page (HTTP 429). If `match.py` prints `skipped N of M files that are not readable images`, fetch those again before ranking.\n- `--images` has no candidate cap and loads everything into memory: keep a run to about 400 images by splitting the bbox.\n- Items without a P18 image are not ranked. Count them (`pool.json` rows minus thumbnails) and say so in the conclusion; they are not excluded.\n- Season, light and side differ, so SIFT inliers stay near zero even for the right item: compare the top 24 in `pool_rank.jpg` by plan, tower, roof and windows; a shortlisted item goes on the board and needs ≥3 invariant features (SKILL step 5).\n- `P279*` already covers subclasses (cathedrals, chapels under church building `wd:Q16970`). If nothing fits, widen to `wd:Q1370598` (structure of worship), then drop the class line, before widening the region.\n\n## Common mistakes\n\n- Using only one engine and searching only the whole image; after a failure, not changing the image or the engine.\n- Turning the \"object category\" from an AI or image recognition directly into a location (\"boat-shaped sculpture → tourist city\"); this jump counts only as weak evidence (v005).\n- Getting carried off by the \"most famous similar place\" the search turns up and forgetting to come back and check bearings and details (the main way the AI failed in v010).\n- Two engines each point at a different city, and you draw a big circle around the midpoint of the two and call it done: instead run a discriminating test and pick one as the main answer (SKILL hard rule 8).\n- Image search gave a residential compound or housing-development name, and you didn't get its coordinates.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":17951,"content_sha256":"82c9c2d3606b6ea03f15a0754c4a316ba6b19ee3a8cde2f6e448f7baa9421f20"},{"filename":"references/sky.md","content":"# Sky clues: sun, shadows, satellite dishes\n\nScripts: `scripts/sun.py` (`pos` / `ratio` / `locate` / `when` / `street` / `facing` / `compass` / `dish`). Compared with NREL SPA, the algorithm's error is ≤0.02°.\nWhen there are no landmarks and text only gets you to a province or country, this is the main way to shrink hundreds of kilometers to tens of kilometers (v001: from hundreds of kilometers down to tens).\n\n## When to take this branch\n\n| Condition | What you get |\n|---|---|\n| Clear shadows + known capture time (puzzle setter's hint, EXIF, screenshot status bar, chat log timestamps) | Sun elevation angle → a band on the map; plus direction → a segment of the band |\n| Clear shadows + location already fixed | Capture time and date (two candidate sets) |\n| Satellite TV dishes in the frame | Photo heading (use them as a compass); dish azimuth → longitude range |\n| The sun itself in the frame (sunrise/sunset, backlight) | Lens heading and the true bearings of objects in the frame (`sun.py compass`); given a clock time, you can also exclude countries with the wrong time zone |\n| Overcast, night, indoors | Not usable; don't force a calculation |\n\nBe clear about the capture time: posting time and IP location are not the capture time and place; treat them only as hypotheses to verify.\n\n**Times in saved file names are in the saver's time zone.** Messenger desktop clients (e.g. Telegram Desktop `photo_YYYY-MM-DD_HH-MM-SS.jpg`) write the saving computer's local time. Infer that zone (an inferred clue) and pass its offset: `sun.py pos --at \u003ccandidate> --time 2025-03-14T18:30+03:00 --tz \u003cscene IANA zone>`; an hour's gap can put the sun on the other side of the horizon. The capture is at or before this time.\n\n**Without a time, don't infer road direction, river direction or course from the sun's direction**. The sun at the side of the frame only says the lens is roughly perpendicular to the sun's azimuth; morning, noon and afternoon correspond to completely different directions (the creators in v002, v008 and v012 all reasoned this way and happened to be right; the method doesn't hold).\n\n## 1. Determine the photo heading first\n\nIn order of reliability:\n1. **Satellite dishes**: dishes in the same area point the same way. China's Hu Hu Tong (direct-to-home) small dishes point at ChinaSat 9; in eastern cities they face southwest (about 215°–227°), in the west south or even southeast (see the table in section 5). In Europe dishes generally face south, slightly east.\n2. **Shadows or lit faces + time**: north of the Tropic of Cancer, the noon shadow points due north, northwest in the morning and northeast in the afternoon, turning clockwise through the day. With a date and time, compute it; don't estimate:\n `uv run scripts/sun.py pos --at \u003capprox. location> --time 2025-01-26T08:53 --tz Asia/Shanghai`\n On winter mornings before 9 a.m. the sun's azimuth is about 120°, not due east (v006 took the sun as due east; the river direction could be off by 30°). Whichever face of a building is brightest faces roughly toward the sun.\n3. **Lit faces** (when there's no measurable shadow): a wall is lit if and only if the angle between the sun's azimuth and the wall's normal is less than 90°. Intersect the lit/shaded state of several walls and you get a range for the lens heading:\n `uv run scripts/sun.py facing --at \u003capprox. location> --time \u003ctime> --tz \u003ctime zone> --lit left --shaded camera`\n Describe walls by which way they face in the frame: `camera` faces the lens, `left` faces frame left, `right` faces frame right. At the range boundaries the wall is nearly parallel to the sunlight; don't treat them as hard boundaries.\n **When shadow direction conflicts with wall brightness or the lit side of tree trunks, trust the shadow** (in two cases, reading wall brightness put the sun on the wrong side): white walls and pale bark also look bright in diffuse light. The easiest thing to read is a car parked at the roadside: use the car length of about 4.5 m as a ruler; how far the shadow extends beyond the front of the car and which side of the car it falls toward directly tell you whether the sun is in front or behind, left or right, and roughly how low its elevation angle is. Tree-shadow stripes on sidewalks come from trees outside the frame and their direction is hard to judge; use them only as a reference.\n4. **The sun itself in the frame**: it is the compass. Compute the lens heading from the sun's pixel position; you can also compute the true bearing of any object in the frame (tower, chimney, building corner), to judge \"which block is the tower in the photo\" within a candidate factory site and which side of the building the camera position is on:\n `uv run scripts/sun.py compass --at \u003ccandidate area> --time \u003cclock time> --dates \u003cpossible date range> --tz \u003ctime zone> --sun-x \u003cpixels> --width \u003cimage width> --hfov \u003cFOV range> --x \u003cobject pixels>`\n - Only a time given, no date: compute over the possible date range, and the bearings become ranges; then add `--elev lo:hi` from how high the sun is above the horizon in the frame; dates whose elevation doesn't match are dropped automatically and the range narrows.\n - Neither time nor date: `compass` defaults to the whole year, split into a sunrise set and a sunset set. Around 30°N, the sunset azimuth swings between about 240° and 300° over a year; the difference is enough to make a top-down template not match. **Posting time is not capture time**; don't use the posting month to narrow the date.\n - Telling sunrise from sunset: what the poster says, sky color, shadow changes, road traffic and streetlights. Don't use reasons like \"nobody at the school → after school\", which holidays and weekends overturn (v014). If you can't tell, build templates for both headings.\n - **Glow without the sun in the frame is not a compass.** If the sun is low (elevation above 0, below the frame's top edge), out of frame and nothing in the frame could hide it, the heading lies *outside* sun azimuth ± hfov/2, often with the sun just past one edge. Below the horizon the glow only gives the sun's half of the sky (±90°). Write the heading as a range and let a landmark, the street layout or `geo.py bearings` pick the value.\n5. **Roads on satellite imagery**: once you have location candidates, match the directions of roads, walls and riverbanks in the frame against satellite imagery.\n6. Claims like moss or tree crowns leaning one way are unreliable; don't use them.\n\n## 2. Measure the shadow length ratio\n\n`sun elevation angle = atan(object height / shadow length)`. Only the ratio is needed, not real sizes.\n\n- Pick **vertical** objects (wall corners, utility poles, lamp posts, vertical wall edges) and their shadows on **flat** ground.\n- Prefer shadows parallel to the image plane (spread sideways); a shadow pointing toward or away from the lens is heavily foreshortened, so pick another object.\n- For wide-angle photos, first correct barrel distortion, then draw guide lines along the perspective direction to measure (the v001 creator corrected first, then measured).\n- In the same image, the shadows of vertical objects on flat ground must be parallel to each other. Not parallel = uneven ground, a non-vertical object, an artificial light source or a composite image.\n- Measure with at least two objects and take a range. A 20% shadow-length error gives a 6–8° elevation-angle error near 45°.\n\n```bash\nuv run scripts/sun.py ratio --shadow 1.2 # 1:1.2 → 39.8°\n```\n\n## 3. Known time → a band on the map\n\nThe points where the sun's elevation angle is the same at the same moment form a large circle centered on the subsolar point; adding tolerance makes it a band. Intersect it with the existing candidate areas.\n\n```bash\n# Beijing time 2023-08-15 16:20, shadow ratio 1.2, ±1°, candidate area in North China\nuv run scripts/sun.py locate --time 2023-08-15T16:20 --tz Asia/Shanghai --ratio 1.2 --tol 1 \\\n --bbox 34,110,42,122 --step 0.05 --mosaic north.jpg --out band.jpg\n# Add the shadow bearing (northeast, 60°±10°) and the band shrinks to a segment\nuv run scripts/sun.py locate ... --shadow-bearing 60 --az-tol 10\n```\n\n- When the time is uncertain, add `--time-tol 30` (±30 minutes).\n- `--mosaic` draws the band directly on a low-zoom base map from `tiles.py fetch` (zoom 7–9), for the evidence image.\n- Verified: plugging in a video puzzle's known answer point and the time the puzzle setter gave, the computed shadow ratio matched the creator's measurement.\n\n### China-specific: a single Beijing time = a longitude clue\n\nThe whole country uses UTC+8, yet spans 73°E–135°E. Beijing time of solar noon (shadow pointing due north): Shanghai about 11:56, Lanzhou about 13:06, Lhasa about 13:57, Urumqi about 14:11 (June; it shifts by about another ±15 minutes over the year).\nSo **knowing the clock time + being able to see the shadow direction** directly constrains longitude: if the shadow still points due north at 2 p.m., it can't be in the east.\n\n## 4. Known location → capture time and date\n\n```bash\nuv run scripts/sun.py when --at 30.25,120.16 --date 2024-10-01 --tz Asia/Shanghai --ratio 1.2 --shadow-bearing 30\nuv run scripts/sun.py when --at 30.25,120.16 --dates 2024-01-01:2024-12-31 --tz Asia/Shanghai --elev 40 --shadow-bearing 330\n```\n\n- The same sun position occurs twice a year (symmetric about the summer or winter solstice), so results always come in two sets of dates; pick one using vegetation (fallen leaves, flowering, rice-paddy color) and clothing.\n- Date resolution is high around the equinoxes; for several weeks around the solstices dates are nearly indistinguishable; say so when reporting.\n- The same elevation angle occurs at two times in a day, morning and afternoon; separate them by shadow **direction** (v007: 08:39 shadow to the northwest, 15:03 shadow to the northeast).\n- **Report the sensitivity before the time**: `when` prints \"1° off ≈ N minutes\". Write out how many minutes a 1 m error in object height and a 1 m error in shadow length each correspond to (v007: each 1 m of object-height error shifted the time by about 13 minutes; the creator's ±5 minutes was luck).\n- Align the shadow tip with the edge that actually casts it (eave, canopy, railing top), not with the base of the wall.\n\n### Known city and date → street orientation\n\nOnly the angle of the shadow's **direction** relative to the street lets you infer the street orientation; shadow length doesn't (v009 inferred the orientation from shadow length; the reasoning doesn't hold).\n\n```bash\n# In the top-down view, turning clockwise from the street direction to the shadow direction is 90° (shadow perpendicular to the street); if you can't tell clockwise from counterclockwise, add --both\nuv run scripts/sun.py street --at 49.25,-123.10 --date 2025-04-01 --tz America/Vancouver --ratio 1.5 --tol 4 --shadow-rel 90\n```\n\nOutputs two sets of directions, morning and afternoon. In a grid city one direction cuts out half the streets; when the date is uncertain, try several days and see how much the direction changes.\n\n## 5. Satellite TV dishes\n\nDishes point at geostationary satellites above the equator; the azimuth is determined by \"the difference between the local longitude and the satellite's longitude\".\n\nChina's Hu Hu Tong / Cun Cun Tong (direct-to-home / village coverage) small dishes (35–60 cm diameter, common on rooftops in rural areas and urban villages) almost all point at ChinaSat 9 (92.2°E):\n\n| City | Azimuth | Elevation |\n|---|---|---|\n| Harbin | 224° | 27° |\n| Beijing | 215° | 38° |\n| Shanghai | 227° | 42° |\n| Guangzhou | 225° | 54° |\n| Lanzhou | 199° | 46° |\n| Chengdu | 202° | 52° |\n| Kunming | 204° | 59° |\n| Lhasa | 178° | 55° |\n| Urumqi | 173° | 39° |\n| Kashgar | 156° | 41° |\n\n```bash\nuv run scripts/sun.py dish --at 31.2,121.5 --sat 92.2 # forward\nuv run scripts/sun.py dish --lat 31.2 --sat 92.2 --azimuth 213 # dish faces 213° → longitude about 109°–113°\n```\n\n- Azimuth readings are more reliable than elevation. Small dishes are mostly offset-feed; the dish face looks more \"upright\" than its actual pointing (offset angle 20°–25°), so don't take the dish face tilt as the elevation angle.\n- Large dishes (1.5 m and up) at institutions and cable TV stations may point at ChinaSat 6B (115.5°) or an AsiaSat satellite; you can't apply the Hu Hu Tong table to them.\n- Common in Europe: Astra 1 (19.2°E, Germany/Austria), Hot Bird (13°E, Italy), Astra 2 (28.2°E, UK). There's a list in `sun.py`.\n\n## Common mistakes\n\n- Wrong time zone or DST: use an IANA name for `--tz` (`Europe/Berlin` handles DST automatically); don't hand-write +1/+2.\n- Taking posting time as capture time, or a saved file name's time as the scene's local time.\n- Setting the lens heading to the sun azimuth because the horizon glows, while the sun itself is out of frame.\n- Measuring shadow length on slopes or steps; taking a slanted shadow (from a non-vertical object) as the shadow of a vertical object.\n- Computing a band and looking only at its center line: the whole band within tolerance must be checked; prioritize the parts that intersect other clues.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":13217,"content_sha256":"23dd598d20d1eba300c89cdde11dd39c9191cfcd0384d1c40de8adb1c8df68b4"},{"filename":"references/skyline-fallback.md","content":"# Skyline fallback: when `terrain.py ridge` misses the ridge\n\n`terrain.py ridge` finds the sky→mountain brightness drop column by column. It works on clear skies; it breaks where cloud sits on the peaks, snow meets a pale sky, or haze flattens the contrast, and then it follows the cloud edge and `fit` never brings the residual down. Always look at `ridge --png` before fitting. If the yellow dots leave the real ridge, work down this list and stop at the first step that puts them back.\n\n## 1. Tune the default detector\n\n- `--ymin ROW` skips a cloud band above the ridge; `--ymax ROW` skips foreground (trees, roofs, a car) that rises into the search.\n- `--x0/--x1` keeps only the clean stretch. A shorter correct ridge beats a longer wrong one.\n- `--drop/--k/--hold` loosen or tighten the brightness test (lower `--drop` for haze).\n\n## 2. Drop the bad points\n\nWhen most points are right and a few columns follow cloud, delete those points from `ridge.json` and pass the file back with `--points ridge.json` (it reads the `ridge` list). Leave the gap; don't move points to where you guess the ridge is.\n\n## 3. Learned detector (optional, needs the hardware below)\n\n[Crestline](https://github.com/ExtReMLapin/crestline) by @ExtReMLapin is a U-Net (ConvNeXt-Base) that segments sky, skyline and inner ridges. On clear skies it matches the default detector to 1–2 px; on snow ranges with cloud it cut the error 2–4× in our tests, and the true camera position ranked as well as or better than with a hand-traced ridge. Weights: [`LPN64/crestline-convnext-base`](https://huggingface.co/LPN64/crestline-convnext-base), CC BY-SA 4.0, downloaded at run time (185 MB, pinned revision below), not shipped with this skill. Trained on Swiss Alps photos only.\n\n**Hardware.** No GPU is required. Long side 1024 px, fp32:\n\n| Device | Peak memory | Time per photo | Source |\n|---|---|---|---|\n| Apple Silicon, CPU | 3.2–3.9 GB RAM | 6–10 s | measured |\n| Apple Silicon, MPS | ~1.3 GB process + ~1.4 GB GPU (shared memory) | \u003c 1 s | measured |\n| NVIDIA GPU | ~0.85 GB VRAM | | reported by the author |\n| x86 Linux CPU | under 2.1 GB (2.1 GB at 1536 px) | | reported by the author |\n\n- A machine with 8 GB RAM runs it on CPU. With less free memory, add `--bf16`: same output, ~1.7 GB on Apple CPU, but 3–6× slower. Don't use fp16.\n- Keep the long side at 1024: the model was trained at ~1024 px wide; 1536 px needs 4–7 GB on macOS CPU and is no more accurate.\n- First run also installs torch (already present if you have run `match.py`).\n\nSave this as `crestline_skyline.py` in your working directory and run `uv run crestline_skyline.py photo.jpg crest.json [--step 20] [--device auto|cpu|mps|cuda] [--bf16]`:\n\n```python\n# /// script\n# requires-python = \">=3.10\"\n# dependencies = [\"torch\", \"segmentation-models-pytorch>=0.5\", \"timm\", \"safetensors\", \"huggingface_hub\", \"pillow\", \"numpy\"]\n# ///\n\"\"\"Crestline skyline -> [[x,y],...] in original pixels, for terrain.py ridge --points.\"\"\"\nimport argparse, json\nimport numpy as np, torch, torch.nn.functional as F\nfrom PIL import Image, ImageOps\nimport segmentation_models_pytorch as smp\nfrom safetensors.torch import load_file\nfrom huggingface_hub import snapshot_download\n\nREPO, REV = \"LPN64/crestline-convnext-base\", \"3e12157a349088dfc8824ff3ddab65ea265fa2f1\"\nap = argparse.ArgumentParser()\nap.add_argument(\"photo\"); ap.add_argument(\"out\")\nap.add_argument(\"--step\", type=int, default=20)\nap.add_argument(\"--device\", default=\"auto\")\nap.add_argument(\"--bf16\", action=\"store_true\")\na = ap.parse_args()\ndev = a.device if a.device != \"auto\" else \"cuda\" if torch.cuda.is_available() else \"mps\" if torch.backends.mps.is_available() else \"cpu\"\ndt = torch.bfloat16 if a.bf16 else torch.float32\nwdir = snapshot_download(REPO, revision=REV, allow_patterns=[\"config.json\", \"model.safetensors\"])\ncfg = json.load(open(f\"{wdir}/config.json\"))\nmodel = smp.Unet(cfg[\"encoder\"], encoder_weights=None, classes=3)\nmodel.load_state_dict({k: v.float() for k, v in load_file(f\"{wdir}/model.safetensors\").items()})\nmodel = model.to(dev, dt).eval()\n\nimg = ImageOps.exif_transpose(Image.open(a.photo)).convert(\"RGB\"); W0, H0 = img.size\nimg.thumbnail((1024, 1024), Image.LANCZOS) # downscale only\nx = ((np.asarray(img, np.float32) / 255 - [0.485, 0.456, 0.406]) / [0.229, 0.224, 0.225]).astype(np.float32)\nx = torch.from_numpy(x).permute(2, 0, 1)[None].to(dev)\nh, w = x.shape[-2:]\nx = F.pad(x, (0, -w % 32, 0, -h % 32), mode=\"reflect\").to(dt)\nwith torch.no_grad():\n prob = torch.sigmoid(model(x).float())[0, :, :h, :w].cpu().numpy() # sky, skyline, inner ridges\n\nrows = np.argmin(np.vstack([prob[0] > 0.5, np.zeros((1, w), bool)]), axis=0) # first non-sky row\nr, cols = np.clip(rows, 0, h - 1), np.arange(w)\nconf = np.stack([prob[1][np.clip(r + d, 0, h - 1), cols] for d in range(-3, 4)]).max(0)\nconf = np.convolve(np.pad(conf, 7, mode=\"edge\"), np.ones(15) / 15, \"valid\") # edge pad keeps border columns\nok = (rows > 0) & (rows \u003c h) & (conf >= 0.6)\npts, dropped = [], []\nfor X in range(0, W0, a.step):\n u = min(int((X + 0.5) * w / W0), w - 1)\n (pts.append([X, round((rows[u] + 0.5) * H0 / h - 0.5)]) if ok[u] else dropped.append(X))\njson.dump(pts, open(a.out, \"w\"))\nprint(f\"{len(pts)} points; {len(dropped)} low-confidence columns dropped: {dropped}\")\n```\n\nThen `terrain.py ridge photo.jpg --points crest.json --png check.png [--hrow ROW ...] --out ridge.json` as usual.\n\n**Known failure: cloud on a peak.** The model then traces the top of the cloud (15–36 px too high in our tests), and the confidence check drops only a few of those columns. Check `--png` at every peak with cloud and remove those points (step 2).\n\n## 4. Hand-pick points (last resort)\n\n`--points \"x,y;x,y;...\"` on the stretches where the real ridge is visible. Skip columns hidden by cloud or haze: in one real case, tracing through cloud put the points on the cloud edge and the residual never came down. For a near range in front of a far one, also use `--near` (`geometry.md` 7.4).\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":5991,"content_sha256":"43d6245497c53d4066215e3ac735180720da275c72051d402d20d6ba7ede3901"},{"filename":"references/verify.md","content":"# Confirmation and falsification\n\nUsed in steps 6–8. In the videos, the creators' most common problem isn't failing to find the place but **confirming too loosely**: declaring after checking only two items (v007), claiming a \"two-meter error\" from one alignment line (v006), dropping a pin and calling it found (v005).\nAI's problem is worse: saying it \"measured on the map\" when it didn't (v010). The rules in this file are mandatory.\n\n## 1. Write falsification conditions before looking at candidates\n\nBefore opening satellite imagery or street view, first write 2–3 \"if this appears, give up\" conditions, for example:\n- \"The road should be on the left of the building; if it's on the right, exclude\"\n- \"The distant mountain should be on the far bank of the river; if it's on the same side, exclude\"\n- \"Sunset has to be visible from here, so the camera heading must lean west\"\n\nIf it appears, give up; don't look for reasons to explain it away. Confirmation bias is the number-one failure mode of this craft: you find a place that \"looks right\" and start making excuses for the differences.\n\n**Falsification conditions themselves must also be verified**, otherwise they backfire and kill the correct answer:\n- \"From here, the landmark must be in the frame\" → first compute it with `geo.py frame` (field-of-view range, occlusion, pixel size); see `geometry.md` section 11.\n- Details inferred from a distant view, like \"the line should turn at the tower\" or \"the building should have 8 floors\" → if not verified, only rank; don't exclude.\n- Record the candidates excluded by falsification conditions; when no candidate matches, first go back and recheck these conditions.\n\n## 2. What to compare, what not to\n\n| Compare (invariant features) | Don't compare (things that change) |\n|---|---|\n| Building outline, number of floors, window positions and spacing, balcony shape | Parked cars, pedestrians |\n| Positions and number of utility poles and streetlights | Signs, ads, awnings (often replaced) |\n| Curbs, wall corners, steps, manhole cover positions | How much foliage, whether flowers are in bloom |\n| Ridgelines, riverbank lines, bridge positions | Construction hoarding, temporary structures |\n\n- Only **independent** items count: the same sign photographed three times is one item, not three.\n- To road level: ≥2 unique features; to building level: ≥3, plus two independent camera-position constraints.\n- Common features like \"factory buildings all faced with yellow tiles\" don't count as unique features.\n\n## 3. Bearing self-check (required before output)\n\nDraw a top-down sketch on satellite imagery: camera position → foreground object → midground object → distant landmark, plus the direction of the sun or shadows. Check each item:\n1. **Left-right order**: if A is left of B in the photo, it must also be so as seen from the candidate camera position.\n2. **Near-far relationship**: if A blocks B in the photo, the candidate camera position, A and B must be roughly collinear, with A nearer.\n3. **Heading and sun**: for a sunset photo the camera must face west; if, from the candidate point, getting the image's elements into one frame requires facing southeast, it can't be a sunset (v010-7: the point the AI chose could only shoot sunrise; off by 850 km).\n4. **Sight line ahead along the road**: a tower straight ahead in a photo taken from inside a car must actually be in that road's direction of travel (v010-4 corrected a misidentified building with this one check).\n5. **Compute the bearing first, then identify the structure**: which block on the satellite image corresponds to the tower, chimney or building corner in the photo is an unverified interpretation. When the sun is in the frame, use `sun.py compass --x \u003cstructure pixel>` to compute its true bearing, and search the satellite image along that bearing from the candidate camera position; don't first decide on the satellite image that \"this is the tower\" and then reason about left and right, and don't judge which block is tall from how the OSM building outlines are cut — for tall structures, look at shadow length and side walls on the satellite image. A misidentification makes the bearing self-check kill the correct candidate.\n Identifying tall structures on satellite imagery: first find a tall building that is certainly vertical and see which of its walls is visible — the satellite is on that side, and the tops of all tall objects lean the opposite way; only blocks with a visible side wall are tall, and the foot of a tower is at the bottom edge of the side wall, where the shadow starts, not at the center of the roof's projection; shadow direction is set by the sun and is not the same thing as lean direction. When both identifications are plausible, draw both sets of camera positions and check each against the bearing computed by `sun.py compass --x`.\n6. **When bearings don't match, check for mirroring and sunrise first**: screenshots of reposted videos are often horizontally flipped. Without text, look at gestures (usually right-handed), clocks, the driver's side of vehicles; when the candidate facility is unique and only left-right doesn't match, build two templates each for \"original / mirrored\" and \"sunset / sunrise\" and check each one, then decide whether to downgrade or widen the search.\n\nIf any item fails, exclude, or downgrade and write down why.\n\n## 4. How to confirm without street view\n\n| Situation | Alternative |\n|---|---|\n| Mountains, outskirts, looking across a river from afar | `terrain.py view --photo` renders the candidate camera position; compare ridgeline outlines; remember the outline only proves \"on this sight line\", you still need a second constraint. With many candidates, batch-score first, then overlay (`geometry.md` 7.4) |\n| Suburban hotels, factory compounds, scenic areas | Map POI photos, real photos of the hotel/scenic area online, tourist photos; compare the type, number and spacing of the pylons on the skyline (v011) |\n| New housing developments, commercial complexes | Development photo albums on real-estate sites (v010-5) |\n| Outside China | `gsv.py near / sheet --headings / render` (Google Street View, no key); when the puzzle image is a street view screenshot, use the watermark year with `sheet --date \u003cyear>` to get the same capture; clouds and season can also be compared |\n| Rural areas, factory compounds, farms outside China, with no street view | **First find a name for the facility**: OSM name, nearby place name + the local-language word for the facility type (sawmill: scierie / aserradero and the like), search the web, and compare facade details in images from news, encyclopedias, company websites and blogs (v013's creator confirmed it this way); then try Wikimedia Commons image search by coordinates (no key, see `data-sources.md`); Mapillary needs a token, KartaView has little coverage |\n| No ground photos of any kind | ≥3 top-down features uniquely matched on satellite imagery + bearing self-check passed: report area level as high, road as medium, building as low, and state that ground-level imagery is missing; don't fall back to district level overall just because there's no street view |\n\n### Finding a storefront along a road\n\nWhen you know the road name (address from a news article, the street given by reverse image search) but not the street number:\n\n```bash\nuv run scripts/osm.py along --bbox \u003cs,w,n,e> --line '[\"highway\"][\"name\"~\"\u003croad name fragment>\"]' --step 120 --out pts.json\nuv run scripts/gsv.py sheet --points pts.json --headings 30,90 --out road.jpg # in China use baidu_pano.py instead\n```\n\n- Spacing ≤150 m: a storefront is only visible within a very narrow angle; at 400 m spacing you can easily miss every single one.\n- Set headings from \"direction of travel + which side the shop is on\": in a right-hand-traffic country, shooting a shop on the right from inside a car while heading north, use 20–90°. To reproduce an upward shot of a storefront from inside a car, add `--pitch 15–20`.\n- To find roadside factory buildings and fields, use satellite thumbnails: `--step 300–500 --side \u003cnorth|south|east|west> --offset \u003cmeters from the road to the building>`, then `tiles.py sheet --zoom 16–17`.\n\n### Picking street view points\n\n- When there are many panorama points, group them by capture date and road, and render the ones with an open view first: waterside paths, bridges, intersections, older captures not blocked by street trees. Newer captures on main roads are often completely blocked by trees (the truly usable ones are often older-capture points on waterside paths and construction access roads).\n- Look around from one point: `baidu_pano.py sheet --ids \u003cid> --headings 0,60,120,180,240,300` (outside China use `gsv.py`).\n- For farmland and countryside in China, first probe one or two points with `baidu_pano.py near \u003clat,lon>` to see whether there's a panorama, then decide whether to `scan`: in a real case a 1.5 km-radius `scan` over farmland ran for 5 minutes and found none.\n\n## 5. Overlay comparison\n\nWhen the two images have different perspectives, crop the sign, utility pole or window from the original, rotate and scale it, and paste it onto the corresponding position in the street view screenshot; whether it's the same object is visible at a glance (v002). Same for water and mountain outlines: rotate and stretch, then overlay onto satellite imagery and compare point by point (v012).\n\n```bash\nuv run scripts/imgprep.py zoom photo.jpg --box ... --scale 2 --out part.png\nuv run scripts/baidu_pano.py render \u003cpanoid> --heading 47 --out sv.jpg\nuv run scripts/evidence.py spec.json --out evidence.jpg # satellite image + camera-position wedge + comparison panels\n```\n\n## 6. Historical imagery\n\n- Street view taken many years before the photo: go by old buildings and permanent structures; new buildings may not exist yet, and signs may have changed (v003: the street view was 10 years older; the comparison used buildings and poles).\n- New facilities (new schools, new factory buildings) aren't in old street view yet: first group by date and look at the newest capture; if it shows demolition, hoarding or tower cranes, that's ground evidence that \"something was built here later\", and it also gives the photo an earliest possible year.\n- Conversely, you can date the photo: find slowly changing things (extent of climbing-plant cover, tree crown size, hoarding, whether a new building has gone up), compare them across street view or satellite imagery from different years, and you can bracket it to within a year or two (v004).\n- Judge the season from the phenology in the photo, not from the month the reference street view was taken (v004's creator called it \"winter\" from the date of a December street view).\n\n## 7. When there are many candidate cities or candidate points\n\n- Make a candidate table: one row per candidate, one column per criterion, fill in \"matches / doesn't match / can't tell\", and in the last column write the reason for exclusion.\n- Look at candidate points one page at a time: `tiles.py sheet --points cands.json --zoom 18`; each cell is centered and numbered, more accurate than finding points in a large mosaic (reading pixels off a large mosaic displayed scaled down is off by 50–100 px).\n- Criteria use only things directly visible within the photo's field of view.\n- Go through all of them once before looking closely at what remains; don't stop at the first one that \"looks like it\".\n- When candidates come from OSM, first check each candidate area's data volume with `osm.py coverage`; an area with sparse data being absent from the candidate table doesn't mean it isn't the one — scan it separately with `tiles.py sheet --grid`.\n\n### Finding a sports field (common in top-down shots from windows)\n\n1. Read the sports-field template from the photo: the angle between the track's long axis and the camera direction (lying across the frame ≈ perpendicular to the camera direction), which side of the track the pitch and basketball courts are on, the shape of nearby buildings (L-shaped, enclosed square), adjacent open ground or slopes.\n2. The camera direction comes from the sun (`sun.py compass`; without a date, compute over the whole year) or from roads or rivers in the frame; if two headings can't be separated, use two templates.\n3. First look at the whole urban area in one z16 mosaic; bright new sports fields stand out at a glance; convert pixels to coordinates with `tiles.py px2ll`. When you need to find all fields, new and old, lay a grid over the candidate district's urban area: `tiles.py sheet --grid \u003curban area s,w,n,e> --zoom 17 --size 320 --cols 5` (one cell ≈ 330 m, 25 cells per page ≈ 1.5 km × 1.5 km), and look for red running tracks cell by cell (a track covered by building shadows shows only a section of arc).\n4. Compare matched fields to the template, then work back from the field toward the camera to find the building the camera is in (which side of the teaching building is visible, which road lies between, how large the depression angle is).\n5. Don't assume \"track lying across the frame = long axis perpendicular to the sight line\": see which end of the track's far edge is lower (closer); if it's slanted, it isn't perpendicular. After identifying ≥4 points (track curves, basketball court corners, building corners), solve heading and camera position with `pose.py solve`; don't compute by hand.\n\n## 8. Final check before concluding\n\n- [ ] Every \"matched\" and \"measured\" has a file actually produced in this session\n- [ ] The error radius comes from intersection geometry or multi-item comparison, not made-up decimal places\n- [ ] All four bearing self-check items passed\n- [ ] After finding an anchor, you didn't go back to generic features to pick another place\n- [ ] Category inference listed all candidates, with exclusion reasons written\n- [ ] Conflicts between hints, IP, EXIF and the conclusion are explained (the image wins)\n- [ ] The key step was redone with a different crop or keywords, with consistent results\n- [ ] If you couldn't pin a point, you wrote \"the level it's determined to + what information is still missing\"\n- [ ] Every exclusion reason was verified: exclusions by \"X isn't in the image\" had the frame and occlusion computed; unverified interpretations were used only for ranking\n- [ ] When the candidates are a few discrete ones, you chose a main answer, not a midpoint with a big circle\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":14465,"content_sha256":"281c827f4997389488cb45b0ec075800aabcc4a2220818ef2b202d7c29840105"},{"filename":"regions/_template/region.json","content":"{\n \"code\": \"XX\",\n \"name\": \"Country name in English\",\n \"names\": [\"Country name in English\", \"Local name\", \"Common alias\"],\n \"status\": \"community\",\n \"maintainers\": [\"@your-github-handle\"],\n \"docs\": [\"clues.md\"]\n}\n","content_type":"application/json; charset=utf-8","language":"json","size":211,"content_sha256":"711335d3e2ad26c0162a3abadf22c8b7e67c4fe2a18769a7ec47a398d63cf2e3"},{"filename":"regions/README.md","content":"# Region packs\n\nEverything that only works inside one country lives here, one folder per country: `regions/\u003ccc>/`, where `cc` is the ISO 3166-1 alpha-2 code in lower case (`cn`, `jp`, `in`). The core (SKILL.md, `references/`, `scripts/`) holds the methods and the cross-country clues, and reads packs through their manifest. Adding or removing a pack never changes the core.\n\n## How the agent uses a pack\n\n1. Country still open: cross-country clues (`references/clues/global.md`), calling codes, driving side. Lookups without `--country` search every pack and return matches from all countries that fit.\n2. A country is a candidate: `regions.py show \u003ccc>` prints the pack's card (lookups, preferred services, tips) and a one-line index of its clue entries with line numbers. Read only the entries you need.\n3. Country settled: `board.py country \u003ccc>`; from then on `board.py apply` uses only that pack, and `board.py next` uses its tips.\n4. No pack for the country: global tools and references only. Lookups for it answer `unsupported`; they never fall back to another country's table.\n\n## What a pack contains\n\n```\nregions/\u003ccc>/\n region.json manifest (required)\n clues.md clue entries (required; more docs can be listed in region.json \"docs\")\n data/*.json lookup tables and an admin table (optional)\n tests.json lookup cases (required when the pack has lookups)\n```\n\n**Packs hold data and documents only: `.json` and `.md`.** The skill runs on users' machines, and a pack is reviewed as data. If a table needs logic the formats below can't express, propose a change to the core format in a separate PR.\n\n### region.json\n\n```json\n{\n \"code\": \"JP\",\n \"name\": \"Japan\",\n \"names\": [\"Japan\", \"日本\", \"Nippon\"],\n \"status\": \"community\",\n \"maintainers\": [\"@your-github-handle\"],\n \"docs\": [\"clues.md\"],\n \"admin_levels\": {\"4\": \"admin1\", \"7\": \"admin2\"},\n \"level_names\": {\"admin1\": \"prefecture\", \"admin2\": \"municipality\"},\n \"name_suffixes\": [],\n \"admin_table\": \"data/admin.json\",\n \"lookups\": {\"area-code\": {\"file\": \"data/area_codes.json\"}},\n \"services\": {\"poi\": {\"sources\": \"osm\", \"country\": \"jp\"}, \"street_view\": \"gsv.py\"},\n \"tips\": {\"livery\": \"…\", \"municipal\": \"…\"}\n}\n```\n\n| Field | Meaning |\n|---|---|\n| `code`, `name`, `names` | ISO code, display name, every name or alias a user or table might use (`--country` matches any of them) |\n| `status` | `maintained` (core maintainers keep it current), `community` (a named maintainer does), `stub` (a few clues, no maintainer) |\n| `docs` | Markdown files the card indexes |\n| `admin_levels` | admin table `level` → board level: `admin1`, `admin2`, `city`, `district` |\n| `level_names` | what the levels are called locally, for the card |\n| `name_suffixes` | generic suffixes ignored when matching names across sources (CN: 市, 省, 区, 县) |\n| `admin_table` | optional; lets `board.py children` check OSM's list and gives candidates stable ids |\n| `also_children` | optional; OSM relation names `board.py children \u003ccountry>` lists next to the first-level divisions when OSM's admin tree leaves them out or puts them at another level (CN: Hong Kong, Macau, Taiwan) |\n| `osm_name_keys` | optional; extra OSM `name:*` tags that place-name lookups also match (CN: `name:zh`, `name:zh-Hans`, because OSM's `name` adds a second script for some regions and simplified forms only exist there). Core scripts match `name` plus every installed pack's keys |\n| `lookups` | lookup kind → prefix table. Use an existing kind name when it means the same thing: `plate`, `area-code`, `postal-code` |\n| `services` | `poi` (sources and Nominatim country for `poi.py --region`; optional `keyword_scripts`, Unicode character-name prefixes such as `CJK UNIFIED IDEOGRAPH`, makes its sources the default for keywords in that script when no `--region` is given), `street_view` (a core script), `datum` (local map datum, if any) |\n| `tips` | replaces the matching line of `board.py next`'s cheap tests (`livery`, `municipal`, …) |\n\n### Prefix tables (`data/*.json` named in `lookups`)\n\n```json\n{\n \"_meta\": {\n \"kind\": \"area-code\",\n \"source\": [\"https://…\"],\n \"fetched\": \"2026-10-08\",\n \"license\": \"CC BY-SA 4.0 (derived from Wikipedia)\",\n \"count\": 2,\n \"normalize\": {\"digits_only\": true, \"ensure_prefix\": \"0\"},\n \"key_pattern\": null,\n \"usage\": \"one line shown on the card: what to type\",\n \"miss_note\": \"what a miss usually means\"\n },\n \"entries\": {\n \"020\": [{\"admin1\": \"Maharashtra\", \"admin2\": \"Pune\"}],\n \"0120\": [{\"admin1\": \"Uttar Pradesh\", \"admin2\": \"Ghaziabad\"}, {\"admin1\": \"Uttar Pradesh\", \"admin2\": \"Gautam Buddh Nagar\", \"note\": \"Noida\"}]\n }\n}\n```\n\n- Lookup = normalize the value (`normalize`: `strip` characters, `digits_only`, `upper`, `ensure_prefix`), cut it with `key_pattern` if given (a value that doesn't fit is \"not this country's format\"), then take the **longest key that is a prefix** of it.\n- Each key maps to a **list**. A key that covers several places lists every one of them; never merge two places into one name, and never pick one. The board keeps them all as hypotheses.\n- A match names admin levels explicitly (`admin1` … `district`) with the names as the admin table or OSM spells them, plus optional `note` and `unverified: true` for rows not backed by the source.\n- A key that covers only part of an area (part of a state, part of a district) names the larger area and says what part in `note`.\n- `_meta.source`, `fetched` and `license` are required. Hand-compiled data says so in `source` and gives the references it was compiled from.\n\n### Admin table (`admin_table`)\n\n`{\"_meta\": {source, fetched, license, count}, \"items\": [{\"name\", \"code\", \"level\", \"parent\"}]}` — `parent` is the parent's `name`; top-level items have `\"parent\": \"\"`. An item may add `\"aliases\": [...]` (a list of non-empty strings) for other spellings `clues.py lookup admin` should accept; `name` stays the one lookup tables and the board use. Every `level` needs an entry in `admin_levels`.\n\n### clues.md\n\nEntries in the shared format (`references/clues/README.md`): Look for / Points to / Strength / Counterexamples / Sources, grouped under `##` topics. An entry belongs here when its \"Points to\" stays inside this country; clues that separate countries go into `references/clues/global.md`.\n\n### tests.json\n\n```json\n[{\"kind\": \"area-code\", \"value\": \"0120-4567890\", \"expect\": [{\"admin1\": \"Uttar Pradesh\", \"admin2\": \"Ghaziabad\"}, {\"admin1\": \"Uttar Pradesh\", \"admin2\": \"Gautam Buddh Nagar\"}]}]\n```\n\nAt least one case per lookup kind; include a key that covers several places and a value that should miss (`\"expect\": []`). Only the level fields are compared.\n\n## Contributing a pack\n\n| Tier | What | Review |\n|---|---|---|\n| 1 | Clue entries only (`region.json` + `clues.md`) | Entry format, sources, counterexamples |\n| 2 | Lookup tables (+ `tests.json`, optionally an admin table) | Tier 1 + data sources, licenses, multi-place keys, tests |\n| 3 | Anything that needs code: a new service adapter, OCR language support, a new table format | Open an issue first; core change in its own PR |\n\nA pack PR touches only `regions/\u003ccc>/`. Start from `regions/_template/`, then run:\n\n```bash\nuv run scripts/regions.py lint \u003ccc> # manifest, sources, entry fields, no code files, tests\nuv run scripts/regions.py show \u003ccc> # what the agent will see\nuv run scripts/clues.py lookup \u003ckind> \u003cvalue> --country \u003ccc>\n```\n\n`lint` must print `ok`. It rejects code files, tables without source/date/license, keys that map to a merged name like \"A / B\" instead of a list, entries missing a field, and failing tests.\n","content_type":"text/markdown; charset=utf-8","language":"markdown","size":7557,"content_sha256":"05b43eb9548940608d84576f1c422e51b40cfe787a568b6a12541a63e19ab16a"},{"filename":"scripts/_browser.py","content":"\"\"\"Shared Chrome / Playwright Chromium launch and actionable setup errors.\"\"\"\nfrom __future__ import annotations\n\nfrom _net import resolve_proxy\n\n\nasync def launch_browser(playwright, proxy: str | None = None):\n value = resolve_proxy(proxy)\n options = {\"headless\": True, \"timeout\": 15000,\n \"args\": [\"--disable-blink-features=AutomationControlled\"]}\n if value:\n options[\"proxy\"] = {\"server\": value.replace(\"socks5h://\", \"socks5://\")}\n else:\n options[\"args\"].append(\"--no-proxy-server\")\n errors = []\n for channel in (\"chrome\", None):\n try:\n browser = await playwright.chromium.launch(\n **({\"channel\": channel} if channel else {}), **options)\n return browser, \"Google Chrome\" if channel else \"Playwright Chromium\"\n except Exception as exc:\n errors.append(f\"{channel or 'chromium'}: {str(exc).splitlines()[0]}\")\n raise RuntimeError(\n \"Neither Google Chrome nor Playwright Chromium could start. \"\n \"Run `uvx playwright install chromium` and retry; on Linux, missing system libraries \"\n \"may require `uvx playwright install --with-deps chromium`. \"\n + \" | \".join(errors))\n","content_type":"text/x-python; charset=utf-8","language":"python","size":1205,"content_sha256":"542c7fd5e3d4ddbcfd2a724c934fb7d873788f5680ac6070ad0e46a251581afe"},{"filename":"scripts/_device.py","content":"\"\"\"Shared torch device selection: CUDA, then MPS, then CPU; GEO_DEVICE overrides.\"\"\"\nfrom __future__ import annotations\n\nimport os\nimport sys\n\nDEVICES = (\"cpu\", \"cuda\", \"mps\")\n\n\ndef _available(torch) -> dict:\n return {\"cpu\": True, \"cuda\": bool(torch.cuda.is_available()), \"mps\": bool(torch.backends.mps.is_available())}\n\n\ndef torch_device(torch=None) -> str:\n \"\"\"Device for the ML models. GEO_DEVICE=cpu|cuda|mps forces one; an invalid or unavailable value exits with one line.\"\"\"\n if torch is None:\n import torch\n have = _available(torch)\n want = os.environ.get(\"GEO_DEVICE\", \"\").strip().lower()\n if not want:\n return \"cuda\" if have[\"cuda\"] else \"mps\" if have[\"mps\"] else \"cpu\"\n report = f\"torch {torch.__version__} reports cuda={have['cuda']}, mps={have['mps']}\"\n if want not in DEVICES:\n sys.exit(f\"GEO_DEVICE={want!r} is not valid: use one of {', '.join(DEVICES)}, or unset it for automatic selection ({report}).\")\n if not have[want]:\n sys.exit(f\"GEO_DEVICE={want} was requested but that device is not available: {report}. Unset GEO_DEVICE or install a torch build that supports it.\")\n return want\n","content_type":"text/x-python; charset=utf-8","language":"python","size":1160,"content_sha256":"0fb06ae8b589abcea09cddeac9568aaedc43ac41222e5ca0f1d303b790b890a3"},{"filename":"scripts/_image.py","content":"\"\"\"Shared reading of user photos and candidate images: an explicit set of raster formats, content sniffing\ninstead of trusting the suffix, RAW rejected with a request to export, and one failure category per file.\n\nScripts that read a photo or candidate images go through `open_image`; scripts that only read their own\noutputs (map tiles, panoramas) keep plain `Image.open`. Callers that import this declare `pillow-heif` in their\nPEP 723 header.\n\nPixel coordinates (crop boxes, OCR boxes, ridge points, pose points) refer to the upright frame, the photo as an\nimage viewer shows it: `open_photo` / `upright` apply the EXIF orientation exactly once. Records a script writes\nwith pixel coordinates in them (ridge.json, a pose record's `_frame`) say so with \"frame\": \"upright\". A record\nwithout the key was written before orientation was applied, when those scripts drew on the stored pixels; it is\nread in that \"stored\" frame whatever the image size (`frame_of`), so old records give the overlays they gave then.\n\"\"\"\nfrom __future__ import annotations\n\nimport sys\nfrom pathlib import Path\n\nfrom PIL import Image, ImageOps, features\n\ntry:\n import pillow_heif\n pillow_heif.register_heif_opener()\n HEIF = True\nexcept ImportError: # without pillow-heif, HEIC/HEIF is reported as a missing decoder; other formats work as usual\n HEIF = False\n\n# Pillow format names accepted as a photo or candidate; anything else Pillow can open is \"unsupported\"\nSUPPORTED = {\"JPEG\", \"MPO\", \"PNG\", \"WEBP\", \"GIF\", \"BMP\", \"TIFF\", \"HEIF\", \"AVIF\"}\n# Directory scans: these suffixes (any case) after the historical *.jp*g then *.png order; other files are sniffed\nSCAN_SUFFIXES = (\".jpg\", \".jpeg\", \".png\", \".webp\", \".heic\", \".heif\", \".avif\")\nRAW_SUFFIXES = {\".3fr\", \".arw\", \".cr2\", \".cr3\", \".crw\", \".dcr\", \".dng\", \".erf\", \".iiq\", \".k25\", \".kdc\", \".mef\",\n \".mos\", \".mrw\", \".nef\", \".nrw\", \".orf\", \".pef\", \".raf\", \".raw\", \".rw2\", \".rwl\", \".sr2\", \".srf\",\n \".srw\", \".x3f\"}\nHEIF_BRANDS = {b\"heic\", b\"heix\", b\"heim\", b\"heis\", b\"hevc\", b\"hevx\", b\"heif\", b\"mif1\", b\"msf1\"}\nAVIF_BRANDS = {b\"avif\", b\"avis\"}\nHEAD = 32 # bytes read to sniff a file\nORIENTATION = 0x0112\n\n\nclass ImageReadError(Exception):\n \"\"\"category: unreadable | empty | raw | not-image | unsupported | missing-decoder | decode-error\"\"\"\n\n def __init__(self, category: str, reason: str):\n super().__init__(reason)\n self.category = category\n self.reason = reason\n\n\ndef sniff(head: bytes, suffix: str = \"\") -> str | None:\n \"\"\"Format from the first bytes: JPEG, PNG, WEBP, GIF, BMP, TIFF, HEIF, AVIF, RAW or TEXT; None if unknown.\"\"\"\n if head.startswith(b\"\\xff\\xd8\\xff\"):\n return \"JPEG\"\n if head.startswith(b\"\\x89PNG\\r\\n\\x1a\\n\"):\n return \"PNG\"\n if head[:4] == b\"RIFF\" and head[8:12] == b\"WEBP\":\n return \"WEBP\"\n if head[:6] in (b\"GIF87a\", b\"GIF89a\"):\n return \"GIF\"\n if head[:2] == b\"BM\" and len(head) >= 14:\n return \"BMP\"\n if head[4:8] == b\"ftyp\":\n brand = head[8:12]\n if brand in HEIF_BRANDS:\n return \"HEIF\"\n if brand in AVIF_BRANDS:\n return \"AVIF\"\n if brand == b\"crx \":\n return \"RAW\" # Canon CR3\n if head.startswith((b\"FUJIFILMCCD-RAW\", b\"IIRO\", b\"IIRS\", b\"MMOR\", b\"IIU\\x00\", b\"FOVb\")):\n return \"RAW\" # Fujifilm RAF, Olympus ORF, Panasonic RW2, Sigma X3F\n if head[:4] in (b\"II*\\x00\", b\"MM\\x00*\"):\n if head[8:10] == b\"CR\" or suffix.lower() in RAW_SUFFIXES:\n return \"RAW\" # Canon CR2; DNG, NEF, ARW, PEF ... are TIFF inside\n return \"TIFF\"\n if head.lstrip()[:1] in (b\"\u003c\", b\"{\", b\"[\"):\n return \"TEXT\"\n return None\n\n\ndef _head(path: Path) -> bytes:\n try:\n with open(path, \"rb\") as fh:\n return fh.read(HEAD)\n except OSError as e:\n raise ImageReadError(\"unreadable\", f\"cannot read the file ({e.strerror or type(e).__name__})\") from None\n\n\ndef _tiff_is_raw(im: Image.Image) -> bool:\n \"\"\"A TIFF that is really a camera RAW: a DNG version tag, raw photometric data, or a first image marked as a\n reduced-resolution preview with the real image in SubIFDs (Pillow would only ever read the preview).\"\"\"\n tags = getattr(im, \"tag_v2\", {})\n preview = int(tags.get(254, 0) or 0) & 1\n return 50706 in tags or tags.get(262) in (32803, 34892) or bool(preview and 330 in tags)\n\n\ndef _first_line(e: Exception) -> str:\n text = str(e).strip().splitlines()\n return f\"{type(e).__name__}: {text[0]}\" if text else type(e).__name__\n\n\ndef open_image(path: Path | str, load: bool = True) -> Image.Image:\n \"\"\"Open a photo or candidate image. Raises ImageReadError with a category and a one-line reason.\n load=True decodes the pixels now, so a damaged file fails here instead of in the middle of a run.\"\"\"\n path = Path(path)\n head = _head(path)\n if not head:\n raise ImageReadError(\"empty\", \"empty file\")\n kind = sniff(head, path.suffix)\n if kind == \"RAW\" or (kind is None and path.suffix.lower() in RAW_SUFFIXES):\n raise ImageReadError(\"raw\", f\"camera RAW ({path.suffix or 'no suffix'}), which is not decoded here: export it as JPEG or PNG first\")\n if kind == \"TEXT\":\n raise ImageReadError(\"not-image\", \"text, not an image (probably an error page saved by a download)\")\n if kind == \"HEIF\" and not HEIF:\n raise ImageReadError(\"missing-decoder\", \"HEIC/HEIF, which needs pillow-heif (run the script with uv run, or convert to JPEG/PNG)\")\n if kind == \"AVIF\" and not features.check(\"avif\"):\n raise ImageReadError(\"missing-decoder\", \"AVIF, which this Pillow build cannot decode (upgrade Pillow, or convert to JPEG/PNG)\")\n try:\n im = Image.open(path)\n if im.format == \"TIFF\" and _tiff_is_raw(im): # checked from the tags, before any pixels are decoded\n im.close()\n raise ImageReadError(\"raw\", f\"camera RAW ({path.suffix or 'no suffix'}, TIFF-based: only its embedded preview would be read), \"\n \"which is not decoded here: export it as JPEG or PNG first\")\n if load:\n im.load()\n except ImageReadError:\n raise\n except Exception as e: # noqa: BLE001\n if kind is None:\n raise ImageReadError(\"not-image\", f\"not a recognised image ({_first_line(e)})\") from None\n raise ImageReadError(\"decode-error\", f\"{kind} that could not be decoded ({_first_line(e)})\") from None\n if im.format not in SUPPORTED:\n fmt = im.format\n im.close()\n raise ImageReadError(\"unsupported\", f\"{fmt} is not a supported photo format (JPEG, PNG, WebP, HEIC/HEIF, AVIF, GIF, BMP, TIFF): convert it first\")\n return im\n\n\ndef open_or_exit(path: Path | str, what: str = \"photo\", load: bool = True) -> Image.Image:\n \"\"\"open_image for a single required input: exit with one line instead of a traceback.\"\"\"\n try:\n return open_image(path, load=load)\n except ImageReadError as e:\n sys.exit(f\"cannot read {what} {path}: {e.reason}\")\n\n\ndef find_images(directory: Path | str) -> list[Path]:\n \"\"\"Image files in a folder, not recursive. Keeps the historical order (*.jp*g, then *.png, each sorted), then\n adds other image suffixes in any case (WebP, HEIC/HEIF, AVIF, uppercase names), then files whose first bytes\n show an image or a camera RAW despite a missing or misleading suffix (RAW is kept so it can be reported).\"\"\"\n d = Path(directory)\n files = sorted(d.glob(\"*.jp*g\")) + sorted(d.glob(\"*.png\"))\n seen = {f.resolve() for f in files}\n rest = sorted(f for f in d.iterdir() if f.is_file() and f.resolve() not in seen)\n by_suffix = [f for f in rest if f.suffix.lower() in SCAN_SUFFIXES or f.suffix.lower() in RAW_SUFFIXES]\n sniffed = []\n for f in rest:\n if f in by_suffix:\n continue\n try:\n with open(f, \"rb\") as fh:\n kind = sniff(fh.read(HEAD), f.suffix)\n except OSError:\n continue\n if kind not in (None, \"TEXT\"):\n sniffed.append(f)\n return files + by_suffix + sniffed\n\n\ndef orientation(im: Image.Image) -> int:\n \"\"\"EXIF orientation (1–8) still to apply to the decoded pixels. HEIF is always 1: libheif applies the file's\n irot/imir transforms and pillow-heif resets the EXIF tag, which the HEIF spec says to ignore.\"\"\"\n if im.format == \"HEIF\":\n return 1\n try:\n o = int(im.getexif().get(ORIENTATION, 1) or 1)\n except Exception: # noqa: BLE001\n return 1\n return o if 1 \u003c= o \u003c= 8 else 1\n\n\ndef upright(im: Image.Image) -> Image.Image:\n \"\"\"The image as a viewer shows it. Returns the same object when there is nothing to apply; otherwise\n exif_transpose, whose result carries no orientation tag, so a second call changes nothing.\"\"\"\n return im if orientation(im) == 1 else ImageOps.exif_transpose(im)\n\n\nFRAMES = (\"upright\", \"stored\")\n\n\ndef frame_of(record: dict, default: str = \"stored\") -> str:\n \"\"\"Frame of the pixel coordinates in a record: its \"frame\" value, else `default`. For records a script wrote\n (ridge.json, a pose record's _frame) the default \"stored\" is the frame every record without the key was written in.\"\"\"\n frame = record.get(\"frame\", default)\n if frame not in FRAMES:\n sys.exit(f'\"frame\" must be one of {\", \".join(FRAMES)}, not {frame!r}')\n return frame\n\n\ndef open_photo(path: Path | str, what: str = \"photo\", frame: str = \"upright\", size=None) -> Image.Image:\n \"\"\"open_or_exit, in the frame of the coordinates that will be drawn on it: \"upright\" (EXIF orientation applied) or\n \"stored\" (the decoded pixels as they are, the frame of records written before orientation was applied).\n size: the image size a record gives, checked against the photo in that frame (records may be scaled, so the\n aspect ratio is what must agree).\"\"\"\n im = open_or_exit(path, what)\n o = orientation(im)\n if frame == \"stored\":\n if o != 1:\n print(f\"note: {path} has EXIF orientation {o}; this record's coordinates are in the stored frame (a record without \"\n f\"\\\"frame\\\": \\\"upright\\\" was written before orientation was applied), so it is drawn on the stored \"\n f\"{im.width}x{im.height} pixels, unrotated\", file=sys.stderr)\n else:\n im = upright(im)\n if size is not None and size[0] > 0 and size[1] > 0 and abs(size[0] / size[1] - im.width / im.height) > 0.01 * im.width / im.height:\n print(f\"warning: the record's image size {size[0]}x{size[1]} doesn't match {path} in the {frame} frame \"\n f\"({im.width}x{im.height}); its points won't line up with the photo. If they were measured in the other frame, \"\n f\"set \\\"frame\\\": \\\"{'stored' if frame == 'upright' else 'upright'}\\\" in the record\", file=sys.stderr)\n return im\n\n\n# Formats every later step and the search engines take as they are; anything else gets a working copy\nNATIVE = {\"JPEG\", \"MPO\", \"PNG\"}\n\n\ndef working_copy_reason(im: Image.Image) -> str | None:\n \"\"\"Why a photo needs a working copy, or None: a format outside NATIVE, or an EXIF orientation still to apply.\"\"\"\n if im.format not in NATIVE:\n return f\"{'HEIC/HEIF' if im.format == 'HEIF' else im.format} decoded to PNG\"\n o = orientation(im)\n return f\"EXIF orientation {o} applied\" if o != 1 else None\n\n\ndef write_working_copy(path: Path | str, out: Path | str) -> dict | None:\n \"\"\"Lossless upright PNG of a photo that needs one (see working_copy_reason), with the original's EXIF kept\n (focal length, time, GPS) and its orientation tag reset. Returns None for photos used as they are; raises\n ImageReadError for unreadable ones. The original file is never touched.\"\"\"\n path, out = Path(path), Path(out)\n im = open_image(path)\n why = working_copy_reason(im)\n if why is None:\n return None\n up = upright(im)\n if up.mode not in (\"RGB\", \"RGBA\", \"L\", \"LA\"):\n up = up.convert(\"RGBA\" if \"A\" in up.getbands() else \"RGB\")\n exif = im.getexif()\n if ORIENTATION in exif:\n exif[ORIENTATION] = 1\n out.parent.mkdir(parents=True, exist_ok=True)\n extra = {\"icc_profile\": im.info[\"icc_profile\"]} if im.info.get(\"icc_profile\") else {}\n up.save(out, format=\"PNG\", exif=exif, **extra)\n return {\"original\": str(path), \"working\": str(out), \"reason\": why, \"format\": im.format,\n \"original_size\": list(im.size), \"working_size\": list(up.size)}\n","content_type":"text/x-python; charset=utf-8","language":"python","size":12521,"content_sha256":"0f79259df40e1dae1733f2071fcfb80f2612ddbc4bb064a9f5a96681295aba13"},{"filename":"scripts/_net.py","content":"\"\"\"HTTP helpers shared by service clients.\"\"\"\nfrom __future__ import annotations\n\nimport os\nimport subprocess\n\nPROXY_HELP = \"proxy URL or 'direct'\"\n\n\ndef resolve_proxy(proxy: str | None = None) -> str | None:\n value = (os.environ.get(\"GEO_PROXY\", \"\") if proxy is None else proxy).strip()\n return None if value.lower() in (\"\", \"direct\") else value\n\n\ndef curl_args(proxy: str | None = None) -> list[str]:\n value = resolve_proxy(proxy)\n # Clear NO_PROXY for an explicit proxy; ignore ambient proxies for direct mode.\n return [\"--proxy\", value, \"--noproxy\", \"\"] if value else [\"--noproxy\", \"*\"]\n\n\ndef fetch_bytes(url: str, proxy: str | None = None, timeout: int = 20,\n headers: dict[str, str] | None = None) -> bytes:\n cmd = [\"curl\", \"-q\", \"-fsSL\", \"--max-time\", str(timeout), *curl_args(proxy)]\n for name, value in (headers or {}).items():\n cmd += [\"-H\", f\"{name}: {value}\"]\n result = subprocess.run(cmd + [url], capture_output=True, timeout=timeout + 5)\n if result.returncode:\n raise RuntimeError(f\"Request failed (curl exit {result.returncode}); run doctor.py --network to check connectivity.\")\n return result.stdout\n\n\ndef model_proxy_env(proxy: str | None = None) -> None:\n \"\"\"Set the model downloader's process environment before importing its HTTP client.\"\"\"\n value = resolve_proxy(proxy)\n for key in (\"HTTP_PROXY\", \"HTTPS_PROXY\", \"ALL_PROXY\", \"NO_PROXY\",\n \"http_proxy\", \"https_proxy\", \"all_proxy\", \"no_proxy\"):\n os.environ.pop(key, None)\n if value:\n os.environ.update(HTTP_PROXY=value, HTTPS_PROXY=value, ALL_PROXY=value)\n else:\n os.environ[\"NO_PROXY\"] = \"*\"\n","content_type":"text/x-python; charset=utf-8","language":"python","size":1672,"content_sha256":"76bf4664e6f96eb8d0df211d72a13e223a5af58a9dfef7a56806a3eee71ae947"},{"filename":"scripts/baidu_pano.py","content":"#!/usr/bin/env python3\n# /// script\n# requires-python = \">=3.10\"\n# dependencies = [\"pillow\"]\n# ///\n\"\"\"Baidu panoramas (street view in China): find points, scan an area, render by heading, build comparison sheets.\n\nUses the public endpoints the Baidu Maps web page itself calls; no key needed.\nThe endpoints are rate-limited: when scanning an area, keep the step at 15 m or more and concurrency at 32 or less.\nBaidu street view in China was mostly captured in 2017–2019; when comparing, look at old buildings, new ones may not be there yet.\n\nExamples:\n baidu_pano.py near 22.6047,114.0523 # nearest panorama point\n baidu_pano.py info 09005700121709201230455178V # date, position, all points on this road\n baidu_pano.py scan 22.6050,114.0535 --radius 300 --out panos.json\n baidu_pano.py render 09005700121709201230455178V --heading 47 --out v.jpg\n baidu_pano.py sheet --panos panos.json --toward 22.6072,114.0564 --offset -12 --out s.jpg\n baidu_pano.py sheet --ids ID1,ID2 --heading 45 --out s.jpg\n baidu_pano.py sheet --ids ID1 --headings 0,60,120,180,240,300 --out around.jpg # look around from a single point\n baidu_pano.py sheet --panos panos.json --road \u003cname>路 --spread 60 --toward 22.6072,114.0564 --out s.jpg # one road only (路 = road), thinned\n baidu_pano.py sample --bbox 22.52,113.90,22.60,114.10 --n 24 --out cityA.jpg # street-view sample of a candidate city: compare guardrails, streetlights, bus stops\n\"\"\"\nfrom __future__ import annotations\n\nimport argparse\nfrom _net import fetch_bytes, PROXY_HELP\nimport io\nimport json\nimport math\nimport os\nimport re\nimport sys\nfrom concurrent.futures import ThreadPoolExecutor\nfrom pathlib import Path\n\nfrom PIL import Image, ImageDraw, ImageFont\n\nsys.path.insert(0, str(Path(__file__).parent))\nimport geo # noqa: E402\n\nAPI = \"https://mapsv0.bdimg.com/\"\n\n\ndef _get(url: str, timeout: int = 20, proxy: str | None = None) -> bytes:\n return fetch_bytes(url, proxy, timeout)\n\n\ndef near_bdmc(x: float, y: float, proxy: str | None = None) -> dict | None:\n try:\n c = json.loads(_get(f\"{API}?qt=qsdata&x={x:.0f}&y={y:.0f}\", proxy=proxy)).get(\"content\")\n except Exception:\n return None\n if not c:\n return None\n bx, by = c[\"x\"] / 100, c[\"y\"] / 100\n lat, lon = geo.convert(bx, by, \"bdmc\", \"wgs\")\n return {\"id\": c[\"id\"], \"bdmc\": [bx, by], \"wgs\": [lat, lon], \"road\": c.get(\"RoadName\", \"\")}\n\n\ndef near(lat: float, lon: float, proxy: str | None = None) -> dict | None:\n return near_bdmc(*geo.convert(lat, lon, \"wgs\", \"bdmc\"), proxy=proxy)\n\n\ndef info(pid: str, proxy: str | None = None) -> dict:\n c = json.loads(_get(f\"{API}?qt=sdata&sid={pid}\", proxy=proxy))[\"content\"][0]\n x, y = c[\"X\"] / 100, c[\"Y\"] / 100\n lat, lon = geo.convert(x, y, \"bdmc\", \"wgs\")\n roads = []\n for r in c.get(\"Roads\") or []:\n roads.append([{\"id\": p[\"PID\"], \"wgs\": list(geo.convert(p[\"X\"] / 100, p[\"Y\"] / 100, \"bdmc\", \"wgs\"))}\n for p in (r.get(\"Panos\") or [])]) # some points (construction access roads, isolated points) have no road-segment info\n return {\"id\": pid, \"date\": c.get(\"Date\"), \"wgs\": [lat, lon], \"bdmc\": [x, y],\n \"move_dir\": c.get(\"MoveDir\"), \"roads\": roads,\n \"timeline\": [t.get(\"ID\") for t in (c.get(\"TimeLine\") or [])]}\n\n\ndef scan(lat: float, lon: float, radius_m: float, step_m: float, workers: int = 24, proxy: str | None = None) -> dict:\n \"\"\"Query the nearest panorama point at each point of a square grid; deduplicate and return {id: {...}}.\"\"\"\n pts = []\n n = int(radius_m // step_m)\n for i in range(-n, n + 1):\n for j in range(-n, n + 1):\n p = geo.dest(geo.dest((lat, lon), 0, i * step_m), 90, j * step_m)\n pts.append(p)\n found: dict = {}\n with ThreadPoolExecutor(workers) as ex:\n for r in ex.map(lambda p: near(*p, proxy=proxy), pts):\n if r:\n found[r[\"id\"]] = r\n return found\n\n\ndef thin(panos: dict, spread_m: float) -> dict:\n \"\"\"Thin by minimum spacing: of points closer than spread_m, keep only the first.\"\"\"\n kept: dict = {}\n for k, v in panos.items():\n if all(geo.distance(tuple(v[\"wgs\"]), tuple(u[\"wgs\"])) >= spread_m for u in kept.values()):\n kept[k] = v\n return kept\n\n\ndef sample(bbox: tuple[float, float, float, float], n: int, spread_m: float, seed: int,\n skip: list[str], named_only: bool = True, workers: int = 16, proxy: str | None = None) -> list[dict]:\n \"\"\"Scatter random points in the area to find panoramas, deduplicate, thin by spacing, take n; the heading is the capture car's direction of travel (along the road).\"\"\"\n import random\n rnd = random.Random(seed)\n s, w, nn, e = bbox\n found: dict = {}\n tries = 0\n while len(found) \u003c n and tries \u003c n * 8:\n batch = [(rnd.uniform(s, nn), rnd.uniform(w, e)) for _ in range(n * 2)]\n tries += len(batch)\n with ThreadPoolExecutor(workers) as ex:\n for r in ex.map(lambda p: near(*p, proxy=proxy), batch):\n if not r or r[\"id\"] in found or any(k in (r.get(\"road\") or \"\") for k in skip):\n continue\n if named_only and not r.get(\"road\"): # points without a road name are mostly internal roads of residential compounds and campuses, with no municipal fixtures in view\n continue\n if not (s \u003c= r[\"wgs\"][0] \u003c= nn and w \u003c= r[\"wgs\"][1] \u003c= e):\n continue\n if all(geo.distance(tuple(r[\"wgs\"]), tuple(u[\"wgs\"])) >= spread_m for u in found.values()):\n found[r[\"id\"]] = r\n if len(found) >= n:\n break\n items = []\n with ThreadPoolExecutor(8) as ex:\n infos = list(ex.map(lambda pid: info(pid, proxy=proxy), list(found)[:n]))\n for inf in infos:\n v = found[inf[\"id\"]]\n items.append({\"id\": inf[\"id\"], \"heading\": (inf.get(\"move_dir\") or 0) % 360, \"pitch\": 0, \"fov\": 60,\n \"date\": inf.get(\"date\"), \"wgs\": v[\"wgs\"], \"road\": v.get(\"road\", \"\"),\n \"label\": f\"{(v.get('road') or '')[:10]} {str(inf.get('date') or '')[:6]}\"})\n return items\n\n\n# The Baidu Maps logo render() gets baked into the bottom-left of every view (x \u003c 120, y > 715 on 1024×768), as fractions\n# x0,y0,x1,y1. match.py keeps it out of SIFT: logo matches logo between any two renders (a wrong candidate got 27 of 30 inliers there).\nLOGO_BOX = (0.0, 0.93, 0.12, 1.0)\n\n\ndef render(pid: str, heading: float, pitch: float = 10, fov: float = 80,\n w: int = 1024, h: int = 768, cache: Path | None = None, proxy: str | None = None) -> Image.Image:\n \"\"\"Render a perspective view at a compass heading. heading 0 = north, positive pitch = looking up, fov is the vertical field of view. Max width 1024.\"\"\"\n w = min(w, 1024)\n key = f\"{pid}_{heading:.0f}_{pitch:.0f}_{fov:.0f}_{w}x{h}.jpg\"\n if cache:\n cache.mkdir(parents=True, exist_ok=True)\n if (cache / key).exists():\n return Image.open(cache / key)\n url = (f\"{API}?qt=pr3d&fovy={fov:.0f}&quality=80&panoid={pid}&heading={heading:.1f}\"\n f\"&pitch={pitch:.1f}&width={w}&height={h}\")\n try:\n data = _get(url, 40, proxy)\n except Exception:\n return Image.new(\"RGB\", (w, h), \"gray\")\n if cache:\n (cache / key).write_bytes(data)\n return Image.open(io.BytesIO(data))\n\n\n# CJK-capable fonts, tried in order; GEO_FONT can point at any other font file.\n_FONT_PATHS = (\n \"C:/Windows/Fonts/msyh.ttc\", \"C:/Windows/Fonts/msyhbd.ttc\",\n \"C:/Windows/Fonts/simhei.ttf\", \"C:/Windows/Fonts/simsun.ttc\",\n \"/System/Library/Fonts/STHeiti Medium.ttc\", \"/System/Library/Fonts/Hiragino Sans GB.ttc\",\n \"/usr/share/fonts/opentype/noto/NotoSansCJK-Regular.ttc\",\n)\n\n\ndef _font(size: int):\n for p in (os.environ.get(\"GEO_FONT\", \"\").strip(), *_FONT_PATHS):\n if not p:\n continue\n try:\n return ImageFont.truetype(p, size)\n except Exception:\n continue\n return ImageFont.load_default()\n\n\ndef sheet(items: list[dict], out: Path, cols: int = 3, tw: int = 480, th: int = 360,\n cache: Path | None = None, proxy: str | None = None) -> None:\n \"\"\"items: [{id, heading, pitch?, fov?, label?}] → numbered contact sheet.\"\"\"\n def one(it):\n return render(it[\"id\"], it[\"heading\"], it.get(\"pitch\", 10), it.get(\"fov\", 80), cache=cache, proxy=proxy)\n\n with ThreadPoolExecutor(12) as ex:\n ims = list(ex.map(one, items))\n rows = (len(items) + cols - 1) // cols\n S = Image.new(\"RGB\", (cols * tw, rows * th), \"black\")\n d = ImageDraw.Draw(S)\n f = _font(16)\n for i, (it, im) in enumerate(zip(items, ims)):\n x, y = (i % cols) * tw, (i // cols) * th\n S.paste(im.convert(\"RGB\").resize((tw, th)), (x, y))\n text = it.get(\"label\") or f\"{i}: …{it['id'][-9:]} h{it['heading']:.0f}\"\n d.rectangle([x, y, x + tw, y + 22], fill=\"black\")\n d.text((x + 4, y + 2), text, fill=\"yellow\", font=f)\n S.save(out, quality=88)\n\n\n\ndef _neg_coords(argv: list[str]) -> list[str]:\n \"\"\"argparse treats negative coordinates like -1.45,-48.5 as option names; prefixing a space makes them plain values (float ignores the space). Needed for every puzzle in the southern or western hemisphere.\"\"\"\n return [\" \" + a if re.match(r\"^-\\d[\\d.]*(,-?[\\d.]+)+$\", a) else a for a in argv]\n\n\ndef main() -> None:\n ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)\n ap.add_argument(\"--proxy\", help=PROXY_HELP)\n ap.add_argument(\"--cache\", type=Path, default=Path(\".geo-cache/pano\"))\n sub = ap.add_subparsers(dest=\"cmd\", required=True)\n\n n = sub.add_parser(\"near\")\n n.add_argument(\"latlon\")\n\n i = sub.add_parser(\"info\")\n i.add_argument(\"id\")\n\n s = sub.add_parser(\"scan\")\n s.add_argument(\"center\")\n s.add_argument(\"--radius\", type=float, default=250)\n s.add_argument(\"--step\", type=float, default=20)\n s.add_argument(\"--out\", type=Path, required=True)\n\n r = sub.add_parser(\"render\")\n r.add_argument(\"id\")\n r.add_argument(\"--heading\", type=float, required=True)\n r.add_argument(\"--pitch\", type=float, default=10)\n r.add_argument(\"--fov\", type=float, default=80)\n r.add_argument(\"--out\", type=Path, required=True)\n\n sh = sub.add_parser(\"sheet\")\n g = sh.add_mutually_exclusive_group(required=True)\n g.add_argument(\"--ids\", help=\"comma-separated panoids\")\n g.add_argument(\"--panos\", type=Path, help=\"JSON output of scan\")\n g.add_argument(\"--spec\", type=Path, help=\"JSON list [{id, heading, pitch, fov, label}]\")\n sh.add_argument(\"--heading\", type=float, help=\"one heading for all\")\n sh.add_argument(\"--headings\", help=\"render these headings at every point, comma-separated, e.g. 0,60,120,180,240,300 (look around from a single point)\")\n sh.add_argument(\"--toward\", help=\"lat,lon: each point faces this target\")\n sh.add_argument(\"--offset\", type=float, default=0, help=\"angle added on top of the toward bearing\")\n sh.add_argument(\"--pitch\", type=float, default=10)\n sh.add_argument(\"--fov\", type=float, default=80)\n sh.add_argument(\"--within\", help=\"lat,lon,radius in meters: only points inside this circle\")\n sh.add_argument(\"--road\", help=\"only points whose road name contains these characters (the road field in scan output); comma-separated for several\")\n sh.add_argument(\"--spread\", type=float, help=\"thinning: minimum meters between neighboring points\")\n sh.add_argument(\"--limit\", type=int, default=12, help=\"max cells per contact sheet; extra cells go to more pages automatically\")\n sh.add_argument(\"--out\", type=Path, required=True)\n\n sp = sub.add_parser(\"sample\", help=\"random street-view sample contact sheet of a candidate city/area: compare the styles of municipal fixtures such as guardrails, streetlights, bus stops, curbs\")\n sp.add_argument(\"--bbox\", required=True, help=\"south,west,north,east (urban area)\")\n sp.add_argument(\"--n\", type=int, default=24, help=\"how many points to sample\")\n sp.add_argument(\"--spread\", type=float, default=300, help=\"minimum spacing between two points, m\")\n sp.add_argument(\"--seed\", type=int, default=1)\n sp.add_argument(\"--skip\", default=\"隧道,高速,高架,匝道,立交\", help=\"skip points whose road name contains these characters, comma-separated (default: tunnel, expressway, elevated road, ramp, interchange)\")\n sp.add_argument(\"--heading-offset\", type=float, default=0, help=\"extra degrees to turn from the along-road direction (90 = look at the roadside)\")\n sp.add_argument(\"--include-unnamed\", action=\"store_true\", help=\"also take points without a road name (internal roads of residential compounds, campuses); skipped by default\")\n sp.add_argument(\"--out\", type=Path, required=True)\n\n for parser in (n, i, s, r, sh, sp):\n parser.add_argument(\"--proxy\", default=argparse.SUPPRESS, help=PROXY_HELP)\n args = ap.parse_args(_neg_coords(sys.argv[1:]))\n if args.cmd == \"sample\":\n bb = tuple(map(float, args.bbox.split(\",\")))\n items = sample(bb, args.n, args.spread, args.seed, [k for k in args.skip.split(\",\") if k],\n named_only=not args.include_unnamed, proxy=args.proxy)\n for it in items:\n it[\"heading\"] = (it[\"heading\"] + args.heading_offset) % 360\n if not items:\n sys.exit(\"no panorama points found in the area (area too small, or no Baidu street view here)\")\n pages = [items[k:k + 12] for k in range(0, len(items), 12)]\n for pi, page in enumerate(pages):\n out = args.out if pi == 0 else args.out.with_name(f\"{args.out.stem}_{pi + 1}{args.out.suffix}\")\n sheet(page, out, cache=args.cache, proxy=args.proxy)\n print(out)\n idx = args.out.with_suffix(\".index.json\")\n idx.write_text(json.dumps(items, ensure_ascii=False, indent=1), encoding=\"utf-8\")\n dates = sorted(str(i.get(\"date\") or \"\")[:4] for i in items)\n print(f\"{len(items)} points, capture years {dates[0]}–{dates[-1]}; index -> {idx}\")\n return\n if args.cmd == \"near\":\n print(json.dumps(near(*map(float, args.latlon.split(\",\")), proxy=args.proxy), ensure_ascii=False))\n elif args.cmd == \"info\":\n print(json.dumps(info(args.id, proxy=args.proxy), ensure_ascii=False, indent=1))\n elif args.cmd == \"scan\":\n lat, lon = map(float, args.center.split(\",\"))\n res = scan(lat, lon, args.radius, args.step, proxy=args.proxy)\n args.out.write_text(json.dumps(res, ensure_ascii=False, indent=1), encoding=\"utf-8\")\n print(f\"{len(res)} panos -> {args.out}\")\n elif args.cmd == \"render\":\n render(args.id, args.heading, args.pitch, args.fov, cache=args.cache, proxy=args.proxy).save(args.out)\n print(args.out)\n elif args.cmd == \"sheet\":\n if args.spec:\n items = json.loads(args.spec.read_text(encoding=\"utf-8\"))\n else:\n if args.ids:\n need_pos = bool(args.within or args.toward)\n panos = {p: (info(p, proxy=args.proxy) if need_pos else None) for p in args.ids.split(\",\")}\n else:\n panos = json.loads(args.panos.read_text(encoding=\"utf-8\"))\n if args.within:\n wl, wo, wr = map(float, args.within.split(\",\"))\n panos = {k: v for k, v in panos.items() if geo.distance((wl, wo), v[\"wgs\"]) \u003c= wr}\n if args.road:\n keys = [k for k in args.road.split(\",\") if k]\n panos = {k: v for k, v in panos.items() if any(x in ((v or {}).get(\"road\") or \"\") for x in keys)}\n print(f\"{len(panos)} points after --road filter\")\n if args.spread:\n panos = thin(panos, args.spread)\n print(f\"{len(panos)} points after --spread thinning\")\n target = tuple(map(float, args.toward.split(\",\"))) if args.toward else None\n items = []\n for pid, v in panos.items():\n if args.headings:\n for hd in (float(x) for x in args.headings.split(\",\")):\n items.append({\"id\": pid, \"heading\": hd % 360, \"pitch\": args.pitch, \"fov\": args.fov})\n continue\n if target:\n hd = geo.bearing(tuple(v[\"wgs\"]), target) + args.offset\n elif args.heading is not None:\n hd = args.heading\n else:\n ap.error(\"need --heading, --headings or --toward\")\n items.append({\"id\": pid, \"heading\": hd % 360, \"pitch\": args.pitch, \"fov\": args.fov})\n pages = [items[k:k + args.limit] for k in range(0, len(items), args.limit)] or [[]]\n for pi, page in enumerate(pages):\n # page 1 is written to --out itself, later pages get _2, _3…, to avoid \"the file given by --out doesn't exist\"\n out = args.out if pi == 0 else args.out.with_name(f\"{args.out.stem}_{pi + 1}{args.out.suffix}\")\n sheet(page, out, cache=args.cache, proxy=args.proxy)\n print(out)\n if len(pages) > 1:\n print(f\"{len(pages)} pages: {args.out.name} plus {args.out.stem}_2 … _{len(pages)}, {args.limit} cells per page\")\n if len(pages) > 1 or args.spec is None:\n idx = args.out.with_suffix(\".index.json\")\n idx.write_text(json.dumps(items, indent=1), encoding=\"utf-8\")\n print(f\"index -> {idx}\")\n\n\nif __name__ == \"__main__\":\n # Chinese-locale Windows writes GBK by default: m², ñ make it crash, and the Chinese the agent reads comes out garbled\n sys.stdout.reconfigure(encoding=\"utf-8\")\n sys.stderr.reconfigure(encoding=\"utf-8\", errors=\"backslashreplace\")\n main()\n","content_type":"text/x-python; charset=utf-8","language":"python","size":17750,"content_sha256":"e835a4eb9cce4f9234f08bd6818dfacb39ac8b58d724d255cc96e5c16c438709"},{"filename":"scripts/board.py","content":"#!/usr/bin/env python3\n# /// script\n# requires-python = \">=3.10\"\n# dependencies = []\n# ///\n\"\"\"Candidate board: every candidate, clue, piece of evidence and exclusion during geolocation is recorded here; ranking and the next step are computed by the script, not from memory.\n\nRules are written as code, not left to self-discipline:\n- There is no \"delete candidate\" operation. Exclusion has exactly one entry point, `exclude`, and it must be given a computed file (output of geo.py frame, terrain.py and the like);\n the clue itself must be text that was read or a computed result; an inference can only down-weight (likelihood ratio clamped to 1/3–3).\n- Exclusion scope must be ≤ evidence scope: for candidates with extent (district/area/road), exclude must state with --covers which stretch the evidence covers;\n coverage under half is rejected. Looking at one point and excluding the whole road (one point standing in for the whole area) is a mistake that recurred twice.\n- Discrete candidates are never averaged to a midpoint: the main answer in report is always first place; the rest go to alternatives.\n- Population and fame are not evidence: the prior is uniform by default (optionally by area); there is no by-population option.\n- Scan order is by \"share ÷ pages\": scan small districts first, put large districts last with a page cap.\n- When discriminating tests can't separate the candidates, next gives a definite next step instead of stalling.\n\n init create board.json\n add add candidates (country/province/city/district/area/point); --from bulk-imports output of poi.py / gazetteer.py / osm.py geom\n children use gazetteer.py to add every subdivision of an admin area as a candidate (\"list the whole category first\")\n clue record a clue: seen / text read / inferred / computed\n evidence a clue's likelihood ratio for some candidates (>1 supports, \u003c1 opposes)\n exclude exclude a candidate (needs a computed file)\n scan-bbox set a candidate's scan extent (built-up area); pages are counted from it\n urban fill scan-bbox automatically with gazetteer.py urban\n falsify write the falsification condition before scanning/confirming\n rank current ranking (score, share, evidence, pages, share/page)\n next suggested next step\n check checklist before the conclusion\n report generate the candidates/alternatives/excluded/unused-clues fields of result.json\n apply lookup clues (calling code, driving side, territories; plates, area codes… from region packs) add candidates and evidence automatically (uses clues.py)\n move put a candidate under another candidate (a point under its road) so its supporting evidence counts there\n country set the country context: apply restricts region-pack lookups to it, next uses its pack's tips\n log print the ledger\n\nExamples:\n board.py init --photo photo.jpg\n board.py children \u003cmunicipality or province name> # all 38 districts become candidates, uniform prior\n board.py add --from pois.json --level area --parent \u003ccity> # fine-level candidates (same-name campuses, OSM walled compounds) all go on the board; don't hand-pick\n board.py clue \"bus yellow on top, green below; green rear stripe curves down\" --kind livery --status observed --file bus_zoom.png\n board.py evidence --clue K1 --for \u003cdistrict A>:5 --for \u003cdistrict B>:2 --why \"compared bus rears one by one across both districts' bus photos\" --file livery_sheet.jpg\n board.py apply --kind plate --value \u003cfirst two plate characters>\n board.py urban \u003cdistrict A> --within \u003cmunicipality or province name>\n board.py rank\n board.py next\n board.py falsify \u003cdistrict A> --text \"give up if the sports field's long axis isn't north-south\"\n board.py exclude \u003cdistrict B> --clue K4 --computed frame.json --why \"by the view-angle range X should be in frame and ≥40px; it isn't in the image\"\n board.py check\n board.py report --merge result.json\n\"\"\"\nfrom __future__ import annotations\n\nimport argparse\nfrom _net import PROXY_HELP\nimport json\nimport math\nimport os\nimport re\nimport subprocess\nimport sys\nfrom datetime import datetime\nfrom pathlib import Path\n\nimport regions\n\nHERE = Path(__file__).parent\n# uv run writes its own path into the UV env var: call child scripts with it, so uv is found even when it isn't on PATH (e.g. just installed, terminal not reopened)\nUV = os.environ.get(\"UV\") or \"uv\"\nLEVELS = [\"country\", \"admin1\", \"admin2\", \"city\", \"district\", \"area\", \"road\", \"point\"]\nLEVEL_LABEL = {\"country\": \"country\", \"admin1\": \"province/state\", \"admin2\": \"prefecture/county\", \"city\": \"city\",\n \"district\": \"district\", \"area\": \"area\", \"road\": \"road\", \"point\": \"point\"}\n# Candidates with extent: exclusion scope must be \u003c= evidence scope. Looking at one point on a road/area and excluding the whole of it is a mistake that recurred twice\nEXTENDED_LEVELS = {\"district\", \"area\", \"road\"}\nCOVERS_MIN = 0.5 # --covers must cover at least this fraction of the candidate's extent before the whole candidate can be excluded\nSTATUS = [\"observed\", \"read\", \"inferred\", \"computed\"]\n# Likelihood ratio caps: inferences can only rank; only text read and computed results can move the score a lot\nLR_CAP = {\"inferred\": 3.0, \"observed\": 5.0, \"read\": 50.0, \"computed\": 50.0}\nGLOBAL_KINDS = (\"calling-code\", \"driving-side\", \"territories\") # clues.py tables in data/, not region packs\nCHEAP_OPS = [\n (\"plate/area-code/calling-code\", \"plate, area code or calling code readable → `clues.py lookup` + `board.py apply`\"),\n (\"terrain\", \"flat vs hill city in the image → `terrain.py view` or z13 satellite imagery per candidate to check terrain; one call rules out a batch\"),\n (\"livery\", \"bus/taxi livery → image search for \\\"\u003ccity> \u003ccolor> bus\\\" in the local language; read route signs from the results, compare the rear waistline stripe district by district\"),\n (\"municipal\", \"guardrails, street lights, bus stops, curb styles → street view of each candidate's built-up area (`gsv.py sheet --points`, or the street-view script the region pack names), one contact sheet per candidate\"),\n (\"network\", \"water/road network template → `tiles.py fetch --zoom 13` per candidate, compare river direction and number of bridges\"),\n (\"phenology\", \"vegetation + month → weak evidence only, can't exclude on its own\"),\n]\n\n\ndef _norm(s: str) -> str:\n return re.sub(r\"[\\s（）()]\", \"\", s or \"\").rstrip(\"市省区县自治州自治县自治区特别行政区\")\n\n\ndef _clue_kind(kind: str) -> str:\n # Accept the long spelling in new clues and in boards written by older versions.\n return \"infra\" if kind.strip().lower() in (\"infra\", \"infrastructure\") else kind\n\n\ndef _load(p: Path) -> dict:\n if not p.exists():\n sys.exit(f\"No {p}: run `board.py init --photo photo.jpg` first\")\n return json.loads(p.read_text(encoding=\"utf-8\"))\n\n\ndef _save(p: Path, b: dict) -> None:\n b[\"updated\"] = datetime.now().isoformat(timespec=\"seconds\")\n p.write_text(json.dumps(b, ensure_ascii=False, indent=1), encoding=\"utf-8\")\n\n\ndef _find(b: dict, name: str) -> str:\n if name in b[\"candidates\"]:\n return name\n hits = [k for k in b[\"candidates\"] if _norm(k) == _norm(name)]\n if len(hits) == 1:\n return hits[0]\n if not hits:\n sys.exit(f\"No candidate \\\"{name}\\\": add it or use children first (existing: {', '.join(list(b['candidates'])[:12])}…)\")\n sys.exit(f\"\\\"{name}\\\" matches several candidates: {hits}; write the full name\")\n\n\ndef _log(b: dict, text: str) -> None:\n b.setdefault(\"log\", []).append(f\"{datetime.now():%H:%M} {text}\")\n\n\ndef _run(cmd: list[str], live: bool = False) -> subprocess.CompletedProcess:\n \"\"\"Run gazetteer.py, clues.py. Child scripts and this one both use UTF-8: Chinese Windows reads and writes GBK by default, and if the two sides differ you get mojibake or crashes.\n live=True passes the child's stderr straight through: Overpass retries can take many minutes, and captured they look like a hang.\"\"\"\n return subprocess.run(cmd, text=True, encoding=\"utf-8\", errors=\"replace\", stdout=subprocess.PIPE,\n stderr=None if live else subprocess.PIPE, env={**os.environ, \"PYTHONUTF8\": \"1\"})\n\n\ndef _bbox_km2(bb: list[float]) -> float:\n s, w, n, e = bb\n return abs(n - s) * 110.574 * abs(e - w) * 111.320 * math.cos(math.radians((s + n) / 2))\n\n\ndef _pages(bb: list[float], zoom: int, cell: int, cols: int) -> int:\n s, w, n, e = bb\n mpp = 156543.03392 * math.cos(math.radians((s + n) / 2)) / 2 ** zoom\n step = cell * mpp * 0.9\n r = max(1, math.ceil((n - s) * 110574 / step))\n c = max(1, math.ceil((e - w) * 111320 * math.cos(math.radians((s + n) / 2)) / step))\n return math.ceil(r * c / (cols * cols))\n\n\ndef _inherited(b: dict, name: str, own_clues: set[str]) -> list[dict]:\n \"\"\"Support flows up, refutation doesn't: a point that matches supports its road, a point that doesn't match says nothing about the rest of the road.\n Per clue the strongest supporting evidence among descendants, for clues the candidate has no evidence of its own.\"\"\"\n kids: dict[str, list[str]] = {}\n for n, c in b[\"candidates\"].items():\n if c.get(\"parent\") and c.get(\"status\") != \"excluded\":\n kids.setdefault(c[\"parent\"], []).append(n)\n seen, todo, best = {name}, list(kids.get(name, [])), {}\n while todo:\n n = todo.pop()\n if n in seen:\n continue\n seen.add(n)\n todo += kids.get(n, [])\n for e in b[\"evidence\"]:\n if e[\"candidate\"] == n and e[\"lr\"] > 1 and e[\"clue\"] not in own_clues and e[\"lr\"] > best.get(e[\"clue\"], {}).get(\"lr\", 1):\n best[e[\"clue\"]] = {**e, \"via\": n}\n return list(best.values())\n\n\ndef _scores(b: dict, level: str, prior_by: str) -> list[dict]:\n rows = []\n for name, c in b[\"candidates\"].items():\n if c[\"level\"] != level:\n continue\n prior = 1.0\n if prior_by == \"area\" and c.get(\"bbox\"):\n prior = max(_bbox_km2(c[\"bbox\"]), 1.0)\n logit = math.log(prior) + math.log(max(c.get(\"prior\", 1.0), 1e-9))\n ev = [e for e in b[\"evidence\"] if e[\"candidate\"] == name]\n ev += _inherited(b, name, {e[\"clue\"] for e in ev})\n for e in ev:\n logit += math.log(max(e[\"lr\"], 1e-9))\n rows.append({\"name\": name, \"logit\": logit, \"n_ev\": len(ev),\n \"n_verified\": sum(1 for e in ev if b[\"clues\"][e[\"clue\"]][\"status\"] in (\"read\", \"computed\")),\n \"excluded\": c.get(\"status\") == \"excluded\", \"c\": c, \"ev\": ev})\n open_rows = [r for r in rows if not r[\"excluded\"]]\n if open_rows:\n m = max(r[\"logit\"] for r in open_rows)\n z = sum(math.exp(r[\"logit\"] - m) for r in open_rows)\n for r in open_rows:\n r[\"share\"] = math.exp(r[\"logit\"] - m) / z\n for r in rows:\n r.setdefault(\"share\", 0.0)\n rows.sort(key=lambda r: (r[\"excluded\"], -r[\"share\"]))\n return rows\n\n\ndef _frontier(b: dict) -> str | None:\n \"\"\"The finest level that still has ≥2 non-excluded candidates; if none, the finest level that has candidates.\"\"\"\n for lv in reversed(LEVELS):\n if sum(1 for c in b[\"candidates\"].values() if c[\"level\"] == lv and c.get(\"status\") != \"excluded\") >= 2:\n return lv\n for lv in reversed(LEVELS):\n if any(c[\"level\"] == lv and c.get(\"status\") != \"excluded\" for c in b[\"candidates\"].values()):\n return lv\n return None\n\n\ndef _cost_of(c: dict, args) -> tuple[int | None, str]:\n bb = c.get(\"scan_bbox\") or c.get(\"bbox\")\n if not bb:\n return None, \"no extent\"\n pages = _pages(bb, args.zoom, args.cell, args.cols)\n return pages, (\"built-up area\" if c.get(\"scan_bbox\") else \"whole-district bbox\")\n\n\n# ---------------------------------------------------------------- subcommands\n\ndef cmd_init(args, p: Path) -> None:\n if p.exists() and not args.force:\n sys.exit(f\"{p} already exists (add --force to overwrite)\")\n b = {\"case\": args.case or Path.cwd().name, \"photo\": args.photo, \"created\": datetime.now().isoformat(timespec=\"seconds\"),\n \"country\": _country_code(args.country) if args.country else None,\n \"candidates\": {}, \"clues\": {}, \"evidence\": [], \"falsify\": {}, \"log\": []}\n _save(p, b)\n print(f\"Created {p}\" + (f\" (country {b['country']})\" if b[\"country\"] else \"\"))\n\n\ndef _country_code(text: str) -> str:\n pk = regions.resolve(text)\n if not pk:\n print(f\"Note: no region pack for '{text}' (installed: {', '.join(sorted(regions.packs())) or 'none'}); \"\n f\"lookups that need a pack will answer unsupported, global tools still work.\")\n return text.upper()\n return pk[\"code\"]\n\n\ndef cmd_country(args, p: Path) -> None:\n b = _load(p)\n if args.code in (\"none\", \"-\", \"\"):\n b[\"country\"] = None\n _log(b, \"country cleared\")\n _save(p, b)\n print(\"Country context cleared: lookups search every region pack.\")\n return\n b[\"country\"] = _country_code(args.code)\n _log(b, f\"country {b['country']}\")\n _save(p, b)\n pk = regions.resolve(b[\"country\"])\n print(f\"Country context {b['country']}: `apply` restricts region-pack lookups to it; `next` uses its tips.\"\n + (f\" Card: `regions.py show {b['country'].lower()}`.\" if pk else \"\"))\n\n\ndef _bbox_around(lat: float, lon: float, r: float) -> list[float]:\n dy, dx = r / 110574, r / (111320 * math.cos(math.radians(lat)))\n return [round(lat - dy, 6), round(lon - dx, 6), round(lat + dy, 6), round(lon + dx, 6)]\n\n\ndef _coords(g) -> list[tuple[float, float]]:\n \"\"\"All (lon, lat) in a GeoJSON geometry.\"\"\"\n if isinstance(g, (list, tuple)) and g and isinstance(g[0], (int, float)):\n return [(float(g[0]), float(g[1]))]\n out = []\n for x in g or []:\n out += _coords(x)\n return out\n\n\ndef _read_from(path: Path, radius: float) -> list[tuple[str, dict]]:\n \"\"\"--from file → [(name, {bbox, center})]. Accepts four forms:\n poi.py/tiles.py {name: [lat, lon]}; gazetteer.py {name: {bbox, center}};\n GeoJSON (osm.py geom, name taken from properties.name); [{name, lat, lon} or {name, bbox}]. Points get an extent from --radius.\"\"\"\n d = json.loads(path.read_text(encoding=\"utf-8\"))\n rows: list[tuple[str, dict]] = []\n\n def one(name: str, v) -> None:\n if isinstance(v, (list, tuple)) and len(v) >= 2 and all(isinstance(x, (int, float)) for x in v[:2]):\n lat, lon = float(v[0]), float(v[1])\n rows.append((name, {\"center\": [lat, lon], \"bbox\": _bbox_around(lat, lon, radius)}))\n elif isinstance(v, dict):\n bb = v.get(\"bbox\")\n c = v.get(\"center\") or ([v[\"lat\"], v[\"lon\"]] if \"lat\" in v and \"lon\" in v else None)\n if bb:\n rows.append((name, {\"center\": c or [(bb[0] + bb[2]) / 2, (bb[1] + bb[3]) / 2], \"bbox\": bb}))\n elif c:\n rows.append((name, {\"center\": c, \"bbox\": _bbox_around(float(c[0]), float(c[1]), radius)}))\n\n if isinstance(d, dict) and d.get(\"type\") == \"FeatureCollection\":\n for i, f in enumerate(d.get(\"features\", []), 1):\n pr = f.get(\"properties\") or {}\n pts = _coords((f.get(\"geometry\") or {}).get(\"coordinates\"))\n if not pts:\n continue\n lons, lats = [q[0] for q in pts], [q[1] for q in pts]\n name = pr.get(\"name\") or f\"{pr.get('@id') or pr.get('id') or f'#{i}'}\"\n bb = [min(lats), min(lons), max(lats), max(lons)]\n if len(pts) == 1:\n bb = _bbox_around(lats[0], lons[0], radius)\n rows.append((name, {\"center\": [round((bb[0] + bb[2]) / 2, 6), round((bb[1] + bb[3]) / 2, 6)],\n \"bbox\": [round(v, 6) for v in bb]}))\n elif isinstance(d, dict):\n for name, v in d.items():\n if not str(name).startswith(\"_\"):\n one(str(name), v)\n elif isinstance(d, list):\n for i, v in enumerate(d, 1):\n if isinstance(v, dict):\n one(str(v.get(\"name\") or f\"#{i}\"), v)\n # Same names (several campuses of one school, several same-name walled areas in OSM) get a sequence number; none are dropped\n seen: dict[str, int] = {}\n out = []\n for name, v in rows:\n seen[name] = seen.get(name, 0) + 1\n out.append((name if seen[name] == 1 else f\"{name}#{seen[name]}\", v))\n return out\n\n\ndef cmd_add(args, p: Path) -> None:\n b = _load(p)\n if args.level not in LEVELS:\n sys.exit(f\"--level must be one of {LEVELS}\")\n if not args.names and not args.from_:\n sys.exit(\"Give candidate names, or --from a file (poi.py --out, gazetteer.py --out, GeoJSON from osm.py geom)\")\n if args.from_:\n rows = _read_from(Path(args.from_), args.radius)\n if not rows:\n sys.exit(f\"No candidates with coordinates read from {args.from_}\")\n n, added = 0, []\n for name, v in rows:\n key = _slot(b, name, args.parent, None, args.level)\n if not key:\n continue\n added.append(key)\n b[\"candidates\"][key] = {\"level\": args.level, \"parent\": args.parent, \"bbox\": v[\"bbox\"], \"scan_bbox\": None,\n \"center\": v[\"center\"], \"prior\": args.prior, \"status\": \"open\",\n \"note\": args.note or \"\", \"from\": str(args.from_)}\n n += 1\n _log(b, f\"add --from {args.from_} → +{n} {args.level}\")\n _save(p, b)\n hints = [h for k in added for h in [_skip_hint(b, k)] if h]\n if hints:\n print(f\"NOTE {hints[0]}\" + (f\" (and {len(hints) - 1} more)\" if len(hints) > 1 else \"\"))\n print(f\"Added {n} {LEVEL_LABEL[args.level]} candidates from {args.from_} ({len(rows)} read; duplicate names skipped). \"\n f\"Same names under another parent get the parent in the key. All on the board with a uniform prior: record evidence for every one you look at (record --against too when it doesn't match); check lists the ones not looked at.\")\n for raw in args.names:\n name = _slot(b, raw, args.parent, None, args.level)\n if not name:\n print(f\"Already have {raw}\" + (f\" under {args.parent}\" if args.parent else \"\") + \", skipping\")\n continue\n c = {\"level\": args.level, \"parent\": args.parent, \"bbox\": None, \"scan_bbox\": None, \"prior\": args.prior,\n \"status\": \"open\", \"note\": args.note or \"\"}\n if args.bbox:\n c[\"bbox\"] = [float(v) for v in args.bbox.split(\",\")]\n b[\"candidates\"][name] = c\n _log(b, f\"add {name} ({args.level})\")\n hint = _skip_hint(b, name)\n if hint:\n print(f\"NOTE {hint}\")\n _save(p, b)\n print(f\"{len(b['candidates'])} candidates\")\n\n\ndef cmd_move(args, p: Path) -> None:\n b = _load(p)\n name = _find(b, args.name)\n parent = _find(b, args.parent)\n if parent == name:\n sys.exit(\"A candidate can't be its own parent\")\n if LEVELS.index(b[\"candidates\"][parent][\"level\"]) >= LEVELS.index(b[\"candidates\"][name][\"level\"]):\n sys.exit(f\"{parent} ({b['candidates'][parent]['level']}) isn't coarser than {name} ({b['candidates'][name]['level']})\")\n old = b[\"candidates\"][name].get(\"parent\")\n b[\"candidates\"][name][\"parent\"] = parent\n _log(b, f\"move {name}: {old} → {parent}\")\n _save(p, b)\n print(f\"{name} now under {parent} (was {old}); its supporting evidence now counts for {parent} in the {b['candidates'][parent]['level']} ranking\")\n hint = _skip_hint(b, name)\n if hint:\n print(f\"NOTE {hint}\")\n\n\ndef cmd_children(args, p: Path) -> None:\n b = _load(p)\n cmd = [UV, \"run\", str(HERE / \"gazetteer.py\"), \"children\", args.parent, \"--out\", str(p.parent / \".gz_children.json\")]\n if args.level:\n cmd += [\"--level\", str(args.level)]\n if args.within:\n cmd += [\"--within\", args.within]\n if args.proxy:\n cmd += [\"--proxy\", args.proxy]\n r = _run(cmd, live=True)\n out = r.stdout.strip(); print(out if len(out) \u003c 1800 else out[:1800].rsplit('\\n', 1)[0] + '\\n …')\n if r.returncode != 0:\n sys.exit(\"gazetteer failed; its messages are above\")\n kids = json.loads((p.parent / \".gz_children.json\").read_text(encoding=\"utf-8\"))\n as_level, why = _child_level(b, args, out)\n # Stable ids from a region pack's admin table, so lookups (plates, area codes) land on these same candidates\n pk, local = regions.admin_children(args.parent, b.get(\"country\"))\n by_local = {}\n if pk:\n chain = regions.admin_chain(pk, regions.admin_find(args.parent, pk[\"code\"])[0][1])\n for it in local:\n by_local[_norm(it[\"name\"])] = \"/\".join([pk[\"code\"], *chain, it[\"name\"]])\n n = 0\n for raw, k in kids.items():\n cid = by_local.get(_norm(raw))\n name = _slot(b, raw, args.parent, cid, as_level)\n if not name:\n continue\n b[\"candidates\"][name] = {\"level\": as_level, \"parent\": args.parent, \"bbox\": k.get(\"bbox\"), \"scan_bbox\": None,\n \"prior\": 1.0, \"status\": \"open\", \"note\": k.get(\"note\", \"\"), \"osm_id\": k.get(\"osm_id\"),\n \"country\": pk[\"code\"] if pk else b.get(\"country\"), \"cid\": cid}\n n += 1\n _log(b, f\"children {args.parent} → +{n} {as_level}\")\n _save(p, b)\n print(f\"Added {n} candidates (level {as_level}: {why}; uniform prior). Population and fame don't enter the score.\")\n\n\n# Parent's level on the board → level the children are recorded as\nNEXT_LEVEL = {\"country\": \"admin1\", \"admin1\": \"admin2\", \"admin2\": \"district\", \"city\": \"district\", \"district\": \"area\",\n \"area\": \"road\", \"road\": \"point\"}\n\n\ndef _child_level(b: dict, args, gz_out: str) -> tuple[str, str]:\n \"\"\"Candidate level for children: use --as-level if given; parent is a country → admin1; parent already on the board → its next level\n (where a province's next level down is directly districts, as with a municipality, record as district); otherwise → district.\"\"\"\n if args.as_level:\n return args.as_level, \"set by --as-level\"\n m = re.search(r\"admin_level (\\d+)\\) children admin_level (\\d+)\", gz_out)\n p_lv, c_lv = (int(m.group(1)), int(m.group(2))) if m else (None, None)\n if p_lv is not None and p_lv \u003c= 2:\n # When jumping straight from a country to a finer level (--level 5/6), don't record as province level\n lv = \"admin1\" if c_lv is None or c_lv \u003c= 4 else (\"admin2\" if c_lv == 5 else \"district\")\n return lv, f\"parent is a country, children admin_level {c_lv}\"\n hits = [k for k in b[\"candidates\"] if _norm(k) == _norm(args.parent)]\n if len(hits) == 1:\n lv = b[\"candidates\"][hits[0]][\"level\"]\n nxt = NEXT_LEVEL.get(lv, \"district\")\n if lv == \"admin1\" and p_lv is not None and c_lv is not None and c_lv >= p_lv + 2:\n nxt = \"district\"\n return nxt, f\"parent {hits[0]} is {lv} on the candidate board\"\n return \"district\", \"default\"\n\n\ndef cmd_clue(args, p: Path) -> None:\n b = _load(p)\n kid = f\"K{len(b['clues']) + 1}\"\n if args.status not in STATUS:\n sys.exit(f\"--status must be one of {STATUS}: observed=shape/color seen directly in the image, read=text/numbers read, inferred=inference (building about 8 floors, road going uphill), computed=computed by a script\")\n resolves = list(dict.fromkeys(getattr(args, \"resolves\", None) or []))\n output = args.file or \"\"\n if resolves:\n if args.status != \"computed\":\n sys.exit(\"--resolves requires --status computed: link a result that actually tests the named clues\")\n unknown = [k for k in resolves if k not in b[\"clues\"]]\n if unknown:\n sys.exit(f\"Unknown clues in --resolves: {', '.join(unknown)}\")\n if not output or not Path(output).is_file():\n sys.exit(\"--resolves requires --file pointing to an existing computed output file\")\n output = str(Path(output).resolve())\n kind = _clue_kind(args.kind)\n b[\"clues\"][kid] = {\"text\": args.text, \"kind\": kind, \"status\": args.status, \"file\": output,\n \"source\": args.source or \"\", \"used\": False}\n if resolves:\n b[\"clues\"][kid][\"resolves\"] = resolves\n _log(b, f\"clue {kid} [{kind}/{args.status}] {args.text}\" + (f\"; resolves {', '.join(resolves)}\" if resolves else \"\"))\n _save(p, b)\n cap = LR_CAP[args.status]\n print(f\"{kid} recorded. Status {args.status}: likelihood ratio cap {cap:g}\" + (\", can only rank, can't exclude\" if args.status in (\"inferred\", \"observed\") else \", usable for exclude (still needs a computed file)\"))\n\n\ndef _parse_lr(items: list[str] | None, default: float | None) -> list[tuple[str, float]]:\n out = []\n for it in items or []:\n if \":\" in it:\n name, v = it.rsplit(\":\", 1)\n out.append((name, float(v)))\n elif default is not None:\n out.append((it, default))\n else:\n sys.exit(f\"Write it as name:likelihood_ratio, e.g. {it}:5\")\n return out\n\n\ndef _put_evidence(b: dict, rec: dict) -> dict | None:\n \"\"\"Revise one clue/candidate pair, including duplicate entries in older boards.\"\"\"\n key = (rec[\"clue\"], rec[\"candidate\"])\n old = next((e for e in b[\"evidence\"] if (e[\"clue\"], e[\"candidate\"]) == key), None)\n if old is not None:\n previous = dict(old)\n old.update(rec)\n b[\"evidence\"] = [e for e in b[\"evidence\"] if e is old or (e[\"clue\"], e[\"candidate\"]) != key]\n return previous\n next_id = max((int(e[\"id\"][1:]) for e in b[\"evidence\"]), default=0) + 1\n b[\"evidence\"].append({\"id\": f\"E{next_id}\", **rec})\n return None\n\n\ndef cmd_evidence(args, p: Path) -> None:\n b = _load(p)\n if args.clue not in b[\"clues\"]:\n sys.exit(f\"No clue {args.clue}; run `board.py clue` first\")\n cl = b[\"clues\"][args.clue]\n cap = LR_CAP[cl[\"status\"]]\n pairs = _parse_lr(args.for_, 3.0) + [(n, v) for n, v in _parse_lr(args.against, 1 / 3)]\n if not pairs:\n sys.exit(\"Give at least one --for name:likelihood_ratio or --against name:likelihood_ratio\")\n clipped, replaced = [], []\n for name, lr in pairs:\n cname = _find(b, name)\n if lr \u003c= 0:\n sys.exit(\"A likelihood ratio of 0 means exclusion; use `board.py exclude` (needs a computed file)\")\n lr2 = min(max(lr, 1 / cap), cap)\n if abs(lr2 - lr) > 1e-9:\n clipped.append(f\"{cname}:{lr:g}→{lr2:g}\")\n rec = {\"clue\": args.clue, \"candidate\": cname, \"lr\": lr2, \"why\": args.why or \"\", \"file\": args.file or \"\", \"cmd\": args.command_ or \"\"}\n # one clue counts once per candidate: recording it again revises that estimate instead of multiplying it in a second time\n old = _put_evidence(b, rec)\n if old:\n replaced.append(f\"{old['id']} {cname}:{old['lr']:g}→{lr2:g}\")\n cl[\"used\"] = True\n if args.file and not Path(args.file).exists():\n print(f\"Note: {args.file} doesn't exist; evidence files must be real outputs of this session\")\n _log(b, f\"evidence {args.clue} → {', '.join(f'{n}:{v:g}' for n, v in pairs)}\")\n _save(p, b)\n if clipped:\n print(f\"Clue {args.clue} is \\\"{cl['status']}\\\", likelihood ratio clamped to 1/{cap:g}–{cap:g}: {'; '.join(clipped)}. \"\n f\"For more weight, first verify the clue as read/computed (read the text, run a script), then record a new clue.\")\n if replaced:\n print(f\"Clue {args.clue} already had evidence for these candidates; replaced it instead of counting it twice: {'; '.join(replaced)}. \"\n f\"An independent observation (another photo detail, another source) is a new clue: `board.py clue` first.\")\n print(\"Recorded.\" + (\"\" if args.why else \" Suggest adding --why to state the basis.\"))\n\n\ndef cmd_exclude(args, p: Path) -> None:\n b = _load(p)\n cname = _find(b, args.name)\n if args.clue not in b[\"clues\"]:\n sys.exit(f\"No clue {args.clue}\")\n cl = b[\"clues\"][args.clue]\n if cl[\"status\"] not in (\"read\", \"computed\"):\n sys.exit(f\"Clue {args.clue} is \\\"{cl['status']}\\\" (inferred/observed): it can't exclude, only `evidence --against {cname}:0.34`. \"\n f\"Hard rule 9: exclusion and confirmation use the same standard.\")\n if not args.computed or not Path(args.computed).exists():\n sys.exit(\"Exclusion must attach a computed file (--computed, e.g. geo.py frame output or a terrain.py comparison image), and the file must actually exist\")\n c = b[\"candidates\"][cname]\n if c[\"level\"] in EXTENDED_LEVELS:\n if not args.covers:\n sys.exit(\n f\"{cname} is a {LEVEL_LABEL[c['level']]}-level candidate (has extent): exclusion needs --covers stating where the evidence actually covers \"\n f\"('lat,lon' or 'lat,lon:lat,lon'). Hard rule 9: exclusion scope must be ≤ evidence scope — \"\n f\"looking at one point and excluding the whole thing is one point standing in for the whole area. If you only checked one stretch, use instead: \"\n f\"board.py evidence --clue {args.clue} --against {cname}:0.34 --file {args.computed}\")\n pts = _parse_covers(args.covers)\n if not pts:\n sys.exit(\"--covers must parse to coordinates: 'lat,lon' (single point) or 'lat,lon:lat,lon' (span)\")\n r = _covers_ratio(c, pts)\n if r is not None and r[0] \u003c COVERS_MIN:\n cov_m = _cov_span_m(pts[0], pts[-1]) if len(pts) >= 2 else 0.0\n sys.exit(\n f\"--covers covers only about {r[0]:.0%} of {cname} (evidence {cov_m:.0f} m / candidate extent diagonal {r[1]:.0f} m): \"\n f\"not enough to exclude the whole thing. Add comparison images for the remaining stretches before excluding, or down-weight first: \"\n f\"board.py evidence --clue {args.clue} --against {cname}:0.34 --file {args.computed}\")\n c[\"status\"] = \"excluded\"\n c[\"excluded_by\"] = {\"clue\": args.clue, \"computed\": args.computed, \"why\": args.why or \"\",\n \"covers\": args.covers or \"\"}\n cl[\"used\"] = True\n _log(b, f\"exclude {cname} by {args.clue} ({args.computed})\")\n _save(p, b)\n print(f\"Excluded {cname}. Excluded candidates stay in the ledger; check looks back at them.\")\n\n\ndef cmd_scan_bbox(args, p: Path) -> None:\n b = _load(p)\n cname = _find(b, args.name)\n b[\"candidates\"][cname][\"scan_bbox\"] = [float(v) for v in args.bbox.split(\",\")]\n _log(b, f\"scan-bbox {cname} {args.bbox}\")\n _save(p, b)\n print(\"ok\")\n\n\ndef cmd_urban(args, p: Path) -> None:\n b = _load(p)\n cname = _find(b, args.name)\n cmd = [UV, \"run\", str(HERE / \"gazetteer.py\"), \"urban\", cname]\n if args.within:\n cmd += [\"--within\", args.within]\n if args.proxy:\n cmd += [\"--proxy\", args.proxy]\n r = _run(cmd, live=True)\n if r.returncode != 0:\n sys.exit(\"gazetteer urban failed; its messages are above\")\n d = json.loads(r.stdout[r.stdout.index(\"{\"):])\n if d.get(\"urban_bbox\"):\n b[\"candidates\"][cname][\"scan_bbox\"] = d[\"urban_bbox\"]\n b[\"candidates\"][cname][\"urban_source\"] = d.get(\"source\")\n _log(b, f\"urban {cname} {d['urban_bbox']} ({d.get('source')})\")\n _save(p, b)\n print(f\"{cname} built-up area {d['urban_bbox']}, about {d.get('urban_bbox_km2')} km² (source {d.get('source')})\"\n + (f\"; {d['note']}\" if d.get(\"note\") else \"\"))\n else:\n print(f\"{cname}: {d.get('note', 'no built-up area data')}; set it by hand with `board.py scan-bbox`\")\n\n\ndef cmd_falsify(args, p: Path) -> None:\n b = _load(p)\n cname = _find(b, args.name)\n b[\"falsify\"].setdefault(cname, []).append(args.text)\n _log(b, f\"falsify {cname}: {args.text}\")\n _save(p, b)\n print(\"Falsification condition recorded. If it shows up, give up; don't look for reasons to explain it away.\")\n\n\ndef cmd_rank(args, p: Path, quiet: bool = False) -> dict:\n b = _load(p)\n out = {}\n for lv in LEVELS:\n rows = _scores(b, lv, args.prior_by)\n if not rows:\n continue\n out[lv] = rows\n if quiet:\n continue\n print(f\"\\n[{LEVEL_LABEL[lv]}] {len(rows)} candidates ({sum(1 for r in rows if not r['excluded'])} not excluded)\")\n print(f\" {'candidate':\u003c14}{'share':>7}{'evidence':>9}{'verified':>9}{'pages':>6} {'share/page':>10} note\")\n for r in rows[: args.limit]:\n pages, src = _cost_of(r[\"c\"], args)\n ratio = (r[\"share\"] / pages) if pages else None\n flag = \"excluded\" if r[\"excluded\"] else \"\"\n print(f\" {r['name']:\u003c14}{r['share']:>7.1%}{r['n_ev']:>9}{r['n_verified']:>9}{(pages if pages is not None else '?'):>6}\"\n f\" {(f'{ratio:.4f}' if ratio is not None else '?'):>10} {flag} {src if pages is not None else ''} {r['c'].get('note', '')[:30]}\")\n if len(rows) > args.limit:\n print(f\" … {len(rows) - args.limit} more (raise --limit)\")\n if not out and not quiet:\n print(\"No candidates: run `board.py add` or `board.py children` first\")\n return out\n\n\ndef _corridor_pending(b: dict) -> list[str]:\n \"\"\"Infrastructure observations without a computed result explicitly addressing them.\"\"\"\n infra = [k for k, c in b[\"clues\"].items() if _clue_kind(c[\"kind\"]) == \"infra\" and c[\"status\"] in (\"observed\", \"read\")]\n done = {k for c in b[\"clues\"].values()\n if c[\"status\"] == \"computed\" and c.get(\"resolves\") and c.get(\"file\") and Path(c[\"file\"]).is_file()\n for k in c.get(\"resolves\", [])}\n return [k for k in infra if k not in done]\n\n\ndef cmd_next(args, p: Path) -> None:\n b = _load(p)\n pend = _corridor_pending(b)\n if pend:\n print(f\"→ First: infrastructure clue {', '.join(pend)} has no linked corridor result yet. `osm.py geom '\u003cfilter>' --bbox \u003cregion> --out lines.geojson` → \"\n f\"`terrain.py scan --lines lines.geojson` (mountains fill the frame: add `--flat-run 0 --min-low-deg 0`) → `terrain.py fit`; \"\n f\"two or three kinds of infrastructure: `osm.py near`. Record a result that actually tests the clue with \"\n f\"`board.py clue '\u003cresult>' --kind corridor --status computed --file \u003coutput> --resolves \u003cclue-id>` \"\n f\"(repeat --resolves for each clue tested).\")\n lv = _frontier(b)\n if not lv:\n print(\"No candidates. First do step 2 (lookup clues) or step 4 (coarse location from the environment) and list the candidates in full (board.py children)\")\n return\n rows = [r for r in _scores(b, lv, args.prior_by) if not r[\"excluded\"]]\n top = rows[0]\n second = rows[1] if len(rows) > 1 else None\n unused = [k for k, c in b[\"clues\"].items() if not c[\"used\"]]\n print(f\"Current frontier: {LEVEL_LABEL[lv]}, {len(rows)} not excluded; first {top['name']} {top['share']:.0%}\"\n + (f\", second {second['name']} {second['share']:.0%}\" if second else \"\"))\n if unused:\n pending = \", \".join(f\"{k}({b['clues'][k]['kind']})\" for k in unused)\n print(f\"Clues not used yet: {pending} → evidence or apply first\")\n separable = (not second) or (top[\"share\"] >= 0.7 and top[\"share\"] / max(second[\"share\"], 1e-9) >= 3)\n if separable and top[\"n_verified\"] == 0 and second:\n print(f\"{top['name']} leads but has no verified evidence at all (read/computed); it's built from inferences only: do one cheap verification before scanning.\")\n if separable:\n pages, src = _cost_of(top[\"c\"], args)\n print(f\"→ Ready to narrow/scan: {top['name']} ({src}, about {pages if pages is not None else '?'} pages).\")\n if src == \"whole-district bbox\":\n print(\" First `board.py urban \u003cname>` or `scan-bbox` to shrink the extent to the built-up area; pages drop by an order of magnitude.\")\n elif pages is None:\n print(f\" It has no extent yet: `board.py urban {top['name']} --within \u003cparent>` or `board.py scan-bbox {top['name']} --bbox s,w,n,e`, otherwise pages can't be computed.\")\n if top[\"name\"] not in b[\"falsify\"]:\n print(f\" Write the falsification condition before scanning: `board.py falsify {top['name']} --text \\\"…\\\"`\")\n print(\" Let the machine rank the scan first: `sat_scan.py grid --bbox \u003cscan_bbox> --preset …`, `osm.py buildings`, `poi.py`; look only at the top 20–30.\")\n return\n print(\"→ Can't separate. Do discriminating tests from cheap to expensive, each on all non-excluded candidates together, not just the leader:\")\n kinds_have = {c[\"kind\"] for c in b[\"clues\"].values()}\n pk = _region_of(b, rows)\n tips = (pk or {}).get(\"tips\") or {}\n if pk:\n print(f\" (region pack {pk['code']}: `regions.py show {pk['code'].lower()}` lists its lookups, services and clue index)\")\n for kind, tip in [(k, tips.get(k, t)) for k, t in CHEAP_OPS]:\n mark = \"(you already have a clue of this kind; record evidence)\" if any(k in kinds_have for k in kind.split(\"/\")) else \"\"\n print(f\" - {kind}: {tip} {mark}\")\n print(\"→ All cheap tests done and still can't separate: don't stop. Scan in \\\"share ÷ pages\\\" order, finish the small ones first, set a page cap on the large ones:\")\n order = []\n for r in rows:\n pages, src = _cost_of(r[\"c\"], args)\n order.append((r[\"share\"] / pages if pages else 0, r, pages, src))\n order.sort(key=lambda t: -t[0])\n for ratio, r, pages, src in order[:8]:\n print(f\" {r['name']:\u003c14} share {r['share']:.0%} pages {pages if pages is not None else '?'} ({src}) share/page {ratio:.4f}\")\n if any(pages is None for _, _, pages, _ in order):\n print(\" Some candidates have no extent: fill it in with `board.py urban` or `scan-bbox`, otherwise they can't be ordered\")\n\n\ndef _region_of(b: dict, rows: list[dict]) -> dict | None:\n \"\"\"The board's country, else the one country all open frontier candidates share.\"\"\"\n if b.get(\"country\"):\n return regions.resolve(b[\"country\"])\n codes = {r[\"c\"].get(\"country\") for r in rows}\n return regions.resolve(codes.pop()) if len(codes) == 1 and None not in codes else None\n\n\ndef _parse_covers(s: str) -> list[tuple[float, float]]:\n \"\"\"--covers: 'lat,lon' or 'lat,lon:lat,lon' (the point/span the evidence actually covers). Returns [] if no coordinates parse.\"\"\"\n pts = []\n for part in str(s).split(\":\"):\n m = re.findall(r\"-?\\d+\\.\\d+|-?\\d+\", part)\n if len(m) >= 2:\n pts.append((float(m[0]), float(m[1])))\n return pts\n\n\ndef _cov_span_m(a: tuple[float, float], b: tuple[float, float]) -> float:\n kx = 111320.0 * math.cos(math.radians((a[0] + b[0]) / 2))\n return math.hypot((b[1] - a[1]) * kx, (b[0] - a[0]) * 110540.0)\n\n\ndef _covers_ratio(c: dict, pts: list[tuple[float, float]]) -> tuple[float, float] | None:\n \"\"\"(evidence coverage length / candidate extent diagonal, diagonal in meters). None if the candidate has no bbox (can't compute, only warn).\"\"\"\n bb = c.get(\"bbox\") or c.get(\"scan_bbox\")\n if not bb or len(bb) != 4:\n return None\n try:\n sw, ww, nn, ee = (float(v) for v in bb)\n except (TypeError, ValueError):\n return None\n diag = _cov_span_m((sw, ww), (nn, ee))\n if diag \u003c= 1.0:\n return None\n cov = _cov_span_m(pts[0], pts[-1]) if len(pts) >= 2 else 0.0\n return cov / diag, diag\n\n\ndef _unseen(rows: list[dict], lv: str) -> list[str]:\n \"\"\"Non-excluded candidates at fine levels (area/road/point) without a single piece of evidence. Coarse levels are scanned in share/pages order, where no evidence is normal, so they don't count.\"\"\"\n if lv not in (\"area\", \"road\", \"point\"):\n return []\n return [r[\"name\"] for r in rows if r[\"n_ev\"] == 0 and not r[\"excluded\"]]\n\n\ndef _center_of(c: dict) -> tuple[float, float] | None:\n if c.get(\"center\"):\n return float(c[\"center\"][0]), float(c[\"center\"][1])\n bb = c.get(\"bbox\")\n return ((bb[0] + bb[2]) / 2, (bb[1] + bb[3]) / 2) if bb else None\n\n\ndef _inside(c: dict, lat: float, lon: float, pad_m: float = 30) -> bool | None:\n \"\"\"Is (lat, lon) inside the candidate's bbox (padded)? None when the candidate has no bbox.\"\"\"\n bb = c.get(\"bbox\")\n if not bb:\n return None\n dy, dx = pad_m / 110574, pad_m / (111320 * math.cos(math.radians(lat)))\n return bb[0] - dy \u003c= lat \u003c= bb[2] + dy and bb[1] - dx \u003c= lon \u003c= bb[3] + dx\n\n\ndef _skipped_level(b: dict, name: str) -> tuple[str, list[str], str | None] | None:\n \"\"\"A candidate that skips a level its siblings use: a point added under a city that also has road candidates.\n Its evidence then never reaches that level's ranking, which goes on ranking the other roads.\n Returns (skipped level, the siblings at that level, the one whose bbox contains it or None).\"\"\"\n c = b[\"candidates\"][name]\n li = LEVELS.index(c[\"level\"])\n for lv in reversed(LEVELS[:li]):\n sibs = [n for n, s in b[\"candidates\"].items() if s[\"level\"] == lv and s.get(\"parent\") == c.get(\"parent\")\n and n != c.get(\"parent\") and s.get(\"status\") != \"excluded\"]\n if not sibs:\n continue\n at = _center_of(c)\n home = next((n for n in sibs if at and _inside(b[\"candidates\"][n], *at)), None)\n return lv, sibs, home\n return None\n\n\ndef _skip_hint(b: dict, name: str) -> str | None:\n gap = _skipped_level(b, name)\n if not gap:\n return None\n lv, sibs, home = gap\n c = b[\"candidates\"][name]\n where = (f\"it lies inside {home} → `board.py move '{name}' --parent '{home}'`\" if home else\n f\"none of them contains it, so that ranking leaves the answer out → `board.py add \u003cits {lv}> --level {lv} --parent '{c.get('parent')}' --bbox s,w,n,e`, \"\n f\"then `board.py move '{name}' --parent \u003cit>`\")\n return (f\"{name} ({c['level']}) sits directly under {c.get('parent')}, which also has {len(sibs)} {lv} candidates; \"\n f\"its evidence never reaches the {lv} ranking: {where}\")\n\n\ndef cmd_check(args, p: Path) -> None:\n b = _load(p)\n ok = True\n print(\"Pre-conclusion check:\")\n for name, c in b[\"candidates\"].items():\n if c.get(\"status\") == \"excluded\":\n ex = c.get(\"excluded_by\", {})\n if not ex.get(\"computed\") or not Path(ex[\"computed\"]).exists():\n ok = False\n print(f\" FAIL file for excluding {name} doesn't exist: {ex.get('computed')}\")\n for name, c in b[\"candidates\"].items():\n if c.get(\"status\") != \"excluded\" or c[\"level\"] not in EXTENDED_LEVELS:\n continue\n ex = c.get(\"excluded_by\", {})\n pts = _parse_covers(ex.get(\"covers\", \"\"))\n if not pts:\n ok = False\n print(f\" FAIL excluding {name} ({LEVEL_LABEL[c['level']]} level) has no --covers recorded: no way to judge how much the evidence covered, \"\n f\"possibly one point standing in for the whole area → re-run exclude with --covers, or switch to evidence --against to down-weight\")\n continue\n r = _covers_ratio(c, pts)\n if r is None:\n print(f\" NOTE evidence for excluding {name} covers {ex['covers']}; the candidate has no bbox, so the coverage ratio can't be computed → \"\n f\"state in the conclusion that only this stretch was checked\")\n elif r[0] \u003c COVERS_MIN:\n ok = False\n print(f\" FAIL evidence for excluding {name} covers only about {r[0]:.0%} (candidate extent diagonal {r[1]:.0f} m): exclusion scope exceeds evidence scope\")\n for name, c in b[\"candidates\"].items():\n if c.get(\"status\") != \"excluded\" and any(e[\"candidate\"] == name and e[\"lr\"] > 1 for e in b[\"evidence\"]):\n hint = _skip_hint(b, name)\n if hint:\n ok = False\n print(f\" FAIL {hint}\")\n lv = _frontier(b)\n if lv:\n rows = [r for r in _scores(b, lv, args.prior_by) if not r[\"excluded\"]]\n top = rows[0]\n if top[\"n_verified\"] == 0:\n ok = False\n print(f\" FAIL main answer {top['name']} has no read/computed evidence, only observations and inferences: you can't claim this level or finer\")\n else:\n print(f\" ok main answer {top['name']}: {top['n_ev']} pieces of evidence, {top['n_verified']} verified\")\n weak = [r[\"name\"] for r in rows[1:] if r[\"share\"] >= 0.15]\n if weak:\n print(f\" WARN alternatives with share ≥15% remain: {', '.join(weak)} → write them into alternatives with a discriminating test; no midpoint\")\n unseen = _unseen(rows, lv)\n if unseen:\n print(f\" WARN {len(unseen)}/{len(rows)} {LEVEL_LABEL[lv]} candidates have no evidence at all, i.e. were never looked at: \"\n f\"{', '.join(unseen[:10])}{' …' if len(unseen) > 10 else ''} → look at each; record evidence --against even when it doesn't match; \"\n f\"unseen ones don't count as excluded; state this in the conclusion\")\n down = [(e[\"candidate\"], e[\"clue\"]) for e in b[\"evidence\"]\n if e[\"lr\"] \u003c 1 and b[\"clues\"][e[\"clue\"]][\"status\"] in (\"inferred\", \"observed\")\n and b[\"candidates\"][e[\"candidate\"]].get(\"status\") != \"excluded\"]\n if down:\n names = sorted({n for n, _ in down})\n print(f\" NOTE candidates down-weighted by inference but not excluded: {', '.join(names[:10])} → when no candidate matches, look back at these first\")\n for k, c in b[\"clues\"].items():\n if c[\"status\"] in (\"read\", \"computed\") and not c.get(\"file\"):\n print(f\" WARN {k} is {c['status']} but has no file: text read needs a zoomed image, computed results need an output file\")\n # fine-level answer needs the evidence image the Output section asks for\n fine = [n for n, c in b[\"candidates\"].items() if c[\"level\"] in (\"area\", \"road\", \"point\") and c.get(\"status\") != \"excluded\"\n and any(e[\"candidate\"] == n and e[\"lr\"] > 1 for e in b[\"evidence\"])]\n if fine:\n ev = Path(args.evidence) if args.evidence else p.parent / \"evidence.jpg\"\n if not ev.exists():\n ok = False\n print(f\" FAIL evidence image {ev} doesn't exist: supporting evidence reaches area/road/point level ({', '.join(fine[:3])}) → make it with evidence.py \"\n f\"(camera + heading wedge on satellite, comparison panels), or pass --evidence \u003cpath>\")\n else:\n print(f\" ok evidence image {ev}\")\n unused = [k for k, c in b[\"clues\"].items() if not c[\"used\"]]\n if unused:\n print(f\" NOTE unused clues: {', '.join(unused)} → write them into unused_clues\")\n for name in [n for n, c in b[\"candidates\"].items() if c.get(\"status\") != \"excluded\"]:\n if b[\"falsify\"].get(name):\n print(f\" ok {name} falsification condition: {'; '.join(b['falsify'][name])}\")\n print(\"Result: \" + (\"pass\" if ok else \"has FAIL, fix those first\"))\n\n\ndef cmd_report(args, p: Path) -> None:\n b = _load(p)\n lv = _frontier(b)\n rep = {\"candidate_levels\": {}, \"main\": None, \"alternatives\": [], \"excluded\": [], \"unused_clues\": [],\n \"downweighted_not_excluded\": [], \"falsify\": b[\"falsify\"]}\n for l2 in LEVELS:\n rows = _scores(b, l2, args.prior_by)\n if rows:\n rep[\"candidate_levels\"][l2] = [{\"name\": r[\"name\"], \"share\": round(r[\"share\"], 3), \"evidence\": r[\"n_ev\"],\n \"verified\": r[\"n_verified\"], \"excluded\": r[\"excluded\"]} for r in rows]\n if lv:\n rows = [r for r in _scores(b, lv, args.prior_by) if not r[\"excluded\"]]\n top = rows[0]\n rep[\"main\"] = {\"level\": lv, \"name\": top[\"name\"], \"share\": round(top[\"share\"], 3),\n \"evidence\": [{\"clue\": b[\"clues\"][e[\"clue\"]][\"text\"], \"status\": b[\"clues\"][e[\"clue\"]][\"status\"],\n \"lr\": e[\"lr\"], \"why\": e[\"why\"], \"file\": e[\"file\"], **({\"via\": e[\"via\"]} if e.get(\"via\") else {})}\n for e in top[\"ev\"]]}\n for r in rows[1:]:\n if r[\"share\"] >= 0.05:\n rep[\"alternatives\"].append({\"name\": r[\"name\"], \"share\": round(r[\"share\"], 3),\n \"how_to_separate\": \"run one cheap test on both together (terrain/livery/municipal fixtures/water template), or scan the first 3 pages of each built-up area\"})\n rep[\"unexamined\"] = _unseen(rows, lv)\n rep[\"level_gaps\"] = [h for n, c in b[\"candidates\"].items() if c.get(\"status\") != \"excluded\"\n and any(e[\"candidate\"] == n and e[\"lr\"] > 1 for e in b[\"evidence\"]) for h in [_skip_hint(b, n)] if h]\n for name, c in b[\"candidates\"].items():\n if c.get(\"status\") == \"excluded\":\n ex = c.get(\"excluded_by\", {})\n rep[\"excluded\"].append({\"name\": name, \"clue\": b[\"clues\"].get(ex.get(\"clue\"), {}).get(\"text\", \"\"),\n \"computed\": ex.get(\"computed\"), \"why\": ex.get(\"why\", \"\")})\n rep[\"unused_clues\"] = [c[\"text\"] for c in b[\"clues\"].values() if not c[\"used\"]]\n rep[\"downweighted_not_excluded\"] = sorted({e[\"candidate\"] for e in b[\"evidence\"] if e[\"lr\"] \u003c 1\n and b[\"clues\"][e[\"clue\"]][\"status\"] in (\"inferred\", \"observed\")\n and b[\"candidates\"][e[\"candidate\"]].get(\"status\") != \"excluded\"})\n if args.merge:\n mp = Path(args.merge)\n base = json.loads(mp.read_text(encoding=\"utf-8\")) if mp.exists() else {}\n if rep[\"main\"] and isinstance(base.get(\"lat\"), (int, float)) and isinstance(base.get(\"lon\"), (int, float)):\n inside = _inside(b[\"candidates\"][rep[\"main\"][\"name\"]], base[\"lat\"], base[\"lon\"], pad_m=max(30, float(base.get(\"radius_m\") or 0)))\n if inside is False:\n rep[\"main_mismatch\"] = (f\"result.json lat/lon is outside the board's main answer {rep['main']['name']} ({rep['main']['level']}): \"\n f\"the answer's candidate isn't on that level, or the board ranking disagrees with the conclusion\")\n base[\"board\"] = rep\n if rep[\"alternatives\"]:\n base.setdefault(\"alternatives\", [])\n base[\"alternatives\"] = [f\"{a['name']} (share {a['share']:.0%}): {a['how_to_separate']}\" for a in rep[\"alternatives\"]] + \\\n [x for x in base.get(\"alternatives\", []) if not isinstance(x, str) or \"(share \" not in x]\n base[\"excluded\"] = [f\"{x['name']}: {x['why']} ({x['computed']})\" for x in rep[\"excluded\"]] or base.get(\"excluded\", [])\n base[\"unused_clues\"] = rep[\"unused_clues\"] or base.get(\"unused_clues\", [])\n mp.write_text(json.dumps(base, ensure_ascii=False, indent=1), encoding=\"utf-8\")\n print(f\"Merged into {mp} (board field + alternatives/excluded/unused_clues)\")\n print(json.dumps(rep, ensure_ascii=False, indent=1)[:4000])\n for w in rep[\"level_gaps\"] + ([rep[\"main_mismatch\"]] if rep.get(\"main_mismatch\") else []):\n print(f\"WARN {w}\")\n if rep[\"main\"] and rep[\"alternatives\"]:\n print(\"\\nReminder: main answer = first place, alternatives listed separately; no midpoint, no big circle.\")\n\n\nSUB_LEVELS = {\"admin2\", \"city\", \"district\"} # children may record a prefecture as district; these count as one tier for name matching\n\n\ndef _same_tier(a: str, b: str) -> bool:\n \"\"\"Levels at which two same-named places could be one place (吉林省 and its 吉林市 share a stem, not a level).\"\"\"\n return a == b or (a in SUB_LEVELS and b in SUB_LEVELS)\n\n\ndef _slot(b: dict, name: str, parent: str | None, cid: str | None, level: str) -> str | None:\n \"\"\"Key for a new candidate, or None when the board already has this place. A same-name place under another parent\n (Gulou District in Nanjing and in Fuzhou) is a different candidate and gets its parent in the key.\"\"\"\n if name in b[\"candidates\"] and not _same_tier(b[\"candidates\"][name][\"level\"], level):\n return f\"{name} ({level})\" if f\"{name} ({level})\" not in b[\"candidates\"] else None\n same = [k for k in b[\"candidates\"] if _norm(k.split(\" (\")[0]) == _norm(name) and _same_tier(b[\"candidates\"][k][\"level\"], level)]\n if not same:\n return name\n for k in same:\n c = b[\"candidates\"][k]\n if cid and c.get(\"cid\"):\n if c[\"cid\"] == cid:\n return None\n continue\n if not parent or not c.get(\"parent\") or _norm(c[\"parent\"]) == _norm(parent):\n return None\n key = f\"{name} ({parent})\"\n return None if key in b[\"candidates\"] else key\n\n\ndef _same_place(c: dict, name: str, parent: str | None, code: str | None, cid: str | None) -> bool:\n \"\"\"Is board candidate c the place a lookup names? Same stable id; or same name with no conflicting id, country or parent.\"\"\"\n if cid and c.get(\"cid\"):\n return c[\"cid\"] == cid\n if code and c.get(\"country\") and c[\"country\"] != code:\n return False\n if parent and c.get(\"parent\") and _norm(c[\"parent\"]) != _norm(parent):\n return False\n return True\n\n\ndef _ensure(b: dict, level: str, name: str, parent: str | None, code: str | None, cid: str | None, why: str) -> str:\n \"\"\"Candidate key for this place, creating it if needed. Same-name places elsewhere get their parent in the key, e.g. 朝阳区 (长春市).\"\"\"\n for k, c in b[\"candidates\"].items():\n if cid and c.get(\"cid\") == cid:\n return k\n for k, c in b[\"candidates\"].items():\n if _norm(k.split(\" (\")[0]) == _norm(name) and _same_tier(c[\"level\"], level) and _same_place(c, name, parent, code, cid):\n if cid and not c.get(\"cid\"):\n c[\"cid\"] = cid\n if code and not c.get(\"country\"):\n c[\"country\"] = code\n return k\n key = name\n if key in b[\"candidates\"]:\n key = f\"{name} ({parent or code})\"\n b[\"candidates\"][key] = {\"level\": level, \"parent\": parent, \"bbox\": None, \"scan_bbox\": None, \"prior\": 1.0, \"status\": \"open\",\n \"note\": why, \"country\": code, \"cid\": cid}\n return key\n\n\ndef _country_key(b: dict, m: dict) -> str | None:\n \"\"\"Existing country-level candidate for a region match (by pack code or any of the pack's names), if any.\"\"\"\n for k, c in b[\"candidates\"].items():\n if c[\"level\"] != \"country\":\n continue\n if c.get(\"country\") == m[\"country_code\"]:\n return k\n p = regions.resolve(k.split(\" (\")[0])\n if p and p[\"code\"] == m[\"country_code\"]:\n c[\"country\"] = m[\"country_code\"]\n return k\n return None\n\n\ndef cmd_apply(args, p: Path) -> None:\n b = _load(p)\n cl = HERE / \"clues.py\"\n country = args.country or b.get(\"country\")\n # The country context only narrows region-pack lookups. Global kinds read --country as an observation target\n # (driving-side --country X = \"which side does X drive on\"), so passing the context there would overwrite the value.\n is_global = args.kind in GLOBAL_KINDS\n cmd = [UV, \"run\", str(cl), \"lookup\", args.kind, args.value, \"--json\"] + ([\"--country\", country] if country and not is_global else [])\n r = _run(cmd)\n if r.returncode != 0:\n sys.exit(f\"clues.py failed: {(r.stderr or r.stdout).strip()[-400:]}\")\n try:\n d = json.loads(r.stdout[r.stdout.index(\"{\"):])\n except Exception: # noqa: BLE001\n sys.exit(f\"clues.py output isn't JSON: {r.stdout[:300]}\")\n matches = d.get(\"matches\") or []\n kid = args.clue\n if not kid:\n kid = f\"K{len(b['clues']) + 1}\"\n b[\"clues\"][kid] = {\"text\": f\"{args.kind} {args.value}\", \"kind\": args.kind, \"status\": \"read\",\n \"file\": args.file or \"\", \"source\": d.get(\"source\", \"\"), \"used\": False}\n if not matches:\n what = \"Lookup not supported\" if d.get(\"status\") == \"unsupported\" else \"Lookup returned nothing\"\n print(f\"{what}: {d.get('note', '')}. Clue {kid} recorded, no evidence added.\")\n _save(p, b)\n return\n lr = args.lr\n touched: dict[tuple[str, str], None] = {}\n why = f\"lookup {args.kind}={args.value}\"\n\n def hit(lv: str, cname: str) -> None:\n lv = b[\"candidates\"][cname][\"level\"] # an existing candidate keeps its level (children may have recorded a prefecture as district)\n if (lv, cname) in touched: # one piece of evidence per place, however many table rows name it\n return\n touched[(lv, cname)] = None\n _put_evidence(b, {\"clue\": kid, \"candidate\": cname, \"lr\": lr,\n \"why\": why, \"file\": args.file or \"\", \"cmd\": f\"clues.py lookup {args.kind} {args.value}\"})\n\n for m in matches:\n code = m.get(\"country_code\")\n if not code: # global tables: country level only\n if m.get(\"country\"):\n hit(\"country\", _ensure(b, \"country\", m[\"country\"], None, None, None, f\"from lookup {args.kind}\"))\n continue\n ck = _country_key(b, m)\n if ck: # only when countries are already being compared; a single-country board isn't given a country row\n hit(\"country\", ck)\n parent = None\n for lv in (\"admin1\", \"admin2\", \"city\", \"district\"):\n name = m.get(lv)\n if not name:\n continue\n hit(lv, _ensure(b, lv, name, parent, code, regions.cid(code, m, upto=lv), f\"from lookup {args.kind}\"))\n parent = name\n # Finest place each match names: candidates inside it (a district of a pointed-to city) aren't \"elsewhere\"\n leaves = {_norm(next((m[lv] for lv in (\"district\", \"city\", \"admin2\", \"admin1\", \"country\") if m.get(lv)), \"\")) for m in matches}\n # Other non-excluded candidates at the same level: against, but not excluded (lookups have exceptions too: out-of-town vehicles,\n # headquarters phone numbers). Only candidates the lookup could have pointed to: a country whose pack has no such table\n # (or isn't installed) was never consulted, so \"not pointed to\" says nothing about it.\n searched = set(d.get(\"searched\") or [])\n ctx = (regions.resolve(country) or {}).get(\"code\") if country else None\n\n def covered(k: str, c: dict) -> bool:\n if is_global:\n return True\n code = c.get(\"country\")\n if c[\"level\"] == \"country\":\n # A country row is its own country: the context mustn't stand in for one we can't identify (India with no pack)\n code = code or (regions.resolve(k.split(\" (\")[0]) or {}).get(\"code\")\n return code in searched\n if code or ctx:\n return (code or ctx) in searched\n # Hand-added candidate of unknown country: covered if a searched pack's admin table knows the name\n return any(pk[\"code\"] in searched for pk, _ in regions.admin_find(k.split(\" (\")[0]))\n\n for lv in {lv for lv, _ in touched}:\n for k, c in b[\"candidates\"].items():\n if c[\"level\"] == lv and (lv, k) not in touched and c.get(\"status\") != \"excluded\" and _norm(c.get(\"parent\") or \"\") not in leaves \\\n and covered(k, c):\n _put_evidence(b, {\"clue\": kid, \"candidate\": k, \"lr\": 1 / lr,\n \"why\": f\"lookup {args.kind}={args.value} doesn't point here\", \"file\": \"\", \"cmd\": \"\"})\n b[\"clues\"][kid][\"used\"] = True\n _log(b, f\"apply {args.kind}={args.value} → {[n for _, n in touched]}\")\n _save(p, b)\n countries = sorted({m.get(\"country_code\") for m in matches if m.get(\"country_code\")})\n print(f\"{kid}: {args.kind}={args.value} → {', '.join(n for _, n in sorted(touched))} (likelihood ratio {lr:g}; others at the same level 1/{lr:g}, not excluded)\"\n + (f\"\\n Matches in several countries ({', '.join(countries)}): all kept as hypotheses. If the country is known, `board.py country \u003ccc>` and re-apply.\" if len(countries) > 1 else \"\"))\n if d.get(\"note\") and len(countries) \u003c= 1:\n print(f\" Note: {d['note'][:300]}\")\n\n\ndef cmd_log(args, p: Path) -> None:\n b = _load(p)\n print(\"\\n\".join(b.get(\"log\", [])[-args.n:]))\n print(f\"\\n{len(b['clues'])} clues, {len(b['evidence'])} pieces of evidence, {len(b['candidates'])} candidates\")\n\n\ndef main() -> None:\n ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)\n ap.add_argument(\"--board\", type=Path, default=Path(\"board.json\"))\n sub = ap.add_subparsers(dest=\"cmd\", required=True)\n\n def rank_opts(sp):\n sp.add_argument(\"--prior-by\", choices=[\"none\", \"area\"], default=\"none\", help=\"prior: uniform (default) or by bbox area. There is no by-population option\")\n sp.add_argument(\"--zoom\", type=int, default=16, help=\"scan pages are computed at this zoom (sports field overview z16, factories z16, buildings z17)\")\n sp.add_argument(\"--cell\", type=int, default=320)\n sp.add_argument(\"--cols\", type=int, default=5)\n\n i = sub.add_parser(\"init\")\n i.add_argument(\"--photo\", required=True)\n i.add_argument(\"--case\")\n i.add_argument(\"--force\", action=\"store_true\")\n i.add_argument(\"--country\", help=\"country context when it's already known (code or name): region-pack lookups use only that pack\")\n\n co = sub.add_parser(\"country\", help=\"set the country context (code or name; 'none' clears it)\")\n co.add_argument(\"code\")\n\n a = sub.add_parser(\"add\")\n a.add_argument(\"names\", nargs=\"*\")\n a.add_argument(\"--from\", dest=\"from_\", help=\"bulk import: JSON from poi.py --out / gazetteer.py --out, or GeoJSON from osm.py geom\")\n a.add_argument(\"--radius\", type=float, default=500, help=\"when --from has only point coordinates, the candidate extent is this many meters around the point (default 500)\")\n a.add_argument(\"--level\", required=True, help=f\"{'/'.join(LEVELS)}\")\n a.add_argument(\"--parent\")\n a.add_argument(\"--bbox\", help=\"s,w,n,e\")\n a.add_argument(\"--prior\", type=float, default=1.0)\n a.add_argument(\"--note\")\n\n ch = sub.add_parser(\"children\", help=\"list every subordinate admin area with gazetteer.py and add them as candidates\")\n ch.add_argument(\"parent\")\n ch.add_argument(\"--level\", type=int, help=\"OSM admin_level (automatic if omitted)\")\n ch.add_argument(\"--within\")\n ch.add_argument(\"--as-level\", help=f\"level to record the candidates as: {'/'.join(LEVELS)}. Automatic if omitted: parent is a country → admin1, \"\n \"parent already on the candidate board → its next level, otherwise district\")\n ch.add_argument(\"--proxy\", default=os.environ.get(\"GEO_PROXY\"), help=PROXY_HELP)\n\n c = sub.add_parser(\"clue\")\n c.add_argument(\"text\")\n c.add_argument(\"--kind\", required=True, help=\"plate/area-code/text/livery/terrain/sun/infra/vegetation/network/municipal/ip/hint/…; infrastructure is an alias for infra\")\n c.add_argument(\"--status\", required=True, help=\"/\".join(STATUS))\n c.add_argument(\"--file\", help=\"zoomed image, script output\")\n c.add_argument(\"--source\", help=\"where in the image, who said it\")\n c.add_argument(\"--resolves\", action=\"append\", metavar=\"CLUE_ID\",\n help=\"earlier clue actually tested by this computed result; repeatable, requires --status computed and an existing --file\")\n\n e = sub.add_parser(\"evidence\")\n e.add_argument(\"--clue\", required=True)\n e.add_argument(\"--for\", dest=\"for_\", action=\"append\", help=\"name:likelihood_ratio (>1), repeatable; name alone defaults to 3\")\n e.add_argument(\"--against\", action=\"append\", help=\"name:likelihood_ratio (\u003c1), repeatable; name alone defaults to 1/3\")\n e.add_argument(\"--why\")\n e.add_argument(\"--file\")\n e.add_argument(\"--command\", dest=\"command_\", help=\"the command that produced this evidence\")\n\n x = sub.add_parser(\"exclude\")\n x.add_argument(\"name\")\n x.add_argument(\"--clue\", required=True)\n x.add_argument(\"--computed\", required=True, help=\"computed file (must exist)\")\n x.add_argument(\"--covers\", help=\"where the evidence actually covers: 'lat,lon' or 'lat,lon:lat,lon'; required for district/area/road candidates\")\n x.add_argument(\"--why\")\n\n sb = sub.add_parser(\"scan-bbox\")\n sb.add_argument(\"name\")\n sb.add_argument(\"--bbox\", required=True)\n\n u = sub.add_parser(\"urban\")\n u.add_argument(\"name\")\n u.add_argument(\"--within\")\n u.add_argument(\"--proxy\", default=os.environ.get(\"GEO_PROXY\"), help=PROXY_HELP)\n\n f = sub.add_parser(\"falsify\")\n f.add_argument(\"name\")\n f.add_argument(\"--text\", required=True)\n\n r = sub.add_parser(\"rank\")\n rank_opts(r)\n r.add_argument(\"--limit\", type=int, default=15)\n\n n = sub.add_parser(\"next\")\n rank_opts(n)\n\n k = sub.add_parser(\"check\")\n rank_opts(k)\n k.add_argument(\"--evidence\", help=\"evidence image path (default: evidence.jpg next to board.json)\")\n\n rp = sub.add_parser(\"report\")\n rank_opts(rp)\n rp.add_argument(\"--merge\", help=\"merge into an existing result.json\")\n\n ap_ = sub.add_parser(\"apply\", help=\"lookup clues add candidates and evidence automatically (clues.py)\")\n ap_.add_argument(\"--kind\", required=True, help=\"calling-code/driving-side/territories, or a region-pack kind (plate, area-code, …: `regions.py list`)\")\n ap_.add_argument(\"--country\", help=\"restrict region-pack lookups to this country (default: the board's country, else every pack)\")\n ap_.add_argument(\"--value\", required=True)\n ap_.add_argument(\"--clue\", help=\"id of an already recorded clue; if omitted, a new read clue is created\")\n ap_.add_argument(\"--file\", help=\"zoomed image the text was read from\")\n ap_.add_argument(\"--lr\", type=float, default=20.0)\n\n mv = sub.add_parser(\"move\", help=\"put a candidate under another candidate (e.g. a point under its road); evidence stays with it\")\n mv.add_argument(\"name\")\n mv.add_argument(\"--parent\", required=True)\n\n lg = sub.add_parser(\"log\")\n lg.add_argument(\"-n\", type=int, default=40)\n\n args = ap.parse_args()\n p = args.board\n fn = {\"init\": cmd_init, \"add\": cmd_add, \"children\": cmd_children, \"clue\": cmd_clue, \"evidence\": cmd_evidence,\n \"exclude\": cmd_exclude, \"scan-bbox\": cmd_scan_bbox, \"urban\": cmd_urban, \"falsify\": cmd_falsify,\n \"rank\": cmd_rank, \"next\": cmd_next, \"check\": cmd_check, \"report\": cmd_report, \"apply\": cmd_apply, \"log\": cmd_log,\n \"country\": cmd_country, \"move\": cmd_move}\n fn[args.cmd](args, p)\n\n\nif __name__ == \"__main__\":\n # Chinese Windows outputs GBK by default: it crashes on m², ñ, and Chinese text the agent reads comes out garbled\n sys.stdout.reconfigure(encoding=\"utf-8\")\n sys.stderr.reconfigure(encoding=\"utf-8\", errors=\"backslashreplace\")\n main()\n","content_type":"text/x-python; charset=utf-8","language":"python","size":66285,"content_sha256":"e3109b74beef931b209bd3ce59cb1b6457b600f69ccde6673ce66808f87f2866"},{"filename":"scripts/clues.py","content":"#!/usr/bin/env python3\n# /// script\n# requires-python = \">=3.10\"\n# dependencies = []\n# ///\n\"\"\"Lookup-table clues. Tables are local JSON; lookups don't go online.\n\nGlobal tables (data/):\n lookup calling-code +594 international calling code → country/region\n lookup driving-side left countries that drive on the left; `driving-side --country Japan` → left\n lookup territories France overseas territories/dependencies of France; `--continent \"South America\"` lists only that continent\nRegion-pack tables (regions/\u003ccc>/, see `regions.py list`):\n lookup \u003ckind> \u003cvalue> [--country cc] kinds the packs declare, e.g. plate, area-code; without --country every pack with\n that kind is searched and matches from several countries are all returned; with --country only\n that pack is used, and a pack that lacks the kind (or isn't installed) answers \"unsupported\"\n lookup admin \u003cname> [--country cc] parent chain from a pack's admin table; `admin --children \u003cname>` lists the children\n list global tables and installed packs with entry counts and fetch dates\n\n--json output follows one contract (used by board.py apply):\n {\"kind\", \"value\", \"status\": ok|no_match|unsupported|error (error = that country's pack is broken), \"matches\": [...], \"searched\": [pack codes], \"source\", \"table_fetched\", \"note\"}\n region matches: {\"country\", \"country_code\", \"admin1\", \"admin2\"?, \"city\"?, \"district\"?, \"note\", \"unverified\"?} — levels are explicit\n global matches: {\"country\", \"continent\"?, \"subregion\"?, \"side\"?, \"utc\"?, \"note\"}\n\nExamples:\n clues.py lookup plate \u003cplate prefix> --country cn\n clues.py lookup area-code 0817 --json\n clues.py lookup calling-code +594\n clues.py lookup territories France --continent \"South America\"\nTables are refreshed by maintainers with refresh.py.\n\"\"\"\nfrom __future__ import annotations\n\nimport argparse\nimport json\nimport re\nimport sys\nfrom pathlib import Path\n\nsys.path.insert(0, str(Path(__file__).parent))\nimport regions # noqa: E402\n\nDATA = Path(__file__).parent.parent / \"data\"\nGLOBAL_TABLES = (\"calling_codes\", \"driving_side\", \"territories\", \"country_names\")\nGLOBAL_KINDS = (\"calling-code\", \"driving-side\", \"territories\")\n\nCONTINENT_ZH = {\"亚洲\": [\"Asia\"], \"欧洲\": [\"Europe\"], \"非洲\": [\"Africa\"], \"大洋洲\": [\"Oceania\"], \"北美洲\": [\"Northern America\", \"North America\"],\n \"南美洲\": [\"South America\"], \"美洲\": [\"Americas\"], \"加勒比\": [\"Caribbean\"], \"中美洲\": [\"Central America\"], \"南极洲\": [\"Antarctica\"]}\n\n# ---------------------------------------------------------------- read/write\n\ndef load(table: str) -> dict:\n f = DATA / f\"{table}.json\"\n if not f.exists():\n sys.exit(f\"{f} not found: reinstall the skill folder, or refetch with `refresh.py {table}`\")\n return json.loads(f.read_text(encoding=\"utf-8\"))\n\n\ndef _result(kind, value, matches, source, fetched, note=\"\", status=None):\n return {\"kind\": kind, \"value\": value, \"status\": status or (\"ok\" if matches else \"no_match\"), \"matches\": matches,\n \"searched\": [], \"source\": source, \"table_fetched\": fetched, \"note\": note}\n\ndef lookup_calling_code(value: str) -> dict:\n d = load(\"calling_codes\")\n src, fetched = d[\"_meta\"][\"source\"][0], d[\"_meta\"][\"fetched\"]\n digits = re.sub(r\"\\D\", \"\", value)\n if digits.startswith(\"00\"):\n digits = digits[2:]\n for L in (3, 2, 1):\n code = digits[:L]\n if code in d:\n hits = d[code]\n # shared codes like 1 and 7: look at the area code that follows\n subs = [k for k in d if k.startswith(code + \"-\") and digits[L:].startswith(k.split(\"-\")[1])]\n if subs:\n hits = [h for k in subs for h in d[k]]\n return _result(\"calling-code\", value, [{\"country\": h[\"country\"], \"utc\": h.get(\"utc\", \"\"), \"note\": \"\"} for h in hits], src, fetched)\n return _result(\"calling-code\", value, [], src, fetched, \"no matching country code\")\n\n\n_CN_NAMES: dict | None = None\n\n\ndef _country_en(name: str) -> str:\n \"\"\"Chinese name / alias → the English name the tables use, from data/country_names.json (300 countries + aliases).\"\"\"\n global _CN_NAMES\n n = name.strip()\n if _CN_NAMES is None:\n f = DATA / \"country_names.json\"\n _CN_NAMES = json.loads(f.read_text(encoding=\"utf-8\")) if f.exists() else {}\n aliases = _CN_NAMES.get(\"aliases\") or {}\n en2zh = _CN_NAMES.get(\"en2zh\") or {}\n if n in aliases:\n return aliases[n]\n for en, zh in en2zh.items():\n if zh == n:\n return en\n return n\n\n\ndef lookup_driving_side(value: str | None, country: str | None) -> dict:\n d = load(\"driving_side\")\n src, fetched = d[\"_meta\"][\"source\"][0], d[\"_meta\"][\"fetched\"]\n if country:\n en = _country_en(country)\n hit = next(((k, v) for k, v in d.items() if k != \"_meta\" and k.lower() == en.lower()), None) or \\\n next(((k, v) for k, v in d.items() if k != \"_meta\" and en.lower() in k.lower()), None)\n if not hit:\n return _result(\"driving-side\", country, [], src, fetched, \"country name not in the table; try the English name\")\n k, v = hit\n return _result(\"driving-side\", country, [{\"country\": k, \"side\": v[\"side\"], \"note\": v.get(\"note\", \"\")}], src, fetched)\n side = \"left\" if (value or \"\").lower().startswith((\"l\", \"左\")) else \"right\"\n ms = [{\"country\": k, \"side\": v[\"side\"], \"note\": v.get(\"note\", \"\")} for k, v in d.items() if k != \"_meta\" and v[\"side\"] == side]\n return _result(\"driving-side\", side, ms, src, fetched)\n\n\ndef lookup_territories(value: str, continent: str | None) -> dict:\n d = load(\"territories\")\n src, fetched = d[\"_meta\"][\"source\"][0], d[\"_meta\"][\"fetched\"]\n en = _country_en(value)\n sovs = [k for k in d[\"by_sovereign\"] if k.lower() == en.lower()] or [k for k in d[\"by_sovereign\"] if en.lower() in k.lower()]\n if not sovs:\n return _result(\"territories\", value, [], src, fetched, \"sovereign state not in the table (or it has no territories); former colonies are not in this table\")\n items = [it for k in sovs for it in d[\"by_sovereign\"][k]]\n if continent:\n keys = [x.lower() for x in CONTINENT_ZH.get(continent, [continent])]\n items = [it for it in items if any(k in (it[\"region\"] + \" \" + it[\"subregion\"]).lower() for k in keys)]\n return _result(\"territories\", value, [{\"country\": it[\"name\"], \"continent\": it[\"region\"], \"subregion\": it[\"subregion\"],\n \"note\": f\"{it['status']}; sovereign state {it['sovereign']}\"} for it in items], src, fetched)\n\n\n# ---------------------------------------------------------------- region-pack lookups\n\ndef lookup_region(kind: str, value: str, country: str | None) -> dict:\n r = regions.lookup(kind, value, country)\n return {\"kind\": kind, \"value\": value, \"status\": r[\"status\"], \"matches\": r[\"matches\"], \"searched\": r.get(\"searched\", []),\n \"source\": r.get(\"source\", \"\"), \"table_fetched\": r.get(\"fetched\", \"\"), \"note\": r.get(\"note\", \"\")}\n\n\ndef lookup_admin(value: str | None, children: str | None, level: str | None, country: str | None) -> dict:\n want = {\"city\": \"admin2\", \"county\": \"district\"}.get(level or \"\", level)\n if children:\n p, kids = regions.admin_children(children.strip(), country)\n if not p:\n return _result(\"admin-children\", children, [], \"\", \"\", \"no installed pack has this admin area (use the full name, or gazetteer.py for OSM)\",\n status=\"unsupported\" if country and not regions.resolve(country) else None)\n rows = []\n for k in kids:\n m = regions.match_from_item(p, k)\n m.update(code=k.get(\"code\", \"\"), level=regions.level_name(p, k[\"level\"]))\n if not want or m[\"level\"] == want:\n rows.append(m)\n meta = regions.load_table(p, p[\"admin_table\"]).get(\"_meta\", {})\n return _result(\"admin-children\", children, rows, \", \".join(meta.get(\"source\") or []), meta.get(\"fetched\", \"\"))\n name = (value or \"\").strip()\n rows, srcs = [], set()\n for p, it in regions.admin_find(name, country):\n m = regions.match_from_item(p, it)\n m.update(chain=regions.admin_chain(p, it), code=it.get(\"code\", \"\"), level=regions.level_name(p, it[\"level\"]))\n rows.append(m)\n srcs.add(\", \".join(regions.load_table(p, p[\"admin_table\"]).get(\"_meta\", {}).get(\"source\") or []))\n return _result(\"admin\", name, rows, \" | \".join(sorted(srcs)), \"\", \"\" if rows else \"no admin area with this name in the installed packs (use the full name)\")\n\n\ndef cmd_lookup(args) -> None:\n k = \"plate\" if args.kind == \"plate-prefix\" else args.kind\n if k == \"calling-code\":\n res = lookup_calling_code(args.value or \"\")\n elif k == \"driving-side\":\n res = lookup_driving_side(args.value, args.country)\n elif k == \"territories\":\n res = lookup_territories(args.value or \"\", args.continent)\n elif k == \"admin\":\n res = lookup_admin(args.value, args.children, args.level, args.country)\n else:\n kinds = sorted({x for p in regions.packs().values() for x in (p.get(\"lookups\") or {})})\n if k not in kinds:\n sys.exit(f\"kind: {' / '.join([*GLOBAL_KINDS, 'admin', *kinds])} (region kinds come from installed packs: `regions.py list`)\")\n res = lookup_region(k, args.value or \"\", args.country)\n if args.json:\n print(json.dumps(res, ensure_ascii=False, indent=1))\n return\n head = f\"[{res['kind']}] {res['value']} → {len(res['matches'])} matches\"\n if res[\"status\"] == \"unsupported\":\n head = f\"[{res['kind']}] {res['value']} → unsupported\"\n if res.get(\"searched\"):\n head += f\" (packs searched: {', '.join(res['searched'])})\"\n print(head + (f\" ({res['note']})\" if res[\"note\"] else \"\"))\n for m in res[\"matches\"][: args.limit]:\n if \"country_code\" in m:\n chain = \" / \".join(m[lv] for lv in regions.MATCH_LEVELS if m.get(lv))\n print(f\" {m['country_code']}: {chain}\" + (f\" [{m['level']}]\" if m.get(\"level\") else \"\") + (\" (unverified)\" if m.get(\"unverified\") else \"\")\n + (f\" {m['note']}\" if m.get(\"note\") else \"\"))\n else:\n print(\" \" + \" / \".join(str(m.get(x)) for x in (\"country\", \"side\", \"continent\", \"subregion\", \"utc\") if m.get(x)) + (f\" {m['note']}\" if m.get(\"note\") else \"\"))\n if len(res[\"matches\"]) > args.limit:\n print(f\" … {len(res['matches'])} in total; raise --limit\")\n if res[\"source\"]:\n print(f\"source {res['source']}\" + (f\" ({res['table_fetched']})\" if res[\"table_fetched\"] else \"\"))\n\n\ndef cmd_list(args) -> None:\n for name in GLOBAL_TABLES:\n f = DATA / f\"{name}.json\"\n if not f.exists():\n print(f\"{name}: missing\")\n continue\n m = json.loads(f.read_text(encoding=\"utf-8\")).get(\"_meta\", {})\n src = m.get(\"source\") or [\"hand-compiled\"]\n print(f\"{name}: {m.get('count')} entries, {f.stat().st_size // 1024} KB, fetched {m.get('fetched')}, source {src[0] if isinstance(src, list) else src}\")\n for code, p in sorted(regions.packs().items()):\n for kind, spec in (p.get(\"lookups\") or {}).items():\n try:\n meta = regions.load_table(p, spec[\"file\"]).get(\"_meta\", {})\n except regions.PACK_ERRORS as exc:\n print(f\"{code.lower()}/{kind}: broken ({type(exc).__name__}); `regions.py lint {code.lower()}`\")\n continue\n print(f\"{code.lower()}/{kind}: {meta.get('count')} keys, fetched {meta.get('fetched')}, source {(meta.get('source') or ['?'])[0]}\")\n if p.get(\"admin_table\"):\n print(f\"{code.lower()}/admin: {len(regions.admin_items(p))} areas\")\n\n\ndef main() -> None:\n ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)\n sub = ap.add_subparsers(dest=\"cmd\", required=True)\n lk = sub.add_parser(\"lookup\")\n lk.add_argument(\"kind\")\n lk.add_argument(\"value\", nargs=\"?\")\n lk.add_argument(\"--country\", help=\"region-pack kinds: restrict to this country's pack (code or name); driving-side: the country to look up\")\n lk.add_argument(\"--continent\")\n lk.add_argument(\"--children\")\n lk.add_argument(\"--level\", help=\"with --children: keep one level (admin2/district; city/county also accepted)\")\n lk.add_argument(\"--json\", action=\"store_true\")\n lk.add_argument(\"--limit\", type=int, default=40)\n sub.add_parser(\"list\")\n args = ap.parse_args()\n {\"lookup\": cmd_lookup, \"list\": cmd_list}[args.cmd](args)\n\n\nif __name__ == \"__main__\":\n # Chinese-locale Windows writes GBK by default: m², ñ make it crash, and the Chinese the agent reads comes out garbled\n sys.stdout.reconfigure(encoding=\"utf-8\")\n sys.stderr.reconfigure(encoding=\"utf-8\", errors=\"backslashreplace\")\n main()\n","content_type":"text/x-python; charset=utf-8","language":"python","size":12885,"content_sha256":"642676fb2c7fcc6118325d94877fab8de5d92118898d507ed76c68b9ae1e0ab9"},{"filename":"scripts/doctor.py","content":"#!/usr/bin/env python3\n# /// script\n# requires-python = \">=3.10\"\n# dependencies = [\"playwright\"]\n# ///\n\"\"\"Check geo-sleuth setup without uploading photos or downloading ML models.\n\n uv run scripts/doctor.py local tools, data, browser, writable directory\n uv run scripts/doctor.py --network also probe public service endpoints\n uv run scripts/doctor.py --json machine-readable results on stdout\n\nuv may install this script's Playwright dependency on first use. Network probes only\ncheck reachability, not image uploads, API stability, coverage, or model inference.\nExit codes: 0 = no failed checks (warnings may remain); 1 = at least one failed check.\n\"\"\"\nfrom __future__ import annotations\n\nimport argparse\nimport asyncio\nimport json\nimport os\nimport platform\nimport shutil\nimport subprocess\nimport sys\nimport tempfile\nfrom concurrent.futures import ThreadPoolExecutor\nfrom pathlib import Path\n\nfrom _browser import launch_browser\nfrom _net import PROXY_HELP, curl_args, resolve_proxy\nimport regions\n\nSERVICES = {\n \"Baidu image search\": \"https://graph.baidu.com/pcpage/index?tpl_from=pc\",\n \"Baidu panoramas\": \"https://mapsv0.bdimg.com/?qt=qsdata&x=0&y=0\",\n \"Yandex Images\": \"https://yandex.com/images/\",\n \"Google satellite tiles\": \"https://mt1.google.com/vt/lyrs=s&x=0&y=0&z=0\",\n \"Overpass\": \"https://overpass-api.de/api/status\",\n \"Elevation tiles\": \"https://s3.amazonaws.com/elevation-tiles-prod/terrarium/0/0/0.png\",\n \"Hugging Face\": \"https://huggingface.co/\",\n}\n\n\n# gsv.py's panorama lookup at a covered point (Shibuya). An HTTP probe isn't enough here: a turned-down endpoint still answers 200.\nSTREET_VIEW = (\"https://maps.googleapis.com/$rpc/google.internal.maps.mapsjs.v1.MapsJsInternalService/SingleImageSearch\",\n '[[\"apiv3\",null,null,null,\"US\",null,null,null,null,null,[[0]]],[[null,null,35.6595,139.7005],50],'\n '[null,[\"en\",\"US\"],null,null,null,null,null,null,[2],null,[[[2,true,2]]]],[[1,2,3,4,8,6]]]')\n\n\ndef check(name: str, status: str, detail: str, fix: str = \"\") -> dict:\n return {\"name\": name, \"status\": status, \"detail\": detail, \"fix\": fix}\n\n\ndef probe(item: tuple[str, str], proxy: str | None) -> dict:\n name, url = item\n try:\n result = subprocess.run(\n [\"curl\", \"-q\", \"-sSL\", \"--max-time\", \"12\", *curl_args(proxy),\n \"-A\", \"geo-sleuth-doctor/1.0\", \"-o\", os.devnull, \"-w\", \"%{http_code}\", url],\n capture_output=True, text=True, encoding=\"utf-8\", errors=\"replace\", timeout=15)\n code = result.stdout.strip()\n if result.returncode:\n return check(name, \"FAIL\", f\"Connection failed (curl exit {result.returncode}).\",\n \"Check DNS, TLS certificates and network access.\")\n if code.isdigit() and 200 \u003c= int(code) \u003c 400:\n return check(name, \"PASS\", f\"HTTP {code}; reachable (a live operation can still be blocked or rate-limited).\")\n return check(name, \"WARN\", f\"HTTP {code}; server reached, but service access was not confirmed.\",\n \"Retry later or inspect the service in a browser. HTTP 403/429 may mean a challenge or rate limit.\")\n except (OSError, subprocess.TimeoutExpired):\n return check(name, \"FAIL\", \"Probe could not complete within its time limit.\", \"Check curl and network access, then retry.\")\n\n\ndef street_view_check(proxy: str | None) -> dict:\n name = \"Google Street View\"\n url, query = STREET_VIEW\n try:\n result = subprocess.run(\n [\"curl\", \"-q\", \"-sS\", \"--max-time\", \"12\", *curl_args(proxy), \"-X\", \"POST\",\n \"-H\", \"content-type: application/json+protobuf\", \"-H\", \"x-user-agent: grpc-web-javascript/0.1\",\n \"--data\", query, url],\n capture_output=True, text=True, encoding=\"utf-8\", errors=\"replace\", timeout=15)\n except (OSError, subprocess.TimeoutExpired):\n return check(name, \"FAIL\", \"Probe could not complete within its time limit.\", \"Check curl and network access, then retry.\")\n if result.returncode:\n return check(name, \"FAIL\", f\"Connection failed (curl exit {result.returncode}).\", \"Check DNS, TLS certificates and network access.\")\n try:\n pano = json.loads(result.stdout)[1][1][1]\n except (ValueError, IndexError, TypeError, KeyError):\n pano = None\n if isinstance(pano, str) and pano:\n return check(name, \"PASS\", \"Panorama lookup returned a panorama at a covered test point.\")\n return check(name, \"FAIL\", f\"Panorama lookup failed: {' '.join(result.stdout.split())[:160] or 'empty reply'}\",\n \"gsv.py near/sheet won't work. If the reply says the service changed or was turned down, gsv.py needs updating.\")\n\n\nasync def browser_check(proxy: str | None) -> dict:\n try:\n from playwright.async_api import async_playwright\n async with async_playwright() as p:\n browser, name = await launch_browser(p, proxy)\n try:\n page = await browser.new_page()\n await page.goto(\"about:blank\")\n return check(\"Reverse-search browser\", \"PASS\", f\"{name} launched successfully.\")\n finally:\n await browser.close()\n except Exception as exc:\n return check(\"Reverse-search browser\", \"WARN\", str(exc),\n \"Install Chrome or run `uvx playwright install chromium`. Until then, use intake.py --no-rev.\")\n\n\ndef diagnose(network: bool, proxy: str | None) -> dict:\n rows = [check(\"Python\", \"PASS\" if sys.version_info >= (3, 10) else \"FAIL\",\n platform.python_version(), \"Use Python 3.10 or later.\" if sys.version_info \u003c (3, 10) else \"\")]\n for tool in (\"uv\", \"curl\"):\n found = (os.environ.get(\"UV\") or shutil.which(\"uv\")) if tool == \"uv\" else shutil.which(tool)\n try:\n result = subprocess.run([found or tool, \"--version\"], capture_output=True, timeout=5)\n ok = result.returncode == 0\n except (OSError, subprocess.TimeoutExpired):\n ok = False\n rows.append(check(tool, \"PASS\" if ok else \"FAIL\", \"Available.\" if ok else \"Not runnable.\",\n \"\" if ok else f\"Install {tool} and reopen your terminal so it is on PATH.\"))\n try:\n with tempfile.TemporaryFile(dir=Path.cwd()) as file:\n file.write(b\"geo-sleuth\")\n rows.append(check(\"Working directory\", \"PASS\", \"Writable; reports and .geo-cache can be created here.\"))\n except OSError:\n rows.append(check(\"Working directory\", \"FAIL\", \"Cannot write here.\", \"Run from a writable folder.\"))\n data = Path(__file__).resolve().parent.parent / \"data\"\n names = (\"calling_codes\", \"driving_side\", \"territories\", \"country_names\")\n bad = []\n for name in names:\n try:\n json.loads((data / f\"{name}.json\").read_text(encoding=\"utf-8\"))\n except (OSError, ValueError):\n bad.append(name)\n rows.append(check(\"Global lookup tables\", \"FAIL\" if bad else \"PASS\", \"Missing or invalid: \" + \", \".join(bad) if bad else f\"All {len(names)} tables are readable.\",\n \"Reinstall the complete skill folder (including data/).\" if bad else \"\"))\n # A broken region pack only disables that country's lookups and tips; it never fails the core\n for code, pack in sorted(regions.packs().items()):\n try:\n errs, _ = regions.lint(pack)\n except regions.PACK_ERRORS as exc:\n errs = [str(exc)]\n rows.append(check(f\"Region pack {code.lower()}\", \"WARN\" if errs else \"PASS\",\n \"; \".join(errs[:2]) if errs else f\"{pack.get('name', code)}: lookups {', '.join(pack.get('lookups') or {}) or 'none'}.\",\n f\"`regions.py lint {code.lower()}` lists every problem; reinstall regions/{code.lower()}/ or remove it.\" if errs else \"\"))\n route = \"configured proxy\" if resolve_proxy(proxy) else \"direct\"\n rows.append(check(\"Service connection\", \"PASS\", f\"Using {route}.\"))\n rows.append(asyncio.run(browser_check(proxy)))\n rows.append(check(\"OCR\", \"INFO\", \"Apple Vision preferred; RapidOCR fallback.\" if sys.platform == \"darwin\" else \"RapidOCR backend.\",\n \"uv run ocr.py \u003cphoto> tests recognition; this check does not install or run OCR.\"))\n rows.append(check(\"ML models\", \"INFO\", \"Not loaded. match.py and sat_scan.py download model weights on first use; allow time and disk space.\"))\n if network and shutil.which(\"curl\"):\n with ThreadPoolExecutor(max_workers=8) as pool:\n rows.extend(pool.map(lambda item: probe(item, proxy), SERVICES.items()))\n rows.append(street_view_check(proxy))\n else:\n rows.append(check(\"Network probes\", \"SKIP\", \"Run with --network to check service reachability.\"))\n return {\"ok\": not any(row[\"status\"] == \"FAIL\" for row in rows), \"connection\": route, \"checks\": rows}\n\n\ndef main() -> None:\n parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)\n parser.add_argument(\"--network\", action=\"store_true\", help=\"probe public endpoints without uploading photos\")\n parser.add_argument(\"--proxy\", help=PROXY_HELP)\n parser.add_argument(\"--json\", action=\"store_true\", help=\"print JSON for issue reports or automation\")\n args = parser.parse_args()\n report = diagnose(args.network, args.proxy)\n if args.json:\n print(json.dumps(report, ensure_ascii=False, indent=2))\n else:\n for row in report[\"checks\"]:\n print(f\"[{row['status']}] {row['name']}: {row['detail']}\")\n if row[\"fix\"]:\n print(f\" Next: {row['fix']}\")\n print(\"\\nNo failed checks. Review warnings before using optional features.\" if report[\"ok\"] else \"\\nSome checks failed. Follow the suggested fixes and rerun doctor.py.\")\n sys.exit(0 if report[\"ok\"] else 1)\n\n\nif __name__ == \"__main__\":\n sys.stdout.reconfigure(encoding=\"utf-8\")\n sys.stderr.reconfigure(encoding=\"utf-8\", errors=\"backslashreplace\")\n main()\n","content_type":"text/x-python; charset=utf-8","language":"python","size":9988,"content_sha256":"fc232d0971d5f931c86f273a862f6c96463ad009cae1f3f12f4f02ed489f54d5"},{"filename":"scripts/evidence.py","content":"#!/usr/bin/env python3\n# /// script\n# requires-python = \">=3.10\"\n# dependencies = [\"pillow\", \"pillow-heif\"]\n# ///\n\"\"\"Make the evidence image: mark the camera position, heading wedge and key features on satellite imagery, with street-view comparison panels below.\n\nInput is a JSON spec file:\n{\n \"map\": \"area.jpg\", # required: output of tiles.py fetch (needs the same-name .json next to it)\n \"crop\": [x0, y0, x1, y1], # optional, crop in original-image pixels\n \"width\": 1280, # output width\n \"camera\": [lat, lon],\n \"heading\": 52, \"hfov\": 54, \"range_m\": 560, # heading wedge\n \"labels\": [{\"at\": [lat, lon], \"text\": \"Xinghe Twin Towers\", \"color\": \"#ffdd55\", \"dx\": 0, \"dy\": 0}],\n \"lines\": [{\"from\": [lat, lon], \"bearing\": 47, \"length_m\": 75, \"color\": \"#00ffff\"}],\n \"panels\": [{\"image\": \"sv1.jpg\", \"caption\": \"Street view: same road, looking northeast\"}]\n}\n\nAll coordinates are WGS84, and the map must be WGS84-aligned imagery as tiles.py fetches it. A GCJ-02 mosaic\n(Amap and other Chinese road maps) shifts every mark by several hundred meters in China; use tiles.py imagery\ninstead. Convert GCJ-02 input coordinates to WGS84 with geo.py convert --from gcj --to wgs. Paths are relative\nto the spec file.\n\nExample: evidence.py spec.json --out evidence.jpg\n\"\"\"\nfrom __future__ import annotations\n\nimport argparse\nfrom _image import open_photo\nimport json\nimport re\nimport sys\nfrom pathlib import Path\n\nfrom PIL import Image, ImageDraw\n\nsys.path.insert(0, str(Path(__file__).parent))\nimport geo # noqa: E402\nfrom baidu_pano import _font # noqa: E402\nfrom tiles import Mosaic # noqa: E402\n\n\n# required keys of each list item in the spec\nITEM_KEYS = {\"labels\": (\"at\", \"text\"), \"lines\": (\"from\", \"bearing\", \"length_m\"), \"panels\": (\"image\",)}\n\n\ndef check_spec(spec: dict, base: Path) -> list[str]:\n \"\"\"Problems that would otherwise surface as a bare KeyError or FileNotFoundError mid-drawing.\"\"\"\n if not isinstance(spec, dict):\n return [\"the spec must be a JSON object\"]\n errs = []\n if \"map\" not in spec:\n errs.append(\"missing 'map': the tiles.py fetch mosaic to draw on\")\n elif not isinstance(spec[\"map\"], str) or not spec[\"map\"].strip():\n errs.append(\"'map' must be a non-empty path string\")\n else:\n map_path = base / spec[\"map\"]\n if not map_path.is_file():\n errs.append(f\"map not found: {map_path}\")\n if not map_path.with_suffix(\".json\").is_file():\n errs.append(f\"map has no georeference: {map_path.with_suffix('.json')} is missing (tiles.py fetch writes it next to the image)\")\n for key, need in ITEM_KEYS.items():\n items = spec.get(key, [])\n if not isinstance(items, list):\n errs.append(f\"'{key}' must be a JSON array\")\n continue\n for i, item in enumerate(items):\n if not isinstance(item, dict):\n errs.append(f\"{key}[{i}] must be a JSON object\")\n continue\n miss = [k for k in need if k not in item]\n if miss:\n errs.append(f\"{key}[{i}] is missing {', '.join(repr(k) for k in miss)}\")\n if key == \"panels\" and \"image\" in item:\n if not isinstance(item[\"image\"], str) or not item[\"image\"].strip():\n errs.append(f\"panels[{i}] 'image' must be a non-empty path string\")\n elif not (base / item[\"image\"]).is_file():\n errs.append(f\"panels[{i}] image not found: {base / item['image']}\")\n return errs\n\n\ndef build(spec: dict, base: Path, out: Path) -> None:\n errs = check_spec(spec, base)\n if errs:\n sys.exit(\"bad spec:\\n \" + \"\\n \".join(errs) + \"\\nspec format: evidence.py --help\")\n map_path = base / spec[\"map\"]\n m = Mosaic(map_path)\n im = Image.open(map_path).convert(\"RGBA\")\n scale_font = max(im.size) / 1400\n\n ov = Image.new(\"RGBA\", im.size, (0, 0, 0, 0))\n d = ImageDraw.Draw(ov)\n if \"camera\" in spec:\n cam = tuple(spec[\"camera\"])\n if \"heading\" in spec:\n h, fov, rng = spec[\"heading\"], spec.get(\"hfov\", 54), spec.get(\"range_m\", 400)\n pts = [m.to_px(*cam)] + [m.to_px(*geo.dest(cam, h - fov / 2 + k * fov / 24, rng)) for k in range(25)]\n d.polygon(pts, fill=(255, 220, 0, 55), outline=(255, 220, 0, 210))\n for ln in spec.get(\"lines\", []):\n a = m.to_px(*ln[\"from\"])\n b = m.to_px(*geo.dest(tuple(ln[\"from\"]), ln[\"bearing\"], ln[\"length_m\"]))\n d.line([a, b], fill=ln.get(\"color\", \"#00ffff\"), width=int(8 * scale_font) or 3)\n if \"camera\" in spec:\n x, y = m.to_px(*spec[\"camera\"])\n r = 18 * scale_font\n d.ellipse([x - r, y - r, x + r, y + r], fill=(255, 40, 40, 255), outline=\"white\", width=4)\n im = Image.alpha_composite(im, ov).convert(\"RGB\")\n\n d = ImageDraw.Draw(im)\n f = _font(int(42 * scale_font))\n for lb in spec.get(\"labels\", []):\n ax, ay = m.to_px(*lb[\"at\"])\n r = 8 * scale_font\n d.ellipse([ax - r, ay - r, ax + r, ay + r], fill=lb.get(\"color\", \"white\"), outline=\"black\") # anchor point\n x, y = ax + lb.get(\"dx\", 12), ay + lb.get(\"dy\", -12)\n bb = d.textbbox((0, 0), lb[\"text\"], font=f)\n tw, th = bb[2] - bb[0], bb[3] - bb[1]\n x = min(max(8, x), im.size[0] - tw - 16) # stay inside the image edges\n y = min(max(8, y), im.size[1] - th - 16)\n d.rectangle([x - 8, y - 6, x + tw + 8, y + th + 10], fill=\"black\")\n d.text((x, y), lb[\"text\"], font=f, fill=lb.get(\"color\", \"white\"))\n\n if spec.get(\"crop\"):\n im = im.crop(tuple(spec[\"crop\"]))\n W = spec.get(\"width\", 1280)\n im = im.resize((W, int(im.size[1] * W / im.size[0])))\n\n panels = spec.get(\"panels\", [])\n if panels:\n pw = W // len(panels)\n ph = int(pw * 3 / 4)\n cf = _font(20)\n lines_per = []\n for p in panels: # wrap captions to the panel width so they don't overlap the neighboring panel\n cap, cur, lines = p.get(\"caption\", \"\"), \"\", []\n for ch in cap:\n if cf.getlength(cur + ch) > pw - 12:\n lines.append(cur)\n cur = ch\n else:\n cur += ch\n lines.append(cur)\n lines_per.append(lines[:3])\n cap_h = 10 + 26 * max(len(x) for x in lines_per)\n S = Image.new(\"RGB\", (W, im.size[1] + ph + cap_h), \"black\")\n S.paste(im, (0, 0))\n d = ImageDraw.Draw(S)\n for k, (p, lines) in enumerate(zip(panels, lines_per)):\n pim = open_photo(base / p[\"image\"], \"panel image\").convert(\"RGB\")\n pim.thumbnail((pw, ph)) # keep aspect ratio, centered with black borders\n S.paste(pim, (k * pw + (pw - pim.width) // 2, im.size[1] + cap_h + (ph - pim.height) // 2))\n for li, text in enumerate(lines):\n d.text((k * pw + 6, im.size[1] + 6 + 26 * li), text, font=cf, fill=\"white\")\n im = S\n im.save(out, quality=88)\n\n\n\ndef _neg_coords(argv: list[str]) -> list[str]:\n \"\"\"argparse takes negative coordinates like -1.45,-48.5 for option names; prefixing a space makes them plain values (float ignores the space). Needed for any case in the southern or western hemisphere.\"\"\"\n return [\" \" + a if re.match(r\"^-\\d[\\d.]*(,-?[\\d.]+)+$\", a) else a for a in argv]\n\n\ndef main() -> None:\n ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)\n ap.add_argument(\"spec\", type=Path)\n ap.add_argument(\"--out\", type=Path, required=True)\n args = ap.parse_args(_neg_coords(sys.argv[1:]))\n build(json.loads(args.spec.read_text(encoding=\"utf-8\")), args.spec.parent, args.out)\n print(args.out)\n\n\nif __name__ == \"__main__\":\n # Chinese Windows outputs GBK by default: it crashes on m², ñ, and Chinese text the agent reads comes out garbled\n sys.stdout.reconfigure(encoding=\"utf-8\")\n sys.stderr.reconfigure(encoding=\"utf-8\", errors=\"backslashreplace\")\n main()\n","content_type":"text/x-python; charset=utf-8","language":"python","size":8098,"content_sha256":"4bb5949afec3eb247e89af3b2991c52786320e0666a0a7d992dfd9b581e1f25c"},{"filename":"scripts/exif.py","content":"#!/usr/bin/env python3\n# /// script\n# requires-python = \">=3.10\"\n# dependencies = [\"pillow\", \"pillow-heif\"]\n# ///\n\"\"\"Read photo metadata: GPS, capture time, camera model, focal length. The first thing to do before geolocating.\n\n exif.py \u003cphoto> [\u003cphoto> ...]\n\n- With GPS: gives WGS84 coordinates and the GCJ-02 conversion (for Amap/Tencent Maps) directly, but you still must check against the image (metadata can be edited).\n- With capture time: can be fed to sun.py (note EXIF time is usually local time at the capture location, without a time zone; see the OffsetTime field).\n- With 35mm equivalent focal length: can be fed to geo.py range / geometry.md to compute the field of view, no need to guess the zoom factor.\n- size is the stored pixel size; a photo with an EXIF orientation also gets orientation and upright_size, the size as a viewer shows it,\n which is the frame the other scripts' pixel coordinates use (imgprep.py boxes, ocr.py, terrain.py ridge, pose.py points).\n- WeChat, QQ and most social platforms strip metadata when forwarding; screenshots and rephotographed images have no metadata. Empty output is normal and means nothing.\n\"\"\"\nfrom __future__ import annotations\n\nimport json\nimport sys\nfrom pathlib import Path\n\nfrom PIL import ExifTags, Image\n\nsys.path.insert(0, str(Path(__file__).parent))\nimport geo # noqa: E402\nfrom _image import orientation # noqa: E402\n\ntry:\n import pillow_heif\n pillow_heif.register_heif_opener()\nexcept ImportError: # without pillow-heif, HEIC can't be read; other formats work as usual\n pass\n\n\ndef _deg(v, ref) -> float:\n d, m, s = (float(x) for x in v)\n val = d + m / 60 + s / 3600\n return -val if ref in (\"S\", \"W\") else val\n\n\ndef read(path: Path) -> dict:\n im = Image.open(path)\n ex = im.getexif()\n out: dict = {\"file\": str(path), \"size\": list(im.size)}\n o = orientation(im)\n if o != 1: # size is the stored pixels; pixel coordinates for the other scripts are in the upright frame\n out[\"orientation\"] = o\n out[\"upright_size\"] = list(im.size) if o \u003c= 4 else list(im.size)[::-1]\n base = {ExifTags.TAGS.get(k, k): v for k, v in ex.items()}\n sub = {ExifTags.TAGS.get(k, k): v for k, v in ex.get_ifd(0x8769).items()}\n for k in (\"Make\", \"Model\", \"Software\"):\n if base.get(k):\n out[k.lower()] = str(base[k]).strip(\"\\x00 \")\n for k in (\"DateTimeOriginal\", \"OffsetTimeOriginal\"):\n if sub.get(k):\n out[k] = str(sub[k])\n if not out.get(\"DateTimeOriginal\") and base.get(\"DateTime\"):\n out[\"DateTime\"] = str(base[\"DateTime\"])\n if sub.get(\"FocalLengthIn35mmFilm\"):\n out[\"focal_35mm\"] = int(sub[\"FocalLengthIn35mmFilm\"])\n lf, sf = geo.fov_from_equiv_focal(out[\"focal_35mm\"])\n out[\"fov_4x3_long_short\"] = [round(lf, 1), round(sf, 1)]\n elif sub.get(\"FocalLength\"):\n out[\"focal_mm_actual\"] = float(sub[\"FocalLength\"])\n g = ex.get_ifd(0x8825)\n if g and 2 in g and 4 in g:\n lat, lon = _deg(g[2], g.get(1, \"N\")), _deg(g[4], g.get(3, \"E\"))\n out[\"gps_wgs84\"] = [round(lat, 7), round(lon, 7)]\n out[\"gps_gcj02\"] = [round(x, 7) for x in geo.convert(lat, lon, \"wgs\", \"gcj\")]\n if 6 in g:\n out[\"altitude_m\"] = round(float(g[6]), 1)\n if 17 in g:\n out[\"image_direction_deg\"] = round(float(g[17]), 1) # some phones write the lens heading\n return out\n\n\ndef main() -> None:\n if len(sys.argv) \u003c 2:\n sys.exit(__doc__)\n for p in sys.argv[1:]:\n try:\n print(json.dumps(read(Path(p)), ensure_ascii=False))\n except Exception as e: # noqa: BLE001\n print(json.dumps({\"file\": p, \"error\": str(e)}, ensure_ascii=False))\n\n\nif __name__ == \"__main__\":\n # Chinese Windows outputs GBK by default: it crashes on m², ñ, and Chinese text the agent reads comes out garbled\n sys.stdout.reconfigure(encoding=\"utf-8\")\n sys.stderr.reconfigure(encoding=\"utf-8\", errors=\"backslashreplace\")\n main()\n","content_type":"text/x-python; charset=utf-8","language":"python","size":3971,"content_sha256":"3d89a690d47bd1a0b3a5c953ed1f30cdad899fccaf97117ad6c2d92bce0b0223"},{"filename":"scripts/gazetteer.py","content":"#!/usr/bin/env python3\n# /// script\n# requires-python = \">=3.10\"\n# dependencies = []\n# ///\n\"\"\"Admin gazetteer: list candidate areas \"in full\" with extent, area and scan cost. Used by board.py; can also be queried on its own.\n\n children list every next-level (or specified-level) admin area of an admin area, with bbox, bbox area, center\n info an admin area's OSM relation, admin_level, bbox, center\n urban an admin area's built-up extent (largest contiguous block of OSM residential/commercial/industrial land use), for scan cost and scan_bbox\n cost how many cells and pages a bbox takes at tiles.py sheet --grid settings\n\nData comes from OSM Overpass (results cached per query in .geo-cache/osm/). OSM admin lists can have gaps:\nchildren merges them with a region pack's admin table when one knows the parent (regions/\u003ccc>/, e.g. China's three levels) and flags entries missing a bbox.\nadmin_level differs by country: China province 4 / prefecture 5 / county 6, France region 4 / department 6, US state 4 / county 6. If unsure, omit --level;\nthe script tries downward from the parent's admin_level and skips levels whose combined extent is under 30% of the parent (China's level 3 is only Hong Kong and Macau).\n\nExamples:\n gazetteer.py children \u003cfull province-level name> --out districts.json # municipality → all districts\n gazetteer.py children \u003ccountry name> --level 4 # country → first-level admin divisions\n gazetteer.py info \u003cadmin area name>\n gazetteer.py urban \u003cdistrict name> --within \u003cfull province-level name> # built-up bbox + area\n gazetteer.py cost --bbox \u003cs,w,n,e> --zoom 16 --cell 320 --cols 5\n\"\"\"\nfrom __future__ import annotations\n\nimport argparse\nfrom functools import lru_cache\nfrom _net import PROXY_HELP\nimport json\nimport math\nimport os\nimport re\nimport sys\nfrom pathlib import Path\n\nsys.path.insert(0, str(Path(__file__).parent))\nimport geo # noqa: E402\nimport osm # noqa: E402\nimport regions # noqa: E402\n\nMIN_COVER = 0.3 # children auto level pick: this level's combined bbox area must be at least this fraction of the parent's bbox\n\n\ndef _cache(args) -> Path:\n return Path(getattr(args, \"cache\", None) or \".geo-cache/osm\")\n\n\ndef _bbox_km2(b: list[float]) -> float:\n s, w, n, e = b\n return abs(n - s) * 110.574 * abs(e - w) * 111.320 * math.cos(math.radians((s + n) / 2))\n\n\ndef _by_name(name: str, extra: str = \"\", scope: str = \"\") -> str:\n \"\"\"Union of admin-relation selectors matching name on `name` and the region packs' extra name keys (regions.osm_name_keys):\n a bilingual OSM name (\"\u003clocal> \u003csecond script>\") never equals the plain name. Indexed tag lookups, fast.\"\"\"\n q = name.replace(\"\\\\\", \"\\\\\\\\\").replace('\"', '\\\\\"')\n return \"(\" + \"\".join(f'rel{scope}[\"{k}\"=\"{q}\"][\"boundary\"=\"administrative\"]{extra};' for k in regions.osm_name_keys()) + \")\"\n\n\n@lru_cache(maxsize=1)\ndef _country_index() -> tuple[dict[str, str], dict[str, list[str]]]:\n \"\"\"Exact names and ISO codes only. Broad clue aliases (e.g. England -> UK) are not boundaries.\"\"\"\n data = json.loads((Path(__file__).resolve().parents[1] / \"data\" / \"country_names.json\").read_text(encoding=\"utf-8\"))\n codes = data[\"iso3166\"]\n index = {}\n for code, names in codes.items():\n for name in [code, *names, *(data[\"en2zh\"].get(n, \"\") for n in names)]:\n if name:\n index[name.casefold()] = code\n return index, codes\n\n\ndef _country(name: str) -> tuple[str, tuple[str, ...]] | None:\n index, codes = _country_index()\n code = index.get(name.strip().casefold())\n if not code:\n # Packs may add local aliases, but are not required to find a country.\n code = (regions.resolve(name) or {}).get(\"code\")\n if code not in codes:\n return None\n return code, tuple(codes[code])\n\n\ndef _by_country(country: tuple[str, tuple[str, ...]], level: int | None = None) -> str:\n code, names = country\n lv = f'[\"admin_level\"=\"{level}\"]' if level else \"\"\n # ISO also identifies territories such as Hong Kong (level 3); do not turn them into countries.\n selectors = [f'rel[\"{key}\"=\"{code}\"][\"boundary\"=\"administrative\"]{lv};'\n for key in (\"ISO3166-1\", \"ISO3166-1:alpha2\")]\n # Some country relations lack the ISO tag. English names are safe only at country level.\n for name in names:\n q = name.replace(\"\\\\\", \"\\\\\\\\\").replace('\"', '\\\\\"')\n selectors.append(f'rel[\"name:en\"=\"{q}\"][\"boundary\"=\"administrative\"][\"admin_level\"=\"2\"];')\n return \"(\" + \"\".join(selectors) + \")\"\n\n\ndef _plain_name(t: dict, fallback: str) -> str:\n \"\"\"A bilingual name tag (\"\u003cChinese> \u003cTibetan>\") → the leading part that is one of the relation's own name:* values.\"\"\"\n name = t.get(\"name\", fallback)\n if \" \" in name:\n heads = [v for k, v in t.items() if k.startswith(\"name:\") and v and name.startswith(v + \" \")]\n if heads:\n return max(heads, key=len)\n return name\n\n\ndef _rel_rows(data: dict) -> list[dict]:\n rows, seen = [], set()\n for el in data.get(\"elements\", []):\n if el.get(\"type\") != \"relation\" or el[\"id\"] in seen:\n continue\n t, b = el.get(\"tags\") or {}, el.get(\"bounds\") or {}\n if not b:\n continue\n seen.add(el[\"id\"])\n bbox = [b[\"minlat\"], b[\"minlon\"], b[\"maxlat\"], b[\"maxlon\"]]\n rows.append({\"name\": _plain_name(t, str(el[\"id\"])), \"name_en\": t.get(\"name:en\", \"\"), \"osm_id\": el[\"id\"],\n \"names\": sorted({v for k, v in t.items() if k in regions.osm_name_keys() and v}),\n \"admin_level\": int(t.get(\"admin_level\") or 0), \"bbox\": [round(v, 5) for v in bbox],\n \"bbox_km2\": round(_bbox_km2(bbox), 1),\n \"center\": [round((bbox[0] + bbox[2]) / 2, 5), round((bbox[1] + bbox[3]) / 2, 5)],\n \"population\": t.get(\"population\", \"\")})\n return rows\n\n\ndef find_relation(name: str, proxy: str | None, cache: Path, within: str | None = None,\n level: int | None = None) -> list[dict]:\n \"\"\"Find admin relations by name (there can be several with the same name: Chaoyang District in Beijing and in Changchun), with bbox.\"\"\"\n lv = f'[\"admin_level\"=\"{level}\"]' if level else \"\"\n country = _country(name) if not within and level in (None, 2) else None\n if within:\n scope_country = _country(within)\n parent = _by_country(scope_country) if scope_country else _by_name(within)\n ql = f'[out:json][timeout:120];{parent};map_to_area->.p;{_by_name(name, lv, \"(area.p)\")};out tags bb;'\n else:\n parent = _by_country(country, level) if country else _by_name(name, lv)\n ql = f'[out:json][timeout:120];{parent};out tags bb;'\n rows = _rel_rows(osm.run(ql, proxy, cache))\n if country and not rows:\n sys.exit(f'OSM has no country/territory boundary for \"{name}\" ({country[0]}). '\n 'For a same-named local place, add --within \u003cparent name> or use info --level \u003cadmin level>.')\n if country and len(rows) > 1:\n sys.exit(f'OSM returned several country/territory boundaries for \"{name}\" ({country[0]}): '\n + \", \".join(f'{r[\"name\"]} (relation {r[\"osm_id\"]}, level {r[\"admin_level\"]})' for r in rows)\n + \". Check the OSM relations before choosing a parent.\")\n rows.sort(key=lambda r: (r[\"admin_level\"], -r[\"bbox_km2\"]))\n return rows\n\n\ndef _filter_known_neighbours(parent: dict, rows: list[dict]) -> list[dict]:\n \"\"\"An area query also returns relations touching the border. Drop only places the same pack locates outside this parent;\n unknown names and ambiguous parents stay in the results, because a partial admin table cannot prove they are outside.\"\"\"\n parents = regions.admin_find(parent[\"name\"])\n if len(parents) != 1:\n return rows\n pack, item = parents[0]\n kept = []\n for row in rows:\n hits = [hit for name in {row[\"name\"], *row.get(\"names\", [])} for _, hit in regions.admin_find(name, pack[\"code\"])]\n if hits and not any(item[\"name\"] in regions.admin_chain(pack, hit)[:-1] for hit in hits):\n print(f\"Skipping {row['name']}: {pack['code']} admin table places it outside {parent['name']}\", file=sys.stderr)\n else:\n kept.append(row)\n return kept\n\n\ndef children(parent: str, proxy: str | None, cache: Path, level: int | None, within: str | None) -> tuple[dict, list[dict]]:\n ps = find_relation(parent, proxy, cache, within)\n if not ps:\n sys.exit(f\"OSM has no admin relation named \\\"{parent}\\\": try the full name (with or without the suffix 市/省/区), or add --within \u003cparent name>\")\n p = ps[0]\n levels = [level] if level else [p[\"admin_level\"] + k for k in (1, 2, 3, 4)]\n first = None\n for lv in levels:\n ql = (f'[out:json][timeout:180];rel({p[\"osm_id\"]});map_to_area->.a;'\n f'rel(area.a)[\"boundary\"=\"administrative\"][\"admin_level\"=\"{lv}\"];out tags bb;')\n rows = _rel_rows(osm.run(ql, proxy, cache))\n rows = _filter_known_neighbours(p, [r for r in rows if r[\"osm_id\"] != p[\"osm_id\"]])\n rows = sorted(rows, key=lambda r: r[\"name\"])\n if level:\n return p, rows\n if len(rows) \u003c 2:\n continue\n # Auto level pick: a level that covers only a small part of the parent isn't the \"next level\" (China's admin_level 3 is only Hong Kong and Macau; provinces are 4)\n cover = sum(r[\"bbox_km2\"] for r in rows) / max(p[\"bbox_km2\"], 1e-9)\n if cover >= MIN_COVER:\n return p, rows\n print(f\"admin_level {lv} has only {len(rows)} ({', '.join(r['name'] for r in rows[:6])}), \"\n f\"together covering only {cover:.1%} of the parent; not treated as the next level, trying lower\", file=sys.stderr)\n first = first or rows\n if first:\n print(\"None of the lower levels covers the parent; falling back to the first level with ≥2; if that's wrong, give --level\", file=sys.stderr)\n return p, first or []\n\n\ndef also_children(parent: dict, rows: list[dict], proxy: str | None, cache: Path) -> tuple[list[dict], list[str]]:\n \"\"\"Areas a region pack lists next to a country's first-level divisions (region.json \"also_children\") that OSM's\n admin tree leaves out or puts at another level. One query for all of them. (rows found, names not found)\"\"\"\n pk = regions.resolve(parent[\"name\"]) if parent.get(\"admin_level\") == 2 else None\n have = {n for r in rows for n in (r[\"name\"], *r.get(\"names\", []))}\n want = [n for n in (pk or {}).get(\"also_children\") or [] if n not in have]\n if not want:\n return [], []\n found = _rel_rows(osm.run(\"[out:json][timeout:120];(\" + \"\".join(_by_name(n) + \";\" for n in want) + \");out tags bb;\", proxy, cache))\n out, missing = [], []\n for n in want:\n hits = sorted((r for r in found if n == r[\"name\"] or n in r[\"names\"]), key=lambda r: (r[\"admin_level\"], -r[\"bbox_km2\"]))\n if not hits:\n missing.append(n)\n elif hits[0][\"osm_id\"] not in {r[\"osm_id\"] for r in rows + out}:\n out.append(hits[0])\n return out, missing\n\n\ndef _local_children(parent: str) -> tuple[list[str], str]:\n \"\"\"Names of parent's direct children in a region pack's admin table, and the pack code ([] and \"\" when no pack knows the parent).\"\"\"\n pk, kids = regions.admin_children(parent)\n return sorted({k[\"name\"] for k in kids}), (pk[\"code\"] if pk else \"\")\n\n\ndef urban(name: str, proxy: str | None, cache: Path, within: str | None) -> dict:\n \"\"\"Built-up area: the largest contiguous block of OSM land-use areas. When land use isn't mapped, fall back to a place=city/town node and estimate a radius from population.\"\"\"\n rs = find_relation(name, proxy, cache, within)\n if not rs:\n sys.exit(f\"OSM has no admin area named \\\"{name}\\\": try the full name or add --within\")\n r = rs[0]\n ql = (f'[out:json][timeout:180];rel({r[\"osm_id\"]});map_to_area->.a;'\n f'way[\"landuse\"~\"^(residential|commercial|retail|industrial)$\"](area.a);out bb;')\n els = osm.run(ql, proxy, cache).get(\"elements\", [])\n boxes = [(e[\"bounds\"][\"minlat\"], e[\"bounds\"][\"minlon\"], e[\"bounds\"][\"maxlat\"], e[\"bounds\"][\"maxlon\"])\n for e in els if e.get(\"bounds\")]\n out = {\"name\": r[\"name\"], \"admin_bbox\": r[\"bbox\"], \"admin_bbox_km2\": r[\"bbox_km2\"], \"landuse_polygons\": len(boxes)}\n # Admin seat: the place=city/town node with the largest population; prefer the land-use block nearest to it (when OSM is only half mapped, the largest block is often another town)\n ql2 = (f'[out:json][timeout:120];rel({r[\"osm_id\"]});map_to_area->.a;'\n f'node[\"place\"~\"^(city|town)$\"](area.a);out;')\n nodes = osm.run(ql2, proxy, cache).get(\"elements\", [])\n seat = None\n if nodes:\n def pop(n):\n return int(re.sub(r\"\\D\", \"\", (n.get(\"tags\") or {}).get(\"population\", \"0\") or \"0\") or 0)\n best = max(nodes, key=lambda n: (pop(n), (n.get(\"tags\") or {}).get(\"place\") == \"city\"))\n seat = {\"name\": (best.get(\"tags\") or {}).get(\"name\", \"\"), \"ll\": (best[\"lat\"], best[\"lon\"]), \"population\": pop(best)}\n out[\"seat\"] = seat\n if len(boxes) >= 3:\n # Connected blocks on a 2 km grid; take the block with the largest area\n cs = 2 / 110.574\n cell_of = {}\n for i, b in enumerate(boxes):\n cell_of.setdefault((int(((b[0] + b[2]) / 2) / cs), int(((b[1] + b[3]) / 2) / cs)), []).append(i)\n seen, blocks = set(), []\n for c0 in cell_of:\n if c0 in seen:\n continue\n stack, comp = [c0], []\n seen.add(c0)\n while stack:\n c = stack.pop()\n comp.extend(cell_of[c])\n for di in (-1, 0, 1):\n for dj in (-1, 0, 1):\n nb = (c[0] + di, c[1] + dj)\n if nb in cell_of and nb not in seen:\n seen.add(nb)\n stack.append(nb)\n blocks.append(comp)\n def block_bbox(comp):\n return [min(boxes[i][0] for i in comp), min(boxes[i][1] for i in comp),\n max(boxes[i][2] for i in comp), max(boxes[i][3] for i in comp)]\n\n def block_area(comp):\n return sum(_bbox_km2(list(boxes[i])) for i in comp)\n\n chosen, how = None, \"\"\n if seat:\n near = [c for c in blocks if geo.distance(seat[\"ll\"], (((bb := block_bbox(c))[0] + bb[2]) / 2, (bb[1] + bb[3]) / 2)) \u003c= 12000]\n if near:\n chosen, how = max(near, key=block_area), f\"contiguous land use nearest the admin seat {seat['name']}\"\n if chosen is None:\n chosen, how = max(blocks, key=block_area), \"largest contiguous land use (no admin seat node found; may be another town)\"\n bb = block_bbox(chosen)\n if seat and not (bb[0] \u003c= seat[\"ll\"][0] \u003c= bb[2] and bb[1] \u003c= seat[\"ll\"][1] \u003c= bb[3]):\n # The block doesn't cover the admin seat: merge in 1.5 km around the seat\n s2, w2 = geo.dest(geo.dest(seat[\"ll\"], 180, 1500), 270, 1500)\n n2, e2 = geo.dest(geo.dest(seat[\"ll\"], 0, 1500), 90, 1500)\n bb = [min(bb[0], s2), min(bb[1], w2), max(bb[2], n2), max(bb[3], e2)]\n how += \"; merged in 1.5 km around the admin seat\"\n out.update({\"urban_bbox\": [round(v, 5) for v in bb], \"urban_bbox_km2\": round(_bbox_km2(bb), 1),\n \"urban_landuse_km2\": round(block_area(chosen), 1), \"blocks\": len(blocks), \"source\": \"landuse\", \"how\": how})\n return out\n if seat:\n pop = seat[\"population\"]\n rad = max(1500.0, min(12000.0, math.sqrt(max(pop, 20000) / 6000 / math.pi) * 1000)) # about 6000 people/km²\n s, w = geo.dest(geo.dest(seat[\"ll\"], 180, rad), 270, rad)\n n, e = geo.dest(geo.dest(seat[\"ll\"], 0, rad), 90, rad)\n out.update({\"urban_bbox\": [round(s, 5), round(w, 5), round(n, 5), round(e, 5)],\n \"urban_bbox_km2\": round(_bbox_km2([s, w, n, e]), 1), \"source\": \"place-node\",\n \"place\": seat[\"name\"], \"population\": pop,\n \"note\": \"OSM has no land use mapped; built-up radius estimated from the place node's population, order of magnitude only\"})\n return out\n out[\"note\"] = \"OSM has neither land use nor a place node: draw scan_bbox yourself on satellite imagery\"\n return out\n\n\ndef cost(bbox: list[float], zoom: int, cell: int, cols: int) -> dict:\n s, w, n, e = bbox\n mpp = 156543.03392 * math.cos(math.radians((s + n) / 2)) / 2 ** zoom\n step = cell * mpp * 0.9\n rows_ = max(1, math.ceil((n - s) * 110574 / step))\n cols_ = max(1, math.ceil((e - w) * 111320 * math.cos(math.radians((s + n) / 2)) / step))\n cells = rows_ * cols_\n return {\"bbox\": bbox, \"km2\": round(_bbox_km2(bbox), 1), \"zoom\": zoom, \"cell_px\": cell, \"cell_m\": round(cell * mpp),\n \"cells\": cells, \"pages\": math.ceil(cells / (cols * cols)), \"per_page\": cols * cols}\n\n\ndef _neg_coords(argv: list[str]) -> list[str]:\n return [\" \" + a if re.match(r\"^-\\d[\\d.]*(,-?[\\d.]+)+$\", a) else a for a in argv]\n\n\ndef main() -> None:\n ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)\n ap.add_argument(\"--proxy\", default=os.environ.get(\"GEO_PROXY\"), help=PROXY_HELP)\n ap.add_argument(\"--cache\", type=Path, default=Path(\".geo-cache/osm\"))\n sub = ap.add_subparsers(dest=\"cmd\", required=True)\n\n def common(sp):\n sp.add_argument(\"--proxy\", default=argparse.SUPPRESS)\n sp.add_argument(\"--cache\", type=Path, default=argparse.SUPPRESS)\n\n c = sub.add_parser(\"children\", help=\"every subordinate admin area of an admin area (with bbox); use when candidates must be listed \\\"in full\\\"\")\n common(c)\n c.add_argument(\"name\")\n c.add_argument(\"--level\", type=int, help=\"OSM admin_level; if omitted, try downward from the parent\")\n c.add_argument(\"--within\", help=\"parent admin area name, to disambiguate same names\")\n c.add_argument(\"--out\", type=Path, help=\"write {name: {bbox, bbox_km2, center, ...}}\")\n\n i = sub.add_parser(\"info\")\n common(i)\n i.add_argument(\"name\")\n i.add_argument(\"--within\")\n i.add_argument(\"--level\", type=int)\n\n u = sub.add_parser(\"urban\", help=\"built-up extent: scan cost is computed from this, not from the whole admin area\")\n common(u)\n u.add_argument(\"name\")\n u.add_argument(\"--within\")\n u.add_argument(\"--out\", type=Path)\n\n k = sub.add_parser(\"cost\", help=\"how many pages a bbox takes at tiles.py sheet --grid settings\")\n k.add_argument(\"--bbox\", required=True)\n k.add_argument(\"--zoom\", type=int, default=16)\n k.add_argument(\"--cell\", type=int, default=320)\n k.add_argument(\"--cols\", type=int, default=5)\n\n args = ap.parse_args(_neg_coords(sys.argv[1:]))\n cache = _cache(args)\n if args.cmd == \"cost\":\n print(json.dumps(cost([float(v) for v in args.bbox.split(\",\")], args.zoom, args.cell, args.cols), ensure_ascii=False))\n return\n if args.cmd == \"info\":\n rows = find_relation(args.name, args.proxy, cache, args.within, args.level)\n if not rows:\n sys.exit(\"Not found; try the full name or add --within\")\n for r in rows[:8]:\n print(json.dumps(r, ensure_ascii=False))\n return\n if args.cmd == \"urban\":\n out = urban(args.name, args.proxy, cache, args.within)\n if args.out:\n args.out.write_text(json.dumps(out, ensure_ascii=False, indent=1), encoding=\"utf-8\")\n print(json.dumps(out, ensure_ascii=False, indent=1))\n return\n if args.cmd == \"children\":\n p, rows = children(args.name, args.proxy, cache, args.level, args.within)\n also, also_missing = also_children(p, rows, args.proxy, cache) if rows and not args.level else ([], [])\n rows += also\n local, code = _local_children(p[\"name\"])\n names = {r[\"name\"] for r in rows}\n pk = regions.resolve(code) or {}\n missing = [n for n in local if n not in names and not any(regions.strip_suffix(n, pk) == regions.strip_suffix(m, pk) for m in names)]\n out = {r[\"name\"]: {k: r[k] for k in (\"osm_id\", \"admin_level\", \"bbox\", \"bbox_km2\", \"center\", \"name_en\")} for r in rows}\n for n in missing:\n out[n] = {\"osm_id\": None, \"admin_level\": None, \"bbox\": None, \"bbox_km2\": None, \"center\": None,\n \"name_en\": \"\", \"note\": \"only in the local admin table, no OSM relation found: fill in the bbox with info/urban, or estimate from the parent's extent for now\"}\n lvl = rows[0][\"admin_level\"] if rows else None\n print(f\"{p['name']} (admin_level {p['admin_level']}) children admin_level {lvl}: OSM {len(rows)}\"\n + (f\", {code} pack's admin table adds {len(missing)} ({', '.join(missing)})\" if missing else \"\")\n + (f\"; region pack also lists {', '.join(r['name'] for r in also)}\" if also else \"\")\n + (f\"; not found in OSM: {', '.join(also_missing)}\" if also_missing else \"\")\n + (\"; no region pack's admin table has this parent, can't check the list is complete\" if not local else \"\"))\n for name, r in sorted(out.items(), key=lambda kv: -(kv[1][\"bbox_km2\"] or 0)):\n b = r[\"bbox\"]\n print(f\" {name:\u003c14} {r['bbox_km2'] or '?':>9} km² {b if b else '(no bbox)'}\")\n if args.out:\n args.out.write_text(json.dumps(out, ensure_ascii=False, indent=1), encoding=\"utf-8\")\n print(f\"-> {args.out}\")\n\n\nif __name__ == \"__main__\":\n # Chinese Windows outputs GBK by default: it crashes on m², ñ, and Chinese text the agent reads comes out garbled\n sys.stdout.reconfigure(encoding=\"utf-8\")\n sys.stderr.reconfigure(encoding=\"utf-8\", errors=\"backslashreplace\")\n main()\n","content_type":"text/x-python; charset=utf-8","language":"python","size":21726,"content_sha256":"82baa7eea1be14138e3c9e8461032ba0b34c7067b5235b158a70af51ae8299c7"},{"filename":"scripts/geo.py","content":"#!/usr/bin/env python3\n# /// script\n# requires-python = \">=3.10\"\n# dependencies = [\"numpy\", \"pillow\"]\n# ///\n\"\"\"Coordinate-system conversion + bearing/distance + camera geometry. All other scripts import this file.\n\nCoordinate systems:\n wgs WGS84: GPS / Google satellite imagery / OSM\n gcj GCJ-02: Amap (Gaode) / Tencent / Google China road map\n bd BD-09 lat/lon: Baidu\n bdmc Baidu Mercator plane coordinates (the @x,y in Baidu Maps URLs, the x,y of the panorama API)\n\nAll lat/lon arguments are always in (lat, lon) order.\n\nCLI examples:\n geo.py convert --from bdmc --to wgs 12697689.83 2568072.49\n geo.py bearing 22.6045,114.0520 22.6072,114.0564\n geo.py dest 22.6045,114.0520 --bearing 47 --dist 120\n geo.py range --real 55 --pixels 195 --image-width 1279 --hfov 53\n geo.py range --real 300 --pixels 420 --image-width 1080 --hfov 12:70 # zoom unknown: field-of-view range gives a distance range\n geo.py line --near 22.6060,114.0550 --far 22.6072,114.0564 --range 50:1500 --out line.json\n geo.py intersect --align1 N1lat,N1lon:F1lat,F1lon --align2 N2lat,N2lon:F2lat,F2lon --sigma 1\n geo.py bearings --at \u003clat,lon> --geojson buildings.geojson --target 306 --tol 8 # bearing first, then identify the structure: which building lies on that bearing\n geo.py frame --at 22.60,114.10 --anchor A:22.61,114.05:px=1000 --width 1080 --hfov 12:70 \\\\\n --pt B:22.62,114.06:h=300:w=60 --pt C:22.58,114.04:h=150:w=40 # compute before excluding: should B and C be in the frame\n geo.py spacing --cols '39,133,219,296,369;745,788,829' --line rail.geojson --line-name 城际 --span 32 \\\\\n --center 35.4983,138.7688 --radius 1500 --grid 50 --headings 55:115 --focals 1200:1500 \\\\\n --cx 640 --out spacing.json # pixel columns of a row of bridge piers → camera position, heading, focal length (7.7)\n\"\"\"\nfrom __future__ import annotations\n\nimport argparse\nimport json\nimport math\nimport os\nimport re\nimport sys\nimport time\nfrom pathlib import Path\n\n# ---------------------------------------------------------------- coordinate-system conversion\n\n_A = 6378245.0\n_EE = 0.00669342162296594323\n\n\ndef _out_of_china(lat: float, lon: float) -> bool:\n return not (73.66 \u003c lon \u003c 135.05 and 3.86 \u003c lat \u003c 53.55)\n\n\ndef _t_lat(x: float, y: float) -> float:\n r = -100 + 2 * x + 3 * y + 0.2 * y * y + 0.1 * x * y + 0.2 * math.sqrt(abs(x))\n r += (20 * math.sin(6 * x * math.pi) + 20 * math.sin(2 * x * math.pi)) * 2 / 3\n r += (20 * math.sin(y * math.pi) + 40 * math.sin(y / 3 * math.pi)) * 2 / 3\n r += (160 * math.sin(y / 12 * math.pi) + 320 * math.sin(y * math.pi / 30)) * 2 / 3\n return r\n\n\ndef _t_lon(x: float, y: float) -> float:\n r = 300 + x + 2 * y + 0.1 * x * x + 0.1 * x * y + 0.1 * math.sqrt(abs(x))\n r += (20 * math.sin(6 * x * math.pi) + 20 * math.sin(2 * x * math.pi)) * 2 / 3\n r += (20 * math.sin(x * math.pi) + 40 * math.sin(x / 3 * math.pi)) * 2 / 3\n r += (150 * math.sin(x / 12 * math.pi) + 300 * math.sin(x / 30 * math.pi)) * 2 / 3\n return r\n\n\ndef _gcj_delta(lat: float, lon: float) -> tuple[float, float]:\n dlat = _t_lat(lon - 105, lat - 35)\n dlon = _t_lon(lon - 105, lat - 35)\n rad = lat / 180 * math.pi\n m = 1 - _EE * math.sin(rad) ** 2\n sm = math.sqrt(m)\n dlat = dlat * 180 / ((_A * (1 - _EE)) / (m * sm) * math.pi)\n dlon = dlon * 180 / (_A / sm * math.cos(rad) * math.pi)\n return dlat, dlon\n\n\ndef wgs2gcj(lat: float, lon: float) -> tuple[float, float]:\n if _out_of_china(lat, lon):\n return lat, lon\n dlat, dlon = _gcj_delta(lat, lon)\n return lat + dlat, lon + dlon\n\n\ndef gcj2wgs(lat: float, lon: float) -> tuple[float, float]:\n \"\"\"Iterative inverse, error \u003c 0.5 m.\"\"\"\n if _out_of_china(lat, lon):\n return lat, lon\n wlat, wlon = lat, lon\n for _ in range(5):\n glat, glon = wgs2gcj(wlat, wlon)\n wlat += lat - glat\n wlon += lon - glon\n return wlat, wlon\n\n\n_XPI = math.pi * 3000 / 180\n\n\ndef gcj2bd(lat: float, lon: float) -> tuple[float, float]:\n z = math.hypot(lon, lat) + 0.00002 * math.sin(lat * _XPI)\n th = math.atan2(lat, lon) + 0.000003 * math.cos(lon * _XPI)\n return z * math.sin(th) + 0.006, z * math.cos(th) + 0.0065\n\n\ndef bd2gcj(lat: float, lon: float) -> tuple[float, float]:\n x, y = lon - 0.0065, lat - 0.006\n z = math.hypot(x, y) - 0.00002 * math.sin(y * _XPI)\n th = math.atan2(y, x) - 0.000003 * math.cos(x * _XPI)\n return z * math.sin(th), z * math.cos(th)\n\n\n_MCBAND = [12890594.86, 8362377.87, 5591021, 3481989.83, 1678043.12, 0]\n_MC2LL = [\n [1.410526172116255e-8, 0.00000898305509648872, -1.9939833816331, 200.9824383106796, -187.2403703815547, 91.6087516669843, -23.38765649603339, 2.57121317296198, -0.03801003308653, 17337981.2],\n [-7.435856389565537e-9, 0.000008983055097726239, -0.78625201886289, 96.32687599759846, -1.85204757529826, -59.36935905485877, 47.40033549296737, -16.50741931063887, 2.28786674699375, 10260144.86],\n [-3.030883460898826e-8, 0.00000898305509983578, 0.30071316287616, 59.74293618442277, 7.357984074871, -25.38371002664745, 13.45380521110908, -3.29883767235584, 0.32710905363475, 6856817.37],\n [-1.981981304930552e-8, 0.000008983055099779535, 0.03278182852591, 40.31678527705744, 0.65659298677277, -4.44255534477492, 0.85341911805263, 0.12923347998204, -0.04625736007561, 4482777.06],\n [3.09191371068437e-9, 0.000008983055096812155, 0.00006995724062, 23.10934304144901, -0.00023663490511, -0.6321817810242, -0.00663494467273, 0.03430082397953, -0.00466043876332, 2555164.4],\n [2.890871144776878e-9, 0.000008983055095805407, -3.068298e-8, 7.47137025468032, -0.00000353937994, -0.02145144861037, -0.00001234426596, 0.00010322952773, -0.00000323890364, 826088.5],\n]\n_LLBAND = [75, 60, 45, 30, 15, 0]\n_LL2MC = [\n [-0.0015702102444, 111320.7020616939, 1704480524535203, -10338987376042340, 26112667856603880, -35149669176653700, 26595700718403920, -10725012454188240, 1800819912950474, 82.5],\n [0.0008277824516172526, 111320.7020463578, 647795574.6671607, -4082003173.641316, 10774905663.51142, -15171875531.51559, 12053065338.62167, -5124939663.577472, 913311935.9512032, 67.5],\n [0.00337398766765, 111320.7020202162, 4481351.045890365, -23393751.19931662, 79682215.47186455, -115964993.2797253, 97236711.15602145, -43661946.33752821, 8477230.501135234, 52.5],\n [0.00220636496208, 111320.7020209128, 51751.86112841131, 3796837.749470245, 992013.7397791013, -1221952.21711287, 1340652.697009075, -620943.6990984312, 144416.9293806241, 37.5],\n [-0.0003441963504368392, 111320.7020576856, 278.2353980772752, 2485758.690035394, 6070.750963243378, 54821.18345352118, 9540.606633304236, -2710.55326746645, 1405.483844121726, 22.5],\n [-0.0003218135878613132, 111320.7020701615, 0.00369383431289, 823725.6402795718, 0.46104986909093, 2351.343141331292, 1.58060784298199, 8.77738589078284, 0.37238884252424, 7.45],\n]\n\n\ndef _poly(x: float, y: float, c: list[float]) -> tuple[float, float]:\n fx = c[0] + c[1] * abs(x)\n t = abs(y) / c[9]\n fy = c[2] + c[3] * t + c[4] * t ** 2 + c[5] * t ** 3 + c[6] * t ** 4 + c[7] * t ** 5 + c[8] * t ** 6\n return (-fx if x \u003c 0 else fx), (-fy if y \u003c 0 else fy)\n\n\ndef bdmc2bd(x: float, y: float) -> tuple[float, float]:\n c = next(_MC2LL[i] for i, b in enumerate(_MCBAND) if abs(y) >= b)\n lon, lat = _poly(x, y, c)\n return lat, lon\n\n\ndef bd2bdmc(lat: float, lon: float) -> tuple[float, float]:\n lat = max(min(lat, 74), -74)\n c = next(_LL2MC[i] for i, b in enumerate(_LLBAND) if abs(lat) >= b)\n return _poly(lon, lat, c)\n\n\ndef convert(a: float, b: float, src: str, dst: str) -> tuple[float, float]:\n \"\"\"a,b are lat,lon for the lat/lon systems; x,y for bdmc.\"\"\"\n if src == dst:\n return a, b\n # normalize to wgs first\n if src == \"wgs\":\n lat, lon = a, b\n elif src == \"gcj\":\n lat, lon = gcj2wgs(a, b)\n elif src == \"bd\":\n lat, lon = gcj2wgs(*bd2gcj(a, b))\n elif src == \"bdmc\":\n lat, lon = gcj2wgs(*bd2gcj(*bdmc2bd(a, b)))\n else:\n raise ValueError(src)\n if dst == \"wgs\":\n return lat, lon\n g = wgs2gcj(lat, lon)\n if dst == \"gcj\":\n return g\n bd = gcj2bd(*g)\n if dst == \"bd\":\n return bd\n if dst == \"bdmc\":\n return bd2bdmc(*bd)\n raise ValueError(dst)\n\n\n# ---------------------------------------------------------------- bearing and distance\n\n_R = 6371008.8\n\n\ndef distance(p: tuple[float, float], q: tuple[float, float]) -> float:\n la1, lo1, la2, lo2 = map(math.radians, (*p, *q))\n h = math.sin((la2 - la1) / 2) ** 2 + math.cos(la1) * math.cos(la2) * math.sin((lo2 - lo1) / 2) ** 2\n return 2 * _R * math.asin(math.sqrt(h))\n\n\ndef bearing(p: tuple[float, float], q: tuple[float, float]) -> float:\n \"\"\"Compass bearing from p looking at q, 0=north, clockwise.\"\"\"\n la1, lo1, la2, lo2 = map(math.radians, (*p, *q))\n y = math.sin(lo2 - lo1) * math.cos(la2)\n x = math.cos(la1) * math.sin(la2) - math.sin(la1) * math.cos(la2) * math.cos(lo2 - lo1)\n return (math.degrees(math.atan2(y, x)) + 360) % 360\n\n\ndef dest(p: tuple[float, float], brg: float, dist_m: float) -> tuple[float, float]:\n la1, lo1 = map(math.radians, p)\n b = math.radians(brg)\n d = dist_m / _R\n la2 = math.asin(math.sin(la1) * math.cos(d) + math.cos(la1) * math.sin(d) * math.cos(b))\n lo2 = lo1 + math.atan2(math.sin(b) * math.sin(d) * math.cos(la1), math.cos(d) - math.sin(la1) * math.sin(la2))\n return math.degrees(la2), math.degrees(lo2)\n\n\n# ---------------------------------------------------------------- Web Mercator tiles\n\ndef ll2px(zoom: int, lat: float, lon: float) -> tuple[float, float]:\n \"\"\"lat/lon → global pixel coordinates at this zoom level (256 tiles).\"\"\"\n n = 256 * 2 ** zoom\n x = (lon + 180) / 360 * n\n y = (1 - math.asinh(math.tan(math.radians(lat))) / math.pi) / 2 * n\n return x, y\n\n\ndef px2ll(zoom: int, x: float, y: float) -> tuple[float, float]:\n n = 256 * 2 ** zoom\n lon = x / n * 360 - 180\n lat = math.degrees(math.atan(math.sinh(math.pi * (1 - 2 * y / n))))\n return lat, lon\n\n\ndef meters_per_px(zoom: int, lat: float) -> float:\n return 156543.03392 * math.cos(math.radians(lat)) / 2 ** zoom\n\n\n# ---------------------------------------------------------------- camera geometry\n\ndef focal_px(image_width_px: float, hfov_deg: float) -> float:\n \"\"\"Focal length in pixels under the pinhole model.\"\"\"\n return (image_width_px / 2) / math.tan(math.radians(hfov_deg) / 2)\n\n\ndef range_from_size(real_size_m: float, size_px: float, image_width_px: float, hfov_deg: float) -> float:\n \"\"\"Known real width of an object and its pixel width in the frame → depth distance (meters).\"\"\"\n return real_size_m * focal_px(image_width_px, hfov_deg) / size_px\n\n\ndef angle_from_center(px: float, center_px: float, image_width_px: float, hfov_deg: float) -> float:\n \"\"\"Horizontal angle of a point in the frame relative to the optical axis (degrees, right positive). Same for vertical: pass vertical pixels.\"\"\"\n return math.degrees(math.atan((px - center_px) / focal_px(image_width_px, hfov_deg)))\n\n\n# Common phone main-camera field of view (35mm equivalent focal length → long-side/short-side field of view for a 4:3 frame)\ndef fov_from_equiv_focal(focal_mm: float, aspect: tuple[int, int] = (4, 3)) -> tuple[float, float]:\n diag = 43.2666\n w, h = aspect\n k = diag / math.hypot(w, h)\n long_side, short_side = w * k, h * k\n return (math.degrees(2 * math.atan(long_side / 2 / focal_mm)),\n math.degrees(2 * math.atan(short_side / 2 / focal_mm)))\n\n\n# ---------------------------------------------------------------- sight-line intersection\n\ndef _enu(p: tuple[float, float], o: tuple[float, float]) -> tuple[float, float]:\n \"\"\"Local plane coordinates with o as origin (meters, east, north). Error is negligible within 50 km.\"\"\"\n return ((p[1] - o[1]) * math.cos(math.radians(o[0])) * 111320.0, (p[0] - o[0]) * 110540.0)\n\n\ndef _ll(xy: tuple[float, float], o: tuple[float, float]) -> tuple[float, float]:\n return (o[0] + xy[1] / 110540.0, o[1] + xy[0] / (math.cos(math.radians(o[0])) * 111320.0))\n\n\ndef ray_from_alignment(near: tuple[float, float], far: tuple[float, float]) -> tuple[tuple[float, float], float]:\n \"\"\"In the frame, near is in front of far (or vertically aligned) → the camera position is on the extension of far→near, beyond near. Returns (origin, bearing).\"\"\"\n return near, bearing(far, near)\n\n\ndef ray_from_sighting(landmark: tuple[float, float], seen_bearing: float) -> tuple[tuple[float, float], float]:\n \"\"\"The bearing from the camera position to landmark is seen_bearing → the camera position is on the ray from landmark in the opposite direction.\"\"\"\n return landmark, (seen_bearing + 180) % 360\n\n\ndef intersect_rays(r1, r2) -> tuple[tuple[float, float], float, float, float] | None:\n \"\"\"Intersect two rays. Returns (intersection, crossing angle°, distance along ray 1 in m, distance along ray 2 in m); returns None if parallel or the intersection is behind a ray.\"\"\"\n (p1, b1), (p2, b2) = r1, r2\n o = p1\n x1, y1 = _enu(p1, o)\n x2, y2 = _enu(p2, o)\n d1 = (math.sin(math.radians(b1)), math.cos(math.radians(b1)))\n d2 = (math.sin(math.radians(b2)), math.cos(math.radians(b2)))\n den = d1[0] * d2[1] - d1[1] * d2[0]\n if abs(den) \u003c 1e-9:\n return None\n t1 = ((x2 - x1) * d2[1] - (y2 - y1) * d2[0]) / den\n t2 = ((x2 - x1) * d1[1] - (y2 - y1) * d1[0]) / den\n if t1 \u003c 0 or t2 \u003c 0:\n return None\n pt = _ll((x1 + t1 * d1[0], y1 + t1 * d1[1]), o)\n ang = abs((b1 - b2 + 180) % 360 - 180)\n return pt, min(ang, 180 - ang), t1, t2\n\n\ndef intersect_with_error(r1, r2, sigma_deg: float) -> dict | None:\n \"\"\"Intersection of two rays, plus the maximum distance the intersection moves when each bearing is offset by ±sigma (used as the error radius).\"\"\"\n base = intersect_rays(r1, r2)\n if base is None:\n return None\n pt, ang, t1, t2 = base\n spread = 0.0\n for s1 in (-sigma_deg, sigma_deg):\n for s2 in (-sigma_deg, sigma_deg):\n q = intersect_rays((r1[0], r1[1] + s1), (r2[0], r2[1] + s2))\n spread = max(spread, math.inf if q is None else distance(pt, q[0]))\n return {\"point\": [round(pt[0], 7), round(pt[1], 7)], \"crossing_angle_deg\": round(ang, 1),\n \"dist_from_ray1_origin_m\": round(t1), \"dist_from_ray2_origin_m\": round(t2),\n \"error_radius_m\": None if math.isinf(spread) else round(spread)}\n\n\n# ---------------------------------------------------------------- frame prediction: should a landmark appear in the frame\n\ndef _band_f(s: str) -> tuple[float, float]:\n \"\"\"'65' → (65, 65); '12:70' → (12, 70).\"\"\"\n if \":\" in s:\n a, b = s.split(\":\")\n return float(a), float(b)\n return float(s), float(s)\n\n\ndef _landmark(s: str) -> dict:\n \"\"\"name:lat,lon[:h=height m][:w=width m][:px=pixel x] → dict.\"\"\"\n parts = s.split(\":\")\n d = {\"name\": parts[0], \"ll\": _pair(parts[1])}\n for kv in parts[2:]:\n k, v = kv.split(\"=\")\n d[k] = float(v)\n return d\n\n\ndef _drop_m(dist_m: float) -> float:\n \"\"\"Height by which Earth curvature + standard atmospheric refraction (k≈0.13) make a distant object look lower.\"\"\"\n return dist_m * dist_m / (2 * _R) * (1 - 0.13)\n\n\ndef frame_predict(cam: tuple[float, float], heading: float, hfov: float, width_px: float,\n pts: list[dict], cam_h: float = 0.0) -> list[dict]:\n \"\"\"Given camera position, heading and horizontal field of view, compute for each landmark: angle from the optical axis, pixel x, whether it is in the frame, angular height and angular width.\"\"\"\n f = focal_px(width_px, hfov)\n out = []\n for p in pts:\n dist = distance(cam, p[\"ll\"])\n rel = (bearing(cam, p[\"ll\"]) - heading + 540) % 360 - 180\n row = {\"name\": p[\"name\"], \"dist_m\": round(dist), \"rel_deg\": round(rel, 2)}\n half_w = math.degrees(math.atan2(p.get(\"w\", 0) / 2, dist)) if dist > 0 else 0\n row[\"half_width_deg\"] = round(half_w, 3)\n if abs(rel) \u003c 90:\n x = width_px / 2 + f * math.tan(math.radians(rel))\n row[\"px\"] = round(x)\n lo = width_px / 2 + f * math.tan(math.radians(rel - half_w))\n hi = width_px / 2 + f * math.tan(math.radians(rel + half_w))\n row[\"in_frame\"] = \"full\" if lo >= 0 and hi \u003c= width_px else (\"partial\" if hi >= 0 and lo \u003c= width_px else \"out\")\n else:\n row[\"px\"], row[\"in_frame\"] = None, \"behind\"\n if \"h\" in p and dist > 0:\n top = p[\"h\"] - _drop_m(dist) - cam_h\n row[\"top_elev_deg\"] = round(math.degrees(math.atan2(top, dist)), 3)\n row[\"angular_height_deg\"] = round(math.degrees(math.atan2(p[\"h\"], dist)), 3)\n row[\"height_px\"] = round(f * math.tan(math.radians(row[\"angular_height_deg\"])))\n out.append(row)\n return out\n\n\ndef occluders(cam: tuple[float, float], pts: list[dict], cam_h: float = 0.0) -> list[str]:\n \"\"\"Pairwise check: can a nearer landmark (with w, h given) fully block a farther one.\"\"\"\n notes = []\n info = []\n for p in pts:\n d = distance(cam, p[\"ll\"])\n info.append((p, d, bearing(cam, p[\"ll\"])))\n for near, dn, bn in info:\n if \"w\" not in near or \"h\" not in near:\n for far, df, bf in info:\n gap = abs((bf - bn + 540) % 360 - 180)\n if far is not near and df > dn and gap \u003c 3:\n notes.append(f\"{far['name']} may be blocked by {near['name']}: their bearings differ by only {gap:.2f}°, \"\n f\"{near['name']} has no w, h, can't compute (add them and rerun)\")\n continue\n for far, df, bf in info:\n if far is near or df \u003c= dn:\n continue\n half_n = math.degrees(math.atan2(near[\"w\"] / 2, dn))\n half_f = math.degrees(math.atan2(far.get(\"w\", 0) / 2, df))\n gap = abs((bf - bn + 540) % 360 - 180)\n top_n = math.degrees(math.atan2(near[\"h\"] - _drop_m(dn) - cam_h, dn))\n top_f = math.degrees(math.atan2(far.get(\"h\", 0) - _drop_m(df) - cam_h, df)) if \"h\" in far else None\n if gap + half_f \u003c= half_n and (top_f is None or top_n >= top_f):\n notes.append(f\"{far['name']} may be fully blocked by {near['name']} (bearing difference {gap:.2f}°, {near['name']} half-width {half_n:.2f}°)\")\n elif gap \u003c half_n + half_f and (top_f is None or top_n >= top_f):\n notes.append(f\"{far['name']} is partly blocked by {near['name']} (bearing difference {gap:.2f}°)\")\n elif gap \u003c half_n + half_f and top_f is not None:\n notes.append(f\"{far['name']} rises above {near['name']} and shows over the top ({top_f:.2f}° > {top_n:.2f}°)\")\n return notes\n\n\ndef _ring_positions(spec: str) -> dict:\n \"\"\"lat,lon:rmin:rmax:rstep:azstep → rings of candidate camera positions centered on a point.\"\"\"\n ll, rmin, rmax, rstep, azstep = spec.split(\":\")\n c = _pair(ll)\n out = {}\n r = float(rmin)\n while r \u003c= float(rmax) + 1e-6:\n az = 0.0\n while az \u003c 360 - 1e-6:\n out[f\"R{int(r)}m@{int(az)}\"] = list(dest(c, az, r))\n az += float(azstep)\n r += float(rstep)\n return out\n\n\n# ---------------------------------------------------------------- camera position from evenly spaced structures (spacing)\n\nSPACING_DOC = \"\"\"When a row of evenly spaced structures in the frame (viaduct piers, utility poles, street lights, guardrail posts) lies on a known line on the map,\nsolve for camera position, heading and focal length. Method in references/geometry.md 7.7.\n\nPrinciple: each structure's pixel column x → offset angle from the frame center atan((x-cx)/f) → a ray; intersect the ray with the GeoJSON polyline\nto get that structure's chainage (distance along the line). When camera position/heading/focal length are all right, the chainage difference between neighbors is constant and equals the standard span (--span).\nScore pier = dispersion CV of the chainage differences + |log(mean chainage difference / span)| (plus --mono-penalty if the chainage is not monotonic).\n\nWith spacing alone, the solution is a band along the sight-line direction (in practice scattered within about 800 m). Give --ridge to score jointly with the skyline\n(as in joint.py: tot = skyline rms + --flat-weight × flat-horizon penalty + --ridge-weight × spacing score),\nwhich in practice narrows it to about 300 m. The skyline part downloads elevation tiles (the AWS Terrain Tiles used by terrain.py, cached in --cache).\n\nGrouping in --cols: when the foreground breaks the structures into segments, group them with ';'; neighbor differences are only computed within a group (the difference across a break is not one span).\nThe monotonicity check still runs across all columns: the chainage of the whole row along the line must increase all the way or decrease all the way.\n\nOutput JSON:\n {\"line\": {…the selected line…},\n \"params\": {…all parameters of this run…},\n \"best\": \u003ccandidates[0]>,\n \"candidates\": [{\"tot\": total score, \"rms\": skyline rms (null without --ridge),\n \"pier\": spacing score, \"cam\": [lat, lon], \"f\": focal length px, \"H\": heading°,\n \"cc\": skyline pitch correction° (null without --ridge),\n \"span_m\": mean solved span m, \"d_first\": distance to the first structure m,\n \"d_last\": distance to the last structure m}, …top --top sorted by tot ascending]}\n The field names cam/f/H/cc follow the output of the script written on the spot in that real case (joint.py), so results from both can be compared directly.\"\"\"\n\n\ndef _spacing_col_val(v) -> float:\n \"\"\"One pixel column: a number, [x, …], or {\"col\": x, \"prominence\": …} as from imgprep.py piers.\"\"\"\n if isinstance(v, dict):\n if \"col\" not in v:\n sys.exit(f\"Structure record in the JSON has no col field: {v}\")\n return float(v[\"col\"])\n if isinstance(v, (list, tuple)):\n return float(v[0])\n return float(v)\n\n\ndef _spacing_cols(spec: str) -> list[list[float]]:\n \"\"\"Pixel-column groups: '39,133,219;745,788' → [[39,133,219],[745,788]].\n\n Also accepts a JSON file ('@path' or ending in .json): list, list[list], list[{\"col\": …}],\n {\"piers\": [{\"col\": …}, …]} (output of imgprep.py piers), {\"groups\": [[…]]}, {\"cols\": […]} are all recognized.\n Key priority is piers > groups > cols: \"cols\" in the imgprep.py piers output is the column search range [x0, x1], not structure columns.\n Each group is sorted ascending; groups are ordered by their first column.\"\"\"\n raw = spec\n if raw.startswith(\"@\") or raw.lower().endswith(\".json\"):\n with open(raw[1:] if raw.startswith(\"@\") else raw, encoding=\"utf-8\") as fh:\n data = json.load(fh)\n if isinstance(data, dict):\n for key in (\"piers\", \"groups\", \"cols\"):\n if key in data:\n data = data[key]\n break\n else:\n sys.exit(\"No piers / groups / cols found in the JSON\")\n if data is None:\n sys.exit(\"piers / groups / cols in the JSON is empty\")\n groups = data if (data and isinstance(data[0], (list, tuple))) else [data]\n groups = [[_spacing_col_val(v) for v in g] for g in groups]\n else:\n groups = [[float(v) for v in part.split(\",\") if v.strip() != \"\"] for part in raw.split(\";\")]\n groups = [sorted(g) for g in groups if g]\n groups.sort(key=lambda g: g[0])\n if not groups:\n sys.exit(\"--cols is empty\")\n return groups\n\n\ndef _spacing_focals(spec: str, step: float) -> list[float]:\n \"\"\"'1200,1281,1350' → list; '1200:1500' → arithmetic sequence with step. That real case used the list 1200,1281,1350,1430,1500.\"\"\"\n if \":\" in spec:\n parts = [float(v) for v in spec.split(\":\")]\n lo, hi = parts[0], parts[1]\n st = parts[2] if len(parts) > 2 else step\n n = int(round((hi - lo) / st))\n return [lo + st * k for k in range(n + 1)]\n return [float(v) for v in spec.split(\",\") if v.strip() != \"\"]\n\n\ndef _spacing_band(spec: str) -> tuple[float, float]:\n a, b = spec.split(\":\")\n return float(a), float(b)\n\n\ndef _spacing_line(path: str, name: str | None, index: int):\n \"\"\"Take one polyline from a GeoJSON, return (Nx2 [lon,lat] list, info dict).\n\n --line-name matches a substring of properties.name; for features with the same name (two tracks, one per direction) use --line-index to pick which one.\n Without a name: if there is only one line use it; if there are several use the longest and print a note.\"\"\"\n with open(path, encoding=\"utf-8\") as fh:\n g = json.load(fh)\n feats = g.get(\"features\") if isinstance(g, dict) and g.get(\"type\") == \"FeatureCollection\" else None\n if feats is None:\n feats = [g] if isinstance(g, dict) else list(g)\n cands = []\n for f in feats:\n geom = f.get(\"geometry\", f) or {}\n props = f.get(\"properties\") or {}\n parts = []\n if geom.get(\"type\") == \"LineString\":\n parts = [geom[\"coordinates\"]]\n elif geom.get(\"type\") == \"MultiLineString\":\n parts = list(geom[\"coordinates\"])\n for c in parts:\n if len(c) \u003c 2:\n continue\n ln = sum(math.dist(c[k], c[k + 1]) for k in range(len(c) - 1)) * 111000\n cands.append({\"name\": str(props.get(\"name\") or \"\"), \"coords\": c, \"length_m\": ln})\n if not cands:\n sys.exit(f\"No LineString in {path}\")\n if name:\n hit = [c for c in cands if name in c[\"name\"]]\n if not hit:\n names = sorted({c[\"name\"] for c in cands if c[\"name\"]})\n sys.exit(f\"No line whose name contains {name!r}; names in the file: {names}\")\n if index >= len(hit):\n sys.exit(f\"{len(hit)} lines have a name containing {name!r}; --line-index can go up to {len(hit) - 1} only\")\n pick = hit[index]\n note = f\"line {index} of the {len(hit)} whose name contains {name!r}\"\n elif len(cands) == 1:\n pick = cands[0]\n note = \"this is the only line in the file\"\n else:\n pick = max(cands, key=lambda c: c[\"length_m\"])\n note = f\"the file has {len(cands)} lines and no --line-name was given, so the longest was used; for another line give --line-name / --line-index\"\n info = {\"name\": pick[\"name\"], \"note\": note, \"points\": len(pick[\"coords\"]),\n \"length_m\": round(pick[\"length_m\"]),\n \"ends\": [[round(pick[\"coords\"][0][1], 6), round(pick[\"coords\"][0][0], 6)],\n [round(pick[\"coords\"][-1][1], 6), round(pick[\"coords\"][-1][0], 6)]]}\n return pick[\"coords\"], info\n\n\ndef _spacing_frame_note(path: str, d) -> None:\n \"\"\"--cols from imgprep.py piers have always been in the upright frame (EXIF orientation applied); a ridge.json\n without \"frame\" was written on the stored pixels. The two agree only when the photo has no EXIF orientation.\"\"\"\n if not isinstance(d, dict) or \"frame\" in d or not d.get(\"photo\"):\n return\n photo = Path(d[\"photo\"])\n if not photo.is_file():\n photo = Path(path).parent / photo\n try:\n from _image import open_image, orientation # noqa: PLC0415 — pillow only when a ridge file is given\n o = orientation(open_image(photo, load=False))\n except Exception: # noqa: BLE001 — the photo is only consulted for this note\n return\n if o != 1:\n print(f\"warning: {path} has no \\\"frame\\\": \\\"upright\\\": its ridge points are in the stored frame of {photo} (EXIF orientation {o}), \"\n f\"while --cols from imgprep.py piers are in the upright frame. Re-run terrain.py ridge so both are in the same frame.\",\n file=sys.stderr)\n\n\ndef _spacing_ridge(path: str) -> dict:\n \"\"\"Read the output of terrain.py ridge {\"ridge\": [[x,y]…], \"flat\": [x0,x1], \"hrow\":…, \"f0\":…}.\n\n Also recognizes two handwritten formats: {\"760\": 826, …} (column → ridgeline row) and a bare [[x,y]…].\"\"\"\n with open(path, encoding=\"utf-8\") as fh:\n d = json.load(fh)\n _spacing_frame_note(path, d)\n out = {\"flat\": None, \"hrow\": None, \"f0\": None}\n if isinstance(d, list):\n pts = d\n else:\n pts = d.get(\"ridge\")\n out[\"flat\"] = d.get(\"flat\")\n out[\"hrow\"] = d.get(\"hrow\")\n out[\"f0\"] = d.get(\"f0\")\n if pts is None:\n pts = [[float(k), float(v)] for k, v in d.items() if str(k).lstrip(\"-\").replace(\".\", \"\", 1).isdigit()]\n if isinstance(pts, dict):\n pts = [[float(k), float(v)] for k, v in pts.items()]\n pts = [[float(p[0]), float(p[1])] for p in pts]\n if len(pts) \u003c 4:\n sys.exit(f\"Too few ridgeline points in {path} ({len(pts)}), need at least 4\")\n out[\"pts\"] = sorted(pts)\n return out\n\n\ndef _spacing_run(args) -> dict:\n \"\"\"Grid search over camera position × heading × focal length, computing the spacing score for each (optionally joint with the skyline). Returns the dict written to --out.\"\"\"\n import numpy as np\n\n groups = _spacing_cols(args.cols)\n cols = [c for g in groups for c in g]\n if len(cols) \u003c 3:\n sys.exit(f\"--cols has only {len(cols)} columns, need at least 3 (geometry.md 7.7 recommends ≥6). \"\n \"If you passed a JSON file: in imgprep.py piers output the structure columns are in the piers field; the cols field is the search range\")\n pier = np.array(cols, float)\n\n pair_a, pair_b, off = [], [], 0 # neighbor pairs within a group: only neighbors in the same segment are one span\n for g in groups:\n pair_a += list(range(off, off + len(g) - 1))\n pair_b += list(range(off + 1, off + len(g)))\n off += len(g)\n if not pair_a:\n sys.exit(\"Every --cols group has only one column; can't compute neighbor spacing\")\n pair_a, pair_b = np.array(pair_a), np.array(pair_b)\n\n lat0, lon0 = args.center\n kx = 111320 * math.cos(math.radians(lat0))\n ky = 110540.0\n\n coords, line_info = _spacing_line(args.line, args.line_name, args.line_index)\n ll = np.array(coords, float)\n P = np.c_[(ll[:, 0] - lon0) * kx, (ll[:, 1] - lat0) * ky] # local meter coordinates (x east, y north)\n seg_a, seg_b = P[:-1], P[1:]\n e = seg_b - seg_a\n seg_len = np.hypot(*e.T)\n chain0 = np.r_[0, np.cumsum(seg_len)][:-1] # chainage at the start of each segment\n\n hs = np.arange(args.headings[0], args.headings[1] + 1e-9, args.heading_step)\n focals = _spacing_focals(args.focals, args.focal_step)\n xs = np.arange(-args.radius, args.radius + 1e-9, args.grid)\n print(f\"Line: {line_info['name'] or '(unnamed)'} {line_info['points']} points {line_info['length_m']} m ({line_info['note']})\")\n print(f\"Camera positions {len(xs)}×{len(xs)} cells × headings {len(hs)} × focal lengths {len(focals)}; {len(cols)} pixel columns in {len(groups)} groups, \"\n f\"standard span {args.span} m\")\n\n dem = ridge = None\n if args.ridge:\n sys.path.insert(0, str(Path(__file__).resolve().parent))\n import terrain # noqa: PLC0415 — only needed for the joint skyline; don't make other subcommands wait for it\n ridge = _spacing_ridge(args.ridge)\n hrow = args.hrow if args.hrow is not None else ridge[\"hrow\"]\n if hrow is None:\n sys.exit(\"The --ridge file has no hrow (the image row of the horizon in the photo); give one with --hrow\")\n rp = np.array(ridge[\"pts\"], float)\n rx, ry = rp[:, 0], rp[:, 1]\n flat = args.flat if args.flat is not None else (tuple(ridge[\"flat\"]) if ridge[\"flat\"] else None)\n flx = np.arange(flat[0], flat[1] + 1e-9, args.flat_step) if flat else None\n az = np.arange(0, 360, args.az_step)\n dist = args.sky_near * (args.sky_range / args.sky_near) ** (np.arange(args.sky_samples) / (args.sky_samples - 1))\n drop = dist ** 2 / (2 * terrain.R_EARTH) * (1 - terrain.K_REFRACTION) # Earth curvature + atmospheric refraction\n dem = terrain.DEM((lat0, lon0), args.dem_range, args.dem_zoom, args.cache, args.proxy)\n print(f\"Joint skyline: ridgeline {len(rx)} points, horizon row {hrow}, \"\n f\"flat-horizon columns {f'{flat[0]:g}–{flat[1]:g}' if flat else 'not used'}, DEM z{args.dem_zoom} radius {args.dem_range} m\")\n\n res, nfit, t0 = [], 0, time.time()\n for iy, yy in enumerate(xs):\n for xx in xs:\n w = seg_a - np.array([xx, yy])\n hor = None\n for f in focals:\n offs = np.degrees(np.arctan((pier - args.cx) / f)) # offset angle of each structure from the frame center\n a = np.radians(hs[:, None] + offs[None, :])\n dx, dy = np.sin(a)[..., None], np.cos(a)[..., None] # (heading, structure, 1)\n den = dx * e[:, 1] - dy * e[:, 0]\n with np.errstate(divide=\"ignore\", invalid=\"ignore\"):\n t = (w[:, 0] * e[:, 1] - w[:, 1] * e[:, 0]) / den # ray parameter = distance m\n u = (w[:, 0] * dy - w[:, 1] * dx) / den # segment parameter 0–1\n ok = (t > args.min_dist) & (u >= 0) & (u \u003c= 1)\n tt = np.where(ok, t, np.inf)\n i = np.argmin(tt, axis=2) # take the nearest intersection\n tmin = np.take_along_axis(tt, i[..., None], 2)[..., 0]\n valid = np.all(np.isfinite(tmin), axis=1)\n if not valid.any():\n continue\n uu = np.take_along_axis(u, i[..., None], 2)[..., 0]\n ch = chain0[i] + uu * seg_len[i] # chainage of each structure\n sp = np.abs(ch[:, pair_b] - ch[:, pair_a])\n m = sp.mean(axis=1)\n cv = sp.std(axis=1) / np.maximum(m, 1e-6)\n dch = np.diff(ch, axis=1)\n mono = np.all(dch > 0, axis=1) | np.all(dch \u003c 0, axis=1)\n ps = cv + np.abs(np.log(np.maximum(m, 1e-3) / args.span)) + (~mono) * args.mono_penalty\n ps[~valid] = np.inf\n k = int(np.argmin(ps))\n if not np.isfinite(ps[k]) or ps[k] > args.pier_max:\n continue\n nfit += 1\n H = float(hs[k])\n clat, clon = lat0 + yy / ky, lon0 + xx / kx\n rms = cc = None\n tot = float(ps[k])\n if dem is not None:\n if hor is None: # compute the horizon once per camera position, shared by all focal lengths\n g0 = float(dem.sample(np.array([clat]), np.array([clon]))[0])\n la, lo = terrain._dest_np(clat, clon, az, dist)\n hh = dem.sample(la, lo)\n hor = np.degrees(np.arctan2(hh - drop[None, :] - g0 - args.eye, dist[None, :])).max(axis=1)\n r_off = np.degrees(np.arctan((rx - args.cx) / f)) # ridgeline points: pixel column → bearing offset\n r_el = np.degrees(np.arctan((hrow - ry) / f)) # pixel row → elevation angle\n mr = np.interp((H + r_off) % 360, az, hor, period=360)\n diff = mr - r_el\n cc = float(np.clip(np.median(diff), -args.cc_max, args.cc_max)) # pitch correction (horizon row estimate was off)\n rms = float(np.sqrt(np.mean((diff - cc) ** 2)))\n fpen = 0.0\n if flx is not None:\n mf = np.interp((H + np.degrees(np.arctan((flx - args.cx) / f))) % 360, az, hor, period=360)\n fpen = float(np.mean(np.clip(mf - cc - args.flat_margin, 0, None))) # the stretch that should be flat must not show mountains\n tot = rms + args.flat_weight * fpen + args.ridge_weight * float(ps[k])\n res.append({\"tot\": tot, \"rms\": rms, \"pier\": float(ps[k]), \"cam\": [round(clat, 5), round(clon, 5)],\n \"f\": int(f) if float(f).is_integer() else f, \"H\": H, \"cc\": cc, \"span_m\": round(float(m[k]), 1),\n \"d_first\": round(float(tmin[k, 0])), \"d_last\": round(float(tmin[k, -1]))})\n if args.progress and (iy + 1) % args.progress == 0:\n print(f\" {iy + 1}/{len(xs)} rows, hits {nfit}, {time.time() - t0:.0f}s\", file=sys.stderr)\n\n res.sort(key=lambda r: r[\"tot\"])\n top = res[:args.top]\n return {\"line\": line_info,\n \"params\": {\"cols\": [[int(c) if float(c).is_integer() else c for c in g] for g in groups],\n \"span\": args.span, \"cx\": args.cx, \"center\": [lat0, lon0], \"radius\": args.radius,\n \"grid\": args.grid, \"headings\": list(args.headings), \"heading_step\": args.heading_step,\n \"focals\": focals, \"pier_max\": args.pier_max, \"min_dist\": args.min_dist,\n \"ridge\": args.ridge, \"ridge_weight\": args.ridge_weight if args.ridge else None,\n \"hrow\": (args.hrow if args.hrow is not None else (ridge[\"hrow\"] if ridge else None)),\n \"dem_zoom\": args.dem_zoom if args.ridge else None},\n \"n_fit\": nfit, \"best\": top[0] if top else None, \"candidates\": top}\n\n\n# ---------------------------------------------------------------- CLI\n\ndef _pair(s: str) -> tuple[float, float]:\n a, b = s.split(\",\")\n return float(a), float(b)\n\n\n\ndef _neg_coords(argv: list[str]) -> list[str]:\n \"\"\"argparse treats negative coordinates like -1.45,-48.5 as option names; prefixing a space makes them plain values (float ignores the space). Needed for any photo in the southern or western hemisphere.\"\"\"\n return [\" \" + a if re.match(r\"^-\\d[\\d.]*(,-?[\\d.]+)+$\", a) else a for a in argv]\n\n\ndef main() -> None:\n ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)\n sub = ap.add_subparsers(dest=\"cmd\", required=True)\n\n c = sub.add_parser(\"convert\", help=\"coordinate-system conversion\")\n c.add_argument(\"--from\", dest=\"src\", required=True, choices=[\"wgs\", \"gcj\", \"bd\", \"bdmc\"])\n c.add_argument(\"--to\", dest=\"dst\", required=True, choices=[\"wgs\", \"gcj\", \"bd\", \"bdmc\"])\n c.add_argument(\"a\", type=float)\n c.add_argument(\"b\", type=float)\n\n b = sub.add_parser(\"bearing\", help=\"bearing and distance between two points (wgs)\")\n b.add_argument(\"p\", type=_pair)\n b.add_argument(\"q\", type=_pair)\n\n d = sub.add_parser(\"dest\", help=\"walk a given number of meters from a point along a bearing (wgs)\")\n d.add_argument(\"p\", type=_pair)\n d.add_argument(\"--bearing\", type=float, required=True)\n d.add_argument(\"--dist\", type=float, required=True)\n\n r = sub.add_parser(\"range\", help=\"estimate distance from an object's real size and pixel size\")\n r.add_argument(\"--real\", type=float, required=True, help=\"real size, meters\")\n r.add_argument(\"--pixels\", type=float, required=True, help=\"pixel size in the frame\")\n r.add_argument(\"--image-width\", type=float, required=True, help=\"pixel count of the side that hfov refers to\")\n r.add_argument(\"--hfov\", help=\"field of view of that side (degrees); if the zoom is unknown give a range, e.g. 12:70, to output a distance range\")\n r.add_argument(\"--equiv-focal\", type=float, help=\"35mm equivalent focal length; if given, the field of view is computed automatically\")\n r.add_argument(\"--side\", choices=[\"long\", \"short\"], default=\"short\", help=\"whether image-width is the long side or the short side\")\n\n f = sub.add_parser(\"fov\", help=\"equivalent focal length → field of view\")\n f.add_argument(\"focal\", type=float)\n\n ln = sub.add_parser(\"line\", help=\"alignment line: near is in front of far → the extension line the camera position is on; outputs sample points along it\")\n ln.add_argument(\"--near\", type=_pair, required=True, help=\"lat,lon of the nearer object in the frame\")\n ln.add_argument(\"--far\", type=_pair, required=True, help=\"lat,lon of the farther object in the frame, aligned with near\")\n ln.add_argument(\"--range\", default=\"0:2000\", help=\"distance range extending outward from near (meters), e.g. 50:1500\")\n ln.add_argument(\"--step\", type=float, default=100)\n ln.add_argument(\"--out\", help=\"write {name:[lat,lon]}, can be passed to tiles.py mark\")\n\n it = sub.add_parser(\"intersect\", help=\"intersect two sight lines (with error radius)\")\n it.add_argument(\"--align1\", help=\"near_lat,near_lon:far_lat,far_lon (first aligned pair)\")\n it.add_argument(\"--align2\", help=\"second aligned pair, same format\")\n it.add_argument(\"--sight1\", help=\"lat,lon@bearing: compass bearing from the camera position to this landmark\")\n it.add_argument(\"--sight2\", help=\"same as above\")\n it.add_argument(\"--sigma\", type=float, default=1.0, help=\"bearing error of each sight line (degrees), default 1\")\n\n fr = sub.add_parser(\"frame\", help=\"frame check before excluding: from a candidate camera position, should a landmark appear in the frame, and would it be blocked\",\n formatter_class=argparse.RawDescriptionHelpFormatter, description=(\n \"Purpose: before you use \\\"X is not in the frame\\\" to exclude a candidate camera position or a whole direction, first compute whether X falls in the frame at all.\\n\"\n \"Heading can be given two ways: --heading directly; or --anchor with an identified landmark and its pixel x in the frame,\\n\"\n \"and the script back-computes the heading for each field of view (the smaller the field of view, the narrower the frame, the easier nearby landmarks fall out of it).\\n\"\n \"If the field of view is unknown (video screenshot, crop, zoom), give a range; the script sweeps it and reports over which field-of-view range each landmark enters the frame.\\n\"\n \"Only a landmark that is \\\"fully in the frame over the whole field-of-view range and not blocked\\\" can have its absence used to exclude.\"))\n pos = fr.add_mutually_exclusive_group(required=True)\n pos.add_argument(\"--at\", action=\"append\", type=_pair, help=\"candidate camera position lat,lon, repeatable\")\n pos.add_argument(\"--points\", help=\"candidate camera positions JSON {name:[lat,lon]}\")\n pos.add_argument(\"--ring\", help=\"lat,lon:min_radius:max_radius:radius_step:bearing_step — place candidate camera positions in rings around a landmark\")\n fr.add_argument(\"--heading\", type=float, help=\"camera heading (compass angle)\")\n fr.add_argument(\"--anchor\", help=\"name:lat,lon:px=pixel_x — an identified landmark and its horizontal pixel position in the frame\")\n fr.add_argument(\"--width\", type=float, required=True, help=\"frame width in pixels (same side as hfov)\")\n fr.add_argument(\"--hfov\", default=\"65\", help=\"horizontal field of view, single value or range 12:70 (default 65)\")\n fr.add_argument(\"--pt\", action=\"append\", default=[], type=_landmark,\n help=\"landmark to check name:lat,lon[:h=height_m][:w=width_m], repeatable; h, w are used for angular height and occlusion\")\n fr.add_argument(\"--cam-h\", type=float, default=0.0, help=\"camera height, same datum as landmark h (both height above ground or both elevation above sea level)\")\n fr.add_argument(\"--min-px\", type=float, default=12, help=\"minimum pixel height of a landmark in the frame to count as \\\"should be visible\\\"\")\n fr.add_argument(\"--out\", help=\"write the full result as JSON\")\n\n bg = sub.add_parser(\"bearings\", help=\"bearing, angular width and distance of each GeoJSON feature from the camera position; with --target, sorted by bearing difference (bearing first, then identify the structure)\")\n bg.add_argument(\"--at\", type=_pair, required=True, help=\"camera position lat,lon\")\n bg.add_argument(\"--geojson\", required=True, help=\"output of osm.py geom (building footprints, lines, points)\")\n bg.add_argument(\"--target\", type=float, help=\"true bearing of the target (the bearing of that tower in the photo, computed by sun.py compass)\")\n bg.add_argument(\"--tol\", type=float, default=8, help=\"mark as candidates those within this many degrees of bearing difference\")\n bg.add_argument(\"--max-dist\", type=float, default=3000, help=\"only consider features within this distance (meters)\")\n bg.add_argument(\"--out\", help=\"write {name: [lat, lon]}, can be passed to tiles.py mark\")\n\n sp = sub.add_parser(\"spacing\", help=\"camera position from a row of evenly spaced structures (bridge piers, utility poles): pixel columns → rays → intersect with the line → neighbor chainage differences should be constant\",\n formatter_class=argparse.RawDescriptionHelpFormatter, description=SPACING_DOC)\n sp.add_argument(\"--cols\", required=True,\n help=\"pixel columns of the structures, comma-separated; when blocked into segments, group with ';' (remember the quotes), e.g. '39,133,219,296,369;745,788,829'. \"\n \"Can also be a JSON file (@cols.json); the output of imgprep.py piers (piers field) works directly. \"\n \"First check the piers --sheet to remove peaks that aren't structures and split into segments by foreground occlusion; these two are the top reasons for getting no solution\")\n sp.add_argument(\"--line\", required=True, help=\"the line the structures are on, GeoJSON (railway/power line fetched by osm.py)\")\n sp.add_argument(\"--line-name\", help=\"pick a line by substring of properties.name; if not given, use the longest line in the file\")\n sp.add_argument(\"--line-index\", type=int, default=0, help=\"which one to pick when several lines share the name (one track per direction), default 0\")\n sp.add_argument(\"--span\", type=float, required=True, help=\"standard span m (high-speed rail simply supported box girder 32, utility poles per local standard); if unsure, run once with each of several values\")\n sp.add_argument(\"--center\", type=_pair, required=True, help=\"center of the candidate area lat,lon (also the origin of the local plane coordinates)\")\n sp.add_argument(\"--radius\", type=float, default=1500, help=\"half side length of the candidate area m, default 1500 (searches a ±radius square)\")\n sp.add_argument(\"--grid\", type=float, default=50, help=\"camera-position grid spacing m, default 50\")\n sp.add_argument(\"--headings\", type=_spacing_band, default=(0.0, 360.0), help=\"heading search range lo:hi (degrees), default 0:360; narrow it if you know the rough heading, much faster\")\n sp.add_argument(\"--heading-step\", type=float, default=0.25, help=\"heading step (degrees), default 0.25\")\n sp.add_argument(\"--focals\", required=True, help=\"focal lengths (pixels), list 1200,1281,1350,1430,1500 or range 1200:1500[:step]\")\n sp.add_argument(\"--focal-step\", type=float, default=50, help=\"step when --focals is a range, default 50\")\n sp.add_argument(\"--cx\", type=float, default=640, help=\"horizontal center pixel of the frame, default 640 (photo 1280 wide); for a cropped photo compute it from the crop\")\n sp.add_argument(\"--min-dist\", type=float, default=100, help=\"minimum distance for an intersection to count m, default 100 (filters out false intersections at the camera's feet)\")\n sp.add_argument(\"--pier-max\", type=float, default=0.08, help=\"solutions with a spacing score above this are dropped, default 0.08\")\n sp.add_argument(\"--mono-penalty\", type=float, default=1.0, help=\"penalty for non-monotonic chainage (rays hitting both sides of the line), default 1.0\")\n sp.add_argument(\"--top\", type=int, default=25, help=\"number of top results to output, default 25\")\n sp.add_argument(\"--progress\", type=int, default=10, help=\"report progress to stderr every this many grid rows, 0=off\")\n sp.add_argument(\"--out\", help=\"write the result as JSON (structure above)\")\n sp.add_argument(\"--ridge\", help=\"ridgeline pixel points JSON (output of terrain.py ridge); if given, score jointly with the skyline, downloads elevation tiles\")\n sp.add_argument(\"--hrow\", type=float, help=\"pixel row of the horizon in the photo; not needed if the --ridge file has hrow\")\n sp.add_argument(\"--flat\", type=_spacing_band, help=\"column range in the frame that should be a flat horizon x0:x1 (e.g. 0:280); defaults to flat from the --ridge file\")\n sp.add_argument(\"--flat-step\", type=float, default=20, help=\"sampling step for flat-horizon columns px, default 20\")\n sp.add_argument(\"--flat-margin\", type=float, default=1.0, help=\"how many degrees the flat horizon may rise above the corrected horizon, default 1.0\")\n sp.add_argument(\"--flat-weight\", type=float, default=1.5, help=\"weight of the flat-horizon penalty, default 1.5\")\n sp.add_argument(\"--ridge-weight\", \"--pier-weight\", dest=\"ridge_weight\", type=float, default=2.0,\n help=\"weight of the spacing score in joint scoring, default 2.0 (tot = skyline rms + flat-weight×flat-horizon penalty + this weight×spacing score)\")\n sp.add_argument(\"--cc-max\", type=float, default=0.8, help=\"maximum absolute pitch correction (degrees), default 0.8\")\n sp.add_argument(\"--eye\", type=float, default=1.6, help=\"camera height above ground m, default 1.6\")\n sp.add_argument(\"--dem-zoom\", type=int, default=13, help=\"elevation tile zoom level, default 13\")\n sp.add_argument(\"--dem-range\", type=float, default=18000, help=\"DEM mosaic radius m, default 18000 (must cover the candidate area + the visible mountains)\")\n sp.add_argument(\"--sky-range\", type=float, default=16000, help=\"how far out the skyline looks m, default 16000\")\n sp.add_argument(\"--sky-near\", type=float, default=100, help=\"how near the skyline sampling starts m, default 100\")\n sp.add_argument(\"--sky-samples\", type=int, default=400, help=\"number of distances sampled per bearing, default 400 (dense near, sparse far)\")\n sp.add_argument(\"--az-step\", type=float, default=0.1, help=\"step of the horizon bearing table (degrees), default 0.1\")\n sp.add_argument(\"--cache\", type=Path, default=Path(\".geo-cache/dem\"), help=\"elevation tile cache directory, default .geo-cache/dem (same as terrain.py)\")\n sp.add_argument(\"--proxy\", default=os.environ.get(\"GEO_PROXY\"), help=\"proxy URL or 'direct'\")\n\n args = ap.parse_args(_neg_coords(sys.argv[1:]))\n if args.cmd == \"convert\":\n x, y = convert(args.a, args.b, args.src, args.dst)\n print(f\"{x:.7f},{y:.7f}\")\n elif args.cmd == \"bearing\":\n print(f\"bearing={bearing(args.p, args.q):.1f}deg distance={distance(args.p, args.q):.1f}m\")\n elif args.cmd == \"dest\":\n la, lo = dest(args.p, args.bearing, args.dist)\n print(f\"{la:.7f},{lo:.7f}\")\n elif args.cmd == \"range\":\n if args.hfov is None:\n if args.equiv_focal is None:\n ap.error(\"--hfov or --equiv-focal is required\")\n lf, sf = fov_from_equiv_focal(args.equiv_focal)\n lo = hi = lf if args.side == \"long\" else sf\n else:\n lo, hi = _band_f(args.hfov)\n if lo == hi:\n dist = range_from_size(args.real, args.pixels, args.image_width, lo)\n print(f\"fov={lo:.1f}deg focal={focal_px(args.image_width, lo):.0f}px range={dist:.0f}m\")\n else:\n steps = 6\n for k in range(steps + 1):\n hf = lo * (hi / lo) ** (k / steps) # geometric sampling over zoom\n dist = range_from_size(args.real, args.pixels, args.image_width, hf)\n print(f\"fov={hf:5.1f}deg focal={focal_px(args.image_width, hf):6.0f}px range={dist:7.0f}m\")\n print(f\"Field of view {lo:g}–{hi:g}° → distance {range_from_size(args.real, args.pixels, args.image_width, hi):.0f}–\"\n f\"{range_from_size(args.real, args.pixels, args.image_width, lo):.0f} m. Until the zoom is pinned down, look for candidates over the whole range\")\n elif args.cmd == \"frame\":\n if args.heading is None and not args.anchor:\n ap.error(\"--heading or --anchor is required\")\n if args.at:\n cams = {f\"P{i + 1}\": list(p) for i, p in enumerate(args.at)}\n elif args.points:\n with open(args.points, encoding=\"utf-8\") as fh:\n cams = {k: v[:2] for k, v in json.load(fh).items()}\n else:\n cams = _ring_positions(args.ring)\n anchor = _landmark(args.anchor) if args.anchor else None\n lo, hi = _band_f(args.hfov)\n hfovs = [lo] if lo == hi else [lo * (hi / lo) ** (k / 40) for k in range(41)]\n pts = args.pt + ([anchor] if anchor else [])\n W = args.width\n full: dict = {}\n summary_rows = []\n for cname, cll in cams.items():\n cam = tuple(cll)\n per_hfov = []\n status: dict[str, list[float]] = {p[\"name\"]: [] for p in args.pt}\n min_px: dict[str, float] = {}\n for hf in hfovs:\n if anchor:\n f = focal_px(W, hf)\n off = math.degrees(math.atan((anchor[\"px\"] - W / 2) / f))\n hd = (bearing(cam, anchor[\"ll\"]) - off) % 360\n else:\n hd = args.heading\n rows = frame_predict(cam, hd, hf, W, pts, args.cam_h)\n per_hfov.append({\"hfov\": round(hf, 2), \"heading\": round(hd, 2), \"points\": rows})\n for r in rows:\n if r[\"name\"] in status and r[\"in_frame\"] == \"full\":\n status[r[\"name\"]].append(hf)\n if \"height_px\" in r:\n min_px[r[\"name\"]] = min(min_px.get(r[\"name\"], 1e9), r[\"height_px\"])\n occ = occluders(cam, pts, args.cam_h)\n full[cname] = {\"ll\": cll, \"frames\": per_hfov, \"occlusion\": occ}\n verdict = {}\n for p in args.pt:\n inside = status[p[\"name\"]]\n hidden = any(o.startswith(p[\"name\"] + \" may be \") for o in occ)\n whole_range = False\n if len(hfovs) == 1:\n v = \"fully in frame\" if inside else \"not fully in frame\"\n elif len(inside) == len(hfovs):\n v = \"in frame over the whole field-of-view range\"\n whole_range = True\n elif inside:\n v = f\"in frame at field of view {min(inside):.0f}–{max(inside):.0f}°, out of frame otherwise\"\n else:\n v = \"not in frame over the whole field-of-view range\"\n if hidden:\n v += \"; may be blocked\"\n small = False\n if inside and \"h\" not in p:\n v += \"; no height h given, not checked whether it is big enough or shows above the foreground\"\n small = True\n elif inside and min_px.get(p[\"name\"], 0) \u003c args.min_px:\n v += f\"; only {min_px.get(p['name'], 0):.0f} px tall at minimum, may not be visible\"\n small = True\n can_exclude = (whole_range or (len(hfovs) == 1 and bool(inside))) \\\n and not hidden and not small\n verdict[p[\"name\"]] = {\"verdict\": v, \"absence_excludes\": can_exclude}\n full[cname][\"verdict\"] = verdict\n summary_rows.append((cname, cll, verdict))\n for cname, cll, verdict in summary_rows[:60]:\n print(f\"{cname} {cll[0]:.5f},{cll[1]:.5f}\")\n frames = full[cname][\"frames\"]\n for n, v in verdict.items():\n print(f\" {n}: {v['verdict']} → absence {'can' if v['absence_excludes'] else 'cannot'} exclude\")\n if len(hfovs) == 1:\n f0 = frames[0]\n print(f\" heading {f0['heading']}°: \" + \"; \".join(\n f\"{r['name']} x={r['px']} {r['in_frame']}\" + (f\" height {r['height_px']}px\" if \"height_px\" in r else \"\")\n for r in f0[\"points\"]))\n else:\n a, b = frames[0], frames[-1]\n print(f\" as field of view goes {a['hfov']}°→{b['hfov']}°, heading is between {a['heading']}°→{b['heading']}°\")\n for o in full[cname][\"occlusion\"]:\n print(f\" occlusion: {o}\")\n if len(summary_rows) > 60:\n print(f\" …{len(summary_rows)} camera positions in total, see --out for the full result\")\n excl = [c for c, _, v in summary_rows if any(x[\"absence_excludes\"] for x in v.values())]\n print(f\"\\nOf {len(summary_rows)} candidate camera positions, {len(excl)} can be excluded by \\\"a landmark is not in the frame\\\"\"\n \" (provided the landmark really is absent from the photo, and no unlisted nearby building blocks that direction)\")\n if args.out:\n with open(args.out, \"w\", encoding=\"utf-8\") as fh:\n json.dump(full, fh, ensure_ascii=False, indent=1)\n excl_pts = {c: cams[c] for c in excl}\n keep_pts = {c: cams[c] for c in cams if c not in excl}\n base = args.out.rsplit(\".\", 1)[0]\n with open(base + \"_keep.json\", \"w\", encoding=\"utf-8\") as fh:\n json.dump(keep_pts, fh, indent=1)\n with open(base + \"_excluded.json\", \"w\", encoding=\"utf-8\") as fh:\n json.dump(excl_pts, fh, indent=1)\n print(f\"-> {args.out} (kept camera positions {base}_keep.json, excludable ones {base}_excluded.json, both can go straight to tiles.py mark)\")\n elif args.cmd == \"bearings\":\n with open(args.geojson, encoding=\"utf-8\") as fh:\n gj = json.load(fh)\n rows = []\n for f in gj.get(\"features\", []):\n g = f.get(\"geometry\") or {}\n t = g.get(\"type\")\n if t == \"Polygon\":\n coords = [(c[1], c[0]) for c in g[\"coordinates\"][0]]\n elif t == \"MultiPolygon\":\n coords = [(c[1], c[0]) for poly in g[\"coordinates\"] for c in poly[0]]\n elif t == \"LineString\":\n coords = [(c[1], c[0]) for c in g[\"coordinates\"]]\n elif t == \"Point\":\n coords = [(g[\"coordinates\"][1], g[\"coordinates\"][0])]\n else:\n continue\n dists = [distance(args.at, c) for c in coords]\n if min(dists) > args.max_dist:\n continue\n brgs = [bearing(args.at, c) for c in coords]\n mx = sum(math.cos(math.radians(b)) for b in brgs)\n my = sum(math.sin(math.radians(b)) for b in brgs)\n center = math.degrees(math.atan2(my, mx)) % 360\n rel = [((b - center + 180) % 360) - 180 for b in brgs]\n props = f.get(\"properties\") or {}\n tags = props.get(\"tags\") or props\n name = tags.get(\"name\") or f.get(\"id\") or \"\"\n rows.append({\"name\": str(name), \"bearing\": round(center, 1), \"span_deg\": round(max(rel) - min(rel), 1),\n \"dist_m\": round(min(dists)), \"height\": tags.get(\"height\") or tags.get(\"building:height\") or \"\",\n \"levels\": tags.get(\"building:levels\") or \"\", \"kind\": tags.get(\"building\") or tags.get(\"man_made\") or tags.get(\"power\") or \"\",\n \"center\": [round(sum(c[0] for c in coords) / len(coords), 6), round(sum(c[1] for c in coords) / len(coords), 6)]})\n if args.target is not None:\n for r in rows:\n r[\"d_bearing\"] = round(((r[\"bearing\"] - args.target + 180) % 360) - 180, 1)\n rows.sort(key=lambda r: abs(r[\"d_bearing\"]))\n else:\n rows.sort(key=lambda r: r[\"bearing\"])\n print(f\"{len(rows)} features (within {args.max_dist:.0f} m)\" + (f\", target bearing {args.target}°, those within ±{args.tol}° marked *\" if args.target is not None else \"\"))\n print(f\"{'':1} {'brg':>6} {'span':>5} {'dist':>6} {'h/lvl':>7} name/kind\")\n for r in rows[:60]:\n star = \"*\" if args.target is not None and abs(r[\"d_bearing\"]) \u003c= args.tol else \" \"\n print(f\"{star} {r['bearing']:>6.1f} {r['span_deg']:>5.1f} {r['dist_m']:>6} {str(r['height'] or r['levels']):>7} {r['name'][:28]} {r['kind']}\")\n if args.target is not None:\n hits = [r for r in rows if abs(r[\"d_bearing\"]) \u003c= args.tol]\n print(f\"{len(hits)} match the bearing\" + (\"; when nothing matches, first suspect the heading and zoom, then suspect OSM hasn't mapped it\" if not hits else \". Nearby buildings with a large angular width block those behind; only tall ones can show above\"))\n if args.out:\n with open(args.out, \"w\", encoding=\"utf-8\") as fh:\n json.dump({(f\"{'*' if args.target is not None and abs(r['d_bearing']) \u003c= args.tol else ''}{r['bearing']:.0f}° {r['name'][:12]}\"): r[\"center\"] for r in rows}, fh, ensure_ascii=False, indent=1)\n print(f\"-> {args.out}\")\n elif args.cmd == \"spacing\":\n out = _spacing_run(args)\n cands = out[\"candidates\"]\n if not cands:\n print(f\"No solution: {out['n_fit']} combinations passed --pier-max {args.pier_max}. Check in this order: \"\n \"① --cols contains peaks that aren't structures (foreground highlights, railings, trees); clean them out against the imgprep.py piers --sheet; \"\n \"② structures broken by the foreground weren't split into segments with ';' (the difference across a break is not one span, so the whole group is ruined); \"\n \"③ loosen --pier-max; ④ check that --span, --cx, --cols were measured on the same image; \"\n \"⑤ only then suspect that --center/--radius encloses the wrong area\")\n else:\n print(f\"{out['n_fit']} passed --pier-max; top {len(cands)} by tot:\")\n print(f\"{'tot':>6} {'rms':>6} {'pier':>6} {'span':>6} {'camera':^17} {'f':>6} {'H':>7} {'cc':>6} {'d_1st':>6} {'d_last':>6}\")\n for r in cands:\n rms = \" -\" if r[\"rms\"] is None else format(r[\"rms\"], \"6.3f\")\n cc = \" -\" if r[\"cc\"] is None else format(r[\"cc\"], \"6.2f\")\n print(f\"{r['tot']:6.3f} {rms} {r['pier']:6.3f} {r['span_m']:6.1f} \"\n f\"{r['cam'][0]:.5f},{r['cam'][1]:.5f} {r['f']:6.0f} {r['H']:7.2f} {cc} \"\n f\"{r['d_first']:6.0f} {r['d_last']:6.0f}\")\n b = cands[0]\n spread = max(distance(b[\"cam\"], r[\"cam\"]) for r in cands)\n print(f\"Best solution: {b['cam'][0]:.5f},{b['cam'][1]:.5f} heading {b['H']:.2f}° focal length {b['f']:.0f}px \"\n f\"mean span {b['span_m']:.1f} m (nominal {args.span:g})\")\n print(f\"Top {len(cands)} scattered within {spread:.0f} m\" + (\n \". With spacing alone the solution is a band along the sight line; giving --ridge for joint scoring with the skyline narrows it (geometry.md 7.7)\"\n if not args.ridge else \". Check the pier positions of the best solution on satellite imagery; don't rely on the score alone\"))\n if args.out:\n with open(args.out, \"w\", encoding=\"utf-8\") as fh:\n json.dump(out, fh, ensure_ascii=False, indent=1)\n print(f\"-> {args.out}\")\n elif args.cmd == \"fov\":\n lf, sf = fov_from_equiv_focal(args.focal)\n print(f\"4:3 long_side={lf:.1f}deg short_side={sf:.1f}deg\")\n elif args.cmd == \"line\":\n origin, brg = ray_from_alignment(args.near, args.far)\n a, b = (float(x) for x in args.range.split(\":\"))\n pts, k, dcur = {}, 0, a\n while dcur \u003c= b + 1e-6:\n pts[f\"L{int(dcur)}m\"] = [round(v, 7) for v in dest(origin, brg, dcur)]\n dcur += args.step\n k += 1\n print(f\"Camera position is on the ray beyond near at bearing {brg:.1f}°; {k} sample points\")\n if args.out:\n with open(args.out, \"w\", encoding=\"utf-8\") as fh:\n json.dump(pts, fh, indent=1)\n print(f\"-> {args.out}\")\n else:\n for name, ll in pts.items():\n print(f\" {name}: {ll[0]},{ll[1]}\")\n elif args.cmd == \"intersect\":\n def ray(align, sight):\n if align:\n n, fr = align.split(\":\")\n return ray_from_alignment(_pair(n), _pair(fr))\n if sight:\n ll, brg = sight.split(\"@\")\n return ray_from_sighting(_pair(ll), float(brg))\n ap.error(\"each line needs --alignN or --sightN\")\n r1, r2 = ray(args.align1, args.sight1), ray(args.align2, args.sight2)\n res = intersect_with_error(r1, r2, args.sigma)\n if res is None:\n print(\"The two lines are parallel, or the intersection is behind the objects (check whether near/far are swapped)\")\n else:\n print(json.dumps(res, ensure_ascii=False))\n if res[\"crossing_angle_deg\"] \u003c 15:\n print(\"Note: the lines cross at less than 15°, so the intersection is very sensitive to bearing error; find another line with a larger crossing angle\")\n\n\nif __name__ == \"__main__\":\n # Chinese-locale Windows outputs GBK by default: it crashes on m², ñ, and the Chinese the agent reads is garbled\n sys.stdout.reconfigure(encoding=\"utf-8\")\n sys.stderr.reconfigure(encoding=\"utf-8\", errors=\"backslashreplace\")\n main()\n","content_type":"text/x-python; charset=utf-8","language":"python","size":65142,"content_sha256":"7ef8c2e51646ac39ab80851ef0faa18eb08dcd149fed8d9509bd5e912ca5fe35"},{"filename":"scripts/gsv.py","content":"#!/usr/bin/env python3\n# /// script\n# requires-python = \">=3.10\"\n# dependencies = [\"pillow\"]\n# ///\n\"\"\"Google Street View (the main source outside China): find points, see dates and nearby points, render by heading, build comparison sheets. Usage mirrors baidu_pano.py.\n\nUses the endpoints the Google Maps web page itself calls; no key needed.\nThere is almost no Google Street View coverage in mainland China; for photos from mainland China use baidu_pano.py.\n\nExamples:\n gsv.py near 35.6595,139.7005 --radius 50 # nearest panorama point: id, coordinates, capture date, historical captures, address, nearby points\n gsv.py render XlVh96-Z9lAI5tKrU2O4Yg --heading 90 --out v.jpg\n gsv.py sheet --at 35.6595,139.7005 --headings 0,60,120,180,240,300 --out around.jpg # look around from a single point\n gsv.py sheet --ids ID1,ID2 --toward 35.6600,139.7010 --out s.jpg # every point faces the same target\n gsv.py sheet --points pts.json --heading 90 --date 2018 --out s2018.jpg # only one capture (the year of the street-view watermark in the photo)\n gsv.py targets --points churches.json --per 2 --out churches.index.json # every target seen from the panoramas around it, for match.py rank --items\n\"\"\"\nfrom __future__ import annotations\n\nimport argparse\nfrom _net import curl_args, PROXY_HELP\nimport json\nimport math\nimport os\nimport re\nimport subprocess\nimport sys\nimport threading\nfrom concurrent.futures import ThreadPoolExecutor\nfrom pathlib import Path\n\nfrom PIL import Image, ImageDraw\n\nsys.path.insert(0, str(Path(__file__).parent))\nimport geo # noqa: E402\nfrom baidu_pano import _font # noqa: E402\n\nUA = \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/152.0.0.0 Safari/537.36\"\n# official Street View car coverage only (user-uploaded panorama photos have ids like CIHM0og…, and the perspective endpoint can't render them)\n# The Maps JS API's own panorama lookup (the GET endpoint GeoPhotoService.SingleImageSearch was turned down in 2026; same response layout)\nMETA = \"https://maps.googleapis.com/$rpc/google.internal.maps.mapsjs.v1.MapsJsInternalService/SingleImageSearch\"\nTHUMB = (\"https://streetviewpixels-pa.googleapis.com/v1/thumbnail?panoid={id}&cb_client=maps_sv.tactile\"\n \"&w={w}&h={h}&yaw={yaw:.1f}&pitch={pitch:.1f}&thumbfov={fov:.0f}\")\n# Coverage layer the Maps web client draws: every official panorama inside one z17 tile (~300 m), one request per tile\nCOVERAGE = \"https://www.google.com/maps/photometa/ac/v1?pb=!1m1!1smaps_sv.tactile!6m3!1i{x}!2i{y}!3i17!8b1\"\n# Bound lock storage while serializing workers that share a coverage tile.\n_COVERAGE_LOCKS = [threading.Lock() for _ in range(64)]\n\n\ndef _curl(url: str, proxy: str | None, out: Path | None = None, data: str | None = None) -> bytes:\n cmd = [\"curl\", \"-q\", \"-s\", \"-m\", \"40\", \"-A\", UA]\n cmd += curl_args(proxy)\n if out:\n cmd += [\"-o\", str(out)]\n if data is not None:\n cmd += [\"-X\", \"POST\", \"-H\", \"content-type: application/json+protobuf\",\n \"-H\", \"x-user-agent: grpc-web-javascript/0.1\", \"--data\", data]\n r = subprocess.run(cmd + [url], capture_output=True)\n return r.stdout\n\n\nclass ServiceError(RuntimeError):\n \"\"\"Google answered with an error instead of a lookup result (endpoint changed, blocked, rate-limited).\"\"\"\n\n\ndef _meta_query(lat: float, lon: float, radius: float) -> str:\n # official Street View car coverage only ([2, true, 2]); the last list asks for address, date, neighbors and history\n return json.dumps([[\"apiv3\", None, None, None, \"US\", None, None, None, None, None, [[0]]],\n [[None, None, lat, lon], radius],\n [None, [\"en\", \"US\"], None, None, None, None, None, None, [2], None, [[[2, True, 2]]]],\n [[1, 2, 3, 4, 8, 6]]])\n\n\ndef near(lat: float, lon: float, radius: float, proxy: str | None) -> dict | None:\n \"\"\"Nearest panorama within radius; None when there is none. Raises ServiceError when the lookup itself fails.\"\"\"\n t = _curl(META, proxy, data=_meta_query(lat, lon, radius)).decode(\"utf-8\", \"replace\")\n try:\n d = json.loads(t)\n except json.JSONDecodeError:\n raise ServiceError(f\"unreadable reply: {' '.join(t.split())[:160] or 'empty (network?)'}\") from None\n try:\n code = d[0][0]\n body = d[1] if len(d) > 1 else None\n except (IndexError, TypeError, KeyError):\n raise ServiceError(f\"unexpected reply: {t[:160]}\") from None\n if body is None:\n if code == 5 and \"no images\" in t.lower(): # [[5, \"generic\", \"Search returned no images.\"]]\n return None\n raise ServiceError(f\"error reply: {t[:160]}\")\n out: dict = {}\n try:\n out[\"id\"] = body[1][1]\n loc = body[5][0][1]\n out[\"wgs\"] = [loc[0][2], loc[0][3]]\n out[\"pano_heading\"] = round(loc[2][0], 1) if loc[2] else None\n except (IndexError, TypeError):\n return None\n try:\n out[\"address\"] = \" / \".join(x[0] for x in body[3][2])\n except (IndexError, TypeError):\n out[\"address\"] = \"\"\n # this panorama's capture date is in body[6][7]; body[5][0][8] holds historical captures [neighbor index, [year, month(, day)]], don't take it as this panorama's date\n try:\n out[\"date\"] = _ym(body[6][7])\n except (IndexError, TypeError):\n out[\"date\"] = None\n nbrs = []\n try:\n for e in body[5][0][3][0]:\n nbrs.append({\"id\": e[0][1], \"wgs\": [e[2][0][2], e[2][0][3]]})\n except (IndexError, TypeError):\n pass\n hist = []\n try:\n for idx, ym, *_ in body[5][0][8] or []:\n # user-uploaded panoramas get mixed into historical captures (perspective renders come out gray): ids start with CIHM0og/CIAB/CAoS, length varies from 22 to 28\n pid = nbrs[idx][\"id\"] if idx \u003c len(nbrs) else \"\"\n if len(pid) == 22 and not pid.startswith((\"CIHM\", \"CIAB\", \"CAoS\")):\n hist.append({**nbrs[idx], \"date\": _ym(ym)})\n except (IndexError, TypeError, ValueError):\n pass\n out[\"history\"] = sorted(hist, key=lambda h: h[\"date\"], reverse=True)\n out[\"dates_seen\"] = sorted({d for d in [out[\"date\"], *(h[\"date\"] for h in hist)] if d})\n out[\"neighbors\"] = nbrs[:40]\n return out\n\n\ndef _ym(v: list) -> str:\n return f\"{v[0]}-{int(v[1]):02d}\"\n\n\ndef pick_date(res: dict, date: str) -> dict | None:\n \"\"\"Pick one capture ('2018' or '2018-07') from a near() result: the first match among this panorama and its historical captures; None if there is none.\"\"\"\n for p in [{\"id\": res[\"id\"], \"wgs\": res[\"wgs\"], \"date\": res[\"date\"]}, *res.get(\"history\", [])]:\n if p[\"date\"] and p[\"date\"].startswith(date):\n return p\n return None\n\n\ndef _coverage_rows(t: str) -> list:\n \"\"\"Rows of one coverage reply. An empty tile (sea, no coverage) is []; anything else that isn't a coverage reply\n (network failure, 429 page, changed layout) raises ServiceError so it isn't mistaken for missing coverage.\"\"\"\n if not t.startswith(\")]}'\"):\n raise ServiceError(f\"unreadable coverage reply: {' '.join(t.split())[:160] or 'empty (network?)'}\")\n try:\n d = json.loads(t[t.index(\"\\n\") + 1:])\n except ValueError:\n raise ServiceError(f\"unreadable coverage reply: {t[:160]}\") from None\n # with panoramas: [null, [null, [rows…]], …]; an empty tile: [null, null, [cells…]]\n if not isinstance(d, list) or len(d) \u003c 2 or not (d[1] is None or isinstance(d[1], list)):\n raise ServiceError(f\"unexpected coverage reply: {t[:160]}\")\n if d[1] is None:\n return []\n rows = d[1][1] if len(d[1]) > 1 else None\n if not (rows is None or isinstance(rows, list)):\n raise ServiceError(f\"unexpected coverage reply: {t[:160]}\")\n return rows or []\n\n\ndef tile_panos(x: int, y: int, proxy: str | None, cache: Path) -> list[tuple[str, float, float]]:\n \"\"\"All official panoramas in z17 tile (x, y) as (id, lat, lon). An empty tile (sea, no coverage) is [].\n Only replies that parse as coverage are cached; a failed lookup raises ServiceError and is retried next time.\"\"\"\n cache.mkdir(parents=True, exist_ok=True)\n x %= 2 ** 17 # tiles wrap at the antimeridian\n p = cache / f\"cov17_{x}_{y}.txt\"\n with _COVERAGE_LOCKS[hash(p.resolve()) % len(_COVERAGE_LOCKS)]:\n try:\n rows = _coverage_rows(p.read_text(encoding=\"utf-8\", errors=\"replace\"))\n except (FileNotFoundError, ServiceError):\n p.unlink(missing_ok=True) # missing, empty, or invalid cache: fetch again\n t = _curl(COVERAGE.format(x=x, y=y), proxy).decode(\"utf-8\", \"replace\")\n rows = _coverage_rows(t)\n # Atomic replacement also protects readers in other processes.\n tmp = p.with_name(f\"{p.name}.{os.getpid()}.{threading.get_ident()}.tmp\")\n tmp.write_text(t, encoding=\"utf-8\")\n os.replace(tmp, p)\n out = []\n for r in rows:\n try:\n pid, ll = r[0][0][1], r[0][2][0]\n # same filter as near(): user-uploaded panoramas can't be rendered as perspective views\n if len(pid) == 22 and not pid.startswith((\"CIHM\", \"CIAB\", \"CAoS\")):\n out.append((pid, float(ll[2]), float(ll[3])))\n except (IndexError, TypeError, ValueError):\n continue\n return out\n\n\ndef tiles_around(lat: float, lon: float, max_d: float) -> list[tuple[int, int]]:\n \"\"\"z17 tiles covering every point within max_d metres of (lat, lon); tiles get narrower towards the poles.\"\"\"\n dlat = math.degrees(max_d / geo._R)\n dlon = math.degrees(max_d / (geo._R * max(math.cos(math.radians(lat)), 1e-6)))\n top = min(lat + dlat, 85.05)\n bottom = max(lat - dlat, -85.05)\n x0, y0 = (int(v // 256) for v in geo.ll2px(17, top, lon - min(dlon, 180)))\n x1, y1 = (int(v // 256) for v in geo.ll2px(17, bottom, lon + min(dlon, 180)))\n n = 2 ** 17\n xs = sorted({x % n for x in range(x0, x1 + 1)})\n return [(x, y) for x in xs for y in range(max(y0, 0), min(y1, n - 1) + 1)]\n\n\ndef panos_around(lat: float, lon: float, min_d: float, max_d: float, per: int, spacing: float,\n proxy: str | None, cache: Path) -> list[tuple[str, float, float, float]]:\n \"\"\"Up to `per` panoramas between min_d and max_d metres from (lat, lon), nearest first and at least `spacing` m apart.\n Returns (id, lat, lon, distance). Raises ServiceError when a coverage lookup fails.\"\"\"\n cands = []\n for tx, ty in tiles_around(lat, lon, max_d):\n for pid, la, lo in tile_panos(tx, ty, proxy, cache):\n d = geo.distance((lat, lon), (la, lo))\n if min_d \u003c= d \u003c= max_d:\n cands.append((d, pid, la, lo))\n picked: list[tuple[str, float, float, float]] = []\n for d, pid, la, lo in sorted(cands):\n if all(geo.distance((la, lo), (p[1], p[2])) >= spacing for p in picked):\n picked.append((pid, la, lo, d))\n if len(picked) >= per:\n break\n return picked\n\n\ndef render(pid: str, heading: float, pitch: float, fov: float, w: int, h: int, proxy: str | None,\n cache: Path) -> Image.Image:\n \"\"\"heading: compass bearing; positive pitch = looking up; fov: horizontal field of view.\"\"\"\n cache.mkdir(parents=True, exist_ok=True)\n p = cache / f\"{pid}_{heading:.0f}_{pitch:.0f}_{fov:.0f}_{w}x{h}.jpg\"\n if not (p.exists() and p.stat().st_size > 2000):\n _curl(THUMB.format(id=pid, w=w, h=h, yaw=heading % 360, pitch=-pitch, fov=fov), proxy, p)\n try:\n return Image.open(p).convert(\"RGB\")\n except Exception: # noqa: BLE001\n return Image.new(\"RGB\", (w, h), \"gray\")\n\n\ndef sheet(items: list[dict], out: Path, proxy: str | None, cache: Path, cols: int = 3, tw: int = 480, th: int = 360) -> None:\n with ThreadPoolExecutor(8) as ex:\n ims = list(ex.map(lambda it: render(it[\"id\"], it[\"heading\"], it.get(\"pitch\", 0), it.get(\"fov\", 90),\n 640, 480, proxy, cache), items))\n rows = (len(items) + cols - 1) // cols\n S = Image.new(\"RGB\", (cols * tw, max(1, rows) * th), \"black\")\n d = ImageDraw.Draw(S)\n f = _font(16)\n for i, (it, im) in enumerate(zip(items, ims)):\n x, y = (i % cols) * tw, (i // cols) * th\n S.paste(im.resize((tw, th)), (x, y))\n d.rectangle([x, y, x + tw, y + 22], fill=\"black\")\n d.text((x + 4, y + 2), it.get(\"label\") or f\"{i}: …{it['id'][-8:]} h{it['heading']:.0f}\", fill=\"yellow\", font=f)\n S.save(out, quality=88)\n\n\n\ndef _neg_coords(argv: list[str]) -> list[str]:\n \"\"\"argparse treats negative coordinates like -1.45,-48.5 as option names; prefixing a space makes them plain values (float ignores the space). Needed for every puzzle in the southern or western hemisphere.\"\"\"\n return [\" \" + a if re.match(r\"^-\\d[\\d.]*(,-?[\\d.]+)+$\", a) else a for a in argv]\n\n\ndef main() -> None:\n ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)\n ap.add_argument(\"--proxy\", default=os.environ.get(\"GEO_PROXY\"), help=PROXY_HELP)\n ap.add_argument(\"--cache\", type=Path, default=Path(\".geo-cache/gsv\"))\n sub = ap.add_subparsers(dest=\"cmd\", required=True)\n\n def common(sp):\n sp.add_argument(\"--proxy\", default=argparse.SUPPRESS, help=\"can go before or after the subcommand\")\n sp.add_argument(\"--cache\", type=Path, default=argparse.SUPPRESS)\n\n n = sub.add_parser(\"near\")\n common(n)\n n.add_argument(\"latlon\")\n n.add_argument(\"--radius\", type=float, default=50)\n\n r = sub.add_parser(\"render\")\n common(r)\n r.add_argument(\"id\")\n r.add_argument(\"--heading\", type=float, required=True)\n r.add_argument(\"--pitch\", type=float, default=0, help=\"positive = looking up\")\n r.add_argument(\"--fov\", type=float, default=90, help=\"horizontal field of view\")\n r.add_argument(\"--width\", type=int, default=1024)\n r.add_argument(\"--height\", type=int, default=768)\n r.add_argument(\"--out\", type=Path, required=True)\n\n s = sub.add_parser(\"sheet\")\n common(s)\n g = s.add_mutually_exclusive_group(required=True)\n g.add_argument(\"--ids\", help=\"comma-separated panoids\")\n g.add_argument(\"--at\", help=\"lat,lon: take the nearest panorama point\")\n g.add_argument(\"--points\", type=Path, help=\"JSON {name:[lat,lon]}: take the nearest panorama point for each\")\n h = s.add_mutually_exclusive_group(required=True)\n h.add_argument(\"--heading\", type=float)\n h.add_argument(\"--headings\", help=\"comma-separated, e.g. 0,60,120,180,240,300\")\n h.add_argument(\"--toward\", help=\"lat,lon: every point faces this target\")\n s.add_argument(\"--offset\", type=float, default=0)\n s.add_argument(\"--pitch\", type=float, default=0)\n s.add_argument(\"--fov\", type=float, default=90)\n s.add_argument(\"--radius\", type=float, default=50)\n s.add_argument(\"--date\", help=\"only this capture (2018 or 2018-07), picked from this panorama and its historical captures; use it for the street-view watermark year in the photo\")\n s.add_argument(\"--limit\", type=int, default=12)\n s.add_argument(\"--out\", type=Path, required=True)\n\n t = sub.add_parser(\"targets\", help=\"for every target point, the panoramas around it rendered facing it -> .index.json for match.py\")\n common(t)\n t.add_argument(\"--points\", type=Path, required=True, help=\"JSON {name:[lat,lon]}, e.g. osm.py find output (every church in a region)\")\n t.add_argument(\"--per\", type=int, default=2, help=\"panoramas per target\")\n t.add_argument(\"--min-dist\", type=float, default=12, help=\"skip panoramas closer than this (m): too close shows only a wall\")\n t.add_argument(\"--max-dist\", type=float, default=70, help=\"m\")\n t.add_argument(\"--spacing\", type=float, default=15, help=\"minimum distance between the panoramas picked for one target (m)\")\n t.add_argument(\"--pitch\", type=float, default=8)\n t.add_argument(\"--fov\", type=float, default=90)\n t.add_argument(\"--sheet\", type=Path, help=\"also draw the first --limit items as a contact sheet\")\n t.add_argument(\"--limit\", type=int, default=12)\n t.add_argument(\"--out\", type=Path, required=True, help=\"writes the .index.json for match.py rank --items\")\n\n args = ap.parse_args(_neg_coords(sys.argv[1:]))\n if args.cmd == \"near\":\n lat, lon = map(float, args.latlon.split(\",\"))\n try:\n res = near(lat, lon, args.radius, args.proxy)\n except ServiceError as e:\n sys.exit(f\"Google Street View lookup failed ({e}); run doctor.py --network to check service access\")\n print(json.dumps(res, ensure_ascii=False, indent=1) if res else f\"no Google Street View panorama within {args.radius:.0f} m (increase --radius or check coverage)\")\n elif args.cmd == \"render\":\n render(args.id, args.heading, args.pitch, args.fov, args.width, args.height, args.proxy, args.cache).save(args.out)\n print(args.out)\n elif args.cmd == \"targets\":\n pts = json.loads(args.points.read_text(encoding=\"utf-8\"))\n\n def around(kv):\n name, (la, lo) = kv\n try:\n return name, (la, lo), panos_around(la, lo, args.min_dist, args.max_dist, args.per, args.spacing,\n args.proxy, args.cache), None\n except ServiceError as e:\n return name, (la, lo), [], e\n\n items, empty, failed = [], [], []\n with ThreadPoolExecutor(8) as ex:\n for name, tgt, picked, err in ex.map(around, pts.items()):\n if err:\n failed.append(name)\n print(f\"{name}: coverage lookup failed ({err})\", file=sys.stderr)\n continue\n if not picked:\n empty.append(name)\n for pid, la, lo, d in picked:\n hd = geo.bearing((la, lo), tgt)\n items.append({\"id\": pid, \"heading\": round(hd, 1), \"pitch\": args.pitch, \"fov\": args.fov,\n \"label\": f\"{len(items)}: {str(name)[:18]} {d:.0f}m h{hd:.0f}\", \"point\": name,\n \"wgs\": [la, lo], \"target\": list(tgt), \"dist\": round(d), \"date\": \"\"})\n out = args.out if args.out.name.endswith(\".index.json\") else args.out.with_suffix(\".index.json\")\n out.write_text(json.dumps(items, ensure_ascii=False, indent=1), encoding=\"utf-8\")\n print(f\"{len(pts)} targets -> {len(items)} views ({len(empty)} targets without a panorama \"\n f\"within {args.min_dist:.0f}–{args.max_dist:.0f} m, {len(failed)} lookups failed) -> {out}\")\n if len(items) > 400:\n print(\"match.py loads all candidate images into memory and caps candidates at 400 by default: \"\n \"split the target points into smaller batches or narrow the area before ranking\", file=sys.stderr)\n if args.sheet and items:\n sheet(items[:args.limit], args.sheet, args.proxy, args.cache)\n print(args.sheet)\n if failed:\n # failed tiles aren't cached, so running the same command again retries only those\n sys.exit(f\"Google Street View coverage lookup failed for {len(failed)} of {len(pts)} targets; \"\n \"rerun later or run doctor.py --network to check service access\")\n else:\n panos: dict[str, dict] = {}\n if args.ids:\n panos = {pid: {\"ll\": None, \"name\": \"\", \"date\": \"\"} for pid in args.ids.split(\",\")}\n else:\n pts = {\"at\": list(map(float, args.at.split(\",\")))} if args.at else json.loads(args.points.read_text(encoding=\"utf-8\"))\n for name, (la, lo) in pts.items():\n try:\n res = near(la, lo, args.radius, args.proxy)\n except ServiceError as e:\n sys.exit(f\"Google Street View lookup failed ({e}); run doctor.py --network to check service access\")\n if res and args.date:\n p = pick_date(res, args.date)\n if not p:\n print(f\"{name}: no {args.date} capture (available: {', '.join(res['dates_seen'])})\", file=sys.stderr)\n continue\n panos.setdefault(p[\"id\"], {\"ll\": p[\"wgs\"], \"name\": name, \"date\": p[\"date\"]})\n elif res:\n panos.setdefault(res[\"id\"], {\"ll\": res[\"wgs\"], \"name\": name, \"date\": res.get(\"date\") or \"\"})\n else:\n print(f\"{name}: no panorama nearby\", file=sys.stderr)\n target = tuple(map(float, args.toward.split(\",\"))) if args.toward else None\n items = []\n for pid, meta in panos.items():\n ll = meta[\"ll\"]\n if args.headings:\n heads = [float(x) for x in args.headings.split(\",\")]\n elif target:\n if ll is None:\n try:\n info = near(*target, 5000, args.proxy)\n except ServiceError:\n info = None\n # with only an id there are no coordinates, so estimate from near the target; --points is recommended\n ll = info[\"wgs\"] if info else list(target)\n heads = [geo.bearing(tuple(ll), target) + args.offset]\n else:\n heads = [args.heading]\n for hd in heads:\n where = f\"{ll[0]:.5f},{ll[1]:.5f}\" if ll else f\"…{pid[-8:]}\"\n label = \" \".join(x for x in (meta[\"name\"][:18], where, meta[\"date\"], f\"h{hd % 360:.0f}\") if x)\n items.append({\"id\": pid, \"heading\": hd % 360, \"pitch\": args.pitch, \"fov\": args.fov,\n \"label\": f\"{len(items)}: {label}\", \"point\": meta[\"name\"], \"wgs\": ll, \"date\": meta[\"date\"]})\n pages = [items[k:k + args.limit] for k in range(0, len(items), args.limit)] or [[]]\n for pi, page in enumerate(pages):\n out = args.out if pi == 0 else args.out.with_name(f\"{args.out.stem}_{pi + 1}{args.out.suffix}\")\n sheet(page, out, args.proxy, args.cache)\n print(out)\n args.out.with_suffix(\".index.json\").write_text(json.dumps(items, indent=1), encoding=\"utf-8\")\n\n\nif __name__ == \"__main__\":\n # Chinese-locale Windows writes GBK by default: m², ñ make it crash, and the Chinese the agent reads comes out garbled\n sys.stdout.reconfigure(encoding=\"utf-8\")\n sys.stderr.reconfigure(encoding=\"utf-8\", errors=\"backslashreplace\")\n main()\n","content_type":"text/x-python; charset=utf-8","language":"python","size":22403,"content_sha256":"8c06f88c0c75d16ae3ba2f9280c09acb13a7750e56555b34005537f5dcc24913"},{"filename":"scripts/imgprep.py","content":"#!/usr/bin/env python3\n# /// script\n# requires-python = \">=3.10\"\n# dependencies = [\"numpy\", \"pillow\", \"pillow-heif\"]\n# ///\n\"\"\"Image processing before looking at and searching with a photo.\n\n zoom crop a region, upscale and sharpen: read small text, count tracks, check plate background color\n edges one upscaled image for each of the four edges and four corners — ropes in corners, a boat bow at the bottom, small road signs are the easiest to miss\n variants variants for reverse image search: tight crop / horizontal flip / enhanced grayscale / color-cast removal / upscale; optional perspective rectification\n grid cut into N×M tiles, for \"search only a part\" in reverse image search\n piers take a brightness profile along a few rows below the bridge deck and find the pixel columns of the bridge piers (for back-solving the camera position)\n\nCoordinates are always pixels of the whole photo as a viewer shows it (EXIF orientation applied), x0,y0,x1,y1 (top-left, bottom-right);\nthe same frame as ocr.py boxes, terrain.py ridge points and pose.py points. Check the photo size first: `exif.py` size, or its upright_size when it reports an EXIF orientation.\n\nExamples:\n imgprep.py zoom photo.jpg --box 820,40,1080,300 --scale 4 --out sign.png\n imgprep.py edges photo.jpg --out-dir edges/\n imgprep.py variants photo.jpg --box 300,120,900,760 --prefix left --out-dir v/ # → v/left_crop.jpg left_flip.jpg …\n imgprep.py variants photo.jpg --persp 312,140,880,95,905,770,290,720 --out-dir v/ # rectify four corners (top-left top-right bottom-right bottom-left)\n imgprep.py grid photo.jpg --rows 2 --cols 3 --out-dir tiles/\n imgprep.py piers photo.jpg --rows 926:940 --out cols.json --sheet piers.jpg\n\npiers output JSON:\n {\"image\": path, \"size\": [W, H], \"rows\": [r0, r1], \"cols\": [x0, x1],\n \"polarity\": \"bright\" | \"dark\", # whether piers are brighter or darker than their surroundings\n \"params\": {\"min_gap\": 20, \"min_prominence\": 12, \"baseline\": 61},\n \"count\": number of peaks,\n \"piers\": [{\"col\": 38, \"prominence\": 100.5, \"dev\": 92.4, \"level\": 183.2}, ...]}\n col is the pixel column (integer, original-image coordinates), prominence is the peak's topographic prominence (the main basis for judging real vs. false),\n dev is the height after baseline removal, level is the column's raw mean brightness over the rows range. piers are sorted by col ascending.\n\"\"\"\nfrom __future__ import annotations\n\nimport argparse\nfrom _image import open_photo\nimport json\nimport sys\nfrom pathlib import Path\n\nfrom PIL import Image, ImageFilter, ImageOps, ImageStat\n\n\ndef _box(s: str) -> tuple[int, int, int, int]:\n x0, y0, x1, y1 = (int(float(v)) for v in s.split(\",\"))\n return min(x0, x1), min(y0, y1), max(x0, x1), max(y0, y1)\n\n\ndef zoom(im: Image.Image, box, scale: float) -> Image.Image:\n c = im.crop(box)\n c = c.resize((max(1, int(c.width * scale)), max(1, int(c.height * scale))), Image.LANCZOS)\n return c.filter(ImageFilter.UnsharpMask(radius=2, percent=120, threshold=2))\n\n\ndef gray_world(im: Image.Image) -> Image.Image:\n \"\"\"Gray-world white balance: use when an old photo has a yellow or purple cast.\"\"\"\n r, g, b = ImageStat.Stat(im.convert(\"RGB\")).mean\n avg = (r + g + b) / 3\n chans = [ch.point(lambda v, k=avg / max(m, 1): max(0, min(255, int(v * k))))\n for ch, m in zip(im.convert(\"RGB\").split(), (r, g, b))]\n return ImageOps.autocontrast(Image.merge(\"RGB\", chans), cutoff=1)\n\n\ndef perspective(im: Image.Image, quad: list[float]) -> Image.Image:\n \"\"\"Four corners (top-left top-right bottom-right bottom-left) → rectangle. Output width and height take the larger of each pair of opposite side lengths.\"\"\"\n (x0, y0), (x1, y1), (x2, y2), (x3, y3) = (quad[i:i + 2] for i in range(0, 8, 2))\n w = int(max(((x1 - x0) ** 2 + (y1 - y0) ** 2) ** .5, ((x2 - x3) ** 2 + (y2 - y3) ** 2) ** .5))\n h = int(max(((x3 - x0) ** 2 + (y3 - y0) ** 2) ** .5, ((x2 - x1) ** 2 + (y2 - y1) ** 2) ** .5))\n # PIL's QUAD transform: the output rectangle's four corners come, in order, from the source's top-left, bottom-left, bottom-right, top-right\n return im.transform((w, h), Image.QUAD, (x0, y0, x3, y3, x2, y2, x1, y1), Image.BICUBIC)\n\n\ndef _range(s: str) -> tuple[int, int]:\n \"\"\"Parse \"a:b\" (a inclusive, b exclusive).\"\"\"\n a, b = s.split(\":\")\n a, b = int(float(a)), int(float(b))\n return (a, b) if a \u003c= b else (b, a)\n\n\ndef _baseline(p, w: int):\n \"\"\"Sliding median baseline: flattens slowly varying brightness such as the bridge body, sky and ground, leaving only thin vertical strips.\"\"\"\n import numpy as np\n h = w // 2\n pad = np.pad(p, (h, h), mode=\"edge\")\n return np.array([float(np.median(pad[i:i + w])) for i in range(len(p))])\n\n\ndef _prominence(d, i: int) -> float:\n \"\"\"Topographic prominence of local maximum i: peak height minus the higher of the two valley floors it descends to on the left and right.\"\"\"\n h = d[i]\n lo = h\n j = i\n while j > 0 and d[j - 1] \u003c= h:\n j -= 1\n lo = min(lo, d[j])\n left = lo\n lo = h\n j = i\n while j \u003c len(d) - 1 and d[j + 1] \u003c= h:\n j += 1\n lo = min(lo, d[j])\n return float(h - max(left, lo))\n\n\ndef _peaks(d, min_gap: int, min_prominence: float) -> list[tuple[int, float]]:\n \"\"\"Find peaks in d: prominence must first pass the threshold, then, from highest to lowest prominence, greedily keep those ≥ min_gap apart.\"\"\"\n cand = [i for i in range(1, len(d) - 1) if d[i] >= d[i - 1] and d[i] > d[i + 1]]\n res = [(i, _prominence(d, i)) for i in cand]\n res = [(i, pr) for i, pr in res if pr >= min_prominence]\n res.sort(key=lambda t: -t[1])\n keep: list[tuple[int, float]] = []\n for i, pr in res:\n if all(abs(i - j) >= min_gap for j, _ in keep):\n keep.append((i, pr))\n keep.sort()\n return keep\n\n\ndef piers(im: Image.Image, rows: tuple[int, int], cols: tuple[int, int] | None,\n min_gap: int, min_prominence: float, baseline_win: int, polarity: str) -> dict:\n \"\"\"Take each column's mean brightness over the rows below the bridge deck → subtract the sliding median baseline → find peaks = bridge pier pixel columns.\n\n In the shadow of a shaded bridge body, piers are usually brighter than their surroundings (polarity=bright); against the light or on overcast days it can be the reverse (dark);\n auto computes both and takes the one with the larger total prominence.\n \"\"\"\n import numpy as np\n W, H = im.size\n r0, r1 = max(0, rows[0]), min(H, rows[1])\n if r1 - r0 \u003c 1:\n raise SystemExit(f\"--rows {rows[0]}:{rows[1]} selects no rows in a {W}x{H} image\")\n x0, x1 = (0, W) if cols is None else (max(0, cols[0]), min(W, cols[1]))\n a = np.asarray(im.convert(\"L\"), dtype=float)\n p = a[r0:r1, x0:x1].mean(axis=0)\n base = _baseline(p, max(3, baseline_win | 1))\n dev = p - base\n opts = {\"bright\": _peaks(dev, min_gap, min_prominence),\n \"dark\": _peaks(-dev, min_gap, min_prominence)} if polarity == \"auto\" \\\n else {polarity: _peaks(dev if polarity == \"bright\" else -dev, min_gap, min_prominence)}\n pol = max(opts, key=lambda k: sum(pr for _, pr in opts[k]))\n sign = 1.0 if pol == \"bright\" else -1.0\n found = [{\"col\": int(i + x0), \"prominence\": round(pr, 2),\n \"dev\": round(float(dev[i]) * sign, 2), \"level\": round(float(p[i]), 2)}\n for i, pr in opts[pol]]\n return {\"size\": [W, H], \"rows\": [r0, r1], \"cols\": [x0, x1], \"polarity\": pol,\n \"params\": {\"min_gap\": min_gap, \"min_prominence\": min_prominence, \"baseline\": baseline_win},\n \"count\": len(found), \"piers\": found}\n\n\ndef piers_sheet(im: Image.Image, r: dict, out: Path) -> None:\n \"\"\"Draw the sampled row band and each pier column (numbered) on the original image, so a person can check at a glance for extra or missing ones.\"\"\"\n from PIL import ImageDraw\n sheet = im.copy()\n d = ImageDraw.Draw(sheet)\n r0, r1 = r[\"rows\"]\n x0, x1 = r[\"cols\"]\n d.rectangle([x0, r0, x1 - 1, r1 - 1], outline=(0, 200, 255), width=1)\n for n, pier in enumerate(r[\"piers\"]):\n x = pier[\"col\"]\n d.line([(x, max(0, r0 - 60)), (x, min(sheet.height, r1 + 60))], fill=(255, 0, 0), width=1)\n d.text((x + 2, max(0, r0 - 72)), str(n), fill=(255, 255, 0))\n sheet.save(out, quality=92)\n\n\ndef main() -> None:\n ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)\n sub = ap.add_subparsers(dest=\"cmd\", required=True)\n\n z = sub.add_parser(\"zoom\")\n z.add_argument(\"image\", type=Path)\n z.add_argument(\"--box\", type=_box, required=True)\n z.add_argument(\"--scale\", type=float, default=3)\n z.add_argument(\"--out\", type=Path, required=True)\n\n e = sub.add_parser(\"edges\")\n e.add_argument(\"image\", type=Path)\n e.add_argument(\"--frac\", type=float, default=0.18, help=\"edge strip as a fraction of the whole image, default 0.18\")\n e.add_argument(\"--out-dir\", type=Path, required=True)\n\n v = sub.add_parser(\"variants\")\n v.add_argument(\"image\", type=Path)\n v.add_argument(\"--box\", type=_box, help=\"search only this region (removes sky and background)\")\n v.add_argument(\"--persp\", help=\"8 numbers: the four corners top-left top-right bottom-right bottom-left, rectified to a front view\")\n v.add_argument(\"--prefix\", help=\"output file name prefix; default is original name + crop box, so variants of different boxes don't overwrite each other\")\n v.add_argument(\"--out-dir\", type=Path, required=True)\n\n g = sub.add_parser(\"grid\")\n g.add_argument(\"image\", type=Path)\n g.add_argument(\"--rows\", type=int, default=2)\n g.add_argument(\"--cols\", type=int, default=2)\n g.add_argument(\"--overlap\", type=float, default=0.15)\n g.add_argument(\"--out-dir\", type=Path, required=True)\n\n pr = sub.add_parser(\"piers\", help=\"find bridge pier pixel columns from the brightness profile of a few rows below the bridge deck\",\n formatter_class=argparse.RawDescriptionHelpFormatter, description=\"\"\"\\\nTake the brightness profile along the --rows rows, subtract the sliding median baseline, and find peaks with prominence ≥ --min-prominence; the pixel columns of the peaks are the pier columns.\nFor sample rows, pick the rows below the bridge deck where the piers are exposed and the background (sky, distant view, water) is continuous; when the deck is tilted in the frame, widen the row band to cover both ends.\n\nOutput JSON (--out):\n {\"image\": path, \"size\": [W, H], \"rows\": [r0, r1], \"cols\": [x0, x1], # cols is the column search range, not the structure columns\n \"polarity\": \"bright\" | \"dark\", # whether piers are brighter or darker than their surroundings\n \"params\": {\"min_gap\": 20, \"min_prominence\": 12, \"baseline\": 61},\n \"count\": number of peaks,\n \"piers\": [{\"col\": 38, \"prominence\": 100.5, \"dev\": 92.4, \"level\": 183.2}, ...]}\n col is the pixel column (integer, original-image coordinates), prominence is the peak's prominence (the main basis for judging real vs. false),\n dev is the height after baseline removal, level is the column's raw mean brightness over the rows range. piers are sorted by col ascending.\n\nThis JSON can go straight to geo.py spacing --cols @cols.json, but **check --sheet first**:\ndrop non-pier peaks (bright foreground spots, railings, trees); structure segments broken by foreground occlusion must be separated with ';' when you hand-write the --cols string.\"\"\")\n pr.add_argument(\"image\", type=Path)\n pr.add_argument(\"--rows\", type=_range, required=True,\n help='sample rows, \"r0:r1\" (r0 inclusive, r1 exclusive). Take the rows below the bridge deck where the piers are exposed; '\n 'look with zoom first; when the deck is tilted, widen the row band to cover both ends')\n pr.add_argument(\"--cols\", type=_range, help='search only within this column range, \"x0:x1\"; default full width')\n pr.add_argument(\"--min-gap\", type=int, default=20, help=\"minimum spacing in pixels between two pier columns, default 20\")\n pr.add_argument(\"--min-prominence\", type=float, default=12,\n help=\"peak prominence threshold (0-255 gray level), default 12; lower gives extra false peaks, higher misses dense piers at the far end\")\n pr.add_argument(\"--baseline\", type=int, default=61,\n help=\"window width of the sliding median baseline, default 61 px; larger than the pier spacing, smaller than the scale of brightness changes along the bridge body\")\n pr.add_argument(\"--polarity\", choices=[\"auto\", \"bright\", \"dark\"], default=\"auto\",\n help=\"whether piers are brighter or darker than their surroundings, default auto (tries both, takes the one with the larger total prominence)\")\n pr.add_argument(\"--out\", type=Path, required=True, help=\"output JSON, structure at the top of --help\")\n pr.add_argument(\"--sheet\", type=Path, help=\"draw the row band and each column (numbered) on the original image, for a person to check\")\n\n args = ap.parse_args()\n im = open_photo(args.image, \"image\").convert(\"RGB\")\n W, H = im.size\n\n if args.cmd == \"zoom\":\n zoom(im, args.box, args.scale).save(args.out)\n print(args.out)\n return\n\n if args.cmd == \"piers\":\n r = piers(im, args.rows, args.cols, args.min_gap, args.min_prominence, args.baseline, args.polarity)\n r = {\"image\": str(args.image), **r}\n args.out.parent.mkdir(parents=True, exist_ok=True)\n args.out.write_text(json.dumps(r, ensure_ascii=False), encoding=\"utf-8\")\n print(f\"rows {r['rows'][0]}:{r['rows'][1]} cols {r['cols'][0]}:{r['cols'][1]} \"\n f\"polarity {r['polarity']} found {r['count']} columns\")\n for n, pier in enumerate(r[\"piers\"]):\n print(f\" {n:>2} col {pier['col']:>5} prominence {pier['prominence']:>6.1f} brightness {pier['level']:>6.1f}\")\n print(\"list: \" + \",\".join(str(p[\"col\"]) for p in r[\"piers\"]))\n print(\"Before feeding geo.py spacing, check --sheet: drop non-pier peaks (bright foreground spots, railings, trees); \"\n \"separate structure segments broken by foreground occlusion with ';', e.g. --cols '38,133,218;745,788,829'\")\n print(args.out)\n if args.sheet:\n args.sheet.parent.mkdir(parents=True, exist_ok=True)\n piers_sheet(im, r, args.sheet)\n print(args.sheet)\n return\n\n args.out_dir.mkdir(parents=True, exist_ok=True)\n stem = args.image.stem\n if args.cmd == \"variants\":\n stem = args.prefix or (f\"{stem}_{'-'.join(map(str, args.box))}\" if args.box else stem)\n outs = []\n if args.cmd == \"edges\":\n fw, fh = int(W * args.frac), int(H * args.frac)\n regions = {\"top\": (0, 0, W, fh), \"bottom\": (0, H - fh, W, H), \"left\": (0, 0, fw, H), \"right\": (W - fw, 0, W, H),\n \"corner_tl\": (0, 0, 2 * fw, 2 * fh), \"corner_tr\": (W - 2 * fw, 0, W, 2 * fh),\n \"corner_bl\": (0, H - 2 * fh, 2 * fw, H), \"corner_br\": (W - 2 * fw, H - 2 * fh, W, H)}\n for name, box in regions.items():\n scale = max(1.0, 1600 / max(box[2] - box[0], box[3] - box[1]))\n p = args.out_dir / f\"{stem}_{name}.jpg\"\n zoom(im, box, scale).save(p, quality=92)\n outs.append(p)\n elif args.cmd == \"variants\":\n base = im\n if args.persp:\n base = perspective(im, [float(x) for x in args.persp.split(\",\")])\n p = args.out_dir / f\"{stem}_persp.jpg\"\n base.save(p, quality=95)\n outs.append(p)\n if args.box:\n base = base.crop(args.box)\n p = args.out_dir / f\"{stem}_crop.jpg\"\n base.save(p, quality=95)\n outs.append(p)\n for name, img in ((\"flip\", ImageOps.mirror(base)),\n (\"gray\", ImageOps.autocontrast(ImageOps.grayscale(base), cutoff=1)),\n (\"wb\", gray_world(base))):\n p = args.out_dir / f\"{stem}_{name}.jpg\"\n img.save(p, quality=95)\n outs.append(p)\n if max(base.size) \u003c 800:\n k = 1024 / max(base.size)\n p = args.out_dir / f\"{stem}_up.jpg\"\n base.resize((int(base.width * k), int(base.height * k)), Image.LANCZOS).save(p, quality=95)\n outs.append(p)\n elif args.cmd == \"grid\":\n tw, th = W / args.cols, H / args.rows\n ox, oy = tw * args.overlap, th * args.overlap\n for r in range(args.rows):\n for c in range(args.cols):\n box = (int(max(0, c * tw - ox)), int(max(0, r * th - oy)), int(min(W, (c + 1) * tw + ox)), int(min(H, (r + 1) * th + oy)))\n p = args.out_dir / f\"{stem}_r{r}c{c}.jpg\"\n im.crop(box).save(p, quality=92)\n outs.append(p)\n for p in outs:\n print(p)\n\n\nif __name__ == \"__main__\":\n # Chinese-locale Windows outputs GBK by default: it crashes on m² or ñ, and any Chinese the agent reads comes out garbled\n sys.stdout.reconfigure(encoding=\"utf-8\")\n sys.stderr.reconfigure(encoding=\"utf-8\", errors=\"backslashreplace\")\n main()\n","content_type":"text/x-python; charset=utf-8","language":"python","size":17107,"content_sha256":"195e671b41ea4036ae1c2d9dddba66f7c432f82536bbc2a7512a66aee1cbe7e7"},{"filename":"scripts/intake.py","content":"#!/usr/bin/env python3\n# /// script\n# requires-python = \">=3.10\"\n# dependencies = [\"pillow\", \"pillow-heif\"]\n# ///\n\"\"\"Steps 0–3 of geolocation in one command: metadata, zoomed four edges and four corners, search variants, OCR, reverse image search, run in parallel, one report.\n\nOutputs (all in --out-dir):\n exif.json edges/ variants/ ocr.json ocr.png rev/\u003cname>_\u003cengine>.png + .json intake.md intake.json\n work/\u003cname>.png only when the photo needs it: HEIC/WebP/... decoded, or an EXIF orientation applied. A lossless upright\n copy with the original's EXIF (orientation reset); every step below then uses it, and intake.json \"input\"\n records it. Metadata is always read from the original. Pixel coordinates are in the upright frame either way.\nExit status: 0 ok or partial (the report says which steps failed), 1 every enabled visual step (edges, variants, OCR)\nfailed, 2 the photo itself can't be read (metadata is still tried and reported). Steps turned off with --no-* don't count.\nintake.md is for humans (and the LLM): metadata, OCR text, Baidu similar images (count, source sites, numbered contact sheet rev/\u003cname>_baidu_similar.jpg),\ntiered vote count of image-search tags, suspected residential compound / housing development / hotel names, list of outputs, items skipped and failed.\nAlways open the image-search screenshots and the similar-image contact sheets; \"searched, nothing found\" and \"not searched\" are written separately in the report.\n\n intake.py photo.jpg --out-dir intake/ [--box x0,y0,x1,y1 ...] [--engines baidu,yandex] [--exclude word1,word2]\n [--no-rev] [--no-ocr] [--max-variants 4]\n\nExamples:\n intake.py photo.jpg --out-dir intake/\n intake.py photo.jpg --out-dir intake/ --box 300,120,900,760 --exclude 网络迷踪,\u003ccreator name> # in blind tests, exclude walkthrough posts (网络迷踪 = photo geolocation)\n intake.py photo.jpg --out-dir intake/ --no-rev # only metadata, edge crops, OCR (under 30 seconds)\n\"\"\"\nfrom __future__ import annotations\n\nimport argparse\nfrom _image import ImageReadError, write_working_copy\nfrom _net import PROXY_HELP\nimport json\nimport os\nimport re\nimport subprocess\nimport sys\nimport time\nfrom collections import Counter\nfrom concurrent.futures import ThreadPoolExecutor\nfrom pathlib import Path\n\nHERE = Path(__file__).parent\n# uv run writes its own path into the UV env var: call child scripts with it, so uv is found even when it isn't on PATH (e.g. just installed, terminal not reopened)\nUV = os.environ.get(\"UV\") or \"uv\"\nCITY_SUFFIX = (\"省\", \"市\", \"区\", \"县\", \"州\", \"盟\", \"旗\", \"国\", \"府\", \"道\", \"自治区\", \"特别行政区\", \"City\", \"Province\", \"County\", \"Prefecture\")\nPLACE_WORDS = (\"公园\", \"大厦\", \"小区\", \"花园\", \"广场\", \"学校\", \"中学\", \"小学\", \"大学\", \"酒店\", \"宾馆\", \"景区\", \"寺\", \"塔\", \"桥\", \"大楼\",\n \"中心\", \"村\", \"镇\", \"街\", \"路\", \"湖\", \"山\", \"站\", \"港\", \"码头\", \"厂\", \"园\", \"苑\", \"府\", \"城\", \"馆\", \"庙\", \"教堂\", \"Park\",\n \"Tower\", \"Hotel\", \"Bridge\", \"Station\", \"Square\", \"Plaza\", \"Church\", \"Temple\", \"Mall\", \"Street\", \"Road\", \"Avenue\")\nKNOWN_CITIES = (\"北京\", \"上海\", \"天津\", \"重庆\", \"广州\", \"深圳\", \"成都\", \"杭州\", \"武汉\", \"西安\", \"南京\", \"苏州\", \"郑州\", \"长沙\", \"青岛\", \"沈阳\",\n \"大连\", \"厦门\", \"福州\", \"济南\", \"合肥\", \"昆明\", \"贵阳\", \"南宁\", \"哈尔滨\", \"长春\", \"石家庄\", \"太原\", \"兰州\", \"乌鲁木齐\", \"拉萨\",\n \"呼和浩特\", \"银川\", \"西宁\", \"海口\", \"三亚\", \"宁波\", \"无锡\", \"东莞\", \"佛山\", \"珠海\", \"香港\", \"澳门\", \"台北\", \"东京\", \"大阪\", \"首尔\",\n \"曼谷\", \"新加坡\", \"吉隆坡\", \"伦敦\", \"巴黎\", \"纽约\", \"洛杉矶\", \"悉尼\", \"墨尔本\", \"温哥华\", \"多伦多\", \"莫斯科\", \"柏林\", \"罗马\", \"马德里\")\n\n\ndef _run(cmd: list[str], cwd: Path | None = None, timeout: int = 900) -> tuple[int, str, str]:\n try:\n # Child scripts and this one both use UTF-8: Chinese Windows reads and writes GBK by default, and if the two sides differ you get mojibake or crashes\n r = subprocess.run(cmd, text=True, encoding=\"utf-8\", errors=\"replace\", capture_output=True, cwd=cwd, timeout=timeout,\n env={**os.environ, \"PYTHONUTF8\": \"1\"})\n return r.returncode, r.stdout, r.stderr\n except subprocess.TimeoutExpired:\n return 124, \"\", f\"timed out after {timeout}s\"\n\n\ndef _script(name: str) -> str:\n return str(HERE / name)\n\n\nGENERIC = (\"城市街道\", \"城市\", \"街道\", \"市区\", \"街景\", \"建筑\", \"楼房\", \"高楼\", \"夜景\", \"风景\", \"天空\", \"道路\", \"马路\", \"小镇\", \"乡村\", \"都市\",\n \"city\", \"street\", \"town\", \"building\", \"skyline\", \"road\", \"urban\")\nKNOWN_CITIES_EN = (\"hong kong\", \"kowloon\", \"tokyo\", \"osaka\", \"kyoto\", \"seoul\", \"bangkok\", \"singapore\", \"kuala lumpur\", \"taipei\", \"shanghai\",\n \"beijing\", \"shenzhen\", \"guangzhou\", \"chengdu\", \"chongqing\", \"london\", \"paris\", \"new york\", \"los angeles\", \"sydney\",\n \"melbourne\", \"vancouver\", \"toronto\", \"moscow\", \"berlin\", \"rome\", \"madrid\", \"dubai\", \"istanbul\", \"mumbai\", \"delhi\",\n \"hanoi\", \"ho chi minh\", \"manila\", \"jakarta\", \"macau\", \"lisbon\", \"barcelona\", \"amsterdam\", \"prague\", \"vienna\", \"cairo\")\n\n\ndef _classify(tag: str) -> str:\n t = tag.strip(\"：: ，,。.\")\n low = t.lower()\n if low in GENERIC or t in GENERIC:\n return \"other\"\n if any(c in low for c in KNOWN_CITIES_EN):\n return \"city\"\n if any(c in t for c in KNOWN_CITIES):\n return \"city\"\n if len(t) >= 3 and any(t.endswith(s) for s in CITY_SUFFIX) and not any(g in t for g in (\"街道\", \"城市\")):\n return \"city\"\n if any(w in t for w in PLACE_WORDS):\n return \"place\"\n return \"other\"\n\n\ndef _collect_rev(rev_dir: Path) -> tuple[list[dict], dict]:\n \"\"\"Read revimg's .json files: guess/links for each image and engine, and do the tiered vote count.\"\"\"\n entries = []\n for jf in sorted(rev_dir.glob(\"*.json\")):\n try:\n d = json.loads(jf.read_text(encoding=\"utf-8\"))\n except Exception: # noqa: BLE001\n continue\n m = re.match(r\"(.+)_(baidu|yandex|bing|baiduimg|sogouimg)$\", jf.stem)\n if not m:\n continue\n simi = d.get(\"similar\") or []\n entries.append({\"variant\": m.group(1), \"engine\": m.group(2), \"guess\": d.get(\"guess\") or [], \"links\": (d.get(\"links\") or [])[:8],\n \"error\": d.get(\"error\"), \"refused\": bool(d.get(\"refused\")), \"shot\": str(jf.with_suffix(\".png\").name),\n \"json\": jf.name, \"similar_n\": len(simi), \"similar_sites\": Counter(x.get(\"site\") or \"?\" for x in simi).most_common(),\n \"similar_sheet\": Path(d[\"similar_sheet\"]).name if d.get(\"similar_sheet\") else None,\n \"similar_note\": d.get(\"similar_sheet_note\")})\n votes: dict = {\"city\": {}, \"place\": {}}\n for e in entries:\n seen_city = set()\n for g in e[\"guess\"]:\n body = re.sub(r\"^(图中可能是|Image appears to contain)[：:]?\", \"\", g).strip()\n for tag in re.split(r\"[、/，,;；|]\", body):\n tag = tag.strip()\n if len(tag) \u003c 2:\n continue\n k = _classify(tag)\n if k == \"city\":\n key = tag\n if (e[\"engine\"], key) in seen_city: # several variants from one engine naming the same city count as one vote\n continue\n seen_city.add((e[\"engine\"], key))\n votes[\"city\"].setdefault(key, set()).add(e[\"engine\"])\n elif k == \"place\":\n votes[\"place\"].setdefault(tag, set()).add(f\"{e['engine']}/{e['variant']}\")\n return entries, {k: {t: sorted(v) for t, v in d.items()} for k, d in votes.items()}\n\n\ndef main() -> None:\n ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)\n ap.add_argument(\"photo\")\n ap.add_argument(\"--out-dir\", type=Path, required=True)\n ap.add_argument(\"--box\", action=\"append\", help=\"x0,y0,x1,y1: tight-crop variant, repeatable\")\n ap.add_argument(\"--engines\", default=\"baidu,yandex\")\n ap.add_argument(\"--exclude\", help=\"words to exclude from reverse image search results (for blind tests)\")\n ap.add_argument(\"--no-rev\", action=\"store_true\")\n ap.add_argument(\"--no-ocr\", action=\"store_true\")\n ap.add_argument(\"--max-variants\", type=int, default=4, help=\"max number of variants each engine searches (besides the original)\")\n ap.add_argument(\"--proxy\", default=os.environ.get(\"GEO_PROXY\"), help=PROXY_HELP)\n args = ap.parse_args()\n\n photo = Path(args.photo).resolve()\n out = args.out_dir\n out.mkdir(parents=True, exist_ok=True)\n (out / \"edges\").mkdir(exist_ok=True)\n (out / \"variants\").mkdir(exist_ok=True)\n t_all = time.time()\n timings: dict = {}\n status: dict = {}\n\n def timed(name, fn):\n t0 = time.time()\n try:\n res = fn()\n status[name] = \"ok\" if res[0] == 0 else f\"failed: {(res[2] or res[1])[-300:].strip()}\"\n except Exception as e: # noqa: BLE001\n status[name] = f\"exception: {e}\"\n res = (1, \"\", str(e))\n timings[name] = round(time.time() - t0, 1)\n return res\n\n # A photo the later steps or the search engines can't take as stored gets one upright lossless working copy\n try:\n work = write_working_copy(photo, out / \"work\" / f\"{photo.stem}.png\")\n except ImageReadError as e:\n _unreadable_input(photo, out, e, timed)\n return\n src = Path(work[\"working\"]) if work else photo\n\n def exif():\n return _run([UV, \"run\", _script(\"exif.py\"), str(photo)])\n\n def edges():\n return _run([UV, \"run\", _script(\"imgprep.py\"), \"edges\", str(src), \"--out-dir\", str(out / \"edges\")])\n\n def variants():\n rc, so, se = _run([UV, \"run\", _script(\"imgprep.py\"), \"variants\", str(src), \"--prefix\", \"full\", \"--out-dir\", str(out / \"variants\")])\n for i, b in enumerate(args.box or [], 1):\n r2 = _run([UV, \"run\", _script(\"imgprep.py\"), \"variants\", str(src), \"--box\", b, \"--prefix\", f\"box{i}\", \"--out-dir\", str(out / \"variants\")])\n rc, so, se = max(rc, r2[0]), so + r2[1], se + r2[2]\n return rc, so, se\n\n def ocr():\n return _run([UV, \"run\", _script(\"ocr.py\"), str(src), \"--out\", str(out / \"ocr.json\"), \"--draw\", str(out / \"ocr.png\")])\n\n # Batch 1: metadata, edges, variants, OCR in parallel\n with ThreadPoolExecutor(4) as ex:\n f_exif = ex.submit(timed, \"exif\", exif)\n f_edges = ex.submit(timed, \"edges\", edges)\n f_var = ex.submit(timed, \"variants\", variants)\n f_ocr = None if args.no_ocr else ex.submit(timed, \"ocr\", ocr)\n exif_res = f_exif.result()\n f_edges.result()\n f_var.result()\n if f_ocr:\n f_ocr.result()\n try:\n exif_json = json.loads(exif_res[1].strip().splitlines()[0]) if exif_res[1].strip() else {}\n except Exception: # noqa: BLE001\n exif_json = {\"raw\": exif_res[1][:500]}\n (out / \"exif.json\").write_text(json.dumps(exif_json, ensure_ascii=False, indent=1), encoding=\"utf-8\")\n\n # Batch 2: reverse image search, two engines in parallel, each searches the original + up to max-variants variants\n rev_dir = out / \"rev\"\n entries, votes = [], {\"city\": {}, \"place\": {}}\n if not args.no_rev:\n rev_dir.mkdir(exist_ok=True)\n var_files = sorted((out / \"variants\").glob(\"*.jpg\")) + sorted((out / \"variants\").glob(\"*.png\"))\n prefer = [f for f in var_files if any(k in f.stem for k in (\"crop\", \"flip\", \"box\"))] + [f for f in var_files if not any(k in f.stem for k in (\"crop\", \"flip\", \"box\"))]\n images = [str(src)] + [str(f) for f in prefer[: args.max_variants]]\n\n def rev(engine):\n cmd = [UV, \"run\", _script(\"revimg.py\"), *images, \"--out-dir\", str(rev_dir), \"--engines\", engine]\n if args.exclude:\n cmd += [\"--exclude\", args.exclude]\n if args.proxy:\n cmd += [\"--proxy\", args.proxy]\n return _run(cmd, timeout=900)\n\n with ThreadPoolExecutor(2) as ex:\n futs = {eng: ex.submit(timed, f\"rev-{eng}\", (lambda e=eng: (lambda: rev(e)))()) for eng in [e for e in args.engines.split(\",\") if e]}\n for eng, f in futs.items():\n f.result()\n entries, votes = _collect_rev(rev_dir)\n\n ocr_items = []\n if (out / \"ocr.json\").exists():\n try:\n ocr_items = json.loads((out / \"ocr.json\").read_text(encoding=\"utf-8\")).get(\"items\", [])\n except Exception: # noqa: BLE001\n pass\n\n # ---- report\n visual = [\"edges\", \"variants\"] + ([] if args.no_ocr else [\"ocr\"])\n failed = [k for k, v in status.items() if v != \"ok\"]\n result = \"failed\" if all(status.get(k) != \"ok\" for k in visual) else \"partial\" if failed else \"ok\"\n L = [f\"# Steps 0–3 report: {photo.name}\", \"\", f\"Total time {time.time() - t_all:.0f}s; per step: \" + \", \".join(f\"{k} {v}s\" for k, v in timings.items()), \"\"]\n if result == \"failed\":\n L += [f\"**Result: failed.** Every enabled visual step failed ({', '.join(visual)}); reasons under Status. Fix that before going on.\", \"\"]\n elif result == \"partial\":\n L += [f\"**Result: partial.** Failed: {', '.join(failed)} (reasons under Status); the other steps' outputs below are usable.\", \"\"]\n L += [\"Next: record clues on board.py (`clue`), lookup clues with `apply`; list the candidates in full first (`children`), then rank.\", \"\"]\n if work:\n L += [f\"Input: {work['reason']}, so the steps below used `work/{Path(work['working']).name}` ({work['working_size'][0]}x{work['working_size'][1]}, lossless, \"\n \"the original's EXIF kept). Metadata was read from the original. Crops and OCR boxes are in this upright frame, the same frame \"\n \"every script uses when given the original, so keep passing the original photo to later commands.\", \"\"]\n L += [\"## Metadata\", \"\"]\n gps = exif_json.get(\"gps\") or exif_json.get(\"GPS\")\n if exif_json and (gps or exif_json.get(\"datetime\") or exif_json.get(\"DateTimeOriginal\")):\n L.append(\"```\\n\" + json.dumps(exif_json, ensure_ascii=False, indent=1)[:1500] + \"\\n```\")\n L.append(\"Metadata is an assumption (it can be edited, it can be stripped): check both GPS and time against the image.\")\n else:\n L.append(\"No metadata (common for forwarded images, screenshots, rephotographed images).\" + (f\" Raw output: `{json.dumps(exif_json, ensure_ascii=False)[:300]}`\" if exif_json else \"\"))\n L += [\"\", \"## OCR text (second reader; pass=up/tile was only read after zooming, treat it as an assumption)\", \"\"]\n if args.no_ocr:\n L.append(\"Not done (--no-ocr).\")\n elif ocr_items:\n L.append(\"| Conf | Pass | Box (px) | Text |\")\n L.append(\"|---|---|---|---|\")\n for t in ocr_items[:40]:\n L.append(f\"| {t['conf']:.2f} | {t['pass']} | {t['box']} | {t['text']} |\")\n L.append(\"\")\n L.append(\"Phone numbers, plates, proper names on road signs, issuing authorities: first `clues.py lookup`, then `board.py apply`.\")\n else:\n L.append(\"No text read\" + (f\" (OCR {status.get('ocr')})\" if status.get(\"ocr\") != \"ok\" else \". Maybe there really is no text, or it's too small: zoom in by hand with imgprep.py zoom and look again\"))\n L += [\"\", \"## Reverse image search\", \"\"]\n if args.no_rev:\n L.append(\"Not done (--no-rev). This is not \\\"searched, nothing found\\\".\")\n else:\n for eng in args.engines.split(\",\"):\n st = status.get(f\"rev-{eng}\", \"not run\")\n L.append(f\"- {eng}: {st}\")\n refused = [e for e in entries if e[\"error\"] or e[\"refused\"]]\n if refused:\n L.append(\"- Items the engine refused or failed (not the same as nothing found): \" + \", \".join(f\"{e['variant']}/{e['engine']}: {e['error'] or 'refused'}\" for e in refused))\n L.append(\"\")\n baidu = sorted([e for e in entries if e[\"engine\"] == \"baidu\"], key=lambda e: e[\"variant\"] != photo.stem)\n if baidu:\n L.append(\"### Similar images (Baidu; open the contact sheet first)\")\n L.append(\"\")\n L.append(\"A near-duplicate photo is the fastest route to a location: the same object or scene has often been photographed by others and posted on Dianping, Douyin, Xiaohongshu, and the source page's shop name, scenic-area name or location tag gives the place directly. \"\n \"Open the contact sheet and compare each cell with the query image in the top-left corner on **fixed features** (shapes of structural parts, soot and damage, the mountains/buildings/bridges/pylons behind); merely the same kind of object doesn't count. \"\n \"For a near-duplicate: number i is `similar[i]` in that image's JSON, `from` is the source page, `site` is the site; look through the other images on the same source page too. \"\n \"For sources that need a login, don't log in and don't bypass verification; instead do a keyword search from image features + site type. Look at the original image's sheet first; look at the variants' sheets only if the original's has no near-duplicate.\")\n L.append(\"\")\n for e in baidu:\n head = f\"**{e['variant']}**: \"\n if e[\"error\"]:\n L.append(f\"- {head}Baidu error ({e['error'][:80]}), no similar images captured; doesn't count as searched with nothing found\")\n elif not e[\"similar_n\"]:\n L.append(f\"- {head}no similar images captured (Baidu returned none, or the page layout changed); open the screenshot `rev/{e['shot']}` and check for a \\\"相似图片\\\" (similar images) section\")\n else:\n sites = \", \".join(f\"{s} {n}\" for s, n in e[\"similar_sites\"][:8])\n sheet = f\"contact sheet `rev/{e['similar_sheet']}` (first {min(24, e['similar_n'])})\" if e[\"similar_sheet\"] else \"no contact sheet\"\n note = f\"; {e['similar_note']}\" if e[\"similar_note\"] else \"\"\n L.append(f\"- {head}{e['similar_n']} images ({sites}); {sheet}{note}; source list in `similar` of `rev/{e['json']}`\")\n L.append(\"\")\n L.append(\"### Tiered vote count (several variants from one engine naming the same city count as one vote; conflicting specific places don't cancel city votes)\")\n L.append(\"\")\n if votes[\"city\"]:\n for t, engs in sorted(votes[\"city\"].items(), key=lambda kv: -len(kv[1])):\n L.append(f\"- City level **{t}**: {len(engs)} votes ({', '.join(engs)})\")\n else:\n L.append(\"- City level: none\")\n if votes[\"place\"]:\n for t, srcs in sorted(votes[\"place\"].items(), key=lambda kv: -len(kv[1])):\n L.append(f\"- Specific place **{t}**: {len(srcs)} times ({', '.join(srcs)}) → `poi.py \\\"{t}\\\" --city \u003ccity> --region \u003ccc>` to get coordinates; list every same-name place, then verify\")\n else:\n L.append(\"- Specific place level: none\")\n L.append(\"\")\n L.append(\"### Each image, each engine\")\n L.append(\"\")\n for e in entries:\n L.append(f\"**{e['variant']} / {e['engine']}** (screenshot `rev/{e['shot']}`, always open it)\")\n if e[\"similar_sheet\"]:\n L.append(f\"- Similar images: {e['similar_n']}, contact sheet `rev/{e['similar_sheet']}` (see above)\")\n for g in e[\"guess\"]:\n L.append(f\"- Tag: {g[:200]}\")\n for ln in e[\"links\"][:8]:\n L.append(f\"- [{ln.get('site', '')}] {ln.get('title', '')[:80]} — {ln.get('url', '')[:100]}\")\n L.append(\"\")\n L += [\"## Output files\", \"\", f\"- Edge crops: `edges/` ({len(list((out / 'edges').glob('*')))} images; look at the four edges and four corners one by one)\",\n f\"- Variants: `variants/` ({len(list((out / 'variants').glob('*')))} images)\", \"- OCR annotated image: `ocr.png`\" if not args.no_ocr else \"\",\n f\"- Image-search screenshots: `rev/` ({len(list(rev_dir.glob('*.png'))) if rev_dir.exists() else 0} images)\" if not args.no_rev else \"\",\n f\"- Baidu similar-image contact sheets: `rev/*_baidu_similar.jpg` ({len(list(rev_dir.glob('*_baidu_similar.jpg'))) if rev_dir.exists() else 0})\" if not args.no_rev else \"\", \"\",\n \"## Status\", \"\"]\n for k, v in status.items():\n L.append(f\"- {k}: {v}\")\n (out / \"intake.md\").write_text(\"\\n\".join(x for x in L if x is not None), encoding=\"utf-8\")\n (out / \"intake.json\").write_text(json.dumps({\"photo\": str(photo), \"input\": work, \"result\": result, \"exif\": exif_json, \"ocr\": ocr_items, \"rev\": entries,\n \"votes\": votes, \"status\": status, \"timings\": timings}, ensure_ascii=False, indent=1), encoding=\"utf-8\")\n print(\"\\n\".join(L[:6]))\n print(f\"-> {out / 'intake.md'} (read this)\")\n if result == \"failed\":\n sys.exit(f\"intake: every enabled visual step failed ({', '.join(visual)}); see {out / 'intake.md'}\")\n\n\ndef _unreadable_input(photo: Path, out: Path, err: ImageReadError, timed) -> None:\n \"\"\"The photo itself can't be decoded: still try its metadata (a RAW often carries GPS and time), write a short report, exit 2.\"\"\"\n res = timed(\"exif\", lambda: _run([UV, \"run\", _script(\"exif.py\"), str(photo)]))\n try:\n exif_json = json.loads(res[1].strip().splitlines()[0]) if res[1].strip() else {}\n except Exception: # noqa: BLE001\n exif_json = {}\n (out / \"exif.json\").write_text(json.dumps(exif_json, ensure_ascii=False, indent=1), encoding=\"utf-8\")\n L = [f\"# Steps 0–3 report: {photo.name}\", \"\", f\"**Result: input unreadable.** {err.reason}.\",\n \"No crops, variants, OCR or image search were done.\", \"\", \"## Metadata\", \"\",\n \"```\\n\" + json.dumps(exif_json, ensure_ascii=False, indent=1)[:1500] + \"\\n```\" if exif_json else \"No metadata could be read either.\"]\n (out / \"intake.md\").write_text(\"\\n\".join(L), encoding=\"utf-8\")\n (out / \"intake.json\").write_text(json.dumps({\"photo\": str(photo), \"input\": {\"error\": err.category, \"reason\": err.reason}, \"result\": \"input unreadable\",\n \"exif\": exif_json}, ensure_ascii=False, indent=1), encoding=\"utf-8\")\n print(\"\\n\".join(L[:4]))\n print(f\"-> {out / 'intake.md'}\")\n sys.exit(2)\n\n\nif __name__ == \"__main__\":\n # Chinese Windows outputs GBK by default: it crashes on m², ñ, and Chinese text the agent reads comes out garbled\n sys.stdout.reconfigure(encoding=\"utf-8\")\n sys.stderr.reconfigure(encoding=\"utf-8\", errors=\"backslashreplace\")\n main()\n","content_type":"text/x-python; charset=utf-8","language":"python","size":22870,"content_sha256":"51552258b37051056ce6492eb9268fc36d1ee6bd7321ea1fd71ceb5bbd292c03"},{"filename":"scripts/match.py","content":"#!/usr/bin/env python3\n# /// script\n# requires-python = \">=3.10\"\n# dependencies = [\"pillow\", \"pillow-heif\", \"numpy\", \"torch\", \"transformers\", \"opencv-python-headless\", \"socksio\", \"pysocks\", \"requests\"]\n# ///\n\"\"\"Similarity ranking of a photo against a batch of candidate real-scene images (street-view renders, satellite thumbnails, reference images): the machine ranks first, you look only at the top few.\n\nUses:\n- Street-view confirmation: with dozens to hundreds of candidate panorama points, sort by similarity first and open only the top 10 to compare invariant features.\n- Satellite thumbnails: compare the top-down views of candidate points with the top-down features visible in the photo (weaker than street view; coarse ranking only).\n- Reference library: compare reference images of bus liveries, streetlight styles, etc. with crops of the photo.\n\n rank score and rank the candidates, output ranked.json + contact sheet of the top N\n index save embeddings of a batch of images to disk (reused repeatedly within the same city)\n\nCandidate source, pick one of three:\n --images \u003cdir or glob> existing images, file name is the id\n --items \u003c.index.json> --render baidu|gsv index written by baidu_pano.py sheet/sample or gsv.py sheet, rendered item by item\n --panos panos.json --toward lat,lon | --headings 0,60,… output of baidu_pano.py scan, rendered by heading (can add --within, --spread)\n\nScoring:\n global descriptor DINOv2 (facebook/dinov2-small, CLS + patch mean) or CLIP (openai/clip-vit-base-patch32) cosine similarity;\n --refine sift re-ranks the top --refine-top by SIFT + RANSAC inlier count (only inliers ≥ 15 count as geometric consistency).\n Rows below --refine-top keep inliers null and refined false: never refined, which is not the same as 0 matches.\n Final order: those with inliers by inlier count, the rest by global score. Scores are only for ordering; whether it is the same place still requires you to compare ≥3 invariant features.\n\nOn first use the model downloads from HuggingFace and is cached in ~/.cache/huggingface.\n\nExamples:\n match.py rank --query photo.jpg --panos panos.json --toward \u003clat,lon> --spread 15 --refine sift --top 10 --out ranked.json --sheet ranked.jpg\n match.py rank --query photo.jpg --query-box 200,100,900,700 --items around.index.json --render baidu --spread-headings -30,0,30 --out r.json --sheet r.jpg\n match.py rank --query photo.jpg --images cands/ --method clip --out r.json\n match.py index --images city_panos/ --out city.npz\n\"\"\"\nfrom __future__ import annotations\n\nimport argparse\nfrom _device import torch_device\nfrom _image import ImageReadError, find_images, open_image, open_photo, upright\nfrom _net import PROXY_HELP\nfrom _net import model_proxy_env\nimport glob\nimport json\nimport os\nimport re\nimport sys\nimport time\nfrom concurrent.futures import ThreadPoolExecutor\nfrom pathlib import Path\n\nimport numpy as np\nfrom PIL import Image, ImageDraw\n\nsys.path.insert(0, str(Path(__file__).parent))\nimport geo # noqa: E402\n\nMODELS = {\"dino\": \"facebook/dinov2-small\", \"clip\": \"openai/clip-vit-base-patch32\"}\n\n\ndef _proxy_env(proxy: str | None) -> None:\n model_proxy_env(proxy)\n\n\nIMNET = ([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])\nCLIPN = ([0.48145466, 0.4578275, 0.40821073], [0.26862954, 0.26130258, 0.27577711])\n\n\ndef _tensor(ims: list[Image.Image], norm: tuple, torch):\n \"\"\"PIL 224×224 → (N,3,224,224) normalized tensor. Does its own preprocessing, no torchvision dependency.\"\"\"\n mean, std = (np.array(v, dtype=np.float32).reshape(1, 3, 1, 1) for v in norm)\n arr = np.stack([np.asarray(im.convert(\"RGB\"), dtype=np.float32) / 255.0 for im in ims]).transpose(0, 3, 1, 2)\n return torch.from_numpy((arr - mean) / std)\n\n\ndef _feat(out):\n \"\"\"In transformers 5, get_*_features returns BaseModelOutputWithPooling with the projected vector in pooler_output; older versions return the tensor directly.\"\"\"\n if hasattr(out, \"shape\"):\n return out\n for k in (\"pooler_output\", \"text_embeds\", \"image_embeds\"):\n v = getattr(out, k, None)\n if v is not None and hasattr(v, \"shape\"):\n return v\n return out[0]\n\n\nclass Embedder:\n \"\"\"Global descriptor. dino: DINOv2 CLS + patch mean; clip: image-tower embedding.\"\"\"\n\n def __init__(self, method: str):\n import torch\n from transformers import AutoModel, CLIPModel\n self.method = method\n self.dev = torch_device(torch)\n t0 = time.time()\n try:\n self.model = (CLIPModel if method == \"clip\" else AutoModel).from_pretrained(MODELS[method]).to(self.dev).eval()\n except Exception as e: # noqa: BLE001\n sys.exit(f\"failed to load model {MODELS[method]}: {str(e)[:300]}\\n\"\n f\"check disk space, model availability and `doctor.py --network`.\")\n self.torch = torch\n print(f\"model {MODELS[method]} ready ({self.dev}, {time.time() - t0:.1f}s)\", file=sys.stderr)\n\n def _views(self, im: Image.Image, multi: bool) -> list[Image.Image]:\n im = im.convert(\"RGB\")\n vs = [im.resize((224, 224), Image.BICUBIC)]\n if multi:\n w, h = im.size\n s = min(w, h)\n vs.append(im.crop(((w - s) // 2, (h - s) // 2, (w + s) // 2, (h + s) // 2)).resize((224, 224), Image.BICUBIC))\n # left and right halves: when the street view and the photo viewpoints are offset, the overlapping half matches better\n vs.append(im.crop((0, 0, int(w * 0.6), h)).resize((224, 224), Image.BICUBIC))\n vs.append(im.crop((int(w * 0.4), 0, w, h)).resize((224, 224), Image.BICUBIC))\n return vs\n\n def embed(self, ims: list[Image.Image], multi: bool = False, batch: int = 32) -> np.ndarray:\n \"\"\"Returns (N, V, D): normalized vectors for the V views of each image.\"\"\"\n views = [self._views(im, multi) for im in ims]\n flat = [v for vs in views for v in vs]\n out = []\n with self.torch.no_grad():\n for i in range(0, len(flat), batch):\n chunk = flat[i:i + batch]\n if self.method == \"clip\":\n x = _tensor(chunk, CLIPN, self.torch).to(self.dev)\n f = _feat(self.model.get_image_features(pixel_values=x))\n else:\n x = _tensor(chunk, IMNET, self.torch).to(self.dev)\n h = self.model(pixel_values=x).last_hidden_state\n # Give CLS and patch mean equal weight regardless of their original vector lengths.\n cls, patch = h[:, 0], h[:, 1:].mean(1)\n f = self.torch.cat([cls / cls.norm(dim=1, keepdim=True), patch / patch.norm(dim=1, keepdim=True)], dim=1)\n f = f / f.norm(dim=1, keepdim=True)\n out.append(f.float().cpu().numpy())\n arr = np.concatenate(out, 0)\n nv = len(views[0]) if views else 1\n return arr.reshape(len(ims), nv, -1)\n\n\ndef sift_inliers(a: Image.Image, b: Image.Image, max_side: int = 1024,\n skip_a: tuple | None = None, skip_b: tuple | None = None) -> tuple[int, int]:\n \"\"\"Inlier count (and match count) from SIFT + ratio test + RANSAC homography.\n skip_a / skip_b: a fractional box x0,y0,x1,y1 of that image where no keypoints are taken (an overlay a renderer bakes in, such as baidu_pano.LOGO_BOX).\"\"\"\n import cv2\n\n def prep(im, skip):\n im = im.convert(\"L\")\n s = max_side / max(im.size)\n if s \u003c 1:\n im = im.resize((int(im.width * s), int(im.height * s)), Image.BICUBIC)\n arr = np.array(im)\n if skip is None:\n return arr, None\n h, w = arr.shape\n mask = np.full((h, w), 255, np.uint8)\n mask[int(skip[1] * h):int(skip[3] * h), int(skip[0] * w):int(skip[2] * w)] = 0\n return arr, mask\n\n sift = cv2.SIFT_create(nfeatures=3000)\n ka, da = sift.detectAndCompute(*prep(a, skip_a))\n kb, db = sift.detectAndCompute(*prep(b, skip_b))\n if da is None or db is None or len(ka) \u003c 8 or len(kb) \u003c 8:\n return 0, 0\n m = cv2.BFMatcher().knnMatch(da, db, k=2)\n good = [x[0] for x in m if len(x) == 2 and x[0].distance \u003c 0.75 * x[1].distance]\n if len(good) \u003c 8:\n return 0, len(good)\n pa = np.float32([ka[g.queryIdx].pt for g in good])\n pb = np.float32([kb[g.trainIdx].pt for g in good])\n H, mask = cv2.findHomography(pa, pb, cv2.RANSAC, 6.0)\n if H is None or mask is None:\n return 0, len(good)\n n = int(mask.sum())\n # For a projective map the local orientation is det(H) / (h3 @ [x,y,1])**3.\n # The upper-left 2x2 determinant alone can reject valid perspective and accept reflections.\n keep = mask.ravel().astype(bool)\n denom = pa[keep] @ H[2, :2] + H[2, 2]\n orientation = np.linalg.det(H) * denom # Same sign, without dividing near a projective horizon.\n distinct = len({(round(float(x)), round(float(y))) for x, y in pb[keep]})\n if not np.all(np.isfinite(orientation) & (orientation > 0)) or distinct \u003c n / 2:\n return 0, len(good)\n return n, len(good)\n\n\n# ---------------------------------------------------------------- candidate sources\n\ndef _items_from_panos(args) -> list[dict]:\n import baidu_pano as bp\n panos = json.loads(Path(args.panos).read_text(encoding=\"utf-8\"))\n if args.within:\n wl, wo, wr = (float(v) for v in args.within.split(\",\"))\n panos = {k: v for k, v in panos.items() if geo.distance((wl, wo), tuple(v[\"wgs\"])) \u003c= wr}\n if args.spread:\n panos = bp.thin(panos, args.spread)\n target = tuple(float(v) for v in args.toward.split(\",\")) if args.toward else None\n items = []\n for pid, v in panos.items():\n heads = [float(x) for x in args.headings.split(\",\")] if args.headings else (\n [geo.bearing(tuple(v[\"wgs\"]), target) + args.offset] if target else None)\n if heads is None:\n sys.exit(\"--panos needs --toward or --headings\")\n for hd in heads:\n items.append({\"id\": pid, \"heading\": hd % 360, \"pitch\": args.pitch, \"fov\": args.fov, \"wgs\": v[\"wgs\"],\n \"road\": v.get(\"road\", \"\"), \"date\": v.get(\"date\", \"\")})\n return items\n\n\ndef _overlay(engine: str | None) -> tuple | None:\n \"\"\"The box a street-view renderer bakes into every view (its LOGO_BOX), to keep out of SIFT; None for plain images.\"\"\"\n if not engine:\n return None\n return getattr(__import__(\"baidu_pano\" if engine == \"baidu\" else \"gsv\"), \"LOGO_BOX\", None)\n\n\ndef _render_items(items: list[dict], engine: str, proxy: str | None, cache: Path) -> list[Image.Image | None]:\n if engine == \"gsv\":\n import gsv\n def one(it):\n try:\n return gsv.render(it[\"id\"], it[\"heading\"], it.get(\"pitch\", 0), it.get(\"fov\", 90), 640, 480, proxy, cache / \"gsv\")\n except Exception: # noqa: BLE001\n return None\n else:\n import baidu_pano as bp\n def one(it):\n try:\n return bp.render(it[\"id\"], it[\"heading\"], it.get(\"pitch\", 10), it.get(\"fov\", 80), cache=cache / \"pano\", proxy=proxy)\n except Exception: # noqa: BLE001\n return None\n with ThreadPoolExecutor(12) as ex:\n return list(ex.map(one, items))\n\n\ndef _open_images(files: list[Path]) -> tuple[list[Path], list[Image.Image]]:\n \"\"\"Decode every candidate up front; skip the files that can't be read as a supported image and say why, instead of failing mid-run.\"\"\"\n ok, ims, bad = [], [], []\n for f in files:\n try:\n im = upright(open_image(f))\n except ImageReadError as e:\n # group by reason; for decode errors drop the exception detail so one cause stays one line\n bad.append((f, e.reason.split(\" (\")[0] if e.category in (\"decode-error\", \"not-image\") else e.reason))\n continue\n ok.append(f)\n ims.append(im)\n if bad:\n kinds: dict[str, int] = {}\n for _, why in bad:\n kinds[why] = kinds.get(why, 0) + 1\n detail = \"; \".join(f\"{n} {why}\" for why, n in sorted(kinds.items(), key=lambda kv: -kv[1]))\n print(f\"skipped {len(bad)} of {len(files)} files that are not readable images: {detail}; e.g. {bad[0][0].name}\", file=sys.stderr)\n if not ims:\n sys.exit(f\"none of the {len(files)} candidate files could be read as an image; see the reasons above\")\n return ok, ims\n\n\ndef _load_candidates(args) -> tuple[list[dict], list[Image.Image]]:\n cache = Path(args.cache)\n if args.images:\n p = Path(args.images)\n files = find_images(p) if p.is_dir() else [Path(x) for x in sorted(glob.glob(args.images))]\n if not files:\n sys.exit(f\"--images has no images: {args.images}\")\n files, ims = _open_images(files)\n return [{\"id\": f.stem, \"file\": str(f)} for f in files], ims\n if args.items:\n items = json.loads(Path(args.items).read_text(encoding=\"utf-8\"))\n if args.spread_headings:\n ds = [float(x) for x in args.spread_headings.split(\",\")]\n items = [dict(it, heading=(it[\"heading\"] + d) % 360) for it in items for d in ds]\n elif args.panos:\n items = _items_from_panos(args)\n else:\n sys.exit(\"candidate source: pick one of --images / --items / --panos\")\n if len(items) > args.max_candidates:\n if args.toward and all(it.get(\"wgs\") for it in items):\n tgt = tuple(float(v) for v in args.toward.split(\",\"))\n items.sort(key=lambda it: geo.distance(tuple(it[\"wgs\"]), tgt))\n how = \"nearest to the --toward target first\"\n else:\n how = \"in list order\"\n print(f\"{len(items)} candidates, more than --max-candidates {args.max_candidates}; taking {args.max_candidates} {how} (narrow first with --within/--spread)\", file=sys.stderr)\n items = items[: args.max_candidates]\n engine = args.render or (\"gsv\" if items and str(items[0].get(\"id\", \"\")).startswith((\"CAoS\", \"CIHM\")) or len(str(items[0].get(\"id\", \"\"))) == 22 else \"baidu\")\n ims = _render_items(items, engine, args.proxy, cache)\n ok_items, ok_ims = [], []\n for it, im in zip(items, ims):\n if im is not None:\n ok_items.append(dict(it, render=engine))\n ok_ims.append(im)\n if not ok_ims:\n sys.exit(\"not a single candidate rendered: check panorama ids, coverage and service availability with doctor.py --network\")\n if len(ok_ims) \u003c len(items):\n print(f\"{len(items) - len(ok_ims)} failed to render, skipped\", file=sys.stderr)\n return ok_items, ok_ims\n\n\ndef _sheet(rows: list[dict], ims: dict, out: Path, cols: int = 3, tw: int = 480, th: int = 360) -> None:\n from baidu_pano import _font\n n = len(rows)\n S = Image.new(\"RGB\", (cols * tw, max(1, (n + cols - 1) // cols) * th), \"black\")\n d = ImageDraw.Draw(S)\n f = _font(16)\n for i, r in enumerate(rows):\n x, y = (i % cols) * tw, (i // cols) * th\n S.paste(ims[r[\"id_key\"]].convert(\"RGB\").resize((tw, th)), (x, y))\n t = f\"#{r['rank']} inliers {r['inliers'] if r['inliers'] is not None else '-'} global {r['score_global']:.3f} …{str(r['id'])[-9:]}\"\n if r.get(\"heading\") is not None:\n t += f\" h{r['heading']:.0f}\"\n d.rectangle([x, y, x + tw, y + 22], fill=\"black\")\n d.text((x + 4, y + 2), t, fill=\"yellow\", font=f)\n S.save(out, quality=88)\n\n\ndef cmd_rank(args) -> None:\n _proxy_env(args.proxy)\n q = open_photo(args.query, \"query\")\n if args.query_box:\n x0, y0, x1, y1 = (int(float(v)) for v in args.query_box.split(\",\"))\n q = q.crop((x0, y0, x1, y1))\n items, ims = _load_candidates(args)\n t0 = time.time()\n methods = [\"dino\", \"clip\"] if args.method == \"both\" else [args.method]\n sims = np.zeros(len(ims))\n for m in methods:\n emb = Embedder(m)\n qf = emb.embed([q], multi=True)[0] # (V, D)\n cf = emb.embed(ims, multi=False)[:, 0] # (N, D)\n s = (cf @ qf.T).max(axis=1) # each candidate takes its max similarity over the query's views\n sims += s / len(methods)\n order = np.argsort(-sims)\n rows = []\n for rk, i in enumerate(order):\n it = items[i]\n rows.append({\"rank\": rk + 1, \"id\": it.get(\"id\"), \"id_key\": i, \"file\": it.get(\"file\"), \"score_global\": round(float(sims[i]), 4),\n \"inliers\": None, \"matches\": None, \"refined\": False, \"heading\": it.get(\"heading\"), \"wgs\": it.get(\"wgs\"),\n \"road\": it.get(\"road\", \"\"), \"date\": it.get(\"date\", \"\"), \"label\": it.get(\"label\", \"\")})\n t1 = time.time()\n if args.refine != \"none\":\n top = rows[: args.refine_top]\n def one(r):\n return sift_inliers(q, ims[r[\"id_key\"]], skip_b=_overlay(items[r[\"id_key\"]].get(\"render\")))\n with ThreadPoolExecutor(4) as ex:\n for r, (inl, mt) in zip(top, ex.map(one, top)):\n r[\"inliers\"], r[\"matches\"], r[\"refined\"] = inl, mt, True\n rows.sort(key=lambda r: (-(r[\"inliers\"] or 0) if (r[\"inliers\"] or 0) >= args.min_inliers else 0, -r[\"score_global\"]))\n for k, r in enumerate(rows):\n r[\"rank\"] = k + 1\n t2 = time.time()\n out_rows = rows[: args.top]\n print(f\"{len(ims)} candidates; global scoring {t1 - t0:.1f}s, refinement {t2 - t1:.1f}s\")\n print(f\"{'#':>3} {'inliers':>7} {'global':>7} id / heading / position\")\n for r in out_rows:\n pos = f\"{r['wgs'][0]:.5f},{r['wgs'][1]:.5f}\" if r.get(\"wgs\") else (r.get(\"file\") or \"\")\n print(f\"{r['rank']:>3} {(r['inliers'] if r['inliers'] is not None else '-'):>7} {r['score_global']:>7.3f} …{str(r['id'])[-10:]}\"\n f\" h{r['heading']:.0f} {pos} {r.get('road', '')}\" if r.get(\"heading\") is not None else\n f\"{r['rank']:>3} {(r['inliers'] if r['inliers'] is not None else '-'):>7} {r['score_global']:>7.3f} {r['id']} {pos}\")\n strong = [r for r in out_rows if (r[\"inliers\"] or 0) >= args.min_inliers]\n if args.refine != \"none\":\n if len(rows) > args.refine_top: # inliers null / \"-\" means \"not refined\", not \"0 matches\" (issue #37)\n print(f\"inliers '-' = ranked below the top {args.refine_top} by global score, so SIFT never ran on it (refined: false in --out); \"\n f\"that is not 0 matches and not a rejection. A candidate that other evidence points to can be checked with a larger \"\n f\"--refine-top or a --query-box on its stable structures\")\n print(f\"{len(strong)} with inliers ≥{args.min_inliers}\" + (\": open these first and compare invariant features\" if strong else \": this does not mean none of them is right — with a season change, an old capture, or the photo taken on the sidewalk while the street view is from the middle of the road, the ground truth also often has only single-digit inliers. First open the top 10 in --sheet and compare invariant features; if none match, change heading (--spread-headings) or widen the area\"))\n if args.out:\n Path(args.out).write_text(json.dumps([{k: v for k, v in r.items() if k != \"id_key\"} for r in rows], ensure_ascii=False, indent=1), encoding=\"utf-8\")\n print(f\"-> {args.out} (all {len(rows)} rows)\")\n if args.sheet:\n _sheet(out_rows, {r[\"id_key\"]: ims[r[\"id_key\"]] for r in out_rows}, Path(args.sheet))\n print(f\"-> {args.sheet}\")\n\n\ndef cmd_index(args) -> None:\n _proxy_env(args.proxy)\n p = Path(args.images)\n files = find_images(p) if p.is_dir() else [Path(x) for x in sorted(glob.glob(args.images))]\n if not files:\n sys.exit(f\"--images has no images: {args.images}\")\n files, ims = _open_images(files)\n emb = Embedder(args.method)\n feats = emb.embed(ims, multi=False)[:, 0]\n np.savez(args.out, ids=np.array([f.stem for f in files]), files=np.array([str(f) for f in files]), feats=feats, method=args.method)\n print(f\"{len(files)} images -> {args.out}\")\n\n\ndef _neg_coords(argv: list[str]) -> list[str]:\n return [\" \" + a if re.match(r\"^-\\d[\\d.]*(,-?[\\d.]+)+$\", a) else a for a in argv]\n\n\ndef main() -> None:\n ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)\n ap.add_argument(\"--proxy\", default=os.environ.get(\"GEO_PROXY\"), help=PROXY_HELP)\n ap.add_argument(\"--cache\", type=Path, default=Path(\".geo-cache\"))\n sub = ap.add_subparsers(dest=\"cmd\", required=True)\n\n r = sub.add_parser(\"rank\")\n r.add_argument(\"--query\", required=True)\n r.add_argument(\"--query-box\", help=\"x0,y0,x1,y1: compare only this region\")\n r.add_argument(\"--images\")\n r.add_argument(\"--items\", help=\".index.json from baidu_pano.py / gsv.py\")\n r.add_argument(\"--render\", choices=[\"baidu\", \"gsv\"], help=\"which engine renders --items; if omitted, guessed from the id format\")\n r.add_argument(\"--spread-headings\", help=\"extra heading offsets for each --items entry, e.g. -30,0,30\")\n r.add_argument(\"--panos\", help=\"output of baidu_pano.py scan\")\n r.add_argument(\"--toward\", help=\"lat,lon\")\n r.add_argument(\"--headings\")\n r.add_argument(\"--offset\", type=float, default=0)\n r.add_argument(\"--within\", help=\"lat,lon,radius in meters\")\n r.add_argument(\"--spread\", type=float, help=\"thinning spacing in meters\")\n r.add_argument(\"--pitch\", type=float, default=10)\n r.add_argument(\"--fov\", type=float, default=80)\n r.add_argument(\"--max-candidates\", type=int, default=400)\n r.add_argument(\"--method\", choices=[\"dino\", \"clip\", \"both\"], default=\"dino\")\n r.add_argument(\"--refine\", choices=[\"none\", \"sift\"], default=\"sift\")\n r.add_argument(\"--refine-top\", type=int, default=30)\n r.add_argument(\"--min-inliers\", type=int, default=15)\n r.add_argument(\"--top\", type=int, default=10)\n r.add_argument(\"--out\")\n r.add_argument(\"--sheet\")\n r.add_argument(\"--proxy\", default=argparse.SUPPRESS)\n r.add_argument(\"--cache\", type=Path, default=argparse.SUPPRESS)\n\n i = sub.add_parser(\"index\")\n i.add_argument(\"--images\", required=True)\n i.add_argument(\"--method\", choices=[\"dino\", \"clip\"], default=\"dino\")\n i.add_argument(\"--out\", required=True)\n i.add_argument(\"--proxy\", default=argparse.SUPPRESS)\n\n args = ap.parse_args(_neg_coords(sys.argv[1:]))\n if args.cmd == \"rank\":\n cmd_rank(args)\n else:\n cmd_index(args)\n\n\nif __name__ == \"__main__\":\n # Chinese-locale Windows writes GBK by default: m², ñ make it crash, and the Chinese the agent reads comes out garbled\n sys.stdout.reconfigure(encoding=\"utf-8\")\n sys.stderr.reconfigure(encoding=\"utf-8\", errors=\"backslashreplace\")\n main()\n","content_type":"text/x-python; charset=utf-8","language":"python","size":22535,"content_sha256":"79a388c78b17f7959d5f6e7614aaeaffd1e364bcb8c53ba934ab6bb0619c3c12"},{"filename":"scripts/ocr.py","content":"#!/usr/bin/env python3\n# /// script\n# requires-python = \">=3.10\"\n# dependencies = [\"pillow\", \"pillow-heif\", \"pyobjc-framework-Vision; sys_platform == 'darwin'\", \"pyobjc-framework-Quartz; sys_platform == 'darwin'\", \"rapidocr-onnxruntime\"]\n# ///\n\"\"\"Read the text in a photo (second reader): runs once each on the whole image, upscaled, and in tiles; merges and dedupes; marks which text was only readable after zooming.\n\nBackends: on macOS, Apple Vision (local, no download, handles Chinese, English, Japanese and Korean); on other systems or when Vision isn't available, RapidOCR.\nText read only after upscaling or tiling (pass = up / tile) is just an assumption; go back to the original image and zoom in to take a look.\n\n ocr.py photo.jpg [--langs zh-Hans,en] [--upscale 2] [--tiles 2x2] [--min-conf 0.3] [--out ocr.json] [--draw ocr.png]\n\nExamples:\n ocr.py photo.jpg --out ocr.json --draw ocr.png\n ocr.py photo.jpg --langs zh-Hans,zh-Hant,en,ja --tiles 3x3 --upscale 3 # lots of small distant text: finer tiles, more upscaling\n\"\"\"\nfrom __future__ import annotations\n\nimport argparse\nfrom _image import open_photo\nimport json\nimport sys\nimport time\nfrom pathlib import Path\n\nfrom PIL import Image, ImageDraw\n\n\ndef _iou(a, b) -> float:\n x0, y0, x1, y1 = max(a[0], b[0]), max(a[1], b[1]), min(a[2], b[2]), min(a[3], b[3])\n inter = max(0, x1 - x0) * max(0, y1 - y0)\n ua = (a[2] - a[0]) * (a[3] - a[1]) + (b[2] - b[0]) * (b[3] - b[1]) - inter\n return inter / ua if ua > 0 else 0.0\n\n\nclass VisionOCR:\n name = \"apple-vision\"\n\n def __init__(self, langs: list[str]):\n import Quartz # noqa: F401\n import Vision\n self.Vision = Vision\n self.langs = langs\n\n def run(self, im: Image.Image) -> list[dict]:\n import io\n from Foundation import NSData\n Vision = self.Vision\n buf = io.BytesIO()\n im.convert(\"RGB\").save(buf, format=\"PNG\")\n data = NSData.dataWithBytes_length_(buf.getvalue(), len(buf.getvalue()))\n handler = Vision.VNImageRequestHandler.alloc().initWithData_options_(data, None)\n req = Vision.VNRecognizeTextRequest.alloc().init()\n req.setRecognitionLevel_(Vision.VNRequestTextRecognitionLevelAccurate)\n req.setUsesLanguageCorrection_(False) # don't \"correct\" proper names away\n try:\n req.setRecognitionLanguages_(self.langs)\n except Exception: # noqa: BLE001\n pass\n ok, err = handler.performRequests_error_([req], None)\n if not ok:\n raise RuntimeError(f\"Vision failed: {err}\")\n W, H = im.size\n out = []\n for obs in req.results() or []:\n cand = obs.topCandidates_(1)\n if not cand:\n continue\n text = str(cand[0].string())\n conf = float(cand[0].confidence())\n bb = obs.boundingBox() # normalized, origin at bottom-left\n x, y, w, h = bb.origin.x, bb.origin.y, bb.size.width, bb.size.height\n out.append({\"text\": text, \"conf\": round(conf, 3), \"box\": [int(x * W), int((1 - y - h) * H), int((x + w) * W), int((1 - y) * H)]})\n return out\n\n\nclass RapidOCRBackend:\n name = \"rapidocr\"\n\n def __init__(self, langs: list[str]):\n from rapidocr_onnxruntime import RapidOCR\n self.ocr = RapidOCR()\n\n def run(self, im: Image.Image) -> list[dict]:\n import numpy as np\n res, _ = self.ocr(np.array(im.convert(\"RGB\"))[:, :, ::-1])\n out = []\n for box, text, conf in res or []:\n xs = [p[0] for p in box]\n ys = [p[1] for p in box]\n out.append({\"text\": str(text), \"conf\": round(float(conf), 3), \"box\": [int(min(xs)), int(min(ys)), int(max(xs)), int(max(ys))]})\n return out\n\n\ndef backend(langs: list[str]):\n if sys.platform == \"darwin\":\n try:\n return VisionOCR(langs)\n except Exception as e: # noqa: BLE001\n print(f\"Apple Vision unavailable ({str(e)[:120]}), using RapidOCR instead\", file=sys.stderr)\n try:\n return RapidOCRBackend(langs)\n except Exception as e: # noqa: BLE001\n sys.exit(f\"No OCR backend available: {e}\")\n\n\ndef recognize(im: Image.Image, be, upscale: float, tiles: tuple[int, int], min_conf: float) -> list[dict]:\n W, H = im.size\n found: list[dict] = []\n\n def add(items, pass_name, ox=0, oy=0, scale=1.0):\n for it in items:\n if it[\"conf\"] \u003c min_conf or not it[\"text\"].strip():\n continue\n b = it[\"box\"]\n box = [int(ox + b[0] / scale), int(oy + b[1] / scale), int(ox + b[2] / scale), int(oy + b[3] / scale)]\n new = {\"text\": it[\"text\"].strip(), \"conf\": it[\"conf\"], \"box\": box, \"pass\": pass_name}\n for old in found:\n if _iou(old[\"box\"], box) > 0.4:\n if new[\"conf\"] > old[\"conf\"] + 0.05 or (len(new[\"text\"]) > len(old[\"text\"]) and new[\"conf\"] >= old[\"conf\"] - 0.1):\n old.update(new)\n break\n else:\n found.append(new)\n\n add(be.run(im), \"full\")\n if upscale and upscale > 1:\n big = im.resize((int(W * upscale), int(H * upscale)), Image.LANCZOS)\n add(be.run(big), \"up\", scale=upscale)\n r, c = tiles\n if r * c > 1:\n tw, th = W / c, H / r\n for i in range(r):\n for j in range(c):\n # Each tile keeps a 10% overlap so text isn't cut at the edges\n x0, y0 = max(0, int(j * tw - tw * 0.1)), max(0, int(i * th - th * 0.1))\n x1, y1 = min(W, int((j + 1) * tw + tw * 0.1)), min(H, int((i + 1) * th + th * 0.1))\n s = max(upscale or 1, 2)\n crop = im.crop((x0, y0, x1, y1)).resize((int((x1 - x0) * s), int((y1 - y0) * s)), Image.LANCZOS)\n add(be.run(crop), \"tile\", ox=x0, oy=y0, scale=s)\n found.sort(key=lambda t: -t[\"conf\"])\n return found\n\n\ndef _where(box, W, H) -> str:\n cx, cy = (box[0] + box[2]) / 2 / W, (box[1] + box[3]) / 2 / H\n v = \"top\" if cy \u003c 0.33 else (\"bottom\" if cy > 0.67 else \"middle\")\n h = \"left\" if cx \u003c 0.33 else (\"right\" if cx > 0.67 else \"center\")\n return f\"{v}-{h}\" if (v, h) != (\"middle\", \"center\") else \"center\"\n\n\ndef main() -> None:\n ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)\n ap.add_argument(\"image\")\n ap.add_argument(\"--langs\", default=\"zh-Hans,zh-Hant,en,ja,ko,fr,es,pt,de,it,ru,th,vi\")\n ap.add_argument(\"--upscale\", type=float, default=2)\n ap.add_argument(\"--tiles\", default=\"2x2\")\n ap.add_argument(\"--min-conf\", type=float, default=0.3)\n ap.add_argument(\"--out\")\n ap.add_argument(\"--draw\")\n args = ap.parse_args()\n im = open_photo(args.image, \"image\")\n t0 = time.time()\n be = backend([x.strip() for x in args.langs.split(\",\") if x.strip()])\n r, c = (int(v) for v in args.tiles.lower().split(\"x\"))\n found = recognize(im, be, args.upscale, (r, c), args.min_conf)\n W, H = im.size\n print(f\"{be.name}: {len(found)} items, {time.time() - t0:.1f}s (image {W}×{H})\")\n print(f\"{'conf':>5} {'pass':\u003c5} {'where':\u003c13} text\")\n for t in found:\n print(f\"{t['conf']:>5.2f} {t['pass']:\u003c5} {_where(t['box'], W, H):\u003c13} {t['text']}\")\n if any(t[\"pass\"] != \"full\" for t in found):\n print(\"Text with pass=up/tile was only read after zooming; treat it as an assumption: go back to the original, zoom in and look before using it\")\n if args.out:\n Path(args.out).write_text(json.dumps({\"image\": args.image, \"size\": [W, H], \"backend\": be.name, \"items\": found}, ensure_ascii=False, indent=1), encoding=\"utf-8\")\n print(f\"-> {args.out}\")\n if args.draw:\n d = ImageDraw.Draw(im)\n for t in found:\n col = \"lime\" if t[\"pass\"] == \"full\" else \"yellow\"\n d.rectangle(t[\"box\"], outline=col, width=2)\n d.text((t[\"box\"][0], max(0, t[\"box\"][1] - 12)), f\"{t['text'][:20]} {t['conf']:.2f}\", fill=col)\n im.save(args.draw)\n print(f\"-> {args.draw} (green = read from the whole image, yellow = read after upscaling/tiling)\")\n\n\nif __name__ == \"__main__\":\n # Chinese Windows outputs GBK by default: it crashes on m², ñ, and Chinese text the agent reads comes out garbled\n sys.stdout.reconfigure(encoding=\"utf-8\")\n sys.stderr.reconfigure(encoding=\"utf-8\", errors=\"backslashreplace\")\n main()\n","content_type":"text/x-python; charset=utf-8","language":"python","size":8380,"content_sha256":"55949b422bffcce6e577b1dea471f8785f0114e48a693a77546b3da72ca6c1f8"},{"filename":"scripts/osm.py","content":"#!/usr/bin/env python3\n# /// script\n# requires-python = \">=3.10\"\n# dependencies = []\n# ///\n\"\"\"OpenStreetMap Overpass queries: find candidate points by \"feature combinations\" and \"linear corridors\" instead of searching the whole map.\n\nSuited to photos with \"no name, but structure\": a power tower next to a high-speed rail bridge, a church at a river bend, a level crossing with four tracks...\nData outside China is very complete; in China, roads, rivers, railways, power lines and large buildings are usable, while small shops and compound interiors are basically missing.\nOSM data in China is incomplete: results can only be a source of candidates, not grounds for exclusion.\n\n find find features of one kind within an area\n near find A that has B within N meters (optionally also C)\n crossings line-to-point: bridges, dams, ferries, level crossings on a river / railway / road (level crossings estimate track count from node count)\n route bus / rail / ferry route number → sample points along the route, turning \"search the whole city\" into \"search along one line\"\n intersect crossing points of two kinds of linear features (railway × power line ...), optionally requiring a bend beyond the crossing (angle tower, river bend)\n street-scan street view geometry template: bearing looking down a street + \"building / no building\" on each side → candidate intersections for a whole town\n along take points at a step along any road / river / line (optionally offset to one side of the road), for satellite thumbnails or street view scans of the roadside\n buildings list large buildings by footprint area (factories, warehouses, farm sheds), marking how sparse the surroundings are\n coverage how many features of a kind each candidate admin area has: check OSM coverage before enumerating candidates\n geom export GeoJSON for any filter (keeps line and polygon geometry) for custom analysis\n raw run your own Overpass QL ({{bbox}} is replaced with s,w,n,e)\n\nRun queries one after another: public Overpass servers reject parallel requests from one address (every mirror then fails), and a bbox query is often faster than --area for a whole province.\n\nOutputs JSON {name or id: [lat, lon]} (WGS84), which can go straight to tiles.py mark --points to draw on satellite imagery.\n\nExamples (江苏省 = Jiangsu Province, 长江 = Yangtze River):\n osm.py find --bbox 30.24,120.12,30.27,120.17 '[\"highway\"=\"street_lamp\"]'\n osm.py near --area 江苏省 --a '[\"railway\"=\"rail\"][\"highspeed\"=\"yes\"][\"bridge\"]' --b '[\"power\"=\"tower\"]' --within 700 --c '[\"waterway\"=\"river\"]' --within-c 100\n osm.py crossings --bbox 31.9,118.4,32.3,119.0 --line '[\"waterway\"=\"river\"][\"name\"=\"长江\"]' --kind bridge,dam,ferry\n osm.py crossings --bbox \u003cs,w,n,e> --line '[\"railway\"=\"rail\"]' --kind level_crossing\n osm.py route --bbox \u003cs,w,n,e> --kind bus --ref \u003croute number> --step 150 --out route.json\n osm.py intersect --area \u003cfull name of province-level admin area> --a '[\"railway\"=\"rail\"][\"electrified\"=\"contact_line\"]' --b '[\"power\"=\"line\"]' --bend-min 25 --rank-near '[\"place\"~\"^(city|town)$\"]'\n osm.py street-scan --bbox \u003ctown s,w,n,e> --bearing 320:80 --right building --left empty --out cands.json\n osm.py geom '[\"building\"]' --bbox 30.25,120.15,30.26,120.16 --out b.geojson\n osm.py along --bbox \u003cs,w,n,e> --line '[\"highway\"][\"ref\"=\"\u003croad number>\"]' --step 400 --side north --offset 80 --out pts.json\n osm.py buildings --bbox \u003cs,w,n,e> --min-area 1500 --sort sparse --out big.json\n osm.py coverage --areas \u003cdistrict A>,\u003cdistrict B>,\u003cdistrict C> --filter '[\"leisure\"~\"^(pitch|track)$\"]'\n osm.py raw query.overpassql --bbox 30.2,120.1,30.3,120.2\n\"\"\"\nfrom __future__ import annotations\n\nimport argparse\nfrom _net import curl_args, resolve_proxy, PROXY_HELP\nimport hashlib\nimport json\nimport math\nimport os\nimport re\nimport subprocess\nimport sys\nimport time\nimport urllib.error\nimport urllib.parse\nimport urllib.request\nfrom pathlib import Path\n\nsys.path.insert(0, str(Path(__file__).parent))\nimport regions # noqa: E402\n\n# Independent services only: overpass.kumi.systems is the old name of private.coffee (same server), so it can't confirm or replace it\nENDPOINTS = [\n \"https://overpass-api.de/api/interpreter\",\n \"https://overpass.private.coffee/api/interpreter\",\n \"https://maps.mail.ru/osm/tools/overpass/api/interpreter\",\n]\n# overpass-api.de rejects curl's default User-Agent with HTTP 406; public instances ask clients to identify themselves\nUA = \"geo-sleuth (+https://github.com/Oldcircle/geo-sleuth)\"\n\n\nAREA_SET = re.compile(r\"(\$(?:area\\[[^;]*\\];)+\$)->\\.(\\w+);\")\n\n\ndef _post(ep: str, ql: str, proxy: str | None, timeout: int) -> tuple[dict | None, str]:\n \"\"\"(reply, \"\") or (None, \"HTTP 504: \u003cwhat the server said>\").\"\"\"\n cmd = [\"curl\", \"-q\", \"-s\", \"-m\", str(timeout + 30), \"-A\", UA, \"-w\", \"\\n%{http_code}\", \"--data-urlencode\", f\"data={ql}\", ep]\n cmd += curl_args(proxy)\n r = subprocess.run(cmd, capture_output=True, text=True, encoding=\"utf-8\", errors=\"replace\")\n body, _, code = (r.stdout or \"\").rpartition(\"\\n\")\n if r.returncode == 35:\n # TLS handshake failed: Windows 10's Schannel curl has no TLS 1.3, and overpass-api.de accepts nothing older; Python's OpenSSL does\n value = resolve_proxy(proxy)\n opener = urllib.request.build_opener(urllib.request.ProxyHandler({\"http\": value, \"https\": value} if value else {}))\n req = urllib.request.Request(ep, data=urllib.parse.urlencode({\"data\": ql}).encode(), headers={\"User-Agent\": UA})\n try:\n with opener.open(req, timeout=timeout + 30) as resp:\n body, code = resp.read().decode(\"utf-8\", \"replace\"), str(resp.status)\n except urllib.error.HTTPError as e: # 4xx/5xx still carry the server's message, as with curl\n body, code = e.read().decode(\"utf-8\", \"replace\"), str(e.code)\n except OSError as e:\n return None, f\"curl exit 35, urllib: {e}\"\n elif r.returncode:\n return None, f\"curl exit {r.returncode}\"\n try:\n data = json.loads(body)\n except json.JSONDecodeError:\n text = \" \".join(re.sub(r\"\u003c[^>]+>\", \" \", body).split())\n return None, f\"HTTP {code}: {text[:160]}\"\n return (data, \"\") if isinstance(data, dict) else (None, f\"HTTP {code}: {str(data)[:160]}\")\n\n\ndef _area_empty(data: dict, ql: str) -> bool:\n \"\"\"An --area query that came back empty (or with a zero area count). Mirrors disagree on these: one answered a whole-province\n power-line query with a clean, empty result while others returned thousands, so an empty answer needs a second mirror.\n The same goes for rel(\u003cid>);map_to_area (gazetteer.py): a false empty level made `children` skip Inner Mongolia's prefectures.\"\"\"\n if not AREA_SET.search(ql) and \"map_to_area\" not in ql:\n return False\n els = data.get(\"elements\") or []\n if els and els[0].get(\"type\") == \"count\": # coverage / _area_count: the first count is the area count\n return int((els[0].get(\"tags\") or {}).get(\"total\", 0)) == 0\n return not els\n\n\ndef run(ql: str, proxy: str | None, cache: Path, timeout: int = 180, rounds: int = 3) -> dict:\n \"\"\"Try each mirror in turn; when servers are busy (common for public Overpass), wait a while and retry the whole round. Results are cached per query.\n Error replies are never cached; an empty --area result is cached only when a second mirror agrees.\"\"\"\n cache.mkdir(parents=True, exist_ok=True)\n key = cache / (hashlib.sha1(ql.encode()).hexdigest()[:16] + \".json\")\n if key.exists():\n cached = json.loads(key.read_text(encoding=\"utf-8\"))\n confirmed = cached.pop(\"_geo_empty_confirmed\", False)\n if not _area_empty(cached, ql) or confirmed is True:\n return cached\n # Older releases cached single-mirror empty results. They need the same check as a fresh response.\n print(\"Rechecking an old empty area cache against the mirrors\", file=sys.stderr)\n errs: dict[str, str] = {}\n empty_from = None\n for rnd in range(rounds):\n for ep in ENDPOINTS:\n if ep == empty_from:\n continue\n data, err = _post(ep, ql, proxy, timeout)\n if data is None:\n errs[ep] = err\n continue\n remark = data.get(\"remark\") or \"\"\n if remark and not data.get(\"elements\"): # runtime error / timeout / out of memory with nothing returned\n errs[ep] = f\"remark: {remark[:160]}\"\n continue\n if _area_empty(data, ql) and empty_from is None:\n empty_from, first_empty = ep, data\n continue\n if remark:\n print(f\"Overpass remark (results may be incomplete): {remark[:200]}\", file=sys.stderr)\n stored = dict(data)\n if empty_from is not None and _area_empty(data, ql):\n stored[\"_geo_empty_confirmed\"] = True\n key.write_text(json.dumps(stored), encoding=\"utf-8\")\n return data\n if empty_from is not None:\n print(f\"Only {empty_from} answered, with an empty --area result; not confirmed by a second mirror, so not cached. \"\n f\"Rerun later, or use --bbox, before treating it as \\\"none\\\".\", file=sys.stderr)\n return first_empty\n last = \"; \".join(f\"{ep.split('/')[2]} {e}\" for ep, e in errs.items())\n if rnd \u003c rounds - 1:\n print(f\"No Overpass mirror returned a result, retrying in {15 * (rnd + 1)} s: {last}\", file=sys.stderr)\n time.sleep(15 * (rnd + 1))\n sys.exit(f\"Overpass query failed on every mirror ({last}). HTTP 504/\\\"too busy\\\": retry later, or use --bbox / a smaller area; \"\n f\"HTTP 400: check the query syntax; curl exit: network or proxy (doctor.py --network)\")\n\n\ndef _area_sel(name: str, var: str, loose: bool = False) -> str:\n \"\"\"Get OSM areas by name and store them in set var.\n Some areas have bilingual names in OSM (\"\u003clocal name> \u003csecond script>\"), so [\"name\"=\"…\"] alone silently returns 0 results;\n the region packs' extra name keys (regions.osm_name_keys) are matched too (indexed, fast).\n loose=True also adds a prefix regex \"name + space + other text\" as a fallback for bilingual names without such a key; it scans all area names, which is slow, so use it only when the lookup above finds nothing.\"\"\"\n q = name.replace(\"\\\\\", \"\\\\\\\\\").replace('\"', '\\\\\"')\n sel = \"\".join(f'area[\"{k}\"=\"{q}\"];' for k in regions.osm_name_keys())\n if loose:\n rx = re.sub(r'([.^$*+?()\\[\\]{}|\\\\])', r\"\\\\\\\\\\1\", name).replace('\"', '\\\\\"')\n sel += f'area[\"name\"~\"^{rx} \"];'\n return f\"({sel})->.{var};\"\n\n\ndef _area_count(name: str, args, loose: bool = False) -> int:\n \"\"\"How many areas OSM returns for this name (used after a 0-result query to tell \"really none\" from \"wrong name\").\"\"\"\n els = run(f\"[out:json][timeout:120];{_area_sel(name, 'a', loose)}.a out count;\", args.proxy, args.cache).get(\"elements\") or []\n return int(((els[0].get(\"tags\") or {}).get(\"total\", 0)) if els else 0)\n\n\ndef _scope(args) -> tuple[str, str]:\n \"\"\"Returns (preamble statement, filter suffix).\"\"\"\n if args.area:\n return _area_sel(args.area, \"searchArea\", getattr(args, \"area_loose\", False)), \"(area.searchArea)\"\n if args.bbox:\n s, w, n, e = args.bbox\n return \"\", f\"({s},{w},{n},{e})\"\n sys.exit(\"Need --bbox or --area\")\n\n\ndef _center(el: dict) -> list[float] | None:\n if el[\"type\"] == \"node\":\n return [el.get(\"lat\"), el.get(\"lon\")]\n c = el.get(\"center\") or {}\n return [c[\"lat\"], c[\"lon\"]] if \"lat\" in c else None\n\n\ndef _points(data: dict) -> dict:\n out = {}\n for el in data.get(\"elements\", []):\n ll = _center(el)\n if not ll or ll[0] is None:\n continue\n name = (el.get(\"tags\") or {}).get(\"name\") or f\"{el['type']}/{el['id']}\"\n if name in out:\n name = f\"{name}#{el['id']}\"\n out[name] = [round(ll[0], 6), round(ll[1], 6)]\n return out\n\n\ndef _dist(a, b) -> float:\n la1, lo1, la2, lo2 = map(math.radians, (*a, *b))\n h = math.sin((la2 - la1) / 2) ** 2 + math.cos(la1) * math.cos(la2) * math.sin((lo2 - lo1) / 2) ** 2\n return 2 * 6371008.8 * math.asin(math.sqrt(h))\n\n\ndef _cluster(items: list[tuple[str, list[float], dict]], radius: float) -> list[dict]:\n \"\"\"Merge results within radius meters into one place (a bridge is often split into several ways; a level crossing has one node per track).\"\"\"\n groups: list[dict] = []\n for name, ll, tags in items:\n for g in groups:\n if _dist(g[\"center\"], ll) \u003c= radius:\n g[\"members\"].append((name, ll, tags))\n n = len(g[\"members\"])\n g[\"center\"] = [(g[\"center\"][0] * (n - 1) + ll[0]) / n, (g[\"center\"][1] * (n - 1) + ll[1]) / n]\n break\n else:\n groups.append({\"center\": list(ll), \"members\": [(name, ll, tags)]})\n return groups\n\n\ndef _sample(elements: list[dict], step: float) -> list[list[float]]:\n \"\"\"Take a point every step meters along the lines from out geom.\"\"\"\n pts = []\n for el in elements:\n geom = el.get(\"geometry\") or []\n acc = 0.0\n for a, b in zip(geom, geom[1:]):\n pa, pb = (a[\"lat\"], a[\"lon\"]), (b[\"lat\"], b[\"lon\"])\n seg = _dist(pa, pb)\n while seg > 0 and acc \u003c= seg:\n t = acc / seg\n pts.append([round(pa[0] + (pb[0] - pa[0]) * t, 6), round(pa[1] + (pb[1] - pa[1]) * t, 6)])\n acc += step\n acc -= seg\n return pts\n\n\ndef cmd_crossings(args) -> dict:\n pre, sc = _scope(args)\n kinds = set(args.kind.split(\",\"))\n buf = f\"{args.buffer:.0f}\"\n parts = []\n if \"bridge\" in kinds:\n parts.append(f'way[\"bridge\"][\"bridge\"!=\"no\"](around.L:{buf}){sc};')\n if \"dam\" in kinds:\n parts.append(f'nwr[\"waterway\"~\"^(dam|weir)$\"](around.L:{buf}){sc};')\n if \"ferry\" in kinds:\n parts.append(f'way[\"route\"=\"ferry\"](around.L:{buf}){sc};')\n if \"level_crossing\" in kinds:\n parts.append(f'node[\"railway\"~\"^(level_crossing|crossing)$\"](around.L:{buf}){sc};')\n if not parts:\n sys.exit(\"--kind only supports bridge,dam,ferry,level_crossing\")\n ql = f\"[out:json][timeout:180];{pre}way{args.line}{sc}->.L;(\" + \"\".join(parts) + \");out center tags;\"\n items = []\n for el in run(ql, args.proxy, args.cache).get(\"elements\", []):\n ll = _center(el)\n if not ll or ll[0] is None:\n continue\n tags = el.get(\"tags\") or {}\n items.append((tags.get(\"name\") or \"\", ll, tags))\n groups = _cluster(items, 25 if kinds == {\"level_crossing\"} else 150)\n print(f\"{len(groups)} places (merged from {len(items)} OSM features)\")\n pts = {}\n for i, g in enumerate(groups, 1):\n tags = [m[2] for m in g[\"members\"]]\n names = sorted({m[0] for m in g[\"members\"] if m[0]})\n if any(t.get(\"railway\") in (\"level_crossing\", \"crossing\") for t in tags):\n kind = f\"level crossing·~{len(g['members'])} tracks\"\n elif any(t.get(\"waterway\") in (\"dam\", \"weir\") for t in tags):\n kind = \"dam/weir\"\n elif any(t.get(\"route\") == \"ferry\" for t in tags):\n kind = \"ferry route\"\n else:\n use = {(\"railway\" if t.get(\"railway\") else \"road\" if t.get(\"highway\") else \"other\") for t in tags}\n kind = \"bridge·\" + \"/\".join(sorted(use))\n label = f\"{i:02d} {kind} {', '.join(names)[:40]}\".strip()\n pts[label] = [round(g[\"center\"][0], 6), round(g[\"center\"][1], 6)]\n return pts\n\n\ndef cmd_route(args) -> dict:\n pre, sc = _scope(args)\n flt = f'[\"type\"=\"route\"][\"route\"=\"{args.kind}\"]'\n if args.ref:\n flt += f'[\"ref\"=\"{args.ref}\"]'\n if args.name:\n flt += f'[\"name\"~\"{args.name}\"]'\n ql = f\"[out:json][timeout:180];{pre}relation{flt}{sc}->.R;.R out tags;way(r.R);out geom;\"\n data = run(ql, args.proxy, args.cache)\n rels = [e for e in data.get(\"elements\", []) if e[\"type\"] == \"relation\"]\n ways = [e for e in data.get(\"elements\", []) if e[\"type\"] == \"way\"]\n for rel in rels:\n t = rel.get(\"tags\") or {}\n print(f\" route {t.get('ref', '')} {t.get('name', '')} {t.get('from', '')}→{t.get('to', '')}\")\n pts = _sample(ways, args.step)\n print(f\"{len(rels)} route relations, {len(ways)} ways; one point every {args.step:.0f} m along the route, {len(pts)} points total\")\n if pts:\n lats, lons = [p[0] for p in pts], [p[1] for p in pts]\n print(f\" route extent bbox: {min(lats):.4f},{min(lons):.4f},{max(lats):.4f},{max(lons):.4f}\")\n return {f\"R{i}\": p for i, p in enumerate(pts)}\n\n\ndef cmd_intersect(args) -> dict:\n \"\"\"Crossing points of two kinds of linear features (railway × power line, river × road ...), optionally requiring B to bend beyond the crossing (angle tower, river bend).\"\"\"\n pre, sc = _scope(args)\n ql = (f\"[out:json][timeout:280];{pre}way{args.a}{sc}->.a;way{args.b}(around.a:30){sc}->.b;\"\n f\".a out geom tags;.b out geom tags;\")\n data = run(ql, args.proxy, args.cache, timeout=280)\n A = [e for e in data.get(\"elements\", []) if e.get(\"geometry\") and _match(e, args.a)]\n B = [e for e in data.get(\"elements\", []) if e.get(\"geometry\") and e not in A]\n if not A or not B:\n print(f\"A {len(A)} lines, B {len(B)} lines: no crossing points possible (change the area or tags)\")\n return {}\n lat0 = A[0][\"geometry\"][0][\"lat\"]\n kx, ky = 111320 * math.cos(math.radians(lat0)), 110540\n\n def pr(g):\n return (g[\"lon\"] * kx, g[\"lat\"] * ky)\n\n cells: dict[tuple[int, int], list] = {}\n for el in A:\n pts = [pr(g) for g in el[\"geometry\"]]\n for s0, s1 in zip(pts, pts[1:]):\n for c in {(int(s0[0] // 500), int(s0[1] // 500)), (int(s1[0] // 500), int(s1[1] // 500))}:\n cells.setdefault(c, []).append((s0, s1, el))\n\n def cross(p1, p2, p3, p4):\n d = (p2[0] - p1[0]) * (p4[1] - p3[1]) - (p2[1] - p1[1]) * (p4[0] - p3[0])\n if abs(d) \u003c 1e-9:\n return None\n t = ((p3[0] - p1[0]) * (p4[1] - p3[1]) - (p3[1] - p1[1]) * (p4[0] - p3[0])) / d\n u = ((p3[0] - p1[0]) * (p2[1] - p1[1]) - (p3[1] - p1[1]) * (p2[0] - p1[0])) / d\n if 0 \u003c= t \u003c= 1 and 0 \u003c= u \u003c= 1:\n return (p1[0] + t * (p2[0] - p1[0]), p1[1] + t * (p2[1] - p1[1]))\n return None\n\n def find_bend(el, hit):\n \"\"\"First bend ≥bend_min in B within bend_within beyond the crossing: returns (distance m, turn angle °) or None.\"\"\"\n if args.bend_min \u003c= 0:\n return None\n lo, hi = _band(args.bend_within)\n pts = [pr(g) for g in el[\"geometry\"]]\n k = min(range(len(pts)), key=lambda i: math.dist(pts[i], hit))\n for direction in (1, -1):\n d = 0.0\n i = k\n while 0 \u003c i + direction \u003c len(pts) - 1:\n nxt = i + direction\n d += math.dist(pts[i], pts[nxt])\n i = nxt\n if d > hi:\n break\n if d \u003c lo:\n continue\n a1 = math.atan2(pts[i][0] - pts[i - 1][0], pts[i][1] - pts[i - 1][1])\n a2 = math.atan2(pts[i + 1][0] - pts[i][0], pts[i + 1][1] - pts[i][1])\n turn = abs(math.degrees(a2 - a1))\n turn = min(turn % 360, 360 - turn % 360)\n if turn >= args.bend_min:\n return round(d), round(turn)\n return None\n\n hits = []\n for el in B:\n pts = [pr(g) for g in el[\"geometry\"]]\n for s0, s1 in zip(pts, pts[1:]):\n for c in {(int(s0[0] // 500), int(s0[1] // 500)), (int(s1[0] // 500), int(s1[1] // 500))}:\n for a0, a1, ael in cells.get(c, []):\n x = cross(a0, a1, s0, s1)\n if not x:\n continue\n ta, tb = ael.get(\"tags\") or {}, el.get(\"tags\") or {}\n lbl = \" \".join(filter(None, [ta.get(\"name\", \"\"), tb.get(\"name\", \"\"), tb.get(\"voltage\", \"\"),\n ta.get(\"electrified\", \"\")]))\n hits.append((lbl[:40], [x[1] / ky, x[0] / kx], {\"bend\": find_bend(el, x)}))\n groups = _cluster(hits, args.cluster)\n rows = []\n for g in groups:\n bends = [m[2][\"bend\"] for m in g[\"members\"] if m[2][\"bend\"]]\n rows.append({\"label\": g[\"members\"][0][0], \"ll\": [round(g[\"center\"][0], 6), round(g[\"center\"][1], 6)],\n \"bend\": min(bends) if bends else None})\n if args.ring:\n c_ll, rmin, rmax = args.ring.split(\":\")\n c = tuple(map(float, c_ll.split(\",\")))\n before = len(rows)\n rows = [r for r in rows if float(rmin) \u003c= _dist(c, r[\"ll\"]) \u003c= float(rmax)]\n print(f\"--ring: {len(rows)}/{before} places kept in the ring {rmin}–{rmax} m from {c_ll}\")\n print(f\"A {len(A)} lines × B {len(B)} lines → {len(hits)} crossing points, merged into {len(groups)} clusters\")\n if args.bend_min > 0:\n with_bend = [r for r in rows if r[\"bend\"]]\n print(f\" of these, {len(with_bend)} have a bend ≥{args.bend_min:g}° in B within {args.bend_within} m beyond the crossing\"\n f\" (labeled \\\"bend\\\"; used only for ranking, see --bend-filter)\")\n if args.bend_filter:\n dropped = [r for r in rows if not r[\"bend\"]]\n rows = with_bend\n print(f\" --bend-filter: dropped {len(dropped)} crossing points without a bend. The bend comes from interpreting the photo; \"\n f\"if none of the candidates match, look back at this dropped batch first\")\n if args.out and dropped:\n dp = args.out.with_name(args.out.stem + \"_dropped.json\")\n dp.write_text(json.dumps({f\"{i:03d} {r['label']}\".strip(): r[\"ll\"] for i, r in enumerate(dropped, 1)},\n ensure_ascii=False, indent=1), encoding=\"utf-8\")\n print(f\" dropped -> {dp}\")\n if args.rank_near:\n rows = _rank_rows(rows, args.rank_near, args)\n elif args.bend_min > 0:\n rows.sort(key=lambda r: r[\"bend\"] is None)\n out = {}\n for i, r in enumerate(rows, 1):\n extra = []\n if r[\"bend\"]:\n extra.append(f\"bend {r['bend'][0]}m/{r['bend'][1]}°\")\n if r.get(\"near_m\") is not None:\n extra.append(f\"{r['near_m'] / 1000:.1f}km from {r['near_name'][:6]}\")\n out[f\"{i:03d} {r['label']} {' '.join(extra)}\".strip()] = r[\"ll\"]\n return out\n\n\n\n_SIDES = {\"north\": (0.0, 1.0), \"south\": (0.0, -1.0), \"east\": (1.0, 0.0), \"west\": (-1.0, 0.0)}\n\n\ndef cmd_along(args) -> dict:\n \"\"\"Take a point every step meters along any linear feature (road, river, power line filtered by ref / name), optionally shifting all points to one side of the road.\n With --bbox, only points inside the box are kept (Overpass returns the whole length of any road that crosses the box).\n\n For tiles.py sheet (satellite thumbnails scanning roadside factories, shops, fields) and gsv.py / baidu_pano.py sheet --points (street view scanning storefronts).\n \"\"\"\n pre, sc = _scope(args)\n data = run(f\"[out:json][timeout:180];{pre}way{args.line}{sc};out geom;\", args.proxy, args.cache)\n ways = [e for e in data.get(\"elements\", []) if e[\"type\"] == \"way\" and e.get(\"geometry\")]\n side = _SIDES.get(args.side) if args.side else None\n pts, seen = {}, []\n for w in ways:\n g = [(n[\"lat\"], n[\"lon\"]) for n in w[\"geometry\"]]\n acc = 0.0\n for a, b in zip(g, g[1:]):\n seg = _dist(a, b)\n if seg \u003c= 0:\n continue\n kx = 111320 * math.cos(math.radians(a[0]))\n ux, uy = (b[1] - a[1]) * kx / seg, (b[0] - a[0]) * 110574 / seg # unit vector (east, north components)\n while acc \u003c= seg:\n t = acc / seg\n lat, lon = a[0] + (b[0] - a[0]) * t, a[1] + (b[1] - a[1]) * t\n if side and args.offset:\n nx, ny = -uy, ux # left normal\n if nx * side[0] + ny * side[1] \u003c 0:\n nx, ny = uy, -ux\n lat += ny * args.offset / 110574\n lon += nx * args.offset / kx\n ll = (round(lat, 6), round(lon, 6))\n inside = not args.bbox or (args.bbox[0] \u003c= ll[0] \u003c= args.bbox[2] and args.bbox[1] \u003c= ll[1] \u003c= args.bbox[3])\n if inside and not any(_dist(ll, s) \u003c args.step * 0.5 for s in seen[-200:]):\n seen.append(ll)\n name = (w.get(\"tags\") or {}).get(\"ref\") or (w.get(\"tags\") or {}).get(\"name\") or f\"w{w['id']}\"\n pts[f\"{len(pts):04d} {name}\"] = list(ll)\n acc += args.step\n acc -= seg\n print(f\"{len(ways)} ways, one point every {args.step:.0f} m, {len(pts)} points total\" + (f\", offset {args.offset:.0f} m to the {args.side} side\" if side else \"\"))\n if pts:\n lats, lons = [v[0] for v in pts.values()], [v[1] for v in pts.values()]\n print(f\" extent bbox: {min(lats):.4f},{min(lons):.4f},{max(lats):.4f},{max(lons):.4f}\")\n return pts\n\n\ndef _ring_area(coords: list[tuple[float, float]]) -> float:\n if len(coords) \u003c 3:\n return 0.0\n lat0 = sum(c[0] for c in coords) / len(coords)\n kx, ky = 111320 * math.cos(math.radians(lat0)), 110574\n xy = [(c[1] * kx, c[0] * ky) for c in coords]\n return abs(sum(x1 * y2 - x2 * y1 for (x1, y1), (x2, y2) in zip(xy, xy[1:] + xy[:1]))) / 2\n\n\ndef cmd_buildings(args) -> dict:\n \"\"\"Find buildings with large footprints (factories, warehouses, farm sheds, mine facilities): filter by footprint area and mark how many other buildings are around.\n\n In many countries OSM's rural building outlines are traced from satellite imagery, so they have an area even without a name; in China, county and township buildings are often missing, so this can't be grounds for exclusion.\n \"Sparse surroundings\" is used only for ranking (--sort sparse), not for filtering: a factory site itself is often split into a pile of small sheds.\n \"\"\"\n pre, sc = _scope(args)\n flt = args.filter or '[\"building\"]'\n data = run(f\"[out:json][timeout:240];{pre}way{flt}{sc};out geom;\", args.proxy, args.cache)\n rows = []\n for el in data.get(\"elements\", []):\n g = [(n[\"lat\"], n[\"lon\"]) for n in el.get(\"geometry\") or []]\n if len(g) \u003c 4:\n continue\n area = _ring_area(g[:-1] if g[0] == g[-1] else g)\n c = (sum(p[0] for p in g) / len(g), sum(p[1] for p in g) / len(g))\n rows.append({\"id\": el[\"id\"], \"c\": c, \"area\": area, \"tags\": el.get(\"tags\") or {}})\n big = [r for r in rows if r[\"area\"] >= args.min_area]\n cells: dict[tuple[int, int], list] = {}\n cs = args.within / 111000\n for r in rows:\n cells.setdefault((int(r[\"c\"][0] / cs), int(r[\"c\"][1] / cs)), []).append(r)\n for r in big:\n ci, cj = int(r[\"c\"][0] / cs), int(r[\"c\"][1] / cs)\n r[\"neighbors\"] = sum(1 for di in (-1, 0, 1) for dj in (-1, 0, 1) for o in cells.get((ci + di, cj + dj), [])\n if o is not r and _dist(r[\"c\"], o[\"c\"]) \u003c= args.within)\n big.sort(key=(lambda r: (r[\"neighbors\"], -r[\"area\"])) if args.sort == \"sparse\" else (lambda r: -r[\"area\"]))\n print(f\"{len(rows)} buildings, {len(big)} with footprint ≥{args.min_area:.0f} m² (sorted by {'sparse surroundings' if args.sort == 'sparse' else 'area'}; nb=number of other buildings within {args.within:.0f} m)\")\n pts = {}\n for i, r in enumerate(big, 1):\n t = r[\"tags\"]\n kind = t.get(\"building\", \"\")\n label = f\"{i:03d} {int(r['area'])}m² nb{r['neighbors']} {kind if kind != 'yes' else ''} {t.get('name', '')}\"\n pts[\" \".join(label.split())] = [round(r[\"c\"][0], 6), round(r[\"c\"][1], 6)]\n return pts\n\n\ndef cmd_coverage(args) -> dict:\n \"\"\"How many features of a kind each candidate admin area has. Run it before enumerating candidates with OSM: districts with clearly low counts can't be excluded on OSM results and must be scanned separately with a satellite imagery grid.\"\"\"\n rows = []\n for name in [a for a in args.areas.split(\",\") if a]:\n for loose in (False, True): # bilingual names without a pack name key: the second round falls back to the name prefix\n ql = f'[out:json][timeout:120];{_area_sel(name, \"a\", loose)}.a out count;nwr{args.filter}(area.a);out count;'\n els = run(ql, args.proxy, args.cache).get(\"elements\") or []\n counts = [int((e.get(\"tags\") or {}).get(\"total\", 0)) for e in els]\n if counts and counts[0] > 0:\n break\n rows.append((name, counts[1] if len(counts) > 1 and counts[0] > 0 else -1))\n top = max((n for _, n in rows), default=0)\n print(f\"Count of {args.filter} in OSM (compare only admin areas of the same level; a low count may be truly low, or just unmapped):\")\n for name, n in rows:\n if n \u003c 0:\n flag = \" ← no admin area with this name in OSM; try another spelling (with or without the \\\"区/县/市\\\" suffix)\"\n elif n \u003c max(5, top * 0.25):\n flag = \" ← low: OSM results can't exclude this area; scan it with tiles.py sheet --grid instead\"\n else:\n flag = \"\"\n print(f\" {name}: {'?' if n \u003c 0 else n}{flag}\")\n return {}\n\n\ndef _rank_rows(rows: list[dict], flt: str, args) -> list[dict]:\n \"\"\"Rank candidates by \"distance to the nearest feature of a kind (town, service area, station ...)\", nearest first. Ranks only, never deletes.\"\"\"\n pre, sc = _scope(args)\n if args.bbox: # towns just outside the bbox edge count too\n s, w, n, e = args.bbox\n sc = f\"({s - 0.2},{w - 0.2},{n + 0.2},{e + 0.2})\"\n data = run(f\"[out:json][timeout:180];{pre}nwr{flt}{sc};out center tags;\", args.proxy, args.cache)\n refs = []\n for el in data.get(\"elements\", []):\n ll = _center(el)\n if ll and ll[0] is not None:\n refs.append(((el.get(\"tags\") or {}).get(\"name\", \"?\"), ll))\n if not refs:\n print(f\"--rank-near {flt} found no features; not ranking\")\n return rows\n for r in rows:\n name, ll = min(refs, key=lambda t: _dist(r[\"ll\"], t[1]))\n r[\"near_m\"], r[\"near_name\"] = _dist(r[\"ll\"], ll), name\n rows.sort(key=lambda r: r[\"near_m\"])\n print(f\" ranked by distance to the nearest {flt} ({len(refs)} found), nearest first\")\n return rows\n\n\ndef _match(el: dict, flt: str) -> bool:\n \"\"\"Roughly check whether a feature matches A's tag filter (looks only at key=value conditions).\"\"\"\n tags = el.get(\"tags\") or {}\n for k, v in re.findall(r'\\[\"([^\"]+)\"=\"([^\"]+)\"\\]', flt):\n if tags.get(k) != v:\n return False\n for k in re.findall(r'\\[\"([^\"=\\]]+)\"\\]', flt):\n if k not in tags:\n return False\n return True\n\n\ndef cmd_geom(args) -> dict:\n \"\"\"Any filter → GeoJSON (keeps geometry for lines, polygons and points), for custom analysis or overlays.\"\"\"\n pre, sc = _scope(args)\n data = run(f\"[out:json][timeout:180];{pre}nwr{args.filter}{sc};out geom tags;\", args.proxy, args.cache)\n feats = []\n for el in data.get(\"elements\", []):\n tags = el.get(\"tags\") or {}\n if el[\"type\"] == \"node\":\n geom = {\"type\": \"Point\", \"coordinates\": [el[\"lon\"], el[\"lat\"]]}\n elif el.get(\"geometry\"):\n coords = [[g[\"lon\"], g[\"lat\"]] for g in el[\"geometry\"]]\n closed = len(coords) > 3 and coords[0] == coords[-1]\n geom = {\"type\": \"Polygon\", \"coordinates\": [coords]} if closed and (\"building\" in tags or \"area\" in tags) \\\n else {\"type\": \"LineString\", \"coordinates\": coords}\n else:\n continue\n feats.append({\"type\": \"Feature\", \"id\": f\"{el['type']}/{el['id']}\", \"properties\": tags, \"geometry\": geom})\n gj = {\"type\": \"FeatureCollection\", \"features\": feats}\n out = args.out or Path(\"osm_geom.geojson\")\n out.write_text(json.dumps(gj, ensure_ascii=False), encoding=\"utf-8\")\n if not feats and args.area and not getattr(args, \"area_loose\", False) and _area_count(args.area, args) == 0:\n if _area_count(args.area, args, loose=True) > 0:\n print(f\"\\\"{args.area}\\\" has a bilingual name in OSM and no matching name key; retrying by name prefix\", file=sys.stderr)\n args.area_loose = True\n return cmd_geom(args)\n print(f\"Note: no area named \\\"{args.area}\\\" found in OSM; 0 results doesn't mean there is nothing here; try another spelling or use --bbox\", file=sys.stderr)\n print(f\"{len(feats)} features -> {out}\")\n args.out = None\n return {}\n\n\ndef _band(s: str) -> tuple[float, float]:\n a, b = s.split(\":\")\n return float(a), float(b)\n\n\ndef cmd_street_scan(args) -> dict:\n \"\"\"Street view geometry template: camera standing at an intersection (or on a road), looking down a street, with a \"building / no building\" pattern on each side → candidate points across the whole town.\"\"\"\n if not args.bbox:\n sys.exit(\"street-scan only supports --bbox (the extent of one town, sides no longer than ~15 km)\")\n s, w, n, e = args.bbox\n if (n - s) > 0.2 or (e - w) > 0.25:\n print(\"Tip: the area is large and the query may time out; running it town by town in several passes is more reliable\", file=sys.stderr)\n excl = \"footway|path|cycleway|steps|track|pedestrian|bridleway|corridor|platform|proposed|construction|elevator\"\n ql = (f'[out:json][timeout:280];way[\"highway\"][\"highway\"!~\"^({excl})$\"]({s},{w},{n},{e});out body geom;'\n f'way[\"building\"]({s},{w},{n},{e});out body geom;')\n data = run(ql, args.proxy, args.cache, timeout=280)\n lat0, lon0 = (s + n) / 2, (w + e) / 2\n kx, ky = 111320 * math.cos(math.radians(lat0)), 110540\n\n def P(g):\n return ((g[\"lon\"] - lon0) * kx, (g[\"lat\"] - lat0) * ky)\n\n types = set(args.types.split(\",\"))\n ways = [el for el in data.get(\"elements\", []) if el[\"type\"] == \"way\" and el.get(\"geometry\")]\n hw = [el for el in ways if \"highway\" in (el.get(\"tags\") or {})]\n bl = [el for el in ways if \"building\" in (el.get(\"tags\") or {})]\n node_ways: dict[int, set] = {}\n for wy in hw:\n for nd in wy[\"nodes\"]:\n node_ways.setdefault(nd, set()).add(wy[\"id\"])\n grid: dict[tuple[int, int], list] = {}\n area: dict[int, float] = {}\n for b in bl:\n pts = [P(g) for g in b[\"geometry\"]]\n area[b[\"id\"]] = abs(sum(x1 * y2 - x2 * y1 for (x1, y1), (x2, y2) in zip(pts, pts[1:]))) / 2 if len(pts) > 2 else 0\n for (x1, y1), (x2, y2) in zip(pts, pts[1:]):\n k = max(1, int(math.hypot(x2 - x1, y2 - y1) / 2))\n for i in range(k + 1):\n x, y = x1 + (x2 - x1) * i / k, y1 + (y2 - y1) * i / k\n grid.setdefault((int(x // 50), int(y // 50)), []).append((x, y, b[\"id\"]))\n b_lo, b_hi = _band(args.bearing)\n near_lo, near_hi = _band(args.band)\n clr_lo, clr_hi = _band(args.clear)\n ah_lo, ah_hi = _band(args.ahead)\n reach = int(max(ah_hi, near_hi, clr_hi) // 50) + 2\n\n def bearing_ok(brg):\n return (b_lo \u003c= brg \u003c= b_hi) if b_lo \u003c= b_hi else (brg >= b_lo or brg \u003c= b_hi)\n\n def along(pts, dist):\n acc = 0.0\n for (x1, y1), (x2, y2) in zip(pts, pts[1:]):\n L = math.hypot(x2 - x1, y2 - y1)\n if L > 0 and acc + L >= dist:\n f = (dist - acc) / L\n return (x1 + (x2 - x1) * f, y1 + (y2 - y1) * f)\n acc += L\n return None\n\n def side_ok(J, ux, uy, want, sign):\n if want == \"any\":\n return True\n nx, ny = uy * sign, -ux * sign # sign=+1 right side, -1 left side\n cx, cy = int(J[0] // 50), int(J[1] // 50)\n hits = set()\n blocked = False\n for gx in range(cx - reach, cx + reach + 1):\n for gy in range(cy - reach, cy + reach + 1):\n for x, y, bid in grid.get((gx, gy), []):\n t = (x - J[0]) * ux + (y - J[1]) * uy\n lat_d = (x - J[0]) * nx + (y - J[1]) * ny\n if ah_lo \u003c= t \u003c= ah_hi:\n if near_lo \u003c= lat_d \u003c= near_hi and area.get(bid, 0) >= args.min_area:\n hits.add(bid)\n if clr_lo \u003c= lat_d \u003c= clr_hi:\n blocked = True\n return bool(hits) if want == \"building\" else not blocked\n\n cands = {}\n for wy in hw:\n if wy[\"tags\"][\"highway\"] not in types:\n continue\n g = [P(x) for x in wy[\"geometry\"]]\n starts = []\n for end in (0, -1):\n if args.anywhere or len(node_ways.get(wy[\"nodes\"][end], ())) >= 2:\n starts.append((g if end == 0 else g[::-1], \"s\" if end == 0 else \"e\", 0.0))\n if args.anywhere:\n total = sum(math.hypot(x2 - x1, y2 - y1) for (x1, y1), (x2, y2) in zip(g, g[1:]))\n d = args.every\n while d \u003c total - args.look:\n starts.append((g, f\"m{int(d)}\", d))\n d += args.every\n for pts, tag, off in starts:\n J = along(pts, off) if off else pts[0]\n p_far, p_mid = along(pts, off + args.look), along(pts, off + args.look / 2)\n if not J or not p_far or not p_mid:\n continue\n ux, uy = p_far[0] - J[0], p_far[1] - J[1]\n L = math.hypot(ux, uy)\n ux, uy = ux / L, uy / L\n brg = math.degrees(math.atan2(ux, uy)) % 360\n if not bearing_ok(brg):\n continue\n b_mid = math.degrees(math.atan2(p_mid[0] - J[0], p_mid[1] - J[1])) % 360\n if abs((b_mid - brg + 180) % 360 - 180) > 15: # the first half must be straight enough\n continue\n if side_ok(J, ux, uy, args.right, +1) and side_ok(J, ux, uy, args.left, -1):\n name = (wy.get(\"tags\") or {}).get(\"name\", \"\")\n key = f\"{len(cands) + 1:03d} {name[:20]} heading {brg:.0f}° way{wy['id']}{tag}\".strip()\n cands[key] = [round(J[1] / ky + lat0, 6), round(J[0] / kx + lon0, 6)]\n print(f\"{len(hw)} roads, {len(bl)} buildings → {len(cands)} candidates (point = camera position at the intersection / on the road)\")\n return cands\n\n\n\ndef _neg_coords(argv: list[str]) -> list[str]:\n \"\"\"argparse treats negative coordinates like -1.45,-48.5 as option names; a leading space makes them plain values (float ignores the space). Every southern- or western-hemisphere case needs this.\"\"\"\n return [\" \" + a if re.match(r\"^-\\d[\\d.]*(,-?[\\d.]+)+$\", a) else a for a in argv]\n\n\ndef main() -> None:\n ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)\n ap.add_argument(\"--proxy\", default=os.environ.get(\"GEO_PROXY\"), help=PROXY_HELP)\n ap.add_argument(\"--cache\", type=Path, default=Path(\".geo-cache/osm\"))\n sub = ap.add_subparsers(dest=\"cmd\", required=True)\n\n def scope(sp):\n sp.add_argument(\"--proxy\", default=argparse.SUPPRESS, help=\"can also go after the subcommand\")\n sp.add_argument(\"--cache\", type=Path, default=argparse.SUPPRESS)\n sp.add_argument(\"--bbox\", type=lambda s: tuple(map(float, s.split(\",\"))), help=\"south,west,north,east\")\n sp.add_argument(\"--area\", help='OSM admin area name, e.g. \"Bayern\", \"江苏省\" (Jiangsu Province), \"深圳市\" (Shenzhen)')\n sp.add_argument(\"--out\", type=Path, help=\"write {name:[lat,lon]}\")\n sp.add_argument(\"--limit\", type=int, default=30, help=\"max rows printed to the terminal\")\n\n f = sub.add_parser(\"find\")\n f.add_argument(\"filter\", help='Overpass tag filter, e.g. \\'[\"amenity\"=\"fuel\"]\\'')\n scope(f)\n\n n = sub.add_parser(\"near\")\n n.add_argument(\"--a\", required=True, help=\"the main feature to find\")\n n.add_argument(\"--b\", required=True, help=\"feature that must be near the main feature\")\n n.add_argument(\"--within\", type=float, default=200, help=\"max distance from A to B (meters)\")\n n.add_argument(\"--c\", help=\"optional: a third required feature\")\n n.add_argument(\"--within-c\", type=float, default=200)\n n.add_argument(\"--report\", type=Path, help=\"write the actual B/C distances and key tags for each A (JSON), for ranking\")\n n.add_argument(\"--rank-near\", help='rank by distance to the nearest feature of a kind, e.g. \\'[\"place\"~\"^(city|town)$\"]\\' (ranks only, never deletes)')\n scope(n)\n\n c = sub.add_parser(\"crossings\")\n c.add_argument(\"--line\", required=True, help='linear feature tags, e.g. \\'[\"waterway\"=\"river\"][\"name\"=\"长江\"]\\' (Yangtze), \\'[\"railway\"=\"rail\"]\\'')\n c.add_argument(\"--kind\", default=\"bridge,dam,ferry\", help=\"bridge,dam,ferry,level_crossing, comma-separated\")\n c.add_argument(\"--buffer\", type=float, default=40, help=\"how many meters from the linear feature still counts as on the line\")\n scope(c)\n\n ro = sub.add_parser(\"route\")\n ro.add_argument(\"--kind\", default=\"bus\", help=\"bus / trolleybus / tram / subway / train / light_rail / ferry\")\n ro.add_argument(\"--ref\", help=\"route number, e.g. 79\")\n ro.add_argument(\"--name\", help=\"regex fragment of the route name\")\n ro.add_argument(\"--step\", type=float, default=200, help=\"take a point every this many meters along the route\")\n scope(ro)\n\n r = sub.add_parser(\"raw\")\n r.add_argument(\"file\", type=Path, help=\"Overpass QL file\")\n scope(r)\n\n it = sub.add_parser(\"intersect\")\n it.add_argument(\"--a\", required=True, help='first kind of line, e.g. \\'[\"railway\"=\"rail\"][\"electrified\"=\"contact_line\"]\\'')\n it.add_argument(\"--b\", required=True, help='second kind of line, e.g. \\'[\"power\"=\"line\"]\\'')\n it.add_argument(\"--bend-min\", type=float, default=0,\n help=\"mark crossing points where B bends ≥ this many degrees beyond the crossing (angle tower / river bend) and rank them first; 0 = ignore bends. By default only marks, never deletes\")\n it.add_argument(\"--bend-within\", default=\"100:1000\", help=\"distance range of the bend from the crossing, m\")\n it.add_argument(\"--bend-filter\", action=\"store_true\",\n help=\"actually delete crossing points without a bend (deleted ones go to \u003cout>_dropped.json). Use only when the bend is a fact you confirmed with your own eyes in the photo\")\n it.add_argument(\"--rank-near\", help='rank by distance to the nearest feature of a kind, e.g. \\'[\"place\"~\"^(city|town)$\"]\\', \\'[\"highway\"=\"services\"]\\'')\n it.add_argument(\"--ring\", help=\"lat,lon:min_m:max_m — keep only crossing points in this distance ring around a point (e.g. a distance range computed from a landmark's pixel size)\")\n it.add_argument(\"--cluster\", type=float, default=100, help=\"crossing points within this many meters merge into one place\")\n scope(it)\n\n cv = sub.add_parser(\"coverage\", help=\"how many features of a kind each candidate admin area has: check OSM coverage before enumerating; blank areas can't be grounds for exclusion\")\n cv.add_argument(\"--areas\", required=True, help='comma-separated OSM admin area names, e.g. \"\u003cname>区,\u003cname>县\"')\n cv.add_argument(\"--filter\", required=True, help='feature filter, e.g. \\'[\"leisure\"~\"^(pitch|track)$\"]\\', \\'[\"building\"]\\'')\n cv.add_argument(\"--proxy\", default=argparse.SUPPRESS)\n cv.add_argument(\"--cache\", type=Path, default=argparse.SUPPRESS)\n\n al = sub.add_parser(\"along\", help=\"take points at a step along a road / river / power line (optionally offset to one side), for satellite thumbnails or street view scans of the roadside\")\n al.add_argument(\"--line\", required=True, help='linear feature filter, e.g. \\'[\"highway\"][\"ref\"=\"\u003croad number>\"]\\', \\'[\"highway\"][\"name\"~\"\u003croad name fragment>\"]\\'')\n al.add_argument(\"--step\", type=float, default=150, help=\"take a point every this many meters; ≤150 for finding storefronts in street view, 300–500 is fine for satellite thumbnails\")\n al.add_argument(\"--side\", choices=list(_SIDES), help=\"which side of the road to offset to (by compass direction)\")\n al.add_argument(\"--offset\", type=float, default=0, help=\"offset in meters; when looking at factories on one side of the road, use the building-to-road distance\")\n scope(al)\n\n bu = sub.add_parser(\"buildings\", help=\"find large buildings by footprint area (factories, warehouses, farm sheds), marking how sparse the surroundings are\")\n bu.add_argument(\"--min-area\", type=float, default=1000, help=\"minimum footprint m²\")\n bu.add_argument(\"--within\", type=float, default=200, help=\"radius for counting surrounding buildings, m\")\n bu.add_argument(\"--sort\", choices=[\"area\", \"sparse\"], default=\"area\", help=\"sparse: fewer surrounding buildings first (isolated rural factories); ranks only, never deletes\")\n bu.add_argument(\"--filter\", help='building filter, default \\'[\"building\"]\\', e.g. \\'[\"building\"~\"industrial|warehouse|farm_auxiliary\"]\\'')\n scope(bu)\n\n gm = sub.add_parser(\"geom\")\n gm.add_argument(\"filter\", help='tag filter, e.g. \\'[\"building\"]\\', \\'[\"highway\"]\\'')\n scope(gm)\n\n ss = sub.add_parser(\"street-scan\")\n ss.add_argument(\"--bearing\", required=True, help=\"bearing range of the street the camera looks down, e.g. 320:80 (may cross north)\")\n ss.add_argument(\"--right\", choices=[\"building\", \"empty\", \"any\"], default=\"any\", help=\"right side of the street\")\n ss.add_argument(\"--left\", choices=[\"building\", \"empty\", \"any\"], default=\"any\", help=\"left side of the street\")\n ss.add_argument(\"--band\", default=\"4:22\", help=\"'building' test band: lateral distance from the street centerline, m\")\n ss.add_argument(\"--clear\", default=\"3:10\", help=\"'empty' test band: no building may be in this lateral range, m\")\n ss.add_argument(\"--ahead\", default=\"0:40\", help=\"how far ahead along the street to look, m\")\n ss.add_argument(\"--min-area\", type=float, default=150, help=\"minimum footprint m² required for 'building'\")\n ss.add_argument(\"--look\", type=float, default=60, help=\"how many meters ahead are used to compute the street bearing\")\n ss.add_argument(\"--types\", default=\"residential,unclassified,living_street,tertiary,secondary,service\")\n ss.add_argument(\"--anywhere\", action=\"store_true\", help=\"the camera isn't necessarily at an intersection: try every --every meters along the street\")\n ss.add_argument(\"--every\", type=float, default=30)\n scope(ss)\n\n args = ap.parse_args(_neg_coords(sys.argv[1:]))\n if args.cmd == \"crossings\":\n pts = cmd_crossings(args)\n elif args.cmd == \"route\":\n pts = cmd_route(args)\n elif args.cmd == \"intersect\":\n pts = cmd_intersect(args)\n elif args.cmd == \"geom\":\n pts = cmd_geom(args)\n elif args.cmd == \"coverage\":\n cmd_coverage(args)\n return\n elif args.cmd == \"along\":\n pts = cmd_along(args)\n elif args.cmd == \"buildings\":\n pts = cmd_buildings(args)\n elif args.cmd == \"street-scan\":\n pts = cmd_street_scan(args)\n else:\n if args.cmd == \"raw\":\n ql = args.file.read_text(encoding=\"utf-8\")\n if args.bbox:\n ql = ql.replace(\"{{bbox}}\", \",\".join(map(str, args.bbox)))\n else:\n pre, sc = _scope(args)\n if args.cmd == \"find\":\n ql = f\"[out:json][timeout:180];{pre}nwr{args.filter}{sc};out center tags;\"\n else:\n ql = f\"[out:json][timeout:180];{pre}nwr{args.b}{sc}->.b;nwr{args.a}(around.b:{args.within:.0f}){sc}->.a;\"\n if args.c:\n ql += f\"nwr{args.c}{sc}->.c;nwr.a(around.c:{args.within_c:.0f})->.a;\"\n ql += \".a out center tags;\"\n pts = _points(run(ql, args.proxy, args.cache))\n print(f\"{len(pts)} results\")\n if not pts:\n print(\" 0 doesn't mean none: OSM may not have mapped this area at all (common for counties and townships in China), so it can't be grounds for exclusion; compare first with osm.py coverage, or use tiles.py sheet --grid instead\")\n if args.cmd == \"near\" and args.rank_near and pts:\n rows = _rank_rows([{\"label\": k, \"ll\": v, \"bend\": None} for k, v in pts.items()], args.rank_near, args)\n pts = {f\"{r['label']} {r['near_m'] / 1000:.1f}km from {r['near_name'][:6]}\" if r.get(\"near_m\") is not None\n else r[\"label\"]: r[\"ll\"] for r in rows}\n if args.cmd == \"near\" and args.report and pts:\n pre2, sc2 = _scope(args)\n rep = {}\n a_ids = f\"nwr{args.b}{sc2}->.b;nwr{args.a}(around.b:{args.within:.0f}){sc2}->.a;\"\n if args.c:\n a_ids += f\"nwr{args.c}{sc2}->.c;nwr.a(around.c:{args.within_c:.0f})->.a;\"\n q2 = (f\"[out:json][timeout:200];{pre2}{a_ids}.a out center tags;\"\n f\"nwr{args.b}(around.a:{args.within:.0f}){sc2};out geom tags;\")\n if args.c:\n q2 += f\"nwr{args.c}(around.a:{args.within_c:.0f}){sc2};out geom tags;\"\n d2 = run(q2, args.proxy, args.cache)\n others = [e for e in d2.get(\"elements\", []) if e.get(\"geometry\") or e[\"type\"] == \"node\"]\n for name, ll in pts.items():\n near_list = []\n for el in others:\n geom = el.get(\"geometry\") or ([{\"lat\": el[\"lat\"], \"lon\": el[\"lon\"]}] if el[\"type\"] == \"node\" else [])\n if not geom:\n continue\n dmin = min(_dist(ll, (g[\"lat\"], g[\"lon\"])) for g in geom)\n t = el.get(\"tags\") or {}\n keep = {k2: v for k2, v in t.items() if k2 in\n (\"name\", \"voltage\", \"electrified\", \"highspeed\", \"railway\", \"power\", \"waterway\", \"highway\", \"tracks\")}\n near_list.append({\"dist_m\": round(dmin), \"tags\": keep})\n near_list.sort(key=lambda x: x[\"dist_m\"])\n rep[name] = {\"ll\": ll, \"nearest\": near_list[:4]}\n args.report.write_text(json.dumps(rep, ensure_ascii=False, indent=1), encoding=\"utf-8\")\n print(f\"report -> {args.report}\")\n for k, (name, ll) in enumerate(pts.items()):\n if k >= args.limit:\n print(\" …\")\n break\n print(f\" {name} {ll[0]},{ll[1]}\")\n if args.out:\n args.out.write_text(json.dumps(pts, ensure_ascii=False, indent=1), encoding=\"utf-8\")\n print(f\"-> {args.out}\")\n\n\nif __name__ == \"__main__\":\n # Chinese Windows outputs GBK by default: it crashes on m² or ñ, and the Chinese the agent reads is garbled\n sys.stdout.reconfigure(encoding=\"utf-8\")\n sys.stderr.reconfigure(encoding=\"utf-8\", errors=\"backslashreplace\")\n main()\n","content_type":"text/x-python; charset=utf-8","language":"python","size":50485,"content_sha256":"08b63a414ce656a828e44cade821cec0b529e7e4eecbc0d90828d8523316a7cd"},{"filename":"scripts/poi.py","content":"#!/usr/bin/env python3\n# /// script\n# requires-python = \">=3.10\"\n# dependencies = []\n# ///\n\"\"\"Place name, residential compound name, housing development name, shop name → coordinate candidates (no API key).\n\nReverse image search often returns \"the photo may be XX花园 (XX Garden)\" or \"XX大厦 (XX Tower)\"; a compound name or hotel name read on a web page must be placed at coordinates before it can be checked.\nThe same name often exists dozens of times across China; the script lists the same-name points in every region, and other clues in the photo pick among them.\n\nSources:\n so 360 Maps search: good coverage of residential compounds, housing developments, shops and organizations, with address; raw coordinates are GCJ-02, converted to WGS84\n osm OpenStreetMap Nominatim: named compounds, parks, roads; coordinates WGS84\n sug Baidu Maps search suggestions: only \"city + district + name\", no coordinates; use it to see which districts in China have this name\n\nExamples:\n poi.py \"\u003ccompound name>\" --city \u003ccity> # all same-name points in the city, outputs {name: [lat, lon]}\n poi.py \"\u003cdistrict> \u003croad name> 学校\" --city \u003cmunicipality or prefecture-level city> # --city only accepts prefecture-level cities; put the district in the keyword (学校 = school)\n poi.py \"\u003ccompound name>\" --city \u003ccity> --region cn # sources from the region pack (CN: 360 Maps + OSM + Baidu suggestions)\n poi.py \"\u003cstreet address or place name>\" --region mx # packs without a poi service: OSM restricted to that country\n poi.py \"\u003ccompound name>\" # no --region: sources of the pack whose keyword script matches (CN: Han characters)\n poi.py \"\u003chotel name>\" --city \u003ccity> --out pois.json && tiles.py sheet --points pois.json --zoom 18 --out pois_sheet.jpg\n\nBaidu Maps web place search requires a CAPTCHA, so the script doesn't use it; the Tencent and Amap APIs require a key, so it doesn't use them either.\n\"\"\"\nfrom __future__ import annotations\n\nimport argparse\nfrom _net import curl_args, PROXY_HELP\nimport json\nimport os\nimport re\nimport subprocess\nimport sys\nimport urllib.parse\nfrom pathlib import Path\n\nsys.path.insert(0, str(Path(__file__).parent))\nimport geo # noqa: E402\nimport regions # noqa: E402\n\nUA = \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0 Safari/537.36\"\n\n\ndef _curl(url: str, proxy: str | None = None, ua: str = UA, timeout: int = 25) -> str:\n cmd = [\"curl\", \"-q\", \"-sS\", \"-m\", str(timeout), \"-A\", ua, url]\n cmd += curl_args(proxy)\n # Pages are UTF-8. Without encoding=, Chinese Windows decodes as GBK, and stdout is None when decoding fails\n r = subprocess.run(cmd, capture_output=True, text=True, encoding=\"utf-8\", errors=\"replace\")\n if r.returncode != 0:\n print(f\"Request failed (curl exit code {r.returncode}): {r.stderr.strip()[:200]}\", file=sys.stderr)\n return r.stdout\n\n\ndef search_so(kw: str, city: str | None, n: int, proxy: str | None = None) -> tuple[list[dict], list[dict]]:\n q = {\"keyword\": kw, \"batch\": 1, \"number\": n, \"ext\": 1, \"sid\": 1000}\n if city:\n q[\"cityname\"] = city\n raw = _curl(\"https://restapi.map.so.com/newapi?\" + urllib.parse.urlencode(q), proxy=proxy)\n try:\n d = json.loads(raw)\n except json.JSONDecodeError:\n print(f\"360 Maps did not return JSON: {raw[:120]}\", file=sys.stderr)\n return [], []\n rows = []\n for p in d.get(\"poi\") or []:\n if not p.get(\"x\") or not p.get(\"y\"):\n continue\n lat, lon = geo.gcj2wgs(float(p[\"y\"]), float(p[\"x\"]))\n rows.append({\"src\": \"so\", \"name\": p.get(\"name\", \"\"), \"city\": p.get(\"city\", \"\"), \"area\": p.get(\"area\", \"\"),\n \"address\": p.get(\"address\", \"\"), \"type\": p.get(\"cat_new_name\") or p.get(\"type\", \"\"),\n \"wgs\": [round(lat, 6), round(lon, 6)]})\n cities = [{\"city\": c.get(\"name\"), \"province\": c.get(\"province\"), \"count\": c.get(\"resultnum\")}\n for c in d.get(\"citysuggestion\") or []]\n return rows, cities\n\n\ndef search_osm(kw: str, city: str | None, n: int, proxy: str | None, country: str) -> list[dict]:\n q = {\"q\": f\"{kw} {city}\" if city else kw, \"format\": \"jsonv2\", \"limit\": n, \"accept-language\": \"en\"}\n if country:\n q[\"countrycodes\"] = country\n raw = _curl(\"https://nominatim.openstreetmap.org/search?\" + urllib.parse.urlencode(q), proxy=proxy,\n ua=\"geo-sleuth/1.0 (photo geolocation research)\")\n try:\n d = json.loads(raw)\n except json.JSONDecodeError:\n print(f\"Nominatim did not return JSON (check service availability with doctor.py --network): {raw[:120]}\", file=sys.stderr)\n return []\n return [{\"src\": \"osm\", \"name\": p.get(\"name\") or p.get(\"display_name\", \"\").split(\",\")[0],\n \"city\": \"\", \"area\": \"\", \"address\": p.get(\"display_name\", \"\"), \"type\": f\"{p.get('category')}/{p.get('type')}\",\n \"wgs\": [round(float(p[\"lat\"]), 6), round(float(p[\"lon\"]), 6)]} for p in d]\n\n\ndef search_sug(kw: str, proxy: str | None = None) -> list[dict]:\n raw = _curl(\"https://map.baidu.com/su?\" + urllib.parse.urlencode({\"wd\": kw, \"cid\": 1, \"type\": 0, \"newmap\": 1, \"ie\": \"utf-8\"}), proxy=proxy)\n try:\n d = json.loads(raw)\n except json.JSONDecodeError:\n return []\n out = []\n for s in d.get(\"s\") or []:\n parts = s.split(\"$\")\n if len(parts) >= 4:\n out.append({\"src\": \"sug\", \"city\": parts[0], \"area\": parts[1], \"name\": parts[3]})\n return out\n\n\n\ndef _neg_coords(argv: list[str]) -> list[str]:\n \"\"\"argparse treats negative coordinates like -1.45,-48.5 as option names; a leading space makes them plain values (float ignores the space). Every southern- or western-hemisphere case needs this.\"\"\"\n return [\" \" + a if re.match(r\"^-\\d[\\d.]*(,-?[\\d.]+)+$\", a) else a for a in argv]\n\n\ndef main() -> None:\n ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)\n ap.add_argument(\"keyword\", help=\"place name, residential compound name, housing development name, or shop name\")\n ap.add_argument(\"--city\", help=\"city name, e.g. \u003cname>市 or \u003cname> (omit to search the whole country)\")\n ap.add_argument(\"--region\", help=\"region pack (code or name): sources and Nominatim country come from its services.poi\")\n ap.add_argument(\"--sources\", help=\"any of so,osm,sug, comma-separated (default: the region pack's; without --region, the sources of a pack whose services.poi.keyword_scripts match the keyword, else osm)\")\n ap.add_argument(\"--limit\", type=int, default=10, help=\"max results per source\")\n ap.add_argument(\"--country\", help=\"Nominatim country code (default: the region pack's; without one, no restriction)\")\n ap.add_argument(\"--proxy\", default=os.environ.get(\"GEO_PROXY\"), help=PROXY_HELP)\n ap.add_argument(\"--out\", type=Path, help=\"write {name: [lat, lon]} (WGS84) for tiles.py mark / sheet\")\n args = ap.parse_args(_neg_coords(sys.argv[1:]))\n svc = {}\n if args.region:\n pk = regions.resolve(args.region)\n if pk:\n svc = (pk.get(\"services\") or {}).get(\"poi\") or {\"sources\": \"osm\", \"country\": pk[\"code\"].lower()}\n elif re.fullmatch(r\"[A-Za-z]{2}\", args.region.strip()):\n # No pack for this country: still restrict Nominatim to it rather than searching the whole world\n svc = {\"sources\": \"osm\", \"country\": args.region.strip().lower()}\n print(f\"No region pack for '{args.region}': OSM only, restricted to countrycodes={svc['country']}\", file=sys.stderr)\n else:\n sys.exit(f\"No region pack for '{args.region}' (installed: {', '.join(sorted(regions.packs()))}): \"\n f\"pass its ISO 3166-1 alpha-2 code (e.g. --region in) so the search stays inside that country\")\n by_script = None if args.region else regions.poi_for_script(args.keyword) # sources only: no country restriction without --region\n args.sources = args.sources or svc.get(\"sources\") or (by_script or {}).get(\"sources\") or \"osm\"\n args.country = args.country if args.country is not None else svc.get(\"country\", \"\")\n\n src = set(args.sources.split(\",\"))\n rows: list[dict] = []\n if \"so\" in src:\n so_rows, cities = search_so(args.keyword, args.city, args.limit, args.proxy)\n if args.city and so_rows and not any(args.city in (r[\"city\"] + r[\"area\"] + r[\"address\"]) for r in so_rows):\n print(f\"Note: none of the 360 Maps results is in \\\"{args.city}\\\" — it doesn't recognize this city name and silently switched to another city. \"\n f\"Pass a prefecture-level city or municipality to --city (e.g. 重庆 (Chongqing)), and put the district name in the keyword (e.g. \\\"\u003cdistrict> \u003cname>\\\")\")\n rows += so_rows\n if cities and not args.city:\n print(\"360 Maps: cities in China with same-name results (result count)\")\n print(\" \" + \", \".join(f\"{c['city']}({c['count']})\" for c in cities[:30]))\n if \"osm\" in src:\n rows += search_osm(args.keyword, args.city, args.limit, args.proxy, args.country)\n if \"sug\" in src:\n sug = search_sug(args.keyword, args.proxy)\n if sug:\n print(\"Baidu suggestions (no coordinates; only shows which districts have this name):\")\n print(\" \" + \"; \".join(f\"{s['city']}{s['area']} {s['name']}\" for s in sug[:15]))\n\n # Deduplicate when two sources return the same place (within 150 m and one name contains the other)\n uniq: list[dict] = []\n for r in rows:\n if any(geo.distance(r[\"wgs\"], u[\"wgs\"]) \u003c 150 and (r[\"name\"] in u[\"name\"] or u[\"name\"] in r[\"name\"]) for u in uniq):\n continue\n uniq.append(r)\n print(f\"\\n{len(uniq)} candidates with coordinates (WGS84):\")\n pts = {}\n for i, r in enumerate(uniq, 1):\n label = f\"{i:02d} {r['name']} {r['area']}\".strip()\n pts[label] = r[\"wgs\"]\n print(f\" {label} {r['wgs'][0]},{r['wgs'][1]} [{r['src']}] {r['type']} {r['address'][:60]}\")\n if not uniq:\n print(\" None. Try another spelling (drop the \\\"小区/花园\\\" (compound/garden) suffix, add the district name), or use revimg.py --query to search the web for an address\")\n if args.out:\n args.out.write_text(json.dumps(pts, ensure_ascii=False, indent=1), encoding=\"utf-8\")\n print(f\"-> {args.out}\")\n if len(uniq) >= 2:\n # Several campuses of one school or several branches of a chain all show up here; the first one is not necessarily the right one\n print(f\"\\nAll {len(uniq)} places must be candidates; don't take only the first one: \"\n + (f\"`board.py add --from {args.out} --level area --parent \u003cparent>` puts them all on the candidate board\" if args.out\n else \"add --out pois.json, then `board.py add --from pois.json --level area` puts them all on the candidate board\"))\n\n\nif __name__ == \"__main__\":\n # Chinese Windows outputs GBK by default: it crashes on m² or ñ, and the Chinese the agent reads is garbled\n sys.stdout.reconfigure(encoding=\"utf-8\")\n sys.stderr.reconfigure(encoding=\"utf-8\", errors=\"backslashreplace\")\n main()\n","content_type":"text/x-python; charset=utf-8","language":"python","size":11138,"content_sha256":"e7e82bf9868b24b75c36594174dfab61b5678742359b3d8054d9774f2a6d4b68"},{"filename":"scripts/pose.py","content":"#!/usr/bin/env python3\n# /// script\n# requires-python = \">=3.10\"\n# dependencies = [\"pillow\", \"pillow-heif\", \"numpy\"]\n# ///\n\"\"\"Solve the camera position (camera resection): ≥4 points in the photo whose locations you can identify → camera lat/lon, height, heading, pitch, roll, field of view + error radius.\n\nSuited to window views, high-rise downward shots, views across a river and other photos where \"you can pick out several points on satellite imagery\". Compared with a two-sight-line intersection it also uses height and pitch information, and can work out the floor.\nLevel shots (standing on the ground photographing a distant tower, bridge, pier) also work: fix height and give the eye height.\n\n solve solve the camera position; the output includes a per-point check (leave_one_out): drop one point at a time, re-solve, and see how far off the other points' prediction of it is\n check score discrete candidate camera positions: use when several places all fit and you must pick one; each candidate solves only the heading, then is rescored with each point dropped in turn;\n only robust Δchi2 > 9 (dropping any one point can't rescue it) counts as inconsistent with the photo\n project with a known camera position, project a set of lat/lon points onto the photo (check whether riverbanks, roads, buildings line up);\n with --horizon \u003csea horizon row> it validates pitch: the target's depression angle is not the camera's pitch; a sea horizon at the frame center means pitch≈0\n\nspec.json format:\n{\n \"frame\": \"upright\",\n \"image_size\": [1476, 827],\n \"points\": [\n {\"name\": \"bridgehead\", \"px\": [212, 431], \"ll\": [lat, lon], \"h\": 470},\n {\"name\": \"spire\", \"px\": [1180, 120], \"ll\": [lat, lon], \"h\": 620}\n ],\n \"init\": {\"at\": [lat, lon], \"height\": 560, \"heading\": 190, \"pitch\": -8, \"hfov\": 65},\n \"fix\": [\"hfov\"]\n}\n- frame: \"upright\" for new specs measured on the photo as a viewer shows it (EXIF orientation applied), matching imgprep.py and ocr.py.\n \"stored\" uses the unrotated stored pixels. Specs without this key keep the legacy \"stored\" frame when solved again.\n The solved pose.json keeps the frame in _frame, and `project` draws in it; a pose.json without the key also uses \"stored\".\n- px and image_size: pixel coordinates and full photo size in the declared frame (top-left is 0,0).\n- h: height of the point, **on the same datum as the camera height**. Using elevation above sea level throughout is the most reliable: ground points via `terrain.py elev`, rooftop = ground elevation + building height.\n- init: rough camera position and heading; if unknown, give the center of the candidate area, and --restarts will start from multiple points within --search-radius.\n- fix: parameters that can be fixed (hfov, roll, height); if you know the focal length, fix hfov; more stable with few points.\n\nExamples:\n pose.py solve spec.json --photo photo.jpg --out pose.png --search-radius 500\n pose.py check spec.json --cands cands.json # cands.json: {\"candA\": [lat, lon], \"candB\": [lat, lon, eye_elevation]}\n pose.py project --pose pose.json --points river.json --photo photo.jpg --out check.png\n\"\"\"\nfrom __future__ import annotations\n\nimport argparse\nfrom _image import frame_of, open_photo\nimport json\nimport math\nimport re\nimport sys\nfrom pathlib import Path\n\nimport numpy as np\nfrom PIL import Image, ImageDraw\n\nNAMES = [\"east_m\", \"north_m\", \"height\", \"heading\", \"pitch\", \"roll\", \"focal_px\"]\n\n\ndef _frame(lat0: float):\n return 111320.0 * math.cos(math.radians(lat0)), 110540.0\n\n\ndef project(params: np.ndarray, pts_enu: np.ndarray, W: int, H: int) -> tuple[np.ndarray, np.ndarray]:\n e, n, u, yaw, pitch, roll, f = params\n ps, ts, rs = map(math.radians, (yaw, pitch, roll))\n fwd = np.array([math.sin(ps) * math.cos(ts), math.cos(ps) * math.cos(ts), math.sin(ts)])\n r0 = np.array([math.cos(ps), -math.sin(ps), 0.0])\n u0 = np.cross(r0, fwd)\n right = r0 * math.cos(rs) + u0 * math.sin(rs)\n up = -r0 * math.sin(rs) + u0 * math.cos(rs)\n v = pts_enu - np.array([e, n, u])\n z = v @ fwd\n x, y = v @ right, v @ up\n zz = np.where(z > 1e-3, z, np.nan)\n return np.stack([W / 2 + f * x / zz, H / 2 - f * y / zz], axis=1), z\n\n\ndef residuals(p, pts, obs, W, H):\n uv, z = project(p, pts, W, H)\n r = (uv - obs).ravel()\n return np.where(np.isnan(r), 5000.0, r)\n\n\ndef lm(p0, free, pts, obs, W, H, iters=200):\n p = p0.astype(float).copy()\n steps = np.array([0.5, 0.5, 0.5, 0.01, 0.01, 0.01, 1.0])\n lam = 1e-2\n r = residuals(p, pts, obs, W, H)\n cost = float(r @ r)\n J = None\n for _ in range(iters):\n J = np.zeros((r.size, len(free)))\n for j, k in enumerate(free):\n dp = p.copy()\n dp[k] += steps[k]\n J[:, j] = (residuals(dp, pts, obs, W, H) - r) / steps[k]\n A = J.T @ J\n g = J.T @ r\n improved = False\n for _ in range(10):\n try:\n delta = -np.linalg.solve(A + lam * np.diag(np.diag(A) + 1e-9), g)\n except np.linalg.LinAlgError:\n lam *= 10\n continue\n cand = p.copy()\n cand[free] += delta\n rc = residuals(cand, pts, obs, W, H)\n cc = float(rc @ rc)\n if cc \u003c cost:\n p, r, cost, lam, improved = cand, rc, cc, max(lam / 3, 1e-7), True\n break\n lam *= 4\n if not improved or float(np.abs(delta).max()) \u003c 1e-4:\n break\n return p, cost, J\n\n\ndef solve(spec: dict, search_radius: float, restarts: int, seed: int = 7,\n pt_sigma: float = 5.0) -> dict:\n W, H = spec[\"image_size\"]\n init = spec.get(\"init\", {})\n lat0, lon0 = init[\"at\"]\n kx, ky = _frame(lat0)\n pts = np.array([[(q[\"ll\"][1] - lon0) * kx, (q[\"ll\"][0] - lat0) * ky, q.get(\"h\", 0.0)] for q in spec[\"points\"]])\n obs = np.array([q[\"px\"] for q in spec[\"points\"]], dtype=float)\n hfov = init.get(\"hfov\", 65)\n f0 = (W / 2) / math.tan(math.radians(hfov / 2))\n fixed = set(spec.get(\"fix\", []))\n free = [i for i, name in enumerate(NAMES)\n if not ((name == \"focal_px\" and \"hfov\" in fixed) or (name == \"roll\" and \"roll\" in fixed)\n or (name == \"height\" and \"height\" in fixed))]\n n_obs, k = obs.size, len(free)\n if n_obs \u003c k:\n raise SystemExit(f\"Too few points: {len(obs)} points give only {n_obs} equations for {k} unknowns. Give at least {math.ceil(k / 2)} points, or fix hfov/roll with fix\")\n base = np.array([0.0, 0.0, init.get(\"height\", 30.0), init.get(\"heading\", 0.0), init.get(\"pitch\", 0.0), 0.0, f0])\n rng = np.random.default_rng(seed)\n best = None\n for t in range(max(1, restarts)):\n p0 = base.copy()\n if t:\n ang = rng.uniform(0, 2 * math.pi)\n rad = search_radius * math.sqrt(rng.uniform())\n p0[0], p0[1] = rad * math.sin(ang), rad * math.cos(ang)\n if \"height\" not in fixed:\n p0[2] = max(1.0, base[2] * rng.uniform(0.5, 1.5))\n p0[3] = (base[3] + rng.uniform(-90, 90)) % 360 if \"heading\" in init else rng.uniform(0, 360)\n p0[4] = base[4] + rng.uniform(-15, 15)\n p, cost, J = lm(p0, free, pts, obs, W, H)\n if best is None or cost \u003c best[1]:\n best = (p, cost, J)\n p, cost, J = best\n dof = n_obs - k\n rms = math.sqrt(cost / len(obs))\n out = {\"camera_ll\": [round(lat0 + p[1] / ky, 7), round(lon0 + p[0] / kx, 7)], \"height\": round(p[2], 1),\n \"heading\": round(p[3] % 360, 2), \"pitch\": round(p[4], 2), \"roll\": round(p[5], 2),\n \"hfov\": round(2 * math.degrees(math.atan((W / 2) / p[6])), 2), \"rms_px\": round(rms, 2),\n \"points\": len(obs), \"unknowns\": k, \"dof\": dof}\n uv, _ = project(p, pts, W, H)\n out[\"residuals_px\"] = {q.get(\"name\", str(i)): [round(float(a), 1), round(float(b), 1)]\n for i, (q, (a, b)) in enumerate(zip(spec[\"points\"], uv - obs))}\n if dof > 0 and J is not None:\n sigma2 = cost / dof\n try:\n cov = sigma2 * np.linalg.inv(J.T @ J)\n idx = {v: i for i, v in enumerate(free)}\n se = [math.sqrt(max(cov[idx[i], idx[i]], 0)) if i in idx else 0.0 for i in range(len(NAMES))]\n out[\"sigma\"] = {\"east_m\": round(se[0], 1), \"north_m\": round(se[1], 1), \"height_m\": round(se[2], 1),\n \"heading_deg\": round(se[3], 2)}\n out[\"radius_px_only_m\"] = round(3 * math.hypot(se[0], se[1]), 1) # 3σ from pixel noise only; doesn't cover the ground truth, don't use it directly\n except np.linalg.LinAlgError:\n out[\"sigma\"] = \"singular: the points are too concentrated (all on one line or in one direction); add points in different directions and at different distances\"\n else:\n out[\"sigma\"] = \"zero degrees of freedom, can't estimate the error; add more points\"\n out[\"_params\"] = [float(x) for x in p]\n out[\"_frame\"] = {\"lat0\": lat0, \"lon0\": lon0, \"image_size\": [W, H], \"frame\": frame_of(spec)}\n mc = mc_radius(p, free, pts, obs, W, H, pt_sigma, rms)\n out[\"radius_m\"] = mc[\"radius_m\"]\n out[\"radius_note\"] = (f\"95% radius, Monte Carlo {mc['trials']} trials: control-point coordinates perturbed by ±{pt_sigma:g} m\"\n f\" (--pt-sigma), pixels by ±{mc['px_sigma_used']:g} px. \"\n f\"Median offset {mc['p50_m']:g} m. The old pixel-noise-only measure is in radius_px_only_m; it doesn't cover the ground truth.\")\n out[\"pt_sigma_m\"] = pt_sigma\n out[\"leave_one_out\"] = leave_one_out(p, free, pts, obs, W, H, spec[\"points\"])\n return out\n\n\ndef mc_radius(p, free, pts, obs, W, H, pt_sigma: float, rms_px: float,\n trials: int = 60, seed: int = 11, px_sigma: float | None = None) -> dict:\n \"\"\"Control-point coordinate error doesn't propagate into the reprojection residuals: shift all points the same way, the camera shifts along, and the residuals don't change at all.\n So the inv(JtJ) covariance is blind in that direction: in tests, with 5 m coordinate noise the 3σ radius covers the ground truth only 67% of the time (should be 97%).\n Here the control points are perturbed by pt_sigma and the pixels by rms, re-solved from the converged solution, and the 95th percentile of the position offset is taken as the radius.\n The pixel path doesn't subtract the coordinate-error contribution: the slight double-counting happens to make up for the underestimate from \"re-solving from the converged solution can't reach large offsets\".\n Calibration in tests: the version that subtracts it on the pixel path covers 91%, without subtracting 95%; validated on three uncalibrated geometries, 286/300 = 95%.\"\"\"\n rng = np.random.default_rng(seed)\n if px_sigma is None:\n px_sigma = max(rms_px, 1.0)\n offs = []\n for _ in range(trials):\n pts_j = pts.copy()\n if pt_sigma > 0:\n pts_j[:, :2] += rng.normal(0, pt_sigma, (len(pts_j), 2))\n q, _, _ = lm(p.copy(), free, pts_j, obs + rng.normal(0, px_sigma, obs.shape), W, H, iters=60)\n offs.append(math.hypot(q[0] - p[0], q[1] - p[1]))\n return {\"radius_m\": round(float(np.percentile(offs, 95)), 1),\n \"p50_m\": round(float(np.percentile(offs, 50)), 1),\n \"q\": {k: round(float(np.percentile(offs, k)), 1) for k in (50, 90, 95, 99)},\n \"px_sigma_used\": round(px_sigma, 1), \"trials\": trials}\n\ndef leave_one_out(p, free, pts, obs, W, H, spec_pts) -> dict:\n \"\"\"Drop one point at a time and re-solve: where the other points \"predict\" this point, how many pixels off, and how far the camera position moved.\n For a misidentified point (wrong feature, wrong coordinates, inconsistent height datum), the remaining points fit very well once it is dropped, but it is itself far off.\"\"\"\n if 2 * (len(obs) - 1) \u003c len(free) + 1:\n return {\"note\": f\"with {len(obs)} points, dropping one leaves no more equations than unknowns, so the per-point check can't be done; add another point, or fix height/roll\"}\n rows, errs = {}, []\n for i in range(len(obs)):\n keep = [j for j in range(len(obs)) if j != i]\n q, cost, _ = lm(p, free, pts[keep], obs[keep], W, H)\n uv, _ = project(q, pts[i:i + 1], W, H)\n e = float(np.hypot(*(uv[0] - obs[i]))) if not np.isnan(uv[0]).any() else float(\"inf\")\n rest = math.sqrt(cost / len(keep))\n rows[spec_pts[i].get(\"name\", str(i))] = {\"pred_err_px\": round(e, 1), \"rest_rms_px\": round(rest, 1),\n \"shift_m\": round(math.hypot(q[0] - p[0], q[1] - p[1]), 1)}\n errs.append((e, rest, i))\n # Flag only one: dropping it gives the lowest rms for the remaining points, clearly below the other drops (0.4× the median), and its own prediction error is more than 3× the remaining rms.\n # Measured on synthetic cases (one point injected with a 40–80 m shift): downward shot, 7 points: false positives 0–2/20, detected 12–13/20; level shot, 6 points: false positives 2–5/20, detected 6–17/20 (unstable).\n dof_after = 2 * (len(obs) - 1) - len(free)\n e, rest, i = min(errs, key=lambda t: t[1])\n med = float(np.median([t[1] for t in errs]))\n flag = [spec_pts[i].get(\"name\", str(i))] if (dof_after >= 2 and e > 3 * max(rest, 2.0) and rest \u003c 0.4 * med) else []\n return {\"points\": rows, \"suspect\": flag,\n \"most_improved\": {\"name\": spec_pts[i].get(\"name\", str(i)), \"rest_rms_px\": round(rest, 1), \"median_rest_rms_px\": round(med, 1)},\n \"rule\": \"flag only one: dropping it gives the lowest remaining rms (\u003c 0.4× the median of the other drops) and its prediction error > 3× the remaining rms; \"\n \"unreliable for level shots and few points; nothing flagged doesn't mean there is no misidentified point; most_improved is listed whether or not it passes the threshold, check it first\"}\n\n\ndef _fit_at(e, n, h, pts, obs, W, H, free, f0, pitch0, p_start=None):\n \"\"\"Camera position fixed at (e, n, h); solve only heading/pitch/roll (and the focal length if not fixed). With p_start, start from it and don't try multiple starts.\"\"\"\n starts = [p_start] if p_start is not None else [np.array([e, n, h, hd, pt, 0.0, f0])\n for hd in range(0, 360, 30) for pt in (pitch0 - 10, pitch0, pitch0 + 10)]\n best = None\n for p0 in starts:\n p, cost, _ = lm(np.array(p0, dtype=float), free, pts, obs, W, H)\n if abs(p[4]) > 89 or abs(p[5]) > 60: # flipped solutions (camera facing backward, upside down) don't count\n continue\n if best is None or cost \u003c best[1]:\n best = (p, cost)\n return best if best else (np.array(starts[0], dtype=float), float(\"inf\"))\n\n\ndef check(spec: dict, cands: dict, px_sigma: float) -> dict:\n \"\"\"Score discrete candidate camera positions: each candidate frees only heading, pitch, roll (and the focal length if not fixed); compare reprojection errors.\n Suited to cases where \"two or three places fit geometrically and you must pick one\", e.g. the viewing platform marked on the map vs another stretch of shore on satellite imagery, or several buildings on the same alignment line.\n A single misidentified point can flip the ranking (synthetic cases: in downward shots, ground truth ranked first dropped from 15/15 to 1/15), so each point is also dropped in turn and rescored:\n robust_delta_chi2 takes the smallest over all drops; only candidates where that is also large count as inconsistent.\"\"\"\n W, H = spec[\"image_size\"]\n init = spec.get(\"init\", {})\n lat0, lon0 = init[\"at\"]\n kx, ky = _frame(lat0)\n pts = np.array([[(q[\"ll\"][1] - lon0) * kx, (q[\"ll\"][0] - lat0) * ky, q.get(\"h\", 0.0)] for q in spec[\"points\"]])\n obs = np.array([q[\"px\"] for q in spec[\"points\"]], dtype=float)\n names = [q.get(\"name\", str(i)) for i, q in enumerate(spec[\"points\"])]\n fixed = set(spec.get(\"fix\", []))\n f0 = (W / 2) / math.tan(math.radians(init.get(\"hfov\", 65) / 2))\n free = [3, 4, 5] + ([] if \"hfov\" in fixed else [6])\n subsets = [(\"all_points\", list(range(len(obs))))]\n if 2 * (len(obs) - 1) >= len(free) + 1:\n subsets += [(f\"drop {names[i]}\", [j for j in range(len(obs)) if j != i]) for i in range(len(obs))]\n res, costs = {}, {}\n for name, c in cands.items():\n ll, h = (c[:2], c[2]) if len(c) > 2 else (c, init.get(\"height\", 30.0))\n e, n = (ll[1] - lon0) * kx, (ll[0] - lat0) * ky\n p, cost = _fit_at(e, n, h, pts, obs, W, H, free, f0, init.get(\"pitch\", 0.0))\n costs[name] = {\"all_points\": cost}\n for sname, keep in subsets[1:]:\n costs[name][sname] = _fit_at(e, n, h, pts[keep], obs[keep], W, H, free, f0, 0, p_start=p)[1]\n uv, _ = project(p, pts, W, H)\n res[name] = {\"rms_px\": round(math.sqrt(cost / len(obs)), 1),\n \"heading\": round(p[3] % 360, 1), \"pitch\": round(p[4], 1), \"roll\": round(p[5], 1),\n \"hfov\": round(2 * math.degrees(math.atan((W / 2) / p[6])), 1),\n \"residuals_px\": {nm: [round(float(a), 1), round(float(b), 1)] for nm, (a, b) in zip(names, uv - obs)}}\n for sname, keep in subsets:\n best = min(costs[c][sname] for c in cands)\n sig2 = max(px_sigma ** 2, best / len(keep)) # when even the best candidate can't fit to px_sigma, loosen to its rms\n for c in cands:\n costs[c][sname] = (costs[c][sname] - best) / sig2\n for c in cands:\n d = costs[c]\n res[c][\"delta_chi2\"] = round(d[\"all_points\"], 1)\n worst_drop = min(d, key=d.get)\n res[c][\"robust_delta_chi2\"] = round(d[worst_drop], 1)\n res[c][\"robust_by\"] = worst_drop\n return dict(sorted(res.items(), key=lambda kv: (kv[1][\"robust_delta_chi2\"], kv[1][\"delta_chi2\"])))\n\n\ndef sanity(pose: dict, pts: list[dict], horizon_row: float | None = None) -> list[str]:\n \"\"\"Self-consistency check before reporting a camera position. Mistakes made before: putting the depression angle of some target in the frame into pitch as the camera's pitch,\n and reporting a camera position whose elevation doesn't match the depression angle in the frame (a 4 m height difference with a 7.6° depression angle, off by a factor of four).\"\"\"\n H = pose[\"_frame\"][\"image_size\"][1]\n e0, n0, h_cam, _yaw, pitch, _roll, f = pose[\"_params\"]\n lat0, lon0 = pose[\"_frame\"][\"lat0\"], pose[\"_frame\"][\"lon0\"]\n kx, ky = _frame(lat0)\n out = [(f\"Self-check pitch={pitch:.1f}°: the true horizon (sea horizon / distant flat horizon) should fall on row \"\n f\"{H / 2 - f * math.tan(math.radians(pitch)):.0f} of the frame ({H} rows, frame center {H // 2}). \"\n f\"If the sea horizon in the photo is not near this row, pitch was filled in wrong: the target's depression angle ≠ the camera's pitch.\")]\n if horizon_row is not None:\n pitch_h = math.degrees(math.atan2(H / 2 - float(horizon_row), f))\n d = abs(pitch_h - pitch)\n out.append(f\" Sea horizon in the photo is on row {float(horizon_row):.0f} → pitch should be {pitch_h:.1f}° (current {pitch:.1f}°, off by {d:.1f}°)\")\n if d > 2.0:\n out.append(f\" !! pitch contradicts the sea horizon by {d:.1f}° > 2°: calibrate pitch from the sea horizon first, then project/solve\")\n out.append(f\"Self-check camera height {h_cam:.1f} (same datum as each point's h); the \\\"expected row\\\" in the table below must match the actual row of that feature in the photo:\")\n for q in pts:\n e = (q[\"ll\"][1] - lon0) * kx - e0\n n = (q[\"ll\"][0] - lat0) * ky - n0\n dist = math.hypot(e, n)\n if dist \u003c 1.0:\n continue\n dh = h_cam - q.get(\"h\", 0.0)\n dep = math.degrees(math.atan2(dh, dist))\n row = H / 2 + f * math.tan(math.radians(dep + pitch))\n note = \"\"\n if \"px\" in q and len(q[\"px\"]) > 1:\n note = f\" actual row {q['px'][1]:.0f}, off by {abs(row - q['px'][1]):.0f} px\"\n out.append(f\" {q.get('name', '?'):\u003c14} horizontal {dist:6.0f} m height diff {dh:6.1f} m geometric depression {dep:5.1f}° expected row {row:6.0f}{note}\")\n return out\n\n\ndef draw(photo: Path, pose: dict, pts: list[dict], out: Path, observed: bool) -> None:\n W, H = pose[\"_frame\"][\"image_size\"]\n im = open_photo(photo, \"photo\", frame=frame_of(pose[\"_frame\"]), size=[W, H]).convert(\"RGB\")\n sx, sy = im.width / W, im.height / H\n lat0, lon0 = pose[\"_frame\"][\"lat0\"], pose[\"_frame\"][\"lon0\"]\n kx, ky = _frame(lat0)\n enu = np.array([[(q[\"ll\"][1] - lon0) * kx, (q[\"ll\"][0] - lat0) * ky, q.get(\"h\", 0.0)] for q in pts])\n uv, z = project(np.array(pose[\"_params\"]), enu, W, H)\n d = ImageDraw.Draw(im)\n for q, (u, v), zz in zip(pts, uv, z):\n if observed and \"px\" in q:\n ox, oy = q[\"px\"][0] * sx, q[\"px\"][1] * sy\n d.line([ox - 9, oy, ox + 9, oy], fill=\"lime\", width=3)\n d.line([ox, oy - 9, ox, oy + 9], fill=\"lime\", width=3)\n if zz > 0 and not math.isnan(u):\n cx, cy = u * sx, v * sy\n d.ellipse([cx - 7, cy - 7, cx + 7, cy + 7], outline=\"red\", width=3)\n d.text((cx + 9, cy - 8), q.get(\"name\", \"\"), fill=\"yellow\")\n im.save(out, quality=90)\n\n\n\ndef _neg_coords(argv: list[str]) -> list[str]:\n \"\"\"argparse treats negative coordinates like -1.45,-48.5 as option names; prefixing a space makes them plain values (float ignores the space). Needed for any photo in the southern or western hemisphere.\"\"\"\n return [\" \" + a if re.match(r\"^-\\d[\\d.]*(,-?[\\d.]+)+$\", a) else a for a in argv]\n\n\ndef main() -> None:\n ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)\n sub = ap.add_subparsers(dest=\"cmd\", required=True)\n s = sub.add_parser(\"solve\")\n s.add_argument(\"spec\", type=Path)\n s.add_argument(\"--search-radius\", type=float, default=300, help=\"radius of uncertainty of the initial position m\")\n s.add_argument(\"--restarts\", type=int, default=40)\n s.add_argument(\"--photo\", type=Path, help=\"draw observed points (green crosses) and reprojected points (red circles)\")\n s.add_argument(\"--out\", type=Path, help=\"overlay output\")\n s.add_argument(\"--save\", type=Path, default=Path(\"pose.json\"))\n s.add_argument(\"--pt-sigma\", type=float, default=5.0,\n help=\"coordinate error of the control points themselves m (clicking a building corner on satellite imagery: typically 5–15 m). The error radius uses it for Monte Carlo; set 0 to fall back to pixel noise only\")\n ck = sub.add_parser(\"check\", help=\"score discrete candidate camera positions: each candidate solves only the heading; compare reprojection errors\")\n ck.add_argument(\"spec\", type=Path)\n ck.add_argument(\"--cands\", type=Path, required=True, help='{\"cand_name\": [lat, lon] or [lat, lon, height]}; without height, init.height is used')\n ck.add_argument(\"--px-sigma\", type=float, default=5.0, help=\"pixel measurement error, used to convert to chi2\")\n ck.add_argument(\"--save\", type=Path, default=Path(\"pose_check.json\"))\n pr = sub.add_parser(\"project\")\n pr.add_argument(\"--pose\", type=Path, required=True)\n pr.add_argument(\"--points\", type=Path, required=True, help='[{\"name\":…,\"ll\":[lat,lon],\"h\":…}] or {name:[lat,lon,h]}')\n pr.add_argument(\"--photo\", type=Path, required=True)\n pr.add_argument(\"--out\", type=Path, required=True)\n pr.add_argument(\"--horizon\", type=float, help=\"row of the sea horizon / distant flat horizon in the photo, in the frame of the pose's points: used to calibrate and validate pitch\")\n args = ap.parse_args(_neg_coords(sys.argv[1:]))\n\n if args.cmd == \"solve\":\n spec = json.loads(args.spec.read_text(encoding=\"utf-8\"))\n pose = solve(spec, args.search_radius, args.restarts, pt_sigma=args.pt_sigma)\n args.save.write_text(json.dumps(pose, ensure_ascii=False, indent=1), encoding=\"utf-8\")\n show = {k: v for k, v in pose.items() if not k.startswith(\"_\")}\n print(json.dumps(show, ensure_ascii=False, indent=1))\n if pose[\"rms_px\"] > 15:\n print(\"Note: reprojection error is high; a point may be mismatched, the height datum inconsistent, or the initial value too far off (increase --search-radius / --restarts)\")\n loo = pose[\"leave_one_out\"]\n if loo.get(\"suspect\"):\n print(f\"Note: the per-point check flagged a suspected misidentified point {loo['suspect']}: go back and verify its feature and coordinates first, then decide whether to keep it\")\n elif loo.get(\"most_improved\"):\n m = loo[\"most_improved\"]\n print(f\"Per-point check: after dropping \\\"{m['name']}\\\" the remaining rms is {m['rest_rms_px']} px (median of other drops {m['median_rest_rms_px']} px); below the suspect threshold, but if the gap is large, still check it first\")\n for line in sanity(pose, spec[\"points\"]):\n print(line)\n if args.photo and args.out:\n draw(args.photo, pose, spec[\"points\"], args.out, observed=True)\n print(f\"overlay -> {args.out}\")\n elif args.cmd == \"check\":\n spec = json.loads(args.spec.read_text(encoding=\"utf-8\"))\n cands = json.loads(args.cands.read_text(encoding=\"utf-8\"))\n out = check(spec, cands, args.px_sigma)\n args.save.write_text(json.dumps(out, ensure_ascii=False, indent=1), encoding=\"utf-8\")\n for name, v in out.items():\n print(f\"{name:24s} rms {v['rms_px']:6.1f} px Δchi2 {v['delta_chi2']:8.1f} robust Δchi2 {v['robust_delta_chi2']:8.1f} ({v['robust_by']})\"\n f\" heading {v['heading']:6.1f} pitch {v['pitch']:5.1f}\")\n print(f\"-> {args.save} (only robust Δchi2 > 9 counts as inconsistent: dropping any one point can't rescue it; when two candidates differ only on one point, verify that point first)\")\n else:\n pose = json.loads(args.pose.read_text(encoding=\"utf-8\"))\n raw = json.loads(args.points.read_text(encoding=\"utf-8\"))\n pts = raw if isinstance(raw, list) else [{\"name\": k, \"ll\": v[:2], \"h\": (v[2] if len(v) > 2 else 0.0)} for k, v in raw.items()]\n draw(args.photo, pose, pts, args.out, observed=False)\n for line in sanity(pose, pts, args.horizon):\n print(line)\n print(args.out)\n\n\nif __name__ == \"__main__\":\n # Chinese-locale Windows outputs GBK by default: it crashes on m², ñ, and the Chinese the agent reads is garbled\n sys.stdout.reconfigure(encoding=\"utf-8\")\n sys.stderr.reconfigure(encoding=\"utf-8\", errors=\"backslashreplace\")\n main()\n","content_type":"text/x-python; charset=utf-8","language":"python","size":26506,"content_sha256":"70b2297bc1a80593c140caad7977c7cb30e83bd5a80e49a8601cb416a04af8fb"},{"filename":"scripts/refresh.py","content":"#!/usr/bin/env python3\n# /// script\n# requires-python = \">=3.10\"\n# dependencies = []\n# ///\n\"\"\"Maintainer tool: re-fetch the bundled lookup tables from their sources and rewrite them in the shared formats.\n\nGlobal tables go to data/; the China pack's tables go to regions/cn/data/ in the prefix-table and admin-table formats (regions/README.md).\nLookups never call this; they read the JSON only. Contributed packs ship their data with `_meta.source` and don't need a parser here.\n\n refresh.py [table|all] tables: calling_codes driving_side territories cn/admin cn/plates cn/area_codes\n refresh.py all --from-dir D parse already-downloaded source files instead of fetching (names in LOCAL_NAMES)\n\ncn/plates and cn/area_codes resolve place names against cn/admin, so refresh cn/admin first (all does).\n\"\"\"\nfrom __future__ import annotations\n\nimport argparse\nfrom _net import curl_args, PROXY_HELP\nimport html as H\nimport json\nimport os\nimport re\nimport subprocess\nimport sys\nfrom datetime import date\nfrom pathlib import Path\n\nHERE = Path(__file__).resolve().parent\nDATA = HERE.parent / \"data\"\nCN = HERE.parent / \"regions\" / \"cn\" / \"data\"\nUA = \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0 Safari/537.36\"\n\nSOURCES = {\n \"calling_codes\": [\"https://en.wikipedia.org/wiki/List_of_telephone_country_codes\"],\n \"driving_side\": [\"https://en.wikipedia.org/wiki/Left-_and_right-hand_traffic\"],\n \"territories\": [\"https://en.wikipedia.org/wiki/List_of_dependent_territories\"],\n \"cn/admin\": [\"https://raw.githubusercontent.com/modood/Administrative-divisions-of-China/master/dist/pca-code.json\"],\n \"cn/plates\": [\"https://zh.wikipedia.org/zh-cn/中华人民共和国民用机动车号牌\"],\n \"cn/area_codes\": [\"https://zh.wikipedia.org/zh-cn/中国大陆固定电话号码\"],\n}\nLOCAL_NAMES = {\"cn/plates\": \"plates_zh.html\", \"cn/area_codes\": \"areacodes2_zh.html\", \"calling_codes\": \"calling_en.html\",\n \"driving_side\": \"driving_en.html\", \"territories\": \"dependent_en.html\", \"cn/admin\": \"pca-code.json\"}\nLICENSE = {\"cn/admin\": \"WTFPL (modood/Administrative-divisions-of-China)\"}\nWIKI_LICENSE = \"Derived from Wikipedia, CC BY-SA 4.0\"\n\n# Letter splits inside municipalities: source is common knowledge of the former prefecture divisions; the Wikipedia page only goes down to “重庆市” (Chongqing). Marked unverified; verify before use.\nMUNICIPAL_LETTER_NOTES = {\n \"渝\": {\"A\": \"main urban districts\", \"B\": \"main urban districts\", \"C\": \"永川、江津、合川、璧山、铜梁、大足、荣昌、潼南 (former 永川 prefecture)\", \"D\": \"main urban districts (added later)\",\n \"F\": \"万州、开州、梁平、忠县、云阳、奉节、巫山、巫溪、城口 (former 万县 prefecture)\", \"G\": \"涪陵、南川、垫江、丰都、武隆 (former 涪陵 prefecture)\",\n \"H\": \"黔江、石柱、秀山、酉阳、彭水 (former 黔江 prefecture)\"},\n}\n\n# Wikipedia's “List of dependent territories” leaves out overseas territories integrated into the home country (French overseas departments, Spain's Canaries, US Hawaii…), but they are exactly the intersection that “IP country × hinted continent” is looking for, so they are added here.\n# Sources: Wikipedia Overseas France / Outermost regions of the EU / per-country articles; region/subregion follow the UN geoscheme.\nINTEGRAL_OVERSEAS = [\n {\"name\": \"French Guiana\", \"sovereign\": \"France\", \"region\": \"Americas\", \"subregion\": \"South America\", \"status\": \"Overseas department and region (integral part of France, EU)\"},\n {\"name\": \"Guadeloupe\", \"sovereign\": \"France\", \"region\": \"Americas\", \"subregion\": \"Caribbean\", \"status\": \"Overseas department and region\"},\n {\"name\": \"Martinique\", \"sovereign\": \"France\", \"region\": \"Americas\", \"subregion\": \"Caribbean\", \"status\": \"Overseas department and region\"},\n {\"name\": \"Réunion\", \"sovereign\": \"France\", \"region\": \"Africa\", \"subregion\": \"Eastern Africa\", \"status\": \"Overseas department and region\"},\n {\"name\": \"Mayotte\", \"sovereign\": \"France\", \"region\": \"Africa\", \"subregion\": \"Eastern Africa\", \"status\": \"Overseas department and region\"},\n {\"name\": \"Saint Martin\", \"sovereign\": \"France\", \"region\": \"Americas\", \"subregion\": \"Caribbean\", \"status\": \"Overseas collectivity\"},\n {\"name\": \"Saint Barthélemy\", \"sovereign\": \"France\", \"region\": \"Americas\", \"subregion\": \"Caribbean\", \"status\": \"Overseas collectivity\"},\n {\"name\": \"Saint Pierre and Miquelon\", \"sovereign\": \"France\", \"region\": \"Americas\", \"subregion\": \"Northern America\", \"status\": \"Overseas collectivity\"},\n {\"name\": \"French Polynesia\", \"sovereign\": \"France\", \"region\": \"Oceania\", \"subregion\": \"Polynesia\", \"status\": \"Overseas collectivity\"},\n {\"name\": \"New Caledonia\", \"sovereign\": \"France\", \"region\": \"Oceania\", \"subregion\": \"Melanesia\", \"status\": \"Sui generis collectivity\"},\n {\"name\": \"Wallis and Futuna\", \"sovereign\": \"France\", \"region\": \"Oceania\", \"subregion\": \"Polynesia\", \"status\": \"Overseas collectivity\"},\n {\"name\": \"Canary Islands\", \"sovereign\": \"Spain\", \"region\": \"Africa\", \"subregion\": \"Northern Africa (Atlantic)\", \"status\": \"Autonomous community (integral part of Spain)\"},\n {\"name\": \"Ceuta\", \"sovereign\": \"Spain\", \"region\": \"Africa\", \"subregion\": \"Northern Africa\", \"status\": \"Autonomous city (integral part of Spain)\"},\n {\"name\": \"Melilla\", \"sovereign\": \"Spain\", \"region\": \"Africa\", \"subregion\": \"Northern Africa\", \"status\": \"Autonomous city (integral part of Spain)\"},\n {\"name\": \"Azores\", \"sovereign\": \"Portugal\", \"region\": \"Europe\", \"subregion\": \"Southern Europe (Atlantic)\", \"status\": \"Autonomous region (integral part of Portugal)\"},\n {\"name\": \"Madeira\", \"sovereign\": \"Portugal\", \"region\": \"Europe\", \"subregion\": \"Southern Europe (Atlantic, off Africa)\", \"status\": \"Autonomous region (integral part of Portugal)\"},\n {\"name\": \"Hawaii\", \"sovereign\": \"United States\", \"region\": \"Oceania\", \"subregion\": \"Polynesia\", \"status\": \"State (integral part of the US)\"},\n {\"name\": \"Alaska\", \"sovereign\": \"United States\", \"region\": \"Americas\", \"subregion\": \"Northern America\", \"status\": \"State (integral part of the US)\"},\n {\"name\": \"Bonaire\", \"sovereign\": \"Netherlands\", \"region\": \"Americas\", \"subregion\": \"Caribbean\", \"status\": \"Special municipality (integral part of the Netherlands)\"},\n {\"name\": \"Sint Eustatius\", \"sovereign\": \"Netherlands\", \"region\": \"Americas\", \"subregion\": \"Caribbean\", \"status\": \"Special municipality\"},\n {\"name\": \"Saba\", \"sovereign\": \"Netherlands\", \"region\": \"Americas\", \"subregion\": \"Caribbean\", \"status\": \"Special municipality\"},\n {\"name\": \"Svalbard\", \"sovereign\": \"Norway\", \"region\": \"Europe\", \"subregion\": \"Northern Europe (Arctic)\", \"status\": \"Unincorporated area (integral part of Norway)\"},\n {\"name\": \"Easter Island\", \"sovereign\": \"Chile\", \"region\": \"Oceania\", \"subregion\": \"Polynesia\", \"status\": \"Special territory (integral part of Chile)\"},\n {\"name\": \"Galápagos Islands\", \"sovereign\": \"Ecuador\", \"region\": \"Americas\", \"subregion\": \"South America (Pacific)\", \"status\": \"Province (integral part of Ecuador)\"},\n {\"name\": \"Andaman and Nicobar Islands\", \"sovereign\": \"India\", \"region\": \"Asia\", \"subregion\": \"Southern Asia (Bay of Bengal)\", \"status\": \"Union territory (integral part of India)\"},\n {\"name\": \"Kaliningrad Oblast\", \"sovereign\": \"Russia\", \"region\": \"Europe\", \"subregion\": \"Eastern Europe (exclave on the Baltic)\", \"status\": \"Oblast (integral part of Russia)\"},\n {\"name\": \"Okinawa\", \"sovereign\": \"Japan\", \"region\": \"Asia\", \"subregion\": \"Eastern Asia\", \"status\": \"Prefecture (integral part of Japan)\"},\n]\n\n# ---------------------------------------------------------------- HTML table parsing (handles rowspan/colspan)\n\ndef _clean(c: str) -> str:\n c = re.sub(r\"\u003csup[^>]*>.*?\u003c/sup>\", \"\", c, flags=re.S)\n c = re.sub(r\"\u003cbr\\s*/?>\", \" | \", c)\n c = H.unescape(re.sub(r\"\u003c[^>]+>\", \"\", c))\n c = re.sub(r\"\\[[^\\]]*\\]\", \"\", c)\n return re.sub(r\"\\s+\", \" \", c).strip()\n\n\ndef _tables(html: str) -> list[list[list[str]]]:\n out = []\n for t in re.findall(r\"\u003ctable[^>]*>(.*?)\u003c/table>\", html, re.S):\n rows, pending = [], {}\n for tr in re.findall(r\"\u003ctr[^>]*>(.*?)\u003c/tr>\", t, re.S):\n cells = re.findall(r\"\u003ct([dh])([^>]*)>(.*?)\u003c/t[dh]>\", tr, re.S)\n row, col, k = [], 0, 0\n while k \u003c len(cells) or col in pending:\n if col in pending:\n text, left = pending[col]\n row.append(text)\n if left \u003c= 1:\n del pending[col]\n else:\n pending[col] = (text, left - 1)\n col += 1\n continue\n _, attrs, body = cells[k]\n k += 1\n text = _clean(body)\n rs = re.search(r'rowspan=\"?(\\d+)', attrs)\n cs = re.search(r'colspan=\"?(\\d+)', attrs)\n n = int(cs.group(1)) if cs else 1\n for _ in range(n):\n row.append(text)\n if rs and int(rs.group(1)) > 1:\n pending[col] = (text, int(rs.group(1)) - 1)\n col += 1\n rows.append(row)\n out.append(rows)\n return out\n\n\ndef _sections_h3(html: str) -> list[tuple[str, str]]:\n parts = re.split(r\"\u003ch3[^>]*>\", html)\n secs = []\n for part in parts[1:]:\n if \"\u003c/h3>\" not in part:\n continue\n title, body = part.split(\"\u003c/h3>\", 1)\n secs.append((_clean(title), body.split(\"\u003ch2\", 1)[0]))\n return secs\n\n\n# ---------------------------------------------------------------- per-table parsers\n\ndef parse_cn_plates(html: str) -> dict:\n out = {}\n for title, body in _sections_h3(html):\n m = re.match(r\"^(.+?)（(.)）$\", title)\n if not m:\n continue\n prov, abbr = m.group(1), m.group(2)\n letters = {}\n for li in re.findall(r\"\u003cli[^>]*>(.*?)\u003c/li>\", body, re.S):\n text = _clean(li)\n text = re.split(r\"[。；;]\", text)[0]\n text = re.sub(r\"^(参见：.*?)(?=[A-Z](?:[/、，,]|\\s))\", \"\", text)\n text = re.sub(r\"^(汽车|小型汽车|大型汽车)\\s*\", \"\", text)\n if \"摩托车\" in text or \"拖拉机\" in text:\n continue\n mm = re.match(r\"^([A-Z](?:\\s*[/、，,–\\-~～至]\\s*[A-Z])*)\\s*[：:]?\\s*(.+)$\", text)\n if not mm:\n continue\n place = mm.group(2).strip()\n spec = mm.group(1)\n Ls: list[str] = []\n for seg in re.split(r\"[/、，,]\", spec):\n seg = seg.strip()\n r = re.match(r\"^([A-Z])\\s*[–\\-~～至]\\s*([A-Z])$\", seg)\n if r:\n Ls += [chr(c) for c in range(ord(r.group(1)), ord(r.group(2)) + 1) if chr(c) not in \"IO\"]\n elif seg:\n Ls.append(seg)\n for L in Ls:\n letters[L] = place\n if letters:\n out[abbr] = {\"province\": prov, \"letters\": letters}\n for abbr, notes in MUNICIPAL_LETTER_NOTES.items():\n if abbr in out:\n out[abbr][\"letter_notes_unverified\"] = notes\n return out\n\n\ndef parse_cn_area_codes(html: str) -> dict:\n out = {}\n for t in _tables(html):\n if not t or not t[0] or t[0][0] != \"区号\":\n continue\n for row in t[1:]:\n if len(row) \u003c 3 or not re.match(r\"^\\d{2,4}$\", row[0]):\n continue\n code = \"0\" + row[0]\n entry = {\"admin1\": row[1], \"admin2\": [x.strip() for x in row[2].split(\"|\") if x.strip()],\n \"digits\": row[3] if len(row) > 3 else \"\", \"note\": row[4] if len(row) > 4 else \"\"}\n # \"/\" in the digits column marks a retired code; the note alone isn't enough (023's note tells the history of 811 being retired)\n if entry[\"digits\"] == \"/\":\n entry[\"deprecated\"] = True\n out[code] = entry\n return out\n\n\ndef parse_calling_codes(html: str) -> dict:\n out = {}\n for t in _tables(html):\n if len(t) \u003c 50 or not t[0] or t[0][0] != \"Serving\":\n continue\n for row in t[1:]:\n if len(row) \u003c 2 or not re.match(r\"^\\d\", row[1]):\n continue\n country = row[0]\n m = re.match(r\"^(\\d+)\\s*(?:\$([^)]*)\$)?\", row[1])\n if not m:\n continue\n code = m.group(1)\n subs = [s.strip() for s in (m.group(2) or \"\").split(\",\") if s.strip()]\n keys = [f\"{code}-{s}\" for s in subs] or [code]\n for k in keys:\n out.setdefault(k, []).append({\"country\": country, \"utc\": row[2] if len(row) > 2 else \"\"})\n return out\n\n\nDRIVING_SUPPLEMENT = { # regions without their own row in the Wikipedia table (source: each region's article, common knowledge)\n \"Hong Kong\": (\"left\", \"kept from British rule, opposite to the mainland\"), \"Macau\": (\"left\", \"kept from Portuguese rule, opposite to the mainland\"), \"Taiwan\": (\"right\", \"\"),\n \"French Guiana\": (\"right\", \"French overseas department\"), \"Guadeloupe\": (\"right\", \"French overseas department\"), \"Martinique\": (\"right\", \"French overseas department\"),\n \"Réunion\": (\"right\", \"French overseas department\"), \"Mayotte\": (\"right\", \"French overseas department\"), \"New Caledonia\": (\"right\", \"French territory\"), \"French Polynesia\": (\"right\", \"French territory\"),\n \"Puerto Rico\": (\"right\", \"US territory\"), \"Guam\": (\"right\", \"US territory\"), \"U.S. Virgin Islands\": (\"left\", \"US territory, a rare left-hand one\"),\n \"American Samoa\": (\"right\", \"US territory\"), \"Northern Mariana Islands\": (\"right\", \"US territory\"),\n \"Greenland\": (\"right\", \"Denmark\"), \"Faroe Islands\": (\"right\", \"Denmark\"), \"Aruba\": (\"right\", \"Netherlands\"), \"Curaçao\": (\"right\", \"Netherlands\"), \"Sint Maarten\": (\"right\", \"Netherlands\"),\n \"Gibraltar\": (\"right\", \"British territory, a rare right-hand one\"), \"Bermuda\": (\"left\", \"British territory\"), \"Cayman Islands\": (\"left\", \"British territory\"),\n \"British Virgin Islands\": (\"left\", \"British territory\"), \"Anguilla\": (\"left\", \"British territory\"), \"Montserrat\": (\"left\", \"British territory\"),\n \"Turks and Caicos Islands\": (\"left\", \"British territory\"), \"Falkland Islands\": (\"left\", \"British territory\"), \"Saint Helena\": (\"left\", \"British territory\"),\n \"Isle of Man\": (\"left\", \"British Crown Dependency\"), \"Jersey\": (\"left\", \"British Crown Dependency\"), \"Guernsey\": (\"left\", \"British Crown Dependency\"),\n \"Cook Islands\": (\"left\", \"New Zealand associated state\"), \"Niue\": (\"left\", \"New Zealand associated state\"), \"Tokelau\": (\"left\", \"New Zealand territory\"),\n \"Canary Islands\": (\"right\", \"Spain\"), \"Azores\": (\"right\", \"Portugal\"), \"Madeira\": (\"right\", \"Portugal\"), \"Svalbard\": (\"right\", \"Norway\"),\n}\n\n\ndef parse_driving_side(html: str) -> dict:\n \"\"\"The table's \"Country\" header spans two columns: sovereign state, then a sub-unit for states listed in parts\n (China → Mainland / Hong Kong / Macau). Keying every row by the first column let the last sub-row overwrite the\n country (China came out as LHT with Macau's note), so sub-rows are resolved explicitly.\"\"\"\n out: dict = {}\n groups: dict[str, list[tuple[str, dict]]] = {}\n for t in _tables(html):\n if len(t) \u003c 100 or not t[0] or t[0][0] != \"Country\":\n continue\n for row in t[1:]:\n if len(row) \u003c 2:\n continue\n idx = next((i for i, c in enumerate(row) if re.match(r\"^(LHT|RHT)\", c.upper())), None)\n if idx is None:\n continue\n e = {\"side\": \"left\" if row[idx].upper().startswith(\"LHT\") else \"right\",\n \"switched\": row[idx + 1] if len(row) > idx + 1 else \"\", \"note\": row[idx + 2] if len(row) > idx + 2 else \"\"}\n if idx >= 2 and row[1] != row[0]:\n groups.setdefault(re.sub(r\" and overseas territories$\", \"\", row[0]), []).append((row[1], e))\n else:\n out[row[0]] = e\n for parent, subs in groups.items():\n main = [e for sub, e in subs if sub == \"Mainland\" or sub.endswith(\" proper\") or sub.startswith(\"Contiguous\")]\n if main:\n out[parent] = main[0]\n elif len({e[\"side\"] for _, e in subs}) == 1:\n # Listed by province/state, all the same side (Canada): the country takes that side, provinces aren't countries\n out[parent] = dict(subs[0][1], note=\"; \".join(f\"{sub}: {e['note']}\" for sub, e in subs if e[\"note\"]))\n continue\n for sub, e in subs:\n if e in main:\n continue\n out[sub] = dict(e, part_of=parent, note=\"; \".join(x for x in (f\"part of {parent}\", e[\"note\"]) if x))\n for k, (side, note) in DRIVING_SUPPLEMENT.items():\n if k not in out:\n out[k] = {\"side\": side, \"switched\": \"\", \"note\": note, \"curated\": True}\n return out\n\n\ndef parse_territories(html: str) -> dict:\n items = []\n for t in _tables(html):\n if not t or not t[0] or t[0][0] != \"Name\" or \"Sovereign state\" not in t[0]:\n continue\n hi = {h: i for i, h in enumerate(t[0])}\n for row in t[1:]:\n if len(row) \u003c len(t[0]) - 1:\n continue\n items.append({\"name\": row[hi[\"Name\"]], \"sovereign\": row[hi[\"Sovereign state\"]], \"region\": row[hi[\"UN region\"]],\n \"subregion\": row[hi[\"UN subregion\"]], \"status\": row[hi.get(\"Legal status\", len(row) - 1)],\n \"population\": row[hi.get(\"Population (2016)\", 1)], \"area_km2\": row[hi.get(\"Area (km)\", 2)]})\n names = {it[\"name\"].lower() for it in items}\n for it in INTEGRAL_OVERSEAS:\n if it[\"name\"].lower() not in names:\n items.append(dict(it, population=\"\", area_km2=\"\", curated=True))\n by_sov: dict = {}\n for it in items:\n by_sov.setdefault(it[\"sovereign\"], []).append(it)\n return {\"by_sovereign\": by_sov, \"count\": len(items),\n \"note\": \"Wikipedia's “List of dependent territories” excludes overseas territories integrated into the home country, such as French overseas departments; INTEGRAL_OVERSEAS adds them (curated=true)\"}\n\n\ndef parse_cn_admin(raw: str) -> dict:\n nodes = json.loads(raw)\n items = []\n\n def walk(ns, parent, level):\n for n in ns:\n name, code = n.get(\"name\", \"\"), n.get(\"code\", \"\")\n ch = n.get(\"children\") or []\n if level == 2 and name in (\"市辖区\", \"县\", \"省直辖县级行政区划\", \"自治区直辖县级行政区划\", \"市\"):\n walk(ch, parent, 3) # dummy level for municipalities / province-administered units: attach children directly to the province\n continue\n items.append({\"name\": name, \"code\": code, \"level\": level, \"parent\": parent})\n walk(ch, name, level + 1)\n\n walk(nodes, \"\", 1)\n return {\"items\": items}\n\n\n\n# ---------------------------------------------------------------- China pack: raw tables → shared formats\n\nCN_ADMIN_LEVELS = {1: \"admin1\", 2: \"admin2\", 3: \"district\"} # must match regions/cn/region.json admin_levels\n\n\nclass _CNAdmin:\n def __init__(self, items: list[dict]):\n self.items = items\n self.by: dict[str, list[dict]] = {}\n for it in items:\n self.by.setdefault(it[\"name\"], []).append(it)\n\n def chain(self, it: dict) -> list[dict]:\n out, cur = [it], it\n while cur.get(\"parent\"):\n nxt = self.by.get(cur[\"parent\"])\n if not nxt:\n break\n cur = nxt[0]\n out.append(cur)\n return list(reversed(out))\n\n def under(self, prov: str, name: str) -> dict | None:\n \"\"\"The one item called name (or name without its 市/区/县 suffix) inside province prov.\"\"\"\n cands = self.by.get(name) or [it for n, its in self.by.items() if n.rstrip(\"市区县\") == name.rstrip(\"市区县\") for it in its]\n cands = [it for it in cands if prov in [c[\"name\"] for c in self.chain(it)]]\n return cands[0] if len(cands) == 1 else None\n\n def match(self, it: dict) -> dict:\n return {CN_ADMIN_LEVELS[c[\"level\"]]: c[\"name\"] for c in self.chain(it)}\n\n\ndef _cn_places(adm: _CNAdmin, prov: str, place: str) -> list[dict]:\n \"\"\"Free-text place from a Wikipedia table cell → matches. Unresolvable text (vehicle classes, \"直辖市全境\", historic areas)\n stays at province level with the text as the note, instead of becoming a fake place.\"\"\"\n place = place.strip()\n if place == prov or place.rstrip(\"市\") == prov.rstrip(\"市\"):\n return [{\"admin1\": prov}]\n m = re.match(r\"^(.+?)（(.+)）$\", place)\n head, inner = (m.group(1), m.group(2)) if m else (place, \"\")\n it = adm.under(prov, head)\n if not it:\n return [{\"admin1\": prov, \"note\": place}]\n if inner:\n subs = [x.strip() for x in re.split(r\"[、，,]\", inner) if x.strip()]\n kids = [adm.under(prov, x) for x in subs]\n if subs and all(kids) and not re.search(r\"不含|部分|地级|地区\", inner):\n return [adm.match(k) for k in kids]\n return [dict(adm.match(it), note=inner)]\n return [adm.match(it)]\n\n\ndef cn_plates_table(raw: dict, adm: _CNAdmin) -> dict:\n entries: dict[str, list[dict]] = {}\n for abbr, v in raw.items():\n prov = v[\"province\"]\n entries[abbr] = [{\"admin1\": prov, \"note\": \"province abbreviation only\"}]\n unv = v.get(\"letter_notes_unverified\") or {}\n for letter, place in v[\"letters\"].items():\n rows = []\n for p in re.split(r\"[、，,/]\", place):\n if not p.strip():\n continue\n for r in _cn_places(adm, prov, p):\n if set(r) \u003c= {\"admin1\", \"note\"} and not r.get(\"note\") and letter in unv:\n r.update(note=f\"whole municipality; district split from common knowledge: {unv[letter]}\", unverified=True)\n elif set(r) \u003c= {\"admin1\", \"note\"} and not r.get(\"note\"):\n r[\"note\"] = \"whole municipality\" if prov.endswith(\"市\") else \"whole province\"\n rows.append(r)\n entries[abbr + letter] = _dedup(rows)\n return entries\n\n\ndef cn_area_codes_table(raw: dict, adm: _CNAdmin) -> dict:\n entries: dict[str, list[dict]] = {}\n for code, e in raw.items():\n note = (\"deprecated; \" if e.get(\"deprecated\") else \"\") + (e.get(\"note\") or \"\")\n rows = []\n for a in e[\"admin2\"] or [e[\"admin1\"]]:\n for r in _cn_places(adm, e[\"admin1\"], a):\n r[\"note\"] = \"; \".join(x for x in (r.get(\"note\", \"\"), note) if x)\n if e.get(\"digits\"):\n r[\"digits\"] = e[\"digits\"]\n rows.append(r)\n entries[code] = _dedup(rows)\n return entries\n\n\ndef _dedup(rows: list[dict]) -> list[dict]:\n \"\"\"One match per place: rows naming the same place (several vehicle classes under one letter) merge, notes joined.\"\"\"\n out: dict[str, dict] = {}\n for r in rows:\n k = json.dumps({lv: r[lv] for lv in (\"admin1\", \"admin2\", \"district\") if r.get(lv)}, ensure_ascii=False, sort_keys=True)\n if k not in out:\n out[k] = dict(r)\n continue\n notes = [x for x in out[k].get(\"note\", \"\").split(\"; \") if x]\n notes += [x for x in r.get(\"note\", \"\").split(\"; \") if x and x not in notes]\n out[k][\"note\"] = \"; \".join(notes)\n return list(out.values())\n\n\nPLATE_META = {\"kind\": \"plate\", \"normalize\": {\"strip\": \" ·-\", \"upper\": True},\n \"key_pattern\": \"^[\\\\u4e00-\\\\u9fff][A-Z]?\",\n \"usage\": \"first Han character (province) + issuing letter, e.g. \u003cabbr>\u003cletter>; the rest of the plate is ignored\",\n \"note\": \"Municipality letter splits come from common knowledge, marked unverified; Hong Kong/Macau/Taiwan, military and police plates are not in the table.\",\n \"miss_note\": \"province abbreviation not in the table\"}\nAREA_META = {\"kind\": \"area-code\", \"normalize\": {\"digits_only\": True, \"ensure_prefix\": \"0\"},\n \"usage\": \"landline area code; accepts 0817, 0817-1234567, (0817) 123. 010/02X are 3 digits, the rest 4\",\n \"miss_note\": \"not a mainland China landline area code (mobile number, 400/800, foreign number)\"}\n\n\n# ---------------------------------------------------------------- fetch / write\n\ndef _fetch(url: str, proxy: str | None) -> str:\n cmd = [\"curl\", \"-q\", \"-s\", \"-m\", \"90\", \"-A\", UA, \"-L\"]\n cmd += curl_args(proxy)\n r = subprocess.run(cmd + [url], capture_output=True)\n if r.returncode != 0 or len(r.stdout) \u003c 1000:\n sys.exit(f\"fetch failed: {url} (check service availability with doctor.py --network)\")\n return r.stdout.decode(\"utf-8\", \"replace\")\n\n\ndef _write(path: Path, payload: dict) -> None:\n path.parent.mkdir(parents=True, exist_ok=True)\n path.write_text(json.dumps(payload, ensure_ascii=False, indent=1), encoding=\"utf-8\")\n print(f\"{path.relative_to(HERE.parent)}: {payload['_meta'].get('count')} entries ({path.stat().st_size // 1024} KB)\")\n\n\ndef refresh(name: str, raw: str) -> None:\n today = date.today().isoformat()\n meta = {\"source\": SOURCES[name], \"fetched\": today, \"license\": LICENSE.get(name, WIKI_LICENSE)}\n if name in (\"calling_codes\", \"driving_side\", \"territories\"):\n data = {\"calling_codes\": parse_calling_codes, \"driving_side\": parse_driving_side, \"territories\": parse_territories}[name](raw)\n _write(DATA / f\"{name}.json\", {\"_meta\": dict(meta, count=data.get(\"count\") or len(data)), **data})\n elif name == \"cn/admin\":\n data = parse_cn_admin(raw)\n _write(CN / \"admin.json\", {\"_meta\": dict(meta, count=len(data[\"items\"])), **data})\n else:\n adm = _CNAdmin(json.loads((CN / \"admin.json\").read_text(encoding=\"utf-8\"))[\"items\"])\n if name == \"cn/plates\":\n entries, extra = cn_plates_table(parse_cn_plates(raw), adm), PLATE_META\n else:\n entries, extra = cn_area_codes_table(parse_cn_area_codes(raw), adm), AREA_META\n _write(CN / f\"{name.split('/')[1]}.json\", {\"_meta\": dict(extra, **meta, count=len(entries)), \"entries\": entries})\n\n\ndef main() -> None:\n ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)\n ap.add_argument(\"table\", nargs=\"?\", default=\"all\", help=f\"all or one of {', '.join(SOURCES)}\")\n ap.add_argument(\"--proxy\", default=os.environ.get(\"GEO_PROXY\"), help=PROXY_HELP)\n ap.add_argument(\"--from-dir\", type=Path, help=\"directory of already-downloaded source files (for development)\")\n args = ap.parse_args()\n names = list(SOURCES) if args.table == \"all\" else [args.table]\n for name in names:\n if name not in SOURCES:\n sys.exit(f\"unknown table {name}; one of {', '.join(SOURCES)}\")\n raw = (args.from_dir / LOCAL_NAMES[name]).read_text(encoding=\"utf-8\", errors=\"replace\") if args.from_dir else _fetch(SOURCES[name][0], args.proxy)\n refresh(name, raw)\n\n\nif __name__ == \"__main__\":\n sys.stdout.reconfigure(encoding=\"utf-8\")\n sys.stderr.reconfigure(encoding=\"utf-8\", errors=\"backslashreplace\")\n main()\n","content_type":"text/x-python; charset=utf-8","language":"python","size":27083,"content_sha256":"ae5cef48be2a66509dd31d9c4ce5b022a37ed9735797cb66f7eec8dafe9b6375"},{"filename":"scripts/regions.py","content":"#!/usr/bin/env python3\n# /// script\n# requires-python = \">=3.10\"\n# dependencies = []\n# ///\n\"\"\"Region packs: country-specific clues, lookup tables and service preferences, kept out of the core.\n\nA pack is a folder regions/\u003ccc>/ (cc = ISO 3166-1 alpha-2, lower case) holding only data and documents:\n region.json manifest: names, admin levels, lookup tables, preferred services, tips (format: regions/README.md)\n clues.md clue entries in the shared entry format\n data/*.json lookup tables in the shared prefix-table format\n tests.json lookup cases: input → expected matches\nPacks never contain code. The core scripts read the manifest; a lookup kind a pack doesn't declare is reported as unsupported, never answered from another country's table.\n\n list installed packs: code, name, status, lookups, clue entries\n show \u003ccc> one pack's card: lookups, services, tips, and a one-line index of its clue entries with line numbers\n lint [cc ...] check packs against the contract (manifest, sources, entry fields, no code, tests); exit 1 on errors\n\nExtra pack folders (private packs, tests) can be added with GEO_SLEUTH_REGIONS=\u003cdir>[:\u003cdir>...]; a pack there overrides a bundled pack with the same code.\n\nExamples:\n regions.py list\n regions.py show cn\n regions.py lint\n\"\"\"\nfrom __future__ import annotations\n\nimport argparse\nimport json\nimport os\nimport re\nimport sys\nimport unicodedata\nfrom pathlib import Path\n\nROOT = Path(__file__).resolve().parent.parent / \"regions\"\nBOARD_LEVELS = [\"country\", \"admin1\", \"admin2\", \"city\", \"district\", \"area\", \"road\", \"point\"]\nMATCH_LEVELS = [\"admin1\", \"admin2\", \"city\", \"district\"] # levels a lookup match may name\nSTATUSES = (\"maintained\", \"community\", \"stub\")\nENTRY_FIELDS = (\"Look for\", \"Points to\", \"Strength\", \"Counterexamples\", \"Sources\")\nALLOWED_SUFFIXES = {\".json\", \".md\"}\nMERGED = re.compile(r\"/|、|;\") # \"Ghaziabad / Noida\" is two places, not one (\"and\"/\"&\" stay legal: Jammu and Kashmir is one)\n\n_CACHE: dict | None = None\n\n\n# ---------------------------------------------------------------- loading\n\ndef _dirs() -> list[Path]:\n extra = [Path(p) for p in os.environ.get(\"GEO_SLEUTH_REGIONS\", \"\").split(os.pathsep) if p.strip()]\n return [ROOT, *extra]\n\n\ndef packs(reload: bool = False) -> dict[str, dict]:\n \"\"\"{code: manifest + {\"dir\": Path}} for every installed pack (folders starting with _ are templates, skipped).\"\"\"\n global _CACHE\n if _CACHE is not None and not reload:\n return _CACHE\n out: dict[str, dict] = {}\n for base in _dirs():\n if not base.is_dir():\n continue\n for d in sorted(base.iterdir()):\n f = d / \"region.json\"\n if d.name.startswith((\"_\", \".\")) or not f.is_file():\n continue\n try:\n m = json.loads(f.read_text(encoding=\"utf-8\"))\n except ValueError:\n continue\n m[\"dir\"] = d\n out[str(m.get(\"code\", d.name)).upper()] = m\n _CACHE = out\n return out\n\n\ndef _fold(s: str) -> str:\n s = unicodedata.normalize(\"NFKC\", s or \"\").strip().lower()\n return re.sub(r\"[\\s.()\\-_'’]\", \"\", s)\n\n\ndef osm_name_keys() -> tuple[str, ...]:\n \"\"\"OSM tags to match a place name on: `name`, plus every installed pack's `osm_name_keys` (CN: name:zh, name:zh-Hans,\n because OSM's `name` adds a second script for some regions and simplified forms live only in name:zh-Hans).\"\"\"\n extra = {k for m in packs().values() if isinstance(m.get(\"osm_name_keys\"), list)\n for k in m[\"osm_name_keys\"] if isinstance(k, str) and k.startswith(\"name:\")}\n return (\"name\", *sorted(extra))\n\n\ndef poi_for_script(text: str) -> dict | None:\n \"\"\"The poi service of the first pack whose services.poi.keyword_scripts (Unicode character-name prefixes, e.g.\n \"CJK UNIFIED IDEOGRAPH\") match a character of text: the default sources when poi.py gets no --region.\"\"\"\n import unicodedata as ud\n names = [ud.name(ch, \"\") for ch in text]\n for _, m in sorted(packs().items()):\n services = m.get(\"services\") or {}\n poi = services.get(\"poi\") if isinstance(services, dict) else None\n scripts = poi.get(\"keyword_scripts\") if isinstance(poi, dict) else None\n if not isinstance(scripts, list) or not all(isinstance(s, str) and s.strip() for s in scripts):\n continue # lint reports malformed settings; other packs must keep working\n if any(n.startswith(tuple(scripts)) for n in names if n):\n return poi\n return None\n\n\ndef resolve(country: str | None) -> dict | None:\n \"\"\"Country code / name / alias → pack, or None when no installed pack matches.\"\"\"\n if not country:\n return None\n key = _fold(country)\n for code, m in packs().items():\n if key == code.lower() or key in {_fold(n) for n in [m.get(\"name\", \"\"), *m.get(\"names\", [])]}:\n return m\n return None\n\n\n_TABLES: dict[Path, dict] = {}\n\n\ndef load_table(pack: dict, rel: str) -> dict:\n f = pack[\"dir\"] / rel\n if f not in _TABLES:\n _TABLES[f] = json.loads(f.read_text(encoding=\"utf-8\"))\n return _TABLES[f]\n\n\n# ---------------------------------------------------------------- prefix-table lookups\n\ndef normalize(value: str, spec: dict) -> str:\n v = unicodedata.normalize(\"NFKC\", value or \"\") if spec.get(\"fullwidth\", True) else (value or \"\")\n for ch in spec.get(\"strip\", \" \"):\n v = v.replace(ch, \"\")\n if spec.get(\"digits_only\"):\n v = re.sub(r\"\\D\", \"\", v)\n if spec.get(\"upper\", True):\n v = v.upper()\n pre = spec.get(\"ensure_prefix\")\n if pre and v and not v.startswith(pre):\n v = pre + v\n return v\n\n\ndef prefix_lookup(pack: dict, kind: str, value: str) -> dict:\n \"\"\"Look a value up in one pack's prefix table.\n Returns {\"status\": ok|no_match|not_applicable|unsupported, \"matches\": [...], \"note\", \"source\", \"fetched\"}.\n not_applicable = the value doesn't have this table's format (e.g. a Latin plate against a table of Han-character prefixes).\"\"\"\n spec = (pack.get(\"lookups\") or {}).get(kind)\n if not spec:\n return {\"status\": \"unsupported\", \"matches\": [], \"note\": f\"{pack['code']} pack has no {kind} table\", \"source\": \"\", \"fetched\": \"\"}\n t = load_table(pack, spec[\"file\"])\n if not isinstance(t, dict) or not isinstance(t.get(\"entries\", {}), dict):\n raise ValueError(f\"{spec['file']} must be an object with an 'entries' object\")\n meta = t.get(\"_meta\", {})\n src = \", \".join(meta.get(\"source\") or []) if isinstance(meta.get(\"source\"), list) else str(meta.get(\"source\", \"\"))\n base = {\"source\": src, \"fetched\": meta.get(\"fetched\", \"\")}\n v = normalize(value, meta.get(\"normalize\") or {})\n target = v\n if meta.get(\"key_pattern\"):\n m = re.match(meta[\"key_pattern\"], v)\n if not m:\n return {\"status\": \"not_applicable\", \"matches\": [], \"note\": \"\", **base}\n target = m.group(0)\n entries = t.get(\"entries\") or {}\n key = next((target[:n] for n in range(len(target), 0, -1) if target[:n] in entries), None)\n if key is None:\n return {\"status\": \"no_match\", \"matches\": [], \"note\": meta.get(\"miss_note\", \"not in the table\"), **base}\n note = \"\"\n if meta.get(\"key_pattern\") and key != target:\n note = f\"only '{key}' matched; '{target[len(key):]}' is not in the table (new series, or table out of date)\"\n matches = []\n for e in entries[key]:\n row = {\"country\": pack.get(\"name\", pack[\"code\"]), \"country_code\": pack[\"code\"], \"key\": key}\n row.update(e)\n if note:\n row[\"note\"] = \"; \".join(x for x in (note, row.get(\"note\", \"\")) if x)\n matches.append(row)\n return {\"status\": \"ok\", \"matches\": matches, \"note\": \"\", **base}\n\n\nPACK_ERRORS = (OSError, ValueError, KeyError, TypeError, AttributeError, IndexError, re.error)\n\n\ndef safe_lookup(pack: dict, kind: str, value: str) -> dict:\n \"\"\"prefix_lookup that turns a broken pack (missing or malformed table) into status \"error\" instead of an exception.\"\"\"\n try:\n return prefix_lookup(pack, kind, value)\n except PACK_ERRORS as exc:\n return {\"status\": \"error\", \"matches\": [], \"source\": \"\", \"fetched\": \"\",\n \"note\": f\"{pack['code']} pack is broken ({type(exc).__name__}: {exc}); run `regions.py lint {pack['code'].lower()}`\"}\n\n\ndef lookup(kind: str, value: str, country: str | None = None) -> dict:\n \"\"\"Prefix lookup across packs. With a country: only that pack (unsupported if it isn't installed or lacks the table).\n Without: every pack that declares the kind; matches from several countries are all returned.\"\"\"\n if country:\n p = resolve(country)\n if not p:\n have = \", \".join(sorted(packs())) or \"none\"\n return {\"status\": \"unsupported\", \"matches\": [], \"searched\": [],\n \"note\": f\"no region pack for '{country}' (installed: {have}); use global tools, or contribute a pack (regions/README.md)\"}\n r = safe_lookup(p, kind, value)\n r[\"searched\"] = [] if r[\"status\"] == \"error\" else [p[\"code\"]]\n if r[\"status\"] == \"unsupported\":\n r[\"note\"] += f\" (its lookups: {', '.join(p.get('lookups') or {}) or 'none'})\"\n return r\n searched, matches, notes, sources, fetched = [], [], [], [], []\n for code, p in packs().items():\n if kind not in (p.get(\"lookups\") or {}):\n continue\n r = safe_lookup(p, kind, value)\n if r[\"status\"] == \"not_applicable\":\n continue\n if r[\"status\"] == \"error\": # one broken pack mustn't take down the other countries' answers\n notes.append(r[\"note\"])\n continue\n searched.append(code)\n matches += r[\"matches\"]\n if r[\"status\"] == \"no_match\":\n notes.append(f\"{code}: {r['note']}\")\n if r[\"source\"]:\n sources.append(r[\"source\"])\n fetched.append(f\"{code} {r['fetched']}\")\n if not searched:\n return {\"status\": \"unsupported\", \"matches\": [], \"searched\": [],\n \"note\": \"; \".join([f\"no installed pack has a {kind} table in this format\", *notes]),\n \"source\": \"\", \"fetched\": \"\"}\n note = \"; \".join(notes)\n if len({m[\"country_code\"] for m in matches}) > 1:\n note = \"; \".join(x for x in (\"matches in several countries: keep each as a hypothesis, or pass --country\", note) if x)\n return {\"status\": \"ok\" if matches else \"no_match\", \"matches\": matches, \"searched\": searched, \"note\": note,\n \"source\": \" | \".join(sources), \"fetched\": \", \".join(fetched)}\n\n\n# ---------------------------------------------------------------- admin tables\n\ndef admin_items(pack: dict) -> list[dict]:\n rel = pack.get(\"admin_table\")\n if not rel or not (pack[\"dir\"] / rel).is_file():\n return []\n try:\n items = load_table(pack, rel).get(\"items\") or []\n if not all(isinstance(it, dict) and \"name\" in it for it in items):\n raise ValueError(f\"{rel}: items must be objects with a name\")\n return items\n except PACK_ERRORS: # broken table: treated as absent here; doctor.py and lint report it\n return []\n\n\ndef level_name(pack: dict, admin_level) -> str:\n return (pack.get(\"admin_levels\") or {}).get(str(admin_level), \"district\")\n\n\ndef strip_suffix(name: str, pack: dict) -> str:\n \"\"\"Name without one of the pack's generic suffixes (CN: 市/省/区/县), for matching names across sources.\"\"\"\n out = name or \"\"\n for s in pack.get(\"name_suffixes\") or []:\n if out.endswith(s) and len(out) > len(s):\n return out[: -len(s)]\n return out\n\n\n_INDEX: dict[str, tuple[list[dict], dict]] = {}\n\n\ndef admin_index(pack: dict) -> tuple[list[dict], dict]:\n key = str(pack[\"dir\"])\n if key not in _INDEX:\n items = admin_items(pack)\n by: dict[str, list[dict]] = {}\n for it in items:\n by.setdefault(it[\"name\"], []).append(it)\n _INDEX[key] = (items, by)\n return _INDEX[key]\n\n\ndef admin_chain(pack: dict, item: dict) -> list[str]:\n _, by = admin_index(pack)\n chain, p = [item[\"name\"]], item.get(\"parent\", \"\")\n while p:\n chain.append(p)\n nxt = by.get(p)\n p = nxt[0].get(\"parent\", \"\") if nxt else \"\"\n return list(reversed(chain))\n\n\ndef valid_alias(a) -> bool:\n return isinstance(a, str) and bool(a.strip())\n\n\ndef aliases(item: dict) -> list[str]:\n \"\"\"An item's aliases; a malformed field is ignored here (lint reports it), never split into characters.\"\"\"\n a = item.get(\"aliases\")\n return [x for x in a if valid_alias(x)] if isinstance(a, list) else []\n\n\ndef admin_find(name: str, country: str | None = None) -> list[tuple[dict, dict]]:\n \"\"\"Exact name first, then a match on the name or an alias ignoring the pack's generic suffixes (市/省/区/县 in CN). [(pack, item)]\"\"\"\n pk = [resolve(country)] if country else list(packs().values())\n out = []\n for p in pk:\n if not p:\n continue\n items, by = admin_index(p)\n want = strip_suffix(name, p)\n hits = by.get(name) or [it for it in items\n if want in {strip_suffix(n, p) for n in [it[\"name\"], *aliases(it)]}]\n out += [(p, it) for it in hits]\n return out\n\n\ndef admin_children(name: str, country: str | None = None) -> tuple[dict | None, list[dict]]:\n \"\"\"Direct children of an admin area in the local tables. (pack, items); (None, []) when no pack knows the parent.\"\"\"\n hits = admin_find(name, country)\n if not hits:\n return None, []\n p, parent = hits[0]\n kids = [it for it in admin_items(p) if it.get(\"parent\") == parent[\"name\"]]\n return p, kids\n\n\ndef match_from_item(pack: dict, item: dict) -> dict:\n \"\"\"Admin-table item → match dict with explicit board levels along its chain.\"\"\"\n _, by = admin_index(pack)\n row = {\"country\": pack.get(\"name\", pack[\"code\"]), \"country_code\": pack[\"code\"]}\n for nm in admin_chain(pack, item):\n lv = by[nm][0][\"level\"] if nm in by else None\n if lv is not None:\n row[level_name(pack, lv)] = nm\n return row\n\n\ndef cid(code: str, match: dict, upto: str | None = None) -> str:\n \"\"\"Stable candidate id: country code + names along the levels the match states, e.g. CN/广东省/深圳市.\"\"\"\n parts = [code]\n for lv in MATCH_LEVELS:\n if match.get(lv):\n parts.append(match[lv])\n if lv == upto:\n break\n return \"/\".join(parts)\n\n\n# ---------------------------------------------------------------- clue index (for show)\n\ndef clue_index(path: Path) -> list[tuple[int, str, str]]:\n \"\"\"[(line number, '##'/'###', text)] — entries get 'title — points to (strength)'.\"\"\"\n out: list[tuple[int, str, str]] = []\n lines = path.read_text(encoding=\"utf-8\").splitlines()\n for i, s in enumerate(lines, 1):\n if s.startswith(\"## \"):\n out.append((i, \"##\", s[3:].strip()))\n elif s.startswith(\"### \"):\n title, points, strength = s[4:].strip(), \"\", \"\"\n for t in lines[i: i + 12]:\n if t.startswith(\"### \") or t.startswith(\"## \"):\n break\n if t.startswith(\"- Points to:\"):\n points = t.split(\":\", 1)[1].strip()\n elif t.startswith(\"- Strength:\"):\n strength = t.split(\":\", 1)[1].strip().split(\" \")[0].strip(\",;\")\n short = points if len(points) \u003c= 70 else points[:68].rsplit(\" \", 1)[0] + \"…\"\n out.append((i, \"###\", f\"{title} — {short}\" + (f\" ({strength})\" if strength else \"\")))\n return out\n\n\n# ---------------------------------------------------------------- lint\n\ndef lint(pack: dict) -> tuple[list[str], list[str]]:\n errs, warns = [], []\n d: Path = pack[\"dir\"]\n code = str(pack.get(\"code\", \"\"))\n if not re.fullmatch(r\"[A-Z]{2}\", code):\n errs.append(f\"code '{code}' must be an ISO 3166-1 alpha-2 code in upper case\")\n if d.name != code.lower():\n errs.append(f\"folder name '{d.name}' must be the lower-case code '{code.lower()}'\")\n for k in (\"name\", \"names\", \"status\", \"docs\"):\n if not pack.get(k):\n errs.append(f\"region.json: '{k}' is required\")\n if pack.get(\"status\") and pack[\"status\"] not in STATUSES:\n errs.append(f\"region.json: status must be one of {STATUSES}\")\n for f in d.rglob(\"*\"):\n if f.is_file() and f.suffix.lower() not in ALLOWED_SUFFIXES:\n errs.append(f\"{f.relative_to(d)}: packs hold data and documents only (.json/.md); code goes into core scripts through a separate PR\")\n for rel in pack.get(\"docs\") or []:\n f = d / rel\n if not f.is_file():\n errs.append(f\"docs: {rel} not found\")\n continue\n _lint_entries(f, rel, errs, warns)\n keys = pack.get(\"osm_name_keys\")\n if keys is not None and not (isinstance(keys, list) and all(isinstance(k, str) and k.startswith(\"name:\") for k in keys)):\n errs.append(\"region.json: 'osm_name_keys' must be a list of OSM name:* tags\")\n services = pack.get(\"services\", {})\n if not isinstance(services, dict):\n errs.append(\"region.json: 'services' must be an object\")\n services = {}\n poi = services.get(\"poi\", {})\n if not isinstance(poi, dict):\n errs.append(\"region.json: 'services.poi' must be an object\")\n poi = {}\n scripts = poi.get(\"keyword_scripts\")\n if scripts is not None and not (isinstance(scripts, list) and all(isinstance(s, str) and s.strip() for s in scripts)):\n errs.append(\"region.json: 'services.poi.keyword_scripts' must be a list of Unicode character-name prefixes\")\n also = pack.get(\"also_children\")\n if also is not None and not (isinstance(also, list) and all(isinstance(n, str) and n.strip() for n in also)):\n errs.append(\"region.json: 'also_children' must be a list of OSM relation names\")\n levels = set((pack.get(\"admin_levels\") or {}).values())\n bad_lv = levels - set(MATCH_LEVELS)\n if bad_lv:\n errs.append(f\"admin_levels: {sorted(bad_lv)} are not board levels {MATCH_LEVELS}\")\n checks = [lambda: _lint_admin(pack, errs)] if pack.get(\"admin_table\") else []\n checks += [lambda k=k, v=v: _lint_table(pack, k, v, errs, warns) for k, v in (pack.get(\"lookups\") or {}).items()]\n checks.append(lambda: _lint_tests(pack, errs))\n for check in checks: # a malformed file is an error to report, not a crash\n try:\n check()\n except PACK_ERRORS as exc:\n errs.append(f\"malformed data ({type(exc).__name__}: {exc})\")\n for svc in (\"street_view\",):\n v = (pack.get(\"services\") or {}).get(svc)\n if v and not (Path(__file__).parent / v).is_file():\n errs.append(f\"services.{svc}: {v} is not a core script\")\n return errs, warns\n\n\ndef _lint_entries(f: Path, rel: str, errs: list, warns: list) -> None:\n lines = f.read_text(encoding=\"utf-8\").splitlines()\n if len(lines) > 400:\n warns.append(f\"{rel}: {len(lines)} lines; split by topic into several docs (each listed in region.json docs)\")\n if not rel.startswith(\"clues\"):\n return\n starts = [i for i, s in enumerate(lines) if s.startswith(\"### \")]\n for n, i in enumerate(starts):\n end = next((j for j in range(i + 1, len(lines)) if lines[j].startswith((\"### \", \"## \"))), len(lines))\n body = lines[i + 1: end]\n missing = [k for k in ENTRY_FIELDS if not any(s.startswith(f\"- {k}:\") for s in body)]\n if missing:\n errs.append(f\"{rel}:{i + 1} '{lines[i][4:].strip()}' lacks {', '.join(missing)}\")\n\n\ndef _lint_table(pack: dict, kind: str, spec: dict, errs: list, warns: list) -> None:\n f = pack[\"dir\"] / spec.get(\"file\", \"\")\n if not f.is_file():\n errs.append(f\"lookups.{kind}: {spec.get('file')} not found\")\n return\n try:\n t = json.loads(f.read_text(encoding=\"utf-8\"))\n except ValueError as exc:\n errs.append(f\"{spec['file']}: invalid JSON ({exc})\")\n return\n if not isinstance(t, dict):\n errs.append(f\"{spec['file']}: must be an object with '_meta' and 'entries'\")\n return\n meta, entries = t.get(\"_meta\") or {}, t.get(\"entries\")\n for k in (\"source\", \"fetched\", \"license\"):\n if not meta.get(k):\n errs.append(f\"{spec['file']}: _meta.{k} is required (where the data comes from, when, under which license)\")\n if meta.get(\"fetched\") and not re.fullmatch(r\"\\d{4}-\\d{2}-\\d{2}\", str(meta[\"fetched\"])):\n errs.append(f\"{spec['file']}: _meta.fetched must be YYYY-MM-DD\")\n if meta.get(\"key_pattern\"):\n try:\n re.compile(meta[\"key_pattern\"])\n except re.error as exc:\n errs.append(f\"{spec['file']}: key_pattern doesn't compile ({exc})\")\n if not isinstance(entries, dict) or not entries:\n errs.append(f\"{spec['file']}: 'entries' must be a non-empty object of key → [match, ...]\")\n return\n if meta.get(\"count\") is not None and meta[\"count\"] != len(entries):\n errs.append(f\"{spec['file']}: _meta.count {meta['count']} != {len(entries)} entries\")\n if f.stat().st_size > 8 * 1024 * 1024:\n warns.append(f\"{spec['file']}: over 8 MB\")\n for key, rows in entries.items():\n if not isinstance(rows, list) or not rows:\n errs.append(f\"{spec['file']}: '{key}' must map to a non-empty list (several places → several matches, never one merged name)\")\n continue\n for r in rows:\n if not isinstance(r, dict) or not any(r.get(lv) for lv in MATCH_LEVELS):\n errs.append(f\"{spec['file']}: '{key}' has a match without any of {MATCH_LEVELS}\")\n break\n merged = [r[lv] for lv in MATCH_LEVELS if r.get(lv) and MERGED.search(str(r[lv]))]\n if merged:\n errs.append(f\"{spec['file']}: '{key}' names {merged}; list each place as its own match\")\n break\n extra = [k for k in r if k in BOARD_LEVELS and k not in MATCH_LEVELS]\n if extra:\n errs.append(f\"{spec['file']}: '{key}' uses {extra}; a match names admin levels only\")\n break\n\n\ndef _lint_admin(pack: dict, errs: list) -> None:\n rel = pack[\"admin_table\"]\n f = pack[\"dir\"] / rel\n if not f.is_file():\n errs.append(f\"admin_table: {rel} not found\")\n return\n t = json.loads(f.read_text(encoding=\"utf-8\"))\n meta = t.get(\"_meta\") or {}\n for k in (\"source\", \"fetched\", \"license\"):\n if not meta.get(k):\n errs.append(f\"{rel}: _meta.{k} is required\")\n items = t.get(\"items\") or []\n names = {it.get(\"name\") for it in items}\n lv = {str(it.get(\"level\")) for it in items}\n unknown = lv - set((pack.get(\"admin_levels\") or {}).keys())\n if unknown:\n errs.append(f\"{rel}: levels {sorted(unknown)} have no entry in admin_levels\")\n bad = [it.get(\"name\") for it in items if \"aliases\" in it\n and not (isinstance(it[\"aliases\"], list) and all(valid_alias(a) for a in it[\"aliases\"]))]\n if bad:\n errs.append(f\"{rel}: aliases must be a list of non-empty strings (e.g. {bad[:3]})\")\n orphans = [it[\"name\"] for it in items if it.get(\"parent\") and it[\"parent\"] not in names]\n if orphans:\n errs.append(f\"{rel}: {len(orphans)} items name a parent that isn't in the table (e.g. {orphans[:3]})\")\n\n\ndef _lint_tests(pack: dict, errs: list) -> None:\n f = pack[\"dir\"] / \"tests.json\"\n kinds = set(pack.get(\"lookups\") or {})\n if not kinds:\n return\n if not f.is_file():\n errs.append(\"tests.json is required when the pack has lookups (at least one case per lookup kind)\")\n return\n cases = json.loads(f.read_text(encoding=\"utf-8\"))\n seen = set()\n for c in cases:\n seen.add(c.get(\"kind\"))\n r = prefix_lookup(pack, c[\"kind\"], c[\"value\"])\n got = sorted(json.dumps({k: m[k] for k in MATCH_LEVELS if m.get(k)}, ensure_ascii=False, sort_keys=True) for m in r[\"matches\"])\n want = sorted(json.dumps({k: m[k] for k in MATCH_LEVELS if m.get(k)}, ensure_ascii=False, sort_keys=True) for m in c.get(\"expect\", []))\n if got != want:\n errs.append(f\"tests.json: {c['kind']} {c['value']} → {got}, expected {want}\")\n for k in sorted(kinds - seen):\n errs.append(f\"tests.json: no case for lookup kind '{k}'\")\n\n\n# ---------------------------------------------------------------- CLI\n\ndef _count_entries(p: dict) -> int:\n n = 0\n for rel in p.get(\"docs\") or []:\n f = p[\"dir\"] / rel\n if f.is_file():\n n += sum(1 for s in f.read_text(encoding=\"utf-8\").splitlines() if s.startswith(\"### \"))\n return n\n\n\ndef cmd_list(_args) -> None:\n ps = packs()\n if not ps:\n print(\"No region packs installed.\")\n return\n for code, p in sorted(ps.items()):\n lk = \", \".join(p.get(\"lookups\") or {}) or \"-\"\n print(f\"{code.lower()} {p.get('name', '')} [{p.get('status', '?')}] lookups: {lk} clue entries: {_count_entries(p)}\")\n print(\"\\n`regions.py show \u003ccc>` for one pack's card and clue index. No pack for a country = global tools and references only.\")\n\n\ndef cmd_show(args) -> None:\n p = resolve(args.code)\n if not p:\n sys.exit(f\"No pack for '{args.code}' (installed: {', '.join(sorted(packs())) or 'none'}). Use global tools; references/clues/global.md has cross-country clues.\")\n d: Path = p[\"dir\"]\n rel_dir = d.relative_to(ROOT.parent) if d.is_relative_to(ROOT.parent) else d\n print(f\"# {p['code']} {p.get('name', '')} [{p.get('status', '?')}] ({rel_dir}/)\")\n if p.get(\"admin_levels\"):\n names = p.get(\"level_names\") or {}\n print(\"Admin levels: \" + \", \".join(f\"{k}={v}\" + (f\" ({names[v]})\" if names.get(v) else \"\") for k, v in p[\"admin_levels\"].items()))\n for kind, spec in (p.get(\"lookups\") or {}).items():\n t = load_table(p, spec[\"file\"])\n meta = t.get(\"_meta\", {})\n print(f\"Lookup {kind}: {len(t.get('entries') or {})} keys, fetched {meta.get('fetched', '?')} → `clues.py lookup {kind} \u003cvalue> --country {p['code'].lower()}`\"\n + (f\" {meta['usage']}\" if meta.get(\"usage\") else \"\"))\n if p.get(\"admin_table\"):\n print(f\"Admin table: {len(admin_items(p))} areas (used by `board.py children` and `clues.py lookup admin`)\")\n for k, v in (p.get(\"services\") or {}).items():\n print(f\"Service {k}: {v if isinstance(v, str) else json.dumps(v, ensure_ascii=False)}\")\n for k, v in (p.get(\"tips\") or {}).items():\n print(f\"Tip {k}: {v}\")\n for rel in p.get(\"docs\") or []:\n f = d / rel\n if not f.is_file():\n continue\n print(f\"\\n## {rel_dir}/{rel} — read only the entries you need (line numbers below)\")\n for ln, kind, text in clue_index(f):\n print(f\" {ln:>4} {text}\" if kind == \"###\" else f\" {ln:>4} [{text}]\")\n\n\ndef cmd_lint(args) -> None:\n ps = packs()\n codes = [c.upper() for c in args.codes] or sorted(ps)\n bad = 0\n for code in codes:\n if code not in ps:\n print(f\"{code}: not installed\")\n bad += 1\n continue\n errs, warns = lint(ps[code])\n for w in warns:\n print(f\"{code} WARN {w}\")\n for e in errs:\n print(f\"{code} ERROR {e}\")\n bad += bool(errs)\n if not errs:\n print(f\"{code}: ok\" + (f\" ({len(warns)} warnings)\" if warns else \"\"))\n sys.exit(1 if bad else 0)\n\n\ndef main() -> None:\n ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)\n sub = ap.add_subparsers(dest=\"cmd\", required=True)\n sub.add_parser(\"list\")\n s = sub.add_parser(\"show\")\n s.add_argument(\"code\")\n li = sub.add_parser(\"lint\")\n li.add_argument(\"codes\", nargs=\"*\")\n args = ap.parse_args()\n {\"list\": cmd_list, \"show\": cmd_show, \"lint\": cmd_lint}[args.cmd](args)\n\n\nif __name__ == \"__main__\":\n sys.stdout.reconfigure(encoding=\"utf-8\")\n sys.stderr.reconfigure(encoding=\"utf-8\", errors=\"backslashreplace\")\n main()\n","content_type":"text/x-python; charset=utf-8","language":"python","size":27905,"content_sha256":"7430eab7475029f009aaa18584d1e9e0d87f9de3baa7b525d9890e7148120e7d"},{"filename":"scripts/revimg.py","content":"#!/usr/bin/env python3\n# /// script\n# requires-python = \">=3.10\"\n# dependencies = [\"playwright\", \"pillow\", \"pillow-heif\"]\n# ///\n\"\"\"Reverse image search + Chinese keyword search: headless Chrome or Chromium opens the search engines, saves result screenshots, extracts guess text and links.\n\n revimg.py \u003cimage> [\u003cimage> ...] --out-dir rev/ [--engines baidu,yandex] reverse image search\n revimg.py --query \"\u003ckeywords>\" [--query ...] --out-dir q/ [--text-engines bing,baiduimg,sogouimg] keyword search\n\nReverse image search:\n- baidu: the main engine for China. Best coverage of Chinese web pages, Weibo, Baijiahao, e-commerce and scenic-area content; gives a one-line \"图中可能是…\" (\"the image may show…\").\n The \"相似图片\" (similar images) cards are not links; they are captured from the pcsimi requests the page sends (it scrolls a few times to get more pages), deduplicated by contsign and written to `similar` in the JSON;\n the first --similar-sheet of them are downloaded into a numbered contact sheet `\u003cname>_baidu_similar.jpg` (first cell is the query image): near-duplicate photos of the same object/scene are the fastest path to a location, always open it.\n- yandex: supplement for buildings, street scenes and foreign content; gives \"Image appears to contain\" tags and source sites.\n- Google Lens asks for human verification from a server egress, so the script doesn't do it; when you have a browser-control tool (e.g. Claude in Chrome), search in the user's own browser, see references/search.md.\n\nKeyword search (use when general web search tools often fail on Chinese domestic content):\n- bing: Bing China web results (title + link).\n- baiduimg / sogouimg: result-page screenshots from Baidu Images and Sogou Images, for photos of similar scenes.\n- Baidu web search pops up a security check, not done.\n\nEach item outputs `\u003cname>_\u003cengine>.png` (result-page screenshot, always open it) and `.json` (guess text + links; Baidu also has `similar` and the similar-images contact sheet).\n\u003cname> is the file stem; inputs that share a stem (from different folders, or differing only in case) get their number in front: 02_photo.\nAn input that can't be read as a photo (camera RAW, a saved error page, a damaged file) is not uploaded: its `.json` records the reason, and revimg exits 2 after the other searches.\nBefore reverse image search, use `imgprep.py variants` to make tight-crop / flipped / color-cast-removed versions and search each one — no hit on the full image is normal.\nRequires Google Chrome or Playwright Chromium. Chrome is tried first, then Chromium.\nRun `uvx playwright install chromium` if neither is installed; `doctor.py` checks that the browser can start.\n\nExamples:\n revimg.py photo.jpg --out-dir rev/\n revimg.py v/left_crop.jpg v/left_flip.jpg --out-dir rev/ --engines baidu\n revimg.py photo.jpg --out-dir rev/ --engines yandex\n revimg.py --query \"蓝色拱形顶棚人行天桥\" --query \"\u003ccity> 出租车颜色\" --out-dir q/ (queries in Chinese: blue arched canopy footbridge; \u003ccity> taxi color)\n\"\"\"\nfrom __future__ import annotations\n\nimport argparse\nfrom _browser import launch_browser\nfrom _image import ImageReadError, open_image, upright, write_working_copy\nfrom _net import fetch_bytes, PROXY_HELP\nimport asyncio\nimport io\nimport json\nimport os\nimport sys\nfrom collections import Counter\nfrom concurrent.futures import ThreadPoolExecutor\nfrom pathlib import Path\nfrom urllib.parse import parse_qs, urlparse\n\nOWN = {\"baidu\": (\"baidu.com\", \"bdimg.com\", \"bdstatic.com\"), \"yandex\": (\"yandex.\", \"ya.ru\", \"yastatic.net\"),\n \"bing\": (\"bing.com\", \"bing.net\", \"microsoft.com\"), \"baiduimg\": (\"baidu.com\", \"bdimg.com\", \"bdstatic.com\"),\n \"sogouimg\": (\"sogou.com\", \"sogoucdn.com\")}\nBAIDU_REFUSED = (\"功能优化中\", \"建议您重新上传其他图片\")\nTEXT_URL = {\"bing\": \"https://cn.bing.com/search?q={q}\", \"baiduimg\": \"https://image.baidu.com/search/index?tn=baiduimage&word={q}\",\n \"sogouimg\": \"https://pic.sogou.com/pics?query={q}\"}\nUA = \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0 Safari/537.36\"\n\n\nasync def _collect(page, engine: str) -> dict:\n text = await page.evaluate(\"document.body ? document.body.innerText : ''\")\n links = await page.evaluate(\"\"\"() => [...document.querySelectorAll('a[href^=\"http\"]')]\n .map(a => ({t: (a.innerText || a.title || '').trim().replace(/\\\\s+/g, ' ').slice(0, 120), h: a.href}))\n .filter(x => x.t.length >= 4)\"\"\")\n seen, out = set(), []\n for ln in links:\n host = urlparse(ln[\"h\"]).netloc\n if any(o in host for o in OWN[engine]) or ln[\"h\"] in seen:\n continue\n seen.add(ln[\"h\"])\n out.append({\"title\": ln[\"t\"], \"url\": ln[\"h\"], \"site\": host})\n lines = [s.strip() for s in text.splitlines() if s.strip()]\n guess = [s for s in lines if s.startswith(\"图中可能是\") or \"appears to contain\" in s.lower()]\n if engine == \"yandex\" and guess: # the tags are the few lines right after \"appears to contain\"\n i = lines.index(guess[0])\n guess = [guess[0] + \": \" + \" / \".join(lines[i + 1:i + 8])]\n return {\"url\": page.url, \"guess\": guess, \"links\": out[:25], \"text_head\": \"\\n\".join(lines[:60])[:2500]}\n\n\ndef _site(url: str) -> str:\n host = urlparse(url).netloc.lower().split(\":\")[0]\n for pre in (\"www.\", \"m.\", \"wap.\", \"mobile.\"):\n if host.startswith(pre) and host.count(\".\") > 1:\n return host[len(pre):]\n return host\n\n\ndef _font(size: int):\n from PIL import ImageFont\n\n for p in (\"/System/Library/Fonts/STHeiti Medium.ttc\", \"/System/Library/Fonts/Hiragino Sans GB.ttc\",\n \"/usr/share/fonts/opentype/noto/NotoSansCJK-Regular.ttc\"):\n try:\n return ImageFont.truetype(p, size)\n except Exception: # noqa: BLE001\n continue\n return ImageFont.load_default()\n\n\ndef _similar_sheet(query: Path, items: list[dict], out: Path, cols: int = 5, tile: int = 300, proxy: str | None = None) -> int:\n \"\"\"Query image + thumbnails of the first N similar images → numbered contact sheet (number = index in the similar array). Returns how many downloaded successfully.\"\"\"\n from PIL import Image, ImageDraw\n\n def fetch(url: str):\n try:\n data = fetch_bytes(url, proxy, 15, {\"User-Agent\": UA, \"Referer\": \"https://graph.baidu.com/\"})\n return Image.open(io.BytesIO(data)).convert(\"RGB\")\n except Exception: # noqa: BLE001\n return None\n\n with ThreadPoolExecutor(8) as ex:\n ims = list(ex.map(fetch, [it[\"thumb\"] for it in items]))\n try:\n q = upright(open_image(query)).convert(\"RGB\")\n except ImageReadError:\n q = None\n tiles = [(q, \"query\", \"cyan\")] + [\n (im, f\"{i:02d} {it['site'] or '?'}\", \"yellow\") for i, (it, im) in enumerate(zip(items, ims))]\n bar = 24\n rows = (len(tiles) + cols - 1) // cols\n S = Image.new(\"RGB\", (cols * tile, rows * (tile + bar)), (40, 40, 40))\n d = ImageDraw.Draw(S)\n f = _font(17)\n for k, (im, label, color) in enumerate(tiles):\n x, y = (k % cols) * tile, (k // cols) * (tile + bar)\n d.rectangle([x, y, x + tile - 1, y + bar - 1], fill=\"black\")\n d.text((x + 4, y + 2), label[:30], fill=color, font=f)\n if im is None:\n d.text((x + 10, y + bar + tile // 2), \"download failed\", fill=\"gray\", font=f)\n continue\n im.thumbnail((tile - 4, tile - 4))\n S.paste(im, (x + (tile - im.width) // 2, y + bar + (tile - im.height) // 2))\n S.save(out, quality=88)\n return sum(im is not None for im in ims)\n\n\nasync def _baidu(ctx, img: Path, shot: Path, similar_pages: int = 2, sheet_n: int = 24, proxy: str | None = None) -> dict:\n page = await ctx.new_page()\n simi: list = [] # \"相似图片\" (similar images) cards aren't DOM links; they come from ajax/pcsimi responses\n\n async def grab(resp):\n try:\n data = (await resp.json()).get(\"data\") or {}\n simi.append((int(parse_qs(urlparse(resp.url).query).get(\"page\", [\"0\"])[0] or 0), data.get(\"list\") or []))\n except Exception: # noqa: BLE001\n simi.append((0, []))\n\n pending: list = []\n page.on(\"response\", lambda r: pending.append(asyncio.ensure_future(grab(r))) if \"graph.baidu.com/ajax/pcsimi\" in r.url else None)\n await page.goto(\"https://graph.baidu.com/pcpage/index?tpl_from=pc\", wait_until=\"domcontentloaded\", timeout=60000)\n await page.wait_for_timeout(2000)\n inp = await page.query_selector(\"input[type=file]\")\n if not inp:\n await page.screenshot(path=str(shot))\n return {\"error\": \"upload input not found (page redesigned or blocked); check the screenshot\"}\n await inp.set_input_files(str(img))\n for _ in range(60):\n await page.wait_for_timeout(500)\n if \"graph.baidu.com/s\" in page.url:\n break\n await page.wait_for_timeout(4500)\n await page.screenshot(path=str(shot), full_page=True, clip={\"x\": 0, \"y\": 0, \"width\": 1400, \"height\": 3200})\n res = await _collect(page, \"baidu\")\n misses, got = 0, 0 # scrolling to the bottom triggers the next page; stop after two misses in a row\n while got \u003c similar_pages and misses \u003c 2 and pending:\n n0 = len(pending)\n await page.evaluate(\"window.scrollTo(0, document.documentElement.scrollHeight)\")\n for _ in range(16):\n await page.wait_for_timeout(250)\n if len(pending) > n0:\n break\n got, misses = (got + 1, 0) if len(pending) > n0 else (got, misses + 1)\n await asyncio.gather(*pending)\n await page.close()\n seen, similar = set(), []\n for _, lst in sorted(simi, key=lambda t: t[0]):\n for it in lst:\n key = it.get(\"contsign\") or it.get(\"thumbUrl\")\n if not it.get(\"thumbUrl\") or key in seen:\n continue\n seen.add(key)\n similar.append({\"thumb\": it[\"thumbUrl\"], \"from\": it.get(\"fromUrl\") or \"\", \"site\": _site(it.get(\"fromUrl\") or \"\")})\n res[\"similar\"] = similar\n if similar and sheet_n > 0:\n sheet = shot.with_name(shot.stem + \"_similar.jpg\")\n ok = await asyncio.to_thread(_similar_sheet, img, similar[:sheet_n], sheet, proxy=proxy)\n res[\"similar_sheet\"] = str(sheet)\n if ok \u003c min(sheet_n, len(similar)):\n res[\"similar_sheet_note\"] = f\"{min(sheet_n, len(similar)) - ok} thumbnails failed to download\"\n if not res[\"guess\"] and any(w in res[\"text_head\"] for w in BAIDU_REFUSED):\n res[\"error\"] = \"Baidu refused to process this image (the \\\"功能优化中\\\" (\\\"under optimization\\\") page); this does not count as searched with no result: try another crop, retry later, or use only Yandex\"\n return res\n\n\nasync def _yandex(ctx, img: Path, shot: Path) -> dict:\n page = await ctx.new_page()\n await page.goto(\"https://yandex.com/images/search?rpt=imageview\", wait_until=\"domcontentloaded\", timeout=60000)\n await page.wait_for_timeout(2500)\n inp = await page.query_selector(\"input[type=file]\")\n if not inp:\n await page.screenshot(path=str(shot))\n return {\"error\": \"upload input not found (possibly asked for verification); check the screenshot\"}\n await inp.set_input_files(str(img))\n for _ in range(60):\n await page.wait_for_timeout(500)\n if \"url=\" in page.url or \"cbir_id\" in page.url:\n break\n await page.wait_for_timeout(4500)\n for label in (\"Allow essential cookies\", \"Only essential\", \"Accept essential\", \"Allow all\", \"Accept all\"):\n try: # the cookie popup covers the bottom right of the screenshot\n btn = page.get_by_role(\"button\", name=label)\n if await btn.count():\n await btn.first.click(timeout=2000)\n await page.wait_for_timeout(800)\n break\n except Exception: # noqa: BLE001\n continue\n await page.screenshot(path=str(shot), full_page=True, clip={\"x\": 0, \"y\": 0, \"width\": 1400, \"height\": 3200})\n res = await _collect(page, \"yandex\")\n await page.close()\n return res\n\n\nasync def _text(ctx, engine: str, query: str, shot: Path) -> dict:\n from urllib.parse import quote\n\n page = await ctx.new_page()\n await page.goto(TEXT_URL[engine].format(q=quote(query)), wait_until=\"domcontentloaded\", timeout=60000)\n await page.wait_for_timeout(3500)\n head = await page.evaluate(\"document.body ? document.body.innerText.slice(0, 400) : ''\")\n if \"安全验证\" in head or \"captcha\" in page.url.lower():\n await page.screenshot(path=str(shot))\n await page.close()\n return {\"error\": \"asked for human verification (don't bypass it); switch to another engine\"}\n await page.screenshot(path=str(shot), full_page=True, clip={\"x\": 0, \"y\": 0, \"width\": 1400, \"height\": 3000})\n if engine == \"bing\":\n rows = await page.evaluate(\"\"\"() => [...document.querySelectorAll('li.b_algo')].map(li => ({\n t: (li.querySelector('h2') || {}).innerText || '',\n h: (li.querySelector('h2 a') || {}).href || '',\n site: (li.querySelector('cite') || {}).innerText || '',\n s: ((li.querySelector('.b_caption p, .b_lineclamp2, .b_lineclamp3, .b_paractl') || {}).innerText || '').slice(0, 200)}))\"\"\")\n links = [{\"title\": r[\"t\"].strip(), \"url\": r[\"h\"], \"site\": r[\"site\"].strip()[:80], \"snippet\": r[\"s\"].strip()} for r in rows if r[\"t\"]]\n res = {\"url\": page.url, \"guess\": [], \"links\": links[:20], \"text_head\": \"\"}\n else:\n res = await _collect(page, engine)\n await page.close()\n return res\n\n\ndef _names(images: list[Path]) -> list[str]:\n \"\"\"One name per input for its screenshots, result JSON and upload copy. A stem shared with another input (from\n another folder, or differing only in case, which Windows and macOS file systems don't tell apart) gets the\n input's number in front; the result is unique without regard to case.\"\"\"\n stems = [i.stem.casefold() for i in images]\n names, used = [], set()\n for k, img in enumerate(images):\n name = img.stem if stems.count(img.stem.casefold()) == 1 else f\"{k + 1:02d}_{img.stem}\"\n while name.casefold() in used:\n name = f\"{k + 1:02d}_{name}\"\n used.add(name.casefold())\n names.append(name)\n return names\n\n\ndef _upload_path(img: Path, out_dir: Path, name: str | None = None) -> Path:\n \"\"\"Search engines take JPEG and PNG as stored. A HEIC, WebP, ... or a photo with an EXIF orientation still to\n apply is uploaded as an upright lossless PNG copy, out_dir/upload/\u003cname>.png; everything else as it is.\n Raises ImageReadError for a file the shared loader rejects: it is never uploaded.\"\"\"\n info = write_working_copy(img, out_dir / \"upload\" / f\"{name or img.stem}.png\")\n return Path(info[\"working\"]) if info else img\n\n\nasync def run(images: list[Path], engines: list[str], out_dir: Path, proxy: str | None,\n queries: list[str] | None = None, text_engines: list[str] | None = None,\n similar_pages: int = 2, sheet_n: int = 24) -> list[dict]:\n out_dir.mkdir(parents=True, exist_ok=True)\n results = []\n names = _names(images)\n uploads: list[Path | None] = []\n for img, name in zip(images, names) if engines else []:\n try:\n uploads.append(_upload_path(img, out_dir, name))\n except ImageReadError as e: # reported once per engine, the way a failed search is, but nothing is sent\n uploads.append(None)\n for eng in engines:\n res = {\"error\": f\"cannot read image: {e.reason}\", \"unreadable\": e.category, \"image\": str(img), \"engine\": eng,\n \"guess\": [], \"links\": []}\n (out_dir / f\"{name}_{eng}.json\").write_text(json.dumps(res, ensure_ascii=False, indent=1), encoding=\"utf-8\")\n results.append(res)\n if not queries and not any(uploads):\n return results\n from playwright.async_api import async_playwright\n\n async with async_playwright() as p:\n async def browser():\n result, _ = await launch_browser(p, proxy)\n return result\n\n if queries:\n b = await browser()\n ctx = await b.new_context(viewport={\"width\": 1400, \"height\": 1000}, locale=\"zh-CN\")\n for k, q in enumerate(queries):\n for eng in text_engines or []:\n shot = out_dir / f\"q{k + 1:02d}_{eng}.png\"\n try:\n res = await _text(ctx, eng, q, shot)\n except Exception as e: # noqa: BLE001\n res = {\"error\": str(e)[:300]}\n res.update({\"query\": q, \"engine\": eng, \"screenshot\": str(shot), \"image\": f\"q{k + 1:02d}\"})\n (out_dir / f\"q{k + 1:02d}_{eng}.json\").write_text(json.dumps(res, ensure_ascii=False, indent=1), encoding=\"utf-8\")\n results.append(res)\n await b.close()\n\n for eng in engines if any(uploads) else []:\n b = await browser()\n ctx = await b.new_context(viewport={\"width\": 1400, \"height\": 1000},\n locale=\"zh-CN\" if eng == \"baidu\" else \"en-US\")\n for img, name, up in zip(images, names, uploads):\n if up is None:\n continue\n shot = out_dir / f\"{name}_{eng}.png\"\n try:\n res = await (_baidu(ctx, up, shot, similar_pages, sheet_n, proxy) if eng == \"baidu\" else _yandex(ctx, up, shot))\n except Exception as e: # noqa: BLE001\n res = {\"error\": str(e)[:300]}\n res.update({\"image\": str(img), \"engine\": eng, \"screenshot\": str(shot)})\n if up != img:\n res[\"uploaded\"] = str(up)\n (out_dir / f\"{name}_{eng}.json\").write_text(json.dumps(res, ensure_ascii=False, indent=1), encoding=\"utf-8\")\n results.append(res)\n await b.close()\n return results\n\n\ndef main() -> None:\n ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)\n ap.add_argument(\"images\", nargs=\"*\", type=Path)\n ap.add_argument(\"--out-dir\", type=Path, required=True)\n ap.add_argument(\"--engines\", default=\"baidu,yandex\", help=\"reverse image search engines\")\n ap.add_argument(\"--query\", action=\"append\", help=\"keyword search, repeatable\")\n ap.add_argument(\"--text-engines\", default=\"bing,baiduimg,sogouimg\", help=\"keyword search engines\")\n ap.add_argument(\"--proxy\", default=os.environ.get(\"GEO_PROXY\"), help=PROXY_HELP)\n ap.add_argument(\"--exclude\", help=\"comma-separated exclude words: results whose title contains them are not listed (e.g. to hide the puzzle's source in blind tests)\")\n ap.add_argument(\"--similar-pages\", type=int, default=2, help=\"how many more pages of Baidu similar images to load by scrolling past the first (about 30 per page)\")\n ap.add_argument(\"--similar-sheet\", type=int, default=24, help=\"how many of the top similar images to download into a contact sheet; 0 = no sheet\")\n args = ap.parse_args()\n excl = [w for w in (args.exclude or \"\").split(\",\") if w]\n if not args.images and not args.query:\n ap.error(\"give image paths for reverse image search, or use --query for keyword search\")\n missing = [str(p) for p in args.images if not p.is_file()]\n if missing:\n ap.error(\"image not found: \" + \", \".join(missing))\n engines = [e for e in args.engines.split(\",\") if e in (\"baidu\", \"yandex\")]\n text_engines = [e for e in args.text_engines.split(\",\") if e in TEXT_URL]\n results = asyncio.run(run(args.images, engines, args.out_dir, args.proxy, args.query, text_engines,\n args.similar_pages, args.similar_sheet))\n for r in results:\n head = f\"[{r['engine']}] {r['query'] if r.get('query') else Path(r['image']).name}\"\n if r.get(\"error\"):\n print(f\"{head} error: {r['error']}\" + (f\" screenshot {r['screenshot']}\" if r.get(\"screenshot\") else \"\"))\n continue\n print(f\"{head} screenshot {r['screenshot']}\")\n for g in r[\"guess\"][:2]:\n print(f\" guess: {g[:160]}\")\n links = [ln for ln in r[\"links\"] if not any(w in ln[\"title\"] for w in excl)]\n if excl and (len(links) \u003c len(r[\"links\"]) or any(w in r.get(\"text_head\", \"\") for w in excl)):\n print(f\" note: exclude words appear in the results; {len(r['links']) - len(links)} hidden; they may still be visible in the screenshot\")\n for ln in links[:8]:\n print(f\" - {ln['title'][:60]} ({ln['site']})\")\n if r.get(\"similar\"):\n tally = \", \".join(f\"{s} {n}\" for s, n in Counter(x[\"site\"] for x in r[\"similar\"]).most_common(6))\n print(f\" similar images: {len(r['similar'])} ({tally})\" + (f\" contact sheet {r['similar_sheet']} (open it and look for the same object/scene)\" if r.get(\"similar_sheet\") else \"\"))\n elif r[\"engine\"] == \"baidu\":\n print(\" similar images: none captured\")\n unreadable = {r[\"image\"]: r[\"error\"].removeprefix(\"cannot read image: \") for r in results if r.get(\"unreadable\")}\n if unreadable:\n print(f\"revimg: {len(unreadable)} of {len(args.images)} images could not be read and were not uploaded: \"\n + \"; \".join(f\"{p} ({why})\" for p, why in unreadable.items()), file=sys.stderr)\n sys.exit(2)\n\n\nif __name__ == \"__main__\":\n # Chinese-locale Windows writes GBK by default: m², ñ make it crash, and the Chinese the agent reads comes out garbled\n sys.stdout.reconfigure(encoding=\"utf-8\")\n sys.stderr.reconfigure(encoding=\"utf-8\", errors=\"backslashreplace\")\n main()\n","content_type":"text/x-python; charset=utf-8","language":"python","size":21786,"content_sha256":"25a515a47d8bf4e866d96964a84a7d2d816c52975c9774ef768a6b4761c6f3f8"},{"filename":"scripts/sat_scan.py","content":"#!/usr/bin/env python3\n# /// script\n# requires-python = \">=3.10\"\n# dependencies = [\"pillow\", \"numpy\", \"torch\", \"transformers\", \"socksio\", \"pysocks\", \"requests\"]\n# ///\n\"\"\"Find a class of target on satellite imagery (running tracks, stadiums, factory buildings, silos, dams, bridges, open-pit mines, solar farms, greenhouses, ports):\nsplit an area into a grid, score each cell with CLIP zero-shot and sort; you look only at the top 20–30 instead of paging through dozens of contact-sheet pages.\n\n grid tile a bbox with a grid and score cell by cell → ranked.json + contact sheet of top-N thumbnails + score heatmap\n points score and rank a given list of candidate points ({name: [lat, lon]} from osm.py buildings / poi.py) point by point\n presets list the presets and their prompts\n\nThe score is an ordering by \"how alike\", not a verdict: you still have to look at the thumbnails of the top few, then check them against the bearing and shape in the photo.\nCells that are almost a single color (Google placeholder, no imagery) are marked blank and scored 0.\n--seeds pois.json: cells near seed points (school, factory POIs) get a bonus; OSM in China maps few running tracks, but 360/Baidu POIs list schools very completely (poi.py).\n\nExamples:\n sat_scan.py grid --bbox \u003cs,w,n,e> --zoom 17 --cell 320 --preset track --top 30 --out ranked.json --sheet top.jpg --heat heat.jpg\n sat_scan.py grid --bbox \u003cs,w,n,e> --preset factory --seeds factories.json --seed-radius 500 --out r.json --sheet r.jpg\n sat_scan.py points --points big_buildings.json --zoom 17 --preset silo --out r.json --sheet r.jpg\n sat_scan.py grid --bbox \u003cs,w,n,e> --query \"aerial view of a red running track\" --neg \"aerial view of houses\" --out r.json\n\"\"\"\nfrom __future__ import annotations\n\nimport argparse\nfrom _device import torch_device\nfrom _net import PROXY_HELP\nfrom _net import model_proxy_env\nimport json\nimport math\nimport os\nimport re\nimport sys\nimport time\nfrom concurrent.futures import ThreadPoolExecutor\nfrom pathlib import Path\n\nimport numpy as np\nfrom PIL import Image, ImageDraw\n\nsys.path.insert(0, str(Path(__file__).parent))\nimport geo # noqa: E402\nimport tiles # noqa: E402\n\nCLIP_ID = \"openai/clip-vit-base-patch32\"\nNEG_COMMON = [\"a satellite image of residential buildings and streets\", \"a satellite image of a road intersection\",\n \"a satellite image of farmland\", \"a satellite image of a river\", \"a satellite image of a parking lot\",\n \"a satellite image of forest\", \"a satellite image of a highway\", \"a satellite image of bare ground\"]\nPRESETS = {\n \"track\": ([\"a satellite image of a school running track\", \"an aerial view of an athletics track around a football field\",\n \"an oval red running track seen from above\", \"a sports ground with a red oval track and green field from above\"],\n NEG_COMMON + [\"a satellite image of a roundabout\", \"a satellite image of a swimming pool\"]),\n \"stadium\": ([\"a satellite image of a large stadium\", \"an aerial view of a stadium with stands around a field\"], NEG_COMMON),\n \"factory\": ([\"a satellite image of a large factory building with a long roof\", \"an aerial view of industrial warehouses\",\n \"an aerial view of an industrial plant with big sheds\"], NEG_COMMON + [\"a satellite image of apartment blocks\"]),\n \"silo\": ([\"a satellite image of grain silos and a drying tower\", \"an aerial view of a rice mill with silos\",\n \"an aerial view of a farm with round silos and long sheds\"], NEG_COMMON),\n \"dam\": ([\"a satellite image of a dam with a reservoir\", \"an aerial view of a hydroelectric dam\"], NEG_COMMON),\n \"bridge\": ([\"a satellite image of a bridge crossing a river\", \"an aerial view of a long bridge over water\"], NEG_COMMON),\n \"quarry\": ([\"a satellite image of an open pit mine\", \"an aerial view of a quarry with terraces\"], NEG_COMMON),\n \"solar\": ([\"a satellite image of a solar farm with rows of panels\"], NEG_COMMON),\n \"greenhouse\": ([\"a satellite image of rows of greenhouses\", \"an aerial view of plastic greenhouses\"], NEG_COMMON),\n \"port\": ([\"a satellite image of a port with container cranes\", \"an aerial view of a harbor with ships and piers\"], NEG_COMMON),\n \"school\": ([\"a satellite image of a school campus with a running track and classroom buildings\"], NEG_COMMON),\n}\n\n\ndef _proxy_env(proxy: str | None) -> None:\n model_proxy_env(proxy)\n\n\ndef cell_image(lat: float, lon: float, zoom: int, size: int, source: str, proxy: str | None, cache: Path) -> Image.Image:\n \"\"\"Stitch a size×size satellite image centered on (lat, lon) (reuses the tile cache).\"\"\"\n gx, gy = geo.ll2px(zoom, lat, lon)\n x0, y0 = gx - size / 2, gy - size / 2\n tile = Image.new(\"RGB\", (size, size), \"gray\")\n for tx in range(int(x0 // 256), int((x0 + size) // 256) + 1):\n for ty in range(int(y0 // 256), int((y0 + size) // 256) + 1):\n p = cache / f\"{source}_{zoom}_{tx}_{ty}.jpg\"\n if tiles._get(tiles.SOURCES[source].format(x=tx, y=ty, z=zoom), p, proxy):\n try:\n tile.paste(Image.open(p), (int(tx * 256 - x0), int(ty * 256 - y0)))\n except Exception: # noqa: BLE001\n pass\n return tile\n\n\ndef prefetch(points: dict, zoom: int, size: int, source: str, proxy: str | None, cache: Path) -> None:\n need = set()\n for lat, lon in points.values():\n gx, gy = geo.ll2px(zoom, lat, lon)\n x0, y0 = gx - size / 2, gy - size / 2\n for tx in range(int(x0 // 256), int((x0 + size) // 256) + 1):\n for ty in range(int(y0 // 256), int((y0 + size) // 256) + 1):\n need.add((tx, ty))\n cache.mkdir(parents=True, exist_ok=True)\n with ThreadPoolExecutor(16) as ex:\n list(ex.map(lambda t: tiles._get(tiles.SOURCES[source].format(x=t[0], y=t[1], z=zoom), cache / f\"{source}_{zoom}_{t[0]}_{t[1]}.jpg\", proxy), need))\n\n\nCLIPN = ([0.48145466, 0.4578275, 0.40821073], [0.26862954, 0.26130258, 0.27577711])\n\n\ndef _tensor(ims: list[Image.Image], torch):\n mean, std = (np.array(v, dtype=np.float32).reshape(1, 3, 1, 1) for v in CLIPN)\n arr = np.stack([np.asarray(im.convert(\"RGB\"), dtype=np.float32) / 255.0 for im in ims]).transpose(0, 3, 1, 2)\n return torch.from_numpy((arr - mean) / std)\n\n\ndef _feat(out):\n \"\"\"In transformers 5, get_*_features returns BaseModelOutputWithPooling with the projected vector in pooler_output; older versions return the tensor directly.\"\"\"\n if hasattr(out, \"shape\"):\n return out\n for k in (\"pooler_output\", \"text_embeds\", \"image_embeds\"):\n v = getattr(out, k, None)\n if v is not None and hasattr(v, \"shape\"):\n return v\n return out[0]\n\n\nclass Scorer:\n def __init__(self, pos: list[str], neg: list[str]):\n import torch\n from transformers import CLIPModel, CLIPTokenizer\n self.dev = torch_device(torch)\n t0 = time.time()\n try:\n tok = CLIPTokenizer.from_pretrained(CLIP_ID)\n self.model = CLIPModel.from_pretrained(CLIP_ID).to(self.dev).eval()\n except Exception as e: # noqa: BLE001\n sys.exit(f\"failed to load CLIP: {str(e)[:300]}\\ncheck disk space, model availability and `doctor.py --network`.\")\n self.torch = torch\n with torch.no_grad():\n t = tok(pos + neg, return_tensors=\"pt\", padding=True).to(self.dev)\n tf = _feat(self.model.get_text_features(**t))\n self.text = (tf / tf.norm(dim=1, keepdim=True)).float()\n self.npos = len(pos)\n print(f\"CLIP ready ({self.dev}, {time.time() - t0:.1f}s), {len(pos)} positive prompts, {len(neg)} negative prompts\", file=sys.stderr)\n\n def score(self, ims: list[Image.Image], multi_scale: bool, batch: int = 48) -> np.ndarray:\n \"\"\"Score per image = sum of positive-prompt probabilities (softmax over all prompts, logit scale 100); multi_scale also looks at 2×2 sub-blocks and takes the max.\"\"\"\n views_per = 5 if multi_scale else 1\n flat = []\n for im in ims:\n im = im.convert(\"RGB\")\n flat.append(im.resize((224, 224), Image.BICUBIC))\n if multi_scale:\n w, h = im.size\n for (a, b) in ((0, 0), (w // 2, 0), (0, h // 2), (w // 2, h // 2)):\n flat.append(im.crop((a, b, a + w // 2, b + h // 2)).resize((224, 224), Image.BICUBIC))\n probs = []\n with self.torch.no_grad():\n for i in range(0, len(flat), batch):\n x = _tensor(flat[i:i + batch], self.torch).to(self.dev)\n f = _feat(self.model.get_image_features(pixel_values=x))\n f = f / f.norm(dim=1, keepdim=True)\n logits = 100.0 * f.float() @ self.text.T\n p = logits.softmax(dim=1)[:, : self.npos].sum(dim=1)\n probs.append(p.cpu().numpy())\n arr = np.concatenate(probs).reshape(len(ims), views_per)\n return arr.max(axis=1)\n\n\ndef _blank(im: Image.Image) -> bool:\n a = np.asarray(im.convert(\"L\").resize((64, 64)), dtype=np.float32)\n return float(a.std()) \u003c 6.0\n\n\ndef _sheet(rows: list[dict], ims: list[Image.Image], out: Path, cols: int, size: int) -> list[Path]:\n from baidu_pano import _font\n f = _font(14)\n per = cols * cols\n pages = []\n for pi in range(0, len(rows), per):\n chunk = rows[pi:pi + per]\n S = Image.new(\"RGB\", (cols * size, ((len(chunk) + cols - 1) // cols) * size), \"black\")\n d = ImageDraw.Draw(S)\n for k, r in enumerate(chunk):\n cx, cy = (k % cols) * size, (k // cols) * size\n S.paste(ims[r[\"_i\"]].resize((size, size)), (cx, cy))\n c = size / 2\n d.line([cx + c - 10, cy + c, cx + c + 10, cy + c], fill=\"red\", width=2)\n d.line([cx + c, cy + c - 10, cx + c, cy + c + 10], fill=\"red\", width=2)\n d.rectangle([cx, cy, cx + size, cy + 20], fill=\"black\")\n d.text((cx + 4, cy + 3), f\"#{r['rank']} {r['score']:.2f} {r['lat']:.5f},{r['lon']:.5f}\" + (\" seed\" if r.get(\"seed_near\") else \"\"), fill=\"yellow\", font=f)\n o = out if pi == 0 else out.with_name(f\"{out.stem}_{pi // per + 1}{out.suffix}\")\n S.save(o, quality=88)\n pages.append(o)\n return pages\n\n\ndef _heat(rows: list[dict], names: list[str], out: Path) -> None:\n \"\"\"Lay the cells out as a matrix by cell name rXXcYY, mapping score to color.\"\"\"\n rc = {}\n for r in rows:\n m = re.match(r\"r(\\d+)c(\\d+)\", r[\"cell\"])\n if m:\n rc[(int(m.group(1)), int(m.group(2)))] = r[\"score\"]\n if not rc:\n return\n R = max(k[0] for k in rc) + 1\n C = max(k[1] for k in rc) + 1\n px = 24\n S = Image.new(\"RGB\", (C * px, R * px), \"black\")\n d = ImageDraw.Draw(S)\n mx = max(rc.values()) or 1\n for (i, j), s in rc.items():\n v = s / mx\n d.rectangle([j * px, i * px, (j + 1) * px - 1, (i + 1) * px - 1], fill=(int(255 * v), int(120 * v), int(255 * (1 - v)) // 3))\n S.save(out)\n\n\ndef run(points: dict, args) -> None:\n _proxy_env(args.proxy)\n cache = Path(args.cache)\n if args.preset:\n pos, neg = PRESETS[args.preset]\n else:\n pos, neg = [], list(NEG_COMMON)\n pos = list(pos) + list(args.query or [])\n neg = list(neg) + list(args.neg or [])\n if not pos:\n sys.exit(\"give --preset or --query\")\n t0 = time.time()\n prefetch(points, args.zoom, args.size, args.source, args.proxy, cache)\n names = list(points)\n with ThreadPoolExecutor(8) as ex:\n ims = list(ex.map(lambda n: cell_image(points[n][0], points[n][1], args.zoom, args.size, args.source, args.proxy, cache), names))\n t1 = time.time()\n sc = Scorer(pos, neg)\n scores = sc.score(ims, args.multi_scale)\n t2 = time.time()\n seeds = json.loads(Path(args.seeds).read_text(encoding=\"utf-8\")) if args.seeds else {}\n rows = []\n for i, n in enumerate(names):\n lat, lon = points[n]\n blank = _blank(ims[i])\n s = 0.0 if blank else float(scores[i])\n near = None\n if seeds:\n for sn, sl in seeds.items():\n if geo.distance((lat, lon), (sl[0], sl[1])) \u003c= args.seed_radius:\n near = sn\n s += args.seed_bonus\n break\n rows.append({\"cell\": n, \"lat\": lat, \"lon\": lon, \"score\": round(s, 4), \"blank\": blank, \"seed_near\": near, \"_i\": i})\n rows.sort(key=lambda r: -r[\"score\"])\n for k, r in enumerate(rows):\n r[\"rank\"] = k + 1\n print(f\"{len(names)} cells; fetch {t1 - t0:.1f}s, scoring {t2 - t1:.1f}s; blank cells {sum(1 for r in rows if r['blank'])}\")\n print(f\"{'#':>3} {'score':>6} cell/name lat,lon\")\n for r in rows[: min(args.top, 20)]:\n print(f\"{r['rank']:>3} {r['score']:>6.3f} {str(r['cell'])[:18]:\u003c18} {r['lat']:.5f},{r['lon']:.5f}\" + (f\" near {r['seed_near']}\" if r[\"seed_near\"] else \"\"))\n if args.out:\n Path(args.out).write_text(json.dumps([{k: v for k, v in r.items() if k != \"_i\"} for r in rows], ensure_ascii=False, indent=1), encoding=\"utf-8\")\n print(f\"-> {args.out}\")\n pts = {f\"#{r['rank']} {r['score']:.2f}\": [r[\"lat\"], r[\"lon\"]] for r in rows[: args.top]}\n Path(args.out).with_suffix(\".top.json\").write_text(json.dumps(pts, ensure_ascii=False, indent=1), encoding=\"utf-8\")\n if args.sheet:\n for p in _sheet(rows[: args.top], ims, Path(args.sheet), args.cols, args.size):\n print(f\"-> {p}\")\n if getattr(args, \"heat\", None):\n _heat(rows, names, Path(args.heat))\n print(f\"-> {args.heat} (rows north to south, columns west to east; brighter = more alike)\")\n\n\ndef _neg_coords(argv: list[str]) -> list[str]:\n return [\" \" + a if re.match(r\"^-\\d[\\d.]*(,-?[\\d.]+)+$\", a) else a for a in argv]\n\n\ndef main() -> None:\n ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)\n sub = ap.add_subparsers(dest=\"cmd\", required=True)\n\n def common(sp):\n sp.add_argument(\"--preset\", choices=list(PRESETS))\n sp.add_argument(\"--query\", action=\"append\", help=\"custom positive prompt (English), repeatable\")\n sp.add_argument(\"--neg\", action=\"append\", help=\"custom negative prompt, repeatable\")\n sp.add_argument(\"--zoom\", type=int, default=17)\n sp.add_argument(\"--size\", \"--cell\", dest=\"size\", type=int, default=320, help=\"pixels per cell\")\n sp.add_argument(\"--multi-scale\", action=\"store_true\", help=\"also look at 2×2 sub-blocks of each cell and take the max (for small targets)\")\n sp.add_argument(\"--seeds\", help=\"{name:[lat,lon]} seed points; nearby cells get a bonus\")\n sp.add_argument(\"--seed-radius\", type=float, default=400)\n sp.add_argument(\"--seed-bonus\", type=float, default=0.15)\n sp.add_argument(\"--top\", type=int, default=30)\n sp.add_argument(\"--cols\", type=int, default=5)\n sp.add_argument(\"--out\")\n sp.add_argument(\"--sheet\")\n sp.add_argument(\"--source\", choices=list(tiles.SOURCES), default=\"google\")\n sp.add_argument(\"--proxy\", default=os.environ.get(\"GEO_PROXY\"), help=PROXY_HELP)\n sp.add_argument(\"--cache\", type=Path, default=Path(\".geo-cache/tiles\"))\n\n g = sub.add_parser(\"grid\")\n g.add_argument(\"--bbox\", required=True, help=\"s,w,n,e\")\n g.add_argument(\"--step\", type=float, help=\"cell spacing in meters; default is 90%% of the width each cell covers\")\n g.add_argument(\"--heat\", help=\"score heatmap output\")\n common(g)\n\n p = sub.add_parser(\"points\")\n p.add_argument(\"--points\", required=True, help=\"{name: [lat, lon]}\")\n common(p)\n\n sub.add_parser(\"presets\")\n\n args = ap.parse_args(_neg_coords(sys.argv[1:]))\n if args.cmd == \"presets\":\n for k, (pos, neg) in PRESETS.items():\n print(f\"{k}:\\n + \" + \"\\n + \".join(pos) + f\"\\n - {len(neg)} negative prompts\")\n return\n if args.cmd == \"grid\":\n pts = tiles.grid_points(args.bbox, args.zoom, args.size, args.step)\n print(f\"grid: {len(pts)} cells (r rows north to south, c columns west to east)\")\n else:\n pts = json.loads(Path(args.points).read_text(encoding=\"utf-8\"))\n run(pts, args)\n\n\nif __name__ == \"__main__\":\n # Chinese-locale Windows writes GBK by default: m², ñ make it crash, and the Chinese the agent reads comes out garbled\n sys.stdout.reconfigure(encoding=\"utf-8\")\n sys.stderr.reconfigure(encoding=\"utf-8\", errors=\"backslashreplace\")\n main()\n","content_type":"text/x-python; charset=utf-8","language":"python","size":16396,"content_sha256":"5ff73aab9e78ba9531dff172c4080a173f7d6880dc30e79058abe205c53278c6"},{"filename":"scripts/sun.py","content":"#!/usr/bin/env python3\n# /// script\n# requires-python = \">=3.10\"\n# dependencies = [\"pillow\"]\n# ///\n\"\"\"Sky geometry: sun position, shadows, satellite TV dishes.\n\nSun position uses the NOAA algorithm (Meeus), error about 0.01° for 1950–2050, with atmospheric refraction correction.\nAzimuths are always compass bearings: 0=north, clockwise. Times can carry a time zone (--tz Asia/Shanghai); the default is UTC.\n\n pos sun azimuth, elevation angle, shadow length of a 1 m object, and shadow bearing at a given place and time\n ratio shadow ratio ↔ sun elevation angle conversion (object height 1, shadow length r)\n locate known capture time + measured elevation angle (or shadow ratio) / shadow bearing → find the matching band within an area\n when known place + elevation angle / shadow bearing → work back to which times of day (or which days in a date range) match\n dish geostationary satellite dish pointing; given the dish azimuth it can also work back to longitude\n street known place and date + angle between shadow and street → street orientation candidates (morning and afternoon listed separately)\n facing which walls are lit and which are shaded → camera heading range (use when there is no measurable shadow)\n compass sun in the frame → camera heading, and the true bearing of any object in the frame (with no time given, split into sunrise and sunset groups)\n\nExamples:\n sun.py pos --at 39.9042,116.4074 --time 2023-08-15T16:20 --tz Asia/Shanghai\n sun.py ratio --shadow 1.2\n sun.py locate --time 2023-08-15T16:20 --tz Asia/Shanghai --ratio 1.2 --tol 1.5 --bbox 34,110,42,122\n sun.py locate --time 2023-08-15T16:20 --tz Asia/Shanghai --elev 40 --shadow-bearing 60 --az-tol 10 \\\n --bbox 34,110,42,122 --mosaic north.jpg --out band.jpg\n sun.py when --at 30.25,120.16 --date 2024-10-01 --tz Asia/Shanghai --ratio 1.2 --shadow-bearing 30\n sun.py when --at 30.25,120.16 --dates 2024-01-01:2024-12-31 --tz Asia/Shanghai --elev 40 --shadow-bearing 330\n sun.py dish --at 31.23,121.47 --sat 92.2\n sun.py dish --lat 31.2 --sat 92.2 --azimuth 215 # dish faces 215°, work back to longitude\n sun.py compass --at \u003clat,lon> --time 07:40 --dates 2024-09-01:2024-10-15 --tz \u003cIANA time zone> --sun-x 1200 --width 4000 --hfov 60:70 --x 2900\n sun.py compass --at \u003clat,lon> --tz \u003cIANA time zone> --sun-x 2600 --width 4032 --x 1500 # no time, no date: morning and evening groups over the whole year\n sun.py street --at 49.25,-123.10 --date 2025-04-01 --tz America/Vancouver --ratio 1.5 --tol 4 --shadow-rel 90\n\"\"\"\nfrom __future__ import annotations\n\nimport argparse\nimport json\nimport math\nimport re\nimport sys\nfrom datetime import date, datetime, timedelta, timezone\nfrom pathlib import Path\nfrom zoneinfo import ZoneInfo\n\nsys.path.insert(0, str(Path(__file__).parent))\n\n# ---------------------------------------------------------------- Sun position\n\n_D = math.degrees\n_R = math.radians\n\n\ndef _sun_core(dt_utc: datetime) -> tuple[float, float]:\n \"\"\"Returns (solar declination °, equation of time in minutes).\"\"\"\n jd = dt_utc.timestamp() / 86400 + 2440587.5\n t = (jd - 2451545) / 36525\n l0 = (280.46646 + t * (36000.76983 + t * 0.0003032)) % 360\n m = 357.52911 + t * (35999.05029 - 0.0001537 * t)\n e = 0.016708634 - t * (0.000042037 + 0.0000001267 * t)\n c = (math.sin(_R(m)) * (1.914602 - t * (0.004817 + 0.000014 * t))\n + math.sin(_R(2 * m)) * (0.019993 - 0.000101 * t) + math.sin(_R(3 * m)) * 0.000289)\n app_long = l0 + c - 0.00569 - 0.00478 * math.sin(_R(125.04 - 1934.136 * t))\n obliq0 = 23 + (26 + (21.448 - t * (46.815 + t * (0.00059 - t * 0.001813))) / 60) / 60\n obliq = obliq0 + 0.00256 * math.cos(_R(125.04 - 1934.136 * t))\n decl = _D(math.asin(math.sin(_R(obliq)) * math.sin(_R(app_long))))\n y = math.tan(_R(obliq / 2)) ** 2\n eqt = 4 * _D(y * math.sin(2 * _R(l0)) - 2 * e * math.sin(_R(m))\n + 4 * e * y * math.sin(_R(m)) * math.cos(2 * _R(l0))\n - 0.5 * y * y * math.sin(4 * _R(l0)) - 1.25 * e * e * math.sin(2 * _R(m)))\n return decl, eqt\n\n\ndef _refraction(h: float) -> float:\n if h > 85:\n return 0.0\n te = math.tan(_R(h))\n if h > 5:\n r = 58.1 / te - 0.07 / te ** 3 + 0.000086 / te ** 5\n elif h > -0.575:\n r = 1735 + h * (-518.2 + h * (103.4 + h * (-12.79 + h * 0.711)))\n else:\n r = -20.772 / te\n return r / 3600\n\n\ndef sun_position(lat: float, lon: float, dt_utc: datetime, refraction: bool = True) -> tuple[float, float]:\n \"\"\"(azimuth °, elevation angle °). dt_utc must carry the UTC time zone.\"\"\"\n decl, eqt = _sun_core(dt_utc)\n minutes = dt_utc.hour * 60 + dt_utc.minute + dt_utc.second / 60 + dt_utc.microsecond / 6e7\n tst = (minutes + eqt + 4 * lon) % 1440\n ha = tst / 4 + 180 if tst / 4 \u003c 0 else tst / 4 - 180\n cz = math.sin(_R(lat)) * math.sin(_R(decl)) + math.cos(_R(lat)) * math.cos(_R(decl)) * math.cos(_R(ha))\n zen = _D(math.acos(max(-1.0, min(1.0, cz))))\n elev = 90 - zen\n denom = math.cos(_R(lat)) * math.sin(_R(zen))\n if abs(denom) \u003c 1e-9:\n az = 180.0\n else:\n ca = max(-1.0, min(1.0, (math.sin(_R(lat)) * math.cos(_R(zen)) - math.sin(_R(decl))) / denom))\n az = (_D(math.acos(ca)) + 180) % 360 if ha > 0 else (540 - _D(math.acos(ca))) % 360\n if refraction:\n elev += _refraction(elev)\n return az, elev\n\n\ndef subsolar_point(dt_utc: datetime) -> tuple[float, float]:\n \"\"\"Subsolar point (lat, lon). At a given moment, the points with the same sun elevation angle form a circle centered on it with radius 90°-elevation angle.\"\"\"\n decl, eqt = _sun_core(dt_utc)\n minutes = dt_utc.hour * 60 + dt_utc.minute + dt_utc.second / 60\n lon = -(minutes + eqt - 720) / 4\n return decl, (lon + 540) % 360 - 180\n\n\ndef shadow_ratio(elev: float) -> float:\n \"\"\"Shadow length (m) of a 1 m vertical object on level ground.\"\"\"\n return math.inf if elev \u003c= 0 else 1 / math.tan(_R(elev))\n\n\ndef elev_from_ratio(ratio: float) -> float:\n return _D(math.atan(1 / ratio))\n\n\ndef _ang_diff(a: float, b: float) -> float:\n return abs((a - b + 180) % 360 - 180)\n\n\n# ---------------------------------------------------------------- Satellite dishes\n\n_RE, _RGEO = 6378.137, 42164.0\n\n\ndef dish_pointing(lat: float, lon: float, sat_lon: float) -> tuple[float, float]:\n \"\"\"Ground point (lat, lon) pointing at the geostationary satellite at longitude sat_lon: (azimuth °, elevation angle °).\"\"\"\n la, lo = _R(lat), _R(lon)\n gx, gy, gz = _RE * math.cos(la) * math.cos(lo), _RE * math.cos(la) * math.sin(lo), _RE * math.sin(la)\n sx, sy, sz = _RGEO * math.cos(_R(sat_lon)), _RGEO * math.sin(_R(sat_lon)), 0.0\n dx, dy, dz = sx - gx, sy - gy, sz - gz\n east = -math.sin(lo) * dx + math.cos(lo) * dy\n north = -math.sin(la) * math.cos(lo) * dx - math.sin(la) * math.sin(lo) * dy + math.cos(la) * dz\n up = math.cos(la) * math.cos(lo) * dx + math.cos(la) * math.sin(lo) * dy + math.sin(la) * dz\n return (_D(math.atan2(east, north)) + 360) % 360, _D(math.atan2(up, math.hypot(east, north)))\n\n\n# Common satellites (longitude, east positive)\nSATELLITES = {\n \"chinasat9\": (92.2, \"ChinaSat-9 (中星9号): small dishes of China's 户户通/村村通 (rural direct-to-home) programs, the most common nationwide\"),\n \"chinasat6b\": (115.5, \"ChinaSat-6B: large dishes at cable TV headends and organizations\"),\n \"asiasat7\": (105.5, \"AsiaSat 7\"),\n \"apstar6c\": (134.0, \"APStar-6C\"),\n \"astra1\": (19.2, \"Astra 1: Germany, Austria and other Central Europe\"),\n \"hotbird\": (13.0, \"Hot Bird: Italy, Poland, etc.\"),\n \"astra2\": (28.2, \"Astra 2: UK, Ireland\"),\n \"eutelsat5w\": (-5.0, \"Eutelsat 5W: France, parts of Spain\"),\n \"nilesat\": (-7.0, \"Nilesat: Middle East, North Africa\"),\n \"turksat\": (42.0, \"Türksat: Turkey\"),\n}\n\n\n# ---------------------------------------------------------------- Input parsing\n\ndef _pair(s: str) -> tuple[float, float]:\n a, b = s.split(\",\")\n return float(a), float(b)\n\n\ndef _to_utc(s: str, tz: str | None) -> datetime:\n dt = datetime.fromisoformat(s)\n if dt.tzinfo is None:\n dt = dt.replace(tzinfo=ZoneInfo(tz) if tz else timezone.utc)\n return dt.astimezone(timezone.utc)\n\n\ndef _target_elev(args) -> float | None:\n if getattr(args, \"ratio\", None) is not None:\n return elev_from_ratio(args.ratio)\n return getattr(args, \"elev\", None)\n\n\ndef _target_az(args) -> float | None:\n \"\"\"Normalize to sun azimuth. Shadow bearing = sun azimuth + 180.\"\"\"\n if getattr(args, \"shadow_bearing\", None) is not None:\n return (args.shadow_bearing + 180) % 360\n return getattr(args, \"sun_azimuth\", None)\n\n\n# ---------------------------------------------------------------- Subcommands\n\ndef cmd_pos(args) -> None:\n lat, lon = args.at\n t = _to_utc(args.time, args.tz)\n az, el = sun_position(lat, lon, t)\n ss = subsolar_point(t)\n print(json.dumps({\n \"utc\": t.isoformat(timespec=\"minutes\"), \"sun_azimuth\": round(az, 2), \"sun_elevation\": round(el, 2),\n \"shadow_len_per_1m\": None if el \u003c= 0 else round(shadow_ratio(el), 3),\n \"shadow_bearing\": round((az + 180) % 360, 1), \"subsolar_point\": [round(ss[0], 3), round(ss[1], 3)],\n }, ensure_ascii=False))\n\n\ndef cmd_ratio(args) -> None:\n if args.shadow is not None:\n print(f\"Shadow ratio 1:{args.shadow} → sun elevation angle {elev_from_ratio(args.shadow):.2f}°\")\n else:\n print(f\"Sun elevation angle {args.elev}° → shadow length of a 1 m object {shadow_ratio(args.elev):.3f} m\")\n\n\ndef _matches(lat, lon, times, tel, tol, taz, az_tol) -> bool:\n for t in times:\n az, el = sun_position(lat, lon, t)\n if tel is not None and abs(el - tel) > tol:\n continue\n if taz is not None and _ang_diff(az, taz) > az_tol:\n continue\n if el \u003c= 0:\n continue\n return True\n return False\n\n\ndef cmd_locate(args) -> None:\n tel, taz = _target_elev(args), _target_az(args)\n if tel is None and taz is None:\n sys.exit(\"Need at least one of --elev / --ratio or --shadow-bearing / --sun-azimuth\")\n t0 = _to_utc(args.time, args.tz)\n k = max(0, int(args.time_tol // 5))\n times = [t0 + timedelta(minutes=5 * i) for i in range(-k, k + 1)] if args.time_tol else [t0]\n s, w, n, e = args.bbox\n step = args.step\n pts = []\n lat = s\n while lat \u003c= n + 1e-9:\n lon = w\n while lon \u003c= e + 1e-9:\n if _matches(lat, lon, times, tel, args.tol, taz, args.az_tol):\n pts.append((round(lat, 4), round(lon, 4)))\n lon += step\n lat += step\n ss = subsolar_point(t0)\n out = {\"utc\": t0.isoformat(timespec=\"minutes\"), \"target_elev\": None if tel is None else round(tel, 2),\n \"target_sun_azimuth\": None if taz is None else round(taz, 1), \"subsolar_point\": [round(ss[0], 3), round(ss[1], 3)],\n \"equal_elev_circle_radius_km\": None if tel is None else round((90 - tel) * 111.195, 0),\n \"grid_step_deg\": step, \"matched_points\": len(pts)}\n if pts:\n lats, lons = [p[0] for p in pts], [p[1] for p in pts]\n out[\"matched_bbox\"] = [min(lats), min(lons), max(lats), max(lons)]\n cols: dict[float, list[float]] = {}\n for la, lo in pts:\n cols.setdefault(round(lo, 2), []).append(la)\n every = max(1, len(cols) // 12)\n out[\"lat_range_by_lon\"] = {f\"{lo:.2f}\": [min(v), max(v)] for i, (lo, v) in enumerate(sorted(cols.items())) if i % every == 0}\n print(json.dumps(out, ensure_ascii=False, indent=1))\n if args.points:\n args.points.write_text(json.dumps({f\"p{i}\": list(p) for i, p in enumerate(pts)}), encoding=\"utf-8\")\n print(f\"points -> {args.points}\")\n if args.mosaic:\n _draw(args.mosaic, pts, step, args.out or args.mosaic.with_name(args.mosaic.stem + \"_sun.jpg\"))\n\n\ndef _draw(mosaic: Path, pts, step, out: Path) -> None:\n from PIL import Image, ImageDraw\n from tiles import Mosaic\n m = Mosaic(mosaic)\n im = Image.open(mosaic).convert(\"RGBA\")\n ov = Image.new(\"RGBA\", im.size, (0, 0, 0, 0))\n d = ImageDraw.Draw(ov)\n for la, lo in pts:\n x0, y0 = m.to_px(la + step / 2, lo - step / 2)\n x1, y1 = m.to_px(la - step / 2, lo + step / 2)\n d.rectangle([x0, y0, x1, y1], fill=(255, 210, 0, 90))\n Image.alpha_composite(im, ov).convert(\"RGB\").save(out, quality=88)\n print(f\"map -> {out}\")\n\n\ndef _scan_day(lat, lon, day: date, tz: ZoneInfo, tel, tol, taz, az_tol, step_min: int):\n \"\"\"Returns the matching time windows in this day [(start, end, midpoint azimuth, midpoint elevation angle)].\"\"\"\n start = datetime(day.year, day.month, day.day, tzinfo=tz)\n wins, cur = [], None\n for i in range(0, 24 * 60, step_min):\n lt = start + timedelta(minutes=i)\n az, el = sun_position(lat, lon, lt.astimezone(timezone.utc))\n ok = el > 0 and (tel is None or abs(el - tel) \u003c= tol) and (taz is None or _ang_diff(az, taz) \u003c= az_tol)\n if ok:\n cur = [lt, lt, []] if cur is None else [cur[0], lt, cur[2]]\n cur[2].append((az, el))\n elif cur is not None:\n wins.append(cur)\n cur = None\n if cur is not None:\n wins.append(cur)\n res = []\n for a, b, vals in wins:\n az, el = vals[len(vals) // 2]\n res.append((a, b, az, el))\n return res\n\n\ndef cmd_when(args) -> None:\n tel, taz = _target_elev(args), _target_az(args)\n if tel is None and taz is None:\n sys.exit(\"Need at least one of --elev / --ratio or --shadow-bearing / --sun-azimuth\")\n tz = ZoneInfo(args.tz) if args.tz else timezone.utc\n lat, lon = args.at\n if args.dates:\n a, b = (date.fromisoformat(x) for x in args.dates.split(\":\"))\n days = [a + timedelta(days=i) for i in range((b - a).days + 1)]\n else:\n days = [date.fromisoformat(args.date)]\n hits = 0\n for d in days:\n for s, e, az, el in _scan_day(lat, lon, d, tz, tel, args.tol, taz, args.az_tol, args.step_min):\n hits += 1\n mid = s + (e - s) / 2\n _, e1 = sun_position(lat, lon, (mid - timedelta(minutes=5)).astimezone(timezone.utc))\n _, e2 = sun_position(lat, lon, (mid + timedelta(minutes=5)).astimezone(timezone.utc))\n rate = abs(e2 - e1) / 10\n sens = f\" elevation angle changes {rate:.2f}° per minute, 1° off ≈ {1 / rate:.0f} minutes\" if rate > 1e-3 else \" elevation angle barely changes around noon; time resolution is poor\"\n print(f\"{d} {s:%H:%M}–{e:%H:%M} sun azimuth {az:5.1f}° elevation {el:4.1f}° shadow bearing {(az + 180) % 360:5.1f}° \"\n f\"shadow ratio 1:{shadow_ratio(el):.2f}{sens}\")\n if not hits:\n print(\"No matching times (loosen --tol / --az-tol, or check whether the time zone or bearing is backwards)\")\n\n\ndef cmd_dish(args) -> None:\n sat = args.sat\n if args.at:\n lat, lon = args.at\n az, el = dish_pointing(lat, lon, sat)\n print(f\"Place {lat},{lon} pointing at the satellite at longitude {sat}°E: azimuth {az:.1f}°, elevation angle {el:.1f}°\")\n return\n if args.azimuth is None or args.lat is None:\n sys.exit(\"Forward calculation: give --at; working back to longitude: give --lat and --azimuth\")\n best = []\n for i in range(-1800, 1801):\n lon = i / 10\n az, el = dish_pointing(args.lat, lon, sat)\n if el > 0 and _ang_diff(az, args.azimuth) \u003c= args.az_tol:\n best.append((lon, az, el))\n if not best:\n print(\"No longitude at this latitude matches (the dish may point at a different satellite)\")\n return\n print(f\"Latitude {args.lat}, dish azimuth {args.azimuth}±{args.az_tol}° to the satellite at longitude {sat}°E → longitude range {best[0][0]}° ~ {best[-1][0]}°, \"\n f\"elevation angle about {best[len(best) // 2][2]:.0f}°\")\n\n\ndef cmd_facing(args) -> None:\n \"\"\"Which walls are lit / shaded → camera heading range. A wall is lit ⟺ |sun azimuth − wall normal| \u003c 90°.\n\n Walls are described by \"which way they face in the frame\": camera=faces the camera (normal = heading+180), left=faces frame left (heading−90),\n right=faces frame right (heading+90), away=faces away from the camera (normal = heading).\n \"\"\"\n lat, lon = args.at\n t = _to_utc(args.time, args.tz)\n az, el = sun_position(lat, lon, t)\n if el \u003c= 0:\n print(f\"The sun is below the horizon at this time (elevation {el:.1f}°); the lit-face method doesn't apply\")\n return\n offs = {\"camera\": 180.0, \"left\": -90.0, \"right\": 90.0, \"away\": 0.0}\n lit = [w for w in (args.lit or \"\").split(\",\") if w]\n shaded = [w for w in (args.shaded or \"\").split(\",\") if w]\n bad = [w for w in lit + shaded if w not in offs]\n if bad:\n sys.exit(f\"Walls can only be camera/left/right/away, got: {bad}\")\n ok = []\n for h in range(0, 360):\n good = True\n for w in lit:\n if _ang_diff(az, (h + offs[w]) % 360) >= 90 - args.margin:\n good = False\n for w in shaded:\n if _ang_diff(az, (h + offs[w]) % 360) \u003c= 90 + args.margin:\n good = False\n if good:\n ok.append(h)\n print(f\"Sun azimuth {az:.1f}°, elevation {el:.1f}° (shadow bearing {(az + 180) % 360:.1f}°)\")\n if not ok:\n print(\"No heading satisfies all these lit/shaded conditions: check whether you misread which wall is lit, or whether the time / time zone is wrong (the photo may also be a composite)\")\n return\n runs, start = [], ok[0]\n for a, b in zip(ok, ok[1:] + [None]):\n if b is None or b != a + 1:\n runs.append((start, a))\n start = b\n if len(runs) > 1 and runs[0][0] == 0 and runs[-1][1] == 359: # merge across north\n runs = [(runs[-1][0] - 360, runs[0][1])] + runs[1:-1]\n print(\"Possible camera heading ranges: \" + \", \".join(f\"{a % 360}°–{b % 360}°\" for a, b in runs))\n print(\"Tip: at the range edges the walls are nearly parallel to the sunlight and the light/dark difference is small; don't treat them as hard edges (--margin sets how conservative)\")\n\n\ndef cmd_street(args) -> None:\n \"\"\"Measure the angle in a top-down view: clockwise from the street direction to the shadow direction. If you can't tell clockwise from counterclockwise, list both (--both).\"\"\"\n tz = ZoneInfo(args.tz) if args.tz else timezone.utc\n lat, lon = args.at\n tel = _target_elev(args)\n day = date.fromisoformat(args.date)\n start = datetime(day.year, day.month, day.day, tzinfo=tz)\n rows = []\n for i in range(0, 24 * 60, args.step_min):\n lt = start + timedelta(minutes=i)\n az, el = sun_position(lat, lon, lt.astimezone(timezone.utc))\n if el \u003c= 2 or (tel is not None and abs(el - tel) > args.tol):\n continue\n shadow = (az + 180) % 360\n cands = {round((shadow - args.shadow_rel) % 180)}\n if args.both:\n cands.add(round((shadow + args.shadow_rel) % 180))\n rows.append((lt, az, el, shadow, sorted(cands)))\n if not rows:\n print(\"No matching times (loosen the elevation angle tolerance, or check the date / time zone)\")\n return\n step = max(1, len(rows) // 16)\n for k, (lt, az, el, shadow, cands) in enumerate(rows):\n if k % step and k != len(rows) - 1:\n continue\n half = \"morning\" if az \u003c 180 else \"afternoon\" # sun east of the meridian; clock noon can be hours off solar noon\n print(f\"{lt:%H:%M} {half} sun {az:5.1f}°/{el:4.1f}° shadow bearing {shadow:5.1f}° → street orientation \"\n + \" or \".join(f\"{c}°–{c + 180}°\" for c in cands))\n print(\"Street orientation is given as 0–180° (47° means northeast–southwest). You must keep both the morning and the afternoon group, unless other evidence tells morning from afternoon.\")\n\n\n\ndef _px_angle(x: float, width: float, hfov: float) -> float:\n \"\"\"Horizontal angle (°, right positive) of pixel column x relative to the frame center.\"\"\"\n f = (width / 2) / math.tan(math.radians(hfov) / 2)\n return math.degrees(math.atan((x - width / 2) / f))\n\n\ndef cmd_compass(args) -> None:\n \"\"\"Sun in the frame → camera heading, and the true bearing of any pixel column in the frame (tower, chimney, intersection).\n\n Time given: use the sun azimuth at that moment directly (if the date is uncertain, give a range with --dates and the bearing becomes a range too).\n No time given: compute the morning and evening stretches of each day in --dates where the sun elevation is between 0 and --elev-max separately; list both the sunrise and the sunset group.\n \"\"\"\n lat, lon = args.at\n tz = ZoneInfo(args.tz) if args.tz else timezone.utc\n if args.dates:\n a, b = (date.fromisoformat(x) for x in args.dates.split(\":\"))\n elif args.time and \"T\" in args.time:\n a = b = datetime.fromisoformat(args.time).date()\n else:\n a, b = date(2025, 1, 1), date(2025, 12, 31)\n print(\"No date given: computing over the whole year. Posting time is not capture time; when phenology, clothing or snow can pin down the season, narrow it with --dates\")\n groups: dict[str, list[float]] = {}\n lo_el, hi_el = (float(x) for x in args.elev.split(\":\")) if args.elev else (None, None)\n dropped: list[date] = []\n d = a\n while d \u003c= b:\n if args.time:\n hh, mm = (args.time.split(\"T\")[-1].split(\":\") + [\"0\"])[:2]\n lt = datetime(d.year, d.month, d.day, int(hh), int(mm), tzinfo=tz)\n az, el = sun_position(lat, lon, lt.astimezone(timezone.utc))\n if (el > -1 if lo_el is None else lo_el \u003c= el \u003c= hi_el):\n groups.setdefault(f\"{int(hh):02d}:{int(mm):02d}\", []).append(az)\n elif lo_el is not None:\n dropped.append(d)\n else:\n start = datetime(d.year, d.month, d.day, tzinfo=tz)\n for i in range(0, 24 * 60, 4):\n lt = start + timedelta(minutes=i)\n az, el = sun_position(lat, lon, lt.astimezone(timezone.utc))\n if (0 \u003c el \u003c= args.elev_max) if lo_el is None else (lo_el \u003c= el \u003c= hi_el):\n groups.setdefault(\"morning (sunrise side)\" if az \u003c 180 else \"evening (sunset side)\", []).append(az)\n d += timedelta(days=max(1, args.step_days))\n if dropped:\n runs, s0 = [], dropped[0]\n for x, y in zip(dropped, dropped[1:] + [None]):\n if y is None or (y - x).days > max(1, args.step_days):\n runs.append((s0, x))\n s0 = y\n print(\"Dates excluded because the sun elevation is outside --elev: \" + \", \".join(f\"{p:%m-%d}–{q:%m-%d}\" for p, q in runs))\n if not groups:\n print(\"No matching sun position on these dates (sun below the horizon, or --elev / --elev-max too narrow)\")\n return\n h0, h1 = (float(x) for x in args.hfov.split(\":\"))\n xs = [float(v) for s in (args.x or []) for v in s.split(\",\")]\n for name, azs in groups.items():\n lo, hi = min(azs), max(azs)\n cands = []\n for hf in (h0, h1):\n off = _px_angle(args.sun_x, args.width, hf)\n cands += [(lo - off) % 360, (hi - off) % 360]\n print(f\"{name}: sun azimuth {lo:.1f}°–{hi:.1f}°; sun at frame x={args.sun_x:.0f} (offset from center \"\n f\"{_px_angle(args.sun_x, args.width, h0):+.1f}° to {_px_angle(args.sun_x, args.width, h1):+.1f}°)\"\n f\" → camera heading about {min(cands):.0f}°–{max(cands):.0f}°\")\n for x in xs:\n bs = []\n for hf in (h0, h1):\n rel = _px_angle(x, args.width, hf) - _px_angle(args.sun_x, args.width, hf)\n bs += [(lo + rel) % 360, (hi + rel) % 360]\n print(f\" object at x={x:.0f}: bearing about {min(bs):.0f}°–{max(bs):.0f}° (relative to the sun \"\n f\"{_px_angle(x, args.width, h0) - _px_angle(args.sun_x, args.width, h0):+.1f}° to \"\n f\"{_px_angle(x, args.width, h1) - _px_angle(args.sun_x, args.width, h1):+.1f}°)\")\n print(\"When a range crosses north (e.g. 350°–10°), read it clockwise. If the field of view is uncertain, widen --hfov; when the sunrise and sunset groups can't be told apart, you must build top-down templates for both headings.\")\n\n\ndef _neg_coords(argv: list[str]) -> list[str]:\n \"\"\"argparse treats negative coordinates like -1.45,-48.5 as option names; a leading space makes them plain values (float ignores the space). Every southern- or western-hemisphere case needs this.\"\"\"\n return [\" \" + a if re.match(r\"^-\\d[\\d.]*(,-?[\\d.]+)+$\", a) else a for a in argv]\n\n\ndef main() -> None:\n ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)\n sub = ap.add_subparsers(dest=\"cmd\", required=True)\n\n p = sub.add_parser(\"pos\", help=\"sun position at a given place and time\")\n p.add_argument(\"--at\", type=_pair, required=True, help=\"lat,lon\")\n p.add_argument(\"--time\", required=True, help=\"2024-06-06T15:39, may include +08:00\")\n p.add_argument(\"--tz\", help=\"IANA time zone, e.g. Asia/Shanghai, Europe/Berlin\")\n\n r = sub.add_parser(\"ratio\", help=\"shadow ratio ↔ elevation angle\")\n g = r.add_mutually_exclusive_group(required=True)\n g.add_argument(\"--shadow\", type=float, help=\"shadow length for an object of height 1\")\n g.add_argument(\"--elev\", type=float)\n\n def targets(sp):\n sp.add_argument(\"--elev\", type=float, help=\"sun elevation angle °\")\n sp.add_argument(\"--ratio\", type=float, help=\"shadow ratio (shadow length for an object of height 1); either this or --elev\")\n sp.add_argument(\"--tol\", type=float, default=2.0, help=\"elevation angle tolerance °, default 2\")\n sp.add_argument(\"--shadow-bearing\", type=float, help=\"compass bearing ° the shadow points to (from the object to the shadow tip)\")\n sp.add_argument(\"--sun-azimuth\", type=float, help=\"sun azimuth °; either this or --shadow-bearing\")\n sp.add_argument(\"--az-tol\", type=float, default=10.0, help=\"azimuth tolerance °, default 10\")\n sp.add_argument(\"--tz\")\n\n lo = sub.add_parser(\"locate\", help=\"known time → find the band that matches the sun conditions\")\n lo.add_argument(\"--time\", required=True)\n targets(lo)\n lo.add_argument(\"--time-tol\", type=float, default=0, help=\"uncertainty of the capture time in minutes (±)\")\n lo.add_argument(\"--bbox\", type=lambda s: tuple(map(float, s.split(\",\"))), required=True, help=\"south,west,north,east\")\n lo.add_argument(\"--step\", type=float, default=0.1, help=\"grid step °, default 0.1 (about 11 km)\")\n lo.add_argument(\"--points\", type=Path, help=\"write matching points as {name:[lat,lon]}, for tiles.py mark\")\n lo.add_argument(\"--mosaic\", type=Path, help=\"base map from tiles.py fetch; the matching band is drawn on it\")\n lo.add_argument(\"--out\", type=Path)\n\n w = sub.add_parser(\"when\", help=\"known place → work back to capture time / date\")\n w.add_argument(\"--at\", type=_pair, required=True)\n dg = w.add_mutually_exclusive_group(required=True)\n dg.add_argument(\"--date\", help=\"single day 2024-10-01\")\n dg.add_argument(\"--dates\", help=\"date range 2024-01-01:2024-12-31\")\n targets(w)\n w.add_argument(\"--step-min\", type=int, default=2)\n\n d = sub.add_parser(\"dish\", help=\"satellite dish pointing / work back to longitude from the pointing\")\n d.add_argument(\"--sat\", type=float, default=92.2, help=\"satellite longitude, default ChinaSat-9 92.2\")\n d.add_argument(\"--at\", type=_pair, help=\"lat,lon: forward calculation\")\n d.add_argument(\"--lat\", type=float, help=\"work back: known latitude\")\n d.add_argument(\"--azimuth\", type=float, help=\"work back: dish azimuth\")\n d.add_argument(\"--az-tol\", type=float, default=5.0)\n\n st = sub.add_parser(\"street\", help=\"angle between shadow and street → street orientation candidates\")\n st.add_argument(\"--at\", type=_pair, required=True, help=\"any point in the city, lat,lon\")\n st.add_argument(\"--date\", required=True)\n st.add_argument(\"--shadow-rel\", type=float, required=True, help=\"angle in a top-down view, clockwise from the street direction to the shadow direction; 90 if the shadow is perpendicular to the street\")\n st.add_argument(\"--both\", action=\"store_true\", help=\"list both when you can't tell clockwise from counterclockwise\")\n st.add_argument(\"--elev\", type=float, help=\"measured sun elevation angle, used to keep only the matching times\")\n st.add_argument(\"--ratio\", type=float, help=\"shadow ratio; either this or --elev\")\n st.add_argument(\"--tol\", type=float, default=3.0)\n st.add_argument(\"--tz\")\n st.add_argument(\"--step-min\", type=int, default=10)\n\n fc = sub.add_parser(\"facing\", help=\"which walls are lit → camera heading range\")\n fc.add_argument(\"--at\", type=_pair, required=True)\n fc.add_argument(\"--time\", required=True)\n fc.add_argument(\"--tz\")\n fc.add_argument(\"--lit\", help=\"lit walls, comma-separated: camera,left,right,away\")\n fc.add_argument(\"--shaded\", help=\"shaded walls, same format\")\n fc.add_argument(\"--margin\", type=float, default=5, help=\"edge margin °, default 5\")\n\n cp = sub.add_parser(\"compass\", help=\"sun in the frame → camera heading, bearings of objects in the frame (lists sunrise and sunset groups when there is no time)\")\n cp.add_argument(\"--at\", type=_pair, required=True, help=\"any point in the candidate region, lat,lon\")\n cp.add_argument(\"--time\", help=\"clock time 07:40, or full 2024-09-20T07:40; if omitted, computes morning and evening groups for a low sun\")\n cp.add_argument(\"--dates\", help=\"date range 2024-09-01:2024-10-15; if omitted, uses the whole year\")\n cp.add_argument(\"--step-days\", type=int, default=3)\n cp.add_argument(\"--tz\")\n cp.add_argument(\"--elev-max\", type=float, default=12, help=\"when no time is given: upper limit of sun elevation ° (use 5–12 when the sun in the frame is close to the horizon)\")\n cp.add_argument(\"--elev\", help=\"sun elevation range ° lo:hi (estimate from how high the sun is above the horizon / tree line in the frame): with a time given, drops dates whose elevation doesn't fit; with no time, replaces 0–--elev-max\")\n cp.add_argument(\"--sun-x\", type=float, required=True, help=\"pixel column of the sun in the frame\")\n cp.add_argument(\"--width\", type=float, required=True, help=\"frame width (pixels)\")\n cp.add_argument(\"--hfov\", default=\"55:75\", help=\"horizontal field of view range °; phone main camera about 65–75 in landscape, about 50–60 in portrait, narrower for crops / zoom\")\n cp.add_argument(\"--x\", action=\"append\", help=\"pixel columns of objects to compute bearings for, comma-separated or repeated\")\n\n args = ap.parse_args(_neg_coords(sys.argv[1:]))\n {\"pos\": cmd_pos, \"ratio\": cmd_ratio, \"locate\": cmd_locate, \"when\": cmd_when, \"dish\": cmd_dish,\n \"street\": cmd_street, \"facing\": cmd_facing, \"compass\": cmd_compass}[args.cmd](args)\n\n\nif __name__ == \"__main__\":\n # Chinese Windows outputs GBK by default: it crashes on m² or ñ, and the Chinese the agent reads is garbled\n sys.stdout.reconfigure(encoding=\"utf-8\")\n sys.stderr.reconfigure(encoding=\"utf-8\", errors=\"backslashreplace\")\n main()\n","content_type":"text/x-python; charset=utf-8","language":"python","size":31171,"content_sha256":"c0e3292de8375abdc5038fc8b5cdaf6f7769ec9de7205d45a03814f377d8d15b"},{"filename":"scripts/terrain.py","content":"#!/usr/bin/env python3\n# /// script\n# requires-python = \">=3.10\"\n# dependencies = [\"pillow\", \"pillow-heif\", \"numpy\"]\n# ///\n\"\"\"Use elevation data to \"look\" out from a candidate camera position: synthesize a layered terrain view and skyline to compare ridgelines with the photo.\n\nWhen there is no street view, only distant mountains (group photos in the mountains, views across a river, aerial shots, a ridgeline outside a window), this replaces Google Earth oblique 3D.\nElevation comes from AWS Terrain Tiles (Terrarium encoding, ~30 m worldwide, no key).\n\n view render from one camera position by heading and field of view: shading layered by distance + red skyline + bearing ticks; can be stacked above/below the photo\n profile output the skyline elevation angle and distance at each bearing (JSON), for numeric comparison\n elev look up the ground elevation at a point\n scan along linear infrastructure (railway, road, power line), compute the 360° horizon point by point, keep the points that are \"flat nearby + a mountain + the mountain right next to a stretch of flat horizon\"\n and cluster them — when the frame shows only one piece of infrastructure and a mountain you can't recognize, use it to cut a whole large region down to a few hundred patches\n ridge sample the photo's ridgeline: per column, find the sky-to-mountain brightness jump; output a pixel point list + flat-horizon column range + horizon row + focal length (ridge.json)\n fit score each scan cluster against ridge.json: lay a camera-position grid in each cluster, search heading × focal length × horizon offset,\n skyline RMS + flat-horizon penalty (+ optional: near layer of a two-layer photo, heading perpendicular to a line, how far the linear\n infrastructure is at the frame's left/center/right), sort by score, --sheet draws overlays of the top N\n\nImportant: a matching skyline outline only shows that the camera position is near some sight line — moving a few hundred meters forward or back along the sight line barely changes the distant mountain outline.\nTo pin a point you need a second independent constraint (another near–far object alignment, a road or riverbank on the map). See references/geometry.md.\n\nExamples:\n terrain.py elev --at 35.3606,138.7274\n terrain.py view --at 35.4983,138.7688 --heading 194 --hfov 50 --range 30000 --out fuji.png # Mt. Fuji from Lake Kawaguchi\n terrain.py view --at \u003ccandidate camera position> --height 20 --heading \u003cheading> --hfov 65 --out v.png --photo photo.jpg\n terrain.py profile --at 35.4983,138.7688 --heading 194 --hfov 50 --out prof.json\n terrain.py scan --lines rail_bridges.geojson --bbox 35.1,138.3,36.0,139.2 --out hits.json # get the lines with osm.py geom first\n terrain.py ridge photo.jpg --x0 760 --x1 1260 --step 20 --flat 0:280 --out ridge.json --png ridge.png\n terrain.py ridge photo.jpg --points \"X,Y;X,Y;...\" --near \"X,Y;X,Y;...\" --near-clear X0:X1@ROW --hrow ROW --out ridge.json # two-layer\n terrain.py fit --hits hits.json --ridge ridge.json --out fit.json --sheet top.jpg # coarse search: per cluster 2 km / 250 m / z11\n terrain.py fit --at 35.4983,138.7688 --ridge ridge.json --radius 800 --grid 100 --zoom 13 --az-step 0.25 \\\\\n --line rail.geojson --line-dist 350-750:550-900:800-1700 --out fine.json # fine search + infrastructure distance constraint\n terrain.py fit --hits hits.json --ridge ridge.json --range 45000 --min-peak 2 --perp-line lines.geojson --out fit.json # far range + wires overhead\n\nFor the fields of ridge.json and fit.json, see each subcommand's --help.\n\"\"\"\nfrom __future__ import annotations\n\nimport argparse\nfrom _image import frame_of, open_image, open_photo\nfrom _net import curl_args, PROXY_HELP\nimport json\nimport math\nimport os\nimport re\nimport subprocess\nimport sys\nimport threading\nimport time\nfrom concurrent.futures import ThreadPoolExecutor\nfrom pathlib import Path\n\nimport numpy as np\nfrom PIL import Image, ImageDraw, ImageFont\n\nsys.path.insert(0, str(Path(__file__).parent))\nimport geo # noqa: E402\n\nTILE = \"https://s3.amazonaws.com/elevation-tiles-prod/terrarium/{z}/{x}/{y}.png\"\nR_EARTH = 6371008.8\nK_REFRACTION = 0.13\nF35_RANGE = (13.0, 135.0) # fit's focal ladder when the photo has no EXIF focal length: 0.5x ultra-wide to ~5x telephoto\nF35_RATIO = 1.18 # step between ladder rungs\n\n\nFETCH_TRIES = 4\nMISSING_HELP = (\"fill tiles that still fail after retries as missing (skipped in scan/fit, see-through in view/profile) instead of exiting; \"\n \"default: any failed tile is an error, because treating it as 0 m silently fakes sea-level terrain\")\n\n\ndef _decode(p: Path) -> np.ndarray | None:\n \"\"\"Decode a cached Terrarium tile; None if it is not a valid 256×256 image (truncated download, HTTP error body).\"\"\"\n try:\n with Image.open(p) as im:\n if im.size != (256, 256):\n return None\n a = np.asarray(im.convert(\"RGB\"), dtype=np.float32)\n except Exception: # noqa: BLE001\n return None\n return a[..., 0] * 256 + a[..., 1] + a[..., 2] / 256 - 32768\n\n\ndef _fetch(z: int, x: int, y: int, cache: Path, proxy: str | None) -> np.ndarray | None:\n \"\"\"Elevation of one tile, or None if it could not be downloaded after FETCH_TRIES attempts. Never caches a bad response.\"\"\"\n p = cache / f\"terrarium_{z}_{x}_{y}.png\"\n if p.exists():\n a = _decode(p)\n if a is not None:\n return a\n p.unlink(missing_ok=True) # bad file left by an older version: download again\n tmp = p.with_name(f\"{p.name}.{os.getpid()}.{threading.get_ident()}.part\")\n cmd = [\"curl\", \"-q\", \"-fsS\", \"-m\", \"60\", \"-o\", str(tmp), TILE.format(z=z, x=x, y=y)] + curl_args(proxy)\n for k in range(FETCH_TRIES):\n if k:\n time.sleep(2 ** (k - 1)) # 1, 2, 4 s backoff\n r = subprocess.run(cmd, check=False, capture_output=True)\n a = _decode(tmp) if r.returncode == 0 else None\n if a is not None:\n os.replace(tmp, p)\n return a\n tmp.unlink(missing_ok=True)\n return None\n\n\ndef _fetch_all(jobs: list[tuple[int, int]], zoom: int, cache: Path, proxy: str | None, threads: int, progress: bool = False):\n \"\"\"Yield ((x, y), array or None) for every tile; download failures come back as None.\"\"\"\n with ThreadPoolExecutor(threads) as ex:\n for i, item in enumerate(zip(jobs, ex.map(lambda t: _fetch(zoom, t[0], t[1], cache, proxy), jobs)), 1):\n if progress and i % 200 == 0:\n print(f\" tiles {i}/{len(jobs)}\", file=sys.stderr)\n yield item\n\n\ndef _report_missing(failed: list[tuple[int, int]], total: int, zoom: int, allow_missing: bool) -> None:\n \"\"\"Fail loudly on missing tiles; with --allow-missing only warn.\"\"\"\n if not failed:\n return\n eg = \", \".join(f\"{zoom}/{x}/{y}\" for x, y in failed[:5]) + (\" …\" if len(failed) > 5 else \"\")\n msg = f\"{len(failed)}/{total} elevation tiles failed to download after {FETCH_TRIES} tries ({eg})\"\n if not allow_missing:\n sys.exit(f\"{msg}. Good tiles are cached, so just re-run (lower --threads if it keeps failing, \"\n \"or check the network with doctor.py --network); --allow-missing proceeds without them\")\n print(f\"WARNING: {msg}; --allow-missing: continuing without them\", file=sys.stderr)\n\n\nclass DEM:\n \"\"\"Elevation mosaic indexed by Web Mercator pixels.\"\"\"\n\n def __init__(self, center: tuple[float, float], radius_m: float, zoom: int, cache: Path, proxy: str | None,\n allow_missing: bool = False):\n cache.mkdir(parents=True, exist_ok=True)\n self.z = zoom\n mpp = geo.meters_per_px(zoom, center[0])\n cx, cy = geo.ll2px(zoom, *center)\n r_px = radius_m / mpp + 256\n self.tx0, self.ty0 = int((cx - r_px) // 256), int((cy - r_px) // 256)\n tx1, ty1 = int((cx + r_px) // 256), int((cy + r_px) // 256)\n nx, ny = tx1 - self.tx0 + 1, ty1 - self.ty0 + 1\n if nx * ny > 400:\n sys.exit(f\"Area too large ({nx}x{ny} tiles); reduce --range or lower --zoom\")\n self.h = np.full((ny * 256, nx * 256), np.nan, dtype=np.float32) # missing tiles stay NaN, never 0 m\n jobs = [(x, y) for y in range(self.ty0, ty1 + 1) for x in range(self.tx0, tx1 + 1)]\n failed = []\n for (x, y), arr in _fetch_all(jobs, zoom, cache, proxy, 8):\n if arr is None:\n failed.append((x, y))\n continue\n self.h[(y - self.ty0) * 256:(y - self.ty0 + 1) * 256, (x - self.tx0) * 256:(x - self.tx0 + 1) * 256] = arr\n _report_missing(failed, len(jobs), zoom, allow_missing)\n\n def sample(self, lat: np.ndarray, lon: np.ndarray) -> np.ndarray:\n n = 256 * 2 ** self.z\n x = (lon + 180) / 360 * n - self.tx0 * 256\n y = (1 - np.arcsinh(np.tan(np.radians(lat))) / np.pi) / 2 * n - self.ty0 * 256\n x0, y0 = np.floor(x).astype(int), np.floor(y).astype(int)\n fx, fy = x - x0, y - y0\n H, W = self.h.shape\n x0, y0 = np.clip(x0, 0, W - 2), np.clip(y0, 0, H - 2)\n a, b = self.h[y0, x0], self.h[y0, x0 + 1]\n c, d = self.h[y0 + 1, x0], self.h[y0 + 1, x0 + 1]\n return (a * (1 - fx) + b * fx) * (1 - fy) + (c * (1 - fx) + d * fx) * fy\n\n\ndef _dest_np(lat0: float, lon0: float, brg: np.ndarray, dist: np.ndarray) -> tuple[np.ndarray, np.ndarray]:\n la1, lo1 = math.radians(lat0), math.radians(lon0)\n b = np.radians(brg)[:, None]\n d = (dist / R_EARTH)[None, :]\n la2 = np.arcsin(math.sin(la1) * np.cos(d) + math.cos(la1) * np.sin(d) * np.cos(b))\n lo2 = lo1 + np.arctan2(np.sin(b) * np.sin(d) * math.cos(la1), np.cos(d) - math.sin(la1) * np.sin(la2))\n return np.degrees(la2), np.degrees(lo2)\n\n\ndef cast(dem: DEM, at, eye_alt: float, azimuths: np.ndarray, rng: float, near: float = 40.0, n: int = 900, keep_nan: bool = False):\n \"\"\"Return (elevation angle ° of each sample in each column, sample distance m). n = samples per sight line (view/profile use 900; fit uses --nsamp).\n Samples on missing tiles (--allow-missing) become -90° so they never form a skyline; keep_nan=True leaves them NaN for the caller to detect.\"\"\"\n dist = near * (rng / near) ** (np.arange(n) / (n - 1)) # dense near, sparse far\n lat, lon = _dest_np(at[0], at[1], azimuths, dist)\n h = dem.sample(lat, lon)\n drop = dist ** 2 / (2 * R_EARTH) * (1 - K_REFRACTION)\n ang = np.degrees(np.arctan2(h - drop[None, :] - eye_alt, dist[None, :]))\n return (ang if keep_nan else np.nan_to_num(ang, nan=-90.0)), dist\n\n\ndef render(ang: np.ndarray, dist: np.ndarray, azimuths: np.ndarray, pitch: float, vfov: float, height: int) -> tuple[Image.Image, np.ndarray, np.ndarray]:\n W = ang.shape[0]\n img = np.zeros((height, W, 3), dtype=np.uint8)\n img[:] = (205, 225, 245) # sky\n top = pitch + vfov / 2\n\n def row(a):\n return (top - a) / vfov * (height - 1)\n\n sky_ang = np.full(W, -90.0)\n sky_dist = np.full(W, np.nan)\n runmax = np.maximum.accumulate(ang, axis=1)\n visible = ang >= np.concatenate([np.full((W, 1), -90.0), runmax[:, :-1]], axis=1)\n logd = np.log(dist / dist[0]) / np.log(dist[-1] / dist[0])\n for c in range(W):\n idx = np.nonzero(visible[c])[0]\n prev = row(-90.0)\n # near to far: each visible sample paints this column from \"its elevation angle\" to \"the previous highest point\", color from dark to light by distance\n lo_row = height\n for i in idx:\n r = row(ang[c, i])\n if r >= lo_row:\n continue\n shade = int(60 + 150 * logd[i])\n r0 = max(0, int(r))\n img[r0:lo_row, c] = (shade - 25, shade, shade - 35)\n lo_row = r0\n if idx.size:\n j = idx[-1]\n sky_ang[c], sky_dist[c] = ang[c, j], dist[j]\n rr = int(round(row(ang[c, j])))\n if 0 \u003c= rr \u003c height:\n img[max(0, rr - 1):rr + 2, c] = (220, 30, 30)\n _ = prev\n return Image.fromarray(img), sky_ang, sky_dist\n\n\ndef _font(size: int):\n for p in (\"/System/Library/Fonts/STHeiti Medium.ttc\", \"/System/Library/Fonts/Hiragino Sans GB.ttc\",\n \"/usr/share/fonts/opentype/noto/NotoSansCJK-Regular.ttc\"):\n try:\n return ImageFont.truetype(p, size)\n except Exception: # noqa: BLE001\n continue\n return ImageFont.load_default()\n\n\ndef annotate(im: Image.Image, azimuths: np.ndarray, pitch: float, vfov: float, title: str) -> Image.Image:\n W, H = im.size\n d = ImageDraw.Draw(im)\n f = _font(16)\n hr = (pitch + vfov / 2) / vfov * (H - 1)\n d.line([(0, hr), (W, hr)], fill=(90, 90, 200), width=1) # horizontal line at eye height\n names = {0: \"N\", 45: \"NE\", 90: \"E\", 135: \"SE\", 180: \"S\", 225: \"SW\", 270: \"W\", 315: \"NW\"}\n span = azimuths[-1] - azimuths[0]\n stepdeg = 5 if span \u003c= 60 else 10 if span \u003c= 150 else 30\n first = math.ceil(azimuths[0] / stepdeg) * stepdeg\n for a in np.arange(first, azimuths[-1] + 1e-6, stepdeg):\n x = (a - azimuths[0]) / span * (W - 1)\n d.line([(x, H - 14), (x, H)], fill=\"black\", width=1)\n lab = f\"{a % 360:.0f}°\" + (names.get(int(a % 360), \"\"))\n d.text((x + 3, H - 32), lab, fill=\"black\", font=f)\n d.rectangle([0, 0, W, 24], fill=(0, 0, 0))\n d.text((6, 3), title, fill=\"yellow\", font=f)\n return im\n\n\n\n# ---------------------------------------------------------------- scan ----\n# Linear infrastructure × terrain filtering. Method in references/corridors.md 4.3.\n# Difference from view/profile: this has to cover a province-sized region, far more tiles than the DEM class's 400-tile limit,\n# so it builds its own large mosaic filled only with \"the tiles actually needed\", sampled nearest-neighbor (one z10 cell is ~150 m; interpolation is pointless).\n\n\ndef _tile_xy(lat: float, lon: float, z: int) -> tuple[float, float]:\n n = 2 ** z\n x = (lon + 180) / 360 * n\n y = (1 - math.asinh(math.tan(math.radians(lat))) / math.pi) / 2 * n\n return x, y\n\n\ndef _haversine(a: tuple[float, float], b: tuple[float, float]) -> float:\n la1, lo1, la2, lo2 = map(math.radians, (a[0], a[1], b[0], b[1]))\n d = math.sin((la2 - la1) / 2) ** 2 + math.cos(la1) * math.cos(la2) * math.sin((lo2 - lo1) / 2) ** 2\n return 2 * R_EARTH * math.asin(math.sqrt(d))\n\n\nclass _Mosaic:\n \"\"\"A large mosaic stitched from a batch of Terrarium tiles, nearest-neighbor sampling. Tiles that failed to download hold MISSING.\"\"\"\n\n MISSING = -32768 # real elevations are clipped to -500…9000, so this never collides\n\n def __init__(self, need: set[tuple[int, int]], zoom: int, cache: Path, proxy: str | None, threads: int,\n allow_missing: bool = False):\n cache.mkdir(parents=True, exist_ok=True)\n self.z = zoom\n self.tx0 = min(t[0] for t in need)\n self.ty0 = min(t[1] for t in need)\n nx = max(t[0] for t in need) - self.tx0 + 1\n ny = max(t[1] for t in need) - self.ty0 + 1\n gb = nx * ny * 256 * 256 * 2 / 1e9\n print(f\"mosaic {nx}x{ny} tiles, {gb:.2f} GB memory\", file=sys.stderr)\n self.h = np.full((ny * 256, nx * 256), self.MISSING, dtype=np.int16)\n jobs = sorted(need)\n failed = []\n for (x, y), arr in _fetch_all(jobs, zoom, cache, proxy, threads, progress=True):\n if arr is None:\n failed.append((x, y))\n continue\n self.h[(y - self.ty0) * 256:(y - self.ty0 + 1) * 256,\n (x - self.tx0) * 256:(x - self.tx0 + 1) * 256] = np.clip(arr, -500, 9000).astype(np.int16)\n self.n_failed = len(failed)\n _report_missing(failed, len(jobs), zoom, allow_missing)\n\n def sample(self, lat: np.ndarray, lon: np.ndarray) -> np.ndarray:\n n = 256 * 2 ** self.z\n x = ((lon + 180) / 360 * n - self.tx0 * 256).astype(int)\n y = ((1 - np.arcsinh(np.tan(np.radians(lat))) / np.pi) / 2 * n - self.ty0 * 256).astype(int)\n x = np.clip(x, 0, self.h.shape[1] - 1)\n y = np.clip(y, 0, self.h.shape[0] - 1)\n return self.h[y, x].astype(np.float32)\n\n\ndef _line_points(gj: dict, step: float, skip: list[tuple[str, str]], bbox: tuple[float, float, float, float] | None):\n \"\"\"Take one point every step meters along each LineString; return [(lat, lon, properties)].\"\"\"\n pts = []\n n_line = n_skip = 0\n for f in gj.get(\"features\", []):\n g = f.get(\"geometry\") or {}\n if g.get(\"type\") != \"LineString\" or len(g.get(\"coordinates\", [])) \u003c 2:\n continue\n tags = f.get(\"properties\", {}) or {}\n n_line += 1\n if any(str(tags.get(k, \"\")) == v for k, v in skip):\n n_skip += 1\n continue\n c = [(y, x) for x, y in g[\"coordinates\"]] # geojson is lon,lat\n if bbox and not (min(p[0] for p in c) \u003c= bbox[2] and max(p[0] for p in c) >= bbox[0]\n and min(p[1] for p in c) \u003c= bbox[3] and max(p[1] for p in c) >= bbox[1]):\n continue\n got = 0\n acc = 0.0\n for i in range(1, len(c)):\n d = _haversine(c[i - 1], c[i])\n for k in range(int((acc + d) // step)):\n t = (step * (k + 1) - acc) / d if d else 0\n pts.append((c[i - 1][0] + (c[i][0] - c[i - 1][0]) * t,\n c[i - 1][1] + (c[i][1] - c[i - 1][1]) * t, tags))\n got += 1\n acc = (acc + d) % step\n # lines shorter than step overall, or whose ends nearly coincide (loops), get one midpoint added so the whole line isn't missed\n if got == 0 or _haversine(c[0], c[-1]) \u003c step:\n pts.append((c[len(c) // 2][0], c[len(c) // 2][1], tags))\n if bbox:\n s, w, n, e = bbox\n pts = [p for p in pts if s \u003c= p[0] \u003c= n and w \u003c= p[1] \u003c= e]\n print(f\"{n_line} lines ({n_skip} skipped by --skip-tag), {len(pts)} sample points\", file=sys.stderr)\n return pts\n\n\ndef _cluster(hits: list[dict], km: float) -> list[dict]:\n \"\"\"Greedy clustering by max_ang from largest to smallest: the steepest point represents the cluster, points within km join it, and the representative gets\n n = number of points in the cluster and extent_m = distance to its farthest member (fit's camera grid has to reach that far to cover the cluster).\"\"\"\n order = sorted(hits, key=lambda h: -h[\"max_ang\"])\n used = [False] * len(order)\n out = []\n for i, p in enumerate(order):\n if used[i]:\n continue\n n = 0\n extent = 0.0\n for j in range(i, len(order)):\n if used[j]:\n continue\n d = _haversine((p[\"lat\"], p[\"lon\"]), (order[j][\"lat\"], order[j][\"lon\"]))\n if d \u003c= km * 1000:\n used[j] = True\n n += 1\n extent = max(extent, d)\n out.append({**p, \"n\": n, \"extent_m\": round(extent)})\n return out\n\n\ndef _cmd_scan(args) -> None:\n az_step = args.az_step\n naz = int(round(360 / az_step))\n if abs(naz * az_step - 360) > 1e-6:\n sys.exit(\"--az-step must divide 360 evenly\")\n if args.step \u003c= 0 or args.near_step \u003c= 0 or args.near_radius \u003c 0:\n sys.exit(\"--step / --near-step must be greater than 0, --near-radius must not be negative\")\n dist = np.array([float(x) for x in args.dist.split(\",\")], dtype=float)\n near = np.arange(0, args.near_radius + 1e-6, args.near_step, dtype=float)\n skip = []\n for s in (args.skip_tag or [\"electrified=no\"]):\n if s.lower() in (\"none\", \"-\"):\n continue\n if \"=\" not in s:\n sys.exit(f\"--skip-tag must be written as key=value, got {s!r}\")\n k, v = s.split(\"=\", 1)\n skip.append((k, v))\n bbox = None\n if args.bbox:\n b = [float(x) for x in args.bbox.split(\",\")]\n if len(b) != 4:\n sys.exit(\"--bbox needs 4 numbers: s,w,n,e\")\n bbox = (min(b[0], b[2]), min(b[1], b[3]), max(b[0], b[2]), max(b[1], b[3]))\n\n gj = json.loads(Path(args.lines).read_text(encoding=\"utf-8\"))\n pts = _line_points(gj, args.step, skip, bbox)\n if not pts:\n sys.exit(\"0 sample points: --bbox encloses nothing, or the geojson has no LineString\")\n\n reach = float(dist.max()) + 500 # half a kilometer extra, enough even when a sample point falls on a tile edge\n need: set[tuple[int, int]] = set()\n for lat, lon, _ in pts:\n x, y = _tile_xy(lat, lon, args.zoom)\n r = reach / (40075016.7 * math.cos(math.radians(lat)) / 2 ** args.zoom)\n for tx in range(int(x - r), int(x + r) + 1):\n for ty in range(int(y - r), int(y + r) + 1):\n need.add((tx, ty))\n print(f\"need {len(need)} z{args.zoom} tiles\", file=sys.stderr)\n if len(need) > args.max_tiles:\n sys.exit(f\"{len(need)} tiles > --max-tiles {args.max_tiles}: shrink --bbox, lower --zoom, or run in chunks\")\n mos = _Mosaic(need, args.zoom, args.cache, args.proxy, args.threads, args.allow_missing)\n\n az = np.arange(0, 360, az_step)\n min_low = int(round((args.min_low_deg if args.min_low_deg is not None else args.flat_run) / az_step))\n need_run = int(round(args.flat_run / az_step))\n cap = int(round(args.flat_run_cap / az_step))\n hits = []\n n_missing = 0\n for idx, (lat, lon, tags) in enumerate(pts):\n if idx and idx % 2000 == 0:\n print(f\" scanned {idx}/{len(pts)}, hits {len(hits)}\", file=sys.stderr)\n ln, lo = _dest_np(lat, lon, az, near)\n hn = mos.sample(ln, lo)\n if (hn == mos.MISSING).any():\n n_missing += 1\n continue\n if hn.max() - hn.min() > args.near_flat: # not flat nearby: a valley or hillside, not the flat ground seen in the frame\n continue\n h0 = float(np.median(hn))\n la, lo = _dest_np(lat, lon, az, dist)\n h = mos.sample(la, lo)\n if (h == mos.MISSING).any(): # a hole would read as a deep valley or flat horizon\n n_missing += 1\n continue\n ang = np.degrees(np.arctan2(h - h0 - args.eye, dist[None, :])) # within a few km earth curvature is \u003c5 m, negligible next to z10 error\n hor = ang.max(axis=1) # horizon elevation angle at each bearing\n rel = (h - h0).max(axis=1)\n mt = hor >= args.min_peak\n low = hor \u003c args.max_low\n if not mt.any() or low.sum() \u003c min_low:\n continue\n best = 0 # how long the flat horizon right next to the mountain is\n for i in range(naz):\n if not mt[i]:\n continue\n for sgn in (-1, 1):\n k = run = 0\n while k \u003c cap:\n jj = (i + sgn * (k + 1)) % naz\n if low[jj]:\n run += 1\n elif run == 0 and k \u003c 2: # allow a 2-cell transition between the mountain foot and flat ground\n pass\n else:\n break\n k += 1\n best = max(best, run)\n if best \u003c need_run:\n continue\n i = int(np.argmax(hor))\n hits.append({\"lat\": round(lat, 5), \"lon\": round(lon, 5), \"h0\": round(h0), \"max_ang\": round(float(hor[i]), 1),\n \"az\": float(az[i]), \"relief\": round(float(rel[i])), \"flat_run_deg\": best * az_step,\n \"name\": tags.get(\"name\", \"\"), \"hs\": tags.get(\"highspeed\", \"\"),\n \"elec\": tags.get(\"electrified\", \"\"), \"id\": tags.get(\"id\", tags.get(\"@id\", \"\"))})\n\n clusters = _cluster(hits, args.cluster_km)\n print(f\"{len(hits)} hit points → {len(clusters)} clusters\", file=sys.stderr)\n if n_missing:\n print(f\"WARNING: {n_missing}/{len(pts)} sample points not scanned because they touch missing tiles\", file=sys.stderr)\n out = {\n \"params\": {\"lines\": str(args.lines), \"bbox\": list(bbox) if bbox else None, \"step_m\": args.step,\n \"zoom\": args.zoom, \"near_flat_m\": args.near_flat, \"near_radius_m\": args.near_radius,\n \"min_peak_deg\": args.min_peak, \"max_low_deg\": args.max_low, \"flat_run_deg\": args.flat_run,\n \"min_low_deg\": args.min_low_deg if args.min_low_deg is not None else args.flat_run,\n \"eye_m\": args.eye, \"az_step_deg\": az_step, \"dist_m\": dist.tolist(),\n \"skip_tag\": [f\"{k}={v}\" for k, v in skip], \"cluster_km\": args.cluster_km},\n \"n_samples\": len(pts), \"n_hits\": len(hits), \"n_clusters\": len(clusters),\n \"n_failed_tiles\": mos.n_failed, \"n_samples_missing_dem\": n_missing,\n \"hits\": hits, \"clusters\": clusters,\n }\n Path(args.out).write_text(json.dumps(out, ensure_ascii=False, indent=0), encoding=\"utf-8\")\n print(f\"→ {args.out}\", file=sys.stderr)\n if args.clusters_out:\n Path(args.clusters_out).write_text(json.dumps(clusters, ensure_ascii=False, indent=0), encoding=\"utf-8\")\n print(f\"→ {args.clusters_out} (cluster list only)\", file=sys.stderr)\n\n\n# ---------------------------------------------------------------- ridge ----\n# Photo ridgeline sampling. Method in references/geometry.md 7.4: per column, top to bottom, find the sky→mountain brightness jump.\n\n\ndef _focal_px(photo: Path, w: int, h: int, f0_arg: float | None, f35_default: float):\n \"\"\"Photo focal length (pixels). Priority: --f0 > EXIF 35mm-equivalent focal length > assumed from --f35.\n The 35mm equivalent converts via the diagonal: f_px = f35 / 43.27 × diagonal pixels; this holds even if the photo was resized.\n Return (f_px, source, f35).\"\"\"\n diag = math.hypot(w, h)\n if f0_arg:\n return float(f0_arg), \"arg\", round(f0_arg * 43.2666 / diag, 1)\n try:\n ifd = open_image(photo, load=False).getexif().get_ifd(0x8769)\n f35 = ifd.get(0xA405) # FocalLengthIn35mmFilm\n if f35 and float(f35) > 0:\n return float(f35) / 43.2666 * diag, \"exif\", float(f35)\n except Exception: # noqa: BLE001\n pass\n return f35_default / 43.2666 * diag, \"assumed\", f35_default\n\n\ndef _parse_pts(spec: str, name: str, W: int, H: int) -> list[list[int]]:\n \"\"\"Pixel points as \"x,y;x,y;...\" or a JSON file holding [[x,y],...].\"\"\"\n p = Path(spec)\n if p.suffix == \".json\" and p.exists():\n raw = json.loads(p.read_text(encoding=\"utf-8\"))\n pts = raw.get(\"ridge\", raw) if isinstance(raw, dict) else raw\n else:\n try:\n pts = [[float(v) for v in part.split(\",\")] for part in spec.split(\";\") if part.strip()]\n except ValueError:\n sys.exit(f\"{name} must be x,y;x,y;... or a .json file of [[x,y],...], got {spec!r}\")\n out = []\n for q in pts:\n if len(q) != 2 or not (0 \u003c= q[0] \u003c W and 0 \u003c= q[1] \u003c H):\n sys.exit(f\"{name}: point {q} is not x,y inside the {W}x{H} photo\")\n out.append([int(round(q[0])), int(round(q[1]))])\n if len(out) \u003c 3:\n sys.exit(f\"{name} needs at least 3 points, got {len(out)}\")\n return sorted(out)\n\n\ndef _sky_edge(prof: np.ndarray, ymin: int, ymax: int, k: int, drop: float, hold: int) -> int | None:\n \"\"\"In one column's brightness profile, top to bottom, find the first row where \"the mean of the k rows above exceeds the mean of the k rows below by drop, and the hold rows further down are still dark\",\n then take the maximum-gradient row within ±k rows of it. Return the row index of the mountain's first row; None if not found.\n hold is there to skip things only a few pixels tall vertically, such as wires and antennas.\"\"\"\n n = len(prof)\n cs = np.concatenate([[0.0], np.cumsum(prof, dtype=np.float64)])\n y = np.arange(max(ymin, k), min(ymax, n - k - hold))\n if y.size == 0:\n return None\n above = (cs[y] - cs[y - k]) / k\n below = (cs[y + k] - cs[y]) / k\n held = (cs[y + k + hold] - cs[y + k]) / hold\n ok = (above - below >= drop) & (above - held >= drop)\n if not ok.any():\n return None\n y0 = int(y[np.argmax(ok)])\n yy = np.arange(max(k, y0 - k), min(n - k, y0 + k + 1))\n g = (cs[yy] - cs[yy - k]) / k - (cs[yy + k] - cs[yy]) / k\n return int(yy[np.argmax(g)])\n\n\ndef _cmd_ridge(args) -> None:\n im = open_photo(args.photo, \"photo\").convert(\"RGB\")\n W, H = im.size\n a = np.asarray(im, dtype=np.float32)\n L = a[..., 0] * 0.299 + a[..., 1] * 0.587 + a[..., 2] * 0.114\n if args.x1 is None:\n args.x1 = W - 1\n if not (0 \u003c= args.x0 \u003c args.x1 \u003c W):\n sys.exit(f\"--x0/--x1 must satisfy 0 ≤ x0 \u003c x1 \u003c width {W}\")\n if args.step \u003c= 0:\n sys.exit(\"--step must be greater than 0\")\n ymin = args.ymin if args.ymin is not None else 0\n ymax = args.ymax if args.ymax is not None else H\n f0, f_src, f35 = _focal_px(args.photo, W, H, args.f0, args.f35)\n\n def edge(x: int) -> int | None:\n x0, x1 = max(0, x - args.halfw), min(W, x + args.halfw + 1)\n return _sky_edge(L[:, x0:x1].mean(axis=1), ymin, ymax, args.k, args.drop, args.hold)\n\n ridge, missing = [], []\n if args.points: # hand-picked skyline replaces detection (cloud on the peaks, haze, snow against sky)\n ridge = _parse_pts(args.points, \"--points\", W, H)\n else:\n for x in range(args.x0, args.x1 + 1, args.step):\n y = edge(x)\n (ridge.append([x, y]) if y is not None else missing.append(x))\n near = _parse_pts(args.near, \"--near\", W, H) if args.near else []\n near_clear = []\n for spec in args.near_clear or []:\n m = re.match(r\"^\\s*(\\d+):(\\d+)@(\\d+(?:\\.\\d+)?)\\s*$\", spec)\n if not m or not 0 \u003c= int(m[1]) \u003c int(m[2]) \u003c W:\n sys.exit(f\"--near-clear must be x0:x1@row with 0 ≤ x0 \u003c x1 \u003c width {W}, got {spec!r}\")\n near_clear.append([int(m[1]), int(m[2]), float(m[3])])\n flat = None\n flat_rows: list[int] = []\n flat_pts: list[list[int]] = [] # for drawing the check image: flat_rows has only row indices and omits columns with no jump found, so index order can't map back to columns\n if args.flat:\n b = args.flat.split(\":\")\n if len(b) != 2:\n sys.exit(\"--flat must be written as x0:x1\")\n flat = [int(b[0]), int(b[1])]\n if not (0 \u003c= flat[0] \u003c flat[1] \u003c W):\n sys.exit(f\"--flat must satisfy 0 ≤ x0 \u003c x1 \u003c width {W}\")\n for x in range(flat[0], flat[1] + 1, args.step):\n y = edge(x)\n if y is not None:\n flat_rows.append(y)\n flat_pts.append([x, y])\n if args.hrow is not None:\n hrow, h_src = float(args.hrow), \"arg\"\n elif flat_rows:\n hrow, h_src = float(np.median(flat_rows)), \"flat\"\n else:\n hrow, h_src = H / 2, \"center\"\n if len(ridge) \u003c 3:\n print(f\"warning: only {len(ridge)} ridge points found ({len(missing)} columns missing), fit needs at least 3; adjust --drop/--ymin/--ymax or check --png\", file=sys.stderr)\n out = {\n \"photo\": str(args.photo), \"image_size\": [W, H], \"frame\": \"upright\", \"cx\": W / 2,\n \"ridge\": ridge, \"missing_cols\": missing, \"ridge_source\": \"points\" if args.points else \"detected\",\n \"near\": near, \"near_clear\": near_clear,\n \"flat\": flat, \"flat_rows\": flat_rows,\n \"hrow\": round(hrow, 1), \"hrow_source\": h_src,\n \"f0\": round(f0, 1), \"f0_source\": f_src, \"f35_equiv\": f35,\n \"params\": {\"x0\": args.x0, \"x1\": args.x1, \"step\": args.step, \"ymin\": ymin, \"ymax\": ymax,\n \"drop\": args.drop, \"k\": args.k, \"hold\": args.hold, \"halfw\": args.halfw},\n }\n Path(args.out).write_text(json.dumps(out, ensure_ascii=False), encoding=\"utf-8\")\n src_txt = {\"arg\": \"--f0\", \"exif\": \"EXIF 35mm equivalent\",\n \"assumed\": f\"no EXIF, assumed {f35} mm equivalent; fit tries {F35_RANGE[0]:g}–{F35_RANGE[1]:g} mm by default\"}[f_src]\n h_txt = {\"arg\": \"--hrow\", \"flat\": f\"median sky→ground row in the --flat columns ({len(flat_rows)} columns)\", \"center\": \"no --flat given, frame midline\"}[h_src]\n print(f\"{W}x{H} ridge {len(ridge)} points ({len(missing)} columns missing) flat-horizon columns {flat} \"\n f\"hrow {hrow:.0f} ({h_txt}) f0 {f0:.0f} px ({src_txt}, horizontal FOV {2 * math.degrees(math.atan(W / 2 / f0)):.1f}°)\"\n + (f\" near layer {len(near)} points, {len(near_clear)} clear ranges\" if near or near_clear else \"\") + f\" → {args.out}\")\n if args.png:\n d = ImageDraw.Draw(im)\n d.line([(0, hrow), (W, hrow)], fill=(80, 80, 255), width=1)\n if flat:\n d.line([(flat[0], hrow), (flat[1], hrow)], fill=(0, 200, 255), width=4)\n for x, y in flat_pts:\n d.ellipse([x - 3, y - 3, x + 3, y + 3], outline=(0, 200, 255), width=2)\n for x, y in ridge:\n d.ellipse([x - 4, y - 4, x + 4, y + 4], fill=(255, 220, 0))\n for x in missing:\n d.line([(x, ymin), (x, ymax - 1)], fill=(255, 0, 0), width=1)\n for x, y in near:\n d.ellipse([x - 4, y - 4, x + 4, y + 4], fill=(255, 120, 0))\n for x0, x1, row in near_clear:\n d.line([(x0, row), (x1, row)], fill=(255, 120, 0), width=3)\n d.text((8, 8), f\"ridge {len(ridge)} pts hrow {hrow:.0f} ({h_src}) f0 {f0:.0f}px ({f_src})\", fill=\"yellow\", font=_font(20))\n im.save(args.png, quality=90)\n print(f\"check image → {args.png} (yellow dots = ridge points, red vertical lines = columns not found, blue line = hrow, cyan line = flat-horizon column range, \"\n \"orange dots = near-layer points, orange lines = near-clear ranges)\")\n\n\n# ------------------------------------------------------------------ fit ----\n# Batch skyline scoring. Method in references/geometry.md 7.4; infrastructure distance constraint in corridors.md 4.3.\n# For each candidate camera position compute a full 360° horizon hor(az); convert photo ridge points to (bearing offset, elevation angle) and compare against all headings at once:\n# rms = RMS of the ridge elevation-angle residuals (the residual median serves as horizon offset cc, clamped to ±cc_max)\n# flatpen = mean of max(0, hor - cc - flat_clear) over the flat-horizon columns\n# score = rms + flat_w × flatpen\n# With --line, also add: for each heading, the nearest distance dL/dC/dR of infrastructure sample points inside the frame's left/center/right bearing windows,\n# line_pen = meters each falls outside its --line-dist interval / --line-scale (+ penalty 1 for wrong order)\n# total = score + line_w × line_pen; without --line, total = score.\n# Two-layer photos (ridge.json has near / near_clear): near(az) = highest angle within D1 m, D1 tried from --near-dist; near points join\n# the far ridge points in the same rms (one cc and roll for both), near_clear columns add nclearpen = mean of max(0, near − row angle).\n# --perp-line keeps only headings within --perp-tol of perpendicular to the nearest line (wires crossing the frame almost level).\n\n\ndef _parse_pairs(s: str, name: str, n: int = 3, sep: str = \",\", rng: str = \":\") -> list[tuple[float, float]]:\n out = []\n for part in s.split(sep):\n b = part.split(rng)\n if len(b) != 2:\n sys.exit(f\"{name}: each segment must be written as a{rng}b, got {part!r}\")\n out.append((float(b[0]), float(b[1])))\n if len(out) != n:\n sys.exit(f\"{name} needs {n} segments, got {len(out)}\")\n return out\n\n\ndef _load_centers(args) -> list[dict]:\n if args.at:\n lat, lon = map(float, args.at.split(\",\"))\n return [{\"lat\": lat, \"lon\": lon, \"name\": \"\", \"src_idx\": 0}]\n j = json.loads(Path(args.hits).read_text(encoding=\"utf-8\"))\n items = j[\"clusters\"] if isinstance(j, dict) and \"clusters\" in j else j\n if not isinstance(items, list):\n sys.exit(\"--hits must be scan output (with a clusters field) or a list of clusters/points\")\n centers = []\n for i, c in enumerate(items):\n if \"lat\" not in c or \"lon\" not in c:\n sys.exit(f\"--hits item {i} has no lat/lon\")\n centers.append({**c, \"src_idx\": c.get(\"src_idx\", i)})\n if args.select:\n want = [int(x) for x in args.select.split(\",\")]\n bad = [i for i in want if not 0 \u003c= i \u003c len(centers)]\n if bad:\n sys.exit(f\"--select indices {bad} out of range 0–{len(centers) - 1}\")\n centers = [centers[i] for i in want]\n return centers\n\n\ndef _skyline_xy(hor: np.ndarray, az: np.ndarray, H: float, f: float, cc: float, cx: float, hrow: float, half_w: float,\n roll: float = 0.0):\n \"\"\"Project the 360° horizon onto photo pixels with the pinhole model fit uses: x = cx + f·tan(bearing offset), y = hrow − f·tan(elevation angle − cc).\n roll = frame roll angle ° (clockwise positive); pass the value fit solved for, otherwise both ends will be off from the photo by a dozen-plus pixels.\"\"\"\n rel = (az - H + 180) % 360 - 180\n lim = math.degrees(math.atan(half_w / f)) + 1\n m = np.abs(rel) \u003c= lim\n x = cx + f * np.tan(np.radians(rel[m]))\n y = hrow - f * np.tan(np.radians(hor[m] - cc)) + math.tan(math.radians(roll)) * (x - cx)\n o = np.argsort(x)\n return list(zip(x[o].tolist(), y[o].tolist()))\n\n\ndef _fit_sheet(recs: list[dict], ridge: dict, photo: Path | None, out: Path, args, az: np.ndarray, dist_n: int) -> None:\n W, H = ridge[\"image_size\"]\n tw = args.sheet_width\n s = tw / W\n th = int(H * s)\n base = None\n if photo and photo.exists():\n base = open_photo(photo, \"photo\", frame=frame_of(ridge), size=[W, H]).convert(\"RGB\").resize((tw, th))\n cols = max(1, args.sheet_cols)\n rows = math.ceil(len(recs) / cols)\n sheet = Image.new(\"RGB\", (tw * cols, th * rows), \"white\")\n font = _font(13)\n for i, r in enumerate(recs):\n tile = base.copy() if base else Image.new(\"RGB\", (tw, th), (120, 120, 120))\n d = ImageDraw.Draw(tile)\n hrow, f, cx = ridge[\"hrow\"], r[\"f\"], ridge.get(\"cx\", W / 2)\n d.line([(0, hrow * s), (tw, hrow * s)], fill=(80, 80, 255), width=1)\n if ridge.get(\"flat\"):\n d.line([(ridge[\"flat\"][0] * s, hrow * s), (ridge[\"flat\"][1] * s, hrow * s)], fill=(0, 200, 255), width=3)\n try:\n dem = DEM(tuple(r[\"cam\"]), args.range + 500, args.zoom, args.cache, args.proxy, args.allow_missing)\n ang, dist_v = cast(dem, tuple(r[\"cam\"]), r[\"g\"] + args.eye, az, args.range, near=args.near, n=dist_n)\n pts = _skyline_xy(ang.max(axis=1), az, r[\"H\"], f, r[\"cc\"], cx, hrow, W / 2, r.get(\"roll\", 0.0))\n d.line([(x * s, y * s) for x, y in pts], fill=(255, 40, 40), width=2)\n if r.get(\"D1\"):\n pts = _skyline_xy(ang[:, dist_v \u003c= r[\"D1\"]].max(axis=1), az, r[\"H\"], f, r[\"cc\"], cx, hrow, W / 2, r.get(\"roll\", 0.0))\n d.line([(x * s, y * s) for x, y in pts], fill=(255, 140, 0), width=2)\n except SystemExit as e:\n d.text((6, th - 20), f\"DEM failed: {e}\", fill=\"red\", font=font)\n for x, y in ridge[\"ridge\"]:\n d.ellipse([x * s - 2, y * s - 2, x * s + 2, y * s + 2], fill=(255, 220, 0))\n for x, y in ridge.get(\"near\") or []:\n d.ellipse([x * s - 2, y * s - 2, x * s + 2, y * s + 2], fill=(0, 220, 255))\n for x0, x1, row in ridge.get(\"near_clear\") or []:\n d.line([(x0 * s, row * s), (x1 * s, row * s)], fill=(0, 220, 255), width=2)\n line_txt = f\" L/C/R {r['dL']:.0f}/{r['dC']:.0f}/{r['dR']:.0f} pen {r['line_pen']:.2f}\" if \"line_pen\" in r else \"\"\n txt = (f\"#{i + 1} hit{r['hit']} {r['name'] or '-'} {r['cam'][0]:.5f},{r['cam'][1]:.5f}\\n\"\n f\"H {r['H']:.1f} f {f:.0f} ({r.get('f35', 0):.0f}mm) cc {r['cc']:+.2f} rms {r['rms']:.3f} flat {r['flatpen']:.3f}{line_txt}\\n\"\n f\"total {r['total']:.3f}\")\n d.rectangle([0, 0, tw, 46], fill=(0, 0, 0))\n d.text((4, 2), txt, fill=\"yellow\", font=font)\n sheet.paste(tile, ((i % cols) * tw, (i // cols) * th))\n sheet.save(out, quality=88)\n print(f\"overlay → {out} (red line = synthetic skyline, orange line = synthetic near layer, yellow dots = photo ridge points, \"\n \"cyan dots/lines = photo near points / near-clear rows, blue line = hrow, cyan line on hrow = flat-horizon columns)\", file=sys.stderr)\n\n\ndef _cmd_fit(args) -> None:\n if bool(args.hits) == bool(args.at):\n sys.exit(\"give exactly one of --hits and --at\")\n ridge = json.loads(Path(args.ridge).read_text(encoding=\"utf-8\"))\n RX = np.array([p[0] for p in ridge[\"ridge\"]], float)\n RY = np.array([p[1] for p in ridge[\"ridge\"]], float)\n if RX.size \u003c 3:\n sys.exit(f\"ridge.json has only {RX.size} ridge points, needs at least 3\")\n W, Hh = ridge[\"image_size\"]\n hrow, f0, cx = float(ridge[\"hrow\"]), float(ridge[\"f0\"]), float(ridge.get(\"cx\", W / 2))\n flat = ridge.get(\"flat\")\n FLX = np.arange(flat[0], flat[1] + 1e-6, args.flat_step, dtype=float) if flat else np.zeros(0)\n NX = np.array([p[0] for p in ridge.get(\"near\") or []], float)\n NY = np.array([p[1] for p in ridge.get(\"near\") or []], float)\n ncl = ridge.get(\"near_clear\") or []\n NCX = np.concatenate([np.arange(a, b + 1e-6, args.flat_step) for a, b, _ in ncl]) if ncl else np.zeros(0)\n NCR = np.concatenate([np.full(len(np.arange(a, b + 1e-6, args.flat_step)), r) for a, b, r in ncl]) if ncl else np.zeros(0)\n two = bool(NX.size or NCX.size)\n near_dists = [float(x) for x in args.near_dist.split(\",\")] if two else [None]\n if two:\n dropped = [d_ for d_ in near_dists if d_ \u003c= args.near]\n near_dists = [d_ for d_ in near_dists if d_ > args.near]\n if not near_dists:\n sys.exit(f\"every --near-dist value ({args.near_dist}) is within --near {args.near:g} m, where sight lines start: \"\n \"give near distances beyond --near\")\n if dropped:\n print(f\"note: --near-dist {', '.join(f'{d_:g}' for d_ in dropped)} m dropped, within --near {args.near:g} m\", file=sys.stderr)\n diag = math.hypot(W, Hh)\n f35_0 = f0 * 43.2666 / diag\n f_src = ridge.get(\"f0_source\", \"assumed\")\n if args.focal_scales != \"auto\" and args.f35_range:\n sys.exit(\"give --focal-scales or --f35-range, not both\")\n if args.focal_scales != \"auto\":\n fscales = [float(x) for x in args.focal_scales.split(\",\")]\n focal_how = \"--focal-scales\"\n elif args.f35_range or f_src == \"assumed\":\n try:\n lo, hi = (float(x) for x in args.f35_range.split(\":\")) if args.f35_range else F35_RANGE\n except ValueError:\n sys.exit(f\"--f35-range must be lo:hi in mm, got {args.f35_range!r}\")\n if not 0 \u003c lo \u003c hi:\n sys.exit(\"--f35-range needs 0 \u003c lo \u003c hi\")\n n = max(2, math.ceil(math.log(hi / lo) / math.log(F35_RATIO)) + 1)\n fscales = [round(lo * (hi / lo) ** (i / (n - 1)) / f35_0, 4) for i in range(n)]\n focal_how = f\"{lo:g}–{hi:g} mm equivalent, {n} steps\"\n else:\n fscales = [0.9, 1.0, 1.12]\n focal_how = f\"focal from {'EXIF' if f_src == 'exif' else '--f0'} × 0.9–1.12\"\n px_metric = args.focal_metric == \"px\" or (args.focal_metric == \"auto\" and focal_how.endswith(\"steps\"))\n print(f\"focal ladder: {focal_how}, compared in {'pixels' if px_metric else 'degrees'} → \" + \", \".join(f\"{f35_0 * x:.0f}\" for x in fscales) + \" mm\"\n + (f\"; two-layer: {NX.size} near points, {NCX.size} near-clear columns, near distance {args.near_dist} m\" if two else \"\"),\n file=sys.stderr)\n if args.grid \u003c= 0 or args.radius \u003c 0 or args.nsamp \u003c 2 or args.near \u003c= 0 or args.range \u003c= args.near:\n sys.exit(\"need --grid > 0, --radius ≥ 0, --nsamp ≥ 2, 0 \u003c --near \u003c --range\")\n az_step = args.az_step\n naz = int(round(360 / az_step))\n if abs(naz * az_step - 360) > 1e-6:\n sys.exit(\"--az-step must divide 360 evenly\")\n if not 0 \u003c= args.roll_max \u003c 15:\n sys.exit(\"--roll-max must be between 0–15° (0 = don't solve roll, same as the old version)\")\n roll_max = math.tan(math.radians(args.roll_max))\n AZ = np.arange(naz) * az_step\n HS = np.arange(naz)\n\n centers = _load_centers(args)\n if not centers:\n sys.exit(\"no candidate clusters\")\n\n PS = None # (n, 4) segments lat1, lon1, lat2, lon2 for the heading constraint\n if args.perp_line:\n segs = []\n for f_ in json.loads(Path(args.perp_line).read_text(encoding=\"utf-8\")).get(\"features\", []):\n g_ = f_.get(\"geometry\") or {}\n parts = [g_[\"coordinates\"]] if g_.get(\"type\") == \"LineString\" else g_.get(\"coordinates\", []) if g_.get(\"type\") == \"MultiLineString\" else []\n for c_ in parts:\n segs += [(y1, x1, y2, x2) for (x1, y1, *_), (x2, y2, *_) in zip(c_, c_[1:])]\n if not segs:\n sys.exit(f\"--perp-line {args.perp_line}: no LineString in it\")\n PS = np.array(segs, float)\n\n # linear infrastructure sample points (optional)\n LP = None\n if args.line:\n if not args.line_dist:\n sys.exit(\"--line requires --line-dist L:C:R (three distance intervals, e.g. 350-750:550-900:800-1700)\")\n wins = _parse_pairs(args.line_win, \"--line-win\", 3, \",\", \":\")\n dwin = _parse_pairs(args.line_dist, \"--line-dist\", 3, \":\", \"-\")\n scales = [float(x) for x in args.line_scale.split(\",\")]\n if len(scales) != 3 or min(scales) \u003c= 0:\n sys.exit(\"--line-scale needs 3 positive numbers, comma-separated\")\n skip = []\n for s_ in (args.skip_tag or [\"electrified=no\"]):\n if s_.lower() in (\"none\", \"-\"):\n continue\n if \"=\" not in s_:\n sys.exit(f\"--skip-tag must be written as key=value, got {s_!r}\")\n skip.append(tuple(s_.split(\"=\", 1)))\n reach_deg = (args.radius + args.line_reach) / 110540 + 0.01\n lat_c = [c[\"lat\"] for c in centers]\n lon_c = [c[\"lon\"] for c in centers]\n bbox = (min(lat_c) - reach_deg, min(lon_c) - reach_deg * 1.2, max(lat_c) + reach_deg, max(lon_c) + reach_deg * 1.2)\n gj = json.loads(Path(args.line).read_text(encoding=\"utf-8\"))\n pts = _line_points(gj, args.line_sample, skip, bbox)\n LP = np.array([[p[0], p[1]] for p in pts], dtype=float) if pts else np.zeros((0, 2))\n print(f\"{len(LP)} infrastructure sample points (every {args.line_sample} m)\", file=sys.stderr)\n\n ring_az = np.arange(0, 360, 45.0)\n ring_d = np.array([0.0, args.cam_flat_radius / 2, args.cam_flat_radius])\n done: set[tuple[int, int]] = set()\n recs: list[dict] = []\n clusters_out: list[dict] = []\n skipped = {\"not_flat\": 0, \"no_peak\": 0, \"no_line\": 0, \"dup\": 0, \"missing_dem\": 0, \"no_perp_line\": 0}\n relaxed_clusters = []\n n_flat_checked = 0 # grid points that reached the flat test (the denominator of its WARNING)\n dem_failed = []\n grid_pts = [(dx, dy) for dy in np.arange(-args.radius, args.radius + 1e-6, args.grid)\n for dx in np.arange(-args.radius, args.radius + 1e-6, args.grid) if dx * dx + dy * dy \u003c= args.radius ** 2]\n if args.radius == 0:\n grid_pts = [(0.0, 0.0)]\n for ci, c in enumerate(centers):\n lat, lon = float(c[\"lat\"]), float(c[\"lon\"])\n name = c.get(\"name\", \"\")\n try:\n dem = DEM((lat, lon), args.radius + args.range + 500, args.zoom, args.cache, args.proxy, args.allow_missing)\n except SystemExit as e:\n print(f\"cluster {c['src_idx']} {name}: DEM failed ({e}), skipped\", file=sys.stderr)\n dem_failed.append(c[\"src_idx\"])\n continue\n kx = 111320 * math.cos(math.radians(lat))\n sub = None\n if LP is not None and len(LP):\n m = (np.abs(LP[:, 0] - lat) * 110540 \u003c args.radius + args.line_reach) & (np.abs(LP[:, 1] - lon) * kx \u003c args.radius + args.line_reach)\n sub = LP[m]\n n_c = 0\n best_c = None\n cams = [] # (dx, dy, lat, lon, ground ring) of new, fully covered grid points\n for dx, dy in grid_pts:\n dx, dy = float(dx), float(dy)\n clat = lat + dy / 110540\n clon = lon + dx / kx\n key = (round(clat * 110540 / args.grid), round(clon * kx / args.grid))\n if key in done:\n skipped[\"dup\"] += 1\n continue\n done.add(key)\n rla, rlo = _dest_np(clat, clon, ring_az, ring_d)\n g = dem.sample(rla, rlo)\n if np.isnan(g).any():\n skipped[\"missing_dem\"] += 1\n continue\n cams.append((dx, dy, clat, clon, g))\n # --cam-flat assumes the camera stood on open flat ground. On a mountain road or ridge no grid point of the cluster may pass, and the cluster\n # silently dropped out; then score its flattest points (relief within --cam-flat of the cluster's minimum) and mark them flat_relaxed.\n # A single-point cluster (--radius 0) has nothing to choose among, so it keeps the plain filter.\n flat_lim = args.cam_flat\n relief = [float(g.max() - g.min()) for *_, g in cams]\n n_flat_checked += len(cams)\n if len(cams) > 1 and min(relief) > args.cam_flat:\n flat_lim = min(relief) + args.cam_flat\n relaxed_clusters.append(c[\"src_idx\"])\n for (dx, dy, clat, clon, g), rel in zip(cams, relief):\n if rel > flat_lim:\n skipped[\"not_flat\"] += 1\n continue\n g0 = float(g[:, 0].mean())\n ang, dist_v = cast(dem, (clat, clon), g0 + args.eye, AZ, args.range, near=args.near, n=args.nsamp, keep_nan=True)\n if np.isnan(ang).any(): # a missing far tile would drop a ridge out of the skyline; don't rank on it\n skipped[\"missing_dem\"] += 1\n continue\n hor = ang.max(axis=1)\n if hor.max() \u003c args.min_peak:\n skipped[\"no_peak\"] += 1\n continue\n h_ok = None\n line_brg = None\n if PS is not None:\n kxc = math.cos(math.radians(clat))\n # distance to each segment, not its midpoint: a long span's midpoint can be km away while the wire passes overhead\n ay, ax = (PS[:, 0] - clat) * 110540, (PS[:, 1] - clon) * 111320 * kxc\n by, bx = (PS[:, 2] - clat) * 110540, (PS[:, 3] - clon) * 111320 * kxc\n L2 = (bx - ax) ** 2 + (by - ay) ** 2\n t = np.clip(-(ax * (bx - ax) + ay * (by - ay)) / np.where(L2 > 0, L2, 1), 0, 1)\n d2 = (ax + t * (bx - ax)) ** 2 + (ay + t * (by - ay)) ** 2\n i_ = int(np.argmin(d2))\n if d2[i_] > args.perp_reach ** 2:\n skipped[\"no_perp_line\"] += 1\n continue\n y1, x1, y2, x2 = PS[i_]\n line_brg = math.degrees(math.atan2((x2 - x1) * kxc, y2 - y1)) % 180\n h_ok = np.abs((AZ - line_brg) % 180 - 90) \u003c= args.perp_tol\n line_pen = None\n dmins = None\n if LP is not None:\n if sub is None or not len(sub):\n skipped[\"no_line\"] += 1\n continue\n by = (sub[:, 0] - clat) * 110540\n bx = (sub[:, 1] - clon) * 111320 * math.cos(math.radians(clat))\n bd = np.hypot(bx, by)\n baz = np.degrees(np.arctan2(bx, by)) % 360\n off = (baz[None, :] - AZ[:, None] + 180) % 360 - 180 # (naz, n points)\n far = bd > args.line_min\n dmins = []\n for lo_, hi_ in wins:\n mm = (off > lo_) & (off \u003c hi_) & far[None, :]\n dmins.append(np.where(mm, bd[None, :], np.inf).min(axis=1))\n line_pen = np.zeros(naz)\n for dm, (dlo, dhi), sc in zip(dmins, dwin, scales):\n line_pen += np.maximum(0, dlo - dm) / sc + np.maximum(0, dm - dhi) / sc\n if args.line_order == \"asc\":\n line_pen += ((dmins[0] >= dmins[1]) | (dmins[1] >= dmins[2])) * 1.0\n elif args.line_order == \"desc\":\n line_pen += ((dmins[0] \u003c= dmins[1]) | (dmins[1] \u003c= dmins[2])) * 1.0\n if not np.isfinite(line_pen).any():\n skipped[\"no_line\"] += 1\n continue\n best = None\n def idx(off):\n return (HS[:, None] + np.round(off / az_step).astype(int)[None, :]) % naz\n nhs = {D1: ang[:, dist_v \u003c= D1].max(axis=1) if D1 else None for D1 in near_dists} # near layer: highest angle within D1 m\n for D1, fs in ((d_, f_) for d_ in near_dists for f_ in fscales):\n nh = nhs[D1]\n f = f0 * fs\n r_off = np.degrees(np.arctan((RX - cx) / f))\n r_el = np.degrees(np.arctan((hrow - RY) / f))\n # solve horizon offset cc (pitch/hrow inaccurate) and roll rr together by least squares: a 1° handheld tilt puts the ridgeline at both ends of the frame\n # off by a dozen-plus pixels; left unsolved, the ground truth gets crowded into the same rms tier as a pile of wrong candidates (one real photo: 11.1 → 6.2 px, only then did the ground truth stand out)\n diff = hor[idx(r_off)] - r_el[None, :]\n if NX.size: # near points share cc and roll with the far ridge\n n_off = np.degrees(np.arctan((NX - cx) / f))\n diff = np.concatenate([diff, nh[idx(n_off)] - np.degrees(np.arctan((hrow - NY) / f))[None, :]], axis=1)\n r_off = np.concatenate([r_off, n_off])\n if roll_max > 0 and (r_off != r_off.mean()).any():\n oc = r_off - r_off.mean()\n dmean = diff.mean(axis=1)\n rr = np.clip(((diff - dmean[:, None]) * oc[None, :]).sum(axis=1) / (oc ** 2).sum(), -roll_max, roll_max)\n cc = np.clip(dmean - rr * r_off.mean(), -args.cc_max, args.cc_max)\n else: # --roll-max 0: like the old version, subtract only the median\n rr = np.zeros(naz)\n cc = np.clip(np.median(diff, axis=1), -args.cc_max, args.cc_max)\n res = diff - cc[:, None] - rr[:, None] * r_off[None, :]\n rms = np.sqrt(np.mean(res ** 2, axis=1))\n rms_near = np.sqrt(np.mean(res[:, RX.size:] ** 2, axis=1)) if NX.size else None\n if FLX.size:\n fl_off = np.degrees(np.arctan((FLX - cx) / f))\n pen = np.mean(np.clip(hor[idx(fl_off)] - cc[:, None] - args.flat_clear, 0, None), axis=1)\n else:\n pen = np.zeros(naz)\n if NCX.size: # near-clear columns: the near layer must stay below the given row\n c_off = np.degrees(np.arctan((NCX - cx) / f))\n c_el = np.degrees(np.arctan((hrow - NCR) / f))\n ncpen = np.mean(np.clip(nh[idx(c_off)] - cc[:, None] - rr[:, None] * c_off[None, :] - c_el[None, :], 0, None), axis=1)\n else:\n ncpen = np.zeros(naz)\n # across a wide focal ladder, compare in pixels (× fs, i.e. degrees at the base focal): in degrees the same pixel mismatch shrinks as f grows, and on a\n # real photo every candidate ran to the longest rung. Within ×0.9–1.12 degrees stay: there pixels pushed focal short and camera positions farther\n score = (rms + args.flat_w * pen + args.near_clear_w * ncpen) * (fs if px_metric else 1.0)\n total = score + args.line_w * line_pen if line_pen is not None else score\n if h_ok is not None:\n total = np.where(h_ok, total, np.inf)\n k = int(np.argmin(total))\n if not np.isfinite(total[k]):\n continue\n if best is None or total[k] \u003c best[\"total\"]:\n best = {\"hit\": c[\"src_idx\"], \"name\": name, \"hit_ll\": [round(lat, 5), round(lon, 5)],\n \"cam\": [round(clat, 5), round(clon, 5)],\n \"d\": round(math.hypot(dx, dy)), \"brg\": round(math.degrees(math.atan2(dx, dy)) % 360),\n \"g\": round(g0, 1), \"H\": round(k * az_step, 2), \"fs\": fs, \"f\": round(f, 1), \"f35\": round(f35_0 * fs, 1),\n \"cc\": round(float(cc[k]), 2), \"roll\": round(math.degrees(math.atan(float(rr[k]))), 2),\n \"rms\": round(float(rms[k]), 3),\n \"rms_px\": round(float(rms[k]) * f * math.pi / 180, 1), \"flatpen\": round(float(pen[k]), 3),\n \"score\": round(float(score[k]), 3)}\n if flat_lim > args.cam_flat:\n best[\"flat_relaxed\"] = round(rel, 1)\n if D1:\n best.update({\"D1\": D1, \"nclearpen\": round(float(ncpen[k]), 3)})\n if rms_near is not None:\n best[\"rms_near_px\"] = round(float(rms_near[k]) * f * math.pi / 180, 1)\n if line_brg is not None:\n best[\"line_brg\"] = round(line_brg, 1)\n if line_pen is not None:\n best.update({\"dL\": round(float(dmins[0][k])), \"dC\": round(float(dmins[1][k])), \"dR\": round(float(dmins[2][k])),\n \"line_pen\": round(float(line_pen[k]), 3)})\n best[\"total\"] = round(float(total[k]), 3)\n if best is None:\n skipped[\"no_line\"] += 1\n continue\n recs.append(best)\n n_c += 1\n if best_c is None or best[\"total\"] \u003c best_c[\"total\"]:\n best_c = best\n clusters_out.append({\"hit\": c[\"src_idx\"], \"name\": name, \"hit_ll\": [round(lat, 5), round(lon, 5)],\n \"n\": c.get(\"n\"), \"max_ang\": c.get(\"max_ang\"), \"n_cams\": n_c, \"best\": best_c})\n print(f\"cluster {ci + 1}/{len(centers)} hit{c['src_idx']} {name or '-'}: scored {n_c} camera positions\"\n + (f\", best total {best_c['total']} H {best_c['H']} cam {best_c['cam']}\" if best_c else \", no camera position passed the filters\"), file=sys.stderr)\n\n recs.sort(key=lambda r: r[\"total\"])\n clusters_out.sort(key=lambda x: x[\"best\"][\"total\"] if x[\"best\"] else float(\"inf\"))\n for i, x in enumerate(clusters_out):\n x[\"rank\"] = i + 1\n hfov = 2 * math.degrees(math.atan(cx / f0))\n lead = [x[\"best\"] for x in clusters_out if x[\"best\"]] if len(clusters_out) > 1 else recs\n lead = lead[:10]\n focal_edge = None\n if len(fscales) > 1 and lead:\n lo_s, hi_s = min(fscales), max(fscales)\n n_long = sum(r[\"fs\"] == hi_s for r in lead)\n n_short = sum(r[\"fs\"] == lo_s for r in lead)\n top1 = \"long\" if lead[0][\"fs\"] == hi_s else \"short\" if lead[0][\"fs\"] == lo_s else None\n if top1 or max(n_long, n_short) * 2 >= len(lead):\n focal_edge = {\"top1\": top1, \"long\": n_long, \"short\": n_short, \"of\": len(lead),\n \"range_f35\": [round(f35_0 * lo_s, 1), round(f35_0 * hi_s, 1)]}\n out = {\n \"params\": {\"hits\": args.hits, \"at\": args.at, \"select\": args.select, \"ridge\": str(args.ridge),\n \"radius_m\": args.radius, \"grid_m\": args.grid, \"zoom\": args.zoom, \"focal_scales\": fscales,\n \"az_step_deg\": az_step, \"near_m\": args.near, \"range_m\": args.range, \"nsamp\": args.nsamp, \"eye_m\": args.eye,\n \"focal\": focal_how, \"focal_metric\": \"px\" if px_metric else \"deg\", \"f35_ladder\": [round(f35_0 * x, 1) for x in fscales],\n \"near_dist_m\": near_dists if two else None, \"near_clear_w\": args.near_clear_w,\n \"perp_line\": args.perp_line, \"perp_tol_deg\": args.perp_tol if args.perp_line else None,\n \"cam_flat_m\": args.cam_flat, \"cam_flat_radius_m\": args.cam_flat_radius, \"min_peak_deg\": args.min_peak,\n \"cc_max_deg\": args.cc_max, \"roll_max_deg\": args.roll_max, \"flat_clear_deg\": args.flat_clear, \"flat_w\": args.flat_w, \"flat_step_px\": args.flat_step,\n \"line\": args.line, \"line_dist\": args.line_dist, \"line_win\": args.line_win if args.line else None,\n \"line_scale\": args.line_scale if args.line else None, \"line_order\": args.line_order if args.line else None,\n \"line_w\": args.line_w, \"line_min_m\": args.line_min, \"line_sample_m\": args.line_sample, \"line_reach_m\": args.line_reach,\n \"photo\": {\"f0_px\": f0, \"hrow\": hrow, \"cx\": cx, \"hfov_deg_at_fs1\": round(hfov, 1), \"n_ridge\": int(RX.size), \"flat\": flat}},\n \"n_clusters\": len(clusters_out), \"n_cams\": len(recs), \"n_skipped\": skipped, \"dem_failed_clusters\": dem_failed,\n \"flat_relaxed_clusters\": relaxed_clusters,\n \"focal_edge\": focal_edge,\n \"clusters\": clusters_out,\n \"cams\": recs[:args.keep],\n }\n Path(args.out).write_text(json.dumps(out, ensure_ascii=False, indent=0), encoding=\"utf-8\")\n print(f\"clusters {len(clusters_out)}, camera positions {len(recs)}, skipped {skipped} → {args.out}\", file=sys.stderr)\n if skipped[\"missing_dem\"]:\n print(f\"WARNING: {skipped['missing_dem']} camera positions not scored because their sight lines touch missing tiles\", file=sys.stderr)\n wide = [c for c in centers if (c.get(\"extent_m\") or 0) > args.radius]\n if wide: # the grid is centred on the representative only; members beyond --radius never get a camera\n print(f\"WARNING: {len(wide)}/{len(centers)} clusters spread wider than --radius {args.radius:g} m (farthest member up to \"\n f\"{max(c['extent_m'] for c in wide) / 1000:.1f} km from its representative): hits beyond the camera grid were not scored. \"\n f\"Raise --radius, or rerun scan with a smaller --cluster-km, before treating a cluster as checked\", file=sys.stderr)\n if relaxed_clusters:\n print(f\"NOTE: {len(relaxed_clusters)}/{len(centers)} clusters had no grid point within --cam-flat {args.cam_flat:g} m over \"\n f\"{args.cam_flat_radius:g} m (slope, roadside, ridge); their flattest points were scored and marked flat_relaxed\", file=sys.stderr)\n if n_flat_checked and skipped[\"not_flat\"] > 0.5 * n_flat_checked:\n print(f\"WARNING: {skipped['not_flat']}/{n_flat_checked} camera positions failed --cam-flat {args.cam_flat:g} m over {args.cam_flat_radius:g} m. \"\n f\"If the camera may stand on a slope (mountain road, roadside pull-off, ridge), rerun with a larger --cam-flat \"\n f\"(e.g. {max(60, args.cam_flat * 8):g}) before reading the ranking\", file=sys.stderr)\n for x in clusters_out[:min(args.top, 10)]:\n b = x[\"best\"]\n if b:\n line_txt = f\" L/C/R {b['dL']}/{b['dC']}/{b['dR']} pen {b['line_pen']}\" if \"line_pen\" in b else \"\"\n near_txt = (f\" D1 {b['D1'] / 1000:g} km near {b.get('rms_near_px', '-')} px clear {b['nclearpen']}\" if \"D1\" in b else \"\")\n print(f\" {x['rank']:>3}. hit{x['hit']} {x['name'] or '-'} cam {b['cam']} H {b['H']} f {b['f']:.0f} ({b['f35']:.0f} mm) \"\n f\"rms {b['rms']} ({b['rms_px']} px) flat {b['flatpen']}{near_txt}{line_txt} total {b['total']}\", file=sys.stderr)\n if focal_edge:\n lo_mm, hi_mm = focal_edge[\"range_f35\"]\n side = focal_edge[\"top1\"] or (\"long\" if focal_edge[\"long\"] >= focal_edge[\"short\"] else \"short\")\n n_side = focal_edge[side]\n if f_src != \"assumed\" and args.focal_scales == \"auto\" and not args.f35_range:\n print(f\"WARNING: focal is known ({'EXIF' if f_src == 'exif' else '--f0'}) yet {n_side}/{focal_edge['of']} leading results sit on the {side} end \"\n f\"of ×0.9–1.12: it is absorbing another error (hrow, ridge points, crop, wrong place). Check ridge --png before widening.\", file=sys.stderr)\n else:\n wider = f\"{hi_mm * 0.6:.0f}:{hi_mm * 2:.0f}\" if side == \"long\" else f\"{max(8, lo_mm / 2):.0f}:{lo_mm * 1.7:.0f}\"\n print(f\"WARNING: best focal sits on the {side} end of the ladder ({hi_mm if side == 'long' else lo_mm:.0f} mm) for {n_side}/{focal_edge['of']} \"\n f\"leading results{' including #1' if focal_edge['top1'] else ''}. That is a signal, not noise: the photo is probably \"\n f\"{'more zoomed in' if side == 'long' else 'wider'} than the ladder. Rerun with --f35-range {wider} before trusting the ranking, \"\n f\"heading or focal; only if it runs to the new edge again is the focal absorbing another error (geometry.md 7.4).\", file=sys.stderr)\n if side == \"short\":\n # in pixels a short focal also shrinks the DEM error, so deep valley floors can win at 13 mm on a photo that is not ultra-wide\n grounds = \", \".join(f\"{r['g']:.0f}\" for r in lead[:5])\n lo2 = max(20, lo_mm * 1.5)\n hi2 = hi_mm if hi_mm > lo2 else lo2 * 2 # a narrow ladder (e.g. 16:18) would otherwise suggest an empty lo > hi range\n print(f\"WARNING: a short-end pile-up can also be a false fit: in pixels a wide lens shrinks the DEM error too, and valley or canyon floors \"\n f\"(leading grounds {grounds} m) fit a ridge seen high above them. If the photo shows no \"\n f\"ultra-wide perspective, also rerun with --f35-range {lo2:.0f}:{hi2:.0f} and compare the leading places.\",\n file=sys.stderr)\n if args.sheet:\n top = [x[\"best\"] for x in clusters_out if x[\"best\"]][:args.top] if len(clusters_out) > 1 else recs[:args.top]\n if not top:\n print(\"no camera positions to draw, no overlay\", file=sys.stderr)\n else:\n photo = Path(args.photo) if args.photo else None\n if photo is None and ridge.get(\"photo\"):\n # ridge.json records the path exactly as given on the command line, possibly relative: try the current directory first, then the directory of ridge.json\n cands = [Path(ridge[\"photo\"]), Path(args.ridge).parent / ridge[\"photo\"]]\n photo = next((p for p in cands if p.exists()), cands[0])\n if photo is None or not photo.exists():\n print(\"photo not found (--overlay not given, and the photo path in ridge.json can't be found either); drawing the overlay on a gray background\", file=sys.stderr)\n photo = None\n _fit_sheet(top, ridge, photo, Path(args.sheet), args, AZ, args.nsamp)\n if dem_failed: # output is written, but the ranking is missing these clusters\n sys.exit(f\"{len(dem_failed)}/{len(centers)} clusters not scored because their DEM failed \"\n f\"(hit {', '.join(map(str, dem_failed))}); re-run to retry them, the ranking above is incomplete\")\n\n\ndef _neg_coords(argv: list[str]) -> list[str]:\n \"\"\"argparse treats negative coordinates like -1.45,-48.5 as option names; prefixing a space makes them plain values (float ignores spaces). Needed for any photo in the southern or western hemisphere.\n fit's --line-win -27:-21,-3:3,18:27 is the same case (a string of numbers separated by commas or colons).\"\"\"\n return [\" \" + a if re.match(r\"^-\\d[\\d.]*([,:]-?[\\d.]+)+$\", a) else a for a in argv]\n\n\ndef main() -> None:\n ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)\n ap.add_argument(\"--proxy\", default=os.environ.get(\"GEO_PROXY\"), help=PROXY_HELP)\n ap.add_argument(\"--cache\", type=Path, default=Path(\".geo-cache/dem\"))\n sub = ap.add_subparsers(dest=\"cmd\", required=True)\n\n e = sub.add_parser(\"elev\")\n e.add_argument(\"--at\", required=True)\n e.add_argument(\"--zoom\", type=int, default=13)\n e.add_argument(\"--allow-missing\", action=\"store_true\", help=MISSING_HELP)\n\n def common(sp):\n sp.add_argument(\"--at\", required=True, help=\"candidate camera position lat,lon (WGS84)\")\n g = sp.add_mutually_exclusive_group()\n g.add_argument(\"--height\", type=float, default=1.6, help=\"height above ground m (for a shot from a building, use floor × 3)\")\n g.add_argument(\"--alt\", type=float, help=\"absolute altitude m (for aerial and mountaintop camera positions)\")\n sp.add_argument(\"--heading\", type=float, required=True, help=\"compass bearing of the frame center\")\n sp.add_argument(\"--hfov\", type=float, default=65, help=\"horizontal field of view; phone main camera about 65 landscape, about 50 portrait\")\n sp.add_argument(\"--range\", type=float, default=30000, help=\"how far to look (m)\")\n sp.add_argument(\"--zoom\", type=int, default=12, help=\"elevation tile zoom level, 12 ≈ 25–30 m per cell\")\n sp.add_argument(\"--width\", type=int, default=1400)\n sp.add_argument(\"--near\", type=float, default=40, help=\"how near to start sampling, m; if spiky false ridgelines appear up close, raise to 150–300\")\n sp.add_argument(\"--out\", type=Path, required=True)\n sp.add_argument(\"--allow-missing\", action=\"store_true\", help=MISSING_HELP)\n\n v = sub.add_parser(\"view\")\n common(v)\n v.add_argument(\"--pitch\", type=float, default=0, help=\"pitch of the frame center, looking up is positive: a photo whose horizon sits below its middle was shot looking up (positive pitch)\")\n v.add_argument(\"--vfov\", type=float, help=\"vertical field of view; default derived from hfov for a 3:2 frame\")\n v.add_argument(\"--photo\", type=Path, help=\"scale the photo to the same width and place it above the render for comparison\")\n v.add_argument(\"--roll\", type=float, default=0, help=\"frame roll angle (clockwise positive), for downward shots from tall buildings and handheld tilt\")\n v.add_argument(\"--overlay\", action=\"store_true\", help=\"also output a separate image: draw the synthetic skyline (red line) directly on the photo with the same heading/pitch/field of view\")\n\n pr = sub.add_parser(\"profile\")\n common(pr)\n\n sc = sub.add_parser(\"scan\", formatter_class=argparse.RawDescriptionHelpFormatter, description=\"\"\"\\\nAlong linear infrastructure, compute the 360° horizon point by point, keep the points that are \"flat nearby + a mountain + the mountain right next to a stretch of flat horizon\", then cluster them.\n\nUse when: the frame shows only one railway/road/power line and a mountain you can't recognize, with no text at all. First\nget the lines with `osm.py geom '\u003cfilter>' --bbox \u003clarge region> --out lines.geojson`, then run this to cut the large region down to a few hundred patches;\nthe selected clusters go into `terrain.py fit` for skyline scoring. Method and the reasoning behind the values: references/corridors.md 4.3.\n\nSet thresholds at about half of the photo estimate: one z10 cell is ~150 m, peaks get smoothed away, and the computed elevation angles come out smaller than the real ones.\nIn a real case, a threshold set at the photo's estimated 6° left not a single point near the ground truth; it only got in after loosening to 4.5° — better too many than a miss; leave the count to the next ranking step.\n\nOutput JSON (--out):\n {\"params\": {...all thresholds used in this run, recorded as is...},\n \"n_samples\": number of sample points, \"n_hits\": number of hit points, \"n_clusters\": number of clusters,\n \"hits\": [{\"lat\",\"lon\",\"h0\": ground elevation at the point m, \"max_ang\": highest horizon elevation angle °,\n \"az\": that bearing °, \"relief\": max relative height difference at that bearing m, \"flat_run_deg\": length of flat horizon right next to the mountain °,\n \"name\",\"hs\"(highspeed),\"elec\"(electrified),\"id\": OSM tags of the line}, ...],\n \"clusters\": [{...same as above..., \"n\": number of points in the cluster, \"extent_m\": distance to the farthest member m}, ...]} # sorted by max_ang from largest to smallest; the cluster representative is the steepest point\n--clusters-out writes a separate JSON with only the cluster list (exactly the clusters field).\"\"\")\n sc.add_argument(\"--lines\", required=True, help=\"linear infrastructure GeoJSON (output of osm.py geom); only LineString is read\")\n sc.add_argument(\"--out\", required=True, help=\"output JSON\")\n sc.add_argument(\"--clusters-out\", help=\"also write a separate JSON with only the cluster list\")\n sc.add_argument(\"--bbox\", help=\"only scan sample points inside this range s,w,n,e (for validation and chunked runs; omitted = the whole geojson)\")\n sc.add_argument(\"--step\", type=float, default=400, help=\"take one point every this many meters along the line (default 400; anything in 300–500 works)\")\n sc.add_argument(\"--zoom\", type=int, default=10, help=\"elevation tile zoom level (default 10, one cell ~150 m, enough for mountain outlines)\")\n sc.add_argument(\"--near-flat\", type=float, default=40,\n help=\"nearby relief limit m: if the height difference within --near-radius of the point exceeds it, the point is not flat and is dropped (default 40; the loosened version used 60)\")\n sc.add_argument(\"--near-radius\", type=float, default=1200, help=\"radius m for judging whether the nearby ground is flat (default 1200)\")\n sc.add_argument(\"--near-step\", type=float, default=300, help=\"nearby sampling interval m (default 300)\")\n sc.add_argument(\"--min-peak\", type=float, default=4.5,\n help=\"mountain: horizon elevation angle of at least this many degrees (default 4.5, the value after loosening in a real case; the original strict value 6, estimated from that photo, filtered out all of the ground truth)\")\n sc.add_argument(\"--max-low\", type=float, default=1.2,\n help=\"flat: a horizon elevation angle below this many degrees counts as flat horizon (default 1.2; the loosened version used 1.5)\")\n sc.add_argument(\"--flat-run\", type=float, default=20,\n help=\"the stretch of flat horizon right next to the mountain must be at least this many degrees (default 20; original strict value 30)\")\n sc.add_argument(\"--min-low-deg\", type=float,\n help=\"total flat horizon around the full circle of at least this many degrees (default same as --flat-run)\")\n sc.add_argument(\"--flat-run-cap\", type=float, default=100, help=\"count flat-horizon length at most this many degrees to one side (default 100)\")\n sc.add_argument(\"--eye\", type=float, default=1.5, help=\"eye height above ground m (default 1.5)\")\n sc.add_argument(\"--az-step\", type=float, default=5, help=\"bearing step ° (default 5, must divide 360 evenly)\")\n sc.add_argument(\"--dist\", default=\"1500,2000,2500,3000,3500,4000,5000,6000,7000,8000,9000\",\n help=\"distance steps measured outward along each bearing m, comma-separated (default 1.5–9 km)\")\n sc.add_argument(\"--skip-tag\", action=\"append\",\n help=\"skip lines carrying this tag, key=value, repeatable (default electrified=no; --skip-tag none means skip none)\")\n sc.add_argument(\"--cluster-km\", type=float, default=3.5, help=\"clustering radius km (default 3.5)\")\n sc.add_argument(\"--threads\", type=int, default=8, help=\"tile download concurrency (default 8; failed tiles are retried with backoff)\")\n sc.add_argument(\"--allow-missing\", action=\"store_true\", help=MISSING_HELP)\n sc.add_argument(\"--max-tiles\", type=int, default=4000,\n help=\"limit on the number of tiles to download; exits immediately if exceeded (default 4000). Memory is set by the bounding rectangle of these tiles, \"\n \"which can be several times the tile count when the line network is sparse; the actual usage is printed before the mosaic is built\")\n\n rg = sub.add_parser(\"ridge\", formatter_class=argparse.RawDescriptionHelpFormatter, description=\"\"\"\\\nPhoto ridgeline sampling: in --x0..--x1, every --step columns, top to bottom, find the sky→mountain brightness jump (the k rows above are brighter than the k rows below by --drop or more,\nand the --hold rows further down are still dark), and take the maximum-gradient row as the ridge pixel. The output is for terrain.py fit.\n\n--flat gives the column range where \"the frame shows flat horizon\" (the stretch on the left or right with no distant mountains); fit uses it to penalize \"a mountain blocking where it should be flat\".\nWithout --hrow, the horizon row is the median of the sky→ground jump rows in the --flat columns; without --flat, it is the frame midline.\nNote that this estimate lands on top of things like distant treetops and bridge decks, 10–30 px above the true 0° horizon (on that photo: auto estimate 909,\nhand-set 935); fit's horizon offset cc only absorbs ±0.7° (about ±16 px at f≈1300 px), so when there are tall objects in the distance, give --hrow by hand.\nFocal length: --f0 > EXIF 35mm-equivalent focal length > assumed from --f35 (default 26 mm, phone main camera), converted to pixels via the diagonal; the source is written to f0_source.\nAn assumed focal is only the base: fit then searches 13–135 mm equivalent by default, so a zoomed shot is not forced into a main-camera fit.\n\n--points replaces detection with hand-picked skyline points (cloud on the peaks, snow against a pale sky).\nTwo-layer photos (a lower near range in front of a higher far range, the near range ending mid-frame so the far range's foot shows):\n --near x,y;... points on the near range's top edge; fit scores them against the highest terrain within a near distance\n --near-clear x0:x1@ROW columns past the near range's end, where nothing near may rise above ROW (the far range's foot / plain)\n\nOutput JSON (--out):\n {\"photo\": photo path (recorded exactly as given on the command line; if fit can't find it, it retries relative to the directory of ridge.json),\n \"image_size\": [width, height], \"cx\": center column,\n \"frame\": \"upright\", # pixel coordinates are in the photo as a viewer shows it (EXIF orientation applied);\n # a ridge.json without this key predates that and is drawn on the stored pixels\n \"ridge\": [[x, y], ...], # ridge pixel points, x column y row (y is the mountain's first row)\n \"missing_cols\": [x, ...], # columns where no jump was found\n \"ridge_source\": \"detected\"|\"points\",\n \"near\": [[x, y], ...], \"near_clear\": [[x0, x1, row], ...], # two-layer input, empty when not given\n \"flat\": [x0, x1] | null, \"flat_rows\": [jump row of each column in that range],\n \"hrow\": horizon row, \"hrow_source\": \"arg\"|\"flat\"|\"center\",\n \"f0\": focal length px, \"f0_source\": \"arg\"|\"exif\"|\"assumed\", \"f35_equiv\": 35mm-equivalent focal length,\n \"params\": {...parameters used in this run...}}\n--png also outputs a check image: yellow dots = ridge points, red vertical lines = columns not found, blue line = hrow, cyan line = flat-horizon column range,\norange dots / lines = near points / near-clear rows. Look at the image before going to fit.\"\"\")\n rg.add_argument(\"photo\", type=Path, help=\"photo\")\n rg.add_argument(\"--x0\", type=int, default=0, help=\"ridge start column (inclusive, default 0)\")\n rg.add_argument(\"--x1\", type=int, help=\"ridge end column (inclusive, default the last column)\")\n rg.add_argument(\"--step\", type=int, default=20, help=\"take one point every this many columns (default 20)\")\n rg.add_argument(\"--flat\", help=\"flat-horizon column range x0:x1 (e.g. 0:280); omit if there is none\")\n rg.add_argument(\"--hrow\", type=float, help=\"row of the horizon; if omitted, estimated from the --flat columns, and failing that, the frame midline\")\n rg.add_argument(\"--f0\", type=float, help=\"focal length px; if given, EXIF is not read\")\n rg.add_argument(\"--f35\", type=float, default=26.0, help=\"assumed 35mm-equivalent focal length mm when there is no EXIF (default 26, phone main camera; 2x telephoto 48–52)\")\n rg.add_argument(\"--ymin\", type=int, help=\"only search below this row (default 0; for clouds in the sky or foreground occlusion)\")\n rg.add_argument(\"--ymax\", type=int, help=\"only search above this row (default the frame bottom)\")\n rg.add_argument(\"--drop\", type=float, default=30, help=\"minimum amount the sky must be brighter than the mountain (0–255 brightness, default 30; in heavy haze lower to 15–20)\")\n rg.add_argument(\"--k\", type=int, default=4, help=\"rows taken above and below each when comparing brightness (default 4)\")\n rg.add_argument(\"--hold\", type=int, default=12, help=\"rows that must stay dark below the jump for it to count as mountain (default 12, used to skip wires and antennas)\")\n rg.add_argument(\"--halfw\", type=int, default=1, help=\"also average this many columns on each side of each column (default 1, i.e. 3 columns)\")\n rg.add_argument(\"--points\", help=\"hand-picked skyline points x,y;x,y;... (or a .json of [[x,y],...]) used instead of detection, \"\n \"when cloud, haze or snow against a pale sky breaks the brightness jump; --x0/--x1 are then ignored\")\n rg.add_argument(\"--near\", help=\"two-layer photos: points x,y;x,y;... on the top edge of the NEAR range where it stands in front of a higher far range \"\n \"(or a .json of [[x,y],...]); fit then scores the near layer too, see geometry.md 7.4\")\n rg.add_argument(\"--near-clear\", action=\"append\",\n help=\"two-layer photos: x0:x1@row = in these columns the near layer ends and the far range's foot or plain shows, \"\n \"so nothing within the near distance may rise above this row; repeatable\")\n rg.add_argument(\"--out\", required=True, help=\"output ridge.json\")\n rg.add_argument(\"--png\", help=\"check image path\")\n\n ft = sub.add_parser(\"fit\", formatter_class=argparse.RawDescriptionHelpFormatter, description=\"\"\"\\\nBatch skyline scoring: in each candidate cluster, lay a camera-position grid (one every --grid meters within --radius), compute the 360° horizon at each camera position,\nconvert ridge.json's ridge points to (bearing offset, elevation angle), and compare against all headings × --focal-scales at once:\n rms = RMS of the ridge elevation-angle residuals (the residual median serves as horizon offset cc, clamped to ±--cc-max)\n flatpen = mean of max(0, horizon elevation angle − cc − --flat-clear) over the flat-horizon columns\n score = rms + --flat-w × flatpen (+ --near-clear-w × nclearpen for two-layer ridge.json)\n rms_px = rms converted to pixels (rms × f × π/180), for judging whether candidates can be separated (on an f35 ladder the score itself is compared in pixels, --focal-metric): when the top few all come out about as large as\n the error of the ridge sampling itself (a few to a dozen or so pixels), the skyline can't separate them; you need a second constraint\nCamera positions first pass two filters: height difference within --cam-flat-radius ≤ --cam-flat (the camera stands on flat ground in the frame),\nhighest horizon elevation angle ≥ --min-peak (there are mountains to compare). A cluster with no flat grid point at all (slope, mountain road) scores its\nflattest points instead and marks them flat_relaxed; a WARNING follows when most positions fail --cam-flat.\nWith --line (GeoJSON from osm.py geom), one more independent constraint is added: for each heading, the nearest distance dL/dC/dR of infrastructure sample points inside the frame's left/center/right bearing windows\n(--line-win); distances outside the --line-dist intervals are penalized by meters/--line-scale (--line-order can also penalize the order),\ntotal = score + --line-w × line_pen; headings where any of the three windows has no infrastructure don't count. Without --line, total = score.\n\nFocal: --focal-scales auto (default) tries ×0.9/1/1.12 around an EXIF or --f0 focal, and a 13–135 mm equivalent ladder when ridge.json's focal was\nassumed (no EXIF). If the leading results sit on an end of the ladder, fit prints a WARNING and writes focal_edge: rerun with --f35-range widened past\nthat end before trusting ranking, heading or focal. Running to the new end again means the focal is absorbing another error (bad ridge points, hrow, wrong place).\nTwo-layer (ridge.json has near / near_clear): the near layer = highest terrain within D1 m, D1 tried from --near-dist; near points share rms, cc and roll\nwith the far ridge. A far-only fit of a two-layer photo can match the far outline at a wrong heading and focal.\n--perp-line keeps headings within --perp-tol of perpendicular to the nearest line (wires crossing the frame almost level).\nDistant ranges: --range (default 15 km) must reach the far ridge; for ranges 20–40 km away use --range 45000 and lower --min-peak.\n\nThe coarse-search defaults are the ones from the real case: 2 km / 250 m / z11 / 0.5°; for fine search switch to --radius 800 --grid 100 --zoom 13 --az-step 0.25.\nThe skyline only gives one sight line (moving a few hundred meters forward or back along it doesn't change the outline); always look at the --sheet overlay for the top few, then pin the point with a second constraint.\n\nTwo things not to treat as conclusions:\n 1. The output H and f have a systematic bias, which comes from hrow in ridge.json. In the same coarse search, hand-set hrow 935 gave H 80.5 / f 1436,\n and the auto-estimated 909 gave H 79.0 / f 1282 — a difference of 1.5° and 12% in focal length (cc can only absorb ±--cc-max). Cluster ranking isn't affected\n (both runs ranked the ground-truth cluster 1st), but before you argue from H/f, get hrow right first (check the blue line on ridge --png).\n 2. Fine search (z13) is not guaranteed to be more accurate than coarse search. In tests on the same pipeline, fine search actually ended up farther from the ground truth than coarse search:\n the z13 flat filter (--cam-flat / --cam-flat-radius) filters out the points near the ground truth, and the skyline only gives one sight line anyway.\n In this pipeline, fine search's job is to give geo.py spacing a reliable --center, not to bring the error down by itself.\nMethod in references/geometry.md 7.4, corridors.md 4.3.\n\nOutput JSON (--out):\n {\"params\": {...all parameters used in this run, including f0/hrow/cx/hfov under photo...},\n \"n_clusters\", \"n_cams\": number of camera positions scored, \"n_skipped\": {\"not_flat\",\"no_peak\",\"no_line\",\"dup\",\"missing_dem\",\"no_perp_line\"},\n \"focal_edge\": null | {\"top1\": \"long\"|\"short\"|null, \"long\", \"short\", \"of\": leading results counted, \"range_f35\": [lo, hi] mm},\n \"dem_failed_clusters\": hit indices whose elevation tiles failed to download (not scored),\n \"clusters\": [{\"hit\": the cluster's index in --hits, \"name\", \"hit_ll\": [lat,lon], \"n\", \"max_ang\", \"n_cams\", \"rank\",\n \"best\": \u003ccamera record>}, ...], # best camera position of each cluster, ascending by total\n \"cams\": [\u003ccamera record>, ...]} # all camera positions ascending by total, at most --keep entries\n camera record: {\"hit\",\"name\",\"hit_ll\", \"cam\": [lat,lon], \"d\": distance from cluster center m, \"brg\": bearing cluster center→camera position °,\n \"g\": ground elevation m, \"H\": heading °, \"fs\": focal scale, \"f\": focal length px, \"f35\": 35mm-equivalent mm, \"cc\": horizon offset °, \"roll\",\n \"rms\", \"rms_px\", \"flatpen\", \"score\", [two-layer: \"D1\" m, \"rms_near_px\", \"nclearpen\"], [--perp-line: \"line_brg\"],\n [with --line: \"dL\",\"dC\",\"dR\" m, \"line_pen\"], \"total\"}\n--sheet: with multiple clusters, draws the top --top of the per-cluster best camera positions; with a single cluster/--at, draws the top --top camera positions; red line = synthetic skyline, orange line = synthetic near layer, yellow dots = photo ridge points,\ncyan dots = photo near points.\"\"\")\n src = ft.add_mutually_exclusive_group(required=True)\n src.add_argument(\"--hits\", help=\"scan output JSON (uses the clusters field) or a cluster-list JSON\")\n src.add_argument(\"--at\", help=\"instead of --hits, give one center lat,lon directly (for fine search)\")\n ft.add_argument(\"--select\", help=\"only run the clusters at these indices in --hits, comma-separated (for validation and chunked runs)\")\n ft.add_argument(\"--ridge\", type=Path, required=True, help=\"output of terrain.py ridge\")\n ft.add_argument(\"--out\", required=True, help=\"output JSON\")\n ft.add_argument(\"--radius\", type=float, default=2000, help=\"camera-position grid radius per cluster m (default 2000; fine search 800)\")\n ft.add_argument(\"--grid\", type=float, default=250, help=\"camera-position grid spacing m (default 250; fine search 100)\")\n ft.add_argument(\"--zoom\", type=int, default=11, help=\"elevation tile zoom level (default 11, one cell ~70 m; fine search 13)\")\n ft.add_argument(\"--focal-scales\", default=\"auto\",\n help=\"focal length scales of ridge.json's f0, comma-separated. Default auto: EXIF or --f0 focal → 0.9,1,1.12; \"\n \"no EXIF → a ladder over --f35-range (13–135 mm equivalent, ~18%% steps), because a guessed main camera \"\n \"misses zoomed shots, which are common for distant mountains\")\n ft.add_argument(\"--f35-range\", help=\"lo:hi 35mm-equivalent focal range for the ladder, e.g. 40:270 after the best focal hit the long end; \"\n \"overrides auto even with EXIF\")\n ft.add_argument(\"--focal-metric\", choices=[\"auto\", \"deg\", \"px\"], default=\"auto\",\n help=\"unit in which scores are compared across focal lengths: deg (as before) or px (score × focal scale). \"\n \"Default auto: px for an f35 ladder, deg for ×0.9–1.12; degrees favour the longest focal on a wide ladder\")\n ft.add_argument(\"--near-dist\", default=\"3000,5000,8000,12000\",\n help=\"two-layer only: candidate distances m out to which the near layer reaches, comma-separated (default 3000,5000,8000,12000)\")\n ft.add_argument(\"--near-clear-w\", type=float, default=1.5, help=\"two-layer only: weight of nclearpen (default 1.5)\")\n ft.add_argument(\"--perp-line\", help=\"linear infrastructure GeoJSON (osm.py geom); keep only headings near perpendicular to the nearest line, \"\n \"for wires crossing the frame almost level overhead\")\n ft.add_argument(\"--perp-tol\", type=float, default=35, help=\"with --perp-line: allowed deviation ° from perpendicular (default 35)\")\n ft.add_argument(\"--perp-reach\", type=float, default=1500, help=\"with --perp-line: camera positions farther than this from any line m are skipped (default 1500)\")\n ft.add_argument(\"--az-step\", type=float, default=0.5, help=\"heading/bearing step ° (default 0.5; fine search 0.25; must divide 360 evenly)\")\n ft.add_argument(\"--near\", type=float, default=150, help=\"how near the sight line starts sampling m (default 150)\")\n ft.add_argument(\"--range\", type=float, default=15000, help=\"how far the sight line looks m (default 15000)\")\n ft.add_argument(\"--nsamp\", type=int, default=260, help=\"samples per sight line, dense near and sparse far (default 260; fine search 400)\")\n ft.add_argument(\"--eye\", type=float, default=1.6, help=\"eye height above ground m (default 1.6)\")\n ft.add_argument(\"--cam-flat\", type=float, default=8, help=\"height difference limit around the camera position m; above it, it isn't a flat-ground camera position (default 8; fine search has used 6). \"\n \"A cluster where no grid point passes scores its flattest points, marked flat_relaxed\")\n ft.add_argument(\"--cam-flat-radius\", type=float, default=300, help=\"radius m for judging whether the camera position is flat (default 300; fine search has used 200)\")\n ft.add_argument(\"--min-peak\", type=float, default=6, help=\"minimum highest horizon elevation angle in degrees for a camera position to be scored (default 6)\")\n ft.add_argument(\"--cc-max\", type=float, default=0.7, help=\"limit of horizon offset cc ° (default 0.7, i.e. hrow may be off by a dozen or so pixels)\")\n ft.add_argument(\"--roll-max\", type=float, default=1.0,\n help=\"roll angle limit ° (default 1.0, the usual range of handheld tilt; 0 = don't solve roll, same as the old version)\")\n ft.add_argument(\"--flat-clear\", type=float, default=1.0, help=\"on the flat-horizon columns, horizon up to this many degrees above cc is not penalized (default 1.0)\")\n ft.add_argument(\"--flat-w\", type=float, default=1.5, help=\"flatpen weight (default 1.5)\")\n ft.add_argument(\"--flat-step\", type=float, default=20, help=\"take one column every this many pixels in the flat-horizon column range (default 20)\")\n ft.add_argument(\"--line\", help=\"linear infrastructure GeoJSON (output of osm.py geom); if given, adds the left/center/right distance constraint\")\n ft.add_argument(\"--line-dist\", help=\"three distance intervals left:center:right m, e.g. 350-750:550-900:800-1700 (required with --line)\")\n ft.add_argument(\"--line-win\", default=\"-27:-21,-3:3,18:27\",\n help=\"three bearing windows left/center/right, offset angle ° from the frame center (default -27:-21,-3:3,18:27, set for a 53° horizontal field of view; \"\n \"writing --line-win=-27:... or putting the value right after both work)\")\n ft.add_argument(\"--line-scale\", default=\"200,200,300\", help=\"penalty scale m of each of the three windows (default 200,200,300: 200 m out of bounds = penalty 1)\")\n ft.add_argument(\"--line-order\", choices=[\"none\", \"asc\", \"desc\"], default=\"none\",\n help=\"order constraint on the three distances: asc = left\u003ccenter\u003cright (the infrastructure crosses the frame diagonally from near on the left to far on the right), desc the reverse; penalty 1 if not met (default none)\")\n ft.add_argument(\"--line-w\", type=float, default=0.3, help=\"line_pen weight (default 0.3)\")\n ft.add_argument(\"--line-min\", type=float, default=150, help=\"infrastructure points closer to the camera position than this distance don't count (default 150)\")\n ft.add_argument(\"--line-sample\", type=float, default=50, help=\"take one point every this many meters along the infrastructure line (default 50)\")\n ft.add_argument(\"--line-reach\", type=float, default=3000, help=\"only consider infrastructure points within this distance of the camera position m (default 3000)\")\n ft.add_argument(\"--skip-tag\", action=\"append\", help=\"in --line, skip lines carrying this tag, key=value, repeatable (default electrified=no; none means skip none)\")\n ft.add_argument(\"--top\", type=int, default=20, help=\"how many top entries to print in the terminal and draw in --sheet (default 20)\")\n ft.add_argument(\"--keep\", type=int, default=5000, help=\"maximum number of entries written to the cams list (default 5000)\")\n ft.add_argument(\"--sheet\", help=\"overlay sheet jpg of the top --top\")\n ft.add_argument(\"--overlay\", \"--photo\", dest=\"photo\", help=\"photo for the overlay; if omitted, uses the path recorded in ridge.json\")\n ft.add_argument(\"--sheet-cols\", type=int, default=4, help=\"overlay images per row (default 4)\")\n ft.add_argument(\"--sheet-width\", type=int, default=360, help=\"width of each overlay image px (default 360)\")\n ft.add_argument(\"--allow-missing\", action=\"store_true\", help=MISSING_HELP)\n\n args = ap.parse_args(_neg_coords(sys.argv[1:]))\n if args.cmd == \"scan\":\n _cmd_scan(args)\n return\n if args.cmd == \"ridge\":\n _cmd_ridge(args)\n return\n if args.cmd == \"fit\":\n _cmd_fit(args)\n return\n lat, lon = map(float, args.at.split(\",\"))\n if args.cmd == \"elev\":\n dem = DEM((lat, lon), 300, args.zoom, args.cache, args.proxy, args.allow_missing)\n h = float(dem.sample(np.array([lat]), np.array([lon]))[0])\n if math.isnan(h):\n sys.exit(\"no elevation here: the tile under this point failed to download\")\n print(f\"{h:.0f} m\")\n return\n\n dem = DEM((lat, lon), args.range, args.zoom, args.cache, args.proxy, args.allow_missing)\n ground = float(dem.sample(np.array([lat]), np.array([lon]))[0])\n if math.isnan(ground):\n sys.exit(\"no elevation at the camera position: the tile under it failed to download\")\n eye = args.alt if args.alt is not None else ground + args.height\n az = np.linspace(args.heading - args.hfov / 2, args.heading + args.hfov / 2, args.width)\n if eye \u003c ground + 1:\n print(f\"note: eye height {eye:.0f} m is below the ground at this point {ground:.0f} m, so nearby terrain will be taken as the skyline (--alt is wrong, or you should use --height)\")\n ang, dist = cast(dem, (lat, lon), eye, az % 360, args.range, near=args.near)\n\n if args.cmd == \"profile\":\n runmax = np.max(ang, axis=1)\n far = dist[np.argmax(ang, axis=1)]\n step = max(1, args.width // 180)\n out = [{\"azimuth\": round(float(az[i] % 360), 2), \"skyline_deg\": round(float(runmax[i]), 3), \"skyline_dist_m\": round(float(far[i]))}\n for i in range(0, args.width, step)]\n args.out.write_text(json.dumps({\"at\": [lat, lon], \"eye_alt_m\": round(eye, 1), \"ground_m\": round(ground, 1), \"profile\": out}, indent=1), encoding=\"utf-8\")\n print(f\"ground {ground:.0f} m, eye height {eye:.0f} m → {args.out}\")\n return\n\n vfov = args.vfov or 2 * math.degrees(math.atan(math.tan(math.radians(args.hfov / 2)) * 2 / 3))\n height = int(args.width * vfov / args.hfov)\n im, sky_ang, sky_dist = render(ang, dist, az, args.pitch, vfov, height)\n title = (f\"camera {lat:.5f},{lon:.5f} ground {ground:.0f} m eye {eye:.0f} m heading {args.heading % 360:.0f}° \"\n f\"hfov {args.hfov:.0f}° farthest skyline {np.nanmax(sky_dist) / 1000:.1f} km\")\n im = annotate(im, az, args.pitch, vfov, title)\n if args.roll:\n im = im.rotate(-args.roll, resample=Image.BICUBIC, fillcolor=(255, 255, 255))\n if args.photo and args.overlay:\n ph = open_photo(args.photo, \"photo\").convert(\"RGB\")\n fpx = (ph.width / 2) / math.tan(math.radians(args.hfov / 2))\n cr, sr = math.cos(math.radians(args.roll)), math.sin(math.radians(args.roll))\n pts = []\n for a, e in zip(az, sky_ang):\n if e \u003c= -89:\n continue\n x = fpx * math.tan(math.radians(a - args.heading))\n y = -fpx * math.tan(math.radians(e - args.pitch)) / math.cos(math.radians(a - args.heading))\n pts.append((ph.width / 2 + x * cr - y * sr, ph.height / 2 + x * sr + y * cr))\n ImageDraw.Draw(ph).line(pts, fill=(255, 40, 40), width=max(2, ph.width // 400))\n ov = args.out.with_name(args.out.stem + \"_overlay\" + args.out.suffix)\n ph.save(ov, quality=90)\n print(f\"skyline overlaid on photo -> {ov} (if the red line doesn't match the photo's ridgeline, first adjust --heading/--pitch/--hfov/--roll, then suspect the camera position)\")\n if args.photo:\n ph = open_photo(args.photo, \"photo\").convert(\"RGB\")\n ph = ph.resize((args.width, int(ph.height * args.width / ph.width)))\n S = Image.new(\"RGB\", (args.width, ph.height + im.height), \"white\")\n S.paste(ph, (0, 0))\n S.paste(im, (0, ph.height))\n im = S\n im.save(args.out)\n print(f\"ground {ground:.0f} m, eye height {eye:.0f} m, skyline distance {np.nanmin(sky_dist) / 1000:.1f}–{np.nanmax(sky_dist) / 1000:.1f} km → {args.out}\")\n\n\nif __name__ == \"__main__\":\n # Chinese-locale Windows outputs GBK by default: it crashes on m² or ñ, and any Chinese the agent reads comes out garbled\n sys.stdout.reconfigure(encoding=\"utf-8\")\n sys.stderr.reconfigure(encoding=\"utf-8\", errors=\"backslashreplace\")\n main()\n","content_type":"text/x-python; charset=utf-8","language":"python","size":99947,"content_sha256":"968dfdeabe66a55174e7124263b26d64d53e1ea689c5816e92b154e7fc8ff781"},{"filename":"scripts/tiles.py","content":"#!/usr/bin/env python3\n# /// script\n# requires-python = \">=3.10\"\n# dependencies = [\"pillow\"]\n# ///\n\"\"\"Download satellite tiles, stitch them into one large image (mosaic), and mark points on it.\n\nA .json with the same name is written next to the mosaic (zoom level, top-left tile index, center point);\nevidence.py and the mark subcommand use it to convert lat/lon to image pixels.\n\nGoogle satellite imagery is the default (WGS84, the same coordinate system as GPS, no offset correction needed in China).\n\nExamples:\n tiles.py fetch 22.6050,114.0540 --zoom 18 --radius 4 --out area.jpg\n tiles.py mark area.jpg --points panos.json --out area_panos.jpg --label\n tiles.py mark area.jpg --points cands.json --geojson power.geojson --geojson rail.geojson \\\n --sector 22.6045,114.0520,265,40,3000 --out area_lines.jpg # overlay line features + camera-position view sector\n tiles.py sheet --points cands.json --zoom 18 --out cands_sheet.jpg # one centered thumbnail per candidate point, numbered\n tiles.py px2ll area.jpg --px 812,440 --px 300,95 # mosaic pixels → lat/lon (for a cropped image add --crop x0,y0 --scale s)\n tiles.py sheet --grid \u003cs,w,n,e> --zoom 17 --size 320 --cols 5 --out town.jpg # tile a whole urban area with a grid and look cell by cell (find running tracks, factory buildings)\n\"\"\"\nfrom __future__ import annotations\n\nimport argparse\nfrom _net import curl_args, PROXY_HELP\nimport json\nimport os\nimport re\nimport subprocess\nimport sys\nfrom concurrent.futures import ThreadPoolExecutor\nfrom pathlib import Path\n\nfrom PIL import Image, ImageDraw\n\nsys.path.insert(0, str(Path(__file__).parent))\nimport geo # noqa: E402\n\nSOURCES = {\n \"google\": \"https://mt1.google.com/vt/lyrs=s&x={x}&y={y}&z={z}\",\n \"esri\": \"https://server.arcgisonline.com/ArcGIS/rest/services/World_Imagery/MapServer/tile/{z}/{y}/{x}\",\n}\n\n\ndef _get(url: str, path: Path, proxy: str | None) -> bool:\n if path.exists() and path.stat().st_size > 1000:\n return True\n cmd = [\"curl\", \"-q\", \"-s\", \"-m\", \"60\", \"-o\", str(path), url]\n cmd += curl_args(proxy)\n subprocess.run(cmd, check=False)\n return path.exists() and path.stat().st_size > 1000\n\n\ndef fetch(center: tuple[float, float], zoom: int, radius: int, out: Path, source: str,\n proxy: str | None, cache: Path) -> dict:\n cache.mkdir(parents=True, exist_ok=True)\n gx, gy = geo.ll2px(zoom, *center)\n cx, cy = int(gx // 256), int(gy // 256)\n xs = range(cx - radius, cx + radius + 1)\n ys = range(cy - radius, cy + radius + 1)\n jobs = [(x, y) for x in xs for y in ys]\n\n def job(t):\n x, y = t\n p = cache / f\"{source}_{zoom}_{x}_{y}.jpg\"\n ok = _get(SOURCES[source].format(x=x, y=y, z=zoom), p, proxy)\n return t, p, ok\n\n img = Image.new(\"RGB\", (256 * len(xs), 256 * len(ys)), \"black\")\n failed = 0\n with ThreadPoolExecutor(16) as ex:\n for (x, y), p, ok in ex.map(job, jobs):\n if not ok:\n failed += 1\n continue\n try:\n img.paste(Image.open(p), ((x - xs[0]) * 256, (y - ys[0]) * 256))\n except Exception:\n failed += 1\n img.save(out, quality=90)\n meta = {\"zoom\": zoom, \"origin_tile\": [xs[0], ys[0]], \"center\": list(center), \"source\": source,\n \"size\": list(img.size), \"m_per_px\": geo.meters_per_px(zoom, center[0]), \"failed_tiles\": failed}\n out.with_suffix(\".json\").write_text(json.dumps(meta, indent=2), encoding=\"utf-8\")\n return meta\n\n\nclass Mosaic:\n \"\"\"lat/lon ↔ mosaic pixels.\"\"\"\n\n def __init__(self, image: Path):\n self.meta = json.loads(Path(image).with_suffix(\".json\").read_text(encoding=\"utf-8\"))\n self.z = self.meta[\"zoom\"]\n self.ox, self.oy = self.meta[\"origin_tile\"]\n\n def to_px(self, lat: float, lon: float) -> tuple[float, float]:\n x, y = geo.ll2px(self.z, lat, lon)\n return x - self.ox * 256, y - self.oy * 256\n\n def to_ll(self, x: float, y: float) -> tuple[float, float]:\n return geo.px2ll(self.z, x + self.ox * 256, y + self.oy * 256)\n\n\nPALETTE = [\"yellow\", \"cyan\", \"magenta\", \"lime\", \"orange\", \"white\", \"red\", \"deepskyblue\"]\n\n\nLINE_COLORS = [\"orange\", \"deepskyblue\", \"magenta\", \"lime\", \"white\", \"red\"]\n\n\ndef _draw_geojson(d: ImageDraw.ImageDraw, m: \"Mosaic\", gj: dict, color: str) -> int:\n \"\"\"Draw GeoJSON lines, polygons and points on the mosaic (line width 3, polygons as outlines only). Returns how many features were drawn.\"\"\"\n n = 0\n for ft in gj.get(\"features\", []):\n g = ft.get(\"geometry\") or {}\n t, cs = g.get(\"type\"), g.get(\"coordinates\")\n if not cs:\n continue\n lines = {\"LineString\": [cs], \"MultiLineString\": cs, \"Polygon\": cs,\n \"MultiPolygon\": [ring for poly in cs for ring in poly]}.get(t)\n if lines is not None:\n for ln in lines:\n xy = [m.to_px(c[1], c[0]) for c in ln]\n if len(xy) >= 2:\n d.line(xy, fill=color, width=3)\n n += 1\n elif t == \"Point\":\n x, y = m.to_px(cs[1], cs[0])\n d.rectangle([x - 3, y - 3, x + 3, y + 3], outline=color, width=2)\n n += 1\n return n\n\n\ndef _draw_sector(d: ImageDraw.ImageDraw, m: \"Mosaic\", spec: str) -> None:\n \"\"\"lat,lon,heading,horizontal FOV,radius m → camera-position view sector.\"\"\"\n lat, lon, hd, fov, rng = map(float, spec.split(\",\"))\n pts = [m.to_px(lat, lon)]\n steps = max(4, int(fov // 3))\n for k in range(steps + 1):\n a = hd - fov / 2 + fov * k / steps\n pts.append(m.to_px(*geo.dest((lat, lon), a, rng)))\n d.line(pts + [pts[0]], fill=\"yellow\", width=3)\n mid = m.to_px(*geo.dest((lat, lon), hd, rng))\n d.line([pts[0], mid], fill=\"yellow\", width=1)\n x, y = pts[0]\n d.ellipse([x - 7, y - 7, x + 7, y + 7], fill=\"red\", outline=\"yellow\", width=2)\n\n\ndef mark(image: Path, points: dict, out: Path, label: bool, geojsons: list[Path] | None = None,\n sectors: list[str] | None = None) -> None:\n \"\"\"points: {name: [lat, lon]} (wgs). Colored by the name prefix before the first space; with --label, writes the first 12 characters of the name.\"\"\"\n m = Mosaic(image)\n img = Image.open(image).convert(\"RGB\")\n d = ImageDraw.Draw(img)\n for i, gp in enumerate(geojsons or []):\n n = _draw_geojson(d, m, json.loads(Path(gp).read_text(encoding=\"utf-8\")), LINE_COLORS[i % len(LINE_COLORS)])\n print(f\"{gp}: {n} features, color {LINE_COLORS[i % len(LINE_COLORS)]}\")\n for sp in sectors or []:\n _draw_sector(d, m, sp)\n groups: dict[str, str] = {}\n for name, ll in points.items():\n lat, lon = ll[0], ll[1]\n prefix = str(name).split(\" \")[0].rstrip(\"0123456789\") or \"_\"\n color = groups.setdefault(prefix, PALETTE[len(groups) % len(PALETTE)])\n x, y = m.to_px(lat, lon)\n d.ellipse([x - 5, y - 5, x + 5, y + 5], fill=color, outline=\"black\")\n if label:\n d.text((x + 7, y - 6), str(name)[:12], fill=color)\n img.save(out, quality=90)\n\n\ndef grid_points(bbox: str, zoom: int, size: int, step: float | None) -> dict:\n \"\"\"Tile s,w,n,e into grid points; default spacing = 90% of the width each cell covers.\"\"\"\n import math\n\n s, w, n, e = (float(v) for v in bbox.split(\",\"))\n mpp = 156543.03392 * math.cos(math.radians((s + n) / 2)) / 2 ** zoom\n step = step or size * mpp * 0.9\n dlat, dlon = step / 110574, step / (111320 * math.cos(math.radians((s + n) / 2)))\n pts, r, lat = {}, 0, n - dlat / 2\n while lat > s:\n c, lon = 0, w + dlon / 2\n while lon \u003c e:\n pts[f\"r{r:02d}c{c:02d}\"] = [round(lat, 6), round(lon, 6)]\n c, lon = c + 1, lon + dlon\n r, lat = r + 1, lat - dlat\n return pts\n\n\ndef sheet(points: dict, zoom: int, size: int, cols: int, out: Path, source: str, proxy: str | None, cache: Path) -> list[Path]:\n \"\"\"One size×size satellite thumbnail centered on each candidate point, with number and name, laid out on one page (more pages automatically when there are too many).\"\"\"\n cache.mkdir(parents=True, exist_ok=True)\n items = list(points.items())\n per = cols * cols\n pages = []\n for pi in range(0, len(items), per):\n chunk = items[pi:pi + per]\n rows = (len(chunk) + cols - 1) // cols\n S = Image.new(\"RGB\", (cols * size, rows * size), \"black\")\n dr = ImageDraw.Draw(S)\n for k, (name, ll) in enumerate(chunk):\n gx, gy = geo.ll2px(zoom, ll[0], ll[1])\n x0, y0 = gx - size / 2, gy - size / 2\n tile = Image.new(\"RGB\", (size, size), \"gray\")\n for tx in range(int(x0 // 256), int((x0 + size) // 256) + 1):\n for ty in range(int(y0 // 256), int((y0 + size) // 256) + 1):\n p = cache / f\"{source}_{zoom}_{tx}_{ty}.jpg\"\n if _get(SOURCES[source].format(x=tx, y=ty, z=zoom), p, proxy):\n try:\n tile.paste(Image.open(p), (int(tx * 256 - x0), int(ty * 256 - y0)))\n except Exception: # noqa: BLE001\n pass\n cx, cy = (k % cols) * size, (k // cols) * size\n S.paste(tile, (cx, cy))\n c = size / 2\n dr.line([cx + c - 12, cy + c, cx + c + 12, cy + c], fill=\"red\", width=2)\n dr.line([cx + c, cy + c - 12, cx + c, cy + c + 12], fill=\"red\", width=2)\n dr.rectangle([cx, cy, cx + size, cy + 20], fill=\"black\")\n dr.text((cx + 4, cy + 3), f\"#{pi + k + 1} {str(name)[:28]}\", fill=\"yellow\")\n o = out if pi == 0 else out.with_name(f\"{out.stem}_{pi // per + 1}{out.suffix}\")\n S.save(o, quality=88)\n pages.append(o)\n return pages\n\n\n\ndef _neg_coords(argv: list[str]) -> list[str]:\n \"\"\"argparse treats negative coordinates like -1.45,-48.5 as option names; prefixing a space makes them plain values (float ignores the space). Needed for every puzzle in the southern or western hemisphere.\"\"\"\n return [\" \" + a if re.match(r\"^-\\d[\\d.]*(,-?[\\d.]+)+$\", a) else a for a in argv]\n\n\ndef main() -> None:\n ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)\n sub = ap.add_subparsers(dest=\"cmd\", required=True)\n\n f = sub.add_parser(\"fetch\")\n f.add_argument(\"center\", help=\"lat,lon (wgs)\")\n f.add_argument(\"--zoom\", type=int, default=18, help=\"17≈1.1m/px for an area, 19≈0.28m/px for a single building\")\n f.add_argument(\"--radius\", type=int, default=4, help=\"rings of tiles around the center tile, 4 → 9x9 tiles\")\n f.add_argument(\"--out\", type=Path, required=True)\n f.add_argument(\"--source\", choices=list(SOURCES), default=\"google\")\n f.add_argument(\"--proxy\", default=os.environ.get(\"GEO_PROXY\"), help=PROXY_HELP)\n f.add_argument(\"--cache\", type=Path, default=Path(\".geo-cache/tiles\"))\n\n mk = sub.add_parser(\"mark\")\n mk.add_argument(\"image\", type=Path)\n mk.add_argument(\"--points\", type=Path, help=\"JSON: {name: [lat, lon]}\")\n mk.add_argument(\"--out\", type=Path, required=True)\n mk.add_argument(\"--label\", action=\"store_true\")\n mk.add_argument(\"--geojson\", type=Path, action=\"append\", help=\"overlay GeoJSON (output of osm.py geom), repeatable, one color per file\")\n mk.add_argument(\"--sector\", action=\"append\", help=\"camera-position view sector lat,lon,heading,horizontal FOV,radius m, repeatable\")\n\n sh = sub.add_parser(\"sheet\", help=\"centered satellite thumbnail for each candidate point, laid out as a numbered comparison page\")\n shg = sh.add_mutually_exclusive_group(required=True)\n shg.add_argument(\"--points\", type=Path, help=\"JSON {name: [lat, lon]}\")\n shg.add_argument(\"--grid\", help=\"s,w,n,e: cover an area with a grid and render cell by cell (cell name r\u003crow>c\u003ccol>: r00 northernmost, c00 westernmost; center coordinates written to \u003cout>.cells.json)\")\n sh.add_argument(\"--step\", type=float, help=\"--grid cell spacing (meters); default leaves 10%% overlap of the area each cell covers\")\n sh.add_argument(\"--zoom\", type=int, default=18)\n sh.add_argument(\"--size\", type=int, default=320, help=\"pixels per cell\")\n sh.add_argument(\"--cols\", type=int, default=4, help=\"cols×cols cells per page\")\n sh.add_argument(\"--out\", type=Path, required=True)\n sh.add_argument(\"--source\", choices=list(SOURCES), default=\"google\")\n sh.add_argument(\"--proxy\", default=os.environ.get(\"GEO_PROXY\"), help=PROXY_HELP)\n sh.add_argument(\"--cache\", type=Path, default=Path(\".geo-cache/tiles\"))\n\n pl = sub.add_parser(\"px2ll\", help=\"pixels on a mosaic (or a cropped/scaled copy of it) → lat/lon\")\n pl.add_argument(\"image\", type=Path, help=\"mosaic from tiles.py fetch (needs the same-name .json next to it)\")\n pl.add_argument(\"--px\", action=\"append\", required=True, help=\"x,y, repeatable; if you measured on a cropped/scaled image, pair with --crop/--scale\")\n pl.add_argument(\"--crop\", default=\"0,0\", help=\"top-left x0,y0, in the original mosaic, of the image you measured pixels on\")\n pl.add_argument(\"--scale\", type=float, default=1.0, help=\"scale of that image relative to the original mosaic (0.5 for half size)\")\n pl.add_argument(\"--out\", type=Path, help=\"write {p1: [lat, lon], ...}\")\n\n args = ap.parse_args(_neg_coords(sys.argv[1:]))\n if args.cmd == \"px2ll\":\n m = Mosaic(args.image)\n x0, y0 = (float(v) for v in args.crop.split(\",\"))\n pts = {}\n for i, s in enumerate(args.px, 1):\n x, y = (float(v) for v in s.split(\",\"))\n lat, lon = m.to_ll(x0 + x / args.scale, y0 + y / args.scale)\n pts[f\"p{i}\"] = [round(lat, 6), round(lon, 6)]\n print(f\"p{i} ({x:.0f},{y:.0f}) → {lat:.6f},{lon:.6f}\")\n if args.out:\n args.out.write_text(json.dumps(pts, indent=1), encoding=\"utf-8\")\n print(f\"-> {args.out}\")\n return\n if args.cmd == \"sheet\":\n pts = json.loads(args.points.read_text(encoding=\"utf-8\")) if args.points else grid_points(args.grid, args.zoom, args.size, args.step)\n if args.grid:\n args.out.with_suffix(\".cells.json\").write_text(json.dumps(pts, indent=1), encoding=\"utf-8\")\n print(f\"grid: {len(pts)} cells (cell name r\u003crow>c\u003ccol>, rows north to south, columns west to east; cell center coordinates -> {args.out.with_suffix('.cells.json')})\")\n for p in sheet(pts, args.zoom, args.size, args.cols, args.out, args.source, args.proxy, args.cache):\n print(p)\n return\n if args.cmd == \"fetch\":\n lat, lon = map(float, args.center.split(\",\"))\n meta = fetch((lat, lon), args.zoom, args.radius, args.out, args.source, args.proxy, args.cache)\n print(json.dumps(meta))\n elif args.cmd == \"mark\":\n if not (args.points or args.geojson or args.sector):\n ap.error(\"mark needs at least one of --points, --geojson, --sector\")\n mark(args.image, json.loads(args.points.read_text(encoding=\"utf-8\")) if args.points else {}, args.out, args.label,\n args.geojson, args.sector)\n print(args.out)\n\n\nif __name__ == \"__main__\":\n # Chinese-locale Windows writes GBK by default: m², ñ make it crash, and the Chinese the agent reads comes out garbled\n sys.stdout.reconfigure(encoding=\"utf-8\")\n sys.stderr.reconfigure(encoding=\"utf-8\", errors=\"backslashreplace\")\n main()\n","content_type":"text/x-python; charset=utf-8","language":"python","size":15333,"content_sha256":"4af1f09393915a036176cf52a7fde8cffd3def05deac916bb8910e9e54472f31"},{"filename":"tests/test_board.py","content":"#!/usr/bin/env python3\n# /// script\n# requires-python = \">=3.10\"\n# dependencies = []\n# ///\n\"\"\"Offline board regressions. Run: uv run skills/geo-sleuth/tests/test_board.py\"\"\"\nfrom __future__ import annotations\n\nimport json\nfrom pathlib import Path\nimport subprocess\nimport sys\nimport tempfile\nimport unittest\n\nSCRIPTS = Path(__file__).resolve().parents[1] / \"scripts\"\nsys.path.insert(0, str(SCRIPTS))\nimport board # noqa: E402\n\n\nclass CorridorPendingTests(unittest.TestCase):\n def setUp(self):\n self.tmp = tempfile.TemporaryDirectory()\n self.addCleanup(self.tmp.cleanup)\n self.root = Path(self.tmp.name)\n self.path = self.root / \"board.json\"\n self.output = self.root / \"scan.json\"\n self.output.write_text('{\"hits\": []}', encoding=\"utf-8\")\n self.run_board(\"init\", \"--photo\", \"photo.jpg\")\n\n def run_board(self, *args, ok=True):\n result = subprocess.run(\n [sys.executable, str(SCRIPTS / \"board.py\"), \"--board\", str(self.path), *args],\n cwd=self.root, capture_output=True, text=True, encoding=\"utf-8\",\n )\n if ok:\n self.assertEqual(result.returncode, 0, result.stdout + result.stderr)\n else:\n self.assertNotEqual(result.returncode, 0, result.stdout + result.stderr)\n return result\n\n def read_board(self):\n return json.loads(self.path.read_text(encoding=\"utf-8\"))\n\n def clue(self, kind=\"infra\", status=\"observed\", *extra):\n self.run_board(\"clue\", \"A spatial observation or its computed test\", \"--kind\", kind,\n \"--status\", status, *extra)\n\n def pending(self):\n return board._corridor_pending(self.read_board())\n\n def test_alias_is_normalized_when_recorded(self):\n self.clue(\"infrastructure\")\n self.assertEqual(self.read_board()[\"clues\"][\"K1\"][\"kind\"], \"infra\")\n self.assertEqual(self.pending(), [\"K1\"])\n self.assertIn(\"infrastructure clue K1\", self.run_board(\"next\").stdout)\n\n def test_legacy_alias_is_detected_without_rewriting_board(self):\n self.clue()\n data = self.read_board()\n data[\"clues\"][\"K1\"][\"kind\"] = \"infrastructure\"\n self.path.write_text(json.dumps(data), encoding=\"utf-8\")\n before = self.path.read_bytes()\n self.assertEqual(self.pending(), [\"K1\"])\n self.assertIn(\"infrastructure clue K1\", self.run_board(\"next\").stdout)\n self.assertEqual(self.path.read_bytes(), before)\n\n def test_unlinked_computations_do_not_clear_infrastructure(self):\n self.clue()\n self.clue(\"infrastructure\", \"read\")\n for kind in (\"terrain\", \"corridor\", \"infra\"):\n with self.subTest(kind=kind):\n self.clue(kind, \"computed\", \"--file\", str(self.output))\n self.assertEqual(self.pending(), [\"K1\", \"K2\"])\n\n def test_computation_clears_only_explicitly_linked_clues(self):\n self.clue()\n self.clue(\"infrastructure\", \"read\")\n self.clue(\"terrain\", \"computed\", \"--file\", str(self.output), \"--resolves\", \"K1\")\n self.assertEqual(self.pending(), [\"K2\"])\n self.assertIn(\"infrastructure clue K2\", self.run_board(\"next\").stdout)\n self.assertEqual(self.read_board()[\"clues\"][\"K3\"][\"resolves\"], [\"K1\"])\n self.clue(\"corridor\", \"computed\", \"--file\", str(self.output),\n \"--resolves\", \"K1\", \"--resolves\", \"K2\")\n self.assertEqual(self.pending(), [])\n self.assertNotIn(\"infrastructure clue\", self.run_board(\"next\").stdout)\n # Recording a computation is separate from scoring candidate evidence.\n self.assertEqual(self.read_board()[\"evidence\"], [])\n\n def test_unrelated_explicit_link_does_not_clear_infrastructure(self):\n self.clue()\n self.clue(\"terrain\")\n self.clue(\"terrain\", \"computed\", \"--file\", str(self.output), \"--resolves\", \"K2\")\n self.assertEqual(self.pending(), [\"K1\"])\n\n def test_invalid_completion_is_rejected_without_mutation(self):\n self.clue()\n before = self.path.read_bytes()\n cases = [\n (\"--status\", \"observed\", \"--file\", str(self.output), \"--resolves\", \"K1\"),\n (\"--status\", \"computed\", \"--file\", str(self.output), \"--resolves\", \"K99\"),\n (\"--status\", \"computed\", \"--file\", str(self.output), \"--resolves\", \"K1\", \"--resolves\", \"K99\"),\n (\"--status\", \"computed\", \"--resolves\", \"K1\"),\n (\"--status\", \"computed\", \"--file\", str(self.root / \"missing.json\"), \"--resolves\", \"K1\"),\n (\"--status\", \"computed\", \"--file\", str(self.root), \"--resolves\", \"K1\"),\n ]\n for args in cases:\n with self.subTest(args=args):\n self.run_board(\"clue\", \"An invalid completion\", \"--kind\", \"terrain\", *args, ok=False)\n self.assertEqual(self.path.read_bytes(), before)\n\n def test_missing_output_no_longer_counts_as_completion(self):\n self.clue()\n self.clue(\"terrain\", \"computed\", \"--file\", str(self.output), \"--resolves\", \"K1\")\n self.assertEqual(self.pending(), [])\n self.output.unlink()\n self.assertEqual(self.pending(), [\"K1\"])\n\n def test_relative_output_link_survives_a_different_working_directory(self):\n self.clue()\n self.clue(\"terrain\", \"computed\", \"--file\", \"scan.json\", \"--resolves\", \"K1\")\n saved = self.read_board()[\"clues\"][\"K2\"][\"file\"]\n self.assertEqual(Path(saved), self.output.resolve())\n self.assertEqual(self.pending(), [])\n result = subprocess.run(\n [sys.executable, str(SCRIPTS / \"board.py\"), \"--board\", str(self.path), \"next\"],\n cwd=self.root.parent, capture_output=True, text=True, encoding=\"utf-8\",\n )\n self.assertEqual(result.returncode, 0, result.stdout + result.stderr)\n self.assertNotIn(\"infrastructure clue\", result.stdout)\n\n\nclass LevelGapTests(unittest.TestCase):\n \"\"\"A point added under the city instead of under its road: its evidence never reached the road ranking, and report named another road.\"\"\"\n\n def setUp(self):\n self.tmp = tempfile.TemporaryDirectory()\n self.addCleanup(self.tmp.cleanup)\n self.root = Path(self.tmp.name)\n self.path = self.root / \"board.json\"\n (self.root / \"match.jpg\").write_bytes(b\"x\")\n (self.root / \"evidence.jpg\").write_bytes(b\"x\")\n run = CorridorPendingTests.run_board\n self.run_board = lambda *a, ok=True: run(self, *a, ok=ok)\n self.run_board(\"init\", \"--photo\", \"photo.jpg\")\n self.run_board(\"add\", \"City\", \"--level\", \"district\")\n self.run_board(\"add\", \"A ST 100\", \"--level\", \"road\", \"--parent\", \"City\", \"--bbox\", \"49.20,-123.20,49.201,-123.19\")\n self.run_board(\"add\", \"B ST 200\", \"--level\", \"road\", \"--parent\", \"City\", \"--bbox\", \"49.21,-123.20,49.211,-123.19\")\n self.run_board(\"clue\", \"street view matches\", \"--kind\", \"municipal\", \"--status\", \"computed\", \"--file\", \"match.jpg\")\n self.run_board(\"evidence\", \"--clue\", \"K1\", \"--for\", \"A ST 100:2\", \"--why\", \"partial\", \"--file\", \"match.jpg\")\n\n def add_point(self, *extra):\n out = self.run_board(\"add\", \"spot\", \"--level\", \"point\", \"--parent\", \"City\", \"--bbox\", \"49.2159,-123.1095,49.2162,-123.1091\", *extra)\n self.run_board(\"evidence\", \"--clue\", \"K1\", \"--for\", \"spot:10\", \"--why\", \"invariant features\", \"--file\", \"match.jpg\")\n return out\n\n def ranking(self):\n return [r[\"name\"] for r in board._scores(json.loads(self.path.read_text(encoding=\"utf-8\")), \"road\", \"uniform\")]\n\n def test_skipped_level_is_flagged_everywhere(self):\n self.assertIn(\"2 road candidates\", self.add_point().stdout)\n check = self.run_board(\"check\")\n self.assertIn(\"FAIL spot (point) sits directly under City\", check.stdout)\n self.assertIn(\"leaves the answer out\", check.stdout)\n result = self.root / \"result.json\"\n result.write_text(json.dumps({\"lat\": 49.216, \"lon\": -123.1093, \"radius_m\": 20}), encoding=\"utf-8\")\n report = self.run_board(\"report\", \"--merge\", str(result)).stdout\n self.assertIn(\"outside the board's main answer A ST 100\", report)\n self.assertTrue(json.loads(result.read_text(encoding=\"utf-8\"))[\"board\"][\"level_gaps\"])\n\n def test_moving_under_its_road_carries_support_up(self):\n self.add_point()\n self.run_board(\"add\", \"W 60 AVE 100\", \"--level\", \"road\", \"--parent\", \"City\", \"--bbox\", \"49.2158,-123.1112,49.2164,-123.1075\")\n self.assertIn(\"lies inside W 60 AVE 100\", self.run_board(\"check\", ok=True).stdout)\n self.run_board(\"move\", \"spot\", \"--parent\", \"W 60 AVE 100\")\n check = self.run_board(\"check\").stdout\n self.assertNotIn(\"FAIL\", check)\n self.assertIn(\"main answer W 60 AVE 100\", check)\n self.assertEqual(self.ranking()[0], \"W 60 AVE 100\")\n result = self.root / \"result.json\"\n result.write_text(json.dumps({\"lat\": 49.216, \"lon\": -123.1093}), encoding=\"utf-8\")\n self.assertNotIn(\"WARN\", self.run_board(\"report\", \"--merge\", str(result)).stdout)\n\n def test_against_on_a_point_does_not_reach_its_road(self):\n self.run_board(\"add\", \"spot2\", \"--level\", \"point\", \"--parent\", \"B ST 200\")\n before = self.ranking()\n self.run_board(\"evidence\", \"--clue\", \"K1\", \"--against\", \"spot2:0.1\", \"--why\", \"no match\", \"--file\", \"match.jpg\")\n self.assertEqual(self.ranking(), before)\n b = json.loads(self.path.read_text(encoding=\"utf-8\"))\n self.assertEqual(board._inherited(b, \"B ST 200\", set()), [])\n\n def test_same_clue_counts_once(self):\n self.run_board(\"add\", \"spot3\", \"--level\", \"point\", \"--parent\", \"A ST 100\")\n self.run_board(\"evidence\", \"--clue\", \"K1\", \"--for\", \"spot3:10\", \"--why\", \"same clue\", \"--file\", \"match.jpg\")\n b = json.loads(self.path.read_text(encoding=\"utf-8\"))\n self.assertEqual(board._inherited(b, \"A ST 100\", {\"K1\"}), []) # A ST 100 already has its own K1 evidence\n\n def test_recording_a_clue_again_replaces_instead_of_stacking(self):\n before = [e for e in json.loads(self.path.read_text(encoding=\"utf-8\"))[\"evidence\"] if e[\"candidate\"] == \"A ST 100\"]\n out = self.run_board(\"evidence\", \"--clue\", \"K1\", \"--for\", \"A ST 100:2\", \"--why\", \"same comparison again\", \"--file\", \"match.jpg\").stdout\n self.assertIn(\"replaced it instead of counting it twice\", out)\n after = [e for e in json.loads(self.path.read_text(encoding=\"utf-8\"))[\"evidence\"] if e[\"candidate\"] == \"A ST 100\"]\n self.assertEqual(len(after), len(before))\n self.run_board(\"evidence\", \"--clue\", \"K1\", \"--for\", \"A ST 100:1.5\", \"--file\", \"match.jpg\")\n b = json.loads(self.path.read_text(encoding=\"utf-8\"))\n self.assertEqual([e[\"lr\"] for e in b[\"evidence\"] if e[\"candidate\"] == \"A ST 100\"], [1.5])\n self.assertEqual(len({e[\"id\"] for e in b[\"evidence\"]}), len(b[\"evidence\"]))\n\n def test_revision_removes_legacy_duplicates_without_reusing_ids(self):\n b = json.loads(self.path.read_text(encoding=\"utf-8\"))\n b[\"evidence\"] += [{**b[\"evidence\"][0], \"id\": \"E2\", \"lr\": 3},\n {**b[\"evidence\"][0], \"id\": \"E3\", \"candidate\": \"B ST 200\", \"lr\": 1}]\n self.path.write_text(json.dumps(b), encoding=\"utf-8\")\n self.run_board(\"evidence\", \"--clue\", \"K1\", \"--against\", \"A ST 100:0.5\", \"--why\", \"corrected\")\n b = json.loads(self.path.read_text(encoding=\"utf-8\"))\n self.assertEqual([e[\"lr\"] for e in b[\"evidence\"] if e[\"candidate\"] == \"A ST 100\"], [0.5])\n self.assertEqual(b[\"evidence\"][0][\"id\"], \"E1\")\n self.run_board(\"clue\", \"Independent observation\", \"--kind\", \"municipal\", \"--status\", \"observed\")\n self.run_board(\"evidence\", \"--clue\", \"K2\", \"--for\", \"B ST 200:2\")\n b = json.loads(self.path.read_text(encoding=\"utf-8\"))\n self.assertEqual(len({e[\"id\"] for e in b[\"evidence\"]}), len(b[\"evidence\"]))\n self.assertEqual(self.ranking()[0], \"B ST 200\")\n\n def test_independent_clues_still_accumulate_and_invalid_revision_is_atomic(self):\n self.run_board(\"clue\", \"Independent observation\", \"--kind\", \"municipal\", \"--status\", \"observed\")\n self.run_board(\"evidence\", \"--clue\", \"K2\", \"--for\", \"A ST 100:3\")\n b = json.loads(self.path.read_text(encoding=\"utf-8\"))\n self.assertEqual([e[\"lr\"] for e in b[\"evidence\"]], [2, 3])\n before = self.path.read_bytes()\n self.run_board(\"evidence\", \"--clue\", \"K1\", \"--for\", \"A ST 100:4\", \"--for\", \"missing:2\", ok=False)\n self.assertEqual(self.path.read_bytes(), before)\n\n def test_apply_can_revise_the_same_clue_and_handles_gaps_in_ids(self):\n b = json.loads(self.path.read_text(encoding=\"utf-8\"))\n b[\"evidence\"][0][\"id\"] = \"E2\"\n self.path.write_text(json.dumps(b), encoding=\"utf-8\")\n self.run_board(\"clue\", \"Telephone prefix\", \"--kind\", \"calling-code\", \"--status\", \"read\")\n args = (\"apply\", \"--kind\", \"calling-code\", \"--value\", \"+33\", \"--clue\", \"K2\")\n self.run_board(*args)\n before = json.loads(self.path.read_text(encoding=\"utf-8\"))[\"evidence\"]\n self.assertEqual(len({e[\"id\"] for e in before}), len(before))\n self.run_board(*args)\n after = json.loads(self.path.read_text(encoding=\"utf-8\"))[\"evidence\"]\n self.assertEqual(after, before)\n\n def test_move_rejects_finer_parent(self):\n self.run_board(\"add\", \"spot4\", \"--level\", \"point\", \"--parent\", \"A ST 100\")\n self.run_board(\"move\", \"A ST 100\", \"--parent\", \"spot4\", ok=False)\n\n\nif __name__ == \"__main__\":\n unittest.main()\n","content_type":"text/x-python; charset=utf-8","language":"python","size":13484,"content_sha256":"05f64df0d12ac6cfc7a9e49469720ecac6161210bd4c004a1fb1aa785d148e5b"},{"filename":"tests/test_gazetteer.py","content":"#!/usr/bin/env python3\n# /// script\n# requires-python = \">=3.10\"\n# dependencies = []\n# ///\n\"\"\"Offline gazetteer regressions: bilingual OSM names and region-pack extra children. Run: uv run skills/geo-sleuth/tests/test_gazetteer.py\"\"\"\nfrom __future__ import annotations\n\nfrom pathlib import Path\nimport json\nimport re\nimport subprocess\nimport sys\nimport tempfile\nimport unittest\nfrom unittest.mock import patch\n\nSCRIPTS = Path(__file__).resolve().parents[1] / \"scripts\"\nsys.path.insert(0, str(SCRIPTS))\nimport gazetteer # noqa: E402\n\nBOUNDS = {\"minlat\": 1.0, \"minlon\": 2.0, \"maxlat\": 3.0, \"maxlon\": 4.0}\n\n\ndef rel(osm_id: int, level: int, **tags) -> dict:\n tags = {k.replace(\"_\", \":\").replace(\":Hans\", \"-Hans\"): v for k, v in tags.items()}\n return {\"type\": \"relation\", \"id\": osm_id, \"bounds\": BOUNDS, \"tags\": {\"admin_level\": str(level), **tags}}\n\n\n# Tags as OSM has them (2026-10): the ethnic autonomous regions and Hong Kong carry a second script in `name`\nINNER_MONGOLIA = rel(161349, 4, name=\"内蒙古自治区 ᠦᠪᠦᠷ ᠮᠣᠩᠭᠤᠯ ᠤᠨ ᠥᠪᠡᠷᠲᠡᠭᠡᠨ ᠵᠠᠰᠠᠬᠣ ᠣᠷᠣᠨ\", name_zh=\"内蒙古自治区\", name_en=\"Inner Mongolia\")\nTIBET = rel(153292, 4, name=\"西藏自治区 བོད་རང་སྐྱོང་ལྗོངས།\", name_zh=\"西藏自治区\", name_en=\"Xizang\")\nHONG_KONG_4 = rel(20044132, 4, name=\"香港 Hong Kong\", name_zh=\"香港\", name_en=\"Hong Kong\")\nHONG_KONG_3 = rel(913110, 3, name=\"香港 Hong Kong\", name_zh=\"香港\", name_en=\"Hong Kong\")\nMACAU = rel(1867188, 3, name=\"澳門 Macau\", name_zh=\"澳門\", name_zh_Hans=\"澳门\", name_en=\"Macau\")\nTAIWAN = rel(449220, 2, name=\"臺灣\", name_zh_Hans=\"台湾\", name_en=\"Taiwan\")\nSICHUAN = rel(913068, 4, name=\"四川省\", name_zh=\"四川省\", name_en=\"Sichuan\")\nRIO = rel(57963, 4, **{\"name\": \"Rio de Janeiro\", \"name:pt\": \"Rio de Janeiro\", \"name:en\": \"Rio de Janeiro\"})\n\n\nclass BilingualNames(unittest.TestCase):\n def test_rows_use_the_plain_name_and_keep_every_variant(self):\n rows = {r[\"osm_id\"]: r for r in gazetteer._rel_rows({\"elements\": [INNER_MONGOLIA, TIBET, HONG_KONG_4, MACAU, SICHUAN, RIO]})}\n self.assertEqual(rows[161349][\"name\"], \"内蒙古自治区\")\n self.assertEqual(rows[153292][\"name\"], \"西藏自治区\")\n self.assertEqual(rows[20044132][\"name\"], \"香港\")\n self.assertEqual(rows[1867188][\"name\"], \"澳門\")\n self.assertIn(\"澳门\", rows[1867188][\"names\"])\n self.assertEqual(rows[913068][\"name\"], \"四川省\")\n self.assertEqual(rows[57963][\"name\"], \"Rio de Janeiro\") # a name with spaces that isn't bilingual stays whole\n\n def test_duplicate_elements_from_a_union_query_are_listed_once(self):\n self.assertEqual(len(gazetteer._rel_rows({\"elements\": [TIBET, TIBET]})), 1)\n\n def test_name_lookup_matches_every_name_key_and_escapes_quotes(self):\n sel = gazetteer._by_name('A \"B\"', '[\"admin_level\"=\"4\"]', \"(area.p)\")\n for key in gazetteer.regions.osm_name_keys():\n self.assertIn(f'rel(area.p)[\"{key}\"=\"A \\\\\"B\\\\\"\"][\"boundary\"=\"administrative\"][\"admin_level\"=\"4\"];', sel)\n\n def test_extra_name_keys_come_from_region_packs_not_core(self):\n self.assertEqual(gazetteer.regions.osm_name_keys(), (\"name\", \"name:zh\", \"name:zh-Hans\")) # declared by the CN pack\n with patch.object(gazetteer.regions, \"packs\", return_value={}):\n self.assertEqual(gazetteer.regions.osm_name_keys(), (\"name\",))\n self.assertEqual(gazetteer._by_name(\"X\"), '(rel[\"name\"=\"X\"][\"boundary\"=\"administrative\"];)')\n self.assertEqual(gazetteer.osm._area_sel(\"X\", \"a\"), '(area[\"name\"=\"X\"];)->.a;')\n\n\nclass AlsoChildren(unittest.TestCase):\n def setUp(self):\n self.china = {\"name\": \"中国\", \"admin_level\": 2, \"osm_id\": 270056}\n self.provinces = gazetteer._rel_rows({\"elements\": [SICHUAN, INNER_MONGOLIA, HONG_KONG_4]})\n\n def test_pack_lists_macau_and_taiwan_once_without_repeating_hong_kong(self):\n reply = {\"elements\": [HONG_KONG_3, MACAU, TAIWAN]}\n with patch.object(gazetteer.osm, \"run\", return_value=reply) as run:\n extra, missing = gazetteer.also_children(self.china, self.provinces, None, Path(\".\"))\n self.assertEqual([r[\"name\"] for r in extra], [\"澳門\", \"臺灣\"])\n self.assertEqual(missing, [])\n query = run.call_args.args[0]\n self.assertIn('\"name:zh-Hans\"=\"澳门\"', query)\n self.assertNotIn('\"香港\"', query) # already on the list, not asked again\n\n def test_names_osm_lacks_are_reported(self):\n with patch.object(gazetteer.osm, \"run\", return_value={\"elements\": [MACAU]}):\n extra, missing = gazetteer.also_children(self.china, self.provinces, None, Path(\".\"))\n self.assertEqual([r[\"name\"] for r in extra], [\"澳門\"])\n self.assertEqual(missing, [\"臺灣\"])\n\n def test_only_a_country_parent_gets_extra_children(self):\n province = {\"name\": \"中国\", \"admin_level\": 4, \"osm_id\": 1}\n with patch.object(gazetteer.osm, \"run\") as run:\n self.assertEqual(gazetteer.also_children(province, self.provinces, None, Path(\".\")), ([], []))\n run.assert_not_called()\n\n\nclass CountryNameResolution(unittest.TestCase):\n \"\"\"Filter the fixture by the actual selectors, so an incorrect query cannot get the right country.\"\"\"\n\n def setUp(self):\n def place(osm_id, level, **tags):\n return rel(osm_id, level, boundary=\"administrative\", **tags)\n self.china = place(270056, 2, name=\"中国\", name_zh=\"中国\", name_en=\"China\", **{\"ISO3166-1\": \"CN\"})\n self.swiss = place(51701, 2, name=\"Schweiz/Suisse/Svizzera/Svizra\", name_en=\"Switzerland\", **{\"ISO3166-1:alpha2\": \"CH\"})\n self.mexico = place(114686, 2, name=\"México\", name_en=\"Mexico\", **{\"ISO3166-1\": \"MX\"})\n self.namesake = place(5605656, 6, name=\"China\", name_en=\"China\")\n self.england = place(58447, 4, name=\"England\", name_en=\"England\")\n self.uk = place(62149, 2, name=\"United Kingdom\", name_en=\"United Kingdom\", **{\"ISO3166-1\": \"GB\"})\n self.hk = place(913110, 3, name=\"香港 Hong Kong\", name_zh=\"香港\", name_en=\"Hong Kong\", **{\"ISO3166-1\": \"HK\"})\n self.elements = [self.namesake, self.mexico, self.china, self.swiss, self.england, self.uk, self.hk]\n self.membership = {self.namesake[\"id\"]: {self.mexico[\"id\"]}, self.england[\"id\"]: {self.uk[\"id\"]}}\n\n def overpass(self, query, *_):\n def select(text, elements):\n selectors = re.findall(r'rel(?:\$area\\.p\$)?((?:\\[\"[^\"\\\\]+\"=\"(?:\\\\.|[^\"\\\\])*\"\\])+);', text)\n self.assertTrue(selectors, text)\n filters = [[(key, json.loads('\"' + value + '\"'))\n for key, value in re.findall(r'\\[\"([^\"\\\\]+)\"=\"((?:\\\\.|[^\"\\\\])*)\"\\]', s)]\n for s in selectors]\n return [el for el in elements if any(all(el[\"tags\"].get(k) == v for k, v in f) for f in filters)]\n if \"map_to_area->.p;\" in query:\n parent_query, child_query = query.split(\"map_to_area->.p;\")\n parent_ids = {el[\"id\"] for el in select(parent_query, self.elements)}\n elements = [el for el in self.elements if self.membership.get(el[\"id\"], set()) & parent_ids]\n return {\"elements\": select(child_query, elements)}\n return {\"elements\": select(query, self.elements)}\n\n def lookup(self, name, **kwargs):\n with patch.object(gazetteer.osm, \"run\", side_effect=self.overpass) as run:\n rows = gazetteer.find_relation(name, None, Path(\".\"), **kwargs)\n self.assertEqual(run.call_count, 1) # no extra network round trip to resolve a country\n return [r[\"osm_id\"] for r in rows]\n\n def test_country_names_codes_and_bilingual_tags_without_any_packs(self):\n with patch.object(gazetteer.regions, \"packs\", return_value={}):\n for name, expected in ((\"China\", 270056), (\"中国\", 270056), (\"CN\", 270056), (\" china \", 270056),\n (\"Switzerland\", 51701), (\"瑞士\", 51701), (\"ch\", 51701)):\n with self.subTest(name=name):\n self.assertEqual(self.lookup(name), [expected])\n\n def test_pack_alias_remains_usable(self):\n self.assertEqual(self.lookup(\"PRC\"), [270056])\n\n def test_english_name_fallback_only_selects_a_country(self):\n del self.china[\"tags\"][\"ISO3166-1\"]\n self.assertEqual(self.lookup(\"China\"), [270056])\n\n def test_missing_country_does_not_return_a_namesake(self):\n self.elements.remove(self.china)\n with self.assertRaisesRegex(SystemExit, \"no country/territory boundary.*China.*CN\"):\n self.lookup(\"China\")\n\n def test_multiple_country_boundaries_are_reported(self):\n copy = {**self.china, \"id\": 270057}\n self.elements.append(copy)\n with self.assertRaisesRegex(SystemExit, \"several country/territory boundaries.*270056.*270057\"):\n self.lookup(\"China\")\n\n def test_within_resolves_the_country_but_keeps_the_local_target(self):\n self.assertEqual(self.lookup(\"China\", within=\"Mexico\"), [5605656])\n self.assertEqual(self.lookup(\"England\", within=\"United Kingdom\"), [58447])\n self.membership[self.namesake[\"id\"]] = {self.china[\"id\"]}\n self.assertEqual(self.lookup(\"China\", within=\"中国\"), [5605656])\n\n def test_missing_scope_country_does_not_search_its_namesake(self):\n self.elements.remove(self.mexico)\n self.assertEqual(self.lookup(\"China\", within=\"Mexico\"), [])\n\n def test_explicit_local_level_is_not_forced_to_a_country(self):\n self.assertEqual(self.lookup(\"China\", level=6), [5605656])\n self.assertEqual(self.lookup(\"China\", level=2), [270056])\n\n def test_subregions_and_iso_territories_keep_their_own_boundaries(self):\n self.assertEqual(self.lookup(\"England\"), [58447]) # broad clue alias must not turn this into the UK\n self.assertEqual(self.lookup(\"香港\"), [913110]) # ISO territory, OSM admin_level 3\n self.assertEqual(self.lookup(\"Hong Kong\"), [913110])\n self.assertIsNone(gazetteer._country(\"Canary Islands\"))\n self.assertIsNone(gazetteer._country(\"Svalbard\"))\n\n def test_country_code_index_is_unambiguous_and_excludes_reserved_codes(self):\n index, codes = gazetteer._country_index()\n self.assertEqual(len(codes), 249)\n for code, names in codes.items():\n self.assertEqual(index[code.lower()], code)\n for name in names:\n self.assertEqual(index[name.casefold()], code)\n self.assertNotIn(\"EU\", codes)\n self.assertNotIn(\"IC\", codes)\n\n def test_unknown_name_keeps_a_single_name_selector(self):\n with patch.object(gazetteer.osm, \"run\", return_value={\"elements\": []}) as run:\n gazetteer.find_relation(\"Nowhere\", None, Path(\".\"))\n self.assertEqual(run.call_args.args[0].count('[\"name\"='), 1)\n\n\nclass LiveMessages(unittest.TestCase):\n def test_live_run_passes_stderr_through_and_still_returns_stdout(self):\n code = \"import sys; print('retrying in 15 s', file=sys.stderr); print('result')\"\n r = subprocess.run([sys.executable, \"-c\",\n f\"import sys; sys.path.insert(0, {str(SCRIPTS)!r}); import board; \"\n f\"r = board._run([sys.executable, '-c', {code!r}], live=True); print('child stdout:', r.stdout.strip(), r.stderr)\"],\n capture_output=True, text=True, encoding=\"utf-8\")\n self.assertEqual(r.returncode, 0, r.stderr)\n self.assertIn(\"retrying in 15 s\", r.stderr) # reached the caller's terminal as it happened\n self.assertIn(\"child stdout: result None\", r.stdout)\n\n\nclass ChildMembership(unittest.TestCase):\n def test_membership_uses_any_pack_and_preserves_unknown_or_ambiguous_names(self):\n with tempfile.TemporaryDirectory() as d:\n root = Path(d)\n items = [{\"name\": n, \"parent\": p, \"level\": lv} for n, p, lv in\n [(\"North\", \"\", 1), (\"South\", \"\", 1), (\"Inside\", \"North\", 2),\n (\"Outside\", \"South\", 2), (\"Shared\", \"North\", 2), (\"Shared\", \"South\", 2)]]\n (root / 'admin.json').write_text(json.dumps({'items': items}), encoding='utf-8')\n pack = {'code': 'QQ', 'dir': root, 'admin_table': 'admin.json'}\n rows = [{'name': n} for n in ('Inside', 'Outside', 'Shared', 'Unmapped')]\n with patch.object(gazetteer.regions, 'packs', return_value={'QQ': pack}):\n kept = gazetteer._filter_known_neighbours({'name': 'North'}, rows)\n self.assertEqual([r['name'] for r in kept], ['Inside', 'Shared', 'Unmapped'])\n\n def test_known_neighbour_is_not_added_to_the_wrong_province(self):\n parent = gazetteer._rel_rows({\"elements\": [INNER_MONGOLIA]})[0]\n data = {\"elements\": [rel(1, 5, name=\"呼和浩特市\"), rel(2, 5, name=\"包头市\"), rel(3, 5, name=\"忻州市\")]}\n with patch.object(gazetteer, \"find_relation\", return_value=[parent]), patch.object(gazetteer.osm, \"run\", return_value=data):\n _, rows = gazetteer.children(parent[\"name\"], None, Path('.'), 5, None)\n self.assertEqual({r['name'] for r in rows}, {\"呼和浩特市\", \"包头市\"})\n\n def test_absent_pack_does_not_silently_filter_osm_children(self):\n parent = gazetteer._rel_rows({\"elements\": [INNER_MONGOLIA]})[0]\n data = {\"elements\": [rel(1, 5, name=\"呼和浩特市\"), rel(2, 5, name=\"包头市\"), rel(3, 5, name=\"忻州市\")]}\n with patch.object(gazetteer.regions, \"packs\", return_value={}), patch.object(gazetteer, \"find_relation\", return_value=[parent]), patch.object(gazetteer.osm, \"run\", return_value=data):\n _, rows = gazetteer.children(parent[\"name\"], None, Path('.'), 5, None)\n self.assertEqual(len(rows), 3)\n\n def test_explicit_level_keeps_a_single_valid_child(self):\n parent = gazetteer._rel_rows({\"elements\": [INNER_MONGOLIA]})[0]\n data = {\"elements\": [rel(1, 5, name=\"呼和浩特市\"), rel(2, 5, name=\"忻州市\")]}\n with patch.object(gazetteer, 'find_relation', return_value=[parent]), patch.object(gazetteer.osm, 'run', return_value=data):\n _, rows = gazetteer.children(parent['name'], None, Path('.'), 5, None)\n self.assertEqual([r['name'] for r in rows], ['呼和浩特市'])\n\n\nif __name__ == \"__main__\":\n unittest.main()\n","content_type":"text/x-python; charset=utf-8","language":"python","size":14379,"content_sha256":"6b8e04c9120b65fbc9521f3bceadbd3215d5ccac1dea7b29e87ad34d829e8041"},{"filename":"tests/test_match.py","content":"#!/usr/bin/env python3\n# /// script\n# requires-python = \">=3.10\"\n# dependencies = [\"pillow\", \"numpy\", \"opencv-python-headless\", \"torch\"]\n# ///\n\"\"\"Offline descriptor and geometric verification regressions; no model download.\"\"\"\nfrom pathlib import Path\nfrom types import SimpleNamespace\nimport sys\nimport unittest\nfrom unittest.mock import patch\n\nimport cv2\nimport numpy as np\nfrom PIL import Image\nimport torch\n\nsys.path.insert(0, str(Path(__file__).resolve().parents[1] / \"scripts\"))\nimport match\n\n\nclass RendererOverlayTests(unittest.TestCase):\n def test_each_renderer_owns_its_overlay_and_plain_images_have_none(self):\n import baidu_pano\n import gsv\n box = (.8, 0., 1., .1)\n self.assertIsNone(match._overlay(None))\n self.assertIsNone(match._overlay('gsv'))\n with patch.object(baidu_pano, 'LOGO_BOX', box):\n self.assertEqual(match._overlay('baidu'), box)\n with patch.object(gsv, 'LOGO_BOX', box, create=True):\n self.assertEqual(match._overlay('gsv'), box)\n\n\nclass SiftTests(unittest.TestCase):\n def projected_matches(self, H, repeated=False):\n pa = np.float32([(x, y) for x in (20, 40, 60, 70) for y in (20, 40, 60)])\n pb = cv2.perspectiveTransform(pa[None], np.asarray(H, np.float64))[0]\n if repeated:\n pb = np.tile(pb[:3], (4, 1))\n descriptors = np.random.default_rng(1).random((len(pa), 128)).astype(np.float32)\n sift = SimpleNamespace(detectAndCompute=lambda *args: None)\n keypoints = lambda pts: [cv2.KeyPoint(float(x), float(y), 2) for x, y in pts]\n im = Image.new(\"RGB\", (1024, 768))\n with patch.object(cv2, \"SIFT_create\", return_value=sift), patch.object(\n sift, \"detectAndCompute\", side_effect=[(keypoints(pa), descriptors), (keypoints(pb), descriptors)]\n ):\n if repeated:\n with patch.object(cv2, \"findHomography\", return_value=(np.eye(3), np.ones((len(pa), 1), np.uint8))):\n return match.sift_inliers(im, im)\n return match.sift_inliers(im, im)\n\n def test_valid_perspective_with_negative_affine_minor_is_kept(self):\n self.assertEqual(self.projected_matches([[-1, 0, 300], [0, 1, 0], [-0.01, 0, 1]]), (12, 12))\n\n def test_mirrored_perspective_with_positive_affine_minor_is_rejected(self):\n self.assertEqual(self.projected_matches([[1, 0, 300], [0, 1, 0], [0.01, 0, 1]]), (0, 12))\n\n def test_affine_and_mirrored_matches(self):\n self.assertEqual(self.projected_matches([[1, 0, 25], [0, 1, 12], [0, 0, 1]]), (12, 12))\n self.assertEqual(self.projected_matches([[-1, 0, 300], [0, 1, 0], [0, 0, 1]]), (0, 12))\n\n def test_collapsed_matches_are_rejected(self):\n self.assertEqual(self.projected_matches(np.eye(3), repeated=True), (0, 12))\n\n def test_real_texture_survives_perspective(self):\n rng = np.random.default_rng(8)\n a = cv2.GaussianBlur(rng.integers(0, 256, (480, 640, 3), np.uint8), (3, 3), 0)\n b = cv2.warpPerspective(a, np.array([[.9, .06, 15], [-.04, .95, 20], [.0001, .0002, 1]]), (640, 480))\n self.assertGreater(match.sift_inliers(Image.fromarray(a), Image.fromarray(b))[0], 100)\n\n def test_logo_only_matches_do_not_verify_a_place(self):\n rng = np.random.default_rng(42)\n a = np.full((768, 1024), 128, np.uint8)\n a[718:768, :120] = cv2.resize(rng.integers(0, 256, (25, 60), np.uint8), (120, 50), interpolation=cv2.INTER_CUBIC)\n im = Image.fromarray(a)\n self.assertGreaterEqual(match.sift_inliers(im, im)[0], 15)\n self.assertEqual(match.sift_inliers(im, im, skip_b=match._overlay(\"baidu\")), (0, 0))\n\n def test_blank_images_have_no_match(self):\n im = Image.new(\"RGB\", (640, 480), \"gray\")\n self.assertEqual(match.sift_inliers(im, im), (0, 0))\n\n\nclass DescriptorTests(unittest.TestCase):\n def test_dino_halves_have_equal_weight_across_batches(self):\n emb = match.Embedder.__new__(match.Embedder)\n emb.method, emb.dev, emb.torch = \"dino\", \"cpu\", torch\n hidden = torch.tensor([[[3., 4.], [0., 2.], [0., 2.]]])\n emb.model = lambda pixel_values: SimpleNamespace(last_hidden_state=hidden.repeat(len(pixel_values), 1, 1))\n images = [Image.new(\"RGB\", (80, 60)), Image.new(\"RGB\", (100, 70))]\n features = emb.embed(images, multi=True, batch=3)\n self.assertEqual(features.shape, (2, 4, 4))\n np.testing.assert_allclose(np.linalg.norm(features, axis=-1), 1, atol=1e-6)\n np.testing.assert_allclose(features[0, 0], np.array([.6, .8, 0, 1]) / np.sqrt(2), atol=1e-6)\n np.testing.assert_allclose(features, np.broadcast_to(features[0, 0], features.shape))\n\n\nif __name__ == \"__main__\":\n unittest.main()\n","content_type":"text/x-python; charset=utf-8","language":"python","size":4729,"content_sha256":"596836d75059815e936969e89d0da4c07b2511682464a87031214b734ab9310e"},{"filename":"tests/test_regions.py","content":"#!/usr/bin/env python3\n# /// script\n# requires-python = \">=3.10\"\n# dependencies = []\n# ///\n\"\"\"Offline regressions for region packs and country context. Run: uv run skills/geo-sleuth/tests/test_regions.py\"\"\"\nfrom __future__ import annotations\n\nimport json\nimport os\nfrom pathlib import Path\nimport subprocess\nimport sys\nimport tempfile\nimport unittest\n\nSKILL = Path(__file__).resolve().parents[1]\nSCRIPTS = SKILL / \"scripts\"\nsys.path.insert(0, str(SCRIPTS))\nimport regions # noqa: E402\n\n# A fake second country whose area code 020 collides with China's, and whose 0120 covers two districts\nFIXTURE = {\n \"region.json\": {\"code\": \"QQ\", \"name\": \"Testland\", \"names\": [\"Testland\", \"TL\"], \"status\": \"community\", \"docs\": [\"clues.md\"],\n \"admin_levels\": {\"1\": \"admin1\", \"2\": \"admin2\"},\n \"lookups\": {\"area-code\": {\"file\": \"data/area_codes.json\"}}},\n \"data/area_codes.json\": {\"_meta\": {\"kind\": \"area-code\", \"source\": [\"https://example.org/codes\"], \"fetched\": \"2026-10-08\",\n \"license\": \"CC0\", \"count\": 3, \"normalize\": {\"digits_only\": True, \"ensure_prefix\": \"0\"}},\n \"entries\": {\"020\": [{\"admin1\": \"North\", \"admin2\": \"Porta\"}],\n \"0120\": [{\"admin1\": \"North\", \"admin2\": \"Alda\"}, {\"admin1\": \"North\", \"admin2\": \"Brin\"}],\n \"0431\": [{\"admin1\": \"吉林省\", \"admin2\": \"长春市\", \"district\": \"朝阳区\"}]}},\n \"tests.json\": [{\"kind\": \"area-code\", \"value\": \"0120-555\", \"expect\": [{\"admin1\": \"North\", \"admin2\": \"Alda\"}, {\"admin1\": \"North\", \"admin2\": \"Brin\"}]}],\n \"clues.md\": \"# Testland\\n\\n## Signs\\n\\n### Blue signs\\n- Look for: blue\\n- Points to: North\\n- Strength: weak\\n- Counterexamples: none seen\\n- Sources: test\\n\",\n}\n\n\ndef write_pack(root: Path, files: dict) -> Path:\n d = root / \"qq\"\n for rel, body in files.items():\n f = d / rel\n f.parent.mkdir(parents=True, exist_ok=True)\n f.write_text(body if isinstance(body, str) else json.dumps(body, ensure_ascii=False), encoding=\"utf-8\")\n return d\n\n\ndef run(script: str, *args: str, env: dict | None = None, cwd: Path | None = None) -> subprocess.CompletedProcess:\n return subprocess.run([sys.executable, str(SCRIPTS / script), *args], capture_output=True, text=True, encoding=\"utf-8\",\n env={**os.environ, \"PYTHONUTF8\": \"1\", **(env or {})}, cwd=cwd)\n\n\nclass BundledPacks(unittest.TestCase):\n def test_bundled_packs_pass_lint(self):\n r = run(\"regions.py\", \"lint\")\n self.assertEqual(r.returncode, 0, r.stdout + r.stderr)\n\n def test_poi_default_sources_come_from_the_pack_that_claims_the_script(self):\n for keyword, sources in ((\"万科城\", \"so,osm,sug\"), (\"東京タワー\", \"so,osm,sug\"), (\"Starbucks\", None), (\"서울역\", None)):\n self.assertEqual((regions.poi_for_script(keyword) or {}).get(\"sources\"), sources, keyword)\n from unittest.mock import patch\n with patch.object(regions, \"packs\", return_value={}):\n self.assertIsNone(regions.poi_for_script(\"万科城\")) # core itself prefers no country\n\n def test_malformed_optional_settings_do_not_break_other_packs(self):\n from unittest.mock import patch\n good = {\"osm_name_keys\": [\"name:fr\"], \"services\": {\"poi\": {\"sources\": \"osm\", \"keyword_scripts\": [\"LATIN\"]}}}\n for bad in ({\"osm_name_keys\": 7}, {\"services\": {\"poi\": {\"keyword_scripts\": [7]}}},\n {\"services\": []}, {\"services\": {\"poi\": [\"osm\"]}}):\n with self.subTest(bad=bad), patch.object(regions, \"packs\", return_value={\"AA\": bad, \"ZZ\": good}):\n self.assertEqual(regions.osm_name_keys(), (\"name\", \"name:fr\"))\n self.assertEqual(regions.poi_for_script(\"Cafe\"), good[\"services\"][\"poi\"])\n\n def test_script_names_must_be_a_list_not_an_accidental_character_prefix_set(self):\n from unittest.mock import patch\n with patch.object(regions, \"packs\", return_value={\"AA\": {\"osm_name_keys\": \"name:fr\", \"services\": {\"poi\": {\"sources\": \"so\", \"keyword_scripts\": \"LATIN\"}}}}):\n self.assertEqual(regions.osm_name_keys(), (\"name\",))\n self.assertIsNone(regions.poi_for_script(\"Cafe\"))\n\n def test_cn_lookups_keep_their_answers(self):\n m = regions.lookup(\"plate\", \"粤B·12345\", \"cn\")[\"matches\"]\n self.assertEqual([(x[\"admin1\"], x.get(\"admin2\")) for x in m], [(\"广东省\", \"深圳市\")])\n m = regions.lookup(\"area-code\", \"0817-1234567\")[\"matches\"]\n self.assertEqual([(x[\"country_code\"], x[\"admin2\"]) for x in m], [(\"CN\", \"南充市\")])\n\n def test_descriptions_are_not_places(self):\n # Wikipedia cells like \"市区普通汽车\" (city-area cars) used to become fake candidates\n m = regions.lookup(\"plate\", \"京A\", \"cn\")[\"matches\"]\n self.assertEqual(len(m), 1)\n self.assertEqual({k for k in regions.MATCH_LEVELS if m[0].get(k)}, {\"admin1\"})\n\n def test_unknown_country_is_unsupported_not_china(self):\n r = run(\"clues.py\", \"lookup\", \"area-code\", \"020\", \"--country\", \"India\", \"--json\")\n d = json.loads(r.stdout)\n self.assertEqual(d[\"status\"], \"unsupported\")\n self.assertEqual(d[\"matches\"], [])\n\n def test_country_without_that_table_is_unsupported(self):\n d = regions.lookup(\"plate\", \"ABC\", \"jp\")\n self.assertEqual(d[\"status\"], \"unsupported\")\n\n def test_foreign_plate_format_is_not_matched_against_cn(self):\n d = regions.lookup(\"plate\", \"KA01AB1234\")\n self.assertEqual(d[\"matches\"], [])\n\n def test_same_name_districts_stay_apart(self):\n rows = run(\"clues.py\", \"lookup\", \"admin\", \"朝阳区\", \"--json\")\n chains = {tuple(m[\"chain\"]) for m in json.loads(rows.stdout)[\"matches\"]}\n self.assertIn((\"北京市\", \"朝阳区\"), chains)\n self.assertIn((\"吉林省\", \"长春市\", \"朝阳区\"), chains)\n\n\nclass FixturePack(unittest.TestCase):\n def setUp(self):\n self.tmp = tempfile.TemporaryDirectory()\n self.root = Path(self.tmp.name)\n write_pack(self.root / \"packs\", FIXTURE)\n self.env = {\"GEO_SLEUTH_REGIONS\": str(self.root / \"packs\")}\n\n def tearDown(self):\n self.tmp.cleanup()\n\n def lookup(self, *args):\n return json.loads(run(\"clues.py\", \"lookup\", *args, \"--json\", env=self.env).stdout)\n\n def test_no_country_returns_every_country(self):\n d = self.lookup(\"area-code\", \"020\")\n self.assertEqual(sorted(m[\"country_code\"] for m in d[\"matches\"]), [\"CN\", \"QQ\"])\n self.assertIn(\"several countries\", d[\"note\"])\n\n def alias_pack(self, **alias) -> tuple[Path, dict]:\n \"\"\"Fixture pack with an admin table whose 'Ålda' item carries `alias` (aliases=...), suffix ' District'.\"\"\"\n files = json.loads(json.dumps(FIXTURE))\n files[\"region.json\"].update(admin_table=\"data/admin.json\", name_suffixes=[\" District\"])\n files[\"data/admin.json\"] = {\"_meta\": {\"source\": [\"test\"], \"fetched\": \"2026-10-18\", \"license\": \"CC0\"},\n \"items\": [{\"name\": \"North\", \"code\": \"1\", \"level\": 1, \"parent\": \"\"},\n {\"name\": \"Ålda\", \"code\": \"11\", \"level\": 2, \"parent\": \"North\", **alias}]}\n d = write_pack(self.root / \"alias\", files)\n return d, {\"GEO_SLEUTH_REGIONS\": str(self.root / \"alias\")}\n\n def admin(self, q, env, *country):\n r = run(\"clues.py\", \"lookup\", \"admin\", q, *country, \"--json\", env=env)\n self.assertNotIn(\"Traceback\", r.stderr, q)\n return json.loads(r.stdout)[\"matches\"]\n\n def test_admin_alias_finds_the_table_name(self):\n _, env = self.alias_pack(aliases=[\"Alda District\"])\n for q in (\"Ålda\", \"Ålda District\", \"Alda\", \"Alda District\"): # suffix stripped on the query and on the alias\n self.assertEqual([m[\"admin2\"] for m in self.admin(q, env, \"--country\", \"qq\")], [\"Ålda\"], q)\n\n def test_malformed_aliases_fail_lint_and_leave_lookups_working(self):\n for bad in (\"Alda\", [7], [\"\"], [\"Alda\", None], {\"Alda\": 1}, None):\n d, env = self.alias_pack(aliases=bad)\n errs, _ = regions.lint(dict(json.loads((d / \"region.json\").read_text(encoding=\"utf-8\")), dir=d))\n self.assertTrue(any(\"aliases must be a list\" in e for e in errs), (bad, errs))\n # an unrelated pack stays queryable while the malformed one is scanned\n self.assertIn((\"北京市\", \"朝阳区\"), {tuple(m[\"chain\"]) for m in self.admin(\"朝阳区\", env)}, bad)\n self.assertEqual([m[\"admin2\"] for m in self.admin(\"Ålda\", env, \"--country\", \"qq\")], [\"Ålda\"], bad)\n self.assertEqual(self.admin(\"A\", env, \"--country\", \"qq\"), [], bad) # a bare string isn't split into characters\n for ok in ({}, {\"aliases\": []}):\n d, _ = self.alias_pack(**ok)\n errs, _ = regions.lint(dict(json.loads((d / \"region.json\").read_text(encoding=\"utf-8\")), dir=d))\n self.assertFalse(any(\"aliases\" in e for e in errs), (ok, errs))\n\n def test_country_restricts(self):\n d = self.lookup(\"area-code\", \"020\", \"--country\", \"Testland\")\n self.assertEqual([m[\"admin2\"] for m in d[\"matches\"]], [\"Porta\"])\n d = self.lookup(\"area-code\", \"020\", \"--country\", \"cn\")\n self.assertEqual([m[\"admin2\"] for m in d[\"matches\"]], [\"广州市\"])\n\n def test_removing_the_pack_leaves_cn_unchanged(self):\n with_pack = [m for m in self.lookup(\"area-code\", \"0817\")[\"matches\"] if m[\"country_code\"] == \"CN\"]\n without = json.loads(run(\"clues.py\", \"lookup\", \"area-code\", \"0817\", \"--json\").stdout)[\"matches\"]\n self.assertEqual(with_pack, without)\n\n def test_multi_place_key_gives_one_candidate_each(self):\n board = self.root / \"board.json\"\n run(\"board.py\", \"--board\", str(board), \"init\", \"--photo\", \"x.jpg\", env=self.env)\n r = run(\"board.py\", \"--board\", str(board), \"apply\", \"--kind\", \"area-code\", \"--value\", \"0120-555\", \"--country\", \"qq\", env=self.env)\n self.assertEqual(r.returncode, 0, r.stderr)\n b = json.loads(board.read_text(encoding=\"utf-8\"))\n self.assertIn(\"Alda\", b[\"candidates\"])\n self.assertIn(\"Brin\", b[\"candidates\"])\n self.assertEqual(b[\"candidates\"][\"Alda\"][\"cid\"], \"QQ/North/Alda\")\n # one piece of evidence per place\n self.assertEqual(sum(1 for e in b[\"evidence\"] if e[\"candidate\"] == \"North\"), 1)\n\n def test_siblings_listed_by_children_are_down_weighted(self):\n # children records candidates with parent = the listed area; a lookup naming one of them must still count against the rest\n board = self.root / \"board.json\"\n run(\"board.py\", \"--board\", str(board), \"init\", \"--photo\", \"x.jpg\", env=self.env)\n run(\"board.py\", \"--board\", str(board), \"add\", \"Porta\", \"Cora\", \"--level\", \"admin2\", \"--parent\", \"North\", env=self.env)\n run(\"board.py\", \"--board\", str(board), \"apply\", \"--kind\", \"area-code\", \"--value\", \"020\", \"--country\", \"qq\", env=self.env)\n b = json.loads(board.read_text(encoding=\"utf-8\"))\n lr = {e[\"candidate\"]: e[\"lr\"] for e in b[\"evidence\"]}\n self.assertGreater(lr[\"Porta\"], 1)\n self.assertLess(lr[\"Cora\"], 1)\n\n def test_same_name_elsewhere_gets_its_own_candidate(self):\n board = self.root / \"board.json\"\n run(\"board.py\", \"--board\", str(board), \"init\", \"--photo\", \"x.jpg\", env=self.env)\n run(\"board.py\", \"--board\", str(board), \"add\", \"朝阳区\", \"--level\", \"district\", \"--parent\", \"北京市\", env=self.env)\n r = run(\"board.py\", \"--board\", str(board), \"apply\", \"--kind\", \"area-code\", \"--value\", \"0431\", \"--country\", \"qq\", env=self.env)\n self.assertEqual(r.returncode, 0, r.stderr)\n b = json.loads(board.read_text(encoding=\"utf-8\"))\n self.assertIn(\"朝阳区 (长春市)\", b[\"candidates\"])\n beijing = [e for e in b[\"evidence\"] if e[\"candidate\"] == \"朝阳区\"]\n self.assertTrue(all(e[\"lr\"] \u003c 1 for e in beijing), beijing)\n\n def test_board_country_drives_apply_and_next(self):\n board = self.root / \"board.json\"\n run(\"board.py\", \"--board\", str(board), \"init\", \"--photo\", \"x.jpg\", \"--country\", \"cn\", env=self.env)\n run(\"board.py\", \"--board\", str(board), \"apply\", \"--kind\", \"area-code\", \"--value\", \"020\", env=self.env)\n b = json.loads(board.read_text(encoding=\"utf-8\"))\n self.assertNotIn(\"Porta\", b[\"candidates\"])\n run(\"board.py\", \"--board\", str(board), \"add\", \"A区\", \"B区\", \"--level\", \"district\", env=self.env)\n out = run(\"board.py\", \"--board\", str(board), \"next\", env=self.env).stdout\n self.assertIn(\"baidu_pano.py sample\", out)\n run(\"board.py\", \"--board\", str(board), \"country\", \"none\", env=self.env)\n out = run(\"board.py\", \"--board\", str(board), \"next\", env=self.env).stdout\n self.assertIn(\"gsv.py sheet\", out)\n\n def board(self, *args):\n return run(\"board.py\", \"--board\", str(self.root / \"board.json\"), *args, env=self.env)\n\n def read_board(self):\n return json.loads((self.root / \"board.json\").read_text(encoding=\"utf-8\"))\n\n def test_country_without_the_table_is_not_down_weighted(self):\n # India has no pack here: an area code that isn't Chinese says nothing about India\n self.board(\"init\", \"--photo\", \"x.jpg\")\n self.board(\"add\", \"China\", \"India\", \"Testland\", \"--level\", \"country\")\n self.board(\"apply\", \"--kind\", \"area-code\", \"--value\", \"0817\")\n lr = {}\n for e in self.read_board()[\"evidence\"]:\n lr.setdefault(e[\"candidate\"], []).append(e[\"lr\"])\n self.assertEqual(lr[\"China\"], [20.0])\n self.assertNotIn(\"India\", lr)\n self.assertEqual(lr[\"Testland\"], [0.05]) # its table was searched and had no 0817\n\n def test_country_context_doesnt_stand_in_for_unknown_countries(self):\n for how in ((\"init\", \"--photo\", \"x.jpg\", \"--country\", \"cn\"), (\"init\", \"--photo\", \"x.jpg\")):\n (self.root / \"board.json\").unlink(missing_ok=True)\n self.board(*how)\n self.board(\"add\", \"China\", \"India\", \"--level\", \"country\")\n self.board(\"apply\", \"--kind\", \"area-code\", \"--value\", \"0817\", \"--country\", \"cn\")\n cands = {e[\"candidate\"] for e in self.read_board()[\"evidence\"]}\n self.assertIn(\"China\", cands)\n self.assertNotIn(\"India\", cands, how)\n\n def test_province_and_same_stem_city_are_two_places(self):\n self.board(\"init\", \"--photo\", \"x.jpg\")\n self.board(\"add\", \"吉林省\", \"--level\", \"admin1\")\n self.board(\"add\", \"吉林市\", \"--level\", \"admin2\", \"--parent\", \"吉林省\")\n self.board(\"apply\", \"--kind\", \"plate\", \"--value\", \"吉B\", \"--country\", \"cn\")\n b = self.read_board()\n self.assertEqual(sorted(b[\"candidates\"]), [\"吉林市\", \"吉林省\"])\n self.assertEqual(b[\"candidates\"][\"吉林市\"][\"level\"], \"admin2\")\n ev = {e[\"candidate\"] for e in b[\"evidence\"] if e[\"lr\"] > 1}\n self.assertEqual(ev, {\"吉林市\", \"吉林省\"})\n\n def test_country_context_doesnt_overwrite_global_observations(self):\n self.board(\"init\", \"--photo\", \"x.jpg\", \"--country\", \"us\")\n self.board(\"add\", \"United States\", \"Japan\", \"--level\", \"country\")\n self.board(\"apply\", \"--kind\", \"driving-side\", \"--value\", \"left\")\n lr = {e[\"candidate\"]: e[\"lr\"] for e in self.read_board()[\"evidence\"]}\n self.assertGreater(lr[\"Japan\"], 1)\n self.assertLess(lr[\"United States\"], 1)\n self.board(\"country\", \"cn\")\n r = self.board(\"apply\", \"--kind\", \"driving-side\", \"--value\", \"right\")\n self.assertNotIn(\"returned nothing\", r.stdout)\n\n def test_broken_pack_doesnt_break_other_countries(self):\n bad = self.root / \"packs\" / \"zz\"\n bad.mkdir(parents=True)\n (bad / \"region.json\").write_text(json.dumps({\"code\": \"ZZ\", \"name\": \"Brokenland\", \"names\": [\"Brokenland\"], \"status\": \"stub\",\n \"docs\": [], \"lookups\": {\"area-code\": {\"file\": \"data/missing.json\"}}}), encoding=\"utf-8\")\n d = self.lookup(\"area-code\", \"0817\")\n self.assertEqual([m[\"country_code\"] for m in d[\"matches\"]], [\"CN\"])\n self.assertIn(\"ZZ pack is broken\", d[\"note\"])\n self.assertEqual(self.lookup(\"area-code\", \"0817\", \"--country\", \"zz\")[\"status\"], \"error\")\n # valid JSON of the wrong shape: same isolation, and lint reports it instead of crashing\n (bad / \"data\").mkdir()\n (bad / \"data\" / \"missing.json\").write_text(\"[]\", encoding=\"utf-8\")\n d = self.lookup(\"area-code\", \"0817\")\n self.assertEqual([m[\"country_code\"] for m in d[\"matches\"]], [\"CN\"])\n r = run(\"regions.py\", \"lint\", \"zz\", env=self.env)\n self.assertEqual(r.returncode, 1)\n self.assertIn(\"must be an object\", r.stdout)\n self.assertNotIn(\"Traceback\", r.stderr)\n\n def test_lint_rejects_code_merged_names_and_missing_sources(self):\n bad = json.loads(json.dumps(FIXTURE))\n bad[\"data/area_codes.json\"][\"_meta\"].pop(\"source\")\n bad[\"data/area_codes.json\"][\"entries\"][\"0120\"] = [{\"admin1\": \"North\", \"admin2\": \"Alda / Brin\"}]\n bad[\"helper.py\"] = \"print('hi')\\n\"\n bad[\"clues.md\"] = FIXTURE[\"clues.md\"].replace(\"- Counterexamples: none seen\\n\", \"\")\n d = write_pack(self.root / \"bad\", bad)\n regions.packs(reload=True)\n errs, _ = regions.lint(dict(json.loads((d / \"region.json\").read_text(encoding=\"utf-8\")), dir=d))\n text = \"\\n\".join(errs)\n self.assertIn(\"helper.py\", text)\n self.assertIn(\"_meta.source\", text)\n self.assertIn(\"Alda / Brin\", text)\n self.assertIn(\"Counterexamples\", text)\n\n def test_lint_reports_malformed_optional_manifest_settings(self):\n for field, value in [('osm_name_keys', 7), ('services', []), ('services', {'poi': ['osm']}),\n ('services', {'poi': {'keyword_scripts': 'LATIN'}}), ('also_children', 'Elsewhere')]:\n with self.subTest(field=field, value=value):\n files = json.loads(json.dumps(FIXTURE))\n files['region.json'][field] = value\n d = write_pack(self.root / 'invalid-options', files)\n errs, _ = regions.lint(dict(files['region.json'], dir=d))\n self.assertTrue(any(field in e for e in errs), errs)\n\n\nclass GlobalTables(unittest.TestCase):\n def test_driving_side_of_split_countries(self):\n import clues\n side = {c: clues.lookup_driving_side(None, c)[\"matches\"][0][\"side\"] for c in\n (\"China\", \"Hong Kong\", \"Macau\", \"United Kingdom\", \"Japan\", \"India\", \"United States\", \"Canada\", \"Ireland\")}\n self.assertEqual(side, {\"China\": \"right\", \"Hong Kong\": \"left\", \"Macau\": \"left\", \"United Kingdom\": \"left\", \"Japan\": \"left\",\n \"India\": \"left\", \"United States\": \"right\", \"Canada\": \"right\", \"Ireland\": \"left\"})\n\n\nclass SameNamesAndPoi(unittest.TestCase):\n def test_same_name_under_another_parent_is_kept(self):\n import board\n b = {\"candidates\": {}}\n for parent, cid in ((\"南京市\", \"CN/江苏省/南京市/鼓楼区\"), (\"福州市\", \"CN/福建省/福州市/鼓楼区\")):\n key = board._slot(b, \"鼓楼区\", parent, cid, \"district\")\n self.assertIsNotNone(key)\n b[\"candidates\"][key] = {\"parent\": parent, \"cid\": cid, \"level\": \"district\"}\n self.assertEqual(sorted(b[\"candidates\"]), [\"鼓楼区\", \"鼓楼区 (福州市)\"])\n self.assertIsNone(board._slot(b, \"鼓楼区\", \"福州市\", \"CN/福建省/福州市/鼓楼区\", \"district\"))\n self.assertIsNone(board._slot(b, \"鼓楼区\", \"南京市\", None, \"district\"))\n\n def test_poi_region_without_pack_keeps_the_country_filter(self):\n import poi\n from unittest.mock import patch\n seen = {}\n\n def fake_osm(kw, city, n, proxy, country):\n seen[\"country\"] = country\n return []\n with patch.object(poi, \"search_osm\", fake_osm), patch.object(poi, \"search_so\", side_effect=AssertionError(\"360 used\")), \\\n patch.object(sys, \"argv\", [\"poi.py\", \"Connaught Place\", \"--region\", \"in\"]):\n poi.main()\n self.assertEqual(seen[\"country\"], \"in\")\n with patch.object(sys, \"argv\", [\"poi.py\", \"x\", \"--region\", \"Narnia\"]), self.assertRaises(SystemExit):\n poi.main()\n\n\nif __name__ == \"__main__\":\n unittest.main(verbosity=2)\n","content_type":"text/x-python; charset=utf-8","language":"python","size":20347,"content_sha256":"f258d598a3b848acfe347ff2ac14a8230542dc66af3f441ccf467c445b715db1"},{"filename":"tests/test_runtime.py","content":"#!/usr/bin/env python3\n# /// script\n# requires-python = \">=3.10\"\n# dependencies = [\"pillow\", \"pillow-heif\", \"numpy\", \"pyyaml\"]\n# ///\n\"\"\"Offline runtime regressions. Run: uv run skills/geo-sleuth/tests/test_runtime.py\"\"\"\nfrom __future__ import annotations\n\nimport asyncio\nimport contextlib\nfrom concurrent.futures import ThreadPoolExecutor, TimeoutError\nimport http.server\nimport io\nimport json\nimport math\nimport os\nfrom pathlib import Path\nimport re\nimport shutil\nimport subprocess\nimport sys\nimport tempfile\nimport threading\nfrom types import SimpleNamespace\nimport unittest\nfrom unittest.mock import AsyncMock, patch\nimport urllib.parse\n\nimport numpy as np\nfrom PIL import Image, ImageDraw\n\nimport yaml\n\nsys.path.insert(0, str(Path(__file__).resolve().parents[1] / \"scripts\"))\nimport _net\nimport _browser\nimport _device\nimport _image\nimport match\nimport pose\nimport baidu_pano\nimport doctor\nimport evidence\nimport gsv\nimport osm\nimport poi\nimport revimg\n\n\nclass NetworkTests(unittest.TestCase):\n def test_route_precedence(self):\n with patch.dict(os.environ, {\"GEO_PROXY\": \"http://localhost:8123\"}, clear=True):\n self.assertEqual(_net.resolve_proxy(), \"http://localhost:8123\")\n self.assertEqual(_net.resolve_proxy(\"http://localhost:8234\"), \"http://localhost:8234\")\n self.assertIsNone(_net.resolve_proxy(\"direct\"))\n self.assertIsNone(_net.resolve_proxy(\"\"))\n with patch.dict(os.environ, {\"HTTPS_PROXY\": \"http://localhost:9999\"}, clear=True):\n self.assertIsNone(_net.resolve_proxy())\n\n @unittest.skipUnless(shutil.which(\"curl\"), \"curl required\")\n def test_curl_reaches_direct_and_explicit_proxy_despite_ambient_settings(self):\n paths = []\n class Handler(http.server.BaseHTTPRequestHandler):\n def do_GET(self):\n paths.append(self.path)\n self.send_response(200)\n self.end_headers()\n self.wfile.write(b\"ok\")\n def log_message(self, *args):\n pass\n server = http.server.ThreadingHTTPServer((\"127.0.0.1\", 0), Handler)\n thread = threading.Thread(target=server.serve_forever, daemon=True)\n thread.start()\n base = f\"http://127.0.0.1:{server.server_port}\"\n try:\n with patch.dict(os.environ, {\"HTTP_PROXY\": \"http://127.0.0.1:1\", \"http_proxy\": \"http://127.0.0.1:1\",\n \"ALL_PROXY\": \"http://127.0.0.1:1\", \"NO_PROXY\": \"*\", \"GEO_PROXY\": \"http://127.0.0.1:1\"}):\n self.assertEqual(_net.fetch_bytes(base + \"/direct\", \"direct\"), b\"ok\")\n self.assertEqual(_net.fetch_bytes(\"http://example.invalid/proxied\", base), b\"ok\")\n self.assertEqual(paths, [\"/direct\", \"http://example.invalid/proxied\"])\n finally:\n server.shutdown()\n server.server_close()\n thread.join()\n\n def test_model_route_overrides_ambient_variables(self):\n with patch.dict(os.environ, {\"GEO_PROXY\": \"http://chosen:8080\", \"https_proxy\": \"http://old:8080\", \"NO_PROXY\": \"*\"}, clear=True):\n _net.model_proxy_env()\n self.assertEqual(os.environ[\"HTTPS_PROXY\"], \"http://chosen:8080\")\n if os.name != \"nt\": # Windows environment variable names are case-insensitive\n self.assertNotIn(\"https_proxy\", os.environ)\n self.assertNotIn(\"NO_PROXY\", os.environ)\n _net.model_proxy_env(\"direct\")\n self.assertNotIn(\"HTTPS_PROXY\", os.environ)\n self.assertEqual(os.environ[\"NO_PROXY\"], \"*\")\n\n def test_baidu_near_forwards_proxy(self):\n with patch.object(baidu_pano, \"fetch_bytes\", return_value=b'{\"content\":null}') as fetch:\n self.assertIsNone(baidu_pano.near(35, 110, proxy=\"http://chosen:8080\"))\n self.assertEqual(fetch.call_args.args[1], \"http://chosen:8080\")\n\n def test_all_poi_sources_forward_proxy(self):\n with patch.object(poi, \"_curl\", return_value='{}') as fetch:\n poi.search_so(\"school\", None, 2, \"direct\")\n self.assertEqual(fetch.call_args.kwargs[\"proxy\"], \"direct\")\n poi.search_sug(\"school\", \"http://chosen:8080\")\n self.assertEqual(fetch.call_args.kwargs[\"proxy\"], \"http://chosen:8080\")\n with patch.object(poi, \"_curl\", return_value='[]') as fetch:\n poi.search_osm(\"school\", None, 2, \"http://chosen:8080\", \"us\")\n self.assertEqual(fetch.call_args.kwargs[\"proxy\"], \"http://chosen:8080\")\n\n def test_intake_forwards_route_to_both_engines(self):\n import intake\n from PIL import Image\n commands = []\n def run(cmd, **kwargs):\n commands.append(cmd)\n return 0, \"\", \"\"\n with tempfile.TemporaryDirectory() as folder:\n root = Path(folder)\n photo = root / \"photo.jpg\"\n Image.new(\"RGB\", (100, 100), \"white\").save(photo)\n # Preparation writes files used by intake; run the actual small local helpers.\n original = intake._run\n def local_or_search(cmd, **kwargs):\n if any(str(part).endswith(\"revimg.py\") for part in cmd):\n return run(cmd, **kwargs)\n return original([sys.executable, *cmd[2:]], **kwargs)\n with patch.object(intake, \"_run\", side_effect=local_or_search), patch.object(sys, \"argv\", [\"intake.py\", str(photo), \"--out-dir\", str(root / \"out\"), \"--no-ocr\", \"--max-variants\", \"1\", \"--proxy\", \"direct\"]):\n intake.main()\n self.assertEqual(len(commands), 2)\n for cmd in commands:\n self.assertEqual(cmd[cmd.index(\"--proxy\") + 1], \"direct\")\n self.assertEqual({cmd[cmd.index(\"--engines\") + 1] for cmd in commands}, {\"baidu\", \"yandex\"})\n\n\nclass BrowserTests(unittest.IsolatedAsyncioTestCase):\n async def test_chromium_fallback_uses_same_route(self):\n browser = object()\n launch = AsyncMock(side_effect=[RuntimeError(\"Chrome absent\"), browser])\n p = SimpleNamespace(chromium=SimpleNamespace(launch=launch))\n actual, label = await _browser.launch_browser(p, \"socks5h://localhost:8123\")\n self.assertIs(actual, browser)\n self.assertEqual(label, \"Playwright Chromium\")\n for call in launch.call_args_list:\n self.assertEqual(call.kwargs[\"proxy\"][\"server\"], \"socks5://localhost:8123\")\n self.assertEqual(launch.call_args_list[0].kwargs[\"channel\"], \"chrome\")\n self.assertNotIn(\"channel\", launch.call_args_list[1].kwargs)\n\n async def test_explicit_direct_overrides_saved_browser_proxy(self):\n launch = AsyncMock(return_value=object())\n p = SimpleNamespace(chromium=SimpleNamespace(launch=launch))\n with patch.dict(os.environ, {\"GEO_PROXY\": \"http://old:8080\"}):\n await _browser.launch_browser(p, \"direct\")\n self.assertNotIn(\"proxy\", launch.call_args.kwargs)\n self.assertIn(\"--no-proxy-server\", launch.call_args.kwargs[\"args\"])\n\n async def test_both_browsers_missing_gives_install_fix(self):\n p = SimpleNamespace(chromium=SimpleNamespace(launch=AsyncMock(side_effect=RuntimeError(\"missing\"))))\n with self.assertRaisesRegex(RuntimeError, \"uvx playwright install chromium\"):\n await _browser.launch_browser(p, \"direct\")\n\n async def test_reverse_search_all_engines_get_selected_route(self):\n browser = SimpleNamespace(new_context=AsyncMock(), close=AsyncMock())\n manager = AsyncMock()\n manager.__aenter__.return_value = object()\n # Avoid requiring Playwright in this offline test's environment.\n fake = SimpleNamespace(async_playwright=lambda: manager)\n with tempfile.TemporaryDirectory() as folder, patch.dict(sys.modules, {\"playwright.async_api\": fake}), patch.object(revimg, \"launch_browser\", AsyncMock(return_value=(browser, \"test\"))) as launch, patch.object(revimg, \"_text\", AsyncMock(return_value={\"links\": []})), patch.object(revimg, \"_baidu\", AsyncMock(return_value={\"links\": []})) as baidu, patch.object(revimg, \"_yandex\", AsyncMock(return_value={\"links\": []})):\n Image.new(\"RGB\", (8, 8)).save(Path(folder) / \"photo.jpg\")\n await revimg.run([Path(folder) / \"photo.jpg\"], [\"baidu\", \"yandex\"], Path(folder), \"direct\", [\"school\"], [\"bing\"])\n self.assertEqual(launch.await_count, 3)\n self.assertTrue(all(call.args[1] == \"direct\" for call in launch.await_args_list))\n self.assertEqual(baidu.await_args.args[-1], \"direct\")\n\n\nclass DoctorTests(unittest.TestCase):\n def test_probe_distinguishes_transport_failure_and_service_block(self):\n for returncode, stdout, status in [(0, \"200\", \"PASS\"), (0, \"429\", \"WARN\"), (7, \"000\", \"FAIL\")]:\n with self.subTest(status=status), patch.object(doctor.subprocess, \"run\", return_value=SimpleNamespace(returncode=returncode, stdout=stdout)):\n row = doctor.probe((\"service\", \"https://example.invalid\"), \"direct\")\n self.assertEqual(row[\"status\"], status)\n\n def test_local_checks_do_not_probe_network_or_print_proxy_credentials(self):\n with tempfile.TemporaryDirectory() as folder, patch.object(doctor, \"browser_check\", AsyncMock(return_value=doctor.check(\"browser\", \"PASS\", \"ready\"))), patch.object(doctor, \"probe\") as probe, patch.dict(os.environ, {\"GEO_PROXY\": \"http://user:secret@localhost:8080\"}):\n report = doctor.diagnose(False, None)\n probe.assert_not_called()\n self.assertTrue(report[\"ok\"])\n self.assertNotIn(\"secret\", json.dumps(report))\n self.assertEqual(report[\"connection\"], \"configured proxy\")\n\n\nQH = '[out:json][timeout:180];(area[\"name\"=\"青海省\"];area[\"name:zh\"=\"青海省\"];area[\"name:zh-Hans\"=\"青海省\"];)->.searchArea;nwr[\"power\"=\"line\"](area.searchArea);out geom tags;'\nLINE = {\"type\": \"way\", \"id\": 1, \"geometry\": [{\"lat\": 37.7, \"lon\": 95.3}, {\"lat\": 37.8, \"lon\": 95.4}], \"tags\": {\"power\": \"line\"}}\n\n\ndef count(n):\n return {\"elements\": [{\"type\": \"count\", \"tags\": {\"total\": str(n)}}]}\n\n\nclass OverpassMirrorTests(unittest.TestCase):\n \"\"\"One mirror answered a whole-province --area query with a clean empty result while others had thousands of lines; it got cached.\"\"\"\n\n def run_with(self, replies, ql=QH):\n calls = []\n\n def post(ep, q, proxy, timeout):\n calls.append(ep)\n return replies(ep, q)\n\n with tempfile.TemporaryDirectory() as d, patch.object(osm, \"_post\", side_effect=post), patch.object(osm.time, \"sleep\"):\n data = osm.run(ql, None, Path(d))\n cached = list(Path(d).glob(\"*.json\"))\n return data, calls, cached\n\n def test_empty_from_one_mirror_is_checked_on_another(self):\n data, calls, cached = self.run_with(lambda ep, q: ({\"elements\": []} if ep == osm.ENDPOINTS[0] else {\"elements\": [LINE]}, \"\"))\n self.assertEqual(len(data[\"elements\"]), 1)\n self.assertEqual(calls, osm.ENDPOINTS[:2])\n self.assertEqual(len(cached), 1)\n\n def test_empty_confirmed_by_two_mirrors_is_cached(self):\n data, calls, cached = self.run_with(lambda ep, q: ({\"elements\": []}, \"\"))\n self.assertEqual(data[\"elements\"], [])\n self.assertEqual(len(calls), 2)\n self.assertEqual(len(cached), 1)\n\n def test_unconfirmed_empty_is_returned_but_not_cached(self):\n data, calls, cached = self.run_with(lambda ep, q: ({\"elements\": []}, \"\") if ep == osm.ENDPOINTS[0] else (None, \"503\"))\n self.assertEqual(data[\"elements\"], [])\n self.assertEqual(cached, [])\n self.assertEqual(len(calls), len(osm.ENDPOINTS)) # one round, no retry sleeps\n\n def test_zero_area_count_needs_a_second_mirror(self):\n ql = '[out:json][timeout:120];(area[\"name\"=\"青海省\"];)->.a;.a out count;'\n data, _, _ = self.run_with(lambda ep, q: (count(0) if ep == osm.ENDPOINTS[0] else count(1), \"\"), ql)\n self.assertEqual(data[\"elements\"][0][\"tags\"][\"total\"], \"1\")\n\n def test_empty_admin_level_inside_a_relation_needs_a_second_mirror(self):\n ql = ('[out:json][timeout:180];rel(161349);map_to_area->.a;'\n 'rel(area.a)[\"boundary\"=\"administrative\"][\"admin_level\"=\"5\"];out tags bb;')\n data, calls, cached = self.run_with(lambda ep, q: ({\"elements\": []} if ep == osm.ENDPOINTS[0] else {\"elements\": [LINE]}, \"\"), ql)\n self.assertEqual(len(data[\"elements\"]), 1)\n self.assertEqual(calls, osm.ENDPOINTS[:2])\n\n def test_legacy_empty_area_cache_is_rechecked_and_replaced(self):\n import hashlib\n ql = '[out:json];rel(161349);map_to_area->.a;rel(area.a);out tags bb;'\n with tempfile.TemporaryDirectory() as d:\n cache = Path(d)\n key = cache / (hashlib.sha1(ql.encode()).hexdigest()[:16] + '.json')\n key.write_text(json.dumps({'elements': []}), encoding='utf-8')\n with patch.object(osm, '_post', return_value=({'elements': [LINE]}, '')) as post:\n self.assertEqual(osm.run(ql, None, cache)['elements'], [LINE])\n self.assertEqual(post.call_count, 1)\n with patch.object(osm, '_post', side_effect=AssertionError('nonempty cache missed')):\n self.assertEqual(osm.run(ql, None, cache)['elements'], [LINE])\n\n def test_confirmed_empty_cache_is_reused_without_leaking_cache_metadata(self):\n with tempfile.TemporaryDirectory() as d:\n with patch.object(osm, '_post', return_value=({'elements': []}, '')) as post:\n self.assertEqual(osm.run(QH, None, Path(d)), {'elements': []})\n self.assertEqual(post.call_count, 2)\n with patch.object(osm, '_post', side_effect=AssertionError('confirmed cache missed')):\n self.assertEqual(osm.run(QH, None, Path(d)), {'elements': []})\n\n def test_unconfirmed_refresh_does_not_turn_legacy_cache_into_confirmed_empty(self):\n import hashlib\n with tempfile.TemporaryDirectory() as d:\n cache = Path(d)\n key = cache / (hashlib.sha1(QH.encode()).hexdigest()[:16] + '.json')\n key.write_text(json.dumps({'elements': []}), encoding='utf-8')\n with patch.object(osm, '_post', side_effect=lambda ep, *a: ({'elements': []}, '') if ep == osm.ENDPOINTS[0] else (None, '503')):\n self.assertEqual(osm.run(QH, None, cache), {'elements': []})\n with patch.object(osm, '_post', return_value=({'elements': [LINE]}, '')) as post:\n self.assertEqual(osm.run(QH, None, cache)['elements'], [LINE])\n self.assertEqual(post.call_count, 1)\n\n def test_bbox_empty_is_trusted(self):\n ql = '[out:json][timeout:180];nwr[\"power\"=\"line\"](37.6,95.2,37.8,95.4);out geom tags;'\n _, calls, cached = self.run_with(lambda ep, q: ({\"elements\": []}, \"\"), ql)\n self.assertEqual(len(calls), 1)\n self.assertEqual(len(cached), 1)\n\n def test_failure_names_every_mirror(self):\n replies = {osm.ENDPOINTS[0]: (None, \"HTTP 406: Not Acceptable\"), osm.ENDPOINTS[1]: (None, \"HTTP 500: Internal Server Error\"),\n osm.ENDPOINTS[2]: (None, \"HTTP 504: server too busy\")}\n with self.assertRaises(SystemExit) as cm:\n self.run_with(lambda ep, q: replies[ep])\n for code in (\"406\", \"500\", \"504\"):\n self.assertIn(code, str(cm.exception))\n\n def test_error_remark_not_cached(self):\n def replies(ep, q):\n if ep == osm.ENDPOINTS[0]:\n return {\"elements\": [], \"remark\": \"runtime error: Query timed out\"}, \"\"\n return {\"elements\": [LINE]}, \"\"\n data, _, cached = self.run_with(replies)\n self.assertEqual(len(data[\"elements\"]), 1)\n self.assertEqual(len(cached), 1)\n\n def test_tls_failure_resends_the_same_post_without_curl(self):\n \"\"\"Windows 10's Schannel curl has no TLS 1.3, which overpass-api.de requires, so curl exits 35 before any HTTP.\"\"\"\n got = []\n class Handler(http.server.BaseHTTPRequestHandler):\n def do_POST(self):\n body = self.rfile.read(int(self.headers[\"Content-Length\"])).decode()\n got.append((self.headers[\"User-Agent\"], urllib.parse.parse_qs(body)[\"data\"]))\n busy = self.path == \"/busy\"\n self.send_response(504 if busy else 200)\n self.end_headers()\n self.wfile.write(b\"server too busy\" if busy else json.dumps({\"elements\": [LINE]}).encode())\n def log_message(self, *args):\n pass\n server = http.server.ThreadingHTTPServer((\"127.0.0.1\", 0), Handler)\n thread = threading.Thread(target=server.serve_forever, daemon=True)\n thread.start()\n base = f\"http://127.0.0.1:{server.server_port}\"\n try:\n with patch.object(osm.subprocess, \"run\", return_value=SimpleNamespace(returncode=35, stdout=\"\")):\n self.assertEqual(osm._post(base + \"/ok\", QH, \"direct\", 5), ({\"elements\": [LINE]}, \"\"))\n self.assertEqual(osm._post(base + \"/busy\", QH, \"direct\", 5), (None, \"HTTP 504: server too busy\"))\n self.assertEqual(got, [(osm.UA, [QH])] * 2)\n finally:\n server.shutdown()\n server.server_close()\n thread.join()\n\n\nclass StreetViewTests(unittest.TestCase):\n def test_lookup_replies(self):\n ok = json.dumps([[0], [[1], [2, \"Z4kP7jsHKunVOqfsg_khyw\"], None, [None, None, [[\"168 W 60th Ave\"], [\"Vancouver\"]]], None,\n [[None, [[None, None, 49.21598, -123.10927], None, [90.4]], None, [[]], None, None, None, None, []]],\n [None] * 7 + [[2024, 5]]]])\n cases = {\n ok: \"Z4kP7jsHKunVOqfsg_khyw\",\n '[[5,\"generic\",\"Search returned no images.\"]]': None,\n }\n for reply, want in cases.items():\n with patch.object(gsv, \"_curl\", return_value=reply.encode()):\n res = gsv.near(49.2161, -123.1093, 50, None)\n self.assertEqual(res and res[\"id\"], want)\n ok_res = None\n with patch.object(gsv, \"_curl\", return_value=ok.encode()):\n ok_res = gsv.near(49.2161, -123.1093, 50, None)\n self.assertEqual(ok_res[\"date\"], \"2024-05\")\n for bad in ('[[5,\"generic\",\"GeoPhotoService.SingleImageSearch is decommissioned and turned down.\"]]',\n '[3,\"Invalid JSON payload received.\"]', \"\u003chtml>blocked\u003c/html>\", \"\"):\n with patch.object(gsv, \"_curl\", return_value=bad.encode()), self.assertRaises(gsv.ServiceError):\n gsv.near(49.2161, -123.1093, 50, None)\n\n\n\ndef _coverage(panos: list[tuple[str, float, float]]) -> str:\n \"\"\"A coverage tile reply in the layout photometa/ac/v1 returns; no panoramas gives the empty-tile layout.\"\"\"\n if not panos:\n return \")]}'\\n\" + json.dumps([None, None, [[\"ChA0YTkw\", [None, None, [None, None, 40.0, -40.0]]]]])\n rows = [[[[2, pid], None, [[None, None, la, lo], [19.2, None, 63.4], [45.9, 0.27, 1.5]]], [27, 113]] for pid, la, lo in panos]\n return \")]}'\\n\" + json.dumps([None, [None, rows]])\n\n\nclass CoverageTests(unittest.TestCase):\n PID = \"77-Z-M-G0QlSNisjusa5DA\"\n\n def setUp(self):\n self.tmp = tempfile.TemporaryDirectory()\n self.cache = Path(self.tmp.name)\n\n def tearDown(self):\n self.tmp.cleanup()\n\n def world(self, panos):\n \"\"\"_curl stand-in serving each z17 tile only the panoramas inside it; records requested tiles.\"\"\"\n asked = []\n\n def curl(url, proxy, out=None, data=None):\n x, y = map(int, re.search(r\"!1i(\\d+)!2i(\\d+)\", url).groups())\n asked.append((x, y))\n inside = [(pid, la, lo) for pid, la, lo in panos\n if tuple(int(v // 256) for v in gsv.geo.ll2px(17, la, lo)) == (x, y)]\n return _coverage(inside).encode()\n return curl, asked\n\n def test_empty_tile_vs_failed_lookup(self):\n with patch.object(gsv, \"_curl\", return_value=_coverage([]).encode()):\n self.assertEqual(gsv.tile_panos(1, 1, None, self.cache), [])\n with patch.object(gsv, \"_curl\", return_value=_coverage([(self.PID, 51.44, 5.47)]).encode()):\n self.assertEqual(gsv.tile_panos(2, 2, None, self.cache), [(self.PID, 51.44, 5.47)])\n for bad in (\"\u003chtml>429 Too Many Requests\u003c/html>\", \"\", \")]}'\\n\u003chtml>429 Too Many Requests\u003c/html>\",\n \")]}'\\n[3,\\\"Invalid request\\\"]\", \")]}'\\n{\\\"error\\\": 1}\"):\n with patch.object(gsv, \"_curl\", return_value=bad.encode()), self.assertRaises(gsv.ServiceError):\n gsv.tile_panos(3, 3, None, self.cache)\n\n def test_failed_reply_is_not_cached(self):\n bad = \")]}'\\n\u003chtml>429 Too Many Requests\u003c/html>\"\n with patch.object(gsv, \"_curl\", return_value=bad.encode()), self.assertRaises(gsv.ServiceError):\n gsv.tile_panos(5, 5, None, self.cache)\n self.assertEqual([f.name for f in self.cache.iterdir()], [])\n good = _coverage([(self.PID, 51.44, 5.47)]).encode()\n with patch.object(gsv, \"_curl\", return_value=good) as m:\n self.assertEqual(len(gsv.tile_panos(5, 5, None, self.cache)), 1)\n self.assertEqual(m.call_count, 1) # retried, not served from cache\n self.assertEqual(len(gsv.tile_panos(5, 5, None, self.cache)), 1)\n self.assertEqual(m.call_count, 1) # a valid reply is cached\n # an error reply left in the cache by an older version is fetched again\n (self.cache / \"cov17_6_6.txt\").write_text(bad, encoding=\"utf-8\")\n with patch.object(gsv, \"_curl\", return_value=good) as m:\n self.assertEqual(len(gsv.tile_panos(6, 6, None, self.cache)), 1)\n self.assertEqual(m.call_count, 1)\n\n def test_disappearing_coverage_cache_is_refetched(self):\n cached = self.cache / \"cov17_7_7.txt\"\n cached.write_text(\")]}'\\n\u003chtml>429 Too Many Requests\u003c/html>\", encoding=\"utf-8\")\n read_text = Path.read_text\n\n def read(path, *args, **kwargs):\n if path == cached:\n path.unlink() # another worker removed a stale entry before this read\n return read_text(path, *args, **kwargs)\n\n good = _coverage([(self.PID, 51.44, 5.47)]).encode()\n with patch.object(Path, \"read_text\", read), patch.object(gsv, \"_curl\", return_value=good) as fetch:\n self.assertEqual(gsv.tile_panos(7, 7, None, self.cache), [(self.PID, 51.44, 5.47)])\n self.assertEqual(fetch.call_count, 1)\n self.assertEqual(gsv._coverage_rows(cached.read_text(encoding=\"utf-8\")), gsv._coverage_rows(good.decode()))\n\n def test_shared_coverage_tile_uses_one_fetch(self):\n for stale in (False, True):\n with self.subTest(stale=stale):\n cached = self.cache / \"cov17_8_8.txt\"\n if stale:\n cached.write_text(\")]}'\\n\u003chtml>429 Too Many Requests\u003c/html>\", encoding=\"utf-8\")\n fetching, release, started = threading.Event(), threading.Event(), threading.Event()\n good = _coverage([(self.PID, 51.44, 5.47)]).encode()\n\n def curl(*args, **kwargs):\n fetching.set()\n if not release.wait(5):\n raise RuntimeError(\"test did not release coverage request\")\n return good\n\n def second():\n started.set()\n return gsv.tile_panos(8, 8, None, self.cache)\n\n with patch.object(gsv, \"_curl\", side_effect=curl) as fetch, ThreadPoolExecutor(2) as ex:\n first = ex.submit(gsv.tile_panos, 8, 8, None, self.cache)\n try:\n self.assertTrue(fetching.wait(5))\n other = ex.submit(second)\n self.assertTrue(started.wait(5))\n with self.assertRaises(TimeoutError):\n other.result(timeout=0.1)\n finally:\n release.set()\n expected = [(self.PID, 51.44, 5.47)]\n self.assertEqual(first.result(timeout=5), expected)\n self.assertEqual(other.result(timeout=5), expected)\n self.assertEqual(fetch.call_count, 1)\n cached.unlink()\n\n def test_targets_reports_failed_lookups(self):\n pts = self.cache / \"pts.json\"\n pts.write_text(json.dumps({\"church\": [51.44, 5.47]}), encoding=\"utf-8\")\n out = self.cache / \"t.index.json\"\n argv = [\"gsv.py\", \"targets\", \"--points\", str(pts), \"--out\", str(out), \"--cache\", str(self.cache / \"c\")]\n with patch.object(gsv, \"_curl\", return_value=b\"\u003chtml>429 Too Many Requests\u003c/html>\"), \\\n patch.object(sys, \"argv\", argv), patch(\"sys.stdout\", new=io.StringIO()) as stdout, \\\n patch(\"sys.stderr\", new=io.StringIO()), self.assertRaises(SystemExit) as e:\n gsv.main()\n self.assertIn(\"lookup failed\", str(e.exception.code))\n self.assertIn(\"0 targets without a panorama\", stdout.getvalue())\n self.assertIn(\"1 lookups failed\", stdout.getvalue())\n\n def _edge_target(self, lat):\n \"\"\"A target 1 px inside the eastern edge of its z17 tile.\"\"\"\n x, y = gsv.geo.ll2px(17, lat, 5.0)\n tx = int(x // 256)\n tlat, tlon = gsv.geo.px2ll(17, (tx + 1) * 256 - 1, y)\n return tx, tlat, tlon\n\n def _east(self, lat, lon, metres):\n return lon + math.degrees(metres / (gsv.geo._R * math.cos(math.radians(lat))))\n\n def test_radius_wider_than_one_tile(self):\n tx, lat, lon = self._edge_target(51.5)\n plon = self._east(lat, lon, 250)\n self.assertEqual(int(gsv.geo.ll2px(17, lat, plon)[0] // 256), tx + 2) # outside the old 3x3 block\n curl, asked = self.world([(self.PID, lat, plon)])\n with patch.object(gsv, \"_curl\", side_effect=curl):\n got = gsv.panos_around(lat, lon, 12, 300, 2, 15, None, self.cache)\n self.assertEqual([g[0] for g in got], [self.PID])\n self.assertAlmostEqual(got[0][3], 250, delta=1)\n self.assertIn(tx + 2, {a[0] for a in asked})\n with patch.object(gsv, \"_curl\", side_effect=curl): # the exact distance filter still applies\n self.assertEqual(gsv.panos_around(lat, lon, 12, 200, 2, 15, None, self.cache), [])\n\n def test_high_latitude_default_radius(self):\n tx, lat, lon = self._edge_target(78.2)\n plon = self._east(lat, lon, 69)\n self.assertEqual(int(gsv.geo.ll2px(17, lat, plon)[0] // 256), tx + 2)\n curl, _ = self.world([(self.PID, lat, plon)])\n with patch.object(gsv, \"_curl\", side_effect=curl):\n got = gsv.panos_around(lat, lon, 12, 70, 2, 15, None, self.cache)\n self.assertEqual([g[0] for g in got], [self.PID])\n\n def test_tile_range_stays_small(self):\n self.assertLessEqual(len(gsv.tiles_around(51.5, 5.0, 70)), 4) # a 70 m radius spans at most 2x2 tiles of ~190 m\n tiles = gsv.tiles_around(0.0, 179.9999, 300) # wraps across the antimeridian\n self.assertTrue(all(0 \u003c= x \u003c 2 ** 17 for x, _ in tiles))\n self.assertIn(0, {x for x, _ in tiles})\n\nclass FontTests(unittest.TestCase):\n def _tried_paths(self, env):\n tried = []\n\n def fake_truetype(path, size):\n tried.append(path)\n raise OSError(\"not found\")\n\n with patch.dict(os.environ, env, clear=True), patch.object(baidu_pano.ImageFont, \"truetype\", side_effect=fake_truetype), patch.object(baidu_pano.ImageFont, \"load_default\", return_value=\"default\"):\n self.assertEqual(baidu_pano._font(16), \"default\")\n return tried\n\n def test_windows_cjk_fonts_are_tried(self):\n tried = self._tried_paths({})\n self.assertIn(\"C:/Windows/Fonts/msyh.ttc\", tried)\n self.assertIn(\"C:/Windows/Fonts/msyhbd.ttc\", tried)\n self.assertIn(\"C:/Windows/Fonts/simsun.ttc\", tried)\n\n def test_geo_font_is_tried_first(self):\n tried = self._tried_paths({\"GEO_FONT\": \"D:/fonts/custom.ttf\"})\n self.assertEqual(tried[0], \"D:/fonts/custom.ttf\")\n\n def test_first_available_font_is_used(self):\n def fake_truetype(path, size):\n if path != \"C:/Windows/Fonts/simhei.ttf\":\n raise OSError(\"not found\")\n return path\n\n with patch.dict(os.environ, {}, clear=True), patch.object(baidu_pano.ImageFont, \"truetype\", side_effect=fake_truetype):\n self.assertEqual(baidu_pano._font(16), \"C:/Windows/Fonts/simhei.ttf\")\n\n\nclass SkillMetadataTests(unittest.TestCase):\n def test_frontmatter_meets_agent_skills_spec(self):\n root = Path(__file__).resolve().parents[1]\n m = re.match(r\"---\\n(.*?)\\n---\\n\", (root / \"SKILL.md\").read_text(encoding=\"utf-8\"), re.S)\n self.assertIsNotNone(m, \"frontmatter must open and close with --- lines\")\n meta = yaml.safe_load(m.group(1)) # strict parser: a ': ' inside the description breaks it\n self.assertIsInstance(meta, dict)\n self.assertEqual(meta[\"name\"], root.name)\n self.assertIsInstance(meta[\"description\"], str)\n self.assertTrue(meta[\"description\"].strip())\n self.assertLessEqual(len(meta[\"description\"]), 1024) # https://agentskills.io/specification\n\n\nclass EvidenceSpecTests(unittest.TestCase):\n def setUp(self):\n self.tmp = tempfile.TemporaryDirectory()\n self.base = Path(self.tmp.name)\n (self.base / \"area.jpg\").write_bytes(b\"jpg\")\n (self.base / \"area.json\").write_text(\"{}\", encoding=\"utf-8\")\n\n def tearDown(self):\n self.tmp.cleanup()\n\n def test_missing_map_is_named_instead_of_key_error(self):\n with self.assertRaises(SystemExit) as e:\n evidence.build({\"camera\": [1, 2]}, self.base, self.base / \"out.jpg\")\n self.assertIn(\"missing 'map'\", str(e.exception.code))\n\n def test_map_without_georeference(self):\n (self.base / \"area.json\").unlink()\n self.assertIn(\"no georeference\", evidence.check_spec({\"map\": \"area.jpg\"}, self.base)[0])\n\n def test_item_keys_and_panel_images(self):\n errs = evidence.check_spec({\"map\": \"area.jpg\", \"labels\": [{\"at\": [1, 2]}], \"lines\": [{\"from\": [1, 2], \"bearing\": 3}],\n \"panels\": [{\"image\": \"sv1.jpg\"}]}, self.base)\n self.assertEqual(len(errs), 3)\n self.assertIn(\"labels[0] is missing 'text'\", errs[0])\n self.assertIn(\"lines[0] is missing 'length_m'\", errs[1])\n self.assertIn(\"panels[0] image not found\", errs[2])\n\n def test_valid_spec(self):\n self.assertEqual(evidence.check_spec({\"map\": \"area.jpg\", \"labels\": [{\"at\": [1, 2], \"text\": \"x\"}]}, self.base), [])\n\n def test_invalid_path_and_item_types_are_named(self):\n cases = [\n ([], \"JSON object\"),\n *[({\"map\": value}, \"'map'\") for value in (None, 7, \"\")],\n *[({\"map\": \"area.jpg\", key: value}, repr(key))\n for key, value in ((\"labels\", None), (\"labels\", {}), (\"lines\", \"bad\"), (\"panels\", None))],\n *[({\"map\": \"area.jpg\", key: [value]}, f\"{key}[0]\")\n for key, value in ((\"labels\", None), (\"labels\", \"at text\"), (\"lines\", 3))],\n ({\"map\": \"area.jpg\", \"panels\": [{\"image\": None}]}, \"panels[0]\"),\n ]\n for spec, expected in cases:\n with self.subTest(spec=spec):\n self.assertTrue(any(expected in error for error in evidence.check_spec(spec, self.base)))\n\n def test_missing_files_are_reported_together(self):\n errors = evidence.check_spec({\"map\": \"missing.jpg\", \"panels\": [{\"image\": \"missing-panel.jpg\"}]}, self.base)\n self.assertEqual(len(errors), 3)\n self.assertIn(\"map not found\", errors[0])\n self.assertIn(\"no georeference\", errors[1])\n self.assertIn(\"panels[0] image not found\", errors[2])\n\n def test_invalid_spec_cli_exits_without_traceback(self):\n spec_path = self.base / \"bad.json\"\n spec_path.write_text(json.dumps({\"map\": None, \"labels\": None, \"panels\": [{\"image\": None}]}), encoding=\"utf-8\")\n out = self.base / \"out.jpg\"\n result = subprocess.run([sys.executable, str(Path(evidence.__file__)), str(spec_path), \"--out\", str(out)],\n capture_output=True, text=True, encoding=\"utf-8\")\n self.assertEqual(result.returncode, 1)\n for expected in (\"'map'\", \"'labels'\", \"panels[0]\", \"evidence.py --help\"):\n self.assertIn(expected, result.stderr)\n self.assertNotIn(\"Traceback\", result.stderr)\n self.assertFalse(out.exists())\n\n\nclass DeviceTests(unittest.TestCase):\n @staticmethod\n def _torch(cuda=False, mps=False):\n return SimpleNamespace(__version__=\"2.0.0+test\",\n cuda=SimpleNamespace(is_available=lambda: cuda),\n backends=SimpleNamespace(mps=SimpleNamespace(is_available=lambda: mps)))\n\n def _pick(self, value=None, **have):\n env = {} if value is None else {\"GEO_DEVICE\": value}\n with patch.dict(os.environ, env, clear=True):\n return _device.torch_device(self._torch(**have))\n\n def _exit_message(self, value, **have):\n with self.assertRaises(SystemExit) as caught:\n self._pick(value, **have)\n message = str(caught.exception.code)\n self.assertEqual(len(message.splitlines()), 1)\n self.assertIn(\"torch 2.0.0+test\", message)\n return message\n\n def test_auto_selection_order(self):\n self.assertEqual(self._pick(cuda=True, mps=True), \"cuda\")\n self.assertEqual(self._pick(cuda=True), \"cuda\")\n self.assertEqual(self._pick(mps=True), \"mps\")\n self.assertEqual(self._pick(), \"cpu\")\n self.assertEqual(self._pick(\"\", cuda=True), \"cuda\")\n\n def test_explicit_override(self):\n self.assertEqual(self._pick(\"cpu\", cuda=True, mps=True), \"cpu\")\n self.assertEqual(self._pick(\"cuda\", cuda=True, mps=True), \"cuda\")\n self.assertEqual(self._pick(\"mps\", cuda=True, mps=True), \"mps\")\n self.assertEqual(self._pick(\" CUDA \", cuda=True), \"cuda\")\n\n def test_invalid_value_exits_with_one_line(self):\n for value in (\"gpu\", \"cuda:0\", \"auto\"):\n message = self._exit_message(value, cuda=True)\n self.assertIn(repr(value), message)\n self.assertIn(\"cpu, cuda, mps\", message)\n\n def test_unavailable_device_exits_instead_of_falling_back(self):\n message = self._exit_message(\"cuda\", mps=True)\n self.assertIn(\"GEO_DEVICE=cuda\", message)\n self.assertIn(\"cuda=False, mps=True\", message)\n message = self._exit_message(\"mps\", cuda=True)\n self.assertIn(\"GEO_DEVICE=mps\", message)\n self.assertIn(\"cuda=True, mps=False\", message)\n\n def test_both_model_scripts_use_the_shared_helper(self):\n scripts = Path(_device.__file__).parent\n for name in (\"match.py\", \"sat_scan.py\"):\n text = (scripts / name).read_text(encoding=\"utf-8\")\n self.assertIn(\"self.dev = torch_device(torch)\", text)\n self.assertNotIn(\"mps.is_available\", text)\n\n\nHEIC_HEAD = b\"\\x00\\x00\\x00\\x1cftypheic\" + b\"\\x00\" * 64\n\n\nclass ImageLoadingTests(unittest.TestCase):\n \"\"\"scripts/_image.py: explicit formats, content sniffing, RAW rejection, failure categories, folder discovery.\"\"\"\n\n def setUp(self):\n self.tmp = tempfile.TemporaryDirectory()\n self.d = Path(self.tmp.name)\n self.addCleanup(self.tmp.cleanup)\n\n def _save(self, name, fmt=\"JPEG\", color=(200, 30, 30), size=(32, 24)):\n Image.new(\"RGB\", size, color).save(self.d / name, format=fmt)\n return self.d / name\n\n def _category(self, path):\n with self.assertRaises(_image.ImageReadError) as caught:\n _image.open_image(path)\n self.assertEqual(len(caught.exception.reason.splitlines()), 1)\n return caught.exception.category, caught.exception.reason\n\n def test_sniff_reads_content_not_suffix(self):\n heads = {\n \"JPEG\": b\"\\xff\\xd8\\xff\\xe0\", \"PNG\": b\"\\x89PNG\\r\\n\\x1a\\n\", \"WEBP\": b\"RIFF\\x00\\x00\\x00\\x00WEBPVP8 \",\n \"GIF\": b\"GIF89a\", \"HEIF\": HEIC_HEAD[:16], \"AVIF\": b\"\\x00\\x00\\x00\\x1cftypavif\", \"TIFF\": b\"II*\\x00\\x08\\x00\\x00\\x00\",\n \"TEXT\": b'{\"status\":400}',\n }\n for kind, head in heads.items():\n self.assertEqual(_image.sniff(head, \".jpg\"), kind)\n self.assertEqual(_image.sniff(b\"II*\\x00\\x10\\x00\\x00\\x00CR\\x02\\x00\", \".jpg\"), \"RAW\") # Canon CR2\n self.assertEqual(_image.sniff(b\"\\x00\\x00\\x00\\x18ftypcrx \", \".cr3\"), \"RAW\") # Canon CR3\n self.assertEqual(_image.sniff(b\"MM\\x00*\\x00\\x00\\x00\\x08\", \".NEF\"), \"RAW\") # TIFF-based RAW by suffix\n self.assertEqual(_image.sniff(b\"FUJIFILMCCD-RAW 0201\", \".raf\"), \"RAW\")\n self.assertIsNone(_image.sniff(b\"\\x00\\x01\\x02\\x03\", \".jpg\"))\n\n def test_supported_formats_open_whatever_the_suffix(self):\n for name, fmt in ((\"a.jpg\", \"JPEG\"), (\"b.png\", \"PNG\"), (\"c.webp\", \"WEBP\"), (\"d.jpg\", \"PNG\"), (\"e\", \"JPEG\"), (\"f.gif\", \"GIF\")):\n with _image.open_image(self._save(name, fmt)) as im:\n self.assertEqual((im.format, im.size), (fmt, (32, 24)))\n\n @unittest.skipUnless(_image.HEIF, \"pillow-heif not installed\")\n def test_heic_decodes_and_keeps_metadata(self):\n ex = Image.Exif()\n ex[0x8769] = {0xA405: 26} # FocalLengthIn35mmFilm\n Image.new(\"RGB\", (40, 30), (0, 0, 255)).save(self.d / \"phone.heic\", format=\"HEIF\", exif=ex)\n im = _image.open_image(self.d / \"phone.heic\")\n self.assertEqual((im.format, im.size), (\"HEIF\", (40, 30)))\n self.assertGreater(im.getpixel((0, 0))[2], 200)\n self.assertEqual(im.getexif().get_ifd(0x8769).get(0xA405), 26)\n\n def test_failure_categories(self):\n (self.d / \"empty.jpg\").write_bytes(b\"\")\n (self.d / \"error.jpg\").write_bytes(b'{\"status\":400,\"msg\":\"bad\"}')\n (self.d / \"noise.jpg\").write_bytes(b\"\\x00\\x01\\x02\\x03\" * 20)\n (self.d / \"IMG_1.CR2\").write_bytes(b\"II*\\x00\\x10\\x00\\x00\\x00CR\\x02\\x00\" + b\"\\x00\" * 64)\n (self.d / \"IMG_2.dng\").write_bytes(b\"II*\\x00\\x08\\x00\\x00\\x00\" + b\"\\x00\" * 64)\n full = self._save(\"full.jpg\").read_bytes()\n (self.d / \"cut.jpg\").write_bytes(full[: len(full) // 2])\n Image.new(\"RGB\", (8, 8)).save(self.d / \"frame.ppm\", format=\"PPM\")\n expect = {\"empty.jpg\": \"empty\", \"error.jpg\": \"not-image\", \"noise.jpg\": \"not-image\", \"IMG_1.CR2\": \"raw\",\n \"IMG_2.dng\": \"raw\", \"cut.jpg\": \"decode-error\", \"frame.ppm\": \"unsupported\"}\n for name, category in expect.items():\n got, reason = self._category(self.d / name)\n self.assertEqual(got, category, f\"{name}: {reason}\")\n self.assertIn(\"export it as JPEG or PNG\", self._category(self.d / \"IMG_1.CR2\")[1])\n\n def test_tiff_based_raw_is_rejected_by_content(self):\n \"\"\"Structure of an iPhone ProRAW DNG: the first image is a full-size 8-bit preview marked reduced-resolution,\n the real 10-bit linear raw sits in a SubIFD. Pillow would read the preview, whatever the suffix.\"\"\"\n from PIL import TiffImagePlugin\n for name, tags in ((\"dng_named.jpg\", {254: 1, 50706: b\"\\x01\\x06\\x00\\x00\"}), (\"preview_only\", {254: 1, 330: 100})):\n info = TiffImagePlugin.ImageFileDirectory_v2()\n for k, v in tags.items():\n info[k] = v\n if 50706 in tags:\n info.tagtype[50706] = 1\n Image.new(\"RGB\", (40, 30), (90, 120, 150)).save(self.d / name, format=\"TIFF\", tiffinfo=info)\n category, reason = self._category(self.d / name)\n self.assertEqual(category, \"raw\", name)\n self.assertIn(\"embedded preview\", reason)\n self._save(\"plain.tif\", \"TIFF\")\n with _image.open_image(self.d / \"plain.tif\") as im:\n self.assertEqual(im.format, \"TIFF\")\n\n def test_heif_failure_keeps_its_cause(self):\n (self.d / \"x.jpg\").write_bytes(HEIC_HEAD)\n with patch.object(_image, \"HEIF\", False):\n self.assertEqual(self._category(self.d / \"x.jpg\")[0], \"missing-decoder\")\n if _image.HEIF:\n category, reason = self._category(self.d / \"x.jpg\")\n self.assertEqual(category, \"decode-error\")\n self.assertIn(\"HEIF that could not be decoded (\", reason) # the decoder's own message, not a guess\n\n def test_find_images_keeps_old_order_then_adds_new_formats(self):\n for name, fmt in ((\"b.jpg\", \"JPEG\"), (\"a.jpeg\", \"JPEG\"), (\"c.png\", \"PNG\"), (\"d.webp\", \"WEBP\"),\n (\"e.HEIC\", \"PNG\"), (\"f.heif\", \"PNG\"), (\"g.bin\", \"PNG\"), (\"h\", \"JPEG\")):\n self._save(name, fmt)\n (self.d / \"notes.json\").write_text(\"{}\", encoding=\"utf-8\")\n (self.d / \"readme.txt\").write_text(\"hello\", encoding=\"utf-8\")\n (self.d / \"IMG_3.CR2\").write_bytes(b\"II*\\x00\\x10\\x00\\x00\\x00CR\\x02\\x00\")\n names = [f.name for f in _image.find_images(self.d)]\n old = sorted([\"b.jpg\", \"a.jpeg\"]) + [\"c.png\"]\n self.assertEqual(names[:3], old)\n self.assertEqual(sorted(names[3:]), sorted([\"d.webp\", \"e.HEIC\", \"f.heif\", \"IMG_3.CR2\", \"g.bin\", \"h\"]))\n self.assertNotIn(\"notes.json\", names)\n self.assertNotIn(\"readme.txt\", names)\n\n def test_open_or_exit_is_one_line(self):\n (self.d / \"IMG_4.NEF\").write_bytes(b\"MM\\x00*\\x00\\x00\\x00\\x08\" + b\"\\x00\" * 64)\n with self.assertRaises(SystemExit) as caught:\n _image.open_or_exit(self.d / \"IMG_4.NEF\", \"photo\")\n message = str(caught.exception.code)\n self.assertEqual(len(message.splitlines()), 1)\n self.assertIn(\"cannot read photo\", message)\n self.assertIn(\"export it as JPEG or PNG\", message)\n\n def test_photo_scripts_exit_cleanly_on_raw_and_read_heic(self):\n script = Path(_image.__file__).with_name(\"imgprep.py\")\n raw = self.d / \"IMG_5.ARW\"\n raw.write_bytes(b\"II*\\x00\\x08\\x00\\x00\\x00\" + b\"\\x00\" * 64)\n r = subprocess.run([sys.executable, str(script), \"zoom\", str(raw), \"--box\", \"0,0,4,4\", \"--out\", str(self.d / \"z.png\")],\n capture_output=True, text=True, encoding=\"utf-8\")\n self.assertNotEqual(r.returncode, 0)\n self.assertNotIn(\"Traceback\", r.stderr)\n self.assertIn(\"export it as JPEG or PNG\", r.stderr)\n if _image.HEIF:\n Image.new(\"RGB\", (40, 30), (0, 200, 0)).save(self.d / \"p.heic\", format=\"HEIF\")\n r = subprocess.run([sys.executable, str(script), \"zoom\", str(self.d / \"p.heic\"), \"--box\", \"0,0,20,20\", \"--out\", str(self.d / \"z.png\")],\n capture_output=True, text=True, encoding=\"utf-8\")\n self.assertEqual(r.returncode, 0, r.stderr)\n self.assertTrue((self.d / \"z.png\").exists())\n\n\nclass MatchImageLoadingTests(unittest.TestCase):\n def setUp(self):\n self.tmp = tempfile.TemporaryDirectory()\n self.d = Path(self.tmp.name)\n self.addCleanup(self.tmp.cleanup)\n\n def _save(self, name, fmt=\"JPEG\", color=(200, 30, 30)):\n Image.new(\"RGB\", (32, 24), color).save(self.d / name, format=fmt)\n\n def _load(self, files):\n err = io.StringIO()\n with contextlib.redirect_stderr(err):\n ok, ims = match._open_images(files)\n return ok, ims, err.getvalue()\n\n def test_unreadable_files_are_skipped_with_reasons(self):\n self._save(\"a.jpg\")\n self._save(\"b.jpg\", color=(30, 200, 30))\n (self.d / \"heic.jpg\").write_bytes(HEIC_HEAD)\n (self.d / \"error.jpg\").write_bytes('{\"status\":400,\"msg\":\"bad\"}'.encode())\n (self.d / \"empty.jpg\").write_bytes(b\"\")\n full = (self.d / \"a.jpg\").read_bytes()\n (self.d / \"cut.jpg\").write_bytes(full[: len(full) // 2])\n ok, ims, err = self._load(sorted(self.d.glob(\"*.jpg\")))\n self.assertEqual([f.name for f in ok], [\"a.jpg\", \"b.jpg\"])\n self.assertEqual([im.size for im in ims], [(32, 24), (32, 24)])\n self.assertEqual(len(err.strip().splitlines()), 1)\n self.assertIn(\"skipped 4 of 6 files\", err)\n self.assertIn(\"text, not an image\", err)\n self.assertIn(\"1 empty file\", err)\n\n def test_missing_decoder_is_reported_as_such(self):\n self._save(\"a.jpg\")\n (self.d / \"h.heic\").write_bytes(HEIC_HEAD)\n with patch.object(_image, \"HEIF\", False):\n ok, _, err = self._load([self.d / \"a.jpg\", self.d / \"h.heic\"])\n self.assertEqual([f.name for f in ok], [\"a.jpg\"])\n self.assertIn(\"needs pillow-heif\", err)\n\n def test_all_unreadable_exits_with_one_line(self):\n (self.d / \"x.jpg\").write_bytes(b\"\u003chtml>blocked\u003c/html>\")\n (self.d / \"y.CR2\").write_bytes(b\"II*\\x00\\x10\\x00\\x00\\x00CR\\x02\\x00\")\n with contextlib.redirect_stderr(io.StringIO()), self.assertRaises(SystemExit) as caught:\n match._open_images([self.d / \"x.jpg\", self.d / \"y.CR2\"])\n self.assertEqual(len(str(caught.exception.code).splitlines()), 1)\n self.assertIn(\"none of the 2 candidate files\", str(caught.exception.code))\n\n def _candidates(self):\n args = SimpleNamespace(images=str(self.d), items=None, panos=None, cache=str(self.d / \".cache\"))\n err = io.StringIO()\n with contextlib.redirect_stderr(err):\n items, ims = match._load_candidates(args)\n return items, ims, err.getvalue()\n\n def test_mixed_folder_keeps_ids_aligned(self):\n self._save(\"first.jpg\", color=(255, 0, 0))\n (self.d / \"middle.jpg\").write_bytes(HEIC_HEAD)\n self._save(\"last.png\", \"PNG\", color=(0, 0, 255))\n self._save(\"UPPER.WEBP\", \"WEBP\", color=(0, 255, 0))\n self._save(\"renamed.bin\", \"PNG\", color=(255, 255, 0))\n items, ims, _ = self._candidates()\n ids = [it[\"id\"] for it in items]\n self.assertEqual(ids[:2], [\"first\", \"last\"])\n self.assertEqual(sorted(ids[2:]), [\"UPPER\", \"renamed\"])\n colors = {it[\"id\"]: im.convert(\"RGB\").getpixel((0, 0)) for it, im in zip(items, ims)}\n self.assertGreater(colors[\"first\"][0], 240)\n self.assertGreater(colors[\"last\"][2], 240)\n self.assertGreater(colors[\"UPPER\"][1], 240)\n\n @unittest.skipUnless(_image.HEIF, \"pillow-heif not installed\")\n def test_heic_only_folder(self):\n for i, c in enumerate(((255, 0, 0), (0, 0, 255))):\n Image.new(\"RGB\", (40, 30), c).save(self.d / f\"p{i}.HEIC\", format=\"HEIF\")\n Image.new(\"RGB\", (40, 30), (0, 200, 0)).save(self.d / \"phone.jpg\", format=\"HEIF\")\n items, ims, err = self._candidates()\n self.assertEqual([it[\"id\"] for it in items], [\"phone\", \"p0\", \"p1\"])\n self.assertEqual([im.size for im in ims], [(40, 30)] * 3)\n self.assertEqual(err, \"\")\n\n\n# EXIF orientation -> the transpose a viewer applies to the stored pixels; storing = applying the inverse\nVIEW = {2: Image.Transpose.FLIP_LEFT_RIGHT, 3: Image.Transpose.ROTATE_180, 4: Image.Transpose.FLIP_TOP_BOTTOM,\n 5: Image.Transpose.TRANSPOSE, 6: Image.Transpose.ROTATE_270, 7: Image.Transpose.TRANSVERSE, 8: Image.Transpose.ROTATE_90}\nINVERSE = {Image.Transpose.ROTATE_270: Image.Transpose.ROTATE_90, Image.Transpose.ROTATE_90: Image.Transpose.ROTATE_270}\n\n\ndef upright_control(w=60, h=40):\n \"\"\"Asymmetric: every flip or rotation of it is a different image.\"\"\"\n im = Image.new(\"RGB\", (w, h), (230, 230, 230))\n im.paste((220, 30, 30), (0, 0, w // 3, h // 2)) # red, top-left\n im.paste((30, 30, 220), (w - w // 4, h // 2, w, h)) # blue, bottom-right\n im.paste((30, 160, 30), (w // 2, 0, w // 2 + 4, h // 4)) # green tick near the top\n return im\n\n\ndef stored_as(control, o):\n if o == 1:\n return control.copy()\n t = VIEW[o]\n return control.transpose(INVERSE.get(t, t))\n\n\ndef save_oriented(control, o, path, fmt=\"PNG\", **kw):\n ex = Image.Exif()\n ex[0x0112] = o\n stored_as(control, o).save(path, format=fmt, exif=ex, **kw)\n\n\nclass OrientationTests(unittest.TestCase):\n \"\"\"Stage 2 of #29: pixel coordinates refer to the upright frame; EXIF orientation is applied exactly once.\"\"\"\n\n def setUp(self):\n self.tmp = tempfile.TemporaryDirectory()\n self.d = Path(self.tmp.name)\n self.addCleanup(self.tmp.cleanup)\n self.control = upright_control()\n\n def test_all_eight_orientations_match_the_upright_control(self):\n for o in range(1, 9):\n save_oriented(self.control, o, self.d / f\"o{o}.png\")\n im = _image.open_photo(self.d / f\"o{o}.png\")\n self.assertEqual(im.size, self.control.size, f\"orientation {o}\")\n self.assertEqual(im.convert(\"RGB\").tobytes(), self.control.tobytes(), f\"orientation {o}\")\n self.assertEqual(_image.upright(im).tobytes(), im.tobytes(), f\"orientation {o} applied twice\")\n\n def test_jpeg_orientations_match_within_compression_noise(self):\n ref = np.asarray(self.control, dtype=np.int16)\n for o in (3, 6, 8):\n save_oriented(self.control, o, self.d / f\"o{o}.jpg\", fmt=\"JPEG\", quality=95)\n got = np.asarray(_image.open_photo(self.d / f\"o{o}.jpg\").convert(\"RGB\"), dtype=np.int16)\n self.assertEqual(got.shape, ref.shape)\n self.assertLess(np.abs(got - ref).mean(), 3, f\"orientation {o}\")\n\n def test_normal_photo_is_returned_untouched(self):\n save_oriented(self.control, 1, self.d / \"o1.png\")\n im = _image.open_image(self.d / \"o1.png\")\n self.assertIs(_image.upright(im), im)\n\n @unittest.skipUnless(_image.HEIF, \"pillow-heif not installed\")\n def test_heif_is_not_rotated_again(self):\n \"\"\"libheif applies irot/imir itself and pillow-heif resets the EXIF tag; the HEIF spec says EXIF orientation is\n ignored. Whatever the EXIF says, the result must be exactly what the HEIF decoder produced.\"\"\"\n for o in (1, 3, 6, 8):\n save_oriented(self.control, o, self.d / f\"h{o}.heic\", fmt=\"HEIF\", quality=-1)\n decoded = Image.open(self.d / f\"h{o}.heic\")\n decoded.load()\n im = _image.open_photo(self.d / f\"h{o}.heic\")\n self.assertEqual(_image.orientation(im), 1)\n self.assertEqual(im.size, decoded.size)\n self.assertEqual(im.tobytes(), decoded.tobytes(), f\"orientation {o}\")\n\n def test_frames_are_read_from_the_record_not_guessed_from_the_size(self):\n self.assertEqual(_image.frame_of({}), \"stored\") # written before orientation was applied\n self.assertEqual(_image.frame_of({\"frame\": \"upright\"}), \"upright\")\n self.assertEqual(_image.frame_of({}, \"upright\"), \"upright\") # callers can explicitly override the default\n with self.assertRaises(SystemExit):\n _image.frame_of({\"frame\": \"sideways\"})\n for control in (self.control, upright_control(40, 40)): # square: no size tells the frames apart\n for o in range(1, 9):\n save_oriented(control, o, self.d / f\"o{o}.png\")\n err = io.StringIO()\n with contextlib.redirect_stderr(err):\n stored = _image.open_photo(self.d / f\"o{o}.png\", frame=\"stored\", size=stored_as(control, o).size)\n self.assertEqual(stored.convert(\"RGB\").tobytes(), stored_as(control, o).tobytes(), f\"orientation {o}\")\n self.assertEqual(\"stored frame\" in err.getvalue(), o != 1, err.getvalue())\n self.assertNotIn(\"warning\", err.getvalue())\n up = _image.open_photo(self.d / f\"o{o}.png\", size=control.size)\n self.assertEqual(up.convert(\"RGB\").tobytes(), control.tobytes(), f\"orientation {o}\")\n\n def test_a_size_that_fits_neither_frame_is_flagged(self):\n save_oriented(self.control, 6, self.d / \"o6.png\") # 40x60 stored, 60x40 upright\n err = io.StringIO()\n with contextlib.redirect_stderr(err):\n _image.open_photo(self.d / \"o6.png\", size=[40, 60]) # a stored-frame size in an upright record\n self.assertIn('set \"frame\": \"stored\"', err.getvalue())\n err = io.StringIO()\n with contextlib.redirect_stderr(err):\n _image.open_photo(self.d / \"o6.png\", size=[120, 80]) # same aspect, scaled: fine\n self.assertEqual(err.getvalue(), \"\")\n\n def test_match_query_and_candidates_use_the_upright_frame(self):\n save_oriented(self.control, 6, self.d / \"q.png\")\n save_oriented(self.control, 8, self.d / \"c8.png\")\n self.control.save(self.d / \"c1.png\")\n self.assertEqual(match.open_photo(self.d / \"q.png\", \"query\").convert(\"RGB\").tobytes(), self.control.tobytes())\n args = SimpleNamespace(images=str(self.d), items=None, panos=None, cache=str(self.d / \".cache\"))\n with contextlib.redirect_stderr(io.StringIO()):\n items, ims = match._load_candidates(args)\n for it, im in zip(items, ims):\n self.assertEqual(im.convert(\"RGB\").tobytes(), self.control.tobytes(), it[\"id\"])\n\n def _run(self, script, *args):\n r = subprocess.run([sys.executable, str(Path(_image.__file__).with_name(script)), *map(str, args)],\n capture_output=True, text=True, encoding=\"utf-8\")\n self.assertEqual(r.returncode, 0, r.stderr)\n return r\n\n def test_imgprep_crops_use_the_upright_frame(self):\n save_oriented(self.control, 6, self.d / \"o6.png\")\n self.control.save(self.d / \"up.png\")\n for name in (\"o6\", \"up\"):\n self._run(\"imgprep.py\", \"zoom\", self.d / f\"{name}.png\", \"--box\", \"0,0,30,20\", \"--scale\", \"2\", \"--out\", self.d / f\"z_{name}.png\")\n self.assertEqual(Image.open(self.d / \"z_o6.png\").tobytes(), Image.open(self.d / \"z_up.png\").tobytes())\n\n def _pose_case(self, w, h):\n lat0, lon0 = 30.0, 120.0\n rec = {\"_frame\": {\"lat0\": lat0, \"lon0\": lon0, \"image_size\": [w, h]},\n \"_params\": [0.0, 0.0, 10.0, 0.0, 0.0, 0.0, 200.0]} # east, north, height, heading, pitch, roll, focal_px\n pts = [{\"name\": \"A\", \"ll\": [lat0 + 0.0009, lon0 + 0.0002], \"h\": 5.0, \"px\": [w * 5 // 8, h * 9 // 16]},\n {\"name\": \"B\", \"ll\": [lat0 + 0.0009, lon0 - 0.0003], \"h\": 0.0, \"px\": [w // 4, h * 5 // 8]}]\n return rec, pts\n\n def _draw(self, photo, rec, pts, out):\n with contextlib.redirect_stderr(io.StringIO()) as err:\n pose.draw(photo, rec, pts, out, observed=True)\n with Image.open(out) as im:\n return im.tobytes(), err.getvalue()\n\n def test_pose_overlay_uses_the_upright_frame(self):\n for w, h in ((240, 160), (160, 160)):\n scene = upright_control(w, h)\n scene.save(self.d / \"up.png\")\n rec, pts = self._pose_case(w, h)\n rec[\"_frame\"][\"frame\"] = \"upright\"\n control, _ = self._draw(self.d / \"up.png\", rec, pts, self.d / \"d_up.png\")\n for o in range(2, 9):\n save_oriented(scene, o, self.d / f\"o{o}.png\")\n got, err = self._draw(self.d / f\"o{o}.png\", rec, pts, self.d / f\"d_o{o}.png\")\n self.assertEqual(got, control, f\"{w}x{h} orientation {o}\")\n self.assertEqual(err, \"\")\n\n def test_old_pose_records_draw_as_they_did(self):\n \"\"\"A pose.json from before #29 has no frame: its points were drawn on the stored pixels, and still are. Covers the\n cases a size check can't tell apart: mirror / 180° (2, 3, 4) keep the size, and a square photo keeps it under 6.\"\"\"\n for (w, h), orients in (((240, 160), (2, 3, 4, 6, 8)), ((160, 160), (6, 3))):\n scene = upright_control(w, h)\n for o in orients:\n stored = stored_as(scene, o)\n stored.save(self.d / \"plain.png\") # what the old pose.draw opened: no EXIF to apply\n save_oriented(scene, o, self.d / f\"o{o}.png\")\n rec, pts = self._pose_case(*stored.size) # recorded on the stored pixels\n old, _ = self._draw(self.d / \"plain.png\", rec, pts, self.d / \"d_plain.png\")\n got, err = self._draw(self.d / f\"o{o}.png\", rec, pts, self.d / f\"d_o{o}.png\")\n self.assertEqual(got, old, f\"{w}x{h} orientation {o}\")\n self.assertIn(f\"EXIF orientation {o}\", err)\n self.assertNotIn(\"warning\", err)\n\n def _pose_spec(self, W=400, H=300):\n lat0, lon0 = 30.0, 120.0\n kx, ky = pose._frame(lat0)\n truth = np.array([3.0, -4.0, 12.0, 20.0, -2.0, 0.0, 350.0])\n enu = np.array([[-60, 220, 0], [40, 260, 25], [90, 180, 5], [-20, 140, 15], [10, 320, 40], [-90, 300, 8]], float)\n uv, _ = pose.project(truth, enu, W, H)\n return {\"image_size\": [W, H], \"init\": {\"at\": [lat0, lon0], \"height\": 10, \"heading\": 15, \"pitch\": 0, \"hfov\": 60},\n \"points\": [{\"name\": f\"P{i}\", \"ll\": [lat0 + n / ky, lon0 + e / kx], \"h\": u, \"px\": [float(x), float(y)]}\n for i, ((e, n, u), (x, y)) in enumerate(zip(enu, uv))]}\n def test_pose_solve_records_the_frame_of_its_points(self):\n spec = self._pose_spec()\n with contextlib.redirect_stdout(io.StringIO()):\n self.assertEqual(pose.solve(spec, 50, 2)[\"_frame\"][\"frame\"], \"stored\")\n self.assertEqual(pose.solve(dict(spec, frame=\"stored\"), 50, 2)[\"_frame\"][\"frame\"], \"stored\")\n self.assertEqual(pose.solve(dict(spec, frame=\"upright\"), 50, 2)[\"_frame\"][\"frame\"], \"upright\")\n\n def _solve_photo(self, spec, photo, name):\n source, saved, overlay = (self.d / f\"{name}{suffix}\" for suffix in (\"_spec.json\", \"_pose.json\", \".png\"))\n source.write_text(json.dumps(spec), encoding=\"utf-8\")\n self._run(\"pose.py\", \"solve\", source, \"--photo\", photo, \"--out\", overlay, \"--save\", saved,\n \"--search-radius\", 50, \"--restarts\", 2)\n with Image.open(overlay) as im:\n pixels = im.tobytes()\n return json.loads(saved.read_text(encoding=\"utf-8\")), pixels\n\n def test_legacy_pose_specs_solve_in_the_stored_frame(self):\n \"\"\"Re-running an old input spec must preserve its overlay, not just reading an old solved pose.json.\"\"\"\n for w, h, o in ((400, 300, 3), (400, 300, 6), (300, 300, 6)):\n with self.subTest(size=(w, h), orientation=o):\n scene = upright_control(w, h)\n stored = stored_as(scene, o)\n stored.save(self.d / \"plain.png\")\n save_oriented(scene, o, self.d / \"oriented.png\")\n spec = self._pose_spec(*stored.size) # legacy input: no frame key\n before, control = self._solve_photo(spec, self.d / \"plain.png\", \"legacy_control\")\n after, actual = self._solve_photo(spec, self.d / \"oriented.png\", \"legacy_oriented\")\n self.assertEqual(after[\"_frame\"][\"frame\"], \"stored\")\n self.assertEqual(after[\"_params\"], before[\"_params\"])\n self.assertEqual(actual, control)\n\n def test_new_pose_specs_solve_in_the_upright_frame(self):\n scene = upright_control(400, 300)\n scene.save(self.d / \"upright.png\")\n spec = dict(self._pose_spec(), frame=\"upright\")\n before, control = self._solve_photo(spec, self.d / \"upright.png\", \"upright_control\")\n for o in (3, 6):\n with self.subTest(orientation=o):\n save_oriented(scene, o, self.d / \"oriented.png\")\n after, actual = self._solve_photo(spec, self.d / \"oriented.png\", \"upright_oriented\")\n self.assertEqual(after[\"_frame\"][\"frame\"], \"upright\")\n self.assertEqual(after[\"_params\"], before[\"_params\"])\n self.assertEqual(actual, control)\n\n def test_terrain_ridge_points_and_overlay_use_the_upright_frame(self):\n w, h = 240, 160\n scene = Image.new(\"RGB\", (w, h), (210, 220, 235)) # sky\n ImageDraw.Draw(scene).polygon([(0, 70), (120, 40), (240, 90), (240, 160), (0, 160)], fill=(60, 70, 60))\n scene.save(self.d / \"up.png\")\n save_oriented(scene, 6, self.d / \"o6.png\")\n save_oriented(scene, 3, self.d / \"o3.png\")\n out = {}\n for name in (\"up\", \"o6\", \"o3\"):\n self._run(\"terrain.py\", \"ridge\", self.d / f\"{name}.png\", \"--x0\", 10, \"--x1\", 230, \"--step\", 20,\n \"--out\", self.d / f\"r_{name}.json\", \"--png\", self.d / f\"r_{name}.png\")\n out[name] = json.loads((self.d / f\"r_{name}.json\").read_text(encoding=\"utf-8\"))\n for name in (\"o6\", \"o3\"):\n self.assertEqual(out[name][\"image_size\"], [w, h])\n self.assertEqual(out[name][\"frame\"], \"upright\")\n self.assertEqual(out[name][\"ridge\"], out[\"up\"][\"ridge\"], name)\n self.assertEqual(Image.open(self.d / f\"r_{name}.png\").tobytes(), Image.open(self.d / \"r_up.png\").tobytes(), name)\n\n def test_exif_gives_the_upright_size_only_when_it_differs_in_frame(self):\n self.control.save(self.d / \"o1.jpg\", quality=95)\n save_oriented(self.control, 6, self.d / \"o6.jpg\", fmt=\"JPEG\", quality=95)\n save_oriented(self.control, 3, self.d / \"o3.jpg\", fmt=\"JPEG\", quality=95)\n out = {}\n for name in (\"o1\", \"o6\", \"o3\"):\n out[name] = json.loads(self._run(\"exif.py\", self.d / f\"{name}.jpg\").stdout.splitlines()[0])\n self.assertEqual(sorted(out[\"o1\"]), [\"file\", \"size\"]) # ordinary photos: output as before\n self.assertEqual((out[\"o6\"][\"size\"], out[\"o6\"][\"orientation\"], out[\"o6\"][\"upright_size\"]), ([40, 60], 6, [60, 40]))\n self.assertEqual((out[\"o3\"][\"size\"], out[\"o3\"][\"orientation\"], out[\"o3\"][\"upright_size\"]), ([60, 40], 3, [60, 40]))\n if _image.HEIF:\n save_oriented(self.control, 6, self.d / \"h6.heic\", fmt=\"HEIF\", quality=-1)\n self.assertNotIn(\"orientation\", json.loads(self._run(\"exif.py\", self.d / \"h6.heic\").stdout.splitlines()[0]))\n\n def _fit_sheet(self, photo, ridge, out):\n import terrain\n rec = {\"f\": 200.0, \"cam\": [30.0, 120.0], \"g\": 0.0, \"H\": 0.0, \"cc\": 0.0, \"hit\": 1, \"name\": \"x\",\n \"rms\": 0.1, \"flatpen\": 0.0, \"total\": 0.1}\n args = SimpleNamespace(sheet_width=120, sheet_cols=1, range=1000, zoom=12, cache=self.d, proxy=None,\n allow_missing=False, eye=1.6, near=40)\n with patch.object(terrain, \"DEM\", side_effect=SystemExit(\"offline\")), contextlib.redirect_stderr(io.StringIO()) as err:\n terrain._fit_sheet([rec], ridge, photo, out, args, np.arange(0, 360, 1.0), 8)\n with Image.open(out) as im:\n return im.tobytes(), err.getvalue()\n\n def test_fit_sheet_draws_old_ridge_records_on_the_stored_pixels(self):\n for w, h in ((240, 160), (160, 160)):\n scene = upright_control(w, h)\n scene.save(self.d / \"up.png\")\n for o in (3, 6):\n save_oriented(scene, o, self.d / f\"o{o}.png\")\n stored = stored_as(scene, o)\n stored.save(self.d / \"plain.png\")\n old = {\"image_size\": list(stored.size), \"hrow\": stored.height / 2, \"cx\": stored.width / 2, \"flat\": None,\n \"ridge\": [[10, 20], [stored.width - 30, 45]]} # ridge.json from before #29: no frame\n want, _ = self._fit_sheet(self.d / \"plain.png\", old, self.d / \"s_plain.jpg\")\n got, err = self._fit_sheet(self.d / f\"o{o}.png\", old, self.d / f\"s_o{o}.jpg\")\n self.assertEqual(got, want, f\"{w}x{h} orientation {o}\")\n self.assertIn(\"stored frame\", err)\n new = dict(old, image_size=[w, h], hrow=h / 2, cx=w / 2, ridge=[[10, 20], [w - 30, 45]], frame=\"upright\")\n want, _ = self._fit_sheet(self.d / \"up.png\", new, self.d / \"s_up.jpg\")\n got, err = self._fit_sheet(self.d / f\"o{o}.png\", new, self.d / f\"s2_o{o}.jpg\")\n self.assertEqual(got, want, f\"{w}x{h} orientation {o}, upright record\")\n self.assertNotIn(\"stored frame\", err)\n\n def test_spacing_warns_when_an_old_ridge_meets_upright_columns(self):\n import geo\n save_oriented(self.control, 6, self.d / \"o6.png\")\n self.control.save(self.d / \"up.png\")\n pts = [[5, 10], [20, 12], [35, 9], [50, 14]]\n cases = {\"old_o6\": ({\"photo\": \"o6.png\", \"ridge\": pts}, True), # photo path relative to the ridge file\n \"new_o6\": ({\"photo\": str(self.d / \"o6.png\"), \"frame\": \"upright\", \"ridge\": pts}, False),\n \"old_up\": ({\"photo\": str(self.d / \"up.png\"), \"ridge\": pts}, False),\n \"handwritten\": ({\"5\": 10, \"20\": 12, \"35\": 9, \"50\": 14}, False)}\n for name, (rec, warn) in cases.items():\n path = self.d / f\"{name}.json\"\n path.write_text(json.dumps(rec), encoding=\"utf-8\")\n with contextlib.redirect_stderr(io.StringIO()) as err:\n self.assertEqual(geo._spacing_ridge(str(path))[\"pts\"], [[float(x), float(y)] for x, y in pts])\n self.assertEqual(\"stored frame\" in err.getvalue(), warn, f\"{name}: {err.getvalue()}\")\n\n\nclass WorkingCopyTests(unittest.TestCase):\n \"\"\"Stage 3 of #29: a lossless upright PNG only for photos that need one, original EXIF kept, uploads compatible.\"\"\"\n\n def setUp(self):\n self.tmp = tempfile.TemporaryDirectory()\n self.d = Path(self.tmp.name)\n self.addCleanup(self.tmp.cleanup)\n self.control = upright_control()\n\n def _exif(self, o):\n ex = Image.Exif()\n ex[0x0112] = o\n ex[0x8769] = {0xA405: 26} # FocalLengthIn35mmFilm\n return ex\n\n def test_native_upright_photos_are_used_as_they_are(self):\n self.control.save(self.d / \"a.jpg\", exif=self._exif(1))\n self.control.save(self.d / \"b.png\")\n self.assertIsNone(_image.write_working_copy(self.d / \"a.jpg\", self.d / \"w\" / \"a.png\"))\n self.assertIsNone(_image.write_working_copy(self.d / \"b.png\", self.d / \"w\" / \"b.png\"))\n self.assertFalse((self.d / \"w\").exists())\n\n def test_oriented_jpeg_gets_an_upright_png_with_its_exif(self):\n stored_as(self.control, 6).save(self.d / \"o6.jpg\", exif=self._exif(6), quality=95)\n before = (self.d / \"o6.jpg\").read_bytes()\n info = _image.write_working_copy(self.d / \"o6.jpg\", self.d / \"w\" / \"o6.png\")\n self.assertEqual((self.d / \"o6.jpg\").read_bytes(), before) # original untouched\n self.assertEqual(info[\"reason\"], \"EXIF orientation 6 applied\")\n self.assertEqual((info[\"original_size\"], info[\"working_size\"]), ([40, 60], [60, 40]))\n with Image.open(info[\"working\"]) as wc:\n self.assertEqual(wc.format, \"PNG\")\n self.assertEqual(wc.tobytes(), _image.open_photo(self.d / \"o6.jpg\").convert(\"RGB\").tobytes()) # lossless\n self.assertEqual(wc.getexif().get(0x0112), 1)\n self.assertEqual(wc.getexif().get_ifd(0x8769).get(0xA405), 26)\n self.assertIsNone(_image.working_copy_reason(wc)) # a copy never needs another\n\n def test_webp_and_heic_get_a_copy(self):\n self.control.save(self.d / \"w.webp\", exif=self._exif(1))\n self.assertEqual(_image.write_working_copy(self.d / \"w.webp\", self.d / \"w\" / \"w.png\")[\"reason\"], \"WEBP decoded to PNG\")\n if _image.HEIF:\n self.control.save(self.d / \"h.heic\", format=\"HEIF\", exif=self._exif(1))\n info = _image.write_working_copy(self.d / \"h.heic\", self.d / \"w\" / \"h.png\")\n self.assertEqual(info[\"reason\"], \"HEIC/HEIF decoded to PNG\")\n with Image.open(info[\"working\"]) as wc:\n self.assertEqual(wc.getexif().get_ifd(0x8769).get(0xA405), 26)\n\n def test_unreadable_photo_raises(self):\n (self.d / \"IMG.CR2\").write_bytes(b\"II*\\x00\\x10\\x00\\x00\\x00CR\\x02\\x00\" + b\"\\x00\" * 64)\n with self.assertRaises(_image.ImageReadError):\n _image.write_working_copy(self.d / \"IMG.CR2\", self.d / \"w\" / \"IMG.png\")\n\n def test_revimg_uploads_a_compatible_copy_only_when_needed(self):\n import revimg\n self.control.save(self.d / \"plain.jpg\")\n stored_as(self.control, 8).save(self.d / \"side.jpg\", exif=self._exif(8))\n self.assertEqual(revimg._upload_path(self.d / \"plain.jpg\", self.d / \"rev\"), self.d / \"plain.jpg\")\n up = revimg._upload_path(self.d / \"side.jpg\", self.d / \"rev\")\n self.assertEqual(up, self.d / \"rev\" / \"upload\" / \"side.png\")\n with Image.open(up) as im:\n self.assertEqual(im.size, (60, 40))\n\n\nclass RevimgInputTests(unittest.TestCase):\n \"\"\"revimg.py: every input gets its own upload copy, and an input the shared loader rejects is reported, never uploaded.\n The engines are fakes that note which file they were handed and what it shows at the moment of the call.\"\"\"\n\n COLORS = {\"red\": (220, 30, 30), \"green\": (30, 160, 30), \"blue\": (30, 30, 220), \"gray\": (128, 128, 128)}\n\n def setUp(self):\n self.tmp = tempfile.TemporaryDirectory()\n self.d = Path(self.tmp.name)\n self.addCleanup(self.tmp.cleanup)\n self.out = self.d / \"rev\"\n\n def _solid(self, color, path, **kw):\n path.parent.mkdir(parents=True, exist_ok=True)\n Image.new(\"RGB\", (48, 32), self.COLORS[color]).save(path, **kw)\n return path\n\n def _color_of(self, path):\n with Image.open(path) as im:\n px = im.convert(\"RGB\").getpixel((24, 16))\n return min(self.COLORS, key=lambda c: sum((a - b) ** 2 for a, b in zip(self.COLORS[c], px)))\n\n def _unreadable(self):\n raw = self.d / \"IMG_1.CR2\"\n raw.write_bytes(b\"II*\\x00\\x10\\x00\\x00\\x00CR\\x02\\x00\" + b\"\\x00\" * 64)\n page = self.d / \"tile.jpg\" # an error reply saved under an image name\n page.write_bytes(b'{\"status\":429,\"msg\":\"rate limited\"}')\n return raw, page\n\n def _search(self, call):\n sent = {\"baidu\": [], \"yandex\": []}\n\n async def baidu(ctx, up, shot, *rest):\n sent[\"baidu\"].append((Path(up), self._color_of(up)))\n return {\"guess\": [], \"links\": []}\n\n async def yandex(ctx, up, shot):\n sent[\"yandex\"].append((Path(up), self._color_of(up)))\n return {\"guess\": [], \"links\": []}\n\n browser = SimpleNamespace(new_context=AsyncMock(), close=AsyncMock())\n manager = AsyncMock()\n manager.__aenter__.return_value = object()\n fake = SimpleNamespace(async_playwright=lambda: manager)\n with patch.dict(sys.modules, {\"playwright.async_api\": fake}), \\\n patch.object(revimg, \"launch_browser\", AsyncMock(return_value=(browser, \"test\"))) as launch, \\\n patch.object(revimg, \"_baidu\", new=baidu), patch.object(revimg, \"_yandex\", new=yandex):\n got = call()\n return got, sent, launch\n\n def _main(self, *argv):\n err, code = io.StringIO(), 0\n with patch.object(sys, \"argv\", [\"revimg.py\", *map(str, argv)]), \\\n contextlib.redirect_stdout(io.StringIO()), contextlib.redirect_stderr(err):\n try:\n revimg.main()\n except SystemExit as e:\n code = e.code\n return code, err.getvalue()\n\n def test_names_are_unique_without_regard_to_case(self):\n self.assertEqual(revimg._names([Path(\"photo.jpg\"), Path(\"v/left_crop.jpg\")]), [\"photo\", \"left_crop\"])\n self.assertEqual(revimg._names([Path(\"a/photo.jpg\"), Path(\"b/PHOTO.png\"), Path(\"c/02_photo.jpg\"), Path(\"x.jpg\")]),\n [\"01_photo\", \"02_PHOTO\", \"03_02_photo\", \"x\"])\n\n def test_same_stems_upload_their_own_pixels(self):\n images = {self._solid(\"red\", self.d / \"a\" / \"photo.webp\", lossless=True): \"red\",\n self._solid(\"green\", self.d / \"b\" / \"Photo.webp\", lossless=True): \"green\", # same stem but for case\n self._solid(\"gray\", self.d / \"c\" / \"photo.jpg\", quality=95): \"gray\"} # needs no copy\n if _image.HEIF:\n images[self._solid(\"blue\", self.d / \"d\" / \"photo.heic\", format=\"HEIF\", quality=-1)] = \"blue\"\n results, sent, _ = self._search(lambda: asyncio.run(revimg.run(list(images), [\"baidu\", \"yandex\"], self.out, \"direct\")))\n for eng, calls in sent.items():\n self.assertEqual([c for _, c in calls], list(images.values()), eng)\n self.assertEqual(len({p for p, _ in calls}), len(images), f\"{eng}: two inputs shared one upload file\")\n self.assertEqual(sent[\"baidu\"][2][0], self.d / \"c\" / \"photo.jpg\")\n records = [json.loads(f.read_text(encoding=\"utf-8\")) for f in sorted(self.out.glob(\"*.json\"))]\n self.assertEqual(len(records), 2 * len(images))\n for r in results + records: # each record names the original whose pixels went out\n self.assertEqual(self._color_of(r.get(\"uploaded\", r[\"image\"])), images[Path(r[\"image\"])], r)\n\n def test_unreadable_inputs_are_reported_and_not_uploaded(self):\n raw, page = self._unreadable()\n good = self._solid(\"red\", self.d / \"good.webp\", lossless=True)\n results, sent, _ = self._search(lambda: asyncio.run(revimg.run([raw, page, good], [\"baidu\", \"yandex\"], self.out, \"direct\")))\n for eng, calls in sent.items():\n self.assertEqual([c for _, c in calls], [\"red\"], eng)\n bad = {(Path(r[\"image\"]).name, r[\"engine\"]): r[\"unreadable\"] for r in results if r.get(\"unreadable\")}\n self.assertEqual(bad, {(\"IMG_1.CR2\", \"baidu\"): \"raw\", (\"IMG_1.CR2\", \"yandex\"): \"raw\",\n (\"tile.jpg\", \"baidu\"): \"not-image\", (\"tile.jpg\", \"yandex\"): \"not-image\"})\n rec = json.loads((self.out / \"IMG_1_baidu.json\").read_text(encoding=\"utf-8\"))\n self.assertIn(\"export it as JPEG or PNG\", rec[\"error\"])\n self.assertNotIn(\"screenshot\", rec)\n self.assertEqual([f.name for f in (self.out / \"upload\").iterdir()], [\"good.png\"])\n\n def test_cli_exits_2_naming_each_unreadable_input(self):\n raw, page = self._unreadable()\n good = self._solid(\"red\", self.d / \"good.jpg\")\n (code, err), sent, _ = self._search(lambda: self._main(raw, page, good, \"--out-dir\", self.out))\n self.assertEqual(code, 2)\n self.assertIn(\"2 of 3 images could not be read and were not uploaded\", err)\n self.assertIn(f\"{raw} (camera RAW\", err)\n self.assertIn(f\"{page} (text, not an image\", err)\n self.assertEqual([p for p, _ in sent[\"baidu\"]], [good])\n\n def test_cli_with_only_unreadable_inputs_starts_no_browser(self):\n raw, page = self._unreadable()\n (code, err), sent, launch = self._search(lambda: self._main(raw, page, \"--out-dir\", self.out))\n self.assertEqual(code, 2)\n self.assertEqual(launch.await_count, 0)\n self.assertEqual(sent, {\"baidu\": [], \"yandex\": []})\n self.assertEqual(sorted(f.name for f in self.out.iterdir()),\n [\"IMG_1_baidu.json\", \"IMG_1_yandex.json\", \"tile_baidu.json\", \"tile_yandex.json\"])\n r = subprocess.run([sys.executable, revimg.__file__, str(raw), \"--out-dir\", str(self.d / \"rev2\")],\n capture_output=True, text=True, encoding=\"utf-8\")\n self.assertEqual(r.returncode, 2, r.stderr)\n self.assertIn(\"1 of 1 images could not be read\", r.stderr)\n self.assertNotIn(\"Traceback\", r.stderr)\n\n\nclass IntakeStatusTests(unittest.TestCase):\n \"\"\"intake.py as a whole: working-copy routing, partial results, exit status. Child scripts run for real, with this Python.\"\"\"\n\n def setUp(self):\n self.tmp = tempfile.TemporaryDirectory()\n self.d = Path(self.tmp.name)\n self.addCleanup(self.tmp.cleanup)\n\n def _intake(self, photo, *extra, fail=()):\n import intake\n original = intake._run\n\n def run(cmd, **kwargs):\n name = Path(str(cmd[2])).name\n if name in fail:\n return 1, \"\", f\"{name}: simulated failure\"\n return original([sys.executable, *cmd[2:]], **kwargs)\n\n out = self.d / \"out\"\n code = 0\n with patch.object(intake, \"_run\", side_effect=run), \\\n patch.object(sys, \"argv\", [\"intake.py\", str(photo), \"--out-dir\", str(out), \"--no-rev\", *extra]), \\\n contextlib.redirect_stdout(io.StringIO()):\n try:\n intake.main()\n except SystemExit as e:\n code = e.code if isinstance(e.code, int) else 1\n return code, out, json.loads((out / \"intake.json\").read_text(encoding=\"utf-8\")), (out / \"intake.md\").read_text(encoding=\"utf-8\")\n\n def test_ordinary_photo_uses_the_original(self):\n upright_control(120, 80).save(self.d / \"photo.jpg\")\n code, out, js, md = self._intake(self.d / \"photo.jpg\", \"--no-ocr\")\n self.assertEqual((code, js[\"result\"], js[\"input\"]), (0, \"ok\", None))\n self.assertFalse((out / \"work\").exists())\n self.assertTrue((out / \"edges\" / \"photo_top.jpg\").exists())\n self.assertNotIn(\"Result:\", md)\n\n def test_oriented_photo_is_routed_through_an_upright_copy(self):\n upright_control(120, 80).save(self.d / \"photo.png\")\n _, out, _, _ = self._intake(self.d / \"photo.png\", \"--no-ocr\")\n control = {f.name: f.read_bytes() for f in (out / \"edges\").glob(\"*.jpg\")}\n (self.d / \"out\").rename(self.d / \"out_control\")\n save_oriented(upright_control(120, 80), 6, self.d / \"photo.png\")\n code, out, js, md = self._intake(self.d / \"photo.png\", \"--no-ocr\")\n self.assertEqual(code, 0)\n self.assertEqual(js[\"input\"][\"reason\"], \"EXIF orientation 6 applied\")\n self.assertTrue((out / \"work\" / \"photo.png\").exists())\n got = {f.name: f.read_bytes() for f in (out / \"edges\").glob(\"*.jpg\")}\n self.assertEqual(len(control), 8)\n self.assertEqual(got, control) # same names, same upright crops as the upright control\n self.assertIn(\"keep passing the original photo\", md)\n\n @unittest.skipUnless(_image.HEIF, \"pillow-heif not installed\")\n def test_heic_photo_runs_every_step_with_metadata_from_the_original(self):\n ex = Image.Exif()\n ex[0x8769] = {0xA405: 24}\n upright_control(120, 80).save(self.d / \"IMG_1.HEIC\", format=\"HEIF\", exif=ex)\n code, out, js, _ = self._intake(self.d / \"IMG_1.HEIC\", \"--no-ocr\")\n self.assertEqual((code, js[\"result\"]), (0, \"ok\"))\n self.assertEqual(js[\"input\"][\"reason\"], \"HEIC/HEIF decoded to PNG\")\n self.assertEqual(len(list((out / \"edges\").glob(\"IMG_1_*.jpg\"))), 8)\n self.assertEqual(js[\"exif\"][\"file\"], str((self.d / \"IMG_1.HEIC\").resolve()))\n\n def test_partial_failure_is_reported_as_partial(self):\n upright_control(120, 80).save(self.d / \"photo.jpg\")\n code, _, js, md = self._intake(self.d / \"photo.jpg\", fail=(\"ocr.py\",))\n self.assertEqual((code, js[\"result\"]), (0, \"partial\"))\n self.assertIn(\"**Result: partial.** Failed: ocr\", md)\n\n def test_all_visual_steps_failing_exits_nonzero_and_keeps_the_report(self):\n upright_control(120, 80).save(self.d / \"photo.jpg\")\n code, _, js, md = self._intake(self.d / \"photo.jpg\", fail=(\"imgprep.py\", \"ocr.py\"))\n self.assertEqual((code, js[\"result\"]), (1, \"failed\"))\n self.assertIn(\"**Result: failed.**\", md)\n self.assertEqual(js[\"status\"][\"exif\"], \"ok\")\n\n def test_disabled_step_is_not_a_failure(self):\n upright_control(120, 80).save(self.d / \"photo.jpg\")\n code, _, js, _ = self._intake(self.d / \"photo.jpg\", \"--no-ocr\", fail=(\"ocr.py\",))\n self.assertEqual((code, js[\"result\"]), (0, \"ok\"))\n self.assertNotIn(\"ocr\", js[\"status\"])\n\n def test_unreadable_photo_exits_2_with_a_report(self):\n (self.d / \"IMG_2.CR2\").write_bytes(b\"II*\\x00\\x10\\x00\\x00\\x00CR\\x02\\x00\" + b\"\\x00\" * 64)\n code, out, js, md = self._intake(self.d / \"IMG_2.CR2\")\n self.assertEqual(code, 2)\n self.assertEqual((js[\"result\"], js[\"input\"][\"error\"]), (\"input unreadable\", \"raw\"))\n self.assertIn(\"export it as JPEG or PNG\", md)\n self.assertFalse(any((out / \"edges\").glob(\"*\")))\n\n\nif __name__ == \"__main__\":\n unittest.main()\n","content_type":"text/x-python; charset=utf-8","language":"python","size":79952,"content_sha256":"c47d61fe6659d640bf1b37b4a5888b6716e94621d528e9b56713ba461bddadfe"},{"filename":"tests/test_sun.py","content":"#!/usr/bin/env python3\n# /// script\n# requires-python = \">=3.10\"\n# dependencies = [\"tzdata\"]\n# ///\n\"\"\"Offline sun.py regressions. Run: uv run skills/geo-sleuth/tests/test_sun.py\"\"\"\nfrom __future__ import annotations\n\nimport argparse\nimport contextlib\nimport io\nfrom pathlib import Path\nimport re\nimport sys\nimport unittest\n\nsys.path.insert(0, str(Path(__file__).resolve().parents[1] / \"scripts\"))\nimport sun\n\n\ndef _compass(**kw) -> str:\n a = dict(at=(64.84, -147.72), tz=\"America/Anchorage\", dates=\"2024-12-10:2024-12-20\", time=None, step_days=3,\n elev=None, elev_max=12.0, sun_x=2000.0, width=4000.0, hfov=\"60:60\", x=None)\n a.update(kw)\n buf = io.StringIO()\n with contextlib.redirect_stdout(buf):\n sun.cmd_compass(argparse.Namespace(**a))\n return buf.getvalue()\n\n\ndef _group(out: str, name: str) -> tuple[float, float]:\n m = re.search(name + r\"[^:]*: sun azimuth ([\\d.]+)°–([\\d.]+)°\", out)\n return float(m.group(1)), float(m.group(2))\n\n\nclass CompassGroups(unittest.TestCase):\n def test_groups_split_at_solar_noon_not_clock_noon(self):\n # Fairbanks in December: solar noon is about 12:45 local, so the sun is still east of south after 12:00\n out = _compass()\n self.assertLessEqual(_group(out, \"morning\")[1], 180)\n self.assertGreaterEqual(_group(out, \"evening\")[0], 180)\n\n def test_far_west_of_time_zone_meridian(self):\n # Kashgar on Beijing time: a late-morning sun at 12:30 is still south-east\n out = _compass(at=(39.47, 75.99), tz=\"Asia/Shanghai\", elev=\"5:20\")\n self.assertLessEqual(_group(out, \"morning\")[1], 180)\n self.assertGreaterEqual(_group(out, \"evening\")[0], 180)\n\n\nclass StreetHalves(unittest.TestCase):\n def test_morning_label_follows_the_sun_side(self):\n a = argparse.Namespace(at=(39.47, 75.99), tz=\"Asia/Shanghai\", date=\"2024-06-21\", shadow_rel=90.0, both=False,\n elev=None, ratio=None, tol=3.0, step_min=10)\n buf = io.StringIO()\n with contextlib.redirect_stdout(buf):\n sun.cmd_street(a)\n rows = re.findall(r\"^\\d\\d:\\d\\d (morning|afternoon) sun\\s+([\\d.]+)°\", buf.getvalue(), re.M)\n self.assertTrue(rows)\n for half, az in rows:\n self.assertEqual(half, \"morning\" if float(az) \u003c 180 else \"afternoon\", (half, az))\n\n\nif __name__ == \"__main__\":\n unittest.main()\n","content_type":"text/x-python; charset=utf-8","language":"python","size":2379,"content_sha256":"e563d883e331f1d9b6416c79e22710c2e4473b3b2689ca949df0bf70ccfc20b3"},{"filename":"tests/test_terrain.py","content":"#!/usr/bin/env python3\n# /// script\n# requires-python = \">=3.10\"\n# dependencies = [\"pillow\", \"numpy\"]\n# ///\n\"\"\"Offline terrain.py fit regressions on synthetic terrain. Run: uv run skills/geo-sleuth/tests/test_terrain.py\"\"\"\nfrom __future__ import annotations\n\nimport contextlib\nimport io\nimport json\nimport math\nfrom pathlib import Path\nimport sys\nimport tempfile\nimport unittest\nfrom unittest.mock import patch\n\nimport numpy as np\n\nsys.path.insert(0, str(Path(__file__).resolve().parents[1] / \"scripts\"))\nimport terrain\n\nLAT0, LON0 = 38.0, 106.0\nW, H, HROW = 2000, 1125, 600.0\n\n\nclass FakeDEM:\n \"\"\"Flat plain; a low near range 6 km west that ends 1 km north of the camera; a high far range 20 km west.\"\"\"\n\n def __init__(self, *a, **k):\n pass\n\n def sample(self, lat, lon):\n y = (np.asarray(lat) - LAT0) * 110540\n x = (np.asarray(lon) - LON0) * 111320 * math.cos(math.radians(LAT0))\n crest = 1500 + 500 * np.sin(y / 2500) + 300 * np.cos(y / 1300)\n far = crest * np.exp(-((x + 20000) / 2500) ** 2)\n near = (350 + 120 * np.sin(y / 700)) * np.exp(-((x + 6000) / 400) ** 2) * (y \u003c 1000) * (y > -4000)\n return np.maximum(far, near)\n\n\ndef synth_ridge(path: Path, f35: float, heading: float, two_layer: bool) -> None:\n \"\"\"Project the fake terrain through a pinhole at f35 / heading, write ridge.json with an assumed 26 mm focal.\"\"\"\n f = f35 / 43.2666 * math.hypot(W, H)\n xs = np.arange(40, W - 40, 80, dtype=float)\n az = (heading + np.degrees(np.arctan((xs - W / 2) / f))) % 360\n ang, dist = terrain.cast(FakeDEM(), (LAT0, LON0), 1.6, az, 30000, near=150, n=600)\n row = lambda a: HROW - f * np.tan(np.radians(a))\n far = row(ang.max(axis=1))\n near_ang = ang[:, dist \u003c= 9000].max(axis=1)\n near = row(near_ang)\n ridge = {\"photo\": \"\", \"image_size\": [W, H], \"cx\": W / 2, \"ridge\": [[int(x), int(round(y))] for x, y in zip(xs, far)],\n \"flat\": None, \"hrow\": HROW, \"f0\": 26 / 43.2666 * math.hypot(W, H), \"f0_source\": \"assumed\", \"f35_equiv\": 26.0,\n \"near\": [], \"near_clear\": []}\n if two_layer:\n stand = near_ang > 0.3 # columns where the near range stands up\n ridge[\"near\"] = [[int(x), int(round(y))] for x, y, s in zip(xs, near, stand) if s]\n clear = xs[~stand & (xs > W / 2)]\n ridge[\"near_clear\"] = [[int(clear.min()), int(clear.max()), float(row(0.25))]]\n path.write_text(json.dumps(ridge))\n\n\ndef fit(ridge: Path, out: Path, *extra: str, dem=None) -> tuple[dict, str]:\n argv = [\"terrain.py\", \"fit\", \"--at\", f\"{LAT0},{LON0}\", \"--radius\", \"0\", \"--ridge\", str(ridge), \"--out\", str(out),\n \"--range\", \"30000\", \"--min-peak\", \"2\", \"--nsamp\", \"400\", *extra]\n err = io.StringIO()\n with patch.object(terrain, \"DEM\", dem or FakeDEM), patch.object(sys, \"argv\", argv), contextlib.redirect_stderr(err):\n terrain.main()\n return json.loads(out.read_text(encoding=\"utf-8\")), err.getvalue()\n\n\nclass FocalTests(unittest.TestCase):\n def setUp(self):\n self.tmp = tempfile.TemporaryDirectory()\n self.d = Path(self.tmp.name)\n\n def tearDown(self):\n self.tmp.cleanup()\n\n def test_assumed_focal_searches_wide_and_finds_telephoto(self):\n synth_ridge(self.d / \"r.json\", 60, 270, two_layer=False)\n res, _ = fit(self.d / \"r.json\", self.d / \"f.json\")\n best = res[\"cams\"][0]\n self.assertGreater(len(res[\"params\"][\"f35_ladder\"]), 10)\n self.assertLess(abs(math.log(best[\"f35\"] / 60)), math.log(1.2))\n self.assertLess(abs((best[\"H\"] - 270 + 180) % 360 - 180), 1.0)\n self.assertIsNone(res[\"focal_edge\"])\n\n def test_main_camera_only_hits_long_edge_and_warns(self):\n synth_ridge(self.d / \"r.json\", 60, 270, two_layer=False)\n res, err = fit(self.d / \"r.json\", self.d / \"f.json\", \"--focal-scales\", \"0.9,1,1.12\")\n self.assertEqual(res[\"focal_edge\"][\"top1\"], \"long\")\n self.assertIn(\"--f35-range\", err)\n\n def test_widened_range_clears_the_edge(self):\n synth_ridge(self.d / \"r.json\", 60, 270, two_layer=False)\n res, _ = fit(self.d / \"r.json\", self.d / \"f.json\", \"--f35-range\", \"20:40\")\n self.assertEqual(res[\"focal_edge\"][\"top1\"], \"long\")\n res, _ = fit(self.d / \"r.json\", self.d / \"f.json\", \"--f35-range\", \"24:80\")\n self.assertIsNone(res[\"focal_edge\"])\n\n def test_short_edge_also_suggests_a_longer_ladder(self):\n synth_ridge(self.d / \"r.json\", 13, 270, two_layer=False)\n res, err = fit(self.d / \"r.json\", self.d / \"f.json\", \"--f35-range\", \"16:40\")\n self.assertEqual(res[\"focal_edge\"][\"top1\"], \"short\")\n self.assertIn(\"--f35-range 8:27\", err)\n self.assertIn(\"--f35-range 24:40\", err)\n\n def test_short_edge_hint_on_a_narrow_ladder_stays_valid(self):\n synth_ridge(self.d / \"r.json\", 13, 270, two_layer=False)\n res, err = fit(self.d / \"r.json\", self.d / \"f.json\", \"--f35-range\", \"16:18\")\n self.assertEqual(res[\"focal_edge\"][\"top1\"], \"short\")\n self.assertIn(\"--f35-range 24:48\", err)\n self.assertNotIn(\"--f35-range 24:18\", err)\n\n def test_two_layer_scores_near_points(self):\n synth_ridge(self.d / \"r.json\", 60, 270, two_layer=True)\n ridge = json.loads((self.d / \"r.json\").read_text(encoding=\"utf-8\"))\n self.assertGreaterEqual(len(ridge[\"near\"]), 3)\n res, _ = fit(self.d / \"r.json\", self.d / \"f.json\")\n best = res[\"cams\"][0]\n self.assertIn(\"D1\", best)\n self.assertLess(best[\"rms_near_px\"], 6)\n self.assertLess(abs((best[\"H\"] - 270 + 180) % 360 - 180), 1.0)\n\n def test_perp_line_restricts_heading(self):\n synth_ridge(self.d / \"r.json\", 60, 270, two_layer=False)\n line = {\"type\": \"FeatureCollection\", \"features\": [{\"type\": \"Feature\", \"properties\": {},\n \"geometry\": {\"type\": \"LineString\", \"coordinates\": [[LON0 + 0.003, LAT0 - 0.02], [LON0 + 0.003, LAT0 + 0.02]]}}]}\n (self.d / \"l.geojson\").write_text(json.dumps(line))\n res, _ = fit(self.d / \"r.json\", self.d / \"f.json\", \"--perp-line\", str(self.d / \"l.geojson\"), \"--perp-tol\", \"10\")\n self.assertLess(abs((res[\"cams\"][0][\"H\"] - 270 + 180) % 360 - 180), 10.5)\n self.assertEqual(res[\"cams\"][0][\"line_brg\"], 0.0)\n\n def _lines(self, parts):\n p = self.d / \"l.geojson\"\n p.write_text(json.dumps({\"type\": \"FeatureCollection\", \"features\": [\n {\"type\": \"Feature\", \"properties\": {}, \"geometry\": {\"type\": \"LineString\", \"coordinates\": c}} for c in parts]}))\n return str(p)\n\n def test_perp_line_distance_ignores_vertex_density(self):\n synth_ridge(self.d / \"r.json\", 60, 270, two_layer=False)\n lon = LON0 + 0.0005\n a, _ = fit(self.d / \"r.json\", self.d / \"a.json\", \"--perp-line\", self._lines([[[lon, LAT0 - 0.002], [lon, LAT0 + 0.04]]]))\n b, _ = fit(self.d / \"r.json\", self.d / \"b.json\", \"--perp-line\", self._lines([[[lon, LAT0 - 0.002], [lon, LAT0], [lon, LAT0 + 0.04]]]))\n self.assertEqual(a[\"n_cams\"], 1)\n self.assertEqual(b[\"n_cams\"], 1)\n\n def test_perp_line_picks_nearest_segment_not_nearest_midpoint(self):\n synth_ridge(self.d / \"r.json\", 60, 270, two_layer=False)\n lon = LON0 + 0.0005\n p = self._lines([[[lon, LAT0 - 0.0001], [lon, LAT0 + 0.005]], [[LON0 - 0.002, LAT0 + 0.001], [LON0 + 0.002, LAT0 + 0.001]]])\n res, _ = fit(self.d / \"r.json\", self.d / \"f.json\", \"--perp-line\", p, \"--perp-tol\", \"10\")\n self.assertEqual(res[\"cams\"][0][\"line_brg\"], 0.0)\n\n def test_near_dist_within_sight_line_start_is_dropped_not_crash(self):\n synth_ridge(self.d / \"r.json\", 60, 270, two_layer=True)\n res, err = fit(self.d / \"r.json\", self.d / \"f.json\", \"--near\", \"4000\")\n self.assertEqual(res[\"n_cams\"], 1)\n self.assertIn(\"dropped\", err)\n with self.assertRaises(SystemExit):\n fit(self.d / \"r.json\", self.d / \"f.json\", \"--near\", \"4000\", \"--near-dist\", \"3000\")\n\n def test_every_rung_recovers_heading(self):\n for f35 in (13, 20, 26, 40, 90, 135):\n with self.subTest(f35=f35):\n synth_ridge(self.d / \"r.json\", f35, 270, two_layer=False)\n res, _ = fit(self.d / \"r.json\", self.d / \"f.json\")\n self.assertLess(abs((res[\"cams\"][0][\"H\"] - 270 + 180) % 360 - 180), 2)\n\n\n\nclass SlopeDEM(FakeDEM):\n \"\"\"FakeDEM on a hillside: about 90 m of relief within 300 m of the camera, so no point is flat by the default --cam-flat.\"\"\"\n\n def sample(self, lat, lon):\n x = (np.asarray(lon) - LON0) * 111320 * math.cos(math.radians(LAT0))\n return super().sample(lat, lon) + 60 * np.tanh(x / 300)\n\n\nclass BumpDEM(FakeDEM):\n \"\"\"FakeDEM with a 50 m knoll 450 m east of the camera: only the grid points whose flat-test rings reach it fail --cam-flat.\"\"\"\n\n def sample(self, lat, lon):\n y = (np.asarray(lat) - LAT0) * 110540\n x = (np.asarray(lon) - LON0) * 111320 * math.cos(math.radians(LAT0))\n return super().sample(lat, lon) + 50 * np.exp(-((x - 450) ** 2 + y ** 2) / 20 ** 2)\n\n\nclass FlatFilterTests(unittest.TestCase):\n def setUp(self):\n self.tmp = tempfile.TemporaryDirectory()\n self.d = Path(self.tmp.name)\n synth_ridge(self.d / \"r.json\", 40, 270, two_layer=False)\n\n def tearDown(self):\n self.tmp.cleanup()\n\n def _fit(self, *extra):\n return fit(self.d / \"r.json\", self.d / \"f.json\", *extra, dem=SlopeDEM)\n\n def test_cluster_without_flat_point_scores_its_flattest(self):\n res, err = self._fit(\"--radius\", \"300\", \"--grid\", \"150\")\n self.assertGreater(res[\"n_cams\"], 0)\n self.assertEqual(res[\"flat_relaxed_clusters\"], [0])\n self.assertTrue(all(\"flat_relaxed\" in c for c in res[\"cams\"]))\n self.assertIn(\"flat_relaxed\", err)\n\n def test_flat_cluster_is_unchanged(self):\n res, _ = fit(self.d / \"r.json\", self.d / \"f.json\", \"--radius\", \"300\", \"--grid\", \"150\")\n self.assertEqual(res[\"flat_relaxed_clusters\"], [])\n self.assertFalse(any(\"flat_relaxed\" in c for c in res[\"cams\"]))\n\n def test_flat_warning_counts_positions_dropped_later(self):\n # every grid point is then dropped by --min-peak; the warning must compare against all points that reached the flat test\n res, err = fit(self.d / \"r.json\", self.d / \"f.json\", \"--radius\", \"300\", \"--grid\", \"150\", \"--min-peak\", \"89\", dem=BumpDEM)\n self.assertEqual(res[\"n_cams\"], 0)\n self.assertGreater(res[\"n_skipped\"][\"not_flat\"], 0)\n self.assertGreater(res[\"n_skipped\"][\"no_peak\"], res[\"n_skipped\"][\"not_flat\"])\n self.assertNotIn(\"larger --cam-flat\", err)\n\n def test_single_point_keeps_filter_and_warns(self):\n res, err = self._fit()\n self.assertEqual(res[\"n_cams\"], 0)\n self.assertEqual(res[\"n_skipped\"][\"not_flat\"], 1)\n self.assertIn(\"larger --cam-flat\", err)\n\n\nclass ClusterCoverageTests(unittest.TestCase):\n def test_cluster_records_farthest_member(self):\n hits = [{\"lat\": LAT0, \"lon\": LON0, \"max_ang\": 9.0}, {\"lat\": LAT0 + 0.02, \"lon\": LON0, \"max_ang\": 5.0},\n {\"lat\": LAT0 + 0.2, \"lon\": LON0, \"max_ang\": 4.0}]\n cl = terrain._cluster(hits, 3.5)\n self.assertEqual([c[\"n\"] for c in cl], [2, 1])\n self.assertAlmostEqual(cl[0][\"extent_m\"], 2224, delta=2)\n self.assertEqual(cl[1][\"extent_m\"], 0)\n\n def test_fit_warns_when_cluster_is_wider_than_grid(self):\n with tempfile.TemporaryDirectory() as t:\n d = Path(t)\n synth_ridge(d / \"r.json\", 40, 270, two_layer=False)\n (d / \"h.json\").write_text(json.dumps([{\"lat\": LAT0, \"lon\": LON0, \"extent_m\": 5000}]))\n argv = [\"terrain.py\", \"fit\", \"--hits\", str(d / \"h.json\"), \"--radius\", \"0\", \"--ridge\", str(d / \"r.json\"), \"--out\", str(d / \"f.json\"),\n \"--range\", \"30000\", \"--min-peak\", \"2\"]\n err = io.StringIO()\n with patch.object(terrain, \"DEM\", FakeDEM), patch.object(sys, \"argv\", argv), contextlib.redirect_stderr(err):\n terrain.main()\n self.assertIn(\"spread wider than --radius\", err.getvalue())\n self.assertEqual(json.loads((d / \"f.json\").read_text(encoding=\"utf-8\"))[\"n_cams\"], 1)\n\n\nclass RidgeTests(unittest.TestCase):\n def test_points_and_near_layer_written(self):\n from PIL import Image\n with tempfile.TemporaryDirectory() as t:\n p = Path(t) / \"p.jpg\"\n Image.new(\"RGB\", (400, 300), (120, 160, 200)).save(p)\n out = Path(t) / \"r.json\"\n with patch.object(sys, \"argv\", [\"terrain.py\", \"ridge\", str(p), \"--points\", \"10,100;200,90;390,110\",\n \"--near\", \"10,140;100,150;180,160\", \"--near-clear\", \"250:390@170\", \"--out\", str(out)]), \\\n contextlib.redirect_stdout(io.StringIO()):\n terrain.main()\n r = json.loads(out.read_text(encoding=\"utf-8\"))\n self.assertEqual(r[\"ridge_source\"], \"points\")\n self.assertEqual(r[\"ridge\"], [[10, 100], [200, 90], [390, 110]])\n self.assertEqual(len(r[\"near\"]), 3)\n self.assertEqual(r[\"near_clear\"], [[250, 390, 170.0]])\n\n\nif __name__ == \"__main__\":\n unittest.main()\n","content_type":"text/x-python; charset=utf-8","language":"python","size":13185,"content_sha256":"95c0f259aa03e3aa3a63a3cfb000354ff5ccf68b9f9dc2aa51bd6f40fba6b724"}],"content_json":{"type":"doc","content":[{"type":"heading","attrs":{"level":1},"content":[{"text":"geo-sleuth (v2)","type":"text"}]},{"type":"paragraph","content":[{"text":"Goal: find a location that ","type":"text"},{"text":"holds up under checking","type":"text","marks":[{"type":"strong"}]},{"text":". Every conclusion must point to the clue it used, the command it ran and the file it produced.","type":"text"}]},{"type":"blockquote","content":[{"type":"paragraph","content":[{"text":"${CLAUDE_SKILL_DIR}","type":"text","marks":[{"type":"code_inline"}]},{"text":" in commands means the directory containing this SKILL.md (the same for commands in references). Claude Code substitutes it automatically; in other agents, first ","type":"text"},{"text":"export CLAUDE_SKILL_DIR=\u003cabsolute path of this directory>","type":"text","marks":[{"type":"code_inline"}]},{"text":" before running (if the shell doesn't keep variables between commands, prefix every command with it), or replace it with that path directly.","type":"text"}]}]},{"type":"paragraph","content":[{"text":"The work is split into three layers. Remember this first, then read the flow:","type":"text"}]},{"type":"table","attrs":{"layout":null},"content":[{"type":"tr","content":[{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Layer","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Who","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Tools","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Decision: which candidates exist, how evidence is scored, whether a candidate can be excluded, what to scan next","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Script","type":"text","marks":[{"type":"strong"}]},{"text":" (candidate board)","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"board.py","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Perception: reading text, table lookups, finding targets in satellite imagery, street-view matching","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Scripts compute and rank first; you look only at the top few","type":"text","marks":[{"type":"strong"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"intake.py","type":"text","marks":[{"type":"code_inline"}]},{"text":" ","type":"text"},{"text":"ocr.py","type":"text","marks":[{"type":"code_inline"}]},{"text":" ","type":"text"},{"text":"clues.py","type":"text","marks":[{"type":"code_inline"}]},{"text":" ","type":"text"},{"text":"sat_scan.py","type":"text","marks":[{"type":"code_inline"}]},{"text":" ","type":"text"},{"text":"match.py","type":"text","marks":[{"type":"code_inline"}]},{"text":" ","type":"text"},{"text":"geo.py bearings","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Judgment: pulling clues out of the image, proposing hypotheses when the tables have nothing, deciding among the machine's top few","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"You","type":"text","marks":[{"type":"strong"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"—","type":"text"}]}]}]}]},{"type":"paragraph","content":[{"text":"The method comes from breaking down 14 photo-geolocation (网络迷踪) creator videos and 22 puzzles, plus several rounds of blind-test comparisons; the breakdown notes are not published with the repo. The lesson of v1: rules written as prose don't get executed, and two runs of the same skill version gave very different results; so v2 puts every rule that can be written as code into ","type":"text"},{"text":"board.py","type":"text","marks":[{"type":"code_inline"}]},{"text":".","type":"text"}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Hard rules (always in force; ★ = enforced by board.py, just follow it)","type":"text"}]},{"type":"ordered_list","attrs":{"order":1,"listStyle":"number"},"content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"If the user says the photo isn't theirs, or the image shows a private residence or minors, ask once what it's for before continuing; when the prompt already states the source or purpose (an evaluation puzzle, a puzzle setter's hint, the user says they took it), don't ask, and go to the finest level as usual.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Don't fabricate verification.","type":"text","marks":[{"type":"strong"}]},{"text":" \"Measured on the map\", \"matched street view\", \"±10 m\" must correspond to commands actually run and files actually produced in this session. If you didn't run it, write \"unverified\".","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Precision needs a source","type":"text","marks":[{"type":"strong"}]},{"text":": an error radius ≤100 m requires the intersection of two independent constraints, or a ground-level match on ≥3 invariant features. Before reporting a camera position, run a self-check: ","type":"text"},{"text":"pose.py project --horizon \u003csea-horizon row>","type":"text","marks":[{"type":"code_inline"}]},{"text":" computes what the pitch should be and each feature's \"expected row\" for the current camera position; reconcile that with the photo (the depression angle to a target ≠ the camera's pitch; look up the camera position's elevation with ","type":"text"},{"text":"terrain.py elev","type":"text","marks":[{"type":"code_inline"}]},{"text":" — if the height difference doesn't match the depression angles in the image, the camera position is wrong).","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Once you have a unique anchor, close the loop","type":"text","marks":[{"type":"strong"}]},{"text":": all later search and geometry starts from the anchor; don't go back to generic features like \"blue-green lake water\" or \"tropical park\" and pick the most famous place of that kind.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Compute bearings first, then identify structures","type":"text","marks":[{"type":"strong"}]},{"text":": which block in satellite imagery is the tower or chimney in the photo is itself an interpretation. First compute its true bearing in the image with ","type":"text"},{"text":"sun.py compass","type":"text","marks":[{"type":"code_inline"}]},{"text":" or a confirmed landmark, then use ","type":"text"},{"text":"geo.py bearings","type":"text","marks":[{"type":"code_inline"}]},{"text":" to see which outline around the candidate camera position falls on that bearing; when nothing fits, first try both templates (\"original/mirrored\", \"sunrise/sunset\"); if not verified, only lower the tier, don't exclude.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"★ ","type":"text"},{"text":"List the whole category before inferring","type":"text","marks":[{"type":"strong"}]},{"text":": when inferring a region from \"hill city\", \"tropical\", \"IP in a municipality\", use ","type":"text"},{"text":"board.py children","type":"text","marks":[{"type":"code_inline"}]},{"text":" to add all subordinate admin divisions as candidates, then rank them with evidence. Don't default to the main urban district, don't pick by population.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Metadata and hints are hypotheses","type":"text","marks":[{"type":"strong"}]},{"text":": when EXIF, IP location, location tags or the puzzle setter's words conflict with the image, the image wins; don't invent a story to reconcile them. IP says country A but the hint or the regulation designs (plates, signs) point to another continent: intersect with ","type":"text"},{"text":"clues.py lookup territories \u003ccountry A> --continent \u003ccontinent>","type":"text","marks":[{"type":"code_inline"}]},{"text":".","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Self-consistency","type":"text","marks":[{"type":"strong"}]},{"text":": redo each key judgment once with a different crop or keyword set; if the two results differ by tens of kilometers or more, lower the tier.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"★ ","type":"text"},{"text":"If you can't pin a point, give a range + what information is missing","type":"text","marks":[{"type":"strong"}]},{"text":"; when the candidates are a few discrete ones, don't take the midpoint: the main answer of ","type":"text"},{"text":"board.py report","type":"text","marks":[{"type":"code_inline"}]},{"text":" is always the top-ranked candidate; the rest go into alternatives with discriminating tests.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"★ ","type":"text"},{"text":"Exclusion and confirmation use the same standard","type":"text","marks":[{"type":"strong"}]},{"text":": ","type":"text"},{"text":"board.py exclude","type":"text","marks":[{"type":"code_inline"}]},{"text":" only accepts read/computed clues + a computed file (output of ","type":"text"},{"text":"geo.py frame","type":"text","marks":[{"type":"code_inline"}]},{"text":", ","type":"text"},{"text":"terrain.py","type":"text","marks":[{"type":"code_inline"}]},{"text":", etc.); observed and inferred clues can only down-weight via ","type":"text"},{"text":"evidence --against","type":"text","marks":[{"type":"code_inline"}]},{"text":", with the likelihood ratio clamped to 1/3–3 (inferred) or 1/5–5 (observed). Generating candidates from \"there's an X nearby\" is also filtering, and X must be something confirmed in the image. Fine-level candidates such as campuses, residential compounds and street segments must go on the board too: add a batch at once with ","type":"text"},{"text":"board.py add --from \u003cpoi.py --out / osm.py geom output> --level area/road","type":"text","marks":[{"type":"code_inline"}]},{"text":", don't hand-pick; when you drop one, write ","type":"text"},{"text":"evidence --against","type":"text","marks":[{"type":"code_inline"}]},{"text":" with a comparison image, and ","type":"text"},{"text":"check","type":"text","marks":[{"type":"code_inline"}]},{"text":" lists candidates without a single piece of evidence (meaning nobody looked at them); a finer candidate goes under the one that contains it (a point under its street segment: ","type":"text"},{"text":"--parent","type":"text","marks":[{"type":"code_inline"}]},{"text":", or ","type":"text"},{"text":"board.py move","type":"text","marks":[{"type":"code_inline"}]},{"text":"), so its support counts in that level's ranking, and ","type":"text"},{"text":"check","type":"text","marks":[{"type":"code_inline"}]},{"text":" fails when it skips a level; labels from third-party data (tree species in a municipal inventory, an exact school-name match) can only down-weight, never justify exclusion (in two cases the ground truth was skipped by hand exactly this way). ","type":"text"},{"text":"The excluded extent must be ≤ the evidence extent","type":"text","marks":[{"type":"strong"}]},{"text":": for candidates with extent, such as districts, areas and roads, ","type":"text"},{"text":"exclude","type":"text","marks":[{"type":"code_inline"}]},{"text":" needs ","type":"text"},{"text":"--covers lat,lon[:lat,lon]","type":"text","marks":[{"type":"code_inline"}]},{"text":" stating which stretch the evidence covers; the script rejects coverage below half (excluding a whole road after looking at one point on it failed in two cases).","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"★ ","type":"text"},{"text":"Population and fame are not evidence","type":"text","marks":[{"type":"strong"}]},{"text":"; scan order follows \"share ÷ pages\" (","type":"text"},{"text":"board.py next","type":"text","marks":[{"type":"code_inline"}]},{"text":"): finish the small districts first, put large districts last with a page cap.","type":"text"}]}]}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Flow","type":"text"}]},{"type":"paragraph","content":[{"text":"You don't have to go through every step: if reverse image search in step 1 hits directly, jump to steps 6 and 7 to confirm. Every step's output goes on the candidate board.","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Step 1: steps 0–3 in one command","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"bash"},"content":[{"text":"uv run ${CLAUDE_SKILL_DIR}/scripts/intake.py photo.jpg --out-dir intake/ [--box x0,y0,x1,y1 ...]\nuv run ${CLAUDE_SKILL_DIR}/scripts/board.py init --photo photo.jpg","type":"text"}]},{"type":"paragraph","content":[{"text":"intake.py","type":"text","marks":[{"type":"code_inline"}]},{"text":" usually takes 1–2 minutes, longer on the first run because it installs dependencies: give the command a generous timeout (10+ minutes) or run it in the background. If a command timeout interrupts it, the reverse-image-search subprocesses may still be writing into ","type":"text"},{"text":"rev/","type":"text","marks":[{"type":"code_inline"}]},{"text":", but ","type":"text"},{"text":"intake.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" won't be generated; don't treat it as finished.","type":"text"}]},{"type":"paragraph","content":[{"text":"intake.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" contains: metadata, OCR text (lines read from upscaled/tiled passes are marked pass=up/tile and are hypotheses), Baidu similar images (source-site counts + numbered contact sheet), reverse-image-search labels with tiered vote counts, likely residential compound/development names, the list of edge crops, and failed items. Then you do four things:","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Look at the image","type":"text","marks":[{"type":"strong"}]},{"text":": go through every crop in ","type":"text"},{"text":"edges/","type":"text","marks":[{"type":"code_inline"}]},{"text":" (four edges, four corners); run through the checklist in ","type":"text"},{"text":"references/observe.md","type":"text","marks":[{"type":"code_inline"}]},{"text":"; log each clue with ","type":"text"},{"text":"board.py clue \"\u003ctext>\" --kind \u003ckind> --status observed|read|inferred|computed --file \u003czoomed crop>","type":"text","marks":[{"type":"code_inline"}]},{"text":". Be honest about status: text you read is read; \"the building is probably 8 floors\", \"the road goes uphill\" are inferred.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"When a computed result actually tests an earlier clue, link it with ","type":"text"},{"text":"clue \"\u003cresult>\" --kind \u003ckind> --status computed --file \u003coutput> --resolves K1","type":"text","marks":[{"type":"code_inline"}]},{"text":" (repeat ","type":"text"},{"text":"--resolves","type":"text","marks":[{"type":"code_inline"}]},{"text":" for multiple clues). An unrelated computation does not resolve a pending infrastructure clue; this link records the test, not support for a location.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Lookup tables","type":"text","marks":[{"type":"strong"}]},{"text":": calling codes, driving side, overseas territories are global; plates, area codes and other country tables come from region packs → ","type":"text"},{"text":"clues.py lookup \u003ckind> \u003cvalue> [--country \u003ccc>]","type":"text","marks":[{"type":"code_inline"}]},{"text":" (without ","type":"text"},{"text":"--country","type":"text","marks":[{"type":"code_inline"}]},{"text":", every pack is searched and matches from several countries are all returned). For those that resolve to an admin division, use ","type":"text"},{"text":"board.py apply --kind plate --value \u003cplate prefix> --file \u003czoomed crop>","type":"text","marks":[{"type":"code_inline"}]},{"text":" directly (adds candidates and evidence automatically; the other candidates at the same level are only down-weighted, not excluded).","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Reverse-image-search results","type":"text","marks":[{"type":"strong"}]},{"text":": first open ","type":"text"},{"text":"rev/\u003cname>_baidu_similar.jpg","type":"text","marks":[{"type":"code_inline"}]},{"text":" (the top-left tile is the query image) and look for near-duplicates of the same object or the same scene; if there are any, go by number to ","type":"text"},{"text":"similar[i].from","type":"text","marks":[{"type":"code_inline"}]},{"text":" in the JSON to see the source page; compound, development or hotel names in the labels → ","type":"text"},{"text":"poi.py \"\u003cname>\" --city \u003ccity> --out pois.json","type":"text","marks":[{"type":"code_inline"}]},{"text":" to get coordinates; for same-name hits (several campuses, several branches) put them all on the board with ","type":"text"},{"text":"board.py add --from pois.json --level area","type":"text","marks":[{"type":"code_inline"}]},{"text":", then check; always open the screenshots, and once a post hits, look through the rest of its photo set. Choose where to search by object type (","type":"text"},{"text":"references/search.md","type":"text","marks":[{"type":"code_inline"}]},{"text":").","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Hints and metadata","type":"text","marks":[{"type":"strong"}]},{"text":": log each as an inferred clue and state how credible it is; IP location only says where the person was when posting, and the posting time is not the capture time.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Country-specific material","type":"text","marks":[{"type":"strong"}]},{"text":": while the country is still open, check ","type":"text"},{"text":"references/clues/global.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" (clues that separate countries). Country-specific clues, tables, preferred services and tips are in region packs (","type":"text"},{"text":"regions/\u003ccc>/","type":"text","marks":[{"type":"code_inline"}]},{"text":"). Once a country is a candidate, run ","type":"text"},{"text":"regions.py show \u003ccc>","type":"text","marks":[{"type":"code_inline"}]},{"text":": it prints the pack's lookups, services and a one-line index of its clue entries with line numbers; read only the entries you need. When the country is settled, ","type":"text"},{"text":"board.py country \u003ccc>","type":"text","marks":[{"type":"code_inline"}]},{"text":" and stop consulting ","type":"text"},{"text":"global.md","type":"text","marks":[{"type":"code_inline"}]},{"text":". ","type":"text"},{"text":"regions.py list","type":"text","marks":[{"type":"code_inline"}]},{"text":" shows which packs exist; with no pack, use the global tools.","type":"text"}]}]}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Step 2: list all candidates, score evidence, check the next step","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"bash"},"content":[{"text":"uv run ${CLAUDE_SKILL_DIR}/scripts/board.py children \u003cparent admin area> # municipality → all districts; country → first-level admin divisions (gazetteer.py queries OSM, with bbox)\nuv run ${CLAUDE_SKILL_DIR}/scripts/board.py evidence --clue K1 --for \u003ccandidate A>:5 --for \u003ccandidate B>:2 --against \u003ccandidate C>:0.3 --why \"…\" --file \u003ccomparison image>\nuv run ${CLAUDE_SKILL_DIR}/scripts/board.py rank\nuv run ${CLAUDE_SKILL_DIR}/scripts/board.py next","type":"text"}]},{"type":"paragraph","content":[{"text":"next","type":"text","marks":[{"type":"code_inline"}]},{"text":" only ever says one of two things:","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Can't separate","type":"text","marks":[{"type":"strong"}]},{"text":": run discriminating tests from cheap to expensive, each one on all candidates at once — lookup tables → terrain (plain vs. hill city, ","type":"text"},{"text":"tiles.py fetch --zoom 13","type":"text","marks":[{"type":"code_inline"}]},{"text":" or ","type":"text"},{"text":"terrain.py","type":"text","marks":[{"type":"code_inline"}]},{"text":") → vehicle livery (image search for \"\u003ccity> \u003ccolor> bus\" in the local language; read route signs from the result images, compare the stripe along the bus rear by district) → street fixtures (street view per candidate built-up area) → river/road-network templates. Still can't separate after all of them: don't stop; scan in the \"share ÷ pages\" order it gives. ","type":"text"},{"text":"next","type":"text","marks":[{"type":"code_inline"}]},{"text":" replaces these lines with the region pack's tips when the board's country is known.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Ready to narrow","type":"text","marks":[{"type":"strong"}]},{"text":": first narrow the extent to the built-up area with ","type":"text"},{"text":"board.py urban \u003ccandidate>","type":"text","marks":[{"type":"code_inline"}]},{"text":" (or give it by hand with ","type":"text"},{"text":"scan-bbox","type":"text","marks":[{"type":"code_inline"}]},{"text":"), then write a falsification condition with ","type":"text"},{"text":"board.py falsify \u003ccandidate> --text \"…\"","type":"text","marks":[{"type":"code_inline"}]},{"text":", then go to step 3.","type":"text"}]}]}]},{"type":"paragraph","content":[{"text":"The coarse environmental-location rules still apply: terrain before river width and building color; phenology must be paired with the month; take the intersection of rare facility combinations; recognizable species are only an exclusion tool; when all you have is a land type, first narrow to that type of land with a land-cover map (","type":"text"},{"text":"references/clues/","type":"text","marks":[{"type":"code_inline"}]},{"text":").","type":"text"}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Step 3: pick a branch to narrow down","type":"text"}]},{"type":"table","attrs":{"layout":null},"content":[{"type":"tr","content":[{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"What you have","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Approach","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Read","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"A unique anchor (building, statue, tower, scenic-area building)","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Anchor geometry: sight-line intersection, alignment lines, tangent lines, filtering buildings by camera height","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"geometry.md","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Distant mountains, skyline","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Render candidate camera positions with ","type":"text"},{"text":"terrain.py view","type":"text","marks":[{"type":"code_inline"}]},{"text":" and compare; this fixes only one sight line, then find a second constraint. No EXIF: don't assume the main camera (distant mountains are often zoomed); a best focal on the edge of the searched range means widen and rerun. Near range in front of a far one: score both layers","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"geometry.md","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Clear shadows, lit faces, the sun in the frame","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"sun.py locate / when / facing / compass","type":"text","marks":[{"type":"code_inline"}]},{"text":": latitude band, time, heading, true bearing of objects in the image","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"sky.md","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Route numbers, railway or power-line specs, a large river","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Linear corridor + line-to-point: ","type":"text"},{"text":"osm.py route / crossings / along","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"corridors.md","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Two or three kinds of infrastructure in one frame","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"osm.py near --report","type":"text","marks":[{"type":"code_inline"}]},{"text":"; crossings of two linear types ","type":"text"},{"text":"osm.py intersect","type":"text","marks":[{"type":"code_inline"}]},{"text":" (bends are only annotated, not removed)","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"corridors.md","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Linear infrastructure + unrecognized mountains, no text","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Compute horizons from elevation along the infrastructure and filter a whole region → score skyline + distance to infrastructure numerically → overlay the top 3–5 → fix the camera position from evenly spaced structures: ","type":"text"},{"text":"terrain.py scan --lines … --out hits.json --clusters-out clusters.json","type":"text","marks":[{"type":"code_inline"}]},{"text":" → ","type":"text"},{"text":"terrain.py ridge photo.jpg --x0 … --x1 … --out ridge.json","type":"text","marks":[{"type":"code_inline"}]},{"text":" → ","type":"text"},{"text":"terrain.py fit --hits hits.json --ridge ridge.json [--line … --line-dist …] --sheet top.jpg --photo photo.jpg","type":"text","marks":[{"type":"code_inline"}]},{"text":" (for the fine search switch to ","type":"text"},{"text":"--at lat,lon --radius 800 --grid 100 --zoom 13","type":"text","marks":[{"type":"code_inline"}]},{"text":") → ","type":"text"},{"text":"imgprep.py piers photo.jpg --rows … --out cols.json --sheet piers.jpg","type":"text","marks":[{"type":"code_inline"}]},{"text":" → ","type":"text"},{"text":"geo.py spacing --cols … --line … --span … --center lat,lon --ridge ridge.json","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"corridors.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" 4.3, ","type":"text"},{"text":"geometry.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" 7.4 / 7.7","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Clear street layout, no anchor","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"osm.py street-scan","type":"text","marks":[{"type":"code_inline"}]},{"text":" to filter intersections → ","type":"text"},{"text":"tiles.py sheet","type":"text","marks":[{"type":"code_inline"}]},{"text":" → street view","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"corridors.md","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"No near-duplicates from reverse image search, no text, only a set of scene elements (road spec + adjacent features + landform)","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Don't hand-pick spots first","type":"text","marks":[{"type":"strong"}]},{"text":": write the scene as one English query and let CLIP rank with ","type":"text"},{"text":"sat_scan.py grid --bbox \u003ccoastal strip/corridor> --zoom 17 --cell 360 --query … --neg …","type":"text","marks":[{"type":"code_inline"}]},{"text":"; look only at the top 20–30 thumbnails; ","type":"text"},{"text":"grid","type":"text","marks":[{"type":"code_inline"}]},{"text":" beats ","type":"text"},{"text":"points","type":"text","marks":[{"type":"code_inline"}]},{"text":" because OSM features like ","type":"text"},{"text":"amenity=parking","type":"text","marks":[{"type":"code_inline"}]},{"text":" may be incomplete","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"search.md","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"≥4 points of known location in the frame (window views, looking down; also level shots of distant towers, bridges, piers — fix the height)","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"pose.py solve","type":"text","marks":[{"type":"code_inline"}]},{"text":" solves camera position, heading and height; when several candidate positions all fit, score them one by one with ","type":"text"},{"text":"pose.py check --cands","type":"text","marks":[{"type":"code_inline"}]},{"text":"; only robust Δchi2 > 9 may go to ","type":"text"},{"text":"board.py exclude --computed","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"geometry.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" section 10","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Want to exclude by \"there's no X in the frame\"","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"geo.py frame","type":"text","marks":[{"type":"code_inline"}]},{"text":" first computes whether X should be inside the frame, large enough and not occluded; only what it says can exclude goes to ","type":"text"},{"text":"board.py exclude --computed","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"geometry.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" section 11","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"An unnamed building of a recognizable class, search region narrowed","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Enumerate with ","type":"text"},{"text":"osm.py find","type":"text","marks":[{"type":"code_inline"}]},{"text":" → nearby views facing each building with ","type":"text"},{"text":"gsv.py targets","type":"text","marks":[{"type":"code_inline"}]},{"text":" → rank with ","type":"text"},{"text":"match.py","type":"text","marks":[{"type":"code_inline"}]},{"text":"; use bounded batches","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"search.md","type":"text","marks":[{"type":"code_inline"}]},{"text":" section 6","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Recognizable facility type (grain dryer tower, feed mill, sugar mill, concrete batching plant)","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Look up the industry's distribution → enumerate large buildings with ","type":"text"},{"text":"osm.py buildings","type":"text","marks":[{"type":"code_inline"}]},{"text":" → rank with ","type":"text"},{"text":"sat_scan.py points","type":"text","marks":[{"type":"code_inline"}]}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"search.md","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Stores of a chain's sub-brand","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"First search opening press releases for the address; use store locators only as a candidate pool → ","type":"text"},{"text":"osm.py along","type":"text","marks":[{"type":"code_inline"}]},{"text":" + street-view sampling","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"search.md","type":"text","marks":[{"type":"code_inline"}]}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Plane window, drone shots looking down","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Aerial branch","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"aerial.md","type":"text","marks":[{"type":"code_inline"}]}]}]}]}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Step 4: finding the spot in satellite imagery — the machine ranks first","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"bash"},"content":[{"text":"uv run ${CLAUDE_SKILL_DIR}/scripts/sat_scan.py grid --bbox \u003cscan_bbox> --zoom 17 --preset track --multi-scale --top 30 --out sat.json --sheet sat_top.jpg --heat heat.jpg\nuv run ${CLAUDE_SKILL_DIR}/scripts/poi.py \"\u003cdistrict> \u003cschool, in the local language>\" --city \u003ccity> --region \u003ccc> --out schools.json # seeds: --seeds schools.json boosts nearby cells\nuv run ${CLAUDE_SKILL_DIR}/scripts/sat_scan.py points --points big.json --preset factory --out r.json --sheet r.jpg # rank candidate points from osm.py buildings / poi.py","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Presets: track (running tracks), stadium, factory, silo, dam, bridge, quarry, solar, greenhouse, port; custom ","type":"text"},{"text":"--query","type":"text","marks":[{"type":"code_inline"}]},{"text":". In tests: across 300-odd cells of an urban area, more than half of the OSM-mapped running tracks made the top 30; in an old town in China where OSM is blank, a school's sports field still ranked first. ","type":"text"},{"text":"It ranks, it doesn't decide","type":"text","marks":[{"type":"strong"}]},{"text":": look at the thumbnails of the top 20–30 cells, then check them against the bearing and shape in the photo.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Translate the image description into top-down features before looking: curved buildings, octagonal pavilion roofs, sports courts, parking spaces perpendicular to a railway; for tall buildings look at the base; imagery has a date.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"With many candidates, make a candidate table: one row per point, one column per criterion directly visible in the photo.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Nothing in the top 30 matches: first go back to the \"candidates down-weighted by inference\" listed by ","type":"text"},{"text":"board.py check","type":"text","marks":[{"type":"code_inline"}]},{"text":" and to the falsification conditions, then switch preset or go to z18, and only then widen the area.","type":"text"}]}]}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Step 5: confirmation — street view is ranked first too","type":"text"}]},{"type":"code_block","attrs":{"wrap":false,"language":"bash"},"content":[{"text":"uv run ${CLAUDE_SKILL_DIR}/scripts/baidu_pano.py scan \u003clat,lon> --radius 300 --out panos.json # the street-view script the region card names (CN: baidu_pano.py); otherwise gsv.py\nuv run ${CLAUDE_SKILL_DIR}/scripts/match.py rank --query photo.jpg --panos panos.json --toward \u003clandmark lat,lon> --spread 15 --refine sift --top 10 --out m.json --sheet m.jpg\nuv run ${CLAUDE_SKILL_DIR}/scripts/match.py rank --query photo.jpg --items around.index.json --render gsv --spread-headings -30,0,30 --out m.json --sheet m.jpg","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"match.py","type":"text","marks":[{"type":"code_inline"}]},{"text":" coarse-ranks by DINOv2 global similarity and fine-ranks by SIFT inliers; in tests, ground-truth street views of the same place from different years all made the top 4. Open only the top 10 and compare ","type":"text"},{"text":"invariant features","type":"text","marks":[{"type":"strong"}]},{"text":" (building outline, window positions, balconies, pole positions, curbs, ridgelines), not vehicles, signs or foliage. ≥2 features for road level, ≥3 for building level. Not a single image with ≥15 inliers ","type":"text"},{"text":"doesn't mean it's wrong","type":"text","marks":[{"type":"strong"}]},{"text":": with a change of season, an old capture, or the photo taken from the sidewalk while street view was shot from the middle of the road, the ground truth had only 5 and 0–8 inliers in tests (two cases); first open the top 10 and compare invariant features, and only if none fit change ","type":"text"},{"text":"--spread-headings","type":"text","marks":[{"type":"code_inline"}]},{"text":" or widen ","type":"text"},{"text":"--within","type":"text","marks":[{"type":"code_inline"}]},{"text":". The global score gets dominated by season (a blossom-season capture ranks high wherever it is), so compare against a historical capture from the same season as the photo when you can (","type":"text"},{"text":"gsv.py sheet --date","type":"text","marks":[{"type":"code_inline"}]},{"text":").","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"With many panorama points, group by date and road; ","type":"text"},{"text":"sheet --road \u003croad name> --spread 60","type":"text","marks":[{"type":"code_inline"}]},{"text":" shows one road only; older captures have a more open view.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"No street view doesn't mean you can't confirm: compare ridgelines with ","type":"text"},{"text":"terrain.py view --photo","type":"text","marks":[{"type":"code_inline"}]},{"text":"; find a name for the candidate facility (OSM name, nearby place name + the facility-type word in the local language) and search news, encyclopedias and official-site photos to compare facade details.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"When street view is many years older than the photo, go by old buildings and permanent structures.","type":"text"}]}]}]},{"type":"heading","attrs":{"level":3},"content":[{"text":"Step 6: fix the camera position and write the output","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Look back: at the matching street-view point, turn 180° and see what's on the photographer's side.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"A camera position needs two independent constraints (","type":"text"},{"text":"geo.py intersect","type":"text","marks":[{"type":"code_inline"}]},{"text":" sight-line intersection, ","type":"text"},{"text":"geo.py line","type":"text","marks":[{"type":"code_inline"}]},{"text":" alignment line, ","type":"text"},{"text":"pose.py","type":"text","marks":[{"type":"code_inline"}]},{"text":" multi-point solve, camera height, reverse street view); with only one, building-level confidence is at most \"medium\".","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"When there's no measurable shadow, use lit faces for the heading: ","type":"text"},{"text":"sun.py facing --lit left --shaded camera","type":"text","marks":[{"type":"code_inline"}]},{"text":". For photos taken from a car, boat or train, state the direction of travel.","type":"text"}]}]}]},{"type":"code_block","attrs":{"wrap":false,"language":"bash"},"content":[{"text":"uv run ${CLAUDE_SKILL_DIR}/scripts/board.py check # before concluding: do exclusions have files, does the main answer have verified evidence, which clues went unused\nuv run ${CLAUDE_SKILL_DIR}/scripts/board.py report --merge result.json # main answer = top-ranked; alternatives, exclusions and unused clues are written into result.json automatically\nuv run ${CLAUDE_SKILL_DIR}/scripts/evidence.py spec.json --out evidence.jpg","type":"text"}]},{"type":"paragraph","content":[{"text":"Show ","type":"text"},{"text":"evidence.jpg","type":"text","marks":[{"type":"code_inline"}]},{"text":" to the user together with the answer, without waiting to be asked. A road-level claim from satellite imagery needs a ≥z18 crop, not a z17 glance.","type":"text"}]},{"type":"paragraph","content":[{"text":"Output:","type":"text"}]},{"type":"ordered_list","attrs":{"order":1,"listStyle":"number"},"content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"One-sentence conclusion: place + camera position + heading (+ direction of travel and capture time, where applicable)","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Coordinates: WGS84, plus the local datum when the region card names one (","type":"text"},{"text":"geo.py convert","type":"text","marks":[{"type":"code_inline"}]},{"text":"), ","type":"text"},{"text":"with error radius","type":"text","marks":[{"type":"strong"}]}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Evidence image (satellite image marking the camera position and heading wedge + comparison images)","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Reasoning chain: clue → inference → ","type":"text"},{"text":"commands actually run and files produced","type":"text","marks":[{"type":"strong"}]},{"text":" → extent","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Tiered confidence (table below). ","type":"text"},{"text":"Rate each level separately; the tier you report is the finest level rated \"medium\" or above","type":"text","marks":[{"type":"strong"}]},{"text":": a low building level doesn't drag down a high area level","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Excluded candidates and reasons, alternatives and discriminating tests, clues not used or not resolved (generated by ","type":"text"},{"text":"board.py report","type":"text","marks":[{"type":"code_inline"}]},{"text":")","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"When you can't pin a point: which level is settled + what information is still needed","type":"text"}]}]}]},{"type":"paragraph","content":[{"text":"Use English for tool messages, report headings and generated labels. Preserve source text (OCR, place names, service responses) verbatim as evidence; explain or translate it for the reader. Write the agent's final report in the user's language. In Chinese, use 高/中/低 for high/medium/low and 城市/片区/路/楼/楼层 for city/area/road/building/floor.","type":"text"}]},{"type":"table","attrs":{"layout":null},"content":[{"type":"tr","content":[{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Level","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"High","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Medium","type":"text"}]}]},{"type":"th","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Low","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"City","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Text, plates, area codes or a confirmed anchor","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Several independent weak clues agree","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"A single weak clue, or only the hint","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Area","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"A unique facility or landmark matches ≥3 top-down features in satellite imagery (whole candidate area checked, no second match)","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"The most similar spot in the candidate area, with candidates still unchecked","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Inferred","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Road","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Ground-level match on ≥2 unique features","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Layout consistent, 1 unique feature; ","type":"text"},{"text":"when there is no ground-level imagery at all","type":"text","marks":[{"type":"strong"}]},{"text":": area level high + bearing self-check passed","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Satellite layout only","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Building","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Two independent constraints + ground-level match on ≥3 features","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"One of the two","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Inferred","type":"text"}]}]}]},{"type":"tr","content":[{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Floor","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Two kinds of reference agree","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"A single reference","type":"text"}]}]},{"type":"td","attrs":{"colspan":1,"rowspan":1,"colwidth":null,"alignment":""},"content":[{"type":"paragraph","content":[{"text":"Don't give","type":"text"}]}]}]}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Budget and stopping conditions","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"intake.py","type":"text","marks":[{"type":"code_inline"}]},{"text":" counts as one call; if reverse image search with 3 different keyword sets finds nothing, stop and go back to the checklist for other clues.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"While candidates still can't be separated, don't scan any point one by one","type":"text","marks":[{"type":"strong"}]},{"text":" (when ","type":"text"},{"text":"board.py next","type":"text","marks":[{"type":"code_inline"}]},{"text":" says \"can't separate\", do the discriminating tests first); if they still can't be separated after the cheap tests, scan in the order it gives, the first 3 pages of each candidate's built-up area first.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Scanning and confirmation always rank first: look at the top 30 cells from ","type":"text"},{"text":"sat_scan.py","type":"text","marks":[{"type":"code_inline"}]},{"text":" and the top 10 images from ","type":"text"},{"text":"match.py","type":"text","marks":[{"type":"code_inline"}]},{"text":"; widen only when all of the top ones are wrong, instead of paging through.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Run ","type":"text"},{"text":"osm.py coverage","type":"text","marks":[{"type":"code_inline"}]},{"text":" before enumerating candidates with OSM: districts with sparse data get silently excluded; switch to ","type":"text"},{"text":"sat_scan.py grid","type":"text","marks":[{"type":"code_inline"}]},{"text":" and say so in the conclusion.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Scan panoramas in batches of at most 500 points per area. When a batch or several areas yield no match, review candidate generation, search coverage and down-weighted candidates; then change the approach or widen the justified search scope. A fixed number of unmatched areas is not a reason to end the investigation.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"A failed or unavailable service blocks that route, not the whole investigation. Consider other feasible sources or methods; do not repeat unchanged failing requests.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Before stopping, ","type":"text"},{"text":"board.py check","type":"text","marks":[{"type":"code_inline"}]},{"text":": go back through the top 20 candidates, in ranking order, that were down-weighted by inference but not excluded.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Conclude when evidence supports the requested location level, the user asks to stop or a user-set budget is reached, or no remaining feasible action is likely to distinguish the candidates. If unresolved, use ","type":"text"},{"text":"board.py report","type":"text","marks":[{"type":"code_inline"}]},{"text":" to state the supported level, actual search coverage, attempted routes, blockers and missing evidence; do not invent a coordinate or precision to finish.","type":"text"}]}]}]},{"type":"heading","attrs":{"level":2},"content":[{"text":"Runtime environment","type":"text"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Python 3.10+, ","type":"text"},{"text":"uv","type":"text","marks":[{"type":"code_inline"}]},{"text":", and ","type":"text"},{"text":"curl","type":"text","marks":[{"type":"code_inline"}]},{"text":". Always use ","type":"text"},{"text":"uv run ${CLAUDE_SKILL_DIR}/scripts/xxx.py","type":"text","marks":[{"type":"code_inline"}]},{"text":"; each script declares its dependencies. ","type":"text"},{"text":"scripts/","type":"text","marks":[{"type":"code_inline"}]},{"text":" in references is relative to this skill directory.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"On first setup or after a runtime failure, run ","type":"text"},{"text":"uv run ${CLAUDE_SKILL_DIR}/scripts/doctor.py","type":"text","marks":[{"type":"code_inline"}]},{"text":"; add ","type":"text"},{"text":"--network","type":"text","marks":[{"type":"code_inline"}]},{"text":" to check service reachability. Read the English checks and fixes before starting an expensive scan. It uploads no photos and does not load ML models.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Reverse image search uses local Google Chrome, with automatic fallback to Playwright Chromium (","type":"text"},{"text":"uvx playwright install chromium","type":"text","marks":[{"type":"code_inline"}]},{"text":"). If neither starts, use ","type":"text"},{"text":"intake.py --no-rev","type":"text","marks":[{"type":"code_inline"}]},{"text":" and report the skipped search. OCR prefers Apple Vision on macOS and uses RapidOCR elsewhere or as a fallback.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"match.py","type":"text","marks":[{"type":"code_inline"}]},{"text":" and ","type":"text"},{"text":"sat_scan.py","type":"text","marks":[{"type":"code_inline"}]},{"text":" install ML dependencies and download model weights on first use. Allow extra time and disk space.","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"Cache goes to ","type":"text"},{"text":".geo-cache/","type":"text","marks":[{"type":"code_inline"}]},{"text":" in the current directory; the candidate board is ","type":"text"},{"text":"board.json","type":"text","marks":[{"type":"code_inline"}]},{"text":" in the current directory. Script list and data sources: ","type":"text"},{"text":"references/data-sources.md","type":"text","marks":[{"type":"code_inline"}]},{"text":".","type":"text"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"text":"macOS has no ","type":"text"},{"text":"timeout","type":"text","marks":[{"type":"code_inline"}]},{"text":" command; in zsh ","type":"text"},{"text":"$var","type":"text","marks":[{"type":"code_inline"}]},{"text":" doesn't word-split, so use ","type":"text"},{"text":"bash -c","type":"text","marks":[{"type":"code_inline"}]},{"text":" or ","type":"text"},{"text":"${=var}","type":"text","marks":[{"type":"code_inline"}]},{"text":" in loops. A province-wide Overpass query can take several minutes; run it in the background.","type":"text"}]}]}]}]},"metadata":{"date":"2026-11-02","name":"geo-sleuth","author":"@skillopedia","source":{"stars":5595,"repo_name":"geo-sleuth","origin_url":"https://github.com/Oldcircle/geo-sleuth/blob/HEAD/skills/geo-sleuth/SKILL.md","repo_owner":"Oldcircle","body_sha256":"18fc803dfbdccb92c7f92d783c266d33440266e5ffc43c79d227e760f1abc480","cluster_key":"1f86490c13cd1d1c651fd61cc73d146a093bea61e9643ad5fb22a8b637ffacef","clean_bundle":{"format":"clean-skill-bundle-v1","source":"Oldcircle/geo-sleuth/skills/geo-sleuth/SKILL.md","text_bytes":1048519,"source_commit":"1a9ecee153b02e2db2e62352002fa5b17bbd819d","skills_sh_rank":5684,"attachment_count":52,"text_attachments":52,"attachment_storage":"skill_attachments","binary_attachments":0},"cluster_size":1,"skill_md_path":"skills/geo-sleuth/SKILL.md","import_metadata":{"date":"2026-11-02","author":"@skillopedia","version":"v1","category":"media-content","category_label":"Media"},"exact_dupes_collapsed_into_this":0},"version":"v1","category":"media-content","import_tag":"skills-sh-top20k-2026-11","description":"Geolocate or chronolocate a photo with tool-verified reasoning (where was this taken / when was it taken / photo geolocation / geo-guessing). Given one or more photos, intake.py does metadata, OCR and reverse image search (Baidu/Yandex) in one command; clues and candidates go on a candidate board (board.py) that ranks them and gives the next step; clues.py looks up calling codes, driving side, plates and area codes from region packs; for satellite and street view the machine ranks first and you look only at the top few (CLIP in sat_scan.py, DINOv2+SIFT in match.py); plus sun and shadow math, OSM Overpass and DEM skylines. Every conclusion is checked against real data; output is coordinates + error radius, evidence images and tiered confidence. Use when the user shares a photo and asks where was this taken / geolocate this / when was this taken / 这是哪 / 在哪拍的 / 帮我定位这张照片 / 网络迷踪 / 图寻 / 几点拍的."}},"renderedAt":1793671706158}

Important: agents should read /llm.txt, /llms.txt, or /.well-known/skills.json to discover the public Skillopedia API.