From 9b262ffa34ae81dc4dc189c75315ca1d28afdafc Mon Sep 17 00:00:00 2001 From: Twarner491 Date: Mon, 8 Jun 2026 12:00:36 -0700 Subject: [PATCH] [DOC] document the illustration regeneration pipeline --- README.md | 19 ++++----- avian/scripts/README.md | 90 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 98 insertions(+), 11 deletions(-) create mode 100644 avian/scripts/README.md diff --git a/README.md b/README.md index c6b1f3f..06da96b 100644 --- a/README.md +++ b/README.md @@ -51,22 +51,19 @@ Collage: `http://birdnet.local/`. Stock BirdNET-Pi UI: `http://birdnet.local/ind ## 3. (Optional) Restyle the illustrations -The repo ships with 450 bundled illustrations. To restyle or add region-specific species: +The repo ships with 498 bundled illustrations (249 species, perched + flight). To restyle them or generate a set for your own region: ```bash +pip install -r ~/BirdNET-Pi/avian/scripts/requirements.txt export GEMINI_API_KEY='your-key' -# Re-render every species in BirdNET-Pi's model: +# generate on a cream ground, cut the ground off, rebuild the collage masks python3 ~/BirdNET-Pi/avian/scripts/pregen.py --labels ~/BirdNET-Pi/model/labels.txt --force - -# Or filter to species observed in your eBird region: -export EBIRD_API_KEY='your-key' -python3 ~/BirdNET-Pi/avian/scripts/pregen.py \ - --labels ~/BirdNET-Pi/model/labels.txt \ - --ebird-region US-CA +python3 ~/BirdNET-Pi/avian/scripts/cutout.py +python3 ~/BirdNET-Pi/avian/scripts/build_masks.py ``` -Style lives in [`avian/scripts/prompt.template.md`](avian/scripts/prompt.template.md). Edit, re-run with `--force`. +Filter to your region with `--ebird-region US-CA` (needs `EBIRD_API_KEY`). The full pipeline, prompt, reference images, and per-species tuning live in [`avian/scripts/README.md`](avian/scripts/README.md). Style lives in [`prompt.template.md`](avian/scripts/prompt.template.md). --- @@ -85,9 +82,9 @@ See [`avian/forwarding/`](avian/forwarding/) for three independent recipes: ``` avian/ # everything we add to BirdNET-Pi ├── frontend/ # static HTML/JS/CSS for the collage -├── assets/ # 450 bundled illustrations + cutouts + masks +├── assets/ # 498 bundled illustrations + photo-cutout fallbacks ├── api/ # PHP shims served by BirdNET-Pi's PHP-FPM -├── scripts/ # pregen.py + editable prompt template +├── scripts/ # generate -> cutout -> masks pipeline + prompt └── forwarding/ # optional HA / MQTT / Cloudflare configs ``` diff --git a/avian/scripts/README.md b/avian/scripts/README.md new file mode 100644 index 0000000..6fe6534 --- /dev/null +++ b/avian/scripts/README.md @@ -0,0 +1,90 @@ +# Generating illustrations + +The collage art is generated, not hand-drawn. The repo ships 498 kachō-e +illustrations (249 species, a perched and a flight pose each). To restyle +them or build a set for your own region, the pipeline is four scripts in +this directory. + +## Pipeline + +1. `pregen.py` renders each bird with Gemini 2.5 Flash Image, on a flat cream ground. +2. `cutout.py` removes the ground with BiRefNet and crops to the bird. +3. `build_masks.py` rebuilds the collage silhouette masks inlined in `apt.js`. +4. `verify.py` (optional) runs an adversarial species-ID + anatomy check. + +```bash +pip install -r requirements.txt +export GEMINI_API_KEY='your-key' + +# 1. generate (cream ground) for your region's species +python3 pregen.py --labels ~/BirdNET-Pi/model/labels.txt --ebird-region US-CA + +# 2. cut the ground off and crop +python3 cutout.py + +# 3. rebuild the collage masks, then bump SKETCH_VERSION + IMG_VERSION in apt.js +python3 build_masks.py +``` + +`--labels` takes any `Sci|Com` per-line file (BirdNET-Pi's `labels.txt` works +directly). `--ebird-region` filters to species actually seen in your region +(needs `EBIRD_API_KEY`). Re-render one bird with +`--species "Calypte anna|Anna's Hummingbird" --force`. + +## Why a cream ground + +The image model can't cut a clean transparent background on its own: it +leaves holes and fringes, worst on pale birds. Rendering on a flat, +consistent cream ground gives a known color that BiRefNet removes cleanly, +and the steady ground also holds the painting style together across the +whole set. `cutout.py` is the step that makes the backgrounds transparent. + +## The prompt + +`prompt.template.md` is the kachō-e prompt, sent verbatim per request with +`{sci_name}`, `{com_name}`, and `{pose}` substituted. Edit it to change the +style. `pregen.py` attaches up to three reference images per request: + +- **Anatomy** (IMAGE 1): a Wikipedia photo of the target species, auto-fetched + and cached in `assets/references/`. Anchors identity and markings. Drop your + own `references/.jpg` to override. +- **Anti-reference** (IMAGE 2, optional): a photo of a look-alike the model + drifts toward, captioned with what NOT to copy. Wired for blue corvids (vs + Blue Jay) and swallows (vs Barn Swallow); add more in the `ANTI_REFS` table + and place photos at `references/_anti_.jpg`. +- **Style** (IMAGE 3, optional): a real Edo-period kachō-e print whose painting + technique is borrowed. The genus-to-print mapping is in `pregen.py`'s + `STYLE_REFS`. The prints are not bundled (they are someone else's art); put + your own in `assets/references/styles/`. The Koson and Yoshida prints used + originally are easy to find on the public web by the filenames in `STYLE_REFS`. + +All three degrade gracefully: a missing reference is simply not attached. + +## Hard species + +`species-notes.json` holds one-line diagnostic addenda for species the model +gets wrong. Each note names the field marks that matter and the look-alikes to +avoid, and is appended to the prompt for that species. Add entries as you find +drift; they carry forward to every future regeneration of that bird. + +## Verifying + +`verify.py` sends each illustration back through Gemini Vision without telling +it the target species, then checks the guess, the wing/leg/tail counts, and +whether a stray perch crept in. It catches drift a quick eyeball misses. + +```bash +python3 verify.py --labels labels.txt # whole library -> verify-results.csv +python3 verify.py --labels labels.txt calypte-anna +``` + +## What actually goes wrong + +- **Sticks.** Perched raptors often come back gripping a twig the prompt + forbade. Generate 2-3 and keep the clean one. +- **Species drift.** The model collapses an uncommon species toward a common + look-alike (a swift becomes a swallow). Fixes, in order: a sharper + `species-notes.json` note with anti-feature language; an anti-reference; a + different style print; a one-off `--species` regen. +- **Matched pair.** The perched and flight poses must read as the same + individual. Review them side by side before locking.