AvianVisitors hardening pass — install via fork's newinstaller

Restructures the install so users run ONE command. The fork's
newinstaller.sh clones this repo (instead of upstream Nachtzuster) and
runs BirdNET-Pi's installer. install_services.sh symlinks the avian/
overlay into the Caddy site root, so http://birdnet.local/avian/ comes
up after reboot with no Caddy config needed.

Reviewer-flagged fixes:

Frontend (avian/frontend/apt.js)
- Switch all API paths from /api/*.json (which didn't route) to relative
  action-based URLs (./api/birdnet-api.php?action=recent etc.).
- Fetch ./masks.json + ./dims.json relatively (were /avian/*.json).
- Replace every innerHTML user-content sink with createElement +
  textContent + setAttribute — species labels are user-editable in
  BirdNET-Pi, so untrusted.
- Add esc() helper for the few remaining innerHTML attribute contexts.
- fetchJson throws on non-200 (was silently parsing 404 HTML as JSON).
- Defensive null-check in loadMask if MASKS hasn't loaded.

Backend (avian/api/*.php)
- birdnet-api.php now derives DB_PATH from dirname(__DIR__, 3) so any
  BirdNET-Pi install user works (no hardcoded /home/birdnet).
- recording/spectrogram/cutout use getenv('HOME') for the same reason.
- cutout.php reworked as the canonical image resolver: bundled
  illustration → bundled cutout → cached rembg → fresh
  Wikipedia+rembg, with the binomial regex guard and atomic rename
  on cache write. Wikipedia URL host pinned to wikimedia.org /
  wikipedia.org to block SSRF.
- Stale 'X-BirdNET-Proxy-Token' auth claims removed everywhere.

Pregen (avian/scripts/pregen.py)
- Gemini key moved from URL query to x-goog-api-key header.
- Model bumped to gemini-2.5-flash-image-preview.
- Bounded retry on 429 + 5xx with Retry-After honoured.
- Default --sleep raised from 1s to 4s (under free-tier RPM).
- Unified parser handles |/_/comma in all three input modes.
- responseModalities now [TEXT, IMAGE] (was IMAGE-only).
- --poses validated against POSES keys.
- ASCII [ok]/[fail] markers (was ✓/✗ — Windows console crash).
- Surfaces finishReason + blockReason on safety blocks.

Prompt template
- 'Two placeholders' typo → three.
- 'warm paper' / 'transparent background' contradiction resolved.

Forwarding
- mqtt-bridge.py: paho-mqtt 2.x CallbackAPIVersion compat shim.
- avian-mqtt.service: dropped broken %i, set User=birdnet + %h path.
- forwarding/README.md uses the actual action-based URLs.

README
- One-line install via curl … newinstaller.sh.
- No more 'install BirdNET-Pi separately' step (our fork does it).
- License section uses absolute github.com links.
- bird.onethreenine.net replaces the not-yet-existent project writeup.
- docs/thumb.png replaced with the 24-bird Twitter capture.

Removed
- avian/caddy/ and avian/scripts/install.sh — no longer needed; the
  overlay symlink in install_services.sh handles everything.

avian/api/birdnet-api.php

@@ -1,26 +1,35 @@
 <?php
-// /home/monalisa/BirdSongs/Extracted/api.php — JSON facade over BirdNET-Pi's
-// birds.db, queryable by the bird.onethreenine.net Cloudflare Worker.
-//
-// Lives in the Caddy file-server root (NOT in BirdNET-Pi/scripts/) so the
-// Sunday auto-update doesn't clobber it. Reachable as /api.php on the Pi.
-//
-// Auth: callers must send X-BirdNET-Proxy-Token matching the Caddy gate set
-// up earlier — Caddy 403s anything missing it before this script runs, so we
-// inherit that protection for free. The Cloudflare Worker is the only thing
-// that ever sets the header.
+// AvianVisitors — JSON facade over BirdNET-Pi's birds.db. Read-only.
+// Symlinked into the BirdNET-Pi Caddy site root at /avian/api/.
 //
 // Endpoints (?action=...):
-//   stats       — totals: detections, unique species, today, last hour, etc.
+//   stats       — totals (detections, unique species, today, last hour)
 //   lifelist    — every species with first_seen, last_seen, total_count
-//   recent      — &hours=N (default 24): every detection in the window
-//   species     — &sci=<sci_name>: detail page for one species
+//   recent      — &hours=N (default 24): species heard in the window
+//   species     — &sci=<sci_name>: per-species detail page
+//   timeseries  — &days=N: daily detection counts per species
+//   firstseen   — every species' earliest detection
+//
+// Default LAN deploy ships without auth. If you've exposed the Pi via
+// Cloudflare or a tunnel, add a Caddy `basic_auth` matcher around the
+// /avian/api/* path — see avian/forwarding/.
 
 declare(strict_types=1);
 header('Content-Type: application/json; charset=utf-8');
 header('Cache-Control: public, max-age=30');
 
-$DB_PATH = '/home/monalisa/BirdNET-Pi/scripts/birds.db';
+// SCRIPT_FILENAME on the Pi resolves through the symlink to
+// $HOME/BirdNET-Pi/avian/api/birdnet-api.php — walk three dirs up to
+// reach the install root, then point at scripts/birds.db. Lets a Pi
+// installed under any username (the BirdNET-Pi installer uses $USER,
+// not a fixed name) work without editing this file.
+$DB_PATH = dirname(__DIR__, 3) . '/scripts/birds.db';
+// Fallback if the symlink layout ever changes — keeps the most common
+// install path working even if SCRIPT_FILENAME oddities trip __DIR__.
+if (!file_exists($DB_PATH)) {
+    $alt = getenv('HOME') . '/BirdNET-Pi/scripts/birds.db';
+    if (file_exists($alt)) $DB_PATH = $alt;
+}
 
 if (!file_exists($DB_PATH)) {
     http_response_code(503);

avian/api/cutout.php

@@ -1,20 +1,17 @@
 <?php
-// /home/monalisa/BirdSongs/Extracted/cutout.php — dynamic background-removed
-// bird-photo facade. Called by the bird.onethreenine.net Cloudflare Worker
-// (which fronts /api/img?sci=<name>).
+// AvianVisitors — bird image resolver.
 //
-// Lives in the Caddy file-server root so the BirdNET-Pi auto-update doesn't
-// touch it. Each new species hits this once: download Wikipedia/Macaulay
-// image → rembg via /usr/local/bin/rembg-cli → cache transparent PNG to
-// disk under cutouts/. Subsequent requests are instant readfile() from cache.
+// Lookup chain for /avian/api/cutout.php?sci=Calypte+anna:
+//   1. ../assets/illustrations/<slug>.png   (450+ bundled kachō-e renders)
+//   2. ../assets/cutouts/<slug>.png         (background-removed photo)
+//   3. cached rembg of a Wikipedia photo at $HOME/BirdSongs/Extracted/cutouts/
+//   4. fresh Wikipedia → rembg → cache (skipped gracefully if rembg unset)
 //
-// Auth: the Caddy site block 403s anything missing X-BirdNET-Proxy-Token at
-// the top, so this script inherits the Worker-only access guarantee.
+// The frontend's <img src> points here for every species — bundled
+// hits return instantly; cold misses fall through to the dynamic path.
 //
-// Pre-reqs (run once via /install/rembg-setup.sh):
-//   - /home/monalisa/rembg-env/bin/rembg            (Python venv)
-//   - /usr/local/bin/rembg-cli                      (wrapper)
-//   - /home/monalisa/BirdSongs/Extracted/cutouts/   (cache dir, monalisa-owned)
+// Default LAN deploy ships without auth. To expose publicly, gate
+// /avian/api/* with basic_auth in your Caddyfile — see avian/forwarding/.
 
 declare(strict_types=1);
 
@@ -24,36 +21,63 @@ if ($sci === '') {
     echo 'sci required';
     exit;
 }
+// Binomial / trinomial pattern. Rejects path-traversal payloads and
+// junk before any filesystem or upstream lookup.
+if (!preg_match('/^[A-Za-z]{2,40}(?:[ ][a-z]{2,40}){1,3}$/', $sci)) {
+    http_response_code(400);
+    echo 'invalid sci';
+    exit;
+}
 
-// Slugify scientific name for the cache filename.
+// Slugify scientific name for filename + cache key.
 $slug = preg_replace('/[^a-z0-9]+/', '-', strtolower($sci));
 $slug = trim((string)$slug, '-');
-$cacheDir = '/home/monalisa/BirdSongs/Extracted/cutouts';
-$cachePath = "$cacheDir/$slug.png";
 
 function serve_png(string $path): void {
     header('Content-Type: image/png');
-    header('Cache-Control: public, max-age=2592000');
-    header('Content-Length: ' . filesize($path));
+    header('Cache-Control: public, max-age=86400');
+    header('Content-Length: ' . (string)filesize($path));
     readfile($path);
     exit;
 }
 
-// Cache hit — short-circuit.
+// 1. Bundled illustration (the kachō-e PNG the repo ships with).
+$bundled = dirname(__DIR__) . "/assets/illustrations/$slug.png";
+if (is_file($bundled) && filesize($bundled) > 1024) {
+    serve_png($bundled);
+}
+// 2. Bundled cutout (background-removed photo, fallback for species
+//    without an illustration).
+$cutout = dirname(__DIR__) . "/assets/cutouts/$slug.png";
+if (is_file($cutout) && filesize($cutout) > 1024) {
+    serve_png($cutout);
+}
+
+// 3. Dynamic cache from a previous Wikipedia + rembg run.
+$cacheDir = getenv('HOME') . '/BirdSongs/Extracted/cutouts';
+$cachePath = "$cacheDir/$slug.png";
 if (is_file($cachePath) && filesize($cachePath) > 1024) {
     serve_png($cachePath);
 }
 
-// Make sure the cache dir exists (first run).
+// 4. Fresh Wikipedia fetch + rembg. Skipped if rembg-cli isn't on
+//    PATH — the resolver simply returns a 404 in that case rather
+//    than burning a Wikipedia request we can't use.
+$rembg = '/usr/local/bin/rembg-cli';
+if (!is_executable($rembg)) {
+    http_response_code(404);
+    echo 'no illustration bundled for ' . htmlspecialchars($sci) . ' (install rembg-cli to enable Wikipedia fallback)';
+    exit;
+}
+
 if (!is_dir($cacheDir)) @mkdir($cacheDir, 0755, true);
 
-// Resolve a source image URL. Wikipedia summary first (free, fast,
-// no auth, has a clean originalimage for most birds).
+// Wikipedia's REST API asks for a contact-able identifier. Override
+// via the AV_USER_AGENT env var (set in /etc/php/*/fpm/pool.d/www.conf
+// or your shell) if your install hammers their endpoint at scale.
+$ua = getenv('AV_USER_AGENT') ?: 'AvianVisitors/1.0 (+https://github.com/Twarner491/AvianVisitors)';
 $ctx = stream_context_create([
-    'http' => [
-        'header'  => "User-Agent: apartment-birds/1.0 (twarner491@gmail.com)\r\n",
-        'timeout' => 12,
-    ],
+    'http' => ['header' => "User-Agent: $ua\r\n", 'timeout' => 12],
 ]);
 $wpUrl = 'https://en.wikipedia.org/api/rest_v1/page/summary/' . rawurlencode($sci);
 $wpJson = @file_get_contents($wpUrl, false, $ctx);
@@ -62,13 +86,20 @@ if ($wpJson !== false) {
     $j = json_decode($wpJson, true);
     $srcUrl = $j['originalimage']['source'] ?? $j['thumbnail']['source'] ?? null;
 }
+// Defensive: only follow URLs on Wikimedia / Wikipedia hosts so a
+// poisoned summary endpoint can't redirect us to arbitrary servers.
+if ($srcUrl !== null) {
+    $host = parse_url((string)$srcUrl, PHP_URL_HOST) ?: '';
+    if (!preg_match('/(?:^|\.)(?:wikimedia\.org|wikipedia\.org)$/i', $host)) {
+        $srcUrl = null;
+    }
+}
 if (!$srcUrl) {
     http_response_code(404);
-    echo 'no upstream image for ' . htmlspecialchars($sci);
+    echo 'no Wikipedia photo for ' . htmlspecialchars($sci);
     exit;
 }
 
-// Download the source image.
 $imgBytes = @file_get_contents($srcUrl, false, $ctx);
 if (!$imgBytes || strlen($imgBytes) < 1024) {
     http_response_code(503);
@@ -76,16 +107,19 @@ if (!$imgBytes || strlen($imgBytes) < 1024) {
     exit;
 }
 
-// Run rembg via the wrapper. We use temp files because rembg's CLI prefers
-// real paths. u2netp is the lightweight model (~50MB peak RAM) — important
-// on the Pi 3B+ (1GB total RAM). Call with --post-process-mask to clean
-// up edges.
-$tmpIn  = tempnam('/tmp', 'rembg-in-')  . '.jpg';
-$tmpOut = tempnam('/tmp', 'rembg-out-') . '.png';
+// rembg via the wrapper. u2netp = lightweight model (~50MB peak RAM —
+// matters on the Pi 3B+). Temp files because rembg's CLI prefers
+// real paths.
+$tmpInBase  = tempnam(sys_get_temp_dir(), 'rembg-in-');
+$tmpOutBase = tempnam(sys_get_temp_dir(), 'rembg-out-');
+@unlink($tmpInBase); @unlink($tmpOutBase);
+$tmpIn  = $tmpInBase  . '.jpg';
+$tmpOut = $tmpOutBase . '.png';
 file_put_contents($tmpIn, $imgBytes);
 
 $cmd = sprintf(
-    '/usr/local/bin/rembg-cli i -m u2netp -ppm %s %s 2>&1',
+    '%s i -m u2netp -ppm %s %s 2>&1',
+    escapeshellarg($rembg),
     escapeshellarg($tmpIn),
     escapeshellarg($tmpOut)
 );
@@ -96,13 +130,13 @@ if (!is_file($tmpOut) || filesize($tmpOut) < 1024) {
     @unlink($tmpOut);
     http_response_code(500);
     header('Content-Type: text/plain');
-    echo "rembg failed:\n" . ($out ?? '(no output)');
+    echo "rembg failed (see your Pi's logs for details)";
+    error_log("rembg failed for $sci: " . ($out ?? '(no output)'));
     exit;
 }
 
-// 1. Alpha-crop to bounding box so each PNG == the bird's actual shape
-//    (no transparent padding around it). Lets the layout pack tiles tight.
-// 2. Resize down to a max edge of 800px so the cache stays small.
+// Tight-crop to the bird's bounding box + downscale to 800px max edge
+// so cache stays small.
 $im = @imagecreatefrompng($tmpOut);
 if ($im !== false) {
     $cropped = @imagecropauto($im, IMG_CROP_TRANSPARENT);
@@ -128,7 +162,8 @@ if ($im !== false) {
     imagedestroy($im);
 }
 
-// Cache + serve.
-copy($tmpOut, $cachePath);
-@unlink($tmpOut);
+// Atomic install: rename is atomic on the same filesystem, so any
+// concurrent reader either sees the old cached file or the new one,
+// never a half-written PNG.
+@rename($tmpOut, $cachePath);
 serve_png($cachePath);

avian/api/recording.php

@@ -1,16 +1,15 @@
 <?php
-// /home/monalisa/BirdSongs/Extracted/recording.php — serves the most-recent
-// detection mp3 for a given scientific name. Called by the Cloudflare Worker
-// at /api/recording?sci=<name>.
+// AvianVisitors — serves the most-recent detection mp3 for a given
+// scientific name. Called by the collage detail modal at
+// /avian/api/recording.php?sci=<name>.
 //
 // BirdNET-Pi writes audio + spectrograms to
-//   ~/BirdSongs/Extracted/By_Date/YYYY-MM-DD/<Common_Name>/<base>.mp3
-// (with a matching .png next to it).  Common_Name is the SPACE-stripped
-// English common name (e.g. "Anna's_Hummingbird"), NOT the scientific name.
-// We resolve sci → common via the same lookup the Pi's web UI uses
-// (birds.json under /scripts/) and then walk the directory tree newest-first.
+//   $HOME/BirdSongs/Extracted/By_Date/YYYY-MM-DD/<Common_Name>/<base>.mp3
+// (with a matching .png next to it). Common_Name is the SPACE-stripped
+// English common name (e.g. "Anna's_Hummingbird"), NOT the scientific
+// name. We resolve sci → common via birds.json under BirdNET-Pi/scripts/
+// and walk the directory tree newest-first.
 //
-// Auth: the Caddy site block 403s anything missing X-BirdNET-Proxy-Token.
 
 declare(strict_types=1);
 
@@ -33,7 +32,7 @@ if ($sci !== '' && !preg_match('/^[A-Za-z]{2,40}(?:[ ][a-z]{2,40}){1,3}$/', $sci
     exit;
 }
 
-$BY_DATE = '/home/monalisa/BirdSongs/Extracted/By_Date';
+$BY_DATE = getenv('HOME') . '/BirdSongs/Extracted/By_Date';
 
 // ---- Direct-by-file lookup ----
 // Used by the atlas detail modal to play any past recording.
@@ -95,14 +94,14 @@ if ($file !== '') {
     exit;
 }
 $BIRDS_JSON_CANDIDATES = [
-    '/home/monalisa/BirdNET-Pi/scripts/birds.json',
-    '/home/monalisa/BirdNET-Pi/model/labels.txt',
+    getenv('HOME') . '/BirdNET-Pi/scripts/birds.json',
+    getenv('HOME') . '/BirdNET-Pi/model/labels.txt',
 ];
 
 // ---- Resolve scientific name → common name (with underscores) ----
 function resolve_common(string $sci): ?string {
     // Try birds.json first (preferred — has clean sci/com pairs).
-    foreach (['/home/monalisa/BirdNET-Pi/scripts/birds.json'] as $f) {
+    foreach ([getenv('HOME') . '/BirdNET-Pi/scripts/birds.json'] as $f) {
         if (is_readable($f)) {
             $list = json_decode((string)file_get_contents($f), true);
             if (is_array($list)) {
@@ -118,7 +117,7 @@ function resolve_common(string $sci): ?string {
         }
     }
     // Fallback: labels.txt has "<sci>_<com>" or "<sci>, <com>" per line.
-    $labels = '/home/monalisa/BirdNET-Pi/model/labels.txt';
+    $labels = getenv('HOME') . '/BirdNET-Pi/model/labels.txt';
     if (is_readable($labels)) {
         foreach (file($labels, FILE_IGNORE_NEW_LINES) as $line) {
             if (strpos($line, '_') !== false) {

avian/api/spectrogram.php

@@ -1,8 +1,8 @@
 <?php
-// /home/monalisa/BirdSongs/Extracted/spectrogram.php — serves the
-// spectrogram PNG that BirdNET-Pi generates alongside each detection mp3.
-// Same lookup logic as recording.php (find the matching file under
-// By_Date/<date>/<Common_Name>/) — just .png instead of .mp3.
+// AvianVisitors — serves the spectrogram PNG that BirdNET-Pi generates
+// alongside each detection mp3. Same lookup logic as recording.php (find
+// the matching file under By_Date/<date>/<Common_Name>/) — just .png
+// instead of .mp3.
 //
 // Endpoints:
 //   ?sci=<sci_name>            → newest spectrogram for that species
@@ -31,7 +31,7 @@ if ($sci !== '' && !preg_match('/^[A-Za-z]{2,40}(?:[ ][a-z]{2,40}){1,3}$/', $sci
     exit;
 }
 
-$BY_DATE = '/home/monalisa/BirdSongs/Extracted/By_Date';
+$BY_DATE = getenv('HOME') . '/BirdSongs/Extracted/By_Date';
 
 // ---- Direct-by-file lookup ----
 // BirdNET-Pi writes <base>.mp3 and <base>.png next to each other under
@@ -93,7 +93,7 @@ if ($file !== '') {
 }
 
 function resolve_common(string $sci): ?string {
-    $f = '/home/monalisa/BirdNET-Pi/scripts/birds.json';
+    $f = getenv('HOME') . '/BirdNET-Pi/scripts/birds.json';
     if (is_readable($f)) {
         $list = json_decode((string)file_get_contents($f), true);
         if (is_array($list)) {
@@ -107,7 +107,7 @@ function resolve_common(string $sci): ?string {
             }
         }
     }
-    $labels = '/home/monalisa/BirdNET-Pi/model/labels.txt';
+    $labels = getenv('HOME') . '/BirdNET-Pi/model/labels.txt';
     if (is_readable($labels)) {
         foreach (file($labels, FILE_IGNORE_NEW_LINES) as $line) {
             if (strpos($line, '_') !== false) {