AvianVisitors hardening pass — install via fork's newinstaller
Restructures the install so users run ONE command. The fork's newinstaller.sh clones this repo (instead of upstream Nachtzuster) and runs BirdNET-Pi's installer. install_services.sh symlinks the avian/ overlay into the Caddy site root, so http://birdnet.local/avian/ comes up after reboot with no Caddy config needed. Reviewer-flagged fixes: Frontend (avian/frontend/apt.js) - Switch all API paths from /api/*.json (which didn't route) to relative action-based URLs (./api/birdnet-api.php?action=recent etc.). - Fetch ./masks.json + ./dims.json relatively (were /avian/*.json). - Replace every innerHTML user-content sink with createElement + textContent + setAttribute — species labels are user-editable in BirdNET-Pi, so untrusted. - Add esc() helper for the few remaining innerHTML attribute contexts. - fetchJson throws on non-200 (was silently parsing 404 HTML as JSON). - Defensive null-check in loadMask if MASKS hasn't loaded. Backend (avian/api/*.php) - birdnet-api.php now derives DB_PATH from dirname(__DIR__, 3) so any BirdNET-Pi install user works (no hardcoded /home/birdnet). - recording/spectrogram/cutout use getenv('HOME') for the same reason. - cutout.php reworked as the canonical image resolver: bundled illustration → bundled cutout → cached rembg → fresh Wikipedia+rembg, with the binomial regex guard and atomic rename on cache write. Wikipedia URL host pinned to wikimedia.org / wikipedia.org to block SSRF. - Stale 'X-BirdNET-Proxy-Token' auth claims removed everywhere. Pregen (avian/scripts/pregen.py) - Gemini key moved from URL query to x-goog-api-key header. - Model bumped to gemini-2.5-flash-image-preview. - Bounded retry on 429 + 5xx with Retry-After honoured. - Default --sleep raised from 1s to 4s (under free-tier RPM). - Unified parser handles |/_/comma in all three input modes. - responseModalities now [TEXT, IMAGE] (was IMAGE-only). - --poses validated against POSES keys. - ASCII [ok]/[fail] markers (was ✓/✗ — Windows console crash). - Surfaces finishReason + blockReason on safety blocks. Prompt template - 'Two placeholders' typo → three. - 'warm paper' / 'transparent background' contradiction resolved. Forwarding - mqtt-bridge.py: paho-mqtt 2.x CallbackAPIVersion compat shim. - avian-mqtt.service: dropped broken %i, set User=birdnet + %h path. - forwarding/README.md uses the actual action-based URLs. README - One-line install via curl … newinstaller.sh. - No more 'install BirdNET-Pi separately' step (our fork does it). - License section uses absolute github.com links. - bird.onethreenine.net replaces the not-yet-existent project writeup. - docs/thumb.png replaced with the 24-bird Twitter capture. Removed - avian/caddy/ and avian/scripts/install.sh — no longer needed; the overlay symlink in install_services.sh handles everything.
This commit is contained in:
Twarner491
+77
-42
@@ -1,20 +1,17 @@
|
||||
<?php
|
||||
// /home/monalisa/BirdSongs/Extracted/cutout.php — dynamic background-removed
|
||||
// bird-photo facade. Called by the bird.onethreenine.net Cloudflare Worker
|
||||
// (which fronts /api/img?sci=<name>).
|
||||
// AvianVisitors — bird image resolver.
|
||||
//
|
||||
// Lives in the Caddy file-server root so the BirdNET-Pi auto-update doesn't
|
||||
// touch it. Each new species hits this once: download Wikipedia/Macaulay
|
||||
// image → rembg via /usr/local/bin/rembg-cli → cache transparent PNG to
|
||||
// disk under cutouts/. Subsequent requests are instant readfile() from cache.
|
||||
// Lookup chain for /avian/api/cutout.php?sci=Calypte+anna:
|
||||
// 1. ../assets/illustrations/<slug>.png (450+ bundled kachō-e renders)
|
||||
// 2. ../assets/cutouts/<slug>.png (background-removed photo)
|
||||
// 3. cached rembg of a Wikipedia photo at $HOME/BirdSongs/Extracted/cutouts/
|
||||
// 4. fresh Wikipedia → rembg → cache (skipped gracefully if rembg unset)
|
||||
//
|
||||
// Auth: the Caddy site block 403s anything missing X-BirdNET-Proxy-Token at
|
||||
// the top, so this script inherits the Worker-only access guarantee.
|
||||
// The frontend's <img src> points here for every species — bundled
|
||||
// hits return instantly; cold misses fall through to the dynamic path.
|
||||
//
|
||||
// Pre-reqs (run once via /install/rembg-setup.sh):
|
||||
// - /home/monalisa/rembg-env/bin/rembg (Python venv)
|
||||
// - /usr/local/bin/rembg-cli (wrapper)
|
||||
// - /home/monalisa/BirdSongs/Extracted/cutouts/ (cache dir, monalisa-owned)
|
||||
// Default LAN deploy ships without auth. To expose publicly, gate
|
||||
// /avian/api/* with basic_auth in your Caddyfile — see avian/forwarding/.
|
||||
|
||||
declare(strict_types=1);
|
||||
|
||||
@@ -24,36 +21,63 @@ if ($sci === '') {
|
||||
echo 'sci required';
|
||||
exit;
|
||||
}
|
||||
// Binomial / trinomial pattern. Rejects path-traversal payloads and
|
||||
// junk before any filesystem or upstream lookup.
|
||||
if (!preg_match('/^[A-Za-z]{2,40}(?:[ ][a-z]{2,40}){1,3}$/', $sci)) {
|
||||
http_response_code(400);
|
||||
echo 'invalid sci';
|
||||
exit;
|
||||
}
|
||||
|
||||
// Slugify scientific name for the cache filename.
|
||||
// Slugify scientific name for filename + cache key.
|
||||
$slug = preg_replace('/[^a-z0-9]+/', '-', strtolower($sci));
|
||||
$slug = trim((string)$slug, '-');
|
||||
$cacheDir = '/home/monalisa/BirdSongs/Extracted/cutouts';
|
||||
$cachePath = "$cacheDir/$slug.png";
|
||||
|
||||
function serve_png(string $path): void {
|
||||
header('Content-Type: image/png');
|
||||
header('Cache-Control: public, max-age=2592000');
|
||||
header('Content-Length: ' . filesize($path));
|
||||
header('Cache-Control: public, max-age=86400');
|
||||
header('Content-Length: ' . (string)filesize($path));
|
||||
readfile($path);
|
||||
exit;
|
||||
}
|
||||
|
||||
// Cache hit — short-circuit.
|
||||
// 1. Bundled illustration (the kachō-e PNG the repo ships with).
|
||||
$bundled = dirname(__DIR__) . "/assets/illustrations/$slug.png";
|
||||
if (is_file($bundled) && filesize($bundled) > 1024) {
|
||||
serve_png($bundled);
|
||||
}
|
||||
// 2. Bundled cutout (background-removed photo, fallback for species
|
||||
// without an illustration).
|
||||
$cutout = dirname(__DIR__) . "/assets/cutouts/$slug.png";
|
||||
if (is_file($cutout) && filesize($cutout) > 1024) {
|
||||
serve_png($cutout);
|
||||
}
|
||||
|
||||
// 3. Dynamic cache from a previous Wikipedia + rembg run.
|
||||
$cacheDir = getenv('HOME') . '/BirdSongs/Extracted/cutouts';
|
||||
$cachePath = "$cacheDir/$slug.png";
|
||||
if (is_file($cachePath) && filesize($cachePath) > 1024) {
|
||||
serve_png($cachePath);
|
||||
}
|
||||
|
||||
// Make sure the cache dir exists (first run).
|
||||
// 4. Fresh Wikipedia fetch + rembg. Skipped if rembg-cli isn't on
|
||||
// PATH — the resolver simply returns a 404 in that case rather
|
||||
// than burning a Wikipedia request we can't use.
|
||||
$rembg = '/usr/local/bin/rembg-cli';
|
||||
if (!is_executable($rembg)) {
|
||||
http_response_code(404);
|
||||
echo 'no illustration bundled for ' . htmlspecialchars($sci) . ' (install rembg-cli to enable Wikipedia fallback)';
|
||||
exit;
|
||||
}
|
||||
|
||||
if (!is_dir($cacheDir)) @mkdir($cacheDir, 0755, true);
|
||||
|
||||
// Resolve a source image URL. Wikipedia summary first (free, fast,
|
||||
// no auth, has a clean originalimage for most birds).
|
||||
// Wikipedia's REST API asks for a contact-able identifier. Override
|
||||
// via the AV_USER_AGENT env var (set in /etc/php/*/fpm/pool.d/www.conf
|
||||
// or your shell) if your install hammers their endpoint at scale.
|
||||
$ua = getenv('AV_USER_AGENT') ?: 'AvianVisitors/1.0 (+https://github.com/Twarner491/AvianVisitors)';
|
||||
$ctx = stream_context_create([
|
||||
'http' => [
|
||||
'header' => "User-Agent: apartment-birds/1.0 (twarner491@gmail.com)\r\n",
|
||||
'timeout' => 12,
|
||||
],
|
||||
'http' => ['header' => "User-Agent: $ua\r\n", 'timeout' => 12],
|
||||
]);
|
||||
$wpUrl = 'https://en.wikipedia.org/api/rest_v1/page/summary/' . rawurlencode($sci);
|
||||
$wpJson = @file_get_contents($wpUrl, false, $ctx);
|
||||
@@ -62,13 +86,20 @@ if ($wpJson !== false) {
|
||||
$j = json_decode($wpJson, true);
|
||||
$srcUrl = $j['originalimage']['source'] ?? $j['thumbnail']['source'] ?? null;
|
||||
}
|
||||
// Defensive: only follow URLs on Wikimedia / Wikipedia hosts so a
|
||||
// poisoned summary endpoint can't redirect us to arbitrary servers.
|
||||
if ($srcUrl !== null) {
|
||||
$host = parse_url((string)$srcUrl, PHP_URL_HOST) ?: '';
|
||||
if (!preg_match('/(?:^|\.)(?:wikimedia\.org|wikipedia\.org)$/i', $host)) {
|
||||
$srcUrl = null;
|
||||
}
|
||||
}
|
||||
if (!$srcUrl) {
|
||||
http_response_code(404);
|
||||
echo 'no upstream image for ' . htmlspecialchars($sci);
|
||||
echo 'no Wikipedia photo for ' . htmlspecialchars($sci);
|
||||
exit;
|
||||
}
|
||||
|
||||
// Download the source image.
|
||||
$imgBytes = @file_get_contents($srcUrl, false, $ctx);
|
||||
if (!$imgBytes || strlen($imgBytes) < 1024) {
|
||||
http_response_code(503);
|
||||
@@ -76,16 +107,19 @@ if (!$imgBytes || strlen($imgBytes) < 1024) {
|
||||
exit;
|
||||
}
|
||||
|
||||
// Run rembg via the wrapper. We use temp files because rembg's CLI prefers
|
||||
// real paths. u2netp is the lightweight model (~50MB peak RAM) — important
|
||||
// on the Pi 3B+ (1GB total RAM). Call with --post-process-mask to clean
|
||||
// up edges.
|
||||
$tmpIn = tempnam('/tmp', 'rembg-in-') . '.jpg';
|
||||
$tmpOut = tempnam('/tmp', 'rembg-out-') . '.png';
|
||||
// rembg via the wrapper. u2netp = lightweight model (~50MB peak RAM —
|
||||
// matters on the Pi 3B+). Temp files because rembg's CLI prefers
|
||||
// real paths.
|
||||
$tmpInBase = tempnam(sys_get_temp_dir(), 'rembg-in-');
|
||||
$tmpOutBase = tempnam(sys_get_temp_dir(), 'rembg-out-');
|
||||
@unlink($tmpInBase); @unlink($tmpOutBase);
|
||||
$tmpIn = $tmpInBase . '.jpg';
|
||||
$tmpOut = $tmpOutBase . '.png';
|
||||
file_put_contents($tmpIn, $imgBytes);
|
||||
|
||||
$cmd = sprintf(
|
||||
'/usr/local/bin/rembg-cli i -m u2netp -ppm %s %s 2>&1',
|
||||
'%s i -m u2netp -ppm %s %s 2>&1',
|
||||
escapeshellarg($rembg),
|
||||
escapeshellarg($tmpIn),
|
||||
escapeshellarg($tmpOut)
|
||||
);
|
||||
@@ -96,13 +130,13 @@ if (!is_file($tmpOut) || filesize($tmpOut) < 1024) {
|
||||
@unlink($tmpOut);
|
||||
http_response_code(500);
|
||||
header('Content-Type: text/plain');
|
||||
echo "rembg failed:\n" . ($out ?? '(no output)');
|
||||
echo "rembg failed (see your Pi's logs for details)";
|
||||
error_log("rembg failed for $sci: " . ($out ?? '(no output)'));
|
||||
exit;
|
||||
}
|
||||
|
||||
// 1. Alpha-crop to bounding box so each PNG == the bird's actual shape
|
||||
// (no transparent padding around it). Lets the layout pack tiles tight.
|
||||
// 2. Resize down to a max edge of 800px so the cache stays small.
|
||||
// Tight-crop to the bird's bounding box + downscale to 800px max edge
|
||||
// so cache stays small.
|
||||
$im = @imagecreatefrompng($tmpOut);
|
||||
if ($im !== false) {
|
||||
$cropped = @imagecropauto($im, IMG_CROP_TRANSPARENT);
|
||||
@@ -128,7 +162,8 @@ if ($im !== false) {
|
||||
imagedestroy($im);
|
||||
}
|
||||
|
||||
// Cache + serve.
|
||||
copy($tmpOut, $cachePath);
|
||||
@unlink($tmpOut);
|
||||
// Atomic install: rename is atomic on the same filesystem, so any
|
||||
// concurrent reader either sees the old cached file or the new one,
|
||||
// never a half-written PNG.
|
||||
@rename($tmpOut, $cachePath);
|
||||
serve_png($cachePath);
|
||||
|
||||
Reference in New Issue
Block a user