feat: introduce server-thin client architecture for mainline.py on ESP32 with ntfy integration and update hardware documentation to reflect this design.

klubhaus-doorbell-hardware.md

@@ -139,25 +139,27 @@ Things that are plausibly relevant but entirely absent from the codebase:
 
 ## Questions for board owner
 
-I'm looking at porting [mainline.py](mainline.py) — a scrolling terminal news/poetry stream with OTF-font rendering, RSS feeds, ANSI gradients, and glitch effects — to run on one of these boards. To figure out the right approach I need a few things only you can answer:
+I'm looking at displaying [mainline.py](mainline.py) — a scrolling news/poetry consciousness stream — on one of these boards. The plan ("Mainline Renderer + ntfy Message Queue for ESP32") uses a **server + thin client** architecture: a Python server pre-renders headlines and serves them via HTTP; the ESP32 just displays pre-rendered bitmaps, applies gradient/glitch effects locally, and polls `ntfy.sh/klubhaus_terminal_mainline` for messages that interrupt the news scroll.
 
-### 1. Which board should I target?
+To build this I need the following from you:
 
-The three boards have very different constraints:
+### 1. Which board?
+
+With a renderer server doing the heavy lifting, all three boards are viable — but the experience differs:
 
 | | Board 1 (2.8″) | Board 2 (4.0″) | Board 3 (4.3″) |
 |---|---|---|---|
 | Resolution | 320 × 240 | 320 × 480 | 800 × 480 |
-| Display bus | SPI (40 MHz) | SPI (40 MHz) | RGB parallel (14 MHz pclk) |
-| Flash | 4 MB | 4 MB | 16 MB |
-| PSRAM | unknown | unknown | 8 MB |
-| Full-screen redraw | ~60 ms+ | ~120 ms+ | near-instant (framebuffer) |
+| Headline buffer (120 items) | ~77 KB | ~77 KB | ~192 KB |
+| Firehose mode | no (too narrow) | no (SPI too slow) | yes |
+| Smooth scroll at 20 FPS | yes (partial updates) | tight (partial updates mandatory) | yes (framebuffer) |
+| Flash for caching | 4 MB (tight) | 4 MB (tight) | 16 MB (9 MB FAT partition) |
 
-Board 3 is the only one with enough RAM and display bandwidth for smooth scrolling with many headlines buffered. Boards 1 & 2 would need aggressive feature cuts. **Which board do you want this on?**
+Board 3 is the most capable. Boards 1 & 2 work but lose firehose mode and need careful partial-update rendering. **Which board do you want this on?**
 
-### 2. PSRAM on your ESP32-32E boards
+### 2. PSRAM on Boards 1 & 2
 
-The build flags say `-DBOARD_HAS_PSRAM` but I can't tell the capacity. Can you check? Easiest way:
+The build flags say `-DBOARD_HAS_PSRAM` but I can't confirm the capacity. Can you check?
 
 ```
 // Add to setup() temporarily:
@@ -165,30 +167,71 @@ Serial.printf("PSRAM size: %d bytes\n", ESP.getPsramSize());
 Serial.printf("Free PSRAM: %d bytes\n", ESP.getFreePsram());
 ```
 
-If PSRAM is 0 on Boards 1 or 2, those boards can only hold a handful of headlines in 520 KB SRAM (WiFi + TLS eat most of it).
+If PSRAM is 0, the board can only buffer ~20 headlines in a ring buffer instead of the full ~120 set. (Board 3's 8 MB PSRAM is confirmed — this only matters if you pick Board 1 or 2.)
 
-### 3. Feature priorities
+### 3. Network & server hosting
 
-mainline.py does a lot of things that don't map directly to an ESP32 + TFT. Which of these matter to you?
+The renderer server (`serve.py`) needs Python 3 + Pillow, internet access (for RSS feeds), and network access to the ESP32.
 
-- **RSS headline scrolling** — the core experience. How many feeds? All ~25, or a curated subset?
-- **OTF font rendering** — mainline uses Pillow to rasterize a custom `.otf` font into half-block characters. On ESP32 this would become either bitmap fonts or a pre-rendered glyph atlas baked into flash. Is the specific font important, or is the aesthetic (large, stylized text) what matters?
-- **Left-to-right color gradient** — the white-hot → green → black fade. Easy to replicate in RGB565 on the TFT. Keep?
-- **Glitch / noise effects** — the ░▒▓█ and katakana rain. Keep?
-- **Mic-reactive glitch intensity** — none of these boards have a microphone. Drop entirely, or substitute with something else (e.g. touch-reactive, or time-of-day reactive)?
-- **Auto-translation** — mainline translates headlines for region-specific sources via Google Translate. This requires HTTPS calls that are expensive on ESP32 (~40–50 KB RAM per TLS connection). Keep, pre-translate on a server, or drop?
-- **Poetry mode** — fetches full Gutenberg texts. These are large (100+ KB each). Cache to SD card, trim to a small set, or drop?
-- **Content filtering** — the sports/vapid regex filter. Trivial to keep.
-- **Boot sequence animation** — the typewriter-style boot log. Keep?
+- **Where will the server run?** Raspberry Pi, NAS, always-on desktop, cloud VM?
+- **Same LAN as the ESP32?** If yes, the ESP32 can use plain HTTP (no TLS overhead). If remote, we'd need HTTPS (~40–50 KB RAM per connection).
+- **Server discovery:** Hardcoded IP in `secrets.h`, mDNS (`mainline.local`), or a DNS name?
+- **WiFi credentials:** Use the existing multi-network setup from the doorbell firmware, or a specific SSID?
 
-### 4. Network environment
+### 4. ESP32 client repo
 
-- Will the board be on a WiFi network that can reach the public internet (RSS feeds, Google Translate, ntfy.sh)?
-- Is there a preferred SSID / network, or should it use the existing multi-network setup from the doorbell firmware?
+The ESP32 sketch reuses `NetManager`, `IDisplayDriver`, and vendored display libraries from klubhaus-doorbell. Two options:
 
-### 5. SD card availability
+- **klubhaus-doorbell repo** — natural fit as a new board target (`boards/esp32-mainline/`). Requires push access or a PR.
+- **mainline repo** — under `firmware/esp32-mainline/` with a vendored copy of KlubhausCore. Self-contained but duplicates shared code.
 
-All three boards have TF card slots but the doorbell firmware doesn't use them. A microSD card would be useful for caching fonts, pre-rendered glyph atlases, or translated headline buffers. **Is there an SD card in the board you'd want to target?**
+**Which repo should host the ESP32 client?**
+
+### 5. Display features
+
+With the renderer handling RSS fetching, translation, content filtering, and font rendering server-side, the remaining feature questions are about what the ESP32 displays locally:
+
+- **Left-to-right color gradient** — the white-hot → green → black fade, applied per-pixel on the ESP32. Keep?
+- **Glitch / noise effects** — random block characters and katakana rain between headlines. Keep?
+- **Glitch reactivity** — mainline.py uses a microphone (none on these boards). Substitute with touch-reactive, time-of-day reactive, or just random?
+- **Firehose mode** — dense data ticker at the bottom (only viable on Board 3). Want it?
+- **Boot sequence animation** — typewriter-style status log during startup. Keep?
+- **Poetry mode** — the server can serve Gutenberg text instead of RSS. Want both modes available, or just news?
+
+### 6. ntfy message queue
+
+The ESP32 polls `ntfy.sh/klubhaus_terminal_mainline` directly for messages that interrupt the news scroll.
+
+- **Is `klubhaus_terminal_mainline` the right topic name?** Or match the doorbell convention (`ALERT_klubhaus_topic`, etc.)?
+- **Who sends messages?** Just you (manual `curl`), a bot, other people?
+- **Display duration:** How long before auto-dismiss? The doorbell uses 120s for alerts. 30s for terminal messages? Touch-to-dismiss regardless?
+- **Priority levels?** ntfy supports 1–5. Should high-priority messages turn on the backlight if the display is OFF, or show in a different color?
+- **Message history on boot?** Show recent messages from the topic, or only messages arriving while running? The doorbell uses `?since=20s`. You might want `?since=5m` for a message board feel.
+
+### 7. Server-offline fallback
+
+If the renderer server goes down, what should the ESP32 do?
+
+- **A.** Show last cached headlines indefinitely (ntfy messages still work independently).
+- **B.** Show a "no signal" screen after a timeout, keep polling.
+- **C.** Fall back to ntfy messages + clock only.
+
+### 8. Scroll and layout
+
+- **Vertical scroll (like mainline.py)?** Or horizontal marquee?
+- **Speed:** mainline.py uses ~3.75s per headline. Same pace, or different for ambient display?
+- **Font height:** The server pre-renders at a configurable pixel height. On Board 3 (480px tall), 32–48px headlines would match mainline.py's feel. On Boards 1/2, 16–24px. What looks right to you?
+
+### 9. Font and aesthetics
+
+mainline.py uses `CSBishopDrawn-Italic.otf`. The server renders bitmaps with this font, so the ESP32 never needs the font file itself.
+
+- **Can you provide this font to whoever hosts the server?** Or should it fall back to a freely available alternative?
+- **Any license concern** with serving the rendered bitmaps (not the font file) over HTTP?
+
+### 10. SD card
+
+All three boards have TF card slots (unused by the doorbell firmware). An SD card would let the ESP32 cache headline bitmaps for instant boot and offline operation. **Is there an SD card in the board you'd pick?**
 
 ---
 
@@ -240,9 +283,8 @@ void loop() {
 
 Board 2 at 20 FPS with a full-screen redraw each frame would have essentially zero headroom. Partial updates (dirty-rect tracking) would be mandatory.
 
-### Updated feature priority question
+### Port-relevant implications
 
-Given the new features, the feature priority question (§3 above) gains two more items:
-
-- **Firehose mode** — the dense data ticker at the bottom. Want it? It's only viable on Board 3.
-- **Headline caching** — persist headlines to flash/SD for instant boot and offline fallback. Recommended regardless of board, but storage medium depends on board choice.
+- **Firehose mode** is only practical on Board 3 (see §5 in Questions). On Boards 1 & 2, consider a single-line ticker instead.
+- **Headline caching** maps to the ESP32 storing bitmap data from the server to flash/SD for instant boot and offline fallback (see §10 in Questions).
+- **20 FPS frame timing** maps cleanly to a `millis()`-based ESP32 main loop. Board 2 would have zero headroom without partial updates.