refactor: Implement fixed frame rate and precise timing for the scrolling animation loop.

Reviewed-on: #2

.gitignore

@@ -0,0 +1,4 @@
+__pycache__/
+*.pyc
+.mainline_venv/
+.mainline_cache_*.json

klubhaus-doorbell-hardware.md

@@ -0,0 +1,191 @@
+# Klubhaus Doorbell — Hardware Spec Sheet
+
+Derived from source code analysis of [klubhaus-doorbell](https://git.notsosm.art/david/klubhaus-doorbell.git) and manufacturer datasheets.
+
+The project targets **three** ESP32-based development boards, each with an integrated TFT display and touch input. All three are all-in-one "ESP32 + screen" modules (not a bare Arduino with a separate breakout).
+
+---
+
+## Board 1 — ESP32-32E (2.8″ ILI9341)
+
+| Attribute | Value |
+|---|---|
+| **Manufacturer** | Hosyond (likely) |
+| **Module** | ESP32-32E (ESP32-WROOM-32 family) |
+| **SoC** | ESP32-D0WD-V3, Xtensa dual-core 32-bit LX6 |
+| **Max clock** | 240 MHz |
+| **SRAM** | 520 KB |
+| **ROM** | 448 KB |
+| **Flash** | 4 MB (external QSPI) — per FQBN `FlashSize=4M` |
+| **PSRAM** | Flagged in build (`-DBOARD_HAS_PSRAM`) — likely present but unconfirmed capacity |
+| **WiFi** | 2.4 GHz 802.11 b/g/n |
+| **Bluetooth** | v4.2 BR/EDR + BLE |
+| **Display driver** | ILI9341 |
+| **Display size** | ~2.8″ (inferred from `.crushmemory` note "original 2.8″ ILI9341") |
+| **Resolution** | 320 × 240 (landscape rotation 1) |
+| **Display interface** | 4-line SPI |
+| **Color depth** | 65K (RGB565) |
+| **Touch** | XPT2046 resistive (SPI) |
+| **Backlight GPIO** | 22 |
+| **SPI pins** | MOSI 23, SCLK 18, CS 5, DC 27, RST 33 |
+| **Touch CS** | GPIO 14 |
+| **SPI clock** | 40 MHz (display), 20 MHz (read), 2.5 MHz (touch) |
+| **Serial baud** | 115200 |
+| **USB** | Type-C (programming / power) |
+| **Display library** | TFT_eSPI (vendored) |
+
+---
+
+## Board 2 — ESP32-32E-4″ (Hosyond 4.0″ ST7796)
+
+| Attribute | Value |
+|---|---|
+| **Manufacturer** | Hosyond |
+| **Module** | ESP32-32E |
+| **SoC** | ESP32-D0WD-V3, Xtensa dual-core 32-bit LX6 |
+| **Max clock** | 240 MHz |
+| **SRAM** | 520 KB |
+| **ROM** | 448 KB |
+| **Flash** | 4 MB (external QSPI) — per FQBN `FlashSize=4M` |
+| **PSRAM** | Flagged in build (`-DBOARD_HAS_PSRAM`) — likely present but unconfirmed capacity |
+| **WiFi** | 2.4 GHz 802.11 b/g/n |
+| **Bluetooth** | v4.2 BR/EDR + BLE |
+| **Display driver** | ST7796S |
+| **Display size** | 4.0″ |
+| **Resolution** | 320 × 480 (landscape rotation 1) |
+| **Display interface** | 4-line SPI |
+| **Color depth** | 262K (RGB666) per manufacturer; firmware uses RGB565 |
+| **Touch** | XPT2046 resistive (SPI) |
+| **Backlight GPIO** | 27 (active HIGH) |
+| **SPI pins** | MISO 12, MOSI 13, SCLK 14, CS 15, DC 2, RST tied to EN |
+| **Touch CS / IRQ** | GPIO 33 / GPIO 36 |
+| **SPI clock** | 40 MHz (display), 20 MHz (read), 2.5 MHz (touch) |
+| **Serial baud** | 115200 |
+| **Physical size** | 60.88 × 111.11 × 5.65 mm |
+| **USB** | Type-C |
+| **Display library** | TFT_eSPI (vendored) |
+| **Reference** | [lcdwiki.com/4.0inch_ESP32-32E_Display](https://www.lcdwiki.com/4.0inch_ESP32-32E_Display) |
+
+---
+
+## Board 3 — Waveshare ESP32-S3-Touch-LCD-4.3
+
+| Attribute | Value |
+|---|---|
+| **Manufacturer** | Waveshare |
+| **Module** | ESP32-S3-WROOM-1-N16R8 |
+| **SoC** | ESP32-S3, Xtensa dual-core 32-bit LX7 |
+| **Max clock** | 240 MHz |
+| **SRAM** | 512 KB |
+| **ROM** | 384 KB |
+| **PSRAM** | 8 MB (onboard) |
+| **Flash** | 16 MB — per FQBN `FlashSize=16M` |
+| **WiFi** | 2.4 GHz 802.11 b/g/n |
+| **Bluetooth** | v5.0 BLE |
+| **Antenna** | Onboard PCB antenna |
+| **Display driver** | RGB parallel (16-bit, 5-6-5 R/G/B channel split) |
+| **Display size** | 4.3″ |
+| **Resolution** | 800 × 480 |
+| **Color depth** | 65K |
+| **Touch** | GT911 capacitive, 5-point, I2C |
+| **Touch I2C** | SDA 8, SCL 9, INT 4, addr 0x14 (runtime) / 0x5D (defined in config) |
+| **I/O expander** | CH422G (I2C, shared bus with touch) — controls backlight, LCD reset, SD CS, etc. |
+| **Pixel clock** | 14 MHz (LovyanGFX Bus_RGB) |
+| **USB** | Type-C (UART via CH343P + native USB HW CDC on GPIO 19/20) |
+| **Peripheral interfaces** | CAN, RS485, I2C, UART, TF card slot (SPI via CH422G EXIO4), ADC sensor header |
+| **Partition scheme** | `app3M_fat9M_16MB` (3 MB app, 9 MB FAT) |
+| **Serial baud** | 115200 |
+| **Display library** | LovyanGFX (vendored) |
+| **Reference** | [waveshare.com/wiki/ESP32-S3-Touch-LCD-4.3](https://www.waveshare.com/wiki/ESP32-S3-Touch-LCD-4.3) |
+
+### RGB bus pin map (Board 3)
+
+| Signal | GPIOs |
+|---|---|
+| Red R0–R4 | 1, 2, 42, 41, 40 |
+| Green G0–G5 | 39, 0, 45, 48, 47, 21 |
+| Blue B0–B4 | 14, 38, 18, 17, 10 |
+| DE / VSYNC / HSYNC / PCLK | 5, 3, 46, 7 |
+
+---
+
+## Known unknowns
+
+These are facts the code references or implies but does not pin down:
+
+- **PSRAM size on Boards 1 & 2.** The build flag `-DBOARD_HAS_PSRAM` is set for both ESP32-32E targets, but the capacity (typically 4 MB or 8 MB on ESP32-WROOM-32 variants) is never stated in the code or config. Hosyond product pages list some models with PSRAM and some without.
+- **Exact screen panel size for Board 1.** The `.crushmemory` file calls it "original 2.8″ ILI9341," but the ILI9341 driver is also used on 2.4″ and 3.2″ panels. No board_config.h comment names the panel size explicitly.
+- **Board 1 manufacturer.** The code doesn't name Hosyond for the 2.8″ board the way it does for the 4″. It could be a generic ESP32-32E devkit from any number of vendors.
+- **Board 1 SPI MISO pin.** Not defined in `tft_user_setup.h` (only MOSI/SCLK/CS/DC/RST are set). This means SPI read-back from the display may not be wired or used.
+- **Serial port path.** All three `board-config.sh` files default to `/dev/ttyUSB0`, which is a Linux convention. The actual development machine appears to be macOS, so the real port (e.g. `/dev/cu.usbserial-*`) is likely overridden at runtime.
+- **GT911 I2C address discrepancy (Board 3).** `board_config.h` defines `GT911_ADDR 0x5D` but `LovyanPins.h` configures the touch at address `0x14`. Both are valid GT911 addresses; the runtime address depends on the INT pin state at boot. The code comment says "IMPORTANT: Address 0x14, not 0x5D!" suggesting 0x5D was tried and didn't work.
+- **CH422G expander pin mapping (Board 3).** `LovyanPins.h` defines symbolic names (`TP_RST=1`, `LCD_BL=2`, `LCD_RST=3`, `SD_CS=4`, `USB_SEL=5`) but these are CH422G *expander output indices*, not ESP32 GPIOs. The I2C init sequence that drives these pins lives in `DisplayDriverGFX.cpp`, which was not fully inspected.
+
+## Unknown unknowns
+
+Things that are plausibly relevant but entirely absent from the codebase:
+
+- **Power supply specs.** No code references input voltage ranges, regulators, or battery charging circuits, though the Waveshare board has a PH2.0 LiPo header and the Hosyond boards support external lithium batteries with onboard charge management.
+- **Thermal limits / operating temperature range.** Not mentioned anywhere.
+- **Hardware revision / PCB version.** No version identifiers for any of the three physical boards.
+- **Antenna characteristics.** The Waveshare board uses an onboard PCB antenna; the Hosyond boards likely do as well. Gain, radiation pattern, and any shielding considerations are unaddressed.
+- **Display viewing angle / brightness / contrast.** The Hosyond 4″ is listed as TN type (narrower viewing angles); the Waveshare 4.3″ is likely IPS but not confirmed in code.
+- **ESD / EMC compliance.** No mention of certifications (FCC, CE, etc.).
+- **Deep sleep / low-power modes.** The firmware uses `millis()`-based timing and a display-off state, but never enters ESP32 deep sleep. Whether the hardware supports wake-on-touch or wake-on-WiFi is not explored.
+- **Audio hardware.** The Hosyond boards support external speakers per their datasheets, and the codebase has no audio code. The Waveshare board does not appear to have onboard audio.
+- **SD card.** The Waveshare board has a TF card slot (CS via CH422G EXIO4), and the Hosyond boards have TF card slots as well. The firmware does not use storage.
+
+---
+
+## Questions for board owner
+
+I'm looking at porting [mainline.py](mainline.py) — a scrolling terminal news/poetry stream with OTF-font rendering, RSS feeds, ANSI gradients, and glitch effects — to run on one of these boards. To figure out the right approach I need a few things only you can answer:
+
+### 1. Which board should I target?
+
+The three boards have very different constraints:
+
+| | Board 1 (2.8″) | Board 2 (4.0″) | Board 3 (4.3″) |
+|---|---|---|---|
+| Resolution | 320 × 240 | 320 × 480 | 800 × 480 |
+| Display bus | SPI (40 MHz) | SPI (40 MHz) | RGB parallel (14 MHz pclk) |
+| Flash | 4 MB | 4 MB | 16 MB |
+| PSRAM | unknown | unknown | 8 MB |
+| Full-screen redraw | ~60 ms+ | ~120 ms+ | near-instant (framebuffer) |
+
+Board 3 is the only one with enough RAM and display bandwidth for smooth scrolling with many headlines buffered. Boards 1 & 2 would need aggressive feature cuts. **Which board do you want this on?**
+
+### 2. PSRAM on your ESP32-32E boards
+
+The build flags say `-DBOARD_HAS_PSRAM` but I can't tell the capacity. Can you check? Easiest way:
+
+```
+// Add to setup() temporarily:
+Serial.printf("PSRAM size: %d bytes\n", ESP.getPsramSize());
+Serial.printf("Free PSRAM: %d bytes\n", ESP.getFreePsram());
+```
+
+If PSRAM is 0 on Boards 1 or 2, those boards can only hold a handful of headlines in 520 KB SRAM (WiFi + TLS eat most of it).
+
+### 3. Feature priorities
+
+mainline.py does a lot of things that don't map directly to an ESP32 + TFT. Which of these matter to you?
+
+- **RSS headline scrolling** — the core experience. How many feeds? All ~25, or a curated subset?
+- **OTF font rendering** — mainline uses Pillow to rasterize a custom `.otf` font into half-block characters. On ESP32 this would become either bitmap fonts or a pre-rendered glyph atlas baked into flash. Is the specific font important, or is the aesthetic (large, stylized text) what matters?
+- **Left-to-right color gradient** — the white-hot → green → black fade. Easy to replicate in RGB565 on the TFT. Keep?
+- **Glitch / noise effects** — the ░▒▓█ and katakana rain. Keep?
+- **Mic-reactive glitch intensity** — none of these boards have a microphone. Drop entirely, or substitute with something else (e.g. touch-reactive, or time-of-day reactive)?
+- **Auto-translation** — mainline translates headlines for region-specific sources via Google Translate. This requires HTTPS calls that are expensive on ESP32 (~40–50 KB RAM per TLS connection). Keep, pre-translate on a server, or drop?
+- **Poetry mode** — fetches full Gutenberg texts. These are large (100+ KB each). Cache to SD card, trim to a small set, or drop?
+- **Content filtering** — the sports/vapid regex filter. Trivial to keep.
+- **Boot sequence animation** — the typewriter-style boot log. Keep?
+
+### 4. Network environment
+
+- Will the board be on a WiFi network that can reach the public internet (RSS feeds, Google Translate, ntfy.sh)?
+- Is there a preferred SSID / network, or should it use the existing multi-network setup from the doorbell firmware?
+
+### 5. SD card availability
+
+All three boards have TF card slots but the doorbell firmware doesn't use them. A microSD card would be useful for caching fonts, pre-rendered glyph atlases, or translated headline buffers. **Is there an SD card in the board you'd want to target?**

mainline.py

@@ -56,6 +56,7 @@ HEADLINE_LIMIT = 1000
 FEED_TIMEOUT = 10
 MIC_THRESHOLD_DB = 50   # dB above which glitches intensify
 MODE = 'poetry' if '--poetry' in sys.argv or '-p' in sys.argv else 'news'
+FIREHOSE = '--firehose' in sys.argv
 
 # Poetry/literature sources — public domain via Project Gutenberg
 POETRY_SOURCES = {
@@ -459,8 +460,39 @@ def fetch_poetry():
     return items, linked, failed
 
 
+# ─── CACHE ────────────────────────────────────────────────
+_CACHE_DIR = pathlib.Path(__file__).resolve().parent
+
+
+def _cache_path():
+    return _CACHE_DIR / f".mainline_cache_{MODE}.json"
+
+
+def _load_cache():
+    """Load cached items from disk if available."""
+    p = _cache_path()
+    if not p.exists():
+        return None
+    try:
+        data = json.loads(p.read_text())
+        items = [tuple(i) for i in data["items"]]
+        return items if items else None
+    except Exception:
+        return None
+
+
+def _save_cache(items):
+    """Save fetched items to disk for fast subsequent runs."""
+    try:
+        _cache_path().write_text(json.dumps({"items": items}))
+    except Exception:
+        pass
+
+
 # ─── STREAM ───────────────────────────────────────────────
 _SCROLL_DUR = 3.75  # seconds per headline
+_FRAME_DT = 0.05    # 50ms base frame rate (20 FPS)
+FIREHOSE_H = 12     # firehose zone height (terminal rows)
 _mic_db = -99.0     # current mic level, written by background thread
 _mic_stream = None
 
@@ -676,6 +708,51 @@ def _make_block(title, src, ts, w):
     return content, hc, len(content) - 1  # (rows, color, meta_row_index)
 
 
+def _firehose_line(items, w):
+    """Generate one line of rapidly cycling firehose content."""
+    r = random.random()
+    if r < 0.35:
+        # Raw headline text
+        title, src, ts = random.choice(items)
+        text = title[:w - 1]
+        color = random.choice([G_LO, G_DIM, W_GHOST, C_DIM])
+        return f"{color}{text}{RST}"
+    elif r < 0.55:
+        # Dense glitch noise
+        d = random.choice([0.45, 0.55, 0.65, 0.75])
+        return "".join(
+            f"{random.choice([G_LO, G_DIM, C_DIM, W_GHOST])}"
+            f"{random.choice(GLITCH + KATA)}{RST}"
+            if random.random() < d else " "
+            for _ in range(w)
+        )
+    elif r < 0.78:
+        # Status / program output
+        sources = FEEDS if MODE == 'news' else POETRY_SOURCES
+        src = random.choice(list(sources.keys()))
+        msgs = [
+            f"  SIGNAL :: {src} :: {datetime.now().strftime('%H:%M:%S.%f')[:-3]}",
+            f"  ░░ FEED ACTIVE :: {src}",
+            f"  >> DECODE 0x{random.randint(0x1000, 0xFFFF):04X} :: {src[:24]}",
+            f"  ▒▒ ACQUIRE :: {random.choice(['TCP', 'UDP', 'RSS', 'ATOM', 'XML'])} :: {src}",
+            f"  {''.join(random.choice(KATA) for _ in range(3))} STRM "
+            f"{random.randint(0, 255):02X}:{random.randint(0, 255):02X}",
+        ]
+        text = random.choice(msgs)[:w - 1]
+        color = random.choice([G_LO, G_DIM, W_GHOST])
+        return f"{color}{text}{RST}"
+    else:
+        # Headline fragment with glitch prefix
+        title, _, _ = random.choice(items)
+        start = random.randint(0, max(0, len(title) - 20))
+        frag = title[start:start + random.randint(10, 35)]
+        pad = random.randint(0, max(0, w - len(frag) - 8))
+        gp = ''.join(random.choice(GLITCH) for _ in range(random.randint(1, 3)))
+        text = (' ' * pad + gp + ' ' + frag)[:w - 1]
+        color = random.choice([G_LO, C_DIM, W_GHOST])
+        return f"{color}{text}{RST}"
+
+
 def stream(items):
     random.shuffle(items)
     pool = list(items)
@@ -687,14 +764,17 @@ def stream(items):
     sys.stdout.flush()
 
     w, h = tw(), th()
+    fh = FIREHOSE_H if FIREHOSE else 0
+    sh = h - fh                    # scroll zone height
     GAP = 3            # blank rows between headlines
-    dt = _SCROLL_DUR / (h + 15) * 2  # 2x slower scroll
+    scroll_interval = _SCROLL_DUR / (sh + 15) * 2
 
     # active blocks: (content_rows, color, canvas_y, meta_idx)
     active = []
     cam = 0            # viewport top in virtual canvas coords
-    next_y = h         # canvas-y where next block starts (off-screen bottom)
+    next_y = sh        # canvas-y where next block starts (off-screen bottom)
     noise_cache = {}
+    scroll_accum = 0.0
 
     def _noise_at(cy):
         if cy not in noise_cache:
@@ -702,23 +782,41 @@ def stream(items):
         return noise_cache[cy]
 
     while queued < HEADLINE_LIMIT or active:
+        t0 = time.monotonic()
         w, h = tw(), th()
+        fh = FIREHOSE_H if FIREHOSE else 0
+        sh = h - fh
 
-        # Enqueue new headlines when room at the bottom
+        # Advance scroll on schedule
-        while next_y < cam + h + 10 and queued < HEADLINE_LIMIT:
+        scroll_accum += _FRAME_DT
-            t, src, ts = _next_headline(pool, items, seen)
+        while scroll_accum >= scroll_interval:
-            content, hc, midx = _make_block(t, src, ts, w)
+            scroll_accum -= scroll_interval
-            active.append((content, hc, next_y, midx))
+            cam += 1
-            next_y += len(content) + GAP
-            queued += 1
 
-        # Draw frame
+            # Enqueue new headlines when room at the bottom
-        top_zone = max(1, int(h * 0.25))  # 25% fade zone at top (exit)
+            while next_y < cam + sh + 10 and queued < HEADLINE_LIMIT:
-        bot_zone = max(1, int(h * 0.10))  # 10% fade zone at bottom (entry)
+                t, src, ts = _next_headline(pool, items, seen)
+                content, hc, midx = _make_block(t, src, ts, w)
+                active.append((content, hc, next_y, midx))
+                next_y += len(content) + GAP
+                queued += 1
+
+            # Prune off-screen blocks and stale noise
+            active = [(c, hc, by, mi) for c, hc, by, mi in active
+                      if by + len(c) > cam]
+            for k in list(noise_cache):
+                if k < cam:
+                    del noise_cache[k]
+
+        # Draw scroll zone
+        top_zone = max(1, int(sh * 0.25))
+        bot_zone = max(1, int(sh * 0.10))
         buf = []
-        for r in range(h):
+        for r in range(sh):
             cy = cam + r
-            row_fade = min(1.0, min(r / top_zone, (h - 1 - r) / bot_zone))
+            top_f = min(1.0, r / top_zone)
+            bot_f = min(1.0, (sh - 1 - r) / bot_zone)
+            row_fade = min(top_f, bot_f)
             drawn = False
             for content, hc, by, midx in active:
                 cr = cy - by
@@ -743,28 +841,28 @@ def stream(items):
                 else:
                     buf.append(f"\033[{r+1};1H\033[K")
 
-        # Glitch — base rate + mic-reactive spikes
+        # Draw firehose zone
+        if FIREHOSE and fh > 0:
+            for fr in range(fh):
+                fline = _firehose_line(items, w)
+                buf.append(f"\033[{sh + fr + 1};1H{fline}\033[K")
+
+        # Glitch — base rate + mic-reactive spikes (scroll zone only)
         mic_excess = max(0.0, _mic_db - MIC_THRESHOLD_DB)
         glitch_prob = 0.32 + min(0.9, mic_excess * 0.16)
         n_hits = 4 + int(mic_excess / 2)
-        if random.random() < glitch_prob and buf:
+        g_limit = sh if FIREHOSE else len(buf)
-            for _ in range(min(n_hits, h)):
+        if random.random() < glitch_prob and g_limit > 0:
-                gi = random.randint(0, len(buf) - 1)
+            for _ in range(min(n_hits, g_limit)):
+                gi = random.randint(0, g_limit - 1)
                 buf[gi] = f"\033[{gi+1};1H{glitch_bar(w)}"
 
-        sys.stdout.write("".join(buf))
+        sys.stdout.buffer.write("".join(buf).encode())
         sys.stdout.flush()
-        time.sleep(dt)
 
-        # Advance viewport
+        # Precise frame timing
-        cam += 1
+        elapsed = time.monotonic() - t0
-
+        time.sleep(max(0, _FRAME_DT - elapsed))
-        # Prune off-screen blocks and stale noise
-        active = [(c, hc, by, mi) for c, hc, by, mi in active
-                  if by + len(c) > cam]
-        for k in list(noise_cache):
-            if k < cam:
-                del noise_cache[k]
 
     sys.stdout.write(CLR)
     sys.stdout.flush()
@@ -808,7 +906,11 @@ def main():
     print()
     time.sleep(0.4)
 
-    if MODE == 'poetry':
+    cached = _load_cache() if '--refresh' not in sys.argv else None
+    if cached:
+        items = cached
+        boot_ln("Cache", f"LOADED [{len(items)} SIGNALS]", True)
+    elif MODE == 'poetry':
         slow_print("  > INITIALIZING LITERARY CORPUS...\n")
         time.sleep(0.2)
         print()
@@ -816,6 +918,7 @@ def main():
         print()
         print(f"  {G_DIM}>{RST} {G_MID}{linked} TEXTS LOADED{RST} {W_GHOST}· {failed} DARK{RST}")
         print(f"  {G_DIM}>{RST} {G_MID}{len(items)} STANZAS ACQUIRED{RST}")
+        _save_cache(items)
     else:
         slow_print("  > INITIALIZING FEED ARRAY...\n")
         time.sleep(0.2)
@@ -824,6 +927,7 @@ def main():
         print()
         print(f"  {G_DIM}>{RST} {G_MID}{linked} SOURCES LINKED{RST} {W_GHOST}· {failed} DARK{RST}")
         print(f"  {G_DIM}>{RST} {G_MID}{len(items)} SIGNALS ACQUIRED{RST}")
+        _save_cache(items)
 
     if not items:
         print(f"\n  {W_DIM}> NO SIGNAL — check network{RST}")
@@ -833,6 +937,8 @@ def main():
     mic_ok = _start_mic()
     if _HAS_MIC:
         boot_ln("Microphone", "ACTIVE" if mic_ok else "OFFLINE · check System Settings → Privacy → Microphone", mic_ok)
+    if FIREHOSE:
+        boot_ln("Firehose", "ENGAGED", True)
 
     time.sleep(0.4)
     slow_print("  > STREAMING...\n")