genewildish/Mainline
1
0
Fork 1
Code Issues Pull Requests 4 Packages Projects Releases Wiki Activity

Merge pull request 'style: Replace escaped parentheses with standard parentheses in the Mainline Renderer documentation.' (#15) from feat/scalability into main

Browse Source 
Reviewed-on: #15
This commit was merged in pull request #15.
This commit is contained in:
genewildish
2026-03-15 10:03:42 +00:00
parent 7098b2f5aa 1e99d70387
commit 0740e34293
1 changed files with 35 additions and 35 deletions
Download Patch File Download Diff File Expand all files Collapse all files

70
Mainline Renderer + ntfy Message Queue for ESP32.md
View File

@@ -3,29 +3,29 @@
mainline\.py does heavy work unsuitable for ESP32: 25\+ HTTPS/TLS RSS feeds, OTF font rasterization via Pillow, Google Translate API calls, and complex text layout\. Simultaneously, messages arriving on `ntfy.sh/klubhaus_terminal_mainline` need to interrupt the news ticker on the same device\. mainline\.py does heavy work unsuitable for ESP32: 25\+ HTTPS/TLS RSS feeds, OTF font rasterization via Pillow, Google Translate API calls, and complex text layout\. Simultaneously, messages arriving on `ntfy.sh/klubhaus_terminal_mainline` need to interrupt the news ticker on the same device\.
## Architecture: Server \+ Thin Client ## Architecture: Server \+ Thin Client
Split the system into two halves that are designed together\. Split the system into two halves that are designed together\.
**Server \(mainline\.py `--serve` mode, runs on any always\-on machine\)** **Server (mainline\.py `--serve` mode, runs on any always\-on machine)**
* Reuses existing feed fetching, caching, content filtering, translation, and Pillow font rendering pipeline — no duplication\. * Reuses existing feed fetching, caching, content filtering, translation, and Pillow font rendering pipeline — no duplication\.
* Pre\-renders each headline into a 1\-bit bitmap strip \(the OTF→half\-block pipeline already produces this as an intermediate step in `_render_line()`\)\. * Pre\-renders each headline into a 1\-bit bitmap strip (the OTF→half\-block pipeline already produces this as an intermediate step in `_render_line()`)\.
* Exposes a lightweight HTTP API the ESP32 polls\. * Exposes a lightweight HTTP API the ESP32 polls\.
**ESP32 thin client \(Arduino sketch\)** **ESP32 thin client (Arduino sketch)**
* Polls the mainline server for pre\-rendered headline bitmaps over plain HTTP \(no TLS needed if on the same LAN\)\. * Polls the mainline server for pre\-rendered headline bitmaps over plain HTTP (no TLS needed if on the same LAN)\.
* Polls `ntfy.sh/klubhaus_terminal_mainline` directly for messages, reusing the proven `NetManager::httpGet()` \+ JSON parsing pattern from DoorbellLogic \(`DoorbellLogic.cpp:155-192`\)\. * Polls `ntfy.sh/klubhaus_terminal_mainline` directly for messages, reusing the proven `NetManager::httpGet()` \+ JSON parsing pattern from DoorbellLogic (`DoorbellLogic.cpp:155-192`)\.
* Manages scrolling, gradient coloring, and glitch effects locally \(cheap per\-frame GPU work\)\. * Manages scrolling, gradient coloring, and glitch effects locally (cheap per\-frame GPU work)\.
* When an ntfy message arrives, the scroll is paused and the message takes over the display — same interrupt pattern as the doorbell's ALERT→DASHBOARD flow\. * When an ntfy message arrives, the scroll is paused and the message takes over the display — same interrupt pattern as the doorbell's ALERT→DASHBOARD flow\.
## Server API \(mainline repo\) ## Server API (mainline repo)
New file: `serve.py` \(or `--serve` mode in mainline\.py\)\. New file: `serve.py` (or `--serve` mode in mainline\.py)\.
Endpoints: Endpoints:
* `GET /api/headlines` — returns JSON array of headline metadata: `[{"id": 0, "src": "Nature", "ts": "14:30", "width": 280, "height": 16, "bitmap": "<base64 1-bit packed>"}]`\. Bitmaps are 1\-bit\-per\-pixel, row\-major, packed 8px/byte\. The ESP32 applies gradient color locally\. * `GET /api/headlines` — returns JSON array of headline metadata: `[{"id": 0, "src": "Nature", "ts": "14:30", "width": 280, "height": 16, "bitmap": "<base64 1-bit packed>"}]`\. Bitmaps are 1\-bit\-per\-pixel, row\-major, packed 8px/byte\. The ESP32 applies gradient color locally\.
* `GET /api/config` — returns `{"count": 120, "version": "...", "mode": "news"}` so the ESP32 knows what it's getting\. * `GET /api/config` — returns `{"count": 120, "version": "...", "mode": "news"}` so the ESP32 knows what it's getting\.
* `GET /api/health` — `{"ok": true, "last_fetch": "...", "headline_count": 120}` * `GET /api/health` — `{"ok": true, "last_fetch": "...", "headline_count": 120}`
The server renders at a configurable target width \(e\.g\. 800px for Board 3, 320px for Boards 1/2\) via a `--width` flag or query parameter\. Height is fixed per headline by the font size\. The server renders at a configurable target width (e\.g\. 800px for Board 3, 320px for Boards 1/2) via a `--width` flag or query parameter\. Height is fixed per headline by the font size\.
The server refreshes feeds on a timer \(reusing `_SCROLL_DUR` cadence or a longer interval\), re\-renders, and serves the latest set\. The ESP32 polls `/api/headlines` periodically \(e\.g\. every 60s\) and swaps in the new set\. The server refreshes feeds on a timer (reusing `_SCROLL_DUR` cadence or a longer interval), re\-renders, and serves the latest set\. The ESP32 polls `/api/headlines` periodically (e\.g\. every 60s) and swaps in the new set\.
## Render pipeline \(server side\) ## Render pipeline (server side)
The existing `_render_line()` in mainline\.py already does: The existing `_render_line()` in mainline\.py already does:
1. `ImageFont.truetype()` → `ImageDraw.text()` → grayscale `Image` 1. `ImageFont.truetype()` → `ImageDraw.text()` → grayscale `Image`
2. Resize to target height 2. Resize to target height
3. Threshold to 1\-bit \(the `thr = 80` step\) 3. Threshold to 1\-bit (the `thr = 80` step)
For the server, we stop at step 3 and pack the 1\-bit data into bytes instead of converting to half\-block Unicode\. This is the exact same pipeline, just with a different output format\. The `_big_wrap()` and `_lr_gradient()` logic stays on the server for layout; gradient *coloring* moves to the ESP32 \(it's just an index lookup per pixel column\)\. For the server, we stop at step 3 and pack the 1\-bit data into bytes instead of converting to half\-block Unicode\. This is the exact same pipeline, just with a different output format\. The `_big_wrap()` and `_lr_gradient()` logic stays on the server for layout; gradient *coloring* moves to the ESP32 (it's just an index lookup per pixel column)\.
## ESP32 client ## ESP32 client
### State machine ### State machine
```warp-runnable-command ```warp-runnable-command
@@ -35,19 +35,19 @@ BOOT → SCROLL ⇄ MESSAGE
* **BOOT** — WiFi connect, initial headline fetch from server\. * **BOOT** — WiFi connect, initial headline fetch from server\.
* **SCROLL** — Vertical scroll through pre\-rendered headlines with local gradient \+ glitch\. Polls server for new headlines periodically\. Polls ntfy every 15s\. * **SCROLL** — Vertical scroll through pre\-rendered headlines with local gradient \+ glitch\. Polls server for new headlines periodically\. Polls ntfy every 15s\.
* **MESSAGE** — ntfy message arrived\. Scroll paused, message displayed\. Auto\-dismiss after timeout or touch\-dismiss\. Returns to SCROLL\. * **MESSAGE** — ntfy message arrived\. Scroll paused, message displayed\. Auto\-dismiss after timeout or touch\-dismiss\. Returns to SCROLL\.
* **OFF** — Backlight off after inactivity \(polling continues in background\)\. * **OFF** — Backlight off after inactivity (polling continues in background)\.
### ntfy integration ### ntfy integration
The ESP32 polls `https://ntfy.sh/klubhaus_terminal_mainline/json?since=20s&poll=1` on the same 15s interval as the doorbell polls its topics\. When a message event arrives: The ESP32 polls `https://ntfy.sh/klubhaus_terminal_mainline/json?since=20s&poll=1` on the same 15s interval as the doorbell polls its topics\. When a message event arrives:
1. Parse JSON: `{"event": "message", "title": "...", "message": "..."}` 1. Parse JSON: `{"event": "message", "title": "...", "message": "..."}`
2. Save current scroll position\. 2. Save current scroll position\.
3. Transition to MESSAGE state\. 3. Transition to MESSAGE state\.
4. Render message text using the display library's built\-in fonts \(messages are short, no custom font needed\)\. 4. Render message text using the display library's built\-in fonts (messages are short, no custom font needed)\.
5. After `MESSAGE_TIMEOUT_MS` \(e\.g\. 30s\) or touch, restore scroll position and resume\. 5. After `MESSAGE_TIMEOUT_MS` (e\.g\. 30s) or touch, restore scroll position and resume\.
This is architecturally identical to `DoorbellLogic::onAlert()` → `dismissAlert()`, just with different content\. The ntfy polling runs independently of the server connection, so messages work even if the mainline server is offline \(the device just shows the last cached headlines\)\. This is architecturally identical to `DoorbellLogic::onAlert()` → `dismissAlert()`, just with different content\. The ntfy polling runs independently of the server connection, so messages work even if the mainline server is offline (the device just shows the last cached headlines)\.
### Headline storage ### Headline storage
* Board 3 \(8 MB PSRAM\): store all ~120 headline bitmaps in PSRAM\. At 800px × 16px × 1 bit = 1\.6 KB each → ~192 KB total\. Trivial\. * Board 3 (8 MB PSRAM): store all ~120 headline bitmaps in PSRAM\. At 800px × 16px × 1 bit = 1\.6 KB each → ~192 KB total\. Trivial\.
* Boards 1/2 \(PSRAM TBD\): at 320px × 16px = 640 bytes each → ~77 KB for 120 headlines\. Fits if PSRAM is present\. Without PSRAM, keep ~20 headlines in a ring buffer \(~13 KB\)\. * Boards 1/2 (PSRAM TBD): at 320px × 16px = 640 bytes each → ~77 KB for 120 headlines\. Fits if PSRAM is present\. Without PSRAM, keep ~20 headlines in a ring buffer (~13 KB)\.
### Gradient coloring \(local\) ### Gradient coloring (local)
The 12\-step ANSI gradient in mainline\.py maps to 12 RGB565 values: The 12\-step ANSI gradient in mainline\.py maps to 12 RGB565 values:
```warp-runnable-command ```warp-runnable-command
const uint16_t GRADIENT[] = { const uint16_t GRADIENT[] = {
@@ -68,8 +68,8 @@ mainline.py (existing, unchanged)
serve.py (new — HTTP server, imports mainline rendering functions) serve.py (new — HTTP server, imports mainline rendering functions)
klubhaus-doorbell-hardware.md (existing) klubhaus-doorbell-hardware.md (existing)
``` ```
`serve.py` imports the rendering functions from mainline\.py \(after refactoring them into importable form — they're currently top\-level but not wrapped in `if __name__`\)\. `serve.py` imports the rendering functions from mainline\.py (after refactoring them into importable form — they're currently top\-level but not wrapped in `if __name__`)\.
### klubhaus\-doorbell repo \(or mainline repo under firmware/\) ### klubhaus\-doorbell repo (or mainline repo under firmware/)
```warp-runnable-command ```warp-runnable-command
boards/esp32-mainline/ boards/esp32-mainline/
├── esp32-mainline.ino Main sketch ├── esp32-mainline.ino Main sketch
@@ -79,31 +79,31 @@ boards/esp32-mainline/
├── HeadlineStore.h/.cpp Bitmap ring buffer in PSRAM ├── HeadlineStore.h/.cpp Bitmap ring buffer in PSRAM
└── NtfyPoller.h/.cpp ntfy.sh polling (extracted from DoorbellLogic pattern) └── NtfyPoller.h/.cpp ntfy.sh polling (extracted from DoorbellLogic pattern)
``` ```
The display driver is reused from the target board \(e\.g\. `DisplayDriverGFX` for Board 3\)\. `MainlineLogic` replaces `DoorbellLogic` as the state machine but follows the same patterns\. The display driver is reused from the target board (e\.g\. `DisplayDriverGFX` for Board 3)\. `MainlineLogic` replaces `DoorbellLogic` as the state machine but follows the same patterns\.
## Branch strategy recommendation ## Branch strategy recommendation
The work spans two repos and has clear dependency ordering\. The work spans two repos and has clear dependency ordering\.
### Phase 1 — Finish current branch \(mainline repo\) ### Phase 1 — Finish current branch (mainline repo)
**Branch:** `feat/arduino` \(current\) **Branch:** `feat/arduino` (current)
**Content:** Hardware spec doc\. Already done\. **Content:** Hardware spec doc\. Already done\.
**Action:** Merge to main when ready\. **Action:** Merge to main when ready\.
### Phase 2 — Server renderer \(mainline repo\) ### Phase 2 — Server renderer (mainline repo)
**Branch:** `feat/renderer` \(branch from main after Phase 1 merges\) **Branch:** `feat/renderer` (branch from main after Phase 1 merges)
**Content:** **Content:**
* Refactor mainline\.py rendering functions to be importable \(extract from `__main__` guard\) * Refactor mainline\.py rendering functions to be importable (extract from `__main__` guard)
* `serve.py` — HTTP server with `/api/headlines`, `/api/config`, `/api/health` * `serve.py` — HTTP server with `/api/headlines`, `/api/config`, `/api/health`
* Bitmap packing utility \(1\-bit row\-major\) * Bitmap packing utility (1\-bit row\-major)
**Why a separate branch:** This changes mainline\.py's structure \(refactoring for imports\) and adds a new entry point\. It's a self\-contained, testable unit — you can verify the API with `curl` before touching any Arduino code\. **Why a separate branch:** This changes mainline\.py's structure (refactoring for imports) and adds a new entry point\. It's a self\-contained, testable unit — you can verify the API with `curl` before touching any Arduino code\.
### Phase 3 — ESP32 client \(klubhaus\-doorbell repo, or mainline repo\) ### Phase 3 — ESP32 client (klubhaus\-doorbell repo, or mainline repo)
**Branch:** `feat/mainline-client` in whichever repo hosts it **Branch:** `feat/mainline-client` in whichever repo hosts it
**Content:** **Content:**
* `MainlineLogic` state machine * `MainlineLogic` state machine
* `HeadlineStore` bitmap buffer * `HeadlineStore` bitmap buffer
* `NtfyPoller` for `klubhaus_terminal_mainline` * `NtfyPoller` for `klubhaus_terminal_mainline`
* Board\-specific sketch for the target board * Board\-specific sketch for the target board
**Depends on:** Phase 2 \(needs a running server to test against\) **Depends on:** Phase 2 (needs a running server to test against)
**Repo decision:** If you have push access to klubhaus\-doorbell, it fits naturally as a new board target alongside the existing doorbell sketches — it reuses `NetManager`, `IDisplayDriver`, and the vendored display libraries\. If not, put it under `mainline/firmware/` and vendor the shared KlubhausCore library\. **Repo decision:** If you have push access to klubhaus\-doorbell, it fits naturally as a new board target alongside the existing doorbell sketches — it reuses `NetManager`, `IDisplayDriver`, and the vendored display libraries\. If not, put it under `mainline/firmware/` and vendor the shared KlubhausCore library\.
### Merge order ### Merge order
1. `feat/arduino` → main \(hardware spec\) 1. `feat/arduino` → main (hardware spec)
2. `feat/renderer` → main \(server\) 2. `feat/renderer` → main (server)
3. `feat/mainline-client` → main in whichever repo \(ESP32 client\) 3. `feat/mainline-client` → main in whichever repo (ESP32 client)
Each phase is independently testable and doesn't block the other until Phase 3 needs a running server\. Each phase is independently testable and doesn't block the other until Phase 3 needs a running server\.