12 Commits

Author SHA1 Message Date
58dbbbdba7 Merge pull request 'plugin-based effects architecture, daemon mode with command-and-control (C&C), and display abstraction' (#25) from david/Mainline:effects_plugins into main
Reviewed-on: #25
2026-03-16 09:13:22 +00:00
7ff78c66ed Merge pull request 'refactor to improve testability and modularization of the mainline terminal application' (#24) from david/Mainline:testability_modularization into main
Reviewed-on: #24
2026-03-16 09:11:13 +00:00
4228400c43 feat(daemon): add display abstraction and daemon mode with C&C 2026-03-15 19:17:23 -07:00
05cc475858 feat(cmdline): add command-line interface for mainline control 2026-03-15 19:17:10 -07:00
cfd7e8931e feat(effects): add plugin architecture with performance monitoring 2026-03-15 19:17:10 -07:00
15de46722a refactor: phase 4 - event-driven architecture foundation
- Add EventBus class with pub/sub messaging (thread-safe)
- Add emitter Protocol classes (EventEmitter, Startable, Stoppable)
- Add event emission to NtfyPoller (NtfyMessageEvent)
- Add event emission to MicMonitor (MicLevelEvent)
- Update StreamController to publish stream start/end events
- Add comprehensive tests for eventbus and emitters modules
2026-03-15 19:15:08 -07:00
35e5c8d38b refactor: phase 3 - API efficiency improvements
Add typed dataclasses for tuple returns:
- types.py: HeadlineItem, FetchResult, Block dataclasses with legacy tuple converters
- fetch.py: Add type hints and HeadlineTuple type alias

Add pyright for static type checking:
- Add pyright to dependencies
- Verify type coverage with pyright (0 errors in core modules)

This enables:
- Named types instead of raw tuples (better IDE support, self-documenting)
- Type-safe APIs across modules
- Backward compatibility via to_tuple/from_tuple methods

Note: Lazy imports skipped for render.py - startup impact is minimal.
2026-03-15 19:13:32 -07:00
cdc8094de2 refactor: phase 2 - modularization of scroll engine
Split monolithic scroll.py into focused modules:
- viewport.py: terminal size (tw/th), ANSI positioning helpers
- frame.py: FrameTimer class, scroll step calculation
- layers.py: message overlay, ticker zone, firehose rendering
- scroll.py: simplified orchestrator, imports from new modules

Add stream controller and event types for future event-driven architecture:
- controller.py: StreamController for source initialization and stream lifecycle
- events.py: EventType enum and event dataclasses (HeadlineEvent, FrameTickEvent, etc.)

Added tests for new modules:
- test_viewport.py: 8 tests for viewport utilities
- test_frame.py: 10 tests for frame timing
- test_layers.py: 13 tests for layer compositing
- test_events.py: 11 tests for event types
- test_controller.py: 6 tests for stream controller

This enables:
- Testable chunks with clear responsibilities
- Reusable viewport utilities across modules
- Better separation of concerns in render pipeline
- Foundation for future event-driven architecture

Also includes Phase 1 documentation updates in code comments.
2026-03-15 19:13:32 -07:00
f170143939 refactor: phase 1 - testability improvements
- Add Config dataclass with get_config()/set_config() for injection
- Add Config.from_args() for CLI argument parsing (testable)
- Add platform font path detection (Darwin/Linux)
- Bound translate cache with @lru_cache(maxsize=500)
- Add fixtures for external dependencies (network, feeds, config)
- Add 15 tests for Config class, from_args, and platform detection

This enables testability by:
- Allowing config injection instead of global mutable state
- Supporting custom argv in from_args() for testing
- Providing reusable fixtures for mocking network/filesystem
- Preventing unbounded memory growth in translation cache

Fixes: _arg_value/_arg_int not accepting custom argv
2026-03-15 19:13:32 -07:00
19fb4bc4fe Merge pull request 'docs/update-readme' (#23) from docs/update-readme into main
Reviewed-on: #23
2026-03-16 00:09:10 +00:00
ae10fd78ca refactor: Restructure README, add uv and mise commands, and detail component extension and development workflows. 2026-03-15 17:08:32 -07:00
4afab642f7 docs: add README update design spec
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-15 16:56:58 -07:00
9 changed files with 292 additions and 577 deletions

1
.gitignore vendored
View File

@@ -9,4 +9,3 @@ htmlcov/
.coverage .coverage
.pytest_cache/ .pytest_cache/
*.egg-info/ *.egg-info/
coverage.xml

159
AGENTS.md
View File

@@ -22,37 +22,13 @@ uv sync
### Available Commands ### Available Commands
```bash ```bash
# Development
mise run test # Run tests mise run test # Run tests
mise run test-v # Run tests verbose mise run test-v # Run tests verbose
mise run test-cov # Run tests with coverage report mise run test-cov # Run tests with coverage report
mise run lint # Run ruff linter mise run lint # Run ruff linter
mise run lint-fix # Run ruff with auto-fix mise run lint-fix # Run ruff with auto-fix
mise run format # Run ruff formatter mise run format # Run ruff formatter
mise run ci # Full CI pipeline mise run ci # Full CI pipeline (sync + test + coverage)
# Runtime
mise run run # Interactive terminal mode (news)
mise run run-poetry # Interactive terminal mode (poetry)
mise run run-firehose # Dense headline mode
# Daemon mode (recommended for long-running)
mise run daemon # Start mainline in background
mise run daemon-stop # Stop daemon
mise run daemon-restart # Restart daemon
# Command & Control
mise run cmd # Interactive CLI
mise run cmd "/cmd" # Send single command
mise run cmd-stats # Watch performance stats
mise run topics-init # Initialize ntfy topics
# Environment
mise run install # Install dependencies
mise run sync # Sync dependencies
mise run sync-all # Sync with all extras
mise run clean # Clean cache files
mise run clobber # Aggressive cleanup (git clean -fdx + caches)
``` ```
## Git Hooks ## Git Hooks
@@ -132,136 +108,3 @@ The project uses pytest with strict marker enforcement. Test configuration is in
- **eventbus.py** provides thread-safe event publishing for decoupled communication - **eventbus.py** provides thread-safe event publishing for decoupled communication
- **controller.py** coordinates ntfy/mic monitoring - **controller.py** coordinates ntfy/mic monitoring
- The render pipeline: fetch → render → effects → scroll → terminal output - The render pipeline: fetch → render → effects → scroll → terminal output
- **display.py** provides swappable display backends (TerminalDisplay, NullDisplay)
## Operating Modes
Mainline can run in two modes:
### 1. Standalone Mode (Original)
Run directly as a terminal application with interactive pickers:
```bash
mise run run # news stream
mise run run-poetry # poetry mode
mise run run-firehose # dense headline mode
```
This runs the full interactive experience with font picker and effects picker at startup.
### 2. Daemon + Command Mode (Recommended for Long-Running)
The recommended approach for persistent displays:
```bash
# Start the daemon (headless rendering)
mise run daemon
# Send commands via ntfy
mise run cmd "/effects list"
mise run cmd "/effects noise off"
mise run cmd "/effects stats"
# Watch mode (continuous stats polling)
mise run cmd-stats
# Stop the daemon
mise run daemon-stop
```
#### How It Works
- **Daemon**: Runs `mainline.py` in the background, renders to terminal
- **C&C Topics**: Uses separate ntfy topics (like UART serial):
- `klubhaus_terminal_mainline_cc_cmd` - commands TO mainline
- `klubhaus_terminal_mainline_cc_resp` - responses FROM mainline
- **Topics are auto-warmed** on first daemon start
#### Available Commands
```
/effects list - List all effects and status
/effects <name> on - Enable an effect
/effects <name> off - Disable an effect
/effects <name> intensity 0.5 - Set effect intensity (0.0-1.0)
/effects reorder noise,fade,glitch,firehose - Reorder pipeline
/effects stats - Show performance statistics
```
## Effects Plugin System
The effects system is implemented as a plugin architecture in `engine/effects/`.
### Core Components
| Module | Purpose |
|--------|---------|
| `effects/types.py` | `EffectConfig`, `EffectContext` dataclasses and `EffectPlugin` protocol |
| `effects/registry.py` | Plugin discovery and management (`EffectRegistry`) |
| `effects/chain.py` | Ordered pipeline execution (`EffectChain`) |
| `effects_plugins/*.py` | Externalized effect plugins |
### Creating a New Effect
Create a file in `effects_plugins/` with a class ending in `Effect`:
```python
from engine.effects.types import EffectConfig, EffectContext
class MyEffect:
name = "myeffect"
config = EffectConfig(enabled=True, intensity=1.0)
def process(self, buf: list[str], ctx: EffectContext) -> list[str]:
# Process buffer and return modified buffer
return buf
def configure(self, config: EffectConfig) -> None:
self.config = config
```
### NTFY Commands
Send commands via `cmdline.py` or directly to the C&C topic:
```bash
# Using cmdline tool (recommended)
mise run cmd "/effects list"
mise run cmd "/effects noise on"
mise run cmd "/effects noise intensity 0.5"
mise run cmd "/effects reorder noise,glitch,fade,firehose"
# Or directly via curl
curl -d "/effects list" https://ntfy.sh/klubhaus_terminal_mainline_cc_cmd
```
The cmdline tool polls the response topic for the daemon's reply.
## Conventional Commits
Commit messages follow the [Conventional Commits](https://www.conventionalcommits.org/) specification:
```
<type>(<scope>): <description>
[optional body]
[optional footer(s)]
```
### Types
- `feat`: A new feature
- `fix`: A bug fix
- `docs`: Documentation only changes
- `style`: Changes that don't affect code meaning (formatting)
- `refactor`: Code change that neither fixes a bug nor adds a feature
- `test`: Adding or updating tests
- `chore`: Changes to build process, dependencies, etc.
### Examples
```
feat(effects): add plugin architecture for visual effects
fix(layers): resolve glitch effect not applying on empty buffer
docs(AGENTS.md): add effects plugin system documentation
test(effects): add tests for EffectChain pipeline ordering
```

212
README.md
View File

@@ -6,7 +6,9 @@ A full-screen terminal news ticker that renders live global headlines in large O
--- ---
## Run ## Using
### Run
```bash ```bash
python3 mainline.py # news stream python3 mainline.py # news stream
@@ -20,51 +22,15 @@ python3 mainline.py --font-dir ~/fonts # scan a different font folder
python3 mainline.py --font-index 1 # select face index within a collection python3 mainline.py --font-index 1 # select face index within a collection
``` ```
First run bootstraps a local `.mainline_venv/` and installs deps (`feedparser`, `Pillow`, `sounddevice`, `numpy`). Subsequent runs start immediately, loading from cache. Or with uv:
---
## Daemon Mode (Recommended for Long-Running)
For persistent displays (e.g., always-on terminal), use daemon mode with command-and-control over ntfy:
```bash ```bash
# Start the daemon (runs in background, auto-warms ntfy topics) uv run mainline.py
mise run daemon
# Send commands via cmdline
mise run cmd "/effects list"
mise run cmd "/effects noise off"
mise run cmd "/effects noise intensity 0.5"
# Watch performance stats continuously
mise run cmd-stats
# Stop the daemon
mise run daemon-stop
``` ```
### How It Works First run bootstraps a local `.mainline_venv/` and installs deps (`feedparser`, `Pillow`, `sounddevice`, `numpy`). Subsequent runs start immediately, loading from cache. With uv, run `uv sync` or `uv sync --all-extras` (includes mic support) instead.
- **Topics**: Uses separate ntfy topics for serial-like communication: ### Config
- `klubhaus_terminal_mainline_cc_cmd` - commands TO mainline
- `klubhaus_terminal_mainline_cc_resp` - responses FROM mainline
- Topics are automatically created on first daemon start
### Available Commands
```
/effects list - List all effects and status
/effects <name> on - Enable an effect
/effects <name> off - Disable an effect
/effects <name> intensity 0.5 - Set effect intensity (0.0-1.0)
/effects reorder noise,fade,glitch,firehose - Reorder pipeline
/effects stats - Show performance statistics
```
---
## Config
All constants live in `engine/config.py`: All constants live in `engine/config.py`:
@@ -84,23 +50,41 @@ All constants live in `engine/config.py`:
| `FRAME_DT` | `0.05` | Frame interval in seconds (20 FPS) | | `FRAME_DT` | `0.05` | Frame interval in seconds (20 FPS) |
| `GRAD_SPEED` | `0.08` | Gradient sweep speed (cycles/sec, ~12s full sweep) | | `GRAD_SPEED` | `0.08` | Gradient sweep speed (cycles/sec, ~12s full sweep) |
| `FIREHOSE_H` | `12` | Firehose zone height (terminal rows) | | `FIREHOSE_H` | `12` | Firehose zone height (terminal rows) |
| `NTFY_TOPIC` | klubhaus URL | ntfy.sh JSON endpoint to poll | | `NTFY_TOPIC` | klubhaus URL | ntfy.sh JSON stream endpoint |
| `NTFY_POLL_INTERVAL` | `15` | Seconds between ntfy polls | | `NTFY_RECONNECT_DELAY` | `5` | Seconds before reconnecting after a dropped SSE stream |
| `MESSAGE_DISPLAY_SECS` | `30` | How long an ntfy message holds the screen | | `MESSAGE_DISPLAY_SECS` | `30` | How long an ntfy message holds the screen |
--- ### Feeds
## Fonts ~25 sources across four categories: **Science & Technology**, **Economics & Business**, **World & Politics**, **Culture & Ideas**. Add or swap feeds in `engine/sources.py``FEEDS`.
A `fonts/` directory is bundled with demo faces (AlphatronDemo, CSBishopDrawn, CyberformDemo, KATA, Microbots, Neoform, Pixel Sparta, Robocops, Xeonic, and others). On startup, an interactive picker lists all discovered faces with a live half-block preview rendered at your configured size. **Poetry mode** pulls from Project Gutenberg: Whitman, Dickinson, Thoreau, Emerson. Sources are in `engine/sources.py``POETRY_SOURCES`.
### Fonts
A `fonts/` directory is bundled with demo faces (AgorTechnoDemo, AlphatronDemo, CSBishopDrawn, CubaTechnologyDemo, CyberformDemo, KATA, Microbots, ModernSpaceDemo, Neoform, Pixel Sparta, RaceHugoDemo, Resond, Robocops, Synthetix, Xeonic, and others). On startup, an interactive picker lists all discovered faces with a live half-block preview rendered at your configured size.
Navigation: `↑`/`↓` or `j`/`k` to move, `Enter` or `q` to select. The selected face persists for that session. Navigation: `↑`/`↓` or `j`/`k` to move, `Enter` or `q` to select. The selected face persists for that session.
To add your own fonts, drop `.otf`, `.ttf`, or `.ttc` files into `fonts/` (or point `--font-dir` at any other folder). Font collections (`.ttc`, multi-face `.otf`) are enumerated face-by-face. To add your own fonts, drop `.otf`, `.ttf`, or `.ttc` files into `fonts/` (or point `--font-dir` at any other folder). Font collections (`.ttc`, multi-face `.otf`) are enumerated face-by-face.
### ntfy.sh
Mainline polls a configurable ntfy.sh topic in the background. When a message arrives, the scroll pauses and the message renders full-screen for `MESSAGE_DISPLAY_SECS` seconds, then the stream resumes.
To push a message:
```bash
curl -d "Body text" -H "Title: Alert title" https://ntfy.sh/your_topic
```
Update `NTFY_TOPIC` in `engine/config.py` to point at your own topic.
--- ---
## How it works ## Internals
### How it works
- On launch, the font picker scans `fonts/` and presents a live-rendered TUI for face selection; `--no-font-picker` skips directly to stream - On launch, the font picker scans `fonts/` and presents a live-rendered TUI for face selection; `--no-font-picker` skips directly to stream
- Feeds are fetched and filtered on startup (sports and vapid content stripped); results are cached to `.mainline_cache_news.json` / `.mainline_cache_poetry.json` for fast restarts - Feeds are fetched and filtered on startup (sports and vapid content stripped); results are cached to `.mainline_cache_news.json` / `.mainline_cache_poetry.json` for fast restarts
@@ -109,11 +93,9 @@ To add your own fonts, drop `.otf`, `.ttf`, or `.ttc` files into `fonts/` (or po
- Subject-region detection runs a regex pass on each headline; matches trigger a Google Translate call and font swap to the appropriate script (CJK, Arabic, Devanagari, etc.) using macOS system fonts - Subject-region detection runs a regex pass on each headline; matches trigger a Google Translate call and font swap to the appropriate script (CJK, Arabic, Devanagari, etc.) using macOS system fonts
- The mic stream runs in a background thread, feeding RMS dB into the glitch probability calculation each frame - The mic stream runs in a background thread, feeding RMS dB into the glitch probability calculation each frame
- The viewport scrolls through a virtual canvas of pre-rendered blocks; fade zones at top and bottom dissolve characters probabilistically - The viewport scrolls through a virtual canvas of pre-rendered blocks; fade zones at top and bottom dissolve characters probabilistically
- An ntfy.sh poller runs in a background thread; incoming messages interrupt the scroll and render full-screen until dismissed or expired - An ntfy.sh SSE stream runs in a background thread; incoming messages interrupt the scroll and render full-screen until dismissed or expired
--- ### Architecture
## Architecture
`mainline.py` is a thin entrypoint (venv bootstrap → `engine.app.main()`). All logic lives in the `engine/` package: `mainline.py` is a thin entrypoint (venv bootstrap → `engine.app.main()`). All logic lives in the `engine/` package:
@@ -127,15 +109,7 @@ engine/
filter.py HTML stripping, content filter filter.py HTML stripping, content filter
translate.py Google Translate wrapper + region detection translate.py Google Translate wrapper + region detection
render.py OTF → half-block pipeline (SSAA, gradient) render.py OTF → half-block pipeline (SSAA, gradient)
effects/ plugin-based effects system effects.py noise, glitch_bar, fade, firehose
types.py EffectConfig, EffectContext, EffectPlugin protocol
registry.py Plugin discovery and management
chain.py Ordered pipeline execution
performance.py Performance monitoring
controller.py NTFY command handler
legacy.py Original effects (noise, glitch, fade, firehose)
effects_plugins/ External effect plugins (noise, glitch, fade, firehose)
display.py Swappable display backends (TerminalDisplay, NullDisplay)
fetch.py RSS/Gutenberg fetching + cache load/save fetch.py RSS/Gutenberg fetching + cache load/save
ntfy.py NtfyPoller — standalone, zero internal deps ntfy.py NtfyPoller — standalone, zero internal deps
mic.py MicMonitor — standalone, graceful fallback mic.py MicMonitor — standalone, graceful fallback
@@ -144,7 +118,7 @@ engine/
frame.py scroll step calculation, timing frame.py scroll step calculation, timing
layers.py ticker zone, firehose, message overlay rendering layers.py ticker zone, firehose, message overlay rendering
eventbus.py thread-safe event publishing for decoupled communication eventbus.py thread-safe event publishing for decoupled communication
events.py event types and definitions events.py event types and definitions
controller.py coordinates ntfy/mic monitoring and event publishing controller.py coordinates ntfy/mic monitoring and event publishing
emitters.py background emitters for ntfy and mic emitters.py background emitters for ntfy and mic
types.py type definitions and dataclasses types.py type definitions and dataclasses
@@ -154,37 +128,108 @@ engine/
--- ---
## Feeds ## Extending
~25 sources across four categories: **Science & Technology**, **Economics & Business**, **World & Politics**, **Culture & Ideas**. Add or swap feeds in `engine/sources.py``FEEDS`. `ntfy.py` and `mic.py` are fully standalone and designed to be reused by any terminal visualizer. `engine.render` is the importable rendering pipeline for non-terminal targets.
**Poetry mode** pulls from Project Gutenberg: Whitman, Dickinson, Thoreau, Emerson. Sources are in `engine/sources.py``POETRY_SOURCES`. ### NtfyPoller
---
## ntfy.sh Integration
Mainline polls a configurable ntfy.sh topic in the background. When a message arrives, the scroll pauses and the message renders full-screen for `MESSAGE_DISPLAY_SECS` seconds, then the stream resumes.
To push a message:
```bash
curl -d "Body text" -H "Title: Alert title" https://ntfy.sh/your_topic
```
Update `NTFY_TOPIC` in `engine/config.py` to point at your own topic. The `NtfyPoller` class is fully standalone and can be reused by other visualizers:
```python ```python
from engine.ntfy import NtfyPoller from engine.ntfy import NtfyPoller
poller = NtfyPoller("https://ntfy.sh/my_topic/json?since=20s&poll=1")
poller = NtfyPoller("https://ntfy.sh/my_topic/json")
poller.start() poller.start()
# in render loop:
msg = poller.get_active_message() # returns (title, body, timestamp) or None # in your render loop:
msg = poller.get_active_message() # → (title, body, timestamp) or None
if msg:
title, body, ts = msg
render_my_message(title, body) # visualizer-specific
``` ```
Dependencies: `urllib.request`, `json`, `threading`, `time` — stdlib only. The `since=` parameter is managed automatically on reconnect.
### MicMonitor
```python
from engine.mic import MicMonitor
mic = MicMonitor(threshold_db=50)
result = mic.start() # None = sounddevice unavailable; False = stream failed; True = ok
if result:
excess = mic.excess # dB above threshold, clamped to 0
db = mic.db # raw RMS dB level
```
Dependencies: `sounddevice`, `numpy` — both optional; degrades gracefully if unavailable.
### Render pipeline
`engine.render` exposes the OTF → raster pipeline independently of the terminal scroll loop. The planned `serve.py` extension will import it directly to pre-render headlines as 1-bit bitmaps for an ESP32 thin client:
```python
# planned — serve.py does not yet exist
from engine.render import render_line, big_wrap
from engine.fetch import fetch_all
headlines = fetch_all()
for h in headlines:
rows = big_wrap(h.text, font, width=800) # list of half-block rows
# threshold to 1-bit, pack bytes, serve over HTTP
```
See `Mainline Renderer + ntfy Message Queue for ESP32.md` for the full server + thin client architecture.
--- ---
## Ideas / Future ## Development
### Setup
Requires Python 3.10+ and [uv](https://docs.astral.sh/uv/).
```bash
uv sync # minimal (no mic)
uv sync --all-extras # with mic support (sounddevice + numpy)
uv sync --all-extras --group dev # full dev environment
```
### Tasks
With [mise](https://mise.jdx.dev/):
```bash
mise run test # run test suite
mise run test-cov # run with coverage report
mise run lint # ruff check
mise run lint-fix # ruff check --fix
mise run format # ruff format
mise run run # uv run mainline.py
mise run run-poetry # uv run mainline.py --poetry
mise run run-firehose # uv run mainline.py --firehose
```
### Testing
Tests live in `tests/` and cover `config`, `filter`, `mic`, `ntfy`, `sources`, and `terminal`.
```bash
uv run pytest
uv run pytest --cov=engine --cov-report=term-missing
```
### Linting
```bash
uv run ruff check engine/ mainline.py
uv run ruff format engine/ mainline.py
```
Pre-commit hooks run lint automatically via `hk`.
---
## Roadmap
### Performance ### Performance
- **Concurrent feed fetching** — startup currently blocks sequentially on ~25 HTTP requests; `concurrent.futures.ThreadPoolExecutor` would cut load time to the slowest single feed - **Concurrent feed fetching** — startup currently blocks sequentially on ~25 HTTP requests; `concurrent.futures.ThreadPoolExecutor` would cut load time to the slowest single feed
@@ -211,5 +256,4 @@ msg = poller.get_active_message() # returns (title, body, timestamp) or None
--- ---
*macOS only (script/system font paths for translation are hardcoded). Primary display font is user-selectable via the bundled `fonts/` picker. Python 3.9+.* *macOS only (script/system font paths for translation are hardcoded). Primary display font is user-selectable via the bundled `fonts/` picker. Python 3.10+.*
# test

View File

@@ -0,0 +1,145 @@
# README Update Design — 2026-03-15
## Goal
Restructure and expand `README.md` to:
1. Align with the current codebase (Python 3.10+, uv/mise/pytest/ruff toolchain, 6 new fonts)
2. Add extensibility-focused content (`Extending` section)
3. Add developer workflow coverage (`Development` section)
4. Improve navigability via top-level grouping (Approach C)
---
## Proposed Structure
```
# MAINLINE
> tagline + description
## Using
### Run
### Config
### Feeds
### Fonts
### ntfy.sh
## Internals
### How it works
### Architecture
## Extending
### NtfyPoller
### MicMonitor
### Render pipeline
## Development
### Setup
### Tasks
### Testing
### Linting
## Roadmap
---
*footer*
```
---
## Section-by-section design
### Using
All existing content preserved verbatim. Two changes:
- **Run**: add `uv run mainline.py` as an alternative invocation; expand bootstrap note to mention `uv sync` / `uv sync --all-extras`
- **ntfy.sh**: remove `NtfyPoller` reuse code example (moves to Extending); keep push instructions and topic config
Subsections moved into Using (currently standalone):
- `Feeds` — it's configuration, not a concept
- `ntfy.sh` (usage half)
### Internals
All existing content preserved verbatim. One change:
- **Architecture**: append `tests/` directory listing to the module tree
### Extending
Entirely new section. Three subsections:
**NtfyPoller**
- Minimal working import + usage example
- Note: stdlib only dependencies
```python
from engine.ntfy import NtfyPoller
poller = NtfyPoller("https://ntfy.sh/my_topic/json?since=20s&poll=1")
poller.start()
# in your render loop:
msg = poller.get_active_message() # → (title, body, timestamp) or None
if msg:
title, body, ts = msg
render_my_message(title, body) # visualizer-specific
```
**MicMonitor**
- Minimal working import + usage example
- Note: sounddevice/numpy optional, degrades gracefully
```python
from engine.mic import MicMonitor
mic = MicMonitor(threshold_db=50)
if mic.start(): # returns False if sounddevice unavailable
excess = mic.excess # dB above threshold, clamped to 0
db = mic.db # raw RMS dB level
```
**Render pipeline**
- Brief prose about `engine.render` as importable pipeline
- Minimal sketch of serve.py / ESP32 usage pattern
- Reference to `Mainline Renderer + ntfy Message Queue for ESP32.md`
### Development
Entirely new section. Four subsections:
**Setup**
- Hard requirements: Python 3.10+, uv
- `uv sync` / `uv sync --all-extras` / `uv sync --group dev`
**Tasks** (via mise)
- `mise run test`, `test-cov`, `lint`, `lint-fix`, `format`, `run`, `run-poetry`, `run-firehose`
**Testing**
- Tests in `tests/` covering config, filter, mic, ntfy, sources, terminal
- `uv run pytest` and `uv run pytest --cov=engine --cov-report=term-missing`
**Linting**
- `uv run ruff check` and `uv run ruff format`
- Note: pre-commit hooks run lint via `hk`
### Roadmap
Existing `## Ideas / Future` content preserved verbatim. Only change: rename heading to `## Roadmap`.
### Footer
Update `Python 3.9+``Python 3.10+`.
---
## Files changed
- `README.md` — restructured and expanded as above
- No other files
---
## What is not changing
- All existing prose, examples, and config table values — preserved verbatim where retained
- The Ideas/Future content — kept intact under the new Roadmap heading
- The cyberpunk voice and terse style of the existing README

View File

@@ -11,8 +11,10 @@ import time
import tty import tty
from engine import config, render from engine import config, render
from engine.controller import StreamController
from engine.fetch import fetch_all, fetch_poetry, load_cache, save_cache from engine.fetch import fetch_all, fetch_poetry, load_cache, save_cache
from engine.mic import MicMonitor
from engine.ntfy import NtfyPoller
from engine.scroll import stream
from engine.terminal import ( from engine.terminal import (
CLR, CLR,
CURSOR_OFF, CURSOR_OFF,
@@ -247,110 +249,6 @@ def pick_font_face():
print() print()
def pick_effects_config():
"""Interactive picker for configuring effects pipeline."""
import effects_plugins
from engine.effects import get_effect_chain, get_registry
effects_plugins.discover_plugins()
registry = get_registry()
chain = get_effect_chain()
chain.set_order(["noise", "fade", "glitch", "firehose"])
effects = list(registry.list_all().values())
if not effects:
return
selected = 0
editing_intensity = False
intensity_value = 1.0
def _draw_effects_picker():
w = tw()
print(CLR, end="")
print("\033[1;1H", end="")
print(" \033[1;38;5;231mEFFECTS CONFIG\033[0m")
print(f" \033[2;38;5;37m{'' * (w - 4)}\033[0m")
print()
for i, effect in enumerate(effects):
prefix = " > " if i == selected else " "
marker = "[*]" if effect.config.enabled else "[ ]"
if editing_intensity and i == selected:
print(
f"{prefix}{marker} \033[1;38;5;82m{effect.name}\033[0m: intensity={intensity_value:.2f} (use +/- to adjust, Enter to confirm)"
)
else:
print(
f"{prefix}{marker} {effect.name}: intensity={effect.config.intensity:.2f}"
)
print()
print(f" \033[2;38;5;37m{'' * (w - 4)}\033[0m")
print(
" \033[38;5;245mControls: space=toggle on/off | +/-=adjust intensity | arrows=move | Enter=next effect | q=done\033[0m"
)
def _read_effects_key():
ch = sys.stdin.read(1)
if ch == "\x03":
return "interrupt"
if ch in ("\r", "\n"):
return "enter"
if ch == " ":
return "toggle"
if ch == "q":
return "quit"
if ch == "+" or ch == "=":
return "up"
if ch == "-" or ch == "_":
return "down"
if ch == "\x1b":
c1 = sys.stdin.read(1)
if c1 != "[":
return None
c2 = sys.stdin.read(1)
if c2 == "A":
return "up"
if c2 == "B":
return "down"
return None
return None
if not sys.stdin.isatty():
return
fd = sys.stdin.fileno()
old_settings = termios.tcgetattr(fd)
try:
tty.setcbreak(fd)
while True:
_draw_effects_picker()
key = _read_effects_key()
if key == "quit" or key == "enter":
break
elif key == "up" and editing_intensity:
intensity_value = min(1.0, intensity_value + 0.1)
effects[selected].config.intensity = intensity_value
elif key == "down" and editing_intensity:
intensity_value = max(0.0, intensity_value - 0.1)
effects[selected].config.intensity = intensity_value
elif key == "up":
selected = max(0, selected - 1)
intensity_value = effects[selected].config.intensity
elif key == "down":
selected = min(len(effects) - 1, selected + 1)
intensity_value = effects[selected].config.intensity
elif key == "toggle":
effects[selected].config.enabled = not effects[selected].config.enabled
elif key == "interrupt":
raise KeyboardInterrupt
finally:
termios.tcsetattr(fd, termios.TCSADRAIN, old_settings)
def main(): def main():
atexit.register(lambda: print(CURSOR_ON, end="", flush=True)) atexit.register(lambda: print(CURSOR_ON, end="", flush=True))
@@ -361,13 +259,10 @@ def main():
signal.signal(signal.SIGINT, handle_sigint) signal.signal(signal.SIGINT, handle_sigint)
StreamController.warmup_topics()
w = tw() w = tw()
print(CLR, end="") print(CLR, end="")
print(CURSOR_OFF, end="") print(CURSOR_OFF, end="")
pick_font_face() pick_font_face()
pick_effects_config()
w = tw() w = tw()
print() print()
time.sleep(0.4) time.sleep(0.4)
@@ -419,10 +314,9 @@ def main():
sys.exit(1) sys.exit(1)
print() print()
controller = StreamController() mic = MicMonitor(threshold_db=config.MIC_THRESHOLD_DB)
mic_ok, ntfy_ok = controller.initialize_sources() mic_ok = mic.start()
if mic.available:
if controller.mic and controller.mic.available:
boot_ln( boot_ln(
"Microphone", "Microphone",
"ACTIVE" "ACTIVE"
@@ -431,6 +325,12 @@ def main():
bool(mic_ok), bool(mic_ok),
) )
ntfy = NtfyPoller(
config.NTFY_TOPIC,
reconnect_delay=config.NTFY_RECONNECT_DELAY,
display_secs=config.MESSAGE_DISPLAY_SECS,
)
ntfy_ok = ntfy.start()
boot_ln("ntfy", "LISTENING" if ntfy_ok else "OFFLINE", ntfy_ok) boot_ln("ntfy", "LISTENING" if ntfy_ok else "OFFLINE", ntfy_ok)
if config.FIREHOSE: if config.FIREHOSE:
@@ -443,7 +343,7 @@ def main():
print() print()
time.sleep(0.4) time.sleep(0.4)
controller.run(items) stream(items, ntfy, mic)
print() print()
print(f" {W_GHOST}{'' * (tw() - 4)}{RST}") print(f" {W_GHOST}{'' * (tw() - 4)}{RST}")

View File

@@ -105,8 +105,6 @@ class Config:
firehose: bool = False firehose: bool = False
ntfy_topic: str = "https://ntfy.sh/klubhaus_terminal_mainline/json" ntfy_topic: str = "https://ntfy.sh/klubhaus_terminal_mainline/json"
ntfy_cc_cmd_topic: str = "https://ntfy.sh/klubhaus_terminal_mainline_cc_cmd/json"
ntfy_cc_resp_topic: str = "https://ntfy.sh/klubhaus_terminal_mainline_cc_resp/json"
ntfy_reconnect_delay: int = 5 ntfy_reconnect_delay: int = 5
message_display_secs: int = 30 message_display_secs: int = 30
@@ -150,8 +148,6 @@ class Config:
mode="poetry" if "--poetry" in argv or "-p" in argv else "news", mode="poetry" if "--poetry" in argv or "-p" in argv else "news",
firehose="--firehose" in argv, firehose="--firehose" in argv,
ntfy_topic="https://ntfy.sh/klubhaus_terminal_mainline/json", ntfy_topic="https://ntfy.sh/klubhaus_terminal_mainline/json",
ntfy_cc_cmd_topic="https://ntfy.sh/klubhaus_terminal_mainline_cc_cmd/json",
ntfy_cc_resp_topic="https://ntfy.sh/klubhaus_terminal_mainline_cc_resp/json",
ntfy_reconnect_delay=5, ntfy_reconnect_delay=5,
message_display_secs=30, message_display_secs=30,
font_dir=font_dir, font_dir=font_dir,
@@ -197,8 +193,6 @@ FIREHOSE = "--firehose" in sys.argv
# ─── NTFY MESSAGE QUEUE ────────────────────────────────── # ─── NTFY MESSAGE QUEUE ──────────────────────────────────
NTFY_TOPIC = "https://ntfy.sh/klubhaus_terminal_mainline/json" NTFY_TOPIC = "https://ntfy.sh/klubhaus_terminal_mainline/json"
NTFY_CC_CMD_TOPIC = "https://ntfy.sh/klubhaus_terminal_mainline_cc_cmd/json"
NTFY_CC_RESP_TOPIC = "https://ntfy.sh/klubhaus_terminal_mainline_cc_resp/json"
NTFY_RECONNECT_DELAY = 5 # seconds before reconnecting after a dropped stream NTFY_RECONNECT_DELAY = 5 # seconds before reconnecting after a dropped stream
MESSAGE_DISPLAY_SECS = 30 # how long a message holds the screen MESSAGE_DISPLAY_SECS = 30 # how long a message holds the screen

View File

@@ -3,7 +3,6 @@ Stream controller - manages input sources and orchestrates the render stream.
""" """
from engine.config import Config, get_config from engine.config import Config, get_config
from engine.effects.controller import handle_effects_command
from engine.eventbus import EventBus from engine.eventbus import EventBus
from engine.events import EventType, StreamEvent from engine.events import EventType, StreamEvent
from engine.mic import MicMonitor from engine.mic import MicMonitor
@@ -14,45 +13,11 @@ from engine.scroll import stream
class StreamController: class StreamController:
"""Controls the stream lifecycle - initializes sources and runs the stream.""" """Controls the stream lifecycle - initializes sources and runs the stream."""
_topics_warmed = False
def __init__(self, config: Config | None = None, event_bus: EventBus | None = None): def __init__(self, config: Config | None = None, event_bus: EventBus | None = None):
self.config = config or get_config() self.config = config or get_config()
self.event_bus = event_bus self.event_bus = event_bus
self.mic: MicMonitor | None = None self.mic: MicMonitor | None = None
self.ntfy: NtfyPoller | None = None self.ntfy: NtfyPoller | None = None
self.ntfy_cc: NtfyPoller | None = None
@classmethod
def warmup_topics(cls) -> None:
"""Warm up ntfy topics lazily (creates them if they don't exist)."""
if cls._topics_warmed:
return
import urllib.request
topics = [
"https://ntfy.sh/klubhaus_terminal_mainline_cc_cmd",
"https://ntfy.sh/klubhaus_terminal_mainline_cc_resp",
"https://ntfy.sh/klubhaus_terminal_mainline",
]
for topic in topics:
try:
req = urllib.request.Request(
topic,
data=b"init",
headers={
"User-Agent": "mainline/0.1",
"Content-Type": "text/plain",
},
method="POST",
)
urllib.request.urlopen(req, timeout=5)
except Exception:
pass
cls._topics_warmed = True
def initialize_sources(self) -> tuple[bool, bool]: def initialize_sources(self) -> tuple[bool, bool]:
"""Initialize microphone and ntfy sources. """Initialize microphone and ntfy sources.
@@ -70,38 +35,7 @@ class StreamController:
) )
ntfy_ok = self.ntfy.start() ntfy_ok = self.ntfy.start()
self.ntfy_cc = NtfyPoller( return bool(mic_ok), ntfy_ok
self.config.ntfy_cc_cmd_topic,
reconnect_delay=self.config.ntfy_reconnect_delay,
display_secs=5,
)
self.ntfy_cc.subscribe(self._handle_cc_message)
ntfy_cc_ok = self.ntfy_cc.start()
return bool(mic_ok), ntfy_ok and ntfy_cc_ok
def _handle_cc_message(self, event) -> None:
"""Handle incoming C&C message - like a serial port control interface."""
import urllib.request
cmd = event.body.strip() if hasattr(event, "body") else str(event).strip()
if not cmd.startswith("/"):
return
response = handle_effects_command(cmd)
topic_url = self.config.ntfy_cc_resp_topic.replace("/json", "")
data = response.encode("utf-8")
req = urllib.request.Request(
topic_url,
data=data,
headers={"User-Agent": "mainline/0.1", "Content-Type": "text/plain"},
method="POST",
)
try:
urllib.request.urlopen(req, timeout=5)
except Exception:
pass
def run(self, items: list) -> None: def run(self, items: list) -> None:
"""Run the stream with initialized sources.""" """Run the stream with initialized sources."""

View File

@@ -25,41 +25,24 @@ run = "uv run mainline.py"
run-poetry = "uv run mainline.py --poetry" run-poetry = "uv run mainline.py --poetry"
run-firehose = "uv run mainline.py --firehose" run-firehose = "uv run mainline.py --firehose"
daemon = "nohup uv run mainline.py > /dev/null 2>&1 &"
daemon-stop = "pkill -f 'uv run mainline.py' 2>/dev/null || true"
daemon-restart = "mise run daemon-stop && sleep 2 && mise run daemon"
# =====================
# Command & Control
# =====================
cmd = "uv run cmdline.py"
cmd-stats = "bash -c 'uv run cmdline.py -w \"/effects stats\"';:"
# Initialize ntfy topics (warm up before first use - also done automatically by mainline)
topics-init = "curl -s -d 'init' https://ntfy.sh/klubhaus_terminal_mainline_cc_cmd > /dev/null && curl -s -d 'init' https://ntfy.sh/klubhaus_terminal_mainline_cc_resp > /dev/null && curl -s -d 'init' https://ntfy.sh/klubhaus_terminal_mainline > /dev/null"
# ===================== # =====================
# Environment # Environment
# ===================== # =====================
sync = "uv sync" sync = "uv sync"
sync-all = "uv sync --all-extras" sync-all = "uv sync --all-extras"
install = "mise run sync" install = "uv sync"
install-dev = "mise run sync && uv sync --group dev" install-dev = "uv sync --group dev"
bootstrap = "uv sync && uv run mainline.py --help" bootstrap = "uv sync && uv run mainline.py --help"
clean = "rm -rf .venv htmlcov .coverage tests/.pytest_cache .mainline_cache_*.json nohup.out" clean = "rm -rf .venv htmlcov .coverage tests/.pytest_cache"
# Aggressive cleanup - removes all generated files, caches, and venv
clobber = "git clean -fdx && rm -rf .venv htmlcov .coverage tests/.pytest_cache .mainline_cache_*.json nohup.out"
# ===================== # =====================
# CI/CD # CI/CD
# ===================== # =====================
ci = "mise run topics-init && uv sync --group dev && uv run pytest --cov=engine --cov-report=term-missing --cov-report=xml" ci = "uv sync --group dev && uv run pytest --cov=engine --cov-report=term-missing --cov-report=xml"
ci-lint = "uv run ruff check engine/ mainline.py" ci-lint = "uv run ruff check engine/ mainline.py"
# ===================== # =====================

View File

@@ -1,127 +0,0 @@
"""
Integration tests for ntfy topics.
"""
import json
import time
import urllib.request
class TestNtfyTopics:
def test_cc_cmd_topic_exists_and_writable(self):
"""Verify C&C CMD topic exists and accepts messages."""
from engine.config import NTFY_CC_CMD_TOPIC
topic_url = NTFY_CC_CMD_TOPIC.replace("/json", "")
test_message = f"test_{int(time.time())}"
req = urllib.request.Request(
topic_url,
data=test_message.encode("utf-8"),
headers={
"User-Agent": "mainline-test/0.1",
"Content-Type": "text/plain",
},
method="POST",
)
try:
with urllib.request.urlopen(req, timeout=10) as resp:
assert resp.status == 200
except Exception as e:
raise AssertionError(f"Failed to write to C&C CMD topic: {e}") from e
def test_cc_resp_topic_exists_and_writable(self):
"""Verify C&C RESP topic exists and accepts messages."""
from engine.config import NTFY_CC_RESP_TOPIC
topic_url = NTFY_CC_RESP_TOPIC.replace("/json", "")
test_message = f"test_{int(time.time())}"
req = urllib.request.Request(
topic_url,
data=test_message.encode("utf-8"),
headers={
"User-Agent": "mainline-test/0.1",
"Content-Type": "text/plain",
},
method="POST",
)
try:
with urllib.request.urlopen(req, timeout=10) as resp:
assert resp.status == 200
except Exception as e:
raise AssertionError(f"Failed to write to C&C RESP topic: {e}") from e
def test_message_topic_exists_and_writable(self):
"""Verify message topic exists and accepts messages."""
from engine.config import NTFY_TOPIC
topic_url = NTFY_TOPIC.replace("/json", "")
test_message = f"test_{int(time.time())}"
req = urllib.request.Request(
topic_url,
data=test_message.encode("utf-8"),
headers={
"User-Agent": "mainline-test/0.1",
"Content-Type": "text/plain",
},
method="POST",
)
try:
with urllib.request.urlopen(req, timeout=10) as resp:
assert resp.status == 200
except Exception as e:
raise AssertionError(f"Failed to write to message topic: {e}") from e
def test_cc_cmd_topic_readable(self):
"""Verify we can read messages from C&C CMD topic."""
from engine.config import NTFY_CC_CMD_TOPIC
test_message = f"integration_test_{int(time.time())}"
topic_url = NTFY_CC_CMD_TOPIC.replace("/json", "")
req = urllib.request.Request(
topic_url,
data=test_message.encode("utf-8"),
headers={
"User-Agent": "mainline-test/0.1",
"Content-Type": "text/plain",
},
method="POST",
)
try:
urllib.request.urlopen(req, timeout=10)
except Exception as e:
raise AssertionError(f"Failed to write to C&C CMD topic: {e}") from e
time.sleep(1)
poll_url = f"{NTFY_CC_CMD_TOPIC}?poll=1&limit=1"
req = urllib.request.Request(
poll_url,
headers={"User-Agent": "mainline-test/0.1"},
)
try:
with urllib.request.urlopen(req, timeout=10) as resp:
body = resp.read().decode("utf-8")
if body.strip():
data = json.loads(body.split("\n")[0])
assert isinstance(data, dict)
except Exception as e:
raise AssertionError(f"Failed to read from C&C CMD topic: {e}") from e
def test_topics_are_different(self):
"""Verify C&C CMD/RESP and message topics are different."""
from engine.config import NTFY_CC_CMD_TOPIC, NTFY_CC_RESP_TOPIC, NTFY_TOPIC
assert NTFY_CC_CMD_TOPIC != NTFY_TOPIC
assert NTFY_CC_RESP_TOPIC != NTFY_TOPIC
assert NTFY_CC_CMD_TOPIC != NTFY_CC_RESP_TOPIC
assert "_cc_cmd" in NTFY_CC_CMD_TOPIC
assert "_cc_resp" in NTFY_CC_RESP_TOPIC