genewildish/Mainline
1
0
Fork 1
Code Issues Pull Requests 4 Packages Projects Releases Wiki Activity

docs: use david's most recent README from feature/vector_display branch

Browse Source
This commit is contained in:
Gene Johnson
2026-03-18 23:20:04 -07:00
parent f40651b34d
commit 509cee6312
1 changed files with 102 additions and 111 deletions
Download Patch File Download Diff File Expand all files Collapse all files

213
README.md
View File

@@ -11,18 +11,15 @@ A full-screen terminal news ticker that renders live global headlines in large O
- [Using](#using)
- [Run](#run)
- [Config](#config)
- [Display Modes](#display-modes)
- [Feeds](#feeds)
- [Fonts](#fonts)
- [Color Schemes](#color-schemes)
- [ntfy.sh](#ntfysh)
- [Figment Mode](#figment-mode)
- [Command & Control](#command--control-cc)
- [Internals](#internals)
- [How it works](#how-it-works)
- [Architecture](#architecture)
- [Extending](#extending)
- [NtfyPoller](#ntfypoller)
- [MicMonitor](#micmonitor)
- [Render pipeline](#render-pipeline)
- [Development](#development)
- [Setup](#setup)
- [Tasks](#tasks)
@@ -47,6 +44,8 @@ python3 mainline.py -p # same
python3 mainline.py --firehose # dense rapid-fire headline mode
python3 mainline.py --figment # enable periodic SVG glyph overlays
python3 mainline.py --figment-interval 30 # figment every 30 seconds (default: 60)
python3 mainline.py --display websocket # web browser display only
python3 mainline.py --display both # terminal + web browser
python3 mainline.py --refresh # force re-fetch (bypass cache)
python3 mainline.py --no-font-picker # skip interactive font picker
python3 mainline.py --font-file path.otf # use a specific font file
@@ -60,7 +59,20 @@ Or with uv:
uv run mainline.py
```
First run bootstraps a local `.mainline_venv/` and installs deps (`feedparser`, `Pillow`, `sounddevice`, `numpy`). Subsequent runs start immediately, loading from cache. With uv, run `uv sync` or `uv sync --all-extras` (includes mic support) instead.
First run bootstraps dependencies. Use `uv sync --all-extras` for mic support.
### Command & Control (C&C)
Control mainline remotely using `cmdline.py`:
```bash
uv run cmdline.py # Interactive TUI
uv run cmdline.py /effects list # List all effects
uv run cmdline.py /effects stats # Show performance stats
uv run cmdline.py -w /effects stats # Watch mode (auto-refresh)
```
Commands are sent via ntfy.sh topics - useful for controlling a daemonized mainline instance.
### Config
@@ -71,21 +83,33 @@ All constants live in `engine/config.py`:
| `HEADLINE_LIMIT` | `1000` | Total headlines per session |
| `FEED_TIMEOUT` | `10` | Per-feed HTTP timeout (seconds) |
| `MIC_THRESHOLD_DB` | `50` | dB floor above which glitches spike |
| `NTFY_TOPIC` | klubhaus URL | ntfy.sh JSON stream for messages |
| `NTFY_CC_CMD_TOPIC` | klubhaus URL | ntfy.sh topic for C&C commands |
| `NTFY_CC_RESP_TOPIC` | klubhaus URL | ntfy.sh topic for C&C responses |
| `NTFY_RECONNECT_DELAY` | `5` | Seconds before reconnecting after dropped SSE |
| `MESSAGE_DISPLAY_SECS` | `30` | How long an ntfy message holds the screen |
| `FONT_DIR` | `fonts/` | Folder scanned for `.otf`, `.ttf`, `.ttc` files |
| `FONT_PATH` | first file in `FONT_DIR` | Active display font (overridden by picker or `--font-file`) |
| `FONT_INDEX` | `0` | Face index within a font collection file |
| `FONT_PICKER` | `True` | Show interactive font picker at boot (`--no-font-picker` to skip) |
| `FONT_PATH` | first file in `FONT_DIR` | Active display font |
| `FONT_PICKER` | `True` | Show interactive font picker at boot |
| `FONT_SZ` | `60` | Font render size (affects block density) |
| `RENDER_H` | `8` | Terminal rows per headline line |
| `SSAA` | `4` | Super-sampling factor (render at 4× then downsample) |
| `SSAA` | `4` | Super-sampling factor |
| `SCROLL_DUR` | `5.625` | Seconds per headline |
| `FRAME_DT` | `0.05` | Frame interval in seconds (20 FPS) |
| `GRAD_SPEED` | `0.08` | Gradient sweep speed (cycles/sec, ~12s full sweep) |
| `FIREHOSE_H` | `12` | Firehose zone height (terminal rows) |
| `GRAD_SPEED` | `0.08` | Gradient sweep speed |
| `FIGMENT_INTERVAL` | `60` | Seconds between figment appearances (set by `--figment-interval`) |
| `NTFY_TOPIC` | klubhaus URL | ntfy.sh JSON stream endpoint |
| `NTFY_RECONNECT_DELAY` | `5` | Seconds before reconnecting after a dropped SSE stream |
| `MESSAGE_DISPLAY_SECS` | `30` | How long an ntfy message holds the screen |
### Display Modes
Mainline supports multiple display backends:
- **Terminal** (`--display terminal`): ANSI terminal output (default)
- **WebSocket** (`--display websocket`): Stream to web browser clients
- **Sixel** (`--display sixel`): Sixel graphics in supported terminals (iTerm2, mintty)
- **Both** (`--display both`): Terminal + WebSocket simultaneously
WebSocket mode serves a web client at http://localhost:8766 with ANSI color support and fullscreen mode.
### Feeds
@@ -95,30 +119,15 @@ All constants live in `engine/config.py`:
### Fonts
A `fonts/` directory is bundled with demo faces (AgorTechnoDemo, AlphatronDemo, CSBishopDrawn, CubaTechnologyDemo, CyberformDemo, KATA, Microbots, ModernSpaceDemo, Neoform, Pixel Sparta, RaceHugoDemo, Resond, Robocops, Synthetix, Xeonic, and others). On startup, an interactive picker lists all discovered faces with a live half-block preview rendered at your configured size.
A `fonts/` directory is bundled with demo faces. On startup, an interactive picker lists all discovered faces with a live half-block preview.
Navigation: `↑`/`↓` or `j`/`k` to move, `Enter` or `q` to select. The selected face persists for that session.
Navigation: `↑`/`↓` or `j`/`k` to move, `Enter` or `q` to select.
To add your own fonts, drop `.otf`, `.ttf`, or `.ttc` files into `fonts/` (or point `--font-dir` at any other folder). Font collections (`.ttc`, multi-face `.otf`) are enumerated face-by-face.
### Color Schemes
Mainline supports three color themes for the scrolling gradient: **Verdant Green**, **Molten Orange**, and **Violet Purple**. Each theme uses a precise color-opposite palette for ntfy message queue rendering (magenta, blue, and yellow respectively).
On startup, an interactive picker presents all available color schemes:
```
[1] Verdant Green (white-hot → deep green)
[2] Molten Orange (white-hot → deep orange)
[3] Violet Purple (white-hot → deep purple)
```
Navigation: `↑`/`↓` or `j`/`k` to move, `Enter` or `q` to select. The selection applies only to the current session; you'll pick a fresh theme each run.
**Note:** The boot UI (title, status lines, font picker menu) uses a hardcoded green accent color for visual continuity. Only the scrolling headlines and incoming messages render in the selected theme gradient.
To add your own fonts, drop `.otf`, `.ttf`, or `.ttc` files into `fonts/`.
### ntfy.sh
Mainline polls a configurable ntfy.sh topic in the background. When a message arrives, the scroll pauses and the message renders full-screen for `MESSAGE_DISPLAY_SECS` seconds, then the stream resumes.
Mainline polls a configurable ntfy.sh topic in the background. When a message arrives, the scroll pauses and the message renders full-screen.
To push a message:
@@ -135,7 +144,7 @@ Figment mode periodically overlays a full-screen SVG glyph on the running ticker
**Enable it** with the `--figment` flag:
```bash
uv run mainline.py --figment # glyph every 60 seconds (default)
uv run mainline.py --figment # glyph every 60 seconds (default)
uv run mainline.py --figment --figment-interval 30 # every 30 seconds
```
@@ -174,24 +183,28 @@ uv sync --extra figment # adds cairosvg
- Subject-region detection runs a regex pass on each headline; matches trigger a Google Translate call and font swap to the appropriate script (CJK, Arabic, Devanagari, etc.) using macOS system fonts
- The mic stream runs in a background thread, feeding RMS dB into the glitch probability calculation each frame
- The viewport scrolls through a virtual canvas of pre-rendered blocks; fade zones at top and bottom dissolve characters probabilistically
- An ntfy.sh SSE stream runs in a background thread; incoming messages interrupt the scroll and render full-screen until dismissed or expired
- An ntfy.sh SSE stream runs in a background thread for messages and C&C commands; incoming messages interrupt the scroll and render full-screen until dismissed or expired
- Figment mode rasterizes SVGs via cairosvg → PIL → greyscale → half-block encode, then overlays them with ANSI cursor-positioning commands between the effect chain and the ntfy message layer
### Architecture
`mainline.py` is a thin entrypoint (venv bootstrap → `engine.app.main()`). All logic lives in the `engine/` package:
```
engine/
__init__.py package marker
app.py main(), font picker TUI, boot sequence, signal handler
app.py main(), font picker TUI, boot sequence, C&C poller
config.py constants, CLI flags, glyph tables
sources.py FEEDS, POETRY_SOURCES, language/script maps
terminal.py ANSI codes, tw/th, type_out, boot_ln
filter.py HTML stripping, content filter
translate.py Google Translate wrapper + region detection
render.py OTF → half-block pipeline (SSAA, gradient)
effects.py noise, glitch_bar, fade, firehose
effects/ plugin architecture for visual effects
types.py EffectPlugin ABC, EffectConfig, EffectContext
registry.py effect registration and lookup
chain.py effect pipeline chaining
controller.py handles /effects commands
performance.py performance monitoring
legacy.py legacy functional effects
fetch.py RSS/Gutenberg fetching + cache load/save
ntfy.py NtfyPoller — standalone, zero internal deps
mic.py MicMonitor — standalone, graceful fallback
@@ -207,6 +220,15 @@ engine/
emitters.py background emitters for ntfy and mic
types.py type definitions and dataclasses
themes.py THEME_REGISTRY — gradient color definitions
display/ Display backend system
__init__.py DisplayRegistry, get_monitor
backends/
terminal.py ANSI terminal display
websocket.py WebSocket server for browser clients
sixel.py Sixel graphics (pure Python)
null.py headless display for testing
multi.py forwards to multiple displays
benchmark.py performance benchmarking tool
effects_plugins/
__init__.py plugin discovery (ABC issubclass scan)
@@ -219,62 +241,6 @@ effects_plugins/
figments/ SVG assets for figment mode
```
`ntfy.py` and `mic.py` have zero internal dependencies and can be imported by any other visualizer.
---
## Extending
`ntfy.py` and `mic.py` are fully standalone and designed to be reused by any terminal visualizer. `engine.render` is the importable rendering pipeline for non-terminal targets.
### NtfyPoller
```python
from engine.ntfy import NtfyPoller
poller = NtfyPoller("https://ntfy.sh/my_topic/json")
poller.start()
# in your render loop:
msg = poller.get_active_message() # → (title, body, timestamp) or None
if msg:
title, body, ts = msg
render_my_message(title, body) # visualizer-specific
```
Dependencies: `urllib.request`, `json`, `threading`, `time` — stdlib only. The `since=` parameter is managed automatically on reconnect.
### MicMonitor
```python
from engine.mic import MicMonitor
mic = MicMonitor(threshold_db=50)
result = mic.start() # None = sounddevice unavailable; False = stream failed; True = ok
if result:
excess = mic.excess # dB above threshold, clamped to 0
db = mic.db # raw RMS dB level
```
Dependencies: `sounddevice`, `numpy` — both optional; degrades gracefully if unavailable.
### Render pipeline
`engine.render` exposes the OTF → raster pipeline independently of the terminal scroll loop. The planned `serve.py` extension will import it directly to pre-render headlines as 1-bit bitmaps for an ESP32 thin client:
```python
# planned — serve.py does not yet exist
from engine.render import render_line, big_wrap
from engine.fetch import fetch_all
headlines = fetch_all()
for h in headlines:
rows = big_wrap(h.text, font, width=800) # list of half-block rows
# threshold to 1-bit, pack bytes, serve over HTTP
```
See `Mainline Renderer + ntfy Message Queue for ESP32.md` for the full server + thin client architecture.
---
## Development
@@ -285,7 +251,7 @@ Requires Python 3.10+ and [uv](https://docs.astral.sh/uv/).
```bash
uv sync # minimal (no mic, no figment)
uv sync --extras mic # with mic support (sounddevice + numpy)
uv sync --extra mic # with mic support (sounddevice + numpy)
uv sync --extra figment # with figment mode (cairosvg + system Cairo)
uv sync --all-extras # all optional features
uv sync --all-extras --group dev # full dev environment
@@ -299,13 +265,25 @@ With [mise](https://mise.jdx.dev/):
```bash
mise run test # run test suite
mise run test-cov # run with coverage report
mise run lint # ruff check
mise run lint-fix # ruff check --fix
mise run format # ruff format
mise run run # uv run mainline.py
mise run run-poetry # uv run mainline.py --poetry
mise run run-firehose # uv run mainline.py --firehose
mise run test-cov # run with coverage report
mise run lint # ruff check
mise run lint-fix # ruff check --fix
mise run format # ruff format
mise run run # terminal display
mise run run-websocket # web display only
mise run run-sixel # sixel graphics
mise run run-both # terminal + web
mise run run-client # both + open browser
mise run cmd # C&C command interface
mise run cmd-stats # watch effects stats
mise run benchmark # run performance benchmarks
mise run benchmark-json # save as JSON
mise run topics-init # initialize ntfy topics
```
### Testing
@@ -315,8 +293,21 @@ Tests live in `tests/` and cover `config`, `filter`, `mic`, `ntfy`, `sources`, `
```bash
uv run pytest
uv run pytest --cov=engine --cov-report=term-missing
# Run with mise
mise run test
mise run test-cov
# Run performance benchmarks
mise run benchmark
mise run benchmark-json
# Run benchmark hook mode (for CI)
uv run python -m engine.benchmark --hook
```
Performance regression tests are in `tests/test_benchmark.py` marked with `@pytest.mark.benchmark`.
### Linting
```bash
@@ -331,15 +322,15 @@ Pre-commit hooks run lint automatically via `hk`.
## Roadmap
### Performance
- **Concurrent feed fetching** — startup currently blocks sequentially on ~25 HTTP requests; `concurrent.futures.ThreadPoolExecutor` would cut load time to the slowest single feed
- **Background refresh** — re-fetch feeds in a daemon thread so a long session stays current without restart
- **Translation pre-fetch** — run translate calls concurrently during the boot sequence rather than on first render
- Concurrent feed fetching with ThreadPoolExecutor
- Background feed refresh daemon
- Translation pre-fetch during boot
### Graphics
- **Matrix rain underlay** — katakana column rain rendered at low opacity beneath the scrolling blocks as a background layer
- **CRT simulation** — subtle dim scanlines every N rows, occasional brightness ripple across the full screen
- **Sixel / iTerm2 inline images** — bypass half-blocks entirely and stream actual bitmap frames for true resolution; would require a capable terminal
- **Parallax secondary column** — a second, dimmer, faster-scrolling stream of ambient text at reduced opacity on one side
- Matrix rain katakana underlay
- CRT scanline simulation
- Sixel/iTerm2 inline images
- Parallax secondary column
### Cyberpunk Vibes
- **Figment intensity wiring** — `config.intensity` currently stored but not yet applied to reveal/dissolve speed or strobe frequency
@@ -357,4 +348,4 @@ Pre-commit hooks run lint automatically via `hk`.
---
*macOS only (script/system font paths for translation are hardcoded). Primary display font is user-selectable via the bundled `fonts/` picker. Python 3.10+.*
*Python 3.10+. Primary display font is user-selectable via bundled `fonts/` picker.*