Gene Johnson
ca1f2718f2
- Add figment mode to intro paragraph - New Figment Mode section: enabling, assets, trigger protocol, deps - Architecture table updated with engine/figment_render.py, figment_trigger.py, and effects_plugins/ directory breakdown - Dev setup: separate uv sync --extras entries (mic vs figment) - Testing section: mentions figment test coverage and Cairo skip - Roadmap: adds figment follow-up items (CLI flag, intensity wiring, ntfy trigger) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
333 lines
16 KiB
Markdown
333 lines
16 KiB
Markdown
# MAINLINE
|
||
|
||
> *Digital consciousness stream. Matrix aesthetic · THX-1138 hue.*
|
||
|
||
A full-screen terminal news ticker that renders live global headlines in large OTF-font block characters with selectable color gradients (Verdant Green, Molten Orange, or Violet Purple). Headlines auto-translate into the native script of their subject region. Ambient mic input warps the glitch rate in real time. A `--poetry` mode replaces the feed with public-domain literary passages. Live messages can be pushed to the display over [ntfy.sh](https://ntfy.sh). **Figment mode** overlays flickery, theme-colored SVG glyphs on the running stream at timed intervals — controllable from any input source via an extensible trigger protocol.
|
||
|
||
---
|
||
|
||
## Using
|
||
|
||
### Run
|
||
|
||
```bash
|
||
python3 mainline.py # news stream
|
||
python3 mainline.py --poetry # literary consciousness mode
|
||
python3 mainline.py -p # same
|
||
python3 mainline.py --firehose # dense rapid-fire headline mode
|
||
python3 mainline.py --refresh # force re-fetch (bypass cache)
|
||
python3 mainline.py --no-font-picker # skip interactive font picker
|
||
python3 mainline.py --font-file path.otf # use a specific font file
|
||
python3 mainline.py --font-dir ~/fonts # scan a different font folder
|
||
python3 mainline.py --font-index 1 # select face index within a collection
|
||
```
|
||
|
||
Or with uv:
|
||
|
||
```bash
|
||
uv run mainline.py
|
||
```
|
||
|
||
First run bootstraps a local `.mainline_venv/` and installs deps (`feedparser`, `Pillow`, `sounddevice`, `numpy`). Subsequent runs start immediately, loading from cache. With uv, run `uv sync` or `uv sync --all-extras` (includes mic support) instead.
|
||
|
||
### Config
|
||
|
||
All constants live in `engine/config.py`:
|
||
|
||
| Constant | Default | What it does |
|
||
|---|---|---|
|
||
| `HEADLINE_LIMIT` | `1000` | Total headlines per session |
|
||
| `FEED_TIMEOUT` | `10` | Per-feed HTTP timeout (seconds) |
|
||
| `MIC_THRESHOLD_DB` | `50` | dB floor above which glitches spike |
|
||
| `FONT_DIR` | `fonts/` | Folder scanned for `.otf`, `.ttf`, `.ttc` files |
|
||
| `FONT_PATH` | first file in `FONT_DIR` | Active display font (overridden by picker or `--font-file`) |
|
||
| `FONT_INDEX` | `0` | Face index within a font collection file |
|
||
| `FONT_PICKER` | `True` | Show interactive font picker at boot (`--no-font-picker` to skip) |
|
||
| `FONT_SZ` | `60` | Font render size (affects block density) |
|
||
| `RENDER_H` | `8` | Terminal rows per headline line |
|
||
| `SSAA` | `4` | Super-sampling factor (render at 4× then downsample) |
|
||
| `SCROLL_DUR` | `5.625` | Seconds per headline |
|
||
| `FRAME_DT` | `0.05` | Frame interval in seconds (20 FPS) |
|
||
| `GRAD_SPEED` | `0.08` | Gradient sweep speed (cycles/sec, ~12s full sweep) |
|
||
| `FIREHOSE_H` | `12` | Firehose zone height (terminal rows) |
|
||
| `NTFY_TOPIC` | klubhaus URL | ntfy.sh JSON stream endpoint |
|
||
| `NTFY_RECONNECT_DELAY` | `5` | Seconds before reconnecting after a dropped SSE stream |
|
||
| `MESSAGE_DISPLAY_SECS` | `30` | How long an ntfy message holds the screen |
|
||
|
||
### Feeds
|
||
|
||
~25 sources across four categories: **Science & Technology**, **Economics & Business**, **World & Politics**, **Culture & Ideas**. Add or swap feeds in `engine/sources.py` → `FEEDS`.
|
||
|
||
**Poetry mode** pulls from Project Gutenberg: Whitman, Dickinson, Thoreau, Emerson. Sources are in `engine/sources.py` → `POETRY_SOURCES`.
|
||
|
||
### Fonts
|
||
|
||
A `fonts/` directory is bundled with demo faces (AgorTechnoDemo, AlphatronDemo, CSBishopDrawn, CubaTechnologyDemo, CyberformDemo, KATA, Microbots, ModernSpaceDemo, Neoform, Pixel Sparta, RaceHugoDemo, Resond, Robocops, Synthetix, Xeonic, and others). On startup, an interactive picker lists all discovered faces with a live half-block preview rendered at your configured size.
|
||
|
||
Navigation: `↑`/`↓` or `j`/`k` to move, `Enter` or `q` to select. The selected face persists for that session.
|
||
|
||
To add your own fonts, drop `.otf`, `.ttf`, or `.ttc` files into `fonts/` (or point `--font-dir` at any other folder). Font collections (`.ttc`, multi-face `.otf`) are enumerated face-by-face.
|
||
|
||
### Color Schemes
|
||
|
||
Mainline supports three color themes for the scrolling gradient: **Verdant Green**, **Molten Orange**, and **Violet Purple**. Each theme uses a precise color-opposite palette for ntfy message queue rendering (magenta, blue, and yellow respectively).
|
||
|
||
On startup, an interactive picker presents all available color schemes:
|
||
```
|
||
[1] Verdant Green (white-hot → deep green)
|
||
[2] Molten Orange (white-hot → deep orange)
|
||
[3] Violet Purple (white-hot → deep purple)
|
||
```
|
||
|
||
Navigation: `↑`/`↓` or `j`/`k` to move, `Enter` or `q` to select. The selection applies only to the current session; you'll pick a fresh theme each run.
|
||
|
||
**Note:** The boot UI (title, status lines, font picker menu) uses a hardcoded green accent color for visual continuity. Only the scrolling headlines and incoming messages render in the selected theme gradient.
|
||
|
||
### ntfy.sh
|
||
|
||
Mainline polls a configurable ntfy.sh topic in the background. When a message arrives, the scroll pauses and the message renders full-screen for `MESSAGE_DISPLAY_SECS` seconds, then the stream resumes.
|
||
|
||
To push a message:
|
||
|
||
```bash
|
||
curl -d "Body text" -H "Title: Alert title" https://ntfy.sh/your_topic
|
||
```
|
||
|
||
Update `NTFY_TOPIC` in `engine/config.py` to point at your own topic.
|
||
|
||
### Figment Mode
|
||
|
||
Figment mode periodically overlays a full-screen SVG glyph on the running ticker — flickering through a reveal → hold (strobe) → dissolve cycle, colored with a randomly selected theme gradient.
|
||
|
||
**Enable it** by setting the plugin's config at runtime (no CLI flag yet):
|
||
|
||
```python
|
||
from engine.effects.registry import get_registry
|
||
plugin = get_registry().get("figment")
|
||
if plugin:
|
||
plugin.config.enabled = True
|
||
plugin.config.params["interval_secs"] = 60 # seconds between appearances
|
||
plugin.config.params["display_secs"] = 4.5 # duration of each appearance
|
||
```
|
||
|
||
**Figment assets** live in `figments/` — drop any `.svg` file there and it will be picked up automatically. The bundled set contains Mayan and Aztec glyphs. Figments are selected randomly, avoiding immediate repeats, and rasterized into half-block terminal art at display time.
|
||
|
||
**Triggering manually** — any object with a `poll() -> FigmentCommand | None` method satisfies the `FigmentTrigger` protocol and can be passed to the plugin:
|
||
|
||
```python
|
||
from engine.figment_trigger import FigmentAction, FigmentCommand
|
||
|
||
class MyTrigger:
|
||
def poll(self):
|
||
if some_condition:
|
||
return FigmentCommand(action=FigmentAction.TRIGGER)
|
||
return None
|
||
```
|
||
|
||
Built-in commands: `TRIGGER`, `SET_INTENSITY`, `SET_INTERVAL`, `SET_COLOR`, `STOP`.
|
||
|
||
**System dependency:** Figment mode requires the Cairo C library (`brew install cairo` on macOS) in addition to the `figment` extras group:
|
||
|
||
```bash
|
||
uv sync --extras figment # adds cairosvg
|
||
```
|
||
|
||
---
|
||
|
||
## Internals
|
||
|
||
### How it works
|
||
|
||
- On launch, the font picker scans `fonts/` and presents a live-rendered TUI for face selection; `--no-font-picker` skips directly to stream
|
||
- Feeds are fetched and filtered on startup (sports and vapid content stripped); results are cached to `.mainline_cache_news.json` / `.mainline_cache_poetry.json` for fast restarts
|
||
- Headlines are rasterized via Pillow with 4× SSAA into half-block characters (`▀▄█ `) at the configured font size
|
||
- The ticker uses a sweeping white-hot → deep green gradient; ntfy messages use a complementary white-hot → magenta/maroon gradient to distinguish them visually
|
||
- Subject-region detection runs a regex pass on each headline; matches trigger a Google Translate call and font swap to the appropriate script (CJK, Arabic, Devanagari, etc.) using macOS system fonts
|
||
- The mic stream runs in a background thread, feeding RMS dB into the glitch probability calculation each frame
|
||
- The viewport scrolls through a virtual canvas of pre-rendered blocks; fade zones at top and bottom dissolve characters probabilistically
|
||
- An ntfy.sh SSE stream runs in a background thread; incoming messages interrupt the scroll and render full-screen until dismissed or expired
|
||
- Figment mode rasterizes SVGs via cairosvg → PIL → greyscale → half-block encode, then overlays them with ANSI cursor-positioning commands between the effect chain and the ntfy message layer
|
||
|
||
### Architecture
|
||
|
||
`mainline.py` is a thin entrypoint (venv bootstrap → `engine.app.main()`). All logic lives in the `engine/` package:
|
||
|
||
```
|
||
engine/
|
||
__init__.py package marker
|
||
app.py main(), font picker TUI, boot sequence, signal handler
|
||
config.py constants, CLI flags, glyph tables
|
||
sources.py FEEDS, POETRY_SOURCES, language/script maps
|
||
terminal.py ANSI codes, tw/th, type_out, boot_ln
|
||
filter.py HTML stripping, content filter
|
||
translate.py Google Translate wrapper + region detection
|
||
render.py OTF → half-block pipeline (SSAA, gradient)
|
||
effects.py noise, glitch_bar, fade, firehose
|
||
fetch.py RSS/Gutenberg fetching + cache load/save
|
||
ntfy.py NtfyPoller — standalone, zero internal deps
|
||
mic.py MicMonitor — standalone, graceful fallback
|
||
scroll.py stream() frame loop + message rendering
|
||
viewport.py terminal dimension tracking (tw/th)
|
||
frame.py scroll step calculation, timing
|
||
layers.py ticker zone, firehose, message + figment overlay rendering
|
||
figment_render.py SVG → cairosvg → PIL → half-block rasterizer with cache
|
||
figment_trigger.py FigmentTrigger protocol, FigmentAction enum, FigmentCommand
|
||
eventbus.py thread-safe event publishing for decoupled communication
|
||
events.py event types and definitions
|
||
controller.py coordinates ntfy/mic monitoring and event publishing
|
||
emitters.py background emitters for ntfy and mic
|
||
types.py type definitions and dataclasses
|
||
themes.py THEME_REGISTRY — gradient color definitions
|
||
|
||
effects_plugins/
|
||
__init__.py plugin discovery (ABC issubclass scan)
|
||
noise.py NoiseEffect — random character noise
|
||
glitch.py GlitchEffect — horizontal glitch bars
|
||
fade.py FadeEffect — edge fade zones
|
||
firehose.py FirehoseEffect — dense bottom ticker strip
|
||
figment.py FigmentEffect — periodic SVG glyph overlay (state machine)
|
||
|
||
figments/ SVG assets for figment mode
|
||
```
|
||
|
||
`ntfy.py` and `mic.py` have zero internal dependencies and can be imported by any other visualizer.
|
||
|
||
---
|
||
|
||
## Extending
|
||
|
||
`ntfy.py` and `mic.py` are fully standalone and designed to be reused by any terminal visualizer. `engine.render` is the importable rendering pipeline for non-terminal targets.
|
||
|
||
### NtfyPoller
|
||
|
||
```python
|
||
from engine.ntfy import NtfyPoller
|
||
|
||
poller = NtfyPoller("https://ntfy.sh/my_topic/json")
|
||
poller.start()
|
||
|
||
# in your render loop:
|
||
msg = poller.get_active_message() # → (title, body, timestamp) or None
|
||
if msg:
|
||
title, body, ts = msg
|
||
render_my_message(title, body) # visualizer-specific
|
||
```
|
||
|
||
Dependencies: `urllib.request`, `json`, `threading`, `time` — stdlib only. The `since=` parameter is managed automatically on reconnect.
|
||
|
||
### MicMonitor
|
||
|
||
```python
|
||
from engine.mic import MicMonitor
|
||
|
||
mic = MicMonitor(threshold_db=50)
|
||
result = mic.start() # None = sounddevice unavailable; False = stream failed; True = ok
|
||
if result:
|
||
excess = mic.excess # dB above threshold, clamped to 0
|
||
db = mic.db # raw RMS dB level
|
||
```
|
||
|
||
Dependencies: `sounddevice`, `numpy` — both optional; degrades gracefully if unavailable.
|
||
|
||
### Render pipeline
|
||
|
||
`engine.render` exposes the OTF → raster pipeline independently of the terminal scroll loop. The planned `serve.py` extension will import it directly to pre-render headlines as 1-bit bitmaps for an ESP32 thin client:
|
||
|
||
```python
|
||
# planned — serve.py does not yet exist
|
||
from engine.render import render_line, big_wrap
|
||
from engine.fetch import fetch_all
|
||
|
||
headlines = fetch_all()
|
||
for h in headlines:
|
||
rows = big_wrap(h.text, font, width=800) # list of half-block rows
|
||
# threshold to 1-bit, pack bytes, serve over HTTP
|
||
```
|
||
|
||
See `Mainline Renderer + ntfy Message Queue for ESP32.md` for the full server + thin client architecture.
|
||
|
||
---
|
||
|
||
## Development
|
||
|
||
### Setup
|
||
|
||
Requires Python 3.10+ and [uv](https://docs.astral.sh/uv/).
|
||
|
||
```bash
|
||
uv sync # minimal (no mic, no figment)
|
||
uv sync --extras mic # with mic support (sounddevice + numpy)
|
||
uv sync --extras figment # with figment mode (cairosvg + system Cairo)
|
||
uv sync --all-extras # all optional features
|
||
uv sync --all-extras --group dev # full dev environment
|
||
```
|
||
|
||
Figment mode also requires the Cairo C library: `brew install cairo` (macOS).
|
||
|
||
### Tasks
|
||
|
||
With [mise](https://mise.jdx.dev/):
|
||
|
||
```bash
|
||
mise run test # run test suite
|
||
mise run test-cov # run with coverage report
|
||
mise run lint # ruff check
|
||
mise run lint-fix # ruff check --fix
|
||
mise run format # ruff format
|
||
mise run run # uv run mainline.py
|
||
mise run run-poetry # uv run mainline.py --poetry
|
||
mise run run-firehose # uv run mainline.py --firehose
|
||
```
|
||
|
||
### Testing
|
||
|
||
Tests live in `tests/` and cover `config`, `filter`, `mic`, `ntfy`, `sources`, `terminal`, and the full figment pipeline (`figment_render`, `figment_trigger`, `figment`, `figment_overlay`). Figment tests are automatically skipped if Cairo is not installed.
|
||
|
||
```bash
|
||
uv run pytest
|
||
uv run pytest --cov=engine --cov-report=term-missing
|
||
```
|
||
|
||
### Linting
|
||
|
||
```bash
|
||
uv run ruff check engine/ mainline.py
|
||
uv run ruff format engine/ mainline.py
|
||
```
|
||
|
||
Pre-commit hooks run lint automatically via `hk`.
|
||
|
||
---
|
||
|
||
## Roadmap
|
||
|
||
### Performance
|
||
- **Concurrent feed fetching** — startup currently blocks sequentially on ~25 HTTP requests; `concurrent.futures.ThreadPoolExecutor` would cut load time to the slowest single feed
|
||
- **Background refresh** — re-fetch feeds in a daemon thread so a long session stays current without restart
|
||
- **Translation pre-fetch** — run translate calls concurrently during the boot sequence rather than on first render
|
||
|
||
### Graphics
|
||
- **Matrix rain underlay** — katakana column rain rendered at low opacity beneath the scrolling blocks as a background layer
|
||
- **CRT simulation** — subtle dim scanlines every N rows, occasional brightness ripple across the full screen
|
||
- **Sixel / iTerm2 inline images** — bypass half-blocks entirely and stream actual bitmap frames for true resolution; would require a capable terminal
|
||
- **Parallax secondary column** — a second, dimmer, faster-scrolling stream of ambient text at reduced opacity on one side
|
||
|
||
### Cyberpunk Vibes
|
||
- **Figment CLI flag** — expose `--figment` and `--figment-interval N` to enable figment mode from the command line without code changes
|
||
- **Figment intensity wiring** — `config.intensity` currently stored but not yet applied to reveal/dissolve speed or strobe frequency
|
||
- **ntfy figment trigger** — built-in `NtfyFigmentTrigger` that listens on a dedicated topic to fire figments on demand
|
||
- **Keyword watch list** — highlight or strobe any headline matching tracked terms (names, topics, tickers)
|
||
- **Breaking interrupt** — full-screen flash + synthesized blip when a high-priority keyword hits
|
||
- **Live data overlay** — secondary ticker strip at screen edge: BTC price, ISS position, geomagnetic index
|
||
- **Theme switcher** — `--amber` (phosphor), `--ice` (electric cyan), `--red` (alert state) palette modes via CLI flag
|
||
- **Persona modes** — `--surveillance`, `--oracle`, `--underground` as feed presets with matching color themes and boot copy
|
||
- **Synthesized audio** — short static bursts tied to glitch events, independent of mic input
|
||
|
||
### Extensibility
|
||
- **serve.py** — HTTP server that imports `engine.render` and `engine.fetch` directly to stream 1-bit bitmaps to an ESP32 display
|
||
- **Rust port** — `ntfy.py` and `render.py` are the natural first targets; clear module boundaries make incremental porting viable
|
||
|
||
---
|
||
|
||
*macOS only (script/system font paths for translation are hardcoded). Primary display font is user-selectable via the bundled `fonts/` picker. Python 3.10+.*
|