sideline/README.md

# MAINLINE

> *Digital consciousness stream. Matrix aesthetic · THX-1138 hue.*

A full-screen terminal news ticker that renders live global headlines in large OTF-font block characters with a white-hot → deep green gradient. Headlines auto-translate into the native script of their subject region. Ambient mic input warps the glitch rate in real time. A `--poetry` mode replaces the feed with public-domain literary passages. Live messages can be pushed to the display over [ntfy.sh](https://ntfy.sh).

---

## Using

### Run

```bash
python3 mainline.py                      # news stream
python3 mainline.py --poetry             # literary consciousness mode
python3 mainline.py -p                   # same
python3 mainline.py --firehose           # dense rapid-fire headline mode
python3 mainline.py --display websocket  # web browser display only
python3 mainline.py --display both       # terminal + web browser
python3 mainline.py --no-font-picker     # skip interactive font picker
python3 mainline.py --font-file path.otf # use a specific font file
python3 mainline.py --font-dir ~/fonts   # scan a different font folder
python3 mainline.py --font-index 1       # select face index within a collection
```

Or with uv:

```bash
uv run mainline.py
```

First run bootstraps dependencies. Use `uv sync --all-extras` for mic support.

### Command & Control (C&C)

Control mainline remotely using `cmdline.py`:

```bash
uv run cmdline.py                    # Interactive TUI
uv run cmdline.py /effects list      # List all effects
uv run cmdline.py /effects stats     # Show performance stats
uv run cmdline.py -w /effects stats  # Watch mode (auto-refresh)
```

Commands are sent via ntfy.sh topics - useful for controlling a daemonized mainline instance.

### Config

All constants live in `engine/config.py`:

| Constant | Default | What it does |
|---|---|---|
| `HEADLINE_LIMIT` | `1000` | Total headlines per session |
| `FEED_TIMEOUT` | `10` | Per-feed HTTP timeout (seconds) |
| `MIC_THRESHOLD_DB` | `50` | dB floor above which glitches spike |
| `NTFY_TOPIC` | klubhaus URL | ntfy.sh JSON stream for messages |
| `NTFY_CC_CMD_TOPIC` | klubhaus URL | ntfy.sh topic for C&C commands |
| `NTFY_CC_RESP_TOPIC` | klubhaus URL | ntfy.sh topic for C&C responses |
| `NTFY_RECONNECT_DELAY` | `5` | Seconds before reconnecting after dropped SSE |
| `MESSAGE_DISPLAY_SECS` | `30` | How long an ntfy message holds the screen |
| `FONT_DIR` | `fonts/` | Folder scanned for `.otf`, `.ttf`, `.ttc` files |
| `FONT_PATH` | first file in `FONT_DIR` | Active display font |
| `FONT_PICKER` | `True` | Show interactive font picker at boot |
| `FONT_SZ` | `60` | Font render size (affects block density) |
| `RENDER_H` | `8` | Terminal rows per headline line |
| `SSAA` | `4` | Super-sampling factor |
| `SCROLL_DUR` | `5.625` | Seconds per headline |
| `FRAME_DT` | `0.05` | Frame interval in seconds (20 FPS) |
| `FIREHOSE_H` | `12` | Firehose zone height (terminal rows) |
| `GRAD_SPEED` | `0.08` | Gradient sweep speed |

### Display Modes

Mainline supports multiple display backends:

- **Terminal** (`--display terminal`): ANSI terminal output (default)
- **WebSocket** (`--display websocket`): Stream to web browser clients
- **Sixel** (`--display sixel`): Sixel graphics in supported terminals (iTerm2, mintty)
- **Both** (`--display both`): Terminal + WebSocket simultaneously

WebSocket mode serves a web client at http://localhost:8766 with ANSI color support and fullscreen mode.

### Feeds

~25 sources across four categories: **Science & Technology**, **Economics & Business**, **World & Politics**, **Culture & Ideas**. Add or swap feeds in `engine/sources.py` → `FEEDS`.

**Poetry mode** pulls from Project Gutenberg: Whitman, Dickinson, Thoreau, Emerson. Sources are in `engine/sources.py` → `POETRY_SOURCES`.

### Fonts

A `fonts/` directory is bundled with demo faces. On startup, an interactive picker lists all discovered faces with a live half-block preview.

Navigation: `↑`/`↓` or `j`/`k` to move, `Enter` or `q` to select.

To add your own fonts, drop `.otf`, `.ttf`, or `.ttc` files into `fonts/`.

### ntfy.sh

Mainline polls a configurable ntfy.sh topic in the background. When a message arrives, the scroll pauses and the message renders full-screen.

To push a message:

```bash
curl -d "Body text" -H "Title: Alert title" https://ntfy.sh/your_topic
```

---

## Internals

### How it works

- On launch, the font picker scans `fonts/` and presents a live-rendered TUI for face selection
- Feeds are fetched and filtered on startup; results are cached for fast restarts
- Headlines are rasterized via Pillow with 4× SSAA into half-block characters
- The ticker uses a sweeping white-hot → deep green gradient
- Subject-region detection triggers Google Translate and font swap for non-Latin scripts
- The mic stream runs in a background thread, feeding RMS dB into glitch probability
- The viewport scrolls through pre-rendered blocks with fade zones
- An ntfy.sh SSE stream runs in a background thread for messages and C&C commands

### Architecture

```
engine/
  __init__.py           package marker
  app.py                main(), font picker TUI, boot sequence, C&C poller
  config.py             constants, CLI flags, glyph tables
  sources.py            FEEDS, POETRY_SOURCES, language/script maps
  terminal.py           ANSI codes, tw/th, type_out, boot_ln
  filter.py             HTML stripping, content filter
  translate.py          Google Translate wrapper + region detection
  render.py             OTF → half-block pipeline (SSAA, gradient)
  effects/              plugin architecture for visual effects
    types.py            EffectPlugin ABC, EffectConfig, EffectContext
    registry.py         effect registration and lookup
    chain.py            effect pipeline chaining
    controller.py       handles /effects commands
    performance.py      performance monitoring
    legacy.py           legacy functional effects
  effects_plugins/      effect plugin implementations
    noise.py            noise effect
    fade.py             fade effect
    glitch.py           glitch effect
    firehose.py         firehose effect
  fetch.py              RSS/Gutenberg fetching + cache
  ntfy.py               NtfyPoller — standalone, zero internal deps
  mic.py                MicMonitor — standalone, graceful fallback
  scroll.py             stream() frame loop + message rendering
  viewport.py           terminal dimension tracking
  frame.py              scroll step calculation, timing
  layers.py             ticker zone, firehose, message overlay
  eventbus.py           thread-safe event publishing
  events.py             event types and definitions
  controller.py         coordinates ntfy/mic monitoring
  emitters.py           background emitters
  types.py              type definitions
  display/              Display backend system
    __init__.py         DisplayRegistry, get_monitor
    backends/
      terminal.py       ANSI terminal display
      websocket.py      WebSocket server for browser clients
      sixel.py         Sixel graphics (pure Python)
      null.py          headless display for testing
      multi.py         forwards to multiple displays
  benchmark.py          performance benchmarking tool
```

---

## Development

### Setup

Requires Python 3.10+ and [uv](https://docs.astral.sh/uv/).

```bash
uv sync                              # minimal (no mic)
uv sync --all-extras                 # with mic support
uv sync --all-extras --group dev     # full dev environment
```

### Tasks

With [mise](https://mise.jdx.dev/):

```bash
mise run test            # run test suite
mise run test-cov       # run with coverage report

mise run lint           # ruff check
mise run lint-fix       # ruff check --fix
mise run format         # ruff format

mise run run            # terminal display
mise run run-websocket  # web display only
mise run run-sixel     # sixel graphics
mise run run-both       # terminal + web
mise run run-client     # both + open browser

mise run cmd            # C&C command interface
mise run cmd-stats      # watch effects stats

mise run benchmark      # run performance benchmarks
mise run benchmark-json # save as JSON

mise run topics-init    # initialize ntfy topics
```

### Testing

```bash
uv run pytest
uv run pytest --cov=engine --cov-report=term-missing

# Run with mise
mise run test
mise run test-cov

# Run performance benchmarks
mise run benchmark
mise run benchmark-json

# Run benchmark hook mode (for CI)
uv run python -m engine.benchmark --hook
```

Performance regression tests are in `tests/test_benchmark.py` marked with `@pytest.mark.benchmark`.

### Linting

```bash
uv run ruff check engine/ mainline.py
uv run ruff format engine/ mainline.py
```

Pre-commit hooks run lint automatically via `hk`.

---

## Roadmap

### Performance
- Concurrent feed fetching with ThreadPoolExecutor
- Background feed refresh daemon
- Translation pre-fetch during boot

### Graphics
- Matrix rain katakana underlay
- CRT scanline simulation
- Sixel/iTerm2 inline images
- Parallax secondary column

### Cyberpunk Vibes
- Keyword watch list with strobe effects
- Breaking interrupt with synthesized audio
- Live data overlay (BTC, ISS position)
- Theme switcher (amber, ice, red)
- Persona modes (surveillance, oracle, underground)

---

*Python 3.10+. Primary display font is user-selectable via bundled `fonts/` picker.*