feat(integration): Complete feature rewrite with pipeline architecture, effects system, and display improvements

Major changes: - Pipeline architecture with capability-based dependency resolution - Effects plugin system with performance monitoring - Display abstraction with multiple backends (terminal, null, websocket) - Camera system for viewport scrolling - Sensor framework for real-time input - Command-and-control system via ntfy - WebSocket display backend for browser clients - Comprehensive test suite and documentation Issue #48: ADR for preset scripting language included This commit consolidates 110 individual commits into a single feature integration that can be reviewed and tested before further refinement.
2026-03-20 04:41:23 -07:00
parent 42aa6f16cc
commit ef98add0c5
179 changed files with 27649 additions and 6552 deletions
--- a/.opencode/skills/mainline-architecture/SKILL.md
+++ b/.opencode/skills/mainline-architecture/SKILL.md
@@ -0,0 +1,97 @@
+---
+name: mainline-architecture
+description: Pipeline stages, capability resolution, and core architecture patterns
+compatibility: opencode
+metadata:
+  audience: developers
+  source_type: codebase
+---
+
+## What This Skill Covers
+
+This skill covers Mainline's pipeline architecture - the Stage-based system for dependency resolution, data flow, and component composition.
+
+## Key Concepts
+
+### Stage Class (engine/pipeline/core.py)
+
+The `Stage` ABC is the foundation. All pipeline components inherit from it:
+
+```python
+class Stage(ABC):
+    name: str
+    category: str  # "source", "effect", "overlay", "display", "camera"
+    optional: bool = False
+    
+    @property
+    def capabilities(self) -> set[str]:
+        """What this stage provides (e.g., 'source.headlines')"""
+        return set()
+    
+    @property
+    def dependencies(self) -> set[str]:
+        """What this stage needs (e.g., {'source'})"""
+        return set()
+```
+
+### Capability-Based Dependencies
+
+The Pipeline resolves dependencies using **prefix matching**:
+- `"source"` matches `"source.headlines"`, `"source.poetry"`, etc.
+- `"camera.state"` matches the camera state capability
+- This allows flexible composition without hardcoding specific stage names
+
+### Minimum Capabilities
+
+The pipeline requires these minimum capabilities to function:
+- `"source"` - Data source capability
+- `"render.output"` - Rendered content capability
+- `"display.output"` - Display output capability
+- `"camera.state"` - Camera state for viewport filtering
+
+These are automatically injected if missing (auto-injection).
+
+### DataType Enum
+
+PureData-style data types for inlet/outlet validation:
+- `SOURCE_ITEMS`: List[SourceItem] - raw items from sources
+- `ITEM_TUPLES`: List[tuple] - (title, source, timestamp) tuples  
+- `TEXT_BUFFER`: List[str] - rendered ANSI buffer
+- `RAW_TEXT`: str - raw text strings
+- `PIL_IMAGE`: PIL Image object
+
+### Pipeline Execution
+
+The Pipeline (engine/pipeline/controller.py):
+1. Collects all stages from StageRegistry
+2. Resolves dependencies using prefix matching
+3. Executes stages in dependency order
+4. Handles errors for non-optional stages
+
+### Canvas & Camera
+
+- **Canvas** (`engine/canvas.py`): 2D rendering surface with dirty region tracking
+- **Camera** (`engine/camera.py`): Viewport controller for scrolling content
+
+Canvas tracks dirty regions automatically when content is written via `put_region`, `put_text`, `fill`, enabling partial buffer updates.
+
+## Adding New Stages
+
+1. Create a class inheriting from `Stage`
+2. Define `capabilities` and `dependencies` properties
+3. Implement required abstract methods
+4. Register in StageRegistry or use as adapter
+
+## Common Patterns
+
+- Use adapters (engine/pipeline/adapters.py) to wrap existing components as stages
+- Set `optional=True` for stages that can fail gracefully
+- Use `stage_type` and `render_order` for execution ordering
+- Clock stages update state independently of data flow
+
+## Sources
+
+- engine/pipeline/core.py - Stage base class
+- engine/pipeline/controller.py - Pipeline implementation
+- engine/pipeline/adapters/ - Stage adapters
+- docs/PIPELINE.md - Pipeline documentation
--- a/.opencode/skills/mainline-display/SKILL.md
+++ b/.opencode/skills/mainline-display/SKILL.md
@@ -0,0 +1,163 @@
+---
+name: mainline-display
+description: Display backend implementation and the Display protocol
+compatibility: opencode
+metadata:
+  audience: developers
+  source_type: codebase
+---
+
+## What This Skill Covers
+
+This skill covers Mainline's display backend system - how to implement new display backends and how the Display protocol works.
+
+## Key Concepts
+
+### Display Protocol
+
+All backends implement a common Display protocol (in `engine/display/__init__.py`):
+
+```python
+class Display(Protocol):
+    width: int
+    height: int
+    
+    def init(self, width: int, height: int, reuse: bool = False) -> None:
+        """Initialize the display"""
+        ...
+    
+    def show(self, buf: list[str], border: bool = False) -> None:
+        """Display the buffer"""
+        ...
+    
+    def clear(self) -> None:
+        """Clear the display"""
+        ...
+    
+    def cleanup(self) -> None:
+        """Clean up resources"""
+        ...
+    
+    def get_dimensions(self) -> tuple[int, int]:
+        """Return (width, height)"""
+        ...
+```
+
+### DisplayRegistry
+
+Discovers and manages backends:
+
+```python
+from engine.display import DisplayRegistry
+display = DisplayRegistry.create("terminal")  # or "websocket", "null", "multi"
+```
+
+### Available Backends
+
+| Backend | File | Description |
+|---------|------|-------------|
+| terminal | backends/terminal.py | ANSI terminal output |
+| websocket | backends/websocket.py | Web browser via WebSocket |
+| null | backends/null.py | Headless for testing |
+| multi | backends/multi.py | Forwards to multiple displays |
+| moderngl | backends/moderngl.py | GPU-accelerated OpenGL rendering (optional) |
+
+### WebSocket Backend
+
+- WebSocket server: port 8765
+- HTTP server: port 8766 (serves client/index.html)
+- Client has ANSI color parsing and fullscreen support
+
+### Multi Backend
+
+Forwards to multiple displays simultaneously - useful for `terminal + websocket`.
+
+## Adding a New Backend
+
+1. Create `engine/display/backends/my_backend.py`
+2. Implement the Display protocol methods
+3. Register in `engine/display/__init__.py`'s `DisplayRegistry`
+
+Required methods:
+- `init(width: int, height: int, reuse: bool = False)` - Initialize display
+- `show(buf: list[str], border: bool = False)` - Display buffer
+- `clear()` - Clear screen
+- `cleanup()` - Clean up resources
+- `get_dimensions() -> tuple[int, int]` - Get terminal dimensions
+
+Optional methods:
+- `title(text: str)` - Set window title
+- `cursor(show: bool)` - Control cursor
+
+## Usage
+
+```bash
+python mainline.py --display terminal  # default
+python mainline.py --display websocket
+python mainline.py --display moderngl  # GPU-accelerated (requires moderngl)
+```
+
+## Common Bugs and Patterns
+
+### BorderMode.OFF Enum Bug
+
+**Problem**: `BorderMode.OFF` has enum value `1` (not `0`), and Python enums are always truthy.
+
+**Incorrect Code**:
+```python
+if border:
+    buffer = render_border(buffer, width, height, fps, frame_time)
+```
+
+**Correct Code**:
+```python
+from engine.display import BorderMode
+if border and border != BorderMode.OFF:
+    buffer = render_border(buffer, width, height, fps, frame_time)
+```
+
+**Why**: Checking `if border:` evaluates to `True` even when `border == BorderMode.OFF` because enum members are always truthy in Python.
+
+### Context Type Mismatch
+
+**Problem**: `PipelineContext` and `EffectContext` have different APIs for storing data.
+
+- `PipelineContext`: Uses `set()`/`get()` for services
+- `EffectContext`: Uses `set_state()`/`get_state()` for state
+
+**Pattern for Passing Data**:
+```python
+# In pipeline setup (uses PipelineContext)
+ctx.set("pipeline_order", pipeline.execution_order)
+
+# In EffectPluginStage (must copy to EffectContext)
+effect_ctx.set_state("pipeline_order", ctx.get("pipeline_order"))
+```
+
+### Terminal Display ANSI Patterns
+
+**Screen Clearing**:
+```python
+output = "\033[H\033[J" + "".join(buffer)
+```
+
+**Cursor Positioning** (used by HUD effect):
+- `\033[row;colH` - Move cursor to row, column
+- Example: `\033[1;1H` - Move to row 1, column 1
+
+**Key Insight**: Terminal display joins buffer lines WITHOUT newlines, relying on ANSI cursor positioning codes to move the cursor to the correct location for each line.
+
+### EffectPluginStage Context Copying
+
+**Problem**: When effects need access to pipeline services (like `pipeline_order`), they must be copied from `PipelineContext` to `EffectContext`.
+
+**Pattern**:
+```python
+# In EffectPluginStage.process()
+# Copy pipeline_order from PipelineContext services to EffectContext state
+pipeline_order = ctx.get("pipeline_order")
+if pipeline_order:
+    effect_ctx.set_state("pipeline_order", pipeline_order)
+```
+
+This ensures effects can access `ctx.get_state("pipeline_order")` in their process method.
--- a/.opencode/skills/mainline-effects/SKILL.md
+++ b/.opencode/skills/mainline-effects/SKILL.md
@@ -0,0 +1,113 @@
+---
+name: mainline-effects
+description: How to add new effect plugins to Mainline's effect system
+compatibility: opencode
+metadata:
+  audience: developers
+  source_type: codebase
+---
+
+## What This Skill Covers
+
+This skill covers Mainline's effect plugin system - how to create, configure, and integrate visual effects into the pipeline.
+
+## Key Concepts
+
+### EffectPlugin ABC (engine/effects/types.py)
+
+All effects must inherit from `EffectPlugin` and implement:
+
+```python
+class EffectPlugin(ABC):
+    name: str
+    config: EffectConfig
+    param_bindings: dict[str, dict[str, str | float]] = {}
+    supports_partial_updates: bool = False
+    
+    @abstractmethod
+    def process(self, buf: list[str], ctx: EffectContext) -> list[str]:
+        """Process buffer with effect applied"""
+        ...
+    
+    @abstractmethod
+    def configure(self, config: EffectConfig) -> None:
+        """Configure the effect"""
+        ...
+```
+
+### EffectContext
+
+Passed to every effect's process method:
+
+```python
+@dataclass
+class EffectContext:
+    terminal_width: int
+    terminal_height: int
+    scroll_cam: int
+    ticker_height: int
+    camera_x: int = 0
+    mic_excess: float = 0.0
+    grad_offset: float = 0.0
+    frame_number: int = 0
+    has_message: bool = False
+    items: list = field(default_factory=list)
+    _state: dict[str, Any] = field(default_factory=dict)
+```
+
+Access sensor values via `ctx.get_sensor_value("sensor_name")`.
+
+### EffectConfig
+
+Configuration dataclass:
+
+```python
+@dataclass
+class EffectConfig:
+    enabled: bool = True
+    intensity: float = 1.0
+    params: dict[str, Any] = field(default_factory=dict)
+```
+
+### Partial Updates
+
+For performance optimization, set `supports_partial_updates = True` and implement `process_partial`:
+
+```python
+class MyEffect(EffectPlugin):
+    supports_partial_updates = True
+    
+    def process_partial(self, buf, ctx, partial: PartialUpdate) -> list[str]:
+        # Only process changed regions
+        ...
+```
+
+## Adding a New Effect
+
+1. Create file in `effects_plugins/my_effect.py`
+2. Inherit from `EffectPlugin`
+3. Implement `process()` and `configure()`
+4. Add to `effects_plugins/__init__.py` (runtime discovery via issubclass checks)
+
+## Param Bindings
+
+Declarative sensor-to-param mappings:
+
+```python
+param_bindings = {
+    "intensity": {"sensor": "mic", "transform": "linear"},
+    "rate": {"sensor": "oscillator", "transform": "exponential"},
+}
+```
+
+Transforms: `linear`, `exponential`, `threshold`
+
+## Effect Chain
+
+Effects are chained via `engine/effects/chain.py` - processes each effect in order, passing output to next.
+
+## Existing Effects
+
+See `effects_plugins/`:
+- noise.py, fade.py, glitch.py, firehose.py
+- border.py, crop.py, tint.py, hud.py
--- a/.opencode/skills/mainline-presets/SKILL.md
+++ b/.opencode/skills/mainline-presets/SKILL.md
@@ -0,0 +1,103 @@
+---
+name: mainline-presets
+description: Creating pipeline presets in TOML format for Mainline
+compatibility: opencode
+metadata:
+  audience: developers
+  source_type: codebase
+---
+
+## What This Skill Covers
+
+This skill covers how to create pipeline presets in TOML format for Mainline's rendering pipeline.
+
+## Key Concepts
+
+### Preset Loading Order
+
+Presets are loaded from multiple locations (later overrides earlier):
+1. Built-in: `engine/presets.toml`
+2. User config: `~/.config/mainline/presets.toml`
+3. Local override: `./presets.toml`
+
+### PipelinePreset Dataclass
+
+```python
+@dataclass
+class PipelinePreset:
+    name: str
+    description: str = ""
+    source: str = "headlines"      # Data source
+    display: str = "terminal"      # Display backend
+    camera: str = "scroll"          # Camera mode
+    effects: list[str] = field(default_factory=list)
+    border: bool = False
+```
+
+### TOML Format
+
+```toml
+[presets.my-preset]
+description = "My custom pipeline"
+source = "headlines"
+display = "terminal"
+camera = "scroll"
+effects = ["noise", "fade"]
+border = true
+```
+
+## Creating a Preset
+
+### Option 1: User Config
+
+Create/edit `~/.config/mainline/presets.toml`:
+
+```toml
+[presets.my-cool-preset]
+description = "Noise and glitch effects"
+source = "headlines"
+display = "terminal"
+effects = ["noise", "glitch"]
+```
+
+### Option 2: Local Override
+
+Create `./presets.toml` in project root:
+
+```toml
+[presets.dev-inspect]
+description = "Pipeline introspection for development"
+source = "headlines"
+display = "terminal"
+effects = ["hud"]
+```
+
+### Option 3: Built-in
+
+Edit `engine/presets.toml` (requires PR to repository).
+
+## Available Sources
+
+- `headlines` - RSS news feeds
+- `poetry` - Literature mode
+- `pipeline-inspect` - Live DAG visualization
+
+## Available Displays
+
+- `terminal` - ANSI terminal
+- `websocket` - Web browser
+- `null` - Headless
+- `moderngl` - GPU-accelerated (optional)
+
+## Available Effects
+
+See `effects_plugins/`:
+- noise, fade, glitch, firehose
+- border, crop, tint, hud
+
+## Validation Functions
+
+Use these from `engine/pipeline/presets.py`:
+- `validate_preset()` - Validate preset structure
+- `validate_signal_path()` - Detect circular dependencies
+- `generate_preset_toml()` - Generate skeleton preset
--- a/.opencode/skills/mainline-sensors/SKILL.md
+++ b/.opencode/skills/mainline-sensors/SKILL.md
@@ -0,0 +1,136 @@
+---
+name: mainline-sensors
+description: Sensor framework for real-time input in Mainline
+compatibility: opencode
+metadata:
+  audience: developers
+  source_type: codebase
+---
+
+## What This Skill Covers
+
+This skill covers Mainline's sensor framework - how to use, create, and integrate sensors for real-time input.
+
+## Key Concepts
+
+### Sensor Base Class (engine/sensors/__init__.py)
+
+```python
+class Sensor(ABC):
+    name: str
+    unit: str = ""
+    
+    @property
+    def available(self) -> bool:
+        """Whether sensor is currently available"""
+        return True
+    
+    @abstractmethod
+    def read(self) -> SensorValue | None:
+        """Read current sensor value"""
+        ...
+    
+    def start(self) -> None:
+        """Initialize sensor (optional)"""
+        pass
+    
+    def stop(self) -> None:
+        """Clean up sensor (optional)"""
+        pass
+```
+
+### SensorValue Dataclass
+
+```python
+@dataclass
+class SensorValue:
+    sensor_name: str
+    value: float
+    timestamp: float
+    unit: str = ""
+```
+
+### SensorRegistry
+
+Discovers and manages sensors globally:
+
+```python
+from engine.sensors import SensorRegistry
+registry = SensorRegistry()
+sensor = registry.get("mic")
+```
+
+### SensorStage
+
+Pipeline adapter that provides sensor values to effects:
+
+```python
+from engine.pipeline.adapters import SensorStage
+stage = SensorStage(sensor_name="mic")
+```
+
+## Built-in Sensors
+
+| Sensor | File | Description |
+|--------|------|-------------|
+| MicSensor | sensors/mic.py | Microphone input (RMS dB) |
+| OscillatorSensor | sensors/oscillator.py | Test sine wave generator |
+| PipelineMetricsSensor | sensors/pipeline_metrics.py | FPS, frame time, etc. |
+
+## Param Bindings
+
+Effects declare sensor-to-param mappings:
+
+```python
+class GlitchEffect(EffectPlugin):
+    param_bindings = {
+        "intensity": {"sensor": "mic", "transform": "linear"},
+    }
+```
+
+### Transform Functions
+
+- `linear` - Direct mapping to param range
+- `exponential` - Exponential scaling
+- `threshold` - Binary on/off
+
+## Adding a New Sensor
+
+1. Create `engine/sensors/my_sensor.py`
+2. Inherit from `Sensor` ABC
+3. Implement required methods
+4. Register in `SensorRegistry`
+
+Example:
+```python
+class MySensor(Sensor):
+    name = "my-sensor"
+    unit = "units"
+    
+    def read(self) -> SensorValue | None:
+        return SensorValue(
+            sensor_name=self.name,
+            value=self._read_hardware(),
+            timestamp=time.time(),
+            unit=self.unit
+        )
+```
+
+## Using Sensors in Effects
+
+Access sensor values via EffectContext:
+
+```python
+def process(self, buf, ctx):
+    mic_level = ctx.get_sensor_value("mic")
+    if mic_level and mic_level > 0.5:
+        # Apply intense effect
+        ...
+```
+
+Or via param_bindings (automatic):
+
+```python
+# If intensity is bound to "mic", it's automatically 
+# available in self.config.intensity
+```
--- a/.opencode/skills/mainline-sources/SKILL.md
+++ b/.opencode/skills/mainline-sources/SKILL.md
@@ -0,0 +1,87 @@
+---
+name: mainline-sources
+description: Adding new RSS feeds and data sources to Mainline
+compatibility: opencode
+metadata:
+  audience: developers
+  source_type: codebase
+---
+
+## What This Skill Covers
+
+This skill covers how to add new data sources (RSS feeds, poetry) to Mainline.
+
+## Key Concepts
+
+### Feeds Dictionary (engine/sources.py)
+
+All feeds are defined in a simple dictionary:
+
+```python
+FEEDS = {
+    "Feed Name": "https://example.com/feed.xml",
+    # Category comments help organize:
+    # Science & Technology
+    # Economics & Business
+    # World & Politics
+    # Culture & Ideas
+}
+```
+
+### Poetry Sources
+
+Project Gutenberg URLs for public domain literature:
+
+```python
+POETRY_SOURCES = {
+    "Author Name": "https://www.gutenberg.org/cache/epub/1234/pg1234.txt",
+}
+```
+
+### Language & Script Mapping
+
+The sources.py also contains language/script detection mappings used for auto-translation and font selection.
+
+## Adding a New RSS Feed
+
+1. Edit `engine/sources.py`
+2. Add entry to `FEEDS` dict under appropriate category:
+   ```python
+   "My Feed": "https://example.com/feed.xml",
+   ```
+3. The feed will be automatically discovered on next run
+
+### Feed Requirements
+
+- Must be valid RSS or Atom XML
+- Should have `<title>` elements for items
+- Must be HTTP/HTTPS accessible
+
+## Adding Poetry Sources
+
+1. Edit `engine/sources.py`
+2. Add to `POETRY_SOURCES` dict:
+   ```python
+   "Author": "https://www.gutenberg.org/cache/epub/XXXX/pgXXXX.txt",
+   ```
+
+### Poetry Requirements
+
+- Plain text (UTF-8)
+- Project Gutenberg format preferred
+- No DRM-protected sources
+
+## Data Flow
+
+Feeds are fetched via `engine/fetch.py`:
+- `fetch_feed(url)` - Fetches and parses RSS/Atom
+- Results cached for fast restarts
+- Filtered via `engine/filter.py` for content cleaning
+
+## Categories
+
+Organize new feeds by category using comments:
+- Science & Technology
+- Economics & Business  
+- World & Politics
+- Culture & Ideas