Mainline/docs/PIPELINE.md

# Mainline Pipeline

## Architecture Overview

The Mainline pipeline uses a **Stage-based architecture** with **capability-based dependency resolution**. Stages declare capabilities (what they provide) and dependencies (what they need), and the Pipeline resolves dependencies using prefix matching.

```
Source Stage → Render Stage → Effect Stages → Display Stage
      ↓
Camera Stage (provides camera.state capability)
```

### Capability-Based Dependency Resolution

Stages declare capabilities and dependencies:
- **Capabilities**: What the stage provides (e.g., `source`, `render.output`, `display.output`, `camera.state`)
- **Dependencies**: What the stage needs (e.g., `source`, `render.output`, `camera.state`)

The Pipeline resolves dependencies using **prefix matching**:
- `"source"` matches `"source.headlines"`, `"source.poetry"`, etc.
- `"camera.state"` matches the camera state capability provided by `CameraClockStage`
- This allows flexible composition without hardcoding specific stage names

### Minimum Capabilities

The pipeline requires these minimum capabilities to function:
- `"source"` - Data source capability (provides raw items)
- `"render.output"` - Rendered content capability
- `"display.output"` - Display output capability
- `"camera.state"` - Camera state for viewport filtering

These are automatically injected if missing by the `ensure_minimum_capabilities()` method.

### Stage Registry

The `StageRegistry` discovers and registers stages automatically:
- Scans `engine/stages/` for stage implementations
- Registers stages by their declared capabilities
- Enables runtime stage discovery and composition

## Stage-Based Pipeline Flow

```mermaid
flowchart TD
    subgraph Stages["Stage Pipeline"]
        subgraph SourceStage["Source Stage (provides: source.*)"]
            Headlines[HeadlinesSource]
            Poetry[PoetrySource]
            Pipeline[PipelineSource]
        end

        subgraph RenderStage["Render Stage (provides: render.*)"]
            Render[RenderStage]
            Canvas[Canvas]
            Camera[Camera]
        end

        subgraph EffectStages["Effect Stages (provides: effect.*)"]
            Noise[NoiseEffect]
            Fade[FadeEffect]
            Glitch[GlitchEffect]
            Firehose[FirehoseEffect]
            Hud[HudEffect]
        end

        subgraph DisplayStage["Display Stage (provides: display.*)"]
            Terminal[TerminalDisplay]
            Pygame[PygameDisplay]
            WebSocket[WebSocketDisplay]
            Null[NullDisplay]
        end
    end

    subgraph Capabilities["Capability Map"]
        SourceCaps["source.headlines<br/>source.poetry<br/>source.pipeline"]
        RenderCaps["render.output<br/>render.canvas"]
        EffectCaps["effect.noise<br/>effect.fade<br/>effect.glitch"]
        DisplayCaps["display.output<br/>display.terminal"]
    end

    SourceStage --> RenderStage
    RenderStage --> EffectStages
    EffectStages --> DisplayStage

    SourceStage --> SourceCaps
    RenderStage --> RenderCaps
    EffectStages --> EffectCaps
    DisplayStage --> DisplayCaps

    style SourceStage fill:#f9f,stroke:#333
    style RenderStage fill:#bbf,stroke:#333
    style EffectStages fill:#fbf,stroke:#333
    style DisplayStage fill:#bfb,stroke:#333
```

## Stage Adapters

Existing components are wrapped as Stages via adapters:

### Source Stage Adapter
- Wraps `HeadlinesDataSource`, `PoetryDataSource`, etc.
- Provides `source.*` capabilities
- Fetches data and outputs to pipeline buffer

### Render Stage Adapter
- Wraps `StreamController`, `Camera`, `render_ticker_zone`
- Provides `render.output` capability
- Processes content and renders to canvas

### Effect Stage Adapter
- Wraps `EffectChain` and individual effect plugins
- Provides `effect.*` capabilities
- Applies visual effects to rendered content

### Display Stage Adapter
- Wraps `TerminalDisplay`, `PygameDisplay`, etc.
- Provides `display.*` capabilities
- Outputs final buffer to display backend

## Pipeline Mutation API

The Pipeline supports dynamic mutation during runtime:

### Core Methods
- `add_stage(name, stage, initialize=True)` - Add a stage
- `remove_stage(name, cleanup=True)` - Remove a stage and rebuild execution order
- `replace_stage(name, new_stage, preserve_state=True)` - Replace a stage
- `swap_stages(name1, name2)` - Swap two stages
- `move_stage(name, after=None, before=None)` - Move a stage in execution order
- `enable_stage(name)` / `disable_stage(name)` - Enable/disable stages

### Safety Checks
- `can_hot_swap(name)` - Check if a stage can be safely hot-swapped
- `cleanup_stage(name)` - Clean up specific stage without removing it

### WebSocket Commands
The mutation API is accessible via WebSocket for remote control:
```json
{"action": "remove_stage", "stage": "stage_name"}
{"action": "swap_stages", "stage1": "name1", "stage2": "name2"}
{"action": "enable_stage", "stage": "stage_name"}
{"action": "cleanup_stage", "stage": "stage_name"}
```

## Camera Modes

The Camera supports the following modes:

- **FEED**: Single item view (static or rapid cycling)
- **SCROLL**: Smooth vertical scrolling (movie credits style)
- **HORIZONTAL**: Left/right movement
- **OMNI**: Combination of vertical and horizontal
- **FLOATING**: Sinusoidal/bobbing motion
- **BOUNCE**: DVD-style bouncing off edges
- **RADIAL**: Polar coordinate scanning (radar sweep)

Note: Camera state is provided by `CameraClockStage` (capability: `camera.state`) which updates independently of data flow. The `CameraStage` applies viewport transformations (capability: `camera`).

## Animation & Presets

```mermaid
flowchart LR
    subgraph Preset["Preset"]
        PP[PipelineParams]
        AC[AnimationController]
    end
    
    subgraph AnimationController["AnimationController"]
        Clock[Clock]
        Events[Events]
        Triggers[Triggers]
    end
    
    subgraph Triggers["Trigger Types"]
        TIME[TIME]
        FRAME[FRAME]
        CYCLE[CYCLE]
        COND[CONDITION]
        MANUAL[MANUAL]
    end
    
    PP --> AC
    Clock --> AC
    Events --> AC
    Triggers --> Events
```

## Camera Modes State Diagram

```mermaid
stateDiagram-v2
    [*] --> Vertical
    Vertical --> Horizontal: mode change
    Horizontal --> Omni: mode change
    Omni --> Floating: mode change
    Floating --> Trace: mode change
    Trace --> Vertical: mode change

    state Vertical {
        [*] --> ScrollUp
        ScrollUp --> ScrollUp: +y each frame
    }

    state Horizontal {
        [*] --> ScrollLeft
        ScrollLeft --> ScrollLeft: +x each frame
    }

    state Omni {
        [*] --> Diagonal
        Diagonal --> Diagonal: +x, +y each frame
    }

    state Floating {
        [*] --> Bobbing
        Bobbing --> Bobbing: sin(time) for x,y
    }

    state Trace {
        [*] --> FollowPath
        FollowPath --> FollowPath: node by node
    }
```