Mainline/docs/PIPELINE.md

# Mainline Pipeline

## Architecture Overview

```
Sources (static/dynamic) → Fetch → Prepare → Scroll → Effects → Render → Display
                                     ↓
                    NtfyPoller ← MicMonitor (async)
```

### Data Source Abstraction (sources_v2.py)

- **Static sources**: Data fetched once and cached (HeadlinesDataSource, PoetryDataSource)
- **Dynamic sources**: Idempotent fetch for runtime updates (PipelineDataSource)
- **SourceRegistry**: Discovery and management of data sources

### Camera Modes

- **Vertical**: Scroll up (default)
- **Horizontal**: Scroll left
- **Omni**: Diagonal scroll
- **Floating**: Sinusoidal bobbing
- **Trace**: Follow network path node-by-node (for pipeline viz)

## Content to Display Rendering Pipeline

```mermaid
flowchart TD
    subgraph Sources["Data Sources (v2)"]
        Headlines[HeadlinesDataSource]
        Poetry[PoetryDataSource]
        Pipeline[PipelineDataSource]
        Registry[SourceRegistry]
    end

    subgraph SourcesLegacy["Data Sources (legacy)"]
        RSS[("RSS Feeds")]
        PoetryFeed[("Poetry Feed")]
        Ntfy[("Ntfy Messages")]
        Mic[("Microphone")]
    end

    subgraph Fetch["Fetch Layer"]
        FC[fetch_all]
        FP[fetch_poetry]
        Cache[(Cache)]
    end

    subgraph Prepare["Prepare Layer"]
        MB[make_block]
        Strip[strip_tags]
        Trans[translate]
    end

    subgraph Scroll["Scroll Engine"]
        SC[StreamController]
        CAM[Camera]
        RTZ[render_ticker_zone]
        Msg[render_message_overlay]
        Grad[lr_gradient]
        VT[vis_trunc / vis_offset]
    end

    subgraph Effects["Effect Pipeline"]
        subgraph EffectsPlugins["Effect Plugins"]
            Noise[NoiseEffect]
            Fade[FadeEffect]
            Glitch[GlitchEffect]
            Firehose[FirehoseEffect]
            Hud[HudEffect]
        end
        EC[EffectChain]
        ER[EffectRegistry]
    end

    subgraph Render["Render Layer"]
        BW[big_wrap]
        RL[render_line]
    end

    subgraph Display["Display Backends"]
        TD[TerminalDisplay]
        PD[PygameDisplay]
        SD[SixelDisplay]
        KD[KittyDisplay]
        WSD[WebSocketDisplay]
        ND[NullDisplay]
    end

    subgraph Async["Async Sources"]
        NTFY[NtfyPoller]
        MIC[MicMonitor]
    end

    subgraph Animation["Animation System"]
        AC[AnimationController]
        PR[Preset]
    end

    Sources --> Fetch
    RSS --> FC
    PoetryFeed --> FP
    FC --> Cache
    FP --> Cache
    Cache --> MB
    Strip --> MB
    Trans --> MB
    MB --> SC
    NTFY --> SC
    SC --> RTZ
    CAM --> RTZ
    Grad --> RTZ
    VT --> RTZ
    RTZ --> EC
    EC --> ER
    ER --> EffectsPlugins
    EffectsPlugins --> BW
    BW --> RL
    RL --> Display
    Ntfy --> RL
    Mic --> RL
    MIC --> RL

    style Sources fill:#f9f,stroke:#333
    style Fetch fill:#bbf,stroke:#333
    style Prepare fill:#bff,stroke:#333
    style Scroll fill:#bfb,stroke:#333
    style Effects fill:#fbf,stroke:#333
    style Render fill:#ffb,stroke:#333
    style Display fill:#bbf,stroke:#333
    style Async fill:#fbb,stroke:#333
    style Animation fill:#bfb,stroke:#333
```

## Animation & Presets

```mermaid
flowchart LR
    subgraph Preset["Preset"]
        PP[PipelineParams]
        AC[AnimationController]
    end
    
    subgraph AnimationController["AnimationController"]
        Clock[Clock]
        Events[Events]
        Triggers[Triggers]
    end
    
    subgraph Triggers["Trigger Types"]
        TIME[TIME]
        FRAME[FRAME]
        CYCLE[CYCLE]
        COND[CONDITION]
        MANUAL[MANUAL]
    end
    
    PP --> AC
    Clock --> AC
    Events --> AC
    Triggers --> Events
```

## Camera Modes

```mermaid
stateDiagram-v2
    [*] --> Vertical
    Vertical --> Horizontal: mode change
    Horizontal --> Omni: mode change
    Omni --> Floating: mode change
    Floating --> Trace: mode change
    Trace --> Vertical: mode change

    state Vertical {
        [*] --> ScrollUp
        ScrollUp --> ScrollUp: +y each frame
    }

    state Horizontal {
        [*] --> ScrollLeft
        ScrollLeft --> ScrollLeft: +x each frame
    }

    state Omni {
        [*] --> Diagonal
        Diagonal --> Diagonal: +x, +y each frame
    }

    state Floating {
        [*] --> Bobbing
        Bobbing --> Bobbing: sin(time) for x,y
    }

    state Trace {
        [*] --> FollowPath
        FollowPath --> FollowPath: node by node
    }
```