sideline/docs/graph-dsl.md

Graph-Based Pipeline DSL

This document describes the new graph-based DSL for defining pipelines in Mainline.

Overview

The graph DSL represents pipelines as nodes and connections, replacing the verbose XYZStage naming convention with a more intuitive graph abstraction.

TOML Syntax

Basic Pipeline

[nodes.source]
type = "source"
source = "headlines"

[nodes.camera]
type = "camera"
mode = "scroll"

[nodes.display]
type = "display"
backend = "terminal"

[connections]
list = ["source -> camera -> display"]

With Effects

[nodes.source]
type = "source"
source = "headlines"

[nodes.noise]
type = "effect"
effect = "noise"
intensity = 0.5

[nodes.fade]
type = "effect"
effect = "fade"
intensity = 0.8

[nodes.display]
type = "display"
backend = "terminal"

[connections]
list = ["source -> noise -> fade -> display"]

With Positioning

[nodes.source]
type = "source"
source = "headlines"

[nodes.position]
type = "position"
mode = "mixed"

[nodes.display]
type = "display"
backend = "terminal"

[connections]
list = ["source -> position -> display"]

Python API

Basic Construction

from engine.pipeline.graph import Graph, NodeType

graph = Graph()
graph.node("source", NodeType.SOURCE, source="headlines")
graph.node("camera", NodeType.CAMERA, mode="scroll")
graph.node("display", NodeType.DISPLAY, backend="terminal")
graph.chain("source", "camera", "display")

pipeline = graph_to_pipeline(graph)

With Effects

from engine.pipeline.graph import Graph, NodeType

graph = Graph()
graph.node("source", NodeType.SOURCE, source="headlines")
graph.node("noise", NodeType.EFFECT, effect="noise", intensity=0.5)
graph.node("fade", NodeType.EFFECT, effect="fade", intensity=0.8)
graph.node("display", NodeType.DISPLAY, backend="terminal")
graph.chain("source", "noise", "fade", "display")

pipeline = graph_to_pipeline(graph)

Dictionary/JSON Input

from engine.pipeline.graph_adapter import dict_to_pipeline

data = {
    "nodes": {
        "source": "headlines",
        "noise": {"type": "effect", "effect": "noise", "intensity": 0.5},
        "display": {"type": "display", "backend": "terminal"}
    },
    "connections": ["source -> noise -> display"]
}

pipeline = dict_to_pipeline(data)

CLI Usage

Using Graph Config File

mainline --graph-config pipeline.toml

Inline Graph Definition

mainline --graph 'source:headlines -> noise:noise:0.5 -> display:terminal'

With Preset Override

mainline --preset demo --graph-modify 'add:noise:0.5 after:source'

Node Types

Type	Description	Config Options
source	Data source	source: "headlines", "poetry", "empty", etc.
camera	Viewport camera	mode: "scroll", "feed", "horizontal", etc. speed: float
effect	Visual effect	effect: effect name, intensity: 0.0-1.0
position	Positioning mode	mode: "absolute", "relative", "mixed"
display	Output backend	backend: "terminal", "null", "websocket", etc.
render	Text rendering	(auto-injected)
overlay	Message overlay	(auto-injected)

Advanced Features

Conditional Connections

[connections]
list = ["source -> camera -> display"]
# Effects can be conditionally enabled/disabled

Parameter Binding

[nodes.noise]
type = "effect"
effect = "noise"
intensity = 1.0
# intensity can be bound to sensor values at runtime

Pipeline Inspection

[nodes.inspect]
type = "pipeline-inspect"
# Renders live pipeline visualization

Comparison with Stage-Based Approach

Old (Stage-Based)

pipeline = Pipeline()
pipeline.add_stage("source", DataSourceStage(HeadlinesDataSource()))
pipeline.add_stage("camera", CameraStage(Camera.scroll()))
pipeline.add_stage("render", FontStage())
pipeline.add_stage("noise", EffectPluginStage(noise_effect))
pipeline.add_stage("display", DisplayStage(terminal_display))

New (Graph-Based)

graph = Graph()
graph.node("source", NodeType.SOURCE, source="headlines")
graph.node("camera", NodeType.CAMERA, mode="scroll")
graph.node("noise", NodeType.EFFECT, effect="noise")
graph.node("display", NodeType.DISPLAY, backend="terminal")
graph.chain("source", "camera", "noise", "display")
pipeline = graph_to_pipeline(graph)

The graph system automatically:

• Inserts the render stage between camera and effects
• Handles capability-based dependency resolution
• Auto-injects required stages (camera_update, render, etc.)