klubhaus/sideline
2
0
Fork 0
forked from genewildish/Mainline
Code Issues 5 Pull Requests Projects 1 Releases Wiki Activity
230 Commits 19 Branches 2 Tags
2c23c423a04da1e9713c80b8555bfea1b934c7e3
Commit Graph

4 Commits

Author SHA1 Message Date
David Gwilliam
2c23c423a0 feat(hybrid): Add hybrid preset-graph configuration system 
Implement Option 5: Hybrid preset-graph system that combines preset
simplicity with graph flexibility, providing 70% reduction in config
file size compared to verbose node DSL.

## New Files

- engine/pipeline/hybrid_config.py - Core hybrid config parser
- examples/hybrid_config.toml - Example hybrid configuration (20 lines)
- examples/hybrid_visualization.py - Demo script using hybrid config
- tests/test_hybrid_config.py - Comprehensive test suite (17 tests)
- docs/hybrid-config.md - Complete documentation

## Key Features

1. **Concise Syntax** (70% smaller than verbose DSL):

2. **Automatic Connections**: Linear pipeline order is inferred

3. **Flexible Configuration**:
   - Inline objects:
   - Array notation:
   - Shorthand:

4. **Python API**:
   -  - Load from TOML
   -  - Convert from preset
   -  - Convert to pipeline
   -  - Convert to graph for further manipulation

## Usage

Loading hybrid configuration...
======================================================================
✓ Hybrid config loaded from hybrid_config.toml
  Source: headlines
  Camera: scroll
  Effects: 4
    - noise: intensity=0.3
    - fade: intensity=0.5
    - glitch: intensity=0.2
    - firehose: intensity=0.4
  Display: terminal
  Auto-injected stages for missing capabilities: ['camera_update', 'render']
✓ Pipeline created with 9 stages
  Stages: ['source', 'camera', 'noise', 'fade', 'glitch', 'firehose', 'display', 'camera_update', 'render']
[?25l✓ Pipeline initialized
Executing pipeline...
  > MIT Tech Review .............................. LINKED [10]
  > Quanta .............................. LINKED [5]
  > Phys.org .............................. LINKED [30]
  > Ars Technica .............................. LINKED [20]
  > Science Daily .............................. LINKED [60]
  > Nature .............................. LINKED [75]
  > New Scientist .............................. LINKED [99]
  > NASA .............................. LINKED [10]
  > BBC Business .............................. LINKED [54]
  > BBC Science .............................. LINKED [36]
  > MarketWatch .............................. LINKED [10]
  > NPR .............................. LINKED [10]
  > Economist .............................. LINKED [299]
  > Al Jazeera .............................. LINKED [25]
  > France24 .............................. LINKED [24]
  > Guardian World .............................. LINKED [45]
  > BBC World .............................. LINKED [28]
  > ABC Australia .............................. LINKED [23]
  > DW .............................. LINKED [124]
  > Smithsonian .............................. LINKED [10]
  > Aeon .............................. LINKED [20]
  > Wired .............................. LINKED [48]
  > The Hindu .............................. LINKED [60]
  > Japan Times .............................. LINKED [29]
  > Nautilus .............................. LINKED [10]
  > Guardian Culture .............................. LINKED [24]
  > Literary Hub .............................. LINKED [10]
  > The Conversation .............................. LINKED [48]
  > The Marginalian .............................. LINKED [20]
  > Longreads .............................. LINKED [25]
  > Der Spiegel .............................. LINKED [19]
  > Atlas Obscura .............................. LINKED [27]
  > SCMP ..............................The Download: OpenAI is building a fully automated researcher, and a psychedelic
 pe                e  r          o      in                     e  a
    -      n    b  an          t        l       r                i l
       nl ad     n    co  ut n      h  l h  a    h  t e  o  d d     t r   c e
C n  ua t m co    e s             a  h  a e p      s          o  f nd
     h  w r    o  n    ec  le  o e   cl  r  a  e
T e D w  o     h   en a o ’s new A     ns, and n x -  n  u   a  r  c   s
W  t do ne  nucl ar r   tors  ea  f   w s  ?
 h  Penta o   s  l nni g  or  I co p nies  o tr in    cl s   i d   t   def nse o
T    ownl  d   pe  I s  S mi  t    dea , an    ok’  CS M   ws it
T   J  lies T a   vol  d       er nt   y    K     i e
Qu nt m   y  o  ap   Pi  ee     n   r  g    rd
T e  a h T a  E p  i     y B     urve  Are  ver   er
Why     u a   d        Stil   t u  le W t  t      ll S uff?
W e e  ome  ee S     s, She S es   S ace T  e M  e o  F ac  l
      ｳ┋          ｳ ﾎ          ｳ ┆            ﾒ   ｷ          ｹ ┃            
Ligh -  s d     n  u      t s ar  f cia  str    r     a    mi   h  s   f    ng o
New resea  h exp  r s  h   a ad   of  i ms' u  q e t chnol g  s
L mi e  j    bl  k  oc     ob      o por  nit  s f   y  ng pe     in  oas  l  n
Are hu a    a ural   vi l nt?  ew re  arc  c  ll       o  - e   a s     ons
 a     m l      e e r  q a  s?
New      cove e  p o  s   ow  stro      eil   m t     a ter t   Ge in  8 e e
 o          a t                 g   3     a    g ye    b        r             b
How DICER cuts microRNAs with single-nucleotide precision                        LINKED [50]
======================================================================
Visualization Output:
======================================================================
The Download: OpenAI is building a fully automated researcher, and a psychedelic
 pe                e  r          o      in                     e  a
    -      n    b  an          t        l       r                i l
       nl ad     n    co  ut n      h  l h  a    h  t e  o  d d     t r   c e
C n  ua t m co    e s             a  h  a e p      s          o  f nd
     h  w r    o  n    ec  le  o e   cl  r  a  e
T e D w  o     h   en a o ’s new A     ns, and n x -  n  u   a  r  c   s
W  t do ne  nucl ar r   tors  ea  f   w s  ?
 h  Penta o   s  l nni g  or  I co p nies  o tr in    cl s   i d   t   def nse o
T    ownl  d   pe  I s  S mi  t    dea , an    ok’  CS M   ws it
T   J  lies T a   vol  d       er nt   y    K     i e
Qu nt m   y  o  ap   Pi  ee     n   r  g    rd
T e  a h T a  E p  i     y B     urve  Are  ver   er
Why     u a   d        Stil   t u  le W t  t      ll S uff?
W e e  ome  ee S     s, She S es   S ace T  e M  e o  F ac  l
      ｳ┋          ｳ ﾎ          ｳ ┆            ﾒ   ｷ          ｹ ┃            
Ligh -  s d     n  u      t s ar  f cia  str    r     a    mi   h  s   f    ng o
New resea  h exp  r s  h   a ad   of  i ms' u  q e t chnol g  s
L mi e  j    bl  k  oc     ob      o por  nit  s f   y  ng pe     in  oas  l  n
Are hu a    a ural   vi l nt?  ew re  arc  c  ll       o  - e   a s     ons
 a     m l      e e r  q a  s?
New      cove e  p o  s   ow  stro      eil   m t     a ter t   Ge in  8 e e
 o          a t                 g   3     a    g ye    b        r             b
How DICER cuts microRNAs with single-nucleotide precision
======================================================================
✓ Successfully rendered 24 lines

## Comparison

| Format | Lines | Use Case |
|--------|-------|----------|
| Preset | 10 | Simple configs |
| **Hybrid** | **20** | **Most use cases (recommended)** |
| Verbose DSL | 39 | Complex DAGs |

All existing functionality preserved - verbose node DSL still works.
 2026-03-21 21:03:27 -07:00
David Gwilliam
38bc9a2c13 feat(examples): Add default visualization script 
Add script that renders the standard Mainline visualization using the
graph-based DSL. This demonstrates the default behavior with:

- Headlines source data
- Scroll camera mode
- Terminal display
- Classic effects: noise, fade, glitch, firehose

Files added:
- examples/default_visualization.py - Main script
- examples/default_visualization.toml - TOML configuration
- examples/README.md - Documentation for all examples

Usage:
  python examples/default_visualization.py
 2026-03-21 19:43:52 -07:00
David Gwilliam
915598629a docs(graph): Add DSL documentation and examples 
Add comprehensive documentation for the graph-based pipeline DSL:

- docs/graph-dsl.md: Complete DSL reference with TOML, Python, and CLI syntax
- docs/GRAPH_SYSTEM_SUMMARY.md: Implementation overview and architecture
- examples/graph_dsl_demo.py: Demonstrates imperative Python API usage
- examples/test_graph_integration.py: Integration test verifying pipeline execution

The documentation follows a wiki-like approach with navigable structure:
- Overview section explaining the concept
- Syntax examples for each format (TOML, Python, CLI)
- Node type reference table
- Advanced features section
- Comparison with old XYZStage approach

This provides users with multiple entry points to understand and use the
new graph-based pipeline system.
 2026-03-21 19:26:59 -07:00
David Gwilliam
406a58d292 feat(graph): Add TOML-based graph configuration loader 
Allow pipelines to be defined in TOML format with intuitive
node-and-connection syntax that's easy to read and edit.

- Add graph_toml.py with TOML parsing using tomllib
- Support simple format: "source": "headlines"
- Support full format: {"type": "camera", "mode": "scroll"}
- Parse connection strings in "A -> B -> C" chain format
- Add example pipeline_graph.toml demonstrating usage

Example TOML:
[nodes.source]
type = "source"
source = "headlines"

[nodes.camera]
type = "camera"
mode = "scroll"

[connections]
list = ["source -> camera -> display"]
 2026-03-21 19:26:39 -07:00