docs: Add ADR-007 Pipeline Introspection and Epic #20

2026-03-18 00:46:15 -07:00
parent 6df619ced8
commit ea56c72b6b
3 changed files with 67 additions and 6 deletions

@@ -0,0 +1,61 @@
# ADR-007: Pipeline Introspection
**Status:** ✅ Done
**Date:** 2026-03-17
**Author:** David G
**Related Issues:** [#22](https://git.notsosm.art/david/Mainline/issues/22), [#20](https://git.notsosm.art/david/Mainline/issues/20)
## Context
During development of the Stage-based pipeline architecture, there was no way to visualize:
- The structure of the pipeline (which stages are connected)
- Signal flow between stages
- Real-time performance metrics (execution times, frame rates)
This made debugging and development of the pipeline system difficult.
## Decision
Implement a `PipelineIntrospectionSource` DataSource that introspects Pipeline instances and renders ASCII visualization:
- **DAG visualization**: ASCII diagram showing stages and their connections
- **Signal flow**: Shows data moving between stages
- **Execution times**: Per-stage timing information
- **Frame history**: Sparkline of recent frame times
- **Stage breakdown**: Bar chart of time spent in each stage
## Implementation
Location: `engine/data_sources/pipeline_introspection.py`
```python
from engine.data_sources.pipeline_introspection import PipelineIntrospectionSource
# Usage
source = PipelineIntrospectionSource(pipeline=my_pipeline)
items = source.fetch() # Returns ASCII visualization lines
```
The source can be used like any other DataSource in the pipeline, enabling debug views to be displayed alongside normal content.
## Alternatives Considered
1. **Separate debug UI**: Would require additional rendering logic
2. **Logging-based metrics**: Less visual, harder to interpret
3. **External tooling**: Would require separate tool, not integrated
## Consequences
- **Positive**: Developers can visualize pipeline structure at runtime
- **Positive**: Performance issues are easier to identify
- **Positive**: Can be used as a DataSource like any other
- **Negative**: Adds small overhead when enabled (negligible)
## References
- `engine/data_sources/pipeline_introspection.py`
- `engine/data_sources/sources.py` (base DataSource class)
- Pipeline Introspection Demo: `engine/pipeline/pipeline_introspection_demo.py`

@@ -10,6 +10,7 @@ Architecture Decision Records documenting the "why" behind our architectural cho
| [ADR-004](ADR-004-Display-Protocol-Pattern) | Display Protocol Pattern | ✅ Done | [#12](https://git.notsosm.art/david/Mainline/issues/12), [#13](https://git.notsosm.art/david/Mainline/issues/13), [#14](https://git.notsosm.art/david/Mainline/issues/14) |
| [ADR-005](ADR-005-Sensor-Framework) | Sensor Framework | ✅ Done | [#15](https://git.notsosm.art/david/Mainline/issues/15), [#16](https://git.notsosm.art/david/Mainline/issues/16), [#18](https://git.notsosm.art/david/Mainline/issues/18) |
| [ADR-006](ADR-006-Preset-TOML-Format) | Preset TOML Format | ✅ Done | Part of [#6](https://git.notsosm.art/david/Mainline/issues/6) (Epic) |
| [ADR-007](ADR-007-Pipeline-Introspection) | Pipeline Introspection | ✅ Done | [#22](https://git.notsosm.art/david/Mainline/issues/22), [#20](https://git.notsosm.art/david/Mainline/issues/20) (Epic) |
## About ADRs

@@ -11,18 +11,17 @@ Major feature areas organized as Epics and Stories.
| [#4](https://git.notsosm.art/david/Mainline/issues/4) | Display Backend Expansion | [#12](https://git.notsosm.art/david/Mainline/issues/12), [#13](https://git.notsosm.art/david/Mainline/issues/13), [#14](https://git.notsosm.art/david/Mainline/issues/14) | ✅ Done |
| [#5](https://git.notsosm.art/david/Mainline/issues/5) | Real-Time Input (Sensors) | [#15](https://git.notsosm.art/david/Mainline/issues/15), [#16](https://git.notsosm.art/david/Mainline/issues/16) | ✅ Done |
| [#6](https://git.notsosm.art/david/Mainline/issues/6) | Effects Plugin System | [#17](https://git.notsosm.art/david/Mainline/issues/17), [#18](https://git.notsosm.art/david/Mainline/issues/18) | ✅ Done |
| [#20](https://git.notsosm.art/david/Mainline/issues/20) | Additional Display Backends & Pipeline Tools | [#21](https://git.notsosm.art/david/Mainline/issues/21), [#22](https://git.notsosm.art/david/Mainline/issues/22), [#23](https://git.notsosm.art/david/Mainline/issues/23), [#24](https://git.notsosm.art/david/Mainline/issues/24) | ✅ Done |
## Incomplete Work
| Issue | Title | Status |
|-------|-------|--------|
| [#19](https://git.notsosm.art/david/Mainline/issues/19) | Legacy Cleanup Phase 5 | ⏳ Not Started |
_(None currently - all work complete!)_
## Story Summary
- **Total Stories:** 12
- **Completed:** 11
- **Incomplete:** 1 (Legacy cleanup)
- **Total Stories:** 16
- **Completed:** 16
- **Incomplete:** 0
## Quick Links