Move internal imports in run_pipeline_mode() to module level to support
proper mocking in integration tests. This enables more effective testing
of the app's initialization and pipeline setup.
Also simplifies the test suite to focus on key integration points.
Changes:
- Moved effects_plugins, DisplayRegistry, PerformanceMonitor, fetch functions,
and pipeline adapters to module-level imports
- Removed duplicate imports from run_pipeline_mode()
- Simplified test_app.py to focus on core functionality
All manual tests still pass (border-test preset works correctly).
Added initial integration test suite structure for engine/app.py covering:
- main() entry point preset loading
- run_pipeline_mode() pipeline setup
- Content fetching for different sources
- Display initialization
- CLI flag overrides
STATUS: Tests are currently failing because imports in run_pipeline_mode()
are internal to the function, making them difficult to patch. This requires:
1. Refactoring imports to module level, OR
2. Using more sophisticated patching strategies (patch at import time)
This provides a foundation for Phase 2. Tests will be fixed in next iteration.
PHASE 2 TASKS:
- Fix app.py test patching issues
- Add data source tests (currently 34% coverage)
- Expand adapter tests (currently 50% coverage)
- Target: 70% coverage on critical paths
BUG FIXES:
1. Border parameter not being passed to display.show()
- Display backends support border parameter but app.py wasn't passing it
- Now app.py passes params.border to display.show(border=params.border)
- Enables border-test preset to actually render borders
2. WebSocket and Multi displays didn't support border parameter
- Updated WebSocket Protocol to include border parameter
- Updated MultiDisplay.show() to accept and forward border parameter
- Updated test to expect border parameter in mock calls
3. app.py didn't properly handle special sources (empty, pipeline-inspect)
- Border-test preset with source='empty' was still fetching headlines
- Pipeline-inspect source was never using the introspection data source
- Now app.py detects special sources and uses appropriate data source stages:
* 'empty' source → EmptyDataSource stage
* 'pipeline-inspect' → PipelineIntrospectionSource stage
* Other sources → traditional items-based approach
- Uses SourceItemsToBufferStage for special sources instead of RenderStage
- Sets pipeline on introspection source after build to avoid circular dependency
TESTING:
- All 463 tests pass
- Linting passes
- Manual test: `uv run mainline.py --preset border-test` now correctly shows empty source
- border-test preset now properly initializes without fetching unnecessary content
The issue was that the enhanced app.py code from the original diff didn't make it into
the refactor commit. This fix restores that functionality.
MAJOR REFACTORING: Consolidate duplicated pipeline code and standardize on
capability-based dependency resolution. This is a significant but backwards-compatible
restructuring that improves maintainability and extensibility.
## ARCHITECTURE CHANGES
### Data Sources Consolidation
- Move engine/sources_v2.py → engine/data_sources/sources.py
- Move engine/pipeline_sources/ → engine/data_sources/
- Create unified DataSource ABC with common interface:
* fetch() - idempotent data retrieval
* get_items() - cached access with automatic refresh
* refresh() - force cache invalidation
* is_dynamic - indicate streaming vs static sources
- Support for SourceItem dataclass (content, source, timestamp, metadata)
### Display Backend Improvements
- Update all 7 display backends to use new import paths
- Terminal: Improve dimension detection and handling
- WebSocket: Better error handling and client lifecycle
- Sixel: Refactor graphics rendering
- Pygame: Modernize event handling
- Kitty: Add protocol support for inline images
- Multi: Ensure proper forwarding to all backends
- Null: Maintain testing backend functionality
### Pipeline Adapter Consolidation
- Refactor adapter stages for clarity and flexibility
- RenderStage now handles both item-based and buffer-based rendering
- Add SourceItemsToBufferStage for converting data source items
- Improve DataSourceStage to work with all source types
- Add DisplayStage wrapper for display backends
### Camera & Viewport Refinements
- Update Camera class for new architecture
- Improve viewport dimension detection
- Better handling of resize events across backends
### New Effect Plugins
- border.py: Frame rendering effect with configurable style
- crop.py: Viewport clipping effect for selective display
- tint.py: Color filtering effect for atmosphere
### Tests & Quality
- Add test_border_effect.py with comprehensive border tests
- Add test_crop_effect.py with viewport clipping tests
- Add test_tint_effect.py with color filtering tests
- Update test_pipeline.py for new architecture
- Update test_pipeline_introspection.py for new data source location
- All 463 tests pass with 56% coverage
- Linting: All checks pass with ruff
### Removals (Code Cleanup)
- Delete engine/benchmark.py (deprecated performance testing)
- Delete engine/pipeline_sources/__init__.py (moved to data_sources)
- Delete engine/sources_v2.py (replaced by data_sources/sources.py)
- Update AGENTS.md to reflect new structure
### Import Path Updates
- Update engine/pipeline/controller.py::create_default_pipeline()
* Old: from engine.sources_v2 import HeadlinesDataSource
* New: from engine.data_sources.sources import HeadlinesDataSource
- All display backends import from new locations
- All tests import from new locations
## BACKWARDS COMPATIBILITY
This refactoring is intended to be backwards compatible:
- Pipeline execution unchanged (DAG-based with capability matching)
- Effect plugins unchanged (EffectPlugin interface same)
- Display protocol unchanged (Display duck-typing works as before)
- Config system unchanged (presets.toml format same)
## TESTING
- 463 tests pass (0 failures, 19 skipped)
- Full linting check passes
- Manual testing on demo, poetry, websocket modes
- All new effect plugins tested
## FILES CHANGED
- 24 files modified/added/deleted
- 723 insertions, 1,461 deletions (net -738 LOC - cleanup!)
- No breaking changes to public APIs
- All transitive imports updated correctly
- Add PipelineIntrospectionSource that renders live ASCII DAG with metrics
- Add PipelineMetricsSensor exposing pipeline performance as sensor values
- Add PipelineIntrospectionDemo controller with 3-phase animation:
- Phase 1: Toggle effects one at a time (3s each)
- Phase 2: LFO drives intensity default→max→min→default
- Phase 3: All effects with shared LFO (infinite loop)
- Add pipeline-inspect preset
- Add get_frame_times() to Pipeline for sparkline data
- Add tests for new components
- Update mise.toml with pipeline-inspect preset task
- Remove ~700 lines of legacy code from app.py (run_demo_mode, run_pipeline_demo,
run_preset_mode, font picker, effects picker)
- HUD now reads metrics from pipeline context (first-class citizen) with fallback
to global monitor for backwards compatibility
- Add validate_signal_flow() for PureData-style type validation in presets
- Update MicSensor documentation (self-contained, doesn't use MicMonitor)
- Delete test_app.py (was testing removed legacy code)
- Update AGENTS.md with pipeline architecture documentation
- Add DataType enum (SOURCE_ITEMS, TEXT_BUFFER, etc.)
- Add inlet_types and outlet_types to Stage
- Add _validate_types() for type checking at build time
- Update tests with proper type annotations
- Add Sensor base class with value emission
- Add SensorRegistry for discovery
- Add SensorStage adapter for pipeline
- Add MicSensor (self-contained, no external deps)
- Add OscillatorSensor for testing
- Add sensor param bindings to effects
- Add TransformDataSource for filtering/mapping source items
- Add MetricsDataSource for rendering live pipeline metrics as ASCII art
- Fix display stage registration in StageRegistry
- Register sources with both class name and simple name aliases
- Fix DisplayStage.init() to pass reuse parameter
- Simplify create_default_pipeline to use DataSourceStage wrapper
- Set pygame as default display
- Remove old pipeline tasks from mise.toml
- Add tests for new pipeline architecture
- Add Camera class with modes: vertical, horizontal, omni, floating
- Refactor scroll.py and demo to use Camera abstraction
- Add vis_offset for horizontal scrolling support
- Add camera_x to EffectContext for effects
- Add pygame window resize handling
- Add HUD effect plugin for demo mode
- Add --demo flag to run demo mode
- Add tests for Camera and vis_offset
- Add HUD effect plugin showing FPS, effect name, intensity bar, pipeline
- Add pygame window resize handling (VIDEORESIZE event)
- Move HUD to end of chain so it renders on top
- Fix monitor stats API (returns dict, not object)
- Add tests/test_hud.py for HUD effect verification
- Add renderer.py with parse_ansi(), get_default_font_path(), render_to_pil()
- Update KittyDisplay and SixelDisplay to use shared renderer
- Enhance parse_ansi to handle full ANSI color codes (4-bit, 256-color)
- Update tests to use shared renderer functions
Add typed dataclasses for tuple returns:
- types.py: HeadlineItem, FetchResult, Block dataclasses with legacy tuple converters
- fetch.py: Add type hints and HeadlineTuple type alias
Add pyright for static type checking:
- Add pyright to dependencies
- Verify type coverage with pyright (0 errors in core modules)
This enables:
- Named types instead of raw tuples (better IDE support, self-documenting)
- Type-safe APIs across modules
- Backward compatibility via to_tuple/from_tuple methods
Note: Lazy imports skipped for render.py - startup impact is minimal.
- Fix import sorting (isort) across all engine modules
- Fix SIM105 try-except-pass patterns (contextlib.suppress)
- Fix nested with statements in tests
- Fix unused loop variables
Run 'uv run pytest' to verify tests still pass.
