forked from genewildish/Mainline
fix: Implement ViewportFilterStage to prevent FontStage performance regression with large datasets
## Summary Fixed critical performance issue where demo/poetry presets would hang for 10+ seconds due to FontStage rendering all 1438+ headline items instead of just the visible ~5 items. ## Changes ### Core Fix: ViewportFilterStage - New pipeline stage that filters items to only those fitting in the viewport - Reduces 1438 items → ~5 items (288x reduction) before FontStage - Prevents expensive PIL font rendering operations on items that won't be displayed - Located: engine/pipeline/adapters.py:348-403 ### Pipeline Integration - Updated app.py to add ViewportFilterStage before FontStage for headlines/poetry sources - Ensures correct data flow: source → viewport_filter → font → camera → effects → display - ViewportFilterStage depends on 'source' capability, providing pass-through filtering ### Display Protocol Enhancement - Added is_quit_requested() and clear_quit_request() method signatures to Display protocol - Documented as optional methods for backends supporting keyboard input - Already implemented by pygame backend, now formally part of protocol ### Debug Infrastructure - Added MAINLINE_DEBUG_DATAFLOW environment variable logging throughout pipeline - Logs stage input/output types and data sizes to stderr (when flag enabled) - Verified working: 1438 → 5 item reduction shown in debug output ### Performance Testing - Added pytest-benchmark (v5.2.3) as dev dependency for statistical benchmarking - Created comprehensive performance regression tests (tests/test_performance_regression.py) - Tests verify: - ViewportFilterStage filters 2000 items efficiently (<1ms) - FontStage processes filtered items quickly (<50ms) - 288x performance improvement ratio maintained - Pipeline doesn't hang with large datasets - All 523 tests passing, including 7 new performance tests ## Performance Impact **Before:** FontStage renders all 1438 items per frame → 10+ second hang **After:** FontStage renders ~5 items per frame → sub-second execution Real-world impact: Demo preset now responsive and usable with news sources. ## Testing - Unit tests: 523 passed, 16 skipped - Regression tests: Catch performance degradation with large datasets - E2E verification: Debug logging confirms correct pipeline flow - Benchmark suite: Statistical performance tracking enabled
This commit is contained in:
@@ -154,8 +154,12 @@ def run_pipeline_mode(preset_name: str = "demo"):
|
||||
|
||||
# Add FontStage for headlines/poetry (default for demo)
|
||||
if preset.source in ["headlines", "poetry"]:
|
||||
from engine.pipeline.adapters import FontStage
|
||||
from engine.pipeline.adapters import FontStage, ViewportFilterStage
|
||||
|
||||
# Add viewport filter to prevent rendering all items
|
||||
pipeline.add_stage(
|
||||
"viewport_filter", ViewportFilterStage(name="viewport-filter")
|
||||
)
|
||||
pipeline.add_stage("font", FontStage(name="font"))
|
||||
else:
|
||||
# Fallback to simple conversion for other sources
|
||||
|
||||
Reference in New Issue
Block a user