fix: Implement ViewportFilterStage to prevent FontStage performance regression with large datasets

## Summary

Fixed critical performance issue where demo/poetry presets would hang for 10+ seconds due to FontStage rendering all 1438+ headline items instead of just the visible ~5 items.

## Changes

### Core Fix: ViewportFilterStage
- New pipeline stage that filters items to only those fitting in the viewport
- Reduces 1438 items → ~5 items (288x reduction) before FontStage
- Prevents expensive PIL font rendering operations on items that won't be displayed
- Located: engine/pipeline/adapters.py:348-403

### Pipeline Integration
- Updated app.py to add ViewportFilterStage before FontStage for headlines/poetry sources
- Ensures correct data flow: source → viewport_filter → font → camera → effects → display
- ViewportFilterStage depends on 'source' capability, providing pass-through filtering

### Display Protocol Enhancement
- Added is_quit_requested() and clear_quit_request() method signatures to Display protocol
- Documented as optional methods for backends supporting keyboard input
- Already implemented by pygame backend, now formally part of protocol

### Debug Infrastructure
- Added MAINLINE_DEBUG_DATAFLOW environment variable logging throughout pipeline
- Logs stage input/output types and data sizes to stderr (when flag enabled)
- Verified working: 1438 → 5 item reduction shown in debug output

### Performance Testing
- Added pytest-benchmark (v5.2.3) as dev dependency for statistical benchmarking
- Created comprehensive performance regression tests (tests/test_performance_regression.py)
- Tests verify:
  - ViewportFilterStage filters 2000 items efficiently (<1ms)
  - FontStage processes filtered items quickly (<50ms)
  - 288x performance improvement ratio maintained
  - Pipeline doesn't hang with large datasets
- All 523 tests passing, including 7 new performance tests

## Performance Impact

**Before:** FontStage renders all 1438 items per frame → 10+ second hang
**After:** FontStage renders ~5 items per frame → sub-second execution

Real-world impact: Demo preset now responsive and usable with news sources.

## Testing

- Unit tests: 523 passed, 16 skipped
- Regression tests: Catch performance degradation with large datasets
- E2E verification: Debug logging confirms correct pipeline flow
- Benchmark suite: Statistical performance tracking enabled
This commit is contained in:
2026-03-16 22:43:53 -07:00
parent 10c1d057a9
commit 4c97cfe6aa
7 changed files with 468 additions and 1 deletions

View File

@@ -45,6 +45,7 @@ browser = [
]
dev = [
"pytest>=8.0.0",
"pytest-benchmark>=4.0.0",
"pytest-cov>=4.1.0",
"pytest-mock>=3.12.0",
"ruff>=0.1.0",
@@ -60,6 +61,7 @@ build-backend = "hatchling.build"
[dependency-groups]
dev = [
"pytest>=8.0.0",
"pytest-benchmark>=4.0.0",
"pytest-cov>=4.1.0",
"pytest-mock>=3.12.0",
"ruff>=0.1.0",