feat(integration): Complete feature rewrite with pipeline architecture, effects system, and display improvements

Major changes:
- Pipeline architecture with capability-based dependency resolution
- Effects plugin system with performance monitoring
- Display abstraction with multiple backends (terminal, null, websocket)
- Camera system for viewport scrolling
- Sensor framework for real-time input
- Command-and-control system via ntfy
- WebSocket display backend for browser clients
- Comprehensive test suite and documentation

Issue #48: ADR for preset scripting language included

This commit consolidates 110 individual commits into a single
feature integration that can be reviewed and tested before
further refinement.
This commit is contained in:
2026-03-20 04:41:23 -07:00
parent 42aa6f16cc
commit ef98add0c5
179 changed files with 27649 additions and 6552 deletions

View File

@@ -0,0 +1,136 @@
---
name: mainline-sensors
description: Sensor framework for real-time input in Mainline
compatibility: opencode
metadata:
audience: developers
source_type: codebase
---
## What This Skill Covers
This skill covers Mainline's sensor framework - how to use, create, and integrate sensors for real-time input.
## Key Concepts
### Sensor Base Class (engine/sensors/__init__.py)
```python
class Sensor(ABC):
name: str
unit: str = ""
@property
def available(self) -> bool:
"""Whether sensor is currently available"""
return True
@abstractmethod
def read(self) -> SensorValue | None:
"""Read current sensor value"""
...
def start(self) -> None:
"""Initialize sensor (optional)"""
pass
def stop(self) -> None:
"""Clean up sensor (optional)"""
pass
```
### SensorValue Dataclass
```python
@dataclass
class SensorValue:
sensor_name: str
value: float
timestamp: float
unit: str = ""
```
### SensorRegistry
Discovers and manages sensors globally:
```python
from engine.sensors import SensorRegistry
registry = SensorRegistry()
sensor = registry.get("mic")
```
### SensorStage
Pipeline adapter that provides sensor values to effects:
```python
from engine.pipeline.adapters import SensorStage
stage = SensorStage(sensor_name="mic")
```
## Built-in Sensors
| Sensor | File | Description |
|--------|------|-------------|
| MicSensor | sensors/mic.py | Microphone input (RMS dB) |
| OscillatorSensor | sensors/oscillator.py | Test sine wave generator |
| PipelineMetricsSensor | sensors/pipeline_metrics.py | FPS, frame time, etc. |
## Param Bindings
Effects declare sensor-to-param mappings:
```python
class GlitchEffect(EffectPlugin):
param_bindings = {
"intensity": {"sensor": "mic", "transform": "linear"},
}
```
### Transform Functions
- `linear` - Direct mapping to param range
- `exponential` - Exponential scaling
- `threshold` - Binary on/off
## Adding a New Sensor
1. Create `engine/sensors/my_sensor.py`
2. Inherit from `Sensor` ABC
3. Implement required methods
4. Register in `SensorRegistry`
Example:
```python
class MySensor(Sensor):
name = "my-sensor"
unit = "units"
def read(self) -> SensorValue | None:
return SensorValue(
sensor_name=self.name,
value=self._read_hardware(),
timestamp=time.time(),
unit=self.unit
)
```
## Using Sensors in Effects
Access sensor values via EffectContext:
```python
def process(self, buf, ctx):
mic_level = ctx.get_sensor_value("mic")
if mic_level and mic_level > 0.5:
# Apply intense effect
...
```
Or via param_bindings (automatic):
```python
# If intensity is bound to "mic", it's automatically
# available in self.config.intensity
```