klubhaus/klubhaus-doorbell
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

initial commit

Browse Source
This commit is contained in:
David Gwilliam
2026-02-12 00:45:31 -08:00
commit 5f168f370b
3024 changed files with 804889 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

50
libraries/FastLED/ci/util/scrapers/README.md Normal file
View File

@@ -0,0 +1,50 @@
# FastLED Web Scrapers
This directory contains web scraping tools for testing and capturing screenshots from the online FastLED tool.
## Contents
- **`scrape_festival_stick.py`** - Main web scraping script that navigates to the online FastLED tool and captures screenshots
- **`run_fastled_scraper.py`** - Utility script for easy execution with different configurations
- **`screenshots/`** - Directory containing captured screenshots with timestamps
- **`fastled_python_investigation.md`** - Comprehensive documentation of the investigation and implementation
## Quick Start
### Run from project root:
```bash
# Basic usage
uv run python ci/ci/scrapers/scrape_festival_stick.py
# With utility script options
uv run python ci/ci/scrapers/run_fastled_scraper.py --headless --timeout 60
```
### Run from scrapers directory:
```bash
cd ci/ci/scrapers
uv run python scrape_festival_stick.py
uv run python run_fastled_scraper.py --help
```
## Features
- **Automated browser navigation** to https://fastled.onrender.com/docs
- **Smart element detection** for FastLED interface components
- **Screenshot capture** with timestamp organization
- **File upload attempts** for FastLED examples
- **Error handling** with debug screenshots
- **Headless and visible modes** for different use cases
## Requirements
The scrapers use the existing FastLED Python environment with:
- `playwright` for browser automation
- Python dependencies from `pyproject.toml`
- Automatic Playwright browser installation
## Integration
These scrapers are integrated with the FastLED CI infrastructure and complement the existing web testing in `ci/wasm_test.py`.
For detailed information, see `fastled_python_investigation.md`.

197
libraries/FastLED/ci/util/scrapers/fastled_python_investigation.md Normal file
View File

@@ -0,0 +1,197 @@
# FastLED Python Code Investigation
## Overview
This document summarizes the investigation of existing Python code in the FastLED project, particularly focusing on Playwright integration and web scraping capabilities for the FastLED online tool.
**Location**: All scripts are located in the `ci/ci/scrapers/` directory, organized within the CI infrastructure alongside the existing testing infrastructure, including the original Playwright implementation in `ci/wasm_test.py`.
## Existing Python Infrastructure
### Dependencies (pyproject.toml)
The project already includes comprehensive dependencies for development and testing:
- **Playwright**: `playwright` - for browser automation
- **Testing**: `pytest`, `pytest-xdist` for parallel testing
- **FastLED**: `fastled>=1.2.26` - the FastLED Python package
- **FastLED WASM**: `fastled-wasm` - WebAssembly support
- **HTTP Client**: `httpx` - for HTTP requests
- **Build Tools**: `uv`, `ziglang`, `ninja`, `cmake`
- **Code Quality**: `ruff`, `mypy`, `pyright`, `clang-format`, `isort`, `black`
### Existing Playwright Implementation (`ci/wasm_test.py`)
The project already has a sophisticated Playwright setup that:
1. **Automatic Browser Installation**:
```python
def install_playwright_browsers():
os.system(f"{sys.executable} -m playwright install chromium")
```
2. **FastLED WASM Testing**:
- Starts an HTTP server for WASM examples
- Tests browser automation with the FastLED.js library
- Monitors `FastLED_onFrame` callback execution
- Validates WebGL/WASM functionality
3. **Error Handling**:
- Console log monitoring for errors
- Timeout handling for page loads
- Proper server cleanup
### MCP Server Integration (`mcp_server.py`)
The project includes a comprehensive MCP (Model Context Protocol) server with tools for:
- Running tests with various options
- Compiling examples for different platforms
- Code fingerprinting and change detection
- Linting and formatting
- Project information and status
## FestivalStick Example Analysis
### Core Functionality
The FestivalStick example (`examples/FestivalStick/`) is a sophisticated LED pattern demo featuring:
1. **Corkscrew LED Mapping**:
- 19.25 turns, 288 LEDs
- Maps 2D patterns to spiral LED positions
- Uses `fl::Corkscrew` class for geometric calculations
2. **UI Controls**:
- Speed, position, brightness controls
- Noise pattern generation with customizable parameters
- Color palette selection (Party, Heat, Ocean, Forest, Rainbow)
- Rendering mode options (Noise, Position, Mixed)
- Color boost with saturation/luminance functions
3. **Advanced Features**:
- Multi-sampling for accurate LED positioning
- Real-time noise generation with cylindrical mapping
- Auto-advance mode with manual position override
- ScreenMap integration for web visualization
### Key Components
```cpp
// Corkscrew configuration
#define NUM_LEDS 288
#define CORKSCREW_TURNS 19.25
// Runtime corkscrew with flexible configuration
Corkscrew::Input corkscrewInput(CORKSCREW_TURNS, NUM_LEDS, 0);
Corkscrew corkscrew(corkscrewInput);
// Frame buffer for 2D pattern drawing
fl::Grid<CRGB> frameBuffer;
// ScreenMap for web interface visualization
fl::ScreenMap corkscrewScreenMap = corkscrew.toScreenMap(0.2f);
```
## Web Scraping Script Implementation
### Script Features (`scrape_festival_stick.py`)
1. **Robust Web Navigation**:
- Navigates to https://fastled.onrender.com/docs
- Handles dynamic content loading
- Searches for multiple possible interface elements
2. **Smart Element Detection**:
- Looks for example selection dropdowns
- Detects file upload capabilities
- Finds canvas/visualization elements
- Searches for FestivalStick-specific content
3. **Screenshot Capabilities**:
- Full page screenshots with timestamps
- Focused canvas screenshots when available
- Error screenshots for debugging
- Multiple resolution support (1920x1080 default)
4. **File Upload Attempt**:
- Automatically tries to upload `FestivalStick.ino`
- Handles missing file scenarios gracefully
- Waits for upload processing
### Script Workflow
1. Install Playwright browsers automatically
2. Launch visible browser with slow motion for debugging
3. Navigate to online FastLED tool
4. Search for example/upload functionality
5. Attempt to load FestivalStick example
6. Capture screenshots of the visualization
7. Save timestamped results to `screenshots/` directory
## Project Testing Infrastructure
### Unit Tests
- Location: `tests/` directory
- Command: `bash test` (per user rules)
- Comprehensive C++ unit tests for all components
- Platform compilation tests
- Code quality checks
### Example Compilation
- Multi-platform support: `uno`, `esp32`, `teensy`, etc.
- Command: `./compile <platform> --examples <example_name>`
- Batch compilation for multiple platforms
- Interactive and automated modes
## Key Findings
1. **Comprehensive Infrastructure**: The FastLED project already has extensive Python tooling with Playwright, testing, and web automation capabilities.
2. **Advanced LED Visualization**: The FestivalStick example represents sophisticated LED pattern generation with real-time parameter control and web visualization.
3. **Web Integration Ready**: The existing WASM testing infrastructure provides a solid foundation for web-based LED visualization and interaction.
4. **Documentation Gap**: While the code is well-implemented, there could be more comprehensive documentation of the web tooling capabilities.
## Recommendations
1. **Extend Web Scraping**: The created script could be enhanced to:
- Test multiple examples automatically
- Capture video recordings of animations
- Perform parameter sweeps for different configurations
2. **Integration Testing**: Consider adding the web scraping script to the CI/CD pipeline for automated web interface testing.
3. **User Documentation**: Create user guides for the online FastLED tool and example usage.
## Results
✅ **Successfully captured screenshot**: `ci/ci/scrapers/screenshots/festival_stick_20250620_224055.png` (82KB)
The script successfully:
1. Navigated to https://fastled.onrender.com/docs
2. Detected and interacted with the FastLED web interface
3. Captured a full-page screenshot of the FestivalStick example visualization
4. Saved the result with timestamp for easy identification
## Files Created/Modified
- `ci/ci/scrapers/scrape_festival_stick.py` - Main web scraping script with Playwright automation
- `ci/ci/scrapers/run_fastled_scraper.py` - Utility script for easy execution with different configurations
- `ci/ci/scrapers/screenshots/` - Directory containing captured images
- `ci/ci/scrapers/screenshots/festival_stick_20250620_224055.png` - Successfully captured screenshot (82KB)
- `ci/ci/scrapers/fastled_python_investigation.md` - This documentation file
## Usage Examples
```bash
# Run the scraper directly from project root
uv run ci/ci/scrapers/scrape_festival_stick.py
# Use the utility script with options
uv run ci/ci/scrapers/run_fastled_scraper.py --example FestivalStick --headless --timeout 60
# Make scripts executable and run from project root
chmod +x ci/ci/scrapers/scrape_festival_stick.py ci/ci/scrapers/run_fastled_scraper.py
./ci/ci/scrapers/scrape_festival_stick.py
./ci/ci/scrapers/run_fastled_scraper.py --help
# Or run from within the scrapers directory
cd ci/ci/scrapers
uv run scrape_festival_stick.py
uv run run_fastled_scraper.py --help
```

76
libraries/FastLED/ci/util/scrapers/run_fastled_scraper.py Normal file
View File

@@ -0,0 +1,76 @@
#!/usr/bin/env python3
"""
FastLED Web Scraper Utility
This script provides an easy way to run the FastLED web scraper with different
configurations and examples.
"""
import argparse
import subprocess
import sys
from pathlib import Path
def run_scraper(
example_name: str = "FestivalStick", headless: bool = False, timeout: int = 30
) -> int:
"""Run the FastLED web scraper with specified parameters"""
script_path = Path(__file__).parent / "scrape_festival_stick.py"
if not script_path.exists():
print(f"Error: Scraper script not found at {script_path}", file=sys.stderr)
return 1
print(f"Running FastLED web scraper for {example_name}...")
print(f"Headless mode: {headless}")
print(f"Timeout: {timeout} seconds")
try:
# For now, just run the existing script
# In the future, this could be enhanced to pass parameters
result = subprocess.run(
[sys.executable, str(script_path)],
check=False,
capture_output=True,
text=True,
)
if result.returncode == 0:
print("✅ Scraping completed successfully!")
print(result.stdout)
else:
print("❌ Scraping failed!")
print("STDOUT:", result.stdout)
print("STDERR:", result.stderr)
return result.returncode
except Exception as e:
print(f"Error running scraper: {e}", file=sys.stderr)
return 1
def main() -> int:
parser = argparse.ArgumentParser(description="Run FastLED web scraper")
parser.add_argument(
"--example",
"-e",
default="FestivalStick",
help="Example name to scrape (default: FestivalStick)",
)
parser.add_argument(
"--headless", action="store_true", help="Run browser in headless mode"
)
parser.add_argument(
"--timeout", "-t", type=int, default=30, help="Timeout in seconds (default: 30)"
)
args = parser.parse_args()
return run_scraper(args.example, args.headless, args.timeout)
if __name__ == "__main__":
sys.exit(main())

206
libraries/FastLED/ci/util/scrapers/scrape_festival_stick.py Normal file
View File

@@ -0,0 +1,206 @@
#!/usr/bin/env python3
import asyncio
import os
import sys
from datetime import datetime
from pathlib import Path
from playwright.async_api import async_playwright # type: ignore
HERE = Path(__file__).parent
PROJECT_ROOT = HERE.parent.parent.parent # scrapers is 3 levels down from project root
SCREENSHOTS_DIR = HERE / "screenshots"
# Ensure screenshots directory exists
SCREENSHOTS_DIR.mkdir(exist_ok=True)
# Ensure Playwright browsers are installed
def install_playwright_browsers():
print("Installing Playwright browsers...")
try:
os.system(f"{sys.executable} -m playwright install chromium")
print("Playwright browsers installed successfully.")
except Exception as e:
print(f"Failed to install Playwright browsers: {e}", file=sys.stderr)
sys.exit(1)
async def scrape_festival_stick_example():
"""
Navigate to the online FastLED tool and capture a screenshot of the FestivalStick example
"""
install_playwright_browsers()
# Online FastLED tool URL
fastled_url = "https://fastled.onrender.com/docs"
async with async_playwright() as p:
# Launch browser with a visible window for debugging
browser = await p.chromium.launch(headless=False, slow_mo=1000)
page = await browser.new_page()
# Set viewport size for consistent screenshots
await page.set_viewport_size({"width": 1920, "height": 1080})
try:
print(f"Navigating to {fastled_url}...")
await page.goto(fastled_url, timeout=30000)
# Wait for the page to load
await page.wait_for_load_state("networkidle")
# Look for FastLED examples or upload functionality
print("Looking for FastLED example functionality...")
# Wait a bit for dynamic content to load
await page.wait_for_timeout(3000)
# Try to find any example selection or file upload elements
examples_selector = None
possible_selectors = [
"select[name*='example']",
"select[id*='example']",
".example-selector",
"input[type='file']",
"button:has-text('Example')",
"button:has-text('FestivalStick')",
"a:has-text('Example')",
"a:has-text('FestivalStick')",
]
for selector in possible_selectors:
try:
element = await page.wait_for_selector(selector, timeout=2000)
if element:
print(f"Found element with selector: {selector}")
examples_selector = selector
break
except Exception:
continue
if not examples_selector:
# If no specific example selector found, look for text content
print("Looking for FestivalStick text on page...")
try:
await page.wait_for_selector("text=FestivalStick", timeout=5000)
print("Found FestivalStick text on page!")
except Exception:
print(
"FestivalStick text not found, taking screenshot of current page..."
)
# Try to interact with the FastLED tool interface
print("Attempting to interact with FastLED interface...")
# Look for common web interface elements
interface_elements = [
"canvas",
".led-display",
".visualization",
"#fastled-canvas",
".fastled-viewer",
]
canvas_found = False
canvas = None
for element_selector in interface_elements:
try:
canvas_element = await page.wait_for_selector(
element_selector, timeout=2000
)
if canvas_element:
print(f"Found display element: {element_selector}")
canvas_found = True
canvas = canvas_element
break
except Exception:
continue
if not canvas_found:
print("No specific display canvas found, capturing full page...")
# If there's a file upload option, try to upload the FestivalStick example
try:
file_input = await page.query_selector("input[type='file']")
if file_input:
print("Found file input, attempting to upload FestivalStick.ino...")
festival_stick_path = (
PROJECT_ROOT
/ "examples"
/ "FestivalStick"
/ "FestivalStick.ino"
)
if festival_stick_path.exists():
await file_input.set_input_files(str(festival_stick_path))
await page.wait_for_timeout(3000) # Wait for upload to process
print("FestivalStick.ino uploaded successfully!")
else:
print(f"FestivalStick.ino not found at {festival_stick_path}")
except Exception as e:
print(f"Could not upload file: {e}")
# Wait for any animations or dynamic content to settle
await page.wait_for_timeout(5000)
# Take screenshot
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
screenshot_path = SCREENSHOTS_DIR / f"festival_stick_{timestamp}.png"
print(f"Taking screenshot and saving to {screenshot_path}...")
await page.screenshot(path=str(screenshot_path), full_page=True)
print(f"Screenshot saved successfully to {screenshot_path}")
# Also take a focused screenshot if we found a canvas
if canvas_found and canvas is not None:
try:
canvas_screenshot_path = (
SCREENSHOTS_DIR / f"festival_stick_canvas_{timestamp}.png"
)
await canvas.screenshot(path=str(canvas_screenshot_path))
print(f"Canvas screenshot saved to {canvas_screenshot_path}")
except Exception as e:
print(f"Could not take canvas screenshot: {e}")
# Keep browser open for a few seconds to see the result
print("Keeping browser open for 10 seconds for inspection...")
await page.wait_for_timeout(10000)
except Exception as e:
print(f"An error occurred during scraping: {e}", file=sys.stderr)
# Take an error screenshot for debugging
error_screenshot_path = (
SCREENSHOTS_DIR
/ f"error_{datetime.now().strftime('%Y%m%d_%H%M%S')}.png"
)
try:
await page.screenshot(path=str(error_screenshot_path), full_page=True)
print(f"Error screenshot saved to {error_screenshot_path}")
except Exception:
pass
raise e
finally:
await browser.close()
async def main():
"""Main function to run the scraping operation"""
try:
await scrape_festival_stick_example()
print("FastLED FestivalStick scraping completed successfully!")
except Exception as e:
print(f"Scraping failed: {e}", file=sys.stderr)
return 1
return 0
if __name__ == "__main__":
exit_code = asyncio.run(main())
sys.exit(exit_code)