1172 lines
39 KiB
Markdown
1172 lines
39 KiB
Markdown
# RMT5 Worker Pool Implementation
|
|
|
|
## Problem Statement
|
|
|
|
ESP32 RMT5 has hardware limitations on the number of LED strips it can control simultaneously:
|
|
- **ESP32**: 8 RMT channels maximum
|
|
- **ESP32-S2/S3**: 4 RMT channels maximum
|
|
- **ESP32-C3/H2**: 2 RMT channels maximum
|
|
|
|
Currently, FastLED's RMT5 implementation creates a one-to-one mapping between `RmtController5` instances and RMT hardware channels. This means you can only control K strips simultaneously, where K is the hardware limit.
|
|
|
|
**Goal**: Implement a worker pool system to support N strips where N > K, allowing any reasonable number of LED strips to be controlled by recycling RMT workers.
|
|
|
|
## Current RMT5 Architecture
|
|
|
|
### Class Hierarchy
|
|
```cpp
|
|
ClocklessController<PIN, T1, T2, T3, RGB_ORDER>
|
|
└── RmtController5 (mRMTController)
|
|
└── IRmtStrip* (mLedStrip)
|
|
└── RmtStrip (concrete implementation)
|
|
└── led_strip_handle_t (mStrip) // ESP-IDF handle
|
|
└── rmt_channel_handle_t // Hardware channel
|
|
```
|
|
|
|
### Current Flow
|
|
1. `RmtController5::loadPixelData()` creates `IRmtStrip` on first call
|
|
2. `IRmtStrip::Create()` calls `led_strip_new_rmt_device()` which allocates a hardware RMT channel
|
|
3. Channel remains allocated until `RmtStrip` destructor calls `led_strip_del()`
|
|
4. No sharing or pooling exists
|
|
|
|
## RMT4 Worker Pool Reference
|
|
|
|
RMT4 implements a sophisticated worker pool system:
|
|
|
|
### Key Components
|
|
- **`gControllers[FASTLED_RMT_MAX_CONTROLLERS]`**: Array of all registered controllers
|
|
- **`gOnChannel[FASTLED_RMT_MAX_CHANNELS]`**: Currently active controllers per channel
|
|
- **Global counters**: `gNumStarted`, `gNumDone`, `gNext` for coordination
|
|
- **Semaphore coordination**: `gTX_sem` for synchronization
|
|
|
|
### RMT4 Worker Pool Flow
|
|
1. All controllers register in `gControllers[]` during construction
|
|
2. `showPixels()` triggers batch processing when `gNumStarted == gNumControllers`
|
|
3. First K controllers start immediately on available channels
|
|
4. Remaining controllers queue until channels become available
|
|
5. `doneOnChannel()` callback releases channels and starts next queued controller
|
|
6. Process continues until all controllers complete
|
|
|
|
## Proposed RMT5 Worker Pool Architecture
|
|
|
|
### Core Design Principles
|
|
|
|
1. **Backward Compatibility**: Existing `RmtController5` API remains unchanged
|
|
2. **Transparent Pooling**: Controllers don't know they're sharing workers
|
|
3. **Automatic Resource Management**: Workers handle setup/teardown automatically
|
|
4. **Thread Safety**: Pool operations are atomic and interrupt-safe
|
|
|
|
### New Components
|
|
|
|
#### 1. RmtWorkerPool (Singleton)
|
|
```cpp
|
|
class RmtWorkerPool {
|
|
public:
|
|
static RmtWorkerPool& getInstance();
|
|
|
|
// Worker management
|
|
RmtWorker* acquireWorker(const RmtWorkerConfig& config);
|
|
void releaseWorker(RmtWorker* worker);
|
|
|
|
// Coordination
|
|
void registerController(RmtController5* controller);
|
|
void unregisterController(RmtController5* controller);
|
|
void executeDrawCycle();
|
|
|
|
private:
|
|
fl::vector<RmtWorker*> mAvailableWorkers;
|
|
fl::vector<RmtWorker*> mBusyWorkers;
|
|
fl::vector<RmtController5*> mRegisteredControllers;
|
|
|
|
// Synchronization
|
|
SemaphoreHandle_t mPoolMutex;
|
|
SemaphoreHandle_t mDrawSemaphore;
|
|
|
|
// State tracking
|
|
int mActiveDrawCount;
|
|
int mCompletedDrawCount;
|
|
};
|
|
```
|
|
|
|
#### 2. RmtWorker (Replaceable RMT Resource)
|
|
```cpp
|
|
class RmtWorker {
|
|
public:
|
|
// Configuration for worker setup
|
|
struct Config {
|
|
int pin;
|
|
uint32_t ledCount;
|
|
bool isRgbw;
|
|
uint32_t t0h, t0l, t1h, t1l, reset;
|
|
IRmtStrip::DmaMode dmaMode;
|
|
};
|
|
|
|
// Worker lifecycle
|
|
bool configure(const Config& config);
|
|
void loadPixelData(fl::PixelIterator& pixels);
|
|
void startTransmission();
|
|
void waitForCompletion();
|
|
|
|
// State management
|
|
bool isAvailable() const;
|
|
bool isConfiguredFor(const Config& config) const;
|
|
void reset();
|
|
|
|
// Callbacks
|
|
void onTransmissionComplete();
|
|
|
|
private:
|
|
IRmtStrip* mCurrentStrip;
|
|
Config mCurrentConfig;
|
|
bool mIsAvailable;
|
|
bool mTransmissionActive;
|
|
RmtWorkerPool* mPool; // Back reference for release
|
|
};
|
|
```
|
|
|
|
#### 3. Modified RmtController5
|
|
```cpp
|
|
class RmtController5 {
|
|
public:
|
|
// Existing API unchanged
|
|
RmtController5(int DATA_PIN, int T1, int T2, int T3, DmaMode dma_mode);
|
|
void loadPixelData(PixelIterator &pixels);
|
|
void showPixels();
|
|
|
|
private:
|
|
// New pooled implementation
|
|
RmtWorkerConfig mWorkerConfig;
|
|
fl::vector<uint8_t> mPixelBuffer; // Persistent buffer
|
|
bool mRegisteredWithPool;
|
|
|
|
// Remove direct IRmtStrip ownership
|
|
// IRmtStrip *mLedStrip = nullptr; // REMOVED
|
|
};
|
|
```
|
|
|
|
### Worker Pool Operation Flow
|
|
|
|
#### Registration Phase (Constructor)
|
|
```cpp
|
|
RmtController5::RmtController5(int DATA_PIN, int T1, int T2, int T3, DmaMode dma_mode)
|
|
: mPin(DATA_PIN), mT1(T1), mT2(T2), mT3(T3), mDmaMode(dma_mode) {
|
|
|
|
// Configure worker requirements
|
|
mWorkerConfig = {DATA_PIN, 0, false, t0h, t0l, t1h, t1l, 280, convertDmaMode(dma_mode)};
|
|
|
|
// Register with pool
|
|
RmtWorkerPool::getInstance().registerController(this);
|
|
mRegisteredWithPool = true;
|
|
}
|
|
```
|
|
|
|
#### Data Loading Phase
|
|
```cpp
|
|
void RmtController5::loadPixelData(PixelIterator &pixels) {
|
|
// Update worker config with actual pixel count
|
|
mWorkerConfig.ledCount = pixels.size();
|
|
mWorkerConfig.isRgbw = pixels.get_rgbw().active();
|
|
|
|
// Store pixel data in persistent buffer
|
|
storePixelData(pixels);
|
|
}
|
|
|
|
void RmtController5::storePixelData(PixelIterator &pixels) {
|
|
const int bytesPerPixel = mWorkerConfig.isRgbw ? 4 : 3;
|
|
const int bufferSize = mWorkerConfig.ledCount * bytesPerPixel;
|
|
|
|
mPixelBuffer.resize(bufferSize);
|
|
|
|
// Copy pixel data to persistent buffer
|
|
uint8_t* bufPtr = mPixelBuffer.data();
|
|
if (mWorkerConfig.isRgbw) {
|
|
while (pixels.has(1)) {
|
|
uint8_t r, g, b, w;
|
|
pixels.loadAndScaleRGBW(&r, &g, &b, &w);
|
|
*bufPtr++ = r; *bufPtr++ = g; *bufPtr++ = b; *bufPtr++ = w;
|
|
pixels.advanceData();
|
|
pixels.stepDithering();
|
|
}
|
|
} else {
|
|
while (pixels.has(1)) {
|
|
uint8_t r, g, b;
|
|
pixels.loadAndScaleRGB(&r, &g, &b);
|
|
*bufPtr++ = r; *bufPtr++ = g; *bufPtr++ = b;
|
|
pixels.advanceData();
|
|
pixels.stepDithering();
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
#### Execution Phase (Coordinated Draw)
|
|
```cpp
|
|
void RmtController5::showPixels() {
|
|
// Trigger coordinated draw cycle
|
|
RmtWorkerPool::getInstance().executeDrawCycle();
|
|
}
|
|
|
|
void RmtWorkerPool::executeDrawCycle() {
|
|
// Similar to RMT4 coordination logic
|
|
mActiveDrawCount = 0;
|
|
mCompletedDrawCount = 0;
|
|
|
|
// Take draw semaphore
|
|
xSemaphoreTake(mDrawSemaphore, portMAX_DELAY);
|
|
|
|
// Start as many controllers as we have workers
|
|
int startedCount = 0;
|
|
for (auto* controller : mRegisteredControllers) {
|
|
if (startedCount < mAvailableWorkers.size()) {
|
|
startController(controller);
|
|
startedCount++;
|
|
}
|
|
}
|
|
|
|
// Wait for all controllers to complete
|
|
while (mCompletedDrawCount < mRegisteredControllers.size()) {
|
|
xSemaphoreTake(mDrawSemaphore, portMAX_DELAY);
|
|
|
|
// Start next queued controller if workers available
|
|
startNextQueuedController();
|
|
|
|
xSemaphoreGive(mDrawSemaphore);
|
|
}
|
|
}
|
|
|
|
void RmtWorkerPool::startController(RmtController5* controller) {
|
|
// Acquire worker from pool
|
|
RmtWorker* worker = acquireWorker(controller->getWorkerConfig());
|
|
if (!worker) {
|
|
// This should not happen if pool is sized correctly
|
|
return;
|
|
}
|
|
|
|
// Configure worker for this controller
|
|
worker->configure(controller->getWorkerConfig());
|
|
|
|
// Load pixel data from controller's persistent buffer
|
|
loadPixelDataToWorker(worker, controller);
|
|
|
|
// Start transmission
|
|
worker->startTransmission();
|
|
|
|
mActiveDrawCount++;
|
|
}
|
|
|
|
void RmtWorkerPool::onWorkerComplete(RmtWorker* worker) {
|
|
// Called from worker's completion callback
|
|
mCompletedDrawCount++;
|
|
|
|
// Release worker back to pool
|
|
releaseWorker(worker);
|
|
|
|
// Signal main thread
|
|
xSemaphoreGive(mDrawSemaphore);
|
|
}
|
|
```
|
|
|
|
### Worker Reconfiguration Strategy
|
|
|
|
#### Efficient Worker Reuse
|
|
```cpp
|
|
bool RmtWorker::configure(const Config& newConfig) {
|
|
// Check if reconfiguration is needed
|
|
if (isConfiguredFor(newConfig)) {
|
|
return true; // Already configured correctly
|
|
}
|
|
|
|
// Tear down current configuration
|
|
if (mCurrentStrip) {
|
|
// Wait for any pending transmission
|
|
if (mTransmissionActive) {
|
|
mCurrentStrip->waitDone();
|
|
}
|
|
|
|
// Clean shutdown
|
|
delete mCurrentStrip;
|
|
mCurrentStrip = nullptr;
|
|
}
|
|
|
|
// Create new strip with new configuration
|
|
mCurrentStrip = IRmtStrip::Create(
|
|
newConfig.pin, newConfig.ledCount, newConfig.isRgbw,
|
|
newConfig.t0h, newConfig.t0l, newConfig.t1h, newConfig.t1l,
|
|
newConfig.reset, newConfig.dmaMode
|
|
);
|
|
|
|
if (!mCurrentStrip) {
|
|
return false;
|
|
}
|
|
|
|
mCurrentConfig = newConfig;
|
|
return true;
|
|
}
|
|
```
|
|
|
|
#### Pin State Management
|
|
```cpp
|
|
void RmtWorker::reset() {
|
|
if (mCurrentStrip) {
|
|
// Ensure transmission is complete
|
|
if (mTransmissionActive) {
|
|
mCurrentStrip->waitDone();
|
|
}
|
|
|
|
// Set pin to safe state before teardown
|
|
gpio_set_level((gpio_num_t)mCurrentConfig.pin, 0);
|
|
gpio_set_direction((gpio_num_t)mCurrentConfig.pin, GPIO_MODE_OUTPUT);
|
|
|
|
// Clean up strip
|
|
delete mCurrentStrip;
|
|
mCurrentStrip = nullptr;
|
|
}
|
|
|
|
mTransmissionActive = false;
|
|
mIsAvailable = true;
|
|
}
|
|
```
|
|
|
|
### Memory Management Strategy
|
|
|
|
#### Persistent Pixel Buffers
|
|
Each `RmtController5` maintains its own pixel buffer to avoid data races:
|
|
|
|
```cpp
|
|
class RmtController5 {
|
|
private:
|
|
fl::vector<uint8_t> mPixelBuffer; // Persistent storage
|
|
RmtWorkerConfig mWorkerConfig; // Configuration cache
|
|
|
|
void storePixelData(PixelIterator& pixels);
|
|
void restorePixelData(RmtWorker* worker);
|
|
};
|
|
```
|
|
|
|
#### Worker Pool Sizing
|
|
```cpp
|
|
void RmtWorkerPool::initialize() {
|
|
// Determine hardware channel count
|
|
int maxChannels = getHardwareChannelCount();
|
|
|
|
// Create one worker per hardware channel
|
|
mAvailableWorkers.reserve(maxChannels);
|
|
for (int i = 0; i < maxChannels; i++) {
|
|
mAvailableWorkers.push_back(new RmtWorker());
|
|
}
|
|
}
|
|
|
|
int RmtWorkerPool::getHardwareChannelCount() {
|
|
#if CONFIG_IDF_TARGET_ESP32
|
|
return 8;
|
|
#elif CONFIG_IDF_TARGET_ESP32S2 || CONFIG_IDF_TARGET_ESP32S3
|
|
return 4;
|
|
#elif CONFIG_IDF_TARGET_ESP32C3 || CONFIG_IDF_TARGET_ESP32H2
|
|
return 2;
|
|
#else
|
|
return 2; // Conservative default
|
|
#endif
|
|
}
|
|
```
|
|
|
|
## Key Implementation Insights
|
|
|
|
### Critical Async Decision Point
|
|
The worker pool must make a **per-controller decision** at `showPixels()` time:
|
|
|
|
```cpp
|
|
void RmtController5::showPixels() {
|
|
RmtWorkerPool& pool = RmtWorkerPool::getInstance();
|
|
|
|
// CRITICAL: This decision determines async vs blocking behavior
|
|
if (pool.hasAvailableWorker()) {
|
|
// ASYNC PATH: Start immediately and return
|
|
pool.startControllerImmediate(this);
|
|
return; // Returns immediately - preserves async!
|
|
} else {
|
|
// BLOCKING PATH: This specific controller must wait
|
|
pool.startControllerQueued(this); // May block with polling
|
|
}
|
|
}
|
|
```
|
|
|
|
### ESP-IDF RMT Channel Management
|
|
Direct integration with ESP-IDF RMT5 APIs instead of using led_strip wrapper:
|
|
|
|
```cpp
|
|
class RmtWorker {
|
|
private:
|
|
rmt_channel_handle_t mChannel;
|
|
rmt_encoder_handle_t mEncoder;
|
|
|
|
public:
|
|
// Direct RMT channel creation for maximum control
|
|
bool createChannel(int pin) {
|
|
rmt_tx_channel_config_t config = {
|
|
.gpio_num = pin,
|
|
.clk_src = RMT_CLK_SRC_DEFAULT,
|
|
.resolution_hz = 10000000,
|
|
.mem_block_symbols = 64,
|
|
.trans_queue_depth = 1, // Single transmission per worker
|
|
};
|
|
|
|
ESP_ERROR_CHECK(rmt_new_tx_channel(&config, &mChannel));
|
|
|
|
// Register callback for async completion
|
|
rmt_tx_event_callbacks_t callbacks = {
|
|
.on_trans_done = onTransComplete,
|
|
};
|
|
ESP_ERROR_CHECK(rmt_tx_register_event_callbacks(mChannel, &callbacks, this));
|
|
|
|
return true;
|
|
}
|
|
|
|
void transmitAsync(uint8_t* pixelData, size_t dataSize) {
|
|
// Direct transmission - bypasses led_strip wrapper
|
|
ESP_ERROR_CHECK(rmt_enable(mChannel));
|
|
ESP_ERROR_CHECK(rmt_transmit(mChannel, mEncoder, pixelData, dataSize, &mTxConfig));
|
|
// Returns immediately - async transmission started
|
|
}
|
|
};
|
|
```
|
|
|
|
### Polling Strategy Implementation
|
|
Use `delayMicroseconds(100)` only for queued controllers:
|
|
|
|
```cpp
|
|
void RmtWorkerPool::startControllerQueued(RmtController5* controller) {
|
|
// Add to queue
|
|
mQueuedControllers.push_back(controller);
|
|
|
|
// Poll until this controller gets a worker
|
|
while (true) {
|
|
if (RmtWorker* worker = tryAcquireWorker()) {
|
|
// Remove from queue and start
|
|
mQueuedControllers.remove(controller);
|
|
startControllerImmediate(controller, worker);
|
|
break;
|
|
}
|
|
|
|
// Brief delay to prevent busy-wait
|
|
delayMicroseconds(100);
|
|
|
|
// Yield periodically for FreeRTOS
|
|
static uint32_t pollCount = 0;
|
|
if (++pollCount % 50 == 0) {
|
|
yield();
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
## Implementation Plan
|
|
|
|
### Phase 1: Core Infrastructure
|
|
1. **Create `RmtWorkerPool` singleton**
|
|
- Basic worker management (acquire/release)
|
|
- Controller registration system
|
|
- Thread-safe operations with mutexes
|
|
|
|
2. **Implement `RmtWorker` class**
|
|
- Worker lifecycle management
|
|
- Configuration and reconfiguration logic
|
|
- Completion callbacks
|
|
|
|
3. **Modify `RmtController5`**
|
|
- Add persistent pixel buffer
|
|
- Integrate with worker pool
|
|
- Maintain backward-compatible API
|
|
|
|
### Phase 2: Coordination Logic
|
|
1. **Implement coordinated draw cycle**
|
|
- Batch processing similar to RMT4
|
|
- Semaphore-based synchronization
|
|
- Queue management for excess controllers
|
|
|
|
2. **Add worker completion handling**
|
|
- Async completion callbacks
|
|
- Automatic worker recycling
|
|
- Next controller startup
|
|
|
|
### Phase 3: Optimization & Safety
|
|
1. **Optimize worker reconfiguration**
|
|
- Minimize teardown/setup when possible
|
|
- Cache compatible configurations
|
|
- Efficient pin state management
|
|
|
|
2. **Add error handling**
|
|
- Worker allocation failures
|
|
- Transmission errors
|
|
- Recovery mechanisms
|
|
|
|
3. **Memory optimization**
|
|
- Minimize buffer copying
|
|
- Efficient pixel data transfer
|
|
- Memory pool for workers
|
|
|
|
### Phase 4: Testing & Integration
|
|
1. **Unit tests**
|
|
- Worker pool operations
|
|
- Controller coordination
|
|
- Error scenarios
|
|
|
|
2. **Integration testing**
|
|
- Multiple strip configurations
|
|
- High load scenarios
|
|
- Hardware limit validation
|
|
|
|
3. **Performance benchmarking**
|
|
- Throughput comparison with RMT4
|
|
- Memory usage analysis
|
|
- Latency measurements
|
|
|
|
## CRITICAL: Async Behavior Preservation
|
|
|
|
**Key Requirement**: RMT5 currently provides async drawing where `endShowLeds()` returns immediately without waiting. This must be preserved when N ≤ K, and only use polling/waiting when N > K.
|
|
|
|
### Current RMT5 Async Flow
|
|
```cpp
|
|
// Current behavior - MUST PRESERVE when N ≤ K
|
|
void ClocklessController::endShowLeds(void *data) {
|
|
CPixelLEDController<RGB_ORDER>::endShowLeds(data);
|
|
mRMTController.showPixels(); // Calls drawAsync() - returns immediately!
|
|
}
|
|
```
|
|
|
|
### Async Strategy for Worker Pool
|
|
|
|
#### When N ≤ K (Preserve Full Async)
|
|
- **Direct Assignment**: Each controller gets dedicated worker immediately
|
|
- **No Waiting**: `endShowLeds()` returns immediately after starting transmission
|
|
- **Callback-Driven**: Use ESP-IDF `rmt_tx_event_callbacks_t::on_trans_done` for completion
|
|
- **Zero Overhead**: Maintain current performance characteristics
|
|
|
|
#### When N > K (Controlled Polling)
|
|
- **Immediate Start**: First K controllers start immediately (async)
|
|
- **Queue Remaining**: Controllers K+1 through N queue for workers
|
|
- **Polling Strategy**: Use `delayMicroseconds(100)` polling for queued controllers
|
|
- **Callback Coordination**: Workers signal completion via callbacks to start next queued controller
|
|
|
|
## ESP-IDF RMT5 Callback Integration
|
|
|
|
### Callback Registration Pattern
|
|
```cpp
|
|
class RmtWorker {
|
|
private:
|
|
rmt_channel_handle_t mRmtChannel;
|
|
RmtWorkerPool* mPool;
|
|
|
|
static bool IRAM_ATTR onTransmissionComplete(
|
|
rmt_channel_handle_t channel,
|
|
const rmt_tx_done_event_data_t *edata,
|
|
void *user_data) {
|
|
|
|
RmtWorker* worker = static_cast<RmtWorker*>(user_data);
|
|
worker->handleTransmissionComplete();
|
|
return false; // No high-priority task woken
|
|
}
|
|
|
|
public:
|
|
bool initialize() {
|
|
// Create RMT channel
|
|
rmt_tx_channel_config_t tx_config = {
|
|
.gpio_num = mPin,
|
|
.clk_src = RMT_CLK_SRC_DEFAULT,
|
|
.resolution_hz = 10000000, // 10MHz
|
|
.mem_block_symbols = 64,
|
|
.trans_queue_depth = 4,
|
|
};
|
|
|
|
if (rmt_new_tx_channel(&tx_config, &mRmtChannel) != ESP_OK) {
|
|
return false;
|
|
}
|
|
|
|
// Register completion callback
|
|
rmt_tx_event_callbacks_t callbacks = {
|
|
.on_trans_done = onTransmissionComplete,
|
|
};
|
|
|
|
return rmt_tx_register_event_callbacks(mRmtChannel, &callbacks, this) == ESP_OK;
|
|
}
|
|
|
|
void handleTransmissionComplete() {
|
|
// Signal pool that this worker is available
|
|
mPool->onWorkerComplete(this);
|
|
}
|
|
};
|
|
```
|
|
|
|
## Revised Worker Pool Architecture
|
|
|
|
### Async-Aware Worker Pool
|
|
```cpp
|
|
class RmtWorkerPool {
|
|
public:
|
|
enum class DrawMode {
|
|
ASYNC_ONLY, // N ≤ K: All controllers async
|
|
MIXED_MODE // N > K: Some async, some polled
|
|
};
|
|
|
|
void executeDrawCycle() {
|
|
const int numControllers = mRegisteredControllers.size();
|
|
const int numWorkers = mAvailableWorkers.size();
|
|
|
|
if (numControllers <= numWorkers) {
|
|
// ASYNC_ONLY mode - preserve full async behavior
|
|
executeAsyncOnlyMode();
|
|
} else {
|
|
// MIXED_MODE - async for first K, polling for rest
|
|
executeMixedMode();
|
|
}
|
|
}
|
|
|
|
private:
|
|
void executeAsyncOnlyMode() {
|
|
// Start all controllers immediately - full async behavior preserved
|
|
for (auto* controller : mRegisteredControllers) {
|
|
RmtWorker* worker = acquireWorker(controller->getWorkerConfig());
|
|
startControllerAsync(controller, worker);
|
|
}
|
|
// Return immediately - no waiting!
|
|
}
|
|
|
|
void executeMixedMode() {
|
|
// Start first K controllers immediately (async)
|
|
int startedCount = 0;
|
|
for (auto* controller : mRegisteredControllers) {
|
|
if (startedCount < mAvailableWorkers.size()) {
|
|
RmtWorker* worker = acquireWorker(controller->getWorkerConfig());
|
|
startControllerAsync(controller, worker);
|
|
startedCount++;
|
|
} else {
|
|
// Queue remaining controllers
|
|
mQueuedControllers.push_back(controller);
|
|
}
|
|
}
|
|
|
|
// Poll for completion of queued controllers
|
|
while (!mQueuedControllers.empty()) {
|
|
delayMicroseconds(100); // Non-blocking poll interval
|
|
// Callback-driven worker completion will process queue
|
|
}
|
|
}
|
|
|
|
void onWorkerComplete(RmtWorker* worker) {
|
|
// Called from ISR context via callback
|
|
releaseWorker(worker);
|
|
|
|
// Start next queued controller if available
|
|
if (!mQueuedControllers.empty()) {
|
|
RmtController5* nextController = mQueuedControllers.front();
|
|
mQueuedControllers.pop_front();
|
|
|
|
// Reconfigure worker and start transmission
|
|
startControllerAsync(nextController, worker);
|
|
}
|
|
}
|
|
};
|
|
```
|
|
|
|
### Modified RmtController5 for Async Preservation
|
|
```cpp
|
|
class RmtController5 {
|
|
public:
|
|
void showPixels() {
|
|
// This method MUST return immediately when N ≤ K
|
|
// Only block when this specific controller is queued (N > K)
|
|
|
|
RmtWorkerPool& pool = RmtWorkerPool::getInstance();
|
|
|
|
if (pool.canStartImmediately(this)) {
|
|
// Async path - return immediately
|
|
pool.startControllerImmediate(this);
|
|
} else {
|
|
// This controller is queued - must wait for worker
|
|
pool.startControllerQueued(this);
|
|
}
|
|
}
|
|
};
|
|
```
|
|
|
|
## Polling Strategy Details
|
|
|
|
### Microsecond Polling Pattern
|
|
```cpp
|
|
void RmtWorkerPool::waitForQueuedControllers() {
|
|
while (!mQueuedControllers.empty()) {
|
|
// Non-blocking check for available workers
|
|
if (hasAvailableWorker()) {
|
|
processNextQueuedController();
|
|
} else {
|
|
// Short delay to prevent busy-waiting
|
|
delayMicroseconds(100); // 100μs polling interval
|
|
}
|
|
|
|
// Yield to other tasks periodically
|
|
static uint32_t pollCount = 0;
|
|
if (++pollCount % 50 == 0) { // Every 5ms (50 * 100μs)
|
|
yield();
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
### Callback-Driven Queue Processing
|
|
```cpp
|
|
void RmtWorker::handleTransmissionComplete() {
|
|
// Called from ISR context - keep minimal
|
|
BaseType_t xHigherPriorityTaskWoken = pdFALSE;
|
|
|
|
// Signal completion to pool
|
|
xSemaphoreGiveFromISR(mPool->getCompletionSemaphore(), &xHigherPriorityTaskWoken);
|
|
|
|
portYIELD_FROM_ISR(xHigherPriorityTaskWoken);
|
|
}
|
|
|
|
void RmtWorkerPool::processCompletionEvents() {
|
|
// Called from main task context
|
|
while (xSemaphoreTake(mCompletionSemaphore, 0) == pdTRUE) {
|
|
// Process one completion event
|
|
if (!mQueuedControllers.empty()) {
|
|
RmtController5* nextController = mQueuedControllers.front();
|
|
mQueuedControllers.pop_front();
|
|
|
|
// Find available worker and start next transmission
|
|
RmtWorker* worker = findAvailableWorker();
|
|
if (worker) {
|
|
startControllerAsync(nextController, worker);
|
|
}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
## Benefits
|
|
|
|
1. **Async Preservation**: Full async behavior maintained when N ≤ K
|
|
2. **Scalability**: Support unlimited LED strips (within memory constraints)
|
|
3. **Backward Compatibility**: Existing code works unchanged
|
|
4. **Resource Efficiency**: Optimal use of limited RMT hardware
|
|
5. **Controlled Blocking**: Only blocks when specific controller is queued
|
|
6. **Callback Efficiency**: ISR-driven completion for minimal latency
|
|
7. **Polling Optimization**: 100μs intervals prevent busy-waiting
|
|
|
|
## Considerations
|
|
|
|
1. **Memory Usage**: Each controller needs persistent pixel buffer
|
|
2. **Latency**: Worker switching adds small overhead
|
|
3. **Complexity**: More complex than direct mapping
|
|
4. **Debugging**: Pool coordination harder to debug than direct control
|
|
|
|
## Migration Path
|
|
|
|
The implementation maintains full backward compatibility. Existing code using `RmtController5` will automatically benefit from the worker pool without any changes required.
|
|
|
|
## CRITICAL: LED Buffer Transfer Analysis - CORRECTED FINDINGS
|
|
|
|
### ESP-IDF LED Buffer Management
|
|
|
|
**Key Finding**: The ESP-IDF led_strip driver **supports external pixel buffers** and **buffer transfer IS possible through RMT channel recreation**.
|
|
|
|
#### Current Buffer Architecture
|
|
```cpp
|
|
typedef struct {
|
|
led_strip_t base;
|
|
rmt_channel_handle_t rmt_chan;
|
|
rmt_encoder_handle_t strip_encoder;
|
|
uint8_t *pixel_buf; // ← Buffer pointer (fixed at creation)
|
|
bool pixel_buf_allocated_internally; // ← Ownership flag
|
|
// ... other fields
|
|
} led_strip_rmt_obj;
|
|
```
|
|
|
|
#### Buffer Creation Options
|
|
```cpp
|
|
led_strip_config_t strip_config = {
|
|
.strip_gpio_num = pin,
|
|
.max_leds = led_count,
|
|
.external_pixel_buf = external_buffer, // ← Can provide external buffer
|
|
// ... other config
|
|
};
|
|
```
|
|
|
|
### Buffer Transfer Solution - RMT Channel Recreation
|
|
|
|
**✅ POSSIBLE**: Buffer transfer through worker reconfiguration
|
|
- **Destroy existing RMT channel** when switching controllers
|
|
- **Create new RMT channel** with different external buffer
|
|
- **Worker pool manages appropriately-sized buffers** for each controller's requirements
|
|
- **RMT channels always use external buffers** owned by the worker pool
|
|
|
|
### Critical Insight: Buffer Size Requirements
|
|
|
|
**The Core Issue**: Different controllers need different buffer sizes:
|
|
- **Controller A**: 100 RGB LEDs → 300 bytes buffer
|
|
- **Controller B**: 200 RGBW LEDs → 800 bytes buffer
|
|
- **RMT Channel**: Must be recreated with appropriate buffer size for each controller
|
|
|
|
**ESP-IDF led_strip_rmt_obj Structure:**
|
|
```cpp
|
|
typedef struct {
|
|
led_strip_t base;
|
|
rmt_channel_handle_t rmt_chan;
|
|
rmt_encoder_handle_t strip_encoder;
|
|
uint32_t strip_len;
|
|
uint8_t bytes_per_pixel;
|
|
led_color_component_format_t component_fmt;
|
|
uint8_t *pixel_buf; // ← MUST be external and pool-managed
|
|
bool pixel_buf_allocated_internally; // ← Always false for worker pool
|
|
} led_strip_rmt_obj;
|
|
```
|
|
|
|
**Available Buffer APIs (Complete List):**
|
|
- `led_strip_set_pixel()` - Writes to existing buffer
|
|
- `led_strip_set_pixel_rgbw()` - Writes to existing buffer
|
|
- `led_strip_clear()` - Zeros existing buffer
|
|
- `led_strip_refresh_async()` - Transmits from existing buffer
|
|
- **KEY**: `led_strip_del()` does NOT free external buffers (pixel_buf_allocated_internally = false)
|
|
|
|
**Confirmed Solution**: Buffer transfer achieved through:
|
|
1. **Worker pool owns all RMT buffers**
|
|
2. **RMT channels destroyed/recreated** with appropriate buffer sizes
|
|
3. **External buffer management** prevents buffer deallocation during RMT destruction
|
|
|
|
### Buffer Transfer Solution: Worker Pool Buffer Management
|
|
|
|
**CORRECTED APPROACH**: Worker pool manages all RMT buffers and handles buffer sizing for different controllers.
|
|
|
|
#### Worker Pool Buffer Management Strategy
|
|
```cpp
|
|
class RmtWorkerPool {
|
|
private:
|
|
struct WorkerState {
|
|
IRmtStrip* strip;
|
|
uint8_t* worker_buffer; // Pool-owned buffer for this worker
|
|
size_t buffer_capacity; // Current buffer size
|
|
WorkerConfig current_config;
|
|
bool is_available;
|
|
bool transmission_active;
|
|
};
|
|
|
|
fl::vector<WorkerState> mWorkers;
|
|
|
|
// Buffer pool for different sizes
|
|
fl::map<size_t, fl::vector<uint8_t*>> mBuffersBySize;
|
|
|
|
public:
|
|
bool assignWorkerToController(RmtController5* controller) {
|
|
const WorkerConfig& config = controller->getWorkerConfig();
|
|
const size_t requiredBufferSize = config.led_count * (config.is_rgbw ? 4 : 3);
|
|
|
|
// Find available worker
|
|
WorkerState* worker = findAvailableWorker();
|
|
if (!worker) return false;
|
|
|
|
// Get appropriately sized buffer from pool
|
|
uint8_t* workerBuffer = acquireBuffer(requiredBufferSize);
|
|
if (!workerBuffer) return false;
|
|
|
|
// CRITICAL: Wait for any active transmission to complete
|
|
if (worker->strip && worker->transmission_active) {
|
|
worker->strip->waitDone();
|
|
worker->transmission_active = false;
|
|
}
|
|
|
|
// Destroy existing RMT channel if configuration changed
|
|
if (worker->strip && (!configCompatible(worker->current_config, config) ||
|
|
worker->buffer_capacity < requiredBufferSize)) {
|
|
delete worker->strip; // Destroy old RMT channel
|
|
worker->strip = nullptr;
|
|
|
|
// Release old buffer
|
|
releaseBuffer(worker->worker_buffer, worker->buffer_capacity);
|
|
}
|
|
|
|
// Copy controller's pixel data to worker's buffer
|
|
memcpy(workerBuffer, controller->getPixelBuffer(), controller->getBufferSize());
|
|
|
|
// Create new RMT channel with worker's external buffer
|
|
if (!worker->strip) {
|
|
worker->strip = IRmtStrip::CreateWithExternalBuffer(
|
|
config.pin, config.led_count, config.is_rgbw,
|
|
config.t0h, config.t0l, config.t1h, config.t1l, config.reset,
|
|
workerBuffer, // Worker's buffer, not controller's buffer
|
|
config.dma_mode
|
|
);
|
|
|
|
if (!worker->strip) {
|
|
releaseBuffer(workerBuffer, requiredBufferSize);
|
|
return false;
|
|
}
|
|
}
|
|
|
|
worker->worker_buffer = workerBuffer;
|
|
worker->buffer_capacity = requiredBufferSize;
|
|
worker->current_config = config;
|
|
worker->is_available = false;
|
|
|
|
// Start transmission
|
|
worker->strip->drawAsync();
|
|
worker->transmission_active = true;
|
|
return true;
|
|
}
|
|
|
|
void onWorkerComplete(WorkerState* worker) {
|
|
// Transmission complete - RMT hardware done with buffer
|
|
worker->transmission_active = false;
|
|
|
|
if (!mQueuedControllers.empty()) {
|
|
// Assign to next waiting controller
|
|
RmtController5* nextController = mQueuedControllers.front();
|
|
mQueuedControllers.pop_front();
|
|
|
|
const WorkerConfig& nextConfig = nextController->getWorkerConfig();
|
|
const size_t nextBufferSize = nextConfig.led_count * (nextConfig.is_rgbw ? 4 : 3);
|
|
|
|
if (worker->buffer_capacity >= nextBufferSize &&
|
|
configCompatible(worker->current_config, nextConfig)) {
|
|
|
|
// OPTIMIZATION: Reuse existing buffer and RMT channel
|
|
memcpy(worker->worker_buffer, nextController->getPixelBuffer(), nextController->getBufferSize());
|
|
worker->strip->drawAsync();
|
|
worker->transmission_active = true;
|
|
|
|
} else {
|
|
// RECONFIGURE: Need different buffer size or RMT configuration
|
|
|
|
// Release current buffer
|
|
releaseBuffer(worker->worker_buffer, worker->buffer_capacity);
|
|
|
|
// Destroy RMT channel
|
|
delete worker->strip;
|
|
worker->strip = nullptr;
|
|
|
|
// Get new appropriately-sized buffer
|
|
worker->worker_buffer = acquireBuffer(nextBufferSize);
|
|
if (!worker->worker_buffer) return; // Failed to get buffer
|
|
|
|
// Copy next controller's data
|
|
memcpy(worker->worker_buffer, nextController->getPixelBuffer(), nextController->getBufferSize());
|
|
|
|
// Create new RMT channel with new buffer
|
|
worker->strip = IRmtStrip::CreateWithExternalBuffer(
|
|
nextConfig.pin, nextConfig.led_count, nextConfig.is_rgbw,
|
|
nextConfig.t0h, nextConfig.t0l, nextConfig.t1h, nextConfig.t1l, nextConfig.reset,
|
|
worker->worker_buffer, // New worker buffer
|
|
nextConfig.dma_mode
|
|
);
|
|
|
|
worker->buffer_capacity = nextBufferSize;
|
|
worker->current_config = nextConfig;
|
|
|
|
// Start transmission
|
|
worker->strip->drawAsync();
|
|
worker->transmission_active = true;
|
|
}
|
|
} else {
|
|
// No waiting controllers - worker becomes available
|
|
worker->is_available = true;
|
|
// Keep buffer and RMT channel configured for potential reuse
|
|
}
|
|
}
|
|
|
|
private:
|
|
uint8_t* acquireBuffer(size_t size) {
|
|
// Round up to nearest power of 2 for efficient pooling
|
|
size_t poolSize = nextPowerOf2(size);
|
|
|
|
auto& buffers = mBuffersBySize[poolSize];
|
|
if (!buffers.empty()) {
|
|
uint8_t* buffer = buffers.back();
|
|
buffers.pop_back();
|
|
return buffer;
|
|
}
|
|
|
|
// Allocate new buffer
|
|
return (uint8_t*)malloc(poolSize);
|
|
}
|
|
|
|
void releaseBuffer(uint8_t* buffer, size_t size) {
|
|
size_t poolSize = nextPowerOf2(size);
|
|
mBuffersBySize[poolSize].push_back(buffer);
|
|
}
|
|
};
|
|
```
|
|
|
|
#### Simplified Controller Implementation
|
|
```cpp
|
|
class RmtController5 {
|
|
private:
|
|
fl::vector<uint8_t> mPixelBuffer; // Controller maintains its own data
|
|
WorkerConfig mWorkerConfig;
|
|
|
|
public:
|
|
void loadPixelData(PixelIterator& pixels) {
|
|
// Store pixel data in persistent buffer (unchanged)
|
|
const int bytesPerPixel = mWorkerConfig.is_rgbw ? 4 : 3;
|
|
const int bufferSize = pixels.size() * bytesPerPixel;
|
|
|
|
mPixelBuffer.resize(bufferSize);
|
|
|
|
// Load pixel data into our persistent buffer
|
|
uint8_t* bufPtr = mPixelBuffer.data();
|
|
if (mWorkerConfig.is_rgbw) {
|
|
while (pixels.has(1)) {
|
|
uint8_t r, g, b, w;
|
|
pixels.loadAndScaleRGBW(&r, &g, &b, &w);
|
|
*bufPtr++ = r; *bufPtr++ = g; *bufPtr++ = b; *bufPtr++ = w;
|
|
pixels.advanceData();
|
|
pixels.stepDithering();
|
|
}
|
|
} else {
|
|
while (pixels.has(1)) {
|
|
uint8_t r, g, b;
|
|
pixels.loadAndScaleRGB(&r, &g, &b);
|
|
*bufPtr++ = r; *bufPtr++ = g; *bufPtr++ = b;
|
|
pixels.advanceData();
|
|
pixels.stepDithering();
|
|
}
|
|
}
|
|
}
|
|
|
|
void showPixels() {
|
|
// Pool handles all buffer management internally
|
|
RmtWorkerPool::getInstance().assignWorkerToController(this);
|
|
}
|
|
|
|
// Provide buffer access for pool management
|
|
uint8_t* getPixelBuffer() { return mPixelBuffer.data(); }
|
|
size_t getBufferSize() const { return mPixelBuffer.size(); }
|
|
};
|
|
```
|
|
|
|
### Optimized Teardown Strategy
|
|
|
|
**Key Insight**: RMT strip teardown should be **conditional** based on worker pool demand:
|
|
|
|
#### When N ≤ K (No Queued Controllers)
|
|
```cpp
|
|
void onWorkerComplete(RmtWorker* worker) {
|
|
if (mQueuedControllers.empty()) {
|
|
// NO TEARDOWN - keep worker configured and ready
|
|
// Worker maintains its led_strip configuration
|
|
// Optimizes for next frame if same controller used again
|
|
releaseWorker(worker);
|
|
}
|
|
}
|
|
```
|
|
|
|
#### When N > K (Queued Controllers Waiting)
|
|
```cpp
|
|
void onWorkerComplete(RmtWorker* worker) {
|
|
if (!mQueuedControllers.empty()) {
|
|
// IMMEDIATE TEARDOWN AND RECONFIGURATION
|
|
// Next controller is waiting - reconfigure immediately
|
|
RmtController5* nextController = mQueuedControllers.front();
|
|
mQueuedControllers.pop_front();
|
|
|
|
// This triggers teardown in reconfigure()
|
|
startController(nextController, worker);
|
|
}
|
|
}
|
|
```
|
|
|
|
#### Buffer Change Requirement
|
|
**CRITICAL**: Even with identical pin/LED count/timing configuration, **teardown is always required** when switching between different controller buffers:
|
|
|
|
```cpp
|
|
// Controller A has buffer at 0x12345678
|
|
// Controller B has buffer at 0x87654321
|
|
// Even if both have same pin/count/timing:
|
|
// - led_strip object MUST be recreated to use new buffer pointer
|
|
// - ESP-IDF has no API to change pixel_buf after creation
|
|
```
|
|
|
|
### Buffer Transfer Implementation Details
|
|
|
|
#### Memory Safety
|
|
- **Controller Ownership**: Each `RmtController5` owns its persistent buffer
|
|
- **External Buffer Contract**: ESP-IDF won't free external buffers
|
|
- **Worker Lifecycle**: Workers destroy/recreate led_strip objects as needed
|
|
- **Buffer Validity**: Controllers must keep buffers valid during transmission
|
|
|
|
#### Performance Considerations
|
|
- **Reconfiguration Cost**: Creating new led_strip objects has overhead
|
|
- **Buffer Copying**: No copying needed - workers use external buffers directly
|
|
- **Memory Efficiency**: Only one buffer per controller (no duplication)
|
|
|
|
#### Error Handling
|
|
- **Reconfiguration Failures**: Handle led_strip creation failures gracefully
|
|
- **Buffer Size Mismatches**: Validate buffer sizes during reconfiguration
|
|
- **Transmission Errors**: Proper cleanup on transmission failures
|
|
|
|
### Buffer Transfer Summary - CORRECTED
|
|
|
|
**✅ SOLUTION**: Worker pool buffer management with RMT channel recreation
|
|
1. **Worker pool owns all RMT buffers** sized appropriately for each controller
|
|
2. **Controllers maintain their own pixel data** in persistent buffers
|
|
3. **Buffer copying required** from controller buffer to worker buffer
|
|
4. **RMT channels destroyed/recreated** with appropriately-sized external buffers
|
|
5. **ESP-IDF transmits from worker's external buffer** (not controller's buffer)
|
|
|
|
**✅ POSSIBLE**: Buffer transfer through worker reconfiguration
|
|
- RMT channels can be destroyed and recreated with different external buffers
|
|
- Worker pool manages buffer allocation and sizing
|
|
- External buffers are not freed by ESP-IDF when RMT channels are destroyed
|
|
|
|
**🔧 IMPLEMENTATION REQUIREMENTS**:
|
|
- Worker pool must manage buffers of different sizes
|
|
- RMT channel recreation required for buffer size changes
|
|
- Transmission completion synchronization before reconfiguration
|
|
- Buffer copying from controller to worker buffers
|
|
|
|
## CORRECTED CONCLUSIONS AND RECOMMENDATIONS
|
|
|
|
### Key Corrections to Original Analysis
|
|
|
|
1. **Buffer Transfer IS Possible**: The original "NOT POSSIBLE" assessment was incorrect. Buffer transfer can be achieved through RMT channel recreation with appropriately-sized external buffers.
|
|
|
|
2. **Worker Pool Must Manage Buffers**: The critical insight is that different controllers need different buffer sizes (RGB vs RGBW, different LED counts), so the worker pool must own and manage all RMT buffers.
|
|
|
|
3. **Buffer Copying Required**: Unlike the original zero-copy approach, buffer copying from controller to worker is necessary to handle different buffer size requirements.
|
|
|
|
4. **RMT Channel Recreation**: Workers must destroy and recreate RMT channels when switching between controllers with different requirements.
|
|
|
|
### Recommended Implementation Strategy
|
|
|
|
**✅ RECOMMENDED**: Worker Pool Buffer Management
|
|
- **Worker pool owns all RMT buffers** sized for different controller requirements
|
|
- **RMT channels use external buffers** managed by the worker pool
|
|
- **Buffer copying** from controller persistent buffers to worker buffers
|
|
- **RMT channel recreation** when buffer size or configuration changes
|
|
- **Transmission synchronization** to ensure safe reconfiguration
|
|
|
|
### Benefits of Corrected Approach
|
|
|
|
- ✅ **Supports Variable Buffer Sizes**: Handles different LED counts and RGB/RGBW modes
|
|
- ✅ **Proper Resource Management**: Worker pool manages buffer allocation/deallocation
|
|
- ✅ **Clean Separation**: Controllers focus on pixel data, workers handle hardware
|
|
- ✅ **Scalable Design**: Pool can optimize buffer reuse and minimize allocations
|
|
- ✅ **Thread Safe**: Proper synchronization prevents buffer access conflicts
|
|
|
|
### Performance Considerations
|
|
|
|
- ❌ **Buffer Copying Overhead**: Required due to different controller buffer sizes
|
|
- ✅ **Buffer Reuse Optimization**: Pool can reuse buffers for compatible configurations
|
|
- ✅ **Minimal RMT Recreation**: Only when buffer size or configuration changes
|
|
- ✅ **Efficient Memory Usage**: Pool manages buffer sizes optimally
|
|
|
|
### Implementation Priority
|
|
|
|
1. **Phase 1**: Implement basic worker pool with buffer management
|
|
2. **Phase 2**: Add buffer size optimization and reuse logic
|
|
3. **Phase 3**: Implement transmission synchronization and callbacks
|
|
4. **Phase 4**: Add performance optimizations and error handling
|
|
|
|
## Future Enhancements
|
|
|
|
1. **Priority System**: Allow high-priority strips to get workers first
|
|
2. **Smart Batching**: Group compatible strips to minimize reconfiguration
|
|
3. **Dynamic Scaling**: Adjust worker count based on usage patterns
|
|
4. **Metrics**: Add performance monitoring and statistics
|
|
5. **Buffer Pool Optimization**: Advanced buffer size prediction and caching |