Repository navigation
Description
Board / MCU
ESP32-S3
LED Chipset
WS2812B / NeoPixel
FastLED Version
master#27a1583
Arduino / PlatformIO version
PlatformIO 6.1.19
Operating System
Windows
Description
Per the close message in #2420 asking to open a new issue if #2420 was not correctly resolved, I am opening this issue to confirm that #2420 was not correctly resolved and that the timing drift first reported in #2420 is very much still present as of master#27a1583.
Per the request in the closure message asking for serial data timing messages, I have provided the timing messages confirming that the drift is not microseconds as suggested but is rather 63~300 milliseconds per sequence iteration.
Please additionally note that the serial output with the exact same minimal sketch on 3.10.3 looks like this:
2495
2495
2495
2495
2495
2495
2495
2495
2495
2495
Considering the only difference between the data posted below and the data posted above is FastLED 3.10.3 versus master#27a1583, this strongly suggests that the underlying timing issue reported in #2420 is as yet unresolved, and is indeed caused by FastLED itself rather than any bug in the program, the millisDelay library, or the ESP-IDF framework.
Steps to reproduce
- Target FastLED
master#27a1583. - Compile minimal sketch.
- See that timing precision is significantly more unreliable than
3.10.3.
Minimal sketch
#include <Arduino.h>
#include <millisDelay.h>
#include <FastLED.h>
uint32_t i_iteration_time = 0;
uint8_t i_post_fade = 255;
CRGB leds[35];
#define LED_PIN 4
#define FAST_LED_UPDATE_MS 5
#define NUM_LEDS 35 // 35-LED NeoPixel ring
uint8_t i_fast_led_delay = FAST_LED_UPDATE_MS;
millisDelay ms_fast_led;
millisDelay ms_delay_post;
void setup() {
FastLED.addLeds<NEOPIXEL, LED_PIN>(leds, NUM_LEDS).setCorrection(TypicalLEDStrip);
FastLED.setMaxRefreshRate(800);
FastLED.show();
#ifdef ESP32
setCpuFrequencyMhz(160);
Serial.begin(115200);
#endif
ms_delay_post.start(0);
ms_fast_led.start(i_fast_led_delay);
}
void updateLEDs() {
if(ms_fast_led.justFinished()) {
FastLED.show();
ms_fast_led.start(i_fast_led_delay);
}
}
void loop() {
if(i_post_fade > 0 && ms_delay_post.justFinished()) {
const uint8_t i_ring_start = 0;
const uint8_t i_ring_end = 34;
const uint8_t i_ring_divisor = 7;
uint8_t i_ring_counter = 0;
i_ring_counter = ((255 - i_post_fade) / i_ring_divisor) + i_ring_start;
if(i_ring_counter <= i_ring_end) {
leds[i_ring_counter] = CRGB::Red;
}
if(i_ring_counter - 1 >= i_ring_start && i_ring_counter - 1 <= i_ring_end) {
leds[i_ring_counter - 1] = CRGB::Black;
}
i_post_fade--;
if(i_post_fade == 0) {
Serial.println(millis() - i_iteration_time);
i_iteration_time = millis();
ms_delay_post.stop();
for(uint8_t i = i_ring_start; i <= i_ring_end; i++) {
leds[i] = CRGB::Black;
}
updateLEDs();
delay(1000);
i_post_fade = 255;
ms_delay_post.start(225);
return;
}
else {
ms_delay_post.start(5);
}
}
updateLEDs();
}Serial / debug output
2563
2654
2752
2738
2752
2750
2579
2563
2564
2563
2564
2563
2564
2563
2564
2563
2564
2563
2564
2563
2564
2563
2564
2563
2592
2638
2566
2745
2582
2575
2750
2747
2564
2563
2564
2563
2564
2563
2564
2563
2564
2563
2564
2563
2564
2563
2564
2563
2564
2563
2564
2563
2564
2563
2564
2563
2564
2563
2564
2563
2564
2563
2564
2563
2564
2563
2564
2563
2564
2563
2564
2563
2564
2563
2564
2563
2564
2563
2564
2563Checklist
- I have searched existing issues for duplicates
- I have tested with the latest FastLED release or git master
- I can reproduce this with a minimal sketch
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fieldsTriage
Activity
zackees commentedon Jun 9, 2026
tapped out of credits until thursday.
feat(autoresearch): wire timingDriftTest RPC for #2994 repro (#3210)
zackees commentedon Jul 29, 2026
Root-caused. This is not microsecond-level drift — it is FreeRTOS tick quantization inside
FastLED.show()'s refresh-rate throttle, and it explains both the magnitude and why 3.10.3 was rock-steady.The regression
git log -Lon the throttle loop insrc/FastLED.cpp.hppgives the exact 3.10.3 → master delta, commit5c8f7a1e7f("try and fix wifi slowdown", refs #2193):3.10.3 spun in a microsecond-accurate busy-wait. Master sleeps.
Why the sleep is far longer than 250 µs
fl::task::run(250, SYSTEM)—src/fl/task/executor.cpp.hpp:135-141. Becausemicroseconds > 0, it callspumpCoroutines(1000). The caller's 250 µs is discarded; 1000 is hardcoded.pumpCoroutineson ESP32 —src/platforms/esp/32/coroutine_esp32.impl.hpp:82-85. It converts µs→ticks and then forces a floor:Your sketch sets
setMaxRefreshRate(800), somNMinMicros= 1250 µs. The throttle needs to wait ~1.25 ms but can only wake on 1 ms tick boundaries — so eachshow()overshoots by a variable 0–1 ms, and it never averages out because it re-quantizes every frame.The numbers match
Your sequence is 2495 ms with
show()every 5 ms ≈ 499 shows per sequence. At 1 ms tick granularity:Predicted 25–299 ms. You reported 63–300 ms. That is the whole effect.
It also explains the "compounded" framing from #2420 — it is not accumulating error, it is ~500 independent quantization events per sequence.
Why a plain revert is wrong
The yield was added deliberately: without it,
show()busy-waits with interrupts effectively starving lower-priority tasks, which caused the WiFi/websocket starvation in #2193 and #2254. Reverting reintroduces those.Suggested fix
Only take the deep yield when there is at least a full tick left to burn; spin the sub-tick remainder:
Long throttles (low
setMaxRefreshRate) keep yielding to WiFi/lwIP exactly as today; short ones stay microsecond-accurate like 3.10.3. Note that at any refresh rate above ~1000 Hz the entire wait is sub-tick, so today it is pure quantization noise.A second, independent bug worth fixing regardless:
executor.cpp.hpp:138ignores itsmicrosecondsargument and hardcodespumpCoroutines(1000). Every caller asking for a shorter yield silently gets 1000 µs (then rounded up to a tick).Caveats
Analysis is from source and git history — I do not have an ESP32-S3 on hand, so I have not measured this on your setup. Two things would confirm it in minutes:
FastLED.setMaxRefreshRate(0)to disable the throttle entirely. If timing snaps back to a steady 2495, this is confirmed.micros()immediately before and afterFastLED.show()— you should see it consistently overshoot to ~1 ms boundaries rather than ~1250 µs.Happy to put up the fix if the maintainers agree with the approach; it needs someone with hardware to validate that WiFi fairness is preserved.
zackees commentedon Jul 29, 2026
Filed #3762 with the full analysis and a proposed fix. Short version:
show()'s refresh throttle is the last call site still taking an unconditional deep yield — the same defect was already fixed in the channel wait loops by9d71de01d0(skip thevTaskDelay(1)floor) and #2817 (NetworkDetectorgating).show()was missed.#3762 also raises whether the yield is needed at all now that transposition is fast, and proposes an ESP32-C6 bench experiment to settle it by measurement rather than argument.
nomakewan commentedon Jul 29, 2026
Setting this did not resolve the issue. Please note that in the meantime between filing this issue and now, a new version of the ESP-IDF framework has been released. I have updated to this latest version, and updating did not have any effect whatsoever on the results. Please see the timing below with
FastLED.setMaxRefreshRate(0);set insetup().nomakewan commentedon Jul 29, 2026
I have tested commit 1e70d60 on real hardware and the problem not only persists, but the timing drift has become measurably worse. Please see the following serial output.
I request this issue be reopened as it has not been resolved.
zackees commentedon Jul 29, 2026
Retracting my earlier root-cause analysis, and reopening — this is not fixed.
I previously concluded the drift came from FreeRTOS tick quantization inside
CFastLED::show()'s refresh-rate throttle, and #3764 was merged on that basis, which auto-closed this issue. That conclusion was wrong.The throttle never runs in your sketch.
show()is gated byms_fast_ledat 5 ms. Subtracting the ~1050 us WS2812 transmit for 35 LEDs, the throttle sees ~3950 us elapsed againstmNMinMicros= 1250 us atsetMaxRefreshRate(800):So the code path #3764 changed is never reached by your sketch and cannot explain what you measured. #3764 is a genuine improvement for sketches that do hit the throttle, but it is not a fix for this.
How I got it wrong, since it is instructive: my supporting arithmetic (predicted 25-299 ms vs your reported 63-300 ms) came from choosing a per-frame jitter fraction freely across an order of magnitude. Three arbitrary multipliers will bracket almost any target. I treated that as confirmation when it was curve-fitting, and I never ran the one check that would have falsified it in seconds — does the throttle condition ever evaluate true here? — despite having your sketch the whole time. Apologies for the noise.
Where the evidence actually points. Your drift is 68-257 ms over 254 fade steps = 268-1012 us per step. A single WS2812
show()for 35 LEDs is ~1050 us — the same order of magnitude. That implicatesshow()'s duration, not throttle quantization: your loop pollsjustFinished()once per iteration, so a longershow()widens the gap between a 5 ms deadline passing and the poll observing it, and that excess compounds across 254 steps.The real question is therefore why
FastLED.show()got slower between 3.10.3 and master. Candidates, unranked and unmeasured: the RMT5/channels driver rewrite, per-frame transposition/encoding cost,EngineEventsbegin/end-frame listener work, or per-frame allocation/logging added since 3.10.3.Tracking the investigation in #3765.
Two things that would help enormously if you have a moment, both about 5 minutes:
Time
show()directly on both versions:uint32_t t0 = micros(); FastLED.show(); Serial.println(micros() - t0);If master reports meaningfully more than 3.10.3 for your 35-LED strip, that confirms the direction and gives us a number to bisect against.
Confirm the throttle is irrelevant by setting
FastLED.setMaxRefreshRate(0)on master. My analysis says your timing will be unchanged. If it does change, my retraction is itself wrong and I want to know.nomakewan commentedon Jul 29, 2026
The test I performed when requesting this issue be reopened was already tested using
FastLED.setMaxRefreshRate(0);insetup(). The numbers posted are therefore already relevant to your request and fulfill the needs of2.Here is the direct
show()timing you have requested for1using the code you provided:nomakewan commentedon Jul 29, 2026
Here are the numbers for
3.10.3using the same code:zackees commentedon Jul 29, 2026
Investigation state — everything needed to finish this
Consolidating here so this issue is the single home. The meta-tracker #3765 has been closed; its retraction, narrowing and tooling work is done and summarized below.
Where the cause is NOT
tests/profile/fastled_show.cpp(added in #3767) measures the CPU-side per-frame path at ~2 us/frame for a 35-LED strip. The drift reported here is 268-1012 us per fade step — three orders of magnitude larger. So the regression is not inEngineEvents, the three-pass controller walk inCFastLED::show(), dithering/scaling, or the #2428 channels dispatch.It is also not the refresh-rate throttle. That path provably never executes in this sketch:
show()is gated at 5 ms, and minus the ~1050 us WS2812 transmit the throttle sees ~3950 us against a 1250 us threshold, so its loop exits on the first check. (That was my earlier wrong root cause; #3764 is a real improvement to that path but does not affect this report.)Where the cause most likely IS
Everything the host benchmark structurally cannot see, because the host driver is a stub:
waitForConditiontiers,Channel::showPixels'swaitForReady)The per-step excess (268-1012 us) is the same order as one 35-LED WS2812 frame (~1050 us of wire time), which is consistent with
show()having become synchronous or gained a per-frame wait where 3.10.3 returned earlier.How to settle it — one command per version
ci/autoresearch/test_timing_drift.py(added in #3768) drives thetimingDriftTestRPC thatAutoResearchTimingDrift.hhas exposed all along. Note it had no host driver until now, which is why this reproducer had never actually been run.It reports two things:
show()duration — min/mean/max. The firmware already returnedshow_min_us/show_max_us/show_total_us; nothing consumed them.Decision rule: the WS2812 wire floor for 35 LEDs is 35 x 24 x 1.25 us = 1050 us. If
show()'s mean on master sits well above that floor and 3.10.3's does not, the excess is the regression, and the delta localizes it. Then bisect between the two tags on that number.Remaining work
Blocker on my side
I could not run it. The ESP32-C6 on this bench dropped off the USB bus during deploy attempts (VID_303A no longer enumerates) and needs a physical replug. The fbuild failures that preceded it — orphaned mkdir install-locks with no staleness timeout, a stale daemon endpoint surviving
daemon stop, and an esptool provisioning failure that killed the daemon — are filed as FastLED/fbuild#1213.@nomakewan — if you would rather not wait on our bench, the two checks in my previous comment are the fastest path: timing
show()directly on both versions, and confirmingsetMaxRefreshRate(0)changes nothing on master.fix(core): adaptive yield in show()'s refresh throttle (FastLED#3762, f…
fix(profile): repair `bash profile`, add a show() CPU-cost benchmark (F…
feat(autoresearch): host driver for the FastLED#2994 timing-drift rep…
zackees commentedon Jul 29, 2026
Reproduced on hardware, and the cost is localized
Board back on the bench. Ran
ci/autoresearch/test_timing_drift.py(#3768) against current master on an ESP32-C6 @ 160 MHz.The bug reproduces
35-LED geometry, 7 usable sequences:
Your report was +68 .. +257 ms. Same signature, so whatever this is, it is not specific to your board or sketch.
show() has a fixed ~1.84 ms per-frame cost
This is the finding. Varying strip length while measuring
show()min:Least-squares fit: 1843 us fixed + 5.29 us/LED.
Two things fall out of that:
Transmission is asynchronous. At 200 LEDs,
show()returns in 2902 us while the wire needs 6000 us. Doubling 35 -> 70 LEDs added only 166 us toshow()while wire time grew 1050 us. Soshow()is not waiting for the wire.There is a ~1.84 ms fixed cost per frame, independent of strip length. The 5.29 us/LED slope is plausible CPU encoding work. The intercept is not explained by anything in the frame's actual work.
For your 35-LED ring that fixed cost is 986 us of excess per show() — which lands squarely inside the 268-1012 us per fade step implied by your drift. That is the mechanism: ~1 ms of fixed overhead per
show(), compounded over 254 steps.At 200 LEDs the mean (4516 us) runs well above the min because wire time now exceeds the 5 ms call interval and
show()starts blocking on the previous frame. That is expected saturation, not the defect.What I have NOT established
The identity of the 1.84 ms. It is suspiciously close to two FreeRTOS ticks at
CONFIG_FREERTOS_HZ=1000, which would point at a tick-quantized wait in the driver path — the same class of mechanism I wrongly attributed to the refresh throttle earlier, but inwaitForCondition/driver teardown rather thanCFastLED::show()'s throttle.Important caveat before anyone acts on that hunch: the AutoResearch firmware links WiFi/BLE, so
NetworkDetector::isAnyNetworkActive()may be returning true here, which routeswaitForCondition's tier-3 yield to the deepvTaskDelay(1)path. Your sketch has no WiFi. If the fixed cost is WiFi-gated, my number would overstate your case — except that your measured drift matches it closely, which argues it is not gated. I am flagging this rather than asserting either way; it needs one more experiment to separate.Next
Bisect the fixed cost between 3.10.3 and master using the same harness, and instrument inside
show()to attribute the 1.84 ms. Tracking on #2994.Raw data and the harness are reproducible with:
zackees commentedon Aug 20, 2026
Burn-down disposition: DEFER ? not resolved, and not a bounded source-only fix.
Evidence reviewed:
show()every ~5 ms, so the 1.25 ms throttle condition is not reached. The reporter then confirmed the regression persisted withsetMaxRefreshRate(0).FastLED.show()around 2.63 ms for 35 WS2812 LEDs, versus a ~1.05 ms wire-time floor. That leaves a real on-device cost to localize.bash profile, add a show() CPU-cost benchmark (#3765) #3767, and feat(autoresearch): host driver for the #2994 timing-drift reproducer (#3765) #3768 fixed independent scheduler/profiling/tooling gaps. The host profile measured only the stub dispatch path and therefore cannot observe ESP32-S3 encoding, DMA, or driver wait time.Restart criteria:
ci.autoresearch.test_timing_driftdriver on FastLED 3.10.3 and currentmasterwith identical ESP-IDF/Arduino core, CPU frequency, wiring, and pixel count; preserveiter_msplusshow_min_us/show_max_us/show_total_us.show()duration delta, then bisect between 3.10.3 and the first bad commit.#3764 is therefore not a resolving PR for this issue; the issue remains open and deferred until the required S3 evidence is available.
nomakewan commentedon Oct 15, 2026
Since there have been several official releases since this was last touched, I wanted to check in to see if some random commit somewhere had corrected this issue. It has not.
I will start by applauding you for managing to get the compile sizes back down under control. Here are 3.10.7 versus 3.10.3 on the ESP32-S3, compiled with PlatformIO and the latest Pioarduino:
Below is the microsecond capture for the "corrected" version of 3.10.7 which actually compiles on ESP32-S3, commit d4a2d3a. Note also that this build is updated to the latest stable version of Pioarduino,
https://github.com/pioarduino/platform-espressif32/releases/download/55.03.312-1/platform-espressif32.zip.As you can see, while the timing has improved by about 300 microseconds from commit 1e70d60, it remains roughly 21x slower than 3.10.3 which seems to explain the timing drift observed in FastLED since that release.
nomakewan commentedon Nov 3, 2026
Since I see there have been two more official releases since my last post, I shall update this once again to show that 3.10.9 does not fix this issue either. It is still 21x slower than 3.10.3.