Real-world platformio.ini sample reveals remaining native compatibility gaps #43

Summary

Sampled 25 public GitHub repos containing platformio.ini from GitHub code search on 2026-04-13.

Classification rule used for this sample:

• analyze each repo's default_envs
• apply basic [env] inheritance
• resolve simple ${section.key} references
• classify against current native fbuild support, not against --platformio

Result in this sample:

• 8/25 supported by the current native path
• 17/25 not supported by the current native path

This issue is meant to track real repo instances from the wild, not abstract feature ideas.

What The Sample Suggests

The current runtime bridge is already enough for a meaningful subset of normal projects, especially when extra_scripts only mutate ordinary flag/path/library state.

The remaining misses cluster into a few recurring buckets:

• Source filtering: src_filter, build_src_filter
• Dynamic script methods outside the shim: Dump, Execute, BoardConfig, PioPlatform, GetProjectConfig, GetBuildType, env.get, etc.
• Build middleware / custom SCons logic: AddBuildMiddleware, SConscript, custom methods, custom builders/targets
• Config-level gaps: build_unflags, build_type=debug
• Out-of-scope targets: custom platform URLs, non-native host/tool platforms

That means the current bottleneck is no longer basic platformio.ini parsing. It is the boundary between:

• declarative config
• simple script-time env/projenv mutations
• fully dynamic SCons behavior

Highest-Value Follow-Ups From This Sample

1. Add native support for source filtering.
2. Add native support for build_unflags.
3. Add native support for build_type=debug / debug_build_flags.
4. Decide whether a few additional script APIs are worth shimming:
   • IsCleanTarget
   • IsIntegrationDump
   • PioPlatform
   • read-only BoardConfig
   • limited env.get(...)
5. Keep hard-failing on heavy SCons behavior:
   • Execute
   • SConscript
   • build middleware
   • custom builder / target logic

Supported Repos In This Sample

These repos appear compatible with the current native support boundary for their default environments:

• MaslowCNC/Firmware
• dmlambo/SplitFlap
• SergiuToporjinschi/espmanager
• TimoKropp/OPENCHESSBOARD_WiFi
• lora-gateway/esxp1302
• andreisperid/E-TKT
• Aask42/IoB_DC28
• HazeCore/Nametag_Firmware

Unsupported Repos By Main Blocker

Source Filtering

• fbiego/esp32-c3-mini
  feature set: board_build.partitions, build_flags, build_src_filter, extends, extra_scripts, lib_deps
• W0rthlessS0ul/nRF24_jammer
  feature set: build_src_filter, extra_scripts
• darkspr1te/stm32f107vc_sd_bootloader
  feature set: build_flags, extra_scripts, src_filter
• sarfata/kbox-firmware
  feature set: build_flags, extra_scripts, lib_deps, lib_ignore, src_filter
• kinsamanka/openplc-stm32-freertos
  feature set: board_build.ldscript, build_flags, build_src_filter, build_src_flags, extra_scripts
• bigtreetech/BIGTREETECH-SKR-E3-Turbo
  feature set: build_flags, extends, extra_scripts, lib_deps, src_filter
• jmamma/MCL
  feature set: board_build.core, board_build.f_cpu, build_flags, build_src_filter, build_unflags, extends, extra_scripts, lib_deps, lib_ignore

Config-Level Gaps

• jmamma/MCL
  also uses build_unflags
• DasBasti/IndiaNavi_Firmware
  feature set: board_build.embed_txtfiles, board_build.partitions, build_flags, build_type
  unsupported surface: build_type=debug

Dynamic Script APIs Outside The Current Shim

• esprfid/esp-rfid
  feature set: board_build.f_cpu, board_build.flash_mode, build_flags, extra_scripts, lib_deps
  unsupported surface: env.Dump()
• DutchDevelop/BLLEDController
  feature set: board_build.filesystem, build_flags, extra_scripts, lib_deps
  unsupported surface: BoardConfig, Execute, Flatten, get
• UtilitechAS/amsreader-firmware
  feature set: board_build.ldscript, build_flags, extra_scripts, lib_deps, lib_ignore
  unsupported surface: Execute, VerboseAction
• xperiments-in/xtouch
  feature set: board_build.filesystem, board_build.partitions, build_flags, extra_scripts, lib_deps
  unsupported surface: IsCleanTarget, IsIntegrationDump
• nettigo/namf
  feature set: board_build.f_cpu, board_build.f_flash, board_build.flash_mode, build_flags, extra_scripts, lib_deps
  unsupported surface: env.Dump(), projenv.Dump()
• S4N-T0S/Open-Source-LDAT
  feature set: build_flags, extra_scripts, lib_deps
  unsupported surface: PioPlatform()
• cyrilpawelko/m5panel
  feature set: board_build.partitions, build_flags, extra_scripts, lib_deps
  unsupported surface: env.get(), rewriting MKSPIFFSTOOL
• gicking/stm8gal
  feature set: build_flags, extra_scripts
  unsupported surface: script rewrites PROGNAME

Heavy SCons / Middleware / Custom Builder Logic

• Andy-Big/Marlin_FB_Reborn_Old
  feature set: build_flags, extra_scripts, lib_deps
  unsupported surface: custom methods, GetProjectConfig, GetBuildType, ParseFlagsExtended, SConscript, dynamic dependency scripts
• kinsamanka/openplc-stm32-freertos
  also uses AddBuildMiddleware, BoardConfig, PioPlatform

Out-Of-Scope Platform / Tool Targets

• gicking/stm8gal
  default env uses linux_x86_64
• bigtreetech/BIGTREETECH-SKR-E3-Turbo
  default env uses custom platform URL https://github.com/p3p/pio-nxplpc-arduino-lpc176x/archive/0.1.3.zip
• jmamma/MCL
  one default env uses custom platform https://github.com/maxgerhardt/platform-raspberrypi.git

Full Sample Matrix

Supported

• MaslowCNC/Firmware
  default envs: megaatmega2560
  classification: supported
• dmlambo/SplitFlap
  default envs: d1_mini
  classification: supported
• SergiuToporjinschi/espmanager
  default envs: esp01_1m
  classification: supported
• TimoKropp/OPENCHESSBOARD_WiFi
  default envs: arduino_nano_esp32
  classification: supported
• lora-gateway/esxp1302
  default envs: esp32dev
  classification: supported
• andreisperid/E-TKT
  default envs: serial-upload
  classification: supported
• Aask42/IoB_DC28
  default envs: internet-of-batteriesv1
  classification: supported
• HazeCore/Nametag_Firmware
  default envs: serial
  classification: supported

Not Supported

• esprfid/esp-rfid
  default envs: generic
  feature set: board_build.f_cpu, board_build.flash_mode, build_flags, extra_scripts, lib_deps
  unsupported surface: [generic] extra_script scripts/GENdeploy.py: env.Dump()
• fbiego/esp32-c3-mini
  default envs: elecrow_c3_1_28
  feature set: board_build.partitions, build_flags, build_src_filter, extends, extra_scripts, lib_deps
  unsupported surface: [elecrow_c3_1_28] uses source filtering
• DutchDevelop/BLLEDController
  default envs: esp32dev
  feature set: board_build.filesystem, build_flags, extra_scripts, lib_deps
  unsupported surface: [esp32dev] extra_script merge_firmware.py: env.BoardConfig(), env.Execute(), env.Flatten(), env.get()
• UtilitechAS/amsreader-firmware
  default envs: esp8266
  feature set: board_build.ldscript, build_flags, extra_scripts, lib_deps, lib_ignore
  unsupported surface: [esp8266] extra_script ...generate_includes.py: env.Execute(), env.VerboseAction()
• xperiments-in/xtouch
  default envs: esp32dev
  feature set: board_build.filesystem, board_build.partitions, build_flags, extra_scripts, lib_deps
  unsupported surface: [esp32dev] extra_script scripts/pre-build.py: env.IsCleanTarget(), env.IsIntegrationDump()
• W0rthlessS0ul/nRF24_jammer
  default envs: 128x32_Flexible
  feature set: build_src_filter, extra_scripts
  unsupported surface: [128x32_Flexible] uses source filtering
• jmamma/MCL
  default envs: megacommand, megacmd, tbd
  feature set: board_build.core, board_build.f_cpu, build_flags, build_src_filter, build_unflags, extends, extra_scripts, lib_deps, lib_ignore
  unsupported surface: build_unflags, source filtering, Execute, Exit, env.get, custom platform URL env
• nettigo/namf
  default envs: lang_pl, lang_en, lang_ro, lang_hu, lang_en_32
  feature set: board_build.f_cpu, board_build.f_flash, board_build.flash_mode, build_flags, extra_scripts, lib_deps
  unsupported surface: env.Dump(), projenv.Dump()
• gicking/stm8gal
  default envs: linux_x86_64
  feature set: build_flags, extra_scripts
  unsupported surface: unsupported platform linux_x86_64, script rewrites PROGNAME
• darkspr1te/stm32f107vc_sd_bootloader
  default envs: mkstft_28_bl
  feature set: build_flags, extra_scripts, src_filter
  unsupported surface: src_filter
• sarfata/kbox-firmware
  default envs: host
  feature set: build_flags, extra_scripts, lib_deps, lib_ignore, src_filter
  unsupported surface: source filtering, projenv.Append on unsupported non-flag key
• Andy-Big/Marlin_FB_Reborn_Old
  default envs: mks_robin_nano_v1_3_f4_x32, mks_robin_nano35_x16, mks_robin_nano_Sv1_3_f4_x16, mks_robin_nano_SMv1_3_f4_x16
  feature set: build_flags, extra_scripts, lib_deps
  unsupported surface: heavy Marlin SCons script stack
• S4N-T0S/Open-Source-LDAT
  default envs: teensy41
  feature set: build_flags, extra_scripts, lib_deps
  unsupported surface: env.PioPlatform()
• DasBasti/IndiaNavi_Firmware
  default envs: esp32dev_debug
  feature set: board_build.embed_txtfiles, board_build.partitions, build_flags, build_type
  unsupported surface: build_type=debug
• cyrilpawelko/m5panel
  default envs: m5stack-fire
  feature set: board_build.partitions, build_flags, extra_scripts, lib_deps
  unsupported surface: env.get(), rewriting MKSPIFFSTOOL
• kinsamanka/openplc-stm32-freertos
  default envs: bluepill
  feature set: board_build.ldscript, build_flags, build_src_filter, build_src_flags, extra_scripts
  unsupported surface: source filtering, build middleware, BoardConfig, PioPlatform
• bigtreetech/BIGTREETECH-SKR-E3-Turbo
  default envs: LPC1769
  feature set: build_flags, extends, extra_scripts, lib_deps, src_filter
  unsupported surface: custom platform URL, source filtering, heavy pre-script logic

Notes

• Version-pinned platform strings like espressif32@6.9.0 were treated as the same native platform family when the underlying platform is already supported by fbuild.
• This classification is intentionally based on the native fbuild path. Any of these projects may still be buildable through the --platformio fallback path.
• This issue should stay focused on feature-family coverage and real repo evidence rather than trying to promise support for every project-specific script.

Action List

TDD RED -> GREEN Execution Plan

• Source filtering
  • RED: add parser tests for build_src_filter / src_filter resolution
  • RED: add scanner tests for +<...> / -<...> matching on sketch sources
  • GREEN: implement config getter and filter parsing
  • GREEN: apply filter to sketch source discovery in native builds
  • GREEN: rerun targeted tests and record the passing set
• build_unflags
  • RED: add tests covering flag removal from compile and link overlays
  • GREEN: implement unflag removal in native flag resolution
  • GREEN: rerun targeted tests and record the passing set
• build_type=debug / debug_build_flags
  • RED: add config + effective-flag tests for debug envs
  • GREEN: implement native debug-mode flag resolution
  • GREEN: rerun targeted tests and record the passing set
• Small script API shims to evaluate individually
  • RED: add focused harness tests for IsCleanTarget
  • RED: add focused harness tests for IsIntegrationDump
  • RED: add focused harness tests for read-only BoardConfig
  • RED: add focused harness tests for limited env.get(...)
  • GREEN: implement only the APIs that stay read-only / deterministic
• Remain explicit non-goals for the native path
  • keep hard-failing on Execute
  • keep hard-failing on SConscript
  • keep hard-failing on build middleware / custom builders / custom targets

Repo-Driven Success Criteria

• Move at least one source-filter repo from the unsupported sample into the supported bucket
• Re-run the sample classification after each feature-family lands
• Keep the issue updated with exact repo instances unlocked by each change

Progress

• build_src_filter / src_filter native support is now implemented as the first TDD slice.

• RED was confirmed with missing get_source_filter and missing filtered scanner APIs.

• GREEN was confirmed with targeted tests:

  • cargo test -p fbuild-config --lib test_get_source_filter -- --nocapture
  • cargo test -p fbuild-build --lib test_scan_sketch_sources_filtered -- --nocapture

• On this workstation, test execution required pinning the MSVC toolchain directly because external cargo/rustc shims on PATH were mixing GNU and MSVC artifacts. The code itself is green under the correct toolchain.

• build_unflags is the next TDD slice in progress.

• Plan: add failing config/build tests first, implement native unflag removal, rerun targeted tests to green, then move to build_type=debug / debug_build_flags.

• build_unflags is now implemented natively for the common subset.

• RED was confirmed with missing get_build_unflags and missing native unflag application.

• GREEN was confirmed with targeted tests:

  • cargo test -p fbuild-config --lib test_get_build_unflags -- --nocapture
  • cargo test -p fbuild-build --lib test_apply_build_unflags -- --nocapture

• Current behavior removes exact tokens plus common option/value pairs from both build_flags and build_src_flags, which matches the highest-value native subset.

• Next slice in progress: build_type=debug and debug_build_flags.

• build_type=debug and debug_build_flags are now supported for the common native subset.

• RED was confirmed with missing get_build_type, missing get_debug_build_flags, and missing native debug-mode application.

• GREEN was confirmed with targeted tests:

  • cargo test -p fbuild-config --lib test_get_build_type_defaults_to_release -- --nocapture
  • cargo test -p fbuild-config --lib test_get_build_type_reads_debug -- --nocapture
  • cargo test -p fbuild-config --lib test_get_debug_build_flags_uses_platformio_defaults -- --nocapture
  • cargo test -p fbuild-config --lib test_get_debug_build_flags_reads_ini_override -- --nocapture
  • cargo test -p fbuild-build --lib test_apply_debug_build_type_replaces_opt_flags_and_adds_debug_define -- --nocapture
  • cargo test -p fbuild-build --lib test_debug_mode_then_build_unflags_can_remove_debug_flags_again -- --nocapture

• Current behavior adds -D__PLATFORMIO_BUILD_DEBUG__, applies default PlatformIO debug flags when unspecified (-Og -g2 -ggdb2), cleans conflicting optimization/debug levels in the native flag vectors, and still allows build_unflags to remove those debug flags afterward.

• Added a small extra_scripts shim set for common no-op/introspection helpers: Dump, IsCleanTarget, IsIntegrationDump, Flatten, VerboseAction, and Execute.

• RED was confirmed with runtime tests failing on unsupported env.Dump and env.IsCleanTarget calls.

• GREEN was confirmed with targeted tests:

  • cargo test -p fbuild-build --lib test_resolve_extra_script_overlay_supports_dump_shim -- --nocapture
  • cargo test -p fbuild-build --lib test_resolve_extra_script_overlay_supports_common_noop_scons_helpers -- --nocapture

• This does not widen native support to arbitrary script execution; it only removes a handful of low-risk blockers that commonly appear in simple PlatformIO scripts.

• Added native BoardConfig() and PioPlatform() shims to the extra-scripts runtime for common metadata lookups.

• GREEN was confirmed with targeted tests:

  • cargo test -p fbuild-build --lib test_resolve_extra_script_overlay_supports_board_config_shim -- --nocapture
  • cargo test -p fbuild-build --lib test_resolve_extra_script_overlay_supports_pio_platform_shim -- --nocapture

• PioPlatform() remains intentionally minimal: platform name, is_embedded(), and get_package_dir(...). More dynamic platform/package behaviors still fall back to --platformio.