platformio.ini advanced build configurations are not yet compatible with PlatformIO/SCons flag workflows #34

Activity

1. zackees commented on Apr 13, 2026

   

   zackees

   on Apr 13, 2026

   MemberAuthor

   More actions

   Investigation split into child issues:

   • Fail fast on unsupported advanced PlatformIO build options in platformio.ini #35 Fail fast on unsupported advanced PlatformIO build options in platformio.ini
   • Add a native compatibility shim for build_type=debug and debug_build_flags #36 Add a native compatibility shim for build_type=debug and debug_build_flags
   • Apply PlatformIO build_unflags to the effective native flag scopes #37 Apply PlatformIO build_unflags to the effective native flag scopes
   • Define native support boundaries for PlatformIO extra_scripts env/projenv mutations #38 Define native support boundaries for PlatformIO extra_scripts env/projenv mutations
   • Research whether PlatformIO __idedata / project metadata can support a scripted-project bridge #39 Research whether PlatformIO __idedata / project metadata can support a scripted-project bridge

   Current read from the research:

   • Declarative features (build_type=debug, debug_build_flags, build_unflags) look like a good fit for native compatibility work.
   • extra_scripts is the real boundary. Once arbitrary Python mutates SCons environments, full parity likely means either a very constrained shim or explicit fallback to --platformio.
   • PlatformIO's metadata dump looks useful for editor/intellisense parity, but probably not rich enough by itself for full build/link parity.

   That points toward option (2) unless the spike in #39 turns up something unexpectedly stronger.
2. zackees commented on Apr 13, 2026

   

   zackees

   on Apr 13, 2026

   MemberAuthor

   More actions

   Additional child issue from deeper investigation: #40 tracks the mismatch between declared-supported PLATFORMIO_* overrides and the config getters that actually consume them.
3. zackees commented on Apr 13, 2026

   

   zackees

   on Apr 13, 2026

   MemberAuthor

   More actions

   Added #41 for the concrete runtime-bridge experiment: execute PlatformIO/SCons just far enough to dump resolved build state, then feed that into native fbuild.
4. zackees commented on Apr 13, 2026

   

   zackees

   on Apr 13, 2026

   MemberAuthor

   More actions

   Runtime-bridge analysis

   I think the promising path is not a parser, but a runtime bridge:

   1. run PlatformIO/SCons far enough for PRE scripts, platform/framework scripts, library logic, and POST scripts to mutate the construction environments
   2. inject one final dump step
   3. serialize the resolved build state into a machine-readable spec
   4. let fbuild perform the real native compile/link/deploy from that spec

   Why this looks feasible

   • PlatformIO already executes PRE scripts, the main build script, and POST scripts before the integration-dump path: .cache/platformio-core/platformio/builder/main.py:167-180, .cache/platformio-core/platformio/builder/main.py:214-227
   • PlatformIO?s docs say POST scripts can access both env and projenv, which is exactly the point where project-vs-global flag mutations have settled: https://docs.platformio.org/en/latest/scripting/construction_environments.html
   • PlatformIO already serializes part of this state today (build_type, defines, includes, cc_flags, cxx_flags, tool paths, targets, flash image metadata): .cache/platformio-core/platformio/builder/tools/piointegration.py:141-173
   • The normal build path already resolves board flags, build flags, framework scripts, debug behavior, and build_unflags in a well-defined order: .cache/platformio-core/platformio/builder/tools/piobuild.py:94-128

   What this should NOT be

   • not a static Python/SCons parser
   • not a giant reimplementation of SCons semantics inside Rust
   • not __idedata as-is, because some scripts may branch on integration/service stages and that dump is editor-oriented, not build-replay-oriented

   What the prototype should dump

   • global env scopes: CPPDEFINES, CPPPATH, CCFLAGS, CFLAGS, CXXFLAGS, ASFLAGS, LINKFLAGS, LIBPATH, LIBS
   • projenv scopes separately
   • tool paths (CC, CXX, AR, linker, objcopy, gdb when present)
   • BUILD_TYPE, linker script info, flash image metadata
   • selected source/object graph (PIOBUILDFILES and related nodes if available)
   • ideally exact compile and link argv after SCons expansion

   Decision impact

   • If this dump can capture exact compile/link commands or a sufficiently rich resolved build spec, then many ?simple SCons flag files? are realistically bridgeable without full SCons hosting.
   • If not, then the right boundary is still native declarative support plus explicit --platformio fallback for scripted projects.

   That makes #41 the key experiment before deciding whether the long-term answer is a stronger version of option (2) or a real option (3).
5. zackees commented on Apr 13, 2026

   

   zackees

   on Apr 13, 2026

   MemberAuthor

   More actions

   PR #42 is up with the native extra_scripts runtime bridge implementation. It covers the constrained runtime path for common env / projenv flag mutations, applies the resulting overlay to compile/link phases, and fails fast on unsupported scripted behavior instead of silently drifting.
6. zackees commented on Apr 13, 2026

   

   zackees

   on Apr 13, 2026

   MemberAuthor

   More actions

   Merged in PR #42: the implementation follows the expanded (2) path discussed here. fbuild now has a constrained extra_scripts runtime bridge that executes simple PlatformIO-style script mutations, captures the resulting compile/link overlays, applies them to the native build, and fails fast for unsupported dynamic behavior instead of silently drifting. This keeps native builds compatible with common SCons flag workflows without committing the project to full SCons re-hosting.

   Key boundary: supported script behavior is limited to straightforward flag/path/library mutations on common scopes (CPPDEFINES, CPPPATH, CCFLAGS, CFLAGS, CXXFLAGS, ASFLAGS, LINKFLAGS, LIBPATH, LIBS) plus env / post-projenv access. More dynamic script behavior should still fall back to --platformio.
7. zackees commented on Apr 17, 2026

   

   zackees

   on Apr 17, 2026

   MemberAuthor

   More actions

   Status — closing umbrella; remaining scope is independently tracked

   The original five asks in this issue are mostly delivered. The remaining surface lives in the two independent child issues #38 and #39, each actionable on its own without needing this umbrella to coordinate them.

   Delivered

   1. Parse and honor build_type, debug_build_flags, build_unflags from platformio.ini — ✅ done in crates/fbuild-build/src/pipeline.rs (BuildContext::new reads all three; build_type = debug triggers apply_debug_build_type; build_unflags is applied across user / src / overlay link scopes)
   2. Map PlatformIO build types to fbuild profiles — ✅ debug path is wired via apply_debug_build_type; release falls back to the existing BuildProfile::Release. A future full mapping (including test) can be filed separately if needed.
   3. Apply build_unflags against framework/toolchain/user flag sets — ✅ closed by Apply PlatformIO build_unflags to the effective native flag scopes #37 (PRs feat(build): apply build_unflags to framework compile flags, ESP32 wired (#37) #63 and feat(build): wire build_unflags through all remaining platform compilers (#37) #64, across all platforms)
   4. Detect extra_scripts and handle them — ✅ partially done (script_runtime::resolve_extra_script_overlay in crates/fbuild-build/src/script_runtime.rs runs scripts via a Python harness); the remaining design work on what script patterns to support is tracked in Define native support boundaries for PlatformIO extra_scripts env/projenv mutations #38
   5. Add tests — ✅ crates/fbuild-build/src/pipeline.rs and crates/fbuild-config/src/ini_parser.rs cover build_type, debug_build_flags, build_unflags; extra_scripts cases covered in script_runtime.rs tests

   Remaining active scope

   • Define native support boundaries for PlatformIO extra_scripts env/projenv mutations #38 — Define the native support boundary for extra_scripts env/projenv mutations (what to support, what to reject, how to diagnose)
   • Research whether PlatformIO __idedata / project metadata can support a scripted-project bridge #39 — Research whether PlatformIO __idedata / project metadata can back a scripted-project bridge

   Both are independent and actionable without needing this umbrella as a coordination point. Closing.
8. zackees

   closed this as completedon Apr 17, 2026