Skip to content

Prototype a PlatformIO/SCons state-dump bridge for scripted-project compatibility #41

Closed

Description

@zackees

Parent: #34

Problem

There may be a practical middle path between a small declarative shim and full SCons hosting: execute PlatformIO/SCons just far enough to resolve extra_scripts, framework/platform scripts, and build graph decisions, dump the resulting state, then let fbuild perform the actual native compile/link/deploy.

Why this is worth testing

  • PlatformIO already executes PRE scripts, the main platform build script, and POST scripts before the integration dump stage: .cache/platformio-core/platformio/builder/main.py:167-180, .cache/platformio-core/platformio/builder/main.py:214-227
  • PlatformIO exposes construction environments (env, projenv) and post-stage scripts run after the build environment is fully constructed: https://docs.platformio.org/en/latest/scripting/launch_types.html, https://docs.platformio.org/en/latest/scripting/construction_environments.html
  • Existing metadata export already proves some state can be serialized (build_type, defines, includes, compile flags, tool paths, targets, flash images): .cache/platformio-core/platformio/builder/tools/piointegration.py:141-173

Core hypothesis

Inject a final POST script (or equivalent harness) that runs after project scripts, serializes the resolved build state to JSON, and exits before actual compilation. fbuild then translates that JSON into a native build spec.

Important caveat

The existing __idedata / IsIntegrationDump() path is probably not sufficient by itself because PlatformIO documents that extra scripts are re-executed during the integration stage and scripts may intentionally return early when env.IsIntegrationDump() is true.

Prototype tasks

  1. Create a minimal injected POST script that runs after project scripts
  2. Dump at least:
    • BUILD_TYPE
    • tool paths (CC, CXX, AR, linker, objcopy when present)
    • env / projenv scopes: CPPDEFINES, CPPPATH, CCFLAGS, CFLAGS, CXXFLAGS, ASFLAGS, LINKFLAGS, LIBPATH, LIBS
    • resolved source/object nodes (PIOBUILDFILES if available)
    • program path / linker script / extra flash image info
  3. Verify whether this dump reflects common extra_scripts mutations on:
    • env.Append(...)
    • projenv.Append(...)
    • build_unflags
    • generated external sources
  4. Assess whether exact compile and link commands can be dumped in addition to flag scopes

Success criteria

  • Prototype works on at least one real scripted project and one minimal synthetic fixture
  • Dumped state is rich enough for fbuild to reproduce compile and link steps without running pio run for the actual build
  • The result clearly states whether this bridge can cover ?simple SCons flag files? in practice

Failure criteria

Activity

  1. zackees commented on Apr 13, 2026

    @zackees
    MemberAuthor

    Expanded implementation notes

    I think this feature is feasible as a runtime bridge, not as a parser.

    Concrete proposal:

    • invoke PlatformIO/SCons normally enough that PRE scripts, framework/platform scripts, library logic, build_unflags, and POST scripts all run
    • inject a final dump hook after user scripts
    • serialize the resolved state into a BuildSpec
    • replay that BuildSpec in fbuild?s native compiler/linker pipeline

    Important nuance:

    • existing __idedata / project metadata proves that some state is exportable, but it is not automatically sufficient because scripts may branch on integration/service mode
    • the bridge should therefore use a dedicated dump mode or injected final script, not rely blindly on IDE metadata

    The dump should include, at minimum:

    • env/projenv scopes: CPPDEFINES, CPPPATH, CCFLAGS, CFLAGS, CXXFLAGS, ASFLAGS, LINKFLAGS, LIBPATH, LIBS
    • tool paths: CC, CXX, AR, linker, objcopy
    • build type, linker script resolution, flash images
    • source/object graph if available
    • exact compile/link argv if available

    If exact argv can be dumped, this becomes much stronger than a flags-only shim and should cover a large class of ?simple SCons flag files?.

    The practical target is not arbitrary SCons completeness; it is deterministic extraction of enough resolved state that fbuild can do the real build itself.

  2. zackees commented on Apr 13, 2026

    @zackees
    MemberAuthor

    Prototype implemented in PR #42. The branch executes supported extra_scripts mutations inside a constrained Python runtime, converts the resulting state into a typed native overlay, and replays that overlay into compile database generation, compilation, and linking.

  3. zackees commented on Apr 13, 2026

    @zackees
    MemberAuthor

    Implemented and merged in PR #42. The prototype direction proved viable as a constrained runtime bridge: fbuild now runs a narrow script harness for common extra_scripts flag/path mutations, converts the resulting state into native compile/link overlays, and rejects unsupported scripting patterns with a clear fallback recommendation to use --platformio.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions