Skip to content

"recursive" Dependency Causes "section too large" Error When Compiling for wasm #13513

New issue
New issue
Closed
#16418
Closed
"recursive" Dependency Causes "section too large" Error When Compiling for wasm#13513
#16418
Labels
bugSomething isn't working

Description

@berkaysynnada
berkaysynnada
opened on Nov 21, 2024

Describe the bug

I am encountering a problem when trying to compile wasmtest with the following command:

RUSTFLAGS="-C link-arg=--max-memory=4294967296" cargo build --release --target wasm32-unknown-unknown

The build fails with the following error:

error: failed to build archive at `.../target/wasm32-unknown-unknown/release/deps/libpsm-cc498cff4b514c30.rlib`: LLVM error: section too large

The following warnings were emitted during compilation:

warning: psm@0.1.24: warning: ...

I suspect the issue is related to the psm dependency, and it is coming with recursive crate, which was introduced in #13310. I am unsure if this error can be resolved without removing or replacing this dependency.

To Reproduce

Go in datafusion/wasmtest and run cargo build --target wasm32-unknown-unknown

Expected behavior

It needs to compile

Additional context

I tried again after removing recursive dep, and it works successfully

Activity

  1. berkaysynnada
    added
    bugSomething isn't working
    on Nov 21, 2024
  2. berkaysynnada
    changed the title [-]recursive Dependency Causes "section too large" Error When Compiling for wasm32-unknown-unknown[/-] [+]"recursive" Dependency Causes "section too large" Error When Compiling for wasm32-unknown-unknown[/+] on Nov 21, 2024
  3. berkaysynnada
    changed the title [-]"recursive" Dependency Causes "section too large" Error When Compiling for wasm32-unknown-unknown[/-] [+]"recursive" Dependency Causes "section too large" Error When Compiling for wasm[/+] on Nov 21, 2024

  4. Eason0729 commented on Nov 26, 2024

    @Eason0729
    Eason0729
    on Nov 26, 2024
    Contributor

    I have try to reproduce the bug by either build wasmtest or build stacker with wasm32-unknown-unknown, but was unable to reproduce the issue.

    Can you provide more information? (cargo and rustc version...)

  5. blaginin commented on Nov 26, 2024

    @blaginin
    blaginin
    on Nov 26, 2024
    Member

    This obviously doesn't resolve the issue, but wasm-bindgen/wasm-pack#1381 (comment) fixed it for me

  6. berkaysynnada commented on Nov 29, 2024

    @berkaysynnada
    berkaysynnada
    on Nov 29, 2024
    ContributorAuthor

    I have applied the @blaginin suggestion:

    berkaysahin@Berkays-MacBook-Pro wasmtest % clang --version                                        
Homebrew clang version 19.1.4
Target: arm64-apple-darwin23.4.0
Thread model: posix
InstalledDir: /opt/homebrew/Cellar/llvm/19.1.4/bin
Configuration file: /opt/homebrew/etc/clang/arm64-apple-darwin23.cfg

    @Eason0729 these are my cargo and rust versions

    berkaysahin@Berkays-MacBook-Pro wasmtest % cargo --version                                        
cargo 1.82.0 (8f40fc59f 2024-08-21)
berkaysahin@Berkays-MacBook-Pro wasmtest % rustc --version                                        
rustc 1.82.0 (f6e511eec 2024-10-15)

    When I build with: cargo build --target wasm32-unknown-unknown ,
    it successfully compiles and generates the binary.

    However, in the release mode: cargo build --release --target wasm32-unknown-unknown,

    Compiling psm v0.1.24
   Compiling arrow-ipc v53.3.0
   Compiling arrow-json v53.3.0
   Compiling arrow-csv v53.3.0
   Compiling stacker v0.1.17
   Compiling recursive v0.1.1
error: failed to build archive at `/Users/berkaysahin/Desktop/datafusion-upstream/target/wasm32-unknown-unknown/release/deps/libpsm-cc498cff4b514c30.rlib`: LLVM error: section too large

The following warnings were emitted during compilation:

warning: psm@0.1.24: warning: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: archive library: /Users/berkaysahin/Desktop/datafusion-upstream/target/wasm32-unknown-unknown/release/build/psm-d273ebae732f9287/out/libpsm_s.a the table of contents is empty (no object file members in the library define global symbols)

error: could not compile `psm` (lib) due to 1 previous error
warning: build failed, waiting for other jobs to finish...

    I don't know why this still refers to Xcode path.

    Since wasm-pack build goes with release flag, we need to solve this (assuming I am not the only one)

  7. Eason0729 commented on Dec 1, 2024

    @Eason0729
    Eason0729
    on Dec 1, 2024
    Contributor

    It compiles under 1.82.0 and 1.83.0(didn't use LLVM 15), I think it's LLVM version related.

    In addition, I may need some help on how to use specific version of LLVM?

    root@73b0a356b106:/stacker/psm# echo $PATH
/usr/lib/llvm-15/bin:/root/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

    root@73b0a356b106:/stacker/psm# clang --version
Debian clang version 15.0.6
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/lib/llvm-15/bin

    root@73b0a356b106:/stacker/psm# rustc --version --verbose 
rustc 1.83.0 (90b35a623 2024-11-26)
binary: rustc
commit-hash: 90b35a6239c3d8bdabc530a6a0816f7ff89a0aaf
commit-date: 2024-11-26
host: x86_64-unknown-linux-gnu
release: 1.83.0
LLVM version: 19.1.1

  8. peter-toth commented on Dec 14, 2024

    @peter-toth
    peter-toth
    on Dec 14, 2024
    Contributor

    @berkaysynnada, is this sill an issue? wasm-bindgen/wasm-pack#1381 (comment) seems to work for me too.

    % export PATH=/opt/homebrew/Cellar/llvm/19.1.4/bin/:$PATH

% clang --version
Homebrew clang version 19.1.4
Target: arm64-apple-darwin23.6.0
Thread model: posix
InstalledDir: /opt/homebrew/Cellar/llvm/19.1.4/bin
Configuration file: /opt/homebrew/etc/clang/arm64-apple-darwin23.cfg

% rustc --version --verbose
rustc 1.83.0 (90b35a623 2024-11-26)
binary: rustc
commit-hash: 90b35a6239c3d8bdabc530a6a0816f7ff89a0aaf
commit-date: 2024-11-26
host: aarch64-apple-darwin
release: 1.83.0
LLVM version: 19.1.1

% cargo clean
...

% cargo build --release --target wasm32-unknown-unknown
...
   Compiling datafusion-wasmtest v43.0.0 (/Users/ptoth/git/apache/datafusion/datafusion/wasmtest)
    Finished `release` profile [optimized] target(s) in 1m 36s

  9. berkaysynnada commented on Dec 14, 2024

    @berkaysynnada
    berkaysynnada
    on Dec 14, 2024
    ContributorAuthor

    That worked for me now (and passed the tests). Perhaps I should have tried with cargo clean 😞

  10. berkaysynnada
    closed this as completedon Dec 14, 2024

  11. robtaylor commented on Jan 22, 2025

    @robtaylor
    robtaylor
    on Jan 22, 2025

    I'm still getting this issue. Mac M4 with Sequoia 15.2.

    % rustc --version --verbose
rustc 1.84.0 (9fc6b4312 2025-01-07)
binary: rustc
commit-hash: 9fc6b43126469e3858e2fe86cafb4f0fd5068869
commit-date: 2025-01-07
host: aarch64-apple-darwin
release: 1.84.0
LLVM version: 19.1.5

    error: failed to build archive at `[cut]/wasm32-unknown-unknown/release/deps/libpsm-5686a1fe6f354cae.rlib`: LLVM error: section too large

warning: psm@0.1.24: warning: /Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: archive library: [cut]wasm32-unknown-unknown/release/build/psm-2da2e2ac984ce6da/out/libpsm_s.a the table of contents is empty (no object file members in the library define global symbols)
error: could not compile `psm` (lib) due to 1 previous error

  12. Eason0729 commented on Jan 24, 2025

    @Eason0729
    Eason0729
    on Jan 24, 2025
    Contributor

    This should works on Mac, but I didn't have one to test.

    We should try figuring out the reason. If we couldn't, maybe just make recursive arch-dependant dependencies(which will be another issue to open).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions