Skip to content

AIX testing strategy #4230

Open
Open

Description

@sxa

At the moment we are exclusively testing AIX PRs against AIX 7.2.
AIX 7.3 has been available since December 2021 and we are not performing any testing on it.
This issue is to decide what approach we should use - how many machines we require etc.

I have confirmed that we can build node.js on AIX 7.3 and testing verification will be performed under

Assuming this works suitably well, we should decide on how many machines we need of AIX 7.2 and 7.3 in order to keep the CI happy.

Performance testing of the builds with different concurrency (make -j values) has been performed in #4226

At present we have four AIX 7.2 test machines - two at OSUOSL and two at IBM. There is another 7.2 system at OSUOSL waiting to be reinstalled and I would propose that we get it reinstalled with AIX 7.3.

We currently have two IBM-hosted machines running AIX 7.3. An older one currently with 8GiB of RAM that is already in the CI at test-ibm-aix73-ppc64_be-1 and the new one that I have been experimenting with recently at test-ibm-aix73-ppc64_be-3

The other decision to make is whether we need a full set of AIX 7.3 machines to test on or whether we just allow each PR test to run on any AIX machine and let it randomly choose between 7.2 and 7.3. This approach would mean we likely wouldn't need as many systems in the set.

Activity

  1. richardlau commented on Feb 3, 2026

    @richardlau
    Member

    whether we just allow each PR test to run on any AIX machine and let it randomly choose between 7.2 and 7.3. This approach would mean we likely wouldn't need as many systems in the set.

    I'm -1 on this. We do not do it anywhere else.
    We also need to plan for upstream V8 dropping support for Power 8 -- we'll need Power 8 machines to continue testing the existing release lines and plan for only running on Power 9 (or later) for Node.js 26 onwards.

  2. richardlau commented on Feb 12, 2026

    @richardlau
    Member

    Current state of our AIX resources:

    AIX Power type IBM OSUOSL notes
    7.2 8 release - 1
    7.2 8 test - 2 (+1 machine at OSUOSL that needs to be rebuilt)
    7.2 8 release 1 -
    7.2 9 test 2 -
    7.3 9 test 1 - Possibly move this as it is in an account we do not control

    The AIX 7.2 Power 9 machines above are currently labelled the same as the Power 8 machines (aix72-ppc64) and will need to be relabelled (e.g. aix72-power9) so that we can distinguish between the Power 8 and Power 9 AIX 7.2 machines.

    How many machines do we think we need for each AIX version/Power arch combination? Two test machines is our usual minimum for platforms that we test on. If we split/relabel the current resources, we'd end up with

    • x2 AIX 7.2 Power 8 test machines (all OSUOSL)
    • x2 AIX 7.2 Power 9 test machines (all IBM Cloud)

    If we're okay with two test machines, then I think we could go with @sxa's plan of reinstalling the currently offline OSUOSL machine with 7.3, giving us x2 AIX 7.3 machines (we need to check if this can be Power 9 given that we're about to drop support for Power 8 in Node.js 26).

  3. changed the title [-]AIX 7.3 testing strategy[/-] [+]AIX testing strategy[/+] on Mar 4, 2026
  4. richardlau commented on Mar 11, 2026

    @richardlau
    Member

    I'm working on splitting off the Power 9 AIX 7.2 machines from the Power 8 ones in Jenkins, similar to what was done for Linux on ppc64le.

    Existing AIX 7.2 Power 8 machines will keep the aix-ppc64 label in Jenkins (this is least disruptive for existing release lines).
    The Power 9 machines will be relabelled aix-power9 in Jenkins, and appropriate changes made to VersionSelectorScript.groovy.

    At the moment we only have one online AIX 7.2 Power 9 machine in Jenkins (the other is offline for Ansible work to add clang/rust) so I'm going to add a third machine at IBM Cloud so that we have two online after I relabel, otherwise builds will get bottlenecked on the one online Power 9 test machine. We can decide later on whether to repurpose it for AIX 7.3, or leave it as extra capacity (on the assumption that eventually the offline one is put back).

  5. sxa commented on May 19, 2026

    @sxa
    MemberAuthor

    With the changes to the playbooks and documentation under #4340 I believe we're in a good shape to add AIX 7.3 testing into the node-test-commit-aix job. I'm doing some performance tests to ensure that we have a suitable JOBS value for these systems so as not to overwhelm them with a plan to make them live tomorrow.

    We currently have five machines. My intention is to enable 3 and 5 for now.

    Machine Specification Notes
    test-ibm-aix73-ppc64_be_1 IBM,9009-22A, 4 PV_9_Compat CPUs, 8GiB Unknown setup, no clang/rust
    test-ibm-aix73-ppc64_be_2 IBM,9009-22A, 4 PV_9_Compat CPUs, 8GiB Unknown setup, no clang/rust
    test-ibm-aix73-ppc64_be_3 IBM,9080-M9S, 4 PV_9_Compat CPUs, 16GiB Working
    test-ibm-aix73-ppc64_be_4 IBM,9009-22A, 2 PV_9_Compat CPUs, 8GiB Gets stuck mid build. make chewing CPU
    test-ibm-aix73-ppc64_be_5 IBM,9009-22A, 6 PV9_Compat CPUs, 16GiB Working
  6. sxa commented on May 20, 2026

    @sxa
    MemberAuthor

    The machines are working well with Node 26+ using clang but hitting a lot of

    virtual memory exhausted: Not enough space
    

    errors on older versions with the GCC compilers (which are 32-bit). So far setting LDR_CNTRL has not resolved this. Continuing to analyse and try different things (including testing -4 with 8GiB of extra swap and -5 with JOBS=1, but we could still enable these as aix73-power9 for v26+ only, similar to what we've done for aix72-power9 for now.

    [EDIT: I think increasing LDR_CNTRL to MAXDATA=0xC0000000 from MAXDATA=0x80000000 may have resolved it]

  7. sxa commented on May 20, 2026

    @sxa
    MemberAuthor

    Also for information here are the times of a set of build runs with Node 26 done on an AIX 7.2 system (Model IBM,9080-M9S - e980) with 2 cores (SMT8) wih different -j values:

    -j Time to build
    10 107m
    9 110m
    8 114m
    7 115m
    6 124m
    5 134m
    4 155m
    3 176m
    2 247m
  8. sxa commented on May 21, 2026

    @sxa
    MemberAuthor

    aix73-power9 now added to node-test-commit-aix - it will take some time for it to populate the ccaches today on each of the three machines.

    Some stats from the different live AIX 7.3 machines. Noting also that aix73-3 is a different machine type (IBM,9080-M9S) from the other two (IBM,9009-22A):

    System Spec v27 v22 Notes
    aix73-3 4C16G -j10 1h56 -j4 2h49 -j8 2h6 -j10 2h00
    aix73-3 4C16G log=NULL -j10 1h36 -j10 1h53
    aix73-4 2C8G log=NULL -j5 3h42 -j5 2h42 -j4 2h56
    aix73-5 6C16G log=NULL -j10 1h24 -j12 1h26 -j10 1h25
    aix72-5 [*] 2C16G -j9 2h27

    [*] - Not AIX 7.3 - included as comparison

  9. sxa commented on May 24, 2026

    @sxa
    MemberAuthor

    Some failures are still occurring on the aix73-4 machine (the one with 8GiB RAM), Build works ok but the following errors are occurring on Node 24 builds. They seem to all be timeouts:

    es-module.test-wasm-web-api
    es-module.test-wasm-web-api
    parallel.test-buffer-tostring-4gb
    pummel.test-buffer-large-size-buffer-alloc
    es-module.test-wasm-web-api
    es-module.test-wasm-web-api
    pummel.test-buffer-large-size-buffer-alloc
    es-module.test-wasm-web-api
    parallel.test-buffer-tostring-4gb
    pummel.test-buffer-large-size-buffer-alloc
    parallel.test-buffer-tostring-4gb

    Some of those test runs were based on the failures seen in https://ci.nodejs.org/job/node-test-commit-aix/nodes=aix73-power9/62811/console: test-ffi-void-parameter.js and test-debugger-probe-timeout.js but they were likely PR specific issues as they weren't seen on the builds from the main branch.

    Top leve Runs on all aix 7.3 machines were:

  10. sxa commented on May 31, 2026

    @sxa
    MemberAuthor

    To follow on from the timings from the 2 core/16GiB AIX 7.2 machine (test-ibm-aix72-5) and the live AIX 7.3 machines I also ran some tests with higher -j values on one of the existing IBM AIX 7.2 machines which had 32GiB of RAM using GCC on Node 25 (aix72-3). It was an IBM,9009-22A model. As expected the core count seems to be the primary limiting factor, and it wasn't worth pushing much above the 10-12 figure which is achievable with 16GiB RAM. Note that this job is only a build and did not have any tests running so the timings aren't directly comparable to some of the previous ones which were done with the stress job:

    Version JOBS time
    v25 6 107m
    v25 8 100m
    v25 10 88m
    v25 12 80m
    v25 16 73m
    v25 20 77m
    v24 20 70m
  11. sxa commented on May 31, 2026

    @sxa
    MemberAuthor

    Rule of thumb for AIX SMT8 systems:

    • There is performance benefit up to 2.5x the number of cores, not too much more above that.
    • For GCC compilations don't go above GB_RAM=1.5*JOBS

    So for the 4 core, 16GB systems it's worth setting JOBS=10. It could be pushed a little further for the 4 core 32GiB systems but probably not a significant improvement. This does also show that it's probably not the best use of resources to provisioning 32GiB systems with only 4 SMT8 cores.

  12. richardlau commented on Sep 8, 2026

    @richardlau
    Member

    Re. #4230 (comment), I tried applying the aix73-power9 label to test-ibm-aix73-ppc64_be-4 but it is still failing these tests (I've since removed the label again):

    main:

    I believe the bench tests may be non-AIX-specific failures.

    v24:

    As #4230 (comment) notes, test-ibm-aix73-ppc64_be-4 has less memory/CPU than -3 and -5 which might account for some of those test failures.

  13. sxa commented on Sep 9, 2026

    @sxa
    MemberAuthor

    Yeah those sound like memory-limited tests!
    So we could temporary mark them flaky until we get the OSUOSL machine online. We could also bump the memory on it which I think can be done when it's shit down.
    How have the queue lengths for the aix73 label been looking this week?

  14. richardlau commented on Sep 10, 2026

    @richardlau
    Member

    How have the queue lengths for the aix73 label been looking this week?

    Hard to tell. CI has been partially suspended on/off to get the 24.x and 26.x releases out. And then there's a Windows issue that's blocking all runs from succeeding. Right now I'm looking at

    • aix72-power9: 3 jobs in progress, queue of 2 jobs
    • aix73-power9: 2jobs in progress, queue of 3 jobs

    Eyeballing the build times, it looks like aix72-power9 builds are completing between 50mins and 1 hour but the aix73-power9 builds are taking longer 1h 5mins to 1hr 30mins (with a few around 2 hours). So a combination of longer build times and less executors means the queue for aix73-power9 is going to grow faster than aix72-power9 for a similar set of jobs being submitted.

  15. richardlau commented on Sep 10, 2026

    @richardlau
    Member

    FWIW now we're at

    • aix72-power9: 3 jobs in progress, queue of 3 jobs
    • aix73-power9: 2jobs in progress, queue of 5 jobs
  16. richardlau commented on Sep 10, 2026

    @richardlau
    Member

    Yeah those sound like memory-limited tests! So we could temporary mark them flaky until we get the OSUOSL machine online. We could also bump the memory on it which I think can be done when it's shit down. How have the queue lengths for the aix73 label been looking this week?

    Tried bumping up the memory on test-ibm-aix73-ppc64_be-4 (from 8G to 16G) but on restart it doesn't appear to have mounted disks

    bash-5.2# /opt/freeware/bin/df -h
    Filesystem      Size  Used Avail Use% Mounted on
    /dev/hd4        1.0G  608M  417M  60% /
    /dev/hd2        6.0G  2.4G  3.7G  39% /usr
    /dev/hd9var     5.0G  2.7G  2.4G  54% /var
    /dev/hd3        2.0G  256M  1.8G  13% /tmp
    /dev/hd1         20G  2.3G   18G  12% /home
    /dev/hd11admin  128M  380K  128M   1% /admin
    /ahafs             -     -     0    - /aha
    /dev/hd10opt     12G   12G  704M  95% /opt
    /dev/livedump   256M  368K  256M   1% /var/adm/ras/livedump
    /dev/rep00      8.1G  8.1G   45M 100% /usr/sys/inst.images
    bash-5.2# lspv
    hdisk0          00c918806c1dfa89                    rootvg          active
    hdisk1          00c9188052cc38ac                    rootvg          active
    hdisk2          00c9188040a3702a                    homevg          active
    bash-5.2#
    

    I'm guessing that homevg should be mounted at /home.
    According to Power Virtual Server the machine should have three disks attached (as per the lspv output):

    Image
  17. richardlau commented on Sep 11, 2026

    @richardlau
  18. richardlau commented on Sep 11, 2026

    @richardlau
    Member

    I'll split fixing test-ibm-aix73-ppc64_be-4 into a separate issue.

  19. richardlau commented on Sep 15, 2026

    @richardlau
    Member

    I'll split fixing test-ibm-aix73-ppc64_be-4 into a separate issue.

    Machine was fixed and readded to the pool so we have three AIX 7.3 machines processing CI jobs. With build and test times (with populated ccache) typically being around 1 hour for AIX 7.2 and over 1 hour for AIX 7.3 I think we'll need to continue with at least three machines to manage the queue sizes.

  20. sxa commented on Oct 2, 2026

    @sxa
    MemberAuthor

    We have a slightly mad mix of AIX machines in the IBM Cloud portal just now and we shold look at rationalising them.

    Machine Type cores RAM JOBS
    release-aix72-1 S922 4 32GB 8
    test-aix72-1 E980 4 32GB 8
    test-aix72-2 S922 4 32GB 8
    test-aix72-3 S922 4 32GB 16GB 8
    test-aix72-4 S922 4 32GB
    test-aix72-5 S922 2 16GB 6
    test-aix73-2 S922 4 32GB
    test-aix73-3 E980 4 16GB 9
    test-aix73-4 S922 2 16GB 5
    test-aix73-5 S922 6 16GB 24GB 10 15

    aix73-power9 trend shows that all three of the AIX 7.3 machines can complete a build+test run in about 1h-1h10 in the best case with their current configuration - about 10 minutes slower than the aix72 ones. While the AIX 7.2 ones mostly have 32GB RAM I'm not sure that quite explains the difference (I've just reduced aix72-3 from 32GB to 16GB to be able to directly compare). There is no indication that the more expensive E980 systems are providing better performance than the S922.

    Memo to self: test-aix72-5 is offline just now as it's the machine I'm doing some experiments on

  21. sxa commented on Oct 2, 2026

    @sxa
    MemberAuthor

    Quick test of a couple of other scearios on aix73-5 with -j15:

    • v26.x:
    12:39:59 + make -j 15
    14:05:21   ln -fs out/Release/node node; fi
    14:09:43 Building addon in /home/iojs/build/workspace/node/test/ffi/fixture_library
    
    • v26.x with journalling disabled (log=NULL)
    14:40:37 + make -j 15
    15:53:07   ln -fs out/Release/node node; fi
    15:57:08 Building addon in /home/iojs/build/workspace/node/test/ffi/fixture_library
    
    • v27.x
    21:38:30 + make -j 15
    22:48:18   ln -fs out/Release/node node; fi
    22:51:43 Building addon in /home/iojs/build/workspace/node/test/ffi/fixture_library
    

    v27.x with -j10

    00:22:30 + make -j 10
    01:44:49   ln -fs out/Release/node node; fi
    01:48:50 Building addon in /home/iojs/build/workspace/node/test/ffi/fixture_library
    
    • v27.x + V8 15.2
    17:21:20 + make -j 15
    18:39:23   ln -fs out/Release/node node; fi
    18:45:22 Building addon in /home/iojs/build/workspace/node/test/ffi/fixture_library
    

    and same with populated ccache:

    19:16:44 + make -j 15
    19:42:49   ln -fs out/Release/node node; fi
    19:49:42 Building addon in /home/iojs/build/workspace/node/test/ffi/fixture_library
    

    I did note that most of the AIX 7.2 machines are using higher performance disks too - that may impact the build times which seem quicker on the 7.2 machines at the moment.

  22. pinned and unpinned a comment on Oct 2, 2026
  23. sxa commented on Oct 5, 2026

    @sxa
    MemberAuthor

    I'm going to do a little bit of testing over the day with the different types of disk. I've attached three new 30G volumes to b3-5: one "tier 1" ($6.18/month), one "tier 3" ($3.09/month) and one "tier 0" ($7.42/month) and I'll try using each of those as the workspace or /home/iojs volume.

    Memo to self on commands to add each to separate volume group

    Assuming the new PV is hdisk2:

    Disk should be at least 52GB in the IBM console to allow chfs -a size=50G from the playbooks to succeed if new FS is mounted directly on /home

    • mkvg -y tier1vg hdisk2
      Then either:
    • mklv -y tier1lv tier1vg 957 # 957 from the PP number from lspv hdisk2
    • crfs -v jfs2 -d tier1lv -m/mnt -Ano -prw -a agblkdsize=4096 -a logname=INLINE -a isnapshot=no
      or
    • crfs -v jfs2 -gtier1vg -a size=29G -m/mnt -Ano -prw -a agblkdsize=4096 -a logname=INLINE -a isnapshot=no -d tier1lv
      Switch to no log with -o log=NULL on a subsequent mount command or in the /etc/filesystems entry.
    • crfs -v jfs2 -m/mnt -Ano -prw -d fastlv
      Final setup:
    mkvg -y fastvg hdisk2
    mklv -y hdisk7lv hdisk7vg 1661# 1661 is the  PP number from: lspv hdisk2
    crfs -v jfs2 -m/home/iojs/build/workspace -d hdisk7lv
    
    Workspace volume ccache volume journalled? Build time for v26.x (*)
    tier0 tier0 yes 16m14 (75m04)
    tier0 tier0 no 16m04
    tier0 default no 15m18
    tier0 default hd8
    tier0 ramdisk no 15m49
    ramdisk ramdisk no 12m00
    tier1 default no 22m54
    tier3 default no 40m23 [+]
    default default yes 24m06 [%]
    default default no 25m40

    (*) Build time here will be defined as the difference between the make -j15 and ln -fs lines shown in the previous comment which was 1h26 or 1h13 with journally disabled). Populated ccache by default, any figures from unpopulated cache in brackets
    [+] - Significant amount of time spent in the linking steps at the end. The tier0 builds take four minutes at this point (as per snippet below), tier1 was about 10 minutes, tier3 it takes over 20

    13:40:43   LINK(target) /home/iojs/build/workspace/node/out/Release/cctest
    13:40:43   LINK(target) /home/iojs/build/workspace/node/out/Release/embedtest
    13:40:43   LINK(target) /home/iojs/build/workspace/node/out/Release/node
    13:44:43 rm 72082faef431[...]
    
    Slow link command `/usr/bin/ld -o node -b64 -bpT -bpD /usr/lib/crt0_64.o /usr/lib/crti_64.o -bcdtors -bnoerrmsg -bbigtoc -blibpath -bE -brtl /home/iojs/build/workspace/node/out/Release/obj.target/node/src/node_main.o /home/iojs/build/workspace/node/out/Release/obj.target/libnode.a /home/iojs/build/workspace/node/out/Release/obj.target/deps/histogram/libhistogram.a /home/iojs/build/workspace/node/out/Release/obj.target/tools/v8_gypfiles/libv8_snapshot.a /home/iojs/build/workspace/node/out/Release/obj.target/tools/v8_gypfiles/libv8_libplatform.a /home/iojs/build/workspace/node/out/Release/obj.target/tools/icu/libicui18n.a /home/iojs/build/workspace/node/out/Release/obj.target/deps/zlib/libzlib.a /home/iojs/build/workspace/node/out/Release/obj.target/deps/llhttp/libllhttp.a /home/iojs/build/workspace/node/out/Release/obj.target/deps/cares/libcares.a /home/iojs/build/workspace/node/out/Release/obj.target/deps/uv/libuv.a /home/iojs/build/workspace/node/out/Release/obj.target/deps/uvwasi/libuvwasi.a /home/iojs/build/workspace/node/out/Release/obj.target/deps/nghttp2/libnghttp2.a /home/iojs/build/workspace/node/out/Release/obj.target/deps/ada/libada.a /home/iojs/build/workspace/node/out/Release/obj.target/deps/merve/libmerve.a /home/iojs/build/workspace/node/out/Release/obj.target/deps/simdjson/libsimdjson.a /home/iojs/build/workspace/node/out/Release/obj.target/tools/v8_gypfiles/libsimdutf.a /home/iojs/build/workspace/node/out/Release/obj.target/deps/brotli/libbrotli.a /home/iojs/build/workspace/node/out/Release/obj.target/deps/sqlite/libsqlite.a /home/iojs/build/workspace/node/out/Release/obj.target/deps/libffi/libffi.a /home/iojs/build/workspace/node/out/Release/obj.target/deps/zstd/libzstd.a /home/iojs/build/workspace/node/out/Release/obj.target/deps/openssl/libopenssl.a /home/iojs/build/workspace/node/out/Release/obj.target/libnode_base.a /home/iojs/build/workspace/node/out/Release/obj.target/tools/v8_gypfiles/libabseil.a /home/iojs/build/workspace/node/out/Release/obj.target/deps/nbytes/libnbytes.a /home/iojs/build/workspace/node/out/Release/obj.target/deps/inspector_protocol/libcrdtp.a /home/iojs/build/workspace/node/out/Release/obj.target/deps/ncrypto/libncrypto.a /home/iojs/build/workspace/node/out/Release/obj.target/deps/ncrypto/libncrypto_engine.a /home/iojs/build/workspace/node/out/Release/obj.target/tools/icu/libicuucx.a /home/iojs/build/workspace/node/out/Release/obj.target/tools/icu/libicudata.a /home/iojs/build/workspace/node/out/Release/obj.target/tools/v8_gypfiles/libv8_base_without_compiler.a /home/iojs/build/workspace/node/out/Release/obj.target/tools/v8_gypfiles/libv8_libbase.a /home/iojs/build/workspace/node/out/Release/obj.target/tools/v8_gypfiles/libhighway.a /home/iojs/build/workspace/node/out/Release/obj.target/tools/v8_gypfiles/libv8_zlib.a /home/iojs/build/workspace/node/out/Release/obj.target/tools/v8_gypfiles/libv8_compiler.a /home/iojs/build/workspace/node/out/Release/obj.target/tools/v8_gypfiles/libv8_initializers.a -ldl -lrt /home/iojs/build/workspace/node/out/Release/obj/gen//release/libnode_crates.a -lm -lperfstat -lc++ -lc++abi /opt/clang+llvm-20.1.7-powerpc64-ibm-aix-7.2/lib/clang/20/lib/aix/libclang_rt.builtins-powerpc64.a -lunwind -lpthreads -lm -lc`

    [%] - This result is interesting since the default /home volume is a tier3 yet it seems faster than the separate tier 3 volume. The home is from a storage pool called General-Flash-75 instead of General-Flash-5 but that seems the only difference. I will note that some of the post-build "Installing" steps were quite a bit slower on this configuration though. Noting that "automatically select from any pool with sufficient capacity" was selected for the pool as opposed to any of the available affinity options.

    Conclusions:

    • The file system used for ccache makes little difference (at least for reads)
    • The tier1 disks will perform a ccached build in almost half the time of the tier3, and about 25% quicker on top of that for tier0 so there is a big benefit to using that configuration for /home/iojs/build/workspace
    • Using a ramdisk for the workspace dir can yield another 25% improvement in build times but that would have an additional overhead in terms of the RAM cost and maintenance overhead.
    • The log=NULL option didn't seem to make a huge difference to the timings but in other tests (extracting a 2GB tarball to the FS) it had a significant boost taking about a minute instead of 5½ so I still believe that is worthwhile.

    Action: I will look at leaving aix73-5 with a 30GB tier0 disk mounted with journalling disabled on /home/iojs/build/workspace

    For v27 with the change applied and the machine reduced in capacity to 16GB RAM and JOBS=10 we get:

    workspace disk ccache populated time
    default no 125m19
    default yes 32m46
    New tier0 no 110m25
    New tier0 yes 17m03
    tier0 JOBS=9 no 115m57
    tier0 JOBS=9 yes 15m14

    Tests with JOBS=9 are to check if they are significantly different from 10 which was a potential problem when performing PR testing for a recent V8 upgrade on the IBM aix73-3 machine. Based on the earlier analysis in this issue it will be a minor reduction in performance for a full build.

  24. sxa commented on Oct 6, 2026

    @sxa
    MemberAuthor

    Other potential remedial actions based on the above:

    • Switch the two E980 systems (aix72-1 and aix73-3) or S922 on IBM cloud (cost savings)
    • Add tier0 disks to the other machines that are using tier3 at present (Note that at present I have not removed the tier 3 70GB PV from aix73-5 which is part of the rootvg volume group - it may not be worth trying to take it away. (CI performance increase) - will pick this option - mkvg -y fastvg hdisk2; mklv -y fastlv fastvg $(lspv hdisk2 | awk '/^TOTAL PPs:/{print$3}'); crfs -v jfs2 -A yes -m/home/iojs/build/workspace -d fastlv then edit /etc/filesystems to have log = NULL and then mount and chown it to iojs
    • Decrease RAM from 32GB to 16GB (cost saving - no significant benefit to having 32) Remaining IBM 32GB machines are aix72-4, aix73-2, aix72-1, aix72-2 and the release machine.
    • Since it looks like JOBS=9 is feasible (10 is a potentially stretch for latest V8, and older GCC builds which typically use more didn't benefit from more than this we could set that universally. (reliability)
    • Look at ibm-aix72-4 and ibm-aix73-2 which are not currently in use in the CI. Provision or shut down? (cost saving) - those are also two of the machines with 32GB RAM
    • Adjust the startup scripts on the jenkins agent to use better values for JOBS as quite a few have now had the default (from server_jobs: in the inventory file) overridden by environment variables in the jenkins agent config.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions