This repository was archived by the owner on Apr 4, 2025. It is now read-only.
Repository navigation
This repository was archived by the owner on Apr 4, 2025. It is now read-only.
build-optimizer: high memory usage (node crash) #240
Description
Bug Report or Feature Request (mark with an x)
- [x] bug report -> please search issues before submitting
- [ ] feature request
Area
- [x] devkit
- [ ] schematics
Versions
macOS 10.13 (High Sierra)
It affects Windows as well, but on mac it is higher (larger than 1.7 gigs, therefore causes crash with node by default)
Repro steps
The problem is happening with the build-optimizer version 0.0.31.
Using @angular/cli 1.5.0-rc.2 and building with
ng build --prod --aot --build-optimizer
The memory usage starts to grow radically at 95% emitting stage. On a semi-large project this reaches 1.7 gigs on mac, which in turn causes a crash. Increasing the memory limit on node solves the issue, but this seems rather high.
Running it without build-optimizer it builds fine without a crash.
The log given by the failure
$ ng build --prod --aot --build-optimizer
95% emitting s <--- Last few GCs --->
[10242:0x103000000] 191148 ms: Mark-sweep 1417.1 (2081.5) -> 1417.1 (2081.5) MB, 753.9 / 0.0 ms allocation failure scavenge might not succeed
[10242:0x103000000] 191836 ms: Mark-sweep 1417.1 (2081.5) -> 1417.1 (2035.5) MB, 687.0 / 0.0 ms last resort GC in old space requested
[10242:0x103000000] 192507 ms: Mark-sweep 1417.1 (2035.5) -> 1417.1 (2013.5) MB, 670.8 / 0.0 ms last resort GC in old space requested
<--- JS stacktrace --->
==== JS stack trace =========================================
Security context: 0x31fd83fa5e91 <JSObject>
1: DoJoin(aka DoJoin) [native array.js:~95] [pc=0x112d8f5ead39](this=0x31fd64e82311 <undefined>,p=0x31fdc377f859 <JSArray[432]>,q=432,E=0x31fd64e823b1 <true>,A=0x31fd83fef239 <String[1]: >,z=0x31fd64e82421 <false>)
2: Join(aka Join) [native array.js:~120] [pc=0x112d8ee542a9](this=0x31fd64e82311 <undefined>,p=0x31fdc377f859 <JSArray[432]>,q=432,A=0x31fd83fef239 <String[1]: >,z=0x31fd64e...
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
1: node::Abort() [/Users/balazsrostas/.nvm/versions/node/v8.8.1/bin/node]
2: node::FatalException(v8::Isolate*, v8::Local<v8::Value>, v8::Local<v8::Message>) [/Users/balazsrostas/.nvm/versions/node/v8.8.1/bin/node]
3: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [/Users/balazsrostas/.nvm/versions/node/v8.8.1/bin/node]
4: v8::internal::Factory::NewRawTwoByteString(int, v8::internal::PretenureFlag) [/Users/balazsrostas/.nvm/versions/node/v8.8.1/bin/node]
5: v8::internal::Runtime_StringBuilderJoin(int, v8::internal::Object**, v8::internal::Isolate*) [/Users/balazsrostas/.nvm/versions/node/v8.8.1/bin/node]
6: 0x112d8ed0463d
error Command failed with signal "SIGABRT".
Desired functionality
Using the build-optimizer shouldn't exceed the default 1.7GB threshold on a semi-large project.
Activity
filipesilva commentedon Nov 17, 2017
Hi @MrBlaise, does this happen on a project I could look at? In the context of angular/angular-cli#5618, I expect Build Optimizer usage to always use some memory, but It's odd that it happens at the 95% stage since most of the work Build Optimizer does is before the 90% phase.
Regardless, in angular/angular-cli#5618 (comment) I also have a report of high memory usage that I can't really reproduce, and in angular/angular-cli#5618 (comment) I have a report of very long times for build optimizer.
If your project isn't available, would you be interested in following some debug steps?
MrBlaise commentedon Nov 17, 2017
Hi @filipesilva ! Unfortunately I can't share the project because it is private but I am more than happy to help with the debugging steps If I can.
filipesilva commentedon Nov 17, 2017
That's great to hear!
I think it can be one of two things: either a transform is taking up a lot of memory, or the purify plugin has a runaway regex.
Lets measure how much memory some runs take. Make a new npm scrip that runs the CLI with increased memory allowance:
This number should be enough but increase it if you need to.
If you're on a mac the easiest way to see peak memory usage is to use
The number you want is
Maximum resident set size (kbytes). Record this number, it will be our baseline.Then do the same with
--build-optimizer=falseand save the number.Testing purify is easiest. Go to
./node_modules/@angular/cli/models/webpack-configs-production.jsand comment offextraPlugins.push(new build_optimizer_1.PurifyPlugin());. Whats the maximum memory used now?I think one of the build optimizer transformations is using up a lot of memory on your codebase, but I can't know which ones. I can tell you how to disable each one individually though.
You can do this in
./node_modules/@angular-devkit/build-optimizer/src/build-optimizer/build-optimizer.js. Each transform is individually added to thegetTransformsarray. You can comment out each of thegetTransforms.push(and that transform will not be added.If you can do this for each one and time me how it impacts memory used, I might be able to figure out which of the transforms is using up a lot of memory.
MrBlaise commentedon Nov 17, 2017
Here are my tests @filipesilva . I don't know it it is helpful, I don't see much difference with all the options :/
Testing purify enabled or disabled:
purify ENABLED
build-optimizer=true(Baseline): 2728394752 bytes (2.73GB) (166s runtime)purify DISABLED
build-optimizer=true: 2676985856 bytes (2.68GB) (152s runtime)purify ENABLED
build-optimizer=false: 2166198272 bytes (2.17GB) (95s runtime)purify DISABLED
build-optimizer=false: 2002616320 bytes (2.00GB) (95s runtime)Testing build-optimizer transformations (
build-optimizer=true):(These are wrapped in an if statement, so I am not sure which of them actually ran)
It seemed like the memory usage is increased when I disabled these, but it might be misleading
due to the lack of samples (I did each step only once)
wrap_enums_1.getWrapEnumsTransformerDISABLED: 2906730496 bytes (2.90GB) (165s runtime)import_tslib_1.getImportTslibTransformerDISABLED: 3338690560 bytes (3.33GB) (165s runtime)prefix_classes_1.getPrefixClassesTransformerDISABLED: 2890100736 bytes (2.89GB) (168s runtime)Side Effect Free Branch:
prefix_functions_1.getPrefixFunctionsTransformerANDscrub_file_1.getScrubFileTransformerANDclass_fold_1.getFoldFileTransformerDISABLED:2673786880 bytes (2.67GB) (155s runtime)
NOT Side Effect Free Branch
scrub_file_1.getScrubFileTransformerANDclass_fold_1.getFoldFileTransformerDISABLED:2874585088 bytes (2.87GB) (150s runtime)
alexeagle commentedon Sep 5, 2018
This issue was moved to angular/angular-cli#12159