AOT build fails due to "JavaScript heap out of memory"

Bug Report or Feature Request (mark with an `x`)

- [X ] bug report -> please search issues before submitting
- [ ] feature request

Versions.

@angular/cli: 1.0.0
node: 7.7.4
os: win32 x64
@angular/animations: 4.0.0
@angular/common: 4.0.0
@angular/compiler: 4.0.0
@angular/core: 4.0.0
@angular/flex-layout: 2.0.0-rc.1
@angular/forms: 4.0.0
@angular/http: 4.0.0
@angular/material: 2.0.0-beta.2
@angular/platform-browser: 4.0.0
@angular/platform-browser-dynamic: 4.0.0
@angular/router: 4.0.0
@angular/cli: 1.0.0
@angular/compiler-cli: 4.0.0

Repro steps.

ng build --prod --aot

The log given by the failure.

/node_modules/@angular/material/aut 92% chunk asset optimization
<--- Last few GCs --->

[6328:0000016AB4D09F00]  1775597 ms: Mark-sweep 1411.2 (1513.5) -> 1411.0 (1513.5) MB, 1240.6 / 0.0 ms  last resort


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0000034B0DD266A1 <JS Object>
    2: optimize [0000000FABC02311 <undefined>:5480] [pc=00000082575652C8](this=000000633199A279 <an AST_Call with map 0000014753FA0291>,compressor=00000226DBDECA61 <a Compressor with map 0000032B1EC8F829>)
    3: before [0000000FABC02311 <undefined>:5463] [pc=0000008258C1818D](this=00000226DBDECA61 <a Compressor with map 0000032B1EC8F829>,node=000000633199A279 <an AST_Call with map 0000014753...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

Desired functionality.

I would like to be able the app using AOT.

Mention any other details that might be useful.

I have tried changing ng.cmd to this:

@IF EXIST "%~dp0\node.exe" (
  "%~dp0\node.exe" --max_old_space_size=8192 "%~dp0\..\@angular\cli\bin\ng" %*
) ELSE (
  @SETLOCAL
  @SET PATHEXT=%PATHEXT:;.JS;=;%
  node --max_old_space_size=8192 "%~dp0\..\@angular\cli\bin\ng" %*
)

and ngc.cmd to:

@IF EXIST "%~dp0\node.exe" (
  "%~dp0\node.exe" --max_old_space_size=8192 "%~dp0\..\@angular\compiler-cli\src\main.js" %*
) ELSE (
  @SETLOCAL
  @SET PATHEXT=%PATHEXT:;.JS;=;%
  node --max_old_space_size=8192 "%~dp0\..\@angular\compiler-cli\src\main.js" %*
)

I have 8GB swap file. The build fails after 20-25 minutes standing on 92% progress.

Same problem here with:

@angular/cli: 1.0.0-rc.2
node: 6.9.2
os: darwin x64
@angular/common: 2.4.10
@angular/compiler: 2.4.10
@angular/core: 2.4.10
@angular/forms: 2.4.10
@angular/http: 2.4.10
@angular/platform-browser: 2.4.10
@angular/platform-browser-dynamic: 2.4.10
@angular/router: 3.4.10
@angular/cli: 1.0.0-rc.2
@angular/compiler-cli: 2.4.10

I have the same problem when running "ng build -w"
@angular/cli: 1.0.0
node: 7.8.0
@angular/* 4.0.2

setting --max_old_space_size=8192 works for me, but it is not really a good solution since I cannot redistribute it with the source code to my developers.

One note though, there are both a ng.cmd and an ng file (the latter one is an sh file), and if you are in windows as we are both can be used depending on if you run ng from git bash or cmd so both needs to be changed.

My solution to the problem will probably be to change the npm build command from ng build to call a copy of the ng bash file which I distribute with my code.

@Ristaaf

@Ristaaf How did you change the ng file? Where did you add --max_old_space_size in this file? Can you please share your file.

@omeralper

@omeralper: This is my solution, it requires people to use git bash as terminal if on windows, but it would be easy to change if needed (just use the cmd file instead):

In my project root I have a folder called scripts and in it a file called ng.sh, which is a copy from node_modules/.bin/ng but with more allowed RAM to be used

#!/bin/sh
basedir=$(dirname "$(echo "$0" | sed -e 's,\\,/,g')")

case `uname` in
    *CYGWIN*) basedir=`cygpath -w "$basedir"`;;
esac

if [ -x "$basedir/node" ]; then
  "$basedir/node" --max_old_space_size=8192 "./node_modules/@angular/cli/bin/ng" "$@"
  ret=$?
else
  node --max_old_space_size=8192 "./node_modules/@angular/cli/bin/ng" "$@"
  ret=$?
fi
exit $ret

Then in my package.json I do:

"scripts": {
    "build-prod": "bash ./scripts/ng.sh build --prod --aot --env=prod"
}

I am experiencing a similar issue resulting in a JavaScript heap out of memory error when calling ng e2e in a project using:

@angular/cli@1.0.1
@angular/*@4.1.0
node@6.10.2

I was able to create a cross-platform work-around by changing the package.json e2e script entry to:

    "test-browser-e2e-cucumber": "node --max_old_space_size=4096 node_modules/@angular/cli/bin/ng e2e --webdriver-update",

@if

On windows
@if EXIST "%~dp0\node.exe" (
"%~dp0\node.exe" --max_old_space_size=4096 "%~dp0\node_modules@angular\cli\bin\ng" %*
) ELSE (
@SETLOCAL
@set PATHEXT=%PATHEXT:;.JS;=;%
node --max_old_space_size=4096 "%~dp0\node_modules@angular\cli\bin\ng" %*
)

check this npm package out https://github.com/endel/increase-memory-limit

I wrote a little script that lets me control the memory limit in the config section of package.json.
https://github.com/isaacplmann/angular-cli-alias

increase-memory-limit looks pretty sweet too. It didn't exist when I wrote my script and it doesn't let you set your own memory limit.

i've been able to repro this with a minimal test case using an out-of-the-box cli app; relevant file here.

basically it seems like trying to match a wildcard without specifiers will blow up the app, which is weird since the test case is pulled almost directly from typescript documentation.

note that "*": ["*"] does not crash the app, only "*": ["*", "$dir/*"]

as a workaround, i've manually specified each of my path maps, which is annoying but lets me compile:

"paths": {
  "app.*": ["app/app.*"],
  "app-*": ["app/app-*"],
  "authentication/*": ["app/authentication/*"],
  "core/*": ["app/core/*"],
  "navigation/*": ["app/navigation/*"],
  "pages/*": ["app/pages/*"]
}

@zackschuster

We also hit that issue in NativeScript when using the @ngtools/webpack plugin with nativescript-dev-webpack. We use tsconfig paths for mapping our tns-core-modules. They look like that:

"paths": {
  "*": [
    "./node_modules/*",
    "./node_modules/tns-core-modules/*"
  ]
}

However, the bundling fails with the mentioned error (JavaScript heap out of memory). Just like @zackschuster , as a workaround we specified each path mapping manually:

"paths": {
  "application": ["node_modules/tns-core-modules/application"],
  "application-settings": ["node_modules/tns-core-modules/application-settings"],
  "camera": ["node_modules/tns-core-modules/camera"],
  "color": ["node_modules/tns-core-modules/color"],
  "connectivity": ["node_modules/tns-core-modules/connectivity"],
  "console": ["node_modules/tns-core-modules/console"],
  "data/*": ["node_modules/tns-core-modules/data/*"],
  "fetch": ["node_modules/tns-core-modules/fetch"],
  "file-system": ["node_modules/tns-core-modules/file-system"],
  "fps-meter": ["node_modules/tns-core-modules/fps-meter"],
  "globals": ["node_modules/tns-core-modules/globals"],
  "http": ["node_modules/tns-core-modules/http"],
  "image-asset": ["node_modules/tns-core-modules/image-asset"],
  "image-source": ["node_modules/tns-core-modules/image-source"],
  "location": ["node_modules/tns-core-modules/location"],
  "platform": ["node_modules/tns-core-modules/platform"],
  "text": ["node_modules/tns-core-modules/text"],
  "timer": ["node_modules/tns-core-modules/timer"],
  "trace": ["node_modules/tns-core-modules/trace"],
  "ui/*": ["node_modules/tns-core-modules/ui/*"],
  "utils/*": ["node_modules/tns-core-modules/utils/*"],
  "xhr": ["node_modules/tns-core-modules/xhr"],
  "xml": ["node_modules/tns-core-modules/xml"]
}

@sis0k0

@sis0k0 what do your recompile times look like with that setup? i was getting 40-50s recompiles (on an absurdly tiny project) until i dropped the path maps, at which point it dropped to ~2s.

@zackschuster

@zackschuster
What do you mean by path maps?

@Zigzag95

@Zigzag95

"paths": {
 // ...
}

@zackschuster

@zackschuster, similar times.

#5618 (comment)

I tried the increase-memory-limit package, and I still get Javascript heap out of memory

It's time to fix this !!! In real big project angular is bad because of that ! Maybe webpack problem !! Thanks

Still nothing on this?

I am using @angular/cli 1.0.0-rc.4, if I upgrade to 1.0.0 it gives me the error of
angular cli FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

Now we understand why Google don't use webpack with ng for its apps

Work Around Fix
I was able to fix the issue by changing my ng.cmd file to
@IF EXIST "%~dp0\node.exe" ( "%~dp0\node.exe" --max_old_space_size=8192 "%~dp0\node_modules\@angular\cli\bin\ng" %* ) ELSE ( @SETLOCAL @SET PATHEXT=%PATHEXT:;.JS;=;% node --max_old_space_size=8192 "%~dp0\node_modules\@angular\cli\bin\ng" %* )

@cimachris

@cimachris this is not exactly a fix but rather a workaround that can possibly postpone the problem

Is that better?

@hansl

@hansl @filipesilva I am not able to upgrade my application to angular/cli 1.0.0, since I am at angular/cli 1.0.0.rc-4, don't want to make the jump to the latest version without testing my components

Have been affected by this issue since @angular/cli 1.0 was released
But currently we are able to workaround it by running
ng build --aot --sourcemaps=false --environment=prod

It's not exactly the same as the --prod parameter but at least it works.

ng e2e also results in a similar error:


 92% chunk asset optimization
<--- Last few GCs --->

[2164:000002C715EFDC90]   117246 ms: Mark-sweep 1401.4 (1543.7) -> 1401.3 (1545.7) MB, 1088.0 / 0.0 ms  allocation failure GC in old space requested
[2164:000002C715EFDC90]   118473 ms: Mark-sweep 1401.3 (1545.7) -> 1401.2 (1504.2) MB, 1227.0 / 0.0 ms  last resort
[2164:000002C715EFDC90]   119591 ms: Mark-sweep 1401.2 (1504.2) -> 1401.1 (1504.2) MB, 1117.8 / 0.0 ms  last resort


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 000000799D9A66A1 <JS Object>
    2: addMapping(aka SourceMapGenerator_addMapping) [xxx\node_modules\source-map\lib\source-map-generator.js:~94] [pc=0000003D3772300C](this=0000022DE262CAF1 <a SourceMapGenerator with map 000000A8E17906F1>,aArgs=0000005D8F791BF9 <an Object with map 00000399007FF561>)
    3: /* anonymous */ [xxx\node_modules\source-map\l...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

@filipesilva

@filipesilva This is pretty serious and the workaround does not allways work - it seems to affect all medium/large projects. Needs solved for angular + angularcli to remain a viable technology for serious web projects.

@angular/cli: 1.2.3
node: 6.10.0
os: linux x64
@angular/animations: 4.3.1
@angular/common: 4.3.1
@angular/compiler: 4.3.1
@angular/core: 4.3.1
@angular/forms: 4.3.1
@angular/http: 4.3.1
@angular/platform-browser: 4.3.1
@angular/platform-browser-dynamic: 4.3.1
@angular/router: 4.3.1
@angular/cli: 1.2.3
@angular/compiler-cli: 4.3.1
@angular/language-service: 4.3.1

App with 3 modules and about 50 components, after switching @angular/cli from 1.1.x to 1.2.x

ng build -target=production

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [@angular/cli]
 2: 0x109bf8c [@angular/cli]
 3: v8::Utils::ReportApiFailure(char const*, char const*) [@angular/cli]
 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [@angular/cli]
 5: v8::internal::Factory::NewInternalizedStringImpl(v8::internal::Handle<v8::internal::String>, int, unsigned int) [@angular/cli]
 6: v8::internal::StringTable::LookupString(v8::internal::Isolate*, v8::internal::Handle<v8::internal::String>) [@angular/cli]
 7: v8::internal::LookupIterator::PropertyOrElement(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, bool*, v8::internal::LookupIterator::Configuration) [@angular/cli]
 8: v8::internal::Runtime::GetObjectProperty(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>) [@angular/cli]
 9: 0xed3501 [@angular/cli]
10: v8::internal::Runtime_KeyedGetProperty(int, v8::internal::Object**, v8::internal::Isolate*) [@angular/cli]
11: 0x2174585092a7
Aborted (core dumped)

root cause analysis:

OOM on CI machine ( also after adding --max-old-space-size=4096 to node ) due to broken symlink

After fixing it:

ng build --prod

ERROR in ./src/main.ts Module not found: Error: Can't resolve './$$_gendir/app/app.module.ngfactory' @ ./src/main.ts 3:0-74 @ multi ./src/main.ts
`
Upgraded @angular/cli to 1.2.4, still getting the same error ( yarn )

After looking at #7125 it looked like it was an aot related, package dependency issue.
Reverting back to npm fixed it.

i've also seen this error pop up when using @ngtools/webpack by itself. interestingly, it doesn't happen until after a few rebuilds, implying there's a huge memory leak somewhere in either webpack (doubtful) or the ngtools plugin.

this is so java :))

Same issue. As soon as project gain size this thing happens.

Exact same error here, but with @ngtools/webpack
I've tried running it with Node LTS 6.11.2 and Node Latest as well 8.2.1
Tried with Typescript 2.3.2 and 2.4.2.
All my angular packages are at 4.3.3

AoT is pretty much necessary for large apps, so it's kinda ironic we can't get it so build for those.

after update to angular/cli@1.3.0 have the same issue.

@angular/cli: 1.3.0
node: 8.2.1
os: darwin x64
@angular/animations: 4.3.3
@angular/common: 4.3.3
@angular/compiler: 4.3.3
@angular/core: 4.3.3
@angular/forms: 4.3.3
@angular/http: 4.3.3
@angular/platform-browser: 4.3.3
@angular/platform-browser-dynamic: 4.3.3
@angular/router: 4.3.3
@angular/cli: 1.3.0
@angular/compiler-cli: 4.3.3

command:
ng build -prod --build-optimizer --aot=true --progress=false --sourcemap

Just as a heads up. It turns out when I use YARN instead of NPM to RESTORE packages ng build completes successfully.

I am using

ng build --env=prod --aot

When I restore packages using npm install -- same error from this issue (out of memory)
When I restore packages using yarn install -- no error - success

Also got this error after updating to @angular/cli 1.3.0.

Problem solved after upgrading typescript both locally and globally to 2.4.2

Also got this error.

Locking @ngtools/webpack to version 1.5.4 helps.

I had to increase memory given to Node, as well as bump TS to 2.4.2, but after that I did get a successful build. looking forward to NG5 AoT compiler

Same problem here after updating to @angular/cli 1.3.0.

@williangd

@williangd also for me with 1.3.0

I've also been getting the out of memory issue when running ng serve. It works fine for hours, but if I've been developing for a while (I've seen it anywhere from 2-5 hours) I will get the out of memory crash. When rerunning the serve command, I can go typically for another 2-5 hours. The first time I saw this was when updating to a beta version of 1.3, but I haven't seen an improvement with the current release.

For me it also happens with the normal "ng serve" after upgrading to 1.3 version, every time the project recompiles, its takes more time than the previous (6s, 10s, 30s, 60s) and after 4 or 5 times, it goes out of memory.

Same issue.

95% emitting
<--- Last few GCs --->

160332 ms: Mark-sweep 1346.8 (1439.3) -> 1346.7 (1439.3) MB, 648.6 / 0.0 ms [allocation failure] [scavenge might not succeed].
160934 ms: Mark-sweep 1346.7 (1439.3) -> 1346.8 (1439.3) MB, 602.0 / 0.0 ms [allocation failure] [scavenge might not succeed].
161564 ms: Mark-sweep 1346.8 (1439.3) -> 1346.8 (1423.3) MB, 629.5 / 0.0 ms [last resort gc].
162181 ms: Mark-sweep 1346.8 (1423.3) -> 1346.7 (1423.3) MB, 616.9 / 0.0 ms [last resort gc].

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0000009DB903FA99
1: DoJoin(aka DoJoin) [native array.js:~129] [pc=000000A40AE7FE82] (this=0000009DB9004241 ,w=00000355FBAF7F99 <JS Array[287]>,x=287,N=0000009DB9004281 ,J=0
000009DB904A369 <String[1]: >,I=00000342545B4E81 <JS Function ConvertToString (SharedFunctionInfo 0000009DB905DBC9)>)
2: Join(aka Join) [native array.js:180] [pc=000000A406C83FF2] (this=0000009DB9004241 ...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

tsc -v
Version 2.4.2

@angular/cli: 1.3.0
node: 6.11.2
os: win32 x64

How is upgrading to typescript 2.4.2 working. You will get build issues from RxJs as they 2.4 broke the typings of rxjs

This problem is happening as well for our project. Can't build aot on build server with the same javascript heap out of memory. We are also using the experimental webpack 3 support.

@sbabeal

@sbabeal just go to RxJS 5.4.2

For us - building locally with
ng build --target=production --environment=prod --base-href /myproject/ --aot --build-optimizer
works just fine, but when we build it on our server, we get the
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
error. However, if we remove the --build-optimizer everything runs fine.

I've tried adding the increase-memory-limit library, but without any luck.

Is there a fix to this that actually works?

Kind regards Chris

Tried latest version of typescript and still didn't help. Locally it builds but consumes 1.5 gigs of memory. Seems quite excessive.

92% chunk asset optimization
<--- Last few GCs --->

[3933:0x2a7d710] 355436 ms: Mark-sweep 1395.8 (1546.4) -> 1395.3 (1546.4) MB, 975.3 / 0.0 ms allocation failure GC in old space requested
[3933:0x2a7d710] 356584 ms: Mark-sweep 1395.3 (1546.4) -> 1395.3 (1546.9) MB, 976.0 / 0.0 ms allocation failure GC in old space requested
[3933:0x2a7d710] 357953 ms: Mark-sweep 1395.3 (1546.9) -> 1395.3 (1539.9) MB, 1195.7 / 0.0 ms last resort
[3933:0x2a7d710] 359143 ms: Mark-sweep 1395.3 (1539.9) -> 1395.3 (1539.9) MB, 1188.8 / 0.0 ms last resort

<--- JS stacktrace --->

==== JS stack trace =========================================

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
1: node::Abort() [@angular/cli]
2: 0x12b717c [@angular/cli]
3: v8::Utils::ReportOOMFailure(char const*, bool) [@angular/cli]
4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [@angular/cli]
5: v8::internal::Factory::NewCode(v8::internal::CodeDesc const&, unsigned int, v8::internal::Handlev8::internal::Object, bool, bool, int, bool) [@angular/cli]
6: v8::internal::CodeGenerator::MakeCodeEpilogue(v8::internal::MacroAssembler*, v8::internal::EhFrameWriter*, v8::internal::CompilationInfo*, v8::internal::Handlev8::internal::Object) [@angular/cli]
7: v8::internal::FullCodeGenerator::MakeCode(v8::internal::CompilationInfo*, unsigned long) [@angular/cli]
8: v8::internal::FullCodegenCompilationJob::ExecuteJobImpl() [@angular/cli]
9: v8::internal::CompilationJob::ExecuteJob() [@angular/cli]
10: 0xd577dc [@angular/cli]
11: 0xd59faa [@angular/cli]
12: 0xd5b7a9 [@angular/cli]
13: v8::internal::Compiler::Compile(v8::internal::Handlev8::internal::JSFunction, v8::internal::Compiler::ClearExceptionFlag) [@angular/cli]
14: v8::internal::Runtime_CompileLazy(int, v8::internal::Object**, v8::internal::Isolate*) [@angular/cli]
15: 0x3e6ef29040c7
Aborted (core dumped)

We seem to face the error even on allocating 12GB of memory for the build.
Is there any way to debug what is causing the memory leak? also, could this be solved by implementing lazy loading of modules, since this is during build time?

Commands used:
1.node --max_old_space_size=12000 ./node_modules/@angular/cli/bin/ng build --prod --aot --env=prod
2. updated the ng to accept the --max_old_space_size parameter

The build is working only with --sourcemap=false and --aot=false; but this is causing memory leaks and page load issues in the browser due to huge application size.

StackTrace:-
[INFO]
[INFO] <--- Last few GCs --->
[INFO]
[INFO] [11944:0x2710b70] 898052 ms: Mark-sweep 11998.2 (13529.5) -> 11998.2 (13529.5) MB, 4991.9 / 0.0 ms allocation failure GC in old space requested
[INFO] [11944:0x2710b70] 903445 ms: Mark-sweep 11998.2 (13529.5) -> 11998.1 (13434.0) MB, 5366.9 / 0.0 ms last resort
[INFO] [11944:0x2710b70] 908749 ms: Mark-sweep 11998.1 (13434.0) -> 11998.1 (13378.0) MB, 5266.9 / 0.0 ms last resort
[INFO]
[INFO]
[INFO] <--- JS stacktrace --->
[INFO]
[INFO] ==== JS stack trace =========================================
[INFO]
[INFO] Security context: 0x2b8a8b3a66a1
[INFO] 1: DoJoin(aka DoJoin) [native array.js:~97] [pc=0x168f8ea1398a](this=0x38d692102311 ,q=0x39d30737fbe1 <JS Array[2905]>,r=2905,F=0x38d6921023b1 ,B=0x38d692157ff9 <String[1]: >,A=0x38d692102421 )
[INFO] 2: Join(aka Join) [native array.js:~122] [pc=0x168f8e9b25e2](this=0x38d692102311 ,q=0x39d30737fbe1 <JS Array[2905]>,r=2905,B=0x38d692157ff9 <String[1]: >,A=0x...
[INFO]

I don't know if it can help someone but we had the same problem after upgrading to version 1.3.0. And we had others problem too : slow compilation and bad names for generated chunk.

All our problems were solved with this : #7338

There's always an option to eject the app.

What do you mean by ejecting the application?

…

On Tue, Aug 22, 2017 at 5:12 PM, Arseny E. Naryshkin < ***@***.***> wrote: There's always an option to eject the app. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#5618 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADuwsXkLSkoHTg9-AMuUESUfM2sCLVbOks5satPRgaJpZM4Mn-Nn> .

Still a problem for us even after CLI upgrade to 1.3.1 and @angular upgrade to 4.3.5

tried typescript 2.5 but run into some compilation problems. we're on 2.4.2.

edited:

On our local machines or our gitlab runner instances we can't reproduce the problem. For some reason the build only fails on our jenkins box. The only difference I've been able to gather is jenkins is on a slightly older version of node/npm version:

v6.9.1
3.10.8

instead of

v6.11.0
3.10.10

fixed by changing our production npm build script to:

"build-prod": "node --max_old_space_size=4096 ./node_modules/@angular/cli/bin/ng build --prod --aot",

Not so much a fix as it is a brute force work-around, but it'll at least buy time while we figure out what's eating up all the heap space.

@notsonotso oh, it will not then.

my app, however, fails to compile by ng with 'heap out of memory' error, but does as ejected.

Here's a related, solved issue in angular-seed. I hope it helps someone.

No amount of setting max-old-space-size helped on our project. However, as we only build AOT in prod, we have temporarily disabled sourcemaps which seemed to fix it for us on our CI server.

@mattem

Like @mattem I have not been able to get the max-old-space-size to work either. I do not if I am fully setting it up right though since the build process I have uses webpack with "ngtools/webpack".

Same issue, but we've found a more reliable workaround to increase the memory limit.

Since Node 8.x, you can set a NODE_OPTIONS environment variable, and pass in the --old-max-space-size flag. We currently need 6GB.

E.g. on Windows CMD: set NODE_OPTIONS=--old-max-space-size=6411 before running the AOT build.

I wonder if there is a child process that doesn't get the arguments that are set on the node executable.

I know, and trust me, I'm just as much struggling with this. Not fun when the build server starts to run out of memory. It's a serious problem that should be addressed. This workaround just helped me to limp along a little farther, just like turning off the source maps. Your mileage may vary.

Hello.
this problem is serious.

On my computer I was limiting to 8Gb and the ng build was working fine.
(with old-max-space-size)

But now I've got a big big project and believe me, out of memory.

I have a code generator that builds everything perfectly.
but the new project has more than 200 entities.
The creation of modules and lazy loading depends on the settings made by the client and their entities.

I would not want to separate in several applications because there is a lot of dependency.
(200 services, 600 components)

The project runs fine with ng serve, but not with ng build.
I rented a virtual machine with 28Gb ram and believe me, did the build using 27Gb of RAM.
The service node occupied exactly 27Gb.

(I put the limit of 28gb)

Everything works fine with ng serve.

can anyone know to tell me if just separating into several lazy loading modules it will need less memory to compile with ng build? or it works only to separate the JS?

I can not rent a virtual machine whenever I need to run ng build.
And the effort of change will be very high.
So I needed to be sure of what to do.

Thanks.

@tolitech Instead of depending on having enough RAM for this to build, just add swap to your machine. The build will be a little slower, but you don't need to be renting machines with 28 GB of RAM.

I too want this to be resolved ASAP, but adding swap space gets around the memory issue.

@joecot

@joecot Thank you.
I'll try to do this and see if it works.
If it works, I do not see any problems if it takes 8 hours.
I leave a night job for publication without problems.

That was a good idea.
I'll try.

Thank you.

@joecot

@joecot @notsonotso
It worked.
I configured 32Gb virtual memory (swap), --max_old_space_size and finally worked.
It used the 28gb (It's too high), but it worked.
Thank you very much for the idea, @joecot

@mhevery

@mhevery @bradlygreen - apologies for pulling you into this one, but with @tolitech's extreme case above, the value proposition of Angular is starting to become seriously undermined.

Our codebase is growing rapidly, and our team is continuously running into these memory issues, where we even reach the limitations of our development and build machines. In all my years of web development there's never been a case that I would run out of the capabilities of the machine, just to compile to several 300k files for my webapp/website. It seems to be a memory optimization or architectural issue that hopefully can be avoided.

We've already lost countless hours on these problems, and the premise that Angular is a framework that scales for large enterprise applications doesn't hold up anymore.

I'd really appreciate if this issue could be prioritized, as I'm really starting to doubt whether our choice of Angular as a suitable framework was correct, considering that only the AOT builds are performant and small enough to ship to customers over the wire, and that we're struggling to get those to work. Thank you.

@bgever

@bgever Apparently google is not using angular-cli/webpack themselves, so they don't run into problems like this. I suspect this is why this problem receives so little attention from the angular developers. If this continues, the community should be publicly warned about angular being unsuitable for large projects.

@bgever

You were perfect in your words, @bgever .
I'm already thinking that Angular does not work for large applications, after all I'm now needing 28 Gb of RAM and the project could grow bigger. (This is the first day of this application).

I'm already thinking about another strategy for this type of project, maybe even change technology because I'm realizing that this problem is occurring with many people and for a long time.

I believe my customer will not accept me saying that he needs 28 Gb RAM to compile his own project.
I'm very worried about this.

I'm just staying in Angular CLI 1.2.7 while they fix this, hope they do it soon tho, but with 1.2.7 I have no memory problems at all.

Same here. Running Angular CLI 1.3.2 and Angular 4.3.6. Added this based on another thread to get it to work. My app isn't very big but CLI clocked in at 4GB to build my app.

"scripts": {
    "build": "node --max_old_space_size=8192 ./node_modules/@angular/cli/bin/ng build --target production --environment prod --aot",
    "ng": "ng",
    "start": "ng serve",
    "test": "ng test",
    "lint": "ng lint",
    "e2e": "ng e2e"
  },

@bjornharvold

@bjornharvold @tolitech We need 1024go ram 32ghz cpu, so 32'000'000'000 calcul per second to build angular but if you learn bazel from google it can be faster 😂

https://github.com/endel/increase-memory-limit

This worked for me

Same problem here.

I had to downgrade to @angular/cli 1.2.6 with TS 2.4.2 for the build to work again.

A shame I can't use the latest angular cli ... Why isn't this considered a severe bug ?

@hugodes

@hugodes

Why isn't this considered a severe bug ?

I think it is because of angular/angular#19058

Lastest version is the same !

Same here, this is ridiculous. I'm using a cheap VPS server to build and it simply crashes with 2GB of memory

What solved it for me was doing ng build --prod instead of ng build --aot --prod.

The --prod option already includes the --aot one, and optimises the build process and I'm not getting memory errors any more.

Hope it helps some of you guys.

Try -sm false

It happens to me when the allowJS parameter is true in tsconfig.json. Apparently, the javascript files are overloading the memory.

This worked for me, add to package.json scripts section. Watching the memory usage in task manager it got up to around 4gb usage. Seems a touch excessive!

"build2": "node --max_old_space_size=8192 ./node_modules/@angular/cli/bin/ng build --prod --aot"

npm run build2

This command works for me:
node --max-old-space-size=4096 ./node_modules/@angular/cli/bin/ng build --prod

"scripts":{
"prod": "node --max-old-space-size=4096 ./node_modules/@angular/cli/bin/ng build --prod"
}

try process.nextTick() ？

With Node 8.x and latest cli version works without any hack :)

Sad to see that months have passed and this issue is still not solved. I'm having the same problem here:

´
ng build --aot

92% chunk asset optimization
<--- Last few GCs --->

191455 ms: Mark-sweep 1296.7 (1433.7) -> 1291.4 (1434.7) MB,
1052.3 / 0.0 ms [allocation failure] [GC in old space request
ed].
192605 ms: Mark-sweep 1291.4 (1434.7) -> 1291.4 (1434.7) MB,
1150.6 / 0.0 ms [allocation failure] [GC in old space request
ed].
193711 ms: Mark-sweep 1291.4 (1434.7) -> 1294.8 (1403.7) MB,
1105.0 / 0.0 ms [last resort gc].
194808 ms: Mark-sweep 1294.8 (1403.7) -> 1298.3 (1403.7) MB,
1097.0 / 0.0 ms [last resort gc].

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 00000146E3DCF791
2: addMapping(aka SourceMapGenerator_addMapping) [D:\Users
\andre.marcondes\projects\lender-platform\node_modules\source-
map\lib\source-map-generator.js:~94] [pc=00000339B436339E] (th
is=0000015776039641 <a SourceMapGenerator with map 00000102C1F
AA599>,aArgs=00000004C17F22D1 <an Object with map 00000194E061
0DD9>)
3: /* anonymous */ [D:\Users\andre.marcondes\projects\lend
er-platform\node...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScrip
t heap out of memory
´

@notsonotso an ng team member told me this has to do with poor recursion handling in @ngtools/webpack, i'm curious what webpack's faults are?

Started happening to me using macOS 10.13
On Windows it builds fine, both machines using the same configuration and have 16 gigs of ram.

I went back to version 1.2.x and didn't have any exceptions there.

1.3.x and 1.4.x all gave the memory exception.

@filipesilva

@filipesilva Is there any news regarding to this issue?
I'm curious why a semi-large project needs more than 2 gigs to build. I am not familiar with the inner workings of the compilation process, but it seems rather high to me.

EDIT:

using the --build-optimizer flag causes the issue, it was working fine, but I recently upgraded the cli and I see that it uses a newer build-optimizer (0.0.31), maybe memory leak on the optimizer?

@MrBlaisee you mean 20* Go

!!

try ng build --aot --sourcemaps=false ,
it seems there is problem with sourcemaps.

Lol with or without sourcemap it depend how long node is working! Hope this will be fixed one day

I am using 6.9.4 Nodejs and Angular 4.5, Angular-cli 1.4.6 but no lucks

We need hacks above.

@if

Got an issue resolved on one of the machine using below:

Try this out:
In "ProjectDirectory\node_modules\.bin" modify webpack.cmd to:

@if EXIST "%~dp0\node.exe" (
"%~dp0\node.exe --max_old_space_size=4096 " "%~dp0..\webpack\bin\webpack.js" %*
) ELSE (
@SETLOCAL
@set PATHEXT=%PATHEXT:;.JS;=;%
node --max_old_space_size=4096 "%~dp0..\webpack\bin\webpack.js" %*
)

and try to build.
Basically, its going to use --max_old_space_size=4096 with node webpack build.

See if that helps.

Angular 5 and angular-cli 1.5. Have problems --aot

95% emitting
<--- Last few GCs --->

  152742 ms: Mark-sweep 1261.4 (1434.1) -> 1261.4 (1434.1) MB, 471.1 / 0.0 ms [allocation failure] [scavenge might not succeed].
  153236 ms: Mark-sweep 1261.4 (1434.1) -> 1261.4 (1434.1) MB, 493.7 / 0.0 ms [allocation failure] [scavenge might not succeed].
  153757 ms: Mark-sweep 1261.4 (1434.1) -> 1268.2 (1418.1) MB, 520.5 / 0.0 ms [last resort gc].
  154333 ms: Mark-sweep 1268.2 (1418.1) -> 1275.0 (1418.1) MB, 575.8 / 0.0 ms [last resort gc].


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 000003AFBC7CFB61 <JS Object>
    1: DoJoin(aka DoJoin) [native array.js:~129] [pc=00000055CD16A7F7] (this=000003AFBC704381 <undefined>,w=0000011B90EE0C21 <JS Array[539]>,x=539,N=000003AFBC
7043C1 <true>,J=000003AFBC7AE4E1 <String[1]:  >,I=000003AFBC7B46F1 <JS Function ConvertToString (SharedFunctionInfo 000003AFBC752DC9)>)
    2: Join(aka Join) [native array.js:180] [pc=00000055CD1D6A52] (this=000003AFBC704381 <undefined>...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

Also hit this memory issue. Can be 'resolved' by using the build script node --max-old-space-size=4096 ./node_modules/@angular/cli/bin/ng build --prod but hopefully this bug will be fixed soon. Why does it need over 2GB of memory?!

Also facing the same issue currently, our app has over 50 components spread across 6 modules.

We are experiencing the same issue... eclipsing 3GB of memory in a medium-large application. We have ~20 modules each with 5-8 components.

Hey all, I'm seeing a lot of new reports of this happening after 1.5 is out. One of the changes we did for 1.5 was to use Build Optimizer by default on Angular version 5 projects. Build optimizer does indeed use a lot of memory (angular/devkit#240).

Can you tell me if turning off build optimizer helps with your memory crashes? You can do it with --build-optimizer=false. Turning off sourcemaps can also help via --sourcemaps=false.

If these options don't help, can you post here the full failure log please, including command run?

--build-optimizer=false helps in my case.

I have a large app (30 modules, 263 components).

With @angular: 4.4.6 ; @angular/cli: 1.4.9 I can get a --prod build with --max_old_space_size=6144 but with @angular: 5.0.0 ; @angular/cli: 1.5.0 I run out of memory even with --max_old_space_size=8192. I can get a build with --max_old_space_size=10240, but my CI does not support that configuration.

> node --max_old_space_size=8192 ./node_modules/@angular/cli/bin/ng build --prod


<--- Last few GCs --->

  387148 ms: Mark-sweep 4983.1 (8230.5) -> 4987.2 (8230.5) MB, 860.4 / 0.0 ms [allocation failure] [scavenge might not succeed].
  387947 ms: Mark-sweep 4987.2 (8230.5) -> 4987.3 (8230.5) MB, 798.9 / 0.0 ms [allocation failure] [scavenge might not succeed].
  388816 ms: Mark-sweep 4987.3 (8230.5) -> 5002.0 (8214.5) MB, 869.3 / 0.0 ms [last resort gc].
  389619 ms: Mark-sweep 5002.0 (8214.5) -> 5017.0 (8214.5) MB, 802.0 / 0.0 ms [last resort gc].


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x2199619cfb39 <JS Object>
    1: DoJoin(aka DoJoin) [native array.js:~129] [pc=0x29bfc23b9949] (this=0x219961904381 <undefined>,w=0x3864fdc19d1 <JS Array[537]>,x=537,N=0x2199619043c1 <true>,J=0x2199619ae4b9 <String[1]:  >,I=0x2199619b46c9 <JS Function ConvertToString (SharedFunctionInfo 0x219961952dc9)>)
    2: Join(aka Join) [native array.js:180] [pc=0x29bfc2ccdb52] (this=0x219961904381 <undefined>,w=0x3864fdc19d1 <JS ...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [/Users/*****/.nvm/versions/node/v6.11.0/bin/node]
 2: node::FatalException(v8::Isolate*, v8::Local<v8::Value>, v8::Local<v8::Message>) [/Users/*****/.nvm/versions/node/v6.11.0/bin/node]
 3: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [/Users/*****/.nvm/versions/node/v6.11.0/bin/node]
 4: v8::internal::Factory::NewRawTwoByteString(int, v8::internal::PretenureFlag) [/Users/*****/.nvm/versions/node/v6.11.0/bin/node]
 5: v8::internal::Runtime_StringBuilderJoin(int, v8::internal::Object**, v8::internal::Isolate*) [/Users/*****/.nvm/versions/node/v6.11.0/bin/node]
 6: 0x29bfae0092a7


Angular CLI: 1.5.0
Node: 6.11.0
OS: darwin x64
Angular: 5.0.0
... animations, common, compiler, compiler-cli, core, forms
... http, language-service, platform-browser
... platform-browser-dynamic, router

@angular/cli: 1.5.0
@angular-devkit/build-optimizer: 0.0.32
@angular-devkit/core: 0.0.20
@angular-devkit/schematics: 0.0.35
@ngtools/json-schema: 1.1.0
@ngtools/webpack: 1.8.0
@schematics/angular: 0.1.0
typescript: 2.4.2
webpack: 3.8.1

However, with --build-optimizer=false I can build with --max_old_space_size=6144

> node --max_old_space_size=6144 ./node_modules/@angular/cli/bin/ng build --prod  --build-optimizer=false

Date: 2017-11-04T23:08:35.111Z
Hash: afdf23d865ad76d6c3a7
Time: 252357ms
chunk {0} 0.5fdb723fb2afdf8bc41a.chunk.js (common) 657 kB  [rendered]
chunk {1} 1.9d7656022b2d3d139dc3.chunk.js () 9.23 MB  [rendered]
chunk {2} 2.0fd8e1693141dc5c5223.chunk.js () 3.15 MB  [rendered]
chunk {3} main.46f18abd9489952e107d.bundle.js (main) 3.43 MB [initial] [rendered]
chunk {4} polyfills.0acd42e0b1fa025cc794.bundle.js (polyfills) 222 kB [initial] [rendered]
chunk {5} styles.27285cfcd50901377a47.bundle.css (styles) 583 kB [initial] [rendered]
chunk {6} vendor.6b76d39b8f45ef045486.bundle.js (vendor) 1.1 MB [initial] [rendered]
chunk {7} inline.e8ad567f621efe38acfe.bundle.js (inline) 1.52 kB [entry] [rendered]

Tests with angular-cli 1.5

ng build --prod build-optimizer=false --sourcemaps=false (I canceled after 12 minutes running)

node --max_old_space_size=4096 node_modules/@angular/cli/bin/ng build --prod build-optimizer=false --sourcemaps=false (work after 4,2 minutes)

@filipesilva

@filipesilva, @hansl, Before anything, thanks so much for this project- Angular has been great for our team. We love the framework and we love the CLI so thanks for all your hard work.

This memory issue seems like it is becoming a real problem. I'm looking for some messaging from the Angular team around expectations here, and I'd be happy to give whatever data is necessary to help move it forward.

Our team is an enterprise team with a medium-large app. We ran into this memory issue a while back, and upped the max_old_space_size to 1GB and then later 2GB. Each time it solved our problem for a few months. But as the app continues to grow we've run into it again now and we can't do this forever. Our toolchain uses Bitbucket pipelines, which like many other CI's maxes out available memory (3GB is our max given our other needs), so we're beginning to run up against a wall.

I've been taking some benchmarks with our app, it looks like you're correct in your post above: 1.5.0, with --build-optimizer=false uses equivalent memory to 1.4.5.

Our app has ~180 components and ~30 services across ~22 modules. I'd like to think it is constructed in a reasonably smart manner, following best practices and general clean code design. The build --aot --build-optimizer=false --sourcemaps=false for this project using 1.5.0 now takes about 2925 GB of memory.

Two general questions for you and others, about (1) expectations and (2) short-term alleviation:

(1) Is Angular built for enterprise development? If so, this issue seems to be of the highest priority and the community deserves more frequent and clear messaging on this. If not, your users need to know this because lots of decisions across hundreds of companies ride on that.

(2) Are there any recommendations from the Angular team about ways to alleviate this issue?

Should we split out our modules into "pre-compiled" libraries?
Is memory usage particularly worse for high Component counts, high Service counts, etc?
Other flags that help trade memory for build-time?

Again, I'd be happy to test any theories/provide data about our large application to help get to the bottom of this, and again, thanks for all the hard work you've done on this.

In our case, the build optimizer nor the sourcemaps flags were effective. Downgrading to @angular/cli 1.4.9 fixed it.

Worked without either flag set false:

ng build -prod

@angular/cli: 1.4.9
node: 6.9.2
os: linux x64
@angular/common: 4.4.6
@angular/compiler: 4.4.6
@angular/core: 4.4.6
@angular/forms: 4.4.6
@angular/http: 4.4.6
@angular/platform-browser: 4.4.6
@angular/platform-browser-dynamic: 4.4.6
@angular/router: 4.4.6
@angular/cli: 1.4.9
@angular/compiler-cli: 4.4.6
typescript: 2.3.4

DIDN'T Work even with both flags set false:

ng build -prod --build-optimizer=false --sourcemaps=false
Angular CLI: 1.5.0
Node: 6.9.2
OS: linux x64
Angular: 4.4.6
... common, compiler, compiler-cli, core, forms, http
... platform-browser, platform-browser-dynamic, router
... tsc-wrapped

@angular/cli: 1.5.0
 @angular-devkit/build-optimizer: 0.0.32
 @angular-devkit/core: 0.0.20
 @angular-devkit/schematics: 0.0.35
 @ngtools/json-schema: 1.1.0
 @ngtools/webpack: 1.8.0
 @schematics/angular: 0.1.0
typescript: 2.3.4
webpack: 3.8.1

...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

We have not tried the hacks above on changing the old space memory... our application isn't apparently as big as some of the others on this thread, but yet fails on cli 1.5.0.

@filipesilva

@filipesilva: --build-optimizer=false and --sourcemaps=false works for me with cli 1.5.
Without them, I was getting "JavaScript heap out of memory".
Thanks for your help!

Happens for me also. Wired part is error thrown after bundle files generated.

Angular CLI: 1.5.0
Node: 8.9.0
OS: win32 x64
Angular: 5.0.0

<--- Last few GCs --->

[12192:00000000004C39A0] 236834 ms: Mark-sweep 1420.3 (2200.9) -> 1420.3 (2159.4) MB, 578.6 / 0.0 ms last resort GC in old space requested
[12192:00000000004C39A0] 237461 ms: Mark-sweep 1420.3 (2159.4) -> 1420.3 (2136.9) MB, 627.2 / 0.0 ms last resort GC in old space requested

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 000000B4C7AA5EC1
1: DoJoin(aka DoJoin) [native array.js:~95] [pc=0000001391AFE299](this=000001A846C82311 ,p=00000050BD62B091 <JSArray[576]>,q=576,E=000001A846C823B1 ,A=000000B4C7AEEC69 <String[1
]: >,z=000001A846C82421 )
2: Join(aka Join) [native array.js:~120] [pc=0000001390C722A9](this=000001A846C82311 ,p=00000050BD62B091 <JSArray[576]>,q=576,A=000000B4C7AEEC69 <Str...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
npm ERR! code ELIFECYCLE
npm ERR! errno 3
npm ERR! mobx-ui-test-app@0.0.0 build:prod: ng build --env=prod --prod --aot --stats-json --named-chunks=true --sourcemaps=false
npm ERR! Exit status 3

==== update ======
Issue for me is --stats-json. Build is fine after remove this flag.

Optimizations are welcome, but on the other hand memory is cheap. Until optimizations are here run the build using

node --max-old-space-size=8192 ./node_modules/@angular/cli/bin/ng build

and append whatever switches you'd like to the end. To make it easier to run, edit your package.json file and add this line to "scripts":

"serve": "node --max-old-space-size=8192 ./node_modules/@angular/cli/bin/ng serve --aot"

Then you can execute it using

npm run serve

I'm an old Java dev, so this seems familiar to me somehow :)

@swftvsn

@swftvsn, memory is cheap if you have control over it, but we deploy to a Docker swarm, and containers have limited memory. This is a pretty serious issue for us. I'd prefer a slower build if that means caching stuff to disk and not running out of memory.

I am getting this issue after upgrading to cli 1.5 and angular 5.0.

I want to highlight the question above:

Is Angular built for enterprise development?

I am so confused why any compiler would need 4 gigs of memory to compile a web app..... To get it to compile aot post upgrade I had to jack the memory up to 4 gigs. Our app has 2 feature module libraries each with about 15 modules, ~60 modules in the main app, and ~15 services. We have a lot left to build as well.

I would imagine it has to do with the obscene amount of dependencies + all the hoops that are done to statistically analyze all the dependencies and your code in order to shake the trees. Take a look at node_modules if you dare. (And source / debug maps and things I don't even know about)

Excessive memory consumption is usually caused by a lot of data and/or abstraction over abstraction over abstraction over... :) Or a bug.

For now I'm happy and I just feed the beast. I do hope it looses some of it's appetite in the coming versions though..

@filipesilva

I'm less concerned about "how could this possibly...?" than I am about the lack of communication.

The compilation itself is doing some pretty heavy lifting right now. It's really great actually. The entire CLI is. It bundles this thing up and serves it pretty compactly. In fact his may be a testament to just how easy Angular makes it to build large scale web apps... their size is outpacing the toolchain.

I'm just looking for the Angular team to set expectations here, that's all:

Does the Angular team consider this a problem? (its been tagged as "required" since March)
Is there any action being taken to fix it? (its been tagged as "investigation" since March)
Is this a fixable issue in the next month? 6 months? year?
If it's not a fixable issue, are there any other build configurations/toolchains they'd recommend?
Are they still researching the answers to the above questions?

Any answer to those questions is okay... but it's important that those answers (and their progress) are communicated to the community in an iterative fashion. My team and many other teams need to make decisions based off those answers, and right now we feel frozen.

This is an open source project that we're all getting for free. It's added tremendous value already even with this issue; so some people are being awfully critical given this. At the same time, open communication is integral to the success of solid open source projects: we're looking to the Angular team for that leadership.

@filipesilva , @hansl?

@dpxxdp

That's an excellent summary @dpxxdp !

Recent upgrade our project to Angular 5 and Angular CLI 1.5.
We have 100+ components and 20+ modules. The build --prod is taking forever and finally throw heap out of memory exception which is very frustrating!

As I mentioned here a few months ago, I have a very large project with several modules too, +120 pages, +500 components (pages are divided into at least 3 components like grid, form).

To get ng build -prod to work I had to test on a machine with 32 Gb of RAM, nodeJS consumed 28 Gb to compile the project. Impressive.

The solution was actually put 32 Gb as virtual memory on the machine (swap).
In addition, of the already known "--max_old_space_size".

The label of this issue should be priority: 1 (urgent) rather than priority: 2 (required)

@swftvsn

@swftvsn I am still not getting it. Maybe it is because I come from .NET where the c# compiler would tear through a project with comparable amounts of classes without breaking a sweat.

@tolitech This is exactly the main point. If we need machines with 32 GB of ram to build the project, some cloud agents won't be able to handle this/aren't configurable to upgrade the hardware. Thus, we would need to make architectural decisions to start building segmented SPAs to keep the size of a project down, catering to compiler limitations.

@dpxxdp While I do agree that it is great that it is free and open source (thanks as well team!), I don't feel that it is a criticism to report a major bug/functional limitation that requires a hardware upgrade as a work around. I don't think anyone is trying to criticize the work that was done, but rather increase the priority on this issue.

guys 2 workaround for heap problem

use custom build with webpack
currently default build use several optimization step + minification.
use command like
ng build --app app -aot=true -e=prod -sm=false -ec=true --named-chunks=false -oh=all -pr=false
this will generate bundles without minification and general size for vendor bundle will increased in my case for 8%. Build time decreased for twice, but minification step take long time 6-7 min in CI machine (2gb). I use gulp for post build actions in my case its uglify js and postcss.

My app
~72 services (CRUD operation + some general logic)
~10 pipe
~15 directive
~170 components
~ 80 model class (data from API)
~ 15 module

as for me i want some flags to set optimization step count, tree shake deep (function, class, variable)

Hi all,

I've bumped the priority on this issue and will be working on it. We didn't consider it to be as urgent before because the solution at the time was to increase the available memory to node, which defaults to 1.7gb. So it was expected that at some point projects would need more memory than the default.

However with the release of 1.5 the memory requirements seem to have increased dramatically. Projects that didn't have any memory problems before now do. 32gb is definitely a ridiculous amount of RAM and shouldn't be needed.

At this point I'm not sure what caused the memory spike but have a few hypothesis. With CLI1.5 + Angular 5 we use a new pipeline that does better type checking by using a full TS program. Maybe there's a bad cache somewhere in there, or the TS program is taking a lot of memory. Another possibility is Uglify, which we had to upgrade as well. Or it might be something in the AOT compiler itself. We also default build optimizer to true now and it's not really optimized for neither speed nor memory.

If you're running into this problem and have time, I'd appreciate if you could answer a couple questions:

What version of the CLI and Angular are you on?
Does this only happen on ng build --prod?
Do you still get the memory error with these build commands:
- ng build --prod --build-optimizer=false
- ng build --aot
- ng build --aot --build-optimizer
If you have a project that I can look at, can you link it please?

In response to above comment:
CLI 1.5.0, Angular 5.0.1

Project:
Services: 9
Components: 43
Models: 28
Lines of ts: 8200
Lines of sass: 5000

Heap error happens with:
ng build --prod: yes
ng build --build-optimizer=false: no
ng build --aot: no
ng build --aot --build-optimizer: no

@filipesilva

@filipesilva
About 20 modules, 30 services, 80 components

CLI 1.4.4, Angular 4.4.4
Only happens when building with --prod, --aot, and --sourcemap (we need it in prod...)
If settings memory to 8gb, we can actually use --prod, --aot, --sourcemap, AND --build-optimizer

Can't show the repo unfortunately

@filipesilva

@filipesilva

What version of the CLI and Angular are you on?
1.5.0 angular 5.0.1
Does this only happen on ng build --prod?
yes
Do you still get the memory error with these build commands:
ng build --build-optimizer=false no
ng build --aot no
ng build --aot --build-optimizer no

@filipesilva

@filipesilva
Windows 7 Pro 64-bit
12 Gigs of ram
CLI 1.5.0, Angular 5.0.0

Project details:
~120 components
15 modules

Do you still get the memory error with these build commands:
ng build --prod: yes
ng build --build-optimizer=false: no
ng build --aot: no
ng build --aot --build-optimizer: yes

On Angular 4.3
with ~180 components
with ~30 services
with ~22 modules

With CLI 1.5
Does this only happen on ng build --prod? Yes

ng build --build-optimizer=false less than 2GB
ng build --aot --build-optimizer=false ~2.8GB
ng build --aot over 3 GB

With CLI 1.4.5
Does this only happen on ng build --prod? Yes

ng build --build-optimizer=false less than 2GB
ng build --aot --build-optimizer=false ~2.8GB
ng build --aot ~2.8 GB

Our repo is private unfortunately

CLI 1.5.0, Angular 5.0.0

Modules: ~50
Services: ~110
Components: ~290

ng build --prod: fail
ng build --prod --build-optimizer=false: fail
ng build --prod --build-optimizer=false --sourcemaps=false: fail
ng build --prod --aot=false: works
ng build: works
ng build --aot=true: works
ng build --aot=true --build-optimizer=true: fail

What version of the CLI and Angular are you on?
@angular/cli@1.5.0 , @angular 5.0

Does this only happen on ng build --prod?
yes

Do you still get the memory error with these build commands:
ng build --build-optimizer=false : NO
ng build --aot : NO
ng build --aot --build-optimizer: NO

@filipesilva

Under cli1.5 & angular5

Ng build —prod : YES
Ng serve —aot : first time NO but after editing file YES it throw error out of memory... blockin at 0%

Ng serve : NO all’s okay and fast (but wait a lot to show result in browser because of JiT compiler)

Cc @filipesilva

The versions I am using
CLI - 1.2.0 and Angular 4.2.5
Modules: ~ 130
Components: ~ 200
Services: ~ 70

getting the build issues with below options.

ng build --prod: fail
ng build --aot=true: fail
ng build --aot=true --sourcemaps=false : Works most of the times and very often it is failing.
ng build --prod --aot=false : works

@cecotw sarcasm is sometimes misinterpreted. And I'm not here to fire up any flamewars, your ecosystem is obviously the better :)

I had experienced the issue FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory with ng build --prod. I have solved the problem by using
node --max_old_space_size=8192 node_modules/@angular/cli/bin/ng build --prod

You can add also in

package.json

as

"scripts": {
    "prod": "node --max_old_space_size=8192 node_modules/@angular/cli/bin/ng  build --prod",
}
npm run prod

What version of the CLI and Angular are you on?

Angular CLI: 1.5.0
Node: 8.5.0
OS: darwin x64
Angular: 5.0.0
... animations, common, compiler, core, forms, http
... platform-browser, platform-browser-dynamic, router

@angular/cdk: 5.0.0-rc0
@angular/cli: 1.5.0
@angular/compiler-cli: 5.0.1
@angular/flex-layout: 2.0.0-beta.10
@angular/language-service: 5.0.1
@angular/material: 5.0.0-rc0
@angular-devkit/build-optimizer: 0.0.32
@angular-devkit/core: 0.0.20
@angular-devkit/schematics: 0.0.35
@ngtools/json-schema: 1.1.0
@ngtools/webpack: 1.8.0
@schematics/angular: 0.1.2
typescript: 2.4.2
webpack: 3.8.1

Does this only happen on ng build --prod? NOT SURE
Do you still get the memory error with these build commands:

ng build --build-optimizer=false WORKS
ng build --aot FAILS
ng build --aot --build-optimizer FAILS

also:

ng build --aot=true --sourcemaps=false WORKS
ng build --prod -sourcemaps=false FAILS
ng build --prod FAILS
ng build --prod --aot=flase --sourcemaps=false FAILS

If you have a project that I can look at, can you link it please?

https://github.com/Alfresco/alfresco-ng2-components/tree/development

we have a gitter channel if you need help to build our project and have a look:

https://gitter.im/Alfresco/alfresco-ng2-components

Same issue. Upgraded to A5. ng build --prod results in out of memory error below. The error does not occur without --prod. Increasing the --max_old_space_size is not really the answer, something is fundamentally wrong and has stopped our A5 upgrade going anywhere.

Please fix!

<--- Last few GCs --->

243344 ms: Mark-sweep 1000.9 (1434.4) -> 1003.2 (1434.4) MB, 452.4 / 0.0 ms [allocation failure] [scavenge might not succeed].
243760 ms: Mark-sweep 1003.2 (1434.4) -> 1003.2 (1434.4) MB, 416.3 / 0.0 ms [allocation failure] [scavenge might not succeed].
244187 ms: Mark-sweep 1003.2 (1434.4) -> 1012.9 (1418.4) MB, 427.0 / 0.0 ms [last resort gc].
244591 ms: Mark-sweep 1012.9 (1418.4) -> 1022.8 (1418.4) MB, 403.8 / 0.0 ms [last resort gc].

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x957966cfb39
1: DoJoin(aka DoJoin) [native array.js:~129] [pc=0xbe4173aee59] (this=0x95796604381 ,w=0x1ca9328ff671 <JS Array[939]>,x=939,N=0x957966043c1 ,J=0x957966ae4b9 <String[1]: >,I=0x957966b46c9 <JS Function ConvertToString (SharedFunctionInfo 0x95796652dc9)>)
2: Join(aka Join) [native array.js:180] [pc=0xbe419f08ab2] (this=0x95796604381 ,w=0x1ca9328ff671 <JS Array[9...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
1: node::Abort() [/usr/local/bin/node]
2: node::FatalException(v8::Isolate*, v8::Localv8::Value, v8::Localv8::Message) [/usr/local/bin/node]
3: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [/usr/local/bin/node]
4: v8::internal::Factory::NewRawTwoByteString(int, v8::internal::PretenureFlag) [/usr/local/bin/node]
5: v8::internal::Runtime_StringBuilderJoin(int, v8::internal::Object**, v8::internal::Isolate*) [/usr/local/bin/node]
6: 0xbe4066092a7
Abort trap: 6

What version of the CLI and Angular are you on?

@angular/cli: 1.4.9
node: 6.11.0
os: win32 x64
@angular/animations: 4.4.6
@angular/cdk: 2.0.0-beta.12
@angular/common: 4.4.6
@angular/compiler: 4.4.6
@angular/core: 4.4.6
@angular/flex-layout: 2.0.0-beta.9
@angular/forms: 4.4.6
@angular/http: 4.4.6
@angular/material: 2.0.0-beta.12
@angular/platform-browser: 4.4.6
@angular/platform-browser-dynamic: 4.4.6
@angular/router: 4.4.6
@angular/cli: 1.4.9
@angular/compiler-cli: 4.4.6
typescript: 2.6.1

ng build --prod - Success
ng build --build-optimizer=false - Success
ng build --aot - Success
ng build --aot --build-optimizer - OOME

A sanity-check local run of ng serve --prod --aot gave me failure at a later time

Date: 2017-11-12T22:52:55.879Z
Hash: 456f57f84be9de32704e
Time: 185523ms
chunk {0} polyfills.5dbd24b37bdff70c0320.bundle.js (polyfills) 66.1 kB {4} [initial] [rendered]
chunk {1} main.a2ba9af755957e20aa31.bundle.js (main) 1.98 MB {3} [initial] [rendered]
chunk {2} styles.2dfc00976daefa57d973.bundle.css (styles) 80.3 kB {4} [initial][rendered]
chunk {3} vendor.c5146d2ca93aff011cac.bundle.js (vendor) 1.51 MB [initial] [rendered]
chunk {4} inline.7e6f8c72074764ba2a48.bundle.js (inline) 1.45 kB [entry] [rendered]

webpack: Compiled successfully.

<--- Last few GCs --->

  277448 ms: Mark-sweep 1225.3 (1434.4) -> 1225.3 (1434.4) MB, 1086.6 / 0.1 ms [allocation failure] [scavenge might not succeed].
  278456 ms: Mark-sweep 1225.3 (1434.4) -> 1225.3 (1434.4) MB, 1008.4 / 0.1 ms [allocation failure] [scavenge might not succeed].
  279488 ms: Mark-sweep 1225.3 (1434.4) -> 1236.2 (1418.4) MB, 1030.8 / 0.1 ms [last resort gc].
  280607 ms: Mark-sweep 1236.2 (1418.4) -> 1247.1 (1418.4) MB, 1117.0 / 0.1 ms [last resort gc].

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 00000126BFCCFB49 <JS Object>
    1: DoJoin(aka DoJoin) [native array.js:~129] [pc=000003FAD779BBD7] (this=00000126BFC04381 <undefined>,w=0000033E0E885D01 <JS Array[334]>,x=334,N=00000126BFC043C1 <true>,J=00000126BFCAE4C9 <String[1]:  >,I=00000126BFCB46D9 <JS Function ConvertToString (SharedFunctionInfo 00000126BFC52DC9)>)
    2: Join(aka Join) [native array.js:180] [pc=000003FAD73F9FB2] (this=00000126BFC04381 <undefined>...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

In my project everything was working fine in --prod --aot mode up until a day ago when I decided to migrate to HttpClient to prepare for the Angular 5 / CLI 1.5 switch. No version changes, no CLI changes were involved.

I upgraded my services and moved some error/auth logic to a couple of interceptors. My vendor dev bundle went from 15MB to 6MB and I was all impressed. Then I decided to sanity check my prod aot build with ng serve --prod --aot and I started getting the OOME.

Seeing the same issue after upgrading to cli 1.5, Angular 1.5 and HttpClient

$ ng build --aot --prod
95% emitting
<--- Last few GCs --->

224719 ms: Mark-sweep 1304.7 (1434.7) -> 1304.7 (1434.7) MB, 485.6 / 0.0 ms [allocation failure] [scavenge might not succeed].
225206 ms: Mark-sweep 1304.7 (1434.7) -> 1304.7 (1434.7) MB, 487.0 / 0.0 ms [allocation failure] [scavenge might not succeed].
225685 ms: Mark-sweep 1304.7 (1434.7) -> 1304.7 (1418.7) MB, 479.3 / 0.0 ms [last resort gc].
226180 ms: Mark-sweep 1304.7 (1418.7) -> 1304.7 (1418.7) MB, 494.5 / 0.0 ms [last resort gc].

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x3e4edad3fa99
1: DoJoin(aka DoJoin) [native array.js:~129] [pc=0xd5adf91512e] (this=0x3e4edad04241 ,w=0x21eed50e5b41 <JS Array[754]>,x=754,N=0x3e4edad04281 ,J=0x3e4edad4a369 <String[1]: >,I=0x3e9e792bc1e9 <JS Function ConvertToString (SharedFunctionInfo 0x3e4edad5dbc9)>)
2: Join(aka Join) [native array.js:180] [pc=0xd5ac9783c52] (this=0x3e4edad04241 ,w=0x21eed50e5b41 <JS ...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
1: node::Abort() [/usr/local/bin/node]
2: node::FatalException(v8::Isolate*, v8::Localv8::Value, v8::Localv8::Message) [/usr/local/bin/node]
3: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [/usr/local/bin/node]
4: v8::internal::Factory::NewRawTwoByteString(int, v8::internal::PretenureFlag) [/usr/local/bin/node]
5: v8::internal::Runtime_StringBuilderJoin(int, v8::internal::Object**, v8::internal::Isolate*) [/usr/local/bin/node]
6: 0xd5ac97060c7
7: 0xd5adf91512e
Abort trap: 6
Jays-MacBook-Pro:ngx-dynamic-dashboard-framework jayhamilton$ ng build --aot --prod
95% emitting
<--- Last few GCs --->

224540 ms: Mark-sweep 1305.3 (1444.7) -> 1305.3 (1444.7) MB, 476.2 / 0.0 ms [allocation failure] [scavenge might not succeed].
225007 ms: Mark-sweep 1305.3 (1444.7) -> 1305.3 (1444.7) MB, 466.7 / 0.0 ms [allocation failure] [scavenge might not succeed].
225479 ms: Mark-sweep 1305.3 (1444.7) -> 1305.3 (1428.7) MB, 471.9 / 0.0 ms [last resort gc].
225970 ms: Mark-sweep 1305.3 (1428.7) -> 1305.3 (1428.7) MB, 491.1 / 0.0 ms [last resort gc].

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x67c5bb3fa99
1: DoJoin(aka DoJoin) [native array.js:~129] [pc=0x337898eca1ce] (this=0x67c5bb04241 ,w=0x3ba9f97f5a79 <JS Array[754]>,x=754,N=0x67c5bb04281 ,J=0x67c5bb4a369 <String[1]: >,I=0x2b42504bc1e9 <JS Function ConvertToString (SharedFunctionInfo 0x67c5bb5dbc9)>)
2: Join(aka Join) [native array.js:180] [pc=0x33787d083c52] (this=0x67c5bb04241 ,w=0x3ba9f97f5a79 <JS Arra...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
1: node::Abort() [/usr/local/bin/node]
2: node::FatalException(v8::Isolate*, v8::Localv8::Value, v8::Localv8::Message) [/usr/local/bin/node]
3: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [/usr/local/bin/node]
4: v8::internal::Factory::NewRawTwoByteString(int, v8::internal::PretenureFlag) [/usr/local/bin/node]
5: v8::internal::Runtime_StringBuilderJoin(int, v8::internal::Object**, v8::internal::Isolate*) [/usr/local/bin/node]
6: 0x33787d0060c7
7: 0x337898eca1ce
Abort trap: 6

PACKAGE.JSON
"dependencies": {
"@angular/animations": "^5.0.1",
"@angular/cdk": "2.0.0-beta.12",
"@angular/common": "^5.0.1",
"@angular/compiler": "^5.0.1",
"@angular/core": "^5.0.1",
"@angular/forms": "^5.0.1",
"@angular/http": "^5.0.1",
"@angular/material": "2.0.0-beta.12",
"@angular/platform-browser": "^5.0.1",
"@angular/platform-browser-dynamic": "^5.0.1",
"@angular/router": "^5.0.1",
"@swimlane/ngx-charts": "6.1.0",
"core-js": "^2.4.1",
"d3": "^4.4.0",
"jquery": "2.2.0",
"ng2-cookies": "1.0.3",
"ng2-dnd": "^2.1.1",
"reflect-metadata": "0.1.8",
"rxjs": "^5.5.2",
"semantic-ui": "2.2.2",
"socket.io-client": "^1.7.2",
"sockjs-client": "^1.1.4",
"stompjs": "^2.3.3",
"zone.js": "^0.8.4"
},
"devDependencies": {
"@angular/cli": "1.5.0",
"@angular/compiler-cli": "^5.0.1",
"@types/jasmine": "2.5.38",
"@types/node": "~6.0.60",
"codelyzer": "~2.0.0",
"jasmine-core": "~2.5.2",
"jasmine-spec-reporter": "~3.2.0",
"karma": "~1.4.1",
"karma-chrome-launcher": "~2.0.0",
"karma-cli": "~1.0.1",
"karma-coverage-istanbul-reporter": "^0.2.0",
"karma-jasmine": "~1.1.0",
"karma-jasmine-html-reporter": "^0.2.2",
"protractor": "~5.1.0",
"ts-node": "~2.0.0",
"tslint": "~4.5.0",
"typescript": "~2.6.1"
}

EDIT working with the --max_old_space_size=8192 fix above**

Also getting the exact same. Angular 4 build time, 2 minutes maybe? New angular 5 build time is 25 to fail with:

ng build --prod --output-path=dist/client

92% chunk asset optimization
<--- Last few GCs --->

1327460 ms: Mark-sweep 1306.9 (1405.9) -> 1306.9 (1414.9) MB, 1826.0 / 0.0 ms [allocation failure] [GC in old space requested].
1329363 ms: Mark-sweep 1306.9 (1414.9) -> 1306.9 (1414.9) MB, 1903.1 / 0.0 ms [allocation failure] [GC in old space requested].
1331374 ms: Mark-sweep 1306.9 (1414.9) -> 1313.4 (1405.9) MB, 2010.3 / 0.0 ms [last resort gc].
1333333 ms: Mark-sweep 1313.4 (1405.9) -> 1319.9 (1405.9) MB, 1958.0 / 0.0 ms [last resort gc].

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 000002E7514CF791
2: with_parens [000002E751404381 :~5813] [pc=000002F02F5400F6] (this=000001E1DBFE4699 <an Object with map 000001764956B2F9>,cont=000000E96E71C3A9 <JS Function (SharedFunctionInfo 0000030749384359)>)
3: /* anonymous /(aka / anonymous */) [000002E751404381 :6817] [pc=000002F02FE08F44] (this=000002E751404381 ,self=00000314C2BB6571 <an AST_Call with map ...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

Any update on this? This issue was raised back in March and is now manifesting itself in Angular 5 (and 4) prod builds. Seems somehow to be related to HttpClient.

Please can you investigate as a priority as we can not go to production using Angular 5, which is currently not fit for purpose.

@JonathanYates

@JonathanYates #5618 (comment)

Yes I know, that was 5 days ago. I am asking for an update.

@filipesilva

Same issues, should build app without problems with ng build --prod

@filipesilva: I'm asking for an update date.

@andremarcondesteixeira

@andremarcondesteixeira we will all do what we please. This isn't the place to try to convince anybody to use another framework.

@JonathanYates

@JonathanYates @MichalEmbiq @andremarcondesteixeira - Have you guys tried what @filipesilva suggested in #5618 (comment)? That seems to have worked for most people and it also worked for me. Seems like a sufficient work-around until the real issue is solved.

Navigation Menu

Repository navigation

AOT build fails due to "JavaScript heap out of memory" #5618

Bug Report or Feature Request (mark with an `x`)

Versions.

Repro steps.

The log given by the failure.

Desired functionality.

Mention any other details that might be useful.

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

AOT build fails due to "JavaScript heap out of memory" #5618

Description

Bug Report or Feature Request (mark with an x)

Versions.

Repro steps.

The log given by the failure.

Desired functionality.

Mention any other details that might be useful.

Activity

marcalj commented on Apr 3, 2017

Ristaaf commented on Apr 19, 2017

omeralper commented on Apr 19, 2017

Ristaaf commented on Apr 21, 2017

KeithGillette commented on May 4, 2017

speedy32 commented on May 26, 2017

dakotamurphyucf commented on May 28, 2017

isaacplmann commented on Jun 1, 2017

zackschuster commented on Jun 1, 2017

sis0k0 commented on Jun 6, 2017

zackschuster commented on Jun 6, 2017

aminebizid commented on Jun 6, 2017

zackschuster commented on Jun 6, 2017

sis0k0 commented on Jun 6, 2017

tgrosh commented on Jun 15, 2017

elvisbegovic commented on Jul 9, 2017

cassmtnr commented on Jul 10, 2017

aminebizid commented on Jul 10, 2017

cimachris commented on Jul 13, 2017

Masquer commented on Jul 13, 2017

cimachris commented on Jul 13, 2017

cassmtnr commented on Jul 13, 2017

edmund-lee commented on Jul 14, 2017

xnnkmd commented on Jul 14, 2017

xnnkmd commented on Jul 14, 2017

emazv72 commented on Jul 26, 2017

emazv72 commented on Jul 26, 2017

zackschuster commented on Jul 26, 2017

RuedigerMoeller commented on Aug 6, 2017

baio commented on Aug 8, 2017

etiennetalbot commented on Aug 8, 2017

amikitevich commented on Aug 10, 2017

brunobertechini commented on Aug 10, 2017

blyndcide commented on Aug 10, 2017

wojciech-wator commented on Aug 11, 2017

josephliccini commented on Aug 11, 2017

williangd commented on Aug 14, 2017

ChenReuven commented on Aug 15, 2017

jloosli commented on Aug 15, 2017

Dunos commented on Aug 16, 2017

AlekBro commented on Aug 17, 2017

sbabeal commented on Aug 18, 2017

sbabeal commented on Aug 18, 2017

josephliccini commented on Aug 18, 2017

chriseenberg commented on Aug 18, 2017

sbabeal commented on Aug 18, 2017

phamtuananh791994 commented on Aug 21, 2017

linkevin777 commented on Aug 21, 2017

loic49 commented on Aug 21, 2017

Masquer commented on Aug 22, 2017

linkevin777 commented on Aug 22, 2017

randyaa commented on Aug 22, 2017

Masquer commented on Aug 22, 2017

mgechev commented on Aug 24, 2017

mattem commented on Aug 24, 2017

timwright35 commented on Aug 24, 2017

bgever commented on Aug 25, 2017

bgever commented on Aug 25, 2017

eduardotolino commented on Aug 31, 2017

joecot commented on Aug 31, 2017

eduardotolino commented on Aug 31, 2017

eduardotolino commented on Sep 1, 2017

bgever commented on Sep 1, 2017

xnnkmd commented on Sep 1, 2017

eduardotolino commented on Sep 1, 2017

Dunos commented on Sep 1, 2017

bjornharvold commented on Sep 7, 2017

elvisbegovic commented on Sep 7, 2017

gtarun commented on Sep 8, 2017

hugodes commented on Sep 14, 2017

alexzuza commented on Sep 14, 2017

Bug Report or Feature Request (mark with an `x`)