Incremental build performance dropped

Hi,

I noticed that our incremental builds started to be considerably slower than they used to, so I ran webpack with --profile --progress and older versions of ts-loader to see if there are differences. It seems that v0.5.2 is twice as fast as the latest version. I'm unsure how to interpret the profiling data and the webpack source code is a complete mess, so it doesn't help much in that regard.

Here are the stats:

// ts-loader v0.5.2
379ms build modules   
19ms seal
39ms optimize
60ms hashing
35ms create chunk assets
2ms additional chunk assets 
0ms optimize chunk assets 
0ms optimize assets 
99ms emit
Hash: fdd096aa48ded051dda3
Version: webpack 1.12.2
Time: 639ms

// ts-loader v0.5.6
357ms build modules   
19ms seal
50ms optimize
51ms hashing
40ms create chunk assets
4ms additional chunk assets 
0ms optimize chunk assets 
826ms optimize assets
78ms emit
Hash: fdd096aa48ded051dda3
Version: webpack 1.12.2
Time: 1429ms

It looks like there is a lot of time spent in "optimize assets". I don't know what's causing this, but I have a feeling that ts-loader is using some extra dependencies now which may have the side-effect of enabling some webpack optimizations that are meant for production, not development.

Hmm... interesting. Thanks for reporting, I'll take a look.

I did a little profiling with node-inspector and it seems that time is spent calling languageService.getSemanticDiagnostics(). Can't figure out from the history, if it's called more often than it's supposed to be or if it's a side-effect of some other change. Any ideas?

Could be this commit: dc53ef6

Before it was checking for "visited" modules.

EDIT: it runs it for every TS file in the project and even lib.d.ts

Yea, that was my thought as well. It was put in place to fix this but I think #68 is the way to go and there is a decent chance that it will be more performant.

Yes, seems a reasonable assumption. Also, in my case there were only ~20 files or so, because the rest are still ye olde' JS files, but in a larger project, this could sum up considerably to the overall incremental build time.

We ran into this very recently. When bundling very large projects with ts-loader, it takes a very long time, which makes iterative development very difficult. tsc doesn't take nearly so long so we've even considered using a gulp task or npm script to pipe tsc first and then webpack the js files just to speed things up. If this can be solved in ts-loader we would be happy! 😀

@use-strict

@use-strict I'm working on reproducing locally. I'm seeing something similar to you, but I just wanted to confirm whether or not the numbers you posted were for the initial build or for a change in watch mode. Here's some numbers that I'm seeing:

// initial build
2589ms build modules    
14ms seal
5ms optimize 
14ms hashing
19ms create chunk assets
4ms additional chunk assets 
0ms optimize chunk assets 
784ms optimize assets
7ms emit 
Hash: 58c08445fb896261fe7e
Version: webpack 1.12.2
Time: 3460ms

// after making a change while in watch mode
85ms build modules    
4ms seal 
5ms optimize 
10ms hashing
11ms create chunk assets
4ms additional chunk assets 
0ms optimize chunk assets 
805ms optimize assets
11ms emit
Hash: 1b9954af6b2aacc4d866
Version: webpack 1.12.2
Time: 1023ms

In other words, for the initial build both "build modules" and "optimize assets" takes a long time, but a change in watch mode results in a small "build modules" but still a large "optimize assets"

Alright, so after playing with this a little bit more my hopes at it being a quick-fix are gone. I was able to get the ~800ms "optimize assets" down to ~500ms in my test. That time is spent generating errors, and I'm not sure how much more I can reduce that (but I haven't given up yet).

However, I did want to point out there are multiple scenarios & timings to take into account here. There is the "first build" vs an "incremental build" in watch mode, and there is the time to emit output for the modules and time to generate errors. As things stand right now the first build scenario performance is pretty abysmal. In my test it takes 2600ms to generate modules for the first build, but only 85ms to generate modules for an incremental build. At my job we have a project that takes around 20s to generate the first build but less than a second for incremental builds. The point here is that if you're not already using watch mode I highly suggest you do so. That said, I think there are definitely performance gains to be made for the first build and I will look deeper into that.

Regarding the time to generate errors, it is a constant for both the first build and incremental builds, but it's also not a great constant.

So to recap:

Scenario	Action	Time	Notes
First Build	Emit	2600ms	This is bad and I think it can be better
First Build	Gen. Errors	500ms	This is also bad but I'm not sure how much I can fix it
Second Build	Emit	85ms	This is working as intended
Second Build	Gen. Errors	500ms	Same as above

Also, I wanted to point out that in my test running tsc takes about 1000ms. So if I can get the first build emit time for my test down to 500ms I would consider that a huge win.

@jbrantly

@jbrantly , initial build time is irrelevant. It's expected to take a long time and it's not a problem as long as I don't have to kill the watch and restart it all the time.

Now some remarks and questions:

v0.5.2 of ts-loader did not have this overhead, provided it was only checking changed files for errors.
Checking ALL of the files should scale linearly with project size, which is very bad for us (and for @SonofNun15 apparently)
Why do we need to check files that have not changed for errors? I know you probably had a good reason for that change, but I need to understand it in the hopes that I can try to provide a better solution. It's also not obvious for me from the linked test.
Wouldn't the fix with program.getSemanticDiagnostics() work?

Checking ALL of the files should scale linearly with project size

You're probably right, but this is something I want to look into more. I don't know if maybe TS has some optimizations in this area that I'm simply missing out on because I'm doing something silly.

Why do we need to check files that have not changed for errors

Consider app.ts -> dep.ts. dep exports something which app uses, but dep changes that export in some way which makes app's usage incompatible. If we only check changed files we won't see the error since app didn't change.

Somewhat related is #52 which deals with the dependency tree. If I'm thinking about things correctly, once that's fixed in the scenario above webpack would consider dep to be a dependency of app, which means if dep changed app would get rebuilt as well. This is bad for timing emit-wise, but could potentially allow going back to the "visited" pattern for diagnostics which might be a net win.

Wouldn't the fix with program.getSemanticDiagnostics() work?

That's what I tried yesterday. It took the time in my test from 800ms to 500ms, so while it was better it was not a magical fix out of the box. But like I mentioned, it's possible I'm doing something silly elsewhere that's causing TS not to be optimized for this case.

FWIW, I'm using this for my test.

Just brainstorming potential fixes here: It might be possible to use transpileModule for all emits while still updating the language service (or just a program). Then after compile get diagnostics like we do today. This could reduce the initial build time, but keep the same amount of time to generate diagnostics. One problem with this approach is that it could fail for some applications since it assumes that isolatedModules is turned on.

I will investigate more today. Looks likeprogram.getSemanticDiagnostics() still iterates through program.getSourceFiles() (https://github.com/Microsoft/TypeScript/blob/master/src/compiler/program.ts#L583-L591)

I'm still not 100% sure of how the entire process works and there is little documentation to go on, so please just go along me:

Webpack starts with an entry point and recursively includes dependencies, following require calls. At least this is what happens for regular *.js files, without any loaders coming into play.

Now if we add ts-loader, this basically means that every *.ts file spawns a ts-loader instance. Even so, ts-loader itself caches a single typescript instance per project. At this point typescript dependencies are different from webpack dependencies.

Typescript needs to follow any dependencies for type-checking, and that's including any files that are not necessarily referenced at runtime via a require. It's these filesthat webpack has no knowledge of.

After ts-loader is done with a file, a *.js file results and webpack follows again the dependencies that are require calls.

For incremental builds, webpack watches all files in the project, so detecting a change is not a problem. The problem is figuring out which files were actually changed since the last incremental build, which webpack cannot help us with because it have no knowledge of them.

So to sum things up, we need to build the list of changed files ourselves, using only information coming from the compiler API and/or filesystem, maybe using webpack just to get the paths that are being watched. And we need to do this faster than it takes to get all errors from all files in the project. Or, we need to find another way of getting the errors in a cheaper way.

Webpack starts with an entry point and recursively includes dependencies, following require calls. At least this is what happens for regular *.js files, without any loaders coming into play.

Yes

Now if we add ts-loader, this basically means that every *.ts file spawns a ts-loader instance.

Roughly. To be pedantic the "loader" function is called for each .ts file.

Even so, ts-loader itself caches a single typescript instance per project.

Yes. (per the "instance" loader option and per compiler instance)

At this point typescript dependencies are different from webpack dependencies.
Typescript needs to follow any dependencies for type-checking, and that's including any files that are not necessarily referenced at runtime via a require. It's these filesthat webpack has no knowledge of.

Right. Simple case would be a .d.ts file which webpack never runs through the loader or attempts to bundle.

After ts-loader is done with a file, a *.js file results and webpack follows again the dependencies that are require calls.

Yup. By the time the second file is put through the loader TypeScript generally has all of the files for the project loaded. This is actually kind of a bad thing though for various reasons.

For incremental builds, webpack watches all files in the project, so detecting a change is not a problem.

Well, to be clear, webpack watches all files that were put through the loader or explicitly added as a dependency using the addDependency API. This can be (and currently is) different from what TypeScript has loaded. Right now we manually add all .d.ts files as dependencies so that they're watched by webpack.

The problem is figuring out which files were actually changed since the last incremental build, which webpack cannot help us with because it have no knowledge of them.
So to sum things up, we need to build the list of changed files ourselves, using only information coming from the compiler API and/or filesystem, maybe using webpack just to get the paths that are being watched.

I'm not sure this is a problem. Webpack can tell us which files were actually changed. For instance it does so today for .d.ts files. The problem is detecting errors in files that were not actually changed but depended on files that were. And webpack could possibly help us there if we instruct it to do so by explicitly adding the deep tree of dependencies that TypeScript knows about to each file.

In other words, instead of calling a blanket getDiagnostics on the entire program, we could potentially build a list of files that changed concatenated with a list of files that depended on those files and call getDiagnostics on those.

As an example, suppose app.ts -> dep.ts.

If dep.ts changes, we need to call getDiagnostics on both app.ts and dep.ts even though only dep.ts changed. However, if app.ts changes then we only need to call getDiagnostics on app.ts. This helps the best-case scenario but doesn't address the worst-case. If you change a file that nothing depends on it will be very fast. But if you change a file that everything depends on then it'll be the same performance as today.

A downside to this approach is that this list of files will be pushed through the loader again which can increase the "build modules" time, but we might be able to put in some optimizations for that.

IMHO, in the end, it still comes down to the performance of the incremental compilation. Anything larger than a few seconds is not acceptable, for me anyway. Skipping error reporting is not the right solution here. Getting the errors in an efficient way is.

I'm not sure in your case if those 6-15 seconds are just typescript compilation or overall time, including time spent by webpack. The usual bottleneck is dependency resolving, so there may be other things causing the large build time.

On a side-node, something to watch out for: WS has its own "language service" / compilation logic. It doesn't use the API exposed by typescript and they said it will never do so, for commercial reasons. Although rare, you might find yourself with compilation errors when the IDE says everything is fine and viceversa. WS10 at least was full of this kind of bugs and it was one of the reasons why I moved away from it.

@Karabur

@Karabur Have you tried using the transpileOnly option?

@use-strict

@use-strict of course it is workaround, not the solution. 6-14 is error processing step only. so, having all incremental time about 7 seconds I have like 1 second without. For relativeliy small project. Other people reported more time on that step.
Your info about WS is outdated and incorrect. You can use exact same TS compiler as ts-loader and have 100% of it's errors. But it is off-topic here, and other editors could have full type checking support as well (why they shouldn't?)
@jbrantly just tried it:
without any options: total build time: 10Sec (9 seconds overhead)
with transpileOnly: 3~4 sec (2-3 seconds overhead)
with errors disabled: 1 sec
transpileOnly seems to give good boost, and, since with transpileOnly there is also no errors printed, I think it is safe and correct to skip after-compile step completely if it is set to true

I've checked code in more details and with transpileOnly options error gather is never called at all. Looks like things was messed up in my test runs, so that 1s time is caused by something else. I'll try to do more isolated and clean performance testing. So far transpileOnly looks like best option.

Right. With transpileOnly the whole after-compile step is completely skipped. transpileOnly basically is the option to skip error checking (there is some minor error checking that goes on but I think it's only syntactic).

I should note (probably in the documentation somewhere) that transpileOnly does have slightly different output behavior:

From here

imports\exports are not elided just because they cannot be resolved (since we have code for only one file)
const enums are treated as regular enums (even for locally defined const enums)
files are always emitted as modules even if they don't have top level export \ import \ require

Any news or workarounds to make latest version of ts-loader as fast as 0.5.2?

@panKt

@panKt Not yet. I believe initial build time is actually faster now than in 0.5.2, but incremental build time still has the issue outlined here.

@panKt

@panKt, As a workaround you could use transpileOnly during development.

I am having this issue too :( incremental compile gets slower and slower as the project gets bigger, I've resorted to using transpileOnly for now but it's bad because I keep having to remember to toggle it to check for errors :(

Is it getting slower in relation to how much code you're adding? This
would be expected then.

On 20 January 2016 at 09:40, Wael notifications@github.com wrote:

I am having this issue too :( incremental compile gets slower and slower
as the project gets bigger, I've resorted to using transpileOnly for now
but it's bad because I keep having to remember to toggle it to check for
errors :(

—
Reply to this email directly or view it on GitHub
#78 (comment)
.

Had the same issue, setting "sourceMap": false in tsconfig.json helped tremendously (much more than expected).

Setting "sourceMap": false doesn't change much in my case. But setting transpileOnly brings down the compile time from 2.5s to 700ms. This might now seem much, but it is actually 3 times faster.

awesome-typescript-loader stores an internal dependency graph. Would this help at all?

FWIW, both ts-loader and awesome-typescript-loader seem to have an issue with optimize-assets taking a long time :(

Navigation Menu

Repository navigation

Incremental build performance dropped #78

29 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Incremental build performance dropped #78

Description

Activity

jbrantly commented on Oct 8, 2015

use-strict commented on Oct 8, 2015

use-strict commented on Oct 8, 2015

jbrantly commented on Oct 8, 2015

use-strict commented on Oct 8, 2015

SonofNun15 commented on Oct 8, 2015

jbrantly commented on Oct 8, 2015

jbrantly commented on Oct 8, 2015

jbrantly commented on Oct 8, 2015

use-strict commented on Oct 9, 2015

jbrantly commented on Oct 9, 2015

jbrantly commented on Oct 9, 2015

use-strict commented on Oct 9, 2015

use-strict commented on Oct 9, 2015

jbrantly commented on Oct 9, 2015

29 remaining items

use-strict commented on Dec 11, 2015

jbrantly commented on Dec 11, 2015

Karabur commented on Dec 11, 2015

Karabur commented on Dec 15, 2015

jbrantly commented on Dec 15, 2015

jbrantly commented on Dec 15, 2015

panKt commented on Jan 12, 2016

jbrantly commented on Jan 13, 2016

NikGovorov commented on Jan 13, 2016

waeljammal commented on Jan 20, 2016

raybooysen commented on Jan 20, 2016

spock123 commented on Feb 2, 2016

kenjiru commented on Mar 20, 2016

jklmli commented on Mar 27, 2016

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions