Make the Compiler and Language Service API Asynchronous

It makes very little sense that IO-bound operations (e.g. readFile, writeFile) are synchronous when we run on a platform whose touted strength is in writing IO-bound applications through asynchronous non-blocking APIs.

Compile times could probably be shaved down by a bit, especially for large projects with many files. Additionally, tooling experience should be as responsive as possible and avoid blocking in the event that IO takes place, enabling several optimizations to free memory on the managed side.

With #1664, this should be significantly easier.

This means that at the very least, the following would change to have asynchronous counterparts.

createSourceFile (because of /// <reference ... /> comments)
createProgram
all of the language service API

Compile times could probably be shaved down by a bit, especially for large projects with many files. Additionally, tooling experience should be as responsive as possible and avoid blocking in the event that IO takes place

👍

Does that mean you will ship and support a promise library?

Honestly after the hell I have gone through to write brackets-typescript, and make languageService works in a webworker, I can only 👍 👍 👍 this one.

Does that mean you will ship and support a promise library?

I think it'd still be bring-your-own-promise; whatever we use would probably not be an official endorsement of what to use.

My own preference would be for us to use a polyfilled ES6 promise so that we can actually get a sense of how we support that scenario.

Just FYI : people will still need to run in a webworker (or separate node process). Making io async alone is not the performance bottleneck : TypeStrong/atom-typescript#36 (comment) Its just the sheer amount of computations that need to be performed.

Right - for an optimal experience, you really need to get some multithreading into the mix. In our case, we do some caching to avoid blocking IO operations, so this would help us in terms of reducing our memory footprint.

👍

Colin Snover (@csnover)

Definitely try https://atom.io/packages/atom-typescript again if you haven't before. Now I'm running a node (or atom on mac) instance just listening on stdin using this : https://github.com/TypeStrong/atom-typescript/blob/df110e7218815dc05be90102f46aad7bd2f04d29/lib/worker/messages.ts#L1-L50

With a parent : https://github.com/TypeStrong/atom-typescript/blob/df110e7218815dc05be90102f46aad7bd2f04d29/lib/worker/messages.ts#L1-L50

And a child: https://github.com/TypeStrong/atom-typescript/blob/df110e7218815dc05be90102f46aad7bd2f04d29/lib/worker/workerProcess.ts#L18-L36

There are still cleanups to do but the experience is awesome.

/cc (and thanks) Colin Snover (@csnover)

Colin Snover (@csnover)

I've wrapped it up in an architecture that is extremely simple to understand / debug / extend. Also documented it (thanks to excellent support by Colin Snover (@csnover)) https://github.com/TypeStrong/atom-typescript/blob/master/CONTRIBUTING.md#architecture

This doesn't actually buy us much because we're not I/O-bound, and we don't have any I/O that can be easily parallelized (we produce a list of files basically serially). The impact on the compiler base would be very large for not any real perf gain.

This isn't about single or multi threading - the sync callback itself, say readFile or getScriptSnapshot could modify state. These are user-written callbacks and can do dangerous mutations synchronously. Any call to a user-provided callback is just as dangerous as an await.

I think Cyrus is talking more about the risks of having to be very intentional in preventing state modification. An arbitrary host can create a language service, bork the state, and if an API consumer filed an issue saying "it hurts when I do this", much of the time we'd probably say "don't do that".

On the other hand, if you allow the API to re-enter asynchronously, that becomes a much more deliberate API commitment - in fact, I think that's probably half the use-case of making the API asynchronous. Even if we tell people "don't do that", it's way too easy for API consumers to accidentally re-enter the API.

So either we would have to harden the API against that, or we'd have to tell people "don't re-enter" which I would think is unfortunate.

This isn't about single or multi threading - the sync callback itself, say readFile or getScriptSnapshot could modify state.

How? These calls just return values that the LS uses to update it's own state. That state is then able to be used for the remainder of hte computation.

any call to a user-provided callback is just as dangerous as an await.

It really isn't. Calls to user-provided callbacks don't generally make reentrancy happen. Calls to awaits do.

Craig P Hicks (@craigphicks)

Craig P Hicks (@craigphicks) I don't really understand the argument that is being made. All i'm saying is this:

The TS compiler/language-server is heavily mutable (by design). The current design presumes that any mutation is safe because it happens, and then all processing that needs to view that mutation will run after it all teh way to completion. In other words, computation that operates on that mutable state will not ever see that mutable state change while it executes.

It get what you are saying - that there are both multi-part read and write operations which must take place atomically - at least atomic writings cannot overlap atomic readings or other atomic writings, although atomic readings can overlap each other without issue.

Moreover, it might be the case that those multi-part atomic write operations in LS include calls to createProgram and emit, so that asynchronous calls to IO under createProgram and emit could result in violating that atomicity.

How about using context (LS vs non-LS) to conditionally call async-IO under createProgram and emit? A hack to enable some optimization until LS atomicity was enabled.

On the subject of atomic writes and reads, that's an issue that atomic-RW supporting databases handle. Perhaps we could avoid reinventing the wheel and use such a database as an intermediary for the language service?

On the subject of atomic writes and reads, that's an issue that atomic-RW supporting databases handle. Perhaps we could avoid reinventing the wheel and use such a database as an intermediary for the language service?

i see no way to do that without enormous complexity. TS is not Roslyn. It cannot trivially fork the state of the LS (esp. given the perf constraints of JS engines). So it has to take a direct mutation approach just to maintain reasonable perf (and even with that, perf is a continual issue). TS's design is heavily predicated on shared mutation. So either we have to:

ensure that shared mutation cannot happen with asynchrony (not sure how). or
ensure that shared mutation doesn't impact resumed computation (hard, but possible). or
redesign not based on a shared mutation model. extremely costly. underpins the entire design of hte TS compiler/LS top to bottom.

On the subject of atomic writes and reads, that's an issue that atomic-RW supporting databases handle. Perhaps we could avoid reinventing the wheel and use such a database as an intermediary for the language service?

i see no way to do that without enormous complexity. TS is not Roslyn. It cannot trivially fork the state of the LS (esp. given the perf constraints of JS engines). So it has to take a direct mutation approach just to maintain reasonable perf (and even with that, perf is a continual issue). TS's design is heavily predicated on shared mutation. So either we have to:
1. ensure that shared mutation cannot happen with asynchrony (not sure how). or

2. ensure that shared mutation doesn't impact resumed computation (hard, but possible).  or

3. redesign not based on a shared mutation model.  extremely costly.  underpins the entire design of hte TS compiler/LS top to bottom.

My thinking may to too simple, or maybe the overhead is too high. Inside LS, an javascript-async should-be-atomic multi-part write op calls a worker to make a worker-thread-synchronous write to the database and awaits the workers return. The use of the database ensures that javascript-async should-be-atomic multi-part reads do not see a half written version.

Ideally, the database would detect that read request overlaps a write in progress (e.g., same file) and would not return with the old version, but would make the read wait until the write was finished.

(No database duplication.)

The overhead of including a database is certainly high - that imposes requirements on the end users too I think - it makes ensuring compatabilty of all parts harder. The only requirement it satisfies is ensuring transactional consistency, and the DB gets blown away when the language server stops.

to make a worker-thread-synchronous write to the database

What database are you referring to? We have no databases here in our impl. As stated above, we have a shared-memory, mutable-state, representation of everything.

The use of the database ensures that javascript-async should-be-atomic multi-part reads do not see a half written version.

So this is either 1 or 3 (closer to 3 from my reading). It would be an entire redesign of how everything works from top to bottom. We'd need to introduce a DB indirection layer into the deepest parts of our system, which have been tuned heavily for perf. There is no indication whatsoever that this could possibly work or meet any sort of perf constraints we have.

The only requirement it satisfies is ensuring transactional consistency,

This is a huge requirement. :)

It's effectively stating: move away from a shared-state/mutable system as our backing store. Yes, that would potentially address things. But it would need an effective full rewrite of major parts of teh stack. And there's no data at all that indicates that this could be implemented in an efficient enough fashion.

If we were to contemplate a (near) full rewrite, i'd rather just go the roslyn approach and have an shared-state/immutable (rather than mutable) model.

The only requirement it satisfies is ensuring transactional consistency,

This is a huge requirement. :)

Unwind my database detour that would not be practical nor easier.

Here is a plan:

protocol for ensuring atomic transaction consistency

At the heart of the matter is the protocol for ensuring atomic transaction consistency.

The per file interface could look like

interface Gatekeeper1 {
    async waitForReadPermission():void
    async readDone():void
    async waitForWritePermission():void
    async writeDone():void
}

or something that shifted the burden of waiting before execution to the gatekeeper such as

type Awaited<T> = T extends PromiseLike<infer U> ? Awaited<U> : T;

interface Gatekeeper2 {
  async waitUntilRead<F extends (...args: any[]) => ReturnType<F>>(anyf: F)
    : (...args: Parameters<F>)=>Promise<Awaited<ReturnType<F>>>
  async waitUntilWritten<F extends (...args: any[]) => ReturnType<F>>(anyf: F)
    : (...args: Parameters<F>)=>Promise<Awaited<ReturnType<F>>>
}

which would return a function that managed holding onto the arguments before calling the function, and also handle the later ...Done() call, while still ensuring type checking on the arguments. That would make it easier on the caller.

Something that would make the caller wait, but would still type check the arguments and would take care of calling ...Done() would look like this:

type Awaited<T> = T extends PromiseLike<infer U> ? Awaited<U> : T;

interface Gatekeeper3 {
  async waitUntilReadStart<F extends (...args: any[]) => ReturnType<F>>(anyf: F)
    : Promise<(...args: Parameters<F>)=>Promise<Awaited<ReturnType<F>>>>
  async waitUntilWriteStart<F extends (...args: any[]) => ReturnType<F>>(anyf: F)
    : Promise<(...args: Parameters<F>)=>Promise<Awaited<ReturnType<F>>>>
}

The caller code could call like this

const result = await((await gatekeeper.waitUntilReadStart(someFuncInT))(.... arguments ))

or

const resultPromise1 = (await gatekeeper.waitUntilReadStart(someFuncInT)(.... arguments )
...
const resultPromise2 = (await gatekeeper.waitUntilWriteStart(anotherFuncInT)(.... arguments )
... 
const result = await Promise.all[resultPromise1, resultPromise2, ....]

The instantiation of the interface would use a couple of queues to keep track of requests, one queue each for reads and writes, with writes taking precedence. Also, it would keep track of the number of currently executing reads/writes. For the next request to start,

a write request can start when the number of currently executing reads and writes are both zero.
a read request can start when the number of currently executing writes are zero (and the write request queue is empty).

The gatekeeper logic is activated on every event - i.e., a new request or a currently executing read or write completing. It's not even necessary to use nodejs messaging - it can be achieved more transparently with promises managed by gatekeeper.

Careful development and testing

~~1. Select a small handful of atomic multi-read / atomic multi-write candidates for testing. Let's call this group T. We want to code so that they run concurrently.~~

Identify ALL atomic multi-read / atomic multi-write operations. Refactor the sync code so that that every individual atomic multi-rw operation is callable as an individual function. Call this set of functions T.
Rewrite the current sync code so that each each atomic read / atomic write in the candidate group T is callable as a function.
Use call tree analysis to identify all functions above T in the call tree and call that entire group ET.
Use the automated sync->async transform software to transform each function in transform ET to async form. Now async is available for coding in our targets T.
Set up gatekeeper protocol for each call to a function in T.

At this point, with gatekeeper in place at the entry to each atomic operation, it should be safe to call LS in a re-entrant manner, which is required for testing. (Mistakes in identifying must-be-atomic operations, or race conditions on global variables would make it unsafe of course, which is one reason for testing.) However, to fully optimize it is also necessary convert the IO operations from sync to async, and group calls to run concurrently if/where it would help:

Within some (more) function in T, change the file-IO functions from sync to async.
Group calls to run concurretly if/where it would help.
Testing.
Back to 6. until optimization and testing is complete.

Note: The automatic transform of sync to async creates a huge amount of code changes across a wide range of code. We don't have to worry about maintaining most of that, e.g., if the sync version master undergoes drastic change, then a new auto async version can be generated. We only have worry about merging the code of functions in T and the immediate callers of functions in T into that new auto-generated version.

What about other should-be-atomic reads/write that haven't yet been gatekeeper protected?

I have modified the plan in "Careful development and testing" so that gatekeeper is enforces proper exclusion across ALL functions in T before (top level LS entry) testing begins.

Conclusion

It is possible to break the problem down into small steps allowing normal development.

That is true despite the fact that

lexically speaking, a very large number of functions that do not seem directly related to our goal have to be transformed from sync to async form.
- That's true, but (conditioned on the existence of software to automate sync to async transformation) humans only have to focus only on small areas of codes which require human attention.
Async is more complex than sync.
- Absolutely true. However, there are rules and mechanisms to isolate that complexity. Step by step the sections of code that need protection can be identified, asynchronized, and tested.

That feels like an effective rewrite of the TypeScript compiler/LS , along with teh development of several new systems and tools needed to support the sync/async hybrid model.

It's also unclear how any of this is enforced. It seems to be a cooperative reader/writer lock approach. But a single errant yield means mutations may happen when problematic. In a codebase the size and complexity of TS, i don't see how that is prevented.

Cyrus Najmabadi (@CyrusNajmabadi)

Cyrus Najmabadi (@CyrusNajmabadi)

It seems to be a cooperative reader/writer lock approach

As described above it is only enforced inside the library. I can see the viewpoint that atomicity must be enforced outside of the LS as well, and that would have to be via the LS interface. For example:

File Rename Action ensemble
- Client calls GetEditsForFileRenameRequest (A read type command)
- The request returns.
- The return values are used to to a set of CombinedCodeActions to mutate multiple source files.
- The mutating command is executed.

Obviously this is an atomic combined read-then-write action. It is guaranteed to be atomically executed when the LS is synchronous, and therefore the client program must be calling it synchronously, provided the client is not calling from multiple separate parallel threads (actual OS level threads). If the client is calling from multiple separate parallel threads then the atomicity is not guaranteed, even in the sync-version of LS. Therefore it is safe to assume that we are only dealing with clients running a in a single thread (actual OS level thread).

Now suppose language server is changed to async. Re-entrant calls could break the atomicity of a single File Rename Action ensemble. Now we are at a fork in the road:

Clients should handle that themselves.
Clients should be provided with help to avoid incorrect re-entrant calls.

I'll provide argument for both sides.

Argument for 1. Clients should handle that themselves.

The advent of async allows concurrent code to be written in way that makes such an assumption reasonable, because it provides a reasonable facsimile of synchronous coding style. Before the advent of async, it would have had to been implemented with Promises/then/etc, in which case it would have been an unreasonable assumption.
Clients have a motivation to use the async version - it will enable then to group suitable actions to run concurrently for better performance - which necessarily requires deliberate and constrained usage anyway. Clients may want to perform concurrent optimizations they know will work safely, and not follows unnecessary constrains.
The LS 'lock' interface (described below) is redundant documentation and interface if the client writer doesn't use it. (They can't forced to use it, and it is possible to write a correctly behaving client without it.

Argument for 2. Clients should be provided with help (a lock interface) to avoid incorrect re-entrant calls.

Helps to avoid stupid mistakes.
If a LS client coder is really confident, they can skip the 'lock' interface provided they are prepared to accept the responsibility, therefore the 'lock' interface poses no burden on those clients who don't need it.

How the LS side assistance (a lock interface) would work

The gatekeeper logic would be extended, and an external interface provided.
The logic is slightly more complex, because internally both reads and mutates would occur within a single client mutate lock scope. However, it is relatively straightforward and condensed into a code-wise very small block of logic.

This extension would allow a client to await for a read or write lock, and to release that lock when it is done. That would be implemented as another command type in protocol.d.ts.
The client would get a uuid as a result of grabbing the lock, and apss that uuid back with each command in the atomic grouping.
That's straightforward to implement. Of course the clients could fail to code for grabbing the lock, or forget to release it. So it is not perfectly client mistake proof.

A modification of the development plan

Cyrus Najmabadi (@CyrusNajmabadi)

In no way am I saying that TS cannot become async. What i'm saying simply is that it is not trivial and would need to be done witha very conscious eye to ensure that either shared state mutations do not happen. Or, if they do, they cannot cause problems to in-flight operations.

Exactly right, I agree.

Cyrus Najmabadi (@CyrusNajmabadi) 2018

This is a solvable problem. In general, by going with the work-queue model. In other words, any LS operation that comes in is put into a queue and only one operation can come off that queue and operate (even asynchronously) at at time.

I believe that my concrete suggestions are basically just filling in details of expressed in your overall 2018 master vision. The gatekeeper interface does utilize read and write queues just as you mention, and as discussed it is possible to provide a client side IF to the gatekeeper to assist the client in grouping their must-be-atomic command groupings (variously read only, or mutating). (Note, the queues contain only promises to enforce waiting, rather than a queue of tasks, but the overall effect is the same.)

At the top level of LS, where commands are received, we (initially) code for just the following 7 cases:

A client read-lock get request (*)
A client read-lock release request (*)
A client mutate-lock get request (*)
A client mutate-lock release request (*)
A client command within a client lock scope (*)
- check that command is allowed by the lock type, e.g., no mutating with a read-lock.
- I don't think we need to worry about enforcing order of commands within the client lock scope, because the client won't proceed to a mutating operation until all the precursor read operations have returned to make that possible.
A client non-lock single command read request
A client non-lock single command mutating request

(* assuming we code for client side locking)

Of course there are more than 7 commands, but each of them would fall within one of those 7 cases,
and there would be only 7 places where the gatekeeper logic was coded, and then logic to execute the correct case depending on the command content.

Referring back to the steps 1- 5 mentioned in my previous post, those 7 are the functions in T, and they are very high up in the program at the top of the call tree. So high up, that automated sync->async software is not absolutely required for that, because the scope of change is small.

In the first pass of development, the code beneath or adjacent to the functions in T do not need to made async. The program will still work and maintain logically safe performance even if they (the code beneath or adjacent to the functions in T) are not async. In this first pass, there is not yet performance improvement.

Later (careful and deliberate) development would improve performance as follows:

Make all of the read-only-IO commands in the LS library async, The improvement would be apparent when client side re-entrant read-only calls were made concurrently. This code change could be done with automating sync->async software.
Manually group read-only-IO commands to run concurrently where it would help. E.g, to read multiple files concurrently within a single command.
Manually make async and then group together mutating-IO calls in the LS library which can run concurrently. E.g., in the case of the above 'File Rename Action ensemble' the write edits to each separate file could be safely executed concurrently. Since that would happen during a top level mutate type of command, gatekeeper is already enforcing the exclusion of concurrent read operations.

I very much appreciate your critical eye helping to push this proposal into better shape. Your judgement is indispensable.

(Aside: One thing I forgot to mention earlier in previous posts is about the need to check for any variables that might be in the a shared top level scope between re-entrant LS calls, but that really should belong to only one calls scope.)

Cyrus Najmabadi (@CyrusNajmabadi)

Cyrus Najmabadi (@CyrusNajmabadi)

That feels like an effective rewrite of the TypeScript compiler/LS , along with teh development of several new systems and tools needed to support the sync/async hybrid model.

I feel strongly the the current sync libraries and the async libraries should be kept separate. All version improvements other than the async conversion changes should be applied to the sync version, and the async conversion branch updated from the sync version.

A client of LS would use one or the other. I don't see any way to mix them for LS. I don't see any reason to mix them for compiler API. If async performance gains were sufficient, adoption would naturally increase, especially new projects. At some point, non-async version up support could be dropped.

I would guess that async is not the only bottleneck. It sometimes seems the compiler is loading a lot a files housing symbols which are never ever actually referenced during createProgram, and at least in debug mode this noticeably slows things down. If the file loading occurred just-in-time when the symbol was needed it might speed things up in some cases (at least debug mode). That comment is off topic here - I don't claim that excessive file loading is really a critical bottleneck - just mentioning it as a possible way in which async might not be the most critical bottleneck -- yet 'async' might be an important precursor to solving other bottlenecks, so it would probably still be worthwhile.

Automated transformation (from sync to async) sounds like a good idea, Is there any tools/project can do or trying to do this?

In TS7, API will be out of proc thus up to the caller to decide synchronicity (in-proc, if you must, will have Go non-coloring semantics, thus this is moot).

Navigation Menu

Repository navigation

Make the Compiler and Language Service API Asynchronous #1857

32 remaining items

protocol for ensuring atomic transaction consistency

Careful development and testing

What about other should-be-atomic reads/write that haven't yet been gatekeeper protected?

Conclusion

Argument for 1. Clients should handle that themselves.

Argument for 2. Clients should be provided with help (a lock interface) to avoid incorrect re-entrant calls.

How the LS side assistance (a lock interface) would work

A modification of the development plan

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Make the Compiler and Language Service API Asynchronous #1857

Description

Activity

NoelAbrahams commented on Jan 30, 2015

fdecampredon commented on Jan 30, 2015

DanielRosenwasser commented on Jan 30, 2015

basarat commented on Feb 2, 2015

DanielRosenwasser commented on Feb 2, 2015

vvakame commented on Feb 4, 2015

basarat commented on Feb 5, 2015

basarat commented on Feb 6, 2015

RyanCavanaugh commented on May 4, 2015

32 remaining items

justinfagnani commented on Aug 2, 2021

DanielRosenwasser commented on Aug 2, 2021

CyrusNajmabadi commented on Aug 2, 2021

craigphicks commented on Aug 2, 2021

CyrusNajmabadi commented on Aug 2, 2021

craigphicks commented on Aug 2, 2021

CyrusNajmabadi commented on Aug 2, 2021

CyrusNajmabadi commented on Aug 2, 2021

craigphicks commented on Aug 3, 2021

protocol for ensuring atomic transaction consistency

Careful development and testing

What about other should-be-atomic reads/write that haven't yet been gatekeeper protected?

Conclusion

CyrusNajmabadi commented on Aug 3, 2021

craigphicks commented on Aug 4, 2021

Argument for 1. Clients should handle that themselves.

Argument for 2. Clients should be provided with help (a lock interface) to avoid incorrect re-entrant calls.

How the LS side assistance (a lock interface) would work

A modification of the development plan

craigphicks commented on Aug 5, 2021

partic2 commented on Aug 10, 2024

RyanCavanaugh commented on Oct 28, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions