Repository navigation
Trade-offs in Control Flow Analysis #9998
Description
Some rough notes from a conversation Anders Hejlsberg (@ahejlsberg) and I had earlier about trade-offs in the control flow analysis work based on running the real-world code (RWC) tests. For obvious reasons I'll be comparing what Flow does with similar examples to compare and contrast possible outcomes.
The primary question is: When a function is invoked, what should we assume its side effects are?
One option is to be pessimistic and reset all narrowings, assuming that any function might mutate any object it could possibly get its hands on. Another option is to be optimistic and assume the function doesn't modify any state. Both of these seem to be bad.
This problem spans both locals (which might be subject to some "closed over or not" analysis) and object fields.
Optimistic: Bad behavior on locals
The TypeScript compiler has code like this:
enum Token { Alpha, Beta, Gamma }
let token = Token.Alpha;
function nextToken() {
token = Token.Beta;
}
function maybeNextToken() {
if (... something ...) {
nextToken();
}
}
function doSomething() {
if (token !== Token.Alpha) {
maybeNextToken();
}
// is this possible?
if (token === Token.Alpha) {
// something happens
}
}Optimistically assuming token isn't modified by maybeNextToken incorrectly flags token === Token.Alpha as an impossibility. However, in other cases, this is a good check to do! See later examples.
Optimistic: Bad behavior on fields
The RWC suite picked up a "bug" that looked like this:
// Function somewhere else
declare function tryDoSomething(x: string, result: { success: boolean; value: number; }): void;
function myFunc(x: string) {
let result = { success: false, value: 0 };
tryDoSomething(x, result);
if (result.success === true) { // %%
return result.value;
}
tryDoSomething(x.trim(), result);
if (result.success === true) { // ??
return result.value;
}
return -1;
}The ?? line here is not a bug in the user code, but we thought it was, because after the %% block runs, the only remaining value in result.success's domain is false.
Pessimistic: Bad behavior on locals
We found actual bugs (several!) in partner code that looked like this:
enum Kind { Good, Bad, Ugly }
let kind: Kind = ...;
function f() {
if (kind) {
log('Doing some work');
switch (kind) {
case Kind.Good:
// unreachable!
}
}
}Here, we detected the bug that Kind.Good (which has the falsy value 0) is not in the domain of kind at the point of the case label. However, if we were fully pessimistic, we couldn't know that the global function log doesn't modify the global variable kind, thus incorrectly allowing this broken code.
Pessimistic: Bad behavior on fields, example 1
A question on flowtype SO is a good example of this
A smaller example that demonstrates the behavior:
function fn(arg: { x: string | null }) {
if (arg.x !== null) {
alert('All is OK!');
// Flow: Not OK, arg.x could be null
console.log(arg.x.substr(3));
}
}The problem here is that, pessimistically, something like this might be happening:
let a = { x: 'ok' };
function alert() {
a.x = null;
}
fn(a);Pessimistic: Bad behavior on fields, example 2
The TS compiler has code that looks like this (simplified):
function visitChildren(node: Node, visit: (node: Node) => void) {
switch(node.kind) {
case SyntaxKind.BinaryExpression:
visit(node.left);
visit(node.right); // Unsafe?
break;
case SyntaxKind.PropertyAccessExpression:
visit(node.expr);
visit(node.name); // Unsafe?
break;
}
}Here, we discriminated the Node union type by its kind. A pessimistic behavior would say that the second invocations are unsafe, because the call to visit may have mutated node.kind through a secondary reference and invalidated the discrimination.
Mitigating with (shallow) inlining / analysis
Flow does some assignment analysis to improve the quality of these errors, but it's obviously short of a full inlining solution, which wouldn't be even remotely practical. Some examples of how to defeat the analysis:
// Non-null assignment can still trigger null warnings
function fn(x: string | null) {
function check1() {
x = 'still OK';
}
if (x !== null) {
check1();
// Flow: Error, x could be null
console.log(x.substr(0));
}
}// Inlining is only one level deep
function fn(x: string | null) {
function check1() {
check2();
}
function check2() {
x = null;
}
if (x !== null) {
check1();
// Flow: No error
console.log(x.substr(0)); // crashes
}
}Mitigating with const parameters
A low-hanging piece of fruit is to allow a const modifier on parameters. This would allow a much faster fix for code that looks like this:
function fn(const x: string | number) {
if (typeof x === 'string') {
thisFunctionCannotMutateX();
x.substr(0); // ok
}
}Mitigating with readonly fields
The visitChildren example above might be mitigated by saying that readonly fields retain their narrowing effects even in the presence of intervening function calls. This is technically unsound as you may have both a readonly and non-readonly alias to the same property, but in practice this is probably very rare.
Mitigating with other options
Random ideas that got thrown out (will add to this list) but are probably bad?
puremodifier on functions that says this function doesn't modify anything. This is a bit impractical as we'd realistically want this on the vast majority of all functions, and it doesn't really solve the problem since lots of functions only modify one thing so you'd really want to say "pureexcept form"volatileproperty modifier that says this "this property will change without notice". We're not C++ and it's perhaps unclear where you'd apply this and where you wouldn't.
Activity
zpdDG4gta8XKpMCd commentedon Jul 28, 2016
with persistent data structures it makes perfect practical sense, amortized time/memory consumption for a typical application would be comparable, it take's some discipline and support from the language ( #6230, #2319, #1213), but it's worth the trouble
with
readonlyfor immutables anspurefor functions it will be blazing fast compared to inlining, because modifiers only need to be checked once for each function/interfaceplease no more half measures based on assumptions for a couple of typical cases and endless exceptions that only worsen soundness
ahejlsberg commentedon Jul 28, 2016
Looking through the RWC tests, I see only one case where the additional narrowing performed by #9407 causes unwanted errors. However, there were scores of real bugs and inconsistencies being caught, such as checking for the same value twice, assuming that enum members with the value 0 pass truthiness checks, dead branches in
ifandswitchstatements, etc. In aggregate, I think our optimistic assumption that type guards are unaffected by intervening function calls is the best compromise.BTW, the compiler itself relies on side effecting changes to a
tokenvariable in the parser. For example:This becomes an error with #9407 because the compiler continues to think
tokenhas the valueSyntaxKind.ExportKeywordfollowing the call tonextToken(and thus reports an error whentokenis compared toSyntaxKind.DefaultKeyword).We will instead be using a function to obtain the current token:
where the function is simply:
Since all modern JavaScript VMs inline such simple functions there is no performance penalty.
I think this pattern of suppressing type narrowing by accessing mutable state using a function is a reasonable one.
gvkhna commentedon Jul 29, 2016
is there any possibility of a const on an argument?
In swift/objc they added the "in", "out", and "inout" meta attributes to arguments so you could see if they meant for a value to be modified. i feel like this could be useful for annotating whether a function will modify a variable it received or not. Not sure if I'm missing something.
kitsonk commentedon Jul 29, 2016
While
pureis all the "rage" these days, I think many JavaScript developers don't understand what it means and could easily lead to surprises and or large scale frustration. Because the inherit universal mutability of JavaScript I thinkpurewould generally be shackles that would potentially go unused because of the over strictness of the concept... It just seems an "all or nothing" concept, when most of JavaScript is shades of grey.I think a argument by argument modifier makes the most sense. Of course
constandreadonlyonly imply disallowing reassignment. Personally, I don't find it confusing of expanding those to inform the flow control that property mutations are "unsafe" as that would be a design time only construct.Is there any benefit in considering a deeply immutable design time keyword to avoid any confusion between
constandreadonly? Potentially evenimmutable(though that is jargoney).zpdDG4gta8XKpMCd commentedon Jul 29, 2016
in my stubborn defense of
pure:[1, 2, 3].map(x => x + 1 /* <-- hi, i am pure function */)i really wish it was it was considered
other options may work out too and should be considered, just not a big fan of half-baked solutions
kitsonk commentedon Jul 29, 2016
I understand your argument, although I think what you would cover with "obviously" pure functions are not ones that suffer from CFA challenges. You example of
[1, 2, 3].map(x => x + 1)does not suffer from CFA function boundary issues, and while it is logically pure, there is not benefit in denoting this at design time.I think your argument might have more merit if you could think of some examples where there are current CFA issues that would be solved by a pure notation that benefitted the developer. My argument isn't against the concept of writing pure functions, it is more of the argument that an "all or nothing" solution likely won't be practical.
One of the biggest current issues with CFA in my opinion is in Promise callbacks. There are many situations where those might not be pure. That is why I think an argument by argument mutability notation would solve significantly more use cases and do so in a more end-developer controllable way.
zpdDG4gta8XKpMCd commentedon Jul 29, 2016
although you are right that the example doesn't target CFA issues, the answer is rather evident:
this is one merit of the pureness, among many others
let me say it again, if a promise callback is pure, all reasoning that was done outside of it's scope or time is still valid inside of it
you might wonder why i am so certain, let me explain, there is no notion of time in math that stands behind the idea of pureness, time doesn't exist there, it's absolutely static, everything existed forever and forever will and cannot ever change (because changes need time) to the point where a pure function can be thrown away and replaced with its result (what's called referential transparency) which for the any given arguments will always be the same (hence a question why not just use a hash-table instead to map arguments to results), so you don't really need a function in the first place, armed with such super static equivalence between results that were calculated vs hardcoded the only merit in using a function is to save some memory that otherwise would be taken by a huge static table mapping all possible arguments to all results, since math is absolute abstraction we only care when it comes to program it
this is what a pure function is, this is why it will always work
i agree that it's rather 'all or nothing', but is the price you have to pay for the peace of mind because in its strictness it solves the CFA problems and many others simply by giving them no place to exist
zpdDG4gta8XKpMCd commentedon Jul 29, 2016
think about it, a "pure" (aka persistent) data structure, say, a linked list, can be deeply copied just by... being assigned:
how cool is that?
in my humble opinion "pure" functions and data structures can easily cover 40% of typical everyday needs without even feeling restricted
as for the rest 60% (since we don't appreciate any sort of extremes) we can use good old mutable impure JavaScript
but mind those 40% percent of no problem ever! that's a darn bargain, just let it happen and enjoy it
sledorze commentedon Jul 29, 2016
totally agree with Aleksey-Bykov arguments in favor of having "pure" introduced into the type checker.
gvkhna commentedon Jul 30, 2016
Just my .02, I think "pure" would be too abstract and too confusing to understand.
I'm actually in favor of apple style semantics, "in", "out", "inout"
Also I think const could lead to confusion since they would be similar in usage but different concepts.
It might be easier for beginners to pick up and understand but the intent simply wouldn't be as clear.
maiermic commentedon Jul 31, 2016
Ryan Cavanaugh (@RyanCavanaugh) The compiler should already be able to answer the question in the example of Optimistic: Bad behavior on locals already with yes. If
doSomethingis called whentokenisToken.Alpha, the conditiontoken !== Token.Alphaof the firstifstatement is false. Hence, the body is skipped andnextTokenis never called.tokenisToken.Alphastill holds when the secondifstatement is reached. Ergo the answer is yes.We can make this question insteresting again by changing
doSomethinglike this:Now it is only possible if
maybeNextTokenchangestokentoToken.Alpha. If you are optimistic you would answer the question with no, since you assume that functions have no side effects like changing variables. However, if you changetokeninnextTokentoToken.Alphainstead ofToken.Betait would be possible if and only if
... something ...is true inmaybeNextToken. If you are optimistic you would still answer the question with no even though the correct answer is yes, maybe.maiermic commentedon Jul 31, 2016
Infer constraints about functions
How about tracking variable changes of functions?
The basic idea is that we know a constraint before a function call and that we can reason about a constraint after a function call.
Note: I write a new introduced term bold in the following when it is mentioned the first time. Jump back to this position to see the definition of this term.
ifstatementsThis idea is quite similar to control flow base type analysis. So let's take a look at the
ifstatement in the example of control flow base type analysis first (before we discover constraints introduced by functions):I rewrite the constraint annotations like this
As usual, a single colon like in
x: string | number | booleanmeans variablexhas typestring | number | boolean.A double colon like in
x :: string | number | booleandescribes a value-typed-binding, which means that the current value ofxis of typestring | number | boolean. The type of the value may be any subtype of the type of the variable. You can for example conclude from the checktypeof x === "string"istruethat we havex :: string(the current value ofxis astring).The constraints of a simple
ifstatement can be described in more general asHere constraint is abbreviated as c (
cBeforestands for constraint before). I name our constraints in the example above like this:I write
c := eto assign a constraint expressioneto the constraint calledc.In the example,
x = 1;is a side effect that causes a change on the constraint we have before. Beforex = 1;we havec1 := (x :: string)and afterwards we havec2 := (x :: number).I write
c with c'for the constraint where all appearances of value-type-bindingsx :: Tofcare overridden with value-type-bindingsx :: T'ofc'. In this examplec1 with c2is(x :: string) with (x :: number)which results in(x :: number), since the old constraint value-type-binding(x :: string)inc1is overridden by(x :: number)oft1.I describe a side effect
sas a constraint transitionc -> c'. A constraint transition takes constraintc(that holds before side effects) and gives you a new constraintc'(that holds after side effects). Given a constraint transitiont: c -> ewhereeis a constraint expression, I use function like notationt(cp)to express constraintc'that is equal toewhere all appearances ofcare replaced bycp.Using this notation, the example from above looks like this:
You can describe reasoning about constraints of a simple
ifstatement in more general like thisI write
c & conditionto describe the case thatconditionistrue. Otherwise, I writec & !conditionto describe the case thatconditionisfalse. In both cases the constraintchad been given before the check. From the check ofconditionyou can often reason about a new constraintc'that holds. Ifconditionistrueimplies thatc'holds, you can rewritec & conditionasc with c'. Similarly, ifconditionisfalseimplies thatc'holds, you can rewritec & !conditionasc with c'. If no constraint follows, you just writecinstead.In the example,
conditionistypeof x === "string". Ifconditionis fulfilled the body is executed. Thereby,c1iscBefore & condition. From fulfilledconditionyou knowtypeof x === "string"holds. From that, you can infer the constraint transitionx :: string. As a result,cBefore & conditionequalscBefore with (x :: string). SincecBeforeisx :: string | number | booleanyou get(x :: string | number | boolean) with (x :: string)that is reduced tox :: string.After the
ifstatement you know thatconditionhad been fulfilled and it's body has been executed that causes side effects described byt1t1(cBefore & condition)and can be reduced tox :: string(like described above)conditionhad been unfulfilledcBefore & !condition!conditionis!(typeof x === "string")which istypeof x !== "string"(x :: string | number | boolean) with !(x :: string)which can be reduced tox :: number | booleanFrom that you can conclude that
cAfterist1(cBefore & condition) | (cBefore & !condition)which is equal to(x :: string) | (x :: number | boolean)and can be reduced tox :: number | boolean.Constraints introduced by functions
Now we have warmed up, we take a look at constraints that are introduced by functions.
(User-Defined) Type Guards
A type guard is a function whose return type is a type predicate:
Those type predicates are constraints that are introduced by a function (here
isFish) and are already part of TypeScript:Let's add constraints as comments:
Before the
ifstatement you know thatpetisFish | Bird. IfisFish(pet)is fulfilled, the type predicatepet is FishofisFishholds for the value of our local variablepet. That means that it adds a constraintpet :: Fishtoc1. Thus, it's ok to callpet.swim();. In the else branch you know that the type predicatepet is Fishis unfulfilled and you can add!(pet :: Fish)toc2, which results inpet :: Bird.Functions in general
In a similar way to type predicates of type guards I introduce constraints that can be automatically inferred from expressions and statements. I will add constraints to the functions one after another in my modified example (see comment above) of the example behavior on locals (see first post) to answer the question
is this possible?in the comment in the functiondoSomething:nextTokennextTokenassignsToken.Alphato the shared local variabletoken. Like the assignmentx = 1;in theifstatement example,token = Token.Alphais a side effect that can be described as a constraint transition:Here
cBeforedescribes the constraints that we have before callingnextTokenandcAfterdescribes the constraints that we have after callingnextToken. Like an assignment, a function can be described as a constraint transition. SincenextTokenis doing just the same as the assignmenttoken = Token.Alpha;it describes the same constraint transition ast1.maybeNextTokenLet's have a look at the caller
maybeNextToken:First you add the constraint transitions (here
t1fornextToken) and afterwards the constraints of theifstatement:Since
conditionis not specified in the original example you don't know its result. I assume that it is an expression without any side effect. To relax this assumption even further I considerconditionis a shared variable of typeboolean. Thereby, you can derivecondition :: trueifconditionis fulfilled andcondition :: falseotherwise. Thereof,cAfterisDespite that simplification
maybeNextTokenis described as the constraint transitioncBefore -> cAfter.doSomethingLike in
maybeNextTokenyou first add constraint transitions and afterwards the constraints of theifstatement:Our goal is to answer the question if it is possible that
token === Token.Alphais fulfilled in the condition of the secondifstatement. This is only possible ifc2contains a constrainttoken :: TwhereTis a type that containsToken.Alpha. Hence, you only look atc1andc2:c1iscBefore & token !== Token.Alpha. Sincetokenis notToken.Alphait only can beToken.BetaorToken.Gamma. This results incBefore with (token :: Token.Beta | Token.Gamma).c2is derived by replacingcint1withc1:Since the type of the value of
tokenincBeforeis overridden bycBefore with (token :: ...)operations, you can leavecBeforeout:I'm only interested in
tokento answer the question. Hence, you can remove allwithoperations of other variables (herecondition):From
c2 = token :: Tokenyou conclude that the answer of the question is:Yes,
tokenmight beToken.Alphain the condition of the second/innerifstatement.yortus commentedon Aug 1, 2016
Michael Maier (@maiermic) your post was pretty complex, so I may have misunderstood it, but it looks like you are describing what control flow anaylsis already does, plus extending it to 'look into' the functions being called within guarded blocks as well. If so, how is this different to the inlining approach mentioned by Ryan Cavanaugh (@RyanCavanaugh) in the OP?
maiermic commentedon Aug 1, 2016
Troy Gerwien (@yortus) That's right, I describe what control flow analysis already does to introduce the notation I use afterwards to explain my idea. However, I'd like to clarify that I don't 'look into' a function every time it is called (if that wasn't clear so far). Instead I infer the constraint transition of a function when I look at it's definition. Therefore, I have to look into the function, but I only have to do this once, since I can carry the constraint transition in the type of the function. When I look at the function call, I can use the constraint transition that I inferred before to calculate the constraint that holds after the function call from the constraint that holds before the function call.
Mitigating with (shallow) inlining / analysis
I don't know how (shallow) inlining/analysis works and Ryan Cavanaugh (@RyanCavanaugh) doesn't explain it. Nevertheless, he shows two examples.
1. Example
Flow claims that
x could be nullafter the call ofcheck1. Flow doesn't look intocheck1. Otherwise, it would know thatxhas typestringand cann't benull.In my approach, the constraint transition of
check1is inferred ast: c -> (x :: string). Since we checked thatx !== nullistruebefore and the type of the variablexisstring | nullwe havex :: stringas constraint that holds before the call ofcheck1. Hence, we passx :: stringtotto get the constraint that holds after the call ofcheck1.t(x :: string)results inx :: string. As a consequence,xcann't benullinconsole.log(x.substr(0));.Note: You can even infer
x :: 'still OK'incheck1in that case. Thereby, you could even determine the type of the return value ofx.substr(0)which is's'. However, this would require further knowledge about the build-in methodsubstr. I didn't cover such an approach in my previous post.2. Example
To come straight to the point, the constraint transition
check2ist2: c -> (x :: null)andcheck1ist1 = t2Before the call of
check1we havex :: string. Afterwards we havet1(x :: string), which results inx :: null. Thus, my approach detects the error and flow doesn't.107 remaining items
refactor(codecatalyst): migrate client to sdkv3 (#6734)
modjke commentedon Jul 21, 2025
Maybe mentioned somewhere but in case of hardcoded values
as typecan be solution to please the compilerFix
Phase 2 follow-ups: deepseek catalog fix, plugin docs, strict-mode TODO
Phase 2 follow-ups: deepseek fix, plugin docs, strict-mode TODO
Hideman42 commentedon Jun 30, 2026
Hi, I got a simple issue like
I found several issues that are all closed as duplicate of this pit-fall issue. I think I understand that the topic seem a lot larger and complex that I can imagine.
Today I have to disable some controls, add extra hacks and all these makes the code harder to understand and loose the benefits of using TS in the first hand.
I kind of wondering if there is any plan to improve the developer experience on this subject? Is it something post TS7.0 or is something we wont be able to see for a while?
laug commentedon Jun 30, 2026
Hideman42 You can easily solve this by using
for...ofinstead of.forEach:Mike-Bell commentedon Jun 30, 2026
The way I usually solve this particular situation is just doing
let bool = false as boolean. Theas booleanseems to tell TS not to do its normal (incorrect) narrowing.fix(v5): 修复 imageBilling.integ 类型错误——canonical typecheck 复绿
chore: Type-aware lint rules, enabled and cleared in one pass (PLA-66) (
fix(pool): bound the launch fill, so a wedged Chromium cannot hold th…
Stop resized Windows terminals from dropping a typed key (#4989)
Stop resized Windows terminals from dropping a typed key (#4989)
fix(math-expression-compiler): a token check reads the state's own st…