Repository navigation
Feature request: lift the circular constraint for conditional types #26980
Description
Search Terms
circular type conditional type
Suggestion
Currently, Last1 is valid, but seemingly-equivalent Last2 is not:
type Head<T extends any[]> = T extends [infer X, ...any[]] ? X : never;
type Tail<T extends any[]> =
((...x: T) => void) extends ((x: any, ...xs: infer XS) => void) ? XS : never;
type Last1<T extends any[]> = {
0: never,
1: Head<T>,
2: Last1<Tail<T>>,
}[T extends [] ? 0 : T extends [any] ? 1 : 2];
type Last2<T extends any[]> =
T extends [] ? never :
T extends [infer R] ? R :
Last2<Tail<T>>;My suggestion is to make Last2 valid, since the recursion isn't necessarily unbounded.
Use Cases
It'd make recursive conditional types a lot easier and more intuitive to write. It'd also eliminate the need to guard against impossible cases when using the workaround like when using Last1 above. If you wanted to mandate termination, all I'd want is direct recursion - this makes it much easier to check for infinite recursion. (Other types are generally assumed to terminate, so you're only assessing control flow.)
Examples
See above in the suggestion summary. Here's a concrete example of how this file (with dependency) would be simplified:
type Last<L extends any[], D = never> =
L extends [] ? D :
L extends [infer H] ? H :
((...l: L) => any) extends ((h: any, ...t: infer T) => any) ? Last<T> :
D;
type Append<T extends any[], H> =
((h: H, ...t: T) => any) extends ((...l: infer L) => any) ? L : never;
type Reverse<L extends any[], R extends any[] = []> =
((...l: L) => any) extends ((h: infer H, ...t: infer T) => any) ?
Reverse<T, Append<R, H>> :
R;
type Compose<L extends any[], V, R extends any[] = []> =
((...l: L) => any) extends ((a: infer H, ...t: infer T) => any) ?
Compose<T, H, Append<R, (x: V) => H>> :
R;
export type PipeFunc<T extends any[], V> =
(...f: Reverse<Compose<T, V>>) => ((x: V) => Last<T, V>);Currently, you could achieve similar via this, but as you can see, it's highly repetitive and boilerplatey:
type Last<L extends any[], D = never> = {
0: D,
1: L extends [infer H] ? H : never,
2: ((...l: L) => any) extends ((h: any, ...t: infer T) => any) ? Last<T> : D,
}[L extends [] ? 0 : L extends [any] ? 1 : 2];
type Append<T extends any[], H> =
((h: H, ...t: T) => any) extends ((...l: infer L) => any) ? L : never;
type Reverse<L extends any[], R extends any[] = []> = {
0: R,
1: ((...l: L) => any) extends ((h: infer H, ...t: infer T) => any) ?
Reverse<T, Append<R, H>> :
never,
}[L extends [any, ...any[]] ? 1 : 0];
type Compose<L extends any[], V, R extends any[] = []> = {
0: R,
1: ((...l: L) => any) extends ((a: infer H, ...t: infer T) => any) ?
Compose<T, H, Append<R, (x: V) => H>>
: never,
}[L extends [any, ...any[]] ? 1 : 0];
export type PipeFunc<T extends any[], V> =
(...f: Reverse<Compose<T, V>>) => ((x: V) => Last<T, V>);(The original code snippet is linked to from here as a possible solution to the issue of _.compose, _.flow, and friends.)
Checklist
My suggestion meets these guidelines:
- This wouldn't be a breaking change in existing TypeScript / JavaScript code
- This wouldn't change the runtime behavior of existing JavaScript code
- This could be implemented without emitting different JS based on the types of the expressions
- This isn't a runtime feature (e.g. new expression-level syntax)
Activity
dead-claudia commentedon Sep 8, 2018
BTW, here's the Wikipedia article on total functional programming, where TS's type system models to a very large extent in my experience (probably unintentionally). And yes, this could enable a type-encoded quicksort across integers and potentially strings if they were made comparable, without even requiring the type system to remain Turing-complete.
After including this, you could then permanently break the Turing-complete aspect by making a similar check with indexed types. Note that AFAICT, the Turing-complete nature is dependent on indexed types not being checked for recursive cases. It may require support for corecursive functions, depending on how much code breaks.
dead-claudia commentedon Sep 8, 2018
Oh, and another reason to address Turing completeness and potentially kill it is because the original "solution" to variadic composition happens to hang when plugged into the playground, due to a bug not very clear when just looking. It maxes out my computer's CPU without taking much memory at all, which is a pretty obvious problem, but it can happen when a type system is Turing-complete, but not sound enough to force you to be explicit about your intent (you see similar with C-like procedural languages and infinite loops with non-trivial logic, hence various control-flow-sensitive lints for it).
jcalz commentedon Sep 14, 2018
(Moving the conversation from #5453)
Regarding the
Last1<>implementation, the official word is something like "don't do it". Anders Hejlsberg (@ahejlsberg) said:I would be enormously in favor of something that lets me express more recursive type functions without running afoul of the current pitfalls (warnings, hanging compiler, crashing compiler). Not sure how to get there from here, though. I suspect if the circularity warning were simply removed, the
Last2<>implementation would bog down the compiler in the same way thatLast1<>does. Assuming that the type checker sometimes treatsT extends U ? X : YlikeX | Yfor testing assignability, any recursion that depends on conditional types to express termination could be a problem.I'm not holding my breath for any solution involving a significant overhaul of the type checker. Can we think of some addition which wouldn't break existing code but would allow some (bounded) type function iteration/recursion?
jcalz commentedon Nov 14, 2018
Copying from the Design Notes for ease of linking.
Daniel Rosenwasser (@DanielRosenwasser) said:
dead-claudia commentedon Mar 28, 2019
May I add another compelling use case: you could lift the similar constraints on unions, intersections, and generic type parameters to address the possible results of
JSON.parse(string)without a reviver:In this case,
JSONResultcan only really be these types, and anything else is entirely unreasonable. You can detect the safety and legality by ensuring that any cycles are abstracted through an object property of some kind. In the case ofRecord<string, JSONResult>, it's a[key: string]property, and in the case ofJSONResult[], it's a[key: number]property.Edit: Missed a type.
YowaiCoder commentedon Jun 28, 2019
jcalz commentedon Jul 2, 2019
cross-linking #30188
And calling @pirix-gh... I think since you are using recursive conditional types inside ts-toolbelt and committed them into the typings for ramda it would be useful for someone to either reiterate that this is a Bad Idea, or give some update if there is one.
And if there ever is an update here past "don't do it" and "we're not ready for this", I'd expect to see some recursive conditional types included in the baseline tests for the language itself, so that future releases won't break them.
millsp commentedon Jul 2, 2019
Can we discuss whether we can have official approval for the types provided by the ts-toolbelt or not?
Here's how the story started:
Performance
I created a heavy test that shows the
Minustype in action. It performs theMinus216000 times in less than 4 seconds. This shows that TS can handle recursive types very well. But this is quite recent.Since when?
This is thanks to Anders 🎉 (#30769). He allowed me to switch from conditional types to indexed conditions (like a switch). And as a matter of fact, it improved performance by x6 (with everything being indexed).
Is it safe?
The lib also makes recursion safe with
Iterationthat will prevent any overflow from TypeScript. In other words, ifIgoes over 40 then it overflows andPos<I>equalsnumber. Thus stopping the recursion safely. Here's an example:Thoughts
Minusoperations to calculate the average cost ofMinus. It took 125 seconds to complete the compilation. So a singleMinusoperation (~30 iterations) takes 9.65e-6 seconds to be performed.What now?
Minus) every time, we could just compute its output once, and this is what would end up in a generatedd.ts. This could (maybe) be determined whether a type is waiting for arguments (can't be computed) or is argument-less (can be computed).Anders Hejlsberg (@ahejlsberg) Ryan Cavanaugh (@RyanCavanaugh) Daniel Rosenwasser (@DanielRosenwasser)
AnyhowStep commentedon Jul 26, 2019
Joe Calzaretta (@jcalz) Circular conditional type police? I would like to turn myself in,
I just wrote a generic
compose()function with no overloads,Playground
[Edit]
Just realized I left in the unnecessary IndexT. It was there because the assertion and actual composition were one type before. I split it into two types and forgot to clean up
[Edit]
Removed
IndexTfromComposeImpl<>7 remaining items
jcalz commentedon Dec 14, 2019
Is there any way to get a version of this that works in the Playground? Or some guidance on how to do this without using an external library? Like maybe we can make a self-contained "ts-toolbelt lite" that can be used inline as opposed to in a module hierarchy? I'm hoping to minimize Stack Overflow and GitHub issue headaches.
AnyhowStep commentedon Dec 14, 2019
Nathan Shively-Sanders (@sandersn) if I made a PR to add trampolines to the tests, would it be accepted?
The following file gives an example of what the test would look like,
https://github.com/AnyhowStep/tsql/blob/master/src/tuple-util/reverse.ts
It doesn't rely on external dependencies to work and can easily go beyond 1000+ iterations/recursive calls
[Edit]
Trampolines saved my life with #33541
#33541 (comment)
Another example of trampolines for recursive types,
https://github.com/AnyhowStep/tsql/pull/65/files#diff-0214b2fefabaf1617e690660683db775R226-R237
millsp commentedon Dec 14, 2019
Joe Calzaretta (@jcalz), I'm working on a script that will make it work on the playground (flattening), will ping when it's done
jack-williams commentedon Dec 16, 2019
There is the possibility that the current implementation relies on variance markers for conditional types, which are known to have bugs. When I've looked at fixing these issues I've run into problems where conditional types that were checked trivially now fail because they are checked structurally.
For instance, when investigating #31277 I got an infinite type error in something originating in
ts-toolbelt. This was pre-3.7, and I haven't looked at how that affects things yet.When I get a chance I'll see if the changes in #31277 caues any regressions in #33810.
sandersn commentedon Dec 16, 2019
Erm, I was the one who merged that test. My intent in adding it to the test suite was not, not to support recursive types. It was to let us (and anybody who wants to follow our user tests) know when we break the latest workarounds. That is, we'll consider that test passing even if there are compilation errors or OOMs.
We have a recursive type limit for a reason — people don't expect a compiler to do arbitrary computation, so compile-time programs that you write need to run fast. This is especially true for commonly-used libraries.
AnyhowStep That particular test is for ts-toolbelt. Is ts-toolbelt trampolined? If not, I'm fine with adding a separate test in index.ts. @pirix-gh is that OK with you?
AnyhowStep commentedon Dec 16, 2019
I don't think ts-toolbelt is trampolined. I can say I have not noticed a negative impact on performance due to trampolines, though.
Trampolines can be an important workaround, even when recursive types are not deeply nested. In the above linked issue (#33541), I only had a recursive type doing two iterations and TS still gave up (unless assigning the result to a temporary variable).
With the trampoline workaround, it works fine with 60+ iterations, whether I assign to a temporary variable or not.
I can make a separate PR that tests trampolines. I didn't even realize it was a thing that could be OK'd
millsp commentedon Dec 16, 2019
Nope, it's not trampolined, one of my goals is keep ts' memory as sane as I can. Recursive types do get limited by the compiler when they get too complex internally, within themselves (instantiation depth). What I do for recursive types, and that is what Nathan Shively-Sanders (@sandersn) means by workaround, is that I use:
This is in a sense a mini-trampoline, to allow combining recursive types together without hitting that instantiation depth error (buildup from the recursive type). I feel like this is already asking a lot from ts. But if you remove the
extends infer Xpart you are left with a fully supported type, as it uses no trick IMHO... However, if I were to remove this on the ts-toolbelt, types likeCurrywould certainly not be possible.So I use
extends infer Xto ask ts to give me more, and is optional when you write your own types.(Just to give an idea, ts-toolbelt standalone weighs
~40MB, and with all its tests, it climbs up to~90MB).millsp commentedon Dec 16, 2019
I think trampolines belongs to a separate PR. I'd be interested to see memory & performance on them AnyhowStep.
AnyhowStep commentedon Dec 16, 2019
I've seen that
infer Xtrick fail in many (of my) use cases. Which is why I opt for a full trampoline nowadays.And another test with,
And the output,
Not bad, considering I just concated two tuples, length 287 each, to become one tuple of length 574 during compile-time. (I don't actually do this in my application code, though).
The most I've done in my projects is concated a tuple of length 50+, I think.
AnyhowStep commentedon Dec 16, 2019
https://github.com/AnyhowStep/ts-trampoline-test
I just realized looking at
Memory usedisn't too helpful for such small examples. When I put theReverseandConcattests together, memory usage was153478K, for a single run.Memory usage for
Concatalone was156981K, for a single run. (Notice it is higher thanConcat + Reversetests combined)Reversealone was85343K, for a single run.@pirix-gh how are you benchmarking performance and memory usage?
And what's your "largest" test? I know you have a hard limit of 50 for each recursive type, but is there a particular test where you compose types until they just become a massive computational nightmare?
jcalz commentedon Jan 11, 2020
I am reminded:
It's too late to decide that we're not ready for recursive conditional types; they're here already. Now let's make them safe and easy to use! Maybe some kind of
Defer<T, D, N>whereTis some arbitrarily self-referencing type,Dis some default type in caseTcan't be resolved by recursing at mostNlevels deep, and you can leave offNand get the maximum value allowed by the compiler, which is something big.Otherwise I either have to keep giving answers like this one and/or impersonating an officer of the law.
jcalz commentedon May 1, 2020
Hmm, look at that! Recursive
flat()! (#32131) Not sure what to make of it.EDIT: #38298 ❗
jcalz commentedon Aug 16, 2020
Oh wow #40002 🌟 🎉 🎆 👍