Repository navigation
rootDirs should merge outputsΒ #44321
Description
Suggestion
π Search Terms
rootDirs merge output
Related: #9875
β Viability Checklist
My suggestion meets these guidelines:
- This wouldn't be a breaking change in existing TypeScript/JavaScript code
- This wouldn't change the runtime behavior of existing JavaScript code
- This could be implemented without emitting different JS based on the types of the expressions
- This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, new syntax sugar for JS, etc.)
- This feature would agree with the rest of TypeScript's Design Goals.
β Suggestion
rootDirs should merge outputs instead of emitting to the folders listed in rootDirs. This would be consistent with rootDir.
π Motivating Example
Starting point: single root
Let's say I start with a project with a src/ folder:
.
βββ src
βββ index.ts
βββ lib
β βββ lib.ts
βββ test
βββ test.ts
It has one rootDir and this config:
{
"compilerOptions": {
"outDir": "./",
"rootDir": "./src",
},
"include": ["src/**/*.ts"],
"exclude": []
}When I compile, the output goes to ./ and mirrors the structure of ./src:
.
βββ lib
β βββ lib.js
βββ src
β βββ index.ts
β βββ lib
β β βββ lib.ts
β βββ test
β βββ test.ts
βββ test
βββ test.js
All well and good.
Friction point: add generated files
Now I want to add some generated .ts files to my project and keep them outside of src/ for easier source control.
I put a .graphql source file in my src/ tree, and set up the generator to output to gen/:
.
βββ gen
β βββ lib
β βββ schema.ts
βββ src
βββ index.ts
βββ lib
β βββ lib.ts
β βββ schema.graphql
βββ test
βββ test.ts
Now I want the same output structure as before, but I want to compile the sources in gen/ and output them as if I had "rootDir": "./gen".
The first thing I try is to replace "rootDir": "./src" with "rootDirs": ["./src", "./gen"]:
{
"compilerOptions": {
"outDir": "./",
"rootDirs": ["./src", "./gen"],
},
"include": ["src/**/*.ts", "gen/**/*.ts"],
"exclude": []
}but this unfortunately defaults rootDir to ./, putting src/ and /gen in the output paths and writing .js files as siblings to their .ts sources:
.
βββ gen
β βββ lib
β βββ schema.js
β βββ schema.ts
βββ src
βββ index.js
βββ index.ts
βββ lib
β βββ lib.js
β βββ lib.ts
β βββ schema.graphql
βββ test
βββ test.ts
βββ test.ts
This is very much not what I want or expect, and a big difference from the behavior of rootDir.
What I expect is that rootDirs acts like a multi-value rootDir, and for each input the root dir that it's in is stripped from the output path.
One additionally confusing aspect of rootDir vs rootDirs is that "rootDir": "./src" is not equivalent to "rootDirs": ["./src"]. The former will cause output files to be siblings to inputs.
It looks like to work around this we need to set "outDir": "./build", and then add an additional build step just to copy the files from ./build/src and ./build/gen into ./. This obviously can work but it's more of a complication than just a simple step: --watch will no longer work correctly unless you wire up the copy step to automatically run when ./build files change. For a project with simple build scripts, this is a pretty big leap in complexity.
Starting package.json:
"scripts": {
"build": "tsc --build"
},Desired addition of a generator:
"scripts": {
"build": "npm run build:graphql && npm run build:ts",
"build:ts": "tsc --build",
"build:graphql": "graphql-codegen"
},Actual addition of a generator, with broken --watch:
"scripts": {
"build": "npm run build:graphql && npm run build:ts && npm run build:copy",
"build:ts": "tsc --build",
"build:graphql": "graphql-codegen",
"build:copy": "cp -r build/{src,gen}/*"
},Actual addition of a generator, with working --watch:
π€·ββοΈ I have to go figure this out still. Also, I'm not sure if this will badly interact with a monorepo, composite projects, and --build. The cross-package import paths are going to be different from the compiler output.
Potential workaround
I tried setting "rootDir": "./(src|gen)", hoping to tell tsc to strip those paths from the output. No luck as that path isn't interpreted as a pattern. That could potentially be a non-breaking way to add merging.
π» Use Cases
Compiling generated files into a common output tree.
Activity
justinfagnani commentedon May 28, 2021
Well, so far it does look like adding the copy steps interferes with composite projects.
To test I made a change to a shared project that causes a break in a project that references it while I was running
tsc --build --watchin the dependent project.When I make the change to the source file, tsc recompiles, but with no errors. (Because the files actually imported by the dependent project are the copies and they haven't changed yet?) If I run the copy step the imported files do change, but the
.tsbuildinfofile hasn't, so tsc doesn't see a change still. If I then kill the watch build and runtsc --build -fthe error is picked up.edit: More complications with a monorepo / composite setup. On a clean build, when a dependent project builds it'll run
tscon the referenced project, but not invoke the copy step, so the imports in the dependent project fail to resolve.It seems like
rootDirsand composite projects just don't mix? It'd be great to see a working example as requested in #37257 I think things would just work ifrootDirsmerged the roots as requested here.RyanCavanaugh commentedon Jun 3, 2021
Stating the obvious, probably: We can't just change this and break everyone's existing builds, so there'd have to be some new opt-in
Less obvious, perhaps: This is tricky. We'd start to have to deal with what to do with conflicting output names. It's also unclear how relative-path module specifiers should work in this world -- does importing
"./foo"mean we'd scan every directory at the same relative point in the hierarchy, or would you only be able to import relative filenames from the input-side path despite there being a larger set of paths that would actually work at runtime?justinfagnani commentedon Jun 3, 2021
Yeah, changing
rootDirswould be a breaking change, I was assuming there'd need to be an opt-in.I think there are good answers to the questions you raise:
This is how other src/gen overlay systems I've used work. It makes imports behave as if the generated files were generated directly into the source folders, which has the essential benefit of abstracting away whether a file is checked-in or generated.