Don't include all @types/ automatically, only dependencies from package.json

[ivogabe]

The compiler currently picks all type declarations from node_modules/@types. This causes that when a dependency adds some @types package, this is automatically added to the compilation. However, this is not desired if the dependency is not used, for instance since it is only used in the build process. This could cause a lot of issues; imagine that a dependency of a dependency (etc.) adds an @types dependency.

Real world example: gulp-typescript is now using @types for its dependencies; @types/node is now a dependency of gulp-typescript. This caused that @types/node is now included in the projects of users of gulp-typescript, and TypeScript will automatically find this.

Currently users can either include all @types (default) or include a specific list (by setting "types": [ ... ]. My suggestion is something between those options:

Suggestion

Instead of picking all files of node_modules/@types, first check if a package.json file exist in the current directory or a parent directory. If so, read that file and include all @types that are registered as a dependency or devDependency. If the package.json file does not exist, fall back to the current behaviour.

If a project does include gulp-typescript for instance, the compiler will read the following line:

/// <reference types="node" />

The compiler will automatically include this when run with the declaration option. So, types will only be added if they are really used either by the project itself, or by one of the dependencies (recursively).

I think that this would solve a lot of issues and make it easier for projects to migrate to @types. What do you guys think?

[blakeembrey]

I like the idea of being more explicit, but personally I'd go further and avoid tying it to package.json (which would be yet another surprising behaviour for people - e.g. when prototyping in my tmp/ts-test directory it might read upward to a tmp/package.json and include types I don't want). I'd love if the default was instead to include no globals and they all had to be explicitly listed, but maybe that's just me 😄

[aluanhaddad]

@blakeembrey Not it is not just you.
I think there is a conceptual question here of how to define what the extent of a project actually is. Is it package.json, tsconfig.json, cwd?

There are so many different interpretations with so many different answers. ES Modules are pretty silent on this so we can't look to them for guidance.
In some ways tsconfig files seem like a natural fit but they are purely a TypeScript convention so they don't help if you are depending on JavaScript projects via allowJs and with many other scenarios. Also they need to be interpreted by many different kinds of tools for different purposes. This relates to #3469 and there is a lot of discussion about the problems with this approach.

What about package.json?. It gets sticky when you consider the scope of tools like npm which are used to install packages but don't really define the scope of a project the way one might want them to. For example, we may use npm to install packages, but it is Node which resolves them, and Node doesn't care about package.json. So if you don't have a dependency on a package listed in your package.json but it happens to, incidentally, exist on disk (for example because of a bad merge resolution or neglecting to prune and install after checkout) Node and TypeScript (in mirroring Node's behavior) will happily resolve it.

Then you have RequireJS, other AMD loaders, and SystemJS. These all have a clearer, although arguably still vague, definition of what a project or package is, with explicitly declared dependencies. This is very different from how Node works as the latter treats the file system as a dependency source, which may make more sense on the server (but is still probably a bad idea), and then falls back to global installations based on environmental variables and so on and so forth.

This all leaves a lot to be desired. Something Allen Wirfs-Brock has talked about is the lack of any metadata for ES Modules, maybe something like that in a future specification could help here, but it is difficult to say.

[ivogabe]

@blakeembrey

I like the idea of being more explicit, but personally I'd go further and avoid tying it to package.json (which would be yet another surprising behaviour for people - e.g. when prototyping in my tmp/ts-test directory it might read upward to a tmp/package.json and include types I don't want)

TypeScript will currently include those types, since it searches for the first node_modules directory in a parent directory. So this surprising behaviour will already occur.

I'd love if the default was instead to include no globals and they all had to be explicitly listed, but maybe that's just me 😄

I agree about that, but I'm afraid that that is annoying to users. For instance when they want to use jQuery, they should install jQuery itself, install @types/jquery and set "types": ["jquery"] (assuming they use jQuery as a global variable). Anyway, you can already opt-in to that behaviour using "types": [] and list all used types between the brackets.

@aluanhaddad I also don't like to add another configuration file to TypeScript, but, package.json will already contain a list of the used @types packages. They are already listed as a dependency or a devDependency.

I think that you should make a difference between explicitly resolving packages (using import or <reference />) and implicitly resolving packages (how global @types packages are resolved). Your examples are only referring to importing packages that are not listed in the package.json file, but that are accidentally in the node_modules directory. This causes that something works, whereas it shouldn't work. For @types this is the other way around: by including some other package, your compilation may fail whereas it would compile if the package was not included.

@types packages can break a compilation; you could get 'duplicate identifier' errors. Node overrides the types for setTimeout and setInterval, which can also break code using these functions.

[aluanhaddad]

@ivogabe Indeed, I was going off on a tangent about how TypeScript and JavaScript lack a formal definition of projects, packages, and scopes. Regardless, I apologize for going off topic.

package.json is the closest thing to defining a project that we have and I think the your suggestion has a lot of merit, but I think transitive dependencies should be resolvable when you navigating the source of direct dependencies. That is there should be an implicit scope for module resolution across packages.

[blakeembrey]

TypeScript will currently include those types

I agree, which there's already dozens of issues about. I'm suggesting trying to avoid adding new surprising behaviour.

Anyway, you can already opt-in to that behaviour using "types": [] and list all used types between the brackets.

I agree again, but this issue is about the defaults and the open issues suggest the current default could use improving. Sure, every user could just do that, but it's not a valuable default in that case.

your compilation may fail whereas it would compile if the package was not included.

I'm not sure I follow this, it's really the same use-case. They are both results of implicit behaviour, but if it helps imagine that I have a parent tmp direct that contains both @types/mocha and @types/jasmine and does fail as a result. I'd love to remove implicit behaviour, as that's what I see as the source of confusion. The current explicit behaviours I don't think are a problem, only the implicit ones.

Edit: To be clearer, I also think your idea is already a better behaviour than the current one, I just want to propose taking it further. Using package.json as the wall isn't terribly consistent with how node resolves.

[reshadi]

I think this issue is not related to 'definition of project' and should not be resolved by looking at package.json, tsconfig.json, or current root dir.

A big part of using es6 module system is to simplify reusing code on client and server. That means you may have one package.json in root folder, a "lib" folder for shared code, a "client" folder for client code, and a "server" folder for server code.
In you build, you will then invoke tsc twice, each time to compile certain "main" file in client/server folders.

The tsc "discovers" the list of files that need to be included in the compilation by walking the import statements. It already has an algorithm for module resolution, which it uses to find the .ts or .d.ts files to be include. All we need here is to add the node_modules/@types/* to the list of directories to search for .d.ts files. In other words, first try to resolve the types in the node_modules/ and then try node_modules/@types with exact same algorithm.
This way, the right types will be loaded on demand and there is no need to make implicit assumptions.

[aluanhaddad]

Those directories are already implicitly in the list of directories that TypeScript enumerates.

It is an implicit assumption to resolve packages from there. That is NodeJS behavior, not JavaScript behavior and not ESModule behavior.

The point about inferring dependency locations by walking import statements breaks down whenever imports are neither relatives nor absolute. This is the case when importing any external dependency, say $ from "jquery". "jquery" is just an abstract identifier and ES Modules don't help us here except in that they allow for configuration mapping.

[reshadi]

What I'm suggesting is that loading of these types can follow --moduleResolution value. If module resolution is "node", then the spec defines (https://www.typescriptlang.org/docs/handbook/module-resolution.html) which folders and in what order are searched. I think if the search include the node_module/@types as well and load types files lazily, this particular problem goes away.

[aluanhaddad]

It already includes node_module/@types as of TypeScript 2.0.

The only available resolution algorithms are "node" and "classic" and the latter is problematic for many reasons.

Could you clarify what you mean by load types files lazily? I'm not sure I follow.

[aluanhaddad]

I would like the ability to disable or configure the recursive nature of the search. For example when I set traceResolution I can observe the language service searching node_modules and node_modules/@types directories that exist many directories above the baseUrl or rootDir(s) I have specified, all the way up to the root of the file system. If I'm not using npm this is pointless. Even if I am using it, it doesn't imply that I want to walk all the way up to the root.

[reshadi]

Could you clarify what you mean by load types files lazily? I'm not sure I follow.

What I mean is that compiler should not load all files in the @types folder. Instead, it should only load the content of node_modules/@types/X only if:

• it sees an "import * as x from 'X';" statement
• it cannot find corresponding .ts or .d.ts files in the node_modules/X folders (according to the search algorithm for "node" resolution mode.

In this way, only the types of the modules that are actually referenced in the code are used.

The example I provided in #11949 shows a simple case where the code itself is compiled with es5, but an independent test runner modules needs to be compiled (later) with es6 target. These are all part of the same project, but as of now, tsc compilation fails.

[ivogabe]

I like the suggestion of @blakeembrey, as it would make everything more predictable. It is more restrictive however. I'd like to hear what others think about it.

@aluanhaddad No problem, it does add some important context to this topic.

@reshadi What you are suggesting is already implemented for modules (packages that you can include/require). The problem here is: how should ambient/global types (libraries that are exposed as global variables) be loaded? If your code contains import * as foo from "foo", the compiler will know that the types for foo are required, but if the code contains process, how should it see that the types for NodeJS are required?

The current solution is to include everything in node_modules/@types. As you pointed out, that means that unused modules are included too. You can disable that in your project by setting "types": [].

My initial suggestion was to read package.json to find those files, @blakeembrey suggested to default to "types": [] (meaning that nothing is imported).

[reshadi]

@ivogabe, in any type of application, we can assume there is a main or top module that imports others. One can explicitly say "import '@types/node';" to get the proper types picked up. Same can be done for a browser application. In fact, in this case, you might say the --types flag is redundant since the application could have a simple file that imports all these @types and pass it to the compiler.

Imagine, you may also want to have a WinJs, Cordova, Chrome Only, .... type of application. In those cases as well, the "main" of the application can import proper types at first and then import the rest of the code.

In fact, we ran into this problem in our project. An external module had an issue with Google Closure. While waiting for upstream to fix it, we use a modified local copy, but for the types, simply use the following to mix local copy with existing types:

import './lib_name';
import '@types/lib_name';

setting "types": [] could be a work around, but requires the knowledge of all necessary types to be imported explicitly.