Reduce installed package size

[sindresorhus]

Latest master (b79fdfd), when installed with npm@3.5.2, is 68 MB. Most of that is Babel stuff.

We should look into ways we can reduce the size.

(Output from DaisyDisk)

Turns out every single Babel package includes a dependency on babel-runtime and for some reason it's not deduped by npm@3, not even with npm dedupe:

~/dev/ava/node_modules master*
❯ find . -type d -name "babel-runtime" -print 
./babel-code-frame/node_modules/babel-runtime
./babel-core/node_modules/babel-runtime
./babel-generator/node_modules/babel-runtime
./babel-helper-builder-binary-assignment-operator-visitor/node_modules/babel-runtime
./babel-helper-call-delegate/node_modules/babel-runtime
./babel-helper-define-map/node_modules/babel-runtime
./babel-helper-explode-assignable-expression/node_modules/babel-runtime
./babel-helper-function-name/node_modules/babel-runtime
./babel-helper-get-function-arity/node_modules/babel-runtime
./babel-helper-hoist-variables/node_modules/babel-runtime
./babel-helper-optimise-call-expression/node_modules/babel-runtime
./babel-helper-regex/node_modules/babel-runtime
./babel-helper-remap-async-to-generator/node_modules/babel-runtime
./babel-helper-replace-supers/node_modules/babel-runtime
./babel-helpers/node_modules/babel-runtime
./babel-messages/node_modules/babel-runtime
./babel-plugin-check-es2015-constants/node_modules/babel-runtime
./babel-plugin-syntax-async-functions/node_modules/babel-runtime
./babel-plugin-syntax-exponentiation-operator/node_modules/babel-runtime
./babel-plugin-syntax-object-rest-spread/node_modules/babel-runtime
./babel-plugin-transform-async-to-generator/node_modules/babel-runtime
./babel-plugin-transform-es2015-arrow-functions/node_modules/babel-runtime
./babel-plugin-transform-es2015-block-scoped-functions/node_modules/babel-runtime
./babel-plugin-transform-es2015-block-scoping/node_modules/babel-runtime
./babel-plugin-transform-es2015-classes/node_modules/babel-runtime
./babel-plugin-transform-es2015-computed-properties/node_modules/babel-runtime
./babel-plugin-transform-es2015-destructuring/node_modules/babel-runtime
./babel-plugin-transform-es2015-for-of/node_modules/babel-runtime
./babel-plugin-transform-es2015-function-name/node_modules/babel-runtime
./babel-plugin-transform-es2015-literals/node_modules/babel-runtime
./babel-plugin-transform-es2015-modules-commonjs/node_modules/babel-runtime
./babel-plugin-transform-es2015-object-super/node_modules/babel-runtime
./babel-plugin-transform-es2015-parameters/node_modules/babel-runtime
./babel-plugin-transform-es2015-shorthand-properties/node_modules/babel-runtime
./babel-plugin-transform-es2015-spread/node_modules/babel-runtime
./babel-plugin-transform-es2015-sticky-regex/node_modules/babel-runtime
./babel-plugin-transform-es2015-template-literals/node_modules/babel-runtime
./babel-plugin-transform-es2015-typeof-symbol/node_modules/babel-runtime
./babel-plugin-transform-es2015-unicode-regex/node_modules/babel-runtime
./babel-plugin-transform-exponentiation-operator/node_modules/babel-runtime
./babel-plugin-transform-object-rest-spread/node_modules/babel-runtime
./babel-plugin-transform-regenerator/node_modules/babel-runtime
./babel-plugin-transform-runtime/node_modules/babel-runtime
./babel-plugin-transform-strict-mode/node_modules/babel-runtime
./babel-register/node_modules/babel-runtime
./babel-runtime
./babel-template/node_modules/babel-runtime
./babel-traverse/node_modules/babel-runtime
./babel-types/node_modules/babel-runtime
./babylon/node_modules/babel-runtime

[ariporad]

@sindresorhus: That appears to be a bug on your machine. On mine it works fine:

ls spits out the size kind of weird (in raw bytes, floored), it's actually 33.2MB:

npm ls:

/Users/Ari/Developer/.tmp
└─┬ ava@0.8.0
  ├── arr-flatten@1.0.1
  ├── arrify@1.0.1
  ├─┬ ava-init@0.1.3
  │ ├── arr-exclude@1.0.0
  │ ├── pify@2.3.0
  │ ├─┬ pinkie-promise@2.0.0
  │ │ └── pinkie@2.0.1
  │ ├─┬ read-pkg-up@1.0.1
  │ │ ├─┬ find-up@1.1.0
  │ │ │ └── path-exists@2.1.0
  │ │ └─┬ read-pkg@1.1.0
  │ │   ├─┬ load-json-file@1.1.0
  │ │   │ ├─┬ parse-json@2.2.0
  │ │   │ │ └─┬ error-ex@1.3.0
  │ │   │ │   └── is-arrayish@0.2.1
  │ │   │ └─┬ strip-bom@2.0.0
  │ │   │   └── is-utf8@0.2.1
  │ │   └── path-type@1.1.0
  │ ├── the-argv@1.0.0
  │ └─┬ write-pkg@1.0.0
  │   └─┬ write-json-file@1.2.0
  │     └─┬ sort-keys@1.1.1
  │       └── is-plain-obj@1.1.0
  ├─┬ babel-core@5.8.34
  │ ├── babel-plugin-constant-folding@1.0.1
  │ ├── babel-plugin-dead-code-elimination@1.0.2
  │ ├── babel-plugin-eval@1.0.1
  │ ├── babel-plugin-inline-environment-variables@1.0.1
  │ ├── babel-plugin-jscript@1.0.4
  │ ├── babel-plugin-member-expression-literals@1.0.1
  │ ├── babel-plugin-property-literals@1.0.1
  │ ├── babel-plugin-proto-to-assign@1.0.4
  │ ├── babel-plugin-react-constant-elements@1.0.3
  │ ├── babel-plugin-react-display-name@1.0.3
  │ ├── babel-plugin-remove-console@1.0.1
  │ ├── babel-plugin-remove-debugger@1.0.1
  │ ├── babel-plugin-runtime@1.0.7
  │ ├─┬ babel-plugin-undeclared-variables-check@1.0.2
  │ │ └── leven@1.0.2
  │ ├── babel-plugin-undefined-to-void@1.1.6
  │ ├── babylon@5.8.34
  │ ├── bluebird@2.10.2
  │ ├── convert-source-map@1.1.2
  │ ├── core-js@1.2.6
  │ ├─┬ detect-indent@3.0.1
  │ │ ├── get-stdin@4.0.1
  │ │ └── minimist@1.2.0
  │ ├── esutils@2.0.2
  │ ├── fs-readdir-recursive@0.1.2
  │ ├── globals@6.4.1
  │ ├─┬ home-or-tmp@1.0.0
  │ │ ├── os-tmpdir@1.0.1
  │ │ └── user-home@1.1.1
  │ ├── is-integer@1.0.6
  │ ├── js-tokens@1.0.1
  │ ├── json5@0.4.0
  │ ├─┬ line-numbers@0.2.0
  │ │ └── left-pad@0.0.3
  │ ├── lodash@3.10.1
  │ ├─┬ minimatch@2.0.10
  │ │ └─┬ brace-expansion@1.1.2
  │ │   ├── balanced-match@0.3.0
  │ │   └── concat-map@0.0.1
  │ ├─┬ output-file-sync@1.1.1
  │ │ └─┬ mkdirp@0.5.1
  │ │   └── minimist@0.0.8
  │ ├── path-exists@1.0.0
  │ ├── path-is-absolute@1.0.0
  │ ├── private@0.1.6
  │ ├─┬ regenerator@0.8.40
  │ │ ├─┬ commoner@0.10.4
  │ │ │ ├─┬ commander@2.9.0
  │ │ │ │ └── graceful-readlink@1.0.1
  │ │ │ ├─┬ detective@4.3.1
  │ │ │ │ ├── acorn@1.2.2
  │ │ │ │ └── defined@1.0.0
  │ │ │ ├── glob@5.0.15
  │ │ │ ├── iconv-lite@0.4.13
  │ │ │ └── q@1.4.1
  │ │ ├─┬ defs@1.1.1
  │ │ │ ├─┬ alter@0.2.0
  │ │ │ │ └── stable@0.1.5
  │ │ │ ├── ast-traverse@0.1.1
  │ │ │ ├── breakable@1.0.0
  │ │ │ ├── simple-fmt@0.1.0
  │ │ │ ├── simple-is@0.2.0
  │ │ │ ├── stringmap@0.2.2
  │ │ │ ├── stringset@0.2.1
  │ │ │ ├── tryor@0.1.2
  │ │ │ └─┬ yargs@3.27.0
  │ │ │   ├── camelcase@1.2.1
  │ │ │   ├─┬ cliui@2.1.0
  │ │ │   │ ├─┬ center-align@0.1.2
  │ │ │   │ │ ├─┬ align-text@0.1.3
  │ │ │   │ │ │ ├─┬ kind-of@2.0.1
  │ │ │   │ │ │ │ └── is-buffer@1.1.0
  │ │ │   │ │ │ └── repeat-string@1.5.2
  │ │ │   │ │ └── lazy-cache@0.2.7
  │ │ │   │ └── right-align@0.1.3
  │ │ │   ├── decamelize@1.1.2
  │ │ │   ├─┬ os-locale@1.4.0
  │ │ │   │ └─┬ lcid@1.0.0
  │ │ │   │   └── invert-kv@1.0.0
  │ │ │   ├── window-size@0.1.4
  │ │ │   └── y18n@3.2.0
  │ │ ├── esprima-fb@15001.1001.0-dev-harmony-fb
  │ │ ├─┬ recast@0.10.33
  │ │ │ └── ast-types@0.8.12
  │ │ └── through@2.3.8
  │ ├─┬ regexpu@1.3.0
  │ │ ├── esprima@2.7.1
  │ │ ├── regenerate@1.2.1
  │ │ ├── regjsgen@0.2.0
  │ │ └─┬ regjsparser@0.1.5
  │ │   └── jsesc@0.5.0
  │ ├── repeating@1.1.3
  │ ├── resolve@1.1.6
  │ ├── shebang-regex@1.0.0
  │ ├── slash@1.0.0
  │ ├── source-map@0.5.3
  │ ├─┬ source-map-support@0.2.10
  │ │ └── source-map@0.1.32
  │ ├── to-fast-properties@1.0.1
  │ ├── trim-right@1.0.1
  │ └── try-resolve@1.0.1
  ├─┬ babel-plugin-espower@1.1.0
  │ ├── array-find@1.0.0
  │ ├─┬ escallmatch@1.4.2
  │ │ ├── deep-equal@1.0.1
  │ │ ├── esprima@2.7.1
  │ │ ├─┬ espurify@1.5.0
  │ │ │ └── isarray@1.0.0
  │ │ └── indexof@0.0.1
  │ ├─┬ espower@1.2.1
  │ │ ├─┬ escodegen@1.7.1
  │ │ │ ├── esprima@1.2.5
  │ │ │ ├── estraverse@1.9.3
  │ │ │ ├─┬ optionator@0.5.0
  │ │ │ │ ├── deep-is@0.1.3
  │ │ │ │ ├── fast-levenshtein@1.0.7
  │ │ │ │ ├── levn@0.2.5
  │ │ │ │ ├── prelude-ls@1.1.2
  │ │ │ │ ├── type-check@0.3.1
  │ │ │ │ └── wordwrap@0.0.2
  │ │ │ └── source-map@0.2.0
  │ │ ├── is-url@1.2.1
  │ │ └── isarray@0.0.1
  │ └── xtend@4.0.1
  ├── babel-runtime@5.8.34
  ├── bluebird@3.1.1
  ├─┬ chalk@1.1.1
  │ ├── ansi-styles@2.1.0
  │ ├── escape-string-regexp@1.0.4
  │ ├─┬ has-ansi@2.0.0
  │ │ └── ansi-regex@2.0.0
  │ ├── strip-ansi@3.0.0
  │ └── supports-color@2.0.0
  ├─┬ co-with-promise@4.6.0
  │ └─┬ pinkie-promise@1.0.0
  │   └── pinkie@1.0.0
  ├─┬ core-assert@0.1.3
  │ └── buf-compare@1.0.0
  ├─┬ debug@2.2.0
  │ └── ms@0.7.1
  ├── deeper@2.1.0
  ├─┬ empower@1.1.0
  │ ├── array-filter@1.0.0
  │ ├── array-foreach@1.0.1
  │ ├── array-map@0.0.0
  │ ├── array-some@1.0.0
  │ ├─┬ define-properties@1.1.2
  │ │ └── foreach@2.0.5
  │ └─┬ object-create@0.1.0
  │   └─┬ object-define-property@0.1.0
  │     ├── function-bind@0.1.0
  │     └── has@0.0.1
  ├─┬ empower-core@0.2.0
  │ └── call-signature@0.0.2
  ├── figures@1.4.0
  ├── fn-name@2.0.1
  ├─┬ globby@4.0.0
  │ ├─┬ array-union@1.0.1
  │ │ └── array-uniq@1.0.2
  │ └─┬ glob@6.0.2
  │   ├─┬ inflight@1.0.4
  │   │ └── wrappy@1.0.1
  │   ├── inherits@2.0.1
  │   └── once@1.3.3
  ├── has-generator@1.0.0
  ├── is-generator-fn@1.0.0
  ├─┬ is-observable@0.1.0
  │ └── symbol-observable@0.1.0
  ├── is-promise@2.1.0
  ├─┬ loud-rejection@1.2.0
  │ └── signal-exit@2.1.2
  ├── max-timeout@1.0.0
  ├─┬ meow@3.6.0
  │ ├─┬ camelcase-keys@2.0.0
  │ │ ├── camelcase@2.0.1
  │ │ └── map-obj@1.0.1
  │ ├── minimist@1.2.0
  │ ├─┬ normalize-package-data@2.3.5
  │ │ ├── hosted-git-info@2.1.4
  │ │ ├─┬ is-builtin-module@1.0.0
  │ │ │ └── builtin-modules@1.1.0
  │ │ ├── semver@5.1.0
  │ │ └─┬ validate-npm-package-license@3.0.1
  │ │   ├─┬ spdx-correct@1.0.2
  │ │   │ └── spdx-license-ids@1.1.0
  │ │   └─┬ spdx-expression-parse@1.0.2
  │ │     └── spdx-exceptions@1.0.4
  │ ├─┬ redent@1.0.0
  │ │ ├─┬ indent-string@2.1.0
  │ │ │ └── repeating@2.0.0
  │ │ └── strip-indent@1.0.1
  │ └── trim-newlines@1.0.0
  ├── object-assign@4.0.1
  ├── observable-to-promise@0.1.0
  ├─┬ plur@2.1.2
  │ └── irregular-plurals@1.1.0
  ├─┬ power-assert-formatter@1.3.2
  │ ├── acorn@2.6.4
  │ ├── acorn-es7-plugin@1.0.11
  │ ├── array-reduce@0.0.0
  │ ├── eastasianwidth@0.1.1
  │ ├── estraverse@4.1.1
  │ ├── googlediff@0.1.0
  │ ├── object-keys@1.0.9
  │ ├─┬ stringifier@1.2.1
  │ │ ├── array-reduce-right@1.0.0
  │ │ └── traverse@0.6.6
  │ └── type-name@1.1.0
  ├── power-assert-renderers@0.1.0
  ├─┬ pretty-ms@2.1.0
  │ ├─┬ is-finite@1.0.1
  │ │ └── number-is-nan@1.0.0
  │ ├── parse-ms@1.0.0
  │ └── plur@1.0.0
  ├── require-from-string@1.1.0
  ├─┬ resolve-cwd@1.0.0
  │ └── resolve-from@2.0.0
  ├── serialize-error@1.1.0
  ├── set-immediate-shim@1.0.1
  ├─┬ source-map-support@0.4.0
  │ └─┬ source-map@0.1.32
  │   └── amdefine@1.0.0
  ├─┬ squeak@1.3.0
  │ ├── console-stream@0.1.1
  │ └─┬ lpad-align@1.1.0
  │   ├── longest@1.0.1
  │   └── lpad@2.0.1
  ├─┬ time-require@0.1.2
  │ ├─┬ chalk@0.4.0
  │ │ ├── ansi-styles@1.0.0
  │ │ ├── has-color@0.1.7
  │ │ └── strip-ansi@0.1.1
  │ ├── date-time@0.1.1
  │ ├─┬ pretty-ms@0.2.2
  │ │ └── parse-ms@0.1.2
  │ └── text-table@0.2.0
  └─┬ update-notifier@0.5.0
    ├─┬ configstore@1.4.0
    │ ├── graceful-fs@4.1.2
    │ ├─┬ osenv@0.1.3
    │ │ └── os-homedir@1.0.1
    │ ├── uuid@2.0.1
    │ ├─┬ write-file-atomic@1.1.4
    │ │ ├── imurmurhash@0.1.4
    │ │ └── slide@1.1.6
    │ └── xdg-basedir@2.0.0
    ├── is-npm@1.0.0
    ├─┬ latest-version@1.0.1
    │ └─┬ package-json@1.2.0
    │   ├─┬ got@3.3.1
    │   │ ├─┬ duplexify@3.4.2
    │   │ │ ├── end-of-stream@1.0.0
    │   │ │ └─┬ readable-stream@2.0.5
    │   │ │   ├── core-util-is@1.0.2
    │   │ │   ├── isarray@0.0.1
    │   │ │   ├── process-nextick-args@1.0.6
    │   │ │   ├── string_decoder@0.10.31
    │   │ │   └── util-deprecate@1.0.2
    │   │ ├── infinity-agent@2.0.3
    │   │ ├── is-redirect@1.0.0
    │   │ ├── is-stream@1.0.1
    │   │ ├── lowercase-keys@1.0.0
    │   │ ├── nested-error-stacks@1.0.2
    │   │ ├── object-assign@3.0.0
    │   │ ├── prepend-http@1.0.3
    │   │ ├─┬ read-all-stream@3.0.1
    │   │ │ └─┬ pinkie-promise@1.0.0
    │   │ │   └── pinkie@1.0.0
    │   │ └── timed-out@2.0.0
    │   └─┬ registry-url@3.0.3
    │     └─┬ rc@1.1.5
    │       ├── deep-extend@0.4.0
    │       ├── ini@1.3.4
    │       ├── minimist@1.2.0
    │       └── strip-json-comments@1.0.4
    ├── semver-diff@2.1.0
    └── string-length@1.0.1

So I'm really not sure why you're seeing that. Maybe file a bug report with npm?

[jamestalmage]

Actually, it is only ~16.5 MB, @ariporad's number includes dev-dependencies. You need to delete node_modules and run

$ npm install --production

If you create a new project and just do this:

$ npm init
# default for every prompt
$ npm i babel-core babel-preset-es2015 babel-preset-stage-2

I get a relatively svelte 6.8 MB.

Adding babel-plugin-espower brings me to 9.8 MB. That seems like a pretty significant jump. @twada - Any ideas how we can reduce this?

Adding babel-runtime@6.X.X does bump me up to 14 MB, but that is due to massive duplication of babel-runtime@5.X.X. It gets copied into the node_modules folder of every babel-plugin-transform-*. This is expected, since current babel plugins are written in ES6, compiled with babel@5, and maintain a dev-dependency on babel-runtime@5. This really does not hurt your install times though, since all those duplicates are definitely coming from the cache (babel-runtime is not re-downloaded 20 times).

2.5 MB for the remaining 30 some dependencies we have seems pretty reasonable.

[ariporad]

@jamestalmage: that's strange, I did go above and beyond my usual intelligence level and create a new project to install AVA, but I'll try again when I'm home.

[novemberborn]

What are we hoping to achieve here? With changing dependencies this is a never ending struggle.

That said, looks like all Babel dependencies require babel-runtime@^5, whereas we explicitly install ^6. There's FIFTY copies of babel-runtime@5.8.35.

Presumably we could downgrade to ^5 for the time being and we'd save a lot space. In my testing that takes us from 77MB to 29MB.

Of course there's other dependencies doing weird stuff, like fsevents which pulls in 7MB just for node-pre-gyp.

---

To measure I did a fresh install from NPM (in an empty directory) and ran du -k -d 1 node_modules | sort -n -r. Also check out snyk-resolve which shows the actual dependencies, not deduped.

[sindresorhus]

@novemberborn There are some things that can be done even if the worst offender, Babel and fsevents, are out of our control. Many packages include junk like tests or examples directory, etc. I did a PR marathon a few years ago for Yeoman on 100+ different packages adding a "files" property to their package.json to reduce the total disk size of the dependency tree. That helped a lot. Pretty much have this still open for when I have time to do the same here.

[novemberborn]

Pretty much have this still open for when I have time to do the same here.

Sure, fair enough. Somebody should write a tool for that 😜

Also thinking about it more I'm doubtful we can change our babel-runtime version. Presumably the runtime transformer plugin builds against v6, not v5.

[novemberborn]

Also thinking about it more I'm doubtful we can change our babel-runtime version. Presumably the runtime transformer plugin builds against v6, not v5.

Actually if we have a dependency which merely reexports babel-runtime it would allow npm to dedupe babel-runtime@5 instead.

[jamestalmage]

Actually if we have a dependency which merely reexports babel-runtime it would allow npm to dedupe babel-runtime@5 instead.

I thought about that. Actually, I think it would be better if Babel did that (all the Babel 6 plugins should depend on a differently named module that exports babel-runtime@5). That way everyone benefits.

[sindresorhus]

Actually, I think it would be better if Babel did that (all the Babel 6 plugins should depend on a differently named module that exports babel-runtime@5). That way everyone benefits.

👍