Webpack without bundling for server-side compilation

[amirmohsen]

Do you want to request a feature or report a bug?
Feature

What is the current behavior?
Webpack always bundles files and this is not a desirable behaviour when compiling server-side code. I know that it sounds like an oxymoron to ask for it not to bundle since webpack is a "bundler".

What is the expected behavior?
It would be great if webpack could accept an option to only compile files and then output them back out while preserving the directory structure, essentially skipping the bundling step.

If this is a feature request, what is motivation or use case for changing the behavior?
When writing a universal react application, the entire react app is shared between client and server. Moreover, may other files such as schemas, validation logic, utils, api clients, etc are also shared and need to be compiled separately for the browser and node.js. Currently, I have two webpack config files, one for client and one for server. This generates two bundled files.

While this setup works fine for now, it does negatively impact my development speed. Node.js code is really not meant to be bundled. For instance, after bundling, you can't use fs.read with a relative path anymore. The biggest issue, however, which has led me on a path to find an alternative is the terrible debugging experience. With recent versions of Node, we can use Chrome Dev Tools to debug our server application. Unfortunately, due to limitations of the dev tools, we don't have acccess to source maps when debugging the server code. Believe it or not, this is a huge nightmare when trying to debug server code and all you have is one massive webpack-generated file. There's an open issue about this problem on node.js' github repo but so far, there doesn't seem to be any fix.

I can use something like gulp-babel to compile my server code but that won't handle assets like webpack does so it won't work for react ssr. Moreover, gulp doesn't "crawl" the import statements like webpack does so I have to completely separate my server and client code into separate folders (or use a silly naming convention like ".server.js") which again causes many issues when it comes to shared files. Probably more than 50% of my application is shared files so you can imagine what a nightmare that would be trying to separate client and server.

If webpack had an option to disable the bundling step and just output files after compilation, then this problem would be resolved.

[DrewML]

@amirmohsen See https://github.com/DrewML/webpack-emit-all-plugin

[mucsi96]

I think this can be useful also for publishing some libraries to npm as not bundled ES6 modules. Then the consumer can do much better tree shaking with own bundler.

[vehsakul]

There is a plenty of posts on the internet saying "Ha-ha! webpack + node + etc. Works like a charm!". Unfortunately it doesn't. Can somebody please share some working dev configs that allow to develop a project using node + ts + express? Not bundling code, of course. Not necessarily webpack. Thank you!

[sandangel]

Hello everyone, may I ask is there any update for this issue?

[frenzzy]

If I understand correctly this feature is available since webpack v5 alpha 3:

SplitChunks for single-file-targets

Targets that only allow to startup a single file (like node, WebWorker, electron main) now support loading the dependent pieces required for bootstrap automatically by the runtime.

This allows to use splitChunks for these targets with chunks: "all".

[andykais]

@frenzzy can you explain how chunks: "all" accomplishes this? I have tried multiple configurations under target: "node" which still bundle the modules. To be clear, this is the goal

given this input:

src/
  index.js
  morecode/
    index.js
    helper.js

webpack should output the transpiled code while preserving directory structure:

dist/
  index.js
  morecode/
    index.js
    helper.js

if it is possible to give us an example repo that would be much appreciated.

webpack-emit-all-plugin is currently busted DrewML/webpack-emit-all-plugin#8 so I can't verify that the plugin method works either.

[andykais]

just want to share with everyone my working solution:

const glob = require('glob')
const path = require('path')

module.exports = {
  entry: glob.sync('./src/**/*.js', { ignore: './src/**/*.test.js' }).reduce((acc, file) => {
    acc[file.replace(/^\.\/src\/(.*?)\.js$/, (_, filename) => filename)] = file
    return acc
  }, {}),
  output: {
    path: path.resolve(__dirname, 'dist'),
    filename: '[name].js'
  }
}

what this does is make an entrypoint for every file in the src/ directory (excluding files that end in .test.js)

[edit]
So this solution does create separate files for each source file, but the downside is there is no shared code between the outputs, which I suppose is to be expected. This means that there is a ton of extra code generated, and if you use singletons in your modules, they will not share state if you require different files from the build folder. Essentially each file will be a self contained project.

[etienne-martin]

@andykais It will still bundle everything together.

[andykais]

@etienne-martin see my comment at the bottom of my previous message. It does in fact create separate output files

but the downside is there is no shared code between the outputs

[korniychuk]

I found a fork of webpack-emit-all-plugin that works with WebPack 4.41.6.
https://github.com/bv-loveholidays/webpack-emit-all-plugin/blob/master/index.js
However, this plugin looks like a hack, it doesn't generate correctly source maps and doesn't resolve all imports properly.

[DanielOrtel]

Just to add to the list of reasons why this would be a useful thing to have:

• Webpack has a mature ecosystem with a number of great tools. It isn't just a bundler tool when considering all of it's plugins, so it makes sense that we'd want to use it to build, but not bundle, the backend code.
• In a monorepo environment, where the frontend, backend, admin apps and custom libraries are all kept in a single repo, using a completely different build process just for the backend, simply because webpack doesn't support not-bundling, can add significant overhead to project maintenance. Plus it introduces additional unnecessary dev dependencies. (ex. libraries and frontend both utilize copy-webpack-plugin, but backend needs to come up with it's own solution).

[andykais]

adding to those reasons:
if you want your webpacked library to emit typescript types, either you have to use one of the declaration bundler tools (which are heavy at best, and buggy at worst), or you will just use tsc and have declaration files matching your src files while your actual code is just a single bundle.

[franleplant]

I support this issue and have been supporting it for a while.

This also relates to libraries, libraries not always need to be bundled (even if they have front end assets), they only need to be compiled (assets inlined, typescript, babel, emotion, etc).

This also allows for a more granular use of the library when needed.
And of course this only works when the consumer app of the library has a bundler in place, for any other consumer you will need effectively a bundle, but given how the former is such a common practice it is arguably a really common use case.

[moltar]

I have a use case where I am bundling a lambda function for database migrations. Each database migration is a separate file. The migration tool scans a directory of files and performs these migrations. Since webpack bundles the whole thing as one file, the tool cannot find these files on disk.

I wish it was possible to opt out from bundling everything and keep files separate.

[RedVelocity]

This feature is so much needed on server side development, currently having to resort to separate babel, rimraf libraries rather than a single webpack config to transpile server side code.

[alexander-akait]

Yes, it will be great feature, we will consider it after the stable webpack 5 release

[odykyi]

Solution for serverless-framework, can be also used for express, koa, etc.. just replace slsw.lib.entries to your entries.
force: true attribute inside CopyPlugin works very nice.
Make sure that your entrypoint file called index.js

const path = require('path');
const slsw = require('serverless-webpack');
const nodeExternals = require('webpack-node-externals');
const CopyPlugin = require("copy-webpack-plugin");

const plugins = [
  new CopyPlugin({
    patterns: Object.keys(slsw.lib.entries).map(el => {
      const srcFolder = slsw.lib.entries[el].replace('index.js', '');
      return { from: srcFolder, 
        to: srcFolder,
        force: true, }
    })
  }),
];
console.log('plugins', plugins[0]);

module.exports = {
  entry: slsw.lib.entries,
  target: 'node',
  optimization: {
    minimize: false,
    runtimeChunk: true,
  },
  mode: slsw.lib.webpack.isLocal ? 'development' : 'production',
  plugins,
  devtool: false,
  externals: [nodeExternals(), 'aws-sdk', 'mjml', 'handlebars'], // exclude Lambda Layers
  output: {
    libraryTarget: 'commonjs',
    path: path.resolve(__dirname, '.webpack'),
    filename: '[name].js',
  },
};

[B4bharat]

@odykyi Firstly, thanks a lot for this. I tried it out, but sadly it didn't work for me. I have post my serverless.ts, tsconfig.json and webpack.config.js file below, kindly let me know what am I doing wrong.

This is the error I get Error: offline: handler 'handler' in /Users/bharatpoptwani/projects/nextsteps/gw-api-public/dist/service/src/index is not a function

webpack.config.js

const serverlessConfiguration: AWS = {
  service: 'gw-api-public',
  frameworkVersion: '2',
  custom: {
    webpack: {
      webpackConfig: './webpack.config.js',
      includeModules: {
        packagePath: 'package.json',
      },
    },
  },
  plugins: ['serverless-webpack', 'serverless-offline'],
  provider: {
    name: 'aws',
    runtime: 'nodejs14.x',
    region: 'ap-southeast-1',
    stage: 'dev',
    apiGateway: {
      minimumCompressionSize: 1024,
      shouldStartNameWithService: true,
    },
    environment: {
      AWS_NODEJS_CONNECTION_REUSE_ENABLED: '1',
    },
    lambdaHashingVersion: '20201221',
  },
  // import the function via paths
  functions: {
    app: {
      handler: 'src/app.handler',
      events: [
        {
          http: {
            method: 'any',
            path: '/',
          },
        },
        {
          http: 'ANY {proxy+}',
        },
      ],
    },
  },
};

module.exports = serverlessConfiguration;

tsconfig.json

{
  "extends": "./tsconfig.paths.json",
  "compilerOptions": {
    "lib": ["ESNext"],
    "moduleResolution": "node",
    "emitDecoratorMetadata": true,
    "experimentalDecorators": true,
    "noUnusedLocals": false,
    "noUnusedParameters": false,
    "removeComments": true,
    "sourceMap": true,
    "allowSyntheticDefaultImports": true,
    "target": "ES2020",
    "outDir": "lib",
    "resolveJsonModule": true
  },
  "include": ["src/**/*.ts", "serverless.ts"],
  "exclude": [
    "node_modules/**/*",
    ".serverless/**/*",
    ".webpack/**/*",
    "_warmup/**/*",
    ".vscode/**/*"
  ],
  "ts-node": {
    "require": ["tsconfig-paths/register"]
  }
}

webpack.config.js

const path = require('path');
const slsw = require('serverless-webpack');
const nodeExternals = require('webpack-node-externals');
const TsconfigPathsPlugin = require('tsconfig-paths-webpack-plugin');
const CircularDependencyPlugin = require('circular-dependency-plugin');
const CopyPlugin = require('copy-webpack-plugin');
const _ = require('lodash');

/*
This line is only required if you are specifying `TS_NODE_PROJECT` for whatever reason.
 */
// delete process.env.TS_NODE_PROJECT;

module.exports = {
  context: __dirname,
  mode: slsw.lib.webpack.isLocal ? 'development' : 'production',
  entry: slsw.lib.entries,
  devtool: slsw.lib.webpack.isLocal
    ? 'eval-cheap-module-source-map'
    : 'source-map',
  resolve: {
    extensions: ['.mjs', '.json', '.ts'],
    symlinks: false,
    cacheWithContext: false,
    plugins: [
      new TsconfigPathsPlugin({
        configFile: './tsconfig.paths.json',
      }),
    ],
  },
  output: {
    libraryTarget: 'commonjs',
    path: path.join(__dirname, 'dist'),
    filename: '[name].js',
  },
  optimization: {
    minimize: false,
    runtimeChunk: true,
  },
  target: 'node',
  externals: [nodeExternals()],
  module: {
    rules: [
      // all files with a `.ts` or `.tsx` extension will be handled by `ts-loader`
      {
        test: /\.(tsx?)$/,
        loader: 'ts-loader',
        exclude: [
          [
            path.resolve(__dirname, 'node_modules'),
            path.resolve(__dirname, '.serverless'),
            path.resolve(__dirname, '.webpack'),
          ],
        ],
        options: {
          transpileOnly: true,
          experimentalWatchApi: true,
        },
      },
    ],
  },
  experiments: {
    topLevelAwait: true,
  },
  plugins: [
    new CircularDependencyPlugin({
      // exclude detection of files based on a RegExp
      exclude: /a\.js|node_modules/,
      // add errors to webpack instead of warnings
      failOnError: true,
      // allow import cycles that include an asyncronous import,
      // e.g. via import(/* webpackMode: "weak" */ './file.js')
      allowAsyncCycles: false,
      // set the current working directory for displaying module paths
      cwd: process.cwd(),
    }),
    new CopyPlugin({
      patterns: Object.keys(slsw.lib.entries).map((el) => {
        const srcFolder = slsw.lib.entries[el].replace('index.js', '');
        return { from: srcFolder, to: srcFolder, force: true };
      }),
    }),
  ],
};

[aberezkin]

@alexander-akait, hi 👋

Stable webpack 5 was there for a while. Can we expect progress on this issue?

[alexander-akait]

Hello, it is on our roadmap, recently we implement support es modules output with small limitations, after fixing limitations we will implement this

[aberezkin]

Nice to hear that, thank you. As you said earlier it will be a great feature :)

[deviant310]

God, I really need this feature!

[roninjin10]

@alexander-akait

This is awesome news! Could you link to the implementation or discussion of this feature?

[alexander-akait]

Right now no, we still need do some improvement, in many places, in near future we will implement ETag and other caches, it requires for fast bundless mode, also it is improve speed loading assets for standard workaround (i.e. bundling)

[aurospire]

Has this been implemented yet?

[falsandtru]

@alexander-akait Can you add this issue to webpack 5 or 6 project board, or set a milestone?

[terozio]

Our use case for this would be nodejs/typescript based lambdas.

I would like to bundle the dependencies in node_modules but keep our own source code as it is. Since the lambdas usually are only some hundreds of lines of code, dependencies usually make 99% of the final package. Reason for keeping everything under '/src/' as it is, is to be able to easily read and modify the code in AWS lambda console. This is especially valuable in dev/test environments.

[spamator12]

Any info on this? This feature is much needed ASAP. Option to bundle/not bundle/bundle some, is crucial.

[david-szabo97]

Is this still on the roadmap?

[licg9999]

Hey, how about using this plugin transpile-webpack-plugin? It collects all the files imported direcly or indirectly by the entry and generates output files seperately without bundling. It works well with resolve aliasing, externals and other webpack config fields. We may setup it as below:

const TranspilePlugin = require('transpile-webpack-plugin');

module.exports = {
  entry: './src/index.js',
  output: {
    path: __dirname + '/dist',
  },
  plugins: [new TranspilePlugin(/* options */)],
};

Assuming the entry src/index.js imports another file src/constants/greeting.js, input files will be src/index.js src/constants/greeting.js. After compilation, output files will be dist/index.js dist/constants/greeting.js. The common dir of input files is used as the base dir to evaluate the relative paths of output files in output dir.

[bptremblay]

I am just learning webpack, so my terminology may be off.
This issue seems to be close to what I'm looking for, but I need to ask some questions.
Before I ask, this is what I want:

• I want webpack to build standard bundles for an ESXXXX (not ts) project.
• When I am in "development watched mode", I do not want to wait many long seconds to bundle; I want to load unbundled ESM modules directly into my browser, via SWC.
• I am willing to put in effort to cache assets that the normal webpack build process produces.

MY QUESTION:
Is it possible to use the webpack dev server for this, or do I need to build my own express server to watch my changes and run SWC?
Thanks,
Ben

[Kreijstal]

@amirmohsen See https://github.com/DrewML/webpack-emit-all-plugin

out of date?