copy directories/files per function

[jamesdixon]

This is a (Bug Report / Feature Proposal)

Question

Description

Hi, is there a way to copy additional directories on a per-function basis? For example, I have a createEmail function located at lib/createEmail that also has a templates directory. I need that copied with the function, but it doesn't copy. However, I also don't want that directory copied to other functions that live in the same project.

Thank you!

For bug reports:

• What went wrong?
• What did you expect should have happened?
• What was the config you used?
• What stacktrace or error message from your provider did you see?

For feature proposals:

• What is the use case that should be solved. The more detail you describe this in the easier it is to understand for us.
• If there is additional config how would it look

Similar or dependent issue(s):

• #12345

Additional Data

• Serverless-Webpack Version you're using:
  5.2.0
• Webpack version you're using:
  4.16
• Serverless Framework Version you're using:
  1.28
• Operating System:
  OSX High Sierra
• Stack Trace (if available):

[jamesdixon]

@HyperBrain I just tried copy-webpack-plugin in accordance with the advice of #361, but I'm not sure to get specific files to be included in the package of specific functions rather than being copied to every function. Any suggestions?

[HyperBrain]

@jamesdixon This is currently not supported. With the CopyWebpack plugin you'll get the modules copy for each function.
@serverless-heaven/serverless-webpack-contributors Any idea, if this can be somehow worked around with existing webpack plugins?

[ceilfors]

I have been using my own written workaround to handle such scenario. See ConditionalPlugin below. Any suggestion on how we can make this as serverless-webpack feature?

const ConditionalPlugin = (condition, plugin) => ({
  apply: compiler => {
    compiler.plugin('emit', (compilation, callback) => {
      if (condition(compiler)) {
        plugin.apply(compiler);
      }
      callback();
    });
  }
});

Inside webpack.config.js used by serverless-webpack:

...
  plugins: [
    ConditionalPlugin(
      compiler => compiler.outputPath.includes('moduleName'), // not function name
      new CopyWebpackPlugin([
        {
          from: 'source directory',
          to: 'lib'
        }
      ])
    )
  ]
...

[jamesdixon]

@ceilfors thanks for this! much appreciated.

I'm running into the following:

/Users/jamesdixon/Projects/scout/platform/functions/email-service/node_modules/copy-webpack-plugin/dist/index.js:163
                for (var _iterator = fileDependencies[Symbol.iterator](), _step; !(_iteratorNormalCompletion = (_step = _iterator.next()).done); _iteratorNormalCompletion = true) {
                                                     ^
TypeError: Cannot read property 'Symbol(Symbol.iterator)' of undefined
    at afterEmit (/Users/jamesdixon/Projects/scout/platform/functions/email-service/node_modules/copy-webpack-plugin/dist/index.js:163:54)
    at AsyncSeriesHook.eval [as callAsync] (eval at create (/Users/jamesdixon/Projects/scout/platform/functions/email-service/node_modules/tapable/lib/HookCodeFactory.js:24:12), <anonymous>:7:1)
    at AsyncSeriesHook.lazyCompileHook [as _callAsync] (/Users/jamesdixon/Projects/scout/platform/functions/email-service/node_modules/tapable/lib/Hook.js:35:21)
    at asyncLib.forEach.err (/Users/jamesdixon/Projects/scout/platform/functions/email-service/node_modules/webpack/lib/Compiler.js:355:27)
    at done (/Users/jamesdixon/Projects/scout/platform/functions/email-service/node_modules/neo-async/async.js:2854:11)
    at /Users/jamesdixon/Projects/scout/platform/functions/email-service/node_modules/neo-async/async.js:2805:7
    at /Users/jamesdixon/Projects/scout/platform/functions/email-service/node_modules/graceful-fs/graceful-fs.js:43:10
    at /Users/jamesdixon/.nvm/versions/node/v8.10.0/lib/node_modules/serverless/node_modules/graceful-fs/graceful-fs.js:43:10
    at FSReqWrap.oncomplete (fs.js:135:15)

Have you had a similar issue?

[ceilfors]

@jamesdixon Unfortunately not. Might be able to help you if you can produce an MVCE.

[jamesdixon]

@ceilfors here's my webpack.config.js for reference:

const nodeExternals = require('webpack-node-externals')
const slsw = require('serverless-webpack')
const CopyWebpackPlugin = require('copy-webpack-plugin')

const ConditionalPlugin = (condition, plugin) => ({
  apply: compiler => {
    compiler.plugin('emit', (compilation, callback) => {
      if (condition(compiler)) {
        plugin.apply(compiler)
      }
      callback()
    })
  }
})

module.exports = {
  entry: slsw.lib.entries,
  mode: slsw.lib.webpack.isLocal ? 'development' : 'production',
  target: 'node',
  devtool: 'source-map',
  optimization: {
    minimize: false
  },
  externals: [nodeExternals()],
  plugins: [
    ConditionalPlugin(
      compiler => compiler.outputPath.includes('createEmail'),
      new CopyWebpackPlugin([
        {
          context: `./lib/createEmail`,
          from: '**/*',
          force: true
        }
      ], {
        copyUnmodified: true
      })
    )
  ]
}

[HyperBrain]

@ceilfors Cool stuff. Maybe the conditional plugin can be integrated into the sls-webpack project, so that you do not need to install it but just use slsw.plugins.ConditionalPlugin in the webpack config?

[hiddestokvis]

@jamesdixon I had the same error, got it working however by using the following code:

const ConditionalPlugin = (condition, plugin) => ({
  apply: compiler => {
    if (condition(compiler)) {
      plugin.apply(compiler)
    }
  }
})

also as a condition I am searching the path compiler.options.output.path instead of compiler.outputPath hope this helps 😄

[jamesdixon]

@hiddestokvis thank you! this fixed my issue 👍

Question: do you have local modules as well? I'm now noticing that paths are off because of the way functions are packaged individually. For example, my functions live under lib/[functionName], so when they are packaged, I end up with that directory structure in the zip file. However, I noticed that no local modules (ex: helpers/) are packaged.

[craigtsmith]

I took the ideas in this thread and extended them so I can set conditionals / config back at the service config level.

Given a serverless.yml with a function like this:

# serverless.yml

...
  handleWebhook:
    handler: handlers/handleWebhook.default
    name: ${self:provider.stage}-${self:service}-handleWebhook
    description: Handle the Webhook
    webpack:
      toggle: true
...

And the conditional plugin tweaked to do this:

const ConditionalPlugin = (condition, plugin) => ({
  apply: compiler => {
    let name = Object.keys(compiler.options.entry)[0].split('/').pop();
    let config = Object.assign({webpack: {}}, slsw.lib.serverless.service.getFunction(name));
    
    if (condition(config)) {
      plugin.apply(compiler)
    }
  }
});

You can configure your functions to have conditional plugins execute based on the configuration in the overall service, like this:

# webpack.config.js

module.exports = {
  entry: slsw.lib.entries,
  target: 'node',
  mode: 'production',
  plugins: [
    ConditionalPlugin(
      ((config) => config.webpack.toggle),
      new CopyWebpackPlugin([{from: 'some/path', to: 'some/other/path'}])
    )
  ],

The only thing specific to my setup is a naming convention, where the function name to serverless is the same as the file name with the lambda handler inside a folder called "handlers".

It seems to be working fine, but YMMV

[buithaibinh]

My solution.

 # serverless.yml
transform:
    handler: src/handler.transform

# webpack.config.js
const ConditionalPlugin = (condition, plugin) => ({
  apply: compiler => {
    if (condition(compiler)) {
      plugin.apply(compiler)
    }
  }
});
...
plugins: [
      ConditionalPlugin(
        compiler => {
          // copy folder templates into webpack
          return compiler.options.output.path.includes("transform")
        },
        new CopyWebpackPlugin([{from:'./templates', to:'templates'} ], {logLevel: "error"}),
      ),
],

[Omicron7]

I wanted to use the packaging config that AWS serverless provides. https://serverless.com/framework/docs/providers/aws/guide/packaging/
Thanks to everyone above, I came up with this.

Individual functions can have

# serverless.yml
functions:
  name:
    handler: path/to/file.handler
    package:
      include:
        - lib/**

I implemented this functionality with

// webpack.config.js
const { find, get } = require('lodash')

module.exports = {
  ...
  plugins: [{
      apply: compiler => {
        const handler = `${Object.keys(compiler.options.entry)[0]}.handler`
        const config = find(slsw.lib.serverless.service.functions, val => val.handler === handler)
        const includePaths = get(config, 'package.include', [])
        if (includePaths.length) {
          new CopyWebpackPlugin(includePaths).apply(compiler)
        }
      }
  }]
}

This takes the array provided to the package.include and sends it to the CopyWebpackPlugin. If anyone knows a better way to find the serverless function config based on the current entrypoint, I'd be grateful. The code above relies on the fact that all of my entrypoints have only a single exported handler name handler.

[cpconcertage]

As someone who treads carefully with webpack, I was thrilled that the solution suggested by @Omicron7 works brilliantly. For some reason (version differences, perhaps), I needed to send the plugin an object with patterns:

new CopyWebpackPlugin({ patterns: includePaths }).apply(compiler);

I don't find it any trouble to call my handler handler, so this solution is a quick and simple fix. Thanks!

[raymond-w-ko]

Minor hack/modification to fix cases where you are specifying a certain file and need to keep the directories.
With @Omicron7 's code, if you have:

  function:
    handler: function.handler
    package:
      include:
        - bin/exe

exe gets copied to the root, and it is not inside a ./bin directory in the deployed zip.
Below code keeps the ./bin/.

const convertPattern = (s) => {
  if (s.endsWith("*")) return s
  if (!s.includes("/")) return s

  const i = s.lastIndexOf("/")
  return {from: s, to: s.substring(0, i) + "/"}
}

module.exports = {
  plugins: [
    {
      apply: (compiler) => {
        const handler = `${Object.keys(compiler.options.entry)[0]}.handler`
        const config = find(
          slwp.lib.serverless.service.functions,
          (val) => val.handler === handler
        )
        let includePaths = get(config, "package.include", [])
        includePaths = _.map(includePaths, convertPattern)
        if (includePaths.length) {
          new CopyWebpackPlugin({patterns: includePaths}).apply(compiler)
        }
      },
    },
  ],
}