Error trying to import ESM Temporal polyfill in a CJS environment

[justingrant]

This polyfill is marked as ESM in package..json via type: 'module', but this is causing an error at runtime in my code which runs in an AWS Lambda function. According to this Stack Overflow answer, AWS Lambda's Node 14 runtime does not support ESM at the moment (or at least didn't as of a few months ago).

It's possible (I haven't tested) that the workaround is what's described in dherault/serverless-offline#1014 (comment) might help:

It's possible to distribute both using exports. Old versions of Node will ignore it and fall back to main while newer versions will load the ES module if the caller supports it.

{
  "type": "module",
  "main": "dist/index.cjs",
  "exports": {
    "import": "./src/index.js",
    "require": "./dist/index.cjs"
  },
  "module": "index.js"
}

Below is the error I'm seeing at runtime when my code is run locally using the serverless-offline package, which is a harness for running AWS Lambda functions locally on my dev machine. This harness doesn't apparently support ESM either, per dherault/serverless-offline#1014.

If any experts in Node's ESM support, Webpack 5, and/or AWS Lambda have any suggestions, I'm all ears!

/usr/local/bin/node --lazy ./../../../../../../usr/local/lib/node_modules/serverless/bin/serverless offline start --noTimeout --dontPrintOutput --httpsProtocol .serverlessoffline

Error: Must use import to load ES Module: /Users/justingrant/Documents/hdev/h3/api/node_modules/@js-temporal/polyfill/dist/index.js
require() of ES modules is not supported.
require() of /Users/justingrant/Documents/hdev/h3/api/node_modules/@js-temporal/polyfill/dist/index.js from /Users/justingrant/Documents/hdev/h3/api/.webpack/service/src/getCurrentUser.js is an ES module file as it is a .js file whose nearest parent package.json contains "type": "module" which defines all .js files in that package scope as ES modules.
Instead rename index.js to end in .cjs, change the requiring code to use import(), or remove "type": "module" from /Users/justingrant/Documents/hdev/h3/api/node_modules/@js-temporal/polyfill/package.json.

    at new NodeError (node:internal/errors:370:5)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1112:13)
    at Module.load (node:internal/modules/cjs/loader:975:32)
    at Function.Module._load (node:internal/modules/cjs/loader:816:12)
    at Module.require (node:internal/modules/cjs/loader:999:19)
    at require (node:internal/modules/cjs/helpers:93:18)
    at Object.@js-temporal/polyfill (/Users/justingrant/Documents/hdev/h3/api/.webpack/service/src/webpack:/h3-api/external "@js-temporal/polyfill":1:1)
    at __webpack_require__ (/Users/justingrant/Documents/hdev/h3/api/.webpack/service/src/webpack:/h3-api/webpack/bootstrap:19:1)
    at Object.../web/src/Serializer.ts (/Users/justingrant/Documents/hdev/h3/api/.webpack/service/src/getCurrentUser.js:1226:79)
    at __webpack_require__ (/Users/justingrant/Documents/hdev/h3/api/.webpack/service/src/webpack:/h3-api/webpack/bootstrap:19:1)
    at Object../src/libs/lambdaHandler.ts (/Users/justingrant/Documents/hdev/h3/api/.webpack/service/src/getCurrentUser.js:55:77)
    at __webpack_require__ (/Users/justingrant/Documents/hdev/h3/api/.webpack/service/src/webpack:/h3-api/webpack/bootstrap:19:1)
    at /Users/justingrant/Documents/hdev/h3/api/.webpack/service/src/getCurrentUser.js:2204:77
    at /Users/justingrant/Documents/hdev/h3/api/.webpack/service/src/getCurrentUser.js:2233:3
    at Object.<anonymous> (/Users/justingrant/Documents/hdev/h3/api/.webpack/service/src/getCurrentUser.js:2238:12)
    at Module._compile (node:internal/modules/cjs/loader:1095:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1124:10)
    at Module.load (node:internal/modules/cjs/loader:975:32)
    at Function.Module._load (node:internal/modules/cjs/loader:816:12)
    at Module.require (node:internal/modules/cjs/loader:999:19)
    at require (node:internal/modules/cjs/helpers:93:18)
    at /Users/justingrant/Documents/hdev/h3/api/node_modules/serverless-offline/dist/lambda/handler-runner/in-process-runner/InProcessRunner.js:157:133
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at InProcessRunner.run (/Users/justingrant/Documents/hdev/h3/api/node_modules/serverless-offline/dist/lambda/handler-runner/in-process-runner/InProcessRunner.js:157:9)

[ryzokuken]

I think you should be able to import the packaged version produced by rollup using CJS though? The source files are ESM indeed but we can investigate a way to use CJS in development...

[ryzokuken]

Actually, I think this should already work, thanks to https://github.com/js-temporal/temporal-polyfill/blob/main/package.json#L6-L8 ?

[justingrant]

I agree that it should work, but it doesn't. If I remove type: 'module', it works great.

I'm not sure if the problem is my webpack config, the serverless-offline package, or some inherent limitation of some environments like AWS Lambda.

[ptomato]

May be related to tc39/proposal-temporal#1499 ?

[12wrigja]

Justin, do you have a minimal repro for this harness/environment I can clone? I'm migrating JSBI to TS and trying to avoid similar issues, so getting to the bottom of this sooner rather than later would be great.

[12wrigja]

I think I found a fix for this - see the referenced PR.

@justingrant can you confirm that your local environment is approximately similiar to this repro I put together? https://github.com/12wrigja/temporal-timeserver-lambda

I'm not sure how webpack comes into play here (though I did see it referenced on the serverless-offline README, I just didn't end up using it.)

[justingrant]

@justingrant can you confirm that your local environment is approximately similiar to this repro I put together? https://github.com/12wrigja/temporal-timeserver-lambda

Nope. Your repro is traditional CJS code that uses require. What I have is this:

• All my lambda handlers are written in TS (and they use import / export
• Those TS files are transpiled by babel as part of bundling by webpack using the serverless-webpack plugin of the serverless package
• Those bundles are then run by serverless-offline (also a serverless plugin) via sls online start (that's the serverless framework CLI)

Below are simplified versions of all the files that should be needed to repro:

Here's a simplified version of my webpack.config.js:

const slsw = require('serverless-webpack');
const nodeExternals = require('webpack-node-externals');
const path = require('path');

module.exports = {
  entry: slsw.lib.entries,
  target: 'node',
  devtool: 'source-map', 
  externals: [nodeExternals()], 
  mode: slsw.lib.webpack.isLocal ? 'development' : 'production',
  optimization: { minimize: false },
  performance: { hints: false } // Turn off size warnings for entry points
  module: {
    rules: [
      {
        test: /\.[jt]s$/,
        loader: 'babel-loader',
        include: [__dirname, path.resolve(__dirname, '../web/src')],  // shared code lives in "../web"
        exclude: [/node_modules/],
        options: {
          // This is a feature of `babel-loader` for webpack (not Babel itself).
          // It enables caching results in ./node_modules/.cache/babel-loader/
          // directory for faster rebuilds.
          cacheDirectory: true,
        },
      },
      {
        enforce: 'pre',
        exclude: /@babel(?:\/|\\{1,2})runtime/,
        test: /\.(js|mjs|jsx|ts|tsx|css)$/,
        use: 'source-map-loader',
      },
    ],
  },
  output:
  {
    libraryTarget: 'commonjs',
    path: path.resolve(__dirname, '.webpack'),  // temp folder to hold build output
    filename: '[name].js',
    devtoolModuleFilenameTemplate: (info) => {  // for sourcemap resolution
      return path.resolve(info.absoluteResourcePath).replace(/\\/g, '/');
    },
  },

My tsconfig.json is this:

{
  "compilerOptions": {
    "baseUrl": ".",
    "paths": {
      "@hshared/*": ["../web/src/*"]
    },
    "rootDirs": ["./", "../web/"],
    "outDir": "build/dist",
    "module": "esnext",
    "target": "es2019",
    "lib": ["es2019", "ES2020.String", "ES2020.Symbol.WellKnown"],
    "watch": true,
    "sourceMap": true,
    "allowJs": true,
    "noEmit": true,
    "esModuleInterop": true,
    "isolatedModules": true,
    "moduleResolution": "node",
    "forceConsistentCasingInFileNames": true,
    "strict": true,
    "noImplicitReturns": true,
    "noUnusedLocals": true,
    "allowSyntheticDefaultImports": true
  },
  "exclude": [
    "node_modules",
    "build",
    "scripts",
    "acceptance-tests",
    "webpack",
    ".awcache",
    ".build",
    ".serverless",
    ".serverlessoffline",
    ".webpack",
    "jest",
    "serverless.ts",
  ]
}

And here's a simplified version of my package.json. Note there's no type, exports, etc because it's never actually being bundled. Instead, each function is bundled separately by serverless-webpack. but can be helpful for knowing the scripts, and dependency versions.

{
  "name": "h3-api",
  "version": "1.1.0",
  "description": "_REDACTED_",
  "main": "handler.js",
  "scripts": {
    "start": "npm run lint && serverless offline start --noTimeout --httpsProtocol .serverlessoffline",
    "debug": "export SLS_DEBUG=* && node --inspect /usr/local/lib/node_modules/serverless/lib/Serverless.js offline start",
    "webpack": "webpack",
    "tsc": "tsc",
  },
  "author": "",
  "license": "MIT",
  "repository": {
    "type": "git",
    "url": "https://github.com/_REDACTED_.git"
  },
  "devDependencies": {
    "@babel/core": "^7.15.0",
    "@babel/plugin-proposal-class-properties": "^7.14.5",
    "@babel/plugin-proposal-object-rest-spread": "^7.14.7",
    "@babel/plugin-transform-runtime": "^7.15.0",
    "@babel/preset-env": "^7.15.0",
    "@babel/preset-stage-3": "^7.8.3",
    "@babel/preset-typescript": "^7.15.0",
    "@types/aws-lambda": "^8.10.82",
    "@types/mongodb": "^4.0.6",
    "@types/node": "^16.6.1",
    "@typescript-eslint/eslint-plugin": "^4.29.2",
    "@typescript-eslint/parser": "^4.29.2",
    "aws-sdk": "^2.971.0",
    "babel-loader": "^8.2.2",
    "babel-plugin-module-resolver": "^4.1.0",
    "babel-plugin-source-map-support": "^2.1.3",
    "prettier": "^2.3.2",
    "serverless": "^2.55.0",
    "serverless-dynamodb-local": "^0.2.40",
    "serverless-offline": "^8.0.0",
    "serverless-webpack": "^5.5.1",
    "source-map-loader": "^3.0.0",
    "typescript": "^4.3.5",
    "webpack": "^5.50.0",
    "webpack-cli": "^4.8.0",
    "webpack-node-externals": "^3.0.0"
  },
  "dependencies": {
    "@babel/runtime": "^7.15.3",
    // a fork of the polyfill that removes type=module
    "@js-temporal/polyfill": "file:../../temporal-polyfill/js-temporal-polyfill-0.2.0.tgz",
    "@types/serverless": "^1.78.34",
    "node-fetch": "^2.6.1",
    "source-map-support": "^0.5.19",
  },
}

Here's my babel.config.json.

{
  "presets": ["@babel/typescript", ["@babel/preset-env", { "targets": { "node": "12" } }]],
  "plugins": [
    [
      "module-resolver",
      {
        "cwd": "packagejson",
        "root": ["./"],
        "alias": { "@hshared": "./../web/src" }
      }
    ],
    "source-map-support",
    "@babel/plugin-transform-runtime",
    "@babel/proposal-class-properties",
    "@babel/proposal-object-rest-spread"
  ]
}

Finally, here's a simplified version of my serverless.yml config:

service: h3-api

# Use the serverless-webpack plugin to transpile ES6
plugins:
  - serverless-webpack
  - serverless-offline

# serverless-webpack configuration
# Enable auto-packing of external modules
custom:
  webpack:
    webpackConfig: ./webpack.config.js
    includeModules: true
  stage: '${opt:stage, self:provider.stage}'
  serverless-offline:
    httpPort: 4000
    corsAllowOrigin: '*'
    corsAllowHeaders: accept,content-type,x-api-key,cognito-identity-id
    stage: prod

provider:
  name: aws
  runtime: nodejs14.x
  versionFunctions: false,
  stage: prod
  region: us-east-1
  environment: ${file(env.yml):${self:provider.stage}}

functions:
  # Defines an HTTP API endpoint that calls the main function in getUser.ts
  # - path: url path is /user
  getUser:
    handler: src/getUser.main
    events:
      - http:
          path: user
          method: get
          cors:
            headers:
              - Content-Type
              - X-Amz-Date
              - Authorization
              - X-Api-Key
              - X-Amz-Security-Token
              - X-Amz-User-Agent
              - cognito-identity-id
          authorizer: aws_iam

If you change your handler file to getUser.ts, you may be able to use the files above to repro the issue. If not, just let me know and I can fork my project and just keep removing stuff until I end up with a minimal repro.

[12wrigja]

I think based on comments left on #38 this should now be fixed? Please re-open if not.