Skip to content

support Electron: respect process.env.npm_config_arch & process.env.npm_config_platform #35

@mifi

Description

@mifi

Because in v4 it only downloads the binary for the current operating system arch, how can I now build an electron app that includes the correct ffmpeg binary according to which target platform I am building for?

For example if I want to run electron-builder --mac --win --linux

Before v4, I would just include the correct binary from the right arch directory when building like this:

electron-builder package.json "build":

    "mac": {
...
      "extraResources": [
        "node_modules/ffmpeg-static/bin/darwin/**",
        "node_modules/ffprobe-static/bin/darwin/**"
      ],
    },
    "win": {
...
      "extraResources": [
        "node_modules/ffmpeg-static/bin/win32/x64/**",
        "node_modules/ffprobe-static/bin/win32/x64/**"
      ],
    },

See
https://github.com/mifi/lossless-cut/blob/74439d716db5ff550a1851fa5b58e30ba188c477/package.json#L108

But now this doesn't work. Any tips of how to cleanly solve this? Or isn't electron really supported by this module anymore?

Activity

derhuerst

derhuerst commented on Feb 11, 2020

@derhuerst
Collaborator

Bundling the binaries with the npm package gradually became too clunky, as outlined in #20 and #33. This breaks Electron for now, sorry. (Keep in mind that the previous hacks were helpful but not very future-proof.)


First, let me put the problem in the perspective of ffmpeg-static:

The new download-based approach is a problem whenever you want to package ffmpeg-static into a platform-independent or per-platform "distributable"; Electron apps are likely the most common case, but .zip archives are another.

I don't see this a problem of `ffmpeg-static though. Developers need to figure this out in their build scripts anyways whenever they use

  • a native dependency (which needs to be compiled for the/each platform/arch), or
  • a dependency with a platform-/arch-specific install script (quite a few npm packages), or
  • other platform-/arch-specific assets, e.g. images.

Therefore, developers should configure their build tooling to set up the right environment for these dependencies to build (themselves) correctly.


I don't have any experience with building Electron apps. It seems like it has a mechanism for this:

Digging into the electron-builder (from electron-builder's install-app-deps command via installOrRebuild via installDependencies to getGypEnv), I stumbled upon the script for installing Electron via npm and the Using Native Node Modules docs.

They which mention the npm_config_platform, npm_config_arch & npm_config_target_arch env vars. It seems like we should respect them here in ffmpeg-static as well.

@mifi Would you mind to add the env vars to install.js in your fork of ffmpeg-static and try it with your electron-builder setup?

pietrop

pietrop commented on Feb 11, 2020

@pietrop

👋 this comment pietrop#1 (comment) and blog post explains how I dealt with making static ffmpeg work with electron build process. Sharing it in case it helps.

derhuerst

derhuerst commented on Feb 12, 2020

@derhuerst
Collaborator

👋 this comment pietrop#1 (comment) and blog post explains how I dealt with making static ffmpeg work with electron build process. Sharing it in case it helps.

Maybe I misunderstood it, but It's not as zero-config as I'd like it to be:

  • you have to manually add a disallow-list to your electron config
  • you have to know about every large file in the ffmpeg-static package.
pietrop

pietrop commented on Feb 12, 2020

@pietrop

Yeah, fair, at the moment is not zero-config but it works.

The main problem is if you want to build for mac, linux, and windows on OSX (I do that on travis CI) then you'd still need all 3 binaries to be able to do the build, but you'd just want to package only the one you need for each distribution to keep the package size contained etc..

Hope this use case helps.

I do agree that having binaries inside a repo is not ideal, and perhaps there could be a way to pull those down eg from github releases section and add them to the module this way as part of a pre installation step. But not sure if that's a possibility?

mifi

mifi commented on Feb 12, 2020

@mifi
Author

It seems that install.js isn't being run when I run a complete build with electron-builder (even for a different OS than my current). I think electron-builder will run node-gyp for all the native dependencies, maybe ffmpeg-static needs to hook into that somehow and do the downloading of the correct binary instead of building

added a commit that references this issue on Feb 16, 2020
derhuerst

derhuerst commented on Feb 21, 2020

@derhuerst
Collaborator

I do agree that having binaries inside a repo is not ideal, and perhaps there could be a way to pull those down eg from github releases section and add them to the module this way as part of a pre installation step. But not sure if that's a possibility?

That is the case right now: The install script defined in package.json downloads the binary for the current platform, according to os.platform() & os.arch().

It seems that install.js isn't being run when I run a complete build with electron-builder (even for a different OS than my current).

Again, I don't know Electron well, but I think running the install script is the job of any tool that installs (and builds) dependencies.

But downloading the intended binaries for a different platform than the current one seems to be the crucial point here.

You could help me by

  1. setting up a clean electron-builder-based project,
  2. adding any npm dependency (e.g. lodash),
  3. add an install script to it inside node_modules, that
  4. prints npm_config_platform,npm_config_arch & npm_config_target_arch env vars, and
  5. run electron-builder to find out if it sets these env vars.
mifi

mifi commented on Feb 22, 2020

@mifi
Author

That’s Essentially what i already did. I tried to modify install.js in ffmpeg-static in node_modules and add console.log, then run electron-builder, but it was never being executed. The install script only gets executed when running npm install / yarn. Electron-builder never runs this afaik. I think wwhat electron builder does is to take modules from node_modules as is and pacakge them up. But first it will run node-gyp to rebuild any binary dependencies inside node_modules to the target arch. If ffmpeg-static was using node-gyp to compile ffmpeg then i think it would just work. So i think if there is some way we can hook into node gyp and make it run the install.js script, that might be a way to do it. But i know next to nothing about node gyp.

derhuerst

derhuerst commented on Feb 22, 2020

@derhuerst
Collaborator

The install script only gets executed when running npm install / yarn. Electron-builder never runs this afaik. [...] If ffmpeg-static was using node-gyp to compile ffmpeg then i think it would just work.

Sorry, I'm not going to do that:

  • The preinstall/install/postinstall scripts are the right place for npm packages to declare that they need to do work to "become functional". If I'm not missing something, electron-builder should add support for getting the right "variant" of node_modules.
  • Abusing mechanisms intended for different things adds a long-term maintenance burden. This mechanism might change in the future.
derhuerst

derhuerst commented on Feb 23, 2020

@derhuerst
Collaborator

Because in v4 it only downloads the binary for the current operating system arch, how can I now build an electron app that includes the correct ffmpeg binary according to which target platform I am building for?

Keep in mind that the electron-builder docs explicitly mention:

Don’t expect that you can build app for all platforms on one platform.
If your app has native dependency, it can be compiled only on the target platform unless prebuild is not used.

prebuild is a solution, but most node modules don’t provide prebuilt binaries.

electron-builder mentions prebuild & prebuild-install as working though. prebuild-install latter respects process.env.npm_config_arch & process.env.npm_config_platform.

PR welcome!

changed the title [-]How to use with electron now?[/-] [+]support Electron: respect process.env.npm_config_arch & process.env.npm_config_platform[/+] on Feb 23, 2020

11 remaining items

derhuerst

derhuerst commented on May 19, 2020

@derhuerst
Collaborator

ffmpeg-static@4.2.2 includes #47, which should allow you to cross-platform-package Electron apps with the mentioned rm -rf node_modules workaround. I have created #48 to make this experience better.

I will close this ticket. Please re-open if there's a bug with @RavisMsk's logic that I have just published.

pietrop

pietrop commented on May 19, 2020

@pietrop

In my current setup I use electron builder on TravisCI to build for Mac, Linux and Windows on a TravisCI OSX image. It then publishes to GitHub release.

The recent change would not work in this “edge” case, right? Coz it would just use the OS of the platform / npm etc..? Or am I missing something?

As mentioned in previous comment I already have a workaround to package the right ffmpeg bin a part of the electron build process
#35 (comment)

But wanted to check if this latest change, changes anything in this context. If it doesn’t no worries.

derhuerst

derhuerst commented on May 19, 2020

@derhuerst
Collaborator

@pietrop Cross-platform-packaging Electron apps should just work with ffmpeg-static@4.2.2:

And modified packaging commands to remove node_modules of app that's using ffmpeg-static:

"package-mac": "rm -rf ./app/node_modules && yarn build && electron-builder build --mac",
"package-linux": "rm -rf ./app/node_modules && yarn build && electron-builder build --linux",
"package-win": "rm -rf ./app/node_modules && yarn build && electron-builder build --win --x64",

In the macOS Travis build, do rm -rf node_modules each time before you call electron-builder.

pietrop

pietrop commented on May 19, 2020

@pietrop

Awesome, thanks will try it out, at the moment I do all 3 in one command but I guess I could split it

delewis13

delewis13 commented on Sep 23, 2021

@delewis13

@RavisMsk i'm a little confused how your commands listed are working O.o Given you delete all your node_modules, how does yarn build [which in turn looking @ https://github.com/electron-react-boilerplate/electron-react-boilerplate/blob/main/package.json runs webpack], actually work, given you just deleted webpack and all ur other deps?

I'm trying to get this to work on my end, but if I rm -rf ./app/node_modules && electron-builder build --win --x64, the build fails as electron-builder node_module was deleted... So I need to yarn install everything which gets me back to square one, given that the necessary environment variables aren't defined at that point

mnvr

mnvr commented on Aug 28, 2024

@mnvr

Maybe this will help someone. This is how we created a macOS universal ffmpeg binary (download for both archs, and then combine with lipo):

npm rebuild --arch=x64 -f ffmpeg-static && mv node_modules/ffmpeg-static/ffmpeg{,-x64}
npm rebuild --arch=arm64 -f ffmpeg-static && mv node_modules/ffmpeg-static/ffmpeg{,-arm64}
cd node_modules/ffmpeg-static
lipo -create ffmpeg-arm64 ffmpeg-x64 -output ffmpeg

Since ffmpeg-static does not overwrite the existing ffmpeg binary, we can just run this as a pre-build step in our GitHub action

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @mifi@RavisMsk@pietrop@derhuerst@delewis13

        Issue actions

          support Electron: respect process.env.npm_config_arch & process.env.npm_config_platform · Issue #35 · eugeneware/ffmpeg-static