Description
- I'd be willing to implement this feature (contributing guide)This feature is important to have in this repository; a contrib plugin wouldn't do
Describe the user story
We all want fast yarn install, and node_modules which don't eat our disk.
Pnp is pretty good solution, but it has compatibility issues: bundlers, resolvers, etc. Migration effort if pretty high.
Describe the solution you'd like
Fuse linker.
Idea consist from three elements:
-
pnpm layout.
I forked pnpm linker, removed hardlinks logic from it.
I've split packages into two dirs, .store-fuse and .store-unplugged.
Unplugged has packages which need postinstalls, they are unpacked to fs.
Fuse has other packages, and mounted as readonly.
If fuse linker is not supported in OS, all packages fallback to unplugged. -
fuse mount of .store-fuse.
Supported os: linux (libfuse), macos 15.4 (fskit). Probably windows with wsl 2.
For unprivileged mounts linux needs fusermount util installed.
Macos need app installed.
I've implemented custom fuse fs, which ended to be a bit complicated. After finished I've found out that I can generateerofs
binary image instead, and reuse erofs-fuse or even erofs native mount on linux (it's supported by kernel, but needs root). -
fetching archives on demand from remote cache (optional)
Fuse fs supports async fs actions, which means that we can add fetching data when its requested.
This all will make yarn install time = resolution + cache fulfil + postinstalls.
Sources: https://github.com/goloveychuk/fuse
Describe the drawbacks of your solution
Needs binaries downloaded, need support by OS, could have bugs.
Describe alternatives you've considered
pnp
@arcanis wdyt?
Activity
arcanis commentedon Jun 26, 2025
I deemed it not viable many years ago due to:
Not sure how much of that changed since last time. That's why I went with the Node.js-based FS layer: harder to maintain, but at least we were in control of how it executed on users' machines (usually).
goloveychuk commentedon Jun 26, 2025
licensing is now fixed, in macos we can now implement fskit extension without relying on macfuse. Thats what I've done in my repo. Libfuse (linked) has LGPL.
this remains. It could be fixed a bit by building go binaries (for linux + wsl2), which are completely statically linked.
But still, linux should have root or fusermount installed, and macos should have extensions installed.
Still, it could fallback to unpack if platform is not supported.
I'm not saying it should replace pnp. But it could be an alternative which has some pros and const.
Also, it could be external berry plugin, but if it would be part of berry builtin plugins, it could simplify an adoption.
Btw, it would help if berry has a plugin interface for Cache. Currently it's hardcoded to zip all over the place.
I can implement this if it fits into design.
goloveychuk commentedon Jun 26, 2025
Btw, about whole pnp + nodejs runtime. I saw your pr to nodejs which adds native zip support, and it's not merged (and probably won't ever).
I was thinking to target this by doing two steps.
packagemaps
I think we can write a RFC for nodejs (mb with @guybedford help) which will define a manifest file which will tell nodejs where to find a package from other package. Very similar to importmaps, but from what I understand importmap won't work with nodejs resolution logic we have now (e.g. package exports).
Idea is to have smth like
This will fix many problems with node_modules, no need for pnpm, no hoisting needed, can reuse global cache.
Implementation in nodejs resolution algo should be pretty easy, just change the "node_modules/package" lookups with json lookup. Other resolution logic will remain the same (exports field).
2) second step is to introduce other protocol, maybe zip, maybe some other, optimized for performance (erofs, squashfs, custom format)
This will require implementing vfs subsystem in nodejs itself, they probably won't accept it, but we can try.