Tarball created on Mac throws errors when extracted on Linux

[sgallagher]

• Version: 10.10.0
• Platform: Linux sgallaghp50.sgallagh.rht 4.17.19-200.fc28.x86_64 deps: update openssl to 1.0.1j #1 SMP Fri Aug 24 15:47:41 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
• Subsystem: Tarball

Download http://nodejs.org/dist/v10.10.0/node-v10.10.0.tar.gz on Linux.
Run tar xvf node-v10.10.0.tar.gz

Results:

tar: Ignoring unknown extended header keyword 'SCHILY.dev'
tar: Ignoring unknown extended header keyword 'SCHILY.ino'
tar: Ignoring unknown extended header keyword 'SCHILY.nlink'
tar: Ignoring unknown extended header keyword 'SCHILY.dev'
tar: Ignoring unknown extended header keyword 'SCHILY.ino'
tar: Ignoring unknown extended header keyword 'SCHILY.nlink'
tar: Ignoring unknown extended header keyword 'SCHILY.dev'
tar: Ignoring unknown extended header keyword 'SCHILY.ino'
tar: Ignoring unknown extended header keyword 'SCHILY.nlink'
tar: Ignoring unknown extended header keyword 'SCHILY.dev'
tar: Ignoring unknown extended header keyword 'SCHILY.ino'
tar: Ignoring unknown extended header keyword 'SCHILY.nlink'
tar: Ignoring unknown extended header keyword 'SCHILY.dev'
tar: Ignoring unknown extended header keyword 'SCHILY.ino'
tar: Ignoring unknown extended header keyword 'SCHILY.nlink'
tar: Ignoring unknown extended header keyword 'SCHILY.dev'
tar: Ignoring unknown extended header keyword 'SCHILY.ino'
tar: Ignoring unknown extended header keyword 'SCHILY.nlink'
tar: Ignoring unknown extended header keyword 'SCHILY.dev'
tar: Ignoring unknown extended header keyword 'SCHILY.ino'
tar: Ignoring unknown extended header keyword 'SCHILY.nlink'
tar: Ignoring unknown extended header keyword 'SCHILY.dev'
tar: Ignoring unknown extended header keyword 'SCHILY.ino'
tar: Ignoring unknown extended header keyword 'SCHILY.nlink'
tar: Ignoring unknown extended header keyword 'SCHILY.dev'
tar: Ignoring unknown extended header keyword 'SCHILY.ino'
tar: Ignoring unknown extended header keyword 'SCHILY.nlink'
tar: Ignoring unknown extended header keyword 'SCHILY.dev'
tar: Ignoring unknown extended header keyword 'SCHILY.ino'
tar: Ignoring unknown extended header keyword 'SCHILY.nlink'
tar: Ignoring unknown extended header keyword 'SCHILY.dev'
tar: Ignoring unknown extended header keyword 'SCHILY.ino'
tar: Ignoring unknown extended header keyword 'SCHILY.nlink'
tar: Ignoring unknown extended header keyword 'SCHILY.dev'
tar: Ignoring unknown extended header keyword 'SCHILY.ino'
tar: Ignoring unknown extended header keyword 'SCHILY.nlink'
tar: Ignoring unknown extended header keyword 'SCHILY.dev'
tar: Ignoring unknown extended header keyword 'SCHILY.ino'
tar: Ignoring unknown extended header keyword 'SCHILY.nlink'
tar: Ignoring unknown extended header keyword 'SCHILY.dev'
tar: Ignoring unknown extended header keyword 'SCHILY.ino'
tar: Ignoring unknown extended header keyword 'SCHILY.nlink'
tar: Ignoring unknown extended header keyword 'SCHILY.dev'
tar: Ignoring unknown extended header keyword 'SCHILY.ino'
tar: Ignoring unknown extended header keyword 'SCHILY.nlink'
tar: Ignoring unknown extended header keyword 'SCHILY.dev'
tar: Ignoring unknown extended header keyword 'SCHILY.ino'
tar: Ignoring unknown extended header keyword 'SCHILY.nlink'
tar: Ignoring unknown extended header keyword 'SCHILY.dev'
tar: Ignoring unknown extended header keyword 'SCHILY.ino'
tar: Ignoring unknown extended header keyword 'SCHILY.nlink'
tar: Ignoring unknown extended header keyword 'SCHILY.dev'
tar: Ignoring unknown extended header keyword 'SCHILY.ino'
tar: Ignoring unknown extended header keyword 'SCHILY.nlink'

It appears that in 10.10.0, Node.js may have started using BSD tar on a mac to create the tarball. It would be much better if it were to use GNU tar, as this will produce a tarball with only the standard headers.

[MylesBorins]

/cc @nodejs/build @nodejs/release

[targos]

I can't reproduce and I have the same version of Fedora as in the OP

[sgallagher]

For the record, I'm using tar-1.30-6.fc29.x86_64

I just realized I'm still reporting a Fedora 28 kernel, but I'm on the Fedora 29 development branch.

[targos]

Sorry. I can indeed reproduce (tar xvf node-v10.10.0.tar.gz | grep Ignoring" to see the same output).

I doesn't really throw an error, though (the exit code is 0). Is there an issue with the extracted files?

[sgallagher]

There's no actual error, but it's noisy. I noticed it because I have a script that goes in and sanitizes the tarball to eliminate all the content that isn't actually redistributable (such as some of the OpenSSL algorithms). My script suppresses stdout but leaves stderr so I can see if anything goes wrong.

It never printed these messages before, so I assume something changed. In general, warnings aren't a great user experience and I'd be concerned about using a non-standard TAR because it might lead to actual incompatibilities in the future.

[targos]

The tarball is created on Mac but that has been the case for previous releases as well.

@nodejs/build Did anything change on the release machine (osx1010-release-tar) between v10.9.0 and v10.10.0?

This is what was executed to create the file:

tar -cf node-v10.10.0.tar node-v10.10.0
gzip -c -f -9 node-v10.10.0.tar > node-v10.10.0.tar.gz

[mscdex]

It seems this happens when you use BSD tar instead of GNU tar.

[MylesBorins]

I think we moved from building on 10.10 to 10.11... potentially with another provider.

…

On Tue, Sep 11, 2018, 11:06 AM mscdex ***@***.***> wrote: It seems this happens when you use BSD tar instead of GNU tar. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#22805 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAecV8wXD3gqEVK3G-tLOAUqSmbppZYMks5uZ9FkgaJpZM4WjPTo> .

[gibfahn]

I don't see an obvious difference between the 10.10 and 10.11 hosts (will update with details).

https://ci-release.nodejs.org/computer/release-requireio-osx1010-x64-1/
https://ci-release.nodejs.org/computer/release-macstadium-macos10.11-x64-1/

# 10.10
Result: bsdtar 2.8.3 - libarchive 2.8.3
# 10.11
Result: bsdtar 2.8.3 - libarchive 2.8.3

The job seems to run make tar-upload.

[sgallagher]

@gibfahn I suspect that on 10.10, gnu tar was also installed and probably owned the tar name. Updating to 10.11 may have replaced this with BSD tar.

[targos]

But Node 10.x releases are still done with 10.10:

https://ci-release.nodejs.org/job/iojs+release/3740/console

Skipping osx1011-release-tar for Node.js 10
Triggering iojs+release » osx1010-release-tar

[refack]

But Node 10.x releases are still done with 10.10:

We had some trouble with release-requireio-osx1010-x64-1 as recorded in:
nodejs/build#1484
nodejs/build#1486
Some there might have been an general software update

[refack]

P.S. might also be coming from the new npm verions (#22591)

refael@refaelux:/mnt/d/code/temp$ tar xzvf node-v10.10.0.tar.gz 2>&1 | grep -C10 Ign
node-v10.10.0/deps/npm/test/npm_cache/index-v5/a6/c9/
node-v10.10.0/deps/npm/test/npm_cache/index-v5/a6/c9/dbe7acc04b7cea7b99c68f3821cf6d7f49caba301e383b31143bd9670f28
node-v10.10.0/deps/npm/test/npm_cache/index-v5/77/52/
node-v10.10.0/deps/npm/test/npm_cache/index-v5/77/52/b7394cb3a27a44448e3ac4d6834e73ca79c301c7108c94b0ba23ca97b71c
node-v10.10.0/deps/npm/test/npm_cache/index-v5/19/db/
node-v10.10.0/deps/npm/test/npm_cache/index-v5/19/db/491ce5fd3b8f5521f62a3d208c8b91325f3f8fe347347649e2e7cb8c10b2
node-v10.10.0/deps/npm/test/npm_cache/content-v2/sha512/
node-v10.10.0/deps/npm/test/npm_cache/content-v2/sha512/76/
node-v10.10.0/deps/npm/test/npm_cache/content-v2/sha512/f3/
node-v10.10.0/deps/npm/test/npm_cache/content-v2/sha512/f3/e8/
tar: Ignoring unknown extended header keyword 'SCHILY.dev'
tar: Ignoring unknown extended header keyword 'SCHILY.ino'
tar: Ignoring unknown extended header keyword 'SCHILY.nlink'
node-v10.10.0/deps/npm/test/npm_cache/content-v2/sha512/f3/e8/2d6a0b75ad5cebd09177d93f572dbd8b877ee9f1505b2e84e03810fa0412e4904060be7d2a4df4221b1bb92884db886826bf8cd26634667a2d103a999438
node-v10.10.0/deps/npm/test/npm_cache/content-v2/sha512/76/39/
tar: Ignoring unknown extended header keyword 'SCHILY.dev'
tar: Ignoring unknown extended header keyword 'SCHILY.ino'
tar: Ignoring unknown extended header keyword 'SCHILY.nlink'
node-v10.10.0/deps/npm/test/npm_cache/content-v2/sha512/76/39/4b378512c68bf209b433e06b71df27a45f7e7be35f174a0f83bce7799628b74dbe993c18b1c12e899a1ed7b159470b382180d1f0a5c4098ac6092cda1a8f
node-v10.10.0/deps/npm/test/npm_cache/_logs/2018-08-29T02_01_27_486Z-debug.log

[gibfahn]

@gibfahn I suspect that on 10.10, gnu tar was also installed and probably owned the tar name. Updating to 10.11 may have replaced this with BSD tar.

@sgallagher type -a tar and type -a gtar produced the same results on both machines, this is the only tar in the path.