Fix "everything passes, travis fails"

[mramato]

1. buildSandcastle is returning a stream that is passed to Promise.join. That means the promise resolves before the stream is finished. Instead it should be returning a promise created from the stream.

2. npm pack may be screwed with travis' console leading to random failures, redirect it's output to /dev/null to see if it addresses our random build failures

3. requirejs was occasionally being killed by travis for using too much memory, reduce concurrency during the build process to prevent this from happening.

[cesium-concierge]

Thanks for the pull request @mramato!

• ✔️ Signed CLA found.
• ❔ CHANGES.md was not updated.
  • If this change updates the public API in any way, please add a bullet point to CHANGES.md.

Reviewers, don't forget to make sure that:

• Cesium Viewer works.
• Works in 2D/CV.
• Works (or fails gracefully) in IE11.

---

I am a bot who helps you make Cesium awesome! Contributions to my configuration are welcome.

🌍 🌎 🌏

[mramato]

Well, that didn't work. Back to the drawing board (unless I'm missing something, this is still a good change)

[mramato]

Hold off on merging this, I have a few additional things I want to try.

[mramato]

Lots of weirdness going on. After some experimentation, I think travis/linux may be killing the requirejs process for using too much memory (or possibly trying to use too many processors). I changes makeZipFile to be less concurrent and more waterfall and we'll see if that resolves the problem. It's unfortunate if this is the case, but whatever makes CI stable is a net win.

[mramato]

Partial success! I also now think that npm pack console output is somehow interfering with travis, so I am attempting to redirect it to /dev/null to see if that fixes it.

[mramato]

Awwww yeah!

[mramato]

Okay, I got everything to pass (once). I just reset and force-pushed my changes to see if everything passes again. Assuming it does, this should be ready for review.

[mramato]

Everything passed twice in a row! @ggetz we should probably review/merge this sooner rather than later since it affects all branches.

[mramato]

Fixes #7076
Fixes #6230

[mramato]

After some back and forth with travis, they think that "npm pack" output is definitely screwing with their log handling/filtering code. They made a change to our org that seems to have fixed the problem in a different way. (We definitely still need this PR because there were real issues it fixes as well)

I asked for more clarification about what they changed. I also mentioned that we had 1500 forks an org-only change probably wouldn't be good enough. Just redirecting npm pack to /dev/null seems to have the same affect.

So ultimately this PR is ready and should be reviewed/merged ASAP.

[mramato]

Just in case we ever run into similar issues again, here's the full details from travis support. npm pack was definitely the main culprit.

So what I did for your organization was to switch the filtering mechanism used in the logs. We started using a new one time ago (more details below) and we've been seeing some corner cases and issues since then. In case you're curious, there's been some information posted publicly on the topic at travis-ci/travis-ci#4704

The two filtering mechanisms that we have in place are the redirect_io mechanism (https://github.com/travis-ci/travis-build/blob/master/public/filter/redirect_io.rb) and the pty one - https://github.com/travis-ci/travis-build/blob/master/public/filter/pty.rb. The default mechanism in Travis currently (since ~a year ago) is the redirect_io strategy and we have been putting some users back on the pty strategy, which was the default issue, as a workaround.

One of the issues we found is indeed directly related to Node (see: nodejs/node#14752). It seems one of the commands that Node runs is changing the file descriptor settings and causing this problem. There was a fix for this issue implemented a couple months ago, but I just noticed it indeed got reverted (nodejs/node#21257) and the issue remains open.

Taking into account this, and since you mention that there might be over a thousand forks of this project affected, I agree that leaving the workaround on npm pack might be the best solution here.

[hpinkos]

Works great, thanks for looking into this @mramato! This has been a huge pain for a while.

[mramato]

Thanks, I still want @ggetz to review the gulp changes, since it does affect the build process a bit.

[hpinkos]

Sounds good. Well I can confirm that each of the tasks still run fine locally.

@ggetz merge when ready

[ggetz]

Looks good! Passing consistently and no decrease in build time from what I'm seeing.