Build complexity (added rsync)

[rvagg]

#1654 added the user survey microsite and the build failed on the server. The reason this happened is that we run the build inside a Docker container for a little layer of security but the build added more complexity than the container is prepared for.

https://github.com/nodejs/build/blob/master/setup/www/resources/scripts/build-site.sh

The script just does a docker run node:latest ... and does some very minor stuff, adding a uid and gid and setting some basic npm bits before running npm install.

#1654 added rsync into an npm script and this isn't available in node:latest. For now I've added apt-get update && apt-get install -y rsync to the beginning of the bash script that's executed inside the container. This adds quite a bit of latency to the build, each commit to this repo will be running that and it's pretty slow.

One solution is to create a persistent docker container that has this in it, but that adds maintenance burden that is likely to be out of mind and will get out of date really quickly. Another solution is to remove rsync and rely on something more simple. Perhaps a cp -a is enough? Or, use https://www.npmjs.com/package/cpr as a build dependency and use that instead?

What do you think @fhemberger? I'd like to remove that additional piece of scripting as soon as possible to speed up deploys, unless you think the additional time cost is worth it.

[gtewallace]

@rvagg Thank you so much for troubleshooting this!!!

There's a few things with the survey that don't seem to be working at the moment.

1. Several of the charts have tabs under them, and many of these are not loading the different views on the data
   the Correlated Technologies (Beta) section isn't loading - that is driven from Graph.Html

Thanks again to @fhemberger and everyone who has helped with this.

[rvagg]

Failed to load prism syntax: console
{ Error: Cannot find module 'prismjs/components/prism-console.js'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:594:15)
    at Function.Module._load (internal/modules/cjs/loader.js:520:25)
    at Module.require (internal/modules/cjs/loader.js:650:17)
    at require (internal/modules/cjs/helpers.js:20:18)
    at requireLanguage (/website/node_modules/metalsmith-prism/lib/index.js:42:11)
    at Object.<anonymous> (/website/node_modules/metalsmith-prism/lib/index.js:86:11)
    at initialize.exports.each (/website/node_modules/cheerio/lib/api/traversing.js:300:24)
    at _.each (/website/node_modules/metalsmith-prism/lib/index.js:62:17)
    at /website/node_modules/metalsmith-prism/node_modules/lodash/lodash.js:4925:15
    at baseForOwn (/website/node_modules/metalsmith-prism/node_modules/lodash/lodash.js:3010:24) code: 'MODULE_NOT_FOUND' }

related to that perhaps? this is in the build log. if this is a native addon then building that will be a problem. or perhaps this is a linux vs macos thing?

[gtewallace]

Don't know - @fhemberger any ideas?

[rvagg]

open dev tools console, there's some d3 javascript errors in there

[gtewallace]

gah - bummer. OK - I will ask the dev who wrote that to have a look

[fhemberger]

Already on it!

[fhemberger]

@rvagg Sorry, didn't know the build was running in a container. D'Oh!
I'm fine with a different solution than rsync, was the first one which came to my mind which allows ignoring files easily.

[rvagg]

We've thought through the URL "user-survey-report" already and how it won't scale to next year without a year number in it? Is that a problem to solve later if/when we publish another one of these?

[gtewallace]

OH thank you!!!!!

[gtewallace]

If it won't be too much work later, my preference is to leave it as is and solve next year. we have a bunch of different documents and announcements using this url already. But we can update them if others feel strongly we should do it now

[fhemberger]

Ok, JS issues are fixed in a558c67, my bad.

@rvagg Regarding the years, I think I'd go with this proposed solution: #1654 (comment)

[gtewallace]

@fhemberger great - thanks!! Do you think it will take a few minutes for the fixed version to load? I'm still not seeing the tabs or the correlated technologies graph.

[gtewallace]

@fhemberger never mind

it's all there - you literally are my hero - both of you

[fhemberger]

Okay, everything seems to be running smoothly now.
I'll be AFK for a while, but if anything comes up just ping me and I'll have a look later.

[rvagg]

heh, yeah, that's what I thought after I posed my question since I was involved in sorting that one out too!

site's looking good, well done folks

[gtewallace]

so, this is my bad but is it possible to add this hashtag to the tweet text? #node2018survey

[gtewallace]

@fhemberger in chatting with @mhdawson we wondered if it might be possible to add something to the survey report, maybe to the table of contents on the left, that says "Click here to get the raw data" and "Click here to get the PDF slides"

what do you think?

[gtewallace]

@fhemberger I also am having trouble getting to the raw data file - this link is giving a 404: https://nodejs.org/en/user-survey-report/2018-nodejs-user-survey-raw-data.xls

[fhemberger]

@gtewallace Its .xlsx ;)

Adding the links and the hashtag now.

[fhemberger]

Just pushed the hashtags and download links.

[gtewallace]

awesome - thanks! not seeing them on the site just yet - should show up soon, right?

[fhemberger]

Sorry, committed a while ago. Build seems to hang again. ¯_(ツ)_/¯

[gtewallace]

@rvagg any chance you can look into why the latest commit build is hanging again?

[ZibbyKeaton]

I'm still getting a 404 when I used the update link for https://nodejs.org/en/user-survey-report/2018-nodejs-user-survey-raw-data.xlsx No rush as I know you are busy and I'm not sending this to media, etc.

[fhemberger]

I'm sorry, I'm too stupid to type today. Named the file .xslx by accident. Will fix it right away!

[fhemberger]

Pushed 22d935d.

[ZibbyKeaton]

Great! Thank you so much for all your help with this @fhemberger really, really appreciate it! And thank you too, @rvagg :)