Building LTS v12.x on Windows Fails The system cannot find the file specified."

[jwyglendowski-precisionlender]

• Version: 12.x
• Platform: Windows
• Subsystem:

I took the documentation specified in Building.md and converted that into a DockerFile. I had to make some additions to supplement the boxstarter scripts because that ran into errors installing the build tools. To get around that issue I installed those first via npm, following guidelines in this post, then called the boxstarter scripts. This enabled me to get passed that particular problem but now I have a new issue which is detailed below in the console output. I have included a dockerfile to reproduce the issue. Any assistance will be greatly appreciated.

Output of the build

C:\node>.\vcbuild full-icu download-all
Looking for Python
Python 2 found in C:\Python27\\python.exe
Looking for NASM
Looking for Visual Studio 2017
Found MSVS version 15.0
configure  "--download=all" --with-intl=full-icu --dest-cpu=x64
INFO: Using floating patch "tools/icu/patches/64/source/common/putil.cpp" from "tools/icu"
INFO: Using floating patch "tools/icu/patches/64/source/i18n/dtptngen.cpp" from "tools/icu"
Warning: Missing input files:
tools\msvs\genfiles\node_etw_provider.rc
tools\msvs\genfiles\node_etw_provider.h
tools\v8_gypfiles\..\..\deps\v8\src\regexp\regexp-special-case.h
INFO: configure completed successfully
Project files generated.
The system cannot find the file specified.

Dockerfile to reproduce the behaviour

ARG version=ltsc2019
FROM mcr.microsoft.com/windows/servercore:$version

ENV chocolateyUseWindowsCompression false

RUN powershell -Command \
    iex ((new-object net.webclient).DownloadString('https://chocolatey.org/install.ps1')); \
    choco feature disable --name showDownloadProgress  

RUN powershell choco install nodejs-lts -y
RUN npm install --global --production windows-build-tools
RUN npm config set msvs_version 2017 -global
RUN npm install --global node-gyp --no-optional
RUN powershell choco install vim -y 


RUN powershell -Command \
    iex ((New-Object System.Net.WebClient).DownloadString('https://boxstarter.org/bootstrapper.ps1')) ; \
    Get-Boxstarter -Force ; \
    Install-BoxstarterPackage https://github.com/raw/nodejs/node/master/tools/bootstrap/windows_boxstarter -DisableReboots

RUN powershell git clone https://github.com/nodejs/node.git
RUN powershell  Set-Location -Path C:\Node; git checkout v12.x    

RUN powershell Set-Location -Path C:\Node; .\vcbuild full-icu download-all

[bzoz]

Building v12 works fine on my local machine.

@jwyglendowski-precisionlender , you say that the Boxstarter script run into some issues. What were those issues exactly?

If for whatever reason installing VS does not work through the Boxstarter but works using npm, there is no need to run the Boxstarter script. You are only missing NASM and git. With:

choco install git -params /GitAndUnixToolsOnPath -y
choco install nasm -y

you should be able to build Node.

Finally, remove

RUN npm install --global node-gyp --no-optional

The node-gyp bundled with Node should work.

If you still are having issues, set DEBUG_HELPER environment variable to 1 before running the vcbuild script. It will produce a lot more output.

[jwyglendowski-precisionlender]

@bzoz Thanks for replying.

Here is the error console log for the boxstarter failure for reference. I will adjustment my DockerFile removing out the boxster bits for the two choco commands. Will run the new docker build which should fail running the node build based on previous runs. That being said I will add the DEBUG_HELPER as you have suggested and report back my findings.

Download of ChannelManifest_81725945.man (69.47 KB) completed.
Download of ChannelManifest_81725945.man (69.47 KB) completed.
Downloading catalog manifest 
  from 'https://download.visualstudio.microsoft.com/download/pr/57c99182-2828-4fc1-8b36-271e2472c838/cc21f47884f69b53a6dee91681ba4c4caa1fe14a282a26e0d13cfbf3b01fad91/VisualStudio.vsman'Downloading catalog manifest 
  from 'https://download.visualstudio.microsoft.com/download/pr/57c99182-2828-4fc1-8b36-271e2472c838/cc21f47884f69b53a6dee91681ba4c4caa1fe14a282a26e0d13cfbf3b01fad91/VisualStudio.vsman'

Download of Catalog_-1171790049.man (7.93 MB) completed.
Download of Catalog_-1171790049.man (7.93 MB) completed.
Installing visualstudio2017-workload-vctools...
Installing visualstudio2017-workload-vctools...

ERROR: Running ["C:\Program Files (x86)\Microsoft Visual Studio\Installer\vs_installer.exe" modify --installPath "C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools" --includeRecommended --norestart --quiet --add Microsoft.VisualStudio.Component.VC.Tools.ARM64 --add Microsoft.VisualStudio.Component.VC.ATL.ARM64 --add Microsoft.VisualStudio.Workload.VCTools] was not successful. Exit code was '1'. See log for possible error messages.
The install of visualstudio2017-workload-vctools was NOT successful.
Error while running 'C:\ProgramData\chocolatey\lib\visualstudio2017-workload-vctools\tools\ChocolateyInstall.ps1'.
 See log for details.

Chocolatey installed 0/1 packages. 1 packages failed.
 See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).

Failures
 - visualstudio2017-workload-vctools (exited 1) - Error while running 'C:\ProgramData\chocolatey\lib\visualstudio2017-workload-vctools\tools\ChocolateyInstall.ps1'.
 See log for details.
Chocolatey reported an unsuccessful exit code of 1. See C:\Users\ContainerAdministrator\AppData\Local\Boxstarter\boxstarter.log for details.
+ Boxstarter finished Calling Chocolatey to install visualstudio2017-workload-vctools. This may take several minutes to complete... 00:04:33.8006399
+ Boxstarter finished Calling Chocolatey to install visualstudio2017-workload-vctools. This may take several minutes to complete... 00:04:33.8006399
+ Boxstarter starting Calling Chocolatey to install nasm. This may take several minutes to complete...
+ Boxstarter starting Calling Chocolatey to install nasm. This may take several minutes to complete...
Installing the following packages:
nasm
By installing you accept licenses for the packages.
[NuGet] Installing 'nasm 2.14.02'.
[NuGet] Successfully installed 'nasm 2.14.02'.

nasm v2.14.02 [Approved]
nasm package files install completed. Performing other installation steps.
Downloading nasm 64 bit
  from 'https://www.nasm.us/pub/nasm/releasebuilds/2.14.02/win64/nasm-2.14.02-installer-x64.exe'
Downloading nasm 64 bit
  from 'https://www.nasm.us/pub/nasm/releasebuilds/2.14.02/win64/nasm-2.14.02-installer-x64.exe'


Download of nasm-2.14.02-installer-x64.exe (933.77 KB) completed.
Download of nasm-2.14.02-installer-x64.exe (933.77 KB) completed.
Hashes match.
Installing nasm...
Installing nasm...
nasm has been installed.
nasm has been installed.
 The install of nasm was successful.
  Software installed as 'exe', install location is likely default.

Chocolatey installed 1/1 packages. 0 packages failed.
 See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).
+ Boxstarter finished Calling Chocolatey to install nasm. This may take several minutes to complete... 00:00:09.6367086
+ Boxstarter finished Calling Chocolatey to install nasm. This may take several minutes to complete... 00:00:09.6367086
Only an exit code of non-zero will fail the package by default. Set
 `--failonstderr` if you want error messages to also fail a script. See
 `choco -h` for details.
Environment Vars (like PATH) have changed. Close/reopen your shell to
 see the changes (or in powershell/cmd.exe just type `refreshenv`).
 The install of tmp6820.tmp was successful.
  Software installed to 'C:\Program Files\Git\'

Chocolatey installed 1/1 packages. 0 packages failed.
 See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).```

[jwyglendowski-precisionlender]

@bzoz I have the full log but I will include just the last bits.

C:\Node>call "C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\VC\\Auxiliary\Build\vcvarsall.bat" amd64
**********************************************************************
** Visual Studio 2017 Developer Command Prompt v15.0
** Copyright (c) 2017 Microsoft Corporation
**********************************************************************
[vcvarsall.bat] Environment initialized for: 'x64'

C:\node>echo Found MSVS version 15.0 
Found MSVS version 15.0

C:\node>set GYP_MSVS_VERSION=2017

C:\node>set PLATFORM_TOOLSET=v141

C:\node>goto msbuild-found

C:\node>set project_generated=

C:\node>if defined noprojgen goto msbuild

C:\node>if defined projgen goto run-configure

C:\node>if not exist node.sln goto run-configure

C:\node>del .tmp_gyp_configure_stamp  2>NUL

C:\node>del .gyp_configure_stamp  2>NUL 

C:\node>echo configure  "--download=all" --with-intl=full-icu --dest-cpu=x64 --verbose
configure  "--download=all" --with-intl=full-icu --dest-cpu=x64 --verbose

C:\node>echo  "--download=all" --with-intl=full-icu --dest-cpu=x64 --verbose 1>.used_configure_flags

C:\node>python configure  "--download=all" --with-intl=full-icu --dest-cpu=x64 --verbose
creating icu_config.gypi
creating icu_config.gypi
 <https://github.com/unicode-org/icu/releases/download/release-64-2/icu4c-64_2-src.tgz>
 ** IOError [Errno socket error] [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:726)

WARNING: * ECMA-402 (Intl) support didn't find ICU in deps/icu..
ERROR: Cannot build Intl without ICU in deps/icu.
       Fix, or disable with "--with-intl=none"

C:\node>if errorlevel 1 goto create-msvs-files-failed

C:\node>echo Failed to create vc project files.
Failed to create vc project files.

C:\node>if 2 == 3 (echo Python 3 is not yet fully supported, to avoid issues Python 2 should be installed. )

C:\node>del .used_configure_flags

C:\node>goto exit

C:\node>if 0 NEQ 0 exit /b 0

C:\node>exit /b 0```

[bzoz]

@jwyglendowski-precisionlender could you try the build again?

I also saw this CERTIFICATE_VERIFY_FAILED, but when I sat down today to investigate it, it no longer reproduces. Could you give it another try?

[jwyglendowski-precisionlender]

@bzoz I am still getting the error. I run the following docker build file and command. Are you running this in Docker during your test?

Docker Build: docker build -t nodejs/12.x:latest --no-cache .
My docker file is the following:

ARG version=ltsc2019
FROM mcr.microsoft.com/windows/servercore:$version

ENV chocolateyUseWindowsCompression false

RUN powershell -Command \
    iex ((new-object net.webclient).DownloadString('https://chocolatey.org/install.ps1')); \
    choco feature disable --name showDownloadProgress  

RUN powershell choco install nodejs-lts -y
RUN npm install --global --production windows-build-tools
RUN npm config set msvs_version 2017 -global
RUN npm install --global node-gyp --no-optional
RUN choco install vim -y 
RUN choco install git -params /GitAndUnixToolsOnPath -y
RUN choco install nasm -y

ENV DEBUG_HELPER 1

# RUN powershell -Command \
#     iex ((New-Object System.Net.WebClient).DownloadString('https://boxstarter.org/bootstrapper.ps1')) ; \
#     Get-Boxstarter -Force ; \
#     Install-BoxstarterPackage https://github.com/raw/nodejs/node/master/tools/bootstrap/windows_boxstarter -DisableReboots

RUN powershell git clone https://github.com/nodejs/node.git
RUN powershell Set-Location -Path C:\Node; git checkout v12.x    

RUN powershell Set-Location -Path C:\Node; .\vcbuild full-icu download-all

Still getting

**********************************************************************
** Visual Studio 2017 Developer Command Prompt v15.0
** Copyright (c) 2017 Microsoft Corporation
**********************************************************************
[vcvarsall.bat] Environment initialized for: 'x64'

C:\node>echo Found MSVS version 15.0 
Found MSVS version 15.0

C:\node>set GYP_MSVS_VERSION=2017

C:\node>set PLATFORM_TOOLSET=v141

C:\node>goto msbuild-found

C:\node>set project_generated=

C:\node>if defined noprojgen goto msbuild 

C:\node>if defined projgen goto run-configure

C:\node>if not exist node.sln goto run-configure

C:\node>del .tmp_gyp_configure_stamp  2>NUL

C:\node>del .gyp_configure_stamp  2>NUL 

C:\node>echo configure  "--download=all" --with-intl=full-icu --dest-cpu=x64 --verbose
configure  "--download=all" --with-intl=full-icu --dest-cpu=x64 --verbose

C:\node>echo  "--download=all" --with-intl=full-icu --dest-cpu=x64 --verbose 1>.used_configure_flags

C:\node>python configure  "--download=all" --with-intl=full-icu --dest-cpu=x64 --verbose
creating icu_config.gypi
 <https://github.com/unicode-org/icu/releases/download/release-64-2/icu4c-64_2-src.tgz>
 ** IOError [Errno socket error] [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:726)

WARNING: * ECMA-402 (Intl) support didn't find ICU in deps/icu..
ERROR: Cannot build Intl without ICU in deps/icu.
       Fix, or disable with "--with-intl=none"

C:\node>if errorlevel 1 goto create-msvs-files-failed

C:\node>echo Failed to create vc project files.
Failed to create vc project files.

C:\node>if 2 == 3 (echo Python 3 is not yet fully supported, to avoid issues Python 2 should be installed. )

C:\node>del .used_configure_flags

C:\node>goto exit

C:\node>if 0 NEQ 0 exit /b 0

C:\node>exit /b 0

[bzoz]

I'm running this from a Windows VM and I saw the cert error last week. It worked today when I tried this on a clean machine, although I skipped RUN npm install --global node-gyp --no-optional since installing node-gyp usually messes things up.

I'll give it a spin in a docker container.

[bzoz]

@jwyglendowski-precisionlender: ok, it reproduces almost 100% of times. The download stage did work once when I tried it in an interactive terminal though.

As a temporary workaround:

• download the ICU package manually
• run this Dockerfile:

ARG version=ltsc2019
# Use image with preinstalled DotNet 4.8, otherwise VS installer
# complains that the machine needs a reboot:
FROM mcr.microsoft.com/dotnet/framework/sdk:4.8-windowsservercore-$version
ENV chocolateyUseWindowsCompression false
ENV PYTHONIOENCODING UTF-8 

SHELL ["cmd", "/s", "/c"]

RUN powershell -Command iex ((new-object net.webclient).DownloadString('https://chocolatey.org/install.ps1'))
RUN choco install visualstudio2017buildtools -y
RUN choco install visualstudio2017-workload-vctools -y --params="--add Microsoft.VisualStudio.Component.VC.Tools.ARM64 --add Microsoft.VisualStudio.Component.VC.ATL.ARM64"
RUN choco install nasm -y
RUN choco install git -params /GitAndUnixToolsOnPath -y
RUN choco install python2 -y

RUN git clone https://github.com/nodejs/node.git
WORKDIR c:\\node
RUN git checkout v12.x

# workaround start
# copy the ICU file and extract it
COPY icu4c-64_2-src.tgz c:\\node\\deps\\icu4c-64_2-src.tgz
WORKDIR c:\\node\\deps
RUN mkdir icu
RUN tar -xzf icu4c-64_2-src.tgz
WORKDIR c:\\node
# workaround end

RUN vcbuild full-icu

@nodejs/docker @nodejs/build @nodejs/platform-windows - for whatever reason building v12.x with full-icu in a Docker container fails when trying to download the ICU files:

C:\node>python configure  "--download=all" --with-intl=full-icu --dest-cpu=x64 --verbose
creating icu_config.gypi
 <https://github.com/unicode-org/icu/releases/download/release-64-2/icu4c-64_2-src.tgz>
 ** IOError [Errno socket error] [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:726)

WARNING: * ECMA-402 (Intl) support didn't find ICU in deps/icu..
ERROR: Cannot build Intl without ICU in deps/icu.
       Fix, or disable with "--with-intl=none"

This does not happen on master since full ICU is now included by default (#29522). Any ideas?

[jwyglendowski-precisionlender]

@bzoz Thanks. I am running that now and as soon as it's complete I will post the results.

[jwyglendowski-precisionlender]

@bzoz I made a minor change to the Dockerfile so that it would download the ICU from the script

ARG version=ltsc2019
# Use image with preinstalled DotNet 4.8, otherwise VS installer
# complains that the machine needs a reboot:
FROM mcr.microsoft.com/dotnet/framework/sdk:4.8-windowsservercore-$version
ENV chocolateyUseWindowsCompression false
ENV PYTHONIOENCODING UTF-8 

SHELL ["cmd", "/s", "/c"]

RUN powershell -Command iex ((new-object net.webclient).DownloadString('https://chocolatey.org/install.ps1'))
RUN choco install visualstudio2017buildtools -y
RUN choco install visualstudio2017-workload-vctools -y --params="--add Microsoft.VisualStudio.Component.VC.Tools.ARM64 --add Microsoft.VisualStudio.Component.VC.ATL.ARM64"
RUN choco install nasm -y
RUN choco install git -params /GitAndUnixToolsOnPath -y
RUN choco install python2 -y

RUN git clone https://github.com/nodejs/node.git
WORKDIR c:\\node
RUN git checkout v12.x

# workaround start
# copy the ICU file and extract it
RUN powershell Invoke-WebRequest "https://github.com/unicode-org/icu/releases/download/release-64-2/icu4c-64_2-src.tgz" -OutFile c:\\node\\deps\\icu4c-64_2-src.tgz
#COPY icu4c-64_2-src.tgz c:\\node\\deps\\icu4c-64_2-src.tgz
WORKDIR c:\\node\\deps
RUN mkdir icu
RUN tar -xzf icu4c-64_2-src.tgz
WORKDIR c:\\node
# workaround end

RUN vcbuild full-icu

So the build version

C:\node\Release>node
Welcome to Node.js v12.16.2-pre.
Type ".help" for more information.
> const january = new Date(9e8);
undefined
> const english = new Intl.DateTimeFormat('en', { month: 'long' });
undefined
> const spanish = new Intl.DateTimeFormat('es', { month: 'long' });
undefined
> console.log(english.format(january));
January
undefined
> console.log(spanish.format(january));
enero
undefined
>

Is there an easy way to build it without the -pre in the version? Is there an ENV variable I can set? Is there an easy way to know which version of FULL-ICU goes with 12.x LTS?

[sam-github]

Is there an easy way to build it without the -pre in the version?

Build from a tag, v12.16.1 (for example), if it says v12.16.2-pre, that's what it is, the 12.x dev branch.

[jwyglendowski-precisionlender]

@sam-github Thanks Sam. That did do the trick on the node side. Do you know of a way to determine which ICU version goes with a particular node version since the vcbuild full-icu download-all doesn't work?

[bzoz]

@jwyglendowski-precisionlender it is in tools/icu/current_ver.dep file.

[jwyglendowski-precisionlender]

@bzoz Cool. So I should be able to create a complete docker file where all I have to specify is a tag version then everything else should theoretically take care of itself. Have you looked into dev containers. It might be useful for others or just have a dockerfile for each platform. Even if you don't this thread will provide documentation on how to do it. Once I have a completed dockerfile that pulls the url from the tools/icu/current_ver.dep file I will post it here.