Skip to content

Run Emscripten tests in a browser #569

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Run Emscripten tests in a browser #569

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions .github/workflows/deploy-pages.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,53 @@ jobs:
../
fi
emmake make -j ${{ env.ncpus }} check-cppinterop
cd ./unittests/CppInterOp/

# Fresh install browsers, and run Emscripten tests in them
# This is to match the Emscripten build instructions, where
# we run in a fresh browser, to stop any extra installed
# stuff interferring with the running of the tests
# Explaination of options for emrun
# --browser (name of browser on path)
# --kill_exit makes it so that when emrun finishes,
# that the headless browser we create is killed along with it
# --timeout 60 is such that emrun is killed after 60 seconds if
# still running. emrun should have finished long before then,
# so if it is still running, something went wrong (such as a test
# which crashed the html file). This will cause the ci to fail,
# as a non 0 value of will be returned.
# In the case of Chrome we have the extra --no-sandbox flag, as on
# Ubuntu Chrome will refuse to run otherwise, as it expects to have
# been installed with admin privileges. This flag allows it to run
# in userspace.

# Install Firefox
wget "https://download.mozilla.org/?product=firefox-latest&os=osx&lang=en-US" -O Firefox-latest.dmg
hdiutil attach Firefox-latest.dmg
cp -r /Volumes/Firefox/Firefox.app $PWD
hdiutil detach /Volumes/Firefox
cd ./Firefox.app/Contents/MacOS/
export PATH="$PWD:$PATH"
cd -

# Install Google Chrome
wget https://dl.google.com/chrome/mac/stable/accept_tos%3Dhttps%253A%252F%252Fwww.google.com%252Fintl%252Fen_ph%252Fchrome%252Fterms%252F%26_and_accept_tos%3Dhttps%253A%252F%252Fpolicies.google.com%252Fterms/googlechrome.pkg
pkgutil --expand-full googlechrome.pkg google-chrome
cd ./google-chrome/GoogleChrome.pkg/Payload/Google\ Chrome.app/Contents/MacOS/
export PATH="$PWD:$PATH"
cd -

# Run tests in browsers
echo "Running CppInterOpTests in Firefox"
emrun --browser="firefox" --kill_exit --timeout 60 --browser-args="--headless" CppInterOpTests.html
echo "Running DynamicLibraryManagerTests in Firefox"
emrun --browser="firefox" --kill_exit --timeout 60 --browser-args="--headless" DynamicLibraryManagerTests.html
echo "Running CppInterOpTests in Google Chrome"
emrun --browser="Google Chrome" --kill_exit --timeout 60 --browser-args="--headless --no-sandbox" CppInterOpTests.html
echo "Running DynamicLibraryManagerTests in Google Chrome"
emrun --browser="Google Chrome" --kill_exit --timeout 60 --browser-args="--headless --no-sandbox" DynamicLibraryManagerTests.html

cd ../../
emmake make -j ${{ env.ncpus }} install

cd ..
Expand Down
87 changes: 87 additions & 0 deletions .github/workflows/emscripten.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -599,6 +599,93 @@ jobs:
fi

emmake make -j ${{ env.ncpus }} check-cppinterop
cd ./unittests/CppInterOp/
# Fresh install browsers, and run Emscripten tests in them
# This is to match the Emscripten build instructions, where
# we run in a fresh browser, to stop any extra installed
# stuff interferring with the running of the tests
# Explaination of options for emrun
# --browser (name of browser on path)
# --kill_exit makes it so that when emrun finishes,
# that the headless browser we create is killed along with it
# --timeout 60 is such that emrun is killed after 60 seconds if
# still running. emrun should have finished long before then,
# so if it is still running, something went wrong (such as a test
# which crashed the html file). This will cause the ci to fail,
# as a non 0 value of will be returned.
# In the case of Chrome we have the extra --no-sandbox flag, as on
# Ubuntu Chrome will refuse to run otherwise, as it expects to have
# been installed with admin privileges. This flag allows it to run
# in userspace.
os="${{ matrix.os }}"
if [[ "${os}" == "macos"* ]]; then
Copy link
Collaborator Author

@mcbarton mcbarton May 2, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason we install a fresh install of Firefox and Chome is to follow what is done in the Emscripten repo (for example here with Firefox https://github.com/emscripten-core/emscripten/blob/38e16464cffc8f886364fe4919a712131a4c9456/.circleci/config.yml#L363 ). It has the added advantage of not using the users installed versions of these browsers (if they have them installed), that there are no add ons, etc that could be causing issues with the tests running. I have made the instructions, such that a developer without admin privileges on there machine can install them.

# Install Firefox
wget "https://download.mozilla.org/?product=firefox-latest&os=osx&lang=en-US" -O Firefox-latest.dmg
hdiutil attach Firefox-latest.dmg
cp -r /Volumes/Firefox/Firefox.app $PWD
hdiutil detach /Volumes/Firefox
cd ./Firefox.app/Contents/MacOS/
export PATH="$PWD:$PATH"
cd -

# Install Google Chrome
wget https://dl.google.com/chrome/mac/stable/accept_tos%3Dhttps%253A%252F%252Fwww.google.com%252Fintl%252Fen_ph%252Fchrome%252Fterms%252F%26_and_accept_tos%3Dhttps%253A%252F%252Fpolicies.google.com%252Fterms/googlechrome.pkg
pkgutil --expand-full googlechrome.pkg google-chrome
cd ./google-chrome/GoogleChrome.pkg/Payload/Google\ Chrome.app/Contents/MacOS/
export PATH="$PWD:$PATH"
cd -

# Run tests in browsers
echo "Running CppInterOpTests in Firefox"
emrun --browser="firefox" --kill_exit --timeout 60 --browser-args="--headless" CppInterOpTests.html
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

emrun allows us to run the created html file

--kill_exit makes it so that when emrun finishes, that the headless browser we create is killed along with it
--timeout 60 is such that emrun is killed after 60 seconds if still running. emrun should have finished long before then, so if it is still running, something went wrong (such as a test which crashed the html file). This will cause the ci to fail, as a non 0 value of will be returned.

echo "Running DynamicLibraryManagerTests in Firefox"
emrun --browser="firefox" --kill_exit --timeout 60 --browser-args="--headless" DynamicLibraryManagerTests.html
echo "Running CppInterOpTests in Google Chrome"
emrun --browser="Google Chrome" --kill_exit --timeout 60 --browser-args="--headless --no-sandbox" CppInterOpTests.html
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the case of Chrome we have the extra --no-sandbox flag, as on Ubuntu Chrome will refuse to run otherwise, as it expects to have been installed with admin privileges. This flag allows it to run in userspace.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you capture these comments as comments in the file?

echo "Running DynamicLibraryManagerTests in Google Chrome"
emrun --browser="Google Chrome" --kill_exit --timeout 60 --browser-args="--headless --no-sandbox" DynamicLibraryManagerTests.html
else
export ARCHITECHURE=$(uname -m)
if [[ "$ARCHITECHURE" != "aarch64" ]]; then
# Install Google Chrome
wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
dpkg-deb -x google-chrome-stable_current_amd64.deb $PWD/chrome
cd ./chrome/opt/google/chrome/
export PATH="$PWD:$PATH"
cd -

# Install Firefox
wget https://ftp.mozilla.org/pub/firefox/releases/138.0.1/linux-x86_64/en-GB/firefox-138.0.1.tar.xz
tar -xJf firefox-138.0.1.tar.xz
cd ./firefox
export PATH="$PWD:$PATH"
cd -

# Run tests in browsers
echo "Running CppInterOpTests in Firefox"
emrun --browser="firefox" --kill_exit --timeout 60 --browser-args="--headless" CppInterOpTests.html
echo "Running DynamicLibraryManagerTests in Firefox"
emrun --browser="firefox" --kill_exit --timeout 60 --browser-args="--headless" DynamicLibraryManagerTests.html
echo "Running CppInterOpTests in Google Chrome"
emrun --browser="google-chrome" --kill_exit --timeout 60 --browser-args="--headless --no-sandbox" CppInterOpTests.html
echo "Running DynamicLibraryManagerTests in Google Chrome"
emrun --browser="google-chrome" --kill_exit --timeout 60 --browser-args="--headless --no-sandbox" DynamicLibraryManagerTests.html
else
# Install Firefox
wget https://ftp.mozilla.org/pub/firefox/releases/138.0.1/linux-aarch64/en-GB/firefox-138.0.1.tar.xz
tar -xJf firefox-138.0.1.tar.xz
cd ./firefox
export PATH="$PWD:$PATH"
cd -

# Run tests in browsers
echo "Running CppInterOpTests in Firefox"
emrun --browser="firefox" --kill_exit --timeout 60 --browser-args="--headless" CppInterOpTests.html
echo "Running DynamicLibraryManagerTests in Firefox"
emrun --browser="firefox" --kill_exit --timeout 60 --browser-args="--headless" DynamicLibraryManagerTests.html
fi
fi
cd ../..
emmake make -j ${{ env.ncpus }} install
cd ..

Expand Down
80 changes: 78 additions & 2 deletions Emscripten-build-instructions.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -165,7 +165,7 @@ $env:CMAKE_PREFIX_PATH=$env:PREFIX
$env:CMAKE_SYSTEM_PREFIX_PATH=$env:PREFIX
```

on Windows. Now to build and test your Emscripten build of CppInterOp on Linux and osx execute the following
on Windows. Now to build and test your Emscripten build of CppInterOp using node on Linux and osx execute the following

```bash
mkdir build
Expand All @@ -182,7 +182,7 @@ emcmake cmake -DCMAKE_BUILD_TYPE=Release \
emmake make -j $(nproc --all) check-cppinterop
```

To build and test your Emscripten build of CppInterOp on Windows execute the following
To build and test your Emscripten build of CppInterOp on using node Windows execute the following

```powershell
emcmake cmake -DCMAKE_BUILD_TYPE=${{ env.BUILD_TYPE }} `
Expand All @@ -200,9 +200,85 @@ emcmake cmake -DCMAKE_BUILD_TYPE=${{ env.BUILD_TYPE }} `
emmake make -j $(nproc --all) check-cppinterop
```

It is possible to run the Emscripten tests in a headless browser on Linux and osx (in future we plan to include instructions on how to run the tests in a browser on Windows too). To do this we will first move to the tests directory


```bash
cd ./unittests/CppInterOp/
```

We will run our tests in a fresh installed browser. Installing the browsers, and running the tests within the installed browsers will be platform dependent. To do this on MacOS execute the following

```bash
wget "https://download.mozilla.org/?product=firefox-latest&os=osx&lang=en-US" -O Firefox-latest.dmg
hdiutil attach Firefox-latest.dmg
cp -r /Volumes/Firefox/Firefox.app $PWD
hdiutil detach /Volumes/Firefox
cd ./Firefox.app/Contents/MacOS/
export PATH="$PWD:$PATH"
cd -

wget https://dl.google.com/chrome/mac/stable/accept_tos%3Dhttps%253A%252F%252Fwww.google.com%252Fintl%252Fen_ph%252Fchrome%252Fterms%252F%26_and_accept_tos%3Dhttps%253A%252F%252Fpolicies.google.com%252Fterms/googlechrome.pkg
pkgutil --expand-full googlechrome.pkg google-chrome
cd ./google-chrome/GoogleChrome.pkg/Payload/Google\ Chrome.app/Contents/MacOS/
export PATH="$PWD:$PATH"
cd -

echo "Running CppInterOpTests in Firefox"
emrun --browser="firefox" --kill_exit --timeout 60 --browser-args="--headless" CppInterOpTests.html
echo "Running DynamicLibraryManagerTests in Firefox"
emrun --browser="firefox" --kill_exit --timeout 60 --browser-args="--headless" DynamicLibraryManagerTests.html
echo "Running CppInterOpTests in Google Chrome"
emrun --browser="Google Chrome" --kill_exit --timeout 60 --browser-args="--headless --no-sandbox" CppInterOpTests.html
echo "Running DynamicLibraryManagerTests in Google Chrome"
emrun --browser="Google Chrome" --kill_exit --timeout 60 --browser-args="--headless --no-sandbox" DynamicLibraryManagerTests.html
```

To do this on Ubuntu x86 execute the following

```bash
wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
dpkg-deb -x google-chrome-stable_current_amd64.deb $PWD/chrome
cd ./chrome/opt/google/chrome/
export PATH="$PWD:$PATH"
cd -

wget https://ftp.mozilla.org/pub/firefox/releases/138.0.1/linux-x86_64/en-GB/firefox-138.0.1.tar.xz
tar -xJf firefox-138.0.1.tar.xz
cd ./firefox
export PATH="$PWD:$PATH"
cd -

echo "Running CppInterOpTests in Firefox"
emrun --browser="firefox" --kill_exit --timeout 60 --browser-args="--headless" CppInterOpTests.html
echo "Running DynamicLibraryManagerTests in Firefox"
emrun --browser="firefox" --kill_exit --timeout 60 --browser-args="--headless" DynamicLibraryManagerTests.html
echo "Running CppInterOpTests in Google Chrome"
emrun --browser="google-chrome" --kill_exit --timeout 60 --browser-args="--headless --no-sandbox" CppInterOpTests.html
echo "Running DynamicLibraryManagerTests in Google Chrome"
emrun --browser="google-chrome" --kill_exit --timeout 60 --browser-args="--headless --no-sandbox" DynamicLibraryManagerTests.html
```

and on Ubuntu Arm execute the following (Google Chrome is not available on Ubuntu arm,
so we currently only run in Firefox on this platform, unlike other plaforms)

```bash
wget https://ftp.mozilla.org/pub/firefox/releases/138.0.1/linux-aarch64/en-GB/firefox-138.0.1.tar.xz
tar -xJf firefox-138.0.1.tar.xz
cd ./firefox
export PATH="$PWD:$PATH"
cd -

echo "Running CppInterOpTests in Firefox"
emrun --browser="firefox" --kill_exit --timeout 60 --browser-args="--headless" CppInterOpTests.html
echo "Running DynamicLibraryManagerTests in Firefox"
emrun --browser="firefox" --kill_exit --timeout 60 --browser-args="--headless" DynamicLibraryManagerTests.html
```

Assuming it passes all test you can install by executing the following

```bash
cd ../..
emmake make -j $(nproc --all) install
```

Expand Down
74 changes: 74 additions & 0 deletions docs/Emscripten-build-instructions.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -224,6 +224,80 @@ To build and test your Emscripten build of CppInterOp on Windows execute the fol
..\
emmake make -j $(nproc --all) check-cppinterop

It is possible to run the Emscripten tests in a headless browser on Linux and osx (in future we plan to include instructions on how to run the tests in a browser on Windows too). To do this we will first move to the tests directory

.. code:: bash

cd ./unittests/CppInterOp/

We will run our tests in a fresh installed browser. Installing the browsers, and running the tests within the installed browsers will be platform dependent. To do this on MacOS execute the following

.. code:: bash

wget "https://download.mozilla.org/?product=firefox-latest&os=osx&lang=en-US" -O Firefox-latest.dmg
hdiutil attach Firefox-latest.dmg
cp -r /Volumes/Firefox/Firefox.app $PWD
hdiutil detach /Volumes/Firefox
cd ./Firefox.app/Contents/MacOS/
export PATH="$PWD:$PATH"
cd -

wget https://dl.google.com/chrome/mac/stable/accept_tos%3Dhttps%253A%252F%252Fwww.google.com%252Fintl%252Fen_ph%252Fchrome%252Fterms%252F%26_and_accept_tos%3Dhttps%253A%252F%252Fpolicies.google.com%252Fterms/googlechrome.pkg
pkgutil --expand-full googlechrome.pkg google-chrome
cd ./google-chrome/GoogleChrome.pkg/Payload/Google\ Chrome.app/Contents/MacOS/
export PATH="$PWD:$PATH"
cd -

echo "Running CppInterOpTests in Firefox"
emrun --browser="firefox" --kill_exit --timeout 60 --browser-args="--headless" CppInterOpTests.html
echo "Running DynamicLibraryManagerTests in Firefox"
emrun --browser="firefox" --kill_exit --timeout 60 --browser-args="--headless" DynamicLibraryManagerTests.html
echo "Running CppInterOpTests in Google Chrome"
emrun --browser="Google Chrome" --kill_exit --timeout 60 --browser-args="--headless --no-sandbox" CppInterOpTests.html
echo "Running DynamicLibraryManagerTests in Google Chrome"
emrun --browser="Google Chrome" --kill_exit --timeout 60 --browser-args="--headless --no-sandbox" DynamicLibraryManagerTests.html

To do this on Ubuntu x86 execute the following

.. code:: bash

wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
dpkg-deb -x google-chrome-stable_current_amd64.deb $PWD/chrome
cd ./chrome/opt/google/chrome/
export PATH="$PWD:$PATH"
cd -

wget https://ftp.mozilla.org/pub/firefox/releases/138.0.1/linux-x86_64/en-GB/firefox-138.0.1.tar.xz
tar -xJf firefox-138.0.1.tar.xz
cd ./firefox
export PATH="$PWD:$PATH"
cd -

echo "Running CppInterOpTests in Firefox"
emrun --browser="firefox" --kill_exit --timeout 60 --browser-args="--headless" CppInterOpTests.html
echo "Running DynamicLibraryManagerTests in Firefox"
emrun --browser="firefox" --kill_exit --timeout 60 --browser-args="--headless" DynamicLibraryManagerTests.html
echo "Running CppInterOpTests in Google Chrome"
emrun --browser="google-chrome" --kill_exit --timeout 60 --browser-args="--headless --no-sandbox" CppInterOpTests.html
echo "Running DynamicLibraryManagerTests in Google Chrome"
emrun --browser="google-chrome" --kill_exit --timeout 60 --browser-args="--headless --no-sandbox" DynamicLibraryManagerTests.html

and on Ubuntu Arm execute the following (Google Chrome is not available on Ubuntu arm,
so we currently only run the tests using Firefox on this platform, unlike other plaforms)

.. code:: bash

wget https://ftp.mozilla.org/pub/firefox/releases/138.0.1/linux-aarch64/en-GB/firefox-138.0.1.tar.xz
tar -xJf firefox-138.0.1.tar.xz
cd ./firefox
export PATH="$PWD:$PATH"
cd -

echo "Running CppInterOpTests in Firefox"
emrun --browser="firefox" --kill_exit --timeout 60 --browser-args="--headless" CppInterOpTests.html
echo "Running DynamicLibraryManagerTests in Firefox"
emrun --browser="firefox" --kill_exit --timeout 60 --browser-args="--headless" DynamicLibraryManagerTests.html

Assuming it passes all test you can install by executing the following.

.. code:: bash
Expand Down
6 changes: 6 additions & 0 deletions unittests/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,13 @@ if(WIN32)
else()
target_link_libraries(${name} PRIVATE ${ARG_LIBRARIES} ${gtest_libs} ${link_pthreads_lib})
endif()
if(EMSCRIPTEN)
# Without this cmake will try and get node to run the html file.
# This guarantees that it runs the js file, and uses emsdks node.
add_test(NAME cppinterop-${name} COMMAND $ENV{EMSDK_NODE} ${name}.js)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without this cmake will try and get node to run the html file. This guarantees that it runs the js file, and uses emsdks node.

else()
add_test(NAME cppinterop-${name} COMMAND ${name})
endif()
set_tests_properties(cppinterop-${name} PROPERTIES
TIMEOUT "${TIMEOUT_VALUE}"
ENVIRONMENT "CPLUS_INCLUDE_PATH=${CMAKE_BINARY_DIR}/etc"
Expand Down
8 changes: 8 additions & 0 deletions unittests/CppInterOp/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
if (EMSCRIPTEN)
# So we create a html file, as well as the javascript fil
set(CMAKE_EXECUTABLE_SUFFIX ".html")
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So that we create a html file, as well as the javascript file.

# Omitting CUDATest.cpp since Emscripten build currently has no GPU support
# For Emscripten builds linking to gtest_main will not suffice for gtest to run
# the tests and an explicitly main.cpp is needed
Expand Down Expand Up @@ -42,6 +44,10 @@ if(EMSCRIPTEN)
#
# --preload-file ${SYSROOT_PATH}/include@/include:
# Preloads the system include directory into the Emscripten virtual filesystem to make headers accessible at runtime.
#
# --emrun
# Makes it so that we run the html file created by this target, that we can capture the set_output_directory
# and output to the terminal
target_link_options(CppInterOpTests
PUBLIC "SHELL: -fexceptions"
PUBLIC "SHELL: -s MAIN_MODULE=1"
Expand All @@ -50,6 +56,7 @@ if(EMSCRIPTEN)
PUBLIC "SHELL: -s STACK_SIZE=32mb"
PUBLIC "SHELL: -s INITIAL_MEMORY=128mb"
PUBLIC "SHELL: --preload-file ${SYSROOT_PATH}/include@/include"
PUBLIC "SHELL: --emrun"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

Rebuild your Emscripten application and add the --emrun [linker flag](https://emscripten.org/docs/tools_reference/emcc.html#emcc-emrun).

This flag injects code into the generated [Module object](https://emscripten.org/docs/api_reference/module.html#module) to enable capture of stdout, stderr and exit().

Note

If you skip this step, you can still run any .html file with emrun, but the capture will not work.

The above is taken from emruns documentation https://emscripten.org/docs/compiling/Running-html-files-with-emrun.html

)
endif()

Expand Down Expand Up @@ -97,6 +104,7 @@ if(EMSCRIPTEN)
PUBLIC "SHELL: -s STACK_SIZE=32mb"
PUBLIC "SHELL: -s INITIAL_MEMORY=128mb"
PUBLIC "SHELL: --preload-file ${SYSROOT_PATH}/include@/include"
PUBLIC "SHELL: --emrun"
)
endif()

Expand Down
Loading