Skip to content

Multithreaded CI builds #3311

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Sep 28, 2023
Merged

Multithreaded CI builds #3311

Changes from 10 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
537ff64
mac and linux threads
netrunnereve Sep 22, 2023
b38e1dc
windows
netrunnereve Sep 22, 2023
ad1e2a7
Update build.yml
netrunnereve Sep 22, 2023
55eb46b
Update build.yml
netrunnereve Sep 22, 2023
25f4e52
Update build.yml
netrunnereve Sep 23, 2023
1e2fec1
automatically get thread count
netrunnereve Sep 23, 2023
05c5ada
windows syntax
netrunnereve Sep 23, 2023
4913506
try to fix freebsd
netrunnereve Sep 23, 2023
45b4697
Update build.yml
netrunnereve Sep 23, 2023
a2c43f8
Update build.yml
netrunnereve Sep 23, 2023
287a0e9
Merge branch 'master' into ci_threads
netrunnereve Sep 28, 2023
5fe3594
Update build.yml
netrunnereve Sep 28, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 16 additions & 16 deletions .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,13 +38,13 @@ jobs:
- name: Build
id: make_build
run: |
CC=gcc-8 make
CC=gcc-8 make -j $(nproc)

- name: Test
id: make_test
run: |
CC=gcc-8 make tests
make test
CC=gcc-8 make tests -j $(nproc)
make test -j $(nproc)

ubuntu-latest-cmake:
runs-on: ubuntu-latest
Expand All @@ -66,7 +66,7 @@ jobs:
mkdir build
cd build
cmake ..
cmake --build . --config Release
cmake --build . --config Release -j $(nproc)

- name: Test
id: cmake_test
Expand Down Expand Up @@ -101,7 +101,7 @@ jobs:
mkdir build
cd build
cmake .. -DLLAMA_SANITIZE_${{ matrix.sanitizer }}=ON -DCMAKE_BUILD_TYPE=${{ matrix.build_type }}
cmake --build . --config ${{ matrix.build_type }}
cmake --build . --config ${{ matrix.build_type }} -j $(nproc)

- name: Test
id: cmake_test
Expand Down Expand Up @@ -135,7 +135,7 @@ jobs:
mkdir build
cd build
cmake -DLLAMA_MPI=ON ..
cmake --build . --config Release
cmake --build . --config Release -j $(nproc)

- name: Test
id: cmake_test
Expand All @@ -160,13 +160,13 @@ jobs:
- name: Build
id: make_build
run: |
make
make -j $(sysctl -n hw.logicalcpu)

- name: Test
id: make_test
run: |
make tests
make test
make tests -j $(sysctl -n hw.logicalcpu)
make test -j $(sysctl -n hw.logicalcpu)

macOS-latest-cmake:
runs-on: macos-latest
Expand All @@ -189,7 +189,7 @@ jobs:
mkdir build
cd build
cmake -DLLAMA_AVX2=OFF -DLLAMA_FMA=OFF ..
cmake --build . --config Release
cmake --build . --config Release -j $(sysctl -n hw.logicalcpu)

- name: Test
id: cmake_test
Expand Down Expand Up @@ -223,7 +223,7 @@ jobs:
-DLLAMA_BUILD_SERVER=OFF \
-DCMAKE_SYSTEM_NAME=iOS \
-DCMAKE_OSX_DEPLOYMENT_TARGET=14.0
cmake --build . --config Release
cmake --build . --config Release -j $(sysctl -n hw.logicalcpu)

macOS-latest-cmake-tvos:
runs-on: macos-latest
Expand Down Expand Up @@ -251,7 +251,7 @@ jobs:
-DLLAMA_BUILD_SERVER=OFF \
-DCMAKE_SYSTEM_NAME=tvOS \
-DCMAKE_OSX_DEPLOYMENT_TARGET=14.0
cmake --build . --config Release
cmake --build . --config Release -j $(sysctl -n hw.logicalcpu)

windows-latest-cmake:
runs-on: windows-latest
Expand Down Expand Up @@ -324,7 +324,7 @@ jobs:
mkdir build
cd build
cmake .. ${{ matrix.defines }}
cmake --build . --config Release
cmake --build . --config Release -j ${env:NUMBER_OF_PROCESSORS}

- name: Add clblast.dll
id: add_clblast_dll
Expand Down Expand Up @@ -415,7 +415,7 @@ jobs:
mkdir build
cd build
cmake .. -DLLAMA_BUILD_SERVER=ON -DLLAMA_CUBLAS=ON -DBUILD_SHARED_LIBS=ON
cmake --build . --config Release
cmake --build . --config Release -j ${env:NUMBER_OF_PROCESSORS}

- name: Determine tag name
id: tag
Expand Down Expand Up @@ -458,7 +458,7 @@ jobs:
cudart-llama-bin-win-cu${{ matrix.cuda }}-x64.zip

freeBSD-latest:
runs-on: macos-12
runs-on: ubuntu-latest
steps:
- name: Clone
uses: actions/checkout@v3
Expand All @@ -472,7 +472,7 @@ jobs:
run: |
sudo pkg update
sudo pkg install -y gmake automake autoconf pkgconf llvm15 clinfo clover opencl clblast openblas
gmake CC=/usr/local/bin/clang15 CXX=/usr/local/bin/clang++15
gmake CC=/usr/local/bin/clang15 CXX=/usr/local/bin/clang++15 -j `sysctl -n hw.ncpu`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any reason you switched syntax for this one?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The FreeBSD shell doesn't support the $() syntax.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. Should we switch all the others then?
This is probably the difference between sh and bash

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is probably the difference between sh and bash

Nope. Modern POSIX sh supports $(), you can try it out for yourself with the dash shell. FreeBSD defaults to tcsh which doesn't support $().


release:
if: ${{ ( github.event_name == 'push' && github.ref == 'refs/heads/master' ) || github.event.inputs.create_release == 'true' }}
Expand Down