Skip to content

add CI workflow to crosscompile all the tools and pubblish them #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 18 commits into from
Mar 26, 2021

Conversation

umbynos
Copy link
Contributor

@umbynos umbynos commented Mar 18, 2021

I've created the workflow to cross-compile all the tools (picotool, elf2uf2. and rp2040load). ⚒️
The workflow is divided in:

  • a job which uses a docker container (called crossbuild) with all the toolchains inside required to cross-compile the two tools from raspberry pi. The binaries produced are as static and self-contained as possible. (Obviously, this is run in parallel to minimize time) 🏃
  • a job that cross-compiles the go tool called rp2040load, cross-compiling this tool is so much easier than compiling the C++ ones. 😅
  • one last job used to move in the correct folders the binaries and to tar.bz2 them and upload them in the release page and on s3 download server 🆙
    image
    image
    In less than 6 5 mins we can produce a release! 📦 🥳
    Release binaries are structured as follows:
release/
├── tools_darwin_amd64
│   ├── elf2uf2
│   ├── picotool
│   └── rp2040load
├── tools_linux_386
│   ├── elf2uf2
│   ├── picotool
│   └── rp2040load
├── tools_linux_amd64
│   ├── elf2uf2
│   ├── picotool
│   └── rp2040load
├── tools_linux_arm
│   ├── elf2uf2
│   ├── picotool
│   └── rp2040load
├── tools_linux_arm64
│   ├── elf2uf2
│   ├── picotool
│   └── rp2040load
└── tools_windows_386
    ├── elf2uf2.exe
    ├── picotool.exe
    └── rp2040load.exe

TODO:

  • Clean up the container as of now it weights almost 10gb (and maybe move it to another repo)
  • Clean up the repo (removing old and useless scripts)
  • Publish artifacts on download servers
  • Patches probably can be removed and added as command-line flags

@umbynos umbynos self-assigned this Mar 18, 2021
@umbynos
Copy link
Contributor Author

umbynos commented Mar 22, 2021

With the optimization from #2, i managed to drop the image size from 9.22GB to 7.76GB and thus cut CI step downloading the container from 4mins to 3 🥳 🎈

@umbynos umbynos force-pushed the umbynos/crossbuild branch from ab84734 to bacd5f4 Compare March 24, 2021 17:09
@umbynos
Copy link
Contributor Author

umbynos commented Mar 24, 2021

apparently aws cli does not give a meaningful message when credentials or bucket are wrong.. The error message was:

fatal error: An error occurred (SignatureDoesNotMatch) when calling the ListObjectsV2 operation: The request signature we calculated does not match the signature you provided. Check your key and signing method.

Reinserting the secrets solved the problem. Thanks @rsora for the help in the troubleshooting

@umbynos umbynos force-pushed the umbynos/crossbuild branch from 5aad3bd to 3ba6d8b Compare March 24, 2021 18:32
@umbynos umbynos force-pushed the umbynos/crossbuild branch from 8d23681 to 6be9216 Compare March 25, 2021 08:57
Comment on lines 91 to 95
if [ "${{ matrix.os }}" = "darwin" ]; then
cmake -DCMAKE_C_COMPILER=${{ matrix.cross_compiler }} -DCMAKE_CXX_COMPILER=${{ matrix.cross_compiler }}++ -DCMAKE_CXX_FLAGS="${CMAKE_CXX_FLAGS} -framework IOKit -framework Cocoa -pthread" -DLIBUSB_LIBRARIES=$LIBUSBUDEV -DLIBUSB_INCLUDE_DIR=$LIBUSB_DIR ..
else
cmake -DCMAKE_C_COMPILER=${{ matrix.cross_compile }}-gcc -DCMAKE_CXX_COMPILER=${{ matrix.cross_compile }}-g++ -DCMAKE_EXE_LINKER_FLAGS="-static-libgcc -static-libstdc++" -DCMAKE_CXX_FLAGS="${CMAKE_CXX_FLAGS} -pthread" -DLIBUSB_LIBRARIES=$LIBUSBUDEV -DLIBUSB_INCLUDE_DIR=$LIBUSB_DIR ..
fi
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know it's not ideal, but IMHO it's better than duplicating the steps, like it were in the beginning.
I tried some tinkering to have the cmake flags as variables but I think it was losing readability..

@umbynos umbynos requested review from facchinm and per1234 March 25, 2021 09:01
@umbynos umbynos requested a review from per1234 March 25, 2021 15:43
Copy link
Contributor

@per1234 per1234 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work Umberto!

@umbynos umbynos merged commit 26f4918 into master Mar 26, 2021
@umbynos umbynos deleted the umbynos/crossbuild branch March 26, 2021 10:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants