[CI] reducing image size #20201
[CI] reducing image size #20201
Conversation
Signed-off-by: Aaron Pham <[email protected]>
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add 🚀 |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Summary of Changes
Hello @aarnphm, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request is focused on optimizing the docker/Dockerfile to reduce the final container image size and improve build efficiency. My primary approach involved consolidating multiple RUN instructions into fewer layers by leveraging Docker's heredoc syntax, aligning with best practices for Dockerfile construction.
Highlights
- Dockerfile Syntax Update: I've explicitly added the Dockerfile syntax directive (
# syntax=docker.io/docker/dockerfile:1.17) to enable newer Dockerfile features and ensure consistent parsing. - Layer Consolidation and Optimization: I've refactored numerous
RUNcommands indocker/Dockerfileby combining chained commands (&&) into multi-lineRUN <<EOF ... EOFblocks. This significantly reduces the number of Docker image layers, contributing to smaller image sizes and potentially faster builds due to improved caching. This includes steps for Python installation,uvinstallation (now integrated into a singleRUNcommand), GCC setup, PyTorch/Triton installation, FlashInfer setup, and development dependency installations. - CI Build Improvements: I've added the
ENV UV_NO_PROGRESS=1environment variable to disableuv's progress bar during builds, which is beneficial for cleaner and more efficient output in Continuous Integration (CI) environments.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
There was a problem hiding this comment.
Code Review
The code changes introduce heredocs to improve readability of the Dockerfile. Consider cleaning up apt caches and removing unnecessary packages within the same RUN layer to further reduce the image size. The rm -rf flashinfer command on line 409 will not work as intended because it's executed from within the flashinfer directory.
| cd flashinfer | ||
| python3 -m flashinfer.aot | ||
| uv pip install --system --no-build-isolation . | ||
| rm -rf flashinfer |
There was a problem hiding this comment.
The rm -rf flashinfer command on this line will not work as intended because it's executed from within the flashinfer directory. This will prevent the cloned repository from being cleaned up, increasing the final image size. Change back to the parent directory before removing it.
cd .. && rm -rf flashinfer
| curl -sS ${GET_PIP_URL} | python${PYTHON_VERSION} | ||
| python3 --version | ||
| python3 -m pip --version | ||
| pip install uv |
There was a problem hiding this comment.
Consider cleaning up the apt cache and removing unnecessary packages within the same RUN layer to further reduce the image size. Use --no-install-recommends with apt-get install to avoid installing optional dependencies1. Add apt-get autoremove -y && apt-get clean && rm -rf /var/lib/apt/lists/* to remove downloaded package files and the apt cache.
pip install uv
apt-get autoremove -y && apt-get clean && rm -rf /var/lib/apt/lists/*
Style Guide References
Footnotes
-
Use the
--no-install-recommendsflag to avoid installing optional dependencies. ↩
| RUN <<EOF | ||
| set -ex | ||
| apt-get install -y gcc-10 g++-10 | ||
| update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-10 110 --slave /usr/bin/g++ g++ /usr/bin/g++-10 | ||
| gcc --version |
There was a problem hiding this comment.
Consider cleaning up apt caches here to reduce the image size1. Add apt-get autoremove -y && apt-get clean && rm -rf /var/lib/apt/lists/* after the other commands. Also, consider adding --no-install-recommends to the apt-get install on line 126.
gcc --version
apt-get autoremove -y && apt-get clean && rm -rf /var/lib/apt/lists/*
Style Guide References
Footnotes
-
Clean up apt caches to reduce image size. ↩
| echo "Installing sccache..." | ||
| curl -L -o sccache.tar.gz ${SCCACHE_DOWNLOAD_URL} | ||
| tar -xzf sccache.tar.gz | ||
| sudo mv sccache-v0.8.1-x86_64-unknown-linux-musl/sccache /usr/bin/sccache |
There was a problem hiding this comment.
Since Docker containers typically run as the root user by default, using sudo here is unnecessary.
mv sccache-v0.8.1-x86_64-unknown-linux-musl/sccache /usr/bin/sccache
| ln -sf /usr/bin/python${PYTHON_VERSION}-config /usr/bin/python3-config | ||
| curl -sS ${GET_PIP_URL} | python${PYTHON_VERSION} | ||
| python3 --version | ||
| python3 -m pip --version |
There was a problem hiding this comment.
Consider cleaning up the apt cache and removing unnecessary packages within the same RUN layer to reduce the image size1. Use the --no-install-recommends flag with apt-get install to avoid installing optional dependencies (on lines 311, 312, and 326). Add apt-get autoremove -y && apt-get clean && rm -rf /var/lib/apt/lists/* to remove downloaded package files and the apt cache.
python3 --version && python3 -m pip --version
apt-get autoremove -y && apt-get clean && rm -rf /var/lib/apt/lists/*
Style Guide References
Footnotes
-
Clean up apt cache to reduce image size. ↩
|
How much smaller does this make the image? |
I'm refactoring for the |
|
ah let me move this to draft first |
This PR aims to reduce the size of the container images, while trying to use some best practices in container.
Signed-off-by: Aaron Pham [email protected]