diff --git a/backends/python/server/requirements-intel.txt b/backends/python/server/requirements-intel.txt index 36b330db..5accea69 100644 --- a/backends/python/server/requirements-intel.txt +++ b/backends/python/server/requirements-intel.txt @@ -42,3 +42,4 @@ win32-setctime==1.1.0 ; python_version >= "3.9" and python_version < "3.13" and wrapt==1.15.0 ; python_version >= "3.9" and python_version < "3.13" transformers==4.40.0 ; python_version >= "3.9" and python_version < "3.13" pyrsistent==0.20.0 ; python_version >= "3.9" and python_version < "3.13" +einops==0.8.0 ; python_version >= "3.9" and python_version < "3.13" \ No newline at end of file diff --git a/docs/source/en/_toctree.yml b/docs/source/en/_toctree.yml index 211d1ca5..68f40ada 100644 --- a/docs/source/en/_toctree.yml +++ b/docs/source/en/_toctree.yml @@ -19,6 +19,8 @@ # title: Using TEI CLI - local: custom_container title: Build custom container for TEI + - local: intel_container + title: Using TEI container with Intel Hardware - local: examples title: Example uses title: Tutorials diff --git a/docs/source/en/intel_container.md b/docs/source/en/intel_container.md new file mode 100644 index 00000000..f260fb4e --- /dev/null +++ b/docs/source/en/intel_container.md @@ -0,0 +1,110 @@ + + +# Using TEI Container with Intel® Hardware + +This guide explains how to build and deploy `text-embeddings-inference` containers optimized for Intel® hardware, including CPUs, XPUs, and HPUs. + +## CPU + +### Build Docker Image + +To build a container optimized for Intel® CPUs, run the following command: + +```shell +platform="cpu" + +docker build . -f Dockerfile-intel --build-arg PLATFORM=$platform -t tei_cpu_ipex +``` + +### Deploy Docker Container + +To deploy your model on an Intel® CPU, use the following command: + +```shell +model='BAAI/bge-large-en-v1.5' +volume=$PWD/data + +docker run -p 8080:80 -v $volume:/data tei_cpu_ipex --model-id $model +``` + +## XPU + +### Build Docker Image + +To build a container optimized for Intel® XPUs, run the following command: + +```shell +platform="xpu" + +docker build . -f Dockerfile-intel --build-arg PLATFORM=$platform -t tei_xpu_ipex +``` + +### Deploy Docker Container + +To deploy your model on an Intel® XPU, use the following command: + +```shell +model='BAAI/bge-large-en-v1.5' +volume=$PWD/data + +docker run -p 8080:80 -v $volume:/data --device=/dev/dri -v /dev/dri/by-path:/dev/dri/by-path tei_xpu_ipex --model-id $model --dtype float16 +``` + +## HPU + +### Build Docker Image + +To build a container optimized for Intel® HPUs (Gaudi), run the following command: + +```shell +platform="hpu" + +docker build . -f Dockerfile-intel --build-arg PLATFORM=$platform -t tei_hpu +``` + +### Deploy Docker Container + +To deploy your model on an Intel® HPU (Gaudi), use the following command: + +```shell +model='BAAI/bge-large-en-v1.5' +volume=$PWD/data + +docker run -p 8080:80 -v $volume:/data --runtime=habana -e HABANA_VISIBLE_DEVICES=all -e MAX_WARMUP_SEQUENCE_LENGTH=512 tei_hpu --model-id $model --dtype bfloat16 +``` + +## Prebuilt Docker Images + +For convenience, prebuilt Docker images are available on GitHub Container Registry (GHCR). You can pull these images directly without the need to build them manually: + +### CPU +To use the prebuilt image optimized for Intel® CPUs, run: +```shell +docker pull ghcr.io/huggingface/text-embeddings-inference:cpu-ipex-latest +``` + +### XPU +To use the prebuilt image optimized for Intel® XPUs, run: +```shell +docker pull ghcr.io/huggingface/text-embeddings-inference:xpu-ipex-latest +``` + +### HPU +To use the prebuilt image optimized for Intel® HPUs (Gaudi), run: +```shell +docker pull ghcr.io/huggingface/text-embeddings-inference:hpu-latest +``` \ No newline at end of file