diff --git a/FinanceAgent/README.md b/FinanceAgent/README.md index 640f7113d0..363c6ef193 100644 --- a/FinanceAgent/README.md +++ b/FinanceAgent/README.md @@ -3,7 +3,6 @@ ## Table of Contents - [Overview](#overview) -- [Problem Motivation](#problem-motivation) - [Architecture](#architecture) - [High-Level Diagram](#high-level-diagram) - [OPEA Microservices Diagram for Data Handling](#opea-microservices-diagram-for-data-handling) @@ -56,9 +55,10 @@ The Question Answering worker agent uses `search_knowledge_base` tool to get rel This Finance Agent example can be deployed manually on Docker Compose. -| Hardware | Deployment Mode | Guide Link | -| :----------------------------- | :------------------- | :----------------------------------------------------------------------- | -| Intel® Gaudi® AI Accelerator | Single Node (Docker) | [Gaudi Docker Compose Guide](./docker_compose/intel/hpu/gaudi/README.md) | +| Hardware | Deployment Mode | Guide Link | +| :--------------------------------- | :------------------------------------------------------------------------------------------ | :----------------------------------------------------------------------- | +| Intel® Gaudi® AI Accelerator | Single Node (Docker) | [Gaudi Docker Compose Guide](./docker_compose/intel/hpu/gaudi/README.md) | +| Intel® Xeon® Scalable processors | Single Node (Docker) [Xeon Docker Compose Guide](./docker_compose/intel/cpu/xeon/README.md) | _Note: Building custom microservice images can be done using the resources in [GenAIComps](https://github.com/opea-project/GenAIComps)._ diff --git a/FinanceAgent/docker_compose/intel/cpu/xeon/README.md b/FinanceAgent/docker_compose/intel/cpu/xeon/README.md new file mode 100644 index 0000000000..33115a7b04 --- /dev/null +++ b/FinanceAgent/docker_compose/intel/cpu/xeon/README.md @@ -0,0 +1,202 @@ +# Deploy Finance Agent on Intel® Xeon® Scalable processors with Docker Compose + +This README provides instructions for deploying the Finance Agent application using Docker Compose on systems equipped with Intel® Xeon® Scalable processors. + +## Table of Contents + +- [Overview](#overview) +- [Prerequisites](#prerequisites) +- [Start Deployment](#start-deployment) +- [Validate Services](#validate-services) +- [Accessing the User Interface (UI)](#accessing-the-user-interface-ui) + +## Overview + +This guide focuses on running the pre-configured Finance Agent service using Docker Compose on Intel® Xeon® Scalable processors. It runs with OpenAI LLM models, along with containers for other microservices like embedding, retrieval, data preparation and the UI. + +## Prerequisites + +- Docker and Docker Compose installed. +- Intel® Xeon® Scalable processors on-prem or from the cloud. +- If running OpenAI models, generate the API key by following these [instructions](https://platform.openai.com/api-keys). If using a remote server i.e. for LLM text generation, have the base URL and API key ready from the cloud service provider or owner of the on-prem machine. +- Git installed (for cloning repository). +- Hugging Face Hub API Token (for downloading models). +- Access to the internet (or a private model cache). +- Finnhub API Key. Go to https://docs.financialdatasets.ai/ to get your free api key. +- Financial Datasets API Key. Go to https://docs.financialdatasets.ai/ to get your free api key. + +Clone the GenAIExamples repository: + +```bash +mkdir /path/to/your/workspace/ +export WORKDIR=/path/to/your/workspace/ +cd $WORKDIR +git clone https://github.com/opea-project/GenAIExamples.git +cd GenAIExamples/FinanceAgent/docker_compose/intel/cpu/xeon +``` + +## Start Deployment + +By default, it will run models from OpenAI. + +### Configure Environment + +Set required environment variables in your shell: + +```bash +# Path to your model cache +export HF_CACHE_DIR="./data" +# Some models from Hugging Face require approval beforehand. Ensure you have the necessary permissions to access them. +export HF_TOKEN="your_huggingface_token" +export OPENAI_API_KEY="your-openai-api-key" +export FINNHUB_API_KEY="your-finnhub-api-key" +export FINANCIAL_DATASETS_API_KEY="your-financial-datasets-api-key" + +# Configure HOST_IP +# Replace with your host's external IP address (do not use localhost or 127.0.0.1). +export HOST_IP=$(hostname -I | awk '{print $1}') + +# Optional: Configure proxy if needed +# export HTTP_PROXY="${http_proxy}" +# export HTTPS_PROXY="${https_proxy}" +# export NO_PROXY="${NO_PROXY},${HOST_IP}" + +source set_env.sh +``` + +Note: The compose file might read additional variables from `set_env.sh`. Ensure all required variables like ports (LLM_SERVICE_PORT, TEI_EMBEDDER_PORT, etc.) are set if not using defaults from the compose file. For instance, edit the `set_env.sh` or overwrite LLM_MODEL_ID to change the LLM model. + +### [Optional] Running Models on a Remote Server + +To run models on a remote server i.e. deployed using Intel® AI for Enterprise Inference, a base URL and an API key are required to access them. To run the Agent microservice on Xeon while using models deployed on a remote server, set additional environment variables shown below. + +```bash +# Overwrite this environment variable previously set in set_env.sh with a new value for the desired model. The default value is gpt-4o-mini-2024-07-18. +export OPENAI_LLM_MODEL_ID= + +# The base URL given from the owner of the on-prem machine or cloud service provider. It will follow this format: "https://". Here is an example: "https://api.inference.example.com". +export REMOTE_ENDPOINT= +``` + +### Start Services + +The following services will be launched: + +- tei-embedding-serving +- redis-vector-db +- redis-kv-store +- dataprep-redis-server-finance +- finqa-agent-endpoint +- research-agent-endpoint +- docsum-vllm-xeon +- supervisor-agent-endpoint +- agent-ui + +Follow **ONE** option below to deploy these services. + +#### Option 1: Deploy with Docker Compose for OpenAI Models + +```bash +docker compose -f compose_openai.yaml up -d +``` + +#### [Optional] Option 2: Deploy with Docker Compose for Models on a Remote Server + +```bash +docker compose -f compose_openai.yaml -f compose_remote.yaml up -d +``` + +#### [Optional] Build docker images + +This is only needed if the Docker image is unavailable or the pull operation fails. + +```bash +cd $WORKDIR/GenAIExamples/FinanceAgent/docker_image_build +# get GenAIComps repo +git clone https://github.com/opea-project/GenAIComps.git +# build the images +docker compose -f build.yaml build --no-cache +``` + +## Validate Services + +Wait several minutes for models to download and services to initialize. Check container logs with this command: + +```bash +docker compose logs -f . +``` + +### Validate Data Services + +Ingest data and retrieval from database + +```bash +python3 $WORKDIR/GenAIExamples/FinanceAgent/tests/test_redis_finance.py --port 6007 --test_option ingest +python3 $WORKDIR/GenAIExamples/FinanceAgent/tests/test_redis_finance.py --port 6007 --test_option get +``` + +### Validate Agents + +FinQA Agent: + +```bash +export agent_port="9095" +prompt="What is Gap's revenue in 2024?" +python3 $WORKDIR/GenAIExamples/FinanceAgent/tests/test.py --prompt "$prompt" --agent_role "worker" --ext_port $agent_port +``` + +Research Agent: + +```bash +export agent_port="9096" +prompt="generate NVDA financial research report" +python3 $WORKDIR/GenAIExamples/FinanceAgent/tests/test.py --prompt "$prompt" --agent_role "worker" --ext_port $agent_port --tool_choice "get_current_date" --tool_choice "get_share_performance" +``` + +Supervisor Agent single turns: + +```bash +export agent_port="9090" +python3 $WORKDIR/GenAIExamples/FinanceAgent/tests/test.py --agent_role "supervisor" --ext_port $agent_port --stream +``` + +Supervisor Agent multi turn: + +```bash +python3 $WORKDIR/GenAIExamples/FinanceAgent/tests/test.py --agent_role "supervisor" --ext_port $agent_port --multi-turn --stream +``` + +## Accessing the User Interface (UI) + +The UI microservice is launched in the previous step with the other microservices. +To see the UI, open a web browser to `http://${HOST_IP}:5175` to access the UI. Note the `HOST_IP` here is the host IP of the UI microservice. + +1. Create Admin Account with a random value + +2. Enter the endpoints in the `Connections` settings + + First, click on the user icon in the upper right corner to open `Settings`. Click on `Admin Settings`. Click on `Connections`. + + Then, enter the supervisor agent endpoint in the `OpenAI API` section: `http://${HOST_IP}:9090/v1`. Enter the API key as "empty". Add an arbitrary model id in `Model IDs`, for example, "opea_agent". The `HOST_IP` here should be the host ip of the agent microservice. + + Then, enter the dataprep endpoint in the `Icloud File API` section. You first need to enable `Icloud File API` by clicking on the button on the right to turn it into green and then enter the endpoint url, for example, `http://${HOST_IP}:6007/v1`. The `HOST_IP` here should be the host ip of the dataprep microservice. + + You should see screen like the screenshot below when the settings are done. + +![opea-agent-setting](../../../../assets/ui_connections_settings.png) + +3. Upload documents with UI + + Click on the `Workplace` icon in the top left corner. Click `Knowledge`. Click on the "+" sign to the right of `iCloud Knowledge`. You can paste an url in the left hand side of the pop-up window, or upload a local file by click on the cloud icon on the right hand side of the pop-up window. Then click on the `Upload Confirm` button. Wait till the processing is done and the pop-up window will be closed on its own when the data ingestion is done. See the screenshot below. + Then, enter the dataprep endpoint in the `iCloud File API` section. You first need to enable `iCloud File API` by clicking on the button on the right to turn it into green and then enter the endpoint url, for example, `http://${HOST_IP}:6007/v1`. The `HOST_IP` here should be the host ip of the dataprep microservice. + Note: the data ingestion may take a few minutes depending on the length of the document. Please wait patiently and do not close the pop-up window. + +![upload-doc-ui](../../../../assets/upload_doc_ui.png) + +4. Test agent with UI + + After the settings are done and documents are ingested, you can start to ask questions to the agent. Click on the `New Chat` icon in the top left corner, and type in your questions in the text box in the middle of the UI. + + The UI will stream the agent's response tokens. You need to expand the `Thinking` tab to see the agent's reasoning process. After the agent made tool calls, you would also see the tool output after the tool returns output to the agent. Note: it may take a while to get the tool output back if the tool execution takes time. + +![opea-agent-test](../../../../assets/opea-agent-test.png) diff --git a/FinanceAgent/docker_compose/intel/cpu/xeon/compose_openai.yaml b/FinanceAgent/docker_compose/intel/cpu/xeon/compose_openai.yaml new file mode 100644 index 0000000000..566f8d6b0f --- /dev/null +++ b/FinanceAgent/docker_compose/intel/cpu/xeon/compose_openai.yaml @@ -0,0 +1,188 @@ +# Copyright (C) 2025 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 + + +x-common-environment: + &common-env + no_proxy: ${NO_PROXY} + http_proxy: ${HTTP_PROXY} + https_proxy: ${HTTPS_PROXY} + +x-common-agent-environment: + &common-agent-env + <<: *common-env + llm_engine: openai + model: ${OPENAI_LLM_MODEL_ID} + OPENAI_API_KEY: ${OPENAI_API_KEY} + REDIS_URL_VECTOR: ${REDIS_URL_VECTOR} + REDIS_URL_KV: ${REDIS_URL_KV} + TEI_EMBEDDING_ENDPOINT: ${TEI_EMBEDDING_ENDPOINT} + ip_address: ${HOST_IP} + strategy: react_llama + require_human_feedback: false + +services: + + tei-embedding-serving: + image: ghcr.io/huggingface/text-embeddings-inference:cpu-1.5 + container_name: tei-embedding-serving + entrypoint: /bin/sh -c "apt-get update && apt-get install -y curl && text-embeddings-router --json-output --model-id ${EMBEDDING_MODEL_ID} --auto-truncate" + ports: + - "${TEI_EMBEDDER_PORT:-10221}:80" + volumes: + - ${HF_CACHE_DIR:-./data}:/data + shm_size: 1g + environment: + <<: *common-env + HF_TOKEN: ${HF_TOKEN} + host_ip: ${HOST_IP} + healthcheck: + test: ["CMD", "curl", "-f", "http://${HOST_IP}:${TEI_EMBEDDER_PORT}/health"] + interval: 10s + timeout: 6s + retries: 48 + + redis-vector-db: + image: redis/redis-stack:7.2.0-v9 + container_name: redis-vector-db + ports: + - "${REDIS_PORT1:-6379}:6379" + - "${REDIS_PORT2:-8001}:8001" + environment: + <<: *common-env + healthcheck: + test: ["CMD", "redis-cli", "ping"] + timeout: 10s + retries: 3 + start_period: 10s + + redis-kv-store: + image: redis/redis-stack:7.2.0-v9 + container_name: redis-kv-store + ports: + - "${REDIS_PORT3:-6380}:6379" + - "${REDIS_PORT4:-8002}:8001" + environment: + <<: *common-env + healthcheck: + test: ["CMD", "redis-cli", "ping"] + timeout: 10s + retries: 3 + start_period: 10s + + dataprep-redis-finance: + image: ${REGISTRY:-opea}/dataprep:${TAG:-latest} + container_name: dataprep-redis-server-finance + depends_on: + redis-vector-db: + condition: service_healthy + redis-kv-store: + condition: service_healthy + tei-embedding-serving: + condition: service_healthy + ports: + - "${DATAPREP_PORT:-6007}:5000" + environment: + <<: *common-env + DATAPREP_COMPONENT_NAME: ${DATAPREP_COMPONENT_NAME} + REDIS_URL_VECTOR: ${REDIS_URL_VECTOR} + REDIS_URL_KV: ${REDIS_URL_KV} + TEI_EMBEDDING_ENDPOINT: ${TEI_EMBEDDING_ENDPOINT} + LLM_ENDPOINT: ${LLM_ENDPOINT} + LLM_MODEL: ${LLM_MODEL_ID} + HUGGINGFACEHUB_API_TOKEN: ${HF_TOKEN} + HF_TOKEN: ${HF_TOKEN} + LOGFLAG: true + + worker-finqa-agent: + image: opea/agent:latest + container_name: finqa-agent-endpoint + volumes: + - ${TOOLSET_PATH}:/home/user/tools/ + - ${PROMPT_PATH}:/home/user/prompts/ + ipc: host + ports: + - "9095:9095" + environment: + <<: *common-agent-env + with_memory: false + recursion_limit: ${RECURSION_LIMIT_WORKER} + temperature: ${TEMPERATURE} + max_new_tokens: ${MAX_TOKENS} + stream: false + tools: /home/user/tools/finqa_agent_tools.yaml + custom_prompt: /home/user/prompts/finqa_prompt.py + port: 9095 + + worker-research-agent: + image: opea/agent:latest + container_name: research-agent-endpoint + volumes: + - ${TOOLSET_PATH}:/home/user/tools/ + - ${PROMPT_PATH}:/home/user/prompts/ + ipc: host + ports: + - "9096:9096" + environment: + <<: *common-agent-env + with_memory: false + recursion_limit: ${RECURSION_LIMIT_WORKER} + stream: false + tools: /home/user/tools/research_agent_tools.yaml + custom_prompt: /home/user/prompts/research_prompt.py + FINNHUB_API_KEY: ${FINNHUB_API_KEY} + FINANCIAL_DATASETS_API_KEY: ${FINANCIAL_DATASETS_API_KEY} + port: 9096 + + docsum-vllm-xeon: + image: opea/llm-docsum:latest + container_name: docsum-vllm-xeon + ports: + - ${DOCSUM_PORT:-9000}:9000 + ipc: host + environment: + <<: *common-env + LLM_ENDPOINT: ${LLM_ENDPOINT} + LLM_MODEL_ID: ${LLM_MODEL_ID} + HF_TOKEN: ${HF_TOKEN} + LOGFLAG: ${LOGFLAG:-False} + MAX_INPUT_TOKENS: ${MAX_INPUT_TOKENS} + MAX_TOTAL_TOKENS: ${MAX_TOTAL_TOKENS} + DocSum_COMPONENT_NAME: ${DOCSUM_COMPONENT_NAME:-OpeaDocSumvLLM} + restart: unless-stopped + + supervisor-react-agent: + image: opea/agent:latest + container_name: supervisor-agent-endpoint + volumes: + - ${TOOLSET_PATH}:/home/user/tools/ + - ${PROMPT_PATH}:/home/user/prompts/ + ipc: host + depends_on: + - worker-finqa-agent + - worker-research-agent + ports: + - "9090:9090" + environment: + <<: *common-agent-env + with_memory: "true" + recursion_limit: ${RECURSION_LIMIT_SUPERVISOR} + temperature: ${TEMPERATURE} + max_new_tokens: ${MAX_TOKENS} + stream: "true" + tools: /home/user/tools/supervisor_agent_tools.yaml + custom_prompt: /home/user/prompts/supervisor_prompt.py + WORKER_FINQA_AGENT_URL: ${WORKER_FINQA_AGENT_URL} + WORKER_RESEARCH_AGENT_URL: ${WORKER_RESEARCH_AGENT_URL} + DOCSUM_ENDPOINT: ${DOCSUM_ENDPOINT} + port: 9090 + + agent-ui: + image: opea/agent-ui:latest + container_name: agent-ui + environment: + <<: *common-env + host_ip: ${HOST_IP} + ports: + - "5175:8080" + ipc: host diff --git a/FinanceAgent/docker_compose/intel/cpu/xeon/compose_remote.yaml b/FinanceAgent/docker_compose/intel/cpu/xeon/compose_remote.yaml new file mode 100644 index 0000000000..283c21c604 --- /dev/null +++ b/FinanceAgent/docker_compose/intel/cpu/xeon/compose_remote.yaml @@ -0,0 +1,18 @@ +# Copyright (C) 2025 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 + +services: + worker-finqa-agent: + environment: + llm_endpoint_url: ${REMOTE_ENDPOINT} + api_key: ${OPENAI_API_KEY} + + worker-research-agent: + environment: + llm_endpoint_url: ${REMOTE_ENDPOINT} + api_key: ${OPENAI_API_KEY} + + supervisor-react-agent: + environment: + llm_endpoint_url: ${REMOTE_ENDPOINT} + api_key: ${OPENAI_API_KEY} diff --git a/FinanceAgent/docker_compose/intel/cpu/xeon/set_env.sh b/FinanceAgent/docker_compose/intel/cpu/xeon/set_env.sh new file mode 100644 index 0000000000..10a6ee67c7 --- /dev/null +++ b/FinanceAgent/docker_compose/intel/cpu/xeon/set_env.sh @@ -0,0 +1,86 @@ +#!/usr/bin/env bash + +# Copyright (C) 2025 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 + +# Navigate to the parent directory and source the environment +pushd "../../../../../" > /dev/null +source .set_env.sh +popd > /dev/null + +# Function to check if a variable is set +check_var() { + local var_name="$1" + local var_value="${!var_name}" + if [ -z "${var_value}" ]; then + echo "Error: ${var_name} is not set. Please set ${var_name}." + return 1 # Return an error code but do not exit the script + fi +} + +# Check critical variables +check_var "HF_TOKEN" +check_var "OPENAI_API_KEY" +check_var "FINNHUB_API_KEY" +check_var "FINANCIAL_DATASETS_API_KEY" +check_var "HOST_IP" + +# OpenAI and agent configuration +export OPENAI_API_KEY="${OPENAI_API_KEY}" +export OPENAI_LLM_MODEL_ID="${OPENAI_LLM_MODEL_ID:-gpt-4o-mini-2024-07-18}" +export RECURSION_LIMIT_WORKER="${RECURSION_LIMIT_WORKER:-12}" +export RECURSION_LIMIT_SUPERVISOR="${RECURSION_LIMIT_SUPERVISOR:-10}" +export TEMPERATURE="${TEMPERATURE:-0.5}" +export MAX_TOKENS="${MAX_TOKENS:-4096}" + +# Hugging Face API token +export HUGGINGFACEHUB_API_TOKEN="${HF_TOKEN}" + +# Dataprep and Docsum LLM configuration +export HF_CACHE_DIR="${HF_CACHE_DIR:-"./data"}" +export LLM_PORT="${LLM_PORT:-8086}" +export LLM_ENDPOINT="http://${HOST_IP}:${LLM_PORT}" +export LLM_MODEL_ID="${LLM_MODEL_ID:-meta-llama/Llama-3.1-8B-Instruct}" +export MAX_INPUT_TOKENS="${MAX_INPUT_TOKENS:-2048}" +export MAX_TOTAL_TOKENS="${MAX_TOTAL_TOKENS:-4096}" + +# Data preparation and embedding configuration +export DATAPREP_PORT="${DATAPREP_PORT:-6007}" +export TEI_EMBEDDER_PORT="${TEI_EMBEDDER_PORT:-10221}" +export REDIS_URL_VECTOR="redis://${HOST_IP}:6379" +export REDIS_URL_KV="redis://${HOST_IP}:6380" +export DATAPREP_COMPONENT_NAME="${DATAPREP_COMPONENT_NAME:-OPEA_DATAPREP_REDIS_FINANCE}" +export EMBEDDING_MODEL_ID="${EMBEDDING_MODEL_ID:-BAAI/bge-base-en-v1.5}" +export TEI_EMBEDDING_ENDPOINT="http://${HOST_IP}:${TEI_EMBEDDER_PORT}" + +# Worker URLs +export WORKER_FINQA_AGENT_URL="http://${HOST_IP}:9095/v1/chat/completions" +export WORKER_RESEARCH_AGENT_URL="http://${HOST_IP}:9096/v1/chat/completions" + +# DocSum configuration +export DOCSUM_COMPONENT_NAME="${DOCSUM_COMPONENT_NAME:-"OpeaDocSumvLLM"}" +export DOCSUM_ENDPOINT="http://${HOST_IP}:9000/v1/docsum" + +# API keys +export FINNHUB_API_KEY="${FINNHUB_API_KEY}" +export FINANCIAL_DATASETS_API_KEY="${FINANCIAL_DATASETS_API_KEY}" + + +# Toolset and prompt paths +if check_var "WORKDIR"; then + export TOOLSET_PATH=$WORKDIR/GenAIExamples/FinanceAgent/tools/ + export PROMPT_PATH=$WORKDIR/GenAIExamples/FinanceAgent/prompts/ + + echo "TOOLSET_PATH=${TOOLSET_PATH}" + echo "PROMPT_PATH=${PROMPT_PATH}" + + # Array of directories to check + REQUIRED_DIRS=("${TOOLSET_PATH}" "${PROMPT_PATH}") + + for dir in "${REQUIRED_DIRS[@]}"; do + if [ ! -d "${dir}" ]; then + echo "Error: Required directory does not exist: ${dir}" + exit 1 + fi + done +fi diff --git a/FinanceAgent/docker_compose/intel/hpu/gaudi/README.md b/FinanceAgent/docker_compose/intel/hpu/gaudi/README.md index 79f0a9dec9..26afa92d0c 100644 --- a/FinanceAgent/docker_compose/intel/hpu/gaudi/README.md +++ b/FinanceAgent/docker_compose/intel/hpu/gaudi/README.md @@ -21,8 +21,8 @@ This guide focuses on running the pre-configured Finance Agent service using Doc - Git installed (for cloning repository). - Hugging Face Hub API Token (for downloading models). - Access to the internet (or a private model cache). -- Finnhub API Key. Go to https://docs.financialdatasets.ai/ to get your free api key -- Financial Datgasets API Key. Go to https://docs.financialdatasets.ai/ to get your free api key +- Finnhub API Key. Go to https://docs.financialdatasets.ai/ to get your free api key. +- Financial Datasets API Key. Go to https://docs.financialdatasets.ai/ to get your free api key. Clone the GenAIExamples repository: @@ -48,21 +48,21 @@ export HF_CACHE_DIR="./data" # Some models from Hugging Face require approval beforehand. Ensure you have the necessary permissions to access them. export HF_TOKEN="your_huggingface_token" export FINNHUB_API_KEY="your-finnhub-api-key" -export FINANCIAL_DATASETS_API_KEY="your-financial-datgasets-api-key" +export FINANCIAL_DATASETS_API_KEY="your-financial-datasets-api-key" -# Optional: Configure HOST_IP if needed +# Configure HOST_IP # Replace with your host's external IP address (do not use localhost or 127.0.0.1). -# export HOST_IP=$(hostname -I | awk '{print $1}') +export HOST_IP=$(hostname -I | awk '{print $1}') # Optional: Configure proxy if needed # export HTTP_PROXY="${http_proxy}" # export HTTPS_PROXY="${https_proxy}" # export NO_PROXY="${NO_PROXY},${HOST_IP}" -source ../../set_env.sh +source set_env.sh ``` -Note: The compose file might read additional variables from set_env.sh. Ensure all required variables like ports (LLM_SERVICE_PORT, TEI_EMBEDDER_PORT, etc.) are set if not using defaults from the compose file. For instance, edit the set_env.sh to change the LLM model: +Note: The compose file might read additional variables from `set_env.sh`. Ensure all required variables like ports (LLM_SERVICE_PORT, TEI_EMBEDDER_PORT, etc.) are set if not using defaults from the compose file. For instance, edit the `set_env.sh` or overwrite LLM_MODEL_ID to change the LLM model. ### Start Services @@ -70,14 +70,13 @@ Note: The compose file might read additional variables from set_env.sh. Ensure a Below is the command to launch services -- vllm-gaudi-server - tei-embedding-serving - redis-vector-db - redis-kv-store - dataprep-redis-server-finance - finqa-agent-endpoint - research-agent-endpoint -- docsum-vllm-gaudi +- docsum-vllm-xeon - supervisor-agent-endpoint - agent-ui diff --git a/FinanceAgent/docker_compose/intel/set_env.sh b/FinanceAgent/docker_compose/intel/hpu/gaudi/set_env.sh similarity index 97% rename from FinanceAgent/docker_compose/intel/set_env.sh rename to FinanceAgent/docker_compose/intel/hpu/gaudi/set_env.sh index c8a36fabb0..0024c525f3 100644 --- a/FinanceAgent/docker_compose/intel/set_env.sh +++ b/FinanceAgent/docker_compose/intel/hpu/gaudi/set_env.sh @@ -1,10 +1,10 @@ #!/usr/bin/env bash -# Copyright (C) 2024 Intel Corporation +# Copyright (C) 2025 Intel Corporation # SPDX-License-Identifier: Apache-2.0 # Navigate to the parent directory and source the environment -pushd "../../" > /dev/null +pushd "../../../../../" > /dev/null source .set_env.sh popd > /dev/null diff --git a/FinanceAgent/tests/_test_compose_openai_on_xeon.sh b/FinanceAgent/tests/_test_compose_openai_on_xeon.sh new file mode 100644 index 0000000000..af92ecc84f --- /dev/null +++ b/FinanceAgent/tests/_test_compose_openai_on_xeon.sh @@ -0,0 +1,110 @@ +#!/bin/bash +# Copyright (C) 2025 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 +set -xe + +export WORKPATH=$(dirname "$PWD") +export WORKDIR=$WORKPATH/../../ +echo "WORKDIR=${WORKDIR}" +export IP_ADDRESS=$(hostname -I | awk '{print $1}') +export HOST_IP=${IP_ADDRESS} +LOG_PATH=$WORKPATH + +# Proxy settings +export NO_PROXY="${NO_PROXY},${HOST_IP}" +export HTTP_PROXY="${http_proxy}" +export HTTPS_PROXY="${https_proxy}" + +export no_proxy="${no_proxy},${HOST_IP}" +export http_proxy="${http_proxy}" +export https_proxy="${https_proxy}" + +# OPENAI and agent configuration +export OPENAI_API_KEY="${OPENAI_API_KEY}" +export OPENAI_LLM_MODEL_ID="${OPENAI_LLM_MODEL_ID:-gpt-4o-mini-2024-07-18}" +export RECURSION_LIMIT_WORKER="${RECURSION_LIMIT_WORKER:-12}" +export RECURSION_LIMIT_SUPERVISOR="${RECURSION_LIMIT_SUPERVISOR:-10}" +export TEMPERATURE="${TEMPERATURE:-0.5}" +export MAX_TOKENS="${MAX_TOKENS:-4096}" + +# Hugging Face API token +export HUGGINGFACEHUB_API_TOKEN="${HF_TOKEN}" + +# Dataprep and Docsum LLM configuration +export HF_CACHE_DIR="${HF_CACHE_DIR:-"./data"}" +export LLM_PORT="${LLLM_PORT:-8086}" +export LLM_ENDPOINT="http://${IP_ADDRESS}:${LLM_PORT}" +export LLM_MODEL_ID="${LLM_MODEL_ID:-meta-llama/Llama-3.1-8B-Instruct}" +export MAX_INPUT_TOKENS="${MAX_INPUT_TOKENS:-2048}" +export MAX_TOTAL_TOKENS="${MAX_TOTAL_TOKENS:-4096}" + +# Worker URLs +export WORKER_FINQA_AGENT_URL="http://${IP_ADDRESS}:9095/v1/chat/completions" +export WORKER_RESEARCH_AGENT_URL="http://${IP_ADDRESS}:9096/v1/chat/completions" + +# DocSum configuration +export DOCSUM_COMPONENT_NAME="${DOCSUM_COMPONENT_NAME:-"OpeaDocSumvLLM"}" +export DOCSUM_ENDPOINT="http://${IP_ADDRESS}:9000/v1/docsum" + +# Toolset and prompt paths +export TOOLSET_PATH=$WORKDIR/GenAIExamples/FinanceAgent/tools/ +export PROMPT_PATH=$WORKDIR/GenAIExamples/FinanceAgent/prompts/ + +#### env vars for dataprep ############# +export DATAPREP_PORT="6007" +export TEI_EMBEDDER_PORT="10221" +export REDIS_URL_VECTOR="redis://${IP_ADDRESS}:6379" +export REDIS_URL_KV="redis://${IP_ADDRESS}:6380" + +export DATAPREP_COMPONENT_NAME="OPEA_DATAPREP_REDIS_FINANCE" +export EMBEDDING_MODEL_ID="BAAI/bge-base-en-v1.5" +export TEI_EMBEDDING_ENDPOINT="http://${IP_ADDRESS}:${TEI_EMBEDDER_PORT}" +####################################### + +function stop_dataprep() { + echo "Stopping databases" + cid=$(docker ps -aq --filter "name=dataprep-redis-server*" --filter "name=redis-*" --filter "name=tei-embedding-*") + if [[ ! -z "$cid" ]]; then docker stop $cid && docker rm $cid && sleep 1s; fi +} + +function stop_agent_docker() { + cd $WORKPATH/docker_compose/intel/cpu/xeon/ + container_list=$(cat compose_openai.yaml | grep container_name | cut -d':' -f2) + for container_name in $container_list; do + cid=$(docker ps -aq --filter "name=$container_name") + echo "Stopping container $container_name" + if [[ ! -z "$cid" ]]; then docker rm $cid -f && sleep 1s; fi + done +} + +echo "workpath: $WORKPATH" +echo "=================== Stop containers ====================" +stop_agent_docker +stop_dataprep + +cd $WORKPATH/tests + +echo "=================== #1 Building docker images ====================" +bash step1_build_images.sh xeon +echo "=================== #1 Building docker images completed ====================" + +echo "=================== #2 Start services ====================" +bash step2_start_services.sh xeon +echo "=================== #2 Endpoints for services started ====================" + +echo "=================== #3 Validate ingest_validate_dataprep ====================" +bash step3_validate_ingest_validate_dataprep.sh +echo "=================== #3 Data ingestion and validation completed ====================" + +echo "=================== #4 Start agents ====================" +bash step4_validate_agent_service.sh +echo "=================== #4 Agent test passed ====================" + +echo "=================== #5 Stop microservices ====================" +stop_agent_docker +stop_dataprep +echo "=================== #5 Microservices stopped ===================" + +echo y | docker system prune + +echo "ALL DONE!!" diff --git a/FinanceAgent/tests/step1_build_images.sh b/FinanceAgent/tests/step1_build_images.sh new file mode 100644 index 0000000000..b148e2b38f --- /dev/null +++ b/FinanceAgent/tests/step1_build_images.sh @@ -0,0 +1,74 @@ +#!/bin/bash +# Copyright (C) 2025 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 +set -xe + +export WORKPATH=$(dirname "$PWD") +export WORKDIR=$WORKPATH/../../ +echo "WORKDIR=${WORKDIR}" +export IP_ADDRESS=$(hostname -I | awk '{print $1}') +export HOST_IP=${IP_ADDRESS} +LOG_PATH=$WORKPATH + +function get_genai_comps() { + if [ ! -d "GenAIComps" ] ; then + git clone --depth 1 --branch ${opea_branch:-"main"} https://github.com/opea-project/GenAIComps.git + fi +} + +function build_dataprep_agent_images() { + cd $WORKDIR/GenAIExamples/FinanceAgent/docker_image_build/ + get_genai_comps + echo "Build agent image with --no-cache..." + docker compose -f build.yaml build --no-cache +} + +function build_agent_image_local(){ + cd $WORKDIR/GenAIComps/ + docker build -t opea/agent:latest -f comps/agent/src/Dockerfile . --build-arg https_proxy=$HTTPS_PROXY --build-arg http_proxy=$HTTP_PROXY +} + +function build_vllm_docker_image() { + echo "Building the vllm docker image" + cd $WORKPATH + echo $WORKPATH + if [ ! -d "./vllm-fork" ]; then + git clone https://github.com/HabanaAI/vllm-fork.git + fi + cd ./vllm-fork + + VLLM_FORK_VER=v0.6.6.post1+Gaudi-1.20.0 + git checkout ${VLLM_FORK_VER} &> /dev/null + docker build --no-cache -f Dockerfile.hpu -t $VLLM_IMAGE --shm-size=128g . --build-arg https_proxy=$HTTPS_PROXY --build-arg http_proxy=$HTTP_PROXY + if [ $? -ne 0 ]; then + echo "$VLLM_IMAGE failed" + exit 1 + else + echo "$VLLM_IMAGE successful" + fi +} + +function main() { + case $1 in + "gaudi_vllm") + echo "==================== Build agent docker image for Gaudi ====================" + echo "Build vLLM docker image if needed" + # build_vllm_docker_image + ;; + "xeon") + echo "==================== Build agent docker image for Xeon ====================" + echo "Nothing specific to build for Xeon" + ;; + *) + echo "Invalid argument" + exit 1 + ;; + esac + + build_dataprep_agent_images + + # ## for local test + # # build_agent_image_local +} + +main $1 diff --git a/FinanceAgent/tests/step2_start_services.sh b/FinanceAgent/tests/step2_start_services.sh new file mode 100644 index 0000000000..5cb26e7223 --- /dev/null +++ b/FinanceAgent/tests/step2_start_services.sh @@ -0,0 +1,62 @@ +#!/bin/bash +# Copyright (C) 2025 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 +set -xe + +export WORKPATH=$(dirname "$PWD") +export WORKDIR=$WORKPATH/../../ +echo "WORKDIR=${WORKDIR}" +export IP_ADDRESS=$(hostname -I | awk '{print $1}') +export HOST_IP=${IP_ADDRESS} +LOG_PATH=$WORKPATH + +function start_all_services() { + case $1 in + "gaudi_vllm") + echo "==================== Start all services for Gaudi ====================" + docker compose -f $WORKPATH/docker_compose/intel/hpu/gaudi/compose.yaml up -d + + until [[ "$n" -ge 200 ]] || [[ $ready == true ]]; do + docker logs vllm-gaudi-server &> ${LOG_PATH}/vllm-gaudi-service.log + n=$((n+1)) + if grep -q "Uvicorn running on" ${LOG_PATH}/vllm-gaudi-service.log; then + break + fi + if grep -q "No such container" ${LOG_PATH}/vllm-gaudi-service.log; then + echo "container vllm-gaudi-server not found" + exit 1 + fi + sleep 10s + done + ;; + "xeon") + echo "==================== Start all services for Xeon ====================" + docker compose -f $WORKPATH/docker_compose/intel/cpu/xeon/compose_openai.yaml up -d + until [[ "$n" -ge 200 ]] || [[ $ready == true ]]; do + docker logs docsum-vllm-xeon &> ${LOG_PATH}/docsum-vllm-xeon-service.log + n=$((n+1)) + if grep -q "Uvicorn running on" ${LOG_PATH}/docsum-vllm-xeon-service.log; then + break + fi + if grep -q "No such container" ${LOG_PATH}/docsum-vllm-xeon-service.log; then + echo "container docsum-vllm-xeon not found" + exit 1 + fi + sleep 10s + done + ;; + *) + echo "Invalid argument" + exit 1 + ;; + esac + + sleep 10s + echo "Service started successfully" +} + +function main() { + start_all_services $1 +} + +main $1 diff --git a/FinanceAgent/tests/step3_validate_ingest_validate_dataprep.sh b/FinanceAgent/tests/step3_validate_ingest_validate_dataprep.sh new file mode 100644 index 0000000000..d39d9cd60d --- /dev/null +++ b/FinanceAgent/tests/step3_validate_ingest_validate_dataprep.sh @@ -0,0 +1,56 @@ +#!/bin/bash +# Copyright (C) 2025 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 +set -xe + +export WORKPATH=$(dirname "$PWD") +export WORKDIR=$WORKPATH/../../ +echo "WORKDIR=${WORKDIR}" +export IP_ADDRESS=$(hostname -I | awk '{print $1}') +export HOST_IP=${IP_ADDRESS} +LOG_PATH=$WORKPATH + +function validate() { + local CONTENT="$1" + local EXPECTED_RESULT="$2" + local SERVICE_NAME="$3" + echo "EXPECTED_RESULT: $EXPECTED_RESULT" + echo "Content: $CONTENT" + if echo "$CONTENT" | grep -q "$EXPECTED_RESULT"; then + echo "[ $SERVICE_NAME ] Content is as expected: $CONTENT" + echo 0 + else + echo "[ $SERVICE_NAME ] Content does not match the expected result: $CONTENT" + echo 1 + fi +} + +function ingest_validate_dataprep() { + # test /v1/dataprep/ingest + echo "=========== Test ingest ===========" + local CONTENT=$(python3 $WORKPATH/tests/test_redis_finance.py --port $DATAPREP_PORT --test_option ingest) + local EXIT_CODE=$(validate "$CONTENT" "200" "dataprep-redis-finance") + echo "$EXIT_CODE" + local EXIT_CODE="${EXIT_CODE:0-1}" + if [ "$EXIT_CODE" == "1" ]; then + docker logs dataprep-redis-server-finance + exit 1 + fi + + # test /v1/dataprep/get + echo "=========== Test get ===========" + local CONTENT=$(python3 $WORKPATH/tests/test_redis_finance.py --port $DATAPREP_PORT --test_option get) + local EXIT_CODE=$(validate "$CONTENT" "Request successful" "dataprep-redis-finance") + echo "$EXIT_CODE" + local EXIT_CODE="${EXIT_CODE:0-1}" + if [ "$EXIT_CODE" == "1" ]; then + docker logs dataprep-redis-server-finance + exit 1 + fi +} + +function main() { + ingest_validate_dataprep +} + +main diff --git a/FinanceAgent/tests/step4_validate_agent_service.sh b/FinanceAgent/tests/step4_validate_agent_service.sh new file mode 100644 index 0000000000..dd5c3dd4fa --- /dev/null +++ b/FinanceAgent/tests/step4_validate_agent_service.sh @@ -0,0 +1,86 @@ +#!/bin/bash +# Copyright (C) 2025 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 +set -xe + +export WORKPATH=$(dirname "$PWD") +export WORKDIR=$WORKPATH/../../ +echo "WORKDIR=${WORKDIR}" +export IP_ADDRESS=$(hostname -I | awk '{print $1}') +export HOST_IP=${IP_ADDRESS} +LOG_PATH=$WORKPATH + +function validate() { + local CONTENT="$1" + local EXPECTED_RESULT="$2" + local SERVICE_NAME="$3" + echo "EXPECTED_RESULT: $EXPECTED_RESULT" + echo "Content: $CONTENT" + if echo "$CONTENT" | grep -q "$EXPECTED_RESULT"; then + echo "[ $SERVICE_NAME ] Content is as expected: $CONTENT" + echo 0 + else + echo "[ $SERVICE_NAME ] Content does not match the expected result: $CONTENT" + echo 1 + fi +} + +function validate_agent_service() { + # test worker finqa agent + echo "======================Testing worker finqa agent======================" + export agent_port="9095" + prompt="What is Gap's revenue in 2024?" + local CONTENT=$(python3 $WORKDIR/GenAIExamples/FinanceAgent/tests/test.py --prompt "$prompt" --agent_role "worker" --ext_port $agent_port) + echo $CONTENT + local EXIT_CODE=$(validate "$CONTENT" "15" "finqa-agent-endpoint") + echo $EXIT_CODE + local EXIT_CODE="${EXIT_CODE:0-1}" + if [ "$EXIT_CODE" == "1" ]; then + docker logs finqa-agent-endpoint + exit 1 + fi + + # test worker research agent + echo "======================Testing worker research agent======================" + export agent_port="9096" + prompt="Johnson & Johnson" + local CONTENT=$(python3 $WORKDIR/GenAIExamples/AgentQnA/tests/test.py --prompt "$prompt" --agent_role "worker" --ext_port $agent_port --tool_choice "get_current_date" --tool_choice "get_share_performance") + local EXIT_CODE=$(validate "$CONTENT" "Johnson" "research-agent-endpoint") + echo $CONTENT + echo $EXIT_CODE + local EXIT_CODE="${EXIT_CODE:0-1}" + if [ "$EXIT_CODE" == "1" ]; then + docker logs research-agent-endpoint + exit 1 + fi + + # test supervisor react agent + echo "======================Testing supervisor agent: single turns ======================" + export agent_port="9090" + local CONTENT=$(python3 $WORKDIR/GenAIExamples/FinanceAgent/tests/test.py --agent_role "supervisor" --ext_port $agent_port --stream) + echo $CONTENT + local EXIT_CODE=$(validate "$CONTENT" "test completed with success" "supervisor-agent-endpoint") + echo $EXIT_CODE + local EXIT_CODE="${EXIT_CODE:0-1}" + if [ "$EXIT_CODE" == "1" ]; then + docker logs supervisor-agent-endpoint + exit 1 + fi + + # echo "======================Testing supervisor agent: multi turns ======================" + local CONTENT=$(python3 $WORKDIR/GenAIExamples/FinanceAgent/tests/test.py --agent_role "supervisor" --ext_port $agent_port --multi-turn --stream) + echo $CONTENT + local EXIT_CODE=$(validate "$CONTENT" "test completed with success" "supervisor-agent-endpoint") + echo $EXIT_CODE + local EXIT_CODE="${EXIT_CODE:0-1}" + if [ "$EXIT_CODE" == "1" ]; then + docker logs supervisor-agent-endpoint + exit 1 + fi +} + +function main() { + validate_agent_service +} + +main diff --git a/FinanceAgent/tests/test_compose_on_gaudi.sh b/FinanceAgent/tests/test_compose_on_gaudi.sh index cb0f594422..4678e6a8da 100644 --- a/FinanceAgent/tests/test_compose_on_gaudi.sh +++ b/FinanceAgent/tests/test_compose_on_gaudi.sh @@ -1,5 +1,5 @@ #!/bin/bash -# Copyright (C) 2024 Intel Corporation +# Copyright (C) 2025 Intel Corporation # SPDX-License-Identifier: Apache-2.0 set -xe @@ -69,168 +69,16 @@ export EMBEDDING_MODEL_ID="BAAI/bge-base-en-v1.5" export TEI_EMBEDDING_ENDPOINT="http://${IP_ADDRESS}:${TEI_EMBEDDER_PORT}" ####################################### -function get_genai_comps() { - if [ ! -d "GenAIComps" ] ; then - git clone --depth 1 --branch ${opea_branch:-"main"} https://github.com/opea-project/GenAIComps.git - fi -} - -function build_dataprep_agent_images() { - cd $WORKDIR/GenAIExamples/FinanceAgent/docker_image_build/ - get_genai_comps - echo "Build agent image with --no-cache..." - docker compose -f build.yaml build --no-cache -} - -function build_agent_image_local(){ - cd $WORKDIR/GenAIComps/ - docker build -t opea/agent:latest -f comps/agent/src/Dockerfile . --build-arg https_proxy=$HTTPS_PROXY --build-arg http_proxy=$HTTP_PROXY -} - -function build_vllm_docker_image() { - echo "Building the vllm docker image" - cd $WORKPATH - echo $WORKPATH - if [ ! -d "./vllm-fork" ]; then - git clone https://github.com/HabanaAI/vllm-fork.git - fi - cd ./vllm-fork - - VLLM_FORK_VER=v0.6.6.post1+Gaudi-1.20.0 - git checkout ${VLLM_FORK_VER} &> /dev/null - docker build --no-cache -f Dockerfile.hpu -t $VLLM_IMAGE --shm-size=128g . --build-arg https_proxy=$HTTPS_PROXY --build-arg http_proxy=$HTTP_PROXY - if [ $? -ne 0 ]; then - echo "$VLLM_IMAGE failed" - exit 1 - else - echo "$VLLM_IMAGE successful" - fi -} - -function stop_llm(){ +function stop_llm() { cid=$(docker ps -aq --filter "name=vllm-gaudi-server") echo "Stopping container $cid" if [[ ! -z "$cid" ]]; then docker rm $cid -f && sleep 1s; fi - -} - -function start_all_services(){ - docker compose -f $WORKPATH/docker_compose/intel/hpu/gaudi/compose.yaml up -d - - until [[ "$n" -ge 200 ]] || [[ $ready == true ]]; do - docker logs vllm-gaudi-server &> ${LOG_PATH}/vllm-gaudi-service.log - n=$((n+1)) - if grep -q "Uvicorn running on" ${LOG_PATH}/vllm-gaudi-service.log; then - break - fi - if grep -q "No such container" ${LOG_PATH}/vllm-gaudi-service.log; then - echo "container vllm-gaudi-server not found" - exit 1 - fi - sleep 10s - done - sleep 10s - echo "Service started successfully" -} - -function validate() { - local CONTENT="$1" - local EXPECTED_RESULT="$2" - local SERVICE_NAME="$3" - echo "EXPECTED_RESULT: $EXPECTED_RESULT" - echo "Content: $CONTENT" - if echo "$CONTENT" | grep -q "$EXPECTED_RESULT"; then - echo "[ $SERVICE_NAME ] Content is as expected: $CONTENT" - echo 0 - else - echo "[ $SERVICE_NAME ] Content does not match the expected result: $CONTENT" - echo 1 - fi -} - -function ingest_validate_dataprep() { - # test /v1/dataprep/ingest - echo "=========== Test ingest ===========" - local CONTENT=$(python $WORKPATH/tests/test_redis_finance.py --port $DATAPREP_PORT --test_option ingest) - local EXIT_CODE=$(validate "$CONTENT" "200" "dataprep-redis-finance") - echo "$EXIT_CODE" - local EXIT_CODE="${EXIT_CODE:0-1}" - if [ "$EXIT_CODE" == "1" ]; then - docker logs dataprep-redis-server-finance - exit 1 - fi - - # test /v1/dataprep/get - echo "=========== Test get ===========" - local CONTENT=$(python $WORKPATH/tests/test_redis_finance.py --port $DATAPREP_PORT --test_option get) - local EXIT_CODE=$(validate "$CONTENT" "Request successful" "dataprep-redis-finance") - echo "$EXIT_CODE" - local EXIT_CODE="${EXIT_CODE:0-1}" - if [ "$EXIT_CODE" == "1" ]; then - docker logs dataprep-redis-server-finance - exit 1 - fi } function stop_dataprep() { echo "Stopping databases" cid=$(docker ps -aq --filter "name=dataprep-redis-server*" --filter "name=redis-*" --filter "name=tei-embedding-*") if [[ ! -z "$cid" ]]; then docker stop $cid && docker rm $cid && sleep 1s; fi - -} - -function validate_agent_service() { - # test worker finqa agent - echo "======================Testing worker finqa agent======================" - export agent_port="9095" - prompt="What is Gap's revenue in 2024?" - local CONTENT=$(python3 $WORKDIR/GenAIExamples/FinanceAgent/tests/test.py --prompt "$prompt" --agent_role "worker" --ext_port $agent_port) - echo $CONTENT - local EXIT_CODE=$(validate "$CONTENT" "15" "finqa-agent-endpoint") - echo $EXIT_CODE - local EXIT_CODE="${EXIT_CODE:0-1}" - if [ "$EXIT_CODE" == "1" ]; then - docker logs finqa-agent-endpoint - exit 1 - fi - - # test worker research agent - echo "======================Testing worker research agent======================" - export agent_port="9096" - prompt="Johnson & Johnson" - local CONTENT=$(python3 $WORKDIR/GenAIExamples/AgentQnA/tests/test.py --prompt "$prompt" --agent_role "worker" --ext_port $agent_port --tool_choice "get_current_date" --tool_choice "get_share_performance") - local EXIT_CODE=$(validate "$CONTENT" "Johnson" "research-agent-endpoint") - echo $CONTENT - echo $EXIT_CODE - local EXIT_CODE="${EXIT_CODE:0-1}" - if [ "$EXIT_CODE" == "1" ]; then - docker logs research-agent-endpoint - exit 1 - fi - - # test supervisor react agent - echo "======================Testing supervisor agent: single turns ======================" - export agent_port="9090" - local CONTENT=$(python3 $WORKDIR/GenAIExamples/FinanceAgent/tests/test.py --agent_role "supervisor" --ext_port $agent_port --stream) - echo $CONTENT - local EXIT_CODE=$(validate "$CONTENT" "test completed with success" "supervisor-agent-endpoint") - echo $EXIT_CODE - local EXIT_CODE="${EXIT_CODE:0-1}" - if [ "$EXIT_CODE" == "1" ]; then - docker logs supervisor-agent-endpoint - exit 1 - fi - - # echo "======================Testing supervisor agent: multi turns ======================" - local CONTENT=$(python3 $WORKDIR/GenAIExamples/FinanceAgent/tests/test.py --agent_role "supervisor" --ext_port $agent_port --multi-turn --stream) - echo $CONTENT - local EXIT_CODE=$(validate "$CONTENT" "test completed with success" "supervisor-agent-endpoint") - echo $EXIT_CODE - local EXIT_CODE="${EXIT_CODE:0-1}" - if [ "$EXIT_CODE" == "1" ]; then - docker logs supervisor-agent-endpoint - exit 1 - fi } function stop_agent_docker() { @@ -251,31 +99,27 @@ stop_dataprep cd $WORKPATH/tests -echo "=================== #1 Building docker images====================" -# build_vllm_docker_image -build_dataprep_agent_images - -# ## for local test -# # build_agent_image_local -echo "=================== #1 Building docker images completed====================" +echo "=================== #1 Building docker images ====================" +bash step1_build_images.sh gaudi_vllm +echo "=================== #1 Building docker images completed ====================" echo "=================== #2 Start services ====================" -start_all_services -echo "=================== #2 Endpoints for services started====================" +bash step2_start_services.sh gaudi_vllm +echo "=================== #2 Endpoints for services started ====================" echo "=================== #3 Validate ingest_validate_dataprep ====================" -ingest_validate_dataprep -echo "=================== #3 Data ingestion and validation completed====================" +bash step3_validate_ingest_validate_dataprep.sh +echo "=================== #3 Data ingestion and validation completed ====================" echo "=================== #4 Start agents ====================" -validate_agent_service +bash step4_validate_agent_service.sh echo "=================== #4 Agent test passed ====================" echo "=================== #5 Stop microservices ====================" stop_agent_docker stop_dataprep stop_llm -echo "=================== #5 Microservices stopped====================" +echo "=================== #5 Microservices stopped ====================" echo y | docker system prune