Skip to content

Agents/finance compose #9

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 16 commits into
base: main
Choose a base branch
from
Open

Agents/finance compose #9

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
b696289
update compose.yaml
MSCetin37 May 22, 2025
f54e7f0
update README
MSCetin37 May 24, 2025
fe5baf8
update compose.yaml
MSCetin37 May 24, 2025
ea35bed
update test variables
MSCetin37 May 27, 2025
cce9031
minor readme changes
MSCetin37 May 28, 2025
f5d435b
Merge branch 'main' into agents/finance_compose
MSCetin37 May 28, 2025
6dbe635
updates for readme and env
MSCetin37 May 28, 2025
99a315b
update the readme file
MSCetin37 May 30, 2025
4707162
Merge branch 'main' into agents/finance_compose
MSCetin37 May 30, 2025
a13f84e
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 30, 2025
4b91d18
update test file
MSCetin37 Jun 1, 2025
6a5e7ad
update test file
MSCetin37 Jun 2, 2025
791e2c5
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jun 2, 2025
ea0f0b6
update test file
MSCetin37 Jun 2, 2025
e5a59e5
update test file
MSCetin37 Jun 2, 2025
1fc84da
Merge branch 'main' into agents/finance_compose
MSCetin37 Jun 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
180 changes: 32 additions & 148 deletions FinanceAgent/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,26 @@
# Finance Agent
# Finance Agent Example

## 1. Overview
## Table of Contents

- [Overview](#overview)
- [Problem Motivation](#problem-motivation)
- [Architecture](#architecture)
- [High-Level Diagram](#high-level-diagram)
- [OPEA Microservices Diagram for Data Handling](#opea-microservices-diagram-for-data-handling)
- [Deployment Options](#deployment-options)
- [Contribution](#contribution)

## Overview

The Finance Agent exemplifies a hierarchical multi-agent system designed to streamline financial document processing and analysis for users. It offers three core functionalities: summarizing lengthy financial documents, answering queries related to these documents, and conducting research to generate investment reports on public companies.

Navigating and analyzing extensive financial documents can be both challenging and time-consuming. Users often need concise summaries, answers to specific queries, or comprehensive investment reports. The Finance Agent effectively addresses these needs by automating document summarization, query answering, and research tasks, thereby enhancing productivity and decision-making efficiency.

Users interact with the system through a graphical user interface (UI), where a supervisor agent manages requests by delegating tasks to worker agents or the summarization microservice. The system also supports document uploads via the UI for processing.

## Architecture

### High-Level Diagram

The architecture of this Finance Agent example is shown in the figure below. The agent is a hierarchical multi-agent system and has 3 main functions:

Expand All @@ -12,6 +32,8 @@ The user interacts with the supervisor agent through the graphical UI. The super

![Finance Agent Architecture](assets/finance_agent_arch.png)

### OPEA Microservices Diagram for Data Handling

The architectural diagram of the `dataprep` microservice is shown below. We use [docling](https://github.com/docling-project/docling) to extract text from PDFs and URLs into markdown format. Both the full document content and tables are extracted. We then use an LLM to extract metadata from the document, including the company name, year, quarter, document type, and document title. The full document markdown then gets chunked, and LLM is used to summarize each chunk, and the summaries are embedded and saved to a vector database. Each table is also summarized by LLM and the summaries are embedded and saved to the vector database. The chunks and tables are also saved into a KV store. The pipeline is designed as such to improve retrieval accuracy of the `search_knowledge_base` tool used by the Question Answering worker agent.

![dataprep architecture](assets/fin_agent_dataprep.png)
Expand All @@ -30,154 +52,16 @@ The Question Answering worker agent uses `search_knowledge_base` tool to get rel

![finqa search tool arch](assets/finqa_tool.png)

## 2. Getting started

### 2.1 Download repos

```bash
mkdir /path/to/your/workspace/
export WORKDIR=/path/to/your/workspace/
cd $WORKDIR
git clone https://github.com/opea-project/GenAIExamples.git
```

### 2.2 Set up env vars

```bash
export ip_address="External_Public_IP"
export no_proxy=${your_no_proxy},${ip_address}
export HF_CACHE_DIR=/path/to/your/model/cache/
export HF_TOKEN=<you-hf-token>
export FINNHUB_API_KEY=<your-finnhub-api-key> # go to https://finnhub.io/ to get your free api key
export FINANCIAL_DATASETS_API_KEY=<your-api-key> # go to https://docs.financialdatasets.ai/ to get your free api key
```

### 2.3 [Optional] Build docker images

Only needed when docker pull failed.

```bash
cd $WORKDIR/GenAIExamples/FinanceAgent/docker_image_build
# get GenAIComps repo
git clone https://github.com/opea-project/GenAIComps.git
# build the images
docker compose -f build.yaml build --no-cache
```

If deploy on Gaudi, also need to build vllm image.

```bash
cd $WORKDIR
git clone https://github.com/HabanaAI/vllm-fork.git
# get the latest release tag of vllm gaudi
cd vllm-fork
VLLM_VER=$(git describe --tags "$(git rev-list --tags --max-count=1)")
echo "Check out vLLM tag ${VLLM_VER}"
git checkout ${VLLM_VER}
docker build --no-cache -f Dockerfile.hpu -t opea/vllm-gaudi:latest --shm-size=128g . --build-arg https_proxy=$https_proxy --build-arg http_proxy=$http_proxy
```

## 3. Deploy with docker compose

### 3.1 Launch vllm endpoint

Below is the command to launch a vllm endpoint on Gaudi that serves `meta-llama/Llama-3.3-70B-Instruct` model on 4 Gaudi cards.

```bash
cd $WORKDIR/GenAIExamples/FinanceAgent/docker_compose/intel/hpu/gaudi
bash launch_vllm.sh
```

### 3.2 Prepare knowledge base

The commands below will upload some example files into the knowledge base. You can also upload files through UI.

First, launch the redis databases and the dataprep microservice.

```bash
# inside $WORKDIR/GenAIExamples/FinanceAgent/docker_compose/intel/hpu/gaudi/
bash launch_dataprep.sh
```

Validate datat ingest data and retrieval from database:

```bash
python $WORKDIR/GenAIExamples/FinanceAgent/tests/test_redis_finance.py --port 6007 --test_option ingest
python $WORKDIR/GenAIExamples/FinanceAgent/tests/test_redis_finance.py --port 6007 --test_option get
```

### 3.3 Launch the multi-agent system

The command below will launch 3 agent microservices, 1 docsum microservice, 1 UI microservice.

```bash
# inside $WORKDIR/GenAIExamples/FinanceAgent/docker_compose/intel/hpu/gaudi/
bash launch_agents.sh
```

### 3.4 Validate agents

FinQA Agent:

```bash
export agent_port="9095"
prompt="What is Gap's revenue in 2024?"
python3 $WORKDIR/GenAIExamples/FinanceAgent/tests/test.py --prompt "$prompt" --agent_role "worker" --ext_port $agent_port
```

Research Agent:

```bash
export agent_port="9096"
prompt="generate NVDA financial research report"
python3 $WORKDIR/GenAIExamples/FinanceAgent/tests/test.py --prompt "$prompt" --agent_role "worker" --ext_port $agent_port --tool_choice "get_current_date" --tool_choice "get_share_performance"
```

Supervisor Agent single turns:

```bash
export agent_port="9090"
python3 $WORKDIR/GenAIExamples/FinanceAgent/tests/test.py --agent_role "supervisor" --ext_port $agent_port --stream
```

Supervisor Agent multi turn:

```bash
python3 $WORKDIR/GenAIExamples/FinanceAgent/tests/test.py --agent_role "supervisor" --ext_port $agent_port --multi-turn --stream

```

## How to interact with the agent system with UI

The UI microservice is launched in the previous step with the other microservices.
To see the UI, open a web browser to `http://${ip_address}:5175` to access the UI. Note the `ip_address` here is the host IP of the UI microservice.

1. Create Admin Account with a random value

2. Enter the endpoints in the `Connections` settings

First, click on the user icon in the upper right corner to open `Settings`. Click on `Admin Settings`. Click on `Connections`.

Then, enter the supervisor agent endpoint in the `OpenAI API` section: `http://${ip_address}:9090/v1`. Enter the API key as "empty". Add an arbitrary model id in `Model IDs`, for example, "opea_agent". The `ip_address` here should be the host ip of the agent microservice.

Then, enter the dataprep endpoint in the `Icloud File API` section. You first need to enable `Icloud File API` by clicking on the button on the right to turn it into green and then enter the endpoint url, for example, `http://${ip_address}:6007/v1`. The `ip_address` here should be the host ip of the dataprep microservice.

You should see screen like the screenshot below when the settings are done.

![opea-agent-setting](assets/ui_connections_settings.png)

3. Upload documents with UI

Click on the `Workplace` icon in the top left corner. Click `Knowledge`. Click on the "+" sign to the right of `Icloud Knowledge`. You can paste an url in the left hand side of the pop-up window, or upload a local file by click on the cloud icon on the right hand side of the pop-up window. Then click on the `Upload Confirm` button. Wait till the processing is done and the pop-up window will be closed on its own when the data ingestion is done. See the screenshot below.

Note: the data ingestion may take a few minutes depending on the length of the document. Please wait patiently and do not close the pop-up window.
## Deployment Options

![upload-doc-ui](assets/upload_doc_ui.png)
This Finance Agent example can be deployed manually on Docker Compose.

4. Test agent with UI
| Hardware | Deployment Mode | Guide Link |
| :----------------------------- | :------------------- | :----------------------------------------------------------------------- |
| Intel® Gaudi® AI Accelerator | Single Node (Docker) | [Gaudi Docker Compose Guide](./docker_compose/intel/hpu/gaudi/README.md) |

After the settings are done and documents are ingested, you can start to ask questions to the agent. Click on the `New Chat` icon in the top left corner, and type in your questions in the text box in the middle of the UI.
_Note: Building custom microservice images can be done using the resources in [GenAIComps](https://github.com/opea-project/GenAIComps)._

The UI will stream the agent's response tokens. You need to expand the `Thinking` tab to see the agent's reasoning process. After the agent made tool calls, you would also see the tool output after the tool returns output to the agent. Note: it may take a while to get the tool output back if the tool execution takes time.
## Contribution

![opea-agent-test](assets/opea-agent-test.png)
We welcome contributions to the OPEA project. Please refer to the [contribution guidelines](https://github.com/opea-project/docs/blob/main/community/CONTRIBUTING.md) for more information.
Loading
Loading