[Bug]: error in step 'run_agent_step' with BedrockConverse

[pazevedo-hyland]

Bug Description

Every now and then the AgentWorkflows get a problem when calling tools with bedrock, I can't reproduce this consistently, it happends every now and then

2025-04-14 13:27:28 llama_index.core.workflow.errors.WorkflowRuntimeError: Error in step 'run_agent_step': An error occurred (ValidationException) when calling the ConverseStream operation: The text field in the ContentBlock object at messages.7.content.0 is blank. Add text to the text field, and try again.

Version

llama-index-llms-bedrock-converse = "0.4.15" llama-index-postprocessor-bedrock-rerank = "0.3.0" llama-index-embeddings-bedrock = "0.5.0" llama-index-tools-mcp = "^0.1.0" llama-index = "0.12.28"

Steps to Reproduce

Create AgentWorkflow from tools with base class and call tools with Bedrock converse, it will break eventually

Relevant Logs/Tracbacks

14 13:27:28 │ /usr/local/lib/python3.12/site-packages/llama_index/core/workflow/context.py:592 in _step_worker │
2025-04-14 13:27:28 │                                                                                                  │
2025-04-14 13:27:28 │   589 │   │   │   │   │   │   raise                                                              │
2025-04-14 13:27:28 │   590 │   │   │   │   │   except Exception as e:                                                 │
2025-04-14 13:27:28 │   591 │   │   │   │   │   │   if config.retry_policy is None:                                    │
2025-04-14 13:27:28 │ ❱ 592 │   │   │   │   │   │   │   raise WorkflowRuntimeError(                                    │
2025-04-14 13:27:28 │   593 │   │   │   │   │   │   │   │   f"Error in step '{name}': {e!s}"                           │
2025-04-14 13:27:28 │   594 │   │   │   │   │   │   │   ) from e                                                       │
2025-04-14 13:27:28 │   595                                                                                            │
2025-04-14 13:27:28 │                                                                                                  │
2025-04-14 13:27:28 │ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
2025-04-14 13:27:28 │ │            attempts = 0                                                                      │ │
2025-04-14 13:27:28 │ │ checkpoint_callback = None                                                                   │ │
2025-04-14 13:27:28 │ │              config = StepConfig(                                                            │ │
2025-04-14 13:27:28 │ │                       │   accepted_events=[                                                  │ │
2025-04-14 13:27:28 │ │                       │   │   <class                                                         │ │
2025-04-14 13:27:28 │ │                       'llama_index.core.agent.workflow.workflow_events.AgentSetup'>          │ │
2025-04-14 13:27:28 │ │                       │   ],                                                                 │ │
2025-04-14 13:27:28 │ │                       │   event_name='ev',                                                   │ │
2025-04-14 13:27:28 │ │                       │   return_types=[                                                     │ │
2025-04-14 13:27:28 │ │                       │   │   <class                                                         │ │
2025-04-14 13:27:28 │ │                       'llama_index.core.agent.workflow.workflow_events.AgentOutput'>         │ │
2025-04-14 13:27:28 │ │                       │   ],                                                                 │ │
2025-04-14 13:27:28 │ │                       │   context_parameter='ctx',                                           │ │
2025-04-14 13:27:28 │ │                       │   num_workers=4,                                                     │ │
2025-04-14 13:27:28 │ │                       │   requested_services=[],                                             │ │
2025-04-14 13:27:28 │ │                       │   retry_policy=None                                                  │ │
2025-04-14 13:27:28 │ │                       )                                                                      │ │
2025-04-14 13:27:28 │ │          dispatcher = Dispatcher(                                                            │ │
2025-04-14 13:27:28 │ │                       │   name='llama_index.core.workflow.workflow',                         │ │
2025-04-14 13:27:28 │ │                       │   event_handlers=[],                                                 │ │
2025-04-14 13:27:28 │ │                       │   span_handlers=[],                                                  │ │
2025-04-14 13:27:28 │ │                       │   parent_name='root',                                                │ │
2025-04-14 13:27:28 │ │                       │   manager=<llama_index.core.instrumentation.dispatcher.Manager       │ │
2025-04-14 13:27:28 │ │                       object at 0xffff686863f0>,                                             │ │
2025-04-14 13:27:28 │ │                       │   root_name='root',                                                  │ │
2025-04-14 13:27:28 │ │                       │   propagate=True,                                                    │ │
2025-04-14 13:27:28 │ │                       │   current_span_ids={}                                                │ │
2025-04-14 13:27:28 │ │                       )                                                                      │ │
2025-04-14 13:27:28 │ │                  ev = AgentSetup(                                                            │ │
2025-04-14 13:27:28 │ │                       │   input=[                                                            │ │
2025-04-14 13:27:28 │ │                       │   │   ChatMessage(                                                   │ │
2025-04-14 13:27:28 │ │                       │   │   │   role=<MessageRole.SYSTEM: 'system'>,                       │ │
2025-04-14 13:27:28 │ │                       │   │   │   additional_kwargs={},                                      │ │
2025-04-14 13:27:28 │ │                       │   │   │   blocks=[                                                   │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   TextBlock(                                             │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   block_type='text',                                 │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   text='You are an intelligent assistant specialized │ │
2025-04-14 13:27:28 │ │                       in claims processing and document m'+2789                              │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   )                                                      │ │
2025-04-14 13:27:28 │ │                       │   │   │   ]                                                          │ │
2025-04-14 13:27:28 │ │                       │   │   ),                                                             │ │
2025-04-14 13:27:28 │ │                       │   │   ChatMessage(                                                   │ │
2025-04-14 13:27:28 │ │                       │   │   │   role=<MessageRole.USER: 'user'>,                           │ │
2025-04-14 13:27:28 │ │                       │   │   │   additional_kwargs={},                                      │ │
2025-04-14 13:27:28 │ │                       │   │   │   blocks=[                                                   │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   TextBlock(                                             │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   block_type='text',                                 │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   text="Can you give me information on Mark Scout's  │ │
2025-04-14 13:27:28 │ │                       latest Auto Insurance Claim?"                                          │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   )                                                      │ │
2025-04-14 13:27:28 │ │                       │   │   │   ]                                                          │ │
2025-04-14 13:27:28 │ │                       │   │   ),                                                             │ │
2025-04-14 13:27:28 │ │                       │   │   ChatMessage(                                                   │ │
2025-04-14 13:27:28 │ │                       │   │   │   role=<MessageRole.ASSISTANT: 'assistant'>,                 │ │
2025-04-14 13:27:28 │ │                       │   │   │   additional_kwargs={},                                      │ │
2025-04-14 13:27:28 │ │                       │   │   │   blocks=[                                                   │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   TextBlock(                                             │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   block_type='text',                                 │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   text='\n\nBased on the information retrieved from  │ │
2025-04-14 13:27:28 │ │                       the semantic search, here are the key '+490                            │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   )                                                      │ │
2025-04-14 13:27:28 │ │                       │   │   │   ]                                                          │ │
2025-04-14 13:27:28 │ │                       │   │   ),                                                             │ │
2025-04-14 13:27:28 │ │                       │   │   ChatMessage(                                                   │ │
2025-04-14 13:27:28 │ │                       │   │   │   role=<MessageRole.USER: 'user'>,                           │ │
2025-04-14 13:27:28 │ │                       │   │   │   additional_kwargs={},                                      │ │
2025-04-14 13:27:28 │ │                       │   │   │   blocks=[                                                   │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   TextBlock(                                             │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   block_type='text',                                 │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   text='Great, on alfresco there is a document named │ │
2025-04-14 13:27:28 │ │                       "claim_details_template.docx" can y'+65                                │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   )                                                      │ │
2025-04-14 13:27:28 │ │                       │   │   │   ]                                                          │ │
2025-04-14 13:27:28 │ │                       │   │   ),                                                             │ │
2025-04-14 13:27:28 │ │                       │   │   ChatMessage(                                                   │ │
2025-04-14 13:27:28 │ │                       │   │   │   role=<MessageRole.ASSISTANT: 'assistant'>,                 │ │
2025-04-14 13:27:28 │ │                       │   │   │   additional_kwargs={},                                      │ │
2025-04-14 13:27:28 │ │                       │   │   │   blocks=[                                                   │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   TextBlock(                                             │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   block_type='text',                                 │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   text='\n\nPopulating the template with the claim   │ │
2025-04-14 13:27:28 │ │                       details:\n\nClaim Details:\n- Claim Submit'+593                        │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   )                                                      │ │
2025-04-14 13:27:28 │ │                       │   │   │   ]                                                          │ │
2025-04-14 13:27:28 │ │                       │   │   ),                                                             │ │
2025-04-14 13:27:28 │ │                       │   │   ChatMessage(                                                   │ │
2025-04-14 13:27:28 │ │                       │   │   │   role=<MessageRole.USER: 'user'>,                           │ │
2025-04-14 13:27:28 │ │                       │   │   │   additional_kwargs={},                                      │ │
2025-04-14 13:27:28 │ │                       │   │   │   blocks=[                                                   │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   TextBlock(                                             │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   block_type='text',                                 │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   text='Can you Generate me a pdf report of that?'   │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   )                                                      │ │
2025-04-14 13:27:28 │ │                       │   │   │   ]                                                          │ │
2025-04-14 13:27:28 │ │                       │   │   ),                                                             │ │
2025-04-14 13:27:28 │ │                       │   │   ChatMessage(                                                   │ │
2025-04-14 13:27:28 │ │                       │   │   │   role=<MessageRole.ASSISTANT: 'assistant'>,                 │ │
2025-04-14 13:27:28 │ │                       │   │   │   additional_kwargs={},                                      │ │
2025-04-14 13:27:28 │ │                       │   │   │   blocks=[                                                   │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   TextBlock(                                             │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   block_type='text',                                 │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   text='\n\nThe PDF report                           │ │
2025-04-14 13:27:28 │ │                       "Claim_Status_Report_Mark_Scout.pdf" has been created and uploa'+75    │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   )                                                      │ │
2025-04-14 13:27:28 │ │                       │   │   │   ]                                                          │ │
2025-04-14 13:27:28 │ │                       │   │   ),                                                             │ │
2025-04-14 13:27:28 │ │                       │   │   ChatMessage(                                                   │ │
2025-04-14 13:27:28 │ │                       │   │   │   role=<MessageRole.USER: 'user'>,                           │ │
2025-04-14 13:27:28 │ │                       │   │   │   additional_kwargs={},                                      │ │
2025-04-14 13:27:28 │ │                       │   │   │   blocks=[                                                   │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   TextBlock(                                             │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   block_type='text',                                 │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   text='Good now add a tag to                        │ │
2025-04-14 13:27:28 │ │                       Claim_Status_Report_Mark_Scout.pdf saying "pending_review"'            │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   )                                                      │ │
2025-04-14 13:27:28 │ │                       │   │   │   ]                                                          │ │
2025-04-14 13:27:28 │ │                       │   │   )                                                              │ │
2025-04-14 13:27:28 │ │                       │   ],                                                                 │ │
2025-04-14 13:27:28 │ │                       │   current_agent_name='Agent'                                         │ │
2025-04-14 13:27:28 │ │                       )                                                                      │ │
2025-04-14 13:27:28 │ │   instrumented_step = <FunctionWrapper at 0xffff4641d2a0 for method at 0xffff5485f880>       │ │
2025-04-14 13:27:28 │ │              kwargs = {                                                                      │ │
2025-04-14 13:27:28 │ │                       │   'ctx': <llama_index.core.workflow.context.Context object at        │ │
2025-04-14 13:27:28 │ │                       0xffff466ddf40>,                                                       │ │
2025-04-14 13:27:28 │ │                       │   'ev': AgentSetup(                                                  │ │
2025-04-14 13:27:28 │ │                       │   │   input=[                                                        │ │
2025-04-14 13:27:28 │ │                       │   │   │   ChatMessage(                                               │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   role=<MessageRole.SYSTEM: 'system'>,                   │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   additional_kwargs={},                                  │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   blocks=[                                               │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   TextBlock(                                         │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   │   block_type='text',                             │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   │   text='You are an intelligent assistant         │ │
2025-04-14 13:27:28 │ │                       specialized in claims processing and document m'+2789                  │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   )                                                  │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   ]                                                      │ │
2025-04-14 13:27:28 │ │                       │   │   │   ),                                                         │ │
2025-04-14 13:27:28 │ │                       │   │   │   ChatMessage(                                               │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   role=<MessageRole.USER: 'user'>,                       │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   additional_kwargs={},                                  │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   blocks=[                                               │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   TextBlock(                                         │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   │   block_type='text',                             │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   │   text="Can you give me information on Mark      │ │
2025-04-14 13:27:28 │ │                       Scout's latest Auto Insurance Claim?"                                  │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   )                                                  │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   ]                                                      │ │
2025-04-14 13:27:28 │ │                       │   │   │   ),                                                         │ │
2025-04-14 13:27:28 │ │                       │   │   │   ChatMessage(                                               │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   role=<MessageRole.ASSISTANT: 'assistant'>,             │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   additional_kwargs={},                                  │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   blocks=[                                               │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   TextBlock(                                         │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   │   block_type='text',                             │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   │   text='\n\nBased on the information retrieved   │ │
2025-04-14 13:27:28 │ │                       from the semantic search, here are the key '+490                       │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   )                                                  │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   ]                                                      │ │
2025-04-14 13:27:28 │ │                       │   │   │   ),                                                         │ │
2025-04-14 13:27:28 │ │                       │   │   │   ChatMessage(                                               │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   role=<MessageRole.USER: 'user'>,                       │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   additional_kwargs={},                                  │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   blocks=[                                               │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   TextBlock(                                         │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   │   block_type='text',                             │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   │   text='Great, on alfresco there is a document   │ │
2025-04-14 13:27:28 │ │                       named "claim_details_template.docx" can y'+65                          │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   )                                                  │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   ]                                                      │ │
2025-04-14 13:27:28 │ │                       │   │   │   ),                                                         │ │
2025-04-14 13:27:28 │ │                       │   │   │   ChatMessage(                                               │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   role=<MessageRole.ASSISTANT: 'assistant'>,             │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   additional_kwargs={},                                  │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   blocks=[                                               │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   TextBlock(                                         │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   │   block_type='text',                             │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   │   text='\n\nPopulating the template with the     │ │
2025-04-14 13:27:28 │ │                       claim details:\n\nClaim Details:\n- Claim Submit'+593                  │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   )                                                  │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   ]                                                      │ │
2025-04-14 13:27:28 │ │                       │   │   │   ),                                                         │ │
2025-04-14 13:27:28 │ │                       │   │   │   ChatMessage(                                               │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   role=<MessageRole.USER: 'user'>,                       │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   additional_kwargs={},                                  │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   blocks=[                                               │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   TextBlock(                                         │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   │   block_type='text',                             │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   │   text='Can you Generate me a pdf report of      │ │
2025-04-14 13:27:28 │ │                       that?'                                                                 │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   )                                                  │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   ]                                                      │ │
2025-04-14 13:27:28 │ │                       │   │   │   ),                                                         │ │
2025-04-14 13:27:28 │ │                       │   │   │   ChatMessage(                                               │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   role=<MessageRole.ASSISTANT: 'assistant'>,             │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   additional_kwargs={},                                  │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   blocks=[                                               │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   TextBlock(                                         │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   │   block_type='text',                             │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   │   text='\n\nThe PDF report                       │ │
2025-04-14 13:27:28 │ │                       "Claim_Status_Report_Mark_Scout.pdf" has been created and uploa'+75    │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   )                                                  │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   ]                                                      │ │
2025-04-14 13:27:28 │ │                       │   │   │   ),                                                         │ │
2025-04-14 13:27:28 │ │                       │   │   │   ChatMessage(                                               │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   role=<MessageRole.USER: 'user'>,                       │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   additional_kwargs={},                                  │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   blocks=[                                               │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   TextBlock(                                         │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   │   block_type='text',                             │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   │   text='Good now add a tag to                    │ │
2025-04-14 13:27:28 │ │                       Claim_Status_Report_Mark_Scout.pdf saying "pending_review"'            │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   )                                                  │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   ]                                                      │ │
2025-04-14 13:27:28 │ │                       │   │   │   )                                                          │ │
2025-04-14 13:27:28 │ │                       │   │   ],                                                             │ │
2025-04-14 13:27:28 │ │                       │   │   current_agent_name='Agent'                                     │ │
2025-04-14 13:27:28 │ │                       │   )                                                                  │ │
2025-04-14 13:27:28 │ │                       }                                                                      │ │
2025-04-14 13:27:28 │ │                name = 'run_agent_step'                                                       │ │
2025-04-14 13:27:28 │ │              new_ev = AgentOutput(                                                           │ │
2025-04-14 13:27:28 │ │                       │   response=ChatMessage(                                              │ │
2025-04-14 13:27:28 │ │                       │   │   role=<MessageRole.ASSISTANT: 'assistant'>,                     │ │
2025-04-14 13:27:28 │ │                       │   │   additional_kwargs={                                            │ │
2025-04-14 13:27:28 │ │                       │   │   │   'tool_calls': [                                            │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   {                                                      │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   'input': '{"document_name":                        │ │
2025-04-14 13:27:28 │ │                       "Claim_Status_Report_Mark_Scout.pdf", "tag": "pending_review"}',       │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   'toolUseId': 'tooluse_UAKdmXOoSvmXRr_2FV3A0A',     │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   │   'name': 'add_tag_to_document'                      │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   }                                                      │ │
2025-04-14 13:27:28 │ │                       │   │   │   ],                                                         │ │
2025-04-14 13:27:28 │ │                       │   │   │   'tool_call_id': [],                                        │ │
2025-04-14 13:27:28 │ │                       │   │   │   'status': []                                               │ │
2025-04-14 13:27:28 │ │                       │   │   },                                                             │ │
2025-04-14 13:27:28 │ │                       │   │   blocks=[TextBlock(block_type='text', text='')]                 │ │
2025-04-14 13:27:28 │ │                       │   ),                                                                 │ │
2025-04-14 13:27:28 │ │                       │   tool_calls=[                                                       │ │
2025-04-14 13:27:28 │ │                       │   │   ToolSelection(                                                 │ │
2025-04-14 13:27:28 │ │                       │   │   │   tool_id='tooluse_UAKdmXOoSvmXRr_2FV3A0A',                  │ │
2025-04-14 13:27:28 │ │                       │   │   │   tool_name='add_tag_to_document',                           │ │
2025-04-14 13:27:28 │ │                       │   │   │   tool_kwargs={                                              │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   'document_name': 'Claim_Status_Report_Mark_Scout.pdf', │ │
2025-04-14 13:27:28 │ │                       │   │   │   │   'tag': 'pending_review'                                │ │
2025-04-14 13:27:28 │ │                       │   │   │   }                                                          │ │
2025-04-14 13:27:28 │ │                       │   │   )                                                              │ │
2025-04-14 13:27:28 │ │                       │   ],                                                                 │ │
2025-04-14 13:27:28 │ │                       │   raw={                                                              │ │
2025-04-14 13:27:28 │ │                       │   │   'contentBlockDelta': {                                         │ │
2025-04-14 13:27:28 │ │                       │   │   │   'delta': {'toolUse': {'input': 'ew"}'}},                   │ │
2025-04-14 13:27:28 │ │                       │   │   │   'contentBlockIndex': 0                                     │ │
2025-04-14 13:27:28 │ │                       │   │   }                                                              │ │
2025-04-14 13:27:28 │ │                       │   },                                                                 │ │
2025-04-14 13:27:28 │ │                       │   current_agent_name='Agent'                                         │ │
2025-04-14 13:27:28 │ │                       )                                                                      │ │
2025-04-14 13:27:28 │ │      retry_start_at = 1744633647.0609717                                                     │ │
2025-04-14 13:27:28 │ │              run_id = 'b2fc4ea4-93bd-4dfb-bfaf-2dc6a078dd80'                                 │ │
2025-04-14 13:27:28 │ │                self = <llama_index.core.workflow.context.Context object at 0xffff466ddf40>   │ │
2025-04-14 13:27:28 │ │     service_manager = <llama_index.core.workflow.service.ServiceManager object at            │ │
2025-04-14 13:27:28 │ │                       0xffff3ec19700>                                                        │ │
2025-04-14 13:27:28 │ │                step = <bound method AgentWorkflow.run_agent_step of                          │ │
2025-04-14 13:27:28 │ │                       <llama_index.core.agent.workflow.multi_agent_workflow.AgentWorkflow    │ │
2025-04-14 13:27:28 │ │                       object at 0xffff55704ef0>>                                             │ │
2025-04-14 13:27:28 │ │            stepwise = False                                                                  │ │
2025-04-14 13:27:28 │ │             verbose = False                                                                  │ │
2025-04-14 13:27:28 │ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
2025-04-14 13:27:28 ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
2025-04-14 13:27:28 WorkflowRuntimeError: Error in step 'run_agent_step': An error occurred (ValidationException) when 
2025-04-14 13:27:28 calling the ConverseStream operation: The text field in the ContentBlock object at 
2025-04-14 13:27:28 messages.7.content.0 is blank. Add text to the text field, and try again.
2025-04-14 13:27:28 
2025-04-14 13:27:28 INFO:     192.168.65.1:64920 - "POST /v1/agents/5f2ae5a2-05b7-47a9-8985-b765ad81caa7/invoke-agent HTTP/1.1" 500 Internal Server Error
2025-04-14 13:27:28 ERROR:    Exception in ASGI application
2025-04-14 13:27:28 Traceback (most recent call last):
2025-04-14 13:27:28   File "/usr/local/lib/python3.12/site-packages/llama_index/core/workflow/context.py", line 583, in _step_worker
2025-04-14 13:27:28     new_ev = await instrumented_step(**kwargs)
2025-04-14 13:27:28              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-04-14 13:27:28   File "/usr/local/lib/python3.12/site-packages/llama_index/core/instrumentation/dispatcher.py", line 368, in async_wrapper
2025-04-14 13:27:28     result = await func(*args, **kwargs)
2025-04-14 13:27:28              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-04-14 13:27:28   File "/usr/local/lib/python3.12/site-packages/llama_index/core/agent/workflow/multi_agent_workflow.py", line 382, in run_agent_step
2025-04-14 13:27:28     agent_output = await agent.take_step(
2025-04-14 13:27:28                    ^^^^^^^^^^^^^^^^^^^^^^
2025-04-14 13:27:28   File "/usr/local/lib/python3.12/site-packages/llama_index/core/agent/workflow/function_agent.py", line 48, in take_step
2025-04-14 13:27:28     async for last_chat_response in response:
2025-04-14 13:27:28   File "/usr/local/lib/python3.12/site-packages/llama_index/core/llms/callbacks.py", line 88, in wrapped_gen
2025-04-14 13:27:28     async for x in f_return_val:
2025-04-14 13:27:28   File "/usr/local/lib/python3.12/site-packages/llama_index/llms/bedrock_converse/base.py", line 507, in gen
2025-04-14 13:27:28     async for chunk in response_gen:
2025-04-14 13:27:28   File "/usr/local/lib/python3.12/site-packages/llama_index/llms/bedrock_converse/utils.py", line 495, in _conversion_stream_with_retry
2025-04-14 13:27:28     response = await client.converse_stream(**kwargs)
2025-04-14 13:27:28                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-04-14 13:27:28   File "/usr/local/lib/python3.12/site-packages/aiobotocore/client.py", line 412, in _make_api_call
2025-04-14 13:27:28     raise error_class(parsed_response, operation_name)
2025-04-14 13:27:28 botocore.errorfactory.ValidationException: An error occurred (ValidationException) when calling the ConverseStream operation: The text field in the ContentBlock object at messages.7.content.0 is blank. Add text to the text field, and try again.
2025-04-14 13:27:28 
2025-04-14 13:27:28 The above exception was the direct cause of the following exception:
2025-04-14 13:27:28 
2025-04-14 13:27:28 Traceback (most recent call last):
2025-04-14 13:27:28   File "/usr/local/lib/python3.12/site-packages/uvicorn/protocols/http/httptools_impl.py", line 409, in run_asgi
2025-04-14 13:27:28     result = await app(  # type: ignore[func-returns-value]
2025-04-14 13:27:28              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-04-14 13:27:28   File "/usr/local/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
2025-04-14 13:27:28     return await self.app(scope, receive, send)
2025-04-14 13:27:28            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-04-14 13:27:28   File "/usr/local/lib/python3.12/site-packages/fastapi/applications.py", line 1054, in __call__
2025-04-14 13:27:28     await super().__call__(scope, receive, send)
2025-04-14 13:27:28   File "/usr/local/lib/python3.12/site-packages/starlette/applications.py", line 112, in __call__
2025-04-14 13:27:28     await self.middleware_stack(scope, receive, send)
2025-04-14 13:27:28   File "/usr/local/lib/python3.12/site-packages/starlette/middleware/errors.py", line 187, in __call__
2025-04-14 13:27:28     raise exc
2025-04-14 13:27:28   File "/usr/local/lib/python3.12/site-packages/starlette/middleware/errors.py", line 165, in __call__
2025-04-14 13:27:28     await self.app(scope, receive, _send)
2025-04-14 13:27:28   File "/usr/local/lib/python3.12/site-packages/starlette/middleware/cors.py", line 85, in __call__
2025-04-14 13:27:28     await self.app(scope, receive, send)
2025-04-14 13:27:28   File "/usr/local/lib/python3.12/site-packages/starlette/middleware/base.py", line 176, in __call__
2025-04-14 13:27:28     with recv_stream, send_stream, collapse_excgroups():
2025-04-14 13:27:28                                    ^^^^^^^^^^^^^^^^^^^^
2025-04-14 13:27:28   File "/usr/local/lib/python3.12/contextlib.py", line 158, in __exit__
2025-04-14 13:27:28     self.gen.throw(value)
2025-04-14 13:27:28   File "/usr/local/lib/python3.12/site-packages/starlette/_utils.py", line 82, in collapse_excgroups
2025-04-14 13:27:28     raise exc
2025-04-14 13:27:28   File "/usr/local/lib/python3.12/site-packages/starlette/middleware/base.py", line 178, in __call__
2025-04-14 13:27:28     response = await self.dispatch_func(request, call_next)
2025-04-14 13:27:28                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-04-14 13:27:28   File "/app/common/logging/logging_config.py", line 31, in dispatch
2025-04-14 13:27:28     response = await call_next(request)
2025-04-14 13:27:28                ^^^^^^^^^^^^^^^^^^^^^^^^
2025-04-14 13:27:28   File "/usr/local/lib/python3.12/site-packages/starlette/middleware/base.py", line 156, in call_next
2025-04-14 13:27:28     raise app_exc
2025-04-14 13:27:28   File "/usr/local/lib/python3.12/site-packages/starlette/middleware/base.py", line 141, in coro
2025-04-14 13:27:28     await self.app(scope, receive_or_disconnect, send_no_error)
2025-04-14 13:27:28   File "/usr/local/lib/python3.12/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
2025-04-14 13:27:28     await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
2025-04-14 13:27:28   File "/usr/local/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
2025-04-14 13:27:28     raise exc
2025-04-14 13:27:28   File "/usr/local/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
2025-04-14 13:27:28     await app(scope, receive, sender)
2025-04-14 13:27:28   File "/usr/local/lib/python3.12/site-packages/starlette/routing.py", line 714, in __call__
2025-04-14 13:27:28     await self.middleware_stack(scope, receive, send)
2025-04-14 13:27:28   File "/usr/local/lib/python3.12/site-packages/starlette/routing.py", line 734, in app
2025-04-14 13:27:28     await route.handle(scope, receive, send)
2025-04-14 13:27:28   File "/usr/local/lib/python3.12/site-packages/starlette/routing.py", line 288, in handle
2025-04-14 13:27:28     await self.app(scope, receive, send)
2025-04-14 13:27:28   File "/usr/local/lib/python3.12/site-packages/starlette/routing.py", line 76, in app
2025-04-14 13:27:28     await wrap_app_handling_exceptions(app, request)(scope, receive, send)
2025-04-14 13:27:28   File "/usr/local/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
2025-04-14 13:27:28     raise exc
2025-04-14 13:27:28   File "/usr/local/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
2025-04-14 13:27:28     await app(scope, receive, sender)
2025-04-14 13:27:28   File "/usr/local/lib/python3.12/site-packages/starlette/routing.py", line 73, in app
2025-04-14 13:27:28     response = await f(request)
2025-04-14 13:27:28                ^^^^^^^^^^^^^^^^
2025-04-14 13:27:28   File "/usr/local/lib/python3.12/site-packages/fastapi/routing.py", line 301, in app
2025-04-14 13:27:28     raw_response = await run_endpoint_function(
2025-04-14 13:27:28                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-04-14 13:27:28   File "/usr/local/lib/python3.12/site-packages/fastapi/routing.py", line 212, in run_endpoint_function
2025-04-14 13:27:28     return await dependant.call(**values)
2025-04-14 13:27:28            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-04-14 13:27:28   File "/app/common/auth/auth_decorator.py", line 111, in async_wrapper
2025-04-14 13:27:28     return await f(*args, user=user, **kwargs)
2025-04-14 13:27:28            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-04-14 13:27:28   File "/app/agent_runtime/v1/invoke_api.py", line 48, in invoke_agent
2025-04-14 13:27:28     prediction = await agent_chat_runtime.predict(agent_request)
2025-04-14 13:27:28                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-04-14 13:27:28   File "/app/agent_runtime/runtimes/agent_chat_runtime.py", line 41, in predict
2025-04-14 13:27:28     return await self.agent.predict(request.messages, agent_config)  # type: ignore[no-any-return]
2025-04-14 13:27:28            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-04-14 13:27:28   File "/app/agent_runtime/agent_models/tool_agent.py", line 428, in predict
2025-04-14 13:27:28     response = await self._run_agent_workflow(
2025-04-14 13:27:28                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-04-14 13:27:28   File "/app/agent_runtime/agent_models/tool_agent.py", line 253, in _run_agent_workflow
2025-04-14 13:27:28     workflow_response = await workflow.run(
2025-04-14 13:27:28                         ^^^^^^^^^^^^^^^^^^^
2025-04-14 13:27:28   File "/usr/local/lib/python3.12/site-packages/llama_index/core/workflow/workflow.py", line 394, in _run_workflow
2025-04-14 13:27:28     raise exception_raised
2025-04-14 13:27:28   File "/usr/local/lib/python3.12/site-packages/llama_index/core/workflow/context.py", line 592, in _step_worker
2025-04-14 13:27:28     raise WorkflowRuntimeError(
2025-04-14 13:27:28 llama_index.core.workflow.errors.WorkflowRuntimeError: Error in step 'run_agent_step': An error occurred (ValidationException) when calling the ConverseStream operation: The text field in the ContentBlock object at messages.7.content.0 is blank. Add text to the text field, and try again.