fix: add traces for tool calls and mcp tool listing

[grs]

What does this PR do?

Adds traces around tool execution and mcp tool listing for better observability.

Closes #3108

Test Plan

Manually examined traces in jaeger to verify the added information was available.

[cdoern]

I think we should hold off on this given we might be deprecating the entire telemetry API cc @iamemilio @ashwinb

[grs]

I think we should hold off on this given we might be deprecating the entire telemetry API cc @iamemilio @ashwinb

I believe this is independent of the telemetry API. Allowing tracing to be collected from Llama Stack seems always likely to be a requirement. I believe most users will use the otel standard to collect them and not use Llama Stack's own telemetry API. I do believe they will want tracing though.

This PR adds more detail to the tracing from the responses API as described. This is similar to what was emitted by the old agents API.

[cdoern]

makes sense @grs , I just had to refactor the agents code though for this reason (#3705) -- the tracing utils depend on some of the telemetry provider functionality/code. I'd be wary of merging something depending on that code if we know we are going to delete it all and just depend on an OTEL middleware.

I could be wrong here though, so I will let others chime in.

[cdoern]

especially with pieces like #3723 for example

[grs]

@cdoern the additions here only use tracing.span(). I believe this is safe whether telemetry is enabled or not. Unlike tracing.get_current_span(), span() will always return an object, and if telemetry is not configured, the context manager will simply not do anything.

Of course if tracing.span() is removed in favour of some other call, then the additions in this PR would also need to be swapped over to the new function. To me that seems like something that would be done more generally across the codebase at the point a change is made. Migrating the additions from this PR will be simple as they all use only the span() function, simpler than the agent_instance.py which uses more parts of the current tracing package.

If there is an imminent change to tracing about to merge, then merging that and requiring this to be rebased and fixed would certainly make sense. It doesn't seem that there is though, #3723 is not yet complete, and so to my way of thinking, the very small addition from this current PR does not make a subsequent change any harder.

[iamemilio]

Yes, sorry. The old telemetry API is going to be deprecated and I am going to replace manual instrumentation with the OTEL python SDK

[iamemilio]

That said, if you want to merge this, I plan to replace all instances of manual instrumentation with open telemetry native code, so I can replace this as well.

[ashwinb]

yep this is good. when the telemetry rewrite arrives this can be migrated to use direct OTEL. we cannot let new observability additions stop