You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Split from #4 (LLM Observability Service). Depends on the base Phoenix tracing slice landing first.
The issue's tracing bullet asks for "latency, token counts, and cost per span." Token counts and latency are captured by the base OpenInference/OTel instrumentation for free. Cost is not: this stack's default inference path is a self-hosted llama-server with no per-token price, so "cost" has no meaning until there's an explicit cost model to attach (e.g. an admin-configured $/1K-tokens figure per provider, relevant mainly if/when an openai-compatible paid provider is configured instead of local llama.cpp).
Needs a design decision on where that cost model lives (likely alongside the existing system_settings provider config) before implementation. Deferred out of the base tracing slice rather than adding a meaningless always-zero "cost" column.
Activity