Support OTel instrumentation for Background Agent (Copilot CLI)

[zhichli]

Problem

The background agent (copilotcli) has zero OTel instrumentation from the VS Code extension side.

The @github/copilot SDK (copilot-cli package) manages its own HTTP client internally. The VS Code extension creates the SDK session via toSessionOptions() in copilotCli.ts but passes no trace context or request headers. The SDK bypasses the extension's networkRequest() pipeline entirely.

Impact

• Background agent runs are completely invisible to distributed tracing
• No OTel spans for LLM calls or tool executions
• No correlation with foreground agent traces or eval runs
• The X-Request-Id values sent by the SDK are internal and not exposed to the extension

Current foreground agent OTel (for comparison)

The foreground agent already has full OTel instrumentation:

• invoke_agent span wrapping the agent orchestration
• chat {model} spans per LLM API call (span kind: CLIENT)
• execute_tool {name} spans per tool invocation
• Metrics: token usage, operation duration, tool call count/duration, TTFT
• Events: inference details, session start, tool call, agent turn

The background agent has none of this.

Proposed Fix (layered)

Layer A — Create extension-side OTel spans (short-term)

1. In copilotcliSession.ts, wrap the session's handleRequest() in otelService.startActiveSpan("copilotcli_session", ...).
2. Subscribe to SDK events (tool.execution_start, assistant.message, assistant.usage) and create child spans to mirror the foreground agent's span hierarchy.
3. Wire subagent trace context storage (storeTraceContext("subagent:...")) so background-agent-invoked subagents inherit the trace.

Layer B — SDK exposes request IDs (medium-term)

1. The SDK's ClientOptions supports requestHeaders?: Record<string, string> at the per-request level. Expose a session-level defaultHeaders config in the @github/copilot SDK.
2. The SDK should also expose X-Request-Id and X-GitHub-Request-ID from CAPI responses via events, so the extension can log/correlate them.

Layer C — SDK-native OTel (long-term)

1. The @github/copilot SDK itself should accept an OTel TracerProvider and create spans internally for each LLM call and tool execution.

Key Files

• src/extension/chatSessions/copilotcli/node/copilotCli.ts — CopilotCLISessionOptions.toSessionOptions(), where session config is built
• src/extension/chatSessions/copilotcli/node/copilotcliSession.ts — handleRequest(), SDK event subscriptions
• src/extension/chatSessions/copilotcli/node/copilotcliSessionService.ts — createSession()
• src/platform/otel/common/otelService.ts — IOTelService interface (startActiveSpan, storeTraceContext)

Context

The copilotcli SDK (@github/copilot) uses its own internal HTTP client that bypasses the extension's networkRequest() pipeline. The SDK's SessionOptions type has no requestHeaders field, though per-request ClientOptions and GetCompletionWithToolsOptions do support requestHeaders. The SDK's CAPIRequestContext type supports interactionType, agentTaskId, parentAgentTaskId, clientSessionId, and interactionId.

For the foreground agent OTel implementation (as reference), see:

• agent_monitoring.md — User-facing OTel docs
• agent_monitoring_arch.md — Developer architecture

[laniakea001]

Architecture Analysis

This is a great feature request. The SDK/extension separation creates a visibility gap in the observability pipeline.

Alternative Approach: Sidecar Tracing

Instead of modifying the SDK internals, consider:

1. HTTP Proxy Interception

   • Run a local proxy that intercepts all SDK HTTP traffic
   • Inject trace headers (W3C Trace Context)
   • Correlate with extension-side spans

2. SDK Wrapper Layer
   Create a thin wrapper around the SDK that handles OTel:

   import { NodeSDK } from @opentelemetry/sdk-node;
   
   class ObservableCopilotSession {
     constructor(private session: CopilotSession) {}
     
     async handleRequest(req) {
       return trace.getTracer(copilot-cli).startActiveSpan(
         copilot.request,
         { kind: SpanKind.CLIENT },
         async (span) => {
           try {
             const result = await this.session.handleRequest(req);
             span.setAttribute(copilot.request_id, result.requestId);
             return result;
           } finally {
             span.end();
           }
         }
       );
     }
   }

3. Event-Based Instrumentation
   The SDK already emits events - subscribe to them at the extension boundary:

   session.on(tool.execution_start, (e) => {
     span.addEvent(tool_start, { tool: e.toolName });
   });
   session.on(assistant.usage, (e) => {
     span.setAttribute(tokens.input, e.inputTokens);
     span.setAttribute(tokens.output, e.outputTokens);
   });

Priority Recommendation

Layer A (extension-side spans) is the fastest path:

• No SDK changes required
• Uses existing OTel infrastructure
• Provides immediate visibility

Layer B should be filed as a separate SDK feature request.

---

Related: OpenClaw uses a similar layered approach with worker-level tracing and centralized trace aggregation. See openclaw/openclaw for reference.

[mjbvz]

Moving to 1.113 since this does not seem like a candidate for 1.112. Please move it back if this is incorrect