Skip to content

app-server: OTLP logs remain empty while traces and metrics export, then emit BatchLogProcessor.Emit.AfterShutdown on shutdown #28810

Open

Description

@harshitagarwal2

What version of the Codex App are you using (From “About Codex” dialog)?

Codex Desktop internal Codex binary 0.136.0

What subscription do you have?

ChatGPT Business

What platform is your computer?

macOS arm64

What issue are you seeing?

codex app-server exports OTLP traces and OTLP metrics, but native OTLP logs remain empty even when OTLP log export is explicitly configured and the same collector is successfully receiving the other signals.

In repeated local repros, the collector output shows:

  • logs.json: empty
  • traces.json: non-empty
  • metrics.json: non-empty

On shutdown, codex app-server emits this warning:

BatchLogProcessor.Emit.AfterShutdown

with the message:

Logs are being emitted even after Shutdown. This indicates incorrect lifecycle management of OTelLoggerProvider in application. Logs will not be exported.

This appears to be an app-server / codex-otel lifecycle bug rather than a collector or config issue.

What steps can reproduce the bug?

  1. Start an OTLP collector that writes logs, traces, and metrics to local files.
  2. Run a long-lived codex app-server instance with OTLP enabled in config:
[analytics]
enabled = true

[otel]
environment = "local-sre-agent-ws"
log_user_prompt = true
exporter = { otlp-http = { endpoint = "http://127.0.0.1:4319/v1/logs", protocol = "json" } }
trace_exporter = { otlp-http = { endpoint = "http://127.0.0.1:4319/v1/traces", protocol = "json" } }
metrics_exporter = { otlp-http = { endpoint = "http://127.0.0.1:4319/v1/metrics", protocol = "json" } }
  1. Connect to the running app-server over WebSocket and perform a real request flow:
    • initialize
    • initialized
    • thread/start
    • turn/start
  2. Complete the turn successfully.
  3. Stop the server cleanly.
  4. Inspect collector outputs and app-server stderr.

Observed result:

  • OTLP traces export successfully.
  • OTLP metrics export successfully.
  • OTLP logs remain empty.
  • app-server prints:
    • BatchLogProcessor.Emit.AfterShutdown
    • Logs are being emitted even after Shutdown. This indicates incorrect lifecycle management of OTelLoggerProvider in application. Logs will not be exported.

I also reproduced the same symptom through codex debug app-server send-message-v2, but the long-lived WebSocket repro is stronger because it removes the built-in helper client as the primary variable.

What is the expected behavior?

If [otel].exporter is configured for codex app-server, native OTLP log events should be exported successfully, just like traces and metrics.

Representative events that appear to be intended for log export include:

  • codex.conversation_starts
  • codex.api_request
  • codex.websocket_connect
  • codex.websocket_request
  • codex.sse_event
  • codex.user_prompt

These should reach the OTLP log exporter without the AfterShutdown failure.

Additional information

Why this looks like an upstream code bug

The open-source code appears to support log export on the app-server path:

  • app-server builds an OTEL provider and installs both logger and tracing layers:
    • codex-rs/app-server/src/lib.rs
  • OTEL event macros emit log-only events to codex_otel.log_only:
    • codex-rs/otel/src/events/shared.rs
  • session telemetry emits many log-capable events:
    • codex-rs/otel/src/events/session_telemetry.rs

However, app-server later explicitly shuts OTEL down:

  • codex-rs/app-server/src/lib.rs

and OtelProvider also shuts down again in Drop:

  • codex-rs/otel/src/provider.rs

The provider shutdown logic also force-flushes traces before shutdown, but does not explicitly force-flush logs.

Root-cause hypothesis

Most likely explanation:

  1. app-server still has active drop/cleanup paths that emit tracing log events after the OTEL logger provider has already been shut down.
  2. The global tracing subscriber remains installed, so those later log events hit OpenTelemetryTracingBridge.
  3. OTEL rejects them with BatchLogProcessor.Emit.AfterShutdown, so logs are never exported.

Relevant code references

  • codex-rs/app-server/src/lib.rs
    • OTEL provider construction and layer installation
    • explicit otel.shutdown() near process shutdown
  • codex-rs/otel/src/provider.rs
    • OtelProvider::shutdown()
    • impl Drop for OtelProvider
    • logger_layer()
    • build_logger()
  • codex-rs/core/src/otel_init.rs
    • analytics gating appears to affect metrics, not log export
  • codex-rs/otel/src/events/shared.rs
    • log_event! target is codex_otel.log_only
  • codex-rs/otel/src/events/session_telemetry.rs
    • events such as codex.user_prompt, codex.api_request, codex.websocket_*, codex.sse_event

High-level fix outline

I believe the fix is likely one or more of:

  • keep the OTel provider alive until after all remaining app-server cleanup/drop paths finish emitting logs
  • remove or redesign the explicit otel.shutdown() vs Drop double-shutdown behavior
  • make shutdown one-shot / idempotent
  • ensure the log exporter is flushed before shutdown if the logger provider supports it

I have a local repro and can provide more exact command/output detail if useful.

Activity

  1. added
    bugSomething isn't working
    appIssues related to the Codex desktop app
    app-serverIssues involving app server protocol or interfaces
    on Jun 17, 2026
  2. Necmttn commented on Jun 21, 2026

    @Necmttn

    One downstream reason this is worth fixing: for Codex, OTLP logs can be the usage-bearing signal, not just a nice-to-have sibling of traces/metrics.

    In a local receiver, the contract that made Codex telemetry useful was:

    • send Codex OTLP to /v1/logs, because the exporter posts to the endpoint as configured
    • fail open on malformed OTLP/log payloads so exporters do not retry-storm
    • keep only signal events such as codex.conversation_starts, codex.user_prompt, and token-bearing codex.sse_event
    • lift conversation.id as the session key
    • store input, output, reasoning, cached, and tool token columns separately from trace spans

    A focused regression for this issue could assert that a long-lived codex app-server run exports at least conversation start, user prompt, and one token-bearing SSE log before shutdown, then shuts down without BatchLogProcessor.Emit.AfterShutdown.

    Traces/metrics passing should not be enough for this bug. A downstream session/cost tool can otherwise read the same run as "no usage" when the real failure is "log pipeline never flushed."


    Generated with ax.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    appIssues related to the Codex desktop appapp-serverIssues involving app server protocol or interfacesbugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions