Repository navigation
Observability: verify and enable TLS termination for the Phoenix Helm chart #33
Description
Split from #4 (LLM Observability Service).
Every other Helm chart in this repo (infra/helm/charts/*) defaults tls.enabled: true and terminates TLS itself, using a cert-manager-issued Certificate mounted into the pod. doc-search's chart does this via its own docker-entrypoint.sh, which conditionally adds --ssl-certfile/--ssl-keyfile to its uvicorn invocation — that works because doc-search is a service we build and control the entrypoint of.
Phoenix (infra/helm/charts/phoenix/, added in #4) is a prebuilt, unmodified upstream image (arizephoenix/phoenix) per the "no Dockerfile of our own for prebuilt OSS images" pattern (same as postgres/minio/keycloak). That means we don't control its process invocation the way we do doc-search's, and #4 shipped its chart with tls.enabled: false rather than guess at flags/env vars that might not match the real image's actual TLS support — this repo is air-gapped, so #4's implementation had no way to check Phoenix's docs/CLI against the real running image without fabricating unverified infra.
To resolve:
- Verify against the actual vendored
arizephoenix/phoeniximage whether it supports native TLS termination (env var, CLI flag, or config file) — check its--helpoutput / startup logs / any bundled docs once the image is pulled in a dev environment with internet access. - If it does: wire
TLS_ENABLED/cert paths into the chart the same shape as every other service, flip the default totls.enabled: true, matching the compliance posture every other chart already has. - If it doesn't: add a TLS-terminating sidecar (nginx or envoy) to the Phoenix pod, terminating the cert-manager-issued certificate and proxying plaintext to Phoenix's container port — the fallback pattern for a backing service that can't speak TLS itself. This is real new implementation surface (proxy config to write, test, and maintain), which is why Observability: base LLM tracing service (Arize Phoenix) with prompt/response inspection #4 didn't default to this speculatively.
Until this lands, Phoenix is the one chart in infra/helm/charts/ with tls.enabled: false by default — a documented, deliberate exception (see docs/OBSERVABILITY.md's "Known gap: TLS" section and infra/helm/charts/phoenix/values.yaml's comment), not an oversight.
Activity
Remove Phoenix's misleading TLS scaffolding
prometheusgr commentedon Sep 28, 2026
Priority context: the workflow-configuration epic #80 uses Phoenix as the review and diagnosis surface for workflow iteration. Together with #90 (Phoenix authentication), this issue is a prerequisite for relying on Phoenix in that loop, because traces carry user content and the playground can call the model directly.