Repository navigation
Observability: configurable trace retention (session-only / N days / forever) #32
Description
Split from #4 (LLM Observability Service). Depends on the base Phoenix tracing slice (#4) landing first — file this immediately after, ahead of #29/#30/#31, since trace data starts accumulating unbounded the moment #4 merges.
This repo already has a documented retention pattern (Phase 4b, docs/SECURITY.md's retention policy table): every persisted store gets a nullable DB override over an env default, resolved per-call by app.services.retention_service.resolve_retention_config (None = keep forever, 0 = purge immediately, same semantics as AGENTS.md's "Retention Field Semantics" section). Trace data needs the same treatment — some deployments will want zero retention past a session, others a project's lifecycle, others 3 months, others forever.
Why this isn't part of #4 as filed: #4's traces live in Phoenix's own storage (a SQLite file on a dedicated volume, per #4's design — deliberately kept out of the shared eaistack Postgres DB and Alembic's schema authority, since Phoenix's trace store isn't this repo's application schema). That means the existing retention_sweep/resolve_retention_config machinery — which purges rows it owns directly via SQLAlchemy DELETE — cannot reach trace data at all. Purging traces means calling out to Phoenix's own deletion/retention surface (its HTTP API, or a native TTL config if the vendored image supports one — needs verification against the actual image, not assumed from training data), which is a meaningfully different integration than every other retention target purged so far.
Design sketch (to be finalized when this issue is picked up):
SystemSettings.trace_retention_days: int | None— sameNone=keep forever /0=purge immediately pattern asconversation_retention_hours,knowledge_base_purge_days,api_key_purge_days.- A new
purge_expired_tracesstep, wired into the existing CronJob sweep (app.cli.retention_sweep) alongside conversations/knowledge_base/api_keys — but implemented via a call to Phoenix's deletion API (or its native retention config, if one exists and is verified against the vendored image) rather than a direct DBDELETE, since the sweep process doesn't own that database. - Admin Settings UI: extend the existing retention-window UI (
frontend/src/components/RetentionSettings.tsxpattern) with a fourth field, including the existing "confirm before shortening a window" safety check. - First step when picked up: confirm what retention/deletion surface the actual vendored Phoenix image exposes (HTTP API for cutoff-based deletion vs. a native env-var TTL) before committing to one of the two mechanisms above.
Out of scope for this issue: trace clustering/search (#29), eval hooks (#30), cost-per-span (#31) — unrelated concerns split from the same original issue (#4).
Activity
prometheusgr commentedon Aug 31, 2026
Clarifying two points raised during #4's planning:
Traces get their own independent retention window, not a shared/blanket policy. This repo's existing retention table (
docs/SECURITY.md) already gives every store its own distinct window — conversations 24h, knowledge-base 30d, api_keys 30d, audit_logs forever/never-purged.trace_retention_daysfollows that same shape: its ownSystemSettingscolumn, its own resolver, its own sweep step, independently configurable from every other window (session-only via 0, a project's lifecycle via some N days, 3 months, or forever via null) — never coupled toconversation_retention_hoursor any other field.Default should not be "forever." Traces captured by #4 include the exact prompt and response content by design (that's the "prompt/response inspection" requirement) — the same sensitivity class as
conversation_threads, which already defaults to a conservative 24h window rather than indefinite retention.trace_retention_days's env-level default should follow that precedent (a bounded number of days, notNone) rather than accumulating unbounded sensitive content by default until an admin opts into shortening it. Exact default value to be decided when this issue is implemented, but "keep forever" is very unlikely to be the right out-of-the-box posture given the SECURITY.md precedent for every other content-bearing store.prometheusgr commentedon Sep 28, 2026
Priority context: the workflow-configuration epic #80 uses Phoenix as the review and diagnosis surface for workflow iteration. Together with #90 (Phoenix authentication), this issue is a prerequisite for relying on Phoenix in that loop, because traces carry user content and the playground can call the model directly.