You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Workflows: create a test case from a production chat turn/trace (with provenance + retention handling) #88
Follow-up to #80. Depends on slice 4 (test cases + eval runs).
Context
The most valuable test cases come from real failures: a bad answer seen in a Phoenix trace should become a test case in one step. But that copies real user content (potentially sensitive) out of conversation_threads, which has a retention window, into test cases, which are kept long-term and aren't covered by the retention sweep.
Scope
"Create test case from this turn": from a chat turn (via its workflow version ID / trace link), an admin creates a test case pre-filled with the input, and edits it before saving.
Explicit, audited action (workflow.test_case_created_from_trace), recording the source thread/turn and the admin.
Provenance label on the test case (authored vs. from_production), shown in the UI.
Retention decision, documented in docs/SECURITY.md's retention table: how long from_production test cases are kept, and whether they're purged when their source user is deleted.
User-visible signal
A "Create test case" action in the admin draft/eval screens, and a provenance badge on each test case.
RBAC
Admin feature. An admin can see other users' conversation content through this path, which is a real privacy expansion. State it explicitly in the PR and in docs/SECURITY.md.
Definition of Done
Action, audit event, provenance label: TDD.
Retention/privacy decision documented.
Standard merge gate from AGENTS.md's Definition of Done applies in full.
Activity