Per-user usage/token accounting for budgeting and chargeback #42

Context

Flagged during a review of standard enterprise functionality against the current stack. Issue #31 (cost-per-span tracking) covers per-trace cost inside Phoenix's tracing UI — useful for debugging a single run, but scoped to Phoenix's data model and blocked on the same "no cost model exists yet for the default self-hosted llama.cpp path" problem noted there. This issue is a different, complementary need: aggregate usage accounting per user/team over time, independent of whether Phoenix tracing is even enabled.

The gap

There is no usage accounting anywhere in the system. ChatState/the chat agent path doesn't persist token counts or request counts per user beyond whatever's inside an individual Phoenix trace (if tracing is on at all — it's off by default, tracing_enabled=False). A business running this deployment for multiple teams has no way to answer "how much has team X used this month," set a budget alert, or produce a chargeback report — all standard requirements once a shared deployment serves more than one team or cost center. This is orthogonal to rate limiting (#25/#38), which protects availability in real time and deliberately isn't audit-logged per-trip; usage accounting is a historical/aggregate record, not a real-time control.

Scope

1. Schema: a new table (e.g. usage_records) capturing at minimum user_id, timestamp, endpoint, and token counts (prompt/completion, if obtainable from the LLM client response — app.core.llm_client's ChatOpenAI wrapper already receives usage metadata from llama-server/OpenAI-compatible responses). Follow docs/DATABASE_MODELS.md's shape exactly.
2. Recording point: hook into the existing chat agent path (app.agents.chat_agent) at the same boundary tracing instruments — a service (docs/BACKEND_SERVICES.md-shaped), not scattered inline recording, so it can later be reused by other agents per docs/AGENT_LIBRARY.md's registry pattern.
3. Retention: usage records are exactly the kind of field docs/SECURITY.md's retention policy table already governs — add them to that table with an explicit window (a business needs monthly/annual aggregates to outlive individual conversation retention, so this is likely a longer or separately-configured window, not reuse of conversation_retention_hours) and wire into the existing retention sweep (app.cli.retention_sweep) rather than inventing a second purge mechanism.
4. Aggregation: a service method to roll up records by user (and, if a team/department concept is added, by team — otherwise per-user is sufficient given Document multi-tenancy scope: single-org isolation only, not multi-tenant SaaS #26's documented single-org-per-deployment scope) over a date range.
5. UI element (required): a "Usage" admin screen (require_admin-gated, alongside Settings) showing per-user request/token counts over a selectable date range (e.g. last 7/30/90 days), following docs/FRONTEND_ARCHITECTURE.md's client → service → hook → component layering like every other admin screen.

Definition of Done

• New usage_records table + Alembic migration, following docs/DATABASE_MODELS.md.
• Recording service integrated into the chat agent path, covered by unit tests (mocking only at the app.core.llm_client boundary, per AGENTS.md).
• Retention window for usage records documented in docs/SECURITY.md's policy table and enforced by the existing retention sweep.
• Admin-only "Usage" UI screen exists, shows per-user usage over a selectable date range, and is reachable from the app's normal navigation.
• docs/BACKEND_SERVICES.md-pattern service with unit test coverage for the aggregation logic.
• Standard merge gate from AGENTS.md's Definition of Done applies in full (tests, lint, e2e for the new UI flow, docs).