Repository navigation
Add rate limiting to chat and auth endpoints #25
Description
Context
Flagged during a review of standard enterprise functionality against the current stack (Phase 4d complete). No rate limiting or throttling exists anywhere in the backend today.
The gap
POST /api/agents/chat and the auth endpoints have no per-user or per-IP request throttling. Each chat request drives an LLM call (llama-server) and, when a tool fires, an embedding + pgvector query (doc-search) — the most expensive path in the system. A single misbehaving client (buggy frontend retry loop, a compromised token, a scripted loop) can saturate llama-server for every other user with no backend-side control to stop it. This is a resource-exhaustion / availability gap, not a data-integrity one — distinct from the guardrails work (#16), which validates content, not request volume.
Scope
- Decide the mechanism and where it lives. Options: in-process (e.g.
slowapi/token-bucket per process) vs. a shared store (Redis) for correctness across multiple backend replicas. Given the retention CronJob precedent (docs/SECURITY.md's "Enforcement" section) of preferring K8s-native/shared-state correctness over per-replica state, evaluate whether an in-process limiter is acceptable for v1 (single replica assumption) or whether shared state is needed from the start — decide deliberately and write down why, same as the CronJob rationale. - Scope limits per identity, not just per IP. The caller's
user_id(from the validated JWT, never request input) is the natural key for/api/agents/chat; unauthenticated endpoints (login-adjacent) need IP-based limits instead. - Distinguish chat/tool-calling endpoints from cheap ones. A blanket global limit is the wrong shape —
/api/agents/chatneeds a tighter budget than, say,GET /api/agents/threads. - Admin-configurable limits, following the established pattern (env default + nullable DB override, resolved per-call, audit-logged on change) already used for retention (Backups: strategy, encryption, and retention reconciliation #12-adjacent), guardrails (Guardrails: make thresholds and heuristics configurable #16), and LLM/embedding provider config — don't invent a new config shape.
- Response shape on trip:
429with aRetry-Afterheader, consistent with how the input guardrail already returns400with a reason code rather than a silent failure. - Document in
docs/SECURITY.mdalongside the existing Guardrails & Compliance section — this is adjacent risk-mitigation, and a reader auditing abuse controls should find both in one place.
Standards
AGENTS.md. TDD as usual; the limiter's core logic (token bucket / sliding window math) is exactly the kind of deterministic, time-dependent logic that should accept now: datetime per docs/TIME_INJECTION.md, not call datetime.now()/time.time() internally.
Done when
Chat and auth endpoints reject excess requests with 429, limits are admin-configurable and audit-logged on change, the mechanism's replica-correctness tradeoff is documented, and docs/SECURITY.md reflects the new control.
Activity
Harden rate limiting (#25) and fix the frontend gap it exposed
Address code review: shared TooltipProvider, correct default, fix e2e…
Fix e2e CI failure: raise the dev-stack auth rate limit, fix toast co…
Add hover-help tooltips to every Settings field and close issue #37
Address code review: shared TooltipProvider, correct default, fix e2e…
Fix e2e CI failure: raise the dev-stack auth rate limit, fix toast co…
Add hover-help tooltips to every Settings field and close issue #37 (#51