Skip to content

[FEATURE]: Server-side cap + cursor pagination for /session/[id]/message to make long sessions loadable #35895

Closed as not planned

Description

@tao501

The problem

GET /session/<id>/message?limit=200 returns one giant JSON array. In a long-running session with images, this hit 1.93 MB raw / 1.11 MB gzipped on my install. On mobile over cellular that one request routinely takes 30-90 seconds. There's no escape hatch — the server has no cap, and the underlying query has no LIMIT clause either.

I dug into what makes the payload so big and found three patterns:

  1. Plain messages with tool calls and reasoning are 5-10 KB each. limit=200 × 8 KB = ~1.6 MB before gzip.
  2. Attachments are stored as inline data: URLs — I found one with a 550 KB state.attachments[0].url value. That's a base64-encoded image inlined into JSON. Three attachments and you're shipping 1.5 MB of mostly-incompressible noise.
  3. There's no server-side ceiling. The limit parameter is only honored client-side. A buggy or malicious client can ask for arbitrarily large limits.

Minimum-viable fix (this issue)

I'd like to ship this in two halves, with the minimum being this issue:

Part 1: server-side clamp

{
  "server": {
    "messageMaxLimit": 200,
    "messageDefaultLimit": 20
  }
}
  • messageMaxLimit default 200, preserves current behaviour
  • messageDefaultLimit default 20, kicks in when the client omits limit
  • When the client asks for more than messageMaxLimit, clamp and add Warning: 199 - "limit clamped to N" header
  • The frontend's hard-coded limit=200 becomes safe automatically — gets clamped, no client change needed

This alone brings the worst case down by ~10x without any client change.

Part 2 (follow-up PR): cursor pagination

GET /session/<id>/message?limit=50&olderThan=<messageID>
→ returns up to 50 messages with id < olderThan
→ adds Link: <?olderThan=<newLastId>&limit=50>; rel="next"

I'd be happy to scope this issue to just Part 1 if maintainers prefer smaller PRs, then file Part 2 as a follow-up.

Non-goals (for separate issues)

  • Streaming mode (SSE) for messages — happy to discuss if there's interest
  • Frontend refactor to actually paginate — UI team's call
  • Attachment lazy-loading (replace inline data: URLs with /attachment/<id> route) — I have thoughts, will file separately

Reproduction

A long session with several image attachments will produce a >1 MB /message response. The symptom is the web UI taking 30+ seconds to render chat history over cellular, with the spinner just frozen on the message list.

Things I'd love feedback on

  • Is messageDefaultLimit=20 too aggressive for first-paint? Many UIs assume more history on first load. Open to 50 or 100.
  • Should the Warning header carry the actual clamped value, or just a fixed 199 code?
  • Should the server emit a deprecation log when a client uses the old ?limit=200 directly, so we can find and update frontends?

Activity

  1. github-actions commented on Jul 8, 2026

    @github-actions
    Contributor

    This issue doesn't fully meet our contributing guidelines.

    What needs to be fixed:

    • The issue does not use one of the required templates (Bug Report, Feature Request, or Question). Since this appears to be a Feature Request, the title should start with [FEATURE]: and include the verification checkbox confirming the feature hasn't been suggested before.

    Please edit this issue to address the above within 2 hours, or it will be automatically closed.


    This issue might be a duplicate of existing issues. Please check:

    If you believe this was flagged incorrectly, please let a maintainer know.

  2. changed the title [-]Server-side cap + cursor pagination for /session/[id]/message to make long sessions loadable[/-] [+][FEATURE]: Server-side cap + cursor pagination for /session/[id]/message to make long sessions loadable[/+] on Jul 8, 2026
  3. github-actions commented on Jul 8, 2026

    @github-actions
    Contributor

    Thanks for updating the issue! It now clearly follows the feature request template with a well-described problem, real data supporting the need, and a concrete proposed solution. The needs:compliance label has been removed. We appreciate the thorough write-up!

  4. github-actions commented on Sep 7, 2026

    @github-actions
    Contributor

    To stay organized issues are automatically closed after 60 days of no activity. If the issue is still relevant please open a new one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions