You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The ate-apiserver today is statically sized. We default to two replicas. This task is to explore how to use HPA to automatically scale it based on demand. The ate-apiserver is designed to scale horizontally as any replica can theoretically handle any request (no sharding or affinity is needed).
This task is to explore what metrics are the best candidates for scaling (cpu utilization, pgx connection pool utilization, etc), how to properly tune HPA targets, and how to plumb everything together.
Horizontal scaling stops improving throughput after a shared dependency (such as PostgreSQL CPU or connection limits) reaches capacity. Size your system infrastructure components and database together.
Each postgres replica opens up to poolMaxConns + 6 connections (owner 2, watch 3, readiness 1, in cmd/ateapi/internal/store/atepg/atepg.go). The sum across replicas must stay under max_connections, so the HPA maxReplicas should be floor((max_connections - reserved) / (poolMaxConns + 6)).
minReplicas >= 2, to keep today's availability
Candidate metrics: CPU utilization and RPC rate per pod.
Activity
laoj2 commentedon Nov 6, 2026
Learnings from https://docs.cloud.google.com/kubernetes-engine/docs/how-to/design-substrate-infrastructure#size-system-infra-components:
poolMaxConns + 6connections (owner 2, watch 3, readiness 1, in cmd/ateapi/internal/store/atepg/atepg.go). The sum across replicas must stay undermax_connections, so the HPAmaxReplicasshould befloor((max_connections - reserved) / (poolMaxConns + 6)).minReplicas >= 2, to keep today's availability