Skip to content

graphBatchLoad's follower refusal cannot name a dialable address: HAServerPlugin exposes no gRPC routing table #6091

Description

@lvca

Carried over from #6083 item 3, the one item of that batch not addressed by #6089. It was split out because it changes a public configuration surface and needs a real multi-node test, neither of which belonged in a defect-fix PR.

Problem

ArcadeDbGrpcService.graphBatchLoad refuses a bulk load on a follower of a replicated database — correctly, since the bulk path mutates shared state only the leader can serialize (#4122). To let the caller redirect, it names the leader. But HAServerPlugin exposes only the leader's HTTP address (getLeaderAddress()), so the refusal message currently has to say:

Reconnect to the leader at '<host:httpPort>' (HTTP address; use its gRPC port) and retry

A client that wants to retry automatically must already know the deployment's port-mapping convention. There is no way for it to learn the leader's gRPC address from the cluster.

Precedent to follow

getBoltRoutingTable() is exactly the shape needed: a protocol-specific routing view (BoltRoutingTable(writer, readers)) derived from a single getLeaderId() read, so a concurrent leader change cannot make the writer and reader sets mutually inconsistent. RaftHAServer.resolveBoltAddress backs it, reading a per-peer bolt: field from the object form of HA_SERVER_LIST and falling back to "peer's Raft host + this node's local Bolt port" with a one-time WARNING when the field is absent.

A getGrpcRoutingTable() alongside it — or a generalisation of the existing record over a protocol enum, which may be the better shape now that there would be two — would let the refusal name an address the caller can dial.

Scope

  1. HAServerPlugin: new default method returning null when HA is inactive (same contract as getBoltRoutingTable()).
  2. RaftHAServer / RaftHAPlugin: resolveGrpcAddress mirroring resolveBoltAddress, plus a grpcAddresses map on ParsedPeerList.
  3. RaftPeerAddressResolver: a grpc: field in the object form (host:{raft:2434,http:2480,bolt:7687,grpc:50051}). Object form only, as with bolt — the positional colon form has no room left.
  4. ArcadeDbGrpcService.graphBatchLoad: use it in the refusal, keeping the current HTTP wording as the fallback when no gRPC address resolves.

Decisions worth making deliberately

  • Generalise or duplicate? A second single-protocol routing record invites a third. Worth deciding now whether BoltRoutingTable becomes RoutingTable(protocol, writer, readers), which touches the Bolt ROUTE path.
  • Derive-only vs. explicit config. The derive-from-local-port fallback is correct for homogeneous deployments (a K8s StatefulSet) and wrong for heterogeneous ones, which is why bolt is declarable. Shipping the derive-only half first would avoid the config change but leaves the heterogeneous case silently wrong — the same reason bolt: exists.
  • Whether other gRPC RPCs that refuse on a follower should use the same table, rather than only graphBatchLoad.

Testing note, recorded in #6083

The auth-before-leader-check ordering here is unobservable on a single server: the channel-level GrpcAuthInterceptor short-circuits a credential-less call, and forcing the follower branch on a single node NPEs on a null ha. This needs a real follower in a cluster, so it lands in the ha-integration-tests lane.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions