Repository navigation
[FEATURE] Move the GCS object-store backend out of core #3675
Description
The GCS backend is still compiled into the core object-store plugin. Remove it once a GCS plugin image built outside core is published and selected by the GKE Component, leaving the plugin gRPC interface as the boundary.
Tracks "Move the GCS object-store backend out of core" in the Provider-Neutral Core workstream of the Extensible Install Roadmap.
Current state (0b635b80)
- GCS code:
internal/objectstore/gcs.go,internal/objectstoreplugin/gcsboundary,internal/objectstoreplugin/gcsissuer,pkg/objectstorage/gcs*.go. - GCS-only flags in
cmd/objectstore-plugin/main.go(--transfer-service-account,--exchange-*). - GCP modules in
go.mod(cloud.google.com/go/storage,google.golang.org/api). - In flight: ate-setup: install a provider's object store plugin with --component and storage.backend=external #3651 installs an external plugin (
storage.backend=external). GCS snapshot tokens are minted locally, so large clusters no longer hit the STS quota #3150, Share GCS HTTP/2 connections across actor snapshot transfers in objectstore-plugin #3588 and Refuse actor JWTs over 8 KiB in the GCS exchange and trust only the CA file for .svc.cluster.local issuers #3650 still change the in-tree GCS code. The direction in objectstore-plugin: Azure Blob Storage backend #3381 is that new backends are out-of-tree plugins.
Required changes
- Port or land the open in-tree GCS changes, then hand the code to the external plugin repository (ai-on-gke/substrate-gke Build and publish the GCS object-store plugin image outside core ai-on-gke/substrate-gke#159, or agent-substrate-plugins).
- Delete the GCS code, the GCS-only flags, and the GCP modules from core.
- Decide whether the S3-compatible backend stays in core as the generic default. kind CI uses it.
Dependencies: the object-store plugin interface contract and conformance suite (#3665); #3651; ai-on-gke/substrate-gke#159 published and selected by the GKE Component.
Breaking change: yes. GCS users must use the external image. Target: before 1.0.
Enforcement
- The core plugin image has no
cloud.google.com/go/storagedependency. - The external image passes the conformance suite.
Parent Issue: #3161
Activity