test: guard GCS bucket remap against drift from idc-index #68

Problem

source=gcs URLs (POST /v3/cohort/manifest.txt, MCP get_cohort_urls) are produced by remap_bucket_for_gcs in src/idc_api/core/services/manifest.py, which uses a hand-copied AWS→GCS bucket table (_AWS_TO_GCS_BUCKET) mirroring idc-index's private IDCClient._replace_aws_with_gcp_buckets. Nothing ties the two together: tests/test_rest.py::test_remap_bucket_for_gcs only checks our copy against itself.

If a future idc-index release adds or renames a bucket, our table silently goes stale and GCS users get URLs that 404 (the AWS bucket name does not exist on GCS for the renamed buckets).

Current state (verified against idc-index v25 / idc-index 0.13.3)

The mapping is correct today. The index has exactly three buckets, and one sample series from each was confirmed present on both clouds via anonymous listing:

For the two renamed buckets, the unmapped (AWS) name on storage.googleapis.com returns 404, so the remap is required.

Proposed fix

Add a drift test: for every distinct aws_bucket in the index, assert that remap_bucket_for_gcs agrees with IDCClient._replace_aws_with_gcp_buckets. An idc-index bump that changes the mapping then fails CI instead of shipping broken GCS URLs. (It touches a private idc-index method, but that is exactly the behavior we mirror; alternatively, ask upstream to expose the mapping publicly and import it.)

Related, lower priority

GCS is only offered on the text manifest. The JSON cohort/manifest rows (SeriesManifestRow: aws_bucket, series_aws_url) and download_info.manifest_preview are always AWS. They are correctly labeled, so this is not a bug, but a source option there (or a GCS URL field) could be considered.

_{🤖 Drafted with Claude Code.}