Publish CAIRO input and output files on public S3 (CIEC-11c) #334

What

Answer CIEC-11c by making the CAIRO input and output files behind the Velez Direct Testimony publicly readable on S3, and give CIEC links that work without AWS credentials.

CIEC asks for all CAIRO input and output files not already provided in Exhibits JPV-3 and JPV-4 or in response to CIEC-1.

Suggested target: Nov 10. Responses are due Nov 12.

Why

The draft plan is to give CIEC S3 links, but the data is private today, so those links wouldn't open. Everything public is effectively permanent once it's been downloaded, so the release has to expose only what we intend.

How (optional)

Current state (read-only check, Nov 5)

• data.sb is fully private: all four Block Public Access settings are on, and there is no bucket policy. Account-level settings weren't checked.
• The bucket also holds unrelated prefixes, including eversource/, bge/, rie/, ny_psc/, il_npa/ and tools/. Don't make the bucket public as a whole.
• The testimony batch is ct_20261015_a (reports/ct_hp_rates/batch_config.yaml, RDP ref e97a3ca). Outputs are at:
  • s3://data.sb/switchbox/cairo/outputs/hp_rates/ct/all_utilities/ct_20261015_a/ (40 objects, 96 MiB)
  • s3://data.sb/switchbox/cairo/outputs/hp_rates/ct/ct_eversource/ct_20261015_a/ (1,602 objects, 6.2 GiB, including a .runs/ folder)

Decide the approach

• Option A (recommended): copy the files to a new, dedicated public bucket. data.sb stays private, and a mistake can't expose anything else.
• Option B: a read-only bucket policy scoped to specific prefixes on data.sb. This requires turning off bucket-level Block Public Access on a bucket holding unrelated data, so one bad policy edit could expose more than intended.

Decide the scope

• List which inputs CIEC's request covers, which may include ResStock loads, marginal costs, tariffs and scenario configs. Inputs live outside the output prefixes above.
• Decide whether all scenario runs are in scope, or only those behind the testimony.
• Exclude anything already provided if counsel is comfortable with that.

Review before anything goes public

• Check every file for NDA-covered or CEII data, personal data and credentials. Inspect .runs/ and the JSON configs in particular.
• Check that license terms allow redistribution, especially tariff data retrieved from Arcadia.
• Get sign-off from JPV and counsel on the file list and the wording of the response.

Publish correctly

• Grant object read only. Don't allow public listing or writes unless there's a deliberate reason for listing.
• Copy files so the published set can't change later, for example with versioning or a dated, read-only prefix. It should match what the testimony used.
• Share HTTPS links, since CIEC may not have AWS tools. Include a manifest with each file's path, description, size and checksum, plus a short README.
• Check expected download costs, and turn on access logging.

Verify

• From a logged-out machine with no AWS credentials, confirm that the manifest links download.
• Confirm that the rest of data.sb is still private, including a sample from eversource/.

Deliverables

• Approved file list and access approach
• Public files with a manifest and README
• Verification that only the intended files are public
• Draft response for CIEC-11c, with links pinned to the ct_eversource_submission tag where it cites code