feat!: Align Support for Storage Emulation

Today, baseUrl is equivalent to the STORAGE_EMULATOR_HOST, when available. This leads to limitations when using the client. Here's the relevant code snippet:

nodejs-storage/src/storage.ts

Lines 650 to 668 in 1447269

    
           let apiEndpoint = 'https://storage.googleapis.com'; 
        
           let customEndpoint = false; 
        
           // Note: EMULATOR_HOST is an experimental configuration variable. Use apiEndpoint instead. 
        
           const EMULATOR_HOST = process.env.STORAGE_EMULATOR_HOST; 
        
           if (typeof EMULATOR_HOST === 'string') { 
        
             apiEndpoint = Storage.sanitizeEndpoint(EMULATOR_HOST); 
        
             customEndpoint = true; 
        
           } 
        
           if (options.apiEndpoint) { 
        
             apiEndpoint = Storage.sanitizeEndpoint(options.apiEndpoint); 
        
             customEndpoint = true; 
        
           } 
        
           options = Object.assign({}, options, {apiEndpoint}); 
        
           // Note: EMULATOR_HOST is an experimental configuration variable. Use apiEndpoint instead. 
        
           const baseUrl = EMULATOR_HOST || `${options.apiEndpoint}/storage/v1`;

Ideally, STORAGE_EMULATOR_HOST should only be a host (without any path associations, e.g. http://localhost:9199) while the client calls the appropriate path on the host (e.g. /storage/v1). That implementation should look something like this: c75b8b8

This requires coordination with Firebase and updates to Firebase Storage Emulator. Additionally, we should explore practices to ensure reliability and a consistent experience between Firebase and Google Cloud Storage (e.g. code consolidation and integration testing).

refactor!: Revamp apiEndpoint #2400

Background:

The Go GCS client looks like it had some similar issues in the past. There's a fair bit in that repo if you search issues for STORAGE_EMULATOR_HOST, but this ticket is the closest to this and has two PRs linked to fix it: googleapis/google-cloud-go#2476

The changes there look functionally equivalent to what I proposed in #2070, so I'm wondering if the firebase emulator may already have problems with the Go client? I have no experience with the firebase emulator, so I couldn't say for sure.

A thought: as a stepping stone to avoid a "hard break", might it be useful to temporarily permit a second environment variable indicating how to interpret STORAGE_EMULATOR_HOST, with the initial default being the current behavior, and some specific value indicating to use the "new style", and at some point that default changes, and later that second env var is dropped?

Friendly check-in to see if there's any progress on decisions for how this might move forwards?

@mgabeler-lee-6rs

@mgabeler-lee-6rs hey, thanks for the inquiry; we plan to sync with peer teams on this and other issues over the next few months.

Just checking in on status and confirming that this is still of interest.

@mgabeler-lee-6rs

@mgabeler-lee-6rs We have alignment between teams and we're likely to resolve this in the current quarter.

Obviously missed Q2, Q3 maybe? :)

@mgabeler-lee-6rs

@mgabeler-lee-6rs yea, I’m aiming for this half :)

Update: we had a few other high priority features to take care of; however we still have this as a key item to get to shortly.

Navigation Menu

Repository navigation

feat!: Align Support for Storage Emulation #2092

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	let apiEndpoint = 'https://storage.googleapis.com';
	let customEndpoint = false;

	// Note: EMULATOR_HOST is an experimental configuration variable. Use apiEndpoint instead.
	const EMULATOR_HOST = process.env.STORAGE_EMULATOR_HOST;
	if (typeof EMULATOR_HOST === 'string') {
	apiEndpoint = Storage.sanitizeEndpoint(EMULATOR_HOST);
	customEndpoint = true;
	}

	if (options.apiEndpoint) {
	apiEndpoint = Storage.sanitizeEndpoint(options.apiEndpoint);
	customEndpoint = true;
	}

	options = Object.assign({}, options, {apiEndpoint});

	// Note: EMULATOR_HOST is an experimental configuration variable. Use apiEndpoint instead.
	const baseUrl = EMULATOR_HOST \|\| `${options.apiEndpoint}/storage/v1`;

feat!: Align Support for Storage Emulation #2092

Description

Activity

mgabeler-lee-6rs commented on Oct 21, 2022

mgabeler-lee-6rs commented on Jan 3, 2023

danielbankhead commented on Jan 3, 2023

mgabeler-lee-6rs commented on May 1, 2023

danielbankhead commented on May 1, 2023

mgabeler-lee-6rs commented on Aug 1, 2023

danielbankhead commented on Aug 7, 2023

danielbankhead commented on Jan 29, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions