Repository navigation
Custom Endpoint Support #1630
Description
Let's add support for any custom endpoint across the board. We already have support for Datastore, PubSub and (soon) Bigtable. For the most part supporting them has just been some minor configuration changes, so we should be able to support it everywhere.
We should have a setting, e.g. options.apiEndpoint = 'http://localhost:8888 as well as environment variables.
- GOOGLE_CLOUD_BIGQUERY_ENDPOINT
- GOOGLE_CLOUD_BIGTABLE_ENDPOINT
- GOOGLE_CLOUD_COMPUTE_ENGINE_ENDPOINT // GCE_ENDPOINT?
- GOOGLE_CLOUD_DATASTORE_ENDPOINT
- GOOGLE_CLOUD_DNS_ENDPOINT
- GOOGLE_CLOUD_LANGUAGE_ENDPOINT
- GOOGLE_CLOUD_LOGGING_ENDPOINT
- GOOGLE_CLOUD_PREDICTION_ENDPOINT
- GOOGLE_CLOUD_PUBSUB_ENDPOINT
- GOOGLE_CLOUD_RESOURCE_MANAGER_ENDPOINT
- GOOGLE_CLOUD_SPEECH_ENDPOINT
- GOOGLE_CLOUD_STORAGE_ENDPOINT
- GOOGLE_CLOUD_TRANSLATE_ENDPOINT - translate: support GOOGLE_CLOUD_TRANSLATE_ENDPOINT env var #1674
- GOOGLE_CLOUD_VISION_ENDPOINT
Reacted by Tobias Furuholm, Chris Hiestand and Josh Dutton
Activity
[-]Emulator Support[/-][+]Custom Endpoint Support[/+]stephenplusplus commentedon Sep 27, 2016
Small title change to make sure it's clear we want to support "http://any-uri" as opposed to only official service emulators.
jmuk commentedon Mar 7, 2017
@danoscarmike might also be interested in the idea like this.
rmanyari commentedon Aug 12, 2017
I'm would be super excited to see this feature merged in the next release (custom endpoints are very very helpful for people who operate behind forward proxies).
I can open a PR in the next days that puts a function very similar to {Datastore|PubSub}.prototype.determineBaseUrl_ in common.utils and then use it across the different services.
Rodrigo
lukesneeringer commentedon Aug 14, 2017
Are these individual environment variables? That seems lousy. I would much rather have this be a parameter that can be passed to clients.
Individual environment variables means we are always chasing the full list of APIs; a consistent parameters is a one-and-done.
rmanyari commentedon Aug 14, 2017
PubSub does expose a parameter (vs. other services that use environment variables):
https://github.com/GoogleCloudPlatform/google-cloud-node/blob/a64ad81517045196cf5a3f468ea15aad1e2c25da/packages/pubsub/src/index.js#L83
I prefer the parameter approach as well. @lukesneeringer any other comments on the suggested changes? I can start writing some code for GCE
lukesneeringer commentedon Aug 14, 2017
The other primary comment I have is whether or not the client objects wrap a common object (I think they do). If they do, then the best answer is to place the
apiEndpointparameter there. If not, then it would need to be applied to each object.It looks like it needs to be explicitly passed -- see here in Spanner, for instance. That value just needs to change to
options.apiEndpoint || 'spanner.googleapis.com'. Similar in every gRPC-based API.stephenplusplus commentedon Aug 14, 2017
I believe this was originally a @jgeewax request to have env vars for each. I think the nice part of env vars is you don't have to modify your code depending on where your app is running.
Either way, we should expose at least an override via
options.apiEndpointfor all APIs. Thank you for volunteering @rmanyari!lukesneeringer commentedon Aug 14, 2017
You can always have your code set a value based on your own environment variable. Not so much the converse. If you are in a situation where environment variables are inconvenient, there is not much you can do.
rmanyari commentedon Aug 15, 2017
Silly question. I'm extracting determineBaseUrl in
./packages/commonand./packages/common-grpc. Before calling common.utils.determineBaseUrl in./packages/compute/src/index.jsI have to do:to pickup my local changes. I'm not 100% sure how the publishing process works for this project. Should I submit a PR for the changes in common and common-grpc first and then bump the version in the other service packages before submitting the "actual" changes or is there a clever way of building everything that I'm missing out?
stephenplusplus commentedon Aug 15, 2017
It's fine to put it in one PR. We don't have automated releases, so I'll be manually testing / publishing anyway :)
Just a tip, we have an npm run script that you can run from the root directory of the project:
4 remaining items
stephenplusplus commentedon Mar 15, 2018
We have 3 different API types:
Not supported
Already supported!
new Firestore({ baseUrl: 'http://my-custom-endpoint' })new Datastore({ servicePath: 'http://my-custom-endpoint' })options.apiEndpointis what our REST APIs have used before, and I'm not sure if any still do.options.baseUrlis what we used incommon-grpc. I believe Firestore is the only API still using that module for API requests.options.servicePathis a term right from gRPC, and thus GAPIC. We transparently accept it, as all user options are passed right through to GAPIC.Does anyone have thoughts on how we should support this concept of a custom API endpoint?
alexander-fenster commentedon Mar 15, 2018
I might be a little bit out of context, but having a possibility to override the endpoint via the environment variable might be valuable (e.g. for testing purposes). I don't think it's a high priority request though.
JustinBeckwith commentedon May 23, 2019
So... good news! This is now a high priority thing 🙃 Let's track it over in #2928
stephenplusplus commentedon May 23, 2019
This issue also covered env vars to change the API endpoint, which was a Googler-requested feature at one point. I can see the value for the user, but it would also be the same as a small modification the user can make themselves with the same net effect:
JustinBeckwith commentedon May 23, 2019
That's a really good question :) I've never heard this request. Is this primarily a testing thing? (cc @broady)
chore(deps): update dependency lru-cache to v10.4.3 (#1630)
docs(samples): add example tags to generated samples (#1630)
chore(deps): update dependency lru-cache to v10.4.3 (#1630)
chore(deps): update dependency @types/node to ^20.16.1 (#1630)