Feature Request: Implement waitUntil-like Functionality in Knative/Serving

Describe the feature

Context

In Vercel Functions, the waitUntil() method is a highly valuable feature. It allows developers to enqueue asynchronous tasks to be performed during the lifecycle of a request. These tasks do not block the response but should complete before the function shuts down. This is particularly useful for tasks such as logging, sending analytics, or updating a cache, which can be done after the response is sent, ensuring the response is not delayed by these operations.

Reference

Here is a description of the waitUntil method in Vercel Functions:

The waitUntil() method enqueues an asynchronous task to be performed during the lifecycle of the request. It doesn't block the response, but should complete before shutting down the function.

It's used to run anything that can be done after the response is sent, such as logging, sending analytics, or updating a cache, without blocking the response from being sent.

The package is supported in Next.js (including Server Actions), Vercel CLI, and other frameworks, and can be used with the Node.js and Edge runtimes.

https://vercel.com/changelog/waituntil-is-now-available-for-vercel-functions

Feature Request

I would like to request a similar feature in Knative/Serving. The implementation of a waitUntil-like method would enable asynchronous tasks to run during the lifecycle of a request without blocking the response. This functionality is essential for many use cases, such as logging, analytics, and cache updates, which need to be performed without delaying the response.

Use Cases

Logging: Perform logging operations after the response is sent to avoid delaying the response time.
Analytics: Send analytics data asynchronously to ensure it does not impact the response time.
Cache Updates: Update cache entries asynchronously to enhance performance without blocking the response.

Benefits

Improved Performance: By allowing asynchronous tasks to run after the response is sent, the response time is not impacted by these tasks.
Better Resource Utilization: Asynchronous tasks can be managed more efficiently, reducing the overall resource usage during the request lifecycle.
Enhanced User Experience: Faster response times lead to a better user experience, as the response is not delayed by additional tasks.

Knative Eventing Consideration

While it is possible to achieve similar functionality using Knative Eventing, the setup and maintenance can be complex and time-consuming. Introducing a native waitUntil-like method in Knative/Serving would simplify the implementation process for developers who need asynchronous task execution without the overhead of configuring and managing Knative Eventing.

Conclusion

Implementing a waitUntil-like method in Knative/Serving would greatly enhance its capabilities and align it with the functionality provided by Vercel Functions. This feature would enable developers to perform necessary asynchronous tasks without impacting response times, thereby improving performance and user experience.

Thank you for considering this feature request.

waitUntil is very specific to the underlying application framework.

I'd imagine you could accomplish something similar by calling some function in your code (that doesn't block the response being sent).

You'd have to configure the revision to stay alive longer than the request which can be done via https://knative.dev/docs/serving/autoscaling/scale-bounds/#scale-down-delay and potentially set a TimeoutSeconds value

@dprotaso

@dprotaso

Using scale-down-delay will indeed introduce a delay even if there's no background processing occurring, which isn't optimal for cost-efficiency. Ideally, the delay should only be extended when background tasks are actually being processed.

A practical approach might be to consider a revision 'alive' based on its activity, such as outputting logs. For instance, you could require that logs be emitted at least once every N seconds during background processing.

Alternatively, the queue-proxy could expose a new API that allows it to be notified of ongoing background tasks. This way, you could make periodic calls (e.g., every 5 seconds) using curl or similar tools to indicate that the revision is still active during the background processing.

This issue is stale because it has been open for 90 days with no
activity. It will automatically close after 30 more days of
inactivity. Reopen the issue with /reopen. Mark the issue as
fresh by adding the comment /remove-lifecycle stale.

/remove-lifecycle stale

This issue is stale because it has been open for 90 days with no
activity. It will automatically close after 30 more days of
inactivity. Reopen the issue with /reopen. Mark the issue as
fresh by adding the comment /remove-lifecycle stale.

/remove-lifecycle stale

Thinking about this -- Knative only really has visibility into network requests in flight. It might be possible to use chunked encoding to send back all the content as it completes, but only send back the closing chunk after the waitUntil method completes. Otherwise, scale-down-delay provides a bit of the mechanism needed, though with some additional "hangover".

Note that Knative does set terminationGracePeriodSeconds on the container to the Revision timeout to ensure that the user container has enough lifetime to flush out all the currently-ongoing requests when the Pod is terminated. I haven't tested whether it will terminate the Pod early if there are no more requests in flight, but I'm not sure whether queue-proxy is in a position to do that.

With respect to the other options:

Knative doesn't see / capture the application logs -- those go to the container runtime independent of the queue proxy or other Knative machinery. Additionally, some applications log in the background on an ongoing basis, so this cure might be worse than the disease.
Feel free to propose an additional API for recording that there is some amount of background work ongoing. One thing to keep in mind is that "requests in flight" serves as both a liveness status as well as measure of load, so you'll need to address how the load from background work is measured.

(Not to toot my own horn, but I talk about some of this in chapter 8 of Building Serverless Applications on Knative, if you want to read more of my thoughts.)

Currently, since Knative can only monitor request processing status, it is difficult to properly evaluate the progress and load of background tasks. Therefore, we propose a mechanism where the application exposes metrics and Knative references this metrics information. This provides flexibility to not only monitor background tasks but also add other items for monitoring in the future.

1. Example Metrics Format

For instance, we could adopt the Prometheus Exposition Format to express background task states as follows:

# HELP background_tasks_total Current number of background tasks
# TYPE background_tasks_total gauge
background_tasks_total{status="running"} <value>
background_tasks_total{status="pending"} <value>
background_tasks_total{status="completed"} <value>

With this format, we can return values for each task state (running, pending, completed), and Knative can understand task execution status based on these values. Additionally, by defining additional indicators such as processing time and error counts as needed, more detailed monitoring becomes possible.

2. Proposal for Generic Spec

Looking ahead to the possibility of referencing other metric items in the future, not just background tasks, we propose defining the spec generically as follows. Here, we group the entire metrics monitoring functionality into a single "monitoring" section and allow specification of various options:

spec:
  metrics:
    monitoring:
      enabled: true
      # endpoint: Application endpoint that exposes metrics
      # *Optional. Default is "/metrics" if not specified.
      endpoint: "/metrics"
      # scrapeInterval: Interval for fetching metrics
      # *Optional. Default is "30s" if not specified.
      scrapeInterval: "30s"

With this configuration, Knative can periodically fetch metrics information from the specified endpoint and use it as a basis for various monitoring and scaling decisions. Furthermore, the design allows for easy expansion in the same format when new metric items become necessary in the future.

@evankanderson

@evankanderson
Thank you for sharing the details about your book—it sounds absolutely fantastic, and I’m really looking forward to studying it in depth.
On a personal note, publishing with O’Reilly has long been one of my dreams, so hearing about your work truly motivates me even more. Your contributions to the community are not only insightful but also genuinely inspiring.
Thanks again for the guidance and for being such a great role model!

This issue is stale because it has been open for 90 days with no
activity. It will automatically close after 30 more days of
inactivity. Reopen the issue with /reopen. Mark the issue as
fresh by adding the comment /remove-lifecycle stale.

/remove-lifecycle stale

This issue is stale because it has been open for 90 days with no
activity. It will automatically close after 30 more days of
inactivity. Reopen the issue with /reopen. Mark the issue as
fresh by adding the comment /remove-lifecycle stale.

/remove-lifecycle stale

Navigation Menu

Repository navigation

Feature Request: Implement waitUntil-like Functionality in Knative/Serving #15345

Describe the feature

Context

Reference

Feature Request

Use Cases

Benefits

Knative Eventing Consideration

Conclusion

1. Example Metrics Format

2. Proposal for Generic Spec

8 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Feature Request: Implement waitUntil-like Functionality in Knative/Serving #15345

Description

Describe the feature

Context

Reference

Feature Request

Use Cases

Benefits

Knative Eventing Consideration

Conclusion

Activity

dprotaso commented on Jun 24, 2024

kahirokunn commented on Jun 25, 2024

github-actions commented on Sep 23, 2024

kahirokunn commented on Sep 23, 2024

github-actions commented on Dec 23, 2024

kahirokunn commented on Jan 12, 2025

evankanderson commented on Feb 10, 2025

kahirokunn commented on Feb 10, 2025

1. Example Metrics Format

2. Proposal for Generic Spec

kahirokunn commented on Feb 10, 2025

github-actions commented on May 12, 2025

8 remaining items

github-actions commented on Nov 12, 2025

github-actions commented on Feb 11, 2026

github-actions commented on May 14, 2026

kahirokunn commented on May 14, 2026

github-actions commented on Aug 13, 2026

kahirokunn commented on Aug 13, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions