Add --idle-timeout to shutdown code-server when there isn't an active connection

@code-asher

If there hasn't been an active connection to code-server in X minutes, we should exit with status 0.

This would replace the heartbeat file (#1115) as a more friendly option.

No it does NOT replace the task of the heartbeat.

The heartbeat to an external file, allows an external monitor to shutdown the docker instance, not shutdown code-server itself. In fact the monitor itself may not even be running on the same machine in a multi-node docker environment.

There can be a lot more to the docker instance than code-server itself, such as an apache web server, or a java script node the user is developing in the isolated environment. A way to determine if the user is still active (via code-server) is an integral part of this, and would be a vastly harder thing to monitor without the availability of the heartbeat file.

In code-server v1 the file was actually a log that updated each time the web client requested an update, letting us know if the user still had a connected web server or not. While the web client was connected (even if the user was away from their terminal), the heartbeat would still update. So in a way heartbeat does not report 'user idle' status but 'connected' status.

So looking at it that way "heartbeat" is slightly different to the true meaning of "idle" as in the user is not really active (making changes, clicks, or typing).

In a more general way. You may not actually want code-server to just shutdown, when not used for a certain length of time. It is only one possible action, in that situation. As mentioned in own case we want to shutdown the docker instance, and not code-server itself.

Now a -idle-timeout is still a good option... And I am certain many people would like to make use of such an option, especially when not using docker containers. But it is not the one needed in my setup!

TLDR... --idle-timeout is a good addition, but it is not a replacement for heartbeat.

@antofthy

@antofthy So instead of checking the heart beat file, you'd just check to see whether code-server or the container has exited successfully.

Can you not do that?

Yes I could, but not as simply...

You can only look at processes from the same node in a docker cluster. As such the check would need to be done by each and every docker node.
You then have to parse the process list, which is always messy, and error prone thing to do.
You also lose information on how idle each container is, which can be important when looking for a time to upgrade the system in some way.

For example with heart beat I can do the following on the master node of cluster, for all containers (docker services)...

Name        Idle  Image        Node 
s1234567 13 secs  web-dev      elf1 
s7654321 18 secs  web-dev      elf1 
s1212121 13 secs  web-dev      elf  
Environments: 3 (3 active)

This shows at this moment I have 3 users using the system, all with a 'web-dev' code server environment (with apache, nodejs software) and they are all active (used system in last 5 minutes)
and only 2 docker nodes are in use at this time (elf, and elf1) That same program also is the one that will shutdown idle docker services after being idle for an hour, when called by cron with the appropriate options.

Time its been idle feedback is lost without heartbeat.

heartbeat is a LOT more versitile!

Why the push to get rid of heartbeat. Is it causing some problem I don't know about?

No push, less features the better.

I understand your concerns, we'll keep both! 🎉

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no activity occurs in the next 5 days.

@nhooyr

@nhooyr your last response sounded like you were going to add an --idle-timeout option, but the current release 3.12.0 only has the heartbeat mechanism, right?

For running code-server in a Docker container, the current situation is very inconvenient: A healthcheck can determine that the age of the heartbeat file exceeds a threshold, but since the main process in a Docker container (PID 1) ignores signals, a healthcheck script has no way to exit the code-server.

#!/bin/bash
now=$(date +%s)
fp=~/.local/share/code-server/heartbeat

# The heartbeat file does not exist before a browser session is connected,
# but we also want to exit if that never happens.
if [[ ! -f $fp ]]; then
    # No session yet.
    # Create the heartbeat file to mark the startup time.
    touch $fp
fi

heartbeat=$(date +%s -r $fp)

# NOTE: This was my original block and doesn't work 👇
#if (( (now-heartbeat) > 300 )); then
#    echo "Heartbeat file older than 5 minutes."
#    # Does not work, because Docker main processes ignore signals: kill -s SIGTERM 1
#    exit 1
#fi

if (( (now-heartbeat) > 300 )); then
    echo "Heartbeat older than 5 minutes. Terminating the code-server process."
    pkill -f code-server
    exit 1
else
    echo "Recent heartbeat detected."
    exit 0
fi

So IIUC there's no easy way to stop the Docker container "from the inside", but an external supervisor is needed. Please advise if there's a way.

That is correct, only the heartbeat mechanism exists at the moment. Could the healthcheck script kill code-server instead? Maybe with a pkill -f. Or perhaps the healthcheck script could wrap/spawn code-server so it can store code-server's PID.

[...] Could the healthcheck script kill code-server instead? Maybe with a pkill -f.

Yes, that works! I don't quite understand why, but fair enough. Thanks a lot!
I updated the code block above according to your solution.

The only way to stop a docker container from the inside (unless special docker control is also given - but that is a special case) is to kill off whatever process the container is waiting on. That is the foreground 'START' process.

Basically that means you need specific knowledge of the set up so as to know what needs to be killed.

The 'heartbeat' file is very useful as if it is mounted from a persistent store (docker volume) it is accessible outside the docker container, where a global supervisory program can handle things.

However a -idle-timeout would be useful, but ONLY works to shutdown a docker container if the container is 'waiting' for code-server.

My problem now is I have one container with BOTH code-server and jupyter notebook running.
The latter does not have a heartbeat file (only a idle timeout), as to do idle properly I would need to idle check BOTH processes, which does not seem possible at this time.

Arrggghhh....

@code-asher

The latter does not have a heartbeat file (only a idle timeout), as to do idle properly I would need to idle check BOTH processes, which does not seem possible at this time.

I'll tag @code-asher to see if he has on ideas on how he would solve this

Is this still a feature that's in the works?

No one is working on this, as far as I know, and it is not on any roadmap. PRs welcome though!

It was added an an alternative to using 'heartbeat'.

The university I work for uses heartbeat to shutdown student virtual docker systems, when users stop using them. It works well, mostly because I can access the heartbeat outside the docker environment. Basically a cron script runs ever 10 minutes looking at active services, to see if heartbeat stopped updating for more than an 1 hour, then shotdown that users docker environment, no matter which machine in the cluster the docker service is running on. It also checks on 'control inputs' and even some web service connection (control server does not open/close connections). The "--idle-timeout" did not provide that level flexibility.

On the other hand there is a bug in that heartbeat stops updating if the user does NOT open new files for editing every so often, (just editing a single file or using console). So even it could use some improvement.

We could use the healthz endpoint to check the status even from outside the VM or container where code-server is running:

As said in the docs' FAQ:

What is the healthz endpoint?

You can use the /healthz endpoint exposed by code-server to check whether code-server is running without triggering a heartbeat. The response includes a status (e.g., alive or expired) and a timestamp for the last heartbeat (the default is 0).

{
  "status": "alive",
  "lastHeartbeat": 1599166210566
}

I just saw this and currently exploring.

Are the heartbeat and automatic stop functions only suitable for serverless?

I understand that the heartbeat is to automatically stop the node service when the code-server has not been used for a long time, and tell the container service in some way that the current pod can be destroyed, so as to save costs.

However, if I deploy the code-server on a server (which will not stop), do I actually not use the heartbeat function? If I don't have to worry about memory consumption for the time being. Because whether the code-server is stopped or not, my server is always running and charging.

If you are happy with the amount of resources code-server uses while idle, then I would say there is no reason to shut it down in your case. If you want to free up those resources while code-server is idle, then it could still make sense to automatically stop it.

How can we "automatically stop" code-server when it is idle? Is there an official way of doing it?

How can we "automatically start" code-server when it is active?

How can we do these in Linux for example?

There is no official way to stop code-server, but looks like #7177 would add it. Otherwise you need to add your own service that checks code-server's heartbeat and shuts it down.

There is no official way to start code-server either, but you could use socket systemd activation or roll your own service. Unless you are spawning code-server for many users and need to worry about the resource usage though, I am not sure it is worth the effort.

@code-asher

Thank you @code-asher :)

Navigation Menu

Repository navigation

Add `--idle-timeout` to shutdown code-server when there isn't an active connection #1636

What is the healthz endpoint?

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Add --idle-timeout to shutdown code-server when there isn't an active connection #1636

Description

Activity

antofthy commented on May 16, 2020

antofthy commented on May 16, 2020

nhooyr commented on May 17, 2020

antofthy commented on May 18, 2020

antofthy commented on May 18, 2020

nhooyr commented on May 18, 2020

stale commented on Oct 27, 2021

michaelosthege commented on Nov 24, 2021

code-asher commented on Nov 24, 2021

michaelosthege commented on Nov 24, 2021

antofthy commented on Jan 10, 2022

jsjoeio commented on Jan 10, 2022

eliliam commented on Aug 11, 2023

code-asher commented on Aug 11, 2023

antofthy commented on Aug 14, 2023

doncadavona commented on Aug 14, 2023

What is the healthz endpoint?

yiliang114 commented on Jan 11, 2025

code-asher commented on Jan 13, 2025

doncadavona commented on Jan 14, 2025

code-asher commented on Jan 22, 2025

doncadavona commented on Jan 29, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Add `--idle-timeout` to shutdown code-server when there isn't an active connection #1636