Design: operating system updates and scheduled node reboots #8260

Software Center updates NethServer core and applications, but the Distribution (operating system) of each node is left to the administrator, with commands like dnf update or apt upgrade. Kernel security fixes take effect only after a reboot, and Rocky Linux minor releases (for example from 9.6 to 9.7) need both the update and a reboot. Today the cluster-admin UI does not show whether a node needs operating system updates or a reboot, and cannot schedule them.

The purpose of this issue is the UI design of operating system updates and node reboots, so administrators can keep nodes secure without logging in to each one.

Proposed solution

Design the cluster-admin pages and workflows to:

• show, for each node, the Distribution name and version, uptime, kernel version, the number of available updates, security updates first if possible, and the last update result and time
• review operating system updates on a node (list of package name and version), and update it alone or all nodes, using the existing update-os action
• alert when a node needs a reboot, for example after a kernel update (decide the alert rule, e.g. 30 days)
• alert when a node Distribution is not updated (e.g. last update older than 30 days)
• reboot a node immediately, or by policy. Schedule a reboot "never" or at certain times of the day, week, month, with a warning about applications running on it
• a worker node reboot schedule may be inherited from leader or be different
• a reboot starts like a normal task: it can be canceled in the toast notification
• handle multi-node clusters reboot order. The backend avoids the reboot of leader and workers at the same time
• scheduled reboot handles concurrency with backup and updates. It waits until they're both finished before rebooting the system
• with a subscription, automatic updates also schedule reboot with the node configured policy

The design output is a mockup. Implementation issues follow.

Alternative solutions

• Rely on Cockpit or other existing UIs: they are per node, and do not know about the cluster and its applications.
• Automatic reboots without UI control: not acceptable for sites with critical services, like NethVoice.

Additional context

Related issues:

• Launch update-os action from Software Center: Launch update-os action from Software Center #7933
• Email summary of automatic and available updates: Email summary of automatic and available updates #7934

This issue also covers the former draft card "Report last apply-updates run status": display the outcome (OK or error) and the time of the last automatic update, for example in the Subscription or Status page. Systemd already records it: systemctl show apply-updates --timestamp=unix -p ExecMainExitTimestamp -p ExecMainStatus.

See also

• Operating system updates, admin manual: https://docs.nethserver.org/docs/tutorial/os_updates
• Discussion https://mattermost.nethesis.it/nethesis/pl/heczqdiq4frbm8tonhiaj7euky
• Archived project draft card