Repository navigation
Consider enabling panic_on_oom #324
Description
Activity
This is very interesting and thanks for bringing to my attention that for the X11 XScreenSaver that:
You would think that the OOM-killer would pick the process using the most memory, but most of the time it seems to pick the process that would be most comically inconvenient, such as your screen locker, or crond (8).
When it comes to the
sysctlproposal of now presumably using by default the strictestvm.panic_on_oom=2setting which would cause the kernel to panic in any OOM scenario I have a few notes. Currently, we are using the kernel defaultvm.panic_on_oom=0whilevm.panic_on_oom=1could also be set as it is less strict as uses pre-set mempolicy/cpusets and so since we are not going to mess with these there really is not that much point in using this middle ground.One related setting we should keep in mind is that we are currently set by default
kernel.panic=-1and so the system instantly reboots upon any single panic.When it comes to this proposal, in theory I agree that this is a great idea as bypassing a lock screen due to nothing more than a OOM situation is wild.
Is this concern still present on Wayland based systems and lock screens? This would be important as forcing OOM-based instant reboots on unaffected users by default would be inconsiderate and ruin their experience. If it turns out that only X11 users are impacted, while we do not have telemetry, it would be nice to know if this is still the majority and so changing the default to something this high impact is merited.
Now let us suppose both X11 and Wayland users are identically impacted for argument sake. The threat model for this requires physical presence as that is what the lock screen protects against. Therefore, we are talking about basically bypassing a single passphrase when the computer is on and with the OS drive already decrypted.
My concern is that setting opens the door to not only DoS attacks that inflate memory but also regular user desktop usage. The first is obvious as any malicious process can force the computer to repeatedly crash on command just by filling memory.
The second is that this would place quite a tight noose around especially VMs with small memory allocations and so rely on the OOM killer to keep the system functional. Note that even bare metal desktop users with full access to all memory would also be negatively impacted by this if they ever reach maximum usage. It is unclear to me if the OOM killer is activated when very large sized items are cached into swap after memory is already filled for file transfers but if so this is even more a concern. Overall this change without very loud advertisement to all users has catastrophic potential to break the user experience and they will not know why as we have all debugging disabled by default.
I think this default seeing change is only appropriate for computers that are performing pre-determined fixed 'assembly line' tasks at all times and so no deviations from this are expected as allocated memory has been deemed sufficient. Once you introduce the randomness of what any human might use the computer for at any give time, causing instant reboots when OOM might be a little to over zealous in our hardening goals.
Therefore as a middle ground maybe we should consider this an optional
sysctl? It is essentially equivalent to the most extreme as possible ban on swap to the point at which the currently setvm.swappiness=1looks very weak indeed.The OOM killer doesn't prevent swapping, it only kicks in when the system legitimately has no more virtual memory left and the options are either "kill something at random" or "crash". In practice on most desktop systems, the Linux kernel tries very very hard to not need to invoke the OOM killer, and part of this trying involves extremely resource-intensive swapping of memory. Ultimately this "thrashing" results in the system hanging entirely, and typically once a system starts thrashing it's dead and won't re-enter a usable state because the kernel will always keep its head barely above water without invoking the OOM killer. Thus I don't see this causing usability issues, once a system or VM thrashes it's basically dead anyway, and this won't avoid the thrashing stage. But if something does maliciously fill memory in an attempt to get the screen locker or compositor killed, this would help.
That makes sense, thanks for the details.
Do you think this should be enabled by default from the start or optional (which can then later of course be enabled by default)?
Also do we know if this issue is applicable to Wayland or does that not matter since you do not expect the setting to cause any usability issues?
Should be enabled by default.
I don't think this is related to Wayland, X11 or getty. That's all user space. But OOM is kernel space.
A general Kicksecure improvement would be to implement measurements to avoid entering a state of trashing and OOM.
Related:
Also do we know if this issue is applicable to Wayland or does that not matter since you do not expect the setting to cause any usability issues?
It is applicable to Wayland. Wayland has measures to keep the screen from becoming unlocked if the screen locker gets killed, but that doesn't prevent the compositor itself from getting killed, which depending on the situation could drop one to a logged-in shell. In Kicksecure's default configuration this probably won't happen because people won't be running
startlxqtwaylandfrom a TTY all the time, but they could.- added a commit that references this issue
on Oct 10, 2025 This has been implemented in git.
Just for the record I am not currently sure that enabling this by default is the best idea give we are already using
kernel.panic=-1meaning that the system will instantly reboot.As a start, I would have gone for the optional approach with the benefits of enabling being clearly documented as we have done with many other settings.
Even in the KSPP issue you posted, Kees Cook is also concerned about userspace being able to DoS the system by filling up both memory and swap in unexpected ways.
Even though the OOM killer resides in kernel space, it is entirely responsive to what goes on in userspace as it is only activated in a OOM situation.
Regardless, I respect your decision and can also see the benefits when it comes to preventing a state of thrashing.
Edit:
Additionally, should this setting also be disabled bydebug-misc?Edit:
Additionally, should this setting also be disabled bydebug-misc?Probably, yes.
Just for the record I am not currently sure that enabling this by default is the best idea give we are already using
kernel.panic=-1meaning that the system will instantly reboot.At least to me, this feels like less of a DoS than locking up on OOM. If someone fails to set up a password on their device and leaves autologin enabled, then this would allow an OOM to effectively unlock the screen by rebooting the system, but one can unlock the screen by sending SIGUSR1 to swaylock if they only have software access, and if an attacker has physical access and the user hasn't set a password, then there's no need to bypass the screen locker, just hit "Enter" and you're in.
I would generally agree that this creates a greater DoS risk. That's what the OOM killer exists to prevent. I suppose one has to determine which is best for them, random processes getting killed, system lockup, or system reboot. There are instances where the first one is best, but I think the last one is probably a decent opinionated default for most security-conscious users.
(A future idea might be to make it so that the system will reboot on OOM only when the screen is locked, and allow the OOM killer to do its thing otherwise. That would give the best of all worlds in some ways.)
I am sure now that this isn't the best idea as of right now. A trusted Qubes-Whonix 18 user reported privately that after starting Tor Browser, their VM restarted. After that, Tor Browser is no longer starting as it, I guess, does not handle forced termination at specific times gracefully. This is a horrible user experience.
Therefore I will revert this now.
This should get re-enabled by default at some point but before that we need better user space OOM handling (and maybe fixing other unknown RAM related issues).
Reacted by Aaron Rainbolt- added a commit that references this issue
on Nov 9, 2025 I agree that this was the correct decision. I had not tested this specific use case myself but scenarios like this is what I predicted would occur as enabling this by default would have left us exposed to all manner of unknowns insider user space (even though this is a kernel space feature).
Inside #327, I have provide included more details regarding this
sysctland our logic.This should get re-enabled by default at some point but before that we need better user space OOM handling (and maybe fixing other unknown RAM related issues).
Yes, once we can account for the randomness attached to any desktop users' workflow this should be revisited.
I wonder if the
panic_on_oomoption doesn't quite do what it says "on the tin". I was under the impression that it would panic the system if and only if a condition was hit where the OOM killer would be activated. But based on some feedback from alimj in the #qubes-public room on Matrix, and from this report, it sounds to me like the system is dying just when memory is full, before resorting to swapping. If that's the case, then this feature should probably be off permanently (or something changed in the kernel to make it more useful).This should get re-enabled by default at some point but before that we need better user space OOM handling (and maybe fixing other unknown RAM related issues).
Yes, once we can account for the randomness attached to any desktop users' workflow this should be revisited.
This could be fixed using a privileged daemon that has configurable OOM and freezes/terminates processes before the kernel OOM is triggered. Such a program might even already exist. Needs research.
Context: https://forums.whonix.org/t/screen-locker-in-security-can-we-disable-these-at-least-4-backdoors/8128/14
Should we enable
panic_on_oomto make sure that security features such as the screen locker, kloak, emerg-shutdown, etc. are not arbitrarily terminated when the system starts running out of memory?