Use `slub_debug=FZ`?

[cynicsketch]

https://tails.net/contribute/design/kernel_hardening/
https://gitlab.tails.boum.org/tails/tails/-/issues/19613
https://kspp.github.io/Recommended_Settings

slub_debug is not apparently used in Kicksecure.

Tails and KSPP, however, do recommend using slub_debug=FZ, which still used in Tails to this day.

The consensus is that slub debugging is not generally harmful because the "information leak" is only to root when kernel lockdown is enabled, and that it therefore doesn't matter that kernel pointer hashing is disabled because root should never be compromised.

Concerns of risk of slub debugging would therefore be overstated.

Not sure about any other contraindictions, though.

[raja-grewal]

Hello,

Thanks for the suggestion!

"information leak" is only to root when kernel lockdown is enabled

While the unhashed kernel addresses would leak to root, I am not sure whether it matters if kernel lockdown mode is enabled or not.

A bit of history first, slub_debug=FZP was added in 02e8888, this was then changed be slub_debug=FZ in bd31b40 (as we upgraded Debian), I then disabled slub_debug=FZ in f572332, then I refactored its presentation in a33d4cd, and finally I removed it in 48e1ac4.

I think the argument rests on the significance of enabling kernel pointer hashing which requires these debugging parameters to be disabled (which is the Linux kernel default) versus the security benefits which are currently advocated by KSPP.

Recall that theses are designed to be debugging options, not security options and so if in the future more debugging functionality is added it may lead to problems. Enabling slub_debug=FZ would require people to keep vigilant on what changes are made moving forward (or at the very least regularly checking with KSPP).

As a first step we could consider potentially reverting 48e1ac4 and adding a link to this discussion.

Also see kernel documentation:
https://www.kernel.org/doc/html/latest/mm/slub.html

[adrelanos]

Great history, thank you, @raja! Yes. At very least we could add a comment. This should be best brought up at KSPP and potentially elsewhere as well.

[adrelanos]

On a second thought, we can defer to KSPP as an authority.

Could you please send a pull request to re-enable?

If someone wishes to argue against slub_debug=FZ, they should bring this up at KSPP or make a compelling argument elsewhere.

[adrelanos]

Also slab vs slub?

[cynicsketch]

Also slab vs slub?

https://www.kernel.org/doc/html/latest/admin-guide/kernel-parameters.html

slab_debug[=options[,slabs][;[options[,slabs]]...] [MM]
...
(slub_debug legacy name also accepted for now)

https://docs.kernel.org/mm/slub.html

"Short users guide for SLUB
...
Some more sophisticated uses of slab_debug:
Parameters may be given to slab_debug ... "

Seems to be just semantics?

[adrelanos]

Done since

• Enable slab_debug=FZ #257

was merged.

[adrelanos]

I am pretty sure this needs to be disabled.

[    0.587254] **********************************************************
[    0.587257] **   NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE   **
[    0.587257] **                                                      **  
[    0.587257] ** This system shows unhashed kernel memory addresses   **  
[    0.587258] ** via the console, logs, and other interfaces. This    **  
[    0.587258] ** might reduce the security of your system.            **  
[    0.587258] **                                                      **  
[    0.587258] ** If you see this message and you are not debugging    **  
[    0.587259] ** the kernel, report this immediately to your system   **  
[    0.587259] ** administrator!                                       **  
[    0.587259] **                                                      **  
[    0.587259] **   NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE   **  
[    0.587260] **********************************************************

After removing slab_debug=FZ from the kernel command line, this message disappeared from the journal log.

Thanks to @ArrayBolt3 for reporting this!

[adrelanos]

Asked KSPP:

• Do not set slub_debug=FZ / slub_debug=ZF? KSPP/kspp.github.io#8

[raja-grewal]

This is very interesting.

See above #253 (comment) where the issue with having unhashed pointers was known and so I removed the kernel parameter. Then in #253 (comment) we decided to re-enable it using KSPP as the authority.

Now thanks to improvements in kernel 6.17 we have access to the hash_pointers=always parameter which we can enable in Debian 14 (circa 2028).

I am not sure how to proceed since to exploit unhashed pointers root access is already required, and so as per KSPP are the the benefits of retaining slab_debug=FZ still worth it prior to such an exploit?

Personally I am leaning towards keeping it enabled. While this reduces security in one way, we still have:

• slab_debug=F: Sanity checks on object allocation and deallocation.
• slab_debug=Z: Red zoning, which fills freed memory with a specific pattern to detect use-after-free bugs.

Maybe could introduce a "TODO: Debian 14" comment to remind us to enable hash_pointers=always one day a few years from now?

[ArrayBolt3]

Based on the KSPP response and the fact that all known ways of getting to the unhashed pointer values requires root, I'm inclined to agree with @raja-grewal here. We should do some research into seeing what interfaces the unhashed pointers may be available through though; if some of them are available via an interface that doesn't require root access, that could be a problem.

In theory it might be possible for us to switch to a backported kernel in the future and potentially get hash_pointers=always sooner. I'm not sure how well-maintained the backported kernel in Debian is maintained though.

[ArrayBolt3]

Actually, we do have an interface whereby a user can get access to the kernel logs without needing root. It's dmesg, for which we have a privleap exception for the sudo and privleap groups and the user account in systemcheck/etc/privleap/conf.d/systemcheck.conf. This is probably one way to harvest unhashed pointers from the kernel. We have similar configuration that allows dumping the entire journalctl log from the current boot in the same file. So it's not true that all known ways of getting to the unhashed pointer values requires root. Crud.

Possible options:

• Can we compartmentalize parts of systemcheck such that we don't have to allow blanket log dump capabilities to non-root users? Perhaps these capabilities can be offered only to a dedicated systemcheck account, under which scripts that read the logs run? Even this allows leaking log data though, if a message with an unhashed pointer happens to not be silenced by systemcheck/etc/systemcheck.d/30_default.conf, it will appear in systemcheck's output even if we make the actual filtering run as a different user.
• Any ideas for how to get that hash_pointers=always fix in Kicksecure sooner rather than later? The backport kernel is one option, we also had hardened-kernel for a while there. I doubt Debian will be willing to backport the feature into the stable kernel, but we could always ask.
• Turn off slub_debug after all?

It's also worth investigating what we think an attacker could really do with this information. I assume that having unhashed pointer values could be used in a form of KASLR bypass, but I don't know if that's actually the danger, and I don't know how an attack would work if that was the danger.

[ArrayBolt3]

Quick conversation on IRC (#debian room on OFTC):

[21:02] <arraybolt3> Two probably silly questions which I think I already know the answers to but am mostly double-checking. 1, is there any good chance of getting a particular patch backported into Trixie's *stable* kernel that technically adds a new feature, if that feature overcomes a serious issue similar to a real bug, and 2, is the backported kernel in Debian maintained as well as the stable kernel?
[21:03] <arraybolt3> (I think the answer to both questions is a firm "no".)
[21:04] <sney> 1 I think depends on ease of backporting and how much it's likely to introduce un/related breakage 
[21:04] <arraybolt3> the patch in particular I'm wishing was in Trixie is https://github.com/torvalds/linux/commit/de1c831a7898f164c1c2703c6b2b9e4fb4bebefc, which basically allows using `slub_debug=FZ` or similar without disabling kernel pointer hashing. `slub_debug` can be used to make some types of memory-management related vulnerabilities much harder to exploit, but only at the cost of allowing unhashed kernel pointers to be accessible from 
[21:04] <arraybolt3> userland which is a security risk (I guess because it would allow a KASLR bypass or similar).
[21:04] <arraybolt3> the patch just lets you say "I want `slub_debug` *and* I want hashed kernel pointers", so that you don't have to figure out which is the lesser of two security holes for your use case.
[21:05] <sney> 2 depends on what you mean by "well"; it doesn't have support from stable-security but does get pretty regular uploads, and it's the same kernel team as handles regular upgrades to the stable kernel outside of -security 
[21:05] <arraybolt3> hmm. At least to me, the patch looks *pretty* safe, and it doesn't do much functionally other than letting you set a new config combo.
[21:05] <arraybolt3> so maybe it would be worth asking about it?
[21:05] <sney> file a bug and make the argument, yeah
[21:06] <arraybolt3> kk. Thanks!
[21:06] <sney> np

Will file a bug against Debian to see if we can get hash_pointers=always into Trixie's stable kernel.

[raja-grewal]

Yes it is a great idea to see whether it is possible backport the new parameter. Until we get some feedback from this end I think we should leave it unmodified.

I will soon submit a PR making note of this new parameter so that it is clear in the documentation that the issue has been addressed for future reference.