Repository navigation
[FEAT] Additional Kernel Boot Parameters #1393
Description
Activity
What's the rationale for each? Particularly
ipv6.disable=1To provide more detail than the above see the below.
Candidates for inclusion by default are:
kfence.sample_interval=100: Enable the kernel "Electric-Fence" sampling-based memory safety error to detect heap out-of-bounds access, use-after-free, and invalid-free errors.vdso32=0: Disable 32-bit Virtual Dynamic Shared Object (vDSO) mappings as these are a legacy compatibility feature for superseded glibc versions.cfi=kcfi: Switch (back) to using kCFI as the default Control Flow Integrity (CFI) implementation as kCFI mandates hash validation at the source making it more difficult to bypass. This is in contrast to FineIBT which was made the default in kernel 6.2 and may result in some performance benefits as it only performs hash checks at the destinations.efi_pstore.pstore_disable=1anderst_disable: Disable both the EFI persistent storage feature and Error Record Serialization Table (ERST) support as a form of defense-in-depth. This prevents the kernel from writing crash logs and other persistent data to the storage backend.
Candidates for optional inclusion are:
ipv6.disable=1: Disable the entire IPv6 stack functionality.
Obviously this is just a guide to open up a discussion and nothing is being concretely requested.
As some form of evidence for their validity and functionality, they are all but the last one enabled by default in Kicksecure/Whonix config. Though it should be noted that all these have not been tested as the upcoming Debian 13 (kernel 6.12) port is yet to be released, hence only those available on Debian 12 (kernel 6.1) have been tested. The ones not tested live are
cfi=kcfiandipv6.disable=1.In order to keep this issue brief, please refer to the linked Kicksecure configs where you can find further details and references regarding each parameter.
I look forward to any feedback!
@raja-grewal Thanks for the suggestions
kfence.sample_interval=100
Already set by default for fedora's kernel.
vdso32=0
Not set by default for fedora's kernel, seems reasonable to add.
cfi=kcfi
This was discussed at some length in our discord already. There are pros and cons of each approach and given that there isn't a clear advantage either way, we opted to defer to the kernel
efi_pstore.pstore_disable=1
Already set by default for fedora's kernel
erst_disable
Strangely, I see no record of ERST being initialized in dmesg logs on secureblue. 🤔
ipv6.disable=1
see #1078, we have no interest in discriminating against IPv6 😅
Thanks for the feedback and review.
Regarding
ci=kcfiin particular, could you please point me roughly where and what was discussed regarding the the pros and cons. I respect your decision to opt with the new kernel default as of version 6.2 and would really appreciate for my own knowledge what went into this choice.Personally, to me it seems clear that for security purposes reverting back to the previous default of
cfi=kcfiis a good idea and so would genuinely appreciate to hear some counterarguments to find out if I have made a mistake.When it comes to
ipv6.disable=1I was not suggesting to disable it be default, only perhaps consider providing the option as you do withia32_emulation=0as a form of attack surface reduction for interested individuals.@raja-grewal While putting together sources from the discussion we had last year, I found this article from this year that's pretty damning for FineIBT:
https://www.phoronix.com/news/Linux-FineIBT-Critically-Flawed
Given this, it seems reasonable to set cfi=kcfi
@raja-grewal However, if they did merge the change to require FRED for FineIBT, then the whole conversation is moot because no intel cpus have support for FRED yet. Let me check.
@raja-grewal have a look here
torvalds/linux@97e5967#diff-69a1bc4a724617e303dd8cba2395d68b06a977953dea7c9c2ff919fbeaf1d93cR1145
They added a new "paranoid" mode for FINEIBT that is the default in the absence of intel FRED:
The fineibt_paranoid caller sequence adds additional caller side
hash validation. This stops such circumvention attacks dead, but at the cost
of adding a load.When it comes to ipv6.disable=1 I was not suggesting to disable it be default, only perhaps consider providing the option as you do with ia32_emulation=0 as a form of attack surface reduction for interested individuals.
I'm fine adding it in this way
Thanks for providing that kernel reference, I was not at all aware of it especially since it only was recently committed on 26 Feb 2025.
Inspecting the C code, it appears that this default is only applied if a CPU does not have the prerequisite X86_FEATURE_FRED feature bit to indicate FRED support. I am not an expert on how these feature bits are managed or even if it is possible for them to be manipulated by a sophisticated adversary.
Therefore, I still think it would be more prudent to revert back to using
cfi=kcfias that way we do not have to be totally reliant on the proper functioning of an additional parameter whose management is outside of the scope of secureblue. If we go back tocfi=kcfiwhich was the default prior to kernel 6.2, we at least have substantially more confidence that it will work as specified given it was globally used for a long time and so has likely been far more thoroughly audited over the years.I'm now running my system (
kinoite-main-hardenedimage) with the kernel argumentscfi=kcfi,vdso32=0, anderst_disablein addition to the ones currently set byujust set-kargs-hardening(including the "unstable" ones andia32_emulation=0, but not includingnosmt=force), and everything seems to be working as before—no immediately apparent issues.X86_FEATURE_FRED feature bit to indicate FRED support. I am not an expert on how these feature bits are managed or even if it is possible for them to be manipulated by a sophisticated adversary.
Intel FRED hasn't shipped on any CPUs yet afaik. It's coming with new CPUs later this fiscal year.
Therefore, I still think it would be more prudent to revert back to using cfi=kcfi as that way we do not have to be totally reliant on the proper functioning of an additional parameter whose management is outside of the scope of secureblue. If we go back to cfi=kcfi which was the default prior to kernel 6.2, we at least have substantially more confidence that it will work as specified given it was globally used for a long time and so has likely been far more thoroughly audited over the years.
I'm not convinced by this. kcfi is already used if fineibt isn't applicable. If fineibt is applicable, a software mitigation (paranoid mode) is applied in the absence of a hardware mitigation (Intel FRED). If Intel FRED is available, then the software mitigation is not needed.
So, it seems like the kernel has this taken care of 🙂
so has likely been far more thoroughly audited over the years.
I believe FineIBT still uses kcfi instrumentation for the software extensions: https://lpc.events/event/16/contributions/1315/attachments/1067/2169/cfi.pdf
More generally though, our policy is to not change any kernel parameters unless there's a quantifiable security advantage to doing so. Then, we can track changes to that quantifiable advantage over time, which gives us objective criteria with which to determine that we should undo our karg setting.
Without this, there are far too many ways we could end up shooting ourselves in the foot. The kernel is constantly changing, and without a way to measure those changes against our decisions, we could end up missing improvements that render our decisions obsolete or worse, a security degradation relative to the default. The recent addition of paranoid mode for FineIBT is a great example of this, in that FineIBT will likely continue to improve, and so at what point would we decide to undo
cfi=kcfi? What's are the criteria? What's the quantifiable advantage ofkcfi?Also @raja-grewal, as @RKNF404 just pointed out on discord, this conversation is somewhat moot since kcfi is only available if the kernel is built with clang. Fedora's is not, and I believe debian's also is not. @raja-grewal is kicksecure building your own kernel with clang? If not, then this conversation is probably moot.
However, this may change with: https://lwn.net/Articles/1035117/
18 remaining items
Yes I agree with the benefits, I think we should include
hash_pointers=always slab_debug=FZ.I would suggest enabling it by default even though as pointed out it will reduce performance depending on the workload. There are no expected breakages in any application.
Recently I became aware that at around Oct 2021 kernel 5.15 disabled by default hardware-based encryption of memory for specific AMD CPUs when it was previously enabled. This was at the time due to issues pertaining to DMA masks on certain hardware.
I would wager this merits serious a need for re-enabling provided the issues appear resolved by our combined testing.
Note that these settings generally apply to more workstation AMD CPUs. Users of consumer-grade AMD CPUs can instead enable TSME in their BIOS/UEFI to achieve the same protection. Likewise, users of Intel CPUs can enable TME in their BIOS/UEFI to get similar benefits.
The kernel boot parameters are:
mem_encrypt=on kvm_amd.sev=1Please see my existing PR in Kicksecure/security-misc#338 for more details and references.
Edit:
In Kicksecure/security-misc#341 I have outlined two additional inclusions being:kvm_amd.sev_es=1 kvm_amd.sev_snp=1Recall our previous inclusion of panicking when 100 oopses or warnings are generated leading then to an immediate reboot #1472.
There is a very important extension to this we should include that has to do with the generating panics when certain other conditions are met. Through the use of kernel taints we can create a user defined security policy to enforce strict kernel operation at runtime.
While this does not allow the setting of limits greater than one using
sysctlsettings, some of these issues are so severe that even one occurrence should be a cause for concern.7What I propose are using one of two options:
- Generate a panic upon using out of specification hardware, bad page states, severe firmware bugs, and kernel live patching.
- Generate a panic with the above and also on the loading of proprietary, out-of-tree, or unsigned modules.
The second should not be all that troublesome as we already use
lockdown=confidentialityandmodule.sig_enforce=1.The kernel boot parameters are:
panic_on_taint=0x8824 panic_on_taint=0xB825These correspond to the two options. The first is will detect very serious hardware flaws that should prohibit any continued operation till resolved (along with kernel live patching). The second will do the same and just add another method to enforce trust in loaded modules.
Testing by others is essential as I can not personally account for all user hardware combinations.
Please see my existing PR in Kicksecure/security-misc#339 for more details and references.
Edit:
Corrected the second bitmask.@raja-grewal where are those two codes documented?
out of specification hardware
for example?
loading of proprietary, out-of-tree,
This won't work for nvidia nor zfs
two codes documented?
they're bitmasks derived from the table: https://www.kernel.org/doc/html/latest/admin-guide/tainted-kernels.html#table-for-decoding-tainted-state
- 0x8824 is:
- 4: kernel running on an out of specification system
- 32: bad page referenced or some unexpected page flags
- 2048: workaround for bug in platform firmware applied
- 32768: kernel has been live patched
- 0xB7C5 is:
- 1: proprietary module was loaded
- 4: kernel running on an out of specification system
- 64: taint requested by userspace application
- 128: kernel died recently, i.e. there was an OOPS or BUG
- 256: ACPI table overridden by user
- 512: kernel issued warning
- 1024: staging driver was loaded
- 4096: externally-built (“out-of-tree”) module was loaded
- 8192: unsigned module was loaded
- 32768: kernel has been live patched
- never enabled:
- 2: module was force loaded
- 8: module was force unloaded
- 16: processor reported a Machine Check Exception (MCE)
- 16384: soft lockup occurred
- 65536: auxiliary taint, defined for and used by distros
- 131072: kernel was built with the struct randomization plugin
- 262144: an in-kernel test has been run
- 524288: userspace used a mutating debug operation in fwctl
@raja-grewal
did you intend for 0xB7C5 to not include 32 and 2048 like 0x8824 does? if not the second one should be 0xBFE3here is a script
#CC0 def printMask(mask): print("- ", mask) count = 1 for x in range(32): if ((mask & count)==count): print(" - ", count) count = count * 2 printMask(0x8824) printMask(0xB7C5) printMask(0xBFE3)
- 34852 - 4 - 32 - 2048 - 32768 - 47045 - 1 - 4 - 64 - 128 - 256 - 512 - 1024 - 4096 - 8192 - 32768 - 49123 - 1 - 2 - 32 - 64 - 128 - 256 - 512 - 1024 - 2048 - 4096 - 8192 - 32768
Reacted by RoyalOughtness and raja-grewal- 0x8824 is:
Thanks for the quick review. Apologies I made a mistake in the second hexadecimal bitmask. My previous post has now been edied to reflect the correct value.
Like you have broken it down it goes like the following:
- Panic on using out of specification hardware: 4 = 0x4.
- Panic on the above and bad page faults or some unexpected page flags: 36 = 0x24.
- Panic on the above and severe firmware bugs: 2084 = 0x824.
- Panic on the above and kernel live patching: 34852 = 0x8824.
- Panic on the above and the loading of proprietary, out-of-tree, or unsigned modules: 47141 = 0xB825.
An easy tool for calculating them is:
https://www.milkywaysys.com/calc/calculator-binary-decimal-hex-calculator.htmlfor example?
Please see the more details section at the bottom in the kernel docs. It shows cases where "if the kernel is running on a processor or system that is out of specification: hardware has been put into an unsupported configuration, therefore proper execution cannot be guaranteed."
This won't work for nvidia nor zfs
Agreed so I think stick with the first option and the second could be an optional inclusion.
Closing this as part of a backlog cleaning but please reopen if there remain missing items
Benefit
I was browsing through the current kernel boot parameters and potentially noticed a few ones that we could consider maybe adding to the config. From those below, some could certainly be optional while others might be candidates for enabling by default after comprehensive testing.
Solution
Candidates for inclusion by default are:
kfence.sample_interval=100,vdso32=0,cfi=kcfi,efi_pstore.pstore_disable=1,erst_disable. Candidates for optional inclusion are:ipv6.disable=1.Alternatives
Retain current kernel boot parameters without modification.
Declaration