Repository navigation
Definition of "logsource" values like product or category. #107
Description
Activity
Hey @Maspital
I'm guessing you're running chainsaw with the
sigma-event-logs-all.ymlmapping file. This mapping file does not filter based on provider name or category which means that some Sigma rules (like the file block one you mentioned above) cause a huge number of false positives.You could either extend the sigma rule to also require a specific event ID (which should cut down the number of FPs), or you could use the
sigma-event-logs-legacy.ymlmapping file instead which supports "Provider" mappings.I don't think either of these two options will completely solve your issue, but it should reduce the number of FPs that you face.
There is also potential to add category support to the mapping to apply 'x' filter when category 'y' is set. But this would require code enhancements.
Yes, I'm using a modified version of
sigma_event_logs.all(because winlogbeat likes to name stuff differently). Changing SIGMA rules themselves even slightly is something I would like to avoid because I want to be able to say "this dataset was created with SIGMA vXX" so it can be reproduced easily (though this is probably the easiest solution). In this case the change would be simple, but I wouldn't say I'm competent enough to confidently say that whatever change I intend to do does not change the intended result/logic of a given SIGMA rule :DUsing the legacy mapping file seems like it both has a lot of duplicate entries and prevents any novel alerts from being shown because they are blocked by the manually set filters(?).
Having category support like you mentioned @alexkornitzer would be a really nice and useful feature imo, but I dont know how much effort this would require on your part.
So adding this code wise is pretty easy the challenge comes from how to extend the mapping schema in a clear way, my current thinking is something like this:
--- name: Chainsaw's groupless Sigma mappings for Event Logs kind: evtx rules: sigma ... extensions: preconditions: - for: category: file_block filter: int(EventID): 27 Provider: Microsoft-Windows-Sysmon ... groups: ...Where the
forblock can consist of different identifiable fields of a Sigma rule.That looks great! Assuming knowledge of Sigma rules in general, this is nice and understandable in my opinion.
So this would work for whatever field may be defined within the
logsourceof a rule?Ah good spot nah I would wanna support things like name too, so it should be this:
--- name: Chainsaw's groupless Sigma mappings for Event Logs kind: evtx rules: sigma ... extensions: preconditions: - for: logsource.category: file_block filter: int(EventID): 27 Provider: Microsoft-Windows-Sysmon ... groups: ...But awesome, okay i'll see if I can do this when I get time otherwise i'll lob it at someone else :)
Reacted by Philipp Bönninghausen and ru37zAny news regarding this? :) @alexkornitzer
- added a commit that references this issue
on Oct 19, 2022 @Maspital can you give the
feature/extensionsbranch a try?- added a commit that references this issue
on Oct 19, 2022 Seems to work nicely, at least for the things I tested 👍 I will try it out in more detail tomorrow
Reacted by Alex KornitzerAwesome. If it all seems fine tomorrow i'll get that merged in and released.
I'm sure I didn't test for everything, but so far everything did what it was supposed to do. Two more things:
-
can the filter handle negated input? Might be useful for certain cases, it's not really important right now. Something like
extensions: preconditions: - for: logsource.category: some_category filter: int(EventID): NOT 1337though there probably exists better/clearer syntax than that
-
One would probably only ever write these filters with one or a few specific rule in mind. Sadly, keywords like "file_block" are basically free text with no hard meaning attached to them - though this is more a flaw of Sigma in general imo. This means there could be other rules using the same keyword that now unintentionally get blocked by this filter
To help with this, would it be possible to additionally declare a set of rules
["Some rule name", "Another rule name"]a filter should be assigned to? Or something similar.
Both of these things are just afterthoughts, I think the new feature perfectly solves the original issue. Thank you so much for implementing it so quickly!
Reacted by Alex Kornitzer-
Thanks for checking that all out. Right so the first one is probably not doable because I don't think I have put support in to handle nested syntax for a filter so this
not(int(EventID))probably won't work. We would need to upgrade the filter to be a full Tau block withconditionsupport, so maybe that can come in later as its not too difficult to add. I should add array support into theforblock as well but that can for now just be handled with copy paste, at the cost of duplication.I don't have a ton of time at the moment so I will merge the current work, and then your two above improvements can be added at a later date. For now I will keep this issue open as a reminder.
Reacted by Philipp BönninghausenReacted by ru37zTriage note: this was implemented. The
extensions.preconditionsschema Alex sketched out is live inmappings/sigma-event-logs-all.ymland parsed insrc/hunt.rs. Thefor: logsource.category: ...form Alex landed on is exactly what's in the file:extensions: preconditions: - for: logsource.category: process_creation filter: - Provider: Microsoft-Windows-Sysmon int(EventID): 1 - Provider: Microsoft-Windows-Security-Auditing int(EventID): 4688
Multiple categories covered (process_creation, network_connection, image_load, driver_load, etc.), so the original FP problem this was meant to address is handled in the default mapping. @Maspital probably safe to close unless you're hitting something the precondition system doesn't cover.
@Fuzzdkk the reason this issue is still open was as a reminder about the 2 additional feature requests:
- negation operations, i.e. would this still not work (off the top of my head I can't remember, would need to check the code):
extensions: preconditions: - for: logsource.category: process_creation filter: not(int(EventID)): 4688
- Additionally the preconditions needs to be extended with the ability to only apply to some Sigma rules.
On (1): doesn't work today. The precondition
filter:is deserialized viacrate::ext::tau::deserialize_expression(hunt.rs:38), which goes straight to tau'sparse_identifier. Thenot(...)and!prefix handling lives inparse_kvinsrc/ext/tau.rs, which preconditions skip. Sonot(int(EventID)): 4688parses as a literal field key callednot(int(EventID))and never matches.Should be a small change: a precondition-specific deserializer that reuses the
parse_kvpath the same way Sigma rule fields do. There are no precondition tests either, worth adding some at the same time.On (2): partially possible already.
for_matches against anythingSigma::Document::findexposes (sigma.rs:44-72), sofor: { id: "abc-..." }orfor: { title: "Some Rule" }works for individual rules today. What's missing is list-of-rules in a single precondition (for: { id: [a, b] }) and the FIXME at hunt.rs:208-209: when two preconditions both match the same rule, the second silently overwrites the first viapreconds.insert(...). Probably wants AND.Want me to take a stab at the negation one?
Yep I am happy with that approach,
parse_kvis what the search feature uses for its-targuments. Using that function the syntax would beint(EventID): !4688which is fine.As noted in the above PR, I changed my mind on this and went with an approach that reduces code and tau-chainsaw language fracturing, so we can now do:
extensions: preconditions: - for: logsource.category: process_creation filter: not int(EventID): 1337
Hey,
I am currently using chainsaw + SIGMA to evaluate log datasets and stumbled upon the following issue:
Certain SIGMA rules produce an abnormally high number of false positives, to the point where I suspect that it just triggers on most events. The rule in question is
I think the problem is that the category in question (
file_block) is not mapped to anything - where and how can I define this?In this example, the category should be
file_blockiff"provider_name": "Microsoft-Windows-Sysmon"andevent_id: 27, which would clearly identify the category.I have a similar problem for several other rules. Am I perhaps misunderstanding how certain things work?
Any help on this would be much appreciated :)