Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/elasticsearch.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ You can configure Elasticsearch by going to [Administration](administration.md)

## Parsing

Elasticsearch receives unparsed logs from [Logstash](logstash.md) or [Elastic Agent](elastic-agent.md). Elasticsearch then parses and stores those logs. Parsers are stored in `/opt/so/conf/elasticsearch/ingest/`. Custom ingest parsers can be placed in `/opt/so/saltstack/local/salt/elasticsearch/files/ingest/`. Files placed here are not detected by [Auto State Apply](salt.md#auto-state-apply), so to make these changes take effect, apply the Elasticsearch state to all nodes running Elasticsearch:
Elasticsearch receives unparsed logs from [Logstash](logstash.md) or [Elastic Agent](elastic-agent.md). Elasticsearch then parses and stores those logs. Parsers are stored in `/opt/so/conf/elasticsearch/ingest/`. Custom ingest parsers can be placed in `/opt/so/saltstack/local/salt/elasticsearch/files/ingest/`. [Auto State Apply](salt.md#auto-state-apply) picks these up within a few minutes. If you don't want to wait, apply the Elasticsearch state to all nodes running Elasticsearch:


```
Expand Down
Binary file modified docs/images/01_grub.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/02_initial_install.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/04_setup_init.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/05_setup_option.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/06_setup_airgap.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/06_setup_type.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/07_setup_license.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/08_setup_hostname.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/09_setup_hostname_conflict.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/10_setup_mn_nic.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/11_setup_mn_int.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/12_setup_cidr.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/13_setup_gateway.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/14_setup_dns_servers.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/15_setup_dns_domain.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/16_setup_docker_range.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/18_setup_direct_proxy.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/20_setup_webuser.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/21_setup_webpass1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/22_setup_webpass2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/23_setup_access_type.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/26_setup_so_allow.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/27_setup_so_allow_input.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/27_telemetry.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/28_setup_summary.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/29_setup_finished.png
Binary file modified docs/images/38_overview.png
Binary file modified docs/images/39_grid.png
Binary file modified docs/images/40_upload.png
Binary file modified docs/images/45_import.png
Binary file modified docs/images/50_alerts.png
Binary file modified docs/images/51_alerts_play.png
Binary file modified docs/images/52_alerts_options.png
Binary file modified docs/images/53_dashboards.png
Binary file modified docs/images/54_dashboards_options.png
Binary file modified docs/images/56_hunt.png
Binary file modified docs/images/57_0_cases.png
Binary file modified docs/images/57_1_cases_options.png
Binary file modified docs/images/57_2_cases_create.png
Binary file modified docs/images/57_detections.png
Binary file modified docs/images/58_detections_options.png
Binary file modified docs/images/59_detection_create.png
Binary file modified docs/images/60_detection_nids.png
Binary file modified docs/images/60_detection_nids_0_comments.png
Binary file modified docs/images/60_detection_nids_1_signature.png
Binary file modified docs/images/60_detection_nids_2_tuning_1.png
Binary file modified docs/images/60_detection_nids_2_tuning_2_add.png
Binary file modified docs/images/60_detection_nids_3_playbook.png
Binary file modified docs/images/60_detection_nids_4_history.png
Binary file modified docs/images/60_detection_sigma.png
Binary file modified docs/images/60_detection_sigma_2_tuning_1.png
Binary file modified docs/images/60_detection_sigma_2_tuning_2_add.png
Binary file modified docs/images/60_detection_yara.png
Binary file modified docs/images/61_actions.png
Binary file modified docs/images/62_pcap.png
Binary file modified docs/images/65_pcap_details.png
Binary file modified docs/images/72_jobs.png
Binary file modified docs/images/73_jobs_add.png
Binary file modified docs/images/75_grid.png
Binary file modified docs/images/76_grid_options.png
Binary file modified docs/images/78_downloads.png
Binary file modified docs/images/81_users.png
Binary file modified docs/images/82_users_detail.png
Binary file modified docs/images/83_users_add.png
Binary file modified docs/images/84_gridmembers.png
Binary file modified docs/images/87_config.png
Binary file modified docs/images/88_config_options.png
Binary file modified docs/images/91_licensekey.png
Binary file modified docs/images/94_usermenu.png
Binary file modified docs/images/config-item-backup.png
Binary file modified docs/images/config-item-bpf.png
Binary file modified docs/images/config-item-elastalert-alerter.png
Binary file modified docs/images/config-item-elastalert.png
Binary file modified docs/images/config-item-elasticfleet.png
Binary file modified docs/images/config-item-elasticsearch.png
Binary file modified docs/images/config-item-firewall.png
Binary file modified docs/images/config-item-global-url.png
Binary file modified docs/images/config-item-global.png
Binary file modified docs/images/config-item-host.png
Binary file modified docs/images/config-item-idh.png
Binary file modified docs/images/config-item-influxdb.png
Binary file modified docs/images/config-item-kafka.png
Binary file modified docs/images/config-item-kibana.png
Binary file modified docs/images/config-item-kratos.png
Binary file modified docs/images/config-item-logstash.png
Binary file modified docs/images/config-item-manager.png
Binary file modified docs/images/config-item-nginx.png
Binary file modified docs/images/config-item-ntp.png
Binary file modified docs/images/config-item-patch.png
Binary file modified docs/images/config-item-redis.png
Binary file modified docs/images/config-item-sensor.png
Binary file modified docs/images/config-item-sensoroni.png
Binary file modified docs/images/config-item-soc-additionalAlerters.png
Binary file modified docs/images/config-item-soc-subgrids.png
Binary file modified docs/images/config-item-soc.png
Binary file modified docs/images/config-item-strelka.png
Binary file modified docs/images/config-item-suricata.png
Binary file modified docs/images/config-item-telegraf.png
Binary file modified docs/images/config-item-versionlock.png
Binary file modified docs/images/config-item-zeek.png
39 changes: 39 additions & 0 deletions docs/kernels.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Kernels

Security Onion standardizes on the Oracle Unbreakable Enterprise Kernel (UEK). Nodes run the UEK8 (6.x) kernel series, and the standard EL9 RedHat compatible kernel (RHCK, 5.14) is removed once a node is actually running UEK8.

This is all automatic and you should not need to do anything. Nodes pick up the UEK8 kernel as part of their normal OS updates, and the stock EL9 kernel is removed for you on the next highstate after the node reboots onto UEK8.

The removal is deliberately deferred until after the reboot. `dnf` refuses to erase the running kernel, and waiting also means the node has proven it boots on UEK8 before its fallback is deleted. Fresh installs reboot at the end of setup and are cleaned up on the first highstate after that; existing nodes are cleaned up whenever you reboot them.

These are the packages removed:

```
kernel kernel-core kernel-modules kernel-modules-core kernel-tools kernel-tools-libs
```

!!! NOTE

Older UEK7 `kernel-uek` 5.x packages are not removed. They age out on their own as newer kernels are installed.

## Doing It Manually

If a node did not end up on UEK8 on its own, you can run the following command on that node:

```
sudo so-kernel-upgrade
```

This installs the UEK8 kernel and makes it the boot default. It does not reboot the node, so you can schedule the reboot yourself. The new kernel does not take effect until you reboot.

!!! NOTE

The manager mirrors the UEK8 packages and serves them to the rest of the grid. If the manager has not synced them yet, `so-kernel-upgrade` on the manager will sync them for you; on any other node it will tell you to sync the manager first.

Similarly, if the stock EL9 kernel is still installed on a node that is already running UEK8, you can run the cleanup directly instead of waiting for the next highstate:

```
sudo so-kernel-upgrade --cleanup
```

This does nothing unless the node is already running UEK8.
334 changes: 334 additions & 0 deletions docs/local-llm.md

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions docs/logstash.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ In [SOC](security-onion-console.md), navigate to [Administration](administration
custom/myfile.conf
```

The configuration will be applied at the next scheduled highstate (see [Highstate Interval](salt.md#highstate-interval)) or you can apply it immediately by clicking the `SYNCHRONIZE GRID` button under the `Options` menu.
[Auto State Apply](salt.md#auto-state-apply) will apply the configuration within a few minutes. If you don't want to wait, click the `SYNCHRONIZE GRID` button under the `Options` menu.

You can monitor events flowing through the output by running the following command on the manager:

Expand All @@ -82,7 +82,7 @@ curl -s localhost:9600/_node/stats | jq .pipelines.manager

## Modified Event Forwarding

To forward events to an external destination AFTER they have traversed the Logstash pipelines (NOT ingest node pipelines), perform the same steps as above but instead of adding the reference for your Logstash output to the `manager` pipeline add it to `search` pipeline instead. The configuration will be applied at the next scheduled highstate (see [Highstate Interval](salt.md#highstate-interval)) or immediately by clicking the `SYNCHRONIZE GRID` button under the `Options` menu.
To forward events to an external destination AFTER they have traversed the Logstash pipelines (NOT ingest node pipelines), perform the same steps as above but instead of adding the reference for your Logstash output to the `manager` pipeline add it to `search` pipeline instead. [Auto State Apply](salt.md#auto-state-apply) will apply the configuration within a few minutes. If you don't want to wait, click the `SYNCHRONIZE GRID` button under the `Options` menu.

You can monitor events flowing through the output by running the following command on the search nodes:

Expand Down
2 changes: 1 addition & 1 deletion docs/manager-of-managers.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ While on the API Client screen, click the ⤓ icon to download the Certificate A

!!! NOTE

Please be aware that any users in the MoM Grid will be able to connect to the subgrid using the permissions defined for the API client. For example, suppose that you create an API client ID in the subgrid called `supermom` and you grant it all permissions. Once the MoM is configured to connect to the subgrid as shown in the next section, then any users in the MoM Grid will connect to the subgrid as `supermom` and have all permissions to the subgrid regardless of whether the user has equivalent permissions in the MoM.
Starting in release 3.3.0, all MoM users that are not already superusers must be granted either the `subgrid-auditor` or `subgrid-superuser` role in order to interact with subgrids. The `subgrid-auditor` role will permit read operations (HTTP GET API requests), and `subgrid-superuser` will permit read and write operations (HTTP GET, POST, PUT, PATCH, and DELETE API requests) against the subgrids. These requests to the subgrid will all utilize the same subgrid API client credentials. As an example, a user with `subgrid-auditor` can see cases on the subgrid, but cannot modify them. Whereas a user with `subgrid-superuser` can modify cases on the subgrid, as well as manage subgrid users and their roles.

### Subgrid Config

Expand Down
62 changes: 61 additions & 1 deletion docs/onion-ai.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ Security Onion now supports local models through any OpenAI-compatible endpoint.

## Hosting Local Models

Hosting your own models requires powerful and expensive hardware. For beginners we recommend using a tool such as LM Studio. **You need at least 96GB of VRAM** to host your own models locally. The speed and accuracy of OnionAI when hosted locally is based on the hardware that you are using. For the most accurate results we recommend using credits with OnionAI.
Hosting your own models requires powerful and expensive hardware. For beginners we recommend using a tool such as LM Studio. The models listed above are large, and **you need at least 96GB of VRAM** to host them locally. Smaller mixture-of-experts models can run in considerably less memory while still offering a context window large enough for the assistant; see [Local LLM Hosting](local-llm.md) for a tested walkthrough on a single machine. The speed and accuracy of OnionAI when hosted locally is based on the hardware that you are using. For the most accurate results we recommend using credits with OnionAI.

## Available Tools

Expand Down Expand Up @@ -155,3 +155,63 @@ Your system prompt addendum will be added after Security Onion's default system
Superusers can review token usage and conversation history for all users by going to Administration --> AI Metrics. This page provides usage statistics for a given date range. The page starts with a table of usage by user. Clicking a user's binoculars icon on the right hand side will show any sessions the user interacted with during the selected date range, even deleted sessions. Clicking on a session's binoculars icon will show the full conversation. Administrators can adjust who has permissions via RBAC roles.

To provide an accurate history, deleted sessions are retained on the metrics page even after being deleted by the user.

## Memory

OnionAI can utilize its memory system to retain and reference facts disclosed by users when chatting with the assistant. Memory works by scanning sessions in the background looking for new facts and storing them in a database. The memory is then used to provide contextual information to the assistant when it responds to a user's message.

### How Memory Works

The memory system scans sessions in the background extracting, embedding, reconciling, and saving memories to later be referenced when chatting with Onion AI.

First, extraction. The selected sessions have their transcripts given to a Memory agent that parses out useful facts. The agent also identifies the scope of the fact: is it specific to the user who owns the session we just scanned or is it applicable to the entire organization.

These facts are then embedded using an Embed agent. This enables SOC to compare memories and is the heart of how memory works.

!!! NOTE

The embedding process is model specific. Memories embedded by one model cannot be accurately compared to memories embedded by any other model. Changing the model used for embedding will render all your previous memories unusable. They will not be deleted and will be accessible if the model is reverted back.

Once embedded, the facts are compared against existing memories on the same topics in a step called Reconcilation. A Reconcile agent will decide how the facts should be added, merged, replaced or removed. The reconcile agent's recommendations are validated before being executed to ensure that user defined memories aren't modified and that the session owner's permissions are respected.

If any facts were rewritten during reconcilation, they are re-embedded before being stored in postgres.

Finally the session is updated indicating that it has been scanned. If new messages are sent in a previously scanned session, memory scans will find and scan only the new messages.

When a user sends a message to the assistant, the memory system will embed the message about to be sent and check for any similar memories. Memories are added to the conversation by being added to the end of the prompt.

### Configuring Memory

| Name | Default Value | Description |
|---|---|---|
| `useMemory` | `true` | Enable memory use when sending a message. |
| `useMemoryScanner` | `false` | Enables the scanning of historical sessions. |
| `dontScanBefore` | `""` | A date in RFC3339 format (2026-08-31T22:05:48Z) to use as a cutoff point for memory scans. |
| `memoryScanIntervalSeconds` | `300` | How long between scans the memory scanner waits before scanning again. |
| `memoryProximityThreshold` | `0.8` | A similarity threshold that determines how similar a fact must be to a previous memory for it to be considered "similar." |
| `messageProximityThreshold` | `0.5` | A similarity threshold that determines how similar a message from a user must be to a stored memory for it to be considered "similar." |
| `maxUserMemoriesToReconcile` | `20` | The maximum number of user-specific memories to reconcile at once. |
| `maxGlobalMemoriesToReconcile` | `20` | The maximum number of global memories to reconcile at once. |
| `maxUserMemoriesToInclude` | `5` | The maximum number of user-specific memories to include when sending a message. |
| `maxGlobalMemoriesToInclude` | `5` | The maximum number of global memories to include when sending a message. |
| `memoryModel` | `gemma@SOAI` | The model used to extract facts from sessions. |
| `memoryPersona` | `""` | Special instructions for the memory agent included in the prompt. |
| `embedModel` | `amazon.titan-embed-text-v2@SOAI` | The model used to embed facts. |
| `reconcileModel` | `gemma@SOAI` | The model used to reconcile facts from sessions with existing memories. |
| `reconcilePersona` | `""` | Special instructions for the reconcile agent included in the prompt. |
| `toolUseTurnAttempts` | `12` | When the assistant requests a read-only tool, SOC can approve it automatically. Because the approval can fire before the original request has finished being written to Elasticsearch, SOC will retry the approval up to this many times before giving up. |
| `toolUseTurnDelayMs` | `175` | The time to wait between auto-approval attempts. Together with the attempts setting, this defines the total grace period SOC allows for the tool request to become available. |
| `maxMemoryRetries` | `2` | The maximum number of times SOC will attempt to extract memories from sessions before marking the session to be ignored. Increasing this value may retry sessions that haven't been attempted in a long time. |

Memory and the Memory Scanner may be enabled independently. Disabling memory will stop applying memories to prompts on outgoing messages. Disabling the memory scanner will prevent the scanner from extracting memories from previous assistant sessions. The memory scanner marks sessions as it extracts facts from them so that they are not re-scanned in future scans. If a memory scan takes longer than the interval between scans, then at most 1 scan will queue up for processing and it'll begin again immediately after the previous scan finishes.

### How to Write a Good Memory

When entering memories in manually in the Agent Studio, it's important to understand that how the memory is written affects how it'll be used by the memory system.

1. **One fact per memory.** If a memory has multiple facts then matching is harder to accomplish resulting in the memory being accessed less.
1. **Make each memory self-contained.** It will be read out of context, alone, possibly months later in a conversation about something else. Every memory should make sense to someone who has only that sentence.
1. **Declarative facts, not instructions.** If you want to give an instruction, then use a prompt instead.
1. **Present tense, absolute references, durable phrasing.** A memory is read long after it's written and is never automatically updated, so describe how things are rather than how they changed. Prefer "PCAP retention is 90 days (set August 2026)" over "We recently changed retention to 90 days."

To help track your most successful memories, the Agent Studio presents how many times a memory has been accessed and when was the last time it was referenced.
Loading
Loading