diff --git a/docs/elasticsearch.md b/docs/elasticsearch.md index b3a666f8..592bda25 100644 --- a/docs/elasticsearch.md +++ b/docs/elasticsearch.md @@ -65,7 +65,7 @@ You can configure Elasticsearch by going to [Administration](administration.md) ## Parsing -Elasticsearch receives unparsed logs from [Logstash](logstash.md) or [Elastic Agent](elastic-agent.md). Elasticsearch then parses and stores those logs. Parsers are stored in `/opt/so/conf/elasticsearch/ingest/`. Custom ingest parsers can be placed in `/opt/so/saltstack/local/salt/elasticsearch/files/ingest/`. Files placed here are not detected by [Auto State Apply](salt.md#auto-state-apply), so to make these changes take effect, apply the Elasticsearch state to all nodes running Elasticsearch: +Elasticsearch receives unparsed logs from [Logstash](logstash.md) or [Elastic Agent](elastic-agent.md). Elasticsearch then parses and stores those logs. Parsers are stored in `/opt/so/conf/elasticsearch/ingest/`. Custom ingest parsers can be placed in `/opt/so/saltstack/local/salt/elasticsearch/files/ingest/`. [Auto State Apply](salt.md#auto-state-apply) picks these up within a few minutes. If you don't want to wait, apply the Elasticsearch state to all nodes running Elasticsearch: ``` diff --git a/docs/images/01_grub.png b/docs/images/01_grub.png index 481d5340..6581a8cc 100644 Binary files a/docs/images/01_grub.png and b/docs/images/01_grub.png differ diff --git a/docs/images/02_initial_install.png b/docs/images/02_initial_install.png index 6c1baa73..6541df23 100644 Binary files a/docs/images/02_initial_install.png and b/docs/images/02_initial_install.png differ diff --git a/docs/images/04_setup_init.png b/docs/images/04_setup_init.png index c6c74433..b138224f 100644 Binary files a/docs/images/04_setup_init.png and b/docs/images/04_setup_init.png differ diff --git a/docs/images/05_setup_option.png b/docs/images/05_setup_option.png index 5894f129..d9a65af8 100644 Binary files a/docs/images/05_setup_option.png and b/docs/images/05_setup_option.png differ diff --git a/docs/images/06_setup_airgap.png b/docs/images/06_setup_airgap.png index 3fc7390b..04be2e3a 100644 Binary files a/docs/images/06_setup_airgap.png and b/docs/images/06_setup_airgap.png differ diff --git a/docs/images/06_setup_type.png b/docs/images/06_setup_type.png index a7b160a2..87291c6f 100644 Binary files a/docs/images/06_setup_type.png and b/docs/images/06_setup_type.png differ diff --git a/docs/images/07_setup_license.png b/docs/images/07_setup_license.png index 03bdc15d..ff925af1 100644 Binary files a/docs/images/07_setup_license.png and b/docs/images/07_setup_license.png differ diff --git a/docs/images/08_setup_hostname.png b/docs/images/08_setup_hostname.png index 6102ba9b..a6b43bf1 100644 Binary files a/docs/images/08_setup_hostname.png and b/docs/images/08_setup_hostname.png differ diff --git a/docs/images/09_setup_hostname_conflict.png b/docs/images/09_setup_hostname_conflict.png index 46ee0c84..c0d3ffd4 100644 Binary files a/docs/images/09_setup_hostname_conflict.png and b/docs/images/09_setup_hostname_conflict.png differ diff --git a/docs/images/10_setup_mn_nic.png b/docs/images/10_setup_mn_nic.png index 11321a6d..92de17f0 100644 Binary files a/docs/images/10_setup_mn_nic.png and b/docs/images/10_setup_mn_nic.png differ diff --git a/docs/images/11_setup_mn_int.png b/docs/images/11_setup_mn_int.png index 0c4897ed..3b7bc004 100644 Binary files a/docs/images/11_setup_mn_int.png and b/docs/images/11_setup_mn_int.png differ diff --git a/docs/images/12_setup_cidr.png b/docs/images/12_setup_cidr.png index 2a6b92bb..2b5442b7 100644 Binary files a/docs/images/12_setup_cidr.png and b/docs/images/12_setup_cidr.png differ diff --git a/docs/images/13_setup_gateway.png b/docs/images/13_setup_gateway.png index 6a1087dd..b56c033b 100644 Binary files a/docs/images/13_setup_gateway.png and b/docs/images/13_setup_gateway.png differ diff --git a/docs/images/14_setup_dns_servers.png b/docs/images/14_setup_dns_servers.png index dc81cf3a..cd2abd23 100644 Binary files a/docs/images/14_setup_dns_servers.png and b/docs/images/14_setup_dns_servers.png differ diff --git a/docs/images/15_setup_dns_domain.png b/docs/images/15_setup_dns_domain.png index 4472b7a2..7d68828f 100644 Binary files a/docs/images/15_setup_dns_domain.png and b/docs/images/15_setup_dns_domain.png differ diff --git a/docs/images/16_setup_docker_range.png b/docs/images/16_setup_docker_range.png index ef6cb650..1d74c3a4 100644 Binary files a/docs/images/16_setup_docker_range.png and b/docs/images/16_setup_docker_range.png differ diff --git a/docs/images/18_setup_direct_proxy.png b/docs/images/18_setup_direct_proxy.png index 7be190e9..d3fd4998 100644 Binary files a/docs/images/18_setup_direct_proxy.png and b/docs/images/18_setup_direct_proxy.png differ diff --git a/docs/images/20_setup_webuser.png b/docs/images/20_setup_webuser.png index d919e639..83878720 100644 Binary files a/docs/images/20_setup_webuser.png and b/docs/images/20_setup_webuser.png differ diff --git a/docs/images/21_setup_webpass1.png b/docs/images/21_setup_webpass1.png index 3bfb35ac..e0614ed9 100644 Binary files a/docs/images/21_setup_webpass1.png and b/docs/images/21_setup_webpass1.png differ diff --git a/docs/images/22_setup_webpass2.png b/docs/images/22_setup_webpass2.png index 903a3082..c227eab0 100644 Binary files a/docs/images/22_setup_webpass2.png and b/docs/images/22_setup_webpass2.png differ diff --git a/docs/images/23_setup_access_type.png b/docs/images/23_setup_access_type.png index d171477c..9c81796b 100644 Binary files a/docs/images/23_setup_access_type.png and b/docs/images/23_setup_access_type.png differ diff --git a/docs/images/26_setup_so_allow.png b/docs/images/26_setup_so_allow.png index 6f2de144..0d9fd451 100644 Binary files a/docs/images/26_setup_so_allow.png and b/docs/images/26_setup_so_allow.png differ diff --git a/docs/images/27_setup_so_allow_input.png b/docs/images/27_setup_so_allow_input.png index ae9e2c44..38381a3e 100644 Binary files a/docs/images/27_setup_so_allow_input.png and b/docs/images/27_setup_so_allow_input.png differ diff --git a/docs/images/27_telemetry.png b/docs/images/27_telemetry.png index 449ab839..9a02d1ec 100644 Binary files a/docs/images/27_telemetry.png and b/docs/images/27_telemetry.png differ diff --git a/docs/images/28_setup_summary.png b/docs/images/28_setup_summary.png index 8ab2373c..4db83921 100644 Binary files a/docs/images/28_setup_summary.png and b/docs/images/28_setup_summary.png differ diff --git a/docs/images/29_setup_finished.png b/docs/images/29_setup_finished.png index a3b216b3..9181678d 100644 Binary files a/docs/images/29_setup_finished.png and b/docs/images/29_setup_finished.png differ diff --git a/docs/images/38_overview.png b/docs/images/38_overview.png index 7f51c050..cef73b8c 100644 Binary files a/docs/images/38_overview.png and b/docs/images/38_overview.png differ diff --git a/docs/images/39_grid.png b/docs/images/39_grid.png index 926a91fb..82b46b79 100644 Binary files a/docs/images/39_grid.png and b/docs/images/39_grid.png differ diff --git a/docs/images/40_upload.png b/docs/images/40_upload.png index 5a6ea9b1..ffa6dd47 100644 Binary files a/docs/images/40_upload.png and b/docs/images/40_upload.png differ diff --git a/docs/images/45_import.png b/docs/images/45_import.png index d1d240ba..dacb0415 100644 Binary files a/docs/images/45_import.png and b/docs/images/45_import.png differ diff --git a/docs/images/50_alerts.png b/docs/images/50_alerts.png index 649afd08..3af3be29 100644 Binary files a/docs/images/50_alerts.png and b/docs/images/50_alerts.png differ diff --git a/docs/images/51_alerts_play.png b/docs/images/51_alerts_play.png index d4799e63..86baf29a 100644 Binary files a/docs/images/51_alerts_play.png and b/docs/images/51_alerts_play.png differ diff --git a/docs/images/52_alerts_options.png b/docs/images/52_alerts_options.png index a68082a1..a5d23571 100644 Binary files a/docs/images/52_alerts_options.png and b/docs/images/52_alerts_options.png differ diff --git a/docs/images/53_dashboards.png b/docs/images/53_dashboards.png index 586d3464..25c36aca 100644 Binary files a/docs/images/53_dashboards.png and b/docs/images/53_dashboards.png differ diff --git a/docs/images/54_dashboards_options.png b/docs/images/54_dashboards_options.png index 1e5e9ce8..58b73276 100644 Binary files a/docs/images/54_dashboards_options.png and b/docs/images/54_dashboards_options.png differ diff --git a/docs/images/56_hunt.png b/docs/images/56_hunt.png index 6a2da734..91f6ce71 100644 Binary files a/docs/images/56_hunt.png and b/docs/images/56_hunt.png differ diff --git a/docs/images/57_0_cases.png b/docs/images/57_0_cases.png index c240eef8..ad326c52 100644 Binary files a/docs/images/57_0_cases.png and b/docs/images/57_0_cases.png differ diff --git a/docs/images/57_1_cases_options.png b/docs/images/57_1_cases_options.png index b94c562e..704d7f8a 100644 Binary files a/docs/images/57_1_cases_options.png and b/docs/images/57_1_cases_options.png differ diff --git a/docs/images/57_2_cases_create.png b/docs/images/57_2_cases_create.png index 48e9fcdb..5f175ef7 100644 Binary files a/docs/images/57_2_cases_create.png and b/docs/images/57_2_cases_create.png differ diff --git a/docs/images/57_detections.png b/docs/images/57_detections.png index 969fe65f..bbd73eb4 100644 Binary files a/docs/images/57_detections.png and b/docs/images/57_detections.png differ diff --git a/docs/images/58_detections_options.png b/docs/images/58_detections_options.png index dea920c4..c94b5f25 100644 Binary files a/docs/images/58_detections_options.png and b/docs/images/58_detections_options.png differ diff --git a/docs/images/59_detection_create.png b/docs/images/59_detection_create.png index 26f598f7..c694d55e 100644 Binary files a/docs/images/59_detection_create.png and b/docs/images/59_detection_create.png differ diff --git a/docs/images/60_detection_nids.png b/docs/images/60_detection_nids.png index 8ae2864c..13cc3a7b 100644 Binary files a/docs/images/60_detection_nids.png and b/docs/images/60_detection_nids.png differ diff --git a/docs/images/60_detection_nids_0_comments.png b/docs/images/60_detection_nids_0_comments.png index 3bd1c35a..097d1cce 100644 Binary files a/docs/images/60_detection_nids_0_comments.png and b/docs/images/60_detection_nids_0_comments.png differ diff --git a/docs/images/60_detection_nids_1_signature.png b/docs/images/60_detection_nids_1_signature.png index ffa18114..d4213a31 100644 Binary files a/docs/images/60_detection_nids_1_signature.png and b/docs/images/60_detection_nids_1_signature.png differ diff --git a/docs/images/60_detection_nids_2_tuning_1.png b/docs/images/60_detection_nids_2_tuning_1.png index a26c60c1..19cb3895 100644 Binary files a/docs/images/60_detection_nids_2_tuning_1.png and b/docs/images/60_detection_nids_2_tuning_1.png differ diff --git a/docs/images/60_detection_nids_2_tuning_2_add.png b/docs/images/60_detection_nids_2_tuning_2_add.png index 908d42b5..71399076 100644 Binary files a/docs/images/60_detection_nids_2_tuning_2_add.png and b/docs/images/60_detection_nids_2_tuning_2_add.png differ diff --git a/docs/images/60_detection_nids_3_playbook.png b/docs/images/60_detection_nids_3_playbook.png index ceb7f339..591191b8 100644 Binary files a/docs/images/60_detection_nids_3_playbook.png and b/docs/images/60_detection_nids_3_playbook.png differ diff --git a/docs/images/60_detection_nids_4_history.png b/docs/images/60_detection_nids_4_history.png index 5f2c9c11..0d4a08e3 100644 Binary files a/docs/images/60_detection_nids_4_history.png and b/docs/images/60_detection_nids_4_history.png differ diff --git a/docs/images/60_detection_sigma.png b/docs/images/60_detection_sigma.png index a29bc4df..06b739b6 100644 Binary files a/docs/images/60_detection_sigma.png and b/docs/images/60_detection_sigma.png differ diff --git a/docs/images/60_detection_sigma_2_tuning_1.png b/docs/images/60_detection_sigma_2_tuning_1.png index 485463b4..8e984836 100644 Binary files a/docs/images/60_detection_sigma_2_tuning_1.png and b/docs/images/60_detection_sigma_2_tuning_1.png differ diff --git a/docs/images/60_detection_sigma_2_tuning_2_add.png b/docs/images/60_detection_sigma_2_tuning_2_add.png index 17dcc266..ec4cc14d 100644 Binary files a/docs/images/60_detection_sigma_2_tuning_2_add.png and b/docs/images/60_detection_sigma_2_tuning_2_add.png differ diff --git a/docs/images/60_detection_yara.png b/docs/images/60_detection_yara.png index b871aafe..dc258ce9 100644 Binary files a/docs/images/60_detection_yara.png and b/docs/images/60_detection_yara.png differ diff --git a/docs/images/61_actions.png b/docs/images/61_actions.png index 505add25..7d97d3b6 100644 Binary files a/docs/images/61_actions.png and b/docs/images/61_actions.png differ diff --git a/docs/images/62_pcap.png b/docs/images/62_pcap.png index 897f7a49..9e1c2f8f 100644 Binary files a/docs/images/62_pcap.png and b/docs/images/62_pcap.png differ diff --git a/docs/images/65_pcap_details.png b/docs/images/65_pcap_details.png index 22595038..7f19876d 100644 Binary files a/docs/images/65_pcap_details.png and b/docs/images/65_pcap_details.png differ diff --git a/docs/images/72_jobs.png b/docs/images/72_jobs.png index 5a4ceeb3..a837a740 100644 Binary files a/docs/images/72_jobs.png and b/docs/images/72_jobs.png differ diff --git a/docs/images/73_jobs_add.png b/docs/images/73_jobs_add.png index fbe03d07..e884b2f2 100644 Binary files a/docs/images/73_jobs_add.png and b/docs/images/73_jobs_add.png differ diff --git a/docs/images/75_grid.png b/docs/images/75_grid.png index 22b61149..52ee88aa 100644 Binary files a/docs/images/75_grid.png and b/docs/images/75_grid.png differ diff --git a/docs/images/76_grid_options.png b/docs/images/76_grid_options.png index 4772fb45..2e755561 100644 Binary files a/docs/images/76_grid_options.png and b/docs/images/76_grid_options.png differ diff --git a/docs/images/78_downloads.png b/docs/images/78_downloads.png index d924609f..5baccc36 100644 Binary files a/docs/images/78_downloads.png and b/docs/images/78_downloads.png differ diff --git a/docs/images/81_users.png b/docs/images/81_users.png index ea4ee58e..3630b8dc 100644 Binary files a/docs/images/81_users.png and b/docs/images/81_users.png differ diff --git a/docs/images/82_users_detail.png b/docs/images/82_users_detail.png index d6dfedb3..2b77c423 100644 Binary files a/docs/images/82_users_detail.png and b/docs/images/82_users_detail.png differ diff --git a/docs/images/83_users_add.png b/docs/images/83_users_add.png index 09a0400e..fa8807b2 100644 Binary files a/docs/images/83_users_add.png and b/docs/images/83_users_add.png differ diff --git a/docs/images/84_gridmembers.png b/docs/images/84_gridmembers.png index f4db6cdc..8851faac 100644 Binary files a/docs/images/84_gridmembers.png and b/docs/images/84_gridmembers.png differ diff --git a/docs/images/87_config.png b/docs/images/87_config.png index c51caf10..deb7b490 100644 Binary files a/docs/images/87_config.png and b/docs/images/87_config.png differ diff --git a/docs/images/88_config_options.png b/docs/images/88_config_options.png index fbd4d620..89d7396f 100644 Binary files a/docs/images/88_config_options.png and b/docs/images/88_config_options.png differ diff --git a/docs/images/91_licensekey.png b/docs/images/91_licensekey.png index 2a06533f..bbf1e247 100644 Binary files a/docs/images/91_licensekey.png and b/docs/images/91_licensekey.png differ diff --git a/docs/images/94_usermenu.png b/docs/images/94_usermenu.png index a3e09cae..a93ea630 100644 Binary files a/docs/images/94_usermenu.png and b/docs/images/94_usermenu.png differ diff --git a/docs/images/config-item-backup.png b/docs/images/config-item-backup.png index 1477d594..41a7f482 100644 Binary files a/docs/images/config-item-backup.png and b/docs/images/config-item-backup.png differ diff --git a/docs/images/config-item-bpf.png b/docs/images/config-item-bpf.png index 76b56483..19666a32 100644 Binary files a/docs/images/config-item-bpf.png and b/docs/images/config-item-bpf.png differ diff --git a/docs/images/config-item-elastalert-alerter.png b/docs/images/config-item-elastalert-alerter.png index b34edf73..73288f79 100644 Binary files a/docs/images/config-item-elastalert-alerter.png and b/docs/images/config-item-elastalert-alerter.png differ diff --git a/docs/images/config-item-elastalert.png b/docs/images/config-item-elastalert.png index 68601de0..354d2175 100644 Binary files a/docs/images/config-item-elastalert.png and b/docs/images/config-item-elastalert.png differ diff --git a/docs/images/config-item-elasticfleet.png b/docs/images/config-item-elasticfleet.png index 9f60eb83..3160a09b 100644 Binary files a/docs/images/config-item-elasticfleet.png and b/docs/images/config-item-elasticfleet.png differ diff --git a/docs/images/config-item-elasticsearch.png b/docs/images/config-item-elasticsearch.png index a5746d6c..b7c8a71d 100644 Binary files a/docs/images/config-item-elasticsearch.png and b/docs/images/config-item-elasticsearch.png differ diff --git a/docs/images/config-item-firewall.png b/docs/images/config-item-firewall.png index a2ceb122..b1a89158 100644 Binary files a/docs/images/config-item-firewall.png and b/docs/images/config-item-firewall.png differ diff --git a/docs/images/config-item-global-url.png b/docs/images/config-item-global-url.png index 93cb9399..f4ae2c6f 100644 Binary files a/docs/images/config-item-global-url.png and b/docs/images/config-item-global-url.png differ diff --git a/docs/images/config-item-global.png b/docs/images/config-item-global.png index 7074ab9c..2114094f 100644 Binary files a/docs/images/config-item-global.png and b/docs/images/config-item-global.png differ diff --git a/docs/images/config-item-host.png b/docs/images/config-item-host.png index c9f21136..9d4fe0af 100644 Binary files a/docs/images/config-item-host.png and b/docs/images/config-item-host.png differ diff --git a/docs/images/config-item-idh.png b/docs/images/config-item-idh.png index 407b643b..4b56a4d5 100644 Binary files a/docs/images/config-item-idh.png and b/docs/images/config-item-idh.png differ diff --git a/docs/images/config-item-influxdb.png b/docs/images/config-item-influxdb.png index 09ee0bfa..b29ea9b3 100644 Binary files a/docs/images/config-item-influxdb.png and b/docs/images/config-item-influxdb.png differ diff --git a/docs/images/config-item-kafka.png b/docs/images/config-item-kafka.png index ec87cc3d..b0e09366 100644 Binary files a/docs/images/config-item-kafka.png and b/docs/images/config-item-kafka.png differ diff --git a/docs/images/config-item-kibana.png b/docs/images/config-item-kibana.png index 5d8fb099..eebd99ea 100644 Binary files a/docs/images/config-item-kibana.png and b/docs/images/config-item-kibana.png differ diff --git a/docs/images/config-item-kratos.png b/docs/images/config-item-kratos.png index 64ed803f..a1bd9bd9 100644 Binary files a/docs/images/config-item-kratos.png and b/docs/images/config-item-kratos.png differ diff --git a/docs/images/config-item-logstash.png b/docs/images/config-item-logstash.png index 4f06bc6c..b3770385 100644 Binary files a/docs/images/config-item-logstash.png and b/docs/images/config-item-logstash.png differ diff --git a/docs/images/config-item-manager.png b/docs/images/config-item-manager.png index c94c133d..0b4d3d0c 100644 Binary files a/docs/images/config-item-manager.png and b/docs/images/config-item-manager.png differ diff --git a/docs/images/config-item-nginx.png b/docs/images/config-item-nginx.png index 7477c1cf..d69b319f 100644 Binary files a/docs/images/config-item-nginx.png and b/docs/images/config-item-nginx.png differ diff --git a/docs/images/config-item-ntp.png b/docs/images/config-item-ntp.png index 641dbed0..7f6a7017 100644 Binary files a/docs/images/config-item-ntp.png and b/docs/images/config-item-ntp.png differ diff --git a/docs/images/config-item-patch.png b/docs/images/config-item-patch.png index 4dc09656..cfd3866f 100644 Binary files a/docs/images/config-item-patch.png and b/docs/images/config-item-patch.png differ diff --git a/docs/images/config-item-redis.png b/docs/images/config-item-redis.png index 4012ef4b..bbf04408 100644 Binary files a/docs/images/config-item-redis.png and b/docs/images/config-item-redis.png differ diff --git a/docs/images/config-item-sensor.png b/docs/images/config-item-sensor.png index c9402ee6..593474b7 100644 Binary files a/docs/images/config-item-sensor.png and b/docs/images/config-item-sensor.png differ diff --git a/docs/images/config-item-sensoroni.png b/docs/images/config-item-sensoroni.png index 1ea67a98..a9a7c34d 100644 Binary files a/docs/images/config-item-sensoroni.png and b/docs/images/config-item-sensoroni.png differ diff --git a/docs/images/config-item-soc-additionalAlerters.png b/docs/images/config-item-soc-additionalAlerters.png index bad6d08c..da4390a8 100644 Binary files a/docs/images/config-item-soc-additionalAlerters.png and b/docs/images/config-item-soc-additionalAlerters.png differ diff --git a/docs/images/config-item-soc-subgrids.png b/docs/images/config-item-soc-subgrids.png index f2d2c1a1..3dec4024 100644 Binary files a/docs/images/config-item-soc-subgrids.png and b/docs/images/config-item-soc-subgrids.png differ diff --git a/docs/images/config-item-soc.png b/docs/images/config-item-soc.png index ef30fab6..76a828f5 100644 Binary files a/docs/images/config-item-soc.png and b/docs/images/config-item-soc.png differ diff --git a/docs/images/config-item-strelka.png b/docs/images/config-item-strelka.png index 2212dce8..e6afd1c7 100644 Binary files a/docs/images/config-item-strelka.png and b/docs/images/config-item-strelka.png differ diff --git a/docs/images/config-item-suricata.png b/docs/images/config-item-suricata.png index 65ed8af9..846222a7 100644 Binary files a/docs/images/config-item-suricata.png and b/docs/images/config-item-suricata.png differ diff --git a/docs/images/config-item-telegraf.png b/docs/images/config-item-telegraf.png index 14db39d2..cb044b7c 100644 Binary files a/docs/images/config-item-telegraf.png and b/docs/images/config-item-telegraf.png differ diff --git a/docs/images/config-item-versionlock.png b/docs/images/config-item-versionlock.png index 0f8bb30e..6a31ea60 100644 Binary files a/docs/images/config-item-versionlock.png and b/docs/images/config-item-versionlock.png differ diff --git a/docs/images/config-item-zeek.png b/docs/images/config-item-zeek.png index 8ea61cf1..18dc603e 100644 Binary files a/docs/images/config-item-zeek.png and b/docs/images/config-item-zeek.png differ diff --git a/docs/kernels.md b/docs/kernels.md new file mode 100644 index 00000000..b2b80d7d --- /dev/null +++ b/docs/kernels.md @@ -0,0 +1,39 @@ +# Kernels + +Security Onion standardizes on the Oracle Unbreakable Enterprise Kernel (UEK). Nodes run the UEK8 (6.x) kernel series, and the standard EL9 RedHat compatible kernel (RHCK, 5.14) is removed once a node is actually running UEK8. + +This is all automatic and you should not need to do anything. Nodes pick up the UEK8 kernel as part of their normal OS updates, and the stock EL9 kernel is removed for you on the next highstate after the node reboots onto UEK8. + +The removal is deliberately deferred until after the reboot. `dnf` refuses to erase the running kernel, and waiting also means the node has proven it boots on UEK8 before its fallback is deleted. Fresh installs reboot at the end of setup and are cleaned up on the first highstate after that; existing nodes are cleaned up whenever you reboot them. + +These are the packages removed: + +``` +kernel kernel-core kernel-modules kernel-modules-core kernel-tools kernel-tools-libs +``` + +!!! NOTE + + Older UEK7 `kernel-uek` 5.x packages are not removed. They age out on their own as newer kernels are installed. + +## Doing It Manually + +If a node did not end up on UEK8 on its own, you can run the following command on that node: + +``` +sudo so-kernel-upgrade +``` + +This installs the UEK8 kernel and makes it the boot default. It does not reboot the node, so you can schedule the reboot yourself. The new kernel does not take effect until you reboot. + +!!! NOTE + + The manager mirrors the UEK8 packages and serves them to the rest of the grid. If the manager has not synced them yet, `so-kernel-upgrade` on the manager will sync them for you; on any other node it will tell you to sync the manager first. + +Similarly, if the stock EL9 kernel is still installed on a node that is already running UEK8, you can run the cleanup directly instead of waiting for the next highstate: + +``` +sudo so-kernel-upgrade --cleanup +``` + +This does nothing unless the node is already running UEK8. diff --git a/docs/local-llm.md b/docs/local-llm.md new file mode 100644 index 00000000..6b2b3d68 --- /dev/null +++ b/docs/local-llm.md @@ -0,0 +1,334 @@ +# Local LLM Hosting + +!!! NOTE + + [Onion AI](onion-ai.md) is an enterprise-level feature of Security Onion. Contact Security Onion Solutions, LLC via our website at for more information about purchasing a [Security Onion Pro](security-onion-pro.md) license. + +[Onion AI](onion-ai.md) can connect to a model you host yourself through the *OpenAI Chat* adapter. This page walks through building such an endpoint on a single AMD Strix Halo machine using `llama.cpp`, which is packaged and GPU-accelerated on that platform. + +The result is an OpenAI-compatible API on your own network, protected by an API key, serving a model with a context window large enough for the assistant to work with. + +!!! NOTE + + Hosting your own model is optional. A local model will not match the accuracy of the proprietary foundational models available through the SOAI adapter. See [Local Model Considerations](onion-ai.md#local-model-considerations) before deciding. + +## Hardware + +The reference platform for this guide is an AMD Strix Halo system (Ryzen AI Max series) with 128GB of unified memory. + +Strix Halo is a good fit for this task because its memory is shared between the CPU and the integrated GPU. Rather than being limited to the memory on a discrete graphics card, the GPU can address the bulk of system RAM, which is what makes it possible to hold a large model and a large context window at once on a single, relatively inexpensive machine. + +This guide assumes the AMD Ryzen AI Developer Platform operating system, which is Debian-based and ships with the ROCm stack and GPU-enabled `llama.cpp` packages already available. + +## Verify GPU Access + +Before installing anything, confirm the GPU is visible to the ROCm backend: + +```bash +llama-cli --list-devices +``` + +You should see a ROCm device with most of your system memory available to it: + +```text +Available devices: + ROCm0: Radeon 8060S Graphics (96454 MiB, 124565 MiB free) +``` + +If no ROCm device is listed, the GPU backend is not working and nothing later in this guide will run on the GPU. + +### Available GPU Memory + +The amount of memory the GPU may address is the GTT (Graphics Translation Table) size: + +```bash +cat /sys/class/drm/card0/device/mem_info_gtt_total +``` + +On the reference platform this reports roughly 101GB out of 128GB of system memory, which is ample and requires no tuning. If your system reports a much smaller value, you can raise it with the `amdgpu.gttsize` and `ttm.pages_limit` kernel parameters, or by increasing the memory allocated to graphics in your system firmware. + +## Install llama.cpp + +```bash +sudo apt install llama.cpp +``` + +This is a small metapackage that pulls in `llama.cpp-tools`, which provides `llama-server` and `llama-cli`. The HIP backend that provides GPU acceleration comes from the same vendor repository, so it stays current through normal system updates. + +The package also installs a `llama-server` systemd service, which is what you will configure below rather than running the server by hand. + +## Choose a Model + +This guide uses **Gemma 4 26B A4B**, a mixture-of-experts model with 26 billion total parameters but only about 4 billion active per token. That combination suits this hardware well: the full model must fit in memory, but the amount of data read per generated token stays small, which is what governs speed on a memory-bandwidth-limited system. + +It also supports a 262,144 token context window, comfortably above the 128k minimum recommended for the assistant. + +Pre-converted GGUF files are published by Unsloth. Pick a quantization based on how you want to trade accuracy against speed and memory: + +| Quantization | Size | Notes | +|---|---|---| +| `unsloth/gemma-4-26B-A4B-it-qat-GGUF:UD-Q4_K_XL` | 14.2 GB | Quantization-aware trained by Google. Best quality available at 4-bit, and the fastest option. | +| `unsloth/gemma-4-26B-A4B-it-GGUF:UD-Q4_K_M` | 16.9 GB | Standard 4-bit. | +| `unsloth/gemma-4-26B-A4B-it-GGUF:Q8_0` | 26.9 GB | Effectively indistinguishable from full precision. The recommended default when you have the memory. | +| `unsloth/gemma-4-26B-A4B-it-GGUF:BF16` | 50.5 GB | Full precision. Fits, but offers no meaningful accuracy gain over `Q8_0` while running slower. | + +!!! TIP + + Start with `Q8_0`. On a 128GB machine it leaves plenty of room for a full-size context window, and accuracy matters more than raw speed for security analysis. Move to the QAT 4-bit build if you need faster responses or want to leave more memory free. + +## Generate an API Key + +Never expose an inference endpoint without authentication. Generate a random key and store it in a file that only the service can read: + +```bash +sudo install -d -m 0755 /etc/llama-server +openssl rand -hex 32 | sudo tee /etc/llama-server/api-key > /dev/null +sudo chown root:_llama-server /etc/llama-server/api-key +sudo chmod 0640 /etc/llama-server/api-key +``` + +Display the key when you need it for the Onion AI adapter configuration: + +```bash +sudo cat /etc/llama-server/api-key +``` + +## Grant the Service Access to the GPU + +The packaged service runs as the unprivileged `_llama-server` user. Interactive login sessions are granted access to the GPU devices automatically through an ACL, but system services are not, so the service account must be added to the `render` and `video` groups: + +```bash +sudo usermod -aG render,video _llama-server +``` + +!!! WARNING + + This step is easy to miss. Without it the service starts and appears healthy, but silently falls back to CPU inference. The giveaway is `failed to initialize ROCm: no ROCm-capable device is detected` in the service log, followed by `no usable GPU found, --gpu-layers option will be ignored`. On this hardware, CPU-only inference is too slow to be usable. + +## Configure the Server + +The service reads its settings from `/etc/default/llama-server` as `LLAMA_ARG_*` environment variables, each corresponding to a `llama-server` command line option. Replace the contents of that file with: + +```bash +LLAMA_ARG_HOST=0.0.0.0 +LLAMA_ARG_PORT=8080 +LLAMA_ARG_HF_REPO=unsloth/gemma-4-26B-A4B-it-GGUF:Q8_0 +LLAMA_ARG_ALIAS=gemma-4-26b +LLAMA_ARG_CTX_SIZE=262144 +LLAMA_ARG_N_GPU_LAYERS=99 +LLAMA_ARG_FLASH_ATTN=on +LLAMA_ARG_N_PARALLEL=1 +LLAMA_ARG_API_KEY_FILE=/etc/llama-server/api-key +``` + +What these do: + +- **`LLAMA_ARG_HOST`** -- binds to all interfaces so the Security Onion manager can reach the endpoint. See [Network Access](#network-access) below. +- **`LLAMA_ARG_HF_REPO`** -- the model to serve. It is downloaded automatically on first start into `/var/cache/llama-server`. +- **`LLAMA_ARG_ALIAS`** -- the name the API reports for the model. This is the value you will enter as the model identifier in Onion AI. +- **`LLAMA_ARG_CTX_SIZE`** -- the context window, in tokens. `262144` is the maximum this model supports. +- **`LLAMA_ARG_N_GPU_LAYERS`** -- any value at or above the model's layer count offloads the whole model to the GPU. +- **`LLAMA_ARG_N_PARALLEL`** -- how many requests are served concurrently. The context window is divided among these slots, so leave it at `1` to give a single conversation the entire window. + +!!! NOTE + + Do not set `LLAMA_ARG_SWA_FULL`. Most of this model's layers use sliding-window attention, and leaving that option off allows the server to allocate a much smaller cache for them. Enabling it would make a 262,144 token context far more expensive in memory for no benefit. + +Tool calling, which the assistant depends on, requires the Jinja chat template engine. It is enabled by default, so no setting is needed, but do not disable it. + +## Start the Server + +```bash +sudo systemctl enable --now llama-server +``` + +The first start downloads the model, which takes a while. Watch its progress: + +```bash +journalctl -u llama-server -f +``` + +Once the model has loaded, confirm the server is answering: + +```bash +curl -s -H "Authorization: Bearer $(sudo cat /etc/llama-server/api-key)" \ + http://127.0.0.1:8080/v1/models +``` + +## Network Access + +The Security Onion manager connects to this endpoint over your network, so the port must be reachable from it. + +!!! WARNING + + The configuration above serves plain HTTP. The API key is sent as a bearer token in clear text and is readable by anyone who can observe the traffic, as are the prompts and responses, which will contain data from your grid. + + Restrict access to the manager and terminate TLS in front of the server before using this outside an isolated lab network. + +Limit which hosts may reach the port. If the machine has no firewall configured, allowing only the manager is a reasonable starting point: + +```bash +sudo nft add table inet filter +sudo nft add chain inet filter input '{ type filter hook input priority 0; }' +sudo nft add rule inet filter input tcp dport 8080 ip saddr != drop +``` + +For anything beyond a lab, put a reverse proxy such as nginx in front of `llama-server` to terminate TLS, and bind `llama-server` itself to `127.0.0.1` by setting `LLAMA_ARG_HOST=127.0.0.1`. + +## Connect to Onion AI + +With the endpoint running, configure Security Onion to use it. This mirrors the general instructions in [Onion AI](onion-ai.md#configuration). + +### Add the Adapter + +Go to Administration --> Configuration, turn on *Show advanced settings* in the Options at the top of the page, and navigate to soc --> config --> server --> modules --> assistant --> adapters. Add an adapter: + +| Field | Value | +|---|---| +| Adapter Name | `local` | +| Protocol | *OpenAI Chat* | +| API Url | `http://:8080/v1/` | +| API Key | the contents of `/etc/llama-server/api-key` | +| Health Timeout Seconds | `120` | + +!!! NOTE + + Use the *OpenAI Chat* adapter, not *OpenAI Responses*. `llama-server` implements the Chat Completions protocol. + + A generous health timeout is appropriate here. A locally hosted model on this class of hardware responds more slowly than a cloud provider, particularly when processing a large context. + +### Add the Model + +Navigate to soc --> config --> server --> client --> assistant --> availableModels and add an entry: + +| Field | Value | +|---|---| +| Name | `Gemma 4 26B (local)` | +| Adapter | `local` | +| Model | `gemma-4-26b` | + +The model identifier must match the `LLAMA_ARG_ALIAS` value you configured, since that is the name the server reports over the API. + +Finally, make sure the assistant itself is enabled at soc --> config --> server --> client --> assistant --> enabled, then open the assistant in SOC and select your new model. + +## Reasoning Output + +Gemma 4 is a reasoning model. It works through a problem before committing to an answer, and `llama-server` returns that internal reasoning in a separate `reasoning_content` field, leaving `content` empty until the reasoning is finished. + +The practical consequence is that a request must be allowed enough tokens to finish thinking *and* answer. If the limit is too low, the response comes back with `finish_reason` of `length`, an empty `content`, and the answer stranded mid-thought. A short question can easily spend a few hundred tokens reasoning before producing a one-sentence reply. + +!!! TIP + + If you see empty responses in the assistant, this is the first thing to check. Raise the token limit rather than assuming the model or the connection is broken. + +## Verify the Endpoint + +Confirm authentication is enforced. A request with no key, or the wrong key, must be rejected: + +```bash +curl -s -o /dev/null -w "%{http_code}\n" \ + -H "Content-Type: application/json" \ + http://127.0.0.1:8080/v1/chat/completions \ + -d '{"model":"gemma-4-26b","messages":[{"role":"user","content":"hi"}],"max_tokens":5}' +``` + +This should print `401`. + +!!! NOTE + + The `/v1/models` endpoint is deliberately not protected by the API key and will answer without one. It exposes only the model name. Inference endpoints require the key. + +Now make a real request: + +```bash +curl -s -H "Authorization: Bearer $(sudo cat /etc/llama-server/api-key)" \ + -H "Content-Type: application/json" \ + http://127.0.0.1:8080/v1/chat/completions \ + -d '{"model":"gemma-4-26b","messages":[{"role":"user","content":"In one sentence, what is Zeek used for?"}],"max_tokens":2000}' +``` + +The response includes a `timings` object reporting `prompt_per_second` and `predicted_per_second`, which is the simplest way to measure throughput on your own hardware. + +### Confirm Tool Calling + +The assistant relies on tools such as `query_events` and `ack_alerts` to reach data in your grid, so tool calling has to work. Send a request with a tool definition and confirm the model responds with a `tool_calls` entry rather than plain text: + +```bash +curl -s -H "Authorization: Bearer $(sudo cat /etc/llama-server/api-key)" \ + -H "Content-Type: application/json" \ + http://127.0.0.1:8080/v1/chat/completions -d '{ + "model": "gemma-4-26b", + "messages": [{"role":"user","content":"Are there alerts from source IP 10.1.2.3 in the last 24 hours? Use the available tools."}], + "tools": [{"type":"function","function":{ + "name":"query_events", + "description":"Query security events from the local Security Onion instance.", + "parameters":{"type":"object","properties":{"query":{"type":"string"},"range":{"type":"string"}},"required":["query"]} + }}], + "max_tokens": 2000 + }' +``` + +A correct result contains a `tool_calls` array naming `query_events` with populated arguments. + +## Performance Expectations + +The following were measured on the reference platform serving the `Q8_0` build with a 262,144 token context window. Your results will vary with hardware and quantization. + +| Measure | Result | +|---|---| +| Memory in use | ~32 GB of unified memory (26.9 GB of weights, the remainder cache and buffers) | +| Generation, short context | ~41 tokens/sec | +| Generation, ~193,000 token context | ~22 tokens/sec | +| Prompt processing, bulk | ~850 tokens/sec initially, ~228 tokens/sec averaged over a 193,000 token prompt | + +Two characteristics of this hardware are worth planning around. + +**Prompt processing slows as the context fills.** Bulk prompt processing starts around 850 tokens/sec but falls steadily with depth. A 193,139 token prompt took roughly 14 minutes to process before generation began, and generation itself ran at about half the speed it does on a short prompt. Large contexts are practical, but they are not fast. This is the main reason to set a generous Health Timeout on the adapter. + +**Memory is not the limiting factor here.** The model and a full-size context together use roughly a third of the available memory, which is why a 262,144 token window is practical on this machine. There is room for a larger model or additional concurrent slots if you need them. + +The context window is usable in practice, not just allocatable: in testing, a fact placed at the very beginning of a 193,139 token prompt was retrieved correctly when asked about at the end. + +!!! NOTE + + `LLAMA_ARG_N_PARALLEL` divides the context window among concurrent slots rather than adding capacity. Two slots on a 262,144 token window give each request 131,072 tokens. Raise the context size alongside the slot count if you need several analysts working at once with large contexts. + +## Troubleshooting + +Check the service log first: + +```bash +journalctl -u llama-server -n 100 --no-pager +``` + +**`failed to initialize ROCm: no ROCm-capable device is detected`**, followed by `no usable GPU found, --gpu-layers option will be ignored` + +The service account cannot reach the GPU devices. Confirm it is in the right groups and restart: + +```bash +id -nG _llama-server +sudo usermod -aG render,video _llama-server +sudo systemctl restart llama-server +``` + +Note that running `llama-cli --list-devices` from your own shell may still show the GPU even while the service cannot use it, because interactive sessions get access through a separate mechanism. Always confirm against the service log. + +**`request (N tokens) exceeds the available context size (262144 tokens)`** + +The prompt is larger than the configured window. Either reduce what is being sent or raise `LLAMA_ARG_CTX_SIZE`, keeping in mind the model's own 262,144 token limit. + +**Responses are empty and `finish_reason` is `length`** + +The token limit was reached while the model was still reasoning. See [Reasoning Output](#reasoning-output). + +**The service starts but the model never loads** + +The first start downloads the model, which is tens of gigabytes and can take a long time. Watch the cache directory grow: + +```bash +sudo du -sh /var/cache/llama-server +``` + +**Onion AI reports the adapter as unhealthy** + +Confirm the manager can reach the port, and raise the adapter's Health Timeout. A local model can take longer to respond than the default timeout allows, particularly on a large context. diff --git a/docs/logstash.md b/docs/logstash.md index 3d7547fd..94a8459b 100644 --- a/docs/logstash.md +++ b/docs/logstash.md @@ -71,7 +71,7 @@ In [SOC](security-onion-console.md), navigate to [Administration](administration custom/myfile.conf ``` -The configuration will be applied at the next scheduled highstate (see [Highstate Interval](salt.md#highstate-interval)) or you can apply it immediately by clicking the `SYNCHRONIZE GRID` button under the `Options` menu. +[Auto State Apply](salt.md#auto-state-apply) will apply the configuration within a few minutes. If you don't want to wait, click the `SYNCHRONIZE GRID` button under the `Options` menu. You can monitor events flowing through the output by running the following command on the manager: @@ -82,7 +82,7 @@ curl -s localhost:9600/_node/stats | jq .pipelines.manager ## Modified Event Forwarding -To forward events to an external destination AFTER they have traversed the Logstash pipelines (NOT ingest node pipelines), perform the same steps as above but instead of adding the reference for your Logstash output to the `manager` pipeline add it to `search` pipeline instead. The configuration will be applied at the next scheduled highstate (see [Highstate Interval](salt.md#highstate-interval)) or immediately by clicking the `SYNCHRONIZE GRID` button under the `Options` menu. +To forward events to an external destination AFTER they have traversed the Logstash pipelines (NOT ingest node pipelines), perform the same steps as above but instead of adding the reference for your Logstash output to the `manager` pipeline add it to `search` pipeline instead. [Auto State Apply](salt.md#auto-state-apply) will apply the configuration within a few minutes. If you don't want to wait, click the `SYNCHRONIZE GRID` button under the `Options` menu. You can monitor events flowing through the output by running the following command on the search nodes: diff --git a/docs/manager-of-managers.md b/docs/manager-of-managers.md index 3b7e3761..49575172 100644 --- a/docs/manager-of-managers.md +++ b/docs/manager-of-managers.md @@ -55,7 +55,7 @@ While on the API Client screen, click the ⤓ icon to download the Certificate A !!! NOTE - Please be aware that any users in the MoM Grid will be able to connect to the subgrid using the permissions defined for the API client. For example, suppose that you create an API client ID in the subgrid called `supermom` and you grant it all permissions. Once the MoM is configured to connect to the subgrid as shown in the next section, then any users in the MoM Grid will connect to the subgrid as `supermom` and have all permissions to the subgrid regardless of whether the user has equivalent permissions in the MoM. + Starting in release 3.3.0, all MoM users that are not already superusers must be granted either the `subgrid-auditor` or `subgrid-superuser` role in order to interact with subgrids. The `subgrid-auditor` role will permit read operations (HTTP GET API requests), and `subgrid-superuser` will permit read and write operations (HTTP GET, POST, PUT, PATCH, and DELETE API requests) against the subgrids. These requests to the subgrid will all utilize the same subgrid API client credentials. As an example, a user with `subgrid-auditor` can see cases on the subgrid, but cannot modify them. Whereas a user with `subgrid-superuser` can modify cases on the subgrid, as well as manage subgrid users and their roles. ### Subgrid Config diff --git a/docs/onion-ai.md b/docs/onion-ai.md index 4ba5c5e0..d5d621b8 100644 --- a/docs/onion-ai.md +++ b/docs/onion-ai.md @@ -103,7 +103,7 @@ Security Onion now supports local models through any OpenAI-compatible endpoint. ## Hosting Local Models -Hosting your own models requires powerful and expensive hardware. For beginners we recommend using a tool such as LM Studio. **You need at least 96GB of VRAM** to host your own models locally. The speed and accuracy of OnionAI when hosted locally is based on the hardware that you are using. For the most accurate results we recommend using credits with OnionAI. +Hosting your own models requires powerful and expensive hardware. For beginners we recommend using a tool such as LM Studio. The models listed above are large, and **you need at least 96GB of VRAM** to host them locally. Smaller mixture-of-experts models can run in considerably less memory while still offering a context window large enough for the assistant; see [Local LLM Hosting](local-llm.md) for a tested walkthrough on a single machine. The speed and accuracy of OnionAI when hosted locally is based on the hardware that you are using. For the most accurate results we recommend using credits with OnionAI. ## Available Tools @@ -155,3 +155,63 @@ Your system prompt addendum will be added after Security Onion's default system Superusers can review token usage and conversation history for all users by going to Administration --> AI Metrics. This page provides usage statistics for a given date range. The page starts with a table of usage by user. Clicking a user's binoculars icon on the right hand side will show any sessions the user interacted with during the selected date range, even deleted sessions. Clicking on a session's binoculars icon will show the full conversation. Administrators can adjust who has permissions via RBAC roles. To provide an accurate history, deleted sessions are retained on the metrics page even after being deleted by the user. + +## Memory + +OnionAI can utilize its memory system to retain and reference facts disclosed by users when chatting with the assistant. Memory works by scanning sessions in the background looking for new facts and storing them in a database. The memory is then used to provide contextual information to the assistant when it responds to a user's message. + +### How Memory Works + +The memory system scans sessions in the background extracting, embedding, reconciling, and saving memories to later be referenced when chatting with Onion AI. + +First, extraction. The selected sessions have their transcripts given to a Memory agent that parses out useful facts. The agent also identifies the scope of the fact: is it specific to the user who owns the session we just scanned or is it applicable to the entire organization. + +These facts are then embedded using an Embed agent. This enables SOC to compare memories and is the heart of how memory works. + +!!! NOTE + + The embedding process is model specific. Memories embedded by one model cannot be accurately compared to memories embedded by any other model. Changing the model used for embedding will render all your previous memories unusable. They will not be deleted and will be accessible if the model is reverted back. + +Once embedded, the facts are compared against existing memories on the same topics in a step called Reconcilation. A Reconcile agent will decide how the facts should be added, merged, replaced or removed. The reconcile agent's recommendations are validated before being executed to ensure that user defined memories aren't modified and that the session owner's permissions are respected. + +If any facts were rewritten during reconcilation, they are re-embedded before being stored in postgres. + +Finally the session is updated indicating that it has been scanned. If new messages are sent in a previously scanned session, memory scans will find and scan only the new messages. + +When a user sends a message to the assistant, the memory system will embed the message about to be sent and check for any similar memories. Memories are added to the conversation by being added to the end of the prompt. + +### Configuring Memory + +| Name | Default Value | Description | +|---|---|---| +| `useMemory` | `true` | Enable memory use when sending a message. | +| `useMemoryScanner` | `false` | Enables the scanning of historical sessions. | +| `dontScanBefore` | `""` | A date in RFC3339 format (2026-08-31T22:05:48Z) to use as a cutoff point for memory scans. | +| `memoryScanIntervalSeconds` | `300` | How long between scans the memory scanner waits before scanning again. | +| `memoryProximityThreshold` | `0.8` | A similarity threshold that determines how similar a fact must be to a previous memory for it to be considered "similar." | +| `messageProximityThreshold` | `0.5` | A similarity threshold that determines how similar a message from a user must be to a stored memory for it to be considered "similar." | +| `maxUserMemoriesToReconcile` | `20` | The maximum number of user-specific memories to reconcile at once. | +| `maxGlobalMemoriesToReconcile` | `20` | The maximum number of global memories to reconcile at once. | +| `maxUserMemoriesToInclude` | `5` | The maximum number of user-specific memories to include when sending a message. | +| `maxGlobalMemoriesToInclude` | `5` | The maximum number of global memories to include when sending a message. | +| `memoryModel` | `gemma@SOAI` | The model used to extract facts from sessions. | +| `memoryPersona` | `""` | Special instructions for the memory agent included in the prompt. | +| `embedModel` | `amazon.titan-embed-text-v2@SOAI` | The model used to embed facts. | +| `reconcileModel` | `gemma@SOAI` | The model used to reconcile facts from sessions with existing memories. | +| `reconcilePersona` | `""` | Special instructions for the reconcile agent included in the prompt. | +| `toolUseTurnAttempts` | `12` | When the assistant requests a read-only tool, SOC can approve it automatically. Because the approval can fire before the original request has finished being written to Elasticsearch, SOC will retry the approval up to this many times before giving up. | +| `toolUseTurnDelayMs` | `175` | The time to wait between auto-approval attempts. Together with the attempts setting, this defines the total grace period SOC allows for the tool request to become available. | +| `maxMemoryRetries` | `2` | The maximum number of times SOC will attempt to extract memories from sessions before marking the session to be ignored. Increasing this value may retry sessions that haven't been attempted in a long time. | + +Memory and the Memory Scanner may be enabled independently. Disabling memory will stop applying memories to prompts on outgoing messages. Disabling the memory scanner will prevent the scanner from extracting memories from previous assistant sessions. The memory scanner marks sessions as it extracts facts from them so that they are not re-scanned in future scans. If a memory scan takes longer than the interval between scans, then at most 1 scan will queue up for processing and it'll begin again immediately after the previous scan finishes. + +### How to Write a Good Memory + +When entering memories in manually in the Agent Studio, it's important to understand that how the memory is written affects how it'll be used by the memory system. + +1. **One fact per memory.** If a memory has multiple facts then matching is harder to accomplish resulting in the memory being accessed less. +1. **Make each memory self-contained.** It will be read out of context, alone, possibly months later in a conversation about something else. Every memory should make sense to someone who has only that sentence. +1. **Declarative facts, not instructions.** If you want to give an instruction, then use a prompt instead. +1. **Present tense, absolute references, durable phrasing.** A memory is read long after it's written and is never automatically updated, so describe how things are rather than how they changed. Prefer "PCAP retention is 90 days (set August 2026)" over "We recently changed retention to 90 days." + +To help track your most successful memories, the Agent Studio presents how many times a memory has been accessed and when was the last time it was referenced. diff --git a/docs/rbac.md b/docs/rbac.md index fea1901c..920611d8 100644 --- a/docs/rbac.md +++ b/docs/rbac.md @@ -6,47 +6,62 @@ RBAC in Security Onion covers both Security Onion privileges and Elastic stack p ## Default Roles -Security Onion ships with the following user roles: `superuser`, `analyst`, `limited-analyst`, `auditor`, and `limited-auditor`. +Security Onion ships with the following user roles: `superuser`, `analyst`, `limited-analyst`, `auditor`, `limited-auditor`, `subgrid-auditor`, and `subgrid-superuser`. See the table below which explains the specific Security Onion privileges granted to each role. -| | superuser | analyst | limited-analyst | auditor | limited-auditor | -|---|:---:|:---:|:---:|:---:|:---:| -| View alerts | X | X | X | X | X | -| Acknowledge alerts | X | X | X | | | -| Escalate alerts and events | X | X | X | | | -| View detections | X | X | X | X | X | -| Modify (and Delete) detections | X | X | | | | -| View events in Hunt | X | X | X | X | X | -| View own PCAP jobs | X | X | X | O | O | -| View all PCAP jobs | X | X | | X | | -| Pivot to PCAP job from event | X | X | X | | | -| Request arbitrary PCAP jobs | X | X | | | | -| Delete own PCAP job | X | X | X | O | O | -| Delete any PCAP job | X | X | | | | -| View all nodes in Grid | X | X | X | X | X | -| View all users | X | X | | X | | -| View all users' roles | X | X | | X | | -| View own user | X | X | X | X | X | -| View own user roles | X | X | X | X | X | -| Change own password | X | X | X | X | X | -| Add, update, and reset SOC users | X | | | | | -| Modify and synchronize Grid config | X | | | | | -| Manage Grid membership of nodes | X | | | | | -| Initiate node actions (Ex: reboot) | X | | | | | -| Manage API clients | X | | | | | -| View and list existing API clients | X | | X | | -| Manage Active Queries | X | | | | | -| View Playbooks | X | X | X | X | X | -| Chat with Onion AI | X | X | | | | -| Delete Own Onion AI Sessions | X | X | | | | -| View Own Onion AI History | X | X | | | | -| View All Users' Onion AI History | X | | | | | +| | superuser | analyst | limited-analyst | auditor | limited-auditor | subgrid-superuser | subgrid-auditor | +|---|:---:|:---:|:---:|:---:|:---:|:---:|:---:| +| View alerts | X | X | X | X | X | | | +| Acknowledge alerts | X | X | X | | | | | +| Escalate alerts and events | X | X | X | | | | | +| View detections | X | X | X | X | X | | | +| Modify (and Delete) detections | X | X | | | | | | +| View events in Hunt | X | X | X | X | X | | | +| View own PCAP jobs | X | X | X | O | O | | | +| View all PCAP jobs | X | X | | X | | | | +| Pivot to PCAP job from event | X | X | X | | | | | +| Request arbitrary PCAP jobs | X | X | | | | | | +| Delete own PCAP job | X | X | X | O | O | | | +| Delete any PCAP job | X | X | | | | | | +| View all nodes in Grid | X | X | X | X | X | | | +| View all users | X | X | | X | | | | +| View all users' roles | X | X | | X | | | | +| View own user | X | X | X | X | X | | | +| View own user roles | X | X | X | X | X | | | +| Change own password | X | X | X | X | X | | | +| Add, update, and reset SOC users | X | | | | | | | +| Modify and synchronize Grid config | X | | | | | | | +| Manage Grid membership of nodes | X | | | | | | | +| Initiate node actions (Ex: reboot) | X | | | | | | | +| Manage API clients | X | | | | | | | +| View and list existing API clients | X | | X | | | | +| Manage Active Queries | X | | | | | | | +| View Playbooks | X | X | X | X | X | | | +| Chat with Onion AI | X | X | | | | | | +| Delete Own Onion AI Sessions | X | X | | | | | | +| View Own Onion AI History | X | X | | | | | | +| View All Users' Onion AI History | X | | | | | | | +| View own Onion AI memories | X | X | | X | | | | +| View global Onion AI memories | X | | | X | | | | +| View all users' Onion AI memories | X | | | X | | | | +| Manage own Onion AI memories | X | X | | | | | | +| Manage global Onion AI memories | X | | | | | | | +| Manage all users' Onion AI memories | X | | | | | | | +| View notifications | X | X | X | X | X | | | +| View all notifications | X | | | X | | | | +| Manage notifications | X | X | X | | | | | +| Read subgrid data | X | | | | | X | X | +| Modify subgrid data | X | | | | | X | | !!! NOTE Both `auditor` and `limited-auditor` roles can interact with previously created PCAPs if they were created before a user was converted to that role (e.g. user was downgraded from `analyst` to `auditor`). This is denoted by **O** in the above table. +!!! NOTE + + The `subgrid-auditor` and `subgrid-superuser` roles are used in [Manager of Managers](manager-of-managers.md) deployments to grant read-only or read/write access to remote subgrids. Users needing subgrid access should be assigned one of these roles in addition to their primary role (e.g., `analyst`). + !!! NOTE A system role called `agent` is used by the Security Onion agent that runs on each node of the Security Onion Grid. This special role is given the *jobs/process*, *nodes/read*, and *nodes/write* permissions (defined at the bottom of this page). Avoid creating custom roles that share the same name as Security Onion-provided roles. @@ -171,7 +186,7 @@ These steps will guide you through an example where we wish to introduce a new r sudo so-elasticsearch-query _security/privilege/kibana-.Kibana | jq '. | map_values(keys)' ``` -3. Role files are not detected by [Auto State Apply](salt.md#auto-state-apply), so this step is required. Run so-checkin from the manager: +3. [Auto State Apply](salt.md#auto-state-apply) picks up the new role file within a few minutes. If you don't want to wait, run so-checkin from the manager: ``` @@ -245,6 +260,17 @@ The available low-level Security Onion privileges are listed in the table below: | *playbooks/read* | View all playbooks | | *playbooks/write* | Currently unused | | *playbooks/delete* | Currently unused | +| *subgrid/read* | Read data from subgrids | +| *subgrid/write* | Modify data on subgrids | +| *notifications/read* | View notifications | +| *notifications/read_all* | View all notifications | +| *notifications/write* | Create and update notifications | +| *memory/read_authored* | View own Onion AI memories | +| *memory/read_global* | View global Onion AI memories | +| *memory/read_all* | View all Onion AI memories | +| *memory/write_self* | Create and update own Onion AI memories | +| *memory/write_global* | Create and update global Onion AI memories | +| *memory/write_all* | Create and update all Onion AI memories | | *assistant/read_authored* | View own Onion AI conversation history | | *assistant/write_authored* | Chat with Onion AI | | *assistant/delete_authored* | Delete own Onion AI conversation history | @@ -281,6 +307,15 @@ These discrete privileges are then collected into privilege groups as defined be | user-monitor | *roles/read*, *users/read* | | playbook-monitor | *playbooks/read* | | playbook-admin | *playbooks/read*, *playbooks/write*, *playbooks/delete* | +| subgrid-admin | *subgrid/read*, *subgrid/write* | +| subgrid-monitor | *subgrid/read* | +| notification-admin | *notifications/read*, *notifications/write* | +| notification-monitor | *notifications/read* | +| notification-auditor | *notifications/read*, *notifications/read_all* | +| memory-user | *memory/read_authored*, *memory/write_self* | +| memory-curator | *memory/read_authored*, *memory/read_global*, *memory/write_self*, *memory/write_global* | +| memory-admin | *memory/read_authored*, *memory/read_global*, *memory/read_all*, *memory/write_self*, *memory/write_global*, *memory/write_all* | +| memory-monitor | *memory/read_authored*, *memory/read_global*, *memory/read_all* | | assistant-user | *assistant/read_authored*, *assistant/write_authored*, *assistant/delete_authored*, *assistant/read_shared* | | assistant-admin | *assistant/read_authored*, *assistant/write_authored*, *assistant/delete_authored*, *assistant/read_shared*, *assistant/read_all*, *assistant/write_all*, *assistant/delete_all* | | assistant-monitor | *assistant/read_authored*, *assistant/read_shared*, *assistant/read_all* | diff --git a/docs/release-notes.md b/docs/release-notes.md index a78510ef..0788dc00 100644 --- a/docs/release-notes.md +++ b/docs/release-notes.md @@ -2,18 +2,53 @@ ### Known Issues -[Auto State Apply](salt.md#auto-state-apply) detects configuration changes saved in [Administration](administration.md) --> Configuration and rule updates on the manager node. Files that you create or edit directly under `/opt/so/saltstack/local/salt/` are not detected. After adding or changing any of the following, apply the relevant state from the manager or wait for the next scheduled highstate (see [Highstate Interval](salt.md#highstate-interval)): - -- [Zeek](zeek.md) intel in `/opt/so/saltstack/local/salt/zeek/policy/intel/` -- [Zeek](zeek.md) custom packages in `/opt/so/saltstack/local/salt/zeek/zkg/` -- [Elasticsearch](elasticsearch.md) custom ingest parsers in `/opt/so/saltstack/local/salt/elasticsearch/files/ingest/` -- [RBAC](rbac.md) custom Elastic stack role files in `/opt/so/saltstack/local/salt/elasticsearch/roles/` -- [Logstash](logstash.md) custom pipeline configuration files in `/opt/so/saltstack/local/salt/logstash/pipelines/config/custom/` - For all other known issues, please see . ### Release History +3.3.0 [20260908] Changes +---------------------- + +- FEATURE: 508 AA - Contrast +- FEATURE: Add subgrid roles for MoM grids +- FEATURE: Advanced Agent Studio +- FEATURE: Agentic memory +- FEATURE: Allow for tuning logstash log.level and log.format #16183 +- FEATURE: Allow for tuning multiple Logstash pipelines in SOC #15090 +- FEATURE: ES Troubleshoot from Grid Screen #15432 +- FEATURE: Improve keyboard trap handling for 508 compliance +- FEATURE: Initial notification framework +- FEATURE: Onion AI Reports Tools +- FEATURE: Reduce reboot frequency by removing non-UEK kernels #16208 +- FEATURE: Support drag/drop for config list ordering, such as custom queries +- FIX: Avoid highlighting multiple menu entries #16129 +- FIX: Correct API /.well-known/* paths to properly resolve #16198 +- FIX: Hypervisor not running first highstate #14947 +- FIX: Improve HTTP authorization and security +- FIX: Improve PCAP parsing and viewing +- FIX: Improve upload size enforcement logic #16197 +- FIX: In Cases -> Events tab the Hunt icons link to ?q=[object+Object] instead of valid hunt query #16177 +- FIX: In Hunt interface the Basic Metrics section minimized state does not persist across page refreshes #16154 +- FIX: Local state files not triggering auto state apply #16136 +- FIX: Logrotate for hypervisor +- FIX: Onion AI Reports Font Key Parsing Error +- FIX: Remove usage of scripted fields +- FIX: Re-running so-setup on airgap installs fails +- FIX: so-boot-highstate.service is never enabled on non-manager nodes #16166 +- FIX: SOC Alerts interface in 3.x field and values not selectable, only values are selectable #16155 +- FIX: Soup error due to template DBs collation mismatch #16138 +- FIX: Soup force remote nodes to highstate #16137 +- FIX: Switch telegraf command vars to array #16020 +- FIX: Zeek LogExpireInterval value available in SOC but requires zeekctl cron to be run manually #16156 +- UPGRADE: CyberChef to 11.3.0 #16128 +- UPGRADE: Elasticsearch 9.4.5 +- UPGRADE: PrismJS to 1.30 #16134 +- UPGRADE: Salt-bootstrap +- UPGRADE: SOC Go dependencies #16186 +- UPGRADE: Strelka Go components to latest commit #16141 +- UPGRADE: Telegraf to 1.39.3 #16196 +- UPGRADE: Zeek to 8.0.10 #16171 + 3.2.0 [20260729] Changes ---------------------- diff --git a/docs/salt.md b/docs/salt.md index 3632043c..78289a77 100644 --- a/docs/salt.md +++ b/docs/salt.md @@ -43,6 +43,18 @@ sudo so-checkin When you save a configuration change in [Administration](administration.md) --> Configuration, or when rules are updated on the manager node, Security Onion detects the change and applies the affected state to only the nodes that need it. This typically happens within a few minutes instead of waiting for the next scheduled highstate. +Files that you create or edit by hand on the manager in any of the following directories are detected the same way: + +| Directory | What it holds | +|-----------|---------------| +| `/opt/so/saltstack/local/salt/zeek/policy/` | [Zeek](zeek.md) intel and custom policies | +| `/opt/so/saltstack/local/salt/zeek/zkg/` | [Zeek](zeek.md) custom packages | +| `/opt/so/saltstack/local/salt/elasticsearch/files/ingest/` | [Elasticsearch](elasticsearch.md) custom ingest parsers | +| `/opt/so/saltstack/local/salt/elasticsearch/roles/` | [RBAC](rbac.md) custom Elastic stack role files | +| `/opt/so/saltstack/local/salt/logstash/pipelines/config/custom/` | [Logstash](logstash.md) custom pipeline configuration files | + +Other files under `/opt/so/saltstack/local/salt/` are not watched and are picked up at the next scheduled highstate (see [Highstate Interval](#highstate-interval)). + Auto State Apply is configured at [Administration](administration.md) --> Configuration --> salt --> auto_apply: | Setting | Default | Description | @@ -93,4 +105,4 @@ The root cause of this error is a state trying to run on a minion when another s !!! NOTE - For more information about Salt, please see . \ No newline at end of file + For more information about Salt, please see . diff --git a/docs/software-bill-of-materials.md b/docs/software-bill-of-materials.md index 8f756daf..1a44a490 100644 --- a/docs/software-bill-of-materials.md +++ b/docs/software-bill-of-materials.md @@ -9,19 +9,19 @@ The following table lists the major software projects integrated into the curren | :---- | ----- | :---- | :---- | :---- | :---- | | Alpine Linux | 3.24.1 | Alpine Linux Development Team | [https://alpinelinux.org/](https://alpinelinux.org/) | GNU GPL Version 3 | Alpine Linux is a security-oriented, lightweight Linux distribution based on musl libc and busybox. | | ATT&CK Navigator | 5.3.0 | MITRE | [https://github.com/mitre-attack/attack-navigator](https://github.com/mitre-attack/attack-navigator) | Apache License 2 | The ATT&CK Navigator is designed to provide basic navigation and annotation of ATT&CK matrices, something that people are already doing today in tools like Excel. | -| CyberChef | 11.2.0 | GCHQ | [https://github.com/gchq/CyberChef](https://github.com/gchq/CyberChef) | Apache License 2 | The "Cyber Swiss Army Knife" \- a web app for encryption, encoding, compression and data analysis. | +| CyberChef | 11.3.0 | GCHQ | [https://github.com/gchq/CyberChef](https://github.com/gchq/CyberChef) | Apache License 2 | The "Cyber Swiss Army Knife" \- a web app for encryption, encoding, compression and data analysis. | | Docker | 29.2.1 | Docker | [https://github.com/docker](https://github.com/docker) | Apache License 2 | Docker is a set of platform as a service products that use OS-level virtualization to deliver software in packages called containers. | | ElastAlert | 2.31.0 | Jason Ertel | [https://github.com/jertel/elastalert2](https://github.com/jertel/elastalert2) | Apache License 2 | ElastAlert is a simple framework for alerting on anomalies, spikes, or other patterns of interest from data in Elasticsearch. | -| Elastic Agent | 9.3.7 | Elastic | [https://github.com/elastic/elastic-agent](https://github.com/elastic/elastic-agent) | Elastic License Version 2 | Beats is a free and open platform for single-purpose data shippers. They send data from hundreds or thousands of machines and systems to Logstash or Elasticsearch. | -| Elasticsearch | 9.3.7 | Elastic | [https://github.com/elastic/elasticsearch](https://github.com/elastic/elasticsearch) | Elastic License Version 2 | Elasticsearch is a distributed, open source search and analytics engine for all types of data, including textual, numerical, geospatial, structured, and unstructured. | +| Elastic Agent | 9.4.5 | Elastic | [https://github.com/elastic/elastic-agent](https://github.com/elastic/elastic-agent) | Elastic License Version 2 | Beats is a free and open platform for single-purpose data shippers. They send data from hundreds or thousands of machines and systems to Logstash or Elasticsearch. | +| Elasticsearch | 9.4.5 | Elastic | [https://github.com/elastic/elasticsearch](https://github.com/elastic/elasticsearch) | Elastic License Version 2 | Elasticsearch is a distributed, open source search and analytics engine for all types of data, including textual, numerical, geospatial, structured, and unstructured. | | evtx | 0.8.9 | Omer Benamram | [https://github.com/omerbenamram/evtx](https://github.com/omerbenamram/evtx) | MIT License | A cross-platform parser for the Windows XML EventLog format. | | evtx2es | 1.5.5 | Shinta Nakano | [https://github.com/Security-Onion-Solutions/evtx2es](https://github.com/Security-Onion-Solutions/evtx2es) | MIT License | A fast library for parsing and importing Windows Event Logs into Elasticsearch. | | ExifTool | 12.60 | Phil Harvey | [https://github.com/exiftool/exiftool](https://github.com/exiftool/exiftool) | GNU GPL Version 3 | ExifTool is a platform-independent Perl library plus a command-line application for reading, writing and editing meta information in a wide variety of files. | | Hydra | 26.2.0 | Ory | [https://github.com/ory/hydra](https://github.com/ory/hydra) | Apache License 2 | Ory Hydra is a hardened, OpenID Certified OAuth 2.0 Server and OpenID Connect Provider optimized for low-latency, high throughput, and low resource consumption | | InfluxDB | 2.9.1 | InfluxData | [https://github.com/influxdata/influxdb/tree/main-2.x](https://github.com/influxdata/influxdb/tree/main-2.x) | MIT License | InfluxDB is an open source time series platform. This includes APIs for storing and querying data, processing it in the background for ETL or monitoring and alerting purposes, user dashboards, and visualizing and exploring the data and more. | -| Kibana | 9.3.7 | Elastic | [https://github.com/elastic/kibana](https://github.com/elastic/kibana) | Elastic License Version 2 | Kibana is an open source frontend application that sits on top of the Elastic Stack, providing search and data visualization capabilities for data indexed in Elasticsearch. | +| Kibana | 9.4.5 | Elastic | [https://github.com/elastic/kibana](https://github.com/elastic/kibana) | Elastic License Version 2 | Kibana is an open source frontend application that sits on top of the Elastic Stack, providing search and data visualization capabilities for data indexed in Elasticsearch. | | Kratos | 26.2.0 | Ory | [https://github.com/ory/kratos](https://github.com/ory/kratos) | Apache License 2 | Ory Kratos is the developer-friendly, security-hardened and battle-tested Identity, User Management and Authentication system for the Cloud. | -| Logstash | 9.3.7 | Elastic | [https://github.com/elastic/logstash](https://github.com/elastic/logstash) | Elastic License Version 2 | Logstash is a server-side data processing pipeline that ingests data from a multitude of sources simultaneously, transforms it, and then sends it to your favorite "stash." | +| Logstash | 9.4.5 | Elastic | [https://github.com/elastic/logstash](https://github.com/elastic/logstash) | Elastic License Version 2 | Logstash is a server-side data processing pipeline that ingests data from a multitude of sources simultaneously, transforms it, and then sends it to your favorite "stash." | | NetworkMiner | 2.8.1 | Netresec | [https://www.netresec.com/?page=NetworkMiner](https://www.netresec.com/?page=NetworkMiner) | GNU GPL Version 2 | NetworkMiner is an open source Network Forensic Analysis Tool (NFAT) and can be used as a passive network sniffer/packet capturing tool in order to detect operating systems, sessions, hostnames, open ports etc. without putting any traffic on the network. NetworkMiner can also parse PCAP files for off-line analysis and to regenerate/reassemble transmitted files and certificates from PCAP files. Only included in Security Onion Desktop.| | Nginx | 1.31.2 | NGINX | [https://github.com/nginxinc/docker-nginx](https://github.com/nginxinc/docker-nginx) | 2-Clause BSD License | Nginx is a web server that can also be used as a reverse proxy, load balancer, mail proxy and HTTP cache. | | OpenCanary | 0.9.8 | Thinkst Canary | [https://github.com/thinkst/opencanary](https://github.com/thinkst/opencanary) | 3-Clause BSD License | OpenCanary is a multi-protocol network honeypot. It's primary use-case is to catch hackers after they've breached non-public networks. | @@ -31,10 +31,10 @@ The following table lists the major software projects integrated into the curren | Python | 3.14 | Python Software Foundation | [https://github.com/python/](https://github.com/python/) | Python Software Foundation License Version 2 | Python is a programming language that lets you work quickly and integrate systems more effectively. | | Redis | 7.4.9 | Redis | [https://github.com/redis/redis](https://github.com/redis/redis) | 3-Clause BSD License | Redis is an open source in-memory data structure store used as a database, cache and message broker. | | Salt | 3006.19 | Salt Project | [https://github.com/saltstack/salt](https://github.com/saltstack/salt) | Apache License 2 | Salt is a distributed remote execution system used to execute commands and query data. It was developed in order to bring the best solutions found in the world of remote execution together and make them better, faster and more malleable. Salt accomplishes this via its ability to handle larger loads of information, and not just dozens, but hundreds or even thousands of individual servers, handle them quickly and through a simple and manageable interface. | -| Security Onion Console | 3.2.0 | Security Onion Solutions | [https://github.com/Security-Onion-Solutions/securityonion-soc](https://github.com/Security-Onion-Solutions/securityonion-soc) | Elastic License Version 2 | Security Onion Console is a web interface to viewing alerts, hunting, and PCAP analysis. | +| Security Onion Console | 3.3.0 | Security Onion Solutions | [https://github.com/Security-Onion-Solutions/securityonion-soc](https://github.com/Security-Onion-Solutions/securityonion-soc) | Elastic License Version 2 | Security Onion Console is a web interface to viewing alerts, hunting, and PCAP analysis. | | Strelka | 1.0.1 | Target | [https://github.com/target/strelka](https://github.com/target/strelka) | Apache License 2 | Strelka is a real-time file scanning system used for threat hunting, threat detection, and incident response. Strelka’s purpose is to perform file extraction and metadata collection at huge scale. | | Suricata | 8.0.6 | OISF | [https://github.com/OISF/suricata](https://github.com/OISF/suricata) | GNU GPL Version 2 | Suricata is a free and open source, mature, fast and robust network threat detection engine. Suricata inspects the network traffic using a powerful and extensive rules and signature language. | -| Telegraf | 1.39.0 | InfluxData | [https://github.com/influxdata/telegraf](https://github.com/influxdata/telegraf) | MIT License | Telegraf is a server-based agent for collecting and sending all metrics and events from databases, systems, and IoT sensors. Telegraf is written in Go and compiles into a single binary with no external dependencies, and requires a very minimal memory footprint. | +| Telegraf | 1.39.3 | InfluxData | [https://github.com/influxdata/telegraf](https://github.com/influxdata/telegraf) | MIT License | Telegraf is a server-based agent for collecting and sending all metrics and events from databases, systems, and IoT sensors. Telegraf is written in Go and compiles into a single binary with no external dependencies, and requires a very minimal memory footprint. | | Yara | 4.5.4 | VirusTotal | [https://github.com/virustotal/yara](https://github.com/virustotal/yara) | 3-Clause BSD License | YARA is a tool aimed at (but not limited to) helping malware researchers to identify and classify malware samples. | | Wireshark | 3.4.10 | Wireshark | [https://github.com/wireshark/wireshark](https://github.com/wireshark/wireshark) | GNU GPL Version 2 | Wireshark is the world’s foremost and widely-used network protocol analyzer. It lets you see what’s happening on your network at a microscopic level and is the de facto (and often de jure) standard across many commercial and non-profit enterprises, government agencies, and educational institutions. Only included in Security Onion Desktop. | -| Zeek | 8.0.9 | Zeek | [https://github.com/zeek/zeek/](https://github.com/zeek/zeek/) | 3-Clause BSD License | A powerful framework for network traffic analysis and security monitoring. | +| Zeek | 8.0.10 | Zeek | [https://github.com/zeek/zeek/](https://github.com/zeek/zeek/) | 3-Clause BSD License | A powerful framework for network traffic analysis and security monitoring. | diff --git a/docs/zeek.md b/docs/zeek.md index b1db11ab..caef1b23 100644 --- a/docs/zeek.md +++ b/docs/zeek.md @@ -117,7 +117,7 @@ Please note that Zeek is very strict about the format of `intel.dat`. When editi The default `intel.dat` file follows these guidelines so you can reference it as an example of the proper format. -Intel files are not detected by [Auto State Apply](salt.md#auto-state-apply), so when finished editing `intel.dat`, run `sudo salt -C 'I@zeek:enabled:true' state.apply zeek` to sync `/opt/so/saltstack/local/salt/zeek/policy/intel/` to `/opt/so/conf/zeek/policy/intel/`. If you have a distributed deployment with separate sensor nodes and you don't run that command yourself, intel will sync to the sensor nodes at their next scheduled highstate (see [Highstate Interval](salt.md#highstate-interval)). +[Auto State Apply](salt.md#auto-state-apply) picks up changes to `intel.dat` within a few minutes. If you don't want to wait, run `sudo salt -C 'I@zeek:enabled:true' state.apply zeek` to sync `/opt/so/saltstack/local/salt/zeek/policy/intel/` to `/opt/so/conf/zeek/policy/intel/`. Either way, intel syncs to every node running Zeek, including separate sensor nodes in a distributed deployment. If you experience an error, or do not notice `/nsm/zeek/logs/current/intel.log` being generated, try having a look in `/nsm/zeek/logs/current/reporter.log` for clues. You may also want to restart Zeek after making changes by running `sudo so-zeek-restart`. @@ -137,7 +137,7 @@ The package directory should contain a valid Zeek package structure (including a zkg requires packages from git repos to be a clone of the repo. The working tree must be clean otherwise it will not install the package. `git status` will tell you if it is clean or not. -Packages are not detected by [Auto State Apply](salt.md#auto-state-apply), so after placing the package, run `sudo salt -C 'I@zeek:enabled:true' state.apply zeek` to sync the packages to the sensor nodes. The packages will be automatically installed with `zkg` each time the Zeek container starts. If you have a distributed deployment with separate sensor nodes and you don't run that command yourself, packages will sync to the sensor nodes at their next scheduled highstate (see [Highstate Interval](salt.md#highstate-interval)). +[Auto State Apply](salt.md#auto-state-apply) picks up new packages within a few minutes. If you don't want to wait, run `sudo salt -C 'I@zeek:enabled:true' state.apply zeek` to sync the packages to the sensor nodes. The packages will be automatically installed with `zkg` each time the Zeek container starts. You can verify that a custom package was installed by checking the Zeek container logs: diff --git a/mkdocs.yml b/mkdocs.yml index b9a56299..03a9cf85 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -189,6 +189,7 @@ nav: - Backup: backup.md - Docker: docker.md - Jupyter: jupyter.md + - Kernels: kernels.md - New Disk: new-disk.md - Network Installation: network-installation.md - PCAPs: pcaps.md @@ -233,6 +234,7 @@ nav: - Hypervisor: hypervisor.md - Reports: reports.md - Onion AI: onion-ai.md + - Local LLM Hosting: local-llm.md - Telemetry: telemetry.md - Security: security.md - Software Bill of Materials: software-bill-of-materials.md diff --git a/specs/openapi.yaml b/specs/openapi.yaml index a4aa0b47..be43a7a9 100644 --- a/specs/openapi.yaml +++ b/specs/openapi.yaml @@ -67,24 +67,46 @@ components: type: string protocol: type: string + supportsEmbeddings: + type: boolean type: object - model.AgentParameters: + model.Agent: properties: agentDescription: + example: Analyzes suspicious binaries and scripts type: string allowedSkills: + example: + - Hunt + - Respond items: type: string type: array uniqueItems: false canDelegateTo: + example: + - Log Analyst items: type: string type: array uniqueItems: false + enabled: + description: A disabled agent is still published so the Agent Studio can + re-enable it. + example: true + type: boolean isOrchestrator: + example: false + type: boolean + isSystem: + example: true type: boolean name: + example: Malware Analyst + type: string + personaAddendum: + description: Admin-authored persona; exposed because an admin wrote it. + example: Prefer static analysis before detonating a sample. type: string type: object model.AlertingParameters: @@ -160,6 +182,27 @@ components: example: 132 type: integer type: object + model.AllocationDecider: + properties: + explanation: + description: The datastore's explanation of why the rule rejected or throttled + the shard. + example: the node is above the low watermark cluster setting + type: string + name: + description: The name of the allocation rule. + example: disk_threshold + type: string + nodes: + description: The names of the nodes the rule rejected or throttled the shard + on. + example: + - so-node-01 + items: + type: string + type: array + uniqueItems: false + type: object model.Artifact: properties: artifactType: @@ -286,6 +329,8 @@ components: agentMapping: additionalProperties: type: string + example: + Malware Analyst: claude-sonnet-4.5@SOAI type: object agentic: type: boolean @@ -296,7 +341,7 @@ components: uniqueItems: false availableAgents: items: - $ref: '#/components/schemas/model.AgentParameters' + $ref: '#/components/schemas/model.Agent' type: array uniqueItems: false availableModels: @@ -306,7 +351,17 @@ components: uniqueItems: false availableSkills: items: - $ref: '#/components/schemas/model.SkillParameters' + $ref: '#/components/schemas/model.Skill' + type: array + uniqueItems: false + availableTools: + description: Tool names an admin-created skill may grant; delegate tools + excluded. + example: + - query_events + - query_cases + items: + type: string type: array uniqueItems: false compressContextPrompt: @@ -315,6 +370,19 @@ components: type: boolean investigationPrompt: type: string + maxDelegationDepth: + description: |- + Delegation guardrails, surfaced so the Agent Studio can show and edit them + without fetching every setting. 0 disables the limit. + example: 3 + type: integer + maxSubSessionTokens: + example: 100000 + type: integer + memoryEnabled: + type: boolean + memoryParams: + $ref: '#/components/schemas/model.MemoryParameters' thresholdColorRatioLow: type: number thresholdColorRatioMax: @@ -363,6 +431,22 @@ components: description: The kind of object. This is a read-only field. example: case type: string + lastMemoryScannedIndex: + description: |- + When Memory is enabled, the scanner will use this field to denote how many + messages in the session have been scanned for memory extraction. Extraction + always begins with the oldest messages in a session. + example: 4 + type: integer + messageCount: + description: |- + A denormalized count of the messages saved to this session, incremented on + each chat save so the memory scanner can find sessions with unscanned + messages in a single query. Not omitempty so new sessions index an explicit + 0; sessions created before this field existed have no value, which the + scanner treats as pending until its index update heals it. + example: 7 + type: integer model: description: |- The model/adapter this session itself runs on (the agent id for delegated @@ -491,6 +575,23 @@ components: - caseId - fields type: object + model.Attachment: + description: Attachment represents a file or graph payload to be sent with a + notification. + properties: + contentType: + description: The MIME content type of the attachment. + example: application/pdf + type: string + filename: + description: The filename of the attachment. + example: alert-report.pdf + type: string + url: + description: Direct download URL if the attachment is hosted by SOC. + example: https://soc.example.com/api/reports/123/download + type: string + type: object model.AuditHistory: properties: duplicatedFromId: @@ -1293,6 +1394,10 @@ components: type: object description: The collection of aggregated metrics associated with this search type: object + timedOut: + description: Did the query that produced these results time out while collecting + them + type: boolean totalEvents: description: The total number of matching events (not necessarily the total number returned) @@ -1391,6 +1496,102 @@ components: example: 1 type: integer type: object + model.EventsHealth: + properties: + errors: + additionalProperties: + type: string + description: The sections that could not be collected, keyed by section + name, with the reason for each failure. + example: + nodes: unavailable + type: object + indicators: + description: The per-subsystem health checks reported by the datastore, + most severe first. + items: + $ref: '#/components/schemas/model.HealthIndicator' + type: array + uniqueItems: false + nodes: + description: The nodes making up the datastore cluster. + items: + $ref: '#/components/schemas/model.EventsHealthNode' + type: array + uniqueItems: false + settings: + $ref: '#/components/schemas/model.EventsHealthSettings' + status: + description: 'The overall datastore health: green, yellow, red, or unknown.' + example: yellow + type: string + unassignedShards: + $ref: '#/components/schemas/model.UnassignedShards' + type: object + model.EventsHealthNode: + properties: + cpu: + description: The percentage of the node's CPU in use. + example: "12" + type: string + diskTotal: + description: The total size of the node's data storage. + example: 1.5tb + type: string + diskUsedPercent: + description: The percentage of the node's data storage in use. + example: "85.5" + type: string + heapPercent: + description: The percentage of the node's heap in use. + example: "62" + type: string + ip: + description: The IP address the node is bound to. + example: 10.0.0.5 + type: string + load1m: + description: The node's one minute load average. + example: "1.42" + type: string + master: + description: An asterisk if the node is the elected master, otherwise a + dash. + example: '*' + type: string + name: + description: The node name. + example: so-node-01 + type: string + ramPercent: + description: The percentage of the node's memory in use. + example: "88" + type: string + roles: + description: The abbreviated roles the node performs in the cluster. + example: dhimrst + type: string + uptime: + description: How long the node has been running. + example: 42d + type: string + version: + description: The datastore version the node is running. + example: 8.14.3 + type: string + type: object + model.EventsHealthSettings: + description: The cluster settings that have been overridden from their defaults. + properties: + persistent: + additionalProperties: {} + description: The settings that survive a cluster restart. + type: object + transient: + additionalProperties: {} + description: The settings that are discarded on a cluster restart. + type: object + type: object model.Filter: description: Optional filter for the job, typically used for packet filtering properties: @@ -1443,6 +1644,23 @@ components: example: 55312 type: integer type: object + model.FindingScope: + description: The group of shards this finding was diagnosed from, when it came + from the shard inventory. + properties: + count: + description: The number of shards in the group. + example: 2 + type: integer + primary: + description: Whether the group contains primary shards rather than replicas. + example: true + type: boolean + reason: + description: The datastore's reason code for why the shards became unassigned. + example: NODE_LEFT + type: string + type: object model.GridMember: properties: fingerprint: @@ -1498,6 +1716,87 @@ components: example: 0 type: integer type: object + model.HealthFinding: + properties: + condition: + description: The identifier for the problem that was diagnosed. + example: disk_threshold + type: string + count: + description: The number of affected resources, when the condition is counted + rather than scoped. + example: 3 + type: integer + detail: + description: The datastore's own explanation of the condition. + example: the node is above the low watermark cluster setting + type: string + nodes: + description: The names of the nodes affected by this condition. + example: + - so-node-01 + items: + type: string + type: array + uniqueItems: false + scope: + $ref: '#/components/schemas/model.FindingScope' + severity: + description: 'The importance of this finding: critical, warning, or info.' + example: critical + type: string + type: object + model.HealthIndicator: + properties: + causes: + description: The datastore's diagnoses of the symptom, along with the resources + each affects. + items: + $ref: '#/components/schemas/model.HealthIndicatorCause' + type: array + uniqueItems: false + findings: + description: Diagnosed problems for this indicator, most severe first + items: + $ref: '#/components/schemas/model.HealthFinding' + type: array + uniqueItems: false + id: + description: The datastore's identifier for the subsystem being checked. + example: shards_availability + type: string + status: + description: 'The health of this subsystem: green, yellow, red, or unknown.' + example: red + type: string + symptom: + description: The datastore's summary of what is wrong with this subsystem. + example: This cluster has 1 unavailable primary shard. + type: string + type: object + model.HealthIndicatorCause: + properties: + cause: + description: The datastore's description of the cause. + example: A node has recently left the cluster. + type: string + indices: + description: The names of the indices affected by this cause. + example: + - so-logs-2026.07.21 + items: + type: string + type: array + uniqueItems: false + nodes: + description: The names of the nodes affected by this cause. + example: + - so-node-01 + items: + type: string + type: array + uniqueItems: false + type: object model.HuntingAction: properties: background: @@ -1631,6 +1930,11 @@ components: forceUserOtp: description: OTP indicator; unavailable to API clients type: boolean + lastUnreadNotificationTime: + description: The timestamp of the most recent unread notification for the + user. + example: "2026-08-20T17:00:00Z" + type: string license: description: The copyright license applicable to the Security Onion software example: Elastic License 2.0 (ELv2) @@ -1646,6 +1950,10 @@ components: description: The MAC address assigned to the management interface. example: 11:22:33:AA:BB:CC type: string + notificationsStarted: + description: Indicates whether the notification subsystem module is started + and available. + type: boolean parameters: $ref: '#/components/schemas/model.ClientParameters' srvToken: @@ -1758,6 +2066,121 @@ components: example: no result type: string type: object + model.MemoryParameters: + properties: + dontScanBefore: + type: string + embedModel: + example: amazon.titan-embed-text-v2@SOAI + type: string + maxGlobalMemoriesToInclude: + example: 5 + type: integer + maxGlobalMemoriesToReconcile: + example: 20 + type: integer + maxUserMemoriesToInclude: + example: 5 + type: integer + maxUserMemoriesToReconcile: + example: 20 + type: integer + memoryModel: + example: sonnet@SOAI + type: string + memoryPersona: + type: string + memoryProximityThreshold: + example: 0.8 + type: number + messageProximityThreshold: + example: 0.5 + type: number + reconcileModel: + example: sonnet@SOAI + type: string + reconcilePersona: + type: string + scanIntervalSeconds: + example: 300 + type: integer + staleMemoryCount: + type: integer + useMemory: + example: true + type: boolean + useMemoryScanner: + example: true + type: boolean + type: object + model.MemoryRecord: + properties: + createTime: + type: string + id: + example: c3d44fb8-3bc2-46e2-a7d2-8a8983556d1a + type: string + lastUsedAt: + type: string + memoryText: + example: The user prefers timestamps in UTC. + type: string + modelId: + example: claude-sonnet-4.5 + type: string + scope: + example: user + type: string + sessionId: + example: chat_1757086398900_ykhmndscn + type: string + similarity: + example: 0.82 + type: number + targetUserId: + example: 8beae4b5-275b-4669-b678-8cff894911b5 + type: string + updateTime: + type: string + usageCount: + example: 4 + type: integer + userDefined: + example: true + type: boolean + type: object + model.MemoryRequest: + properties: + memoryText: + example: The user prefers timestamps in UTC. + type: string + scope: + example: user + type: string + targetUserId: + description: |- + TargetUserId assigns a user-scoped memory to someone else; requires + memory/write_all. Empty means the requestor. + example: 8beae4b5-275b-4669-b678-8cff894911b5 + type: string + type: object + model.MemoryResults: + properties: + limit: + example: 25 + type: integer + memories: + items: + $ref: '#/components/schemas/model.MemoryRecord' + type: array + uniqueItems: false + offset: + example: 0 + type: integer + total: + example: 42 + type: integer + type: object model.Message: description: The message content. properties: @@ -2004,10 +2427,6 @@ components: description: Indicates whether disk encryption is enabled on this node example: 1 type: integer - load1m: - description: The node's 1-minute load metric - example: 0.43 - type: number load5m: description: The node's 5-minute load metric example: 0.96 @@ -2016,6 +2435,10 @@ components: description: The node's 15-minute load metric example: 1.09 type: number + load1m: + description: The node's 1-minute load metric + example: 0.43 + type: number memoryTotalGB: description: Total size, in gigabytes, of the system memory, or RAM example: 33.176731648 @@ -2165,52 +2588,161 @@ components: example: 0 type: number type: object - model.Override: + model.NotificationAuditEntry: + description: NotificationAuditEntry represents a user view or dismissal audit + state record. properties: - count: - description: (suricata only) For treshold overrides, this is the number - of occurrences allowed, within the given seconds interval, before this - detection triggers an alert. Must be non-negative and greater than 0. - example: 10 - type: integer - createdAt: - description: The date and time when this override was created - example: "2024-12-06T14:36:45.579994541Z" - type: string - customFilter: - description: (elastalert only) The custom filter applied to Sigma detections - before the detection will trigger an alert. - example: |- - sofilter: - user.name: dresden - type: string - ip: - description: '(suricata only) The IP address or network value, must be in - CIDR format: x.x.x.x/y' - example: 1.2.3.4/32 + dismissedAt: + description: The timestamp when this user dismissed the notification. + example: "2026-08-17T12:10:00Z" type: string - isEnabled: - description: Indicates whether this override is enabled + isDismissed: + description: Indicates whether this user has dismissed the notification. + example: false + type: boolean + isRead: + description: Indicates whether this user has read the notification. example: true type: boolean - note: - description: An optional operational note for this override - example: Exclude the SMTP server due to FPs + readAt: + description: The timestamp when this user read the notification. + example: "2026-08-17T12:05:00Z" type: string - regex: - description: (suricata only) Regular expression for matching modify overrides - example: content:xyz + userId: + description: The user identifier who read or dismissed the notification. + example: admin@soc.local type: string - seconds: - description: (suricata only) For treshold overrides, this is the number - of seconds that the occurrence threshold must occur within. Must be non-negative - and greater than 0. - example: 120 - type: integer - thresholdType: - description: (suricata only) Threshold type, for threshold overrides - enum: - - threshold + type: object + model.NotificationRecord: + description: NotificationRecord represents a persisted notification with user-specific + state. + properties: + attachments: + description: Optional binary or URL attachments associated with this notification. + items: + $ref: '#/components/schemas/model.Attachment' + type: array + uniqueItems: false + createdAt: + description: The timestamp when this notification was created. + example: "2026-08-17T12:00:00Z" + type: string + dismissedAt: + description: The timestamp when the notification was dismissed. + example: "2026-08-17T12:10:00Z" + type: string + fields: + additionalProperties: + type: string + description: Structured key-value fields providing context (e.g. host, IP, + metric name). + example: + dst_ip: 10.0.0.1 + src_ip: 192.168.1.100 + type: object + id: + description: The unique identifier for this notification. + example: a1b2c3d4-e5f6-7890-abcd-ef1234567890 + type: string + isDismissed: + description: Indicates whether this notification has been dismissed by the + user. + example: false + type: boolean + isRead: + description: Indicates whether this notification has been marked as read + by the user. + example: false + type: boolean + links: + additionalProperties: + type: string + description: Deep-links back to relevant views or dashboards in SOC. + example: + View in SOC: https://soc.example.com/#/alerts/123 + type: object + readAt: + description: The timestamp when the notification was marked as read. + example: "2026-08-17T12:05:00Z" + type: string + severity: + description: The severity classification of this notification. + enum: + - info + - low + - medium + - high + - critical + example: high + type: string + silenceKey: + description: Unique key used for duration silencing and debouncing. + example: detection:sigma:12345:192.168.1.100 + type: string + source: + description: The source subsystem that originated this notification. + enum: + - detection + - metric + - agent_ai + - report + example: detection + type: string + summary: + description: A human-readable summary describing the notification details. + example: Inbound SSH scan detected from 192.168.1.100. + type: string + title: + description: The brief title or headline of the notification. + example: ET SCAN Potential SSH Scan + type: string + type: object + model.Override: + properties: + count: + description: (suricata only) For treshold overrides, this is the number + of occurrences allowed, within the given seconds interval, before this + detection triggers an alert. Must be non-negative and greater than 0. + example: 10 + type: integer + createdAt: + description: The date and time when this override was created + example: "2024-12-06T14:36:45.579994541Z" + type: string + customFilter: + description: (elastalert only) The custom filter applied to Sigma detections + before the detection will trigger an alert. + example: |- + sofilter: + user.name: dresden + type: string + ip: + description: '(suricata only) The IP address or network value, must be in + CIDR format: x.x.x.x/y' + example: 1.2.3.4/32 + type: string + isEnabled: + description: Indicates whether this override is enabled + example: true + type: boolean + note: + description: An optional operational note for this override + example: Exclude the SMTP server due to FPs + type: string + regex: + description: (suricata only) Regular expression for matching modify overrides + example: content:xyz + type: string + seconds: + description: (suricata only) For treshold overrides, this is the number + of seconds that the occurrence threshold must occur within. Must be non-negative + and greater than 0. + example: 120 + type: integer + thresholdType: + description: (suricata only) Threshold type, for threshold overrides + enum: + - threshold - limit - both type: string @@ -2277,6 +2809,10 @@ components: description: The packet destination port example: 55423 type: integer + error: + description: An optional error message if the packet failed to decode + example: Unable to decode EthernetType 49320 + type: string flags: description: 'The optional packet flags. Ex: SYN PSH FIN' example: @@ -2513,6 +3049,9 @@ components: $ref: '#/components/schemas/model.EventRecord' type: array uniqueItems: false + queryTimedOut: + description: Indicates if the QueryResults were cut short by a timeout. + type: boolean question: description: The human readable question to be asked. example: What is the source IP address of the alert? @@ -2809,11 +3348,27 @@ components: - SigLangSigma - SigLangSuricata - SigLangYara - model.SkillParameters: + model.Skill: properties: + enabled: + description: A disabled skill grants nothing, but is still published so + it can be re-enabled. + example: true + type: boolean + isSystem: + example: true + type: boolean name: + example: Threat Intel Lookup + type: string + personaAddendum: + description: Admin-authored guidance; exposed because an admin wrote it. + example: Always cite the source of an indicator. type: string tools: + example: + - query_events + - query_cases items: type: string type: array @@ -2832,6 +3387,46 @@ components: example: my_subgrid_a type: string type: object + model.StoredAgent: + properties: + allowedSkills: + example: + - Hunt + - Respond + items: + type: string + type: array + uniqueItems: false + canDelegateTo: + example: + - Log Analyst + items: + type: string + type: array + uniqueItems: false + description: + example: Analyzes suspicious binaries and scripts + type: string + enabled: + description: Pointer so an absent field means enabled rather than disabled. + example: true + type: boolean + isOrchestrator: + example: false + type: boolean + model: + description: A model selector ("id@adapter" or bare id), not a display name. + example: claude-sonnet-4.5@SOAI + type: string + name: + example: Malware Analyst + type: string + persona: + description: Addendum to the built-in prompt for a system agent; the whole + prompt otherwise. + example: Prefer static analysis before detonating a sample. + type: string + type: object model.StoredMessage: description: A stored message in the chat session. This contains metadata about the message and its context not necessary for the conversation with the Assistant. @@ -2884,6 +3479,31 @@ components: example: socl_my_new_client type: string type: object + model.StoredSkill: + properties: + enabled: + description: Pointer so an absent field means enabled rather than disabled. + example: true + type: boolean + name: + example: Threat Intel Lookup + type: string + persona: + description: Addendum to the built-in guidance for a system skill; all of + it otherwise. + example: Always cite the source of an indicator. + type: string + tools: + description: Ignored for a system skill, whose tool set is fixed by the + built-in. + example: + - query_events + - query_cases + items: + type: string + type: array + uniqueItems: false + type: object model.Subgrid: properties: caCertificate: @@ -2943,6 +3563,30 @@ components: type: string type: object model.ToolRequest: + properties: + auxData: + description: Auxiliary data for certain tools. + type: object + model: + description: The model to use for this tool execution. + example: claude-sonnet-4.5 + type: string + params: + description: The parameters for this tool use. + type: object + rejected: + description: |- + Rejected indicates the user declined this tool: it is not executed; an error + tool_result is recorded instead so the turn (and any parallel siblings) resolves. + type: boolean + sessionId: + description: The sessionId this chat message belongs to. + example: chat_1757086398900_ykhmndscn + type: string + toolUseId: + description: The unique identifier for this tool use. + example: tooluse_mT45or7ISwSEUivo63nqow + type: string type: object model.UiElement: properties: @@ -2994,6 +3638,78 @@ components: user before it can be saved type: boolean type: object + model.UnassignedShardGroup: + properties: + canAllocate: + description: The datastore's verdict on whether the sampled shard can currently + be allocated. + example: "no" + type: string + count: + description: The number of shards in this group. + example: 2 + type: integer + deciders: + description: The allocation rules that rejected or throttled the sampled + shard. + items: + $ref: '#/components/schemas/model.AllocationDecider' + type: array + uniqueItems: false + failureDetails: + description: Present when the sampled shard became unassigned due to an + allocation failure. + example: 'failed shard on node [abc]: shard failure, reason [corrupt file]' + type: string + primary: + description: Whether this group contains primary shards rather than replicas. + example: true + type: boolean + reason: + description: The datastore's reason code for why these shards became unassigned. + example: NODE_LEFT + type: string + sampleIndex: + description: The index of the sampled shard. + example: so-logs-2026.07.21 + type: string + sampleShard: + description: The number of the sampled shard within its index. + example: 0 + type: integer + sampleStatus: + description: Whether the sampled shard was explained, capped before being + explained, or failed to be explained. + example: explained + type: string + since: + description: The date and time that the sampled shard became unassigned. + example: "2026-07-21T20:48:02.943Z" + type: string + type: object + model.UnassignedShards: + description: A summary of the shards that the datastore has been unable to assign + to a node. + properties: + groups: + description: The unassigned shards grouped by reason, primaries first. + items: + $ref: '#/components/schemas/model.UnassignedShardGroup' + type: array + uniqueItems: false + primaries: + description: The number of unassigned shards that are primaries. + example: 2 + type: integer + replicas: + description: The number of unassigned shards that are replicas. + example: 12 + type: integer + total: + description: The total number of unassigned shards. + example: 14 + type: integer + type: object model.UpdateSessionRequest: properties: action: @@ -3197,6 +3913,23 @@ components: example: fb58abf3-0a6d-49af-b1a0-1eeabec07716 type: string type: object + server.ToggleDismissRequest: + description: ToggleDismissRequest specifies the desired dismissal state for + a notification. + properties: + isDismissed: + description: Indicates whether the notification should be dismissed. + example: true + type: boolean + type: object + server.ToggleReadRequest: + description: ToggleReadRequest specifies the desired read state for a notification. + properties: + isRead: + description: Indicates whether the notification should be marked as read. + example: true + type: boolean + type: object securitySchemes: basic: scheme: basic @@ -3256,7 +3989,7 @@ paths: security: - bearer: - assistant/read_all - summary: Get Assistant Sessions + summary: Get Assistant Sessions For a Specific User tags: - Assistant /connect/assistant/admin/{userId}/sessions/{sessionId}/history: @@ -3318,7 +4051,7 @@ paths: security: - bearer: - assistant/read_all - summary: Get Assistant Sessions + summary: Get Assistant Sessions Across Users tags: - Assistant /connect/assistant/admin/stats: @@ -3346,19 +4079,264 @@ paths: in: query name: format required: true - schema: - type: string + schema: + type: string + responses: + "200": + content: + application/json: + schema: + items: + $ref: '#/components/schemas/model.UserUsage' + type: array + description: Usage statistics for the AI assistant + "400": + description: The provided date range is invalid or missing + "401": + description: Request was not properly authenticated + "403": + description: Insufficient permissions for this request + "500": + description: Internal SOC error; review SOC logs + security: + - bearer: + - assistant/read_all + summary: Get Usage Statistics + tags: + - Assistant + /connect/assistant/agents/{name}: + delete: + description: Remove an admin-created agent. System agents can only be disabled. + parameters: + - description: Agent name + in: path + name: name + required: true + schema: + type: string + responses: + "200": + content: + application/json: + schema: + type: object + description: Agent deleted + "401": + description: Request was not properly authenticated + "403": + description: Insufficient permissions, or the agent is system-provided + "404": + description: Agent not found + "500": + description: Internal SOC error; review SOC logs + security: + - bearer: + - config/write + summary: Delete an Assistant Agent + tags: + - Assistant + put: + description: Create or update a single agent definition. The server merges it + into the stored set, so a stale client cannot revert other agents. + parameters: + - description: Agent name + in: path + name: name + required: true + schema: + type: string + requestBody: + content: + application/json: + schema: + $ref: '#/components/schemas/model.StoredAgent' + description: Agent definition + required: true + responses: + "200": + content: + application/json: + schema: + type: object + description: Agent saved + "400": + description: The request body is invalid + "401": + description: Request was not properly authenticated + "403": + description: Insufficient permissions for this request + "500": + description: Internal SOC error; review SOC logs + security: + - bearer: + - config/write + summary: Save an Assistant Agent + tags: + - Assistant + /connect/assistant/balance/{modelAndAdapter}: + get: + description: Retrieve the current balance/usage information for the AI assistant. + parameters: + - description: 'Model selector to get balance for: the id@adapter pair (bare + id allowed, slashes in ids supported), or an agent name in agentic mode' + example: sonnet-4.5@SOAI + in: path + name: modelAndAdapter + required: true + schema: + type: string + responses: + "200": + content: + application/json: + schema: + $ref: '#/components/schemas/model.Usage' + description: Current assistant balance and usage information + "401": + description: Request was not properly authenticated + "403": + description: Insufficient permissions for this request + "500": + description: Internal SOC error; review SOC logs + security: + - bearer: + - assistant/read_authored + - bearer: + - assistant/read_all + summary: Get Assistant Balance + tags: + - Assistant + /connect/assistant/chat: + post: + description: Send a message to the AI assistant and receive a response. Supports + both streaming (SSE) and non-streaming responses. + parameters: + - description: Type of entity associated with this session (e.g., 'alert_investigation') + in: query + name: entityType + schema: + type: string + - description: ID of the entity associated with this session (e.g., alert's + soc_id) + in: query + name: entityId + schema: + type: string + requestBody: + content: + application/json: + schema: + properties: + msg: + type: string + sessionId: + type: string + type: object + description: Chat message object with message text and optional session ID + required: true + responses: + "200": + content: + application/json: + schema: + items: + $ref: '#/components/schemas/model.Message' + type: array + text/event-stream: + schema: + type: string + description: AI assistant response messages + "400": + description: The provided input object or parameters are malformed or invalid + "401": + description: Request was not properly authenticated + "403": + description: Insufficient permissions for this request + "500": + description: Internal SOC error; review SOC logs + security: + - bearer: + - assistant/write_authored + summary: Send Chat Message + tags: + - Assistant + /connect/assistant/memories: + get: + description: Retrieve a page of the memories the requestor is allowed to read. + A query orders results by semantic similarity instead of recency. + parameters: + - description: self, global, or all (default) + example: global + in: query + name: scope + schema: + type: string + - description: Narrow to one user; another user requires memory/read_all + example: 8beae4b5-275b-4669-b678-8cff894911b5 + in: query + name: userId + schema: + type: string + - description: Order results by similarity to this text + example: preferred timezone + in: query + name: q + schema: + type: string + - description: Page size + example: 25 + in: query + name: limit + schema: + type: integer + - description: Page offset + example: 0 + in: query + name: offset + schema: + type: integer + responses: + "200": + content: + application/json: + schema: + $ref: '#/components/schemas/model.MemoryResults' + description: A page of memories + "401": + description: Request was not properly authenticated + "403": + description: Insufficient permissions for this request + "500": + description: Internal SOC error; review SOC logs + security: + - bearer: + - memory/read_authored + - bearer: + - memory/read_global + - bearer: + - memory/read_all + summary: List Assistant Memories + tags: + - Assistant + post: + description: Store a new user-defined memory. User-defined memories are never + rewritten or removed by the memory scanner. + requestBody: + content: + application/json: + schema: + $ref: '#/components/schemas/model.MemoryRequest' + description: Memory to create + required: true responses: "200": content: application/json: schema: - items: - $ref: '#/components/schemas/model.UserUsage' - type: array - description: Usage statistics for the AI assistant + $ref: '#/components/schemas/model.MemoryRecord' + description: The created memory "400": - description: The provided date range is invalid or missing + description: The request body is invalid "401": description: Request was not properly authenticated "403": @@ -3367,19 +4345,20 @@ paths: description: Internal SOC error; review SOC logs security: - bearer: - - assistant/read_all - summary: Get Usage Statistics + - memory/write_self + - bearer: + - memory/write_global + summary: Create an Assistant Memory tags: - Assistant - /connect/assistant/balance/{modelAndAdapter}: - get: - description: Retrieve the current balance/usage information for the AI assistant. + /connect/assistant/memories/{id}: + delete: + description: Permanently remove a memory. parameters: - - description: 'Model selector to get balance for: the model''s display name, - or the legacy id@adapter form (including slashes)' - example: Agent Gemini + - description: Memory ID + example: c3d44fb8-3bc2-46e2-a7d2-8a8983556d1a in: path - name: modelAndAdapter + name: id required: true schema: type: string @@ -3388,74 +4367,69 @@ paths: content: application/json: schema: - $ref: '#/components/schemas/model.Usage' - description: Current assistant balance and usage information + type: object + description: Memory deleted "401": description: Request was not properly authenticated "403": description: Insufficient permissions for this request + "404": + description: Memory not found "500": description: Internal SOC error; review SOC logs security: - bearer: - - assistant/read_authored + - memory/write_self - bearer: - - assistant/read_all - summary: Get Assistant Balance + - memory/write_global + - bearer: + - memory/write_all + summary: Delete an Assistant Memory tags: - Assistant - /connect/assistant/chat: - post: - description: Send a message to the AI assistant and receive a response. Supports - both streaming (SSE) and non-streaming responses. + put: + description: Replace the text or scope of an existing memory, marking it user-defined + so the scanner leaves it alone. parameters: - - description: Type of entity associated with this session (e.g., 'alert_investigation') - in: query - name: entityType - schema: - type: string - - description: ID of the entity associated with this session (e.g., alert's - soc_id) - in: query - name: entityId + - description: Memory ID + example: c3d44fb8-3bc2-46e2-a7d2-8a8983556d1a + in: path + name: id + required: true schema: type: string requestBody: content: application/json: schema: - properties: - msg: - type: string - sessionId: - type: string - type: object - description: Chat message object with message text and optional session ID + $ref: '#/components/schemas/model.MemoryRequest' + description: Replacement memory required: true responses: "200": content: application/json: schema: - items: - $ref: '#/components/schemas/model.Message' - type: array - text/event-stream: - schema: - type: string - description: AI assistant response messages + $ref: '#/components/schemas/model.MemoryRecord' + description: The updated memory "400": - description: The provided input object or parameters are malformed or invalid + description: The request body is invalid "401": description: Request was not properly authenticated "403": description: Insufficient permissions for this request + "404": + description: Memory not found "500": description: Internal SOC error; review SOC logs security: - bearer: - - assistant/write_authored - summary: Send Chat Message + - memory/write_self + - bearer: + - memory/write_global + - bearer: + - memory/write_all + summary: Update an Assistant Memory tags: - Assistant /connect/assistant/sessions: @@ -3480,7 +4454,7 @@ paths: security: - bearer: - assistant/read_authored - summary: Get Assistant Sessions + summary: Get Your Assistant Sessions tags: - Assistant /connect/assistant/sessions/{sessionId}: @@ -3507,7 +4481,7 @@ paths: security: - bearer: - assistant/delete_authored - summary: Delete Session + summary: Delete Your Assistant Session tags: - Assistant get: @@ -3540,7 +4514,7 @@ paths: - assistant/read_authored - bearer: - assistant/read_shared - summary: Get Session Details + summary: Get Session Details For Your Session tags: - Assistant put: @@ -3582,6 +4556,75 @@ paths: summary: Update metadata for a Session tags: - Assistant + /connect/assistant/skills/{name}: + delete: + description: Remove an admin-created skill. System skills can only be disabled. + parameters: + - description: Skill name + in: path + name: name + required: true + schema: + type: string + responses: + "200": + content: + application/json: + schema: + type: object + description: Skill deleted + "401": + description: Request was not properly authenticated + "403": + description: Insufficient permissions, or the skill is system-provided + "404": + description: Skill not found + "500": + description: Internal SOC error; review SOC logs + security: + - bearer: + - config/write + summary: Delete an Assistant Skill + tags: + - Assistant + put: + description: Create or update a single skill definition, merged into the stored + set by the server. + parameters: + - description: Skill name + in: path + name: name + required: true + schema: + type: string + requestBody: + content: + application/json: + schema: + $ref: '#/components/schemas/model.StoredSkill' + description: Skill definition + required: true + responses: + "200": + content: + application/json: + schema: + type: object + description: Skill saved + "400": + description: The request body is invalid + "401": + description: Request was not properly authenticated + "403": + description: Insufficient permissions for this request + "500": + description: Internal SOC error; review SOC logs + security: + - bearer: + - config/write + summary: Save an Assistant Skill + tags: + - Assistant /connect/assistant/tool/{name}: post: description: Execute a tool on behalf of the assistant and continue the conversation @@ -5411,6 +6454,34 @@ paths: summary: Acknowledge Alerts tags: - Query + /connect/events/health: + get: + description: |- + Retrieves a point-in-time diagnostic snapshot of the event datastore: overall status, health + indicators, node listing, and non-default settings. When shard availability is degraded, the + unassigned shards are inventoried and one sampled shard per group is explained. Diagnosed + problems are reported as findings on the indicator they affect, ranked by severity. + responses: + "200": + content: + application/json: + schema: + $ref: '#/components/schemas/model.EventsHealth' + description: The events health snapshot + "401": + description: Request was not properly authenticated + "403": + description: Insufficient permissions for this request + "405": + description: The event module is not loaded on the server + "500": + description: Events health could not be retrieved; review SOC logs + security: + - bearer: + - nodes/read + summary: Get Events Health + tags: + - Query /connect/grid/: get: description: |- @@ -5934,6 +7005,158 @@ paths: summary: Node Check-in tags: - Grid + /connect/notifications: + get: + description: Retrieves notifications for the current user, optionally filtered. + parameters: + - description: Filter parameter (all, unread, dismissed) + in: query + name: filter + schema: + type: string + responses: + "200": + content: + application/json: + schema: + items: + $ref: '#/components/schemas/model.NotificationRecord' + type: array + description: The list of notifications + "400": + description: License is invalid + "401": + description: Request was not properly authenticated + "403": + description: Insufficient permissions for this request + "405": + description: Notification module has not been enabled on the server + "500": + description: Internal SOC error; review SOC logs + security: + - bearer: + - notifications/read + summary: Get Notifications + tags: + - Notifications + /connect/notifications/{id}/audit: + get: + description: Retrieves team user state logs (view/dismiss) for a specific notification. + parameters: + - description: Notification ID + example: notif-1 + in: path + name: id + required: true + schema: + type: string + responses: + "200": + content: + application/json: + schema: + items: + $ref: '#/components/schemas/model.NotificationAuditEntry' + type: array + description: The list of notification user audit records + "400": + description: Invalid parameters + "401": + description: Request was not properly authenticated + "403": + description: Insufficient permissions for this request + "405": + description: Notification module has not been enabled on the server + "500": + description: Internal SOC error; review SOC logs + security: + - bearer: + - notifications/read_all + summary: Get Notification Audit Logs + tags: + - Notifications + /connect/notifications/{id}/dismiss: + put: + description: Marks a notification as dismissed or active for the current user. + parameters: + - description: Notification ID + example: notif-1 + in: path + name: id + required: true + schema: + type: string + requestBody: + content: + application/json: + schema: + $ref: '#/components/schemas/server.ToggleDismissRequest' + description: Payload to toggle dismiss + required: true + responses: + "200": + content: + application/json: + schema: + type: object + description: The notification dismissal state was successfully updated + "400": + description: Invalid request body or parameters + "401": + description: Request was not properly authenticated + "403": + description: Insufficient permissions for this request + "405": + description: Notification module has not been enabled on the server + "500": + description: Internal SOC error; review SOC logs + security: + - bearer: + - notifications/write + summary: Toggle Dismiss Notification + tags: + - Notifications + /connect/notifications/{id}/read: + put: + description: Marks a notification as read or unread for the current user. + parameters: + - description: Notification ID + example: notif-1 + in: path + name: id + required: true + schema: + type: string + requestBody: + content: + application/json: + schema: + $ref: '#/components/schemas/server.ToggleReadRequest' + description: Payload to toggle read + required: true + responses: + "200": + content: + application/json: + schema: + type: object + description: The notification read state was successfully updated + "400": + description: Invalid request body or parameters + "401": + description: Request was not properly authenticated + "403": + description: Insufficient permissions for this request + "405": + description: Notification module has not been enabled on the server + "500": + description: Internal SOC error; review SOC logs + security: + - bearer: + - notifications/write + summary: Toggle Read Notification + tags: + - Notifications /connect/packets/{jobId}: get: description: Retrieves the packets collected and attached to the job represented @@ -5953,6 +7176,13 @@ paths: name: unwrap schema: type: boolean + - description: If true, packet decode failures will be excluded from the returned + results. + example: true + in: query + name: excludeErrors + schema: + type: boolean - description: The starting offset of the packet to retrieve; used for paging large packet results. Defaults to 0. example: 100 @@ -5977,6 +7207,12 @@ paths: $ref: '#/components/schemas/model.Packet' type: array description: The array of retrieved Packet objects + headers: + X-Decode-Errors-Present: + description: Indicates whether any packets in the PCAP capture encountered + decode failures (true or false) + schema: + type: string "400": description: The provided input object or parameters are malformed or invalid "401":