[BUG] Source Sophos XG are disconnected but logs are processed and logs on GUI are empty

[securemeit]

Describe the bug
Source "Sophos XG" are disconneted after a while but logs are processed from agent.

tail -f processed_logs.txt
2024/01/09 23:04:07.5688759 +0100 CET - 58349 logs from firewall_sophos have been processed
2024/01/09 23:04:07.5689122 +0100 CET - 5 logs from beats_linux_agent have been processed
2024/01/09 23:09:07.5693139 +0100 CET - 63980 logs from firewall_sophos have been processed
2024/01/09 23:09:07.5693556 +0100 CET - 6 logs from beats_linux_agent have been processed
2024/01/09 23:14:07.5701356 +0100 CET - 62392 logs from firewall_sophos have been processed
2024/01/09 23:14:07.5701737 +0100 CET - 8 logs from beats_linux_agent have been processed
2024/01/09 23:19:07.5710297 +0100 CET - 44 logs from beats_linux_agent have been processed
2024/01/09 23:19:07.5710533 +0100 CET - 57481 logs from firewall_sophos have been processed
2024/01/09 23:26:45.9562315 +0100 CET - 7444 logs from beats_linux_agent have been processed
2024/01/09 23:26:45.9563218 +0100 CET - 55660 logs from firewall_sophos have been processed

However logs on GUI are empty or very small.

Expected behavior
See logs on the GUI and sources are not disconneted.

Environment

• OS: [Ubuntu 22.04]
• Browser [Firefox, Chrome]
• Version [v10.1.0-202312131645)]
• Latest version of agent: {
  "master_version": "10.1.0",
  "agent_version": "10.1.2",
  "updater_version": "10.1.3",
  "redline_version": "10.1.2"
  }

[jdieguez89]

@secureme71 Thank you for reporting this issue. Have you disabled the Sophos XG integration?

[securemeit]

No I don't disable Sophos XG integration.

[jdieguez89]

@secureme71
Sophos XG integration receives data via Syslog, activates its pipeline when enabled. This pipeline parses the logs, evaluates potential security alerts based on correlation rules, and stores the logs in the data engine. If you disable the integration, UTMStack may continue to receive logs if you do not have to change the settings in the agent to disable the Sophos XG log collector. In such a case, UTMStack would treat Sophos XG logs as generic. To verify this, search for 'log-generic-*' in the "Log Explorer." To completely deactivate this integration, you must change the agent's configuration file and disable the integration.

Future versions will allow managing these configurations from the panel. You will no longer need to modify the agent's settings manually. Instead, you can enable or disable the log collector in the agents directly from the panel.

Set this value to false; if the issue persists, please leave a comment.

To determine if UTMStack is treating the logs as generic, check the Log Explorer and search for Sophos XG logs.

[securemeit]

Same issue. Generic logs is empty. Sophos firewall send logs to 7014 udp port. The file "processed_logs" is not populated.

[Kbayero]

@secureme71 Could you provide us with what resources your UTMStack instance has? Additionally, the logs of the log-auth-proxy service found in your UTMStack instance would be very useful.

[securemeit]

VM has 16vCPU,64Gb ram and 500Gb.

Where do I find the logs of log-auth-proxy service?

[osmontero]

@secureme71 you've been an active member of our community for a while, would you like to join us on Discord so we can move faster in resolving the bugs you're reporting? Of course, you will continue to report problems this way, this is just to be more proactive during the solution. https://discord.gg/gAyAGhNXR7

[Kbayero]

@secureme71 To obtain the log-auth-proxy logs, execute the following command on your UTMStack instance:

docker ps

In the result you can find a line like this:
container_id utmstack.azurecr.io/log-auth-proxy:v10 "/app/server" x days ago Up x days 8080/tcp, 50051/tcp utmstack_log-auth-proxy.1.saf5454ad4ffgh

Run this command, replacing container_id with the container id from the log auth proxy:

docker logs container_id

[securemeit]

Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address

[jdieguez89]

@osmontero @Kbayero Can we close this issue?

[securemeit]

No because I have made more details on Discord.

[osmontero]

Hi @secureme71, we have released an update that may resolve this issue. Can you download the latest installer and run it to get the update? Please let us know if this resolves the issue.

[securemeit]

I have already upgraded to latest version but the issue is the same and logs are the same:

Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address
Post "http://logstash:10010": dial tcp 10.0.1.21:10010: connect: cannot assign requested address

[jdieguez89]

@secureme71 the issue was solved in version 10.2.2; please let us know if you face further problems

[securemeit]

Same issue.

Agent version now is empty.

/opt/utmstack-linux-agent/versions.json
{
"master_version": "",
"agent_version": "",
"updater_version": "",
"redline_version": ""
}

Logs are not processed and incomplete.

[Kbayero]

@secureme71 We have released a new version in which we have worked to fix this error. Could you update and let us know if your problem is fixed?

[securemeit]

I have upgraded to v10.4.1-202405031709. UTMStack agent now works fine but Sophos Firewall v20 are not managed well. I see few events related to theat intelligence and logs are not fully complete. dst_ip doesn't show in the detailed logs.