Greenbone XML Import broken

[Whitehorse2]

There are some problems with the Greenbone import.

Bug description
Problem 1:
When importing via the ‘OpenVAS Parser’ as XML, the scanner delivers a significant high number under Closed Findings than under Open Findings, which is impossible in my case.

Problem 2:
In addition, Hosts/Endpoints is completely empty.

Steps to reproduce
Import Options:

    Data = {
        engagement': engagement_id,
        scan_type': “OpenVAS Parser”,
        scan_date': scan_date,
        verified': ’true
        ‘minimum_severity’: ‘Low’,
        close_old_findings': ’true’
    }

    response = requests.post(f‘{DD_API_URL}import-scan/’, headers=headers, files=files, data=data)

I tested this with the xml files from the old issue. #8761

Expected behavior
Hosts and Endpoints from findings are available at the product page.
Only fixed vulnerabilities are closed/migrated.

Deployment method

• Docker Compose
• Kubernetes
• GoDojo

Environment information

• Operating System: Ubuntu 24.04
• DefectDojo version 2.42.3

[valentijnscholten]

Could you point us to the exact report to reproduce this with? Could you reproduce it the demo instance?

I tried the report from #8761 (comment) in the demo instance and imports as expected with all 44 findings as active: https://demo.defectdojo.org/engagement/27

[Whitehorse2]

I think any report should be sufficient for this, but I have generated data in my lab environment for verification.

It was scanned 2x Metasploitable, once with active (test 1) and once with deactivated Apache (test 2). The results were exported to XML and CSV.

I think this shows the differences:

Differences XML/CSV:

• there is no data under “Hosts/Endpoints” in XML
• there is Critical in XML, but not in CSV
• the data for “Active (Verified)” and “Mitigated” are different, although XML and XSV have the same number of findings in Test 1 and Test 2

Versions:
Greenbone Security Assistant
Version 23.3.0

DefectDojo
v. 2.42.3 ( release mode )

gvm_test_data_metasploitable.tar.gz

I hope this information helps!

[valentijnscholten]

Ah, your running multiple imports sequentially.

Mitigated findings
If you use close_old_findings, Defect Dojo will look at all findings in the product or engagement depending on the engagement settings/import parameters. It will close any findings that are no longer found in the second report. The algorithm to match findings is configured via HASHCODE_FIELDS_PER_SCANNER:

django-DefectDojo/dojo/settings/settings.dist.py

Line 1194 in e0de953

HASHCODE_FIELDS_PER_SCANNER = {

.
OpenVAS doesn't have a configuration, so Defect Dojo will fallback to the default fields ['title', 'cwe', 'line', 'file_path', 'description'].

You may try to configure the hashcode configuration via local_settings.py or the DD_HASHCODE_FIELDS_PER_SCANNER environment variable: https://docs.defectdojo.com/en/open_source/installation/configuration/

If you have a good suggestion for what the config should be, you could raise a PR so we can merge the config into Defect Dojo default config (settings.dist.py).

Endpoints
Currently the OpenVAS parser doesn't create endpoints in Defect Dojo from the report. If it's common in OpenVAS to scan multiple targets at once resulting in a combined report, could you raise a GitHub issue as a feature request? We can look at extending the parser or you're welcome to give it a go yourself :-)

[Whitehorse2]

Thank you very much. With Greenbone/OpenVAS it is normal to have multiple hosts or destinations in one report, even whole subnets are possible.

In the XML import, for example, “Java RMI Server Insecure Default Configuration RCE Vulnerability - Active Check” is incorrectly marked as resolved because the “Destination port” in the description is different.

Presumably it only makes sense with OpenVAS as “HASHCODE_FIELDS_PER_SCANNER”:

• vuln_id_from_tool
• title
• endpoints

“endpoints” is missing, currently there is probably a workaround that it is generated in the title.

@valentijnscholten
HASHCODE_FIELDS_PER_SCANNER: '{"OpenVAS Parser": ["title"]}'

I set this variable in docker-compose.yml for uwsgi, celerybeat, celeryworker and restarted the image with docker compose up -d, but this value is completely ignored during import, but available inside the container.

[valentijnscholten]

The name is DD_HASHCODE_FIELDS_PER_SCANNER (DD_ prefix). Please note setting it like this will "erase" the existing configuration in settings.dist.py, but for testing it's fine.

[Whitehorse2]

@valentijnscholten :
Thanks, the result looks better with this change.

With XML (cvss_base) and CSV (CVSS) you can see a discrepancy in the variable numerical_severity of defectdojo. This means that Critical is not filled. It looks as if the CSV OpenVAS parser is not looking at CVSS, but at Severity.

There is no Critical in OpenVAS, so there is the problem.

django-DefectDojo/dojo/tools/openvas/csv_parser.py

Line 138 in d289361

valid_severity = ("Info", "Low", "Medium", "High", "Critical")

The XML parser generates it from CVSS.

django-DefectDojo/dojo/tools/openvas/xml_parser.py

Line 34 in d289361

severity = self.convert_cvss_score(finding.text)

[Whitehorse2]

@valentijnscholten
Can I point out another problem with OpenVAS, in the "findings" (and in "Hosts/Endpoints" - tested during CSV import, as not yet available in XML) there is also no IP for the target. The overview of all vulnerabilities is not really useful if they cannot be assigned directly to a target by searching.

Perhaps someone knows a solution to this problem?

[valentijnscholten]

Presumably it only makes sense with OpenVAS as “HASHCODE_FIELDS_PER_SCANNER”:

• vuln_id_from_tool
• title
• endpoints

I see this in the xml reports: <result id="b87d3a32-8ff8-4010-aada-4b4f578b084d">. Can this be considered the unique ID of that result which would remain constant over time in subsequent scans? In that case we could map it to the unique_id_from_tool field in defect dojo. If it is not unique, it could go into vuln_id_from_tool.

[valentijnscholten]

Made some fixes in #11955, most importantly the xml parser now generates endpoints. For improvements with regards to unique ids and/or deduplication hash code configuration we need some help from the community as we ourselves are not users of OpenVAS.

@Whitehorse2 As most items are solved here, could you create a new issue if you have specific extra requirements for example around the uniqueids and/or dedupe config?