Skip to content

Difference when run with config file and without #97

Open

Description

So i have a bug where i provide a config file i have a lot of errors in my pipelines.

My Example config file for checkov:

{
    "tools": [
      {
        "tool": {
          "name": "Checkov",
          "version": "Latest"
        },
        "arguments": {
          "DownloadExternalModules": "false",
          "TargetDirectory": "$(Checkov.DefaultTargetDirectory)"
        }
      }
    ]
  }

and everything is fine but i am getting this error, even with false setting:

D:\a\_msdo\packages\nuget\Microsoft.Guardian.CheckovRedist_windows_amd64.3.2.144\tools\dist\checkov.exe --download-external-modules false --directory .\ --output-file-path D:\a\1\s\.gdn\.r\checkov\001\checkov.sarif
##[error]2024-07-08 11:42:17,665 [ThreadPoolEx] [WARNI]  Failed to download module git::https://ADOORG@dev.azure.com/ADOORG/PROJ/_git/kvmodule//src?ref=v0.3:None (for external modules, the --download-external-modules flag is required)

Task still yells that DownloadExternalModules is required even when it is provided and set to false. My pipeline is the simplest template as you can imagine:

parameters:
  TemplatesRepoName: ''

stages:
- stage: Microsoft_Defender
  displayName: Microsoft Defender for Cloud DevOps security
  condition: always()
  pool:
    vmImage: windows-latest
  jobs:
  - job: Microsoft_Defender_Scan
    displayName: Scan
    steps:
    - checkout: ${{ parameters.TemplatesRepoName }}
    - checkout: self
          
    - task: MicrosoftSecurityDevOps@1
      displayName: Microsoft Security DevOps
      inputs:
        config: configs/checkov.gdnconfig

From template above i got a lot of errors:

    Process:
      Convert:
        Converting any raw tool logs to Sarif format ...
        Completed converting raw tool logs to Sarif format.
      Import:
        No tool logs to process. 
    Break:
      Guardian is searching for results that meet the given criteria to break the build.
      Results Query Summary:
        Baselines: default
        Suppression Sets: default
        Policy: azuredevops
      Saved file D:\a\1\a\.gdn\msdo.sarif
      Found no breaking results.
      Active results: 0
      Skipped results: 0
        Baselined results: 0
        Suppressed results: 0
        Results excluded by tool filters: 0
        Results below minimum severity: 0
        Results classified as Pass: 0
        Results in flight: 0
##[error]Error running tool 1 of 1: checkov
##[error]Error running checkov job: 1 of 1
##[error]GuardianErrorExitCodeException: checkov completed with an Error exit code: 1. An error has occurred running the Checkov tool.
##[error]BreakException: Guardian detected one or more breaking results.

When i do not provide config file, no issues, only error that DownloadExternalModules is required.

Why this task behaves differently when config file for checkov is provided?

Activity

  1. ptmetcalf commented on Nov 6, 2024

    @ptmetcalf

    I'm also seeing this same behavior. If I try to pass a config file even in the simplest form it causes it to fail and not upload any results.

    {
      "tools": [
        {
          "tool": {
            "name": "Checkov",
            "version": "Latest"
          },
          "arguments": {
          }
        }
      ]
    }
    
    

    If it helps anyone else, I was able to use a checkov specific config file and it seems the scan picks up and uses that correctly. I just put a .checkov.yml file in my scan directory of my repo. This allowed me to set options as well as choose tests to skip.

    directory: ./
    output: sarif
    soft-fail: true
    framework: terraform
    skip-check:
      - CKV_AZURE_190
      - CKV2_AZURE_1
    

    You can see the details of the checkov specific config file on there site. Configuration using a config file

  2. DimaBir commented on Jul 28, 2026

    @DimaBir
    Collaborator

    We're cleaning up the issue backlog in this repo, and this one has been inactive for a long time, so we're closing it.

    If it's still affecting you, just reopen it or drop a comment — that's enough, and we'll pick it up from there. Nothing else needed from you.

    Sorry we left this sitting.

  3. michasacuer commented on Jul 28, 2026

    @michasacuer
    Author

    We're cleaning up the issue backlog in this repo, and this one has been inactive for a long time, so we're closing it.

    If it's still affecting you, just reopen it or drop a comment — that's enough, and we'll pick it up from there. Nothing else needed from you.

    Sorry we left this sitting.

    I just moved to another product. Why did you close this request? Do you assume it was fixed on its own?

  4. DimaBir commented on Jul 28, 2026

    @DimaBir
    Collaborator

    Thanks for the report, and for the detail in it — the config, the emitted command line, and the log excerpt are what made this diagnosable.

    Confirmed as a bug.

    Root cause. The arguments block in a .gdnconfig replaces the MSDO CLI's default argument set for that tool rather than merging into it. Two defaults you hadn't specified were therefore dropped:

    • OutputType: sarif — without it Checkov emits its default cli format, so nothing SARIF-shaped lands at the --output-file-path that MSDO subsequently reads. That produces No tool logs to process.
    • Soft: true — without it Checkov's normal "failed checks found" exit 1 propagates instead of being handed to MSDO's own policy. That produces GuardianErrorExitCodeException: checkov completed with an Error exit code: 1.

    This is also why the run behaved differently with no config file at all: that path keeps the full default set intact.

    The WARNI ... the --download-external-modules flag is required line is Checkov's expected message when it encounters a git:: module source with downloads disabled, and is unrelated to the failure.

    Workaround. Carry the dropped defaults explicitly:

    {
      "tools": [
        {
          "tool": { "name": "Checkov", "version": "Latest" },
          "arguments": {
            "TargetDirectory": "$(Checkov.DefaultTargetDirectory)",
            "OutputType": "sarif",
            "Soft": true,
            "DownloadExternalModules": "false"
          }
        }
      ]
    }

    Note the key is Soft, not SoftFail — credit to newbloke82 in #130, who recovered the CLI's own default configuration via guardian configure -t checkov and confirmed the correct spelling. Running that command prints the full canonical default argument set for any tool, which is the reliable way to see what a hand-written arguments block would otherwise drop.

    Fix. The merge happens inside the closed-source MSDO/Guardian CLI, so it is not addressable in this repository. It is being routed to that team as a single defect covering the four reports that share this root cause — #47, #97, #104 and #130 — since the same layer also mis-serializes list-valued arguments and mishandles mutually exclusive tool verbs.

    Tracking here until it lands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

area:msdo-cliMSDO CLI integration and executionpriority:highImportant, should be addressed soonstatus:blockedBlocked by external dependency or decisiontype:bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions