What happens
codecov/project reports a whole-project coverage drop on PRs that cannot have caused one, so the check is red on
work that never touched the affected code. On #5685 (a 29-line change confined to
engine/src/main/java/com/arcadedb/index/hash/) Codecov reported:
Files 1770 1770
Lines 150057 150086 +29
Hits 101236 100126 -1110 <-- 1,110 covered lines disappear
A 29-line change cannot delete 1,110 hits. Every one of the 34 files that lost coverage is in ha-raft/:
| Hits lost |
File |
base -> head |
| -295 |
RaftReplicatedDatabase.java |
68.16% -> 29.19% |
| -229 |
RaftHAServer.java |
57.71% -> 38.07% |
| -116 |
ArcadeStateMachine.java |
66.66% -> 54.35% |
| -97 |
PostVerifyDatabaseHandler.java |
60.58% -> 3.52% |
Cause
In .github/workflows/mvn-test.yml, coverage-report is wired to consume HA coverage but never waits for it:
coverage-report:
needs: [ unit-tests, integration-tests, slow-unit-tests ] # ha-integration-tests is absent
...
- name: Download HA integration test coverage reports # line ~757
uses: actions/download-artifact@...
continue-on-error: true # failure is swallowed
with:
name: ha-integration-coverage-reports
ha-integration-tests does upload that artifact (line ~340, if: success() || failure()), so the intent is clearly
that HA coverage be included. But because the job is not in needs, coverage-report starts as soon as the three
listed jobs finish, while ha-integration-tests runs for ~55 minutes - by far the longest job in the workflow. The
artifact does not exist yet, continue-on-error: true hides that, and the merged report silently omits the whole
ha-raft module.
So the project total is computed against a base that included HA coverage and a head that does not. The gap is
roughly -0.76pp, which is enough to trip the threshold.
Why it matters
This is not specific to one PR - it is structural, and it fires on every PR whose other jobs finish before the HA
suite. The practical effect is that codecov/project is red for reasons unrelated to the diff, which trains
reviewers to ignore it. A coverage gate that is always red is worse than no gate: a real coverage regression would
look exactly like the noise.
Note codecov/patch is unaffected and remains meaningful (it scored the #5685 diff at 87.75% and passed), as do
Codacy Diff Coverage and Codacy Coverage Variation.
Suggested fix
Add the missing dependency:
coverage-report:
needs: [ unit-tests, integration-tests, slow-unit-tests, ha-integration-tests ]
coverage-report already has if: success() || failure(), so adding the dependency does not make a failing HA suite
block the coverage report - it only makes it wait, which is what the existing download step assumes.
Two things worth deciding alongside:
continue-on-error: true on the download hides exactly this class of bug. Consider failing the job when an
expected coverage artifact is missing, or at minimum logging which artifacts were skipped, so an incomplete
upload is visible rather than silently reported as a coverage drop.
ha-integration-tests is currently failing on main most of the time (filed separately). While that is true,
its coverage artifact still uploads (if: success() || failure()), so this fix works regardless - but the two
interact, and fixing only this one will surface partial HA coverage from partially-failed runs.
Found while diagnosing a spurious codecov/project failure on #5685 (issue #5677).
What happens
codecov/projectreports a whole-project coverage drop on PRs that cannot have caused one, so the check is red onwork that never touched the affected code. On #5685 (a 29-line change confined to
engine/src/main/java/com/arcadedb/index/hash/) Codecov reported:A 29-line change cannot delete 1,110 hits. Every one of the 34 files that lost coverage is in
ha-raft/:RaftReplicatedDatabase.javaRaftHAServer.javaArcadeStateMachine.javaPostVerifyDatabaseHandler.javaCause
In
.github/workflows/mvn-test.yml,coverage-reportis wired to consume HA coverage but never waits for it:ha-integration-testsdoes upload that artifact (line ~340,if: success() || failure()), so the intent is clearlythat HA coverage be included. But because the job is not in
needs,coverage-reportstarts as soon as the threelisted jobs finish, while
ha-integration-testsruns for ~55 minutes - by far the longest job in the workflow. Theartifact does not exist yet,
continue-on-error: truehides that, and the merged report silently omits the wholeha-raftmodule.So the project total is computed against a base that included HA coverage and a head that does not. The gap is
roughly -0.76pp, which is enough to trip the threshold.
Why it matters
This is not specific to one PR - it is structural, and it fires on every PR whose other jobs finish before the HA
suite. The practical effect is that
codecov/projectis red for reasons unrelated to the diff, which trainsreviewers to ignore it. A coverage gate that is always red is worse than no gate: a real coverage regression would
look exactly like the noise.
Note
codecov/patchis unaffected and remains meaningful (it scored the #5685 diff at 87.75% and passed), as doCodacy Diff CoverageandCodacy Coverage Variation.Suggested fix
Add the missing dependency:
coverage-reportalready hasif: success() || failure(), so adding the dependency does not make a failing HA suiteblock the coverage report - it only makes it wait, which is what the existing download step assumes.
Two things worth deciding alongside:
continue-on-error: trueon the download hides exactly this class of bug. Consider failing the job when anexpected coverage artifact is missing, or at minimum logging which artifacts were skipped, so an incomplete
upload is visible rather than silently reported as a coverage drop.
ha-integration-testsis currently failing onmainmost of the time (filed separately). While that is true,its coverage artifact still uploads (
if: success() || failure()), so this fix works regardless - but the twointeract, and fixing only this one will surface partial HA coverage from partially-failed runs.
Found while diagnosing a spurious
codecov/projectfailure on #5685 (issue #5677).