Skip to content

fix: Bugbot flags a missing Coil no store header despite passing real transport tests (#784) - #800

Merged
efraespada merged 1 commit into
developfrom
bugfix/784-bugbot-flags-a-missing-coil-no-store-header-despite-passing-real-transport-tests
Oct 30, 2026
Merged

efraespada merged 1 commit into
developfrom
bugfix/784-bugbot-flags-a-missing-coil-no-store-header-despite-passing-real-transport-tests

Conversation

@efraespada

@efraespada efraespada commented Oct 30, 2026 •

Copy link
Copy Markdown
Member

Bugbot could claim a transport assertion fails because the application does not assign a header that its resolved dependency generates. The shared review instruction now traces concrete policy/default flags through the request builder and adapters to the transport observation, and reconciles source-bound execution evidence before making that claim.

What changed

  • Deliver the same generic request-transform contract to ordinary/partitioned review and the product benchmark; preserve genuine bypass, overwrite, mock-masking and independent faults.
  • Add 22 regressions and eight corpus scenarios, with compiled synthetic oracles and an explicit unknown-evidence result. Preserve all 98 previous inputs/verdicts and every quality floor.
  • Compact the common prompt wrapper while retaining its contracts and the existing 15,000-character bound. Update the owning SDD, catalog, documentation and generated bundles.

Validation

  • Before guidance, all four delivery regressions fail; afterward, 183 focused cases and 134 evaluator cases pass.
  • The actual baseline product benchmark reproduces the Coil false premise at 0.98 confidence. On identical inputs/settings the corrected sample has zero false positives/negatives and detects all five genuine controls.
  • Full coverage: 800 suites, 14,970 passed tests and 35 pre-existing skips; 98.10% lines / 97.33% statements / 97.97% functions / 95.10% branches. All existing budgets pass. Extension: 267 tests, 100% all metrics.
  • Types, lint, specifications, documentation, web, build, package smoke (CLI + typed API + isolated local web session) and graph update pass. Exact-head CI/platform coverage and complete review remain required before merge.

Review notes

Both actual model scores and all predictions are retained in quality/bugbot/evidence/issue-784.json. The corrected sample still fails the unchanged exact-location gate (1/5); it does not establish all-threshold model quality, publication remapping, historical-finding resolution or statistical accuracy. Reference predictions remain separate. Existing consumer CI was inspected read-only; private source/XML was not copied into this repository or executed here.

Fixes #784.

…g-real-transport-tests: ground request-header claims in dependency behavior (#784)
@docs-page

docs-page Bot commented Oct 30, 2026

Copy link
Copy Markdown

To preview the documentation for this pull request, visit the following URL:

docs.page/vypdev/copilot~800

Documentation is deployed and generated using docs.page

@vypbot vypbot changed the title Ground request-header findings in dependency behavior fix: Bugbot flags a missing Coil no store header despite passing real transport tests (#784) Oct 30, 2026
@vypbot vypbot added the size: XXL Indicates an extremely large task or issue that requires significant time and effort to complete. label Oct 30, 2026
@vypbot
vypbot self-requested a review October 30, 2026 04:45
@vypbot vypbot added this to vypdev Oct 30, 2026
@vypbot vypbot moved this to In Progress in vypdev Oct 30, 2026
@vypbot
vypbot changed the base branch from develop to master October 30, 2026 04:45
@vypbot
vypbot changed the base branch from master to develop October 30, 2026 04:45
@vypbot

vypbot commented Oct 30, 2026

Copy link
Copy Markdown
Contributor

Bugbot: review complete

Current status: No active findings on cec40d3.

Pull request · Verified commit · Workflow run

@vypbot

vypbot commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Copilot approval · Human review needed

Status: A protected workflow, policy, or trust-boundary file changed.
Completed: 1/1 observed checks succeeded; Bugbot complete.
Action required: Ask a human reviewer to decide.
Impact: No new approval is confirmed for this revision.
See checks

Technical evidence

Revision: cec40d3aee37; base: 518d97f6fc50; reason: protected-path.

@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.96%. Comparing base (5b3664f) to head (cec40d3).
⚠️ Report is 85 commits behind head on develop.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #800      +/-   ##
===========================================
+ Coverage    94.50%   94.96%   +0.46%     
===========================================
  Files         1035     1094      +59     
  Lines        33299    35705    +2406     
  Branches      9824    10630     +806     
===========================================
+ Hits         31468    33907    +2439     
+ Misses         699      678      -21     
+ Partials      1132     1120      -12     
Files with missing lines Coverage Δ
src/prompts/bugbot.ts 100.00% <100.00%> (ø)

... and 114 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@efraespada
efraespada merged commit a8e9058 into develop Oct 30, 2026
13 checks passed
@github-project-automation github-project-automation Bot moved this from In Progress to Done in vypdev Oct 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size: XXL Indicates an extremely large task or issue that requires significant time and effort to complete. state:verified

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

🐛🧑‍💻 - Bugbot flags a missing Coil no store header despite passing real transport tests

3 participants