Skip to content

fix(review): ground external package claims in current evidence - #745

Merged
efraespada merged 3 commits into
developfrom
bugfix/721-pr-review-rejects-valid-official-android-37-0-package-and-recommends-an-absent-sdk-id
Oct 27, 2026
Merged

efraespada merged 3 commits into
developfrom
bugfix/721-pr-review-rejects-valid-official-android-37-0-package-and-recommends-an-absent-sdk-id

Conversation

@efraespada

@efraespada efraespada commented Oct 27, 2026 •

Copy link
Copy Markdown
Member

Bugbot now evaluates time-sensitive SDK and package claims against applicable inventory and matching execution evidence. This prevents historical naming assumptions from rejecting valid packages while preserving findings for demonstrated resolution failures.

What changed

  • Added a shared evidence contract for production review and the real-agent benchmark, covering package identifiers, provenance, date, channel, platform, and exact execution evidence.
  • Added five dated synthetic scenarios and eight deterministic regression cases, including controls for unavailable or stale inventory and confirmed missing packages.
  • Updated the Bugbot specification, catalog, detection guidance, quality documentation, and generated Action, CLI, and API bundles.

Validation

Verified combined head be4156cdafad325f029d6ddb73f7c720e4b1e1e5 against develop at 74a2b604c56573569d0f7c967681036d7a286972:

Review notes

  • Current-head CI, complete finding-free PR review, and explicit Codecov 100% patch coverage remain required before merge.

Closes #721

Current recovery result

Published normal merge commit be4156cdafad325f029d6ddb73f7c720e4b1e1e5, parents bdbb1bf04a5f80b8f2294c184c42d6874fc40a5a and 74a2b604c56573569d0f7c967681036d7a286972. GitHub confirms the PR is mergeable with no conflicts. The final diff adds #721 while retaining the already-merged framework/layout corrections from #720 and #722. No branch protection, workflow configuration or verification gate was weakened.

Final remote gates passed for be4156cdafad325f029d6ddb73f7c720e4b1e1e5 against actual develop at 74a2b604c56573569d0f7c967681036d7a286972: CI Check, Ubuntu/Windows/macOS, Branch Sync, Commit and PR workflows succeeded. Bugbot reports complete review/context coverage and zero findings in all seven states; there are no review threads or change requests. Codecov's explicit current-base comparison confirms 100% patch coverage, 2/2 lines, zero misses and zero partials; both commits are complete. The separately initiated base CI also succeeded. Historical comment comparisons using an older base are superseded by this explicit report. The two observed Copilot failures have detailed English evidence in #746 and #622.

Historical verification and recorded recovery plans

Final local coverage on 41845ab80b1b: 724 Jest suites / 12,505 passing tests / 35 existing skips; statements 97.04%, branches 94.50%, functions 97.77%, lines 97.89%; every repository coverage and setup-acceptance gate passed. The extension adds 30 passing files / 267 passing tests with 100% in all four coverage metrics. The Bugbot review for this head is complete with complete context coverage and zero findings in all seven states; no review threads are open. Remote CI/platform checks and explicit Codecov 100% patch coverage remain pending.

Recovery plan after failed Action synchronization

The maintainer requested resolution of the remaining conflicts. The Action-owned synchronization run 40483017730 completed with a verification-environment failure: dependencies were installed on the default checkout before switching to the candidate workspace. The agent resolved the nine conflicts, but extension typecheck lacked its workspace dependencies; the guarded rollback restored the original head. The incident is documented with both reproductions in #746.

The bounded recovery uses the existing managed ref bugfix/721-pr-review-rejects-valid-official-android-37-0-package-and-recommends-an-absent-sdk-id, original head 41845ab80b1b2fc00a6de6bc8ae638e50fdd7749, and diagnosed parent develop at 8de9b63ab4ba1a1a2e61a69349a29229de6a92c0:

  1. Prepare a local merge of these exact revisions, preserving the 🐛🧑‍💻 - PR review rejects valid official android 37.0 package and recommends an absent SDK ID #721 external-package evidence contract and 🐛🧑‍💻 - PR review misses ComposeViewport owned cleanup and falsely reports a permanent loading screen #722 framework/startup semantics contract.
  2. Retain the union of their corpus/reference cases (48), both test allocations and both SDD/documentation extensions; regenerate the catalog and all generated bundles from the combined source.
  3. Install the declared frozen workspace dependency graph locally. Run focused regressions, types, documentation/catalog and full coverage, plus the normal unmodified commit hooks. Do not weaken gates or change shared workflow/credential settings.
  4. Recheck the remote child and parent before publication. Publish only a normal fast-forward push of the reviewed recovery commit to that same ref; stop and re-evaluate if either revision moves. Never force-push, create a replacement remote branch or delete a ref.
  5. Let the normal Action observer reconcile the published branch, then require fresh CI, complete finding-free review and explicit Codecov 100% patch coverage for the final head and actual parent before merging.

Conflict recovery result

Published normal merge commit bdbb1bf04a5f80b8f2294c184c42d6874fc40a5a with parents 41845ab80b1b2fc00a6de6bc8ae638e50fdd7749 and 8de9b63ab4ba1a1a2e61a69349a29229de6a92c0. All nine conflicts are resolved. Both grounding instructions, all 48 corpus cases/reference predictions, the #722 evidence file, both test allocations and documentation/SDD contracts are preserved. Catalog and three bundles were regenerated from the combined source. GitHub now reports the PR mergeable against the current develop.

Verification of this combined revision:

  • Normal unmodified hooks passed: build, 724 suites / 12,512 tests passed, 35 existing skips, and lint.
  • Full coverage and every configured budget passed: statements 97.04%, branches 94.50%, functions 97.77%, lines 97.89%; setup acceptance 336/350 passed with 14 explicitly open.
  • Extension: 30 files / 267 tests passed, 100% statements, branches, functions and lines.
  • 31 focused regression tests, all 48 reference-evaluation cases, types including extension, specification/catalog, documentation, package contract and generated bundle parity passed; Graphify AST updated.
  • A new isolated real-agent sample using the combined prompt passed all original thresholds on the five 🐛🧑‍💻 - PR review rejects valid official android 37.0 package and recommends an absent SDK ID #721 cases (precision/recall/F1 1.000, zero false positives/negatives). Both demonstrated package failures remain medium/reliability. This is a scoped synthetic observation, not a general accuracy guarantee.

The Action dependency-setup failure is documented in English with the independent conflicted-path reproduction in #746. Fresh remote CI, review and Codecov evidence for this new head are still required before merge; the earlier head's successful checks do not satisfy that gate.

Recovery continuation after parent advanced

develop advanced through #744 to 74a2b604c56573569d0f7c967681036d7a286972 while the previous recovery head bdbb1bf04a5f80b8f2294c184c42d6874fc40a5a was completing verification. Its review is complete without findings and Codecov confirms 100% (2/2 modified coverable lines), but strict CI now requires the new parent.

The new Action observer run 40489588063 failed while reconciling closed-issue metadata with workflow.stale. The observer detects drift and processes dependency results independently; this failure does not establish that every other dependency observation was skipped, and it never performs automatic branch merging. The documented command path still has the independently reproduced workspace-dependency setup defect in #746, so repeating it unchanged is not a recovery.

The observer evidence is also recorded in #622, without claiming a proven root cause. The same bounded, maintainer-authorized recovery continues on the existing managed ref with exact child bdbb1bf04a5f80b8f2294c184c42d6874fc40a5a and exact parent 74a2b604c56573569d0f7c967681036d7a286972. Prepare and validate their normal merge locally, preserving the union of all #720, #721 and #722 fixtures (53 unique cases), all actual evidence, prompt contracts and tests, documentation and SDD content; regenerate the catalog and bundles. Require focused evaluation, types and normal full hooks, plus coverage for any changed source. Recheck both remote heads before a normal push to the same ref. No force-push, new remote ref, protection/workflow change, or skipped verification. Fresh CI, finding-free review and Codecov for the resulting head and current parent remain mandatory before merge.

…recommends-an-absent-sdk-id: ground external package claims in current evidence
@docs-page

docs-page Bot commented Oct 27, 2026

Copy link
Copy Markdown

To preview the documentation for this pull request, visit the following URL:

docs.page/vypdev/copilot~745

Documentation is deployed and generated using docs.page

@vypbot vypbot added size: XL Indicates that the issue or task is extra large in scope and complexity. state:ai-processing labels Oct 27, 2026
@vypbot
vypbot self-requested a review October 27, 2026 09:20
@vypbot vypbot added the 85% Progress: 85% label Oct 27, 2026
@vypbot vypbot added this to vypdev Oct 27, 2026
@vypbot vypbot moved this to In Progress in vypdev Oct 27, 2026
@vypbot
vypbot changed the base branch from develop to master October 27, 2026 09:21
@vypbot
vypbot changed the base branch from master to develop October 27, 2026 09:21
@vypbot

vypbot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Bugbot: review complete

Current status: No active findings on be4156c.

Pull request · Verified commit · Workflow run

@vypbot

vypbot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Copilot approval · Human review needed

Status: A protected workflow, policy, or trust-boundary file changed.
Completed: 1/1 observed checks succeeded; Bugbot complete.
Action required: Ask a human reviewer to decide.
Impact: No new approval is confirmed for this revision.
See checks

Technical evidence

Revision: be4156cdafad; base: 74a2b604c565; reason: protected-path.

@codecov-commenter

codecov-commenter commented Oct 27, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.50%. Comparing base (8de9b63) to head (be4156c).
⚠️ Report is 3 commits behind head on develop.

Additional details and impacted files
@@           Coverage Diff            @@
##           develop     #745   +/-   ##
========================================
  Coverage    94.50%   94.50%           
========================================
  Files         1035     1035           
  Lines        33298    33299    +1     
  Branches      9824     9824           
========================================
+ Hits         31467    31468    +1     
  Misses         699      699           
  Partials      1132     1132           
Files with missing lines Coverage Δ
src/prompts/bugbot.ts 100.00% <100.00%> (ø)
src/tooling/bugbot_benchmark_runner.ts 77.77% <100.00%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@efraespada

Copy link
Copy Markdown
Member Author

/copilot sync-branch

@vypbot

vypbot commented Oct 27, 2026

Copy link
Copy Markdown
Contributor

Request could not be completed

Impact: The workflow could not complete the requested operation.

Action: Inspect the current state and retry the failed step.

Retained state: Existing persisted state and completed external effects were preserved.

Error code: workflow.failed

Retryable: Yes

Reference: 515b1d00-c2b6-4e4a-9222-bb2aa2185144

…recommends-an-absent-sdk-id: Merge develop into issue 721 and preserve both evidence regressions
…recommends-an-absent-sdk-id: Merge latest develop while preserving package and Compose review contracts
@vypbot vypbot added state:ai-processing 95% Progress: 95% and removed state:ai-processing 85% Progress: 85% labels Oct 27, 2026
@efraespada
efraespada merged commit 5b3664f into develop Oct 27, 2026
12 checks passed
@github-project-automation github-project-automation Bot moved this from In Progress to Done in vypdev Oct 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

95% Progress: 95% size: XL Indicates that the issue or task is extra large in scope and complexity. state:verified

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

🐛🧑‍💻 - PR review rejects valid official android 37.0 package and recommends an absent SDK ID

3 participants