Skip to content

[BUG] Clean grok-cloud install fails audit after replay selects a different target #2816

Description

Describe the bug
A clean local skill installed with --target grok-cloud fails apm audit --ci: audit replays the skill into Copilot's .agents/skills/ tree and reports that unrelated location as unintegrated, although the installed .grok/skills/ file and lockfile agree.

An unrelated .github/workflows/ file supplies a target-detection signal. Saving apm config set target grok-cloud does not correct audit replay. The usual durable-target workaround, targets: [grok-cloud] in the consumer manifest, is explicitly rejected by manifest parsing even after experimental enablement. This leaves an unresolved install/audit target-contract mismatch; it is not evidence of actual content tampering.

To Reproduce
Use a disposable consumer directory and isolated HOME.

  1. Create a local skill package at an absolute path /ABS/package:
package/
  apm.yml
  skills/example/SKILL.md

apm.yml:

name: example-package
version: 0.1.0

skills/example/SKILL.md:

---
name: example
description: Experimental target audit reproduction
---
# Clean source content
  1. Create the consumer's apm.yml, substituting the actual absolute package path:
name: consumer
version: 0.1.0
dependencies:
  apm:
    - path: /ABS/package
  1. Add an unrelated .github/workflows/unrelated.yml file to the consumer (for example, name: unrelated). Do not author a Copilot skill or request a Copilot deployment.
  2. Run:
apm experimental enable grok-cloud
apm config set target grok-cloud
apm install --target grok-cloud --no-policy --parallel-downloads 0
apm audit --ci --no-policy --format json
  1. Install exits 0, writes .grok/skills/example/SKILL.md, and records grok-cloud deployment ownership and the matching content hash. Audit exits 1 because replay expects .agents/skills/example/SKILL.md instead.

Additional diagnostic control: declaring targets: [grok-cloud] in the consumer manifest makes install exit 2 with Unknown target 'grok-cloud', despite the enabled flag. That unsuccessful fixture workaround was not retained in the interaction regression.

Expected behavior
A clean installed target should round-trip through audit without false drift caused by an unrelated harness-directory signal. Define a consistent target-intent contract for experimental targets across install and audit, including how manifest restrictions, explicit installation targets, configured defaults, and deployment ownership interact.

Do not suppress the drift check, discard ownership assertions, or broadly ignore experimental-target findings to make the case pass.

Environment (please complete the following information):

  • OS: macOS
  • Python Version: 3.12.10, locked development dependencies
  • APM Version: 0.29.1, source commit 23e921750e5bdb25d02c7b0c71a6b97bd7b851ac, editable installed CLI
  • VSCode Version (if relevant): N/A

Logs
The real fixture uses a longer unique skill name; the relevant audit output is:

{
  "passed": false,
  "checks": [
    {
      "name": "content-integrity",
      "passed": true,
      "message": "No critical hidden Unicode or hash drift detected"
    },
    {
      "name": "drift",
      "passed": false,
      "message": "drift detected: 1 file(s): .agents/skills/example/SKILL.md",
      "details": ["unintegrated: .agents/skills/example/SKILL.md"]
    }
  ]
}

The actual result has ten checks: all nine non-drift checks pass, including ref consistency, deployment-ledger owners, deployed-file presence, and content integrity.

Additional context
Discovered while independently hardening APMLifecycle after #2809 / during rework of #2813. The unskipped, locally reproduced interaction is tests/integration/test_primitive_target_covering_array.py::test_primitive_target_covering_array[interaction-b8852029871d9058] in the current not-yet-published test-only worktree. The final four-shard run reproduces the failure, and the combined evidence gate correctly refuses to credit its interaction pairs.

Source boundaries at the tested commit: install/drift.py::_read_apm_yml_target and run_replay select replay targets from the manifest or detection; core/apm_yml.py::_validate_canonical rejects experimental capabilities in manifest target fields. reference/cli/config.md currently documents the configured default as an install/update setting, so simply assuming audit already promises to honor it is not a fix or a sufficient contract analysis.

Generalized properties: routing.authorized_targets_only, outcome.status_matches_state, clean install/audit consistency. Production changes and Phase 3 transaction redesign remain separately approval-gated.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

status/acceptedDirection approved, safe to start work.status/shepherdingActively being driven by an APM shepherd runtype/bugSomething does not work as documented.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions