Skip to content

fix: Get hooks working in Copilot again - #2719

Merged
Tom Meschter (tmeschter) merged 4 commits into
microsoft:mainfrom
tmeschter:260629-FixTelemetryHooks
Jun 30, 2026
Merged

fix: Get hooks working in Copilot again#2719
Tom Meschter (tmeschter) merged 4 commits into
microsoft:mainfrom
tmeschter:260629-FixTelemetryHooks

Conversation

@tmeschter

@tmeschter Tom Meschter (tmeschter) commented Jun 29, 2026

Copy link
Copy Markdown
Member

The different coding agents (VS Code, Copilot, Claude etc.) have varying levels of support for the Open Plugins spec and varying fallback mechanisms to try and support plugins designed for other coding agents. This makes for a bit of a mess when it comes to supporting hooks.

For one thing, all the coding agents (or at least VS Code, Copilot CLI, and Claude) agree that hooks/hooks.json is a default location to find hooks. However, they do not agree on the format of that file. Further, they do not agree on a mechanism for excluding that file from the set of hooks files they read.

VS Code attempts to solve this by trying to determine if a plugin is following the Open Plugin spec, the Copilot spec, or the Claude spec, and varying its behavior accordingly. For example, for an "open" plugin it defines the PLUGIN_ROOT variable; for Claude CLAUDE_PLUGIN_ROOT, and for Copilot plugins... neither. However, Copilot itself defines both PLUGIN_ROOT and CLAUDE_PLUGIN_ROOT at all times. So a "Copilot" plugin may not work in VS Code, even though it is explicitly trying to make it work the same way.

Here's where we currently are regarding hooks:

  • In VS Code they work. It picks up the .plugin/plugin.json file, causing it to treat this as an "open" plugin. It honors using hooks/hooks.json as the exclusive source of hook.
  • In Copilot hooks don't work at all. It, too, picks up .plugin/plugin.json but it doesn't understand the structure of the hooks section at all (it doesn't support the exclusive annotation) and so loads no hooks.
  • Claude sort of works. It will pick up .claude-plugin/plugin.json, and since that does not specify a hooks file it will use hooks/hooks.json as the default. However, spaces anywhere in the expanded ${CLAUDE_PLUGIN_ROOT} will break the call to track-telemetry.sh.

Here we make changes so that VS Code and Copilot will continue to use .plugin/plugin.json, but we remove the hooks section entirely. They will both fall back to the default of hooks/hooks.json. VS Code will use the windows, osx and linux commands, while Copilot will use bash and powershell instead.

Claude is pointed to hooks/claude-hooks.json which is in the format it expects and adds in the necessary double-quotes to prevent whitespace issues. Note that it will still try to load hooks/hooks.json as well and and raise an error about it not being in the right format; this seems to be harmless.

Also note that Claude provides no good way to automatically varying behavior between operating systems, so the telemetry reporting effectively only works on system where "bash" is installed (effectively every Mac and Linux system, but only some Windows systems).

Experimentally this allows VS Code and Copilot to report telemetry on Windows systems. I am unable to test Claude fully but it does "see" the hook in claude-hooks.json.

Description

Checklist

  • Tests pass locally (cd tests && npm test)
  • Title has one of the prefixes: fix:, feat:, feature:, chore:, misc:, test:, eval:
  • If modifying skill descriptions: verified routing correctness with integration tests (In tests/, npm run test:integration -- <skill> or npm run test:vally -- --skill <skill>)

Related Issues

The different coding agents (VS Code, Copilot, Claude etc.) have varying levels of support for the [Open Plugins spec](https://open-plugins.com/) and varying fallback mechanisms to try and support plugins designed for other coding agents. This makes for a bit of a mess when it comes to supporting hooks.

For one thing, all the coding agents (or at least VS Code, Copilot CLI, and Claude) agree that hooks/hooks.json is a default location to find hooks. However, they do not agree on the format of that file. Further, they do not agree on a mechanism for excluding that file from the set of hooks files they read.

VS Code attempts to solve this by trying to determine if a plugin is following the Open Plugin spec, the Copilot spec, or the Claude spec, and varying its behavior accordingly. For example, for an "open" plugin it defines the `PLUGIN_ROOT` variable; for Claude `CLAUDE_PLUGIN_ROOT`, and for Copilot plugins... neither. However, Copilot itself defines both `PLUGIN_ROOT` _and_ `CLAUDE_PLUGIN_ROOT` at all times. So a "Copilot" plugin may not work in VS Code, even though it is explicitly trying to make it work the same way.

Here's where we currently are regarding hooks:
- In VS Code they work. It picks up the .plugin/plugin.json file, causing it to treat this as an "open" plugin. It honors using hooks/hooks.json as the exclusive source of hook.
- In Copilot hooks don't work at all. It, too, picks up .plugin/plugin.json but it doesn't understand the structure of the `hooks` section at all (it doesn't support the `exclusive` annotation) and so loads no hooks.
- Claude sort of works. It will pick up .claude-plugin/plugin.json, and since that does not specify a hooks file it will use hooks/hooks.json as the default. However, spaces anywhere in the expanded `${CLAUDE_PLUGIN_ROOT}` will break the call to track-telemetry.sh.

Here we make changes so that VS Code and Copilot will continue to use .plugin/plugin.json, but we remove the `hooks` section entirely. They will both fall back to the default of hooks/hooks.json. VS Code will use the `windows`, `osx` and `linux` commands, while Copilot will use `bash` and `powershell` instead.

Claude is pointed to hooks/claude-hooks.json which is in the format it expects and adds in the necessary double-quotes to prevent whitespace issues. Note that it will still try to load hooks/hooks.json as well and and raise an error about it not being in the right format; this seems to be harmless.

Also note that Claude provides no good way to automatically varying behavior between operating systems, so the telemetry reporting effectively only works on system where "bash" is installed (effectively every Mac and Linux system, but only some Windows systems).

Experimentally this allows VS Code and Copilot to report telemetry on Windows systems. I am unable to test Claude fully but it does "see" the hook in claude-hooks.json.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates plugin hook configuration to restore telemetry hooks across different coding agents by aligning each agent’s manifest and hook-file format expectations.

Changes:

  • Simplifies plugin/.plugin/plugin.json by removing the hooks section so clients fall back to the default hooks/hooks.json.
  • Reworks plugin/hooks/hooks.json into a single file intended to work for both VS Code (OS-specific keys) and Copilot (shell-specific keys).
  • Adds a Claude-specific hooks file (plugin/hooks/claude-hooks.json) and points the Claude manifest at it; removes the old Copilot-specific hooks file.
Show a summary per file
File Description
plugin/hooks/hooks.json Restructures the default hooks definition to include OS- and shell-specific command entries.
plugin/hooks/copilot-hooks.json Removes the Copilot-specific hooks file (intended to rely on default hooks discovery).
plugin/hooks/claude-hooks.json Adds a Claude-specific hooks file and command string intended to handle whitespace in paths.
plugin/.plugin/plugin.json Removes explicit hooks config to trigger default hooks file fallback behavior.
plugin/.claude-plugin/plugin.json Configures Claude to use the new Claude-specific hooks file.

Review details

  • Files reviewed: 5/5 changed files
  • Comments generated: 1
  • Review effort level: Low

Comment thread plugin/hooks/claude-hooks.json

@RickWinter Rick Winter (RickWinter) left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot looks to be using claude schema.

@RickWinter Rick Winter (RickWinter) left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

README is stale and needs updating with this PR

@jongio Jon Gallant (jongio) left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The approach here is well-reasoned and the PR description does a great job explaining the compatibility matrix.

Two items:

  1. The hooks compatibility table in README.md (line 137-138) will be stale after this lands. Copilot CLI row still references plugin/hooks/copilot-hooks.json and Claude Code row says hooks/hooks.json (no custom hooks). Both are now wrong. Worth updating here or in a quick follow-up.

  2. Nit: claude-hooks.json is missing a trailing newline.

Comment thread plugin/hooks/claude-hooks.json Outdated
@tmeschter

Copy link
Copy Markdown
Member Author

Copilot looks to be using claude schema.

Rick Winter (@RickWinter) In what sense?

Update the Claude `command` for `PostToolUse` to explicitly include "bash"; this avoids the quotes becoming part of the executable name as well as any potential issues with the .sh script being executable or not.

@jongio Jon Gallant (jongio) left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified the full changeset. The path quoting asymmetry between hooks.json (unquoted ) and claude-hooks.json (quoted ) is correct: Copilot and VS Code resolve paths internally via structured keys, while Claude passes the command value to a shell as-is, requiring explicit quotes and the bash prefix. No blocking issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants