Foundry skill dataset related improvements by XOEEst · Pull Request #1294 · microsoft/GitHub-Copilot-for-Azure

Luffy Chen (XOEEst) · 2026-03-15T06:20:40Z

This pull request significantly updates the documentation and workflow for dataset generation, evaluator selection, and metadata conventions for Microsoft Foundry agents. The changes clarify and formalize the process for generating, naming, registering, and tracking evaluation datasets, with a new emphasis on reproducibility, metadata alignment, and responsible evaluator use. A dedicated guide for seed dataset generation is also introduced.

Key changes include:

Dataset Generation & Registration Workflow

Added a comprehensive guide (generate-seed-dataset.md) detailing how to generate, structure, and register a seed evaluation dataset, including schema requirements, coverage targets, and explicit metadata tracking steps. This formalizes local dataset creation and Foundry registration, emphasizing reproducibility and alignment between local and remote references.
Updated the deployment workflow to require local generation of seed datasets, registration in Foundry, and consistent metadata updates in both agent-metadata.yaml and .foundry/datasets/manifest.json. The workflow now includes explicit Azure Storage and Foundry registration steps, and mandates the use of the expected_behavior field. [1] [2]

Metadata & Naming Conventions

Introduced detailed dataset naming and metadata conventions, specifying how local filenames and Foundry dataset names should be constructed and tracked. Required metadata fields (agent, stage, version, datasetUri) are now documented and must be persisted for every dataset.
Updated references and examples throughout the docs to use the new naming conventions and clarify the alignment between local cache and Foundry registration.

Evaluator Selection & Caching

Refined the evaluator selection process: the workflow now distinguishes between built-in and custom evaluators, mandates discovery and reuse of existing evaluators, and recommends a minimal default set for initial runs. The process for refreshing or reusing local cache assets is clarified.

Tooling and API Category Corrections

Standardized terminology and API usage for storage connections, replacing references to AzureBlob with AzureStorageAccount in both documentation and code snippets for project connection discovery and creation. [1] [2] [3]

Other Updates

Added links and references to new and updated documentation sections, such as the seed dataset generation guide and updated dataset comparison pipelines.
Bumped the skill version in SKILL.md to 1.0.6.

…OPILOT_GITHUB_TOKEN` (#6) * Initial plan * Fix issue triage workflow: add github.token fallback for COPILOT_GITHUB_TOKEN The Issue Triage workflow was failing at the secret validation step because COPILOT_GITHUB_TOKEN was not configured. This adds github.token as a fallback in all 4 places where COPILOT_GITHUB_TOKEN is used for authentication: - agent job: validate-secret step and Execute step - detection job: validate-secret step and Execute step This is consistent with the existing fallback patterns in the workflow (e.g., secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN) Co-authored-by: XOEEst <18523445+XOEEst@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: XOEEst <18523445+XOEEst@users.noreply.github.com>

* Fix broken auto-create evaluators step in deploy/observe loop The 'Auto-create evaluators & evaluation dataset' step was being skipped when the monolithic agent-observability-loop skill was split into separate deploy and observe skills. Neither skill owned the auto-create step, causing post-deploy users to jump directly to evaluation. Changes: - deploy.md: Replace generic 'set up evaluation?' prompt with automatic 6-step evaluator & dataset creation matching the reference behavior - observe.md: Add Loop Overview, fix entry points to route post-deploy users through auto-setup, add evaluator existence check - deploy-and-setup.md: Make auto-create primary content, demote deploy section to prerequisites Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add content tests for observe/deploy loop logic Tests verify: - observe.md has Loop Overview, post-deploy entry points, evaluator existence checks, behavioral rules, and all reference files - deploy.md has auto-create evaluators section that is automatic (not optional), includes evaluator categories, LLM-judge, artifact persistence, and routes to observe skill Step 2 - deploy-and-setup.md has auto-create as primary content with proper evaluator selection, dataset generation, and user prompt 49 tests total (29 observe + 20 deploy), all passing. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * chore: trigger CI checks * Fix * add local dataset gen enforcement * Merge * feat: prefer monitor_resource_log_query and local datasets - Replace azure-kusto delegation with monitor_resource_log_query for App Insights KQL queries in trace.md and troubleshoot.md - Mark evaluation_dataset_create as not available (MCP upload not ready) - Replace server-side dataset sections with local JSONL workflow - Update mcp-gap-analysis.md to reflect practical tool availability Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: make dataset upload restriction more agent-proof - Add Do NOT section at top of trace-to-dataset.md (before Overview) - Add behavioral rule #7 to eval-datasets.md: never upload to cloud - Remove Option A/B structure; Step 4 is now local JSONL only - Eliminates subtle strikethrough formatting that agents miss Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix link * fix: make auto-create evaluators an explicit numbered step - Hosted workflow: add Step 10 after Step 9 with DO NOT stop gate - Prompt workflow: add Step 5 after Step 4 with DO NOT stop gate - Both link to existing After Deployment section as implementation - Prevents agents from treating evaluator setup as optional appendix Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat: add dataset update loop with optimization guardrails - Add Dataset Update Loop (eval→compare→analyze→optimize→re-eval) to dataset-versioning.md after Creating a New Version - Add guardrails: never remove dataset rows or weaken evaluators to recover scores after dataset expansion - Add same guardrail to observe optimize-deploy.md Step 6 - Add behavioral rule #8 to eval-datasets.md Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: add subscription parameter warning to trace-related skills Always pass subscription explicitly to Azure MCP tools like monitor_resource_log_query — they don't extract it from resource IDs. Added to trace.md, troubleshoot.md, and trace-to-dataset.md. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: make customEvents-to-traces eval correlation more obvious - Add Key Concept section to trace-to-dataset.md explaining that eval results live in customEvents (not dependencies) and the join key is gen_ai.response.id - Add table showing dependencies vs customEvents join pattern - Cross-reference trace skill's eval-correlation.md from both trace-to-dataset.md and eval-datasets.md Related Skills Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: improve cross-references and add KQL parse_json warning 1. Add parse_json(customDimensions) warning to Do NOT section 2. Add Related References section with skill-root paths 3. Add skill-root path hints to all cross-skill links 4. Add observe + trace to SKILL.md sub-skill routing table Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: improve hosted agent KQL patterns and content extraction - Add Hosted Agent Harvest template (requests→dependencies join) - Fix Hosted Agent Attributes: appear on both requests and traces - Add gen_ai.agent.name duality callout (Foundry name vs class name) - Remove incorrect azure.ai.agentserver.agent_name fallback from dependencies queries - Document gen_ai.input.messages/gen_ai.output.messages as content source - Add operation_ParentId join example to Span Correlation section - Update search-traces.md hosted agent query to use requests entry point Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: improve trace sub-skills for hosted agent KQL patterns - search-traces: fix hosted agent query to group by operation_ParentId - conversation-detail: add content extraction from invoke_agent spans (gen_ai.input.messages / gen_ai.output.messages) - analyze-failures: add hosted agent gen_ai.agent.name duality warning and hosted agent variant query using requests→dependencies join - analyze-latency: same hosted agent warning and variant query - kql-templates: expand requests table description as preferred entry point; add gen_ai.input/output.messages to attributes table - trace.md: reword rule 6 to clarify hosted vs prompt agent filtering Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: restore routing keywords and update trigger snapshots - Add back critical routing keywords to SKILL.md description (578→779 chars): role assignment, permissions, capacity, region, deployment failure, AI Services, Cognitive Services, provision, knowledge index, monitoring, customize, onboard, availability - Update trigger test snapshots for new keyword set (24 snapshots) - Fix deploy trigger test: Docker IS our capability (remove false negative) - Fix customize-deployment tests: ensure prompts have ≥2 keyword matches - Fix deploy-model-optimal-region tests: use longer prompts for HA/PTU Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: add 'create AI Services' to description for resource/create test Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * chore: bump microsoft-foundry version to 1.0.2 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat(eval-datasets): enable Foundry dataset sync via MCP tools - Add Step 5 (Sync to Foundry) to trace-to-dataset pipeline using evaluation_dataset_create with connectionName and project_connection tools - Add server-side version discovery via evaluation_dataset_versions_get - Add dual experiment types to dataset-comparison (agent vs dataset comparison) - Update mcp-gap-analysis: mark resolved tools, update workarounds - Add AzureBlob to project connections reference - Bump microsoft-foundry version to 1.0.3 - Fix upstream section heading changes in unit tests - Update trigger snapshots for upstream keyword changes Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * refactor(dataset-comparison): focus on dataset-version comparison only Remove agent comparison experiment type from dataset-comparison flow. Agent comparison belongs in the observe/eval loop, not the dataset skill. Update all examples to use dataset versions as baseline/treatment. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Remove Playwright MCP server until skills require it (microsoft#1200) * Collapse token analysis comment (microsoft#1147) * update region-availability in prepare/validate/deploy skill (microsoft#1083) * update region-availability in prepare/deploy skill * update * update * fix * update date * Update plugin/skills/azure-deploy/references/region-availability.md * fix ci failure * bump version * build(deps): bump @github/copilot and @github/copilot-sdk in /tests (microsoft#1201) Bumps [@github/copilot](https://github.com/github/copilot-cli) to 1.0.2 and updates ancestor dependency [@github/copilot-sdk](https://github.com/github/copilot-sdk). These dependencies need to be updated together. Updates `@github/copilot` from 0.0.414 to 1.0.2 - [Release notes](https://github.com/github/copilot-cli/releases) - [Changelog](https://github.com/github/copilot-cli/blob/main/changelog.md) - [Commits](github/copilot-cli@v0.0.414...v1.0.2) Updates `@github/copilot-sdk` from 0.1.26 to 0.1.32 - [Release notes](https://github.com/github/copilot-sdk/releases) - [Changelog](https://github.com/github/copilot-sdk/blob/main/CHANGELOG.md) - [Commits](https://github.com/github/copilot-sdk/commits/v0.1.32) --- updated-dependencies: - dependency-name: "@github/copilot" dependency-version: 1.0.2 dependency-type: indirect - dependency-name: "@github/copilot-sdk" dependency-version: 0.1.32 dependency-type: direct:development ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * specify application path in prompt (microsoft#1204) * Add AVM (Azure Verified Modules) integration tests (microsoft#1171) * Add AVM (Azure Verified Modules) integration tests Add 3 integration tests validating the AVM module selection hierarchy for Bicep infrastructure generation: - avm-module-priority: Verifies AVM modules prioritized over non-AVM - avm-fallback-behavior: Verifies fallback stays within AVM ecosystem - avm-azd-pattern-preference: Verifies AZD pattern modules preferred Tests validate that the azure-deploy skill enforces the mandatory AVM selection order: Pattern modules > Resource modules > Utility modules, and never falls back to non-AVM alternatives. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add output assertions to AVM integration tests Address Copilot review feedback: add keyword-based output assertions using getAllAssistantMessages/getAllToolText to verify agent responses contain AVM hierarchy terms, not just skill invocation. Includes non-AVM fallback negative check. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Strengthen AVM test output assertions per Copilot review - Split keyword checks into critical-term + context assertions - Add resource-before-utility ordering assertion for fallback test - Expand non-AVM negative check to use regex patterns - Require core keywords (avm+pattern, azd+pattern) explicitly Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: address Copilot round 3 — ordering assertions and context-aware non-AVM check - Add hierarchy ordering assertion to test 1 (pattern before resource/utility) - Make non-AVM detection context-aware: skip matches preceded by negation words (e.g., 'never fall back to non-AVM' is correct behavior, not a false positive) - Add pattern-before-resource ordering assertion to test 3 (AZD pattern preference) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * refactor: move AVM integration tests to avm/ subdirectory Move tests/azure-deploy/avm-integration.test.ts to tests/azure-deploy/avm/integration.test.ts so the file matches the **/integration.test.ts glob used by the custom ESLint rule (integration-test-name) and follows the subdirectory convention established by tests/microsoft-foundry/ (e.g. foundry-agent/). Import paths updated from ../utils/ to ../../utils/ to reflect the new depth. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: address round 4 Copilot review feedback - Add 'fall back'/'fall-back' keyword variants for resilience - Extend non-AVM negation check to also scan following context - Use regex for AZD ordering assertion to match plural/prefixed variants Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Update github workflows to use best practices (microsoft#1149) * Early terminate azure-deploy, azure-validate tests (microsoft#1205) Add comment for early termination to help AI grader * Replace script inline parameters with env var (microsoft#1209) * Early terminate azure-deploy tests on deploy link (microsoft#1208) * Early terminate azure-deploy tests on deploy link * Fix lint issue * Reduce char count of existing skills (microsoft#1210) * Reduce char count of existing skills * Update ci tests and snapshots * Enhance benchmark ci run script (microsoft#1176) * Add msbench_benchmarks repo clone to get model definition * Remove unused vars * Use mcp-pr repo before MI has access to msbench-benchmarks repo * Address copilot feedback * Change back to msbench-benchmarks repo * Get ADO token for repo clone * Fix line continuation character * Add run for all interested models * Extract run IDs * Fix yaml format issue * Schedule it to run nightly * Address copilot feedbacks * formalize .foundry and multi-environment support * fix * Feature/azure quotas (microsoft#1137) * update for using azure-quotas in skill * test update * unit test update * path update * add skill in skills.json * skills.json update * reduce the text * version update * skill version * skill description update * reduce text size * 1.0.4 for next prepare version * upload snap shot * update version * test update --------- Co-authored-by: Yinghui Dong <yinghuidong@microsoft.com> * build(deps-dev): bump simple-git from 3.30.0 to 3.32.3 in /tests (microsoft#1213) Bumps [simple-git](https://github.com/steveukx/git-js/tree/HEAD/simple-git) from 3.30.0 to 3.32.3. - [Release notes](https://github.com/steveukx/git-js/releases) - [Changelog](https://github.com/steveukx/git-js/blob/main/simple-git/CHANGELOG.md) - [Commits](https://github.com/steveukx/git-js/commits/simple-git@3.32.3/simple-git) --- updated-dependencies: - dependency-name: simple-git dependency-version: 3.32.3 dependency-type: direct:development ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Improve azure-compliance invocation rate (microsoft#1214) * Improve azure-compliance invocation rate * Race condition free report writing * Fix debug logging for report location * Bump skill version * Fix suffix base value * fix * llm judge model and eval group improvement --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Chris Harris <charris@microsoft.com> Co-authored-by: JasonYeMSFT <chuye@microsoft.com> Co-authored-by: xfz11 <81600993+xfz11@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Juan Ospina <70209456+jeo02@users.noreply.github.com> Co-authored-by: Jon Gallant <2163001+jongio@users.noreply.github.com> Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> Co-authored-by: Fan Yang <52458914+fanyang-mono@users.noreply.github.com> Co-authored-by: rakal-dyh <33503911+rakal-dyh@users.noreply.github.com> Co-authored-by: Yinghui Dong <yinghuidong@microsoft.com>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> # Conflicts: # tests/microsoft-foundry/__snapshots__/triggers.test.ts.snap # tests/microsoft-foundry/foundry-agent/create/__snapshots__/triggers.test.ts.snap # tests/microsoft-foundry/foundry-agent/deploy/__snapshots__/triggers.test.ts.snap # tests/microsoft-foundry/foundry-agent/invoke/__snapshots__/triggers.test.ts.snap # tests/microsoft-foundry/foundry-agent/observe/__snapshots__/triggers.test.ts.snap # tests/microsoft-foundry/foundry-agent/trace/__snapshots__/triggers.test.ts.snap # tests/microsoft-foundry/foundry-agent/troubleshoot/__snapshots__/triggers.test.ts.snap # tests/microsoft-foundry/models/deploy/capacity/__snapshots__/triggers.test.ts.snap # tests/microsoft-foundry/models/deploy/customize-deployment/__snapshots__/triggers.test.ts.snap # tests/microsoft-foundry/models/deploy/deploy-model-optimal-region/__snapshots__/triggers.test.ts.snap # tests/microsoft-foundry/models/deploy/deploy-model/__snapshots__/triggers.test.ts.snap # tests/microsoft-foundry/resource/create/__snapshots__/triggers.test.ts.snap

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

# Conflicts: # plugin/skills/microsoft-foundry/foundry-agent/observe/observe.md

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* dataset fix * fix: restore foundry dataset guidance Restore the explicit seed dataset registration guidance in the deploy skill and align dataset docs with the current evaluation_dataset_create MCP surface. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

This PR updates the Microsoft Foundry skill documentation and associated unit tests to formalize evaluation dataset generation/registration, tighten evaluator-selection guidance, and improve hosted-agent trace discovery guidance.

Changes:

Added and integrated a seed evaluation dataset workflow (local generation + Foundry registration + metadata tracking via datasetUri).
Introduced a built-in-first, two-phase evaluator strategy with stronger reuse/discovery guidance (evaluator_catalog_get) and expected_behavior emphasis.
Updated trace KQL guidance for hosted agents (requests-first identity + operation_Id expansion + optional version enrichment).

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
tests/microsoft-foundry/foundry-agent/trace/unit.test.ts	Adds assertions that trace docs keep hosted-agent filtering/versioning guidance consistent.
tests/microsoft-foundry/foundry-agent/observe/unit.test.ts	Expands doc-contract tests for evaluator strategy, dataset registration, and analysis guidance.
tests/microsoft-foundry/foundry-agent/eval-datasets/unit.test.ts	Updates doc-contract tests for new naming/metadata conventions and storage connection category.
tests/microsoft-foundry/foundry-agent/deploy/unit.test.ts	Updates doc-contract tests for two-phase evaluators and dataset registration expectations.
plugin/skills/microsoft-foundry/references/agent-metadata-contract.md	Extends contract/examples to include `datasetVersion` + `datasetUri` and local/remote alignment guidance.
plugin/skills/microsoft-foundry/project/connections.md	Renames storage connection category example to `AzureStorageAccount`.
plugin/skills/microsoft-foundry/foundry-agent/trace/trace.md	Adds hosted-agent identity + `operation_Id` fan-out behavioral rules.
plugin/skills/microsoft-foundry/foundry-agent/trace/references/search-traces.md	Updates hosted-agent KQL templates (requests-first + join on `operation_Id`) and adds version column.
plugin/skills/microsoft-foundry/foundry-agent/trace/references/kql-templates.md	Replaces parent-id join with `operation_Id` join and documents optional version parsing.
plugin/skills/microsoft-foundry/foundry-agent/observe/references/evaluate-step.md	Updates dataset schema guidance to require `expected_behavior`.
plugin/skills/microsoft-foundry/foundry-agent/observe/references/deploy-and-setup.md	Adds evaluator discovery step + seed dataset registration workflow + metadata requirements.
plugin/skills/microsoft-foundry/foundry-agent/observe/references/analyze-results.md	Replaces SDK-based download guidance with REST API pagination/auth details and custom-evaluator parsing.
plugin/skills/microsoft-foundry/foundry-agent/observe/observe.md	Adds rules for evaluator reuse/deletion + introduces the two-phase evaluator strategy section.
plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/references/trace-to-dataset.md	Updates storage connection category to `AzureStorageAccount` and strengthens registration guidance.
plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/references/generate-seed-dataset.md	Introduces a dedicated seed dataset generation + registration + metadata update guide.
plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/eval-datasets.md	Adds naming/metadata conventions and links the new seed-dataset guide.
plugin/skills/microsoft-foundry/foundry-agent/deploy/deploy.md	Updates post-deploy workflow to reuse cache, discover evaluators, generate/register seed datasets, and persist `datasetUri`.
plugin/skills/microsoft-foundry/SKILL.md	Bumps skill version to 1.0.6.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Updates the Microsoft Foundry skill documentation and supporting unit tests to formalize dataset generation/registration workflows, evaluator selection strategy, and storage connection terminology, with an emphasis on reproducible seed datasets and metadata alignment.

Changes:

Added seed dataset generation + registration guidance (including expected_behavior, datasetUri, and cache/manifest updates) and expanded evaluator strategy documentation (built-in first, reuse catalog).
Updated trace guidance for hosted agents to prefer requests-scoped identity + operation_Id fan-out, including agent version enrichment from gen_ai.agent.id.
Aligned docs/tests to use the azure MCP server and AzureStorageAccount connection category; bumped skill version and updated CODEOWNERS.

Reviewed changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
tests/microsoft-foundry/foundry-agent/trace/unit.test.ts	Adds assertions for hosted-agent identity/version and `operation_Id` join guidance in trace docs.
tests/microsoft-foundry/foundry-agent/observe/unit.test.ts	Expands assertions for evaluator strategy, `expected_behavior`, dataset registration, and results-download guidance.
tests/microsoft-foundry/foundry-agent/eval-datasets/unit.test.ts	Updates expectations for `azure` MCP server, dataset naming/metadata, and storage connection terminology.
tests/microsoft-foundry/foundry-agent/deploy/unit.test.ts	Updates expectations for evaluator strategy, dataset registration, and metadata persistence.
plugin/skills/microsoft-foundry/references/agent-metadata-contract.md	Extends contract/example to include `datasetUri` and seed dataset version fields.
plugin/skills/microsoft-foundry/project/connections.md	Renames storage connection category reference to `AzureStorageAccount`.
plugin/skills/microsoft-foundry/foundry-agent/trace/trace.md	Adds hosted-agent identity + `operation_Id` span expansion behavioral rules.
plugin/skills/microsoft-foundry/foundry-agent/trace/references/search-traces.md	Reworks hosted-agent search to start from `requests`, enrich version, and join via `operation_Id`.
plugin/skills/microsoft-foundry/foundry-agent/trace/references/kql-templates.md	Updates KQL templates to use `operation_Id` joins and optional agent version parsing.
plugin/skills/microsoft-foundry/foundry-agent/observe/references/evaluate-step.md	Requires `expected_behavior` in local dataset rows passed via `inputData`.
plugin/skills/microsoft-foundry/foundry-agent/observe/references/deploy-and-setup.md	Adds evaluator catalog reuse, built-in-first strategy, seed dataset generation + Foundry registration steps.
plugin/skills/microsoft-foundry/foundry-agent/observe/references/analyze-results.md	Replaces SDK-based download guidance with REST pagination guidance and parsing notes.
plugin/skills/microsoft-foundry/foundry-agent/observe/observe.md	Adds evaluator catalog checks, deletion parameter rules, and two-phase evaluator strategy section.
plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/references/trace-to-dataset.md	Updates storage connection category to `AzureStorageAccount` and clarifies filename guidance.
plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/references/generate-seed-dataset.md	New guide describing seed dataset schema, coverage targets, local save, Foundry registration, and metadata updates.
plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/eval-datasets.md	Switches MCP server to `azure`, adds dataset naming/metadata conventions and seed dataset entry point.
plugin/skills/microsoft-foundry/foundry-agent/deploy/deploy.md	Switches MCP server to `azure`, adds cache reuse, evaluator discovery, seed dataset generation + registration workflow.
plugin/skills/microsoft-foundry/SKILL.md	Bumps skill version to 1.0.6.
.github/CODEOWNERS	Adds an additional code owner for the microsoft-foundry skill.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

This PR updates the Microsoft Foundry skill documentation to formalize evaluation dataset generation/registration, tighten evaluator selection guidance (built-in first), and improve hosted-agent trace querying patterns. It also updates unit tests to enforce the new documentation expectations and bumps the skill version.

Changes:

Add a new seed dataset generation + Foundry registration guide (including expected_behavior, metadata, and datasetUri tracking).
Standardize dataset naming/versioning + metadata conventions across docs (local cache vs Foundry dataset name/version).
Update trace guidance to resolve hosted-agent identity from requests and fan out via operation_Id, plus refresh evaluator selection guidance to start with <=5 built-ins and reuse existing evaluators.

Reviewed changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/microsoft-foundry/foundry-agent/trace/unit.test.ts	Adds assertions to keep hosted-agent trace guidance aligned (requests-first identity, `operation_Id` joins, version parsing).
tests/microsoft-foundry/foundry-agent/observe/unit.test.ts	Expands doc contract tests for evaluator discovery, two-phase strategy, `expected_behavior`, and REST-based results download guidance.
tests/microsoft-foundry/foundry-agent/eval-datasets/unit.test.ts	Updates expectations for MCP server (`azure`), naming/versioning conventions, and AzureStorageAccount dataset registration flow.
tests/microsoft-foundry/foundry-agent/deploy/unit.test.ts	Updates deploy doc assertions for evaluator strategy, dataset metadata persistence, and seed dataset canonical flow.
plugin/skills/microsoft-foundry/references/agent-metadata-contract.md	Extends the metadata contract with dataset version/URI fields and stronger guidance on stable dataset naming + local/remote linkage.
plugin/skills/microsoft-foundry/project/connections.md	Updates storage connection category terminology to `AzureStorageAccount`.
plugin/skills/microsoft-foundry/foundry-agent/trace/trace.md	Adds hosted-agent identity/version + `operation_Id` fan-out behavioral rules.
plugin/skills/microsoft-foundry/foundry-agent/trace/references/search-traces.md	Reworks hosted-agent KQL to materialize `requests`, parse `gen_ai.agent.id`, and join downstream via `operation_Id`.
plugin/skills/microsoft-foundry/foundry-agent/trace/references/kql-templates.md	Updates KQL templates and clarifies semantics of agent fields and hosted-agent join strategy.
plugin/skills/microsoft-foundry/foundry-agent/observe/references/evaluate-step.md	Requires `expected_behavior` in local dataset rows passed via `inputData`.
plugin/skills/microsoft-foundry/foundry-agent/observe/references/deploy-and-setup.md	Adds evaluator catalog discovery step + two-phase strategy and seed dataset generation/registration workflow alignment.
plugin/skills/microsoft-foundry/foundry-agent/observe/references/analyze-results.md	Switches “download detailed results” guidance to Foundry OpenAI evals REST API with pagination + parsing notes.
plugin/skills/microsoft-foundry/foundry-agent/observe/observe.md	Adds evaluator catalog guardrails, evaluator deletion parameter rules, and two-phase evaluator strategy section.
plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/references/trace-to-dataset.md	Updates storage category terminology + dataset version format (`v<N>`), and reinforces `connectionName` usage.
plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/references/generate-seed-dataset.md	New guide: seed dataset schema, coverage targets, upload/registration steps, and metadata persistence requirements.
plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/references/eval-lineage.md	Aligns manifest example dataset name/version conventions (versionless name + `vN` version).
plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/references/dataset-versioning.md	Aligns dataset name/version conventions with new metadata rules.
plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/eval-datasets.md	Updates MCP server to `azure`, adds seed dataset entry point, and documents naming/metadata conventions + `datasetUri` tracking.
plugin/skills/microsoft-foundry/foundry-agent/deploy/deploy.md	Aligns post-deploy workflow with evaluator discovery + seed dataset guide and requires `datasetUri` persistence.
plugin/skills/microsoft-foundry/SKILL.md	Bumps skill version to `1.0.6`.
.github/CODEOWNERS	Adds an additional code owner for the microsoft-foundry skill directory.

Copilot

Pull request overview

Updates the Microsoft Foundry skill documentation and accompanying unit tests to formalize seed dataset generation/registration, tighten evaluator selection guidance (built-in-first), and standardize dataset naming/metadata conventions across deploy/observe/eval-datasets/trace workflows.

Changes:

Added a dedicated seed dataset generation + Foundry registration guide and updated deploy/observe workflows to reference it, including expected_behavior and datasetUri tracking.
Standardized dataset naming/versioning conventions (versionless dataset name + separate v<N> version) and updated metadata contract examples accordingly.
Refined hosted-agent trace discovery guidance (requests-first identity, operation_Id joins, optional agent version enrichment) and expanded unit tests to lock in the new guidance.

Reviewed changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
tests/microsoft-foundry/foundry-agent/trace/unit.test.ts	Adds assertions for hosted-agent identity/version parsing and `operation_Id` join guidance in trace docs.
tests/microsoft-foundry/foundry-agent/observe/unit.test.ts	Expands assertions for evaluator strategy, seed dataset workflow, metadata persistence, and results-download guidance.
tests/microsoft-foundry/foundry-agent/eval-datasets/unit.test.ts	Updates assertions for MCP server (`azure`), naming/versioning rules, metadata conventions, and seed guide linkage.
tests/microsoft-foundry/foundry-agent/deploy/unit.test.ts	Updates assertions for MCP server (`azure`), two-phase evaluator strategy, seed dataset flow, and `datasetUri` persistence.
plugin/skills/microsoft-foundry/references/agent-metadata-contract.md	Updates the metadata contract to include stable dataset names, `datasetVersion`, local `datasetFile`, and remote `datasetUri`.
plugin/skills/microsoft-foundry/project/connections.md	Renames storage connection category reference from `AzureBlob` to `AzureStorageAccount`.
plugin/skills/microsoft-foundry/foundry-agent/trace/trace.md	Adds hosted-agent rules for requests-first identity, optional version parsing, and `operation_Id` fan-out.
plugin/skills/microsoft-foundry/foundry-agent/trace/references/search-traces.md	Reworks hosted-agent KQL to materialize requests, parse optional version, and join via `operation_Id`; updates summary table to include agent version.
plugin/skills/microsoft-foundry/foundry-agent/trace/references/kql-templates.md	Updates hosted-agent template to `operation_Id` join flow and documents optional version enrichment.
plugin/skills/microsoft-foundry/foundry-agent/observe/references/evaluate-step.md	Requires `expected_behavior` in preferred local dataset rows passed via `inputData`.
plugin/skills/microsoft-foundry/foundry-agent/observe/references/deploy-and-setup.md	Adds evaluator discovery (`evaluator_catalog_get`), built-in-first guidance, seed dataset rules, and metadata fields (`datasetVersion`, `datasetUri`).
plugin/skills/microsoft-foundry/foundry-agent/observe/references/analyze-results.md	Documents downloading per-row eval output items via Azure AI Projects Python SDK and explains custom-evaluator dual-entry parsing.
plugin/skills/microsoft-foundry/foundry-agent/observe/observe.md	Adds evaluator discovery rule, deletion parameter rule, two-phase evaluator strategy section, and knowledge-cutoff mitigation.
plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/references/trace-to-dataset.md	Aligns naming/versioning and storage connection category; requires `connectionName`; switches dataset version formatting to `v<N>`.
plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/references/generate-seed-dataset.md	New end-to-end guide for generating seed datasets with `expected_behavior`, uploading, registering, and persisting metadata including `datasetUri`.
plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/references/eval-lineage.md	Updates manifest examples to keep dataset names versionless and store versions separately (`v<N>`).
plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/references/dataset-versioning.md	Updates naming conventions to drop explicit environment key and adopt `v<N>` version formatting.
plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/references/dataset-curation.md	Updates local filename conventions to match new `<agent-name>-<source>-v<N>` pattern.
plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/eval-datasets.md	Switches MCP server to `azure`, adds seed dataset entry point, and documents dataset naming + required metadata fields including `datasetUri`.
plugin/skills/microsoft-foundry/foundry-agent/deploy/deploy.md	Switches MCP server to `azure`, adds evaluator discovery + two-phase strategy, and makes seed dataset generation/registration mandatory with `datasetUri` persistence.
plugin/skills/microsoft-foundry/SKILL.md	Bumps skill metadata version to `1.0.6`.
.github/CODEOWNERS	Adds an additional code owner for the microsoft-foundry skill directory.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

This PR updates the Microsoft Foundry skill documentation and its unit tests to formalize evaluation dataset generation/registration, evaluator selection strategy, and trace querying guidance—aiming to make Foundry eval workflows more reproducible and metadata-aligned.

Changes:

Introduces a seed evaluation dataset generation + registration guide and threads expected_behavior, datasetUri, and dataset version/name conventions through the workflow docs.
Refines evaluator selection guidance (built-in-first/two-phase strategy, catalog discovery/reuse) and updates related unit tests to enforce the new expectations.
Updates trace documentation and KQL templates to improve hosted-agent identity/version parsing and span discovery via operation_Id.

Reviewed changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
tests/microsoft-foundry/foundry-agent/trace/unit.test.ts	Adds assertions that trace docs keep hosted-agent filtering on `requests` and surface version parsing/join patterns.
tests/microsoft-foundry/foundry-agent/observe/unit.test.ts	Expands content assertions for evaluator strategy, seed dataset workflow, metadata persistence, and results-download guidance.
tests/microsoft-foundry/foundry-agent/eval-datasets/unit.test.ts	Updates expectations for naming/versioning conventions and verifies new seed guide + storage connection terminology.
tests/microsoft-foundry/foundry-agent/deploy/unit.test.ts	Updates expectations for evaluator selection strategy and seed dataset registration flow, including `datasetUri`.
plugin/skills/microsoft-foundry/references/agent-metadata-contract.md	Updates metadata contract/examples to separate dataset name vs version and persist `datasetUri`.
plugin/skills/microsoft-foundry/project/connections.md	Renames storage connection category to `AzureStorageAccount` for dataset upload workflows.
plugin/skills/microsoft-foundry/foundry-agent/trace/trace.md	Adds hosted-agent identity/version and `operation_Id` join behavioral rules.
plugin/skills/microsoft-foundry/foundry-agent/trace/references/search-traces.md	Updates hosted-agent search workflow to start from `requests`, parse version, and join by `operation_Id`.
plugin/skills/microsoft-foundry/foundry-agent/trace/references/kql-templates.md	Replaces parent/child join guidance with `operation_Id` join + version enrichment template.
plugin/skills/microsoft-foundry/foundry-agent/observe/references/evaluate-step.md	Requires `expected_behavior` in local dataset rows passed via `inputData`.
plugin/skills/microsoft-foundry/foundry-agent/observe/references/deploy-and-setup.md	Adds evaluator catalog discovery step + two-phase evaluator strategy + seed dataset generation/registration guidance.
plugin/skills/microsoft-foundry/foundry-agent/observe/references/analyze-results.md	Reworks results download guidance to use Azure AI Projects Python SDK and documents judge knowledge-cutoff pitfalls + custom evaluator parsing.
plugin/skills/microsoft-foundry/foundry-agent/observe/observe.md	Adds evaluator-catalog-first rule, deletion parameter guidance, and formalizes the two-phase evaluator strategy section.
plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/references/trace-to-dataset.md	Updates dataset naming/version fields, storage connection category, and emphasizes `connectionName` when registering datasets.
plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/references/generate-seed-dataset.md	Adds a dedicated seed dataset generation + registration + metadata update guide.
plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/references/eval-lineage.md	Updates lineage examples to use stable dataset names with separate version fields.
plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/references/dataset-versioning.md	Updates naming/version-discovery guidance and partially updates manifest examples.
plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/references/dataset-curation.md	Aligns curated dataset filenames with the updated naming convention.
plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/eval-datasets.md	Updates Quick Reference/server terminology and documents dataset naming + required metadata fields.
plugin/skills/microsoft-foundry/foundry-agent/deploy/deploy.md	Updates post-deploy workflow to reuse cache, discover evaluators, and use the seed dataset guide as canonical flow.
plugin/skills/microsoft-foundry/SKILL.md	Bumps skill version to 1.0.6.
.github/CODEOWNERS	Adds an additional code owner for the microsoft-foundry skill.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

This PR formalizes Microsoft Foundry agent evaluation dataset workflows by tightening documentation around seed dataset generation/registration, evaluator selection/reuse, and dataset metadata conventions, and by updating unit tests to enforce the new guidance.

Changes:

Added a dedicated seed dataset generation + Foundry registration guide (including expected_behavior, versioning, and metadata persistence).
Updated deploy/observe/eval-datasets/trace docs to align on evaluator discovery (evaluator_catalog_get), two-phase evaluator strategy, and dataset naming/metadata conventions (including datasetUri).
Expanded Foundry skill unit tests to validate the new documentation requirements and examples.

Reviewed changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
tests/microsoft-foundry/foundry-agent/trace/unit.test.ts	Adds doc-content assertions for hosted-agent identity/version parsing and operation_Id-based span expansion guidance.
tests/microsoft-foundry/foundry-agent/observe/unit.test.ts	Strengthens tests to enforce two-phase evaluator strategy, `expected_behavior` usage, and seed dataset registration guidance.
tests/microsoft-foundry/foundry-agent/eval-datasets/unit.test.ts	Adds tests for new naming/metadata conventions, AzureStorageAccount usage, and versionless dataset names + separate version metadata.
tests/microsoft-foundry/foundry-agent/deploy/unit.test.ts	Updates expectations for evaluator discovery/selection and seed dataset registration flow references.
plugin/skills/microsoft-foundry/references/agent-metadata-contract.md	Updates examples and guidance to include `datasetVersion` + `datasetUri` and versionless dataset naming.
plugin/skills/microsoft-foundry/project/connections.md	Renames storage connection category reference from `AzureBlob` to `AzureStorageAccount`.
plugin/skills/microsoft-foundry/foundry-agent/trace/trace.md	Adds hosted-agent identity/version and operation_Id span fan-out behavioral rules.
plugin/skills/microsoft-foundry/foundry-agent/trace/references/search-traces.md	Updates hosted-agent query patterns to start from `requests`, enrich agent version, and join via `operation_Id`.
plugin/skills/microsoft-foundry/foundry-agent/trace/references/kql-templates.md	Reworks templates to use `requests` + `operation_Id` joins and adds optional hosted-agent version enrichment.
plugin/skills/microsoft-foundry/foundry-agent/observe/references/evaluate-step.md	Updates preferred local dataset schema to require `expected_behavior` in input rows.
plugin/skills/microsoft-foundry/foundry-agent/observe/references/deploy-and-setup.md	Adds evaluator catalog discovery step, two-phase strategy, and seed dataset generation + registration guidance.
plugin/skills/microsoft-foundry/foundry-agent/observe/references/analyze-results.md	Expands results-download guidance using Azure AI Projects Python SDK and adds custom-evaluator parsing notes.
plugin/skills/microsoft-foundry/foundry-agent/observe/observe.md	Adds evaluator reuse/deletion guardrails and documents the two-phase evaluator strategy and knowledge-cutoff mitigation.
plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/references/trace-to-dataset.md	Aligns naming/version fields and updates storage connection category to `AzureStorageAccount`.
plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/references/generate-seed-dataset.md	Introduces the new canonical seed dataset generation + upload + registration + metadata update workflow.
plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/references/eval-lineage.md	Updates manifest example to versionless dataset names with `vN` versions.
plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/references/dataset-versioning.md	Updates naming conventions and manifest examples to use stable names + `vN` versions.
plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/references/dataset-curation.md	Aligns candidate and curated dataset filename patterns with agentName-first naming.
plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/eval-datasets.md	Switches MCP server reference to `azure` and adds naming/metadata conventions + seed dataset entry point.
plugin/skills/microsoft-foundry/foundry-agent/deploy/deploy.md	Incorporates evaluator discovery + two-phase strategy and makes seed dataset guide the canonical registration flow.
.github/CODEOWNERS	Adds an additional code owner for the microsoft-foundry skill area.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

This PR updates the Microsoft Foundry skill documentation and associated unit tests to formalize evaluation dataset generation/registration, tighten metadata conventions (local + Foundry references), and refine evaluator selection guidance (built-in-first, reuse-before-create).

Changes:

Introduces/standardizes dataset naming + metadata tracking (stable dataset name, separate version, persist datasetUri, and required metadata fields).
Adds a dedicated seed dataset generation + Foundry registration guide and wires deploy/observe workflows to treat it as canonical.
Updates trace KQL guidance for hosted agents (requests-first identity resolution, optional version parsing, operation_Id-based fan-out joins) and expands tests to assert the new guidance.

Reviewed changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
tests/microsoft-foundry/foundry-agent/trace/unit.test.ts	Adds assertions that trace docs preserve hosted-agent identity + span discovery guidance.
tests/microsoft-foundry/foundry-agent/observe/unit.test.ts	Updates assertions for two-phase evaluator strategy, expected_behavior usage, and seed dataset guidance.
tests/microsoft-foundry/foundry-agent/eval-datasets/unit.test.ts	Expands assertions for dataset naming/versioning conventions and seed guide registration requirements.
tests/microsoft-foundry/foundry-agent/deploy/unit.test.ts	Updates assertions to match new seed dataset + evaluator selection workflow in deploy docs.
plugin/skills/microsoft-foundry/references/agent-metadata-contract.md	Updates test case contract/examples to include `datasetVersion` + `datasetUri` and stable dataset naming.
plugin/skills/microsoft-foundry/project/connections.md	Renames storage connection category to `AzureStorageAccount`.
plugin/skills/microsoft-foundry/foundry-agent/trace/trace.md	Adds hosted-agent identity/versioning and `operation_Id` fan-out behavioral rules.
plugin/skills/microsoft-foundry/foundry-agent/trace/references/search-traces.md	Updates hosted-agent KQL to requests-first + `operation_Id` joins; adds optional version enrichment.
plugin/skills/microsoft-foundry/foundry-agent/trace/references/kql-templates.md	Updates templates to use `operation_Id` joins and optional `gen_ai.agent.id` parsing for version enrichment.
plugin/skills/microsoft-foundry/foundry-agent/observe/references/evaluate-step.md	Updates preferred local dataset schema to include `expected_behavior`.
plugin/skills/microsoft-foundry/foundry-agent/observe/references/deploy-and-setup.md	Adds evaluator discovery/reuse guidance; shifts seed generation to local + requires `expected_behavior`; links canonical seed guide.
plugin/skills/microsoft-foundry/foundry-agent/observe/references/analyze-results.md	Reworks result download guidance using Azure AI Projects Python SDK; adds custom evaluator parsing notes and knowledge-cutoff mitigation.
plugin/skills/microsoft-foundry/foundry-agent/observe/observe.md	Adds evaluator reuse rule, two-phase evaluator strategy, deletion parameter guidance, and knowledge-cutoff mitigation.
plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/references/trace-to-dataset.md	Aligns dataset naming/versioning + storage connection category and registration guidance.
plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/references/generate-seed-dataset.md	New canonical seed dataset generation + registration guide (schema, coverage targets, metadata persistence).
plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/references/eval-lineage.md	Updates manifest lineage example to stable name + `v<N>` versions.
plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/references/dataset-versioning.md	Updates naming convention to agentName-based prefix; documents required metadata including `datasetUri`.
plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/references/dataset-curation.md	Aligns candidate/versioned filenames to the new naming conventions.
plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/eval-datasets.md	Updates quick reference + adds dataset naming/metadata conventions and seed dataset guide entry point.
plugin/skills/microsoft-foundry/foundry-agent/deploy/deploy.md	Updates post-deploy workflow to reuse cache, discover evaluators, and use canonical seed dataset registration flow.
.github/CODEOWNERS	Adds an additional code owner for microsoft-foundry skill directory.

JasonYeMSFT (JasonYeMSFT) · 2026-03-19T20:04:44Z

Ankit Sinha (@ankitbko) Christopher T Earley (@tendau) Can you help review the code. You are listed as the code owner of this touched files and the default policy requires at least one of your approvals to be merged.

* Fix Issue Triage workflow: add `github.token` fallback for missing `COPILOT_GITHUB_TOKEN` (microsoft#6) * Initial plan * Fix issue triage workflow: add github.token fallback for COPILOT_GITHUB_TOKEN The Issue Triage workflow was failing at the secret validation step because COPILOT_GITHUB_TOKEN was not configured. This adds github.token as a fallback in all 4 places where COPILOT_GITHUB_TOKEN is used for authentication: - agent job: validate-secret step and Execute step - detection job: validate-secret step and Execute step This is consistent with the existing fallback patterns in the workflow (e.g., secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN) Co-authored-by: XOEEst <18523445+XOEEst@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: XOEEst <18523445+XOEEst@users.noreply.github.com> * Fix/observe deploy loop logic (microsoft#7) * Fix broken auto-create evaluators step in deploy/observe loop The 'Auto-create evaluators & evaluation dataset' step was being skipped when the monolithic agent-observability-loop skill was split into separate deploy and observe skills. Neither skill owned the auto-create step, causing post-deploy users to jump directly to evaluation. Changes: - deploy.md: Replace generic 'set up evaluation?' prompt with automatic 6-step evaluator & dataset creation matching the reference behavior - observe.md: Add Loop Overview, fix entry points to route post-deploy users through auto-setup, add evaluator existence check - deploy-and-setup.md: Make auto-create primary content, demote deploy section to prerequisites Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add content tests for observe/deploy loop logic Tests verify: - observe.md has Loop Overview, post-deploy entry points, evaluator existence checks, behavioral rules, and all reference files - deploy.md has auto-create evaluators section that is automatic (not optional), includes evaluator categories, LLM-judge, artifact persistence, and routes to observe skill Step 2 - deploy-and-setup.md has auto-create as primary content with proper evaluator selection, dataset generation, and user prompt 49 tests total (29 observe + 20 deploy), all passing. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * chore: trigger CI checks * Fix * add local dataset gen enforcement * Merge * feat: prefer monitor_resource_log_query and local datasets - Replace azure-kusto delegation with monitor_resource_log_query for App Insights KQL queries in trace.md and troubleshoot.md - Mark evaluation_dataset_create as not available (MCP upload not ready) - Replace server-side dataset sections with local JSONL workflow - Update mcp-gap-analysis.md to reflect practical tool availability Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: make dataset upload restriction more agent-proof - Add Do NOT section at top of trace-to-dataset.md (before Overview) - Add behavioral rule microsoft#7 to eval-datasets.md: never upload to cloud - Remove Option A/B structure; Step 4 is now local JSONL only - Eliminates subtle strikethrough formatting that agents miss Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix link * fix: make auto-create evaluators an explicit numbered step - Hosted workflow: add Step 10 after Step 9 with DO NOT stop gate - Prompt workflow: add Step 5 after Step 4 with DO NOT stop gate - Both link to existing After Deployment section as implementation - Prevents agents from treating evaluator setup as optional appendix Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat: add dataset update loop with optimization guardrails - Add Dataset Update Loop (eval→compare→analyze→optimize→re-eval) to dataset-versioning.md after Creating a New Version - Add guardrails: never remove dataset rows or weaken evaluators to recover scores after dataset expansion - Add same guardrail to observe optimize-deploy.md Step 6 - Add behavioral rule microsoft#8 to eval-datasets.md Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: add subscription parameter warning to trace-related skills Always pass subscription explicitly to Azure MCP tools like monitor_resource_log_query — they don't extract it from resource IDs. Added to trace.md, troubleshoot.md, and trace-to-dataset.md. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: make customEvents-to-traces eval correlation more obvious - Add Key Concept section to trace-to-dataset.md explaining that eval results live in customEvents (not dependencies) and the join key is gen_ai.response.id - Add table showing dependencies vs customEvents join pattern - Cross-reference trace skill's eval-correlation.md from both trace-to-dataset.md and eval-datasets.md Related Skills Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: improve cross-references and add KQL parse_json warning 1. Add parse_json(customDimensions) warning to Do NOT section 2. Add Related References section with skill-root paths 3. Add skill-root path hints to all cross-skill links 4. Add observe + trace to SKILL.md sub-skill routing table Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: improve hosted agent KQL patterns and content extraction - Add Hosted Agent Harvest template (requests→dependencies join) - Fix Hosted Agent Attributes: appear on both requests and traces - Add gen_ai.agent.name duality callout (Foundry name vs class name) - Remove incorrect azure.ai.agentserver.agent_name fallback from dependencies queries - Document gen_ai.input.messages/gen_ai.output.messages as content source - Add operation_ParentId join example to Span Correlation section - Update search-traces.md hosted agent query to use requests entry point Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: improve trace sub-skills for hosted agent KQL patterns - search-traces: fix hosted agent query to group by operation_ParentId - conversation-detail: add content extraction from invoke_agent spans (gen_ai.input.messages / gen_ai.output.messages) - analyze-failures: add hosted agent gen_ai.agent.name duality warning and hosted agent variant query using requests→dependencies join - analyze-latency: same hosted agent warning and variant query - kql-templates: expand requests table description as preferred entry point; add gen_ai.input/output.messages to attributes table - trace.md: reword rule 6 to clarify hosted vs prompt agent filtering Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: restore routing keywords and update trigger snapshots - Add back critical routing keywords to SKILL.md description (578→779 chars): role assignment, permissions, capacity, region, deployment failure, AI Services, Cognitive Services, provision, knowledge index, monitoring, customize, onboard, availability - Update trigger test snapshots for new keyword set (24 snapshots) - Fix deploy trigger test: Docker IS our capability (remove false negative) - Fix customize-deployment tests: ensure prompts have ≥2 keyword matches - Fix deploy-model-optimal-region tests: use longer prompts for HA/PTU Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: add 'create AI Services' to description for resource/create test Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * chore: bump microsoft-foundry version to 1.0.2 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat(eval-datasets): enable Foundry dataset sync via MCP tools - Add Step 5 (Sync to Foundry) to trace-to-dataset pipeline using evaluation_dataset_create with connectionName and project_connection tools - Add server-side version discovery via evaluation_dataset_versions_get - Add dual experiment types to dataset-comparison (agent vs dataset comparison) - Update mcp-gap-analysis: mark resolved tools, update workarounds - Add AzureBlob to project connections reference - Bump microsoft-foundry version to 1.0.3 - Fix upstream section heading changes in unit tests - Update trigger snapshots for upstream keyword changes Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * refactor(dataset-comparison): focus on dataset-version comparison only Remove agent comparison experiment type from dataset-comparison flow. Agent comparison belongs in the observe/eval loop, not the dataset skill. Update all examples to use dataset versions as baseline/treatment. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Remove Playwright MCP server until skills require it (microsoft#1200) * Collapse token analysis comment (microsoft#1147) * update region-availability in prepare/validate/deploy skill (microsoft#1083) * update region-availability in prepare/deploy skill * update * update * fix * update date * Update plugin/skills/azure-deploy/references/region-availability.md * fix ci failure * bump version * build(deps): bump @github/copilot and @github/copilot-sdk in /tests (microsoft#1201) Bumps [@github/copilot](https://github.com/github/copilot-cli) to 1.0.2 and updates ancestor dependency [@github/copilot-sdk](https://github.com/github/copilot-sdk). These dependencies need to be updated together. Updates `@github/copilot` from 0.0.414 to 1.0.2 - [Release notes](https://github.com/github/copilot-cli/releases) - [Changelog](https://github.com/github/copilot-cli/blob/main/changelog.md) - [Commits](github/copilot-cli@v0.0.414...v1.0.2) Updates `@github/copilot-sdk` from 0.1.26 to 0.1.32 - [Release notes](https://github.com/github/copilot-sdk/releases) - [Changelog](https://github.com/github/copilot-sdk/blob/main/CHANGELOG.md) - [Commits](https://github.com/github/copilot-sdk/commits/v0.1.32) --- updated-dependencies: - dependency-name: "@github/copilot" dependency-version: 1.0.2 dependency-type: indirect - dependency-name: "@github/copilot-sdk" dependency-version: 0.1.32 dependency-type: direct:development ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * specify application path in prompt (microsoft#1204) * Add AVM (Azure Verified Modules) integration tests (microsoft#1171) * Add AVM (Azure Verified Modules) integration tests Add 3 integration tests validating the AVM module selection hierarchy for Bicep infrastructure generation: - avm-module-priority: Verifies AVM modules prioritized over non-AVM - avm-fallback-behavior: Verifies fallback stays within AVM ecosystem - avm-azd-pattern-preference: Verifies AZD pattern modules preferred Tests validate that the azure-deploy skill enforces the mandatory AVM selection order: Pattern modules > Resource modules > Utility modules, and never falls back to non-AVM alternatives. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add output assertions to AVM integration tests Address Copilot review feedback: add keyword-based output assertions using getAllAssistantMessages/getAllToolText to verify agent responses contain AVM hierarchy terms, not just skill invocation. Includes non-AVM fallback negative check. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Strengthen AVM test output assertions per Copilot review - Split keyword checks into critical-term + context assertions - Add resource-before-utility ordering assertion for fallback test - Expand non-AVM negative check to use regex patterns - Require core keywords (avm+pattern, azd+pattern) explicitly Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: address Copilot round 3 — ordering assertions and context-aware non-AVM check - Add hierarchy ordering assertion to test 1 (pattern before resource/utility) - Make non-AVM detection context-aware: skip matches preceded by negation words (e.g., 'never fall back to non-AVM' is correct behavior, not a false positive) - Add pattern-before-resource ordering assertion to test 3 (AZD pattern preference) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * refactor: move AVM integration tests to avm/ subdirectory Move tests/azure-deploy/avm-integration.test.ts to tests/azure-deploy/avm/integration.test.ts so the file matches the **/integration.test.ts glob used by the custom ESLint rule (integration-test-name) and follows the subdirectory convention established by tests/microsoft-foundry/ (e.g. foundry-agent/). Import paths updated from ../utils/ to ../../utils/ to reflect the new depth. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: address round 4 Copilot review feedback - Add 'fall back'/'fall-back' keyword variants for resilience - Extend non-AVM negation check to also scan following context - Use regex for AZD ordering assertion to match plural/prefixed variants Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Update github workflows to use best practices (microsoft#1149) * Early terminate azure-deploy, azure-validate tests (microsoft#1205) Add comment for early termination to help AI grader * Replace script inline parameters with env var (microsoft#1209) * Early terminate azure-deploy tests on deploy link (microsoft#1208) * Early terminate azure-deploy tests on deploy link * Fix lint issue * Reduce char count of existing skills (microsoft#1210) * Reduce char count of existing skills * Update ci tests and snapshots * Enhance benchmark ci run script (microsoft#1176) * Add msbench_benchmarks repo clone to get model definition * Remove unused vars * Use mcp-pr repo before MI has access to msbench-benchmarks repo * Address copilot feedback * Change back to msbench-benchmarks repo * Get ADO token for repo clone * Fix line continuation character * Add run for all interested models * Extract run IDs * Fix yaml format issue * Schedule it to run nightly * Address copilot feedbacks * formalize .foundry and multi-environment support * fix * Feature/azure quotas (microsoft#1137) * update for using azure-quotas in skill * test update * unit test update * path update * add skill in skills.json * skills.json update * reduce the text * version update * skill version * skill description update * reduce text size * 1.0.4 for next prepare version * upload snap shot * update version * test update --------- Co-authored-by: Yinghui Dong <yinghuidong@microsoft.com> * build(deps-dev): bump simple-git from 3.30.0 to 3.32.3 in /tests (microsoft#1213) Bumps [simple-git](https://github.com/steveukx/git-js/tree/HEAD/simple-git) from 3.30.0 to 3.32.3. - [Release notes](https://github.com/steveukx/git-js/releases) - [Changelog](https://github.com/steveukx/git-js/blob/main/simple-git/CHANGELOG.md) - [Commits](https://github.com/steveukx/git-js/commits/simple-git@3.32.3/simple-git) --- updated-dependencies: - dependency-name: simple-git dependency-version: 3.32.3 dependency-type: direct:development ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Improve azure-compliance invocation rate (microsoft#1214) * Improve azure-compliance invocation rate * Race condition free report writing * Fix debug logging for report location * Bump skill version * Fix suffix base value * fix * llm judge model and eval group improvement --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Chris Harris <charris@microsoft.com> Co-authored-by: JasonYeMSFT <chuye@microsoft.com> Co-authored-by: xfz11 <81600993+xfz11@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Juan Ospina <70209456+jeo02@users.noreply.github.com> Co-authored-by: Jon Gallant <2163001+jongio@users.noreply.github.com> Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> Co-authored-by: Fan Yang <52458914+fanyang-mono@users.noreply.github.com> Co-authored-by: rakal-dyh <33503911+rakal-dyh@users.noreply.github.com> Co-authored-by: Yinghui Dong <yinghuidong@microsoft.com> * fix: expose observe prompt optimization routing Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> # Conflicts: # tests/microsoft-foundry/__snapshots__/triggers.test.ts.snap # tests/microsoft-foundry/foundry-agent/create/__snapshots__/triggers.test.ts.snap # tests/microsoft-foundry/foundry-agent/deploy/__snapshots__/triggers.test.ts.snap # tests/microsoft-foundry/foundry-agent/invoke/__snapshots__/triggers.test.ts.snap # tests/microsoft-foundry/foundry-agent/observe/__snapshots__/triggers.test.ts.snap # tests/microsoft-foundry/foundry-agent/trace/__snapshots__/triggers.test.ts.snap # tests/microsoft-foundry/foundry-agent/troubleshoot/__snapshots__/triggers.test.ts.snap # tests/microsoft-foundry/models/deploy/capacity/__snapshots__/triggers.test.ts.snap # tests/microsoft-foundry/models/deploy/customize-deployment/__snapshots__/triggers.test.ts.snap # tests/microsoft-foundry/models/deploy/deploy-model-optimal-region/__snapshots__/triggers.test.ts.snap # tests/microsoft-foundry/models/deploy/deploy-model/__snapshots__/triggers.test.ts.snap # tests/microsoft-foundry/resource/create/__snapshots__/triggers.test.ts.snap * chore: drop issue triage token fallback Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: simplify deploy P0 test case guidance Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * chore: align microsoft-foundry version Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix comments * test: move eval-datasets invocation tests Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix comments * bump version * dataset fix (microsoft#9) * dataset fix * fix: restore foundry dataset guidance Restore the explicit seed dataset registration guidance in the deploy skill and align dataset docs with the current evaluation_dataset_create MCP surface. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * chore: add codeowner for microsoft-foundry Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: address foundry PR review comments Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: normalize foundry dataset versioning docs Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Deduplicate Foundry dataset registration docs Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * use sdk for eval results * Fix comments * Align observe test with SDK guidance Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Remove redundant Foundry MCP tips Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix dataset manifest version examples Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix * Align eval-datasets metadata docs Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Merge remote-tracking branch 'upstream/main' --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com> Co-authored-by: XOEEst <18523445+XOEEst@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Chris Harris <charris@microsoft.com> Co-authored-by: JasonYeMSFT <chuye@microsoft.com> Co-authored-by: xfz11 <81600993+xfz11@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Juan Ospina <70209456+jeo02@users.noreply.github.com> Co-authored-by: Jon Gallant <2163001+jongio@users.noreply.github.com> Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com> Co-authored-by: Fan Yang <52458914+fanyang-mono@users.noreply.github.com> Co-authored-by: rakal-dyh <33503911+rakal-dyh@users.noreply.github.com> Co-authored-by: Yinghui Dong <yinghuidong@microsoft.com>

Copilot AI and others added 13 commits March 10, 2026 23:35

chore: drop issue triage token fallback

7e19dd0

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

docs: simplify deploy P0 test case guidance

b45bda1

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

chore: align microsoft-foundry version

67d153c

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Fix comments

4dcf65a

Merge remote-tracking branch 'upstream/main'

abaa1a3

# Conflicts: # plugin/skills/microsoft-foundry/foundry-agent/observe/observe.md

test: move eval-datasets invocation tests

6862366

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Fix comments

85eefe5

bump version

c6dc5b4

Merge remote-tracking branch 'upstream/main'

f6cf2f6

Luffy Chen (XOEEst) requested a review from Ankit Sinha (ankitbko) as a code owner March 15, 2026 06:20

Copilot AI review requested due to automatic review settings March 15, 2026 06:20

Copilot started reviewing on behalf of Luffy Chen (XOEEst) March 15, 2026 06:21 View session

Copilot AI reviewed Mar 15, 2026

View reviewed changes

Luffy Chen (XOEEst) and others added 2 commits March 14, 2026 23:29

chore: add codeowner for microsoft-foundry

d3cf117

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

fix: address foundry PR review comments

2d79f12

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings March 15, 2026 06:35

Copilot started reviewing on behalf of Luffy Chen (XOEEst) March 15, 2026 06:36 View session

Copilot AI reviewed Mar 15, 2026

View reviewed changes

Comment thread plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/eval-datasets.md Outdated

Comment thread plugin/skills/microsoft-foundry/references/agent-metadata-contract.md

Comment thread plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/references/trace-to-dataset.md

fix: normalize foundry dataset versioning docs

3c003a6

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

JasonYeMSFT (JasonYeMSFT) reviewed Mar 16, 2026

View reviewed changes

Comment thread plugin/skills/microsoft-foundry/foundry-agent/observe/references/analyze-results.md Outdated

JasonYeMSFT (JasonYeMSFT) reviewed Mar 16, 2026

View reviewed changes

Comment thread plugin/skills/microsoft-foundry/foundry-agent/observe/references/deploy-and-setup.md Outdated

JasonYeMSFT (JasonYeMSFT) previously approved these changes Mar 16, 2026

View reviewed changes

Ankit Sinha (ankitbko) previously approved these changes Mar 17, 2026

View reviewed changes

Deduplicate Foundry dataset registration docs

80ab8f6

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings March 17, 2026 05:34

Luffy Chen (XOEEst) dismissed Ankit Sinha (ankitbko)’s stale review via 80ab8f6 March 17, 2026 05:34

Copilot started reviewing on behalf of Luffy Chen (XOEEst) March 17, 2026 05:35 View session

Copilot AI reviewed Mar 17, 2026

View reviewed changes

Comment thread plugin/skills/microsoft-foundry/foundry-agent/observe/references/analyze-results.md Outdated

Comment thread plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/eval-datasets.md Outdated

Luffy Chen (XOEEst) added 2 commits March 16, 2026 22:57

use sdk for eval results

adaebc2

Fix comments

70c36f1

Copilot AI review requested due to automatic review settings March 17, 2026 06:55

Copilot started reviewing on behalf of Luffy Chen (XOEEst) March 17, 2026 06:56 View session

Copilot AI reviewed Mar 17, 2026

View reviewed changes

Comment thread tests/microsoft-foundry/foundry-agent/observe/unit.test.ts Outdated

Align observe test with SDK guidance

abb562b

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Luffy Chen (XOEEst) requested a review from Christopher T Earley (tendau) as a code owner March 17, 2026 17:05

Christopher T Earley (tendau) reviewed Mar 17, 2026

View reviewed changes

Comment thread plugin/skills/microsoft-foundry/foundry-agent/deploy/deploy.md Outdated

Comment thread plugin/skills/microsoft-foundry/foundry-agent/observe/references/analyze-results.md

Remove redundant Foundry MCP tips

3f87f9f

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings March 17, 2026 21:52

Copilot started reviewing on behalf of Luffy Chen (XOEEst) March 17, 2026 21:53 View session

Copilot AI reviewed Mar 17, 2026

View reviewed changes

Luffy Chen (XOEEst) and others added 2 commits March 17, 2026 16:19

Fix dataset manifest version examples

cb2d005

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Merge upstream/main into main

6bff838

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings March 18, 2026 17:07

Copilot started reviewing on behalf of Luffy Chen (XOEEst) March 18, 2026 17:08 View session

Copilot AI reviewed Mar 18, 2026

View reviewed changes

Luffy Chen (XOEEst) and others added 2 commits March 18, 2026 11:44

fix

960811e

Align eval-datasets metadata docs

4ea88b4

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings March 19, 2026 02:09

Copilot AI reviewed Mar 19, 2026

View reviewed changes

Comment thread plugin/skills/microsoft-foundry/foundry-agent/eval-datasets/references/trace-to-dataset.md

Luffy Chen (XOEEst) added 2 commits March 19, 2026 11:25

Merge remote-tracking branch 'upstream/main'

8baaf41

Merge remote-tracking branch 'upstream/main'

f5e5859

JasonYeMSFT (JasonYeMSFT) approved these changes Mar 19, 2026

View reviewed changes

Christopher T Earley (tendau) approved these changes Mar 19, 2026

View reviewed changes

Luffy Chen (XOEEst) merged commit 13e7683 into microsoft:main Mar 19, 2026
12 checks passed

Uh oh!

Conversation

Luffy Chen (XOEEst) commented Mar 15, 2026

Dataset Generation & Registration Workflow

Metadata & Naming Conventions

Evaluator Selection & Caching

Tooling and API Category Corrections

Other Updates

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

JasonYeMSFT (JasonYeMSFT) commented Mar 19, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants