Skip to content

FAOS optimize preparation skill - #2174

Merged
Luffy Chen (XOEEst) merged 2 commits into
microsoft:mainfrom
XOEEst:main
May 6, 2026
Merged

FAOS optimize preparation skill#2174
Luffy Chen (XOEEst) merged 2 commits into
microsoft:mainfrom
XOEEst:main

Conversation

@XOEEst

@XOEEst Luffy Chen (XOEEst) commented May 6, 2026

Copy link
Copy Markdown
Contributor

This pull request introduces a new FAOS optimization workflow for Python agents in the Microsoft Foundry skill set. It adds comprehensive documentation and usage patterns for converting existing Python agent code to be optimization-ready, including a step-by-step guide, best practices, and framework-neutral code patterns. The documentation ensures that users can expose agent configuration knobs for optimization, review changes before deployment, and preserve existing runtime behavior.

Key additions and changes:

New FAOS Optimization Workflow Documentation

  • Added a new sub-skill, faos-optimize, to the Foundry agent skills, enabling users to convert Python agent code for FAOS optimization with a required review step before deployment. [1] [2]
  • Created detailed documentation in faos-optimize.md, outlining a 12-step workflow for identifying, wiring, and exposing configuration knobs (like instructions, model, and temperature) for FAOS, with guardrails and review checkpoints to ensure safe and targeted changes.
  • Provided a quick reference table summarizing supported languages, required patterns and knobs, and review/deployment workflow.

Python Implementation Patterns

  • Added python-patterns.md, detailing framework-neutral code snippets and best practices for wiring FAOS optimization knobs in single-agent and multi-agent Python architectures, including guidance for Microsoft Agent Framework, FastAPI, LangGraph, and custom runtimes.
  • Included environment variable and dependency management recommendations, as well as a verification checklist to ensure agent integrity after conversion.

User Guidance and Workflow Integration

  • Updated the main skill documentation to include the new faos-optimize workflow and clarified when to use it in the agent lifecycle, integrating it into the user intent-to-workflow mapping. [1] [2]

These changes provide clear, actionable guidance for making Python agents optimization-ready within the Microsoft Foundry ecosystem, ensuring best practices and safe adoption of FAOS optimization features.## Description

Checklist

  • Tests pass locally (cd tests && npm test)
  • If modifying skill descriptions: verified routing correctness with integration tests (npm run test:skills:integration -- <skill>)
  • If modifying skill USE FOR / DO NOT USE FOR / PREFER OVER clauses: confirmed no routing regressions for competing skills

Related Issues

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new faos-optimize workflow under the microsoft-foundry skill to guide users through converting existing Python agent code into a FAOS optimization-ready form, with a mandatory review checkpoint before deployment.

Changes:

  • Registers a new faos-optimize sub-skill in the main microsoft-foundry skill and intent-to-workflow routing table.
  • Adds a 12-step faos-optimize workflow document covering targeting, wiring config knobs, verification, and review gating.
  • Adds a framework-neutral Python patterns reference for implementing the FAOS config contract across common Python agent architectures.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
plugin/skills/microsoft-foundry/SKILL.md Adds faos-optimize to the sub-skill list and agent lifecycle routing table.
plugin/skills/microsoft-foundry/foundry-agent/faos-optimize/faos-optimize.md New end-to-end FAOS optimization-readiness workflow doc (with review gate before deploy).
plugin/skills/microsoft-foundry/foundry-agent/faos-optimize/references/python-patterns.md New Python implementation patterns reference for wiring FAOS knobs while preserving runtime behavior.

Comment thread plugin/skills/microsoft-foundry/foundry-agent/faos-optimize/faos-optimize.md Outdated

@jongio Jon Gallant (jongio) left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One item the bot didn't catch: FAOS is never expanded anywhere in the new docs. Worth defining once at the top so the agent can answer if a user asks what it stands for.

Comment thread plugin/skills/microsoft-foundry/foundry-agent/faos-optimize/faos-optimize.md Outdated
Address PR microsoft#2174 review comments by expanding FAOS as Foundry Agent Optimization Service on first use and making the Step 8 Python config snippet copy-paste safe.

The minimum contract example now imports os and preserves the app's existing model-selection fallback instead of hard-coding MODEL_DEPLOYMENT_NAME unless that is already what the app uses.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@XOEEst
Luffy Chen (XOEEst) merged commit 0833487 into microsoft:main May 6, 2026
12 checks passed
Luffy Chen (XOEEst) added a commit that referenced this pull request May 13, 2026
)

* FAOS optimize preparation skill

* docs(microsoft-foundry): clarify FAOS optimize guidance

Address PR #2174 review comments by expanding FAOS as Foundry Agent Optimization Service on first use and making the Step 8 Python config snippet copy-paste safe.

The minimum contract example now imports os and preserves the app's existing model-selection fallback instead of hard-coding MODEL_DEPLOYMENT_NAME unless that is already what the app uses.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(microsoft-foundry): clarify custom evaluator output contract

When creating custom evaluator prompts, treat the runtime-enforced JSON schema as authoritative: result plus reason. Preserve user-provided rubric text, but remove or normalize conflicting output schemas such as score/reasoning or duplicate OUTPUT FORMAT blocks before calling evaluator_catalog_create.

Add observe skill test coverage for the promptText guardrail so future edits keep the result/reason contract visible.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(microsoft-foundry): keep evaluator example scale-neutral

Remove the 0-to-1 scoringType/minScore/maxScore details from the sample custom evaluator prompt. The important guardrail is the output contract: preserve the rubric, but avoid conflicting output schemas because the runtime enforces result/reason.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix

* Revert microsoft-foundry SKILL.md to upstream

Restore plugin/skills/microsoft-foundry/SKILL.md from upstream/main so this PR no longer carries a line-ending-only change for the top-level skill file.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix faos skill

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Luffy Chen (XOEEst) added a commit that referenced this pull request May 23, 2026
…2286)

* FAOS optimize preparation skill

* docs(microsoft-foundry): clarify FAOS optimize guidance

Address PR #2174 review comments by expanding FAOS as Foundry Agent Optimization Service on first use and making the Step 8 Python config snippet copy-paste safe.

The minimum contract example now imports os and preserves the app's existing model-selection fallback instead of hard-coding MODEL_DEPLOYMENT_NAME unless that is already what the app uses.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(microsoft-foundry): clarify custom evaluator output contract

When creating custom evaluator prompts, treat the runtime-enforced JSON schema as authoritative: result plus reason. Preserve user-provided rubric text, but remove or normalize conflicting output schemas such as score/reasoning or duplicate OUTPUT FORMAT blocks before calling evaluator_catalog_create.

Add observe skill test coverage for the promptText guardrail so future edits keep the result/reason contract visible.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(microsoft-foundry): keep evaluator example scale-neutral

Remove the 0-to-1 scoringType/minScore/maxScore details from the sample custom evaluator prompt. The important guardrail is the output contract: preserve the rubric, but avoid conflicting output schemas because the runtime enforces result/reason.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix

* Revert microsoft-foundry SKILL.md to upstream

Restore plugin/skills/microsoft-foundry/SKILL.md from upstream/main so this PR no longer carries a line-ending-only change for the top-level skill file.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix faos skill

* Eval suite support

* Fix skills

* Add faos update

* Fix comments

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Luffy Chen (XOEEst) added a commit that referenced this pull request May 27, 2026
…veraging azd (#2383)

* FAOS optimize preparation skill

* docs(microsoft-foundry): clarify FAOS optimize guidance

Address PR #2174 review comments by expanding FAOS as Foundry Agent Optimization Service on first use and making the Step 8 Python config snippet copy-paste safe.

The minimum contract example now imports os and preserves the app's existing model-selection fallback instead of hard-coding MODEL_DEPLOYMENT_NAME unless that is already what the app uses.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(microsoft-foundry): clarify custom evaluator output contract

When creating custom evaluator prompts, treat the runtime-enforced JSON schema as authoritative: result plus reason. Preserve user-provided rubric text, but remove or normalize conflicting output schemas such as score/reasoning or duplicate OUTPUT FORMAT blocks before calling evaluator_catalog_create.

Add observe skill test coverage for the promptText guardrail so future edits keep the result/reason contract visible.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(microsoft-foundry): keep evaluator example scale-neutral

Remove the 0-to-1 scoringType/minScore/maxScore details from the sample custom evaluator prompt. The important guardrail is the output contract: preserve the rubric, but avoid conflicting output schemas because the runtime enforces result/reason.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix

* Revert microsoft-foundry SKILL.md to upstream

Restore plugin/skills/microsoft-foundry/SKILL.md from upstream/main so this PR no longer carries a line-ending-only change for the top-level skill file.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix faos skill

* Eval suite support

* Fix skills

* Add faos update

* Fix comments

* latest faos scaffolding

* Support faos scaffolding and azd alignments

* Fix

* Add temp whl path

* Remove unnecessary env vars in scaffold

* Remove private whl path

* Fix comments

* Fix

* Fix Foundry optimizer eval config

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix comments.

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Luffy Chen (XOEEst) added a commit that referenced this pull request Jun 2, 2026
* FAOS optimize preparation skill

* docs(microsoft-foundry): clarify FAOS optimize guidance

Address PR #2174 review comments by expanding FAOS as Foundry Agent Optimization Service on first use and making the Step 8 Python config snippet copy-paste safe.

The minimum contract example now imports os and preserves the app's existing model-selection fallback instead of hard-coding MODEL_DEPLOYMENT_NAME unless that is already what the app uses.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(microsoft-foundry): clarify custom evaluator output contract

When creating custom evaluator prompts, treat the runtime-enforced JSON schema as authoritative: result plus reason. Preserve user-provided rubric text, but remove or normalize conflicting output schemas such as score/reasoning or duplicate OUTPUT FORMAT blocks before calling evaluator_catalog_create.

Add observe skill test coverage for the promptText guardrail so future edits keep the result/reason contract visible.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(microsoft-foundry): keep evaluator example scale-neutral

Remove the 0-to-1 scoringType/minScore/maxScore details from the sample custom evaluator prompt. The important guardrail is the output contract: preserve the rubric, but avoid conflicting output schemas because the runtime enforces result/reason.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix

* Revert microsoft-foundry SKILL.md to upstream

Restore plugin/skills/microsoft-foundry/SKILL.md from upstream/main so this PR no longer carries a line-ending-only change for the top-level skill file.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix faos skill

* Eval suite support

* Fix skills

* Add faos update

* Fix comments

* latest faos scaffolding

* Support faos scaffolding and azd alignments

* Fix

* Add temp whl path

* Remove unnecessary env vars in scaffold

* Remove private whl path

* Fix comments

* Fix

* Fix Foundry optimizer eval config

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix comments.

* Resolve Foundry skill merge conflict

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* faos package update and minor fixes

* fix

* update eval.yaml

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Luffy Chen (XOEEst) added a commit that referenced this pull request Jun 17, 2026
* FAOS optimize preparation skill

* docs(microsoft-foundry): clarify FAOS optimize guidance

Address PR #2174 review comments by expanding FAOS as Foundry Agent Optimization Service on first use and making the Step 8 Python config snippet copy-paste safe.

The minimum contract example now imports os and preserves the app's existing model-selection fallback instead of hard-coding MODEL_DEPLOYMENT_NAME unless that is already what the app uses.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(microsoft-foundry): clarify custom evaluator output contract

When creating custom evaluator prompts, treat the runtime-enforced JSON schema as authoritative: result plus reason. Preserve user-provided rubric text, but remove or normalize conflicting output schemas such as score/reasoning or duplicate OUTPUT FORMAT blocks before calling evaluator_catalog_create.

Add observe skill test coverage for the promptText guardrail so future edits keep the result/reason contract visible.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(microsoft-foundry): keep evaluator example scale-neutral

Remove the 0-to-1 scoringType/minScore/maxScore details from the sample custom evaluator prompt. The important guardrail is the output contract: preserve the rubric, but avoid conflicting output schemas because the runtime enforces result/reason.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix

* Revert microsoft-foundry SKILL.md to upstream

Restore plugin/skills/microsoft-foundry/SKILL.md from upstream/main so this PR no longer carries a line-ending-only change for the top-level skill file.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix faos skill

* Eval suite support

* Fix skills

* Add faos update

* Fix comments

* latest faos scaffolding

* Support faos scaffolding and azd alignments

* Fix

* Add temp whl path

* Remove unnecessary env vars in scaffold

* Remove private whl path

* Fix comments

* Fix

* Fix Foundry optimizer eval config

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix comments.

* Resolve Foundry skill merge conflict

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* faos package update and minor fixes

* fix

* update eval.yaml

* Update azd eval commands

* Fix comments

* Fix comments

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants