FAOS optimize preparation skill - #2174
Merged
Merged
Conversation
Luffy Chen (XOEEst)
marked this pull request as ready for review
May 6, 2026 06:39
Luffy Chen (XOEEst)
requested review from
Xiaofu Huang (XiaofuHuang),
anchenyi,
Ankit Sinha (ankitbko),
Jon Gallant (jongio) and
Christopher T Earley (tendau)
as code owners
May 6, 2026 06:39
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new faos-optimize workflow under the microsoft-foundry skill to guide users through converting existing Python agent code into a FAOS optimization-ready form, with a mandatory review checkpoint before deployment.
Changes:
- Registers a new
faos-optimizesub-skill in the mainmicrosoft-foundryskill and intent-to-workflow routing table. - Adds a 12-step
faos-optimizeworkflow document covering targeting, wiring config knobs, verification, and review gating. - Adds a framework-neutral Python patterns reference for implementing the FAOS config contract across common Python agent architectures.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| plugin/skills/microsoft-foundry/SKILL.md | Adds faos-optimize to the sub-skill list and agent lifecycle routing table. |
| plugin/skills/microsoft-foundry/foundry-agent/faos-optimize/faos-optimize.md | New end-to-end FAOS optimization-readiness workflow doc (with review gate before deploy). |
| plugin/skills/microsoft-foundry/foundry-agent/faos-optimize/references/python-patterns.md | New Python implementation patterns reference for wiring FAOS knobs while preserving runtime behavior. |
Jon Gallant (jongio)
left a comment
Collaborator
There was a problem hiding this comment.
One item the bot didn't catch: FAOS is never expanded anywhere in the new docs. Worth defining once at the top so the agent can answer if a user asks what it stands for.
Address PR microsoft#2174 review comments by expanding FAOS as Foundry Agent Optimization Service on first use and making the Step 8 Python config snippet copy-paste safe. The minimum contract example now imports os and preserves the app's existing model-selection fallback instead of hard-coding MODEL_DEPLOYMENT_NAME unless that is already what the app uses. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Christopher T Earley (tendau)
approved these changes
May 6, 2026
This was referenced May 7, 2026
Luffy Chen (XOEEst)
added a commit
that referenced
this pull request
May 13, 2026
) * FAOS optimize preparation skill * docs(microsoft-foundry): clarify FAOS optimize guidance Address PR #2174 review comments by expanding FAOS as Foundry Agent Optimization Service on first use and making the Step 8 Python config snippet copy-paste safe. The minimum contract example now imports os and preserves the app's existing model-selection fallback instead of hard-coding MODEL_DEPLOYMENT_NAME unless that is already what the app uses. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs(microsoft-foundry): clarify custom evaluator output contract When creating custom evaluator prompts, treat the runtime-enforced JSON schema as authoritative: result plus reason. Preserve user-provided rubric text, but remove or normalize conflicting output schemas such as score/reasoning or duplicate OUTPUT FORMAT blocks before calling evaluator_catalog_create. Add observe skill test coverage for the promptText guardrail so future edits keep the result/reason contract visible. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs(microsoft-foundry): keep evaluator example scale-neutral Remove the 0-to-1 scoringType/minScore/maxScore details from the sample custom evaluator prompt. The important guardrail is the output contract: preserve the rubric, but avoid conflicting output schemas because the runtime enforces result/reason. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix * Revert microsoft-foundry SKILL.md to upstream Restore plugin/skills/microsoft-foundry/SKILL.md from upstream/main so this PR no longer carries a line-ending-only change for the top-level skill file. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix faos skill --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Luffy Chen (XOEEst)
added a commit
that referenced
this pull request
May 23, 2026
…2286) * FAOS optimize preparation skill * docs(microsoft-foundry): clarify FAOS optimize guidance Address PR #2174 review comments by expanding FAOS as Foundry Agent Optimization Service on first use and making the Step 8 Python config snippet copy-paste safe. The minimum contract example now imports os and preserves the app's existing model-selection fallback instead of hard-coding MODEL_DEPLOYMENT_NAME unless that is already what the app uses. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs(microsoft-foundry): clarify custom evaluator output contract When creating custom evaluator prompts, treat the runtime-enforced JSON schema as authoritative: result plus reason. Preserve user-provided rubric text, but remove or normalize conflicting output schemas such as score/reasoning or duplicate OUTPUT FORMAT blocks before calling evaluator_catalog_create. Add observe skill test coverage for the promptText guardrail so future edits keep the result/reason contract visible. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs(microsoft-foundry): keep evaluator example scale-neutral Remove the 0-to-1 scoringType/minScore/maxScore details from the sample custom evaluator prompt. The important guardrail is the output contract: preserve the rubric, but avoid conflicting output schemas because the runtime enforces result/reason. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix * Revert microsoft-foundry SKILL.md to upstream Restore plugin/skills/microsoft-foundry/SKILL.md from upstream/main so this PR no longer carries a line-ending-only change for the top-level skill file. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix faos skill * Eval suite support * Fix skills * Add faos update * Fix comments --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Luffy Chen (XOEEst)
added a commit
that referenced
this pull request
May 27, 2026
…veraging azd (#2383) * FAOS optimize preparation skill * docs(microsoft-foundry): clarify FAOS optimize guidance Address PR #2174 review comments by expanding FAOS as Foundry Agent Optimization Service on first use and making the Step 8 Python config snippet copy-paste safe. The minimum contract example now imports os and preserves the app's existing model-selection fallback instead of hard-coding MODEL_DEPLOYMENT_NAME unless that is already what the app uses. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs(microsoft-foundry): clarify custom evaluator output contract When creating custom evaluator prompts, treat the runtime-enforced JSON schema as authoritative: result plus reason. Preserve user-provided rubric text, but remove or normalize conflicting output schemas such as score/reasoning or duplicate OUTPUT FORMAT blocks before calling evaluator_catalog_create. Add observe skill test coverage for the promptText guardrail so future edits keep the result/reason contract visible. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs(microsoft-foundry): keep evaluator example scale-neutral Remove the 0-to-1 scoringType/minScore/maxScore details from the sample custom evaluator prompt. The important guardrail is the output contract: preserve the rubric, but avoid conflicting output schemas because the runtime enforces result/reason. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix * Revert microsoft-foundry SKILL.md to upstream Restore plugin/skills/microsoft-foundry/SKILL.md from upstream/main so this PR no longer carries a line-ending-only change for the top-level skill file. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix faos skill * Eval suite support * Fix skills * Add faos update * Fix comments * latest faos scaffolding * Support faos scaffolding and azd alignments * Fix * Add temp whl path * Remove unnecessary env vars in scaffold * Remove private whl path * Fix comments * Fix * Fix Foundry optimizer eval config Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix comments. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Luffy Chen (XOEEst)
added a commit
that referenced
this pull request
Jun 2, 2026
* FAOS optimize preparation skill * docs(microsoft-foundry): clarify FAOS optimize guidance Address PR #2174 review comments by expanding FAOS as Foundry Agent Optimization Service on first use and making the Step 8 Python config snippet copy-paste safe. The minimum contract example now imports os and preserves the app's existing model-selection fallback instead of hard-coding MODEL_DEPLOYMENT_NAME unless that is already what the app uses. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs(microsoft-foundry): clarify custom evaluator output contract When creating custom evaluator prompts, treat the runtime-enforced JSON schema as authoritative: result plus reason. Preserve user-provided rubric text, but remove or normalize conflicting output schemas such as score/reasoning or duplicate OUTPUT FORMAT blocks before calling evaluator_catalog_create. Add observe skill test coverage for the promptText guardrail so future edits keep the result/reason contract visible. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs(microsoft-foundry): keep evaluator example scale-neutral Remove the 0-to-1 scoringType/minScore/maxScore details from the sample custom evaluator prompt. The important guardrail is the output contract: preserve the rubric, but avoid conflicting output schemas because the runtime enforces result/reason. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix * Revert microsoft-foundry SKILL.md to upstream Restore plugin/skills/microsoft-foundry/SKILL.md from upstream/main so this PR no longer carries a line-ending-only change for the top-level skill file. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix faos skill * Eval suite support * Fix skills * Add faos update * Fix comments * latest faos scaffolding * Support faos scaffolding and azd alignments * Fix * Add temp whl path * Remove unnecessary env vars in scaffold * Remove private whl path * Fix comments * Fix * Fix Foundry optimizer eval config Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix comments. * Resolve Foundry skill merge conflict Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * faos package update and minor fixes * fix * update eval.yaml --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Luffy Chen (XOEEst)
added a commit
that referenced
this pull request
Jun 17, 2026
* FAOS optimize preparation skill * docs(microsoft-foundry): clarify FAOS optimize guidance Address PR #2174 review comments by expanding FAOS as Foundry Agent Optimization Service on first use and making the Step 8 Python config snippet copy-paste safe. The minimum contract example now imports os and preserves the app's existing model-selection fallback instead of hard-coding MODEL_DEPLOYMENT_NAME unless that is already what the app uses. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs(microsoft-foundry): clarify custom evaluator output contract When creating custom evaluator prompts, treat the runtime-enforced JSON schema as authoritative: result plus reason. Preserve user-provided rubric text, but remove or normalize conflicting output schemas such as score/reasoning or duplicate OUTPUT FORMAT blocks before calling evaluator_catalog_create. Add observe skill test coverage for the promptText guardrail so future edits keep the result/reason contract visible. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs(microsoft-foundry): keep evaluator example scale-neutral Remove the 0-to-1 scoringType/minScore/maxScore details from the sample custom evaluator prompt. The important guardrail is the output contract: preserve the rubric, but avoid conflicting output schemas because the runtime enforces result/reason. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix * Revert microsoft-foundry SKILL.md to upstream Restore plugin/skills/microsoft-foundry/SKILL.md from upstream/main so this PR no longer carries a line-ending-only change for the top-level skill file. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix faos skill * Eval suite support * Fix skills * Add faos update * Fix comments * latest faos scaffolding * Support faos scaffolding and azd alignments * Fix * Add temp whl path * Remove unnecessary env vars in scaffold * Remove private whl path * Fix comments * Fix * Fix Foundry optimizer eval config Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix comments. * Resolve Foundry skill merge conflict Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * faos package update and minor fixes * fix * update eval.yaml * Update azd eval commands * Fix comments * Fix comments --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces a new FAOS optimization workflow for Python agents in the Microsoft Foundry skill set. It adds comprehensive documentation and usage patterns for converting existing Python agent code to be optimization-ready, including a step-by-step guide, best practices, and framework-neutral code patterns. The documentation ensures that users can expose agent configuration knobs for optimization, review changes before deployment, and preserve existing runtime behavior.
Key additions and changes:
New FAOS Optimization Workflow Documentation
faos-optimize, to the Foundry agent skills, enabling users to convert Python agent code for FAOS optimization with a required review step before deployment. [1] [2]faos-optimize.md, outlining a 12-step workflow for identifying, wiring, and exposing configuration knobs (like instructions, model, and temperature) for FAOS, with guardrails and review checkpoints to ensure safe and targeted changes.Python Implementation Patterns
python-patterns.md, detailing framework-neutral code snippets and best practices for wiring FAOS optimization knobs in single-agent and multi-agent Python architectures, including guidance for Microsoft Agent Framework, FastAPI, LangGraph, and custom runtimes.User Guidance and Workflow Integration
faos-optimizeworkflow and clarified when to use it in the agent lifecycle, integrating it into the user intent-to-workflow mapping. [1] [2]These changes provide clear, actionable guidance for making Python agents optimization-ready within the Microsoft Foundry ecosystem, ensuring best practices and safe adoption of FAOS optimization features.## Description
Checklist
cd tests && npm test)npm run test:skills:integration -- <skill>)USE FOR/DO NOT USE FOR/PREFER OVERclauses: confirmed no routing regressions for competing skillsRelated Issues