Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion plugin/skills/microsoft-foundry/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ description: "Deploy, evaluate, and manage Foundry agents end-to-end: Docker bui
license: MIT
metadata:
author: Microsoft
version: "1.0.7"
version: "1.0.8"
---

# Microsoft Foundry Skill
Expand Down
2 changes: 1 addition & 1 deletion plugin/skills/microsoft-foundry/quota/quota.md
Original file line number Diff line number Diff line change
Expand Up @@ -145,7 +145,7 @@ Quota freed **immediately**. Re-run Workflow #1 to verify.
```
Production [workload type] using [model] in [region].
Expected traffic: [X requests/day] with [Y tokens/request].
Requires [Z TPM] capacity. Current [N TPM] insufficient.
Calculated required TPM: [Z TPM]. Current [N TPM] insufficient.
Request increase to [M TPM]. Deployment target: [date].
```

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ Scenario: 1M requests/day, average 1,000 tokens per request

## Production Workload Examples

Real-world production scenarios with capacity calculations for gpt-4, version 0613 (from Azure Foundry Portal calculator):
To estimate quota requirements, use real-world production scenarios with capacity calculations for gpt-4, version 0613 (from Azure Foundry Portal calculator):

| Workload Type | Calls/Min | Prompt Tokens | Response Tokens | Cache Hit % | Total Tokens/Min | PTU Required | TPM Equivalent |
|---------------|-----------|---------------|-----------------|-------------|------------------|--------------|----------------|
Expand All @@ -65,7 +65,9 @@ Real-world production scenarios with capacity calculations for gpt-4, version 06
| **Summarization** | 10 | 5,000 | 300 | 20% | 53,000 | 100 | 53K TPM |
| **Classification** | 10 | 3,800 | 10 | 20% | 38,100 | 100 | 38K TPM |

**How to Calculate Your Needs:**
**How to Estimate Your Production Quota Requirements:**

To calculate your quota needs for production deployments, follow these steps:

1. **Determine your peak calls per minute**: Monitor or estimate maximum concurrent requests
2. **Measure token usage**: Average prompt size + response size
Expand Down Expand Up @@ -117,7 +119,9 @@ For the combined workload (40 calls/min, 135K tokens/min total), use **200 PTU**

**Capacity Planning Approach** (from [PTU onboarding guide](https://learn.microsoft.com/en-us/azure/ai-foundry/openai/how-to/provisioned-throughput-onboarding)):

1. **Understand your TPM requirements**: Calculate expected tokens per minute based on workload
To calculate and estimate your capacity requirements:

1. **Calculate your TPM requirements**: Determine required tokens per minute based on your expected workload
2. **Use the built-in capacity planner**: Available in Azure AI Foundry portal (Microsoft Foundry → Operate → Quota → Provisioned Throughput Unit tab)
3. **Input your metrics**: Enter input TPM and output TPM based on your workload characteristics
4. **Get PTU recommendation**: The calculator provides PTU allocation recommendation
Expand Down
53 changes: 48 additions & 5 deletions tests/microsoft-foundry/quota/integration.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,8 @@ describeIntegration(`${SKILL_NAME}_quota - Integration Tests`, () => {
prompt: "How much quota do I need for a production Foundry deployment?"
});

const hasCapacityGuidance = doesAssistantMessageIncludeKeyword(
// Require at least one quota-specific term
const hasQuotaTerm = doesAssistantMessageIncludeKeyword(
agentMetadata,
"TPM"
) || doesAssistantMessageIncludeKeyword(
Expand All @@ -101,8 +102,27 @@ describeIntegration(`${SKILL_NAME}_quota - Integration Tests`, () => {
) || doesAssistantMessageIncludeKeyword(
agentMetadata,
"capacity"
) || doesAssistantMessageIncludeKeyword(
agentMetadata,
"tokens per minute"
);

// Require at least one calculation verb
const hasCalculationVerb = doesAssistantMessageIncludeKeyword(
agentMetadata,
"calculate"
) || doesAssistantMessageIncludeKeyword(
agentMetadata,
"estimate"
) || doesAssistantMessageIncludeKeyword(
agentMetadata,
"calculation"
) || doesAssistantMessageIncludeKeyword(
agentMetadata,
"quantify"
);
expect(hasCapacityGuidance).toBe(true);

expect(hasQuotaTerm && hasCalculationVerb).toBe(true);
}));
});

Expand Down Expand Up @@ -253,14 +273,37 @@ describeIntegration(`${SKILL_NAME}_quota - Integration Tests`, () => {
const isSkillUsed = isSkillInvoked(agentMetadata, SKILL_NAME);
expect(isSkillUsed).toBe(true);

const hasPlanning = doesAssistantMessageIncludeKeyword(
// Require at least one quota-specific term
const hasQuotaTerm = doesAssistantMessageIncludeKeyword(
agentMetadata,
"TPM"
) || doesAssistantMessageIncludeKeyword(
agentMetadata,
"PTU"
) || doesAssistantMessageIncludeKeyword(
agentMetadata,
"capacity"
) || doesAssistantMessageIncludeKeyword(
agentMetadata,
"tokens per minute"
);

// Require at least one calculation verb
const hasCalculationVerb = doesAssistantMessageIncludeKeyword(
agentMetadata,
"calculate"
) || doesAssistantMessageIncludeKeyword(
agentMetadata,
"TPM"
"estimate"
) || doesAssistantMessageIncludeKeyword(
agentMetadata,
"calculation"
) || doesAssistantMessageIncludeKeyword(
agentMetadata,
"quantify"
);
expect(hasPlanning).toBe(true);

expect(hasQuotaTerm && hasCalculationVerb).toBe(true);
}));

test("provides best practices", () => withTestResult(async () => {
Expand Down
Loading