mirror of
https://github.com/microsoft/agent-framework.git
synced 2026-06-16 21:04:09 +08:00
4b3df9ad89
* Initial plan * Add Foundry evaluation samples for Red Teaming and Self-Reflection Co-authored-by: rogerbarreto <19890735+rogerbarreto@users.noreply.github.com> * Refactor evaluation samples with real implementations in local functions Co-authored-by: rogerbarreto <19890735+rogerbarreto@users.noreply.github.com> * Uncomment function signatures and bodies, keep only invocations commented Co-authored-by: rogerbarreto <19890735+rogerbarreto@users.noreply.github.com> * Update Foundry evaluation samples with observability support * Restructure evaluation samples to follow FoundryAgents naming convention - Rename Evaluation/Evaluation_StepXX to FoundryAgents_Evaluations_StepXX - Add evaluation projects to slnx - Fix var usage, apply dotnet format, use DefaultAzureCredential - Add try/finally for agent cleanup - Fix evaluator deployment name separation in Step02 - Update README references Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Rewrite Step01 to use Azure.AI.Projects RedTeam API and address review comments - Replace safety evaluator sample with actual Red Teaming using AIProjectClient.RedTeams - Use AttackStrategy (Easy, Moderate, Jailbreak) and RiskCategory from Azure.AI.Projects - Remove Microsoft.Extensions.AI.Evaluation.Safety dependency from Step01 - Add DefaultAzureCredential warning comments to Step02 - Remove unused bestResponse variable in Step02 - Add session isolation comments in self-reflection loop - Fix stale directory references in READMEs - Fix misleading evaluation overview link in main README Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add note about agent-targeted red teaming limitations in README The .NET RedTeam API currently only supports model deployment targets via AzureOpenAIModelConfiguration. Agent-targeted red teaming with AzureAIAgentTarget is documented in concept docs but not yet available in the SDK's RedTeam constructor. Results appear in classic portal view. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add classic Foundry disclaimer to red teaming sample README Clarify that this sample uses the classic Azure AI Foundry red teaming API (/redTeams/runs). The new Foundry portal uses a separate evaluation- based API not yet available in the .NET SDK. AzureAIAgentTarget exists in the SDK but is consumed by the Evaluation Taxonomy API, not RedTeam. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Address PR review comments on Step02 SelfReflection - Pass full prompt (with context) to evaluator messages instead of just the question, so evaluator input matches what the agent received - Include previous response text in self-reflection refinement prompt so the LLM can meaningfully improve its answer across iterations - Inline CreateKnowledgeAgent helper (single use, single statement) - Add comment clarifying why RunCombinedQualityAndSafetyEvaluation intentionally passes only the question (no context) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: rogerbarreto <19890735+rogerbarreto@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Red Teaming with Azure AI Foundry (Classic)
Important
This sample uses the classic Azure AI Foundry red teaming API (
/redTeams/runs) viaAzure.AI.Projects. Results are viewable in the classic Foundry portal experience. The new Foundry portal's red teaming feature uses a different evaluation-based API that is not yet available in the .NET SDK.
This sample demonstrates how to use Azure AI Foundry's Red Teaming service to assess the safety and resilience of an AI model against adversarial attacks.
What this sample demonstrates
- Configuring a red team run targeting an Azure OpenAI model deployment
- Using multiple
AttackStrategyoptions (Easy, Moderate, Jailbreak) - Evaluating across
RiskCategorycategories (Violence, HateUnfairness, Sexual, SelfHarm) - Submitting a red team scan and polling for completion
- Reviewing results in the Azure AI Foundry portal
Prerequisites
Before you begin, ensure you have the following prerequisites:
- .NET 10 SDK or later
- Azure AI Foundry project (hub and project created)
- Azure OpenAI deployment (e.g., gpt-4o or gpt-4o-mini)
- Azure CLI installed and authenticated (for Azure credential authentication)
Regional Requirements
Red teaming is only available in regions that support risk and safety evaluators:
- East US 2, Sweden Central, US North Central, France Central, Switzerland West
Environment Variables
Set the following environment variables:
$env:AZURE_FOUNDRY_PROJECT_ENDPOINT="https://your-project.services.ai.azure.com/api/projects/your-project" # Replace with your Azure Foundry project endpoint
$env:AZURE_FOUNDRY_PROJECT_DEPLOYMENT_NAME="gpt-4o-mini" # Optional, defaults to gpt-4o-mini
Run the sample
Navigate to the sample directory and run:
cd dotnet/samples/GettingStarted/FoundryAgents/FoundryAgents_Evaluations_Step01_RedTeaming
dotnet run
Expected behavior
The sample will:
- Configure a
RedTeamrun targeting the specified model deployment - Define risk categories and attack strategies
- Submit the scan to Azure AI Foundry's Red Teaming service
- Poll for completion (this may take several minutes)
- Display the run status and direct you to the Azure AI Foundry portal for detailed results
Understanding Red Teaming
Attack Strategies
| Strategy | Description |
|---|---|
| Easy | Simple encoding/obfuscation attacks (ROT13, Leetspeak, etc.) |
| Moderate | Moderate complexity attacks requiring an LLM for orchestration |
| Jailbreak | Crafted prompts designed to bypass AI safeguards (UPIA) |
Risk Categories
| Category | Description |
|---|---|
| Violence | Content related to violence |
| HateUnfairness | Hate speech or unfair content |
| Sexual | Sexual content |
| SelfHarm | Self-harm related content |
Interpreting Results
- Results are available in the Azure AI Foundry portal (classic view — toggle at top-right) under the red teaming section
- Lower Attack Success Rate (ASR) is better — target ASR < 5% for production
- Review individual attack conversations to understand vulnerabilities
Current Limitations
Note
- The .NET Red Teaming API (
Azure.AI.Projects) currently supports targeting model deployments only viaAzureOpenAIModelConfiguration. TheAzureAIAgentTargettype exists in the SDK but is consumed by the Evaluation Taxonomy API (/evaluationtaxonomies), not by the Red Teaming API (/redTeams/runs).- Agent-targeted red teaming with agent-specific risk categories (Prohibited actions, Sensitive data leakage, Task adherence) is documented in the concept docs but is not yet available via the public REST API or .NET SDK.
- Results from this API appear in the classic Azure AI Foundry portal view. The new Foundry portal uses a separate evaluation-based system with
eval_*identifiers.
Related Resources
Next Steps
After running red teaming:
- Review attack results and strengthen agent guardrails
- Explore the Self-Reflection sample (FoundryAgents_Evaluations_Step02_SelfReflection) for quality assessment
- Set up continuous red teaming in your CI/CD pipeline