agent/agent-framework
2
0
Fork 0
mirror of https://github.com/microsoft/agent-framework.git synced 2026-06-16 21:04:09 +08:00
Code Issues Packages Projects Releases Wiki Activity
Files
pr-4526-python-skill-validation
agent-framework/dotnet/samples/02-agents/FoundryAgents/FoundryAgents_Evaluations_Step01_RedTeaming
T
History
Roger BarretoGitHubCopilot
 e7961571a8 .NET: Update Azure.AI.Projects 2.0.0-beta.1 (#4270) 
* Update Microsoft.Agents.AI.AzureAI for Azure.AI.Projects SDK 2.0.0

- Bump Azure.AI.Projects to 2.0.0-alpha.20260213.1
- Bump Azure.AI.Projects.OpenAI to 2.0.0-alpha.20260213.1
- Bump System.ClientModel to 1.9.0 (transitive dependency)
- Switch both GetAgent and CreateAgentVersion to protocol methods
  with MEAI user-agent policy injection via RequestOptions
- Migrate 29 CREATE-path tests from FakeAgentClient to HttpHandlerAssert
  pattern for real HTTP pipeline testing
- Fix StructuredOutputDefinition constructor (BinaryData -> IDictionary)
- Fix responses endpoint path (openai/responses -> /responses)
- Add local-packages NuGet source for pre-release nupkgs

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Update Azure.AI.Projects to 2.0.0-beta.1 from NuGet.org

- Update Azure.AI.Projects and Azure.AI.Projects.OpenAI to 2.0.0-beta.1
- Remove local-packages NuGet source (packages now on nuget.org)
- Fix MemorySearchTool -> MemorySearchPreviewTool rename
- Fix RedTeams.CreateAsync ambiguous call
- Fix CreateAgentVersion/Async signature change (BinaryData -> string)
- Suppress AAIP001 experimental warning for WorkflowAgentDefinition

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Move s_modelWriterOptionsWire field before methods that use it

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix flaky test: prevent spurious workflow_invoke Activity on timeout wake-up

The StreamingRunEventStream run loop uses a 1-second timeout on
WaitForInputAsync. When the timeout fires before the consumer calls
StopAsync, the loop would create a spurious workflow_invoke Activity
even though no actual input was provided. This caused the
WorkflowRunActivity_IsStopped_Streaming_OffThread_MultiTurnAsync test
to intermittently fail (expecting 2 activities but finding 3).

Fix: guard the loop body with a HasUnprocessedMessages check. On
timeout wake-ups with no work, the loop waits again without creating
an activity or changing the run status.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix epoch race condition causing unit tests to hang on net10.0 and net472

The HasUnprocessedMessages guard (previous commit) correctly prevents
spurious workflow_invoke Activity creation on timeout wake-ups, but
exposed a latent race in the epoch-based signal filtering.

The race: when the run loop processes messages quickly and calls
Interlocked.Increment(ref _completionEpoch) before the consumer calls
TakeEventStreamAsync, the consumer reads the already-incremented epoch
and sets myEpoch = epoch + 1. This causes the consumer to skip the
valid InternalHaltSignal (its epoch < myEpoch) and block forever
waiting for a signal that will never arrive (since the guard prevents
spurious signal generation).

Fix: read _completionEpoch without +1. The +1 was originally needed to
filter stale signals from timeout-driven spurious loop iterations, but
those no longer exist thanks to the HasUnprocessedMessages guard.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Revert "Fix epoch race condition causing unit tests to hang on net10.0 and net472"

This reverts commit 6ce7f01be8.

* Revert "Fix flaky test: prevent spurious workflow_invoke Activity on timeout wake-up"

This reverts commit 98963e17f2.

* Skip hanging multi-turn declarative integration tests

The ValidateMultiTurnAsync tests (ConfirmInput.yaml, RequestExternalInput.yaml)
hang indefinitely in CI, blocking the merge queue. The hang is SDK-independent
(reproduces with both Azure.AI.Projects 1.2.0-beta.5 and 2.0.0-beta.1) and
is a pre-existing issue in the declarative workflow multi-turn test logic.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Remove unused using directive in IntegrationTest.cs

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Restore Azure.AI.Projects 2.0.0-beta.1 version bump

The merge from main accidentally reverted the package versions back to
1.2.0-beta.5. This is the primary change of this PR.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address merge conflict

* Skip flaky WorkflowRunActivity_IsStopped_Streaming_OffThread_MultiTurnAsync test

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Skip CheckSystem test cases temporarily

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-04 11:36:39 +00:00
..
FoundryAgents_Evaluations_Step01_RedTeaming.csproj
Python / .NET Samples - Restructure and Improve Samples (Feature Branc… (#4092)
2026-02-26 00:56:10 +00:00
Program.cs
.NET: Update Azure.AI.Projects 2.0.0-beta.1 (#4270)
2026-03-04 11:36:39 +00:00
README.md
Python / .NET Samples - Restructure and Improve Samples (Feature Branc… (#4092)
2026-02-26 00:56:10 +00:00

README.md

Red Teaming with Azure AI Foundry (Classic)

Important

This sample uses the classic Azure AI Foundry red teaming API (/redTeams/runs) via Azure.AI.Projects. Results are viewable in the classic Foundry portal experience. The new Foundry portal's red teaming feature uses a different evaluation-based API that is not yet available in the .NET SDK.

This sample demonstrates how to use Azure AI Foundry's Red Teaming service to assess the safety and resilience of an AI model against adversarial attacks.

What this sample demonstrates

  • Configuring a red team run targeting an Azure OpenAI model deployment
  • Using multiple AttackStrategy options (Easy, Moderate, Jailbreak)
  • Evaluating across RiskCategory categories (Violence, HateUnfairness, Sexual, SelfHarm)
  • Submitting a red team scan and polling for completion
  • Reviewing results in the Azure AI Foundry portal

Prerequisites

Before you begin, ensure you have the following prerequisites:

  • .NET 10 SDK or later
  • Azure AI Foundry project (hub and project created)
  • Azure OpenAI deployment (e.g., gpt-4o or gpt-4o-mini)
  • Azure CLI installed and authenticated (for Azure credential authentication)

Regional Requirements

Red teaming is only available in regions that support risk and safety evaluators:

  • East US 2, Sweden Central, US North Central, France Central, Switzerland West

Environment Variables

Set the following environment variables:

$env:AZURE_AI_PROJECT_ENDPOINT="https://your-project.services.ai.azure.com/api/projects/your-project" # Replace with your Azure Foundry project endpoint
$env:AZURE_AI_MODEL_DEPLOYMENT_NAME="gpt-4o-mini"  # Optional, defaults to gpt-4o-mini

Run the sample

Navigate to the sample directory and run:

cd dotnet/samples/02-agents/FoundryAgents/FoundryAgents_Evaluations_Step01_RedTeaming
dotnet run

Expected behavior

The sample will:

  1. Configure a RedTeam run targeting the specified model deployment
  2. Define risk categories and attack strategies
  3. Submit the scan to Azure AI Foundry's Red Teaming service
  4. Poll for completion (this may take several minutes)
  5. Display the run status and direct you to the Azure AI Foundry portal for detailed results

Understanding Red Teaming

Attack Strategies

Strategy Description
Easy Simple encoding/obfuscation attacks (ROT13, Leetspeak, etc.)
Moderate Moderate complexity attacks requiring an LLM for orchestration
Jailbreak Crafted prompts designed to bypass AI safeguards (UPIA)

Risk Categories

Category Description
Violence Content related to violence
HateUnfairness Hate speech or unfair content
Sexual Sexual content
SelfHarm Self-harm related content

Interpreting Results

  • Results are available in the Azure AI Foundry portal (classic view — toggle at top-right) under the red teaming section
  • Lower Attack Success Rate (ASR) is better — target ASR < 5% for production
  • Review individual attack conversations to understand vulnerabilities

Current Limitations

Note

  • The .NET Red Teaming API (Azure.AI.Projects) currently supports targeting model deployments only via AzureOpenAIModelConfiguration. The AzureAIAgentTarget type exists in the SDK but is consumed by the Evaluation Taxonomy API (/evaluationtaxonomies), not by the Red Teaming API (/redTeams/runs).
  • Agent-targeted red teaming with agent-specific risk categories (Prohibited actions, Sensitive data leakage, Task adherence) is documented in the concept docs but is not yet available via the public REST API or .NET SDK.
  • Results from this API appear in the classic Azure AI Foundry portal view. The new Foundry portal uses a separate evaluation-based system with eval_* identifiers.

Related Resources

Next Steps

After running red teaming:

  1. Review attack results and strengthen agent guardrails
  2. Explore the Self-Reflection sample (FoundryAgents_Evaluations_Step02_SelfReflection) for quality assessment
  3. Set up continuous red teaming in your CI/CD pipeline
Reference in New Issue View Git Blame Copy Permalink