Compare commits

...
Author SHA1 Message Date
Tao Chen 1347a00ca8 Hard code model for sheel tool samples 2026-04-01 16:59:06 -07:00
Tao Chen 0ed522cbee Fix env vars for SK 2026-04-01 16:49:32 -07:00
Tao Chen c39cbfe677 Fix SK migration samples 2026-04-01 16:45:06 -07:00
westeyandGitHub 15e435b472 Fix telemetry sample bug (#5037) 2026-04-01 19:30:45 +00:00
Eduard van ValkenburgandGitHub 519bb0cb2b Python: updated declarative samples and handling of non-pydantic response formats (#5022)
* updated declarative samples and handling of non-pydantic response formats

* fixed from comments

* update docstring
2026-04-01 19:16:00 +00:00
6acab3d1d6 Python: [BREAKING] Standardize model selection on model (#4999)
* Refactor Anthropic model option and provider clients

Rename the Anthropic client model option from model_id to model, add provider-specific Anthropic wrappers for Foundry, Bedrock, and Vertex, and expose them through the Anthropic, Foundry, Amazon, and Google namespaces. Update core option handling, docs, samples, and tests accordingly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix Anthropic skills sample typing

Cast the Anthropic beta client to Any in the skills sample so the pre-commit sample pyright check no longer fails on beta skills and files endpoints that are not exposed by the current SDK stubs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* undo sample mypy

* Retry CI after transient external failures

Retrigger PR validation after an unrelated Copilot review workflow SAML failure and a transient external tau2 git fetch failure in the Windows Python test setup.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address review feedback on model option merging

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address Anthropic compatibility review feedback

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* moved all to `model`

* fixes for azure ai search

* Python: standardize remaining sample env var names

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: fix foundry-local pyright compatibility

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* updated env vars in cicd

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-01 19:00:18 +00:00
Tao ChenGitHubTaoChenOSUcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
95550dd0dc Python: Enforce Foundry package unit test coverage (#5036)
* Enforce Foundry package unit test coverage

* Sort ENFORCED_TARGETS alphabetically in python-check-coverage.py

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/ed0b81ed-c267-4ee0-9655-56c4b3066fad

Co-authored-by: TaoChenOSU <12570346+TaoChenOSU@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: TaoChenOSU <12570346+TaoChenOSU@users.noreply.github.com>
2026-04-01 17:37:27 +00:00
86f8efc8ff Python: Fix observability samples (#5016)
* Fix observability samples

* Update python/samples/02-agents/observability/configure_otel_providers_with_parameters.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update python/samples/02-agents/observability/agent_observability.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update python/samples/02-agents/observability/agent_observability.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-01 17:12:03 +00:00
b065a4ce51 Python: [BREAKING] update context provider APIs, middleware, and per-service-call history persistence (#4992)
* Rename provider base APIs

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Allow provider-added chat and function middleware

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Simulate service-stored history per model call

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix typing regressions in CI

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix response ID suppression review feedback

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Rename per-service-call history persistence APIs

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address context persistence review feedback

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Stabilize markdown sample docs

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Persist service continuation state per call

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-01 16:13:11 +00:00
Peter IbekweandGitHub 38de991481 .NET: Fix RequestInfoEvent lost when resuming workflow from checkpoint (#4955)
* Fix RequestInfoEvent lost when resuming workflow from checkpoint

* Fix streaming run double disposal in tests and lockstep republishing before Started event is emitted.

* Fix bug to remove messages after sending to avoid losing messages on send failure.

* Fix declarative test harness
2026-04-01 15:38:48 +00:00
25696a72dc .NET: Replace Azure Foundry/Azure AI Foundry with Microsoft Foundry in .NET samples (#5032)
* Replace Azure Foundry/Azure AI Foundry with Microsoft Foundry in samples

Update all .cs, .md, and .yaml files in dotnet/samples/ to use
'Microsoft Foundry' instead of 'Azure Foundry' and 'Azure AI Foundry'.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Update dotnet/samples/02-agents/AgentWithMemory/AgentWithMemory_Step04_MemoryUsingFoundry/Program.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update dotnet/samples/02-agents/Agents/Agent_Step15_DeepResearch/Program.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update dotnet/samples/05-end-to-end/A2AClientServer/README.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update dotnet/samples/02-agents/Agents/Agent_Step15_DeepResearch/README.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update dotnet/samples/03-workflows/Agents/FoundryAgent/Program.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update dotnet/samples/02-agents/AgentProviders/Agent_With_AzureAIProject/Program.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Fix grammar: 'an Microsoft' -> 'a Microsoft', 'agents ids' -> 'agent IDs'

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-01 15:18:33 +00:00
Eduard van ValkenburgandGitHub 2cb78ea12e fix and unify devui samples (#5025) 2026-04-01 13:47:20 +00:00
Eduard van ValkenburgandGitHub cee0a458fe Python: fixed middleware samples (#5026)
* fixed samples

* small update to explanation

* add snippet fix on root readme
2026-04-01 13:40:27 +00:00
Eduard van ValkenburgandGitHub 4b9856e66f Python: updated azure ai inference sample (#5028)
* updated azure ai inference sample

* openai multimodel fix

* update language
2026-04-01 13:35:56 +00:00
westeyandGitHub acaadc9c45 .NET: Add a verify-samples tool and skill (#5005)
* Add a verify-samples tool and skill

* Address PR comments

* Move verify-samples to eng folder and improve definitions
2026-04-01 10:34:41 +00:00
Christian GlessnerandGitHub 34329840e1 Add Neo4j GraphRAG samples (#4994)
* Add Neo4j GraphRAG samples

* Fix sample CI issues

* Address sample review feedback

* Move Neo4j Python sample to end-to-end

* Make Neo4j GraphRAG sample self-contained

* Remove unused central package versions
2026-04-01 10:23:04 +00:00
Eduard van ValkenburgandGitHub 2a8c3e2dcf fixes to azure ai search init, samples (#5021) 2026-04-01 09:59:52 +00:00
Tao ChenandGitHub e43fc8ccec Python: Fix migration samples (#5015)
* Fix migration samples

* Fix migration samples 2

* Fix formatting

* Comments
2026-04-01 06:32:30 +00:00
d992febe9b Python: Fix agent_with_hosted_mcp sample to use Foundry client for MCP tools (#4867)
* Fix agent_with_hosted_mcp sample to use AzureOpenAIResponsesClient (#4861)

The agent_with_hosted_mcp sample used AzureOpenAIChatClient with an MCP tool
dict, but the Chat Completions API only supports 'function' and 'custom' tool
types, not 'mcp'. This caused a 400 error at runtime.

Switch the sample to AzureOpenAIResponsesClient which natively supports MCP
tools via the Responses API. Use get_mcp_tool() to construct the tool config.

Changes:
- main.py: Replace AzureOpenAIChatClient with AzureOpenAIResponsesClient
- requirements.txt: Update azure-ai-agentserver-agentframework to 1.0.0b16
  and use agent-framework-azure-ai package
- agent.yaml: Use AZURE_OPENAI_RESPONSES_DEPLOYMENT_NAME env var
- Add regression test documenting chat client MCP tool passthrough behavior

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Fix agent_with_hosted_mcp sample to use Responses API client for MCP tools

Fixes #4861

* Remove REPRODUCTION_REPORT.md investigation artifact (#4861)

Remove the reproduction report markdown file from the test directory.
Investigation notes belong in the GitHub issue or PR description,
not as committed files in the source tree. The regression test in
test_openai_chat_client.py already provides automated verification.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add MCP tool API rejection regression test (#4861)

Add test_mcp_tool_dict_causes_api_rejection to verify that MCP tool
dicts passed through to the Chat Completions API result in a clear
ChatClientException rather than being silently dropped. This completes
the regression test coverage requested in code review.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* small fix

* Revert deletion of dotnet local.settings.json files

Restore the two local.settings.json files that were accidentally deleted in this PR.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-31 22:04:54 +00:00
651e317907 Python: Fix _add_text_reasoning_content to preserve id during coalescing (#4862)
* Fix _add_text_reasoning_content dropping id during coalescing (#4852)

Preserve the id field (rs_* identifier) when coalescing text_reasoning
Content objects by passing id=self.id or other.id to the Content
constructor. This fixes the encrypted reasoning round-trip where the
missing id prevented _prepare_content_for_openai from including it in
the serialized reasoning item.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Fix `_add_text_reasoning_content` to preserve `id` during coalescing

Fixes #4852

* Raise AdditionItemMismatch on conflicting text_reasoning ids (#4852)

Detect when both operands have different non-empty ids during
text_reasoning Content coalescing and raise AdditionItemMismatch
instead of silently keeping one. This prevents mis-associating
encrypted_content during round-trips.

Also adds tests for conflicting ids and the neither-has-id edge case.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address review feedback for #4852: Python: [Bug]: Content._add_text_reasoning_content drops id during coalescing, breaking encrypted reasoning round-trip

* test fix

---------

Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-31 21:58:02 +00:00
1e527a328c Python: Remove unsupported memory scoping params from mem0/redis samples and docs (#4367)
* Python: Remove unsupported memory scoping params from samples and docs

Fixes #4353

The `Mem0ContextProvider` and `RedisContextProvider` no longer support
`thread_id` or `scope_to_per_operation_thread_id` parameters. This commit
updates the affected samples and READMEs to use only the currently
supported API (`user_id`, `agent_id`, `application_id`).

Changes:
- mem0_sessions.py: Remove `thread_id` and
  `scope_to_per_operation_thread_id` from examples 1 and 2, rewrite to
  demonstrate user-scoped and agent-scoped memory patterns
- redis_sessions.py: Update module docstring to remove references to
  removed thread scoping params
- mem0/README.md: Update Memory Scoping docs to reflect current API
- redis/README.md: Remove `thread_id` and
  `scope_to_per_operation_thread_id` references from docs

* Address Copilot review: rename thread_scope functions, fix docstring

- Rename `example_global_thread_scope` -> `example_global_memory_scope`
- Rename `example_per_operation_thread_scope` -> `example_agent_scoped_memory`
- Update example 2 docstring to mention `application_id` alongside
  `user_id` and `agent_id` since it's set in the provider config
- Update module docstring scenario 2 to include `application_id`

* fix: rebase onto main, address giles17 review feedback

- Resolve merge conflicts by rebasing all 4 original files onto current main
- Address giles17's agent review suggestions:
  - mem0_basic.py: update comment to remove thread_id from scoping list
  - mem0_oss.py: update comment to remove thread_id from scoping list
  - redis_sessions.py: rename Example 2 from "Agent-Scoped Memory" to
    "Hybrid Vector Search" to accurately describe what it demonstrates
  - redis/README.md: update Example 2 description to match renamed example

---------

Co-authored-by: Tao Chen <taochen@microsoft.com>
Co-authored-by: Giles Odigwe <79032838+giles17@users.noreply.github.com>
2026-03-31 21:57:23 +00:00
352 changed files with 9404 additions and 2160 deletions
+5 -4
View File
@@ -34,14 +34,15 @@ from dataclasses import dataclass
# (e.g., "packages/core/agent_framework/observability.py")
# =============================================================================
ENFORCED_TARGETS: set[str] = {
# Packages
# Packages (sorted alphabetically)
"packages.anthropic.agent_framework_anthropic",
"packages.azure-ai-search.agent_framework_azure_ai_search",
"packages.azure-ai.agent_framework_azure_ai",
"packages.core.agent_framework",
"packages.core.agent_framework._workflows",
"packages.purview.agent_framework_purview",
"packages.anthropic.agent_framework_anthropic",
"packages.azure-ai-search.agent_framework_azure_ai_search",
"packages.foundry.agent_framework_foundry",
"packages.openai.agent_framework_openai",
"packages.purview.agent_framework_purview",
# Individual files (if you want to enforce specific files instead of whole packages)
"packages/core/agent_framework/observability.py",
# Add more targets here as coverage improves
@@ -95,10 +95,10 @@ jobs:
environment: integration
timeout-minutes: 60
env:
AZURE_OPENAI_CHAT_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__CHATDEPLOYMENTNAME }}
AZURE_OPENAI_RESPONSES_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
AZURE_OPENAI_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME: ${{ vars.AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME }}
AZURE_OPENAI_CHAT_MODEL: ${{ vars.AZUREOPENAI__CHATDEPLOYMENTNAME }}
AZURE_OPENAI_RESPONSES_MODEL: ${{ vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
AZURE_OPENAI_MODEL: ${{ vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
AZURE_OPENAI_EMBEDDING_MODEL: ${{ vars.AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME }}
AZURE_OPENAI_ENDPOINT: ${{ vars.AZUREOPENAI__ENDPOINT }}
defaults:
run:
@@ -139,7 +139,7 @@ jobs:
timeout-minutes: 60
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
ANTHROPIC_CHAT_MODEL_ID: ${{ vars.ANTHROPIC_CHAT_MODEL_ID }}
ANTHROPIC_CHAT_MODEL: ${{ vars.ANTHROPIC_CHAT_MODEL_ID }}
LOCAL_MCP_URL: ${{ vars.LOCAL_MCP__URL }}
defaults:
run:
@@ -207,8 +207,8 @@ jobs:
OPENAI_EMBEDDING_MODEL: ${{ vars.OPENAI_EMBEDDING_MODEL_ID }}
OPENAI_API_KEY: ${{ secrets.OPENAI__APIKEY }}
AZURE_OPENAI_ENDPOINT: ${{ vars.AZUREOPENAI__ENDPOINT }}
AZURE_OPENAI_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
AZURE_OPENAI_CHAT_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__CHATDEPLOYMENTNAME }}
AZURE_OPENAI_MODEL: ${{ vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
AZURE_OPENAI_CHAT_MODEL: ${{ vars.AZUREOPENAI__CHATDEPLOYMENTNAME }}
FOUNDRY_PROJECT_ENDPOINT: ${{ vars.FOUNDRY_PROJECT_ENDPOINT }}
FOUNDRY_MODEL: ${{ vars.FOUNDRY_MODEL }}
FUNCTIONS_WORKER_RUNTIME: "python"
@@ -256,7 +256,7 @@ jobs:
timeout-minutes: 60
env:
AZURE_AI_PROJECT_ENDPOINT: ${{ secrets.AZUREAI__ENDPOINT }}
AZURE_AI_MODEL_DEPLOYMENT_NAME: ${{ vars.AZUREAI__DEPLOYMENTNAME }}
AZURE_AI_MODEL: ${{ vars.AZUREAI__DEPLOYMENTNAME }}
FOUNDRY_PROJECT_ENDPOINT: ${{ vars.FOUNDRY_PROJECT_ENDPOINT }}
FOUNDRY_MODEL: ${{ vars.FOUNDRY_MODEL }}
FOUNDRY_AGENT_NAME: ${{ vars.FOUNDRY_AGENT_NAME }}
+8 -8
View File
@@ -192,10 +192,10 @@ jobs:
runs-on: ubuntu-latest
environment: integration
env:
AZURE_OPENAI_CHAT_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__CHATDEPLOYMENTNAME }}
AZURE_OPENAI_RESPONSES_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
AZURE_OPENAI_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME: ${{ vars.AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME }}
AZURE_OPENAI_CHAT_MODEL: ${{ vars.AZUREOPENAI__CHATDEPLOYMENTNAME }}
AZURE_OPENAI_RESPONSES_MODEL: ${{ vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
AZURE_OPENAI_MODEL: ${{ vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
AZURE_OPENAI_EMBEDDING_MODEL: ${{ vars.AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME }}
AZURE_OPENAI_ENDPOINT: ${{ vars.AZUREOPENAI__ENDPOINT }}
defaults:
run:
@@ -255,7 +255,7 @@ jobs:
environment: integration
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
ANTHROPIC_CHAT_MODEL_ID: ${{ vars.ANTHROPIC_CHAT_MODEL_ID }}
ANTHROPIC_CHAT_MODEL: ${{ vars.ANTHROPIC_CHAT_MODEL_ID }}
LOCAL_MCP_URL: ${{ vars.LOCAL_MCP__URL }}
defaults:
run:
@@ -336,8 +336,8 @@ jobs:
OPENAI_EMBEDDING_MODEL: ${{ vars.OPENAI_EMBEDDING_MODEL_ID }}
OPENAI_API_KEY: ${{ secrets.OPENAI__APIKEY }}
AZURE_OPENAI_ENDPOINT: ${{ vars.AZUREOPENAI__ENDPOINT }}
AZURE_OPENAI_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
AZURE_OPENAI_CHAT_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__CHATDEPLOYMENTNAME }}
AZURE_OPENAI_MODEL: ${{ vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
AZURE_OPENAI_CHAT_MODEL: ${{ vars.AZUREOPENAI__CHATDEPLOYMENTNAME }}
FOUNDRY_PROJECT_ENDPOINT: ${{ vars.FOUNDRY_PROJECT_ENDPOINT }}
FOUNDRY_MODEL: ${{ vars.FOUNDRY_MODEL }}
FUNCTIONS_WORKER_RUNTIME: "python"
@@ -398,7 +398,7 @@ jobs:
environment: integration
env:
AZURE_AI_PROJECT_ENDPOINT: ${{ secrets.AZUREAI__ENDPOINT }}
AZURE_AI_MODEL_DEPLOYMENT_NAME: ${{ vars.AZUREAI__DEPLOYMENTNAME }}
AZURE_AI_MODEL: ${{ vars.AZUREAI__DEPLOYMENTNAME }}
FOUNDRY_PROJECT_ENDPOINT: ${{ vars.FOUNDRY_PROJECT_ENDPOINT }}
FOUNDRY_MODEL: ${{ vars.FOUNDRY_MODEL }}
FOUNDRY_AGENT_NAME: ${{ vars.FOUNDRY_AGENT_NAME }}
+39 -32
View File
@@ -65,12 +65,13 @@ jobs:
FOUNDRY_MODEL: ${{ vars.FOUNDRY_MODEL || vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
# Azure OpenAI configuration
AZURE_OPENAI_ENDPOINT: ${{ vars.AZUREOPENAI__ENDPOINT }}
AZURE_OPENAI_DEPLOYMENT_NAME: ${{ vars.AZURE_OPENAI_DEPLOYMENT_NAME || vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME: ${{ vars.AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME || vars.AZUREOPENAI__EMBEDDINGDEPLOYMENTNAME }}
AZURE_OPENAI_MODEL: ${{ vars.AZURE_OPENAI_DEPLOYMENT_NAME || vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
AZURE_OPENAI_RESPONSES_MODEL: ${{ vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
AZURE_OPENAI_EMBEDDING_MODEL: ${{ vars.AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME || vars.AZUREOPENAI__EMBEDDINGDEPLOYMENTNAME }}
# OpenAI configuration
OPENAI_API_KEY: ${{ secrets.OPENAI__APIKEY }}
OPENAI_CHAT_MODEL_ID: ${{ vars.OPENAI__CHATMODELID }}
OPENAI_RESPONSES_MODEL_ID: ${{ vars.OPENAI__RESPONSESMODELID }}
OPENAI_CHAT_MODEL: ${{ vars.OPENAI__CHATMODELID }}
OPENAI_RESPONSES_MODEL: ${{ vars.OPENAI__RESPONSESMODELID }}
# GitHub MCP
GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
OPENAI_MODEL: ${{ vars.OPENAI__RESPONSESMODELID }}
@@ -95,11 +96,12 @@ jobs:
echo "FOUNDRY_PROJECT_ENDPOINT=$FOUNDRY_PROJECT_ENDPOINT" >> .env
echo "FOUNDRY_MODEL=$FOUNDRY_MODEL" >> .env
echo "AZURE_OPENAI_ENDPOINT=$AZURE_OPENAI_ENDPOINT" >> .env
echo "AZURE_OPENAI_DEPLOYMENT_NAME=$AZURE_OPENAI_DEPLOYMENT_NAME" >> .env
echo "AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME=$AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME" >> .env
echo "AZURE_OPENAI_MODEL=$AZURE_OPENAI_MODEL" >> .env
echo "AZURE_OPENAI_RESPONSES_MODEL=$AZURE_OPENAI_RESPONSES_MODEL" >> .env
echo "AZURE_OPENAI_EMBEDDING_MODEL=$AZURE_OPENAI_EMBEDDING_MODEL" >> .env
echo "OPENAI_API_KEY=$OPENAI_API_KEY" >> .env
echo "OPENAI_CHAT_MODEL_ID=$OPENAI_CHAT_MODEL_ID" >> .env
echo "OPENAI_RESPONSES_MODEL_ID=$OPENAI_RESPONSES_MODEL_ID" >> .env
echo "OPENAI_CHAT_MODEL=$OPENAI_CHAT_MODEL" >> .env
echo "OPENAI_RESPONSES_MODEL=$OPENAI_RESPONSES_MODEL" >> .env
echo "GITHUB_PAT=$GITHUB_PAT" >> .env
- name: Run sample validation
@@ -120,8 +122,8 @@ jobs:
env:
OPENAI_API_KEY: ${{ secrets.OPENAI__APIKEY }}
OPENAI_MODEL: ${{ vars.OPENAI__CHATMODELID }}
OPENAI_CHAT_MODEL_ID: ${{ vars.OPENAI__CHATMODELID }}
OPENAI_RESPONSES_MODEL_ID: ${{ vars.OPENAI__RESPONSESMODELID }}
OPENAI_CHAT_MODEL: ${{ vars.OPENAI__CHATMODELID }}
OPENAI_RESPONSES_MODEL: ${{ vars.OPENAI__RESPONSESMODELID }}
defaults:
run:
working-directory: python
@@ -140,8 +142,8 @@ jobs:
run: |
echo "OPENAI_API_KEY=$OPENAI_API_KEY" >> .env
echo "OPENAI_MODEL=$OPENAI_MODEL" >> .env
echo "OPENAI_CHAT_MODEL_ID=$OPENAI_CHAT_MODEL_ID" >> .env
echo "OPENAI_RESPONSES_MODEL_ID=$OPENAI_RESPONSES_MODEL_ID" >> .env
echo "OPENAI_CHAT_MODEL=$OPENAI_CHAT_MODEL" >> .env
echo "OPENAI_RESPONSES_MODEL=$OPENAI_RESPONSES_MODEL" >> .env
- name: Run sample validation
run: |
@@ -160,7 +162,7 @@ jobs:
environment: integration
env:
AZURE_OPENAI_ENDPOINT: ${{ vars.AZUREOPENAI__ENDPOINT }}
AZURE_OPENAI_DEPLOYMENT_NAME: ${{ vars.AZURE_OPENAI_DEPLOYMENT_NAME || vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
AZURE_OPENAI_MODEL: ${{ vars.AZURE_OPENAI_DEPLOYMENT_NAME || vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
AZURE_OPENAI_API_VERSION: ${{ vars.AZURE_OPENAI_API_VERSION || '' }}
defaults:
run:
@@ -179,7 +181,7 @@ jobs:
- name: Create .env for samples
run: |
echo "AZURE_OPENAI_ENDPOINT=$AZURE_OPENAI_ENDPOINT" >> .env
echo "AZURE_OPENAI_DEPLOYMENT_NAME=$AZURE_OPENAI_DEPLOYMENT_NAME" >> .env
echo "AZURE_OPENAI_MODEL=$AZURE_OPENAI_MODEL" >> .env
echo "AZURE_OPENAI_API_VERSION=$AZURE_OPENAI_API_VERSION" >> .env
- name: Run sample validation
@@ -199,7 +201,7 @@ jobs:
environment: integration
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
ANTHROPIC_CHAT_MODEL_ID: ${{ vars.ANTHROPIC_CHAT_MODEL_ID }}
ANTHROPIC_CHAT_MODEL: ${{ vars.ANTHROPIC_CHAT_MODEL_ID }}
defaults:
run:
working-directory: python
@@ -217,7 +219,7 @@ jobs:
- name: Create .env for samples
run: |
echo "ANTHROPIC_API_KEY=$ANTHROPIC_API_KEY" >> .env
echo "ANTHROPIC_CHAT_MODEL_ID=$ANTHROPIC_CHAT_MODEL_ID" >> .env
echo "ANTHROPIC_CHAT_MODEL=$ANTHROPIC_CHAT_MODEL" >> .env
- name: Run sample validation
run: |
@@ -265,7 +267,7 @@ jobs:
runs-on: ubuntu-latest
environment: integration
env:
BEDROCK_CHAT_MODEL_ID: ${{ vars.BEDROCK__CHATMODELID }}
BEDROCK_CHAT_MODEL: ${{ vars.BEDROCK__CHATMODELID }}
defaults:
run:
working-directory: python
@@ -518,7 +520,7 @@ jobs:
FOUNDRY_MODEL: ${{ vars.FOUNDRY_MODEL || vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
# Azure OpenAI configuration
AZURE_OPENAI_ENDPOINT: ${{ vars.AZUREOPENAI__ENDPOINT }}
AZURE_OPENAI_DEPLOYMENT_NAME: ${{ vars.AZURE_OPENAI_DEPLOYMENT_NAME || vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
AZURE_OPENAI_MODEL: ${{ vars.AZURE_OPENAI_DEPLOYMENT_NAME || vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
# Azure AI Search (for evaluation samples)
AZURE_SEARCH_ENDPOINT: ${{ secrets.AZURE_SEARCH_ENDPOINT }}
AZURE_SEARCH_API_KEY: ${{ secrets.AZURE_SEARCH_API_KEY }}
@@ -560,11 +562,11 @@ jobs:
FOUNDRY_MODEL: ${{ vars.FOUNDRY_MODEL || vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
# Azure OpenAI configuration
AZURE_OPENAI_ENDPOINT: ${{ vars.AZUREOPENAI__ENDPOINT }}
AZURE_OPENAI_DEPLOYMENT_NAME: ${{ vars.AZURE_OPENAI_DEPLOYMENT_NAME || vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
AZURE_OPENAI_MODEL: ${{ vars.AZURE_OPENAI_DEPLOYMENT_NAME || vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
# OpenAI configuration
OPENAI_API_KEY: ${{ secrets.OPENAI__APIKEY }}
OPENAI_CHAT_MODEL_ID: ${{ vars.OPENAI__CHATMODELID }}
OPENAI_RESPONSES_MODEL_ID: ${{ vars.OPENAI__RESPONSESMODELID }}
OPENAI_CHAT_MODEL: ${{ vars.OPENAI__CHATMODELID }}
OPENAI_RESPONSES_MODEL: ${{ vars.OPENAI__RESPONSESMODELID }}
OPENAI_MODEL: ${{ vars.OPENAI__RESPONSESMODELID }}
defaults:
run:
@@ -585,10 +587,10 @@ jobs:
echo "FOUNDRY_PROJECT_ENDPOINT=$FOUNDRY_PROJECT_ENDPOINT" >> .env
echo "FOUNDRY_MODEL=$FOUNDRY_MODEL" >> .env
echo "AZURE_OPENAI_ENDPOINT=$AZURE_OPENAI_ENDPOINT" >> .env
echo "AZURE_OPENAI_DEPLOYMENT_NAME=$AZURE_OPENAI_DEPLOYMENT_NAME" >> .env
echo "AZURE_OPENAI_MODEL=$AZURE_OPENAI_MODEL" >> .env
echo "OPENAI_API_KEY=$OPENAI_API_KEY" >> .env
echo "OPENAI_CHAT_MODEL_ID=$OPENAI_CHAT_MODEL_ID" >> .env
echo "OPENAI_RESPONSES_MODEL_ID=$OPENAI_RESPONSES_MODEL_ID" >> .env
echo "OPENAI_CHAT_MODEL=$OPENAI_CHAT_MODEL" >> .env
echo "OPENAI_RESPONSES_MODEL=$OPENAI_RESPONSES_MODEL" >> .env
- name: Run sample validation
run: |
@@ -608,14 +610,19 @@ jobs:
env:
FOUNDRY_PROJECT_ENDPOINT: ${{ vars.FOUNDRY_PROJECT_ENDPOINT || vars.AZURE_AI_PROJECT_ENDPOINT }}
FOUNDRY_MODEL: ${{ vars.FOUNDRY_MODEL || vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
# Azure OpenAI configuration
# Azure OpenAI configuration for AF
AZURE_OPENAI_ENDPOINT: ${{ vars.AZUREOPENAI__ENDPOINT }}
AZURE_OPENAI_DEPLOYMENT_NAME: ${{ vars.AZURE_OPENAI_DEPLOYMENT_NAME || vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
# OpenAI configuration
AZURE_OPENAI_MODEL: ${{ vars.AZURE_OPENAI_DEPLOYMENT_NAME || vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
# Azure OpenAI configuration for SK
AZURE_OPENAI_CHAT_DEPLOYMENT_NAME: ${{ vars.AZURE_OPENAI_DEPLOYMENT_NAME }}
# OpenAI key
OPENAI_API_KEY: ${{ secrets.OPENAI__APIKEY }}
OPENAI_CHAT_MODEL_ID: ${{ vars.OPENAI__CHATMODELID }}
OPENAI_RESPONSES_MODEL_ID: ${{ vars.OPENAI__RESPONSESMODELID }}
# OpenAI configuration for AF
OPENAI_CHAT_MODEL: ${{ vars.OPENAI__CHATMODELID }}
OPENAI_RESPONSES_MODEL: ${{ vars.OPENAI__RESPONSESMODELID }}
OPENAI_MODEL: ${{ vars.OPENAI__RESPONSESMODELID }}
# OpenAI configuration for SK
OPENAI_CHAT_MODEL_ID: ${{ vars.OPENAI__CHATMODELID }}
# Copilot Studio
COPILOTSTUDIOAGENT__ENVIRONMENTID: ${{ secrets.COPILOTSTUDIOAGENT__ENVIRONMENTID }}
COPILOTSTUDIOAGENT__SCHEMANAME: ${{ secrets.COPILOTSTUDIOAGENT__SCHEMANAME }}
@@ -640,10 +647,10 @@ jobs:
echo "FOUNDRY_PROJECT_ENDPOINT=$FOUNDRY_PROJECT_ENDPOINT" >> .env
echo "FOUNDRY_MODEL=$FOUNDRY_MODEL" >> .env
echo "AZURE_OPENAI_ENDPOINT=$AZURE_OPENAI_ENDPOINT" >> .env
echo "AZURE_OPENAI_DEPLOYMENT_NAME=$AZURE_OPENAI_DEPLOYMENT_NAME" >> .env
echo "AZURE_OPENAI_MODEL=$AZURE_OPENAI_MODEL" >> .env
echo "OPENAI_API_KEY=$OPENAI_API_KEY" >> .env
echo "OPENAI_CHAT_MODEL_ID=$OPENAI_CHAT_MODEL_ID" >> .env
echo "OPENAI_RESPONSES_MODEL_ID=$OPENAI_RESPONSES_MODEL_ID" >> .env
echo "OPENAI_CHAT_MODEL=$OPENAI_CHAT_MODEL" >> .env
echo "OPENAI_RESPONSES_MODEL=$OPENAI_RESPONSES_MODEL" >> .env
echo "COPILOTSTUDIOAGENT__ENVIRONMENTID=$COPILOTSTUDIOAGENT__ENVIRONMENTID" >> .env
echo "COPILOTSTUDIOAGENT__SCHEMANAME=$COPILOTSTUDIOAGENT__SCHEMANAME" >> .env
echo "COPILOTSTUDIOAGENT__TENANTID=$COPILOTSTUDIOAGENT__TENANTID" >> .env
+12 -12
View File
@@ -94,23 +94,23 @@ Create a simple Azure Responses Agent that writes a haiku about the Microsoft Ag
# Use `az login` to authenticate with Azure CLI
import os
import asyncio
from agent_framework.azure import AzureOpenAIResponsesClient
from agent_framework import Agent
from agent_framework.foundry import FoundryChatClient
from azure.identity import AzureCliCredential
async def main():
# Initialize a chat agent with Azure OpenAI Responses
# Initialize a chat agent with Microsoft Foundry
# the endpoint, deployment name, and api version can be set via environment variables
# or they can be passed in directly to the AzureOpenAIResponsesClient constructor
agent = AzureOpenAIResponsesClient(
# endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
# deployment_name=os.environ["AZURE_OPENAI_RESPONSES_DEPLOYMENT_NAME"],
# api_version=os.environ["AZURE_OPENAI_API_VERSION"],
# api_key=os.environ["AZURE_OPENAI_API_KEY"], # Optional if using AzureCliCredential
credential=AzureCliCredential(), # Optional, if using api_key
).as_agent(
name="HaikuBot",
instructions="You are an upbeat assistant that writes beautifully.",
# or they can be passed in directly to the FoundryChatClient constructor
agent = Agent(
client=FoundryChatClient(
credential=AzureCliCredential(),
# project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"],
# model=os.environ["FOUNDRY_MODEL_DEPLOYMENT_NAME"],
),
name="HaikuBot",
instructions="You are an upbeat assistant that writes beautifully.",
)
print(await agent.run("Write a haiku about Microsoft Agent Framework."))
@@ -3,13 +3,13 @@ name: MicrosoftLearnAgent
description: Microsoft Learn Agent
instructions: You answer questions by searching the Microsoft Learn content only.
model:
id: =Env.AZURE_FOUNDRY_PROJECT_MODEL_ID
id: =Env.FOUNDRY_MODEL
options:
temperature: 0.9
topP: 0.95
connection:
kind: remote
endpoint: =Env.AZURE_FOUNDRY_PROJECT_ENDPOINT
endpoint: =Env.FOUNDRY_PROJECT_ENDPOINT
tools:
- kind: mcp
name: microsoft_learn
+213
View File
@@ -0,0 +1,213 @@
---
name: verify-samples-tool
description: How to use the verify-samples tool to run, verify, and manage sample definitions in the Agent Framework repository. Use this when adding, updating, or running sample verification.
---
# verify-samples Tool
The `verify-samples` project (`dotnet/eng/verify-samples/`) is an automated tool that runs sample projects and verifies their output using deterministic checks and AI-powered verification.
## Running verify-samples
```bash
cd dotnet
# Run all samples across all categories
dotnet run --project eng/verify-samples -- --log results.log --csv results.csv
# Run a specific category
dotnet run --project eng/verify-samples -- --category 02-agents --log results.log
# Run specific samples by name
dotnet run --project eng/verify-samples -- Agent_Step02_StructuredOutput Agent_Step09_AsFunctionTool
# Control parallelism (default 8)
dotnet run --project eng/verify-samples -- --parallel 8 --log results.log
# Combine options
dotnet run --project eng/verify-samples -- --category 03-workflows --parallel 4 --log results.log --csv results.csv
```
### Required Environment Variables
The tool itself needs:
- `AZURE_OPENAI_ENDPOINT` — for the AI verification agent
- `AZURE_OPENAI_DEPLOYMENT_NAME` (optional, defaults to `gpt-5-mini`)
Individual samples require their own env vars (e.g., `AZURE_AI_PROJECT_ENDPOINT`). The tool automatically checks and skips samples with missing env vars.
### Output Files
- `--log results.log` — detailed per-sample log with stdout/stderr, AI reasoning, and a summary
- `--csv results.csv` — tabular summary with Sample, ProjectPath, Status, FailedChecks, and Failures columns
## Sample Categories
Definitions are in the `dotnet/eng/verify-samples/` directory:
| Category | Config File | Registered Key |
|----------|-------------|----------------|
| 01-get-started | `GetStartedSamples.cs` | `01-get-started` |
| 02-agents | `AgentsSamples.cs` | `02-agents` |
| 03-workflows | `WorkflowSamples.cs` | `03-workflows` |
Categories are registered in `VerifyOptions.cs` in the `s_sampleSets` dictionary.
## SampleDefinition Properties
Each sample is defined as a `SampleDefinition` in the appropriate config file. Key properties:
```csharp
new SampleDefinition
{
// Required: Display name for the sample
Name = "Agent_Step02_StructuredOutput",
// Required: Relative path from dotnet/ to the sample project directory
ProjectPath = "samples/02-agents/Agents/Agent_Step02_StructuredOutput",
// Environment variables the sample requires (throws if missing)
RequiredEnvironmentVariables = ["AZURE_OPENAI_ENDPOINT"],
// Environment variables with defaults that would prompt on console if unset
OptionalEnvironmentVariables = ["AZURE_OPENAI_DEPLOYMENT_NAME"],
// Skip this sample with a reason (for structural issues only)
SkipReason = null, // or "Requires external service X."
// Deterministic checks: substrings that must appear in stdout
MustContain = ["=== Section Header ==="],
// Substrings that must NOT appear in stdout
MustNotContain = [],
// If true, only MustContain checks are used (no AI verification)
IsDeterministic = false,
// AI verification: natural-language descriptions of expected output
// Each entry describes one aspect to verify independently
ExpectedOutputDescription =
[
"The output should show structured person information with Name, Age, and Occupation fields.",
"The output should not contain error messages or stack traces.",
],
// Stdin inputs to feed to the sample (for interactive samples)
Inputs = ["Y", "Y", "Y"],
// Delay between stdin inputs in ms (default 2000, increase for LLM calls between inputs)
InputDelayMs = 3000,
}
```
## How to Add a New Sample Definition
1. **Check the sample's Program.cs** to understand:
- What environment variables it reads (look for `GetEnvironmentVariable`)
- Whether it needs stdin input (look for `Console.ReadLine`, `Application.GetInput`)
- Whether it has an external loop (look for `EXIT` patterns in YAML workflows)
- What output it produces (section headers, markers, expected behavior)
- Whether it exits on its own or runs as a server
2. **Choose the right verification strategy:**
- **Deterministic** (`IsDeterministic = true`): Use `MustContain` for samples with fixed output strings. No AI verification.
- **AI-verified** (default): Use `ExpectedOutputDescription` with semantic descriptions. Write expectations that are flexible enough for non-deterministic LLM output.
- **Both**: Use `MustContain` for fixed markers AND `ExpectedOutputDescription` for LLM-generated content.
3. **Set `SkipReason` only for structural issues:**
- Web servers that don't exit
- Multi-process client/server architectures
- Samples requiring external infrastructure (MCP servers you can't reach, Docker, etc.)
- Do NOT skip for missing env vars — the tool checks those dynamically.
4. **For interactive samples, provide `Inputs`:**
- Samples using `Application.GetInput(args)` need one initial input
- Samples with `Console.ReadLine()` approval loops need `"Y"` inputs
- YAML workflows with `externalLoop` need `"EXIT"` as the last input
- Set `InputDelayMs` to 3000-8000ms for samples with LLM calls between inputs
5. **Add the definition** to the appropriate config file (e.g., `AgentsSamples.cs`) in the `All` list.
6. **Register new categories** (if needed) in `VerifyOptions.cs` `s_sampleSets` dictionary.
### Writing Good ExpectedOutputDescription
- Write descriptions that are **semantically flexible** — LLM output varies between runs
- Each array entry should describe **one independent aspect** to verify
- Always include `"The output should not contain error messages or stack traces."` as the last entry
- Avoid exact wording expectations — use "should mention", "should contain information about", "should show"
- Bad: `"The output should say 'The weather in Amsterdam is cloudy with a high of 15°C'"`
- Good: `"The output should contain weather information about Amsterdam mentioning cloudy weather with a high of 15°C."`
### Example: Simple LLM Sample
```csharp
new SampleDefinition
{
Name = "Agent_With_AzureOpenAIChatCompletion",
ProjectPath = "samples/02-agents/AgentProviders/Agent_With_AzureOpenAIChatCompletion",
RequiredEnvironmentVariables = ["AZURE_OPENAI_ENDPOINT"],
OptionalEnvironmentVariables = ["AZURE_OPENAI_DEPLOYMENT_NAME"],
ExpectedOutputDescription =
[
"The output should contain a joke about a pirate.",
"The output should not contain error messages or stack traces.",
],
},
```
### Example: Deterministic Sample
```csharp
new SampleDefinition
{
Name = "Workflow_Declarative_GenerateCode",
ProjectPath = "samples/03-workflows/Declarative/GenerateCode",
IsDeterministic = true,
MustContain = ["WORKFLOW: Parsing", "WORKFLOW: Defined"],
ExpectedOutputDescription = ["The output should show a YAML workflow being parsed and C# code being generated from it."],
},
```
### Example: Interactive Sample with Approval Loop
```csharp
new SampleDefinition
{
Name = "FoundryAgent_Hosted_MCP",
ProjectPath = "samples/02-agents/ModelContextProtocol/FoundryAgent_Hosted_MCP",
RequiredEnvironmentVariables = ["AZURE_AI_PROJECT_ENDPOINT"],
OptionalEnvironmentVariables = ["AZURE_AI_MODEL_DEPLOYMENT_NAME"],
Inputs = ["Y", "Y", "Y", "Y", "Y"],
InputDelayMs = 5000,
ExpectedOutputDescription = ["The output should show an agent using the Microsoft Learn MCP tool with approval prompts."],
},
```
### Example: Declarative Workflow with External Loop
```csharp
new SampleDefinition
{
Name = "Workflow_Declarative_FunctionTools",
ProjectPath = "samples/03-workflows/Declarative/FunctionTools",
RequiredEnvironmentVariables = ["AZURE_AI_PROJECT_ENDPOINT"],
OptionalEnvironmentVariables = ["AZURE_AI_MODEL_DEPLOYMENT_NAME"],
Inputs = ["What are today's specials?", "EXIT"],
InputDelayMs = 8000,
ExpectedOutputDescription = ["The output should show a workflow calling function tools to answer a question about restaurant specials."],
},
```
### Example: Skipped Sample
```csharp
new SampleDefinition
{
Name = "Agent_MCP_Server",
ProjectPath = "samples/02-agents/ModelContextProtocol/Agent_MCP_Server",
RequiredEnvironmentVariables = ["AZURE_OPENAI_ENDPOINT"],
OptionalEnvironmentVariables = ["AZURE_OPENAI_DEPLOYMENT_NAME"],
SkipReason = "Runs as an MCP stdio server that does not exit on its own.",
},
```
+2
View File
@@ -7,6 +7,7 @@
<Folder Name="/Samples/">
<File Path="samples/AGENTS.md" />
<File Path="samples/README.md" />
<Project Path="eng/verify-samples/verify-samples.csproj" />
</Folder>
<Folder Name="/Samples/01-get-started/">
<Project Path="samples/01-get-started/01_hello_agent/01_hello_agent.csproj" />
@@ -171,6 +172,7 @@
<Project Path="samples/02-agents/AgentWithRAG/AgentWithRAG_Step02_CustomVectorStoreRAG/AgentWithRAG_Step02_CustomVectorStoreRAG.csproj" />
<Project Path="samples/02-agents/AgentWithRAG/AgentWithRAG_Step03_CustomRAGDataSource/AgentWithRAG_Step03_CustomRAGDataSource.csproj" />
<Project Path="samples/02-agents/AgentWithRAG/AgentWithRAG_Step04_FoundryServiceRAG/AgentWithRAG_Step04_FoundryServiceRAG.csproj" />
<Project Path="samples/02-agents/AgentWithRAG/AgentWithRAG_Step05_Neo4jGraphRAG/AgentWithRAG_Step05_Neo4jGraphRAG.csproj" />
</Folder>
<Folder Name="/Samples/02-agents/ModelContextProtocol/">
<File Path="samples/02-agents/ModelContextProtocol/README.md" />
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,95 @@
// Copyright (c) Microsoft. All rights reserved.
namespace VerifySamples;
/// <summary>
/// Thread-safe console output with sample-name prefixes and colored status.
/// </summary>
internal sealed class ConsoleReporter
{
private readonly object _lock = new();
/// <summary>
/// Writes a complete prefixed line atomically to the console.
/// </summary>
public void WriteLineWithPrefix(string sampleName, string message, ConsoleColor? color = null)
{
lock (this._lock)
{
Console.ForegroundColor = ConsoleColor.Cyan;
Console.Write($"[{sampleName}] ");
if (color.HasValue)
{
Console.ForegroundColor = color.Value;
}
else
{
Console.ResetColor();
}
Console.WriteLine(message);
Console.ResetColor();
}
}
/// <summary>
/// Prints the final summary table and elapsed time to the console.
/// </summary>
public void PrintSummary(
IReadOnlyList<VerificationResult> orderedResults,
IReadOnlyList<(string Name, string Reason)> skipped,
TimeSpan elapsed)
{
var passCount = orderedResults.Count(r => r.Passed);
var failCount = orderedResults.Count(r => !r.Passed);
Console.WriteLine();
Console.WriteLine(new string('─', 60));
Console.ForegroundColor = ConsoleColor.White;
Console.WriteLine("SUMMARY");
Console.ResetColor();
foreach (var result in orderedResults)
{
Console.ForegroundColor = result.Passed ? ConsoleColor.Green : ConsoleColor.Red;
Console.Write(result.Passed ? " ✓ " : " ✗ ");
Console.ResetColor();
Console.WriteLine($"{result.SampleName}: {result.Summary}");
}
foreach (var (name, reason) in skipped)
{
Console.ForegroundColor = ConsoleColor.Yellow;
Console.Write(" ○ ");
Console.ResetColor();
Console.WriteLine($"{name}: Skipped — {reason}");
}
Console.WriteLine();
Console.Write("Results: ");
Console.ForegroundColor = ConsoleColor.Green;
Console.Write($"{passCount} passed");
Console.ResetColor();
if (failCount > 0)
{
Console.Write(", ");
Console.ForegroundColor = ConsoleColor.Red;
Console.Write($"{failCount} failed");
Console.ResetColor();
}
if (skipped.Count > 0)
{
Console.Write(", ");
Console.ForegroundColor = ConsoleColor.Yellow;
Console.Write($"{skipped.Count} skipped");
Console.ResetColor();
}
Console.WriteLine();
Console.ForegroundColor = ConsoleColor.DarkGray;
Console.WriteLine($"Elapsed: {elapsed.Hours:D2}:{elapsed.Minutes:D2}:{elapsed.Seconds:D2}");
Console.ResetColor();
}
}
@@ -0,0 +1,56 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Text;
namespace VerifySamples;
/// <summary>
/// Writes a CSV summary of sample verification results.
/// </summary>
internal static class CsvResultWriter
{
/// <summary>
/// Writes the results to a CSV file at the specified path.
/// </summary>
public static async Task WriteAsync(
string path,
IReadOnlyList<VerificationResult> orderedResults,
IReadOnlyList<(string Name, string Reason)> skipped,
IReadOnlyList<SampleDefinition> samples)
{
var pathLookup = samples.ToDictionary(s => s.Name, s => s.ProjectPath);
var sb = new StringBuilder();
sb.AppendLine("Sample,ProjectPath,Status,FailedChecks,Failures");
foreach (var result in orderedResults)
{
var status = result.Passed ? "PASSED" : "FAILED";
var failedChecks = result.Failures.Count;
var failures = string.Join("; ", result.Failures);
pathLookup.TryGetValue(result.SampleName, out var projectPath);
sb.AppendLine($"{CsvEscape(result.SampleName)},{CsvEscape(projectPath ?? "")},{status},{failedChecks},{CsvEscape(failures)}");
}
foreach (var (name, reason) in skipped)
{
pathLookup.TryGetValue(name, out var projectPath);
sb.AppendLine($"{CsvEscape(name)},{CsvEscape(projectPath ?? "")},SKIPPED,0,{CsvEscape(reason)}");
}
await File.WriteAllTextAsync(path, sb.ToString());
}
/// <summary>
/// Escapes a value for CSV: wraps in quotes if it contains commas, quotes, or newlines.
/// </summary>
private static string CsvEscape(string value)
{
if (value.Contains('"') || value.Contains(',') || value.Contains('\n') || value.Contains('\r'))
{
return $"\"{value.Replace("\"", "\"\"")}\"";
}
return value;
}
}
@@ -0,0 +1,105 @@
// Copyright (c) Microsoft. All rights reserved.
namespace VerifySamples;
/// <summary>
/// Defines the expected behavior for each sample in 01-get-started.
/// </summary>
internal static class GetStartedSamples
{
public static IReadOnlyList<SampleDefinition> All { get; } =
[
new SampleDefinition
{
Name = "05_first_workflow",
ProjectPath = "samples/01-get-started/05_first_workflow",
RequiredEnvironmentVariables = [],
IsDeterministic = true,
MustContain =
[
"UppercaseExecutor: HELLO, WORLD!",
"ReverseTextExecutor: !DLROW ,OLLEH",
],
},
new SampleDefinition
{
Name = "01_hello_agent",
ProjectPath = "samples/01-get-started/01_hello_agent",
RequiredEnvironmentVariables = ["AZURE_OPENAI_ENDPOINT"],
OptionalEnvironmentVariables = ["AZURE_OPENAI_DEPLOYMENT_NAME"],
ExpectedOutputDescription =
[
"The output should contain a joke about a pirate.",
"There should be two separate joke responses — one from a non-streaming call and one from a streaming call.",
"The output should not contain error messages or stack traces.",
],
},
new SampleDefinition
{
Name = "02_add_tools",
ProjectPath = "samples/01-get-started/02_add_tools",
RequiredEnvironmentVariables = ["AZURE_OPENAI_ENDPOINT"],
OptionalEnvironmentVariables = ["AZURE_OPENAI_DEPLOYMENT_NAME"],
MustContain = [],
ExpectedOutputDescription =
[
"The output should contain information about the weather in Amsterdam.",
"The response should mention that it is cloudy with a high of 15°C (or equivalent), since this comes from a tool that returns a canned response.",
"There should be two responses — one from a non-streaming call and one from a streaming call.",
"The output should not contain error messages or stack traces.",
],
},
new SampleDefinition
{
Name = "03_multi_turn",
ProjectPath = "samples/01-get-started/03_multi_turn",
RequiredEnvironmentVariables = ["AZURE_OPENAI_ENDPOINT"],
OptionalEnvironmentVariables = ["AZURE_OPENAI_DEPLOYMENT_NAME"],
ExpectedOutputDescription =
[
"The output should contain a joke about a pirate.",
"After the initial joke, there should be a modified version that includes emojis and is told in the voice of a pirate's parrot.",
"The pattern repeats: first a non-streaming pirate joke + parrot version, then a streaming pirate joke + parrot version.",
"The output should not contain error messages or stack traces.",
],
},
new SampleDefinition
{
Name = "04_memory",
ProjectPath = "samples/01-get-started/04_memory",
RequiredEnvironmentVariables = ["AZURE_OPENAI_ENDPOINT"],
OptionalEnvironmentVariables = ["AZURE_OPENAI_DEPLOYMENT_NAME"],
MustContain =
[
">> Use session with blank memory",
">> Use deserialized session with previously created memories",
">> Read memories using memory component",
"MEMORY - User Name:",
"MEMORY - User Age:",
">> Use new session with previously created memories",
],
ExpectedOutputDescription =
[
"In the 'Use session with blank memory' section, the agent should respond to the user's messages. It may ask for the user's name or age if not yet known.",
"In the 'Use deserialized session with previously created memories' section, the agent should correctly recall that the user's name is Ruaidhrí and age is 20.",
"The 'MEMORY - User Name:' line should show 'Ruaidhrí' (or a close transliteration).",
"The 'MEMORY - User Age:' line should show '20'.",
"In the 'Use new session with previously created memories' section, the agent should know the user's name and age from the transferred memory.",
"The output should not contain error messages or stack traces.",
],
},
new SampleDefinition
{
Name = "06_host_your_agent",
ProjectPath = "samples/01-get-started/06_host_your_agent",
RequiredEnvironmentVariables = ["AZURE_OPENAI_ENDPOINT"],
OptionalEnvironmentVariables = ["AZURE_OPENAI_DEPLOYMENT_NAME"],
SkipReason = "Requires Azure Functions Core Tools runtime and starts a web server.",
},
];
}
+153
View File
@@ -0,0 +1,153 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Text;
namespace VerifySamples;
/// <summary>
/// Incrementally writes a sequential (non-interleaved) log file, appending after each sample completes.
/// Thread-safe: multiple parallel tasks may call write methods concurrently.
/// </summary>
internal sealed class LogFileWriter : IDisposable
{
private readonly string _path;
private readonly SemaphoreSlim _writeLock = new(1, 1);
public LogFileWriter(string path)
{
this._path = path;
}
/// <inheritdoc />
public void Dispose()
{
this._writeLock.Dispose();
}
/// <summary>
/// Writes the log file header. Call once at the start of the run.
/// </summary>
public async Task WriteHeaderAsync()
{
var sb = new StringBuilder();
sb.AppendLine($"Sample Verification Log — {DateTime.UtcNow:yyyy-MM-dd HH:mm:ss} UTC");
sb.AppendLine(new string('═', 72));
sb.AppendLine();
await File.WriteAllTextAsync(this._path, sb.ToString());
}
/// <summary>
/// Appends a skipped-sample entry to the log file.
/// </summary>
public async Task WriteSkippedAsync(string name, string reason)
{
var sb = new StringBuilder();
sb.AppendLine($"── {name} ──");
sb.AppendLine($"Status: SKIPPED — {reason}");
sb.AppendLine();
await this.AppendAsync(sb.ToString());
}
/// <summary>
/// Appends a completed sample's full output section to the log file.
/// </summary>
public async Task WriteSampleResultAsync(VerificationResult result)
{
var sb = new StringBuilder();
sb.AppendLine(new string('─', 72));
sb.AppendLine($"── {result.SampleName} ──");
sb.AppendLine($"Status: {(result.Passed ? "PASSED" : "FAILED")}");
sb.AppendLine();
foreach (var line in result.LogLines)
{
sb.AppendLine(line);
}
sb.AppendLine();
if (!string.IsNullOrWhiteSpace(result.Stdout))
{
sb.AppendLine("--- stdout ---");
sb.AppendLine(result.Stdout.TrimEnd());
sb.AppendLine("--- end stdout ---");
sb.AppendLine();
}
if (!string.IsNullOrWhiteSpace(result.Stderr))
{
sb.AppendLine("--- stderr ---");
sb.AppendLine(result.Stderr.TrimEnd());
sb.AppendLine("--- end stderr ---");
sb.AppendLine();
}
if (result.Failures.Count > 0)
{
sb.AppendLine("Failures:");
foreach (var failure in result.Failures)
{
sb.AppendLine($" ✗ {failure}");
}
sb.AppendLine();
}
if (result.AIReasoning is not null)
{
sb.AppendLine("AI Reasoning:");
sb.AppendLine(result.AIReasoning);
sb.AppendLine();
}
await this.AppendAsync(sb.ToString());
}
/// <summary>
/// Appends the final summary section and elapsed time to the log file.
/// </summary>
public async Task WriteSummaryAsync(
IReadOnlyList<VerificationResult> orderedResults,
IReadOnlyList<(string Name, string Reason)> skipped,
TimeSpan elapsed)
{
var passCount = orderedResults.Count(r => r.Passed);
var failCount = orderedResults.Count(r => !r.Passed);
var sb = new StringBuilder();
sb.AppendLine(new string('═', 72));
sb.AppendLine("SUMMARY");
sb.AppendLine();
foreach (var result in orderedResults)
{
sb.AppendLine($" {(result.Passed ? "" : "")} {result.SampleName}: {result.Summary}");
}
foreach (var (name, reason) in skipped)
{
sb.AppendLine($" ○ {name}: Skipped — {reason}");
}
sb.AppendLine();
sb.AppendLine($"Results: {passCount} passed{(failCount > 0 ? $", {failCount} failed" : "")}{(skipped.Count > 0 ? $", {skipped.Count} skipped" : "")}");
sb.AppendLine($"Elapsed: {elapsed.Hours:D2}:{elapsed.Minutes:D2}:{elapsed.Seconds:D2}");
await this.AppendAsync(sb.ToString());
}
private async Task AppendAsync(string text)
{
await this._writeLock.WaitAsync();
try
{
await File.AppendAllTextAsync(this._path, text);
}
finally
{
this._writeLock.Release();
}
}
}
+98
View File
@@ -0,0 +1,98 @@
// Copyright (c) Microsoft. All rights reserved.
// This tool runs the 01-get-started, 02-agents, and 03-workflows samples and verifies their output.
// Deterministic samples are verified with exact string matching.
// Non-deterministic (LLM) samples are verified using an agent-framework agent.
//
// Usage:
// dotnet run # Run all samples
// dotnet run -- 01_hello_agent 05_first_workflow # Run specific samples by name
// dotnet run -- --category 01-get-started # Run the 01-get-started category
// dotnet run -- --category 02-agents # Run the 02-agents category
// dotnet run -- --category 03-workflows # Run the 03-workflows category
// dotnet run -- --parallel 16 # Run up to 16 samples concurrently
// dotnet run -- --log results.log # Write sequential log to file
// dotnet run -- --csv results.csv # Write CSV summary to file
//
// Required environment variables (for AI-powered samples):
// AZURE_OPENAI_ENDPOINT
// AZURE_OPENAI_DEPLOYMENT_NAME (optional, defaults to gpt-5-mini)
using System.Diagnostics;
using Azure.AI.OpenAI;
using Azure.Identity;
using VerifySamples;
var options = VerifyOptions.Parse(args);
if (options is null)
{
return 1;
}
var stopwatch = Stopwatch.StartNew();
// Resolve the dotnet/ root directory (verify-samples is at dotnet/eng/verify-samples/)
var dotnetRoot = Path.GetFullPath(Path.Combine(AppContext.BaseDirectory, "..", "..", "..", "..", ".."));
if (!File.Exists(Path.Combine(dotnetRoot, "agent-framework-dotnet.slnx")))
{
dotnetRoot = Path.GetFullPath(Path.Combine(Directory.GetCurrentDirectory(), "..", ".."));
}
// Set up the AI verifier
var endpoint = Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT");
var deploymentName = Environment.GetEnvironmentVariable("AZURE_OPENAI_DEPLOYMENT_NAME") ?? "gpt-5-mini";
OpenAI.Chat.ChatClient? chatClient = null;
if (!string.IsNullOrEmpty(endpoint))
{
chatClient = new AzureOpenAIClient(new Uri(endpoint), new DefaultAzureCredential())
.GetChatClient(deploymentName);
}
// Set up optional log file writer
LogFileWriter? logWriter = null;
if (options.LogFilePath is not null)
{
logWriter = new LogFileWriter(options.LogFilePath);
await logWriter.WriteHeaderAsync();
}
try
{
// Run all samples
var reporter = new ConsoleReporter();
var verifier = new SampleVerifier(chatClient);
var orchestrator = new VerificationOrchestrator(verifier, reporter, dotnetRoot, TimeSpan.FromMinutes(3), logWriter);
var run = await orchestrator.RunAllAsync(options.Samples, options.MaxParallelism);
stopwatch.Stop();
// Print summary
var orderedResults = run.SampleOrder
.Where(run.Results.ContainsKey)
.Select(name => run.Results[name])
.ToList();
reporter.PrintSummary(orderedResults, run.Skipped, stopwatch.Elapsed);
// Write log file summary
if (logWriter is not null)
{
await logWriter.WriteSummaryAsync(orderedResults, run.Skipped, stopwatch.Elapsed);
Console.WriteLine($"Log written to: {options.LogFilePath}");
}
// Write CSV summary
if (options.CsvFilePath is not null)
{
await CsvResultWriter.WriteAsync(options.CsvFilePath, orderedResults, run.Skipped, options.Samples);
Console.WriteLine($"CSV written to: {options.CsvFilePath}");
}
return orderedResults.Any(r => !r.Passed) ? 1 : 0;
}
finally
{
logWriter?.Dispose();
}
@@ -0,0 +1,79 @@
// Copyright (c) Microsoft. All rights reserved.
namespace VerifySamples;
/// <summary>
/// Describes a sample to verify, including its expected output.
/// </summary>
internal sealed class SampleDefinition
{
/// <summary>
/// Display name for the sample (e.g., "01_hello_agent").
/// </summary>
public required string Name { get; init; }
/// <summary>
/// Relative path from the dotnet/ directory to the sample project directory.
/// </summary>
public required string ProjectPath { get; init; }
/// <summary>
/// Environment variables that the sample requires for a meaningful run.
/// The runner checks these before running and will skip the sample if any are unset,
/// recording a skip reason that indicates which required variables are missing.
/// </summary>
public string[] RequiredEnvironmentVariables { get; init; } = [];
/// <summary>
/// Environment variables that the sample can use but typically has fallbacks or defaults for.
/// If these are not set, the sample might prompt or behave interactively, which could cause
/// automated verification to hang. The runner checks these and skips the sample if they are unset
/// to avoid non-deterministic or blocking behavior in automated runs.
/// </summary>
public string[] OptionalEnvironmentVariables { get; init; } = [];
/// <summary>
/// If set, the sample is skipped with this reason.
/// Use only for structural reasons (e.g., web server, multi-process, needs external service).
/// Do NOT use for missing environment variables — those are checked dynamically.
/// </summary>
public string? SkipReason { get; init; }
/// <summary>
/// Substrings that must appear in stdout for the sample to pass.
/// Used for deterministic verification.
/// </summary>
public string[] MustContain { get; init; } = [];
/// <summary>
/// Substrings that must not appear in stdout for the sample to pass.
/// </summary>
public string[] MustNotContain { get; init; } = [];
/// <summary>
/// If true, <see cref="MustContain"/> entries cover the entire expected output —
/// no AI verification is needed.
/// </summary>
public bool IsDeterministic { get; init; }
/// <summary>
/// Natural-language description of what the sample output should look like.
/// Used by the AI verifier for non-deterministic samples.
/// Each entry describes one aspect of the expected output that should be verified.
/// </summary>
public string[] ExpectedOutputDescription { get; init; } = [];
/// <summary>
/// Sequence of stdin inputs to feed to the sample process.
/// Each entry is written as a line (followed by newline) to the process stdin.
/// A <c>null</c> entry inserts a delay without writing anything.
/// Inputs are sent with a short delay between each to allow the process to prompt.
/// </summary>
public string?[] Inputs { get; init; } = [];
/// <summary>
/// Delay in milliseconds between each input line. Default is 2000ms.
/// Increase for samples that need more time between prompts (e.g., LLM calls between inputs).
/// </summary>
public int InputDelayMs { get; init; } = 2000;
}
+132
View File
@@ -0,0 +1,132 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Diagnostics;
namespace VerifySamples;
/// <summary>
/// Result of running a sample process.
/// </summary>
internal sealed record SampleRunResult(
string Stdout,
string Stderr,
int ExitCode,
TimeSpan Elapsed);
/// <summary>
/// Runs a sample project via <c>dotnet run</c> and captures its output.
/// </summary>
internal static class SampleRunner
{
/// <summary>
/// Runs <c>dotnet run --framework net10.0</c> in the given project directory.
/// </summary>
public static Task<SampleRunResult> RunAsync(
string projectPath,
TimeSpan timeout,
CancellationToken cancellationToken = default)
=> RunAsync(projectPath, "run --framework net10.0", timeout, inputs: null, inputDelayMs: 0, cancellationToken: cancellationToken);
/// <summary>
/// Runs <c>dotnet run --framework net10.0</c> with stdin inputs.
/// </summary>
public static Task<SampleRunResult> RunAsync(
string projectPath,
TimeSpan timeout,
string?[]? inputs,
int inputDelayMs = 2000,
CancellationToken cancellationToken = default)
=> RunAsync(projectPath, "run --framework net10.0", timeout, inputs, inputDelayMs, cancellationToken);
/// <summary>
/// Runs an arbitrary <c>dotnet</c> command in the given working directory.
/// </summary>
public static async Task<SampleRunResult> RunAsync(
string workingDirectory,
string dotnetArgs,
TimeSpan timeout,
string?[]? inputs = null,
int inputDelayMs = 0,
CancellationToken cancellationToken = default)
{
var psi = new ProcessStartInfo
{
FileName = "dotnet",
Arguments = dotnetArgs,
WorkingDirectory = workingDirectory,
RedirectStandardOutput = true,
RedirectStandardError = true,
RedirectStandardInput = inputs is { Length: > 0 },
UseShellExecute = false,
CreateNoWindow = true,
};
var sw = Stopwatch.StartNew();
using var process = new Process { StartInfo = psi };
process.Start();
var stdoutTask = process.StandardOutput.ReadToEndAsync(cancellationToken);
var stderrTask = process.StandardError.ReadToEndAsync(cancellationToken);
// Feed stdin inputs with delays if configured
if (inputs is { Length: > 0 })
{
_ = Task.Run(async () =>
{
try
{
foreach (var input in inputs)
{
await Task.Delay(inputDelayMs, cancellationToken);
if (input is not null)
{
await process.StandardInput.WriteLineAsync(input.AsMemory(), cancellationToken);
await process.StandardInput.FlushAsync(cancellationToken);
}
}
process.StandardInput.Close();
}
catch (Exception ex) when (ex is IOException or ObjectDisposedException or OperationCanceledException)
{
// Process may have exited before all inputs were sent
}
}, cancellationToken);
}
using var cts = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
cts.CancelAfter(timeout);
try
{
await process.WaitForExitAsync(cts.Token);
}
catch (OperationCanceledException) when (!cancellationToken.IsCancellationRequested)
{
// Timeout — kill the process
try
{
process.Kill(entireProcessTree: true);
}
catch
{
// Best effort
}
sw.Stop();
return new SampleRunResult(
Stdout: await stdoutTask,
Stderr: $"TIMEOUT: Sample did not complete within {timeout.TotalSeconds}s.\n{await stderrTask}",
ExitCode: -1,
Elapsed: sw.Elapsed);
}
sw.Stop();
return new SampleRunResult(
Stdout: await stdoutTask,
Stderr: await stderrTask,
ExitCode: process.ExitCode,
Elapsed: sw.Elapsed);
}
}
+202
View File
@@ -0,0 +1,202 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Text.Json.Serialization;
using Microsoft.Agents.AI;
using Microsoft.Extensions.AI;
using OpenAI.Chat;
namespace VerifySamples;
/// <summary>
/// Verifies sample output using deterministic checks and an AI agent
/// for non-deterministic output validation.
/// </summary>
internal sealed class SampleVerifier
{
private readonly AIAgent? _verifierAgent;
/// <summary>
/// Creates a verifier. If <paramref name="chatClient"/> is provided,
/// AI-based verification is available for non-deterministic samples.
/// </summary>
public SampleVerifier(ChatClient? chatClient = null)
{
if (chatClient is not null)
{
this._verifierAgent = chatClient.AsAIAgent(
instructions: """
You are a test output verifier. You will be given:
1. The actual stdout output of a program
2. A list of expectations about what the output should contain or demonstrate
Your job is to determine whether the actual output satisfies each expectation.
Be reasonable the output comes from an LLM so exact wording won't match, but the
semantic intent should be clearly satisfied.
""",
name: "OutputVerifier");
}
}
/// <summary>
/// Verifies the output of a sample run against its definition.
/// </summary>
public async Task<VerificationResult> VerifyAsync(SampleDefinition sample, SampleRunResult run)
{
var failures = new List<string>();
// 1. Exit code check
if (run.ExitCode != 0)
{
failures.Add($"Exit code was {run.ExitCode}, expected 0. Stderr: {Truncate(run.Stderr, 500)}");
}
// 2. Must-contain checks
foreach (var expected in sample.MustContain)
{
if (!run.Stdout.Contains(expected, StringComparison.Ordinal))
{
failures.Add($"Output missing expected substring: \"{expected}\"");
}
}
// 3. Must-not-contain checks
foreach (var unexpected in sample.MustNotContain)
{
if (run.Stdout.Contains(unexpected, StringComparison.Ordinal))
{
failures.Add($"Output contains unexpected substring: \"{unexpected}\"");
}
}
// 4. AI verification for non-deterministic samples
string? aiReasoning = null;
if (!sample.IsDeterministic && sample.ExpectedOutputDescription.Length > 0)
{
if (this._verifierAgent is null)
{
failures.Add("AI verification required but no AI agent configured (missing AZURE_OPENAI_ENDPOINT).");
}
else
{
var aiResult = await this.VerifyWithAIAsync(run.Stdout, sample.ExpectedOutputDescription);
aiReasoning = aiResult.Reasoning;
foreach (var unmet in aiResult.UnmetExpectations)
{
failures.Add($"AI expectation not met: {unmet}");
}
}
}
bool passed = failures.Count == 0;
return new VerificationResult
{
SampleName = sample.Name,
Passed = passed,
Summary = passed ? "All checks passed" : $"{failures.Count} check(s) failed",
Failures = failures,
AIReasoning = aiReasoning,
};
}
private async Task<(string Reasoning, List<string> UnmetExpectations)> VerifyWithAIAsync(
string actualOutput,
string[] expectations)
{
var expectationList = string.Join("\n", expectations.Select((e, i) => $" {i + 1}. {e}"));
var prompt = $"""
Actual program output:
---
{Truncate(actualOutput, 4000)}
---
Expectations to verify:
{expectationList}
Does the output satisfy all expectations?
""";
try
{
var response = await this._verifierAgent!.RunAsync<AIVerificationResponse>(prompt);
var result = response.Result;
if (result is null)
{
return ($"AI verification returned null result. Raw: {response.Text}", ["AI verification returned null result."]);
}
var reasoning = result.Reasoning ?? "(no reasoning provided)";
// Collect unmet expectations as individual failures
var unmet = new List<string>();
if (result.ExpectationResults is { Count: > 0 })
{
foreach (var er in result.ExpectationResults.Where(er => !er.Met))
{
var detail = string.IsNullOrWhiteSpace(er.Detail) ? er.Expectation : $"{er.Expectation} — {er.Detail}";
unmet.Add(detail ?? "Unknown expectation");
}
// If the model flagged overall failure but all individual expectations were met,
// still treat as failure using the overall reasoning.
if (unmet.Count == 0 && !result.Pass)
{
unmet.Add(reasoning);
}
}
else if (!result.Pass)
{
// Fallback: no per-expectation detail but overall pass is false
unmet.Add(reasoning);
}
return (reasoning, unmet);
}
catch (Exception ex)
{
return ($"AI verification error: {ex.Message}", [$"AI verification error: {ex.Message}"]);
}
}
private static string Truncate(string text, int maxLength)
=> text.Length <= maxLength ? text : text[..maxLength] + "... (truncated)";
}
/// <summary>
/// Structured response from the AI verification agent.
/// </summary>
[System.Diagnostics.CodeAnalysis.SuppressMessage("Performance", "CA1812:Avoid uninstantiated internal classes", Justification = "Instantiated by JSON deserialization via RunAsync<T>.")]
internal sealed class AIVerificationResponse
{
/// <summary>Whether all expectations were met.</summary>
[JsonPropertyName("pass")]
public bool Pass { get; set; }
/// <summary>Brief explanation of the overall assessment.</summary>
[JsonPropertyName("reasoning")]
public string? Reasoning { get; set; }
/// <summary>Per-expectation results.</summary>
[JsonPropertyName("expectation_results")]
public List<ExpectationResult>? ExpectationResults { get; set; }
}
/// <summary>
/// Result for an individual expectation check.
/// </summary>
[System.Diagnostics.CodeAnalysis.SuppressMessage("Performance", "CA1812:Avoid uninstantiated internal classes", Justification = "Instantiated by JSON deserialization via RunAsync<T>.")]
internal sealed class ExpectationResult
{
/// <summary>The expectation text that was evaluated.</summary>
[JsonPropertyName("expectation")]
public string? Expectation { get; set; }
/// <summary>Whether this expectation was met.</summary>
[JsonPropertyName("met")]
public bool Met { get; set; }
/// <summary>Detail about how the expectation was or was not met.</summary>
[JsonPropertyName("detail")]
public string? Detail { get; set; }
}
@@ -0,0 +1,197 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Collections.Concurrent;
namespace VerifySamples;
/// <summary>
/// Orchestrates sample verification: filters, runs in parallel, and collects results.
/// </summary>
internal sealed class VerificationOrchestrator
{
private readonly SampleVerifier _verifier;
private readonly ConsoleReporter _reporter;
private readonly LogFileWriter? _logWriter;
private readonly string _dotnetRoot;
private readonly TimeSpan _timeout;
public VerificationOrchestrator(
SampleVerifier verifier,
ConsoleReporter reporter,
string dotnetRoot,
TimeSpan timeout,
LogFileWriter? logWriter = null)
{
this._verifier = verifier;
this._reporter = reporter;
this._logWriter = logWriter;
this._dotnetRoot = dotnetRoot;
this._timeout = timeout;
}
/// <summary>
/// The result of running all samples through the orchestrator.
/// </summary>
internal sealed record RunAllResult(
ConcurrentDictionary<string, VerificationResult> Results,
List<(string Name, string Reason)> Skipped,
List<string> SampleOrder);
/// <summary>
/// Filters samples, runs the runnable ones in parallel, and returns all results.
/// </summary>
public async Task<RunAllResult> RunAllAsync(
IReadOnlyList<SampleDefinition> samples,
int maxParallelism)
{
var skipped = new List<(string Name, string Reason)>();
var runnableSamples = new List<SampleDefinition>();
var sampleOrder = new List<string>();
// Separate samples into skipped and runnable
foreach (var sample in samples)
{
sampleOrder.Add(sample.Name);
if (sample.SkipReason is not null)
{
skipped.Add((sample.Name, sample.SkipReason));
this._reporter.WriteLineWithPrefix(sample.Name, $"SKIPPED — {sample.SkipReason}", ConsoleColor.Yellow);
if (this._logWriter is not null)
{
await this._logWriter.WriteSkippedAsync(sample.Name, sample.SkipReason);
}
continue;
}
var missingRequired = sample.RequiredEnvironmentVariables
.Where(v => string.IsNullOrEmpty(Environment.GetEnvironmentVariable(v)))
.ToList();
var missingOptional = sample.OptionalEnvironmentVariables
.Where(v => string.IsNullOrEmpty(Environment.GetEnvironmentVariable(v)))
.ToList();
if (missingRequired.Count > 0 || missingOptional.Count > 0)
{
var reasons = new List<string>();
if (missingRequired.Count > 0)
{
reasons.Add($"Missing required: {string.Join(", ", missingRequired)}");
}
if (missingOptional.Count > 0)
{
reasons.Add($"Missing optional (would cause console prompt hang): {string.Join(", ", missingOptional)}");
}
var skipReason = string.Join("; ", reasons);
skipped.Add((sample.Name, skipReason));
this._reporter.WriteLineWithPrefix(sample.Name, $"SKIPPED — {skipReason}", ConsoleColor.Yellow);
if (this._logWriter is not null)
{
await this._logWriter.WriteSkippedAsync(sample.Name, skipReason);
}
continue;
}
runnableSamples.Add(sample);
}
// Run samples in parallel
var results = new ConcurrentDictionary<string, VerificationResult>();
var semaphore = new SemaphoreSlim(maxParallelism);
this._reporter.WriteLineWithPrefix(
"runner", $"Running {runnableSamples.Count} samples (max {maxParallelism} parallel)...");
try
{
var tasks = runnableSamples.Select(sample => this.RunSingleAsync(sample, results, semaphore)).ToArray();
await Task.WhenAll(tasks);
}
finally
{
semaphore.Dispose();
}
return new RunAllResult(results, skipped, sampleOrder);
}
private async Task RunSingleAsync(
SampleDefinition sample,
ConcurrentDictionary<string, VerificationResult> results,
SemaphoreSlim semaphore)
{
await semaphore.WaitAsync();
try
{
var log = new List<string>();
log.Add($"[{sample.Name}] Running...");
this._reporter.WriteLineWithPrefix(sample.Name, "Running...");
var projectPath = Path.Combine(this._dotnetRoot, sample.ProjectPath);
var run = sample.Inputs.Length > 0
? await SampleRunner.RunAsync(projectPath, this._timeout, sample.Inputs, sample.InputDelayMs)
: await SampleRunner.RunAsync(projectPath, this._timeout);
log.Add($"[{sample.Name}] Completed ({run.Elapsed.TotalSeconds:F1}s, exit={run.ExitCode})");
this._reporter.WriteLineWithPrefix(
sample.Name, $"Completed ({run.Elapsed.TotalSeconds:F1}s, exit={run.ExitCode}). Verifying...");
var result = await this._verifier.VerifyAsync(sample, run);
if (result.Passed)
{
log.Add($"[{sample.Name}] PASSED");
this._reporter.WriteLineWithPrefix(sample.Name, "PASSED", ConsoleColor.Green);
}
else
{
log.Add($"[{sample.Name}] FAILED");
this._reporter.WriteLineWithPrefix(sample.Name, "FAILED", ConsoleColor.Red);
foreach (var failure in result.Failures)
{
log.Add($"[{sample.Name}] ✗ {failure}");
this._reporter.WriteLineWithPrefix(sample.Name, $" ✗ {failure}", ConsoleColor.Red);
}
}
if (result.AIReasoning is not null)
{
log.Add($"[{sample.Name}] AI: {result.AIReasoning}");
this._reporter.WriteLineWithPrefix(
sample.Name, $" AI: {Truncate(result.AIReasoning, 300)}", ConsoleColor.DarkGray);
}
var verificationResult = new VerificationResult
{
SampleName = result.SampleName,
Passed = result.Passed,
Summary = result.Summary,
Failures = result.Failures,
AIReasoning = result.AIReasoning,
Stdout = run.Stdout,
Stderr = run.Stderr,
LogLines = log,
};
results[sample.Name] = verificationResult;
if (this._logWriter is not null)
{
await this._logWriter.WriteSampleResultAsync(verificationResult);
}
}
finally
{
semaphore.Release();
}
}
private static string Truncate(string text, int maxLength)
=> text.Length <= maxLength ? text : text[..maxLength] + "...";
}
@@ -0,0 +1,31 @@
// Copyright (c) Microsoft. All rights reserved.
namespace VerifySamples;
/// <summary>
/// The result of verifying a single sample.
/// </summary>
internal sealed class VerificationResult
{
public required string SampleName { get; init; }
public required bool Passed { get; init; }
public required string Summary { get; init; }
public List<string> Failures { get; init; } = [];
public string? AIReasoning { get; init; }
/// <summary>
/// The sample's stdout output, captured for log file output.
/// </summary>
public string? Stdout { get; init; }
/// <summary>
/// The sample's stderr output, captured for log file output.
/// </summary>
public string? Stderr { get; init; }
/// <summary>
/// Per-sample log lines, buffered during parallel execution
/// and written sequentially to the log file.
/// </summary>
public List<string> LogLines { get; init; } = [];
}
+124
View File
@@ -0,0 +1,124 @@
// Copyright (c) Microsoft. All rights reserved.
namespace VerifySamples;
/// <summary>
/// Parsed command-line options for the sample verification tool.
/// </summary>
internal sealed class VerifyOptions
{
/// <summary>
/// Maximum number of samples to run concurrently.
/// </summary>
public int MaxParallelism { get; init; } = 8;
/// <summary>
/// Path to write a CSV summary file, or <c>null</c> to skip.
/// </summary>
public string? CsvFilePath { get; init; }
/// <summary>
/// Path to write a sequential log file, or <c>null</c> to skip.
/// </summary>
public string? LogFilePath { get; init; }
/// <summary>
/// The filtered list of samples to process.
/// </summary>
public required IReadOnlyList<SampleDefinition> Samples { get; init; }
/// <summary>
/// All known sample set registries, keyed by category name.
/// </summary>
private static readonly Dictionary<string, IReadOnlyList<SampleDefinition>> s_sampleSets =
new(StringComparer.OrdinalIgnoreCase)
{
["01-get-started"] = GetStartedSamples.All,
["02-agents"] = AgentsSamples.All,
["03-workflows"] = WorkflowSamples.All,
};
/// <summary>
/// Parses command-line arguments and resolves the sample list.
/// Returns <c>null</c> and writes to stderr if the arguments are invalid.
/// </summary>
public static VerifyOptions? Parse(string[] args)
{
var argList = args.ToList();
var categoryFilter = ExtractArg(argList, "--category");
var logFilePath = ExtractArg(argList, "--log");
var csvFilePath = ExtractArg(argList, "--csv");
int maxParallelism = 8;
var parallelArg = ExtractArg(argList, "--parallel");
if (parallelArg is not null && int.TryParse(parallelArg, out var p) && p > 0)
{
maxParallelism = p;
}
HashSet<string>? nameFilter = null;
if (argList.Count > 0)
{
nameFilter = argList.ToHashSet(StringComparer.OrdinalIgnoreCase);
}
// Build the sample list
IReadOnlyList<SampleDefinition> samples;
if (categoryFilter is not null)
{
if (!s_sampleSets.TryGetValue(categoryFilter, out var categoryList))
{
Console.Error.WriteLine(
$"Unknown category '{categoryFilter}'. Available: {string.Join(", ", s_sampleSets.Keys)}");
return null;
}
samples = categoryList;
}
else
{
samples = s_sampleSets.Values.SelectMany(s => s).ToList();
}
if (nameFilter is not null)
{
samples = samples.Where(s => nameFilter.Contains(s.Name)).ToList();
}
if (samples.Count == 0)
{
var allNames = s_sampleSets.Values.SelectMany(s => s).Select(s => s.Name);
Console.Error.WriteLine($"No matching samples found. Available: {string.Join(", ", allNames)}");
return null;
}
return new VerifyOptions
{
MaxParallelism = maxParallelism,
LogFilePath = logFilePath,
CsvFilePath = csvFilePath,
Samples = samples,
};
}
private static string? ExtractArg(List<string> list, string flag)
{
var idx = list.IndexOf(flag);
if (idx < 0)
{
return null;
}
if (idx + 1 >= list.Count)
{
Console.Error.WriteLine($"Missing value for {flag}.");
list.RemoveAt(idx);
return null;
}
var value = list[idx + 1];
list.RemoveRange(idx, 2);
return value;
}
}
@@ -0,0 +1,525 @@
// Copyright (c) Microsoft. All rights reserved.
namespace VerifySamples;
/// <summary>
/// Defines the expected behavior for each sample in 03-workflows.
/// </summary>
internal static class WorkflowSamples
{
public static IReadOnlyList<SampleDefinition> All { get; } =
[
// ───────────────────────────────────────────────────────────────────
// _StartHere
// ───────────────────────────────────────────────────────────────────
new SampleDefinition
{
Name = "Workflow_StartHere_01_Streaming",
ProjectPath = "samples/03-workflows/_StartHere/01_Streaming",
RequiredEnvironmentVariables = [],
IsDeterministic = true,
MustContain =
[
"UppercaseExecutor: HELLO, WORLD!",
"ReverseTextExecutor: !DLROW ,OLLEH",
],
},
new SampleDefinition
{
Name = "Workflow_StartHere_02_AgentsInWorkflows",
ProjectPath = "samples/03-workflows/_StartHere/02_AgentsInWorkflows",
RequiredEnvironmentVariables = ["AZURE_OPENAI_ENDPOINT"],
OptionalEnvironmentVariables = ["AZURE_OPENAI_DEPLOYMENT_NAME"],
ExpectedOutputDescription =
[
"The output should show agent responses from a translation workflow.",
"The output should not contain error messages or stack traces.",
],
},
new SampleDefinition
{
Name = "Workflow_StartHere_03_AgentWorkflowPatterns",
ProjectPath = "samples/03-workflows/_StartHere/03_AgentWorkflowPatterns",
RequiredEnvironmentVariables = ["AZURE_OPENAI_ENDPOINT"],
OptionalEnvironmentVariables = ["AZURE_OPENAI_DEPLOYMENT_NAME"],
Inputs = ["sequential"],
InputDelayMs = 3000,
ExpectedOutputDescription =
[
"The output should show a sequential workflow pattern with multiple agents executing tasks in order.",
"The output should not contain error messages or stack traces.",
],
},
new SampleDefinition
{
Name = "Workflow_StartHere_04_MultiModelService",
ProjectPath = "samples/03-workflows/_StartHere/04_MultiModelService",
RequiredEnvironmentVariables = ["BEDROCK_ACCESS_KEY", "BEDROCK_SECRET_KEY", "ANTHROPIC_API_KEY", "OPENAI_API_KEY"],
SkipReason = "Requires multiple external provider API keys (Bedrock, Anthropic, OpenAI).",
},
new SampleDefinition
{
Name = "Workflow_StartHere_05_SubWorkflows",
ProjectPath = "samples/03-workflows/_StartHere/05_SubWorkflows",
RequiredEnvironmentVariables = [],
IsDeterministic = true,
MustContain =
[
"=== Sub-Workflow Demonstration ===",
"Final Output:",
"=== Main Workflow Completed ===",
"Sample Complete: Workflows can be composed hierarchically using sub-workflows",
],
},
new SampleDefinition
{
Name = "Workflow_StartHere_06_MixedWorkflowAgentsAndExecutors",
ProjectPath = "samples/03-workflows/_StartHere/06_MixedWorkflowAgentsAndExecutors",
RequiredEnvironmentVariables = ["AZURE_OPENAI_ENDPOINT"],
OptionalEnvironmentVariables = ["AZURE_OPENAI_DEPLOYMENT_NAME"],
Inputs = ["What is 2 plus 2?"],
InputDelayMs = 3000,
ExpectedOutputDescription =
[
"The output should show agents and executors working together to process a user question.",
"The output should not contain error messages or stack traces.",
],
},
new SampleDefinition
{
Name = "Workflow_StartHere_07_WriterCriticWorkflow",
ProjectPath = "samples/03-workflows/_StartHere/07_WriterCriticWorkflow",
RequiredEnvironmentVariables = ["AZURE_OPENAI_ENDPOINT"],
OptionalEnvironmentVariables = ["AZURE_OPENAI_DEPLOYMENT_NAME"],
MustContain = ["=== Writer-Critic Iteration Workflow ==="],
ExpectedOutputDescription =
[
"The output should show a writer-critic iteration workflow with writer and critic sections.",
"The critic should either approve or request revisions.",
"The output should not contain error messages or stack traces.",
],
},
// ───────────────────────────────────────────────────────────────────
// Agents
// ───────────────────────────────────────────────────────────────────
new SampleDefinition
{
Name = "Workflow_Agents_CustomAgentExecutors",
ProjectPath = "samples/03-workflows/Agents/CustomAgentExecutors",
RequiredEnvironmentVariables = ["AZURE_OPENAI_ENDPOINT"],
OptionalEnvironmentVariables = ["AZURE_OPENAI_DEPLOYMENT_NAME"],
ExpectedOutputDescription =
[
"The output should show custom workflow events including slogan generation and feedback.",
"The output should not contain error messages or stack traces.",
],
},
new SampleDefinition
{
Name = "Workflow_Agents_FoundryAgent",
ProjectPath = "samples/03-workflows/Agents/FoundryAgent",
RequiredEnvironmentVariables = ["AZURE_AI_PROJECT_ENDPOINT"],
OptionalEnvironmentVariables = ["AZURE_AI_MODEL_DEPLOYMENT_NAME"],
SkipReason = "Requires Azure AI Foundry project endpoint.",
},
new SampleDefinition
{
Name = "Workflow_Agents_GroupChatToolApproval",
ProjectPath = "samples/03-workflows/Agents/GroupChatToolApproval",
RequiredEnvironmentVariables = ["AZURE_OPENAI_ENDPOINT"],
OptionalEnvironmentVariables = ["AZURE_OPENAI_DEPLOYMENT_NAME"],
MustContain = ["Starting group chat workflow for software deployment..."],
ExpectedOutputDescription =
[
"The output should show a group chat workflow with QA and DevOps agents for software deployment.",
"There should be approval requests for tool calls.",
"The workflow should show interaction between QA and DevOps agents toward deployment.",
"The output should not contain error messages or stack traces.",
],
},
new SampleDefinition
{
Name = "Workflow_Agents_WorkflowAsAnAgent",
ProjectPath = "samples/03-workflows/Agents/WorkflowAsAnAgent",
RequiredEnvironmentVariables = ["AZURE_OPENAI_ENDPOINT"],
OptionalEnvironmentVariables = ["AZURE_OPENAI_DEPLOYMENT_NAME"],
Inputs = ["hello", "exit"],
InputDelayMs = 5000,
ExpectedOutputDescription =
[
"The output should show a conversational workflow responding to the user's hello message.",
"The output should not contain error messages or stack traces.",
],
},
// ───────────────────────────────────────────────────────────────────
// Checkpoint
// ───────────────────────────────────────────────────────────────────
new SampleDefinition
{
Name = "Workflow_Checkpoint_CheckpointAndRehydrate",
ProjectPath = "samples/03-workflows/Checkpoint/CheckpointAndRehydrate",
RequiredEnvironmentVariables = [],
IsDeterministic = true,
MustContain =
[
"Workflow completed with result:",
"Number of checkpoints created:",
"Hydrating a new workflow instance from the 6th checkpoint.",
],
},
new SampleDefinition
{
Name = "Workflow_Checkpoint_CheckpointAndResume",
ProjectPath = "samples/03-workflows/Checkpoint/CheckpointAndResume",
RequiredEnvironmentVariables = [],
IsDeterministic = true,
MustContain =
[
"Workflow completed with result:",
"Number of checkpoints created:",
"Restoring from the 6th checkpoint.",
],
},
new SampleDefinition
{
Name = "Workflow_Checkpoint_CheckpointWithHumanInTheLoop",
ProjectPath = "samples/03-workflows/Checkpoint/CheckpointWithHumanInTheLoop",
RequiredEnvironmentVariables = [],
Inputs = ["50", "25", "40", "45", "42", "50", "25", "40", "45", "42"],
InputDelayMs = 1000,
MustContain = ["found in"],
ExpectedOutputDescription =
[
"The output should show a number guessing game with higher/lower hints that eventually reaches the correct number.",
"The output should demonstrate checkpoint save and restore behavior.",
],
},
// ───────────────────────────────────────────────────────────────────
// Concurrent
// ───────────────────────────────────────────────────────────────────
new SampleDefinition
{
Name = "Workflow_Concurrent_Concurrent",
ProjectPath = "samples/03-workflows/Concurrent/Concurrent",
RequiredEnvironmentVariables = ["AZURE_OPENAI_ENDPOINT"],
OptionalEnvironmentVariables = ["AZURE_OPENAI_DEPLOYMENT_NAME"],
ExpectedOutputDescription =
[
"The output should show results from concurrent agent processing.",
"The output should not contain error messages or stack traces.",
],
},
new SampleDefinition
{
Name = "Workflow_Concurrent_MapReduce",
ProjectPath = "samples/03-workflows/Concurrent/MapReduce",
RequiredEnvironmentVariables = [],
MustContain =
[
"=== RUNNING WORKFLOW ===",
],
},
// ───────────────────────────────────────────────────────────────────
// ConditionalEdges
// ───────────────────────────────────────────────────────────────────
new SampleDefinition
{
Name = "Workflow_ConditionalEdges_01_EdgeCondition",
ProjectPath = "samples/03-workflows/ConditionalEdges/01_EdgeCondition",
RequiredEnvironmentVariables = ["AZURE_OPENAI_ENDPOINT"],
OptionalEnvironmentVariables = ["AZURE_OPENAI_DEPLOYMENT_NAME"],
ExpectedOutputDescription =
[
"The output should show an email being classified as spam or not spam and processed accordingly.",
"The output should not contain error messages or stack traces.",
],
},
new SampleDefinition
{
Name = "Workflow_ConditionalEdges_02_SwitchCase",
ProjectPath = "samples/03-workflows/ConditionalEdges/02_SwitchCase",
RequiredEnvironmentVariables = ["AZURE_OPENAI_ENDPOINT"],
OptionalEnvironmentVariables = ["AZURE_OPENAI_DEPLOYMENT_NAME"],
ExpectedOutputDescription =
[
"The output should show an ambiguous email being classified as spam, not spam, or uncertain.",
"The output should not contain error messages or stack traces.",
],
},
new SampleDefinition
{
Name = "Workflow_ConditionalEdges_03_MultiSelection",
ProjectPath = "samples/03-workflows/ConditionalEdges/03_MultiSelection",
RequiredEnvironmentVariables = ["AZURE_OPENAI_ENDPOINT"],
OptionalEnvironmentVariables = ["AZURE_OPENAI_DEPLOYMENT_NAME"],
ExpectedOutputDescription =
[
"The output should show an email being classified and potentially routed to multiple handlers.",
"The output should not contain error messages or stack traces.",
],
},
// ───────────────────────────────────────────────────────────────────
// HumanInTheLoop
// ───────────────────────────────────────────────────────────────────
new SampleDefinition
{
Name = "Workflow_HumanInTheLoop_Basic",
ProjectPath = "samples/03-workflows/HumanInTheLoop/HumanInTheLoopBasic",
RequiredEnvironmentVariables = [],
Inputs = ["50", "25", "40", "45", "42"],
InputDelayMs = 1000,
MustContain = ["found in"],
ExpectedOutputDescription =
[
"The output should show a number guessing game with higher/lower hints that eventually reaches the correct number 42.",
],
},
// ───────────────────────────────────────────────────────────────────
// Loop
// ───────────────────────────────────────────────────────────────────
new SampleDefinition
{
Name = "Workflow_Loop",
ProjectPath = "samples/03-workflows/Loop",
RequiredEnvironmentVariables = [],
MustContain = ["Result:"],
},
// ───────────────────────────────────────────────────────────────────
// SharedStates
// ───────────────────────────────────────────────────────────────────
new SampleDefinition
{
Name = "Workflow_SharedStates",
ProjectPath = "samples/03-workflows/SharedStates",
RequiredEnvironmentVariables = [],
IsDeterministic = true,
MustContain =
[
"Total Paragraphs:",
"Total Words:",
],
},
// ───────────────────────────────────────────────────────────────────
// Visualization
// ───────────────────────────────────────────────────────────────────
new SampleDefinition
{
Name = "Workflow_Visualization",
ProjectPath = "samples/03-workflows/Visualization",
RequiredEnvironmentVariables = [],
IsDeterministic = true,
MustContain =
[
"Generating workflow visualization...",
"Mermaid string:",
"DiGraph string:",
],
},
// ───────────────────────────────────────────────────────────────────
// Observability
// ───────────────────────────────────────────────────────────────────
new SampleDefinition
{
Name = "Workflow_Observability_ApplicationInsights",
ProjectPath = "samples/03-workflows/Observability/ApplicationInsights",
RequiredEnvironmentVariables = ["APPLICATIONINSIGHTS_CONNECTION_STRING"],
SkipReason = "Requires Application Insights connection string.",
},
new SampleDefinition
{
Name = "Workflow_Observability_AspireDashboard",
ProjectPath = "samples/03-workflows/Observability/AspireDashboard",
RequiredEnvironmentVariables = [],
SkipReason = "Requires Aspire Dashboard / OTLP endpoint.",
},
new SampleDefinition
{
Name = "Workflow_Observability_WorkflowAsAnAgent",
ProjectPath = "samples/03-workflows/Observability/WorkflowAsAnAgent",
RequiredEnvironmentVariables = ["AZURE_OPENAI_ENDPOINT"],
OptionalEnvironmentVariables = ["AZURE_OPENAI_DEPLOYMENT_NAME"],
SkipReason = "Interactive console with ReadLine loop; requires OTLP endpoint.",
},
// ───────────────────────────────────────────────────────────────────
// Declarative
// ───────────────────────────────────────────────────────────────────
new SampleDefinition
{
Name = "Workflow_Declarative_ConfirmInput",
ProjectPath = "samples/03-workflows/Declarative/ConfirmInput",
RequiredEnvironmentVariables = ["AZURE_AI_PROJECT_ENDPOINT"],
Inputs = ["hello", "hello"],
InputDelayMs = 8000,
ExpectedOutputDescription = ["The output should show a confirmation prompt and a user response."],
},
new SampleDefinition
{
Name = "Workflow_Declarative_CustomerSupport",
ProjectPath = "samples/03-workflows/Declarative/CustomerSupport",
RequiredEnvironmentVariables = ["AZURE_AI_PROJECT_ENDPOINT"],
OptionalEnvironmentVariables = ["AZURE_AI_MODEL_DEPLOYMENT_NAME"],
Inputs = ["My laptop won't start"],
InputDelayMs = 3000,
ExpectedOutputDescription = ["The output should show a customer support workflow processing a laptop issue, with agent responses providing troubleshooting or support."],
},
new SampleDefinition
{
Name = "Workflow_Declarative_DeepResearch",
ProjectPath = "samples/03-workflows/Declarative/DeepResearch",
RequiredEnvironmentVariables = ["AZURE_AI_PROJECT_ENDPOINT"],
OptionalEnvironmentVariables = ["AZURE_AI_MODEL_DEPLOYMENT_NAME"],
SkipReason = "Requires external weather API (wttr.in).",
},
new SampleDefinition
{
Name = "Workflow_Declarative_ExecuteCode",
ProjectPath = "samples/03-workflows/Declarative/ExecuteCode",
RequiredEnvironmentVariables = ["AZURE_AI_PROJECT_ENDPOINT"],
Inputs = ["What is 12 * 34?"],
InputDelayMs = 5000,
ExpectedOutputDescription = ["The output should show a declarative workflow executing generated code, processing a math question and producing a result."],
},
new SampleDefinition
{
Name = "Workflow_Declarative_ExecuteWorkflow",
ProjectPath = "samples/03-workflows/Declarative/ExecuteWorkflow",
RequiredEnvironmentVariables = ["AZURE_AI_PROJECT_ENDPOINT"],
SkipReason = "Requires a workflow file path as a CLI argument.",
},
new SampleDefinition
{
Name = "Workflow_Declarative_FunctionTools",
ProjectPath = "samples/03-workflows/Declarative/FunctionTools",
RequiredEnvironmentVariables = ["AZURE_AI_PROJECT_ENDPOINT"],
OptionalEnvironmentVariables = ["AZURE_AI_MODEL_DEPLOYMENT_NAME"],
Inputs = ["What are today's specials?", "EXIT"],
InputDelayMs = 8000,
ExpectedOutputDescription = ["The output should show a workflow calling function tools (e.g. a menu plugin) to answer a question about restaurant specials."],
},
new SampleDefinition
{
Name = "Workflow_Declarative_GenerateCode",
ProjectPath = "samples/03-workflows/Declarative/GenerateCode",
IsDeterministic = true,
MustContain = ["WORKFLOW: Parsing", "WORKFLOW: Defined"],
ExpectedOutputDescription = ["The output should show a YAML workflow being parsed and C# code being generated from it."],
},
new SampleDefinition
{
Name = "Workflow_Declarative_HostedWorkflow",
ProjectPath = "samples/03-workflows/Declarative/HostedWorkflow",
RequiredEnvironmentVariables = ["AZURE_AI_PROJECT_ENDPOINT"],
SkipReason = "Hosts a persistent workflow server that does not exit.",
},
new SampleDefinition
{
Name = "Workflow_Declarative_InputArguments",
ProjectPath = "samples/03-workflows/Declarative/InputArguments",
RequiredEnvironmentVariables = ["AZURE_AI_PROJECT_ENDPOINT"],
OptionalEnvironmentVariables = ["AZURE_AI_MODEL_DEPLOYMENT_NAME"],
Inputs = ["I'd like to visit Seattle", "EXIT"],
InputDelayMs = 8000,
ExpectedOutputDescription = ["The output should show a workflow capturing location input and providing travel-related information about Seattle."],
},
new SampleDefinition
{
Name = "Workflow_Declarative_InvokeFunctionTool",
ProjectPath = "samples/03-workflows/Declarative/InvokeFunctionTool",
RequiredEnvironmentVariables = ["AZURE_AI_PROJECT_ENDPOINT"],
OptionalEnvironmentVariables = ["AZURE_AI_MODEL_DEPLOYMENT_NAME"],
Inputs = ["What's the soup of the day?", "EXIT"],
InputDelayMs = 8000,
ExpectedOutputDescription = ["The output should show a workflow invoking a function tool (e.g. a menu plugin) to answer a question about the soup of the day."],
},
new SampleDefinition
{
Name = "Workflow_Declarative_InvokeMcpTool",
ProjectPath = "samples/03-workflows/Declarative/InvokeMcpTool",
RequiredEnvironmentVariables = ["AZURE_AI_PROJECT_ENDPOINT"],
OptionalEnvironmentVariables = ["AZURE_AI_MODEL_DEPLOYMENT_NAME"],
Inputs = ["Search for .NET tutorials on Microsoft Learn"],
InputDelayMs = 3000,
ExpectedOutputDescription = ["The output should show a workflow using MCP tools to search Microsoft Learn documentation and provide a summary of results."],
},
new SampleDefinition
{
Name = "Workflow_Declarative_Marketing",
ProjectPath = "samples/03-workflows/Declarative/Marketing",
RequiredEnvironmentVariables = ["AZURE_AI_PROJECT_ENDPOINT"],
OptionalEnvironmentVariables = ["AZURE_AI_MODEL_DEPLOYMENT_NAME"],
Inputs = ["A smart water bottle that tracks hydration"],
InputDelayMs = 3000,
ExpectedOutputDescription = ["The output should show a marketing workflow generating content about a smart water bottle product."],
},
new SampleDefinition
{
Name = "Workflow_Declarative_StudentTeacher",
ProjectPath = "samples/03-workflows/Declarative/StudentTeacher",
RequiredEnvironmentVariables = ["AZURE_AI_PROJECT_ENDPOINT"],
OptionalEnvironmentVariables = ["AZURE_AI_MODEL_DEPLOYMENT_NAME"],
Inputs = ["What is 18 + 27?"],
InputDelayMs = 3000,
ExpectedOutputDescription = ["The output should show a student-teacher workflow where a student asks a math question and a teacher provides the answer."],
},
new SampleDefinition
{
Name = "Workflow_Declarative_ToolApproval",
ProjectPath = "samples/03-workflows/Declarative/ToolApproval",
RequiredEnvironmentVariables = ["AZURE_AI_PROJECT_ENDPOINT"],
OptionalEnvironmentVariables = ["AZURE_AI_MODEL_DEPLOYMENT_NAME"],
Inputs = ["Search for .NET tutorials", "EXIT"],
InputDelayMs = 8000,
ExpectedOutputDescription = ["The output should show a workflow using an MCP tool with approval to search Microsoft Learn, followed by an exit from the input loop."],
},
];
}
@@ -0,0 +1,24 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFrameworks>net10.0</TargetFrameworks>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<IsPackable>false</IsPackable>
<IsAotCompatible>false</IsAotCompatible>
<!-- This is a top-level console app; ConfigureAwait is unnecessary -->
<NoWarn>$(NoWarn);CA2007</NoWarn>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Azure.AI.OpenAI" />
<PackageReference Include="Azure.Identity" />
<PackageReference Include="Microsoft.Extensions.AI.OpenAI" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\..\src\Microsoft.Agents.AI.OpenAI\Microsoft.Agents.AI.OpenAI.csproj" />
</ItemGroup>
</Project>
@@ -23,7 +23,7 @@ const string SourceName = "OpenTelemetryAspire.ConsoleApp";
const string ServiceName = "AgentOpenTelemetry";
// Configure OpenTelemetry for Aspire dashboard
var otlpEndpoint = Environment.GetEnvironmentVariable("OTEL_EXPORTER_OTLP_ENDPOINT") ?? "http://localhost:4318";
var otlpEndpoint = Environment.GetEnvironmentVariable("OTEL_EXPORTER_OTLP_ENDPOINT") ?? "http://localhost:4317";
var applicationInsightsConnectionString = Environment.GetEnvironmentVariable("APPLICATIONINSIGHTS_CONNECTION_STRING");
@@ -5,8 +5,8 @@ This sample demonstrates how to create an AIAgent using Anthropic Claude models
The sample supports three deployment scenarios:
1. **Anthropic Public API** - Direct connection to Anthropic's public API
2. **Azure Foundry with API Key** - Anthropic models deployed through Azure Foundry using API key authentication
3. **Azure Foundry with Azure CLI** - Anthropic models deployed through Azure Foundry using Azure CLI credentials
2. **Microsoft Foundry with API Key** - Anthropic models deployed through Microsoft Foundry using API key authentication
3. **Microsoft Foundry with Azure CLI** - Anthropic models deployed through Microsoft Foundry using Azure CLI credentials
## Prerequisites
@@ -25,29 +25,29 @@ $env:ANTHROPIC_API_KEY="your-anthropic-api-key" # Replace with your Anthropic A
$env:ANTHROPIC_CHAT_MODEL_NAME="claude-haiku-4-5" # Optional, defaults to claude-haiku-4-5
```
### For Azure Foundry with API Key
### For Microsoft Foundry with API Key
- Azure Foundry service endpoint and deployment configured
- Microsoft Foundry service endpoint and deployment configured
- Anthropic API key
Set the following environment variables:
```powershell
$env:ANTHROPIC_RESOURCE="your-foundry-resource-name" # Replace with your Azure Foundry resource name (subdomain before .services.ai.azure.com)
$env:ANTHROPIC_RESOURCE="your-foundry-resource-name" # Replace with your Microsoft Foundry resource name (subdomain before .services.ai.azure.com)
$env:ANTHROPIC_API_KEY="your-anthropic-api-key" # Replace with your Anthropic API key
$env:ANTHROPIC_CHAT_MODEL_NAME="claude-haiku-4-5" # Optional, defaults to claude-haiku-4-5
```
### For Azure Foundry with Azure CLI
### For Microsoft Foundry with Azure CLI
- Azure Foundry service endpoint and deployment configured
- Microsoft Foundry service endpoint and deployment configured
- Azure CLI installed and authenticated (for Azure credential authentication)
Set the following environment variables:
```powershell
$env:ANTHROPIC_RESOURCE="your-foundry-resource-name" # Replace with your Azure Foundry resource name (subdomain before .services.ai.azure.com)
$env:ANTHROPIC_RESOURCE="your-foundry-resource-name" # Replace with your Microsoft Foundry resource name (subdomain before .services.ai.azure.com)
$env:ANTHROPIC_CHAT_MODEL_NAME="claude-haiku-4-5" # Optional, defaults to claude-haiku-4-5
```
**Note**: When using Azure Foundry with Azure CLI, make sure you're logged in with `az login` and have access to the Azure Foundry resource. For more information, see the [Azure CLI documentation](https://learn.microsoft.com/cli/azure/authenticate-azure-cli-interactively).
**Note**: When using Microsoft Foundry with Azure CLI, make sure you're logged in with `az login` and have access to the Microsoft Foundry resource. For more information, see the [Azure CLI documentation](https://learn.microsoft.com/cli/azure/authenticate-azure-cli-interactively).
@@ -2,7 +2,7 @@
#pragma warning disable CS0618 // Type or member is obsolete - sample uses deprecated PersistentAgentsClientExtensions
// This sample shows how to create and use a simple AI agent with Azure Foundry Agents as the backend.
// This sample shows how to create and use a simple AI agent with Microsoft Foundry Agents as the backend.
using Azure.AI.Agents.Persistent;
using Azure.Identity;
@@ -13,14 +13,14 @@ Below is a comparison between the classic and new Foundry Agents approaches:
Before you begin, ensure you have the following prerequisites:
- .NET 10 SDK or later
- Azure Foundry service endpoint and deployment configured
- Microsoft Foundry service endpoint and deployment configured
- Azure CLI installed and authenticated (for Azure credential authentication)
**Note**: This demo uses Azure CLI credentials for authentication. Make sure you're logged in with `az login` and have access to the Azure Foundry resource. For more information, see the [Azure CLI documentation](https://learn.microsoft.com/cli/azure/authenticate-azure-cli-interactively).
**Note**: This demo uses Azure CLI credentials for authentication. Make sure you're logged in with `az login` and have access to the Microsoft Foundry resource. For more information, see the [Azure CLI documentation](https://learn.microsoft.com/cli/azure/authenticate-azure-cli-interactively).
Set the following environment variables:
```powershell
$env:AZURE_AI_PROJECT_ENDPOINT="https://your-foundry-service.services.ai.azure.com/api/projects/your-foundry-project" # Replace with your Azure Foundry resource endpoint
$env:AZURE_AI_PROJECT_ENDPOINT="https://your-foundry-service.services.ai.azure.com/api/projects/your-foundry-project" # Replace with your Microsoft Foundry resource endpoint
$env:AZURE_AI_MODEL_DEPLOYMENT_NAME="gpt-4o-mini" # Optional, defaults to gpt-4o-mini
```
@@ -1,6 +1,6 @@
// Copyright (c) Microsoft. All rights reserved.
// This sample shows how to create and use a AI agents with Azure Foundry Agents as the backend.
// This sample shows how to create and use AI agents with Microsoft Foundry Agents as the backend.
using Azure.AI.Projects;
using Azure.AI.Projects.Agents;
@@ -13,7 +13,7 @@ var deploymentName = Environment.GetEnvironmentVariable("AZURE_AI_MODEL_DEPLOYME
const string JokerName = "JokerAgent";
// Get a client to create/retrieve/delete server side agents with Azure Foundry Agents.
// Get a client to create/retrieve/delete server side agents with Microsoft Foundry Agents.
// WARNING: DefaultAzureCredential is convenient for development but requires careful consideration in production.
// In production, consider using a specific credential (e.g., ManagedIdentityCredential) to avoid
// latency issues, unintended credential probing, and potential security risks from fallback mechanisms.
@@ -13,14 +13,14 @@ Below is a comparison between the classic and new Foundry Agents approaches:
Before you begin, ensure you have the following prerequisites:
- .NET 10 SDK or later
- Azure Foundry service endpoint and deployment configured
- Microsoft Foundry service endpoint and deployment configured
- Azure CLI installed and authenticated (for Azure credential authentication)
**Note**: This demo uses Azure CLI credentials for authentication. Make sure you're logged in with `az login` and have access to the Azure Foundry resource. For more information, see the [Azure CLI documentation](https://learn.microsoft.com/cli/azure/authenticate-azure-cli-interactively).
**Note**: This demo uses Azure CLI credentials for authentication. Make sure you're logged in with `az login` and have access to the Microsoft Foundry resource. For more information, see the [Azure CLI documentation](https://learn.microsoft.com/cli/azure/authenticate-azure-cli-interactively).
Set the following environment variables:
```powershell
$env:AZURE_AI_PROJECT_ENDPOINT="https://your-foundry-service.services.ai.azure.com/api/projects/your-foundry-project" # Replace with your Azure Foundry resource endpoint
$env:AZURE_AI_PROJECT_ENDPOINT="https://your-foundry-service.services.ai.azure.com/api/projects/your-foundry-project" # Replace with your Microsoft Foundry resource endpoint
$env:AZURE_AI_MODEL_DEPLOYMENT_NAME="gpt-4o-mini" # Optional, defaults to gpt-4o-mini
```
@@ -1,7 +1,7 @@
// Copyright (c) Microsoft. All rights reserved.
// This sample shows how to use the OpenAI SDK to create and use a simple AI agent with any model hosted in Azure AI Foundry.
// You could use models from Microsoft, OpenAI, DeepSeek, Hugging Face, Meta, xAI or any other model you have deployed in your Azure AI Foundry resource.
// This sample shows how to use the OpenAI SDK to create and use a simple AI agent with any model hosted in Microsoft Foundry.
// You could use models from Microsoft, OpenAI, DeepSeek, Hugging Face, Meta, xAI or any other model you have deployed in your Microsoft Foundry resource.
// Note: Ensure that you pick a model that suits your needs. For example, if you want to use function calling, ensure that the model you pick supports function calling.
using System.ClientModel;
@@ -15,7 +15,7 @@ var endpoint = Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT") ?? th
var apiKey = Environment.GetEnvironmentVariable("AZURE_OPENAI_API_KEY");
var model = Environment.GetEnvironmentVariable("AZURE_AI_MODEL_DEPLOYMENT_NAME") ?? "Phi-4-mini-instruct";
// Since we are using the OpenAI Client SDK, we need to override the default endpoint to point to Azure Foundry.
// Since we are using the OpenAI Client SDK, we need to override the default endpoint to point to Microsoft Foundry.
var clientOptions = new OpenAIClientOptions() { Endpoint = new Uri(endpoint) };
// Create the OpenAI client with either an API key or Azure CLI credential.
@@ -1,8 +1,8 @@
## Overview
This sample shows how to use the OpenAI SDK to create and use a simple AI agent with any model hosted in Azure AI Foundry.
This sample shows how to use the OpenAI SDK to create and use a simple AI agent with any model hosted in Microsoft Foundry.
You could use models from Microsoft, OpenAI, DeepSeek, Hugging Face, Meta, xAI or any other model you have deployed in Azure AI Foundry.
You could use models from Microsoft, OpenAI, DeepSeek, Hugging Face, Meta, xAI or any other model you have deployed in Microsoft Foundry.
**Note**: Ensure that you pick a model that suits your needs. For example, if you want to use function calling, ensure that the model you pick supports function calling.
@@ -11,19 +11,19 @@ You could use models from Microsoft, OpenAI, DeepSeek, Hugging Face, Meta, xAI o
Before you begin, ensure you have the following prerequisites:
- .NET 10 SDK or later
- Azure AI Foundry resource
- A model deployment in your Azure AI Foundry resource. This example defaults to using the `Phi-4-mini-instruct` model,
- Microsoft Foundry resource
- A model deployment in your Microsoft Foundry resource. This example defaults to using the `Phi-4-mini-instruct` model,
so if you want to use a different model, ensure that you set your `AZURE_AI_MODEL_DEPLOYMENT_NAME` environment
variable to the name of your deployed model.
- An API key or role based authentication to access the Azure AI Foundry resource
- An API key or role based authentication to access the Microsoft Foundry resource
See [here](https://learn.microsoft.com/en-us/azure/ai-foundry/quickstarts/get-started-code?tabs=csharp) for more info on setting up these prerequisites
Set the following environment variables:
```powershell
# Replace with your Azure AI Foundry resource endpoint
# Ensure that you have the "/openai/v1/" path in the URL, since this is required when using the OpenAI SDK to access Azure Foundry models.
# Replace with your Microsoft Foundry resource endpoint
# Ensure that you have the "/openai/v1/" path in the URL, since this is required when using the OpenAI SDK to access Microsoft Foundry models.
$env:AZURE_OPENAI_ENDPOINT="https://ai-foundry-<myresourcename>.services.ai.azure.com/openai/v1/"
# Optional, defaults to using Azure CLI for authentication if not provided
@@ -18,7 +18,7 @@ See the README.md for each sample for the prerequisites for that sample.
|[Creating an AIAgent with Anthropic](./Agent_With_Anthropic/)|This sample demonstrates how to create an AIAgent using Anthropic Claude models as the underlying inference service|
|[Creating an AIAgent with Foundry Agents using Azure.AI.Agents.Persistent](./Agent_With_AzureAIAgentsPersistent/)|This sample demonstrates how to create a Foundry Persistent agent and expose it as an AIAgent using the Azure.AI.Agents.Persistent SDK|
|[Creating an AIAgent with Foundry Agents using Azure.AI.Project](./Agent_With_AzureAIProject/)|This sample demonstrates how to create an Foundry Project agent and expose it as an AIAgent using the Azure.AI.Project SDK|
|[Creating an AIAgent with AzureFoundry Model](./Agent_With_AzureFoundryModel/)|This sample demonstrates how to use any model deployed to Azure Foundry to create an AIAgent|
|[Creating an AIAgent with Foundry Model](./Agent_With_AzureFoundryModel/)|This sample demonstrates how to use any model deployed to Microsoft Foundry to create an AIAgent|
|[Creating an AIAgent with Azure OpenAI ChatCompletion](./Agent_With_AzureOpenAIChatCompletion/)|This sample demonstrates how to create an AIAgent using Azure OpenAI ChatCompletion as the underlying inference service|
|[Creating an AIAgent with Azure OpenAI Responses](./Agent_With_AzureOpenAIResponses/)|This sample demonstrates how to create an AIAgent using Azure OpenAI Responses as the underlying inference service|
|[Creating an AIAgent with a custom implementation](./Agent_With_CustomImplementation/)|This sample demonstrates how to create an AIAgent with a custom implementation|
@@ -18,9 +18,9 @@ Before you begin, ensure you have the following prerequisites:
**Note**: These samples use Anthropic Claude models. For more information, see [Anthropic documentation](https://docs.anthropic.com/).
## Using Anthropic with Azure Foundry
## Using Anthropic with Microsoft Foundry
To use Anthropic with Azure Foundry, you can check the sample [AgentProviders/Agent_With_Anthropic](../AgentProviders/Agent_With_Anthropic/README.md) for more details.
To use Anthropic with Microsoft Foundry, you can check the sample [AgentProviders/Agent_With_Anthropic](../AgentProviders/Agent_With_Anthropic/README.md) for more details.
## Samples
@@ -1,10 +1,10 @@
// Copyright (c) Microsoft. All rights reserved.
// This sample shows how to use the FoundryMemoryProvider to persist and recall memories for an agent.
// The sample stores conversation messages in an Azure AI Foundry memory store and retrieves relevant
// The sample stores conversation messages in a Microsoft Foundry memory store and retrieves relevant
// memories for subsequent invocations, even across new sessions.
//
// Note: Memory extraction in Azure AI Foundry is asynchronous and takes time. This sample demonstrates
// Note: Memory extraction in Microsoft Foundry is asynchronous and takes time. This sample demonstrates
// a simple polling approach to wait for memory updates to complete before querying.
using System.Text.Json;
@@ -62,7 +62,7 @@ await memoryProvider.EnsureStoredMemoriesDeletedAsync(session);
Console.WriteLine(await agent.RunAsync("Hi there! My name is Taylor and I'm planning a hiking trip to Patagonia in November.", session));
Console.WriteLine(await agent.RunAsync("I'm travelling with my sister and we love finding scenic viewpoints.", session));
// Memory extraction in Azure AI Foundry is asynchronous and takes time to process.
// Memory extraction in Microsoft Foundry is asynchronous and takes time to process.
// WhenUpdatesCompletedAsync polls all pending updates and waits for them to complete.
Console.WriteLine("\nWaiting for Foundry Memory to process updates...");
await memoryProvider.WhenUpdatesCompletedAsync();
@@ -1,6 +1,6 @@
# Agent with Memory Using Azure AI Foundry
# Agent with Memory Using Microsoft Foundry
This sample demonstrates how to create and run an agent that uses Azure AI Foundry's managed memory service to extract and retrieve individual memories across sessions.
This sample demonstrates how to create and run an agent that uses Microsoft Foundry's managed memory service to extract and retrieve individual memories across sessions.
## Features Demonstrated
@@ -13,7 +13,7 @@ This sample demonstrates how to create and run an agent that uses Azure AI Found
## Prerequisites
1. Azure subscription with Azure AI Foundry project
1. Azure subscription with Microsoft Foundry project
2. Azure OpenAI resource with a chat model deployment (e.g., gpt-4o-mini) and an embedding model deployment (e.g., text-embedding-ada-002)
3. .NET 10.0 SDK
4. Azure CLI logged in (`az login`)
@@ -21,7 +21,7 @@ This sample demonstrates how to create and run an agent that uses Azure AI Found
## Environment Variables
```bash
# Azure AI Foundry project endpoint and memory store name
# Microsoft Foundry project endpoint and memory store name
export AZURE_AI_PROJECT_ENDPOINT="https://your-account.services.ai.azure.com/api/projects/your-project"
export AZURE_AI_MEMORY_STORE_ID="my_memory_store"
@@ -48,10 +48,10 @@ The agent will:
## Key Differences from Mem0
| Aspect | Mem0 | Azure AI Foundry Memory |
| Aspect | Mem0 | Microsoft Foundry Memory |
|--------|------|------------------------|
| Authentication | API Key | Azure Identity (DefaultAzureCredential) |
| Scope | ApplicationId, UserId, AgentId, ThreadId | Single `Scope` string |
| Memory Types | Single memory store | User Profile + Chat Summary |
| Hosting | Mem0 cloud or self-hosted | Azure AI Foundry managed service |
| Hosting | Mem0 cloud or self-hosted | Microsoft Foundry managed service |
| Store Creation | N/A (automatic) | Explicit via `EnsureMemoryStoreCreatedAsync` |
@@ -7,7 +7,7 @@ These samples show how to create an agent with the Agent Framework that uses Mem
|[Chat History memory](./AgentWithMemory_Step01_ChatHistoryMemory/)|This sample demonstrates how to enable an agent to remember messages from previous conversations.|
|[Memory with MemoryStore](./AgentWithMemory_Step02_MemoryUsingMem0/)|This sample demonstrates how to create and run an agent that uses the Mem0 service to extract and retrieve individual memories.|
|[Custom Memory Implementation](../../01-get-started/04_memory/)|This sample demonstrates how to create a custom memory component and attach it to an agent.|
|[Memory with Azure AI Foundry](./AgentWithMemory_Step04_MemoryUsingFoundry/)|This sample demonstrates how to create and run an agent that uses Azure AI Foundry's managed memory service to extract and retrieve individual memories.|
|[Memory with Microsoft Foundry](./AgentWithMemory_Step04_MemoryUsingFoundry/)|This sample demonstrates how to create and run an agent that uses Microsoft Foundry's managed memory service to extract and retrieve individual memories.|
|[Bounded Chat History with Overflow](./AgentWithMemory_Step05_BoundedChatHistory/)|This sample demonstrates how to create a bounded chat history provider that overflows older messages to a vector store and recalls them as memories.|
> **See also**: [Memory Search with Foundry Agents](../AgentsWithFoundry/Agent_Step22_MemorySearch/) - demonstrates using the built-in Memory Search tool with Azure Foundry agents.
> **See also**: [Memory Search with Foundry Agents](../AgentsWithFoundry/Agent_Step22_MemorySearch/) - demonstrates using the built-in Memory Search tool with Microsoft Foundry agents.
@@ -13,7 +13,7 @@ This sample uses Qdrant for the vector store, but this can easily be swapped out
- User has the `Cognitive Services OpenAI Contributor` role for the Azure OpenAI resource.
- An existing Qdrant instance. You can use a managed service or run a local instance using Docker, but the sample assumes the instance is running locally.
**Note**: These samples use Azure OpenAI models. For more information, see [how to deploy Azure OpenAI models with Azure AI Foundry](https://learn.microsoft.com/en-us/azure/ai-foundry/how-to/deploy-models-openai).
**Note**: These samples use Azure OpenAI models. For more information, see [how to deploy Azure OpenAI models with Microsoft Foundry](https://learn.microsoft.com/en-us/azure/ai-foundry/how-to/deploy-models-openai).
**Note**: These samples use Azure CLI credentials for authentication. Make sure you're logged in with `az login` and have access to the Azure OpenAI resource and have the `Cognitive Services OpenAI Contributor` role. For more information, see the [Azure CLI documentation](https://learn.microsoft.com/cli/azure/authenticate-azure-cli-interactively).
@@ -0,0 +1,54 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFrameworks>net10.0</TargetFrameworks>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<ManagePackageVersionsCentrally>false</ManagePackageVersionsCentrally>
</PropertyGroup>
<ItemGroup>
<PackageReference Remove="Microsoft.CodeAnalysis.NetAnalyzers" />
<PackageReference Remove="Microsoft.VisualStudio.Threading.Analyzers" />
<PackageReference Remove="xunit.analyzers" />
<PackageReference Remove="Moq.Analyzers" />
<PackageReference Remove="Roslynator.Analyzers" />
<PackageReference Remove="Roslynator.CodeAnalysis.Analyzers" />
<PackageReference Remove="Roslynator.Formatting.Analyzers" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="Azure.AI.OpenAI" Version="2.9.0-beta.1" />
<PackageReference Include="Azure.Identity" Version="1.19.0" />
<PackageReference Include="Microsoft.Agents.AI.OpenAI" Version="1.0.0-rc4" />
<PackageReference Include="Microsoft.Extensions.AI.OpenAI" Version="10.4.0" />
<PackageReference Include="Neo4j.AgentFramework.GraphRAG" Version="0.1.0-preview.2" />
<PackageReference Include="Neo4j.Driver" Version="5.28.0" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="Microsoft.CodeAnalysis.NetAnalyzers" Version="10.0.100">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
<PackageReference Include="Microsoft.VisualStudio.Threading.Analyzers" Version="17.14.15">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
<PackageReference Include="Roslynator.Analyzers" Version="4.14.1">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
<PackageReference Include="Roslynator.CodeAnalysis.Analyzers" Version="4.14.1">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
<PackageReference Include="Roslynator.Formatting.Analyzers" Version="4.14.1">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
</ItemGroup>
</Project>
@@ -0,0 +1,77 @@
// Copyright (c) Microsoft. All rights reserved.
using Azure.AI.OpenAI;
using Azure.Identity;
using Microsoft.Agents.AI;
using Microsoft.Extensions.AI;
using Neo4j.AgentFramework.GraphRAG;
using Neo4j.Driver;
var endpoint = Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT") ?? throw new InvalidOperationException("AZURE_OPENAI_ENDPOINT is not set.");
var deploymentName = Environment.GetEnvironmentVariable("AZURE_OPENAI_DEPLOYMENT_NAME") ?? "gpt-4o-mini";
var neo4jUri = Environment.GetEnvironmentVariable("NEO4J_URI") ?? throw new InvalidOperationException("NEO4J_URI is not set.");
var neo4jUsername = Environment.GetEnvironmentVariable("NEO4J_USERNAME") ?? "neo4j";
var neo4jPassword = Environment.GetEnvironmentVariable("NEO4J_PASSWORD") ?? throw new InvalidOperationException("NEO4J_PASSWORD is not set.");
var fulltextIndex = Environment.GetEnvironmentVariable("NEO4J_FULLTEXT_INDEX_NAME") ?? "search_chunks";
const string RetrievalQuery = """
MATCH (node)-[:FROM_DOCUMENT]->(doc:Document)<-[:FILED]-(company:Company)
OPTIONAL MATCH (company)-[:FACES_RISK]->(risk:RiskFactor)
WITH node, score, company, doc, collect(DISTINCT risk.name)[0..5] AS risks
OPTIONAL MATCH (company)-[:MENTIONS]->(product:Product)
WITH node, score, company, doc, risks, collect(DISTINCT product.name)[0..5] AS products
RETURN
node.text AS text,
score,
company.name AS company,
company.ticker AS ticker,
doc.title AS title,
risks,
products
ORDER BY score DESC
""";
await using var driver = GraphDatabase.Driver(new Uri(neo4jUri), AuthTokens.Basic(neo4jUsername, neo4jPassword));
await driver.VerifyConnectivityAsync();
await using var provider = new Neo4jContextProvider(
driver,
new Neo4jContextProviderOptions
{
IndexName = fulltextIndex,
IndexType = IndexType.Fulltext,
RetrievalQuery = RetrievalQuery,
TopK = 5,
ContextPrompt = "Use the retrieved Neo4j graph context to answer accurately and call out when context is missing."
});
// WARNING: DefaultAzureCredential is convenient for development but requires careful consideration in production.
// In production, consider using a specific credential (e.g., ManagedIdentityCredential) to avoid
// latency issues, unintended credential probing, and potential security risks from fallback mechanisms.
AIAgent agent = new AzureOpenAIClient(
new Uri(endpoint),
new DefaultAzureCredential())
.GetChatClient(deploymentName)
.AsIChatClient()
.AsAIAgent(new ChatClientAgentOptions
{
ChatOptions = new()
{
Instructions = "You are a helpful assistant that answers questions using Neo4j graph context."
},
AIContextProviders = [provider]
});
AgentSession session = await agent.CreateSessionAsync();
foreach (var question in new[]
{
"What products does Microsoft offer?",
"What risks does Apple face?",
"Tell me about NVIDIA's AI business and risk factors."
})
{
Console.WriteLine($">> {question}\n");
Console.WriteLine(await agent.RunAsync(question, session));
Console.WriteLine();
}
@@ -0,0 +1,32 @@
# Agent Framework Retrieval Augmented Generation (RAG) with Neo4j GraphRAG
This sample demonstrates how to create and run an agent that uses the [Neo4j GraphRAG context provider](https://github.com/neo4j-labs/neo4j-maf-provider) with Microsoft Agent Framework for .NET.
The sample uses a Neo4j fulltext index for retrieval and a Cypher `RetrievalQuery` to enrich results with related companies, products, and risk factors.
## Prerequisites
- .NET 10 SDK or later
- Azure OpenAI endpoint and chat deployment
- Azure CLI installed and authenticated
- A Neo4j database with chunked documents and a fulltext index such as `search_chunks`
## Environment variables
```powershell
$env:AZURE_OPENAI_ENDPOINT="https://your-resource.openai.azure.com/"
$env:AZURE_OPENAI_DEPLOYMENT_NAME="gpt-4o-mini"
$env:NEO4J_URI="neo4j+s://your-instance.databases.neo4j.io"
$env:NEO4J_USERNAME="neo4j"
$env:NEO4J_PASSWORD="your-password"
$env:NEO4J_FULLTEXT_INDEX_NAME="search_chunks"
```
## Build and run
```powershell
dotnet build
dotnet run --framework net10.0 --no-build
```
The sample issues a few questions against the graph-backed retrieval provider and prints the responses to the console.
@@ -8,3 +8,4 @@ These samples show how to create an agent with the Agent Framework that uses Ret
|[RAG with Vector Store and custom schema](./AgentWithRAG_Step02_CustomVectorStoreRAG/)|This sample demonstrates how to create and run an agent that uses Retrieval Augmented Generation (RAG) with a vector store. It also uses a custom schema for the documents stored in the vector store.|
|[RAG with custom RAG data source](./AgentWithRAG_Step03_CustomRAGDataSource/)|This sample demonstrates how to create and run an agent that uses Retrieval Augmented Generation (RAG) with a custom RAG data source.|
|[RAG with Foundry VectorStore service](./AgentWithRAG_Step04_FoundryServiceRAG/)|This sample demonstrates how to create and run an agent that uses Retrieval Augmented Generation (RAG) with the Foundry VectorStore service.|
|[RAG with Neo4j GraphRAG](./AgentWithRAG_Step05_Neo4jGraphRAG/)|This sample demonstrates how to create and run an agent that uses a Neo4j-backed GraphRAG context provider with graph-enriched retrieval.|
@@ -18,7 +18,7 @@ Before you begin, ensure you have the following prerequisites:
- Azure CLI installed and authenticated (for Azure credential authentication)
- User has the `Cognitive Services OpenAI Contributor` role for the Azure OpenAI resource
**Note**: This sample uses Azure OpenAI models. For more information, see [how to deploy Azure OpenAI models with Azure AI Foundry](https://learn.microsoft.com/en-us/azure/ai-foundry/how-to/deploy-models-openai).
**Note**: This sample uses Azure OpenAI models. For more information, see [how to deploy Azure OpenAI models with Microsoft Foundry](https://learn.microsoft.com/en-us/azure/ai-foundry/how-to/deploy-models-openai).
**Note**: This demo uses Azure CLI credentials for authentication. Make sure you're logged in with `az login` and have access to the Azure OpenAI resource and have the `Cognitive Services OpenAI Contributor` role. For more information, see the [Azure CLI documentation](https://learn.microsoft.com/cli/azure/authenticate-azure-cli-interactively).
@@ -20,8 +20,8 @@ To use the [MCP Inspector](https://modelcontextprotocol.io/docs/tools/inspector)
MCP Inspector is up and running at http://127.0.0.1:6274
```
1. Open a web browser and navigate to the URL displayed in the terminal. If not opened automatically, this will open the MCP Inspector interface.
1. In the MCP Inspector interface, add the following environment variables to allow your MCP server to access Azure AI Foundry Project to create and run the agent:
- AZURE_AI_PROJECT_ENDPOINT = https://your-resource.openai.azure.com/ # Replace with your Azure AI Foundry Project endpoint
1. In the MCP Inspector interface, add the following environment variables to allow your MCP server to access Microsoft Foundry Project to create and run the agent:
- AZURE_AI_PROJECT_ENDPOINT = https://your-resource.openai.azure.com/ # Replace with your Microsoft Foundry Project endpoint
- AZURE_AI_MODEL_DEPLOYMENT_NAME = gpt-4o-mini # Replace with your model deployment name
1. Find and click the `Connect` button in the MCP Inspector interface to connect to the MCP server.
1. As soon as the connection is established, open the `Tools` tab in the MCP Inspector interface and select the `Joker` tool from the list.
@@ -13,7 +13,7 @@ using Azure.Identity;
using Microsoft.Agents.AI;
using Microsoft.Extensions.AI;
// Get Azure AI Foundry configuration from environment variables
// Get Microsoft Foundry configuration from environment variables
var endpoint = Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT") ?? throw new InvalidOperationException("AZURE_OPENAI_ENDPOINT is not set.");
var deploymentName = System.Environment.GetEnvironmentVariable("AZURE_OPENAI_DEPLOYMENT_NAME") ?? "gpt-4o";
@@ -3,7 +3,7 @@
// This sample shows how to use a chat history reducer to keep the context within model size limits.
// Any implementation of Microsoft.Extensions.AI.IChatReducer can be used to customize how the chat history is reduced.
// NOTE: this feature is only supported where the chat history is stored locally, such as with OpenAI Chat Completion.
// Where the chat history is stored server side, such as with Azure Foundry Agents, the service must manage the chat history size.
// Where the chat history is stored server side, such as with Microsoft Foundry Agents, the service must manage the chat history size.
using Azure.AI.OpenAI;
using Azure.Identity;
@@ -2,7 +2,7 @@
#pragma warning disable CS0618 // Type or member is obsolete - sample uses deprecated PersistentAgentsClientExtensions
// This sample shows how to create an Azure AI Foundry Agent with the Deep Research Tool.
// This sample shows how to create a Microsoft Foundry Agent with the Deep Research Tool.
using Azure.AI.Agents.Persistent;
using Azure.Identity;
@@ -11,10 +11,10 @@ Key features:
Before running this sample, ensure you have:
1. An Azure AI Foundry project set up
1. A Microsoft Foundry project set up
2. A deep research model deployment (e.g., o3-deep-research)
3. A model deployment (e.g., gpt-4o)
4. A Bing Connection configured in your Azure AI Foundry project
4. A Bing Connection configured in your Microsoft Foundry project
5. Azure CLI installed and authenticated
**Important**: Please visit the following documentation for detailed setup instructions:
@@ -29,14 +29,14 @@ Pay special attention to the purple `Note` boxes in the Azure documentation.
/subscriptions/<sub-id>/resourceGroups/<rg>/providers/Microsoft.CognitiveServices/accounts/<account>/projects/<project>/connections/<connection-name>
```
You can find this in the Azure AI Foundry portal under **Management > Connected resources**, or retrieve it programmatically via the connections API (`.id` property).
You can find this in the Microsoft Foundry portal under **Management > Connected resources**, or retrieve it programmatically via the connections API (`.id` property).
## Environment Variables
Set the following environment variables:
```powershell
# Replace with your Azure AI Foundry project endpoint
# Replace with your Microsoft Foundry project endpoint
$env:AZURE_AI_PROJECT_ENDPOINT="https://your-project.services.ai.azure.com/"
# Replace with your Bing Grounding connection ID (full ARM resource URI)
+1 -1
View File
@@ -18,7 +18,7 @@ Before you begin, ensure you have the following prerequisites:
- Azure CLI installed and authenticated (for Azure credential authentication)
- User has the `Cognitive Services OpenAI Contributor` role for the Azure OpenAI resource.
**Note**: These samples use Azure OpenAI models. For more information, see [how to deploy Azure OpenAI models with Azure AI Foundry](https://learn.microsoft.com/en-us/azure/ai-foundry/how-to/deploy-models-openai).
**Note**: These samples use Azure OpenAI models. For more information, see [how to deploy Azure OpenAI models with Microsoft Foundry](https://learn.microsoft.com/en-us/azure/ai-foundry/how-to/deploy-models-openai).
**Note**: These samples use Azure CLI credentials for authentication. Make sure you're logged in with `az login` and have access to the Azure OpenAI resource and have the `Cognitive Services OpenAI Contributor` role. For more information, see the [Azure CLI documentation](https://learn.microsoft.com/cli/azure/authenticate-azure-cli-interactively).
@@ -1,7 +1,7 @@
// Copyright (c) Microsoft. All rights reserved.
// This sample shows how to create, use, and clean up a FoundryAgent backed by a server-side
// versioned agent in Azure AI Foundry. It demonstrates the full lifecycle:
// versioned agent in Microsoft Foundry. It demonstrates the full lifecycle:
// create agent version -> wrap as FoundryAgent -> run -> delete.
using Azure.AI.Projects;
@@ -1,6 +1,6 @@
# Getting started with Foundry Agents
These samples demonstrate how to use Azure AI Foundry with Agent Framework.
These samples demonstrate how to use Microsoft Foundry with Agent Framework.
## Quick start
@@ -1,7 +1,7 @@
// Copyright (c) Microsoft. All rights reserved.
// This sample shows how to create and use a simple AI agent with Azure Foundry Agents as the backend, that uses a Hosted MCP Tool.
// In this case the Azure Foundry Agents service will invoke any MCP tools as required. MCP tools are not invoked by the Agent Framework.
// This sample shows how to create and use a simple AI agent with Microsoft Foundry Agents as the backend, that uses a Hosted MCP Tool.
// In this case the Microsoft Foundry Agents service will invoke any MCP tools as required. MCP tools are not invoked by the Agent Framework.
// The sample first shows how to use MCP tools with auto approval, and then how to set up a tool that requires approval before it can be invoked and how to approve such a tool.
using Azure.AI.Projects;
@@ -3,14 +3,14 @@
Before you begin, ensure you have the following prerequisites:
- .NET 10 SDK or later
- Azure Foundry service endpoint and deployment configured
- Microsoft Foundry service endpoint and deployment configured
- Azure CLI installed and authenticated (for Azure credential authentication)
**Note**: This demo uses Azure CLI credentials for authentication. Make sure you're logged in with `az login` and have access to the Azure Foundry resource. For more information, see the [Azure CLI documentation](https://learn.microsoft.com/cli/azure/authenticate-azure-cli-interactively).
**Note**: This demo uses Azure CLI credentials for authentication. Make sure you're logged in with `az login` and have access to the Microsoft Foundry resource. For more information, see the [Azure CLI documentation](https://learn.microsoft.com/cli/azure/authenticate-azure-cli-interactively).
Set the following environment variables:
```powershell
$env:AZURE_AI_PROJECT_ENDPOINT="https://your-foundry-service.services.ai.azure.com/api/projects/your-foundry-project" # Replace with your Azure Foundry resource endpoint
$env:AZURE_AI_PROJECT_ENDPOINT="https://your-foundry-service.services.ai.azure.com/api/projects/your-foundry-project" # Replace with your Microsoft Foundry resource endpoint
$env:AZURE_AI_MODEL_DEPLOYMENT_NAME="gpt-4.1-mini" # Optional, defaults to gpt-4.1-mini
```
@@ -11,7 +11,7 @@ Before you begin, ensure you have the following prerequisites:
- Azure CLI installed and authenticated (for Azure credential authentication)
- User has the `Cognitive Services OpenAI Contributor` role for the Azure OpenAI resource.
**Note**: These samples use Azure OpenAI models. For more information, see [how to deploy Azure OpenAI models with Azure AI Foundry](https://learn.microsoft.com/en-us/azure/ai-foundry/how-to/deploy-models-openai).
**Note**: These samples use Azure OpenAI models. For more information, see [how to deploy Azure OpenAI models with Microsoft Foundry](https://learn.microsoft.com/en-us/azure/ai-foundry/how-to/deploy-models-openai).
**Note**: These samples use Azure CLI credentials for authentication. Make sure you're logged in with `az login` and have access to the Azure OpenAI resource and have the `Cognitive Services OpenAI Contributor` role. For more information, see the [Azure CLI documentation](https://learn.microsoft.com/cli/azure/authenticate-azure-cli-interactively).
@@ -11,12 +11,12 @@ using Microsoft.Extensions.AI;
namespace WorkflowFoundryAgentSample;
/// <summary>
/// This sample shows how to use Azure Foundry Agents within a workflow.
/// This sample shows how to use Microsoft Foundry Agents within a workflow.
/// </summary>
/// <remarks>
/// Pre-requisites:
/// - Foundational samples should be completed first.
/// - An Azure Foundry project endpoint and model id.
/// - A Microsoft Foundry project endpoint and model ID.
/// </remarks>
public static class Program
{
@@ -30,7 +30,7 @@ namespace Demo.Workflows.Declarative.InvokeMcpTool;
/// <item>Integrating with MCP-compatible services</item>
/// </list>
/// <para>
/// This sample uses the Microsoft Learn MCP server to search Azure documentation and the Azure foundry MCP server to get AI model details.
/// This sample uses the Microsoft Learn MCP server to search Azure documentation and the Microsoft Foundry MCP server to get AI model details.
/// When you run the sample, provide an AI model (e.g. gpt-4.1-mini) as input,
/// The workflow will use the MCP tools to find relevant information about the model from Microsoft Learn and foundry, then an agent will summarize the results.
/// </para>
@@ -6,7 +6,7 @@ to build a `Workflow` that may be executed using the same pattern as any code-ba
## Configuration
These samples must be configured to create and use agents your
[Azure Foundry Project](https://learn.microsoft.com/azure/ai-foundry).
[Microsoft Foundry Project](https://learn.microsoft.com/azure/ai-foundry).
### Settings
@@ -18,9 +18,9 @@ The configuraton required by the samples is:
|Setting Name| Description|
|:--|:--|
|AZURE_AI_PROJECT_ENDPOINT| The endpoint URL of your Azure Foundry Project.|
|AZURE_AI_PROJECT_ENDPOINT| The endpoint URL of your Microsoft Foundry Project.|
|AZURE_AI_MODEL_DEPLOYMENT_NAME| The name of the model deployment to use
|AZURE_AI_BING_CONNECTION_ID| The name of the Bing Grounding connection configured in your Azure Foundry Project.|
|AZURE_AI_BING_CONNECTION_ID| The name of the Bing Grounding connection configured in your Microsoft Foundry Project.|
To set your secrets with .NET Secret Manager:
@@ -42,13 +42,13 @@ To set your secrets with .NET Secret Manager:
dotnet user-secrets init
```
4. Define setting that identifies your Azure Foundry Project (endpoint):
4. Define setting that identifies your Microsoft Foundry Project (endpoint):
```
dotnet user-secrets set "AZURE_AI_PROJECT_ENDPOINT" "https://..."
```
5. Define setting that identifies your Azure Foundry Model Deployment (endpoint):
5. Define setting that identifies your Microsoft Foundry Model Deployment (endpoint):
```
dotnet user-secrets set "AZURE_AI_MODEL_DEPLOYMENT_NAME" "gpt-5"
@@ -70,7 +70,7 @@ $env:AZURE_AI_BING_CONNECTION_ID="mybinggrounding"
### Authorization
Use [_Azure CLI_](https://learn.microsoft.com/cli/azure/authenticate-azure-cli) to authorize access to your Azure Foundry Project:
Use [_Azure CLI_](https://learn.microsoft.com/cli/azure/authenticate-azure-cli) to authorize access to your Microsoft Foundry Project:
```
az login
+1 -1
View File
@@ -26,7 +26,7 @@ Once completed, please proceed to the other samples listed below.
| Sample | Concepts |
|--------|----------|
| [Foundry Agents in Workflows](./Agents/FoundryAgent) | Demonstrates using Azure Foundry agents in a workflow through `ChatClientAgent` |
| [Foundry Agents in Workflows](./Agents/FoundryAgent) | Demonstrates using Microsoft Foundry agents in a workflow through `ChatClientAgent` |
| [Custom Agent Executors](./Agents/CustomAgentExecutors) | Shows how to create a custom agent executor for more complex scenarios |
| [Workflow as an Agent](./Agents/WorkflowAsAnAgent) | Illustrates how to encapsulate a workflow as an agent |
| [Group Chat with Tool Approval](./Agents/GroupChatToolApproval) | Shows multi-agent group chat with tool approval requests and human-in-the-loop interaction |
@@ -51,7 +51,7 @@ dotnet run --urls "http://localhost:5002;https://localhost:5012" --agentType "lo
### Configuring for use with Azure AI Agents
You must create the agents in an Azure AI Foundry project and then provide the project endpoint and agents ids. The instructions for each agent are as follows:
You must create the agents in a Microsoft Foundry project and then provide the project endpoint and agent IDs. The instructions for each agent are as follows:
- Invoice Agent
```
@@ -1,7 +1,7 @@
// Copyright (c) Microsoft. All rights reserved.
// Seattle Hotel Agent - A simple agent with a tool to find hotels in Seattle.
// Uses Microsoft Agent Framework with Azure AI Foundry.
// Uses Microsoft Agent Framework with Microsoft Foundry.
// Ready for deployment to Foundry Hosted Agent service.
using System.ClientModel.Primitives;
@@ -4,7 +4,7 @@ This sample demonstrates how to build a hosted agent that uses local C# function
Key features:
- Defining local C# functions as agent tools using `AIFunctionFactory`
- Using `AIProjectClient` to discover the OpenAI connection from the Azure AI Foundry project
- Using `AIProjectClient` to discover the OpenAI connection from the Microsoft Foundry project
- Building a `ChatClientAgent` with custom instructions and tools
- Deploying to the Foundry Hosted Agent service
@@ -15,7 +15,7 @@ Key features:
Before running this sample, ensure you have:
1. .NET 10 SDK installed
2. An Azure AI Foundry Project with a chat model deployed (e.g., gpt-4o-mini)
2. A Microsoft Foundry Project with a chat model deployed (e.g., gpt-4o-mini)
3. Azure CLI installed and authenticated (`az login`)
## Environment Variables
@@ -23,7 +23,7 @@ Before running this sample, ensure you have:
Set the following environment variables:
```powershell
# Replace with your Azure AI Foundry project endpoint
# Replace with your Microsoft Foundry project endpoint
$env:AZURE_AI_PROJECT_ENDPOINT="https://your-project.services.ai.azure.com/api/projects/your-project-name"
# Optional, defaults to gpt-4o-mini
@@ -1,7 +1,7 @@
// Copyright (c) Microsoft. All rights reserved.
// This sample demonstrates a multi-agent workflow with Writer and Reviewer agents
// using Azure AI Foundry AIProjectClient and the Agent Framework WorkflowBuilder.
// using Microsoft Foundry AIProjectClient and the Agent Framework WorkflowBuilder.
#pragma warning disable CA2252 // AIProjectClient and Agents API require opting into preview features
@@ -42,7 +42,7 @@ which provisions a REST API endpoint compatible with the OpenAI Responses protoc
Before running this sample, ensure you have:
1. **Azure AI Foundry Project**
1. **Microsoft Foundry Project**
- Project created.
- Chat model deployed (e.g., `gpt-4o` or `gpt-4.1`)
- Note your project endpoint URL and model deployment name
@@ -4,7 +4,7 @@ name: FoundryMultiAgent
displayName: "Foundry Multi-Agent Workflow"
description: >
A multi-agent workflow featuring a Writer and Reviewer that collaborate
to create and refine content using Azure AI Foundry PersistentAgentsClient.
to create and refine content using Microsoft Foundry PersistentAgentsClient.
metadata:
authors:
- Microsoft Agent Framework Team
@@ -1,7 +1,7 @@
// Copyright (c) Microsoft. All rights reserved.
// Seattle Hotel Agent - A simple agent with a tool to find hotels in Seattle.
// Uses Microsoft Agent Framework with Azure AI Foundry.
// Uses Microsoft Agent Framework with Microsoft Foundry.
// Ready for deployment to Foundry Hosted Agent service.
#pragma warning disable CA2252 // AIProjectClient and Agents API require opting into preview features
@@ -39,7 +39,7 @@ which provisions a REST API endpoint compatible with the OpenAI Responses protoc
Before running this sample, ensure you have:
1. **Azure AI Foundry Project**
1. **Microsoft Foundry Project**
- Project created.
- Chat model deployed (e.g., `gpt-4o` or `gpt-4.1`)
- Note your project endpoint URL and model deployment name
@@ -57,7 +57,7 @@ Before running this sample, ensure you have:
Set the following environment variables (matching `agent.yaml`):
- `AZURE_AI_PROJECT_ENDPOINT` - Your Azure AI Foundry project endpoint URL (required)
- `AZURE_AI_PROJECT_ENDPOINT` - Your Microsoft Foundry project endpoint URL (required)
- `MODEL_DEPLOYMENT_NAME` - The deployment name for your chat model (defaults to `gpt-4o-mini`)
**PowerShell:**
@@ -20,7 +20,7 @@ Before running any sample, ensure you have:
1. **.NET 10 SDK** or later — [Download](https://dotnet.microsoft.com/download/dotnet/10.0)
2. **Azure CLI** installed — [Install guide](https://learn.microsoft.com/cli/azure/install-azure-cli)
3. **Azure OpenAI** or **Azure AI Foundry project** with a chat model deployed (e.g., `gpt-4o-mini`)
3. **Azure OpenAI** or **Microsoft Foundry project** with a chat model deployed (e.g., `gpt-4o-mini`)
### Authenticate with Azure CLI
@@ -39,14 +39,14 @@ Most samples require one or more of these environment variables:
|----------|---------|-------------|
| `AZURE_OPENAI_ENDPOINT` | Most samples | Your Azure OpenAI resource endpoint URL |
| `AZURE_OPENAI_DEPLOYMENT_NAME` | Most samples | Chat model deployment name (defaults to `gpt-4o-mini`) |
| `AZURE_AI_PROJECT_ENDPOINT` | AgentWithLocalTools, FoundryMultiAgent, FoundrySingleAgent | Azure AI Foundry project endpoint |
| `AZURE_AI_PROJECT_ENDPOINT` | AgentWithLocalTools, FoundryMultiAgent, FoundrySingleAgent | Microsoft Foundry project endpoint |
| `MODEL_DEPLOYMENT_NAME` | AgentWithLocalTools, FoundryMultiAgent, FoundrySingleAgent | Chat model deployment name (defaults to `gpt-4o-mini`) |
See each sample's README for the specific variables required.
## Azure AI Foundry Setup (for samples that use Foundry)
## Microsoft Foundry Setup (for samples that use Foundry)
Some samples (`AgentWithLocalTools`, `FoundrySingleAgent`, `FoundryMultiAgent`) connect to an Azure AI Foundry project. If you're using these samples, you'll need additional setup.
Some samples (`AgentWithLocalTools`, `FoundrySingleAgent`, `FoundryMultiAgent`) connect to a Microsoft Foundry project. If you're using these samples, you'll need additional setup.
### Azure AI Developer Role
@@ -61,7 +61,7 @@ az role assignment create `
> **Note**: You need **Owner** or **User Access Administrator** permissions on the resource to assign roles. If you don't have this, you may need to request JIT (Just-In-Time) elevated access via [Azure PIM](https://portal.azure.com/#view/Microsoft_Azure_PIMCommon/ActivationMenuBlade/~/aadmigratedresource).
For more details on permissions, see [Azure AI Foundry Permissions](https://aka.ms/FoundryPermissions).
For more details on permissions, see [Microsoft Foundry Permissions](https://aka.ms/FoundryPermissions).
## Running a Sample
+1 -1
View File
@@ -28,7 +28,7 @@ dotnet/samples/
│ ├── AGUI/ # AG-UI protocol samples
│ ├── DeclarativeAgents/ # Declarative agent definitions
│ ├── DevUI/ # DevUI samples
│ ├── AgentsWithFoundry/ # Azure AI Foundry samples (FoundryAgent + AsAIAgent extensions)
│ ├── AgentsWithFoundry/ # Microsoft Foundry samples (FoundryAgent + AsAIAgent extensions)
│ └── ModelContextProtocol/ # MCP server/client patterns
├── 03-workflows/ # Workflow patterns
│ ├── _StartHere/ # Introductory workflow samples
@@ -21,6 +21,15 @@ internal interface ICheckpointingHandle
/// <summary>
/// Restores the system state from the specified checkpoint asynchronously.
/// </summary>
/// <remarks>
/// This contract is used by live runtime restore paths. Implementations may re-emit pending
/// external request events as part of the restore once the active event stream is ready to
/// observe them.
///
/// Initial resume paths that create a new event stream should restore state first and defer
/// any replay until after the subscriber is attached, rather than calling this contract
/// directly before the stream is ready.
/// </remarks>
/// <param name="checkpointInfo">The checkpoint information that identifies the state to restore. Cannot be null.</param>
/// <param name="cancellationToken">A cancellation token that can be used to cancel the restore operation.</param>
/// <returns>A <see cref="ValueTask"/> that represents the asynchronous restore operation.</returns>
@@ -36,9 +36,10 @@ internal sealed class AsyncRunHandle : ICheckpointingHandle, IAsyncDisposable
this._eventStream.Start();
// If there are already unprocessed messages (e.g., from a checkpoint restore that happened
// before this handle was created), signal the run loop to start processing them
if (stepRunner.HasUnprocessedMessages)
// If there are already unprocessed messages or unserviced requests (e.g., from a
// checkpoint restore that happened before this handle was created), signal the run
// loop to start processing them
if (stepRunner.HasUnprocessedMessages || stepRunner.HasUnservicedRequests)
{
this.SignalInputToRunLoop();
}
@@ -192,13 +193,17 @@ internal sealed class AsyncRunHandle : ICheckpointingHandle, IAsyncDisposable
{
streamingEventStream.ClearBufferedEvents();
}
else if (this._eventStream is LockstepRunEventStream lockstepEventStream)
{
lockstepEventStream.ClearBufferedEvents();
}
// Restore the workflow state - this will republish unserviced requests as new events
// Restore the workflow state through the live runtime-restore path.
// This can re-emit pending requests into the already-active event stream.
await this._checkpointingHandle.RestoreCheckpointAsync(checkpointInfo, cancellationToken).ConfigureAwait(false);
// After restore, signal the run loop to process any restored messages
// This is necessary because ClearBufferedEvents() doesn't signal, and the restored
// queued messages won't automatically wake up the run loop
// After restore, signal the run loop to process any restored messages. Initial resume
// paths handle this separately when they create the event stream after restoring state.
this.SignalInputToRunLoop();
}
}
@@ -27,6 +27,14 @@ internal interface ISuperStepRunner
ConcurrentEventSink OutgoingEvents { get; }
/// <summary>
/// Re-emits <see cref="RequestInfoEvent"/>s for any pending external requests.
/// Called by event streams after subscribing to <see cref="OutgoingEvents"/> so that
/// requests restored from a checkpoint are observable even when the restore happened
/// before the subscription was active.
/// </summary>
ValueTask RepublishPendingEventsAsync(CancellationToken cancellationToken = default);
ValueTask<bool> RunSuperStepAsync(CancellationToken cancellationToken);
// This cannot be cancelled
@@ -15,6 +15,7 @@ internal sealed class LockstepRunEventStream : IRunEventStream
{
private readonly CancellationTokenSource _stopCancellation = new();
private readonly InputWaiter _inputWaiter = new();
private ConcurrentQueue<WorkflowEvent> _eventSink = new();
private int _isDisposed;
private readonly ISuperStepRunner _stepRunner;
@@ -35,6 +36,8 @@ internal sealed class LockstepRunEventStream : IRunEventStream
// doesn't leak into caller code via AsyncLocal.
Activity? previousActivity = Activity.Current;
this._stepRunner.OutgoingEvents.EventRaised += this.OnWorkflowEventAsync;
this._sessionActivity = this._stepRunner.TelemetryContext.StartWorkflowSessionActivity();
this._sessionActivity?.SetTag(Tags.WorkflowId, this._stepRunner.StartExecutorId)
.SetTag(Tags.SessionId, this._stepRunner.SessionId);
@@ -56,10 +59,6 @@ internal sealed class LockstepRunEventStream : IRunEventStream
using CancellationTokenSource linkedSource = CancellationTokenSource.CreateLinkedTokenSource(this._stopCancellation.Token, cancellationToken);
ConcurrentQueue<WorkflowEvent> eventSink = [];
this._stepRunner.OutgoingEvents.EventRaised += OnWorkflowEventAsync;
// Re-establish session as parent so the run activity nests correctly.
Activity.Current = this._sessionActivity;
@@ -73,7 +72,31 @@ internal sealed class LockstepRunEventStream : IRunEventStream
runActivity?.AddEvent(new ActivityEvent(EventNames.WorkflowStarted));
// Emit WorkflowStartedEvent to the event stream for consumers
eventSink.Enqueue(new WorkflowStartedEvent());
this._eventSink.Enqueue(new WorkflowStartedEvent());
// Re-emit any pending external requests that were restored from a checkpoint
// before this subscription was active. For non-resume starts this is a no-op.
// This runs after WorkflowStartedEvent so consumers always see the started event first.
await this._stepRunner.RepublishPendingEventsAsync(linkedSource.Token).ConfigureAwait(false);
// When resuming from a checkpoint with only pending requests (no queued messages),
// the inner processing loop won't execute, so we must drain events now.
// For normal starts this is a no-op since the inner loop handles the drain.
if (!this._stepRunner.HasUnprocessedMessages)
{
var (drainedEvents, shouldHalt) = this.DrainAndFilterEvents();
foreach (WorkflowEvent raisedEvent in drainedEvents)
{
yield return raisedEvent;
}
if (shouldHalt)
{
yield break;
}
this.RunStatus = this._stepRunner.HasUnservicedRequests ? RunStatus.PendingRequests : RunStatus.Idle;
}
do
{
@@ -107,26 +130,19 @@ internal sealed class LockstepRunEventStream : IRunEventStream
yield break; // Exit if cancellation is requested
}
bool hadRequestHaltEvent = false;
foreach (WorkflowEvent raisedEvent in Interlocked.Exchange(ref eventSink, []))
var (drainedEvents, shouldHalt) = this.DrainAndFilterEvents();
foreach (WorkflowEvent raisedEvent in drainedEvents)
{
if (linkedSource.Token.IsCancellationRequested)
{
yield break; // Exit if cancellation is requested
}
// TODO: Do we actually want to interpret this as a termination request?
if (raisedEvent is RequestHaltEvent)
{
hadRequestHaltEvent = true;
}
else
{
yield return raisedEvent;
}
yield return raisedEvent;
}
if (hadRequestHaltEvent || linkedSource.Token.IsCancellationRequested)
if (shouldHalt || linkedSource.Token.IsCancellationRequested)
{
// If we had a completion event, we are done.
yield break;
@@ -151,25 +167,23 @@ internal sealed class LockstepRunEventStream : IRunEventStream
finally
{
this.RunStatus = this._stepRunner.HasUnservicedRequests ? RunStatus.PendingRequests : RunStatus.Idle;
this._stepRunner.OutgoingEvents.EventRaised -= OnWorkflowEventAsync;
// Explicitly dispose the Activity so Activity.Stop fires deterministically,
// regardless of how the async iterator enumerator is disposed.
runActivity?.Dispose();
}
ValueTask OnWorkflowEventAsync(object? sender, WorkflowEvent e)
{
eventSink.Enqueue(e);
return default;
}
// If we are Idle or Ended, we should break out of the loop
// If we are PendingRequests and not blocking on pending requests, we should break out of the loop
// If cancellation is requested, we should break out of the loop
bool ShouldBreak() => this.RunStatus is RunStatus.Idle or RunStatus.Ended ||
(this.RunStatus == RunStatus.PendingRequests && !blockOnPendingRequest) ||
linkedSource.Token.IsCancellationRequested;
(this.RunStatus == RunStatus.PendingRequests && !blockOnPendingRequest) ||
linkedSource.Token.IsCancellationRequested;
}
internal void ClearBufferedEvents()
{
Interlocked.Exchange(ref this._eventSink, new ConcurrentQueue<WorkflowEvent>());
}
/// <summary>
@@ -192,6 +206,7 @@ internal sealed class LockstepRunEventStream : IRunEventStream
if (Interlocked.Exchange(ref this._isDisposed, 1) == 0)
{
this._stopCancellation.Cancel();
this._stepRunner.OutgoingEvents.EventRaised -= this.OnWorkflowEventAsync;
// Stop the session activity
if (this._sessionActivity is not null)
@@ -207,4 +222,32 @@ internal sealed class LockstepRunEventStream : IRunEventStream
return default;
}
private ValueTask OnWorkflowEventAsync(object? sender, WorkflowEvent e)
{
this._eventSink.Enqueue(e);
return default;
}
// Atomically drains the event sink and separates workflow events from halt signals.
// Used by both the early-drain (resume with pending requests only) and
// the inner superstep drain to keep halt-detection logic in one place.
private (List<WorkflowEvent> Events, bool ShouldHalt) DrainAndFilterEvents()
{
List<WorkflowEvent> events = [];
bool shouldHalt = false;
foreach (WorkflowEvent e in Interlocked.Exchange(ref this._eventSink, new ConcurrentQueue<WorkflowEvent>()))
{
if (e is RequestHaltEvent)
{
shouldHalt = true;
}
else
{
events.Add(e);
}
}
return (events, shouldHalt);
}
}
@@ -60,6 +60,10 @@ internal sealed class StreamingRunEventStream : IRunEventStream
// Subscribe to events - they will flow directly to the channel as they're raised
this._stepRunner.OutgoingEvents.EventRaised += OnEventRaisedAsync;
// Re-emit any pending external requests that were restored from a checkpoint
// before this subscription was active. For non-resume starts this is a no-op.
await this._stepRunner.RepublishPendingEventsAsync(linkedSource.Token).ConfigureAwait(false);
// Start the session-level activity that spans the entire run loop lifetime.
// Individual run-stage activities are nested within this session activity.
Activity? sessionActivity = this._stepRunner.TelemetryContext.StartWorkflowSessionActivity();
@@ -50,10 +50,13 @@ public sealed class InProcessExecutionEnvironment : IWorkflowExecutionEnvironmen
return runner.BeginStreamAsync(this.ExecutionMode, cancellationToken);
}
internal ValueTask<AsyncRunHandle> ResumeRunAsync(Workflow workflow, CheckpointInfo fromCheckpoint, IEnumerable<Type> knownValidInputTypes, CancellationToken cancellationToken)
internal ValueTask<AsyncRunHandle> ResumeRunAsync(Workflow workflow, CheckpointInfo fromCheckpoint, IEnumerable<Type> knownValidInputTypes, CancellationToken cancellationToken = default)
=> this.ResumeRunAsync(workflow, fromCheckpoint, knownValidInputTypes, republishPendingEvents: true, cancellationToken);
internal ValueTask<AsyncRunHandle> ResumeRunAsync(Workflow workflow, CheckpointInfo fromCheckpoint, IEnumerable<Type> knownValidInputTypes, bool republishPendingEvents, CancellationToken cancellationToken = default)
{
InProcessRunner runner = InProcessRunner.CreateTopLevelRunner(workflow, this.CheckpointManager, fromCheckpoint.SessionId, this.EnableConcurrentRuns, knownValidInputTypes);
return runner.ResumeStreamAsync(this.ExecutionMode, fromCheckpoint, cancellationToken);
return runner.ResumeStreamAsync(this.ExecutionMode, fromCheckpoint, republishPendingEvents, cancellationToken);
}
/// <inheritdoc/>
@@ -104,6 +107,32 @@ public sealed class InProcessExecutionEnvironment : IWorkflowExecutionEnvironmen
return new(runHandle);
}
/// <summary>
/// Resumes a streaming workflow run from a checkpoint with control over whether
/// pending request events are republished through the event stream.
/// </summary>
/// <param name="workflow">The workflow to resume.</param>
/// <param name="fromCheckpoint">The checkpoint to resume from.</param>
/// <param name="republishPendingEvents">
/// When <see langword="true"/>, any pending request events are republished through the event
/// stream after subscribing. When <see langword="false"/>, the caller is responsible for
/// handling pending requests (e.g., <see cref="WorkflowSession"/> already sends responses).
/// </param>
/// <param name="cancellationToken">Cancellation token.</param>
internal async ValueTask<StreamingRun> ResumeStreamingInternalAsync(
Workflow workflow,
CheckpointInfo fromCheckpoint,
bool republishPendingEvents,
CancellationToken cancellationToken = default)
{
this.VerifyCheckpointingConfigured();
AsyncRunHandle runHandle = await this.ResumeRunAsync(workflow, fromCheckpoint, [], republishPendingEvents, cancellationToken)
.ConfigureAwait(false);
return new(runHandle);
}
private async ValueTask<AsyncRunHandle> BeginRunHandlingChatProtocolAsync<TInput>(Workflow workflow,
TInput input,
string? sessionId = null,
@@ -71,6 +71,28 @@ internal sealed class InProcessRunner : ISuperStepRunner, ICheckpointingHandle
/// <inheritdoc cref="ISuperStepRunner.StartExecutorId"/>
public string StartExecutorId { get; }
/// <summary>
/// Gating flag for deferred event republishing after checkpoint restore.
/// </summary>
/// <remarks>
/// <para>
/// Written with <see cref="Volatile.Write(ref int, int)"/> in <see cref="ResumeStreamAsync(ExecutionMode, CheckpointInfo, bool, CancellationToken)"/>
/// and consumed atomically with <see cref="Interlocked.Exchange(ref int, int)"/> in
/// <see cref="ISuperStepRunner.RepublishPendingEventsAsync"/>. The write does not need a full
/// memory barrier because it is sequenced before the <see cref="AsyncRunHandle"/> constructor
/// by the <see langword="await"/> in <see cref="ResumeStreamAsync(ExecutionMode, CheckpointInfo, bool, CancellationToken)"/>. The constructor is the
/// only code path that triggers consumption (via the event stream's subscribe and republish flow).
/// </para>
/// <para>
/// Note: <see cref="AsyncRunHandle"/> also reads <see cref="ISuperStepRunner.HasUnservicedRequests"/>
/// in its constructor to signal the run loop, but that property reads from
/// <see cref="InProcessRunnerContext"/>'s request dictionary (restored during
/// <see cref="RestoreCheckpointCoreAsync"/>), not from this flag. The two are independent:
/// <c>HasUnservicedRequests</c> triggers the run loop; <c>_needsRepublish</c> triggers event emission.
/// </para>
/// </remarks>
private int _needsRepublish;
/// <inheritdoc cref="ISuperStepRunner.TelemetryContext"/>
public WorkflowTelemetryContext TelemetryContext => this.Workflow.TelemetryContext;
@@ -145,7 +167,10 @@ internal sealed class InProcessRunner : ISuperStepRunner, ICheckpointingHandle
return new(new AsyncRunHandle(this, this, mode));
}
public async ValueTask<AsyncRunHandle> ResumeStreamAsync(ExecutionMode mode, CheckpointInfo fromCheckpoint, CancellationToken cancellationToken = default)
public ValueTask<AsyncRunHandle> ResumeStreamAsync(ExecutionMode mode, CheckpointInfo fromCheckpoint, CancellationToken cancellationToken = default)
=> this.ResumeStreamAsync(mode, fromCheckpoint, republishPendingEvents: true, cancellationToken);
public async ValueTask<AsyncRunHandle> ResumeStreamAsync(ExecutionMode mode, CheckpointInfo fromCheckpoint, bool republishPendingEvents, CancellationToken cancellationToken = default)
{
this.RunContext.CheckEnded();
Throw.IfNull(fromCheckpoint);
@@ -154,7 +179,18 @@ internal sealed class InProcessRunner : ISuperStepRunner, ICheckpointingHandle
throw new InvalidOperationException("This runner was not configured with a CheckpointManager, so it cannot restore checkpoints.");
}
await this.RestoreCheckpointAsync(fromCheckpoint, cancellationToken).ConfigureAwait(false);
// Restore checkpoint state without republishing pending request events.
// The event stream will republish them after subscribing so that events
// are never lost to an absent subscriber.
await this.RestoreCheckpointCoreAsync(fromCheckpoint, cancellationToken).ConfigureAwait(false);
if (republishPendingEvents)
{
// Signal the event stream to republish pending requests after subscribing.
// This is consumed atomically by RepublishPendingEventsAsync.
Volatile.Write(ref this._needsRepublish, 1);
}
return new AsyncRunHandle(this, this, mode);
}
@@ -163,6 +199,16 @@ internal sealed class InProcessRunner : ISuperStepRunner, ICheckpointingHandle
bool ISuperStepRunner.TryGetResponsePortExecutorId(string portId, out string? executorId)
=> this.RunContext.TryGetResponsePortExecutorId(portId, out executorId);
ValueTask ISuperStepRunner.RepublishPendingEventsAsync(CancellationToken cancellationToken)
{
if (Interlocked.Exchange(ref this._needsRepublish, 0) != 0)
{
return this.RunContext.RepublishUnservicedRequestsAsync(cancellationToken);
}
return default;
}
public bool IsCheckpointingEnabled => this.RunContext.IsCheckpointingEnabled;
public IReadOnlyList<CheckpointInfo> Checkpoints => this._checkpoints;
@@ -310,7 +356,31 @@ internal sealed class InProcessRunner : ISuperStepRunner, ICheckpointingHandle
this._checkpoints.Add(this._lastCheckpointInfo);
}
/// <summary>
/// Restores checkpoint state and re-emits any pending external request events.
/// </summary>
/// <remarks>
/// This is the <see cref="ICheckpointingHandle"/> implementation used for runtime restores
/// where the event stream subscription is already active. For initial resumes,
/// <see cref="ResumeStreamAsync(ExecutionMode, CheckpointInfo, CancellationToken)"/> calls
/// <see cref="RestoreCheckpointCoreAsync"/> directly and defers republishing to the event stream.
/// </remarks>
public async ValueTask RestoreCheckpointAsync(CheckpointInfo checkpointInfo, CancellationToken cancellationToken = default)
{
await this.RestoreCheckpointCoreAsync(checkpointInfo, cancellationToken).ConfigureAwait(false);
// Republish pending request events. This is safe for runtime restores where
// the event stream is already subscribed. For initial resumes the event stream
// handles republishing itself, so ResumeStreamAsync calls RestoreCheckpointCoreAsync directly.
await this.RunContext.RepublishUnservicedRequestsAsync(cancellationToken).ConfigureAwait(false);
}
/// <summary>
/// Restores checkpoint state (queued messages, executor state, edge state, etc.)
/// without republishing pending request events. The caller is responsible for
/// ensuring events are republished after an event subscriber is attached.
/// </summary>
private async ValueTask RestoreCheckpointCoreAsync(CheckpointInfo checkpointInfo, CancellationToken cancellationToken = default)
{
this.RunContext.CheckEnded();
Throw.IfNull(checkpointInfo);
@@ -335,11 +405,9 @@ internal sealed class InProcessRunner : ISuperStepRunner, ICheckpointingHandle
await this.RunContext.ImportStateAsync(checkpoint).ConfigureAwait(false);
Task executorNotifyTask = this.RunContext.NotifyCheckpointLoadedAsync(cancellationToken);
ValueTask republishRequestsTask = this.RunContext.RepublishUnservicedRequestsAsync(cancellationToken);
await this.EdgeMap.ImportStateAsync(checkpoint).ConfigureAwait(false);
await Task.WhenAll(executorNotifyTask,
republishRequestsTask.AsTask(),
restoreCheckpointIndexTask.AsTask()).ConfigureAwait(false);
this._lastCheckpointInfo = checkpointInfo;
@@ -14,6 +14,7 @@ internal sealed class RequestPortOptions;
internal sealed class RequestInfoExecutor : Executor
{
private const string WrappedRequestsStateKey = nameof(WrappedRequestsStateKey);
private readonly Dictionary<string, ExternalRequest> _wrappedRequests = [];
private RequestPort Port { get; }
private IExternalRequestSink? RequestSink { get; set; }
@@ -124,22 +125,46 @@ internal sealed class RequestInfoExecutor : Executor
return null;
}
if (this._allowWrapped && this._wrappedRequests.TryGetValue(message.RequestId, out ExternalRequest? originalRequest))
{
await context.SendMessageAsync(originalRequest.RewrapResponse(message), cancellationToken: cancellationToken).ConfigureAwait(false);
}
else
{
await context.SendMessageAsync(message, cancellationToken: cancellationToken).ConfigureAwait(false);
}
if (!message.Data.IsType(this.Port.Response, out object? data))
{
throw this.Port.CreateExceptionForType(message);
}
await context.SendMessageAsync(data, cancellationToken: cancellationToken).ConfigureAwait(false);
if (this._allowWrapped && this._wrappedRequests.TryGetValue(message.RequestId, out ExternalRequest? originalRequest))
{
await context.SendMessageAsync(originalRequest.RewrapResponse(message), cancellationToken: cancellationToken).ConfigureAwait(false);
this._wrappedRequests.Remove(message.RequestId);
}
else
{
await context.SendMessageAsync(message, cancellationToken: cancellationToken).ConfigureAwait(false);
await context.SendMessageAsync(data, cancellationToken: cancellationToken).ConfigureAwait(false);
}
return message;
}
protected internal override async ValueTask OnCheckpointingAsync(IWorkflowContext context, CancellationToken cancellationToken = default)
{
await context.QueueStateUpdateAsync(WrappedRequestsStateKey,
new Dictionary<string, ExternalRequest>(this._wrappedRequests, StringComparer.Ordinal),
cancellationToken: cancellationToken).ConfigureAwait(false);
await base.OnCheckpointingAsync(context, cancellationToken).ConfigureAwait(false);
}
protected internal override async ValueTask OnCheckpointRestoredAsync(IWorkflowContext context, CancellationToken cancellationToken = default)
{
await base.OnCheckpointRestoredAsync(context, cancellationToken).ConfigureAwait(false);
this._wrappedRequests.Clear();
Dictionary<string, ExternalRequest> wrappedRequests =
await context.ReadStateAsync<Dictionary<string, ExternalRequest>>(WrappedRequestsStateKey, cancellationToken: cancellationToken)
.ConfigureAwait(false) ?? [];
foreach (KeyValuePair<string, ExternalRequest> wrappedRequest in wrappedRequests)
{
this._wrappedRequests[wrappedRequest.Key] = wrappedRequest.Value;
}
}
}
@@ -1,6 +1,7 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Diagnostics;
using System.Diagnostics.CodeAnalysis;
@@ -23,6 +24,7 @@ internal class WorkflowHostExecutor : Executor, IAsyncDisposable
private InProcessRunner? _activeRunner;
private InMemoryCheckpointManager? _checkpointManager;
private readonly ExecutorOptions _options;
private readonly ConcurrentDictionary<string, RequestPortInfo> _pendingResponsePorts = new(StringComparer.Ordinal);
private ISuperStepJoinContext? _joinContext;
private string? _joinId;
@@ -163,6 +165,11 @@ internal class WorkflowHostExecutor : Executor, IAsyncDisposable
private ExternalResponse? CheckAndUnqualifyResponse([DisallowNull] ExternalResponse response)
{
if (this._pendingResponsePorts.TryRemove(response.RequestId, out RequestPortInfo? originalPort))
{
return response with { PortInfo = originalPort };
}
if (!Throw.IfNull(response).PortInfo.PortId.StartsWith($"{this.Id}.", StringComparison.Ordinal))
{
return null;
@@ -193,6 +200,7 @@ internal class WorkflowHostExecutor : Executor, IAsyncDisposable
break;
case RequestInfoEvent requestInfoEvt:
ExternalRequest request = requestInfoEvt.Request;
this._pendingResponsePorts[request.RequestId] = request.PortInfo;
resultTask = this._joinContext?.SendMessageAsync(this.Id, this.QualifyRequestPortId(request)).AsTask() ?? Task.CompletedTask;
break;
case WorkflowErrorEvent errorEvent:
@@ -246,9 +254,13 @@ internal class WorkflowHostExecutor : Executor, IAsyncDisposable
}
private const string CheckpointManagerStateKey = nameof(CheckpointManager);
private const string PendingResponsePortsStateKey = nameof(PendingResponsePortsStateKey);
protected internal override async ValueTask OnCheckpointingAsync(IWorkflowContext context, CancellationToken cancellationToken = default)
{
await context.QueueStateUpdateAsync(CheckpointManagerStateKey, this._checkpointManager, cancellationToken: cancellationToken).ConfigureAwait(false);
await context.QueueStateUpdateAsync(PendingResponsePortsStateKey,
new Dictionary<string, RequestPortInfo>(this._pendingResponsePorts, StringComparer.Ordinal),
cancellationToken: cancellationToken).ConfigureAwait(false);
await base.OnCheckpointingAsync(context, cancellationToken).ConfigureAwait(false);
}
@@ -269,6 +281,15 @@ internal class WorkflowHostExecutor : Executor, IAsyncDisposable
await this.ResetAsync().ConfigureAwait(false);
}
this._pendingResponsePorts.Clear();
Dictionary<string, RequestPortInfo> pendingResponsePorts =
await context.ReadStateAsync<Dictionary<string, RequestPortInfo>>(PendingResponsePortsStateKey, cancellationToken: cancellationToken)
.ConfigureAwait(false) ?? [];
foreach (KeyValuePair<string, RequestPortInfo> pendingResponsePort in pendingResponsePorts)
{
this._pendingResponsePorts[pendingResponsePort.Key] = pendingResponsePort.Value;
}
await this.EnsureRunSendMessageAsync(resume: true, cancellationToken: cancellationToken).ConfigureAwait(false);
}
@@ -280,6 +301,8 @@ internal class WorkflowHostExecutor : Executor, IAsyncDisposable
this._run = null;
}
this._pendingResponsePorts.Clear();
if (this._activeRunner != null)
{
this._activeRunner.OutgoingEvents.EventRaised -= this.ForwardWorkflowEventAsync;
@@ -19,7 +19,14 @@ namespace Microsoft.Agents.AI.Workflows;
internal sealed class WorkflowSession : AgentSession
{
private readonly Workflow _workflow;
private readonly IWorkflowExecutionEnvironment _executionEnvironment;
/// <summary>
/// The execution environment for this session. Concrete type is required because
/// <see cref="CreateOrResumeRunAsync"/> uses the internal
/// <see cref="InProcessExecutionEnvironment.ResumeStreamingInternalAsync"/> API.
/// </summary>
private readonly InProcessExecutionEnvironment _inProcEnvironment;
private readonly bool _includeExceptionDetails;
private readonly bool _includeWorkflowOutputsInResponse;
@@ -63,17 +70,22 @@ internal sealed class WorkflowSession : AgentSession
public WorkflowSession(Workflow workflow, string sessionId, IWorkflowExecutionEnvironment executionEnvironment, bool includeExceptionDetails = false, bool includeWorkflowOutputsInResponse = false)
{
this._workflow = Throw.IfNull(workflow);
this._executionEnvironment = Throw.IfNull(executionEnvironment);
this._includeExceptionDetails = includeExceptionDetails;
this._includeWorkflowOutputsInResponse = includeWorkflowOutputsInResponse;
if (VerifyCheckpointingConfiguration(executionEnvironment, out InProcessExecutionEnvironment? inProcEnv))
IWorkflowExecutionEnvironment env = Throw.IfNull(executionEnvironment);
if (VerifyCheckpointingConfiguration(env, out InProcessExecutionEnvironment? inProcEnv))
{
// We have an InProcessExecutionEnvironment which is not configured for checkpointing. Ensure it has an externalizable checkpoint manager,
// since we are responsible for maintaining the state.
this._executionEnvironment = inProcEnv.WithCheckpointing(this.EnsureExternalizedInMemoryCheckpointing());
env = inProcEnv.WithCheckpointing(this.EnsureExternalizedInMemoryCheckpointing());
}
this._inProcEnvironment = env as InProcessExecutionEnvironment
?? throw new InvalidOperationException(
$"WorkflowSession requires an {nameof(InProcessExecutionEnvironment)}, " +
$"but received {env.GetType().Name}.");
this.SessionId = Throw.IfNullOrEmpty(sessionId);
this.ChatHistoryProvider = new WorkflowChatHistoryProvider();
}
@@ -86,24 +98,30 @@ internal sealed class WorkflowSession : AgentSession
public WorkflowSession(Workflow workflow, JsonElement serializedSession, IWorkflowExecutionEnvironment executionEnvironment, bool includeExceptionDetails = false, bool includeWorkflowOutputsInResponse = false, JsonSerializerOptions? jsonSerializerOptions = null)
{
this._workflow = Throw.IfNull(workflow);
this._executionEnvironment = Throw.IfNull(executionEnvironment);
this._includeExceptionDetails = includeExceptionDetails;
this._includeWorkflowOutputsInResponse = includeWorkflowOutputsInResponse;
IWorkflowExecutionEnvironment env = Throw.IfNull(executionEnvironment);
JsonMarshaller marshaller = new(jsonSerializerOptions);
SessionState sessionState = marshaller.Marshal<SessionState>(serializedSession);
this._inMemoryCheckpointManager = sessionState.CheckpointManager;
if (this._inMemoryCheckpointManager != null &&
VerifyCheckpointingConfiguration(executionEnvironment, out InProcessExecutionEnvironment? inProcEnv))
VerifyCheckpointingConfiguration(env, out InProcessExecutionEnvironment? inProcEnv))
{
this._executionEnvironment = inProcEnv.WithCheckpointing(this.EnsureExternalizedInMemoryCheckpointing());
env = inProcEnv.WithCheckpointing(this.EnsureExternalizedInMemoryCheckpointing());
}
else if (this._inMemoryCheckpointManager != null)
{
throw new ArgumentException("The session was saved with an externalized checkpoint manager, but the incoming execution environment does not support it.", nameof(executionEnvironment));
}
this._inProcEnvironment = env as InProcessExecutionEnvironment
?? throw new InvalidOperationException(
$"WorkflowSession requires an {nameof(InProcessExecutionEnvironment)}, " +
$"but received {env.GetType().Name}.");
this.SessionId = sessionState.SessionId;
this.ChatHistoryProvider = new WorkflowChatHistoryProvider();
@@ -160,10 +178,15 @@ internal sealed class WorkflowSession : AgentSession
// and does not need to be checked again here.
if (this.LastCheckpoint is not null)
{
// Use the internal resume path that suppresses pending request republishing.
// WorkflowSession handles pending requests itself by converting matching responses
// via SendMessagesWithResponseConversionAsync, so event-stream republishing would
// cause unwanted duplicate events visible to the consumer.
StreamingRun run =
await this._executionEnvironment
.ResumeStreamingAsync(this._workflow,
await this._inProcEnvironment
.ResumeStreamingInternalAsync(this._workflow,
this.LastCheckpoint,
republishPendingEvents: false,
cancellationToken)
.ConfigureAwait(false);
@@ -172,7 +195,7 @@ internal sealed class WorkflowSession : AgentSession
return new ResumeRunResult(run, dispatchInfo);
}
StreamingRun newRun = await this._executionEnvironment
StreamingRun newRun = await this._inProcEnvironment
.RunStreamingAsync(this._workflow,
messages,
this.SessionId,
@@ -126,6 +126,13 @@ internal sealed class WorkflowHarness(Workflow workflow, string runId)
{
hasRequest = true;
}
else
{
// This is a republished event for the request we're already responding to
// (emitted by RepublishUnservicedRequestsAsync during checkpoint resume).
// Skip yielding it so downstream code doesn't treat it as a new pending request.
continue;
}
break;
case ConversationUpdateEvent conversationEvent:
@@ -0,0 +1,445 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using FluentAssertions;
using Microsoft.Agents.AI.Workflows.InProc;
using Microsoft.Agents.AI.Workflows.Sample;
namespace Microsoft.Agents.AI.Workflows.UnitTests;
/// <summary>
/// Regression tests for GH-2485: pending <see cref="RequestInfoEvent"/> objects must be
/// re-emitted after resuming a workflow from a checkpoint.
/// </summary>
public class CheckpointResumeTests
{
/// <summary>
/// Verifies that a resumed workflow re-emits <see cref="RequestInfoEvent"/>s for
/// pending external requests that existed at the time of the checkpoint.
/// </summary>
[Theory]
[InlineData(ExecutionEnvironment.InProcess_OffThread)]
[InlineData(ExecutionEnvironment.InProcess_Lockstep)]
internal async Task Checkpoint_Resume_WithPendingRequests_RepublishesRequestInfoEventsAsync(ExecutionEnvironment environment)
{
// Arrange
RequestPort<string, string> requestPort = RequestPort.Create<string, string>("TestPort");
ForwardMessageExecutor<string> processor = new("Processor");
Workflow workflow = new WorkflowBuilder(requestPort)
.AddEdge(requestPort, processor)
.Build();
CheckpointManager checkpointManager = CheckpointManager.CreateInMemory();
InProcessExecutionEnvironment env = environment.ToWorkflowExecutionEnvironment();
// Act 1: Run workflow, collect pending requests and a checkpoint.
List<ExternalRequest> originalRequests = [];
CheckpointInfo? checkpoint = null;
await using (StreamingRun firstRun = await env.WithCheckpointing(checkpointManager)
.RunStreamingAsync(workflow, "Hello"))
{
await foreach (WorkflowEvent evt in firstRun.WatchStreamAsync(blockOnPendingRequest: false))
{
if (evt is RequestInfoEvent requestInfo)
{
originalRequests.Add(requestInfo.Request);
}
if (evt is SuperStepCompletedEvent step && step.CompletionInfo?.Checkpoint is { } cp)
{
checkpoint = cp;
}
}
originalRequests.Should().NotBeEmpty("the workflow should have created at least one external request");
checkpoint.Should().NotBeNull("a checkpoint should have been created");
}
// Act 2: Resume from the checkpoint.
await using StreamingRun resumed = await env.WithCheckpointing(checkpointManager)
.ResumeStreamingAsync(workflow, checkpoint!);
// Assert: The pending requests should be re-emitted.
List<ExternalRequest> reEmittedRequests = [];
using CancellationTokenSource cts = new(TimeSpan.FromSeconds(10));
await foreach (WorkflowEvent evt in resumed.WatchStreamAsync(blockOnPendingRequest: false, cts.Token))
{
if (evt is RequestInfoEvent requestInfo)
{
reEmittedRequests.Add(requestInfo.Request);
}
}
reEmittedRequests.Should().HaveCount(originalRequests.Count,
"all pending requests from the checkpoint should be re-emitted after resume");
reEmittedRequests.Select(r => r.RequestId)
.Should().BeEquivalentTo(originalRequests.Select(r => r.RequestId),
"the re-emitted request IDs should match the original pending request IDs");
}
/// <summary>
/// Verifies that <see cref="RunStatus"/> transitions to <see cref="RunStatus.PendingRequests"/>
/// after resuming from a checkpoint with pending external requests (not stuck at NotStarted).
/// </summary>
[Theory]
[InlineData(ExecutionEnvironment.InProcess_OffThread)]
[InlineData(ExecutionEnvironment.InProcess_Lockstep)]
internal async Task Checkpoint_Resume_WithPendingRequests_RunStatusIsPendingRequestsAsync(ExecutionEnvironment environment)
{
// Arrange
RequestPort<string, string> requestPort = RequestPort.Create<string, string>("TestPort");
ForwardMessageExecutor<string> processor = new("Processor");
Workflow workflow = new WorkflowBuilder(requestPort)
.AddEdge(requestPort, processor)
.Build();
CheckpointManager checkpointManager = CheckpointManager.CreateInMemory();
InProcessExecutionEnvironment env = environment.ToWorkflowExecutionEnvironment();
// First run: collect a checkpoint with pending requests.
CheckpointInfo? checkpoint = null;
await using (StreamingRun firstRun = await env.WithCheckpointing(checkpointManager)
.RunStreamingAsync(workflow, "Hello"))
{
await foreach (WorkflowEvent evt in firstRun.WatchStreamAsync(blockOnPendingRequest: false))
{
if (evt is SuperStepCompletedEvent step && step.CompletionInfo?.Checkpoint is { } cp)
{
checkpoint = cp;
}
}
checkpoint.Should().NotBeNull();
}
// Act: Resume from the checkpoint and consume events so the run loop processes.
await using StreamingRun resumed = await env.WithCheckpointing(checkpointManager)
.ResumeStreamingAsync(workflow, checkpoint!);
using CancellationTokenSource cts = new(TimeSpan.FromSeconds(10));
await foreach (WorkflowEvent _ in resumed.WatchStreamAsync(blockOnPendingRequest: false, cts.Token))
{
// Consume all events until the stream completes.
}
// Assert
RunStatus status = await resumed.GetStatusAsync();
status.Should().Be(RunStatus.PendingRequests,
"the resumed workflow should report PendingRequests after rehydration");
}
/// <summary>
/// Verifies the full roundtrip: resume from checkpoint, observe the re-emitted request,
/// send a response, and verify the workflow completes without duplicating the request.
/// </summary>
[Theory]
[InlineData(ExecutionEnvironment.InProcess_OffThread)]
[InlineData(ExecutionEnvironment.InProcess_Lockstep)]
internal async Task Checkpoint_Resume_RespondToPendingRequest_CompletesWithoutDuplicateAsync(ExecutionEnvironment environment)
{
// Arrange
RequestPort<string, string> requestPort = RequestPort.Create<string, string>("TestPort");
ForwardMessageExecutor<string> processor = new("Processor");
Workflow workflow = new WorkflowBuilder(requestPort)
.AddEdge(requestPort, processor)
.Build();
CheckpointManager checkpointManager = CheckpointManager.CreateInMemory();
InProcessExecutionEnvironment env = environment.ToWorkflowExecutionEnvironment();
// First run: collect checkpoint + pending request.
ExternalRequest? pendingRequest = null;
CheckpointInfo? checkpoint = null;
await using (StreamingRun firstRun = await env.WithCheckpointing(checkpointManager)
.RunStreamingAsync(workflow, "Hello"))
{
await foreach (WorkflowEvent evt in firstRun.WatchStreamAsync(blockOnPendingRequest: false))
{
if (evt is RequestInfoEvent requestInfo)
{
pendingRequest = requestInfo.Request;
}
if (evt is SuperStepCompletedEvent step && step.CompletionInfo?.Checkpoint is { } cp)
{
checkpoint = cp;
}
}
pendingRequest.Should().NotBeNull();
checkpoint.Should().NotBeNull();
}
// Act: Resume and respond to the restored request.
await using StreamingRun resumed = await env.WithCheckpointing(checkpointManager)
.ResumeStreamingAsync(workflow, checkpoint!);
int requestEventCount = 0;
using CancellationTokenSource cts = new(TimeSpan.FromSeconds(10));
// Use blockOnPendingRequest: false for the first pass to see the re-emitted requests.
await foreach (WorkflowEvent evt in resumed.WatchStreamAsync(blockOnPendingRequest: false, cts.Token))
{
if (evt is RequestInfoEvent requestInfo)
{
requestEventCount++;
requestInfo.Request.RequestId.Should().Be(pendingRequest!.RequestId,
"the re-emitted request should match the original");
}
}
requestEventCount.Should().Be(1,
"the pending request should be emitted exactly once (no duplicates)");
// Assert intermediate state before responding: the run should be in PendingRequests
// and we should have observed the re-emitted request. If the first WatchStreamAsync
// didn't complete or yielded nothing, these assertions catch it with a clear message.
RunStatus statusBeforeResponse = await resumed.GetStatusAsync();
statusBeforeResponse.Should().Be(RunStatus.PendingRequests,
"the run should be in PendingRequests state before we send a response");
// Now send the response and verify the workflow processes it.
ExternalResponse response = pendingRequest!.CreateResponse("World");
await resumed.SendResponseAsync(response);
// Consume the resulting events to verify the workflow progresses without errors.
List<WorkflowEvent> postResponseEvents = [];
using CancellationTokenSource cts2 = new(TimeSpan.FromSeconds(10));
await foreach (WorkflowEvent evt in resumed.WatchStreamAsync(blockOnPendingRequest: false, cts2.Token))
{
postResponseEvents.Add(evt);
}
postResponseEvents.Should().NotBeEmpty(
"the workflow should process the response and produce events");
postResponseEvents.OfType<WorkflowErrorEvent>().Should().BeEmpty(
"no errors should occur when processing the restored request's response");
}
/// <summary>
/// Verifies that restoring a live run to a checkpoint re-emits pending requests and allows
/// the workflow to continue from that restored point.
/// </summary>
[Theory]
[InlineData(ExecutionEnvironment.InProcess_OffThread)]
[InlineData(ExecutionEnvironment.InProcess_Lockstep)]
internal async Task Checkpoint_Restore_WithPendingRequests_RepublishesRequestInfoEventsAsync(ExecutionEnvironment environment)
{
// Arrange
Workflow workflow = CreateSimpleRequestWorkflow();
CheckpointManager checkpointManager = CheckpointManager.CreateInMemory();
InProcessExecutionEnvironment env = environment.ToWorkflowExecutionEnvironment();
await using StreamingRun run = await env.WithCheckpointing(checkpointManager)
.RunStreamingAsync(workflow, "Hello");
(ExternalRequest pendingRequest, CheckpointInfo checkpoint) = await CapturePendingRequestAndCheckpointAsync(run);
// Advance the run past the checkpoint so the restore has meaningful work to undo.
await run.SendResponseAsync(pendingRequest.CreateResponse("World"));
List<WorkflowEvent> firstCompletionEvents = await ReadToHaltAsync(run);
firstCompletionEvents.OfType<WorkflowErrorEvent>().Should().BeEmpty(
"the workflow should continue cleanly before we restore");
RunStatus statusAfterFirstResponse = await run.GetStatusAsync();
statusAfterFirstResponse.Should().Be(RunStatus.Idle,
"the workflow should finish processing the first response before we restore");
// Act
await run.RestoreCheckpointAsync(checkpoint);
// Assert
List<WorkflowEvent> restoredEvents = await ReadToHaltAsync(run);
ExternalRequest[] replayedRequests = [.. restoredEvents.OfType<RequestInfoEvent>().Select(evt => evt.Request)];
replayedRequests.Should().ContainSingle("runtime restore should re-emit the restored pending request");
replayedRequests[0].RequestId.Should().Be(pendingRequest.RequestId,
"the replayed request should match the request captured at the checkpoint");
await run.SendResponseAsync(replayedRequests[0].CreateResponse("Again"));
List<WorkflowEvent> secondCompletionEvents = await ReadToHaltAsync(run);
secondCompletionEvents.OfType<WorkflowErrorEvent>().Should().BeEmpty(
"runtime restore replay should not introduce workflow errors");
RunStatus statusAfterRestoreResponse = await run.GetStatusAsync();
statusAfterRestoreResponse.Should().Be(RunStatus.Idle,
"the workflow should be able to continue after the runtime restore replay");
}
/// <summary>
/// Verifies that a resumed parent workflow re-emits pending requests that originated in a subworkflow.
/// </summary>
[Theory]
[InlineData(ExecutionEnvironment.InProcess_OffThread)]
[InlineData(ExecutionEnvironment.InProcess_Lockstep)]
internal async Task Checkpoint_Resume_SubworkflowWithPendingRequests_RepublishesQualifiedRequestInfoEventsAsync(ExecutionEnvironment environment)
{
// Arrange
Workflow workflow = CreateCheckpointedSubworkflowRequestWorkflow();
CheckpointManager checkpointManager = CheckpointManager.CreateInMemory();
InProcessExecutionEnvironment env = environment.ToWorkflowExecutionEnvironment();
ExternalRequest pendingRequest;
CheckpointInfo checkpoint;
await using (StreamingRun firstRun = await env.WithCheckpointing(checkpointManager)
.RunStreamingAsync(workflow, "Hello"))
{
(pendingRequest, checkpoint) = await CapturePendingRequestAndCheckpointAsync(firstRun);
}
// Act
await using StreamingRun resumed = await env.WithCheckpointing(checkpointManager)
.ResumeStreamingAsync(workflow, checkpoint);
// Assert
List<WorkflowEvent> resumedEvents = await ReadToHaltAsync(resumed);
ExternalRequest[] replayedRequests = [.. resumedEvents.OfType<RequestInfoEvent>().Select(evt => evt.Request)];
replayedRequests.Should().ContainSingle("the resumed parent workflow should surface the subworkflow request once");
replayedRequests[0].RequestId.Should().Be(pendingRequest.RequestId,
"the replayed subworkflow request should match the checkpointed request");
replayedRequests[0].PortInfo.PortId.Should().Be(pendingRequest.PortInfo.PortId,
"the replayed request should remain qualified through the subworkflow boundary");
await resumed.SendResponseAsync(replayedRequests[0].CreateResponse("World"));
List<WorkflowEvent> completionEvents = await ReadToHaltAsync(resumed);
completionEvents.OfType<RequestInfoEvent>().Should().BeEmpty(
"the resumed subworkflow request should not be replayed twice");
completionEvents.OfType<WorkflowErrorEvent>().Should().BeEmpty(
"subworkflow replay should not introduce workflow errors");
RunStatus statusAfterSubworkflowResponse = await resumed.GetStatusAsync();
statusAfterSubworkflowResponse.Should().Be(RunStatus.Idle,
"the resumed subworkflow should continue after responding to the replayed request");
}
/// <summary>
/// Verifies that when <c>republishPendingEvents</c> is <see langword="false"/>,
/// no <see cref="RequestInfoEvent"/> is re-emitted after resuming from a checkpoint.
/// </summary>
[Theory]
[InlineData(ExecutionEnvironment.InProcess_OffThread)]
[InlineData(ExecutionEnvironment.InProcess_Lockstep)]
internal async Task Checkpoint_Resume_WithRepublishDisabled_DoesNotEmitRequestInfoEventsAsync(ExecutionEnvironment environment)
{
// Arrange
RequestPort<string, string> requestPort = RequestPort.Create<string, string>("TestPort");
ForwardMessageExecutor<string> processor = new("Processor");
Workflow workflow = new WorkflowBuilder(requestPort)
.AddEdge(requestPort, processor)
.Build();
CheckpointManager checkpointManager = CheckpointManager.CreateInMemory();
InProcessExecutionEnvironment env = environment.ToWorkflowExecutionEnvironment();
// First run: collect a checkpoint with pending requests.
CheckpointInfo? checkpoint = null;
await using (StreamingRun firstRun = await env.WithCheckpointing(checkpointManager)
.RunStreamingAsync(workflow, "Hello"))
{
await foreach (WorkflowEvent evt in firstRun.WatchStreamAsync(blockOnPendingRequest: false))
{
if (evt is SuperStepCompletedEvent step && step.CompletionInfo?.Checkpoint is { } cp)
{
checkpoint = cp;
}
}
checkpoint.Should().NotBeNull();
}
// Act: Resume with republishPendingEvents: false via the internal API.
await using StreamingRun resumed = await env.WithCheckpointing(checkpointManager)
.ResumeStreamingInternalAsync(workflow, checkpoint!, republishPendingEvents: false);
// Assert: No RequestInfoEvent should appear in the event stream.
int requestEventCount = 0;
using CancellationTokenSource cts = new(TimeSpan.FromSeconds(10));
await foreach (WorkflowEvent evt in resumed.WatchStreamAsync(blockOnPendingRequest: false, cts.Token))
{
if (evt is RequestInfoEvent)
{
requestEventCount++;
}
}
requestEventCount.Should().Be(0,
"no RequestInfoEvent should be emitted when republishPendingEvents is false");
}
private static Workflow CreateSimpleRequestWorkflow(
string requestPortId = "TestPort",
string processorId = "Processor")
{
RequestPort<string, string> requestPort = RequestPort.Create<string, string>(requestPortId);
ForwardMessageExecutor<string> processor = new(processorId);
return new WorkflowBuilder(requestPort)
.AddEdge(requestPort, processor)
.Build();
}
private static Workflow CreateCheckpointedSubworkflowRequestWorkflow()
{
ExecutorBinding subworkflow = CreateSimpleRequestWorkflow(
requestPortId: "InnerTestPort",
processorId: "InnerProcessor")
.BindAsExecutor("Subworkflow");
return new WorkflowBuilder(subworkflow)
.AddExternalRequest<string, string>(subworkflow, id: "ForwardedSubworkflowRequest")
.Build();
}
private static async ValueTask<(ExternalRequest PendingRequest, CheckpointInfo Checkpoint)> CapturePendingRequestAndCheckpointAsync(StreamingRun run)
{
ExternalRequest? pendingRequest = null;
CheckpointInfo? checkpoint = null;
await foreach (WorkflowEvent evt in run.WatchStreamAsync(blockOnPendingRequest: false))
{
if (evt is RequestInfoEvent requestInfo)
{
pendingRequest ??= requestInfo.Request;
}
if (evt is SuperStepCompletedEvent step && step.CompletionInfo?.Checkpoint is { } cp)
{
checkpoint = cp;
}
}
pendingRequest.Should().NotBeNull("the workflow should have emitted a pending request");
checkpoint.Should().NotBeNull("the workflow should have produced a checkpoint");
return (pendingRequest!, checkpoint!);
}
private static async ValueTask<List<WorkflowEvent>> ReadToHaltAsync(StreamingRun run)
{
List<WorkflowEvent> events = [];
using CancellationTokenSource cts = new(TimeSpan.FromSeconds(10));
await foreach (WorkflowEvent evt in run.WatchStreamAsync(blockOnPendingRequest: false, cts.Token))
{
events.Add(evt);
}
return events;
}
}
+2 -2
View File
@@ -17,8 +17,8 @@ OPENAI_CHAT_MODEL=""
OPENAI_RESPONSES_MODEL=""
# Azure OpenAI
AZURE_OPENAI_ENDPOINT=""
AZURE_OPENAI_CHAT_DEPLOYMENT_NAME=""
AZURE_OPENAI_RESPONSES_DEPLOYMENT_NAME=""
AZURE_OPENAI_CHAT_MODEL=""
AZURE_OPENAI_RESPONSES_MODEL=""
# Mem0
MEM0_API_KEY=""
# Copilot Studio
+4 -4
View File
@@ -480,7 +480,7 @@ A more complete example with keyword arguments and code samples:
```python
def create_client(
model_id: str | None = None,
model: str | None = None,
*,
timeout: float | None = None,
env_file_path: str | None = None,
@@ -489,7 +489,7 @@ def create_client(
"""Create a new client with the specified configuration.
Args:
model_id: The model ID to use. If not provided,
model: The model ID to use. If not provided,
it will be loaded from settings.
Keyword Args:
@@ -501,14 +501,14 @@ def create_client(
A configured client instance.
Raises:
ValueError: If the model_id is invalid.
ValueError: If the model is invalid.
Examples:
.. code-block:: python
# Create a client with default settings:
client = create_client(model_id="gpt-4o")
client = create_client(model="gpt-4o")
# Or load from environment:
client = create_client(env_file_path=".env")
+1 -1
View File
@@ -51,7 +51,7 @@ OPENAI_MODEL=...
...
AZURE_OPENAI_API_KEY=...
AZURE_OPENAI_ENDPOINT=...
AZURE_OPENAI_DEPLOYMENT_NAME=...
AZURE_OPENAI_MODEL=...
...
FOUNDRY_PROJECT_ENDPOINT=...
FOUNDRY_MODEL=...
@@ -35,9 +35,9 @@ from agent_framework import (
AgentResponseUpdate,
AgentSession,
BaseAgent,
BaseHistoryProvider,
Content,
ContinuationToken,
HistoryProvider,
Message,
ResponseStream,
SessionContext,
@@ -353,7 +353,7 @@ class A2AAgent(AgentTelemetryLayer, BaseAgent):
# Run before_run providers (forward order)
for provider in self.context_providers:
if isinstance(provider, BaseHistoryProvider) and not provider.load_messages:
if isinstance(provider, HistoryProvider) and not provider.load_messages:
continue
if session is None:
raise RuntimeError("Provider session must be available when context providers are configured.")
+2 -2
View File
@@ -24,8 +24,8 @@ from agent_framework import (
AgentResponse,
AgentResponseUpdate,
AgentSession,
BaseContextProvider,
Content,
ContextProvider,
Message,
SessionContext,
)
@@ -869,7 +869,7 @@ async def test_poll_task_completed(a2a_agent: A2AAgent, mock_a2a_client: MockA2A
# region Context Provider Tests
class TrackingContextProvider(BaseContextProvider):
class TrackingContextProvider(ContextProvider):
"""A context provider that records when before_run and after_run are called."""
def __init__(self) -> None:
@@ -108,7 +108,7 @@ class AGUIChatOptions(ChatOptions[ResponseModelT], Generic[ResponseModelT], tota
Keys:
# Inherited from ChatOptions (forwarded to remote server):
model_id: The model identifier (forwarded as-is to server).
model: The model identifier (forwarded as-is to server).
temperature: Sampling temperature.
top_p: Nucleus sampling parameter.
max_tokens: Maximum tokens to generate.
@@ -191,13 +191,13 @@ from fastapi import FastAPI
# Read required configuration
endpoint = os.environ.get("AZURE_OPENAI_ENDPOINT")
deployment_name = os.environ.get("AZURE_OPENAI_DEPLOYMENT_NAME")
model = os.environ.get("AZURE_OPENAI_MODEL")
api_key = os.environ.get("AZURE_OPENAI_API_KEY")
if not endpoint:
raise ValueError("AZURE_OPENAI_ENDPOINT environment variable is required")
if not deployment_name:
raise ValueError("AZURE_OPENAI_DEPLOYMENT_NAME environment variable is required")
if not model:
raise ValueError("AZURE_OPENAI_MODEL environment variable is required")
if not api_key:
raise ValueError("AZURE_OPENAI_API_KEY environment variable is required")
@@ -207,7 +207,7 @@ agent = Agent(
instructions="You are a helpful assistant.",
client=OpenAIChatCompletionClient(
azure_endpoint=endpoint,
model=deployment_name,
model=model,
api_key=api_key,
),
)
@@ -230,7 +230,7 @@ if __name__ == "__main__":
- **`Agent`**: The agent that will handle incoming requests
- **FastAPI Integration**: Uses FastAPI's native async support for streaming responses
- **Instructions**: The agent is created with default instructions, which can be overridden by client messages
- **Configuration**: `OpenAIChatCompletionClient` can read from environment variables (`AZURE_OPENAI_ENDPOINT`, `AZURE_OPENAI_DEPLOYMENT_NAME`, `AZURE_OPENAI_API_KEY`) or accept parameters directly
- **Configuration**: `OpenAIChatCompletionClient` can read from environment variables (`AZURE_OPENAI_ENDPOINT`, `AZURE_OPENAI_MODEL`, `AZURE_OPENAI_API_KEY`) or accept parameters directly
**Alternative (simpler)**: Use environment variables only:
@@ -249,7 +249,7 @@ Set the required environment variables:
```bash
export AZURE_OPENAI_ENDPOINT="https://your-resource.openai.azure.com/"
export AZURE_OPENAI_DEPLOYMENT_NAME="gpt-4o-mini"
export AZURE_OPENAI_MODEL="gpt-4o-mini"
# Optional: Set API key if not using DefaultAzureCredential
# export AZURE_OPENAI_API_KEY="your-api-key"
```
@@ -26,12 +26,12 @@ logger = logging.getLogger(__name__)
# Read required configuration
endpoint = os.environ.get("AZURE_OPENAI_ENDPOINT")
deployment_name = os.environ.get("AZURE_OPENAI_DEPLOYMENT_NAME")
model = os.environ.get("AZURE_OPENAI_MODEL")
if not endpoint:
raise ValueError("AZURE_OPENAI_ENDPOINT environment variable is required")
if not deployment_name:
raise ValueError("AZURE_OPENAI_DEPLOYMENT_NAME environment variable is required")
if not model:
raise ValueError("AZURE_OPENAI_MODEL environment variable is required")
# ============================================================================
@@ -121,7 +121,7 @@ agent = Agent(
instructions="You are a helpful assistant. Use get_weather for weather and get_time_zone for time zones.",
client=OpenAIChatCompletionClient(
azure_endpoint=endpoint,
model=deployment_name,
model=model,
),
tools=[get_time_zone], # ONLY server-side tools
)
+4 -1
View File
@@ -5,6 +5,9 @@ Integration with Anthropic's Claude API.
## Main Classes
- **`AnthropicClient`** - Chat client for Anthropic Claude models
- **`AnthropicFoundryClient`** - Anthropic chat client for Azure AI Foundry's Anthropic-compatible endpoint
- **`AnthropicBedrockClient`** - Anthropic chat client for Amazon Bedrock
- **`AnthropicVertexClient`** - Anthropic chat client for Google Vertex AI
- **`AnthropicChatOptions`** - Options TypedDict for Anthropic-specific parameters
## Usage
@@ -12,7 +15,7 @@ Integration with Anthropic's Claude API.
```python
from agent_framework.anthropic import AnthropicClient
client = AnthropicClient(model_id="claude-sonnet-4-20250514")
client = AnthropicClient(model="claude-sonnet-4-20250514")
response = await client.get_response("Hello")
```
+6
View File
@@ -10,6 +10,12 @@ pip install agent-framework-anthropic --pre
The Anthropic integration enables communication with the Anthropic API, allowing your Agent Framework applications to leverage Anthropic's capabilities.
The package also includes Anthropic-hosted transport wrappers for:
- Azure AI Foundry via `AnthropicFoundryClient`
- Amazon Bedrock via `AnthropicBedrockClient`
- Google Vertex AI via `AnthropicVertexClient`
### Basic Usage Example
See the [Anthropic agent examples](../../samples/02-agents/providers/anthropic/) which demonstrate:
@@ -2,7 +2,10 @@
import importlib.metadata
from ._bedrock_client import AnthropicBedrockClient, RawAnthropicBedrockClient
from ._chat_client import AnthropicChatOptions, AnthropicClient, RawAnthropicClient
from ._foundry_client import AnthropicFoundryClient, RawAnthropicFoundryClient
from ._vertex_client import AnthropicVertexClient, RawAnthropicVertexClient
try:
__version__ = importlib.metadata.version(__name__)
@@ -10,8 +13,14 @@ except importlib.metadata.PackageNotFoundError:
__version__ = "0.0.0" # Fallback for development mode
__all__ = [
"AnthropicBedrockClient",
"AnthropicChatOptions",
"AnthropicClient",
"AnthropicFoundryClient",
"AnthropicVertexClient",
"RawAnthropicBedrockClient",
"RawAnthropicClient",
"RawAnthropicFoundryClient",
"RawAnthropicVertexClient",
"__version__",
]
@@ -0,0 +1,168 @@
# Copyright (c) Microsoft. All rights reserved.
from __future__ import annotations
from collections.abc import Sequence
from typing import Any, ClassVar, Generic, TypedDict
from agent_framework import (
AGENT_FRAMEWORK_USER_AGENT,
ChatAndFunctionMiddlewareTypes,
ChatMiddlewareLayer,
FunctionInvocationConfiguration,
FunctionInvocationLayer,
)
from agent_framework._settings import SecretString, load_settings
from agent_framework.observability import ChatTelemetryLayer
from anthropic import AsyncAnthropicBedrock
from ._chat_client import AnthropicOptionsT, RawAnthropicClient
class AnthropicBedrockSettings(TypedDict, total=False):
"""Resolved settings for Anthropic Bedrock wrappers."""
aws_access_key_id: SecretString | None
aws_secret_access_key: SecretString | None
aws_region: str | None
aws_profile: str | None
aws_session_token: SecretString | None
anthropic_bedrock_base_url: str | None
anthropic_chat_model: str | None
class RawAnthropicBedrockClient(RawAnthropicClient[AnthropicOptionsT], Generic[AnthropicOptionsT]):
"""Raw Anthropic Bedrock chat client without middleware, telemetry, or function invocation support."""
OTEL_PROVIDER_NAME: ClassVar[str] = "aws.bedrock" # type: ignore[reportIncompatibleVariableOverride, misc]
def __init__(
self,
*,
model: str | None = None,
aws_secret_key: str | None = None,
aws_access_key: str | None = None,
aws_region: str | None = None,
aws_profile: str | None = None,
aws_session_token: str | None = None,
base_url: str | None = None,
anthropic_client: AsyncAnthropicBedrock | None = None,
additional_beta_flags: list[str] | None = None,
additional_properties: dict[str, Any] | None = None,
env_file_path: str | None = None,
env_file_encoding: str | None = None,
) -> None:
"""Initialize a raw Anthropic Bedrock client.
Keyword Args:
model: The Anthropic model to use.
aws_secret_key: AWS secret access key.
aws_access_key: AWS access key ID.
aws_region: AWS region.
aws_profile: AWS profile name.
aws_session_token: AWS session token.
base_url: Optional custom Anthropic Bedrock base URL.
anthropic_client: Existing AsyncAnthropicBedrock client to reuse.
additional_beta_flags: Additional beta flags to enable on the client.
additional_properties: Additional properties stored on the client instance.
env_file_path: Path to environment file for loading settings.
env_file_encoding: Encoding of the environment file.
"""
settings = load_settings(
AnthropicBedrockSettings,
env_prefix="",
aws_access_key_id=aws_access_key,
aws_secret_access_key=aws_secret_key,
aws_region=aws_region,
aws_profile=aws_profile,
aws_session_token=aws_session_token,
anthropic_bedrock_base_url=base_url,
anthropic_chat_model=model,
env_file_path=env_file_path,
env_file_encoding=env_file_encoding,
)
model_setting = settings.get("anthropic_chat_model")
access_key_secret = settings.get("aws_access_key_id")
secret_key_secret = settings.get("aws_secret_access_key")
session_token_secret = settings.get("aws_session_token")
if anthropic_client is None:
anthropic_client = AsyncAnthropicBedrock(
aws_secret_key=secret_key_secret.get_secret_value() if secret_key_secret is not None else None,
aws_access_key=access_key_secret.get_secret_value() if access_key_secret is not None else None,
aws_region=settings.get("aws_region"),
aws_profile=settings.get("aws_profile"),
aws_session_token=session_token_secret.get_secret_value() if session_token_secret is not None else None,
base_url=settings.get("anthropic_bedrock_base_url"),
default_headers={"User-Agent": AGENT_FRAMEWORK_USER_AGENT},
)
super().__init__(
model=model_setting,
anthropic_client=anthropic_client,
additional_beta_flags=additional_beta_flags,
additional_properties=additional_properties,
)
class AnthropicBedrockClient( # type: ignore[misc]
FunctionInvocationLayer[AnthropicOptionsT],
ChatMiddlewareLayer[AnthropicOptionsT],
ChatTelemetryLayer[AnthropicOptionsT],
RawAnthropicBedrockClient[AnthropicOptionsT],
Generic[AnthropicOptionsT],
):
"""Anthropic Bedrock chat client with middleware, telemetry, and function invocation support."""
def __init__(
self,
*,
model: str | None = None,
aws_secret_key: str | None = None,
aws_access_key: str | None = None,
aws_region: str | None = None,
aws_profile: str | None = None,
aws_session_token: str | None = None,
base_url: str | None = None,
anthropic_client: AsyncAnthropicBedrock | None = None,
additional_beta_flags: list[str] | None = None,
additional_properties: dict[str, Any] | None = None,
middleware: Sequence[ChatAndFunctionMiddlewareTypes] | None = None,
function_invocation_configuration: FunctionInvocationConfiguration | None = None,
env_file_path: str | None = None,
env_file_encoding: str | None = None,
) -> None:
"""Initialize an Anthropic Bedrock client.
Keyword Args:
model: The Anthropic model to use.
aws_secret_key: AWS secret access key.
aws_access_key: AWS access key ID.
aws_region: AWS region.
aws_profile: AWS profile name.
aws_session_token: AWS session token.
base_url: Optional custom Anthropic Bedrock base URL.
anthropic_client: Existing AsyncAnthropicBedrock client to reuse.
additional_beta_flags: Additional beta flags to enable on the client.
additional_properties: Additional properties stored on the client instance.
middleware: Optional middleware to apply to the client.
function_invocation_configuration: Optional function invocation configuration override.
env_file_path: Path to environment file for loading settings.
env_file_encoding: Encoding of the environment file.
"""
super().__init__(
model=model,
aws_secret_key=aws_secret_key,
aws_access_key=aws_access_key,
aws_region=aws_region,
aws_profile=aws_profile,
aws_session_token=aws_session_token,
base_url=base_url,
anthropic_client=anthropic_client,
additional_beta_flags=additional_beta_flags,
additional_properties=additional_properties,
middleware=middleware,
function_invocation_configuration=function_invocation_configuration,
env_file_path=env_file_path,
env_file_encoding=env_file_encoding,
)
@@ -31,7 +31,7 @@ from agent_framework._settings import SecretString, load_settings
from agent_framework._tools import SHELL_TOOL_KIND_VALUE
from agent_framework._types import _get_data_bytes_as_str # type: ignore
from agent_framework.observability import ChatTelemetryLayer
from anthropic import AsyncAnthropic
from anthropic import AsyncAnthropic, AsyncAnthropicBedrock, AsyncAnthropicFoundry, AsyncAnthropicVertex
from anthropic.types.beta import (
BetaContentBlock,
BetaMessage,
@@ -79,6 +79,7 @@ BETA_FLAGS: Final[list[str]] = ["mcp-client-2025-04-04", "code-execution-2025-08
STRUCTURED_OUTPUTS_BETA_FLAG: Final[str] = "structured-outputs-2025-11-13"
ResponseModelT = TypeVar("ResponseModelT", bound=BaseModel | None, default=None)
AnthropicAsyncClient = AsyncAnthropic | AsyncAnthropicBedrock | AsyncAnthropicFoundry | AsyncAnthropicVertex
# region Anthropic Chat Options TypedDict
@@ -113,8 +114,6 @@ class AnthropicChatOptions(ChatOptions[ResponseModelT], Generic[ResponseModelT],
a default of 1024 will be used.
Keys:
model_id: The model to use for the request,
translates to ``model`` in Anthropic API.
temperature: Sampling temperature between 0 and 1.
top_p: Nucleus sampling parameter.
max_tokens: Maximum number of tokens to generate (REQUIRED).
@@ -169,12 +168,24 @@ AnthropicOptionsT = TypeVar(
# Translation between framework options keys and Anthropic Messages API
OPTION_TRANSLATIONS: dict[str, str] = {
"model_id": "model",
"stop": "stop_sequences",
"instructions": "system",
}
def _apply_option_translations(options: dict[str, Any]) -> None:
"""Translate framework option keys to Anthropic request keys in-place.
When both the old and new key are present, the new key wins and the old key
is discarded to preserve explicit overrides.
"""
for old_key, new_key in OPTION_TRANSLATIONS.items():
if old_key not in options or old_key == new_key:
continue
old_value = options.pop(old_key)
options.setdefault(new_key, old_value)
# region Role and Finish Reason Maps
@@ -204,11 +215,11 @@ class AnthropicSettings(TypedDict, total=False):
Keys:
api_key: The Anthropic API key.
chat_model_id: The Anthropic chat model ID.
chat_model: The Anthropic chat model.
"""
api_key: SecretString | None
chat_model_id: str | None
chat_model: str | None
class RawAnthropicClient(
@@ -236,8 +247,8 @@ class RawAnthropicClient(
self,
*,
api_key: str | None = None,
model_id: str | None = None,
anthropic_client: AsyncAnthropic | None = None,
model: str | None = None,
anthropic_client: AnthropicAsyncClient | None = None,
additional_beta_flags: list[str] | None = None,
additional_properties: dict[str, Any] | None = None,
env_file_path: str | None = None,
@@ -247,7 +258,7 @@ class RawAnthropicClient(
Keyword Args:
api_key: The Anthropic API key to use for authentication.
model_id: The ID of the model to use.
model: The model to use.
anthropic_client: An existing Anthropic client to use. If not provided, one will be created.
This can be used to further configure the client before passing it in.
For instance if you need to set a different base_url for testing or private deployments.
@@ -265,11 +276,11 @@ class RawAnthropicClient(
# Using environment variables
# Set ANTHROPIC_API_KEY=your_anthropic_api_key
# ANTHROPIC_CHAT_MODEL_ID=claude-sonnet-4-5-20250929
# ANTHROPIC_CHAT_MODEL=claude-sonnet-4-5-20250929
# Or passing parameters directly
client = RawAnthropicClient(
model_id="claude-sonnet-4-5-20250929",
model="claude-sonnet-4-5-20250929",
api_key="your_anthropic_api_key",
)
@@ -283,7 +294,7 @@ class RawAnthropicClient(
api_key="your_anthropic_api_key", base_url="https://custom-anthropic-endpoint.com"
)
client = RawAnthropicClient(
model_id="claude-sonnet-4-5-20250929",
model="claude-sonnet-4-5-20250929",
anthropic_client=anthropic_client,
)
@@ -296,7 +307,7 @@ class RawAnthropicClient(
my_custom_option: str
client: RawAnthropicClient[MyOptions] = RawAnthropicClient(model_id="claude-sonnet-4-5-20250929")
client: RawAnthropicClient[MyOptions] = RawAnthropicClient(model="claude-sonnet-4-5-20250929")
response = await client.get_response("Hello", options={"my_custom_option": "value"})
"""
@@ -304,13 +315,13 @@ class RawAnthropicClient(
AnthropicSettings,
env_prefix="ANTHROPIC_",
api_key=api_key,
chat_model_id=model_id,
chat_model=model,
env_file_path=env_file_path,
env_file_encoding=env_file_encoding,
)
api_key_secret = anthropic_settings.get("api_key")
model_id_setting = anthropic_settings.get("chat_model_id")
model_setting = anthropic_settings.get("chat_model")
if anthropic_client is None:
if api_key_secret is None:
@@ -332,7 +343,7 @@ class RawAnthropicClient(
# Initialize instance variables
self.anthropic_client = anthropic_client
self.additional_beta_flags = additional_beta_flags or []
self.model_id = model_id_setting
self.model = model_setting
# streaming requires tracking the last function call ID, name, and content type
self._last_call_id_name: tuple[str, str] | None = None
self._last_call_content_type: str | None = None
@@ -513,7 +524,7 @@ class RawAnthropicClient(
if stream:
# Streaming mode
async def _stream() -> AsyncIterable[ChatResponseUpdate]:
async for chunk in await self.anthropic_client.beta.messages.create(**run_options, stream=True):
async for chunk in await self.anthropic_client.beta.messages.create(**run_options, stream=True): # type: ignore[misc]
parsed_chunk = self._process_stream_event(chunk)
if parsed_chunk:
yield parsed_chunk
@@ -522,7 +533,7 @@ class RawAnthropicClient(
# Non-streaming mode
async def _get_response() -> ChatResponse:
message = await self.anthropic_client.beta.messages.create(**run_options, stream=False)
message = await self.anthropic_client.beta.messages.create(**run_options, stream=False) # type: ignore[misc]
return self._process_message(message, options)
return _get_response()
@@ -561,16 +572,22 @@ class RawAnthropicClient(
# Stream mode is controlled explicitly at call sites.
run_options.pop("stream", None)
# Translation between options keys and Anthropic Messages API
for old_key, new_key in OPTION_TRANSLATIONS.items():
if old_key in run_options and old_key != new_key:
run_options[new_key] = run_options.pop(old_key)
_apply_option_translations(run_options)
# model id
# Filter out framework kwargs that should not be passed to the Anthropic API.
# This includes underscore-prefixed internal objects (like _function_middleware_pipeline)
# and framework kwargs like 'thread' and 'middleware'.
filtered_kwargs = {
k: v for k, v in kwargs.items() if not k.startswith("_") and k not in {"thread", "middleware"}
}
_apply_option_translations(filtered_kwargs)
run_options.update(filtered_kwargs)
# model
if not run_options.get("model"):
if not self.model_id:
raise ValueError("model_id must be a non-empty string")
run_options["model"] = self.model_id
if not self.model:
raise ValueError("model must be a non-empty string")
run_options["model"] = self.model
# max_tokens - Anthropic requires this, default if not provided
if not run_options.get("max_tokens"):
@@ -607,13 +624,6 @@ class RawAnthropicClient(
# Add the structured outputs beta flag
run_options["betas"].add(STRUCTURED_OUTPUTS_BETA_FLAG)
# Filter out framework kwargs that should not be passed to the Anthropic API.
# This includes underscore-prefixed internal objects (like _function_middleware_pipeline)
# and framework kwargs like 'thread' and 'middleware'.
filtered_kwargs = {
k: v for k, v in kwargs.items() if not k.startswith("_") and k not in {"thread", "middleware"}
}
run_options.update(filtered_kwargs)
return run_options
def _prepare_betas(self, options: Mapping[str, Any]) -> set[str]:
@@ -918,7 +928,7 @@ class RawAnthropicClient(
)
],
usage_details=self._parse_usage_from_anthropic(message.usage),
model_id=message.model,
model=message.model,
finish_reason=FINISH_REASON_MAP.get(message.stop_reason) if message.stop_reason else None,
response_format=options.get("response_format"),
raw_representation=message,
@@ -946,7 +956,7 @@ class RawAnthropicClient(
*self._parse_contents_from_anthropic(event.message.content),
*usage_details,
],
model_id=event.message.model,
model=event.message.model,
finish_reason=FINISH_REASON_MAP.get(event.message.stop_reason)
if event.message.stop_reason
else None,
@@ -1396,8 +1406,8 @@ class AnthropicClient(
self,
*,
api_key: str | None = None,
model_id: str | None = None,
anthropic_client: AsyncAnthropic | None = None,
model: str | None = None,
anthropic_client: AnthropicAsyncClient | None = None,
additional_beta_flags: list[str] | None = None,
additional_properties: dict[str, Any] | None = None,
middleware: Sequence[ChatAndFunctionMiddlewareTypes] | None = None,
@@ -1409,7 +1419,7 @@ class AnthropicClient(
Keyword Args:
api_key: The Anthropic API key to use for authentication.
model_id: The ID of the model to use.
model: The model to use.
anthropic_client: An existing Anthropic client to use. If not provided, one will be created.
This can be used to further configure the client before passing it in.
For instance if you need to set a different base_url for testing or private deployments.
@@ -1428,11 +1438,11 @@ class AnthropicClient(
# Using environment variables
# Set ANTHROPIC_API_KEY=your_anthropic_api_key
# ANTHROPIC_CHAT_MODEL_ID=claude-sonnet-4-5-20250929
# ANTHROPIC_CHAT_MODEL=claude-sonnet-4-5-20250929
# Or passing parameters directly
client = AnthropicClient(
model_id="claude-sonnet-4-5-20250929",
model="claude-sonnet-4-5-20250929",
api_key="your_anthropic_api_key",
)
@@ -1446,7 +1456,7 @@ class AnthropicClient(
api_key="your_anthropic_api_key", base_url="https://custom-anthropic-endpoint.com"
)
client = AnthropicClient(
model_id="claude-sonnet-4-5-20250929",
model="claude-sonnet-4-5-20250929",
anthropic_client=anthropic_client,
)
@@ -1459,12 +1469,12 @@ class AnthropicClient(
my_custom_option: str
client: AnthropicClient[MyOptions] = AnthropicClient(model_id="claude-sonnet-4-5-20250929")
client: AnthropicClient[MyOptions] = AnthropicClient(model="claude-sonnet-4-5-20250929")
response = await client.get_response("Hello", options={"my_custom_option": "value"})
"""
super().__init__(
api_key=api_key,
model_id=model_id,
model=model,
anthropic_client=anthropic_client,
additional_beta_flags=additional_beta_flags,
additional_properties=additional_properties,
@@ -0,0 +1,166 @@
# Copyright (c) Microsoft. All rights reserved.
from __future__ import annotations
from collections.abc import Awaitable, Callable, Sequence
from typing import Any, ClassVar, Generic, TypedDict
from agent_framework import (
AGENT_FRAMEWORK_USER_AGENT,
ChatAndFunctionMiddlewareTypes,
ChatMiddlewareLayer,
FunctionInvocationConfiguration,
FunctionInvocationLayer,
)
from agent_framework._settings import SecretString, load_settings
from agent_framework.observability import ChatTelemetryLayer
from anthropic import AsyncAnthropicFoundry
from ._chat_client import AnthropicOptionsT, RawAnthropicClient
AnthropicFoundryAzureADTokenProvider = Callable[[], str | Awaitable[str]]
class AnthropicFoundrySettings(TypedDict, total=False):
"""Resolved settings for Anthropic Foundry wrappers."""
anthropic_foundry_api_key: SecretString | None
anthropic_foundry_resource: str | None
anthropic_foundry_base_url: str | None
anthropic_chat_model: str | None
class RawAnthropicFoundryClient(RawAnthropicClient[AnthropicOptionsT], Generic[AnthropicOptionsT]):
"""Raw Anthropic Foundry chat client without middleware, telemetry, or function invocation support."""
OTEL_PROVIDER_NAME: ClassVar[str] = "azure.ai.foundry" # type: ignore[reportIncompatibleVariableOverride, misc]
def __init__(
self,
*,
model: str | None = None,
resource: str | None = None,
api_key: str | None = None,
azure_ad_token_provider: AnthropicFoundryAzureADTokenProvider | None = None,
base_url: str | None = None,
anthropic_client: AsyncAnthropicFoundry | None = None,
additional_beta_flags: list[str] | None = None,
additional_properties: dict[str, Any] | None = None,
env_file_path: str | None = None,
env_file_encoding: str | None = None,
) -> None:
"""Initialize a raw Anthropic Foundry client.
Keyword Args:
model: The Anthropic model to use.
resource: The Foundry resource name.
api_key: The Foundry Anthropic API key.
azure_ad_token_provider: Azure AD token provider used by the Anthropic SDK.
base_url: Full Anthropic-compatible Foundry base URL.
anthropic_client: Existing AsyncAnthropicFoundry client to reuse.
additional_beta_flags: Additional beta flags to enable on the client.
additional_properties: Additional properties stored on the client instance.
env_file_path: Path to environment file for loading settings.
env_file_encoding: Encoding of the environment file.
"""
settings = load_settings(
AnthropicFoundrySettings,
env_prefix="",
anthropic_foundry_api_key=api_key,
anthropic_foundry_resource=resource,
anthropic_foundry_base_url=base_url,
anthropic_chat_model=model,
env_file_path=env_file_path,
env_file_encoding=env_file_encoding,
)
api_key_secret = settings.get("anthropic_foundry_api_key")
model_setting = settings.get("anthropic_chat_model")
resource_setting = settings.get("anthropic_foundry_resource")
base_url_setting = settings.get("anthropic_foundry_base_url")
api_key_value = api_key_secret.get_secret_value() if api_key_secret is not None else None
if anthropic_client is None:
if base_url_setting is None and resource_setting is None:
message = (
"Anthropic Foundry requires either `resource`/`ANTHROPIC_FOUNDRY_RESOURCE` "
"or `base_url`/`ANTHROPIC_FOUNDRY_BASE_URL`."
)
raise ValueError(message)
if base_url_setting is not None:
anthropic_client = AsyncAnthropicFoundry(
base_url=base_url_setting,
api_key=api_key_value,
azure_ad_token_provider=azure_ad_token_provider,
default_headers={"User-Agent": AGENT_FRAMEWORK_USER_AGENT},
)
else:
anthropic_client = AsyncAnthropicFoundry(
resource=resource_setting,
api_key=api_key_value,
azure_ad_token_provider=azure_ad_token_provider,
default_headers={"User-Agent": AGENT_FRAMEWORK_USER_AGENT},
)
super().__init__(
model=model_setting,
anthropic_client=anthropic_client,
additional_beta_flags=additional_beta_flags,
additional_properties=additional_properties,
)
class AnthropicFoundryClient( # type: ignore[misc]
FunctionInvocationLayer[AnthropicOptionsT],
ChatMiddlewareLayer[AnthropicOptionsT],
ChatTelemetryLayer[AnthropicOptionsT],
RawAnthropicFoundryClient[AnthropicOptionsT],
Generic[AnthropicOptionsT],
):
"""Anthropic Foundry chat client with middleware, telemetry, and function invocation support."""
def __init__(
self,
*,
model: str | None = None,
resource: str | None = None,
api_key: str | None = None,
azure_ad_token_provider: AnthropicFoundryAzureADTokenProvider | None = None,
base_url: str | None = None,
anthropic_client: AsyncAnthropicFoundry | None = None,
additional_beta_flags: list[str] | None = None,
additional_properties: dict[str, Any] | None = None,
middleware: Sequence[ChatAndFunctionMiddlewareTypes] | None = None,
function_invocation_configuration: FunctionInvocationConfiguration | None = None,
env_file_path: str | None = None,
env_file_encoding: str | None = None,
) -> None:
"""Initialize an Anthropic Foundry client.
Keyword Args:
model: The Anthropic model to use.
resource: The Foundry resource name.
api_key: The Foundry Anthropic API key.
azure_ad_token_provider: Azure AD token provider used by the Anthropic SDK.
base_url: Full Anthropic-compatible Foundry base URL.
anthropic_client: Existing AsyncAnthropicFoundry client to reuse.
additional_beta_flags: Additional beta flags to enable on the client.
additional_properties: Additional properties stored on the client instance.
middleware: Optional middleware to apply to the client.
function_invocation_configuration: Optional function invocation configuration override.
env_file_path: Path to environment file for loading settings.
env_file_encoding: Encoding of the environment file.
"""
super().__init__(
model=model,
resource=resource,
api_key=api_key,
azure_ad_token_provider=azure_ad_token_provider,
base_url=base_url,
anthropic_client=anthropic_client,
additional_beta_flags=additional_beta_flags,
additional_properties=additional_properties,
middleware=middleware,
function_invocation_configuration=function_invocation_configuration,
env_file_path=env_file_path,
env_file_encoding=env_file_encoding,
)
@@ -0,0 +1,160 @@
# Copyright (c) Microsoft. All rights reserved.
from __future__ import annotations
from collections.abc import Sequence
from typing import TYPE_CHECKING, Any, ClassVar, Generic, TypedDict
from agent_framework import (
AGENT_FRAMEWORK_USER_AGENT,
ChatAndFunctionMiddlewareTypes,
ChatMiddlewareLayer,
FunctionInvocationConfiguration,
FunctionInvocationLayer,
)
from agent_framework._settings import load_settings
from agent_framework.observability import ChatTelemetryLayer
from anthropic import NOT_GIVEN, AsyncAnthropicVertex
from ._chat_client import AnthropicOptionsT, RawAnthropicClient
if TYPE_CHECKING:
from google.auth.credentials import Credentials as GoogleCredentials
class AnthropicVertexSettings(TypedDict, total=False):
"""Resolved settings for Anthropic Vertex wrappers."""
cloud_ml_region: str | None
anthropic_vertex_project_id: str | None
anthropic_vertex_base_url: str | None
anthropic_chat_model: str | None
class RawAnthropicVertexClient(RawAnthropicClient[AnthropicOptionsT], Generic[AnthropicOptionsT]):
"""Raw Anthropic Vertex chat client without middleware, telemetry, or function invocation support."""
OTEL_PROVIDER_NAME: ClassVar[str] = "google.vertex.ai" # type: ignore[reportIncompatibleVariableOverride, misc]
def __init__(
self,
*,
model: str | None = None,
region: str | None = None,
project_id: str | None = None,
access_token: str | None = None,
credentials: GoogleCredentials | None = None,
base_url: str | None = None,
anthropic_client: AsyncAnthropicVertex | None = None,
additional_beta_flags: list[str] | None = None,
additional_properties: dict[str, Any] | None = None,
env_file_path: str | None = None,
env_file_encoding: str | None = None,
) -> None:
"""Initialize a raw Anthropic Vertex client.
Keyword Args:
model: The Anthropic model to use.
region: Vertex region. Falls back to `CLOUD_ML_REGION`.
project_id: Vertex project ID. Falls back to `ANTHROPIC_VERTEX_PROJECT_ID`.
access_token: Explicit OAuth access token.
credentials: Google credentials object.
base_url: Optional custom Anthropic Vertex base URL.
anthropic_client: Existing AsyncAnthropicVertex client to reuse.
additional_beta_flags: Additional beta flags to enable on the client.
additional_properties: Additional properties stored on the client instance.
env_file_path: Path to environment file for loading settings.
env_file_encoding: Encoding of the environment file.
"""
settings = load_settings(
AnthropicVertexSettings,
env_prefix="",
cloud_ml_region=region,
anthropic_vertex_project_id=project_id,
anthropic_vertex_base_url=base_url,
anthropic_chat_model=model,
env_file_path=env_file_path,
env_file_encoding=env_file_encoding,
)
model_setting = settings.get("anthropic_chat_model")
region_setting = settings.get("cloud_ml_region")
project_id_setting = settings.get("anthropic_vertex_project_id")
if anthropic_client is None:
resolved_region = region_setting if region_setting is not None else NOT_GIVEN
resolved_project_id = project_id_setting if project_id_setting is not None else NOT_GIVEN
anthropic_client = AsyncAnthropicVertex(
region=resolved_region,
project_id=resolved_project_id,
access_token=access_token,
credentials=credentials,
base_url=settings.get("anthropic_vertex_base_url"),
default_headers={"User-Agent": AGENT_FRAMEWORK_USER_AGENT},
)
super().__init__(
model=model_setting,
anthropic_client=anthropic_client,
additional_beta_flags=additional_beta_flags,
additional_properties=additional_properties,
)
class AnthropicVertexClient( # type: ignore[misc]
FunctionInvocationLayer[AnthropicOptionsT],
ChatMiddlewareLayer[AnthropicOptionsT],
ChatTelemetryLayer[AnthropicOptionsT],
RawAnthropicVertexClient[AnthropicOptionsT],
Generic[AnthropicOptionsT],
):
"""Anthropic Vertex chat client with middleware, telemetry, and function invocation support."""
def __init__(
self,
*,
model: str | None = None,
region: str | None = None,
project_id: str | None = None,
access_token: str | None = None,
credentials: GoogleCredentials | None = None,
base_url: str | None = None,
anthropic_client: AsyncAnthropicVertex | None = None,
additional_beta_flags: list[str] | None = None,
additional_properties: dict[str, Any] | None = None,
middleware: Sequence[ChatAndFunctionMiddlewareTypes] | None = None,
function_invocation_configuration: FunctionInvocationConfiguration | None = None,
env_file_path: str | None = None,
env_file_encoding: str | None = None,
) -> None:
"""Initialize an Anthropic Vertex client.
Keyword Args:
model: The Anthropic model to use.
region: Vertex region. Falls back to `CLOUD_ML_REGION`.
project_id: Vertex project ID. Falls back to `ANTHROPIC_VERTEX_PROJECT_ID`.
access_token: Explicit OAuth access token.
credentials: Google credentials object.
base_url: Optional custom Anthropic Vertex base URL.
anthropic_client: Existing AsyncAnthropicVertex client to reuse.
additional_beta_flags: Additional beta flags to enable on the client.
additional_properties: Additional properties stored on the client instance.
middleware: Optional middleware to apply to the client.
function_invocation_configuration: Optional function invocation configuration override.
env_file_path: Path to environment file for loading settings.
env_file_encoding: Encoding of the environment file.
"""
super().__init__(
model=model,
region=region,
project_id=project_id,
access_token=access_token,
credentials=credentials,
base_url=base_url,
anthropic_client=anthropic_client,
additional_beta_flags=additional_beta_flags,
additional_properties=additional_properties,
middleware=middleware,
function_invocation_configuration=function_invocation_configuration,
env_file_path=env_file_path,
env_file_encoding=env_file_encoding,
)
+1 -1
View File
@@ -28,7 +28,7 @@ def anthropic_unit_test_env(monkeypatch, exclude_list, override_env_param_dict):
env_vars = {
"ANTHROPIC_API_KEY": "test-api-key-12345",
"ANTHROPIC_CHAT_MODEL_ID": "claude-3-5-sonnet-20241022",
"ANTHROPIC_CHAT_MODEL": "claude-3-5-sonnet-20241022",
}
env_vars.update(override_env_param_dict) # type: ignore
@@ -40,7 +40,7 @@ skip_if_anthropic_integration_tests_disabled = pytest.mark.skipif(
def create_test_anthropic_client(
mock_anthropic_client: MagicMock,
model_id: str | None = None,
model: str | None = None,
anthropic_settings: AnthropicSettings | None = None,
) -> AnthropicClient:
"""Helper function to create AnthropicClient instances for testing, bypassing normal validation."""
@@ -51,7 +51,7 @@ def create_test_anthropic_client(
AnthropicSettings,
env_prefix="ANTHROPIC_",
api_key="test-api-key-12345",
chat_model_id="claude-3-5-sonnet-20241022",
chat_model="claude-3-5-sonnet-20241022",
)
# Create client instance directly
@@ -59,7 +59,7 @@ def create_test_anthropic_client(
# Set attributes directly
client.anthropic_client = mock_anthropic_client
client.model_id = model_id or anthropic_settings["chat_model_id"]
client.model = model or anthropic_settings["chat_model"]
client._last_call_id_name = None
client._tool_name_aliases = {}
client.additional_properties = {}
@@ -83,7 +83,7 @@ def test_anthropic_settings_init(anthropic_unit_test_env: dict[str, str]) -> Non
assert settings["api_key"] is not None
assert settings["api_key"].get_secret_value() == anthropic_unit_test_env["ANTHROPIC_API_KEY"]
assert settings["chat_model_id"] == anthropic_unit_test_env["ANTHROPIC_CHAT_MODEL_ID"]
assert settings["chat_model"] == anthropic_unit_test_env["ANTHROPIC_CHAT_MODEL"]
def test_anthropic_settings_init_with_explicit_values() -> None:
@@ -92,12 +92,12 @@ def test_anthropic_settings_init_with_explicit_values() -> None:
AnthropicSettings,
env_prefix="ANTHROPIC_",
api_key="custom-api-key",
chat_model_id="claude-3-opus-20240229",
chat_model="claude-3-opus-20240229",
)
assert settings["api_key"] is not None
assert settings["api_key"].get_secret_value() == "custom-api-key"
assert settings["chat_model_id"] == "claude-3-opus-20240229"
assert settings["chat_model"] == "claude-3-opus-20240229"
@pytest.mark.parametrize("exclude_list", [["ANTHROPIC_API_KEY"]], indirect=True)
@@ -107,7 +107,7 @@ def test_anthropic_settings_missing_api_key(
"""Test AnthropicSettings when API key is missing."""
settings = load_settings(AnthropicSettings, env_prefix="ANTHROPIC_")
assert settings["api_key"] is None
assert settings["chat_model_id"] == anthropic_unit_test_env["ANTHROPIC_CHAT_MODEL_ID"]
assert settings["chat_model"] == anthropic_unit_test_env["ANTHROPIC_CHAT_MODEL"]
# Client Initialization Tests
@@ -115,10 +115,10 @@ def test_anthropic_settings_missing_api_key(
def test_anthropic_client_init_with_client(mock_anthropic_client: MagicMock) -> None:
"""Test AnthropicClient initialization with existing anthropic_client."""
client = create_test_anthropic_client(mock_anthropic_client, model_id="claude-3-5-sonnet-20241022")
client = create_test_anthropic_client(mock_anthropic_client, model="claude-3-5-sonnet-20241022")
assert client.anthropic_client is mock_anthropic_client
assert client.model_id == "claude-3-5-sonnet-20241022"
assert client.model == "claude-3-5-sonnet-20241022"
assert isinstance(client, SupportsChatGetResponse)
@@ -141,11 +141,11 @@ def test_anthropic_client_init_auto_create_client(
"""Test AnthropicClient initialization with auto-created anthropic_client."""
client = AnthropicClient(
api_key=anthropic_unit_test_env["ANTHROPIC_API_KEY"],
model_id=anthropic_unit_test_env["ANTHROPIC_CHAT_MODEL_ID"],
model=anthropic_unit_test_env["ANTHROPIC_CHAT_MODEL"],
)
assert client.anthropic_client is not None
assert client.model_id == anthropic_unit_test_env["ANTHROPIC_CHAT_MODEL_ID"]
assert client.model == anthropic_unit_test_env["ANTHROPIC_CHAT_MODEL"]
def test_anthropic_client_init_missing_api_key() -> None:
@@ -153,7 +153,7 @@ def test_anthropic_client_init_missing_api_key() -> None:
with patch("agent_framework_anthropic._chat_client.load_settings") as mock_load:
mock_load.return_value = {
"api_key": None,
"chat_model_id": "claude-3-5-sonnet-20241022",
"chat_model": "claude-3-5-sonnet-20241022",
}
with pytest.raises(ValueError, match="Anthropic API key is required"):
@@ -740,7 +740,7 @@ async def test_prepare_options_basic(mock_anthropic_client: MagicMock) -> None:
run_options = client._prepare_options(messages, chat_options)
assert run_options["model"] == client.model_id
assert run_options["model"] == client.model
assert run_options["max_tokens"] == 100
assert run_options["temperature"] == 0.7
assert "messages" in run_options
@@ -980,7 +980,7 @@ def test_process_message_basic(mock_anthropic_client: MagicMock) -> None:
response = client._process_message(mock_message, {})
assert response.response_id == "msg_123"
assert response.model_id == "claude-3-5-sonnet-20241022"
assert response.model == "claude-3-5-sonnet-20241022"
assert len(response.messages) == 1
assert response.messages[0].role == "assistant"
assert len(response.messages[0].contents) == 1
@@ -992,6 +992,27 @@ def test_process_message_basic(mock_anthropic_client: MagicMock) -> None:
assert response.usage_details["output_token_count"] == 5
def test_process_message_with_dict_response_format(mock_anthropic_client: MagicMock) -> None:
"""_process_message should preserve dict response_format values for response.value parsing."""
client = create_test_anthropic_client(mock_anthropic_client)
mock_message = MagicMock(spec=BetaMessage)
mock_message.id = "msg_123"
mock_message.model = "claude-3-5-sonnet-20241022"
mock_message.content = [BetaTextBlock(type="text", text='{"greeting": "Hello"}')]
mock_message.usage = BetaUsage(input_tokens=10, output_tokens=5)
mock_message.stop_reason = "end_turn"
response = client._process_message(
mock_message,
options={"response_format": {"type": "object", "properties": {"greeting": {"type": "string"}}}},
)
assert response.value is not None
assert isinstance(response.value, dict)
assert response.value["greeting"] == "Hello"
def test_process_message_with_tool_use(mock_anthropic_client: MagicMock) -> None:
"""Test _process_message with tool use."""
client = create_test_anthropic_client(mock_anthropic_client)
@@ -2036,10 +2057,10 @@ def test_prepare_options_with_instructions(mock_anthropic_client: MagicMock) ->
assert result["max_tokens"] == 1024
def test_prepare_options_missing_model_id(mock_anthropic_client: MagicMock) -> None:
"""Test prepare_options raises error when model_id is missing."""
def test_prepare_options_missing_model(mock_anthropic_client: MagicMock) -> None:
"""Test prepare_options raises error when model is missing."""
client = create_test_anthropic_client(mock_anthropic_client)
client.model_id = "" # Set empty model_id
client.model = "" # Set empty model
messages = [Message(role="user", contents=[Content.from_text("Hello")])]
options = {}
@@ -2048,7 +2069,31 @@ def test_prepare_options_missing_model_id(mock_anthropic_client: MagicMock) -> N
client._prepare_options(messages, options)
raise AssertionError("Expected ValueError")
except ValueError as e:
assert "model_id must be a non-empty string" in str(e)
assert "model must be a non-empty string" in str(e)
def test_prepare_options_translates_model_option(mock_anthropic_client: MagicMock) -> None:
"""Test prepare_options translates model to model for runtime option compatibility."""
client = create_test_anthropic_client(mock_anthropic_client)
messages = [Message(role="user", contents=[Content.from_text("Hello")])]
result = client._prepare_options(messages, {"model": "claude-3-5-sonnet-20241022"})
assert result["model"] == "claude-3-5-sonnet-20241022"
assert "model_id" not in result
def test_prepare_options_translates_model_kwarg(mock_anthropic_client: MagicMock) -> None:
"""Test prepare_options translates model passed as a direct keyword argument."""
client = create_test_anthropic_client(mock_anthropic_client)
messages = [Message(role="user", contents=[Content.from_text("Hello")])]
result = client._prepare_options(messages, {}, model="claude-3-5-sonnet-20241022")
assert result["model"] == "claude-3-5-sonnet-20241022"
assert "model_id" not in result
def test_prepare_options_with_user_metadata(mock_anthropic_client: MagicMock) -> None:
@@ -0,0 +1,156 @@
# Copyright (c) Microsoft. All rights reserved.
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from agent_framework import AGENT_FRAMEWORK_USER_AGENT, ChatMiddlewareLayer, FunctionInvocationLayer
from agent_framework.observability import ChatTelemetryLayer
from agent_framework_anthropic import (
AnthropicBedrockClient,
AnthropicFoundryClient,
AnthropicVertexClient,
RawAnthropicBedrockClient,
RawAnthropicFoundryClient,
RawAnthropicVertexClient,
)
def _create_mock_transport(base_url: str) -> MagicMock:
transport = MagicMock()
transport.base_url = base_url
transport.beta = MagicMock()
transport.beta.messages = MagicMock()
transport.beta.messages.create = AsyncMock()
return transport
@pytest.mark.parametrize(
("public_client", "raw_client"),
[
(AnthropicFoundryClient, RawAnthropicFoundryClient),
(AnthropicBedrockClient, RawAnthropicBedrockClient),
(AnthropicVertexClient, RawAnthropicVertexClient),
],
)
def test_provider_client_wraps_raw_client_with_standard_layer_order(public_client, raw_client) -> None:
assert issubclass(public_client, raw_client)
mro = public_client.__mro__
assert mro.index(FunctionInvocationLayer) < mro.index(ChatMiddlewareLayer)
assert mro.index(ChatMiddlewareLayer) < mro.index(ChatTelemetryLayer)
assert mro.index(ChatTelemetryLayer) < mro.index(raw_client)
def test_raw_anthropic_foundry_client_creates_sdk_client_from_settings(tmp_path) -> None:
env_file = tmp_path / ".env"
env_file.write_text(
"ANTHROPIC_CHAT_MODEL=claude-foundry-test\n"
"ANTHROPIC_FOUNDRY_API_KEY=test-key\n"
"ANTHROPIC_FOUNDRY_RESOURCE=test-resource\n"
)
mock_transport = _create_mock_transport("https://test-resource.services.ai.azure.com/anthropic/")
with patch(
"agent_framework_anthropic._foundry_client.AsyncAnthropicFoundry", return_value=mock_transport
) as factory:
client = RawAnthropicFoundryClient(env_file_path=str(env_file))
assert client.model == "claude-foundry-test"
assert client.anthropic_client is mock_transport
factory.assert_called_once_with(
resource="test-resource",
api_key="test-key",
azure_ad_token_provider=None,
default_headers={"User-Agent": AGENT_FRAMEWORK_USER_AGENT},
)
def test_raw_anthropic_foundry_client_creates_sdk_client_from_base_url_settings(tmp_path) -> None:
env_file = tmp_path / ".env"
env_file.write_text(
"ANTHROPIC_CHAT_MODEL=claude-foundry-test\n"
"ANTHROPIC_FOUNDRY_API_KEY=test-key\n"
"ANTHROPIC_FOUNDRY_BASE_URL=https://test-resource.services.ai.azure.com/anthropic/\n"
)
mock_transport = _create_mock_transport("https://test-resource.services.ai.azure.com/anthropic/")
with patch(
"agent_framework_anthropic._foundry_client.AsyncAnthropicFoundry", return_value=mock_transport
) as factory:
client = RawAnthropicFoundryClient(env_file_path=str(env_file))
assert client.model == "claude-foundry-test"
assert client.anthropic_client is mock_transport
factory.assert_called_once_with(
base_url="https://test-resource.services.ai.azure.com/anthropic/",
api_key="test-key",
azure_ad_token_provider=None,
default_headers={"User-Agent": AGENT_FRAMEWORK_USER_AGENT},
)
def test_raw_anthropic_foundry_client_requires_resource_or_base_url() -> None:
with patch("agent_framework_anthropic._foundry_client.load_settings") as mock_load:
mock_load.return_value = {
"anthropic_foundry_api_key": None,
"anthropic_foundry_resource": None,
"anthropic_foundry_base_url": None,
"anthropic_chat_model": None,
}
with pytest.raises(
ValueError,
match=(
"Anthropic Foundry requires either `resource`/`ANTHROPIC_FOUNDRY_RESOURCE` "
"or `base_url`/`ANTHROPIC_FOUNDRY_BASE_URL`\\."
),
):
RawAnthropicFoundryClient()
def test_raw_anthropic_bedrock_client_creates_sdk_client_from_arguments() -> None:
mock_transport = _create_mock_transport("https://bedrock-runtime.us-east-1.amazonaws.com")
with patch(
"agent_framework_anthropic._bedrock_client.AsyncAnthropicBedrock", return_value=mock_transport
) as factory:
client = RawAnthropicBedrockClient(
model="claude-bedrock-test",
aws_access_key="access-key",
aws_secret_key="secret-key",
aws_region="us-east-1",
)
assert client.model == "claude-bedrock-test"
assert client.anthropic_client is mock_transport
factory.assert_called_once_with(
aws_secret_key="secret-key",
aws_access_key="access-key",
aws_region="us-east-1",
aws_profile=None,
aws_session_token=None,
base_url=None,
default_headers={"User-Agent": AGENT_FRAMEWORK_USER_AGENT},
)
def test_raw_anthropic_vertex_client_creates_sdk_client_from_arguments() -> None:
mock_transport = _create_mock_transport("https://us-central1-aiplatform.googleapis.com/v1")
with patch("agent_framework_anthropic._vertex_client.AsyncAnthropicVertex", return_value=mock_transport) as factory:
client = RawAnthropicVertexClient(
model="claude-vertex-test",
region="us-central1",
project_id="test-project",
)
assert client.model == "claude-vertex-test"
assert client.anthropic_client is mock_transport
factory.assert_called_once_with(
region="us-central1",
project_id="test-project",
access_token=None,
credentials=None,
base_url=None,
default_headers={"User-Agent": AGENT_FRAMEWORK_USER_AGENT},
)
@@ -1,9 +1,9 @@
# Copyright (c) Microsoft. All rights reserved.
"""New-pattern Azure AI Search context provider using BaseContextProvider.
"""New-pattern Azure AI Search context provider using ContextProvider.
This module provides ``AzureAISearchContextProvider``, built on the new
:class:`BaseContextProvider` hooks pattern.
:class:`ContextProvider` hooks pattern.
"""
from __future__ import annotations
@@ -11,11 +11,21 @@ from __future__ import annotations
import logging
import sys
from collections.abc import Awaitable, Callable
from typing import TYPE_CHECKING, Any, ClassVar, Literal, TypedDict
from typing import TYPE_CHECKING, Any, ClassVar, Literal, TypedDict, overload
from agent_framework import AGENT_FRAMEWORK_USER_AGENT, Annotation, Content, Message, SupportsGetEmbeddings
from agent_framework._sessions import AgentSession, BaseContextProvider, SessionContext
from agent_framework._settings import SecretString, load_settings
from agent_framework import (
AGENT_FRAMEWORK_USER_AGENT,
AgentSession,
Annotation,
Content,
ContextProvider,
Message,
SecretString,
SessionContext,
SupportsGetEmbeddings,
load_settings,
)
from agent_framework.exceptions import SettingNotFoundError
from azure.core.credentials import AzureKeyCredential, TokenCredential
from azure.core.credentials_async import AsyncTokenCredential
from azure.core.exceptions import ResourceNotFoundError
@@ -111,6 +121,9 @@ except ImportError:
_agentic_retrieval_available = False
AzureCredentialTypes = TokenCredential | AsyncTokenCredential
EmbeddingFunction = Callable[[str], Awaitable[list[float]]] | SupportsGetEmbeddings[str, list[float], Any]
KnowledgeBaseOutputModeLiteral = Literal["extractive_data", "answer_synthesis"]
RetrievalReasoningEffortLiteral = Literal["minimal", "medium", "low"]
logger = logging.getLogger("agent_framework.azure_ai_search")
@@ -141,8 +154,8 @@ class AzureAISearchSettings(TypedDict, total=False):
api_key: SecretString | None
class AzureAISearchContextProvider(BaseContextProvider):
"""Azure AI Search context provider using the new BaseContextProvider hooks pattern.
class AzureAISearchContextProvider(ContextProvider):
"""Azure AI Search context provider using the new ContextProvider hooks pattern.
Retrieves relevant context from Azure AI Search using semantic or agentic search
modes.
@@ -151,6 +164,222 @@ class AzureAISearchContextProvider(BaseContextProvider):
_DEFAULT_SEARCH_CONTEXT_PROMPT: ClassVar[str] = "Use the following context to answer the question:"
DEFAULT_SOURCE_ID: ClassVar[str] = "azure_ai_search"
@overload
def __init__(
self,
source_id: str = DEFAULT_SOURCE_ID,
endpoint: str | None = None,
index_name: str | None = None,
api_key: str | AzureKeyCredential | None = None,
credential: AzureCredentialTypes | None = None,
*,
mode: Literal["semantic"] = "semantic",
top_k: int = 5,
semantic_configuration_name: str | None = None,
vector_field_name: str | None = None,
embedding_function: EmbeddingFunction | None = None,
context_prompt: str | None = None,
azure_openai_resource_url: str | None = None,
model: str | None = None,
knowledge_base_name: None = None,
retrieval_instructions: str | None = None,
azure_openai_api_key: str | None = None,
knowledge_base_output_mode: KnowledgeBaseOutputModeLiteral = "extractive_data",
retrieval_reasoning_effort: RetrievalReasoningEffortLiteral = "minimal",
agentic_message_history_count: int = _DEFAULT_AGENTIC_MESSAGE_HISTORY_COUNT,
env_file_path: str | None = None,
env_file_encoding: str | None = None,
) -> None:
"""Initialize a semantic Azure AI Search context provider.
Keyword Args:
source_id: Unique identifier for this provider instance.
endpoint: Azure AI Search endpoint URL.
index_name: Name of the search index to query.
api_key: API key for authentication.
credential: Azure credential for managed identity authentication.
mode: Must be ``"semantic"`` for this overload.
top_k: Maximum number of documents to retrieve.
semantic_configuration_name: Name of the semantic configuration in the index.
vector_field_name: Name of the vector field in the index.
embedding_function: Embedding provider used for vector search.
context_prompt: Custom prompt to prepend to retrieved context.
azure_openai_resource_url: Unused in semantic mode.
model: Unused in semantic mode.
knowledge_base_name: Must be ``None`` for this overload.
retrieval_instructions: Unused in semantic mode.
azure_openai_api_key: Unused in semantic mode.
knowledge_base_output_mode: Unused in semantic mode.
retrieval_reasoning_effort: Unused in semantic mode.
agentic_message_history_count: Unused in semantic mode.
env_file_path: Optional ``.env`` file checked before process environment variables.
env_file_encoding: Encoding for the ``.env`` file.
"""
...
@overload
def __init__(
self,
source_id: str = DEFAULT_SOURCE_ID,
endpoint: str | None = None,
index_name: str | None = None,
api_key: str | AzureKeyCredential | None = None,
credential: AzureCredentialTypes | None = None,
*,
mode: Literal["agentic"],
top_k: int = 5,
semantic_configuration_name: str | None = None,
vector_field_name: str | None = None,
embedding_function: EmbeddingFunction | None = None,
context_prompt: str | None = None,
azure_openai_resource_url: str,
model: str,
knowledge_base_name: None = None,
retrieval_instructions: str | None = None,
azure_openai_api_key: str | None = None,
knowledge_base_output_mode: KnowledgeBaseOutputModeLiteral = "extractive_data",
retrieval_reasoning_effort: RetrievalReasoningEffortLiteral = "minimal",
agentic_message_history_count: int = _DEFAULT_AGENTIC_MESSAGE_HISTORY_COUNT,
env_file_path: str | None = None,
env_file_encoding: str | None = None,
) -> None:
"""Initialize an agentic provider that creates a Knowledge Base from an index.
Keyword Args:
source_id: Unique identifier for this provider instance.
endpoint: Azure AI Search endpoint URL.
index_name: Name of the search index used to create the Knowledge Base.
api_key: API key for authentication.
credential: Azure credential for managed identity authentication.
mode: Must be ``"agentic"`` for this overload.
top_k: Maximum number of documents to retrieve.
semantic_configuration_name: Semantic configuration name used by hybrid search operations.
vector_field_name: Vector field name used by hybrid search operations.
embedding_function: Embedding provider used for vector search.
context_prompt: Custom prompt to prepend to retrieved context.
azure_openai_resource_url: Azure OpenAI resource URL for Knowledge Base creation.
model: Model used by the generated Knowledge Base.
knowledge_base_name: Must be ``None`` for this overload.
retrieval_instructions: Custom instructions for Knowledge Base retrieval.
azure_openai_api_key: Optional Azure OpenAI API key for Knowledge Base creation.
knowledge_base_output_mode: Output mode for Knowledge Base retrieval.
retrieval_reasoning_effort: Reasoning effort for query planning.
agentic_message_history_count: Number of recent messages included in retrieval.
env_file_path: Optional ``.env`` file checked before process environment variables.
env_file_encoding: Encoding for the ``.env`` file.
"""
...
@overload
def __init__(
self,
source_id: str = DEFAULT_SOURCE_ID,
endpoint: str | None = None,
index_name: None = None,
api_key: str | AzureKeyCredential | None = None,
credential: AzureCredentialTypes | None = None,
*,
mode: Literal["agentic"],
top_k: int = 5,
semantic_configuration_name: str | None = None,
vector_field_name: str | None = None,
embedding_function: EmbeddingFunction | None = None,
context_prompt: str | None = None,
azure_openai_resource_url: str | None = None,
model: str | None = None,
knowledge_base_name: str,
retrieval_instructions: str | None = None,
azure_openai_api_key: str | None = None,
knowledge_base_output_mode: KnowledgeBaseOutputModeLiteral = "extractive_data",
retrieval_reasoning_effort: RetrievalReasoningEffortLiteral = "minimal",
agentic_message_history_count: int = _DEFAULT_AGENTIC_MESSAGE_HISTORY_COUNT,
env_file_path: str | None = None,
env_file_encoding: str | None = None,
) -> None:
"""Initialize an agentic provider that connects to an existing Knowledge Base.
Keyword Args:
source_id: Unique identifier for this provider instance.
endpoint: Azure AI Search endpoint URL.
index_name: Must be ``None`` for this overload.
knowledge_base_name: Name of the existing Knowledge Base to use.
api_key: API key for authentication.
credential: Azure credential for managed identity authentication.
mode: Must be ``"agentic"`` for this overload.
top_k: Maximum number of documents to retrieve.
semantic_configuration_name: Semantic configuration name used by hybrid search operations.
vector_field_name: Vector field name used by hybrid search operations.
embedding_function: Embedding provider used for vector search.
context_prompt: Custom prompt to prepend to retrieved context.
azure_openai_resource_url: Unused when connecting to an existing Knowledge Base.
model: Unused when connecting to an existing Knowledge Base.
retrieval_instructions: Custom instructions for Knowledge Base retrieval.
azure_openai_api_key: Unused when connecting to an existing Knowledge Base.
knowledge_base_output_mode: Output mode for Knowledge Base retrieval.
retrieval_reasoning_effort: Reasoning effort for query planning.
agentic_message_history_count: Number of recent messages included in retrieval.
env_file_path: Optional ``.env`` file checked before process environment variables.
env_file_encoding: Encoding for the ``.env`` file.
"""
...
@overload
def __init__(
self,
source_id: str = DEFAULT_SOURCE_ID,
endpoint: str | None = None,
index_name: None = None,
api_key: str | AzureKeyCredential | None = None,
credential: AzureCredentialTypes | None = None,
*,
mode: Literal["agentic"],
top_k: int = 5,
semantic_configuration_name: str | None = None,
vector_field_name: str | None = None,
embedding_function: EmbeddingFunction | None = None,
context_prompt: str | None = None,
azure_openai_resource_url: str | None = None,
model: str | None = None,
knowledge_base_name: None = None,
retrieval_instructions: str | None = None,
azure_openai_api_key: str | None = None,
knowledge_base_output_mode: KnowledgeBaseOutputModeLiteral = "extractive_data",
retrieval_reasoning_effort: RetrievalReasoningEffortLiteral = "minimal",
agentic_message_history_count: int = _DEFAULT_AGENTIC_MESSAGE_HISTORY_COUNT,
env_file_path: str | None = None,
env_file_encoding: str | None = None,
) -> None:
"""Initialize an agentic provider using environment-resolved setup.
This overload is for agentic initialization where ``index_name`` or
``knowledge_base_name`` is supplied by ``env_file_path`` or the
``AZURE_SEARCH_*`` environment variables.
Keyword Args:
source_id: Unique identifier for this provider instance.
endpoint: Azure AI Search endpoint URL.
index_name: Resolved from ``env_file_path`` or ``AZURE_SEARCH_INDEX_NAME``.
api_key: API key for authentication.
credential: Azure credential for managed identity authentication.
mode: Must be ``"agentic"`` for this overload.
top_k: Maximum number of documents to retrieve.
semantic_configuration_name: Semantic configuration name used by hybrid search operations.
vector_field_name: Vector field name used by hybrid search operations.
embedding_function: Embedding provider used for vector search.
context_prompt: Custom prompt to prepend to retrieved context.
azure_openai_resource_url: Azure OpenAI resource URL when creating a Knowledge Base from an index.
model: Model used when creating a Knowledge Base from an index.
knowledge_base_name: Resolved from ``env_file_path`` or ``AZURE_SEARCH_KNOWLEDGE_BASE_NAME``.
retrieval_instructions: Custom instructions for Knowledge Base retrieval.
azure_openai_api_key: Optional Azure OpenAI API key for Knowledge Base creation.
knowledge_base_output_mode: Output mode for Knowledge Base retrieval.
retrieval_reasoning_effort: Reasoning effort for query planning.
agentic_message_history_count: Number of recent messages included in retrieval.
env_file_path: Optional ``.env`` file checked before process environment variables.
env_file_encoding: Encoding for the ``.env`` file.
"""
...
def __init__(
self,
source_id: str = DEFAULT_SOURCE_ID,
@@ -163,18 +392,15 @@ class AzureAISearchContextProvider(BaseContextProvider):
top_k: int = 5,
semantic_configuration_name: str | None = None,
vector_field_name: str | None = None,
embedding_function: Callable[[str], Awaitable[list[float]]]
| SupportsGetEmbeddings[str, list[float], Any]
| None = None,
embedding_function: EmbeddingFunction | None = None,
context_prompt: str | None = None,
azure_openai_resource_url: str | None = None,
model_deployment_name: str | None = None,
model_name: str | None = None,
model: str | None = None,
knowledge_base_name: str | None = None,
retrieval_instructions: str | None = None,
azure_openai_api_key: str | None = None,
knowledge_base_output_mode: Literal["extractive_data", "answer_synthesis"] = "extractive_data",
retrieval_reasoning_effort: Literal["minimal", "medium", "low"] = "minimal",
knowledge_base_output_mode: KnowledgeBaseOutputModeLiteral = "extractive_data",
retrieval_reasoning_effort: RetrievalReasoningEffortLiteral = "minimal",
agentic_message_history_count: int = _DEFAULT_AGENTIC_MESSAGE_HISTORY_COUNT,
env_file_path: str | None = None,
env_file_encoding: str | None = None,
@@ -184,7 +410,9 @@ class AzureAISearchContextProvider(BaseContextProvider):
Args:
source_id: Unique identifier for this provider instance.
endpoint: Azure AI Search endpoint URL.
index_name: Name of the search index to query.
index_name: Name of the search index to query. In agentic mode, providing this
explicitly selects the index-backed setup and ignores any environment-provided
knowledge base name.
api_key: API key for authentication.
credential: Azure credential for managed identity authentication.
Accepts a TokenCredential, AsyncTokenCredential, or a callable token provider.
@@ -195,8 +423,7 @@ class AzureAISearchContextProvider(BaseContextProvider):
embedding_function: Async function to generate embeddings or a SupportsGetEmbeddings instance.
context_prompt: Custom prompt to prepend to retrieved context.
azure_openai_resource_url: Azure OpenAI resource URL for Knowledge Base.
model_deployment_name: Model deployment name in Azure OpenAI.
model_name: The underlying model name.
model: Model name to use for Azure OpenAI vectorization.
knowledge_base_name: Name of an existing Knowledge Base to use.
retrieval_instructions: Custom instructions for Knowledge Base retrieval.
azure_openai_api_key: Azure OpenAI API key.
@@ -208,12 +435,26 @@ class AzureAISearchContextProvider(BaseContextProvider):
"""
super().__init__(source_id)
# Determine which fields are required based on mode
required: list[str | tuple[str, ...]] = ["endpoint"]
required: list[str | tuple[str, ...]]
ignored_agentic_field: Literal["index_name", "knowledge_base_name"] | None = None
explicit_index_name = index_name is not None
explicit_knowledge_base_name = knowledge_base_name is not None
if mode == "semantic":
required.append("index_name")
elif mode == "agentic":
required.append(("index_name", "knowledge_base_name"))
required = ["endpoint", "index_name"]
elif explicit_index_name and explicit_knowledge_base_name:
raise SettingNotFoundError(
"Only one of 'index_name', 'knowledge_base_name' may be provided, "
"but multiple were set: 'index_name', 'knowledge_base_name'."
)
elif explicit_index_name:
required = ["endpoint", "index_name"]
ignored_agentic_field = "knowledge_base_name"
elif explicit_knowledge_base_name:
required = ["endpoint", "knowledge_base_name"]
ignored_agentic_field = "index_name"
else:
required = ["endpoint", ("index_name", "knowledge_base_name")]
# Load settings from environment/file
settings = load_settings(
@@ -227,11 +468,11 @@ class AzureAISearchContextProvider(BaseContextProvider):
env_file_path=env_file_path,
env_file_encoding=env_file_encoding,
)
if ignored_agentic_field is not None:
settings[ignored_agentic_field] = None
if mode == "agentic" and settings.get("index_name") and not model_deployment_name:
raise ValueError(
"model_deployment_name is required for agentic mode when creating Knowledge Base from index."
)
if mode == "agentic" and settings.get("index_name") and not model:
raise ValueError("model is required for agentic mode when creating Knowledge Base from index.")
resolved_credential: AzureKeyCredential | AsyncTokenCredential
if credential:
@@ -257,8 +498,7 @@ class AzureAISearchContextProvider(BaseContextProvider):
self.context_prompt = context_prompt or self._DEFAULT_SEARCH_CONTEXT_PROMPT
self.azure_openai_resource_url = azure_openai_resource_url
self.azure_openai_deployment_name = model_deployment_name
self.model_name = model_name or model_deployment_name
self.azure_openai_model = model
self.knowledge_base_name = settings.get("knowledge_base_name")
self.retrieval_instructions = retrieval_instructions
self.azure_openai_api_key = azure_openai_api_key
@@ -507,8 +747,8 @@ class AzureAISearchContextProvider(BaseContextProvider):
raise ValueError("Index client is required when creating Knowledge Base from index")
if not self.azure_openai_resource_url:
raise ValueError("azure_openai_resource_url is required when creating Knowledge Base from index")
if not self.azure_openai_deployment_name:
raise ValueError("model_deployment_name is required when creating Knowledge Base from index")
if not self.azure_openai_model:
raise ValueError("model is required when creating Knowledge Base from index")
if not self.index_name:
raise ValueError("index_name is required when creating Knowledge Base from index")
@@ -527,8 +767,8 @@ class AzureAISearchContextProvider(BaseContextProvider):
aoai_params = AzureOpenAIVectorizerParameters(
resource_url=self.azure_openai_resource_url,
deployment_name=self.azure_openai_deployment_name,
model_name=self.model_name,
deployment_name=self.azure_openai_model,
model_name=self.azure_openai_model,
api_key=self.azure_openai_api_key,
)
@@ -155,14 +155,14 @@ class TestInitSemantic:
provider = _make_provider(context_prompt="Custom prompt:")
assert provider.context_prompt == "Custom prompt:"
def test_model_name_falls_back_to_deployment_name(self) -> None:
"""model_name defaults to model_deployment_name when not explicitly set."""
provider = _make_provider(model_deployment_name="my-deploy")
assert provider.model_name == "my-deploy"
def test_model_is_stored(self) -> None:
"""Model is stored on the provider for Azure OpenAI vectorization."""
provider = _make_provider(model="my-deploy")
assert provider.azure_openai_model == "my-deploy"
def test_model_name_explicit(self) -> None:
provider = _make_provider(model_deployment_name="deploy", model_name="gpt-4")
assert provider.model_name == "gpt-4"
def test_model_explicit(self) -> None:
provider = _make_provider(model="gpt-4")
assert provider.azure_openai_model == "gpt-4"
# -- Initialization: credential resolution ------------------------------------
@@ -214,7 +214,7 @@ class TestInitAgenticValidation:
knowledge_base_name="kb",
api_key="key",
mode="agentic",
model_deployment_name="deploy",
model="deploy",
azure_openai_resource_url="https://aoai.openai.azure.com",
)
@@ -227,8 +227,8 @@ class TestInitAgenticValidation:
mode="agentic",
)
def test_missing_model_deployment_name_raises(self) -> None:
with pytest.raises(ValueError, match="model_deployment_name"):
def test_missing_model_raises(self) -> None:
with pytest.raises(ValueError, match="model"):
AzureAISearchContextProvider(
source_id="s",
endpoint="https://test.search.windows.net",
@@ -256,7 +256,7 @@ class TestInitAgenticValidation:
index_name="idx",
api_key="key",
mode="agentic",
model_deployment_name="deploy",
model="deploy",
)
def test_agentic_with_kb_name_sets_use_existing(self) -> None:
@@ -277,12 +277,43 @@ class TestInitAgenticValidation:
index_name="idx",
api_key="key",
mode="agentic",
model_deployment_name="deploy",
model="deploy",
azure_openai_resource_url="https://aoai.openai.azure.com",
)
assert provider._use_existing_knowledge_base is False
assert provider.knowledge_base_name == "idx-kb"
def test_agentic_explicit_kb_ignores_env_index_name(self) -> None:
with patch.dict(os.environ, {"AZURE_SEARCH_INDEX_NAME": "env-index"}, clear=False):
provider = AzureAISearchContextProvider(
source_id="s",
endpoint="https://test.search.windows.net",
knowledge_base_name="my-kb",
api_key="key",
mode="agentic",
)
assert provider.index_name is None
assert provider.knowledge_base_name == "my-kb"
assert provider._use_existing_knowledge_base is True
assert provider._search_client is None
def test_agentic_explicit_index_ignores_env_kb_name(self) -> None:
with patch.dict(os.environ, {"AZURE_SEARCH_KNOWLEDGE_BASE_NAME": "env-kb"}, clear=False):
provider = AzureAISearchContextProvider(
source_id="s",
endpoint="https://test.search.windows.net",
index_name="idx",
api_key="key",
mode="agentic",
model="deploy",
azure_openai_resource_url="https://aoai.openai.azure.com",
)
assert provider.index_name == "idx"
assert provider.knowledge_base_name == "idx-kb"
assert provider._use_existing_knowledge_base is False
# -- __aenter__ / __aexit__ ---------------------------------------------------
@@ -980,9 +1011,9 @@ class TestEnsureKnowledgeBase:
provider.knowledge_base_name = "test-kb"
provider._index_client = AsyncMock()
provider.azure_openai_resource_url = "https://aoai.openai.azure.com"
provider.azure_openai_deployment_name = None
provider.azure_openai_model = None
with pytest.raises(ValueError, match="model_deployment_name is required"):
with pytest.raises(ValueError, match="model is required"):
await provider._ensure_knowledge_base()
async def test_missing_index_name_raises(self) -> None:
@@ -992,7 +1023,7 @@ class TestEnsureKnowledgeBase:
provider.knowledge_base_name = "test-kb"
provider._index_client = AsyncMock()
provider.azure_openai_resource_url = "https://aoai.openai.azure.com"
provider.azure_openai_deployment_name = "deploy"
provider.azure_openai_model = "deploy"
provider.index_name = None
with pytest.raises(ValueError, match="index_name is required"):
@@ -1006,8 +1037,7 @@ class TestEnsureKnowledgeBase:
provider._use_existing_knowledge_base = False
provider.knowledge_base_name = "test-kb"
provider.azure_openai_resource_url = "https://aoai.openai.azure.com"
provider.azure_openai_deployment_name = "deploy"
provider.model_name = "gpt-4"
provider.azure_openai_model = "gpt-4"
provider.index_name = "test-index"
mock_index_client = AsyncMock()
@@ -1030,8 +1060,7 @@ class TestEnsureKnowledgeBase:
provider._use_existing_knowledge_base = False
provider.knowledge_base_name = "test-kb"
provider.azure_openai_resource_url = "https://aoai.openai.azure.com"
provider.azure_openai_deployment_name = "deploy"
provider.model_name = "gpt-4"
provider.azure_openai_model = "gpt-4"
provider.index_name = "test-index"
mock_index_client = AsyncMock()
@@ -1052,8 +1081,7 @@ class TestEnsureKnowledgeBase:
provider._use_existing_knowledge_base = False
provider.knowledge_base_name = "test-kb"
provider.azure_openai_resource_url = "https://aoai.openai.azure.com"
provider.azure_openai_deployment_name = "deploy"
provider.model_name = "gpt-4"
provider.azure_openai_model = "gpt-4"
provider.index_name = "test-index"
provider.knowledge_base_output_mode = "answer_synthesis"
@@ -1074,8 +1102,7 @@ class TestEnsureKnowledgeBase:
provider._use_existing_knowledge_base = False
provider.knowledge_base_name = "test-kb"
provider.azure_openai_resource_url = "https://aoai.openai.azure.com"
provider.azure_openai_deployment_name = "deploy"
provider.model_name = "gpt-4"
provider.azure_openai_model = "gpt-4"
provider.index_name = "test-index"
provider.retrieval_reasoning_effort = "medium"
+1 -1
View File
@@ -18,7 +18,7 @@ from agent_framework_azure_ai import AzureAIInferenceEmbeddingClient
client = AzureAIInferenceEmbeddingClient(
endpoint="https://<resource>.inference.ai.azure.com",
api_key="...",
model_id="text-embedding-3-large",
model="text-embedding-3-large",
)
result = await client.get_embeddings(["Hello"])
```
@@ -44,7 +44,7 @@ class AzureAIInferenceEmbeddingOptions(EmbeddingGenerationOptions, total=False):
from agent_framework_azure_ai import AzureAIInferenceEmbeddingOptions
options: AzureAIInferenceEmbeddingOptions = {
"model_id": "text-embedding-3-small",
"model": "text-embedding-3-small",
"dimensions": 1536,
"input_type": "document",
"encoding_format": "float",
@@ -54,8 +54,8 @@ class AzureAIInferenceEmbeddingOptions(EmbeddingGenerationOptions, total=False):
input_type: str
"""Input type hint for the model. Common values: ``"text"``, ``"query"``, ``"document"``."""
image_model_id: str
"""Override model for image embeddings. Falls back to the client's ``image_model_id``."""
image_model: str
"""Override model for image embeddings. Falls back to the client's ``image_model``."""
encoding_format: str
"""Output encoding format.
@@ -81,8 +81,8 @@ class AzureAIInferenceEmbeddingSettings(TypedDict, total=False):
endpoint: str | None
api_key: str | None
embedding_model_id: str | None
image_embedding_model_id: str | None
embedding_model: str | None
image_embedding_model: str | None
class RawAzureAIInferenceEmbeddingClient(
@@ -97,11 +97,11 @@ class RawAzureAIInferenceEmbeddingClient(
are reassembled in the original input order.
Keyword Args:
model_id: The text embedding model deployment name (e.g. "text-embedding-3-small").
Can also be set via environment variable AZURE_AI_INFERENCE_EMBEDDING_MODEL_ID.
image_model_id: The image embedding model deployment name (e.g. "Cohere-embed-v3-english").
Can also be set via environment variable AZURE_AI_INFERENCE_IMAGE_EMBEDDING_MODEL_ID.
Falls back to ``model_id`` if not provided.
model: The text embedding model (e.g. "text-embedding-3-small").
Can also be set via environment variable AZURE_AI_INFERENCE_EMBEDDING_MODEL.
image_model: The image embedding model (e.g. "Cohere-embed-v3-english").
Can also be set via environment variable AZURE_AI_INFERENCE_IMAGE_EMBEDDING_MODEL.
Falls back to ``model`` if not provided.
endpoint: The Azure AI Inference endpoint URL.
Can also be set via environment variable AZURE_AI_INFERENCE_ENDPOINT.
api_key: API key for authentication.
@@ -117,8 +117,8 @@ class RawAzureAIInferenceEmbeddingClient(
def __init__(
self,
*,
model_id: str | None = None,
image_model_id: str | None = None,
model: str | None = None,
image_model: str | None = None,
endpoint: str | None = None,
api_key: str | None = None,
text_client: EmbeddingsClient | None = None,
@@ -132,17 +132,17 @@ class RawAzureAIInferenceEmbeddingClient(
settings = load_settings(
AzureAIInferenceEmbeddingSettings,
env_prefix="AZURE_AI_INFERENCE_",
required_fields=["endpoint", "embedding_model_id"],
required_fields=["endpoint", "embedding_model"],
endpoint=endpoint,
api_key=api_key,
embedding_model_id=model_id,
image_embedding_model_id=image_model_id,
embedding_model=model,
image_embedding_model=image_model,
env_file_path=env_file_path,
env_file_encoding=env_file_encoding,
)
self.model_id = settings["embedding_model_id"] # type: ignore[reportTypedDictNotRequiredAccess]
self.image_model_id: str = settings.get("image_embedding_model_id") or self.model_id # type: ignore[assignment]
self.model = settings["embedding_model"] # type: ignore[reportTypedDictNotRequiredAccess]
self.image_model: str = settings.get("image_embedding_model") or self.model # type: ignore[assignment]
resolved_endpoint = settings["endpoint"] # type: ignore[reportTypedDictNotRequiredAccess]
if credential is None and settings.get("api_key"):
@@ -202,7 +202,7 @@ class RawAzureAIInferenceEmbeddingClient(
Generated embeddings with usage metadata.
Raises:
ValueError: If model_id is not provided or an unsupported content type is encountered.
ValueError: If model is not provided or an unsupported content type is encountered.
"""
if not values:
return GeneratedEmbeddings([], options=options) # type: ignore[reportReturnType]
@@ -254,8 +254,8 @@ class RawAzureAIInferenceEmbeddingClient(
# Embed text inputs.
if text_items:
if not (text_model := opts.get("model_id") or self.model_id):
raise ValueError("An model_id is required, either in the client or options, for text inputs.")
if not (text_model := opts.get("model") or self.model):
raise ValueError("A model is required, either in the client or options, for text inputs.")
text_inputs = [t for _, t in text_items]
response = await self._text_client.embed(
input=text_inputs,
@@ -268,7 +268,7 @@ class RawAzureAIInferenceEmbeddingClient(
embeddings[original_idx] = Embedding(
vector=vector,
dimensions=len(vector),
model_id=response.model or text_model,
model=response.model or text_model,
)
if response.usage:
usage_details["input_token_count"] = (usage_details.get("input_token_count") or 0) + (
@@ -280,8 +280,8 @@ class RawAzureAIInferenceEmbeddingClient(
# Embed image inputs.
if image_items:
if not (image_model := opts.get("image_model_id") or self.image_model_id):
raise ValueError("An image_model_id is required, either in the client or options, for image inputs.")
if not (image_model := opts.get("image_model") or self.image_model):
raise ValueError("An image_model is required, either in the client or options, for image inputs.")
image_inputs = [img for _, img in image_items]
response = await self._image_client.embed(
input=image_inputs,
@@ -294,7 +294,7 @@ class RawAzureAIInferenceEmbeddingClient(
embeddings[original_idx] = Embedding(
vector=image_vector,
dimensions=len(image_vector),
model_id=response.model or image_model,
model=response.model or image_model,
)
if response.usage:
usage_details["input_token_count"] = (usage_details.get("input_token_count") or 0) + (
@@ -322,11 +322,11 @@ class AzureAIInferenceEmbeddingClient(
``Content.from_data()``.
Keyword Args:
model_id: The text embedding model deployment name (e.g. "text-embedding-3-small").
Can also be set via environment variable AZURE_AI_INFERENCE_EMBEDDING_MODEL_ID.
image_model_id: The image embedding model deployment name
model: The text embedding model (e.g. "text-embedding-3-small").
Can also be set via environment variable AZURE_AI_INFERENCE_EMBEDDING_MODEL.
image_model: The image embedding model
(e.g. "Cohere-embed-v3-english"). Can also be set via environment variable
AZURE_AI_INFERENCE_IMAGE_EMBEDDING_MODEL_ID. Falls back to ``model_id``.
AZURE_AI_INFERENCE_IMAGE_EMBEDDING_MODEL. Falls back to ``model``.
endpoint: The Azure AI Inference endpoint URL.
Can also be set via environment variable AZURE_AI_INFERENCE_ENDPOINT.
api_key: API key for authentication.
@@ -346,8 +346,8 @@ class AzureAIInferenceEmbeddingClient(
# Using environment variables
# Set AZURE_AI_INFERENCE_ENDPOINT=https://your-endpoint.inference.ai.azure.com
# Set AZURE_AI_INFERENCE_API_KEY=your-key
# Set AZURE_AI_INFERENCE_EMBEDDING_MODEL_ID=text-embedding-3-small
# Set AZURE_AI_INFERENCE_IMAGE_EMBEDDING_MODEL_ID=Cohere-embed-v3-english
# Set AZURE_AI_INFERENCE_EMBEDDING_MODEL=text-embedding-3-small
# Set AZURE_AI_INFERENCE_IMAGE_EMBEDDING_MODEL=Cohere-embed-v3-english
client = AzureAIInferenceEmbeddingClient()
# Text embeddings
@@ -368,8 +368,8 @@ class AzureAIInferenceEmbeddingClient(
def __init__(
self,
*,
model_id: str | None = None,
image_model_id: str | None = None,
model: str | None = None,
image_model: str | None = None,
endpoint: str | None = None,
api_key: str | None = None,
text_client: EmbeddingsClient | None = None,
@@ -382,8 +382,8 @@ class AzureAIInferenceEmbeddingClient(
) -> None:
"""Initialize an Azure AI Inference embedding client."""
super().__init__(
model_id=model_id,
image_model_id=image_model_id,
model=model,
image_model=image_model,
endpoint=endpoint,
api_key=api_key,
text_client=text_client,

Some files were not shown because too many files have changed in this diff Show More