* .NET: feat(evals): add ground_truth/expected_output support for workflow eval
Brings .NET to parity with Python PR #5234 for issue #5135:
- Add expectedOutput parameter to Run.EvaluateAsync (workflow) and stamp on the overall EvalItem.ExpectedOutput.
- Map EvalItem.ExpectedOutput -> ground_truth in the Foundry JSONL payload, item_schema, and data_mapping for similarity.
- Add GroundTruthEvaluators set (currently builtin.similarity) and a FindMissingGroundTruthEvaluators helper.
- Fail fast with InvalidOperationException when a ground-truth evaluator is selected but no item provides an ExpectedOutput, instead of surfacing a remote provider error.
- Add tests in FoundryEvalConverterTests and WorkflowEvaluationTests.
- Add Evaluation_WorkflowExpectedOutputs sample (workflow + Foundry similarity).
Fixesmicrosoft/agent-framework#5135 (.NET side).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address review: relax BuildOverallItem events to IReadOnlyList<WorkflowEvent>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Sample: disable per-agent breakdown when using reference-based evaluator
Per-agent EvalItems are intentionally left without ExpectedOutput, so the new fail-fast validation in FoundryEvals would throw when Similarity is invoked for per-agent items. Pass includePerAgent: false in the workflow + similarity sample, and document this gotcha in the EvaluateAsync XML doc.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix BuildOverallItem: fall back to last ExecutorCompletedEvent
AgentResponseEvent is only emitted when AIAgentHostOptions.EmitAgentResponseEvents is enabled, which is not the default for WorkflowBuilder(agent).AddEdge(...). When it is absent, fall back to the last non-internal ExecutorCompletedEvent whose Data is an AgentResponse / ChatMessage / string so the overall EvalItem (and any expectedOutput) is produced. Without this, samples wired up the standard way returned 0 evaluation items.
Update test to cover the fallback path.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Sample: enable EmitAgentResponseEvents; eval throws clear error when no overall response found
Root cause of '0 results': AIAgentHostExecutor only emits AgentResponseEvent when AIAgentHostOptions.EmitAgentResponseEvents is true (default false). For ordinary AIAgent executors the runtime's ExecutorCompletedEvent.Data is null, so the prior fallback couldn't find a final response either.
Sample now builds executors with EmitAgentResponseEvents=true via BindAsExecutor(hostOptions). EvaluateAsync now throws InvalidOperationException with a remediation hint when the user supplies expectedOutput but no overall final response can be located, instead of silently returning 0/0.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Guard against null sample/error/usage/datasource_item in ParseDetailedItem
Foundry eval responses can have these properties present with JSON null
or non-object values, which caused JsonElement.TryGetProperty to throw
'requires Object, has Null'. Check ValueKind == Object before drilling in.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address PR review: reorder expectedOutput, tighten ground-truth check, add fail-fast test
* WorkflowEvaluationExtensions.EvaluateAsync: move 'expectedOutput' to
after 'splitter' so the original positional contract of (splitter,
cancellationToken) is preserved for existing callers.
* FoundryEvals: require ALL items to carry ExpectedOutput when a
ground-truth evaluator is selected (e.g. similarity), not just any.
Reference-based evaluators score per-item, so a single missing GT
would still surface as a provider-side validation error. Updated
fail-fast message accordingly.
* WorkflowEvaluationTests: add EvaluateAsync_WithExpectedOutputButNoFinalResponse_ThrowsAsync
to verify the InvalidOperationException is thrown (and that the
message mentions EmitAgentResponseEvents).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fail-fast on missing overall item regardless of expectedOutput; harden BuildOverallItem default
* EvaluateAsync now throws InvalidOperationException whenever 'includeOverall'
is requested but BuildOverallItem cannot produce an item, instead of only
when 'expectedOutput' is supplied. Same misconfiguration (agents not bound
with EmitAgentResponseEvents) used to silently return empty results — now
it surfaces a clear, actionable error in both cases.
* BuildOverallItem switch default now throws instead of returning null. The
preceding for-loop already constrains Data to AgentResponse/ChatMessage/
string, so reaching default would indicate a contract drift; throw to make
the bug visible.
* Test renamed and broadened to verify the throw fires without expectedOutput.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: alliscode <25218250+alliscode@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* test: Split out Handoff Orchestration tests
* fix: Synthesized Handoff FunctionResult is never sent to agent
When we receive a handoff request from the agent, we need to service it outside of the Agent Loop to terminate the loop. What this means is that we take ownership of terminating the call by feeding the result back into the agent on a subsequent invocation.
When we refactored Handoff to support HITL and make use of AgentSession, we inadvertantly removed this step, causing subsequent invocations to the Handoff agent to fail (first works, but breaks the state).
The fix is to be more precise about the agent's bookmark when concatenating the result of agent invocation to the shared conversation history.
* test: Add unit tests for Handoff FunctionCall/Result matching fix
When MultiPartyConversation gets saved during checkpointing, the data for the chat history is not persisted, resulting in failures to deserialize after. The fix is to make the history visible to the source generated serialization code.
* refactor: remove dead code
* refactor: remove ignore YieldsMessageAttribute
- the correct one to use is YieldsOutputAttribute
- fixes a comment that mistakenly refers to `.YieldsMessage()` which does not exist.
* fix: ChatForwardingExecutor does not use correct role for string messages
- make ChatForwardingExecutor use its configured role for string messages rather than always use ChatRole.User
- add ChatForwardingExecutor tests
* fixup: remove unused attribute
* test: Add tests for failure when .AsAgent used on a non-ChatProtocol workflow
* test: Add FunctionExecutor tests
- also fixes Send and YieldOutput type registration for synchronous output-returning delegates
* test: Suppress CodeCoverage for obsolete names
* fix: Re-add Obsolete attributes
- avoid hard-breaking change
- properly notify users that these attributes get ignored
Some providers, e.g. Gemini, do not use the CallId mechanism to disambiguate simultaneous function calls. This can result in message lists containing multiple turn to fail to filter properly.
The fix is to take advantage of the expectation that Handoff Orchestration is a "single-speaker" flow, which only has a single active AIAgent per "turn" and an agent's turn is not finished until all outstanding function calls are finished.
This allows us to expect that any ambiguous-CallId FunctionCallContent are either in separate turns or will have had a response before the next issued call with the same Id.
* fix: Add session support for Handoff-hosted Agents
In order to better support using `Workflows` hosted as `AIAgents` inside of Handoff workflows, we need to make proper use of AgentSession. This causes potential issues around checkpointing and making sure that we properly compute only the new incoming messages for each agent invocation.
* fix: AgentSession checkpointing using AIAgent's Serialize/Deserialize methods
We cannot rely on implicit serialization through `HandoffHostState` because we are missing type information.
* fix: Thread safety issue in `MultiPartyConversation.AllMessages`
* fix: Enable unwrapping of FunctionResultContent when ExternalRequest was wrapped into FunctionCallContent
* fix: Foundry Agents without description in Handoff
Foundry Agents without a description set will return an empty string (rather than null) for the description. This was breaking the fallback logic for `handoffReason`.
* test: Add unit tests
* Foundry Evals integration for .NET
- Core evaluation framework: EvalItem, LocalEvaluator, FunctionEvaluator, EvalChecks
- IAgentEvaluator interface with MeaiEvaluatorAdapter bridge
- AgentEvaluationExtensions for agent.EvaluateAsync() overloads
- FoundryEvals wrapping MEAI quality/safety evaluators
- ConversationSplitters (LastTurn, Full) and IConversationSplitter
- EvalItem.PerTurnItems() for multi-turn decomposition
- HasImageContent for multimodal content detection
- WorkflowEvaluationExtensions for per-agent workflow evaluation
- 7 eval samples mirroring Python parity:
02-agents/Evaluation: SimpleEval, ExpectedOutputs, Multimodal
03-workflows/Evaluation: WorkflowEval
05-end-to-end/Evaluation: FoundryQuality, MixedProviders, ConversationSplits
- Comprehensive unit tests (1958 passing)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Rewrite FoundryEvals to use real Foundry Evals API
Replace MEAI evaluator shim with actual OpenAI EvaluationClient protocol
methods. FoundryEvals now creates eval definitions, submits runs, polls
for completion, and fetches per-item results server-side.
- New constructor: FoundryEvals(AIProjectClient, model, evaluators)
- Add FoundryEvalConverter for MEAI ChatMessage -> Foundry JSON format
- Add EvalId, RunId, ReportUrl to AgentEvaluationResults
- All 20 built-in evaluator constants now work (agent, tool, quality, safety)
- Remove Microsoft.Extensions.AI.Evaluation.Quality/Safety dependencies
- Update all samples for new constructor (no more ChatConfiguration)
- Replace BuildEvaluators tests with ResolveEvaluator tests
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add response output to CustomEvals and ExpectedOutputs samples
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address review: pagination, validation, error handling, tests
FoundryEvals fixes:
- Add pagination for output items (has_more/after cursor)
- Add guard clauses for pollIntervalSeconds/timeoutSeconds <= 0
- Fix double TryGetProperty for passed field parsing
- Throw on all-tool-evaluators with no tool definitions
- Fix XML doc (default 300s, not 180s)
New tests (30 added, 1989 total):
- EvalChecks: NonEmpty, ContainsExpected (pass/fail/skip/case),
HasImageContent, ToolCallsPresent
- FoundryEvalConverter: ConvertMessage (text, image, function call,
function results fan-out, empty fallback, mixed content),
ConvertEvalItem, BuildTestingCriteria (quality/agent/tool/groundedness
data mappings), BuildItemSchema
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix review: null-refs, Data.ToString() bug, ContainsExpected, add tests
- Fix NullReferenceException in sample Response display (pattern matching)
- Fix WorkflowEvaluationExtensions Data?.ToString() producing type names
instead of message text (pattern-match ChatMessage/AgentResponse/list)
- Change EvalChecks.ContainsExpected to return Passed=false when no
ExpectedOutput (was silently passing, masking misconfiguration)
- Add EvalItem constructor tests with LastTurn/Full/null splitters
- Add FoundryEvalConverter.ConvertMessage DataContent (base64 image) test
- Add ExtractAgentData tests with ChatMessage, list, and AgentResponse data
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix review: conversation fidelity, eval caching, fallback tests
- WorkflowEvaluationExtensions: preserve full response messages (tool calls,
intermediate) instead of synthetic 2-message conversation. Cast completed
Data to AgentResponse and use Messages when available, fallback to text.
- FoundryEvals: cache evalId per schema shape (hasContext, hasTools) so
subsequent EvaluateAsync calls create runs under the same eval definition.
- MeaiEvaluatorAdapter: code already correctly passes queryMessages (not full
conversation) to IEvaluator — no change needed, verified by inspection.
- Add tests: AgentResponse full messages preservation, unknown object
ToString() fallback for ExtractAgentData.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Rename AzureAI→Foundry: move eval files, update references
- Move FoundryEvals.cs and FoundryEvalConverter.cs from
Microsoft.Agents.AI.AzureAI to Microsoft.Agents.AI.Foundry
- Update namespace from AzureAI to Foundry in both files
- Add explicit usings required by Foundry project (no implicit usings)
- Move FoundryEvalConverter tests to Foundry.UnitTests project
(avoids ReplacingRedactor type conflict from dual project refs)
- Update all sample csproj references and using statements
- Remove Foundry project reference from AI UnitTests
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* PR review round 4: wire up tool extraction, remove eval cache, fix null safety
- BuildEvalItem: extract tools from agent via GetService<ChatOptions>() into EvalItem.Tools (Python parity)
- FoundryEvals: remove eval ID cache - each call creates fresh definition (matches Python behavior)
- FoundryEvals: replace null-forgiving operators with descriptive InvalidOperationException
- MixedProviders sample: remove unnecessary explicit PackageReferences (transitively provided)
- FoundryEvalConverter: document that tool results take precedence over text content
- Add LocalEvaluator zero-checks test documenting 0 metrics = failed behavior
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python-dotnet parity: 9 feature gaps filled
New checks:
- ToolCallArgsMatch() — verify tool call names + argument subset match
- ToolCalledCheck(ToolCalledMode.Any, ...) — match any of the specified tools
- ToolCalledMode enum (All/Any)
FoundryEvals enhancements:
- Default evaluators now [Relevance, Coherence, TaskAdherence] (was Relevance, Coherence)
- Auto-add ToolCallAccuracy when items have tool definitions
- EvaluateTracesAsync — evaluate by response_ids, trace_ids, or agent_id
- EvaluateFoundryTargetAsync — evaluate deployed Foundry targets
Result type enrichment:
- AgentEvaluationResults: added Status, Error, PerEvaluator, DetailedItems
- New EvalItemResult/EvalScoreResult/PerEvaluatorResult types
- FoundryEvals populates all new fields from API responses
Workflow fix:
- Skip internal executors (_*, input-conversation, end-conversation, end)
Tests: 8 new tests covering ToolCallArgsMatch, ToolCalledMode.Any, internal executor filtering
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add MeaiEvaluatorAdapter and PerTurnItems edge case tests
- 3 tests for MeaiEvaluatorAdapter: query message forwarding, synthetic
response fallback, multiple items aggregation
- 3 tests for EvalItem.PerTurnItems: empty conversation, no user messages,
system+assistant only
- StubEvaluator and StubChatClient test helpers
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Blocking link check for outdated package in DevUI.
* Replace Dictionary<string, object> payloads with typed wire models
Introduce internal FoundryEvalWireModels.cs with compile-time-safe types
for the OpenAI Evals API wire format. The OpenAI .NET SDK (2.9.1) only
provides protocol-level methods with BinaryContent/ClientResult — no
typed request models. These internal models replace scattered dictionary
literals with [JsonPropertyName]-annotated classes, giving:
- Compile-time safety (typos become build errors)
- Single point of change when the API evolves
- IntelliSense discoverability
- Cleaner serialization via JsonPolymorphic for content items
Models: WireContentItem hierarchy (text, image, tool_call, tool_result),
WireMessage, WireEvalItemPayload, WireTestingCriterion, WireItemSchema,
WireCreateEvalRequest, WireCreateRunRequest, and data source variants.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Skip metric when Foundry returns neither score nor passed
When an evaluator returns no score and no passed value, the previous
code created BooleanMetric(name, false), which falsely failed items
via ItemPassed. Now we skip the MEAI metric entirely for indeterminate
results — the raw data remains available in DetailedItems for diagnostics.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address PR #4914 review comments: fix tool evaluator bug and add tests
- Fix duplicate ToolCallAccuracy: resolve evaluator names before checking
against ToolEvaluators set (Comment 2)
- Make FilterToolEvaluators internal for testability; add tests for the
ArgumentException edge case when all evaluators are tool-type (Comment 3)
- Add CancellationToken test for LocalEvaluator (Comment 4)
- Add EvaluateAsync integration test on Run with sequential workflow and
per-agent SubResults verification (Comment 5)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address Peter's review comments on PR #4914
- Add trailing newline to Evaluation_FoundryQuality.csproj (Comment 6)
- Make evaluator name lookups case-insensitive: switch BuiltinEvaluators,
ToolEvaluators, AgentEvaluators, and ResolveEvaluator's StartsWith check
from Ordinal to OrdinalIgnoreCase (Comment 7)
- Add Trace.TraceWarning when Foundry returns fewer results than submitted
items, indicating expected vs actual count before padding (Comment 8)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add Microsoft.Extensions.AI.Evaluation packages to Directory.Packages.props
These were removed in #5269 as unused, but are needed by the Foundry
and core evaluation integration added in this PR.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: alliscode <bentho@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Improve workflow unit tests
* Update test name prefix for clarity.
* Update tests to surface any errors.
* fix check-point restore-time race in off-thread workflow event stream
* Fixes an intermittent checkpoint-restore race in in-process workflow runs.
* feat: Refactor Handoff Orchestration and add HITL support
* Change HandoffAgentExecutor to use factory-based instantiation
* Extract shared request collection logic in AIAgentUnservicedRequestsCollector
* Refactor HandoffAgentExecutor to use the "ContinueTurn" pattern as in AIAgentHostExecutor
* fix: Remove '$' from exception strings
* fix: Remove Timeout from InputWait in StreamingRunEventStream
* fix: Race condition when the workflow executes to halt before TakeEventStream
* test: Make the OffThread Delay test more nimble
* fix: Remove slight window where runStatus could be stale
* Fix RequestInfoEvent lost when resuming workflow from checkpoint
* Fix streaming run double disposal in tests and lockstep republishing before Started event is emitted.
* Fix bug to remove messages after sending to avoid losing messages on send failure.
* Fix declarative test harness
* Stage
* Add FoundryAgentClient, model param, chatClientFactory, and RAPI samples
- Add model parameter to FoundryAgentClient simple constructor
- Add chatClientFactory parameter to both constructors
- Switch to OpenAI.GetProjectResponsesClientForModel for direct Responses API usage
- Add FoundryAgents-RAPI samples (Step01 Basics, Step02 Multiturn, Step03 FunctionTools)
- Add solution folder entry for FoundryAgents-RAPI samples
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add auto-discovery constructor and simplify RAPI samples
- Add FoundryAgentClient constructor that reads AZURE_AI_PROJECT_ENDPOINT and
AZURE_AI_MODEL_DEPLOYMENT_NAME from environment variables with DefaultAzureCredential
- Simplify RAPI samples to use auto-discovery (no env var or credential code)
- Remove Azure.Identity direct references from sample csproj files
- Update READMEs to document environment variable requirements
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add remaining RAPI samples (Step04-Step12)
- Step04: Function tools with human-in-the-loop approvals
- Step05: Structured output with typed responses
- Step06: Persisted conversations with session serialization
- Step07: Observability with OpenTelemetry
- Step08: Dependency injection with hosted service
- Step10: Image multi-modality
- Step11: Agent as function tool (agent composition)
- Step12: Middleware (PII, guardrails, function logging, HITL approval)
- Update solution file and folder README with all new samples
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add all RAPI samples (Step09-Step23) and switch to AzureCliCredential
- Step09: MCP client as tools (GitHub server via stdio)
- Step13: Plugins with dependency injection
- Step14: Code Interpreter tool
- Step15: Computer Use tool with screenshot simulation
- Step16: File Search with vector stores
- Step17: OpenAPI tools (REST Countries API)
- Step18: Bing Custom Search
- Step19: SharePoint grounding
- Step20: Microsoft Fabric
- Step21: Web Search with citations
- Step22: Memory Search with multi-turn conversations
- Step23: Local MCP via HTTP (Microsoft Learn)
- Switch all samples (Step04-Step12) to use AzureCliCredential with env vars
- Update solution file and README with all 23 samples
- All 23 samples build successfully, tested Step05/06/11/13/21
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Switch Step01-03 samples to AzureCliCredential for consistency
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Clarify connection ID format in SharePoint and Fabric READMEs
Document that SHAREPOINT_PROJECT_CONNECTION_ID and FABRIC_PROJECT_CONNECTION_ID
should use the connection name (e.g., 'SharepointTestTool'), not the full ARM
resource URI.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Normalize env vars, fix structured output, update READMEs with connection ID formats
- Normalize AZURE_FOUNDRY_PROJECT_* env vars to AZURE_AI_PROJECT_ENDPOINT / AZURE_AI_MODEL_DEPLOYMENT_NAME across all samples (Steps 18-22 READMEs + Steps 19-20 Program.cs)
- Fix RAPI Step05 StructuredOutput to use full constructor with ResponseFormat for streaming JSON
- Update Deep Research sample to use AzureCliCredential
- Enrich Bing Grounding README with full ARM resource URI format
- Fix Bing Custom Search README env var mismatch (BING_CUSTOM_SEARCH_* -> AZURE_AI_CUSTOM_SEARCH_*)
- Add finding instructions for connection ID and instance name in Bing Custom Search READMEs
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Refactor memory samples and switch to DefaultAzureCredential
- Refactor RAPI Step22 MemorySearch: extract store setup to EnsureMemoryStoreAsync local function
- Refactor non-RAPI Step22 MemorySearch: same pattern with explicit memory lifecycle
- Set UpdateDelay=0 on MemoryUpdateOptions and MemorySearchPreviewTool for faster ingestion
- Use WaitForMemoriesUpdateAsync with 500ms polling interval
- Switch Step19 SharePoint, Step20 Fabric, Step22 MemorySearch (both) to DefaultAzureCredential
- Remove SearchOptions from MemorySearchPreviewTool (causes unknown parameter error)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Switch all RAPI samples to DefaultAzureCredential and format
- Replace AzureCliCredential with DefaultAzureCredential across all 20 RAPI samples
- Run dotnet format on all RAPI and non-RAPI Foundry samples
- AzureAI unit tests: 341 passed (net10.0 + net472)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Rename to Microsoft Foundry, add metadata, rename RAPI folder
- Replace 'Azure AI Foundry' / 'Azure Foundry' with 'Microsoft Foundry' in all docs, comments, and XML docs
- Update FoundryAgentClient metadata provider name to 'microsoft.foundry'
- Rename FoundryAgents-RAPI folder to FoundryResponseAgents
- Rewrite FoundryResponseAgents README with comparison table vs Foundry Agents
- Update slnx and parent README with new folder references
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address PR review: simplify sample comments and fix DeepResearch credential
- Remove 'no server-side agent' and 'Responses API directly' phrasing from comments
- Simplify to 'Create a FoundryAgentClient' per review feedback
- Switch Agent_Step15_DeepResearch to DefaultAzureCredential
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Restore full DefaultAzureCredential warning comment in DeepResearch sample
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add ADR 0020: Foundry agent type naming convention
Proposes naming options for a new MAF type wrapping versioned
Foundry agents (Prompt, ContainerApp, Hosted, Workflow) to
distinguish from the existing FoundryResponsesAgent (RAPI path).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Simplify FoundryResponsesAgent samples with env-var constructors and rename folders
- Add env-var constructors to FoundryResponsesAgent (simple + options-based)
- Fix Constructor 1 model optionality (no longer throws on missing AZURE_AI_MODEL_DEPLOYMENT_NAME)
- Add ApplyModelDeploymentFallback helper for options-based constructor
- Update all 23 FoundryResponseAgents samples to remove Environment.GetEnvironmentVariable boilerplate
- Condense 6 simple samples to one-liner constructor calls
- Add XML doc remarks about auto-resolved parameters on all constructors
- Rename FoundryAgents -> FoundryVersionedAgents (server-side, versioned)
- Rename FoundryResponseAgents -> FoundryAgents (now the default path forward)
- Update .slnx and README cross-references for new folder names
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add FoundryAITool factory, rename RAPI folders, and clean up references
- Create FoundryAITool static factory class with 17 methods wrapping AgentTool.Create* and ResponseTool.Create* into AITool returns
- Rename 23 FoundryAgentsRAPI_* subfolders to FoundryAgents_* (drop RAPI prefix)
- Rename .csproj files and update .slnx references accordingly
- Update 12 samples (6 FoundryAgents + 6 FoundryVersionedAgents) to use FoundryAITool
- Replace all FoundryResponsesAgent references with FoundryAgent in comments and READMEs
- Update sample READMEs to reference FoundryAITool methods
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Rename FoundryVersionedAgents subfolders from FoundryAgents_* to FoundryVersionedAgents_*
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add FoundryVersionedAgent class and refactor extension method internals
- Create FoundryVersionedAgent with private ctor and async static factory methods
(CreateAIAgentAsync/GetAIAgentAsync) with env-var and explicit endpoint tiers
- Extract shared internal helpers from AzureAIProjectChatClientExtensions:
CreateChatClientAgent, CreateAgentVersionFromOptionsAsync,
CreateAgentVersionWithProtocolAsync (tools overload),
CreateChatClientAgentOptions, GetAgentRecordByNameAsync, ThrowIfInvalidAgentName
- Extension methods now delegate to shared internal helpers
- All 49 existing samples continue to build successfully
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add CreateConversationSessionAsync, DeleteAIAgentAsync, auto-resolve model, simplify samples
- Add CreateConversationSessionAsync to FoundryAgent and FoundryVersionedAgent
(returns ChatClientAgentSession, creates server-side conversation + session in one call)
- Add DeleteAIAgentAsync static method to FoundryVersionedAgent
- Make model parameter optional in env-var factory overloads (auto-resolves from
AZURE_AI_MODEL_DEPLOYMENT_NAME)
- Update all FoundryVersionedAgents samples to use DeleteAIAgentAsync
- Remove deploymentName env var from samples where only used for model parameter
- Use CreateConversationSessionAsync in Step02_MultiturnConversation
- Use explicit types instead of var for agent/session variables
- All 49 samples build successfully
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Remove manual AIProjectClient construction from FoundryVersionedAgents samples
- Replace manual AIProjectClient construction with GetService<AIProjectClient>()
from the FoundryVersionedAgent in all dual-option and tool-specific samples
- Remove AZURE_AI_PROJECT_ENDPOINT env var reads from updated samples
- Remove Azure.Identity usings where no longer needed
- Only Step01.1, Step01.2, Eval_Step01 retain manual construction (pedagogical samples)
- All 49 samples build successfully
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Replace aiProjectClient extension calls with FoundryVersionedAgent factories in all samples
- Replace aiProjectClient.CreateAIAgentAsync with FoundryVersionedAgent.CreateAIAgentAsync
in Option 2 (Native SDK) paths across Steps 14-21
- Replace aiProjectClient.Agents.DeleteAgentAsync with FoundryVersionedAgent.DeleteAIAgentAsync
- Remove unused AIProjectClient variables and using directives
- Only Step01.1, Step01.2, Eval_Step01 retain direct AIProjectClient usage (pedagogical)
- Step16, Step22 use GetService<AIProjectClient>() for file/memory operations
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Remove unused using directives from Step01.2, Step09, Eval_Step02
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Update ADR 0020 with accepted decision: Option 6
- Add Option 6 detailing FoundryAgent, FoundryVersionedAgent, FoundryAITool,
env-var auto-discovery, and self-contained factory patterns
- Mark decision as accepted with rationale
- Update current state and metadata sections
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Update Step01 basics samples to use FoundryVersionedAgent factories
- Step01.1: Replace manual AIProjectClient/AsAIAgent with FoundryVersionedAgent.CreateAIAgentAsync/GetAIAgentAsync/DeleteAIAgentAsync
- Step01.2: Replace manual AIProjectClient with FoundryVersionedAgent.CreateAIAgentAsync/DeleteAIAgentAsync
- Remove env var boilerplate and Azure.Identity dependency
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add DeleteAIAgentVersionAsync to FoundryVersionedAgent
- DeleteAIAgentAsync: deletes the agent and all its versions (existing)
- DeleteAIAgentVersionAsync: deletes only the specific version associated with the agent instance
- Internally delegates to Agents.DeleteAgentAsync vs Agents.DeleteAgentVersionAsync
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix cleanup comments: DeleteAIAgentAsync deletes the agent and all its versions
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Update all FoundryVersionedAgents READMEs for FoundryVersionedAgent and auto-discovery
- Rewrite main README with FoundryVersionedAgent usage, auto-discovery table, code example
- Fix sample table links from FoundryAgents_Step* to FoundryVersionedAgents_Step*
- Add FoundryAITool references in tool-specific sample descriptions
- Update individual READMEs: fix stale paths, add auto-discovery note after env var blocks
- Update tool references: AgentTool/ResponseTool -> FoundryAITool
- Update parent 02-agents/README.md with FoundryVersionedAgent description
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Revert unrelated AGUI and Hosting.OpenAI formatting changes to main
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Remove env-var auto-discovery, add AsAIAgent, mark extensions Obsolete
- Remove 2 env-var constructors from FoundryAgent (keep explicit endpoint ctors)
- Remove 5 env-var factory methods from FoundryVersionedAgent (keep explicit ones)
- Add 3 AsAIAgent static methods to FoundryVersionedAgent (AgentVersion/AgentRecord/AgentReference)
- Mark all 8 AIProjectClient extension methods as [Obsolete] pointing to FoundryVersionedAgent
- Remove ApplyModelDeploymentFallback, env var constants, Azure.Identity usings from source
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Update all samples to use explicit endpoint, credential, and model parameters
- Add explicit Environment.GetEnvironmentVariable reads for AZURE_AI_PROJECT_ENDPOINT
and AZURE_AI_MODEL_DEPLOYMENT_NAME to all 48 sample files
- Pass new Uri(endpoint), new DefaultAzureCredential(), deploymentName to
FoundryAgent constructors and FoundryVersionedAgent factory methods
- Add using Azure.Identity where missing
- Matches repo-wide pattern used by other non-Foundry samples
- All 49 samples build successfully
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Migrate remaining samples and source from obsoleted extension methods
- Migrate AgentProviders, AgentWithRAG, AgentWithMemory, HostedWorkflow samples to FoundryVersionedAgent
- Migrate AzureAgentProvider.cs to FoundryVersionedAgent.AsAIAgent
- Migrate AzureAIProjectChatClientTests.cs to FoundryVersionedAgent.GetAIAgentAsync
- Remove pragma suppressions from migrated files
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add unit tests for FoundryAgent and FoundryVersionedAgent
- FoundryAgentTests.cs: 14 tests covering constructors, validation,
properties, metadata, GetService, chat client factory, user-agent header
- FoundryVersionedAgentTests.cs: 31 tests covering CreateAIAgentAsync,
GetAIAgentAsync, AsAIAgent (3 overloads), DeleteAIAgentAsync,
DeleteAIAgentVersionAsync, validation, invalid names, metadata, GetService
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Finalize Foundry agent migration
Align FoundryAgent and FoundryVersionedAgent samples, docs, and tests with the explicit configuration model, clean up stale README guidance, and fix AzureAI unit test validation/build issues.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Apply formatter cleanup after validation
Capture the dotnet format follow-up changes produced during branch validation so the committed state matches the successfully built and tested branch.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add integration tests for FoundryAgent and FoundryVersionedAgent
Mark old AIProjectClient extension-method integration tests as obsolete and add new integration test suites for both FoundryAgent (Responses API) and FoundryVersionedAgent (versioned agents). All 71 non-skipped tests pass against the live Foundry service.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Update ADR 0020 with test coverage details
Add integration test coverage note to the Current State section of ADR 0020.
* Simplify Foundry agents and validate moved samples
* Rename FoundryAgent integration tests to ResponsesAgent
The test classes exercise the non-versioned Responses path via
AIProjectClient.AsAIAgent(), not the removed FoundryAgent wrapper type.
Rename files and class names to reflect the actual test surface.
* Update documentation for ChatClientAgent usage
Added example usage of ChatClientAgent with JokerAgent.
* Refactor ChatClientAgent instantiation for clarity
* Revise agent type naming and usage examples
Updated documentation to reflect changes in agent creation methods and added examples for using `ChatClientAgent`.
* Fix Azure SDK namespace migration after rebase
Update Azure.AI.Projects.OpenAI references to Azure.AI.Projects.Agents
and Azure.AI.Extensions.OpenAI to match Azure.AI.Projects 2.0.0-beta.2.
- Replace deprecated namespace across samples, tests, and src
- Fix renamed types: OpenAPIFunctionDefinition -> OpenApiFunctionDefinition,
BingCustomSearchToolParameters -> BingCustomSearchToolOptions,
BrowserAutomationToolParameters -> BrowserAutomationToolOptions
- Fix API changes: AgentRecord.Versions -> GetLatestVersion(),
ResponsesClient constructor, FunctionApprovalRequestContent ->
ToolApprovalRequestContent
- Apply dotnet format
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address merge markers
* Replace obsolete GetAIAgentAsync with AsAIAgent in samples
Switch Agent_Step07_AsMcpTool and A2AServer to use the non-obsolete
PersistentAgentsClient.AsAIAgent(PersistentAgent) extension instead
of the deprecated GetAIAgentAsync, fixing CS0618 build errors.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix broken markdown links in Responses sample READMEs
Replace stale ChatClientAgents_Step* folder references with the
correct Agent_Step* names across all Responses sample READMEs.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix format errors and address PR review comments
- Fix charset and remove unused using in AzureAIProjectResponsesChatClient
- Fix doc comment tags (code -> c) in FoundryAITool
- Fix stray period in LocalMCP sample comment
- Fix grammar in FoundryMemoryProvider xmldoc
- Fix AIProjectClientAgentRunStreamingConversationTests base class
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Apply dotnet format fixes to PR-changed files
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix build errors from format pass and apply naming conventions
- Fix static call to CreateSessionAsync in Step02 samples and extension tests
- Use expression-bodied lambda in FoundryMemoryProvider (RCS1021)
- Apply PascalCase naming to const fields in ResponsesAgentExtensionCreateTests (IDE1006)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Introduce FoundryAgent sealed type and update AsAIAgent extensions
- Add FoundryAgent sealed class wrapping ChatClientAgent with:
- Public ctors: (projectEndpoint, credential, model, instructions) and (agentEndpoint, credential)
- Internal ctor: (AIProjectClient, ChatClientAgent) for extension use
- CreateConversationSessionAsync() for server-side conversations
- GetService<ChatClientAgent>() and GetService<AIProjectClient>()
- MEAI user-agent policy on internally-created AIProjectClient
- Change all AsAIAgent extension return types from ChatClientAgent to FoundryAgent
- Update all samples and tests to use FoundryAgent type
- Add 16 FoundryAgentTests covering ctors, GetService, UserAgent, RunAsync
- Fix pre-existing Agent_Step12_Plugins build error
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Collapse sample folders and add FoundryAgent_Step01 sample
- Move all Responses/* samples up to AgentsWithFoundry/ (flat structure)
- Remove entire Versioned/ folder (26 samples)
- Add FoundryAgent_Step01 sample showing direct FoundryAgent ctor usage
- Update slnx to reflect flat folder structure
- Fix csproj ProjectReference paths for new depth
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Update READMEs for flat AgentsWithFoundry structure
- Rewrite AgentsWithFoundry/README.md with FoundryAgent quick start
- Fix cd commands and paths in 11 sample READMEs
- Update 02-agents/README.md to single Foundry link
- Update AGENTS.md tree to flat structure
- Fix AgentWithMemory cross-reference
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix FoundryAgent_Step01 sample with full create/run/delete lifecycle
Show the complete server-side agent lifecycle: create version with
native SDK, wrap as FoundryAgent via AsAIAgent, run, then delete.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Revert RAPI samples to use AIAgent instead of FoundryAgent
RAPI samples should not reference FoundryAgent directly. Restored
original sample code with only ChatClientAgent -> AIAgent type change
to accommodate the AsAIAgent return type.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Convert versioned-pattern samples to pure RAPI
Step09, Step13, Step17, Step22 were using CreateAgentVersionAsync +
PromptAgentDefinition which is the versioned pattern. Converted to
use AsAIAgent(model, instructions, tools) which is the RAPI path.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix format issues from Docker CI check
- FoundryAgent_Step01: CRLF -> LF
- Agent_Step09: missing final newline
- Agent_Step11_Middleware: add internal modifier, final newline
- Agent_Step02: remove redundant cast (IDE0004)
- Agent_Step08: simplify name (IDE0001)
- FoundryAgentTests: s_ prefix, Async suffix naming conventions
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Switch Step09 MCP sample to Microsoft Learn HTTP endpoint
Replace npx stdio GitHub MCP server with the public Microsoft Learn
MCP endpoint (https://learn.microsoft.com/api/mcp) using HTTP transport.
No external tooling required to run.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix missing final newline in Step09 MCP sample
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address PR review: use DelegatingAIAgent, clean up Step01 sample
- FoundryAgent now inherits DelegatingAIAgent instead of AIAgent,
removing manual delegation boilerplate (westey-m feedback)
- Simplified Agent_Step01_Basics to single agent creation path,
moved composable IChatClient approach to README (westey-m feedback)
- Fixed FoundryAgentTests param name assertion
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Update sample using Project specialized type instead
* Address PR review feedback: DefaultAzureCredential warnings, sample simplifications, format fixes
- Add DefaultAzureCredential production warning comments to ~25 samples
- Simplify Anthropic and OpenAI Step01 samples to single agent
- Convert Step11 Middleware regex patterns to [GeneratedRegex]
- Remove unnecessary cleanup comment from Step06
- Fix Step09 README MCP transport description
- Enhance FoundryAgent xmldoc with non-persistent agent comparison
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Split Step02, simplify RAG Step04, sharpen Step23 differentiation
- Split Step02 into 02.1 (simple multi-turn via sessions) and 02.2 (server-side conversations via CreateConversationSessionAsync)
- RAG Step04: replace HostedFileSearchTool + MEAI wrapping with native OpenAI FileSearchTool
- Step23: clarify DelegatingAIFunction wrapping pattern vs Step09 basic MCP
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix Hosted MCP sample: use ResponseTool.CreateMcpTool and move tool to PromptAgentDefinition
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix broken README link after Step02 split
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address Sergey round 3 feedback: branding, README nav, sample rename
- Replace 'Azure AI Foundry' with 'Microsoft Foundry' in ADR 0020
- Fix 3 READMEs: 'ChatClientAgents' → 'AgentsWithFoundry' sample directory
- Rename FoundryAgent_Step01 → Agent_Step00_FoundryAgentLifecycle for naming consistency
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat: Implement return-to-previous routing in handoff workflow
- Also obsoletes HandoffsWorkflowBuilder => HandoffWorkflowBuilder (no "s")
* refactor: Remove instance-shared current agent tracking in handoffs
Because the tracker was instance-shared between the start and end executors, it would be shared between all sessions, resulting in incorrect behaviour.
The corect way to do this is to keep the data in a shared executor scope, which is per-session.
* fix: Fix test logic for Handoff to correctly use checkpointing for multiturn
westeyandGitHub
