* Add LoopAgent capability for Harnesses
* Address PR comments.
* Add support for returning user messages and response aggregation
* Support fresh context per iteration with input sessions via cloning
* Add ability to receive newly created sessions via callback
* Address PR comments
* Add judge criteria
* Address PR comments
* Add reasoning option to request chat options in ChatClientAgent
* Add tests for ChatOptions reasoning merging in ChatClientAgent
---------
Co-authored-by: Roger Barreto <19890735+rogerbarreto@users.noreply.github.com>
* Restore UTF-8 BOMs and fix BuildScriptSchemasBlock doc comment
- Restore UTF-8 BOM on all changed files to match repo convention
- Fix XML doc: <schema name=...> -> <schema script=...> to match emitted output
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address PR review comments: fix doc remarks and rename tests
- Update script doc remarks to clarify only parameter schemas are included
- Fix grammar: 'arguments format' -> 'argument format'
- Rename misleading test methods to match actual assertions
- Clarify comment about removed wrapper element
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: SergeyMenshykh <SergeMenshikh@outlook.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Refactor AgentFileSkillsSource to use filter predicates and add AgentFileSkillFilterContext
- Replace hardcoded script/resource directory lists with configurable ScriptFilter and ResourceFilter predicates
- Add AgentFileSkillFilterContext class to provide contextual file information to filter predicates
- Replace MaxSearchDepth constant with configurable SearchDepth option
- Update AgentFileSkillsSourceOptions with new filter and search depth properties
- Update tests to reflect the new filtering approach
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Log '(none)' instead of empty string for missing file extensions in debug output
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Align c# and python TodoProvider tool names
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Address PR review: remove __slots__ and add typed schemas for tool params
- Remove __slots__ from TodoItem, TodoInput, and TodoCompleteInput classes
(not needed for low-instance-count objects and hinders dev scenarios)
- Add _TodoAddItemSchema and _TodoCompleteItemSchema TypedDicts to provide
proper JSON schema for todos_add and todos_complete tool parameters
- Use typing_extensions for Python 3.10 compatibility
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* .NET: Refactor AgentSkill API to async resource and script lookup
Replace property-based AgentSkill.Content, Resources, and Scripts with
async-by-name lookup methods plus boolean availability flags:
- Content (string getter) -> GetContentAsync(CancellationToken)
- Resources (full list) -> HasResources + GetResourceAsync(name, ct)
- Scripts (full list) -> HasScripts + GetScriptAsync(name, ct)
This makes the API friendlier for sources like MCP where enumerating all
resources up front is expensive or impossible, and allows skill implementations
to fetch content lazily.
Subclass changes:
- AgentFileSkill and AgentInlineSkill implement the new async API while
preserving content caching.
- AgentClassSkill<TSelf> keeps virtual Resources/Scripts properties for
reflection-based discovery and seals the new HasResources/HasScripts/
GetResourceAsync/GetScriptAsync overrides. Its previously non-thread-safe
lazy initialization is replaced with Lazy<T> (default thread-safety) wired
up in a new protected constructor, so concurrent first-access from multiple
threads is safe.
- AgentSkillsProvider calls the new async API and exposes
ead_skill_resource
/ load_skill /
un_skill_script tools that await the per-name lookups.
Includes baseline CompatibilitySuppressions.xml entries for the removed
property getters.
Tests:
- Direct coverage for HasResources, HasScripts, GetResourceAsync, and
GetScriptAsync on all three skill implementations (positive, missing-name,
and no-resources/no-scripts cases).
- Thread-safety regression test for AgentClassSkill<TSelf> that exercises
concurrent first-access to Resources, Scripts, and GetContentAsync from
many tasks and asserts all observers see the same cached instance.
- Provider-level coverage for the
ead_skill_resource tool (invocation +
error paths) and for the previously untested error paths of load_skill
and
un_skill_script (empty names, skill/resource/script not found).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address PR review comments
- Move GetScriptAsync inside try/catch in RunSkillScriptAsync for error-handling parity
- Remove dead _reflectedResources branch from AgentSkillTestExtensions
- Fix XML docs to reference virtual Resources/Scripts properties (not sealed methods)
- Add Async suffix to async test methods per naming convention
- Make no-await tests synchronous to eliminate CS1998
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix formatting: add UTF-8 BOM and remove unused using
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix XML cref: Resources/Scripts are on AgentClassSkill<TSelf>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Remove HasResources and HasScripts properties from AgentSkill
Drop the virtual HasResources and HasScripts properties from AgentSkill
and all concrete subclasses (AgentFileSkill, AgentInlineSkill,
AgentClassSkill). AgentSkillsProvider now always includes all three
tools (load_skill, read_skill_resource, run_skill_script) and both
instruction blocks, since the tools already handle missing
resources/scripts gracefully.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add blank line for readability in file-based skills sample
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix HostedAgentSkillsPatternTests for always-included tools
Update assertions to expect read_skill_resource and run_skill_script
tools are always present, matching the new behavior.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* .NET: Add Hosted-AgentSkills sample for Foundry Skills integration
Add a new hosted agent sample that demonstrates how to load behavioral
guidelines from Foundry Skills at startup using AgentSkillsProvider and
the progressive disclosure pattern (advertise -> load on demand).
The sample:
- Downloads SKILL.md files from Foundry via ProjectAgentSkills SDK
- Extracts ZIP archives with zip-slip protection
- Wires skills into AgentSkillsProvider as an AIContextProvider
- Hosts the agent via the Responses protocol
Ships two Contoso Outdoors skills matching the Python sample (PR #5822):
- support-style: tone, formatting, signature guidelines
- escalation-policy: when and how to escalate tickets
Includes convenience provisioning gated behind PROVISION_SAMPLE_SKILLS
env var, clearly documented as NOT a production pattern.
Closes#5776
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* .NET: Add unit tests and integration test for Hosted-AgentSkills
Unit tests (14 tests, all passing):
- ZIP extraction with zip-slip guard (valid archive, traversal attack,
sibling-prefix attack, directory entries)
- Skill name validation (rejects dots, separators, traversal patterns)
- AgentSkillsProvider with downloaded skills (advertises both skills,
load_skill returns canary tokens, unknown skill returns error)
Container integration test:
- New 'agent-skills' scenario in the test container that creates
Contoso Outdoors skills on disk and wires AgentSkillsProvider
- AgentSkillsHostedAgentFixture + 4 integration tests verifying:
- Routine questions load support-style skill (STYLE-CANARY-3318)
- Escalation triggers load escalation-policy (ESC-CANARY-7742)
- Skills are advertised in system prompt
- load_skill tool is invoked via FunctionCallContent
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* .NET: Add smoke test, bootstrap, and docs for agent-skills integration
- Add scripts/smoke.ps1 for local Docker smoke testing: builds the
contributor image, runs the container, verifies both skills are loaded
via canary tokens (STYLE-CANARY-3318, ESC-CANARY-7742)
- Add 'agent-skills' to the bootstrap script scenario list
- Add agent-skills row to the integration test README scenarios table
- Exclude HostedAgentSkillsPatternTests from net472 (uses net8.0+ APIs)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* .NET: Update commented-out package versions to latest across all hosted samples
Update the end-user PackageReference versions (in the commented-out
sections) from 1.0.0 to the current latest NuGet versions:
- Microsoft.Agents.AI: 1.6.1
- Microsoft.Agents.AI.Foundry: 1.6.1-preview.260514.1
- Microsoft.Agents.AI.Foundry.Hosting: 1.6.1-preview.260514.1
- Microsoft.Agents.AI.Hosting: 1.6.1-preview.260514.1
- Microsoft.Agents.AI.OpenAI: 1.6.1
- Microsoft.Agents.AI.Workflows: 1.6.1
Also adds explicit versions to Hosted-Workflow-Handoff which had bare
PackageReference entries without Version attributes.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* .NET: Fix broken markdown links in Hosted-AgentSkills README
Remove references to non-existent ../../README.md. Replace with
inline instructions matching other hosted samples that don't have
a parent README.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* .NET: Use OS-appropriate string comparison in zip-slip guard
Use Ordinal on Unix (case-sensitive FS) and OrdinalIgnoreCase on
Windows to prevent case-based path bypass on Linux containers.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Initial plan
* .NET: Auto-wire ChatClient with OpenTelemetryChatClient in OpenTelemetryAgent
Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/96dd033a-0c48-4d3f-9148-324bfd436b75
Co-authored-by: rogerbarreto <19890735+rogerbarreto@users.noreply.github.com>
* Address review: remove extension overload; honor UseProvidedChatClientAsIs; drop redundant check
Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/6ac3f75d-eeb7-4811-8043-9a27511b0a8b
Co-authored-by: rogerbarreto <19890735+rogerbarreto@users.noreply.github.com>
* Resolve ChatClientAgent via GetService before checking options/chat client
Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/008d914d-8cbb-4e9f-81b6-f8c3c8bd8d04
Co-authored-by: rogerbarreto <19890735+rogerbarreto@users.noreply.github.com>
* Split OpenTelemetryAgent ctor to preserve original (innerAgent, sourceName) signature
Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/a890c9a7-0b77-40ab-802c-dfbf09f8c260
Co-authored-by: rogerbarreto <19890735+rogerbarreto@users.noreply.github.com>
* Preserve base AgentRunOptions properties and avoid double-wrap on user factory
Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/3afbf18c-de22-4236-a2f2-02ca1e98ae21
Co-authored-by: rogerbarreto <19890735+rogerbarreto@users.noreply.github.com>
* .NET: OpenTelemetryAgent normalize sourceName once and add OTEL wiring path coverage
Normalize the configured source name once in the constructor so the outer OpenTelemetryChatClient and the auto-wired inner OpenTelemetryChatClient always emit spans on the same ActivitySource. A caller passing an empty string previously produced agent-level spans on DefaultSourceName but auto-wired chat spans on the empty source, causing the chat spans to be silently dropped by exporters subscribed to the default source.
Tests added to cover the previously unexercised OTEL wiring branches:
- Ctor_NullOrEmptySourceName_AutoWiredChatClientUsesDefaultSource_Async (Theory: null and empty)
- AutoWireChatClient_PlainAgentRunOptions_PreservesContinuationToken_Async
- AutoWireChatClient_ChatClientAgentRunOptions_NoUserFactory_PreservesChatOptions_Async
- AutoWireChatClient_StreamingDisabled_DoesNotEmitChatSpan_Async
* .NET: Mark OpenTelemetryAgent autoWireChatClient ctor as [Experimental]
Annotate the new 3-arg OpenTelemetryAgent(AIAgent, string?, bool) constructor with [Experimental(DiagnosticIds.Experiments.AgentsAIExperiments)] (MAAI001) so callers must explicitly opt in to the auto-wire toggle. The original 2-arg constructor stays non-experimental and delegates with autoWireChatClient: true; the delegating call is locally suppressed so the existing source compatibility surface is preserved.
* .NET: OpenTelemetryAgent address westey-m PR review
- Use string.IsNullOrWhiteSpace (not IsNullOrEmpty) when normalizing the constructor sourceName, so callers passing whitespace-only strings still land on OpenTelemetryConsts.DefaultSourceName instead of an unsubscribed ActivitySource.
- Fix the misleading pragma comment on the 2-arg ctor delegating call: auto-wiring is the new default, it does not preserve the original (pre-PR) behavior.
- Expand the GetRunOptionsWithChatClientWiring XML doc to spell out that a base AgentRunOptions (not ChatClientAgentRunOptions) is also accepted: it is converted to ChatClientAgentRunOptions with the auto-wire factory installed and base properties copied.
- Tests: extend the source-name normalization Theory with whitespace cases (' ' and '\t'); add end-to-end coverage for plain AgentRunOptions over a real ChatClientAgent (sync + streaming) asserting the inner chat client is invoked and both invoke_agent + chat spans are emitted.
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: rogerbarreto <19890735+rogerbarreto@users.noreply.github.com>
* .NET: Persist input messages on streaming errors in PerServiceCallChatHistoryPersistingChatClient
When the underlying chat service emits an in-stream error (for example a
`response.error` SSE event from the OpenAI Responses API on rate limit),
the OpenAI client surfaces it as an `ErrorContent` update and ends the
stream without throwing. Previously, `PerServiceCallChatHistoryPersistingChatClient`
only persisted history when the streaming loop completed successfully and
`NotifyProvidersOfNewMessagesAsync` was called at the end. On the
in-stream-error path, the input messages handed to that iteration -
typically `FunctionResultContent` produced by `FunctionInvokingChatClient`
in the previous iteration - were never persisted. The next run would
replay session history with a dangling `FunctionCallContent` and the
service would reject the request with `No tool output found for function
call <id>`.
This change:
- Adds a `PersistInputOnErrorAsync` helper that persists the input
messages (with no response messages) so function-call/function-result
pairings are not split across failures.
- Calls the helper from every error path: pre-loop enumerator creation,
the first `MoveNextAsync`, the in-loop `MoveNextAsync`, and a new
`finally` that handles abnormal iterator disposal.
- After the streaming loop, scans the assembled response for any
`ErrorContent` and, if present, persists the input, notifies
providers of failure, and throws `InvalidOperationException` so the
error is surfaced to the caller instead of silently corrupting history.
- Hardens `InMemoryChatHistoryProvider.StoreChatHistoryAsync` to treat
a null `RequestMessages` as empty, since the new error path can
invoke it with no response messages.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix dropped FunctionResultContent on streaming pipeline early-disposal
When a consumer of ChatClientAgent.RunStreamingAsync stops iterating early
(e.g. ToolApprovalAgent yields the approval request and then `yield break`),
the framework cascades DisposeAsync down the stream. C# async iterators do
not auto-dispose IAsyncDisposable locals, so the inner enumerator returned
by IChatClient.GetStreamingResponseAsync(...).GetAsyncEnumerator(ct) was
left suspended. That suspended FunctionInvokingChatClient downstream, which
suspended PerServiceCallChatHistoryPersistingChatClient at its `yield
return`, so its finally block never ran and the in-flight
FunctionResultContent for the just-completed tool call was not persisted
to chat history. The next turn then loaded a session that contained a
FunctionCallContent with no matching FunctionResultContent and the model
returned HTTP 400 `No tool output found for function call`.
Fixes:
* ChatClientAgent.RunStreamingAsync: wrap the iteration in
try/finally that disposes the inner enumerator. Disposal now cascades
through the pipeline and PerService's finally runs on early exit.
* PerServiceCallChatHistoryPersistingChatClient: in the streaming path,
snapshot input messages with `messages.ToList()` (the caller, FICC,
reuses a single mutable buffer across iterations and may mutate it
before our finally / error path persists), wrap GetAsyncEnumerator,
the first MoveNextAsync, and in-loop MoveNextAsync in try/catch each
calling PersistInputOnErrorAsync + NotifyProvidersOfFailureAsync, and
add a finally that calls PersistInputOnErrorAsync when the loop did
not exit normally so per-iteration FRCs are persisted on early
disposal as well as on errors.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* .NET: Add tests for PerService streaming error/dispose persistence paths
Adds five regression tests covering the new error-path persistence in
PerServiceCallChatHistoryPersistingChatClient.GetStreamingResponseInnerAsync:
- Persists input messages when GetStreamingResponseAsync throws synchronously.
- Persists input messages when the first MoveNextAsync throws.
- Persists input messages when a mid-stream MoveNextAsync throws.
- Persists input messages when the consumer abandons enumeration early
(the ToolApprovalAgent yield-break / disposal-cascade case).
- Throws and persists input when the stream emits an in-band ErrorContent.
All 66 tests in the class pass on net10.0 and net472.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* .NET: Address PR feedback on PerService streaming error persistence
Two follow-ups from PR #5744 review:
1. Prevent duplicate persistence on the in-loop MoveNextAsync catch path.
The inner catch persists input messages, then rethrows, which propagates
through the surrounding try/finally where loopExitedNormally is still false,
causing the finally to persist again. Introduced an inputPersisted flag
that the inner catch sets after persisting; the finally now skips when
inputPersisted is true.
2. Use the caller's CancellationToken in the abnormal-exit finally instead
of CancellationToken.None, so cleanup remains responsive to cancellation.
Fall back to CancellationToken.None only when the caller's token is
already canceled (otherwise the persist call would observe the
cancellation, throw, and mask the original early-exit reason).
Tightened all five new streaming-error tests from Times.AtLeastOnce to
Times.Once on the input-persistence matcher to regression-guard against
duplicate persistence. All 66 tests in the class still pass (net10.0 + net472).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* .NET: Scope PerService streaming changes to cooperative early-exit only
Per discussion on PR #5744, scope this PR back to fix only the original
ToolApprovalAgent dropped-FunctionResultContent bug and address the
enumerator-disposal review comment. Specifically:
- Remove input-message persistence from the GetAsyncEnumerator and
MoveNextAsync error paths. Routing failed service calls through the
success notification channel was breaking the provider contract; we
will instead rely on inner-agent retries for transient errors. Failure
paths still call NotifyProvidersOfFailureAsync as before.
- Remove the in-stream ErrorContent detection block (same rationale).
- Keep the try/finally that calls the (now narrower) early-exit input
notification on cooperative disposal (e.g. ToolApprovalAgent yield
break). A new serviceErrorOccurred flag ensures we do NOT renotify
on exception paths.
- Always DisposeAsync the underlying enumerator on every exit path,
addressing the copilot-reviewer comment about leaked HTTP/streams.
- Rename PersistInputOnErrorAsync -> NotifyProvidersOfEarlyExitInputAsync
to better reflect what it does and when it runs (rogerbarreto nit).
- Apply rogerbarreto nit on InMemoryChatHistoryProvider null-coalescing.
- Drop the four tests that covered the removed error-path behavior;
keep RunStreamingAsync_PersistsInputMessages_WhenConsumerAbandons
EnumerationAsync (regression guard for the cooperative-pause path).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: alliscode <bentho@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Adding the ability to inject messages during the function call loop
* Split message injection functionality
* Remove interface, since it is not required not that we split the chat client.
* Address conversation id propogation
* Fix formatting issue
* Bump MEAI to 10.5.1 and add per-call x-client header support
Replaces the brittle UserAgentResponsesClient subclass with a clean
per-call x-client-* header pipeline built on the new Microsoft.Extensions.AI
10.5.1 OpenAIRequestPolicies hook.
Public surface (Microsoft.Agents.AI.Foundry, [Experimental(MAAI001)]):
* chatOptions.WithClientHeader(name, value) and .WithClientHeaders(IEnumerable)
validate the x-client- prefix (case-insensitive), apply all-or-nothing on
bulk, and throw InvalidOperationException on foreign-typed slot collision
* myAgent.AsBuilder().UseClientHeaders().Build() opts a customer-built agent
into the pipeline; idempotent via agent.GetService<ClientHeadersAgent>()
* Foundry-built agents (FoundryAgent.Create*) pre-wire automatically
Internals:
* ClientHeadersAgent decorator snapshots the dict at scope-push time so
concurrent runs sharing a ChatOptions reference do not leak headers
* ClientHeadersScope is an AsyncLocal<IReadOnlyDictionary<string,string>?>
with LIFO push/dispose semantics
* ClientHeadersPolicy singleton stamps headers via Headers.Set so per-call
values overwrite any same-name header from earlier policies and so
duplicate registration is value-stable
* OpenAIRequestPoliciesReflection dedups against MEAI's private _entries
field and falls back to AddPolicy on any reflection failure; a CI test
asserts the field shape on every MEAI bump
Hosting cleanup:
* Deleted UserAgentResponsesClient and its dummy throwing pipeline
* HostedAgentUserAgentPolicy is now registered via OpenAIRequestPolicies
in FoundryHostingExtensions.TryApplyUserAgent
Tests:
* 19 new unit tests in ClientHeadersExtensionsTests.cs covering validation,
AsyncLocal isolation, snapshot semantics, end-to-end wire stamping, and
shared-chat-client dedup
* Updated OpenTelemetryAgentTests for MEAI 10.5.1 changes to web_search
serialization and the reduced tool definition payload when sensitive
data capture is disabled
Microsoft.Extensions.Compliance.Abstractions stays at 10.5.0 because no
10.5.1 release exists on nuget.org.
* Address PR review: pre-wire AsAIAgent path and dedup TryApplyUserAgent
* FoundryAgent: extract WireClientHeaders helper and call it from the
internal (AIProjectClient, ChatClientAgent) constructor used by
AzureAIProjectChatClientExtensions.AsAIAgent so those Foundry-built
agents also pre-wire the x-client header pipeline.
* Foundry.Hosting TryApplyUserAgent: dedup HostedAgentUserAgentPolicy
registration per OpenAIRequestPolicies instance via
ConditionalWeakTable so per-request resolution does not grow the
policy list unboundedly on singleton agents.
* Add tests covering AsAIAgent pre-wire and TryApplyUserAgent dedup
Backs the PR review fixes from a4c8f91 with regression tests:
* ClientHeadersExtensionsTests: AsAIAgent_FoundryAgent_HasPreWiredClientHeadersAgent
asserts the FoundryAgent built via AzureAIProjectChatClientExtensions.AsAIAgent
contains a ClientHeadersAgent in its delegating chain (catches future
regressions of the bypass).
* ClientHeadersExtensionsTests: FoundryAgent_PublicConstructor_HasPreWiredClientHeadersAgent
covers the public constructor path the same way.
* ClientHeadersExtensionsTests: UseClientHeaders_RepeatedRegistrations_OnSameChatClient_OnlyRegistersOnce
invokes UseClientHeaders 25 times on a shared chat client and asserts via
reflection that OpenAIRequestPolicies._entries length is exactly 1.
* HostedTryApplyUserAgentDedupTests: two tests asserting
FoundryHostingExtensions.TryApplyUserAgent stays at one entry per
OpenAIRequestPolicies instance after 50 calls on the same agent and across
distinct agents on different chat clients.
* Move tests next to their SUT
Removes the dedicated HostedTryApplyUserAgentDedupTests.cs test class.
Tests are co-located with the SUT they exercise:
* FoundryAgentTests.cs gains the Constructor_PreWiresClientHeadersAgent
and Constructor_FromAsAIAgentExtension_PreWiresClientHeadersAgent
cases, since FoundryAgent is the SUT for the pre-wire behavior.
* HostedOutboundUserAgentTests.cs gains the two TryApplyUserAgent dedup
cases, since FoundryHostingExtensions.TryApplyUserAgent is the SUT
it already covers.
* ClientHeadersExtensionsTests.cs keeps only the
UseClientHeaders_RepeatedRegistrations_OnSameChatClient_OnlyRegistersOnce
case, which exercises the public ClientHeadersExtensions surface.
* Remove redundant WithCancellation on inner streaming call
ct is already passed to InnerAgent.RunStreamingAsync, so
.WithCancellation(ct) on the resulting IAsyncEnumerable is a no-op.
Caught by Sergey on PR review.
* Address PR review: surface downstream MEAI experimental ID
* Add AIOpenAIRequestPolicies = MEAIExperiments alias to
DiagnosticIds.Experiments (matches the existing AIResponseContinuations,
AIMcpServers, AIFunctionApprovals pattern).
* Mark public ClientHeadersExtensions with [Experimental(AIOpenAIRequestPolicies)]
instead of AgentsAIExperiments. Consumers now see the MEAI001 warning,
surfacing the dependency on MEAI's experimental OpenAIRequestPolicies hook.
* Mark internal OpenAIRequestPoliciesReflection with the same alias to
suppress warnings at the source rather than via project-wide NoWarn.
* Remove MEAI001 from Foundry csproj NoWarn (kept on Foundry.Hosting where
pre-PR usages remain).
* Clarify ClientHeadersScope XML doc: AsyncLocal flows values forward but
does NOT auto-restore on method return; explicit using/Dispose is what
gives stack-style LIFO semantics.
* Foundry Evals integration for .NET
- Core evaluation framework: EvalItem, LocalEvaluator, FunctionEvaluator, EvalChecks
- IAgentEvaluator interface with MeaiEvaluatorAdapter bridge
- AgentEvaluationExtensions for agent.EvaluateAsync() overloads
- FoundryEvals wrapping MEAI quality/safety evaluators
- ConversationSplitters (LastTurn, Full) and IConversationSplitter
- EvalItem.PerTurnItems() for multi-turn decomposition
- HasImageContent for multimodal content detection
- WorkflowEvaluationExtensions for per-agent workflow evaluation
- 7 eval samples mirroring Python parity:
02-agents/Evaluation: SimpleEval, ExpectedOutputs, Multimodal
03-workflows/Evaluation: WorkflowEval
05-end-to-end/Evaluation: FoundryQuality, MixedProviders, ConversationSplits
- Comprehensive unit tests (1958 passing)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Rewrite FoundryEvals to use real Foundry Evals API
Replace MEAI evaluator shim with actual OpenAI EvaluationClient protocol
methods. FoundryEvals now creates eval definitions, submits runs, polls
for completion, and fetches per-item results server-side.
- New constructor: FoundryEvals(AIProjectClient, model, evaluators)
- Add FoundryEvalConverter for MEAI ChatMessage -> Foundry JSON format
- Add EvalId, RunId, ReportUrl to AgentEvaluationResults
- All 20 built-in evaluator constants now work (agent, tool, quality, safety)
- Remove Microsoft.Extensions.AI.Evaluation.Quality/Safety dependencies
- Update all samples for new constructor (no more ChatConfiguration)
- Replace BuildEvaluators tests with ResolveEvaluator tests
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add response output to CustomEvals and ExpectedOutputs samples
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address review: pagination, validation, error handling, tests
FoundryEvals fixes:
- Add pagination for output items (has_more/after cursor)
- Add guard clauses for pollIntervalSeconds/timeoutSeconds <= 0
- Fix double TryGetProperty for passed field parsing
- Throw on all-tool-evaluators with no tool definitions
- Fix XML doc (default 300s, not 180s)
New tests (30 added, 1989 total):
- EvalChecks: NonEmpty, ContainsExpected (pass/fail/skip/case),
HasImageContent, ToolCallsPresent
- FoundryEvalConverter: ConvertMessage (text, image, function call,
function results fan-out, empty fallback, mixed content),
ConvertEvalItem, BuildTestingCriteria (quality/agent/tool/groundedness
data mappings), BuildItemSchema
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix review: null-refs, Data.ToString() bug, ContainsExpected, add tests
- Fix NullReferenceException in sample Response display (pattern matching)
- Fix WorkflowEvaluationExtensions Data?.ToString() producing type names
instead of message text (pattern-match ChatMessage/AgentResponse/list)
- Change EvalChecks.ContainsExpected to return Passed=false when no
ExpectedOutput (was silently passing, masking misconfiguration)
- Add EvalItem constructor tests with LastTurn/Full/null splitters
- Add FoundryEvalConverter.ConvertMessage DataContent (base64 image) test
- Add ExtractAgentData tests with ChatMessage, list, and AgentResponse data
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix review: conversation fidelity, eval caching, fallback tests
- WorkflowEvaluationExtensions: preserve full response messages (tool calls,
intermediate) instead of synthetic 2-message conversation. Cast completed
Data to AgentResponse and use Messages when available, fallback to text.
- FoundryEvals: cache evalId per schema shape (hasContext, hasTools) so
subsequent EvaluateAsync calls create runs under the same eval definition.
- MeaiEvaluatorAdapter: code already correctly passes queryMessages (not full
conversation) to IEvaluator — no change needed, verified by inspection.
- Add tests: AgentResponse full messages preservation, unknown object
ToString() fallback for ExtractAgentData.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Rename AzureAI→Foundry: move eval files, update references
- Move FoundryEvals.cs and FoundryEvalConverter.cs from
Microsoft.Agents.AI.AzureAI to Microsoft.Agents.AI.Foundry
- Update namespace from AzureAI to Foundry in both files
- Add explicit usings required by Foundry project (no implicit usings)
- Move FoundryEvalConverter tests to Foundry.UnitTests project
(avoids ReplacingRedactor type conflict from dual project refs)
- Update all sample csproj references and using statements
- Remove Foundry project reference from AI UnitTests
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* PR review round 4: wire up tool extraction, remove eval cache, fix null safety
- BuildEvalItem: extract tools from agent via GetService<ChatOptions>() into EvalItem.Tools (Python parity)
- FoundryEvals: remove eval ID cache - each call creates fresh definition (matches Python behavior)
- FoundryEvals: replace null-forgiving operators with descriptive InvalidOperationException
- MixedProviders sample: remove unnecessary explicit PackageReferences (transitively provided)
- FoundryEvalConverter: document that tool results take precedence over text content
- Add LocalEvaluator zero-checks test documenting 0 metrics = failed behavior
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python-dotnet parity: 9 feature gaps filled
New checks:
- ToolCallArgsMatch() — verify tool call names + argument subset match
- ToolCalledCheck(ToolCalledMode.Any, ...) — match any of the specified tools
- ToolCalledMode enum (All/Any)
FoundryEvals enhancements:
- Default evaluators now [Relevance, Coherence, TaskAdherence] (was Relevance, Coherence)
- Auto-add ToolCallAccuracy when items have tool definitions
- EvaluateTracesAsync — evaluate by response_ids, trace_ids, or agent_id
- EvaluateFoundryTargetAsync — evaluate deployed Foundry targets
Result type enrichment:
- AgentEvaluationResults: added Status, Error, PerEvaluator, DetailedItems
- New EvalItemResult/EvalScoreResult/PerEvaluatorResult types
- FoundryEvals populates all new fields from API responses
Workflow fix:
- Skip internal executors (_*, input-conversation, end-conversation, end)
Tests: 8 new tests covering ToolCallArgsMatch, ToolCalledMode.Any, internal executor filtering
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add MeaiEvaluatorAdapter and PerTurnItems edge case tests
- 3 tests for MeaiEvaluatorAdapter: query message forwarding, synthetic
response fallback, multiple items aggregation
- 3 tests for EvalItem.PerTurnItems: empty conversation, no user messages,
system+assistant only
- StubEvaluator and StubChatClient test helpers
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Blocking link check for outdated package in DevUI.
* Replace Dictionary<string, object> payloads with typed wire models
Introduce internal FoundryEvalWireModels.cs with compile-time-safe types
for the OpenAI Evals API wire format. The OpenAI .NET SDK (2.9.1) only
provides protocol-level methods with BinaryContent/ClientResult — no
typed request models. These internal models replace scattered dictionary
literals with [JsonPropertyName]-annotated classes, giving:
- Compile-time safety (typos become build errors)
- Single point of change when the API evolves
- IntelliSense discoverability
- Cleaner serialization via JsonPolymorphic for content items
Models: WireContentItem hierarchy (text, image, tool_call, tool_result),
WireMessage, WireEvalItemPayload, WireTestingCriterion, WireItemSchema,
WireCreateEvalRequest, WireCreateRunRequest, and data source variants.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Skip metric when Foundry returns neither score nor passed
When an evaluator returns no score and no passed value, the previous
code created BooleanMetric(name, false), which falsely failed items
via ItemPassed. Now we skip the MEAI metric entirely for indeterminate
results — the raw data remains available in DetailedItems for diagnostics.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address PR #4914 review comments: fix tool evaluator bug and add tests
- Fix duplicate ToolCallAccuracy: resolve evaluator names before checking
against ToolEvaluators set (Comment 2)
- Make FilterToolEvaluators internal for testability; add tests for the
ArgumentException edge case when all evaluators are tool-type (Comment 3)
- Add CancellationToken test for LocalEvaluator (Comment 4)
- Add EvaluateAsync integration test on Run with sequential workflow and
per-agent SubResults verification (Comment 5)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address Peter's review comments on PR #4914
- Add trailing newline to Evaluation_FoundryQuality.csproj (Comment 6)
- Make evaluator name lookups case-insensitive: switch BuiltinEvaluators,
ToolEvaluators, AgentEvaluators, and ResolveEvaluator's StartsWith check
from Ordinal to OrdinalIgnoreCase (Comment 7)
- Add Trace.TraceWarning when Foundry returns fewer results than submitted
items, indicating expected vs actual count before padding (Comment 8)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add Microsoft.Extensions.AI.Evaluation packages to Directory.Packages.props
These were removed in #5269 as unused, but are needed by the Foundry
and core evaluation integration added in this PR.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: alliscode <bentho@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Rename authored identifiers, XML docs, log messages, and comments
from 'folder' to 'directory' across the file skills codebase for
consistency with the agentskills.io specification and .NET conventions.
Public API changes (experimental):
- ScriptFolders → ScriptDirectories
- ResourceFolders → ResourceDirectories
.NET BCL API calls (Directory.Exists, Path.GetDirectoryName, etc.)
were already using 'directory' and are unchanged.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* support reflection for discovery of resources and scripts in class-based skills
* fix format issues
* refactor samples to use reflection
* Validate resource member signatures during discovery
Add discovery-time validation in AgentClassSkill.DiscoverResources() to
fail fast when [AgentSkillResource] is applied to members with incompatible
signatures:
- Reject indexer properties (getter has parameters)
- Reject methods with parameters other than IServiceProvider or
CancellationToken
Throws InvalidOperationException with actionable error messages instead of
allowing silent runtime failures when ReadAsync invokes the AIFunction with
no named arguments.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* prevent duplicates
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* .NET: Add JsonSerializerOptions support to programmatic skill APIs
Allow callers to pass custom JsonSerializerOptions when creating inline
resources and scripts via AgentInlineSkill, AgentClassSkill,
AgentInlineSkillResource, and AgentInlineSkillScript. A skill-level
default can be set on AgentInlineSkill and overridden per-resource/
script call.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Update dotnet/tests/Microsoft.Agents.AI.UnitTests/AgentSkills/TestSkillTypes.cs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* add class-based skills
* address formating issues
* Remove generated filtered-unit.slnx and add to .gitignore
The filtered solution file is generated dynamically by
eng/scripts/New-FilteredSolution.ps1 during CI. Checking it in
risks it becoming stale and out-of-sync with the real solution.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Remove generated filtered-unit.slnx and add to .gitignore
The filtered solution file is generated dynamically by
eng/scripts/New-FilteredSolution.ps1 during CI. Checking it in
risks it becoming stale and out-of-sync with the real solution.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* discover scripts and resource from folders defined in spec
* Remove Step05 and Step06 DI skill samples
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* address review comments
* fix build error
* Fix mixed path separators in skill folder discovery on .NET Framework
Path.Combine with forward-slash folder names (e.g. "scripts/f1") produces
mixed separators on Windows, causing the StartsWith containment check to
fail against Path.GetFullPath-resolved file paths. Wrap in Path.GetFullPath
to canonicalize separators before the containment comparison.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* address comment
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* add class-based skills
* address formating issues
* Remove generated filtered-unit.slnx and add to .gitignore
The filtered solution file is generated dynamically by
eng/scripts/New-FilteredSolution.ps1 during CI. Checking it in
risks it becoming stale and out-of-sync with the real solution.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Remove generated filtered-unit.slnx and add to .gitignore
The filtered solution file is generated dynamically by
eng/scripts/New-FilteredSolution.ps1 during CI. Checking it in
risks it becoming stale and out-of-sync with the real solution.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* consolidate DI samples into one
* fix file encoding
* suppress compatibility warning
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* add inline skills
* Fix IDE1006 and IDE0004 formatting errors in test files
- Add 'Async' suffix to async test methods in FilteringAgentSkillsSourceTests,
DeduplicatingAgentSkillsSourceTests, and AgentInMemorySkillsSourceTests
- Use pragma to suppress false-positive IDE0004 on casts needed for overload
disambiguation in AgentInlineSkillTests and AgentInlineSkillResourceTests
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* address issues
* address comments
* make inline skills script and resource model classes internal
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Changes
* Fix ChatClientAgent streaming responses missing MessageId
Generate fallback MessageId in ChatClientAgent.RunCoreStreamingAsync when
the underlying LLM provider does not set ChatResponseUpdate.MessageId.
Without a MessageId the AGUI converter's null==null check silently drops
all text content, causing CopilotKit Zod validation errors.
Changes:
- ChatClientAgent: generate msg_{Guid} fallback via ??= in streaming loop
- AgentResponseExtensions: sync wrapper MessageId back to RawRepresentation
in AsChatResponseUpdate() so downstream consumers see the value
- Add unit tests for both fixes and AGUI streaming MessageId scenarios
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address PR #4615 review comments
- Fix MessageId seeding: use first-seen provider MessageId (or generate
fallback) and apply consistently to all chunks in the stream, preventing
message splitting when providers set MessageId only on the first chunk
- Add test for mixed MessageId scenario (first chunk only)
- Fix skipped TextStreaming test: assert Empty (not NotEmpty) to match
actual null==null behavior
- Fix skipped ToolCalls test: assert empty ParentMessageId to match
actual empty-string passthrough behavior
* Handle empty MessageId in AsChatResponseUpdate sync
Treat empty/whitespace MessageId the same as null when syncing from
the AgentResponseUpdate wrapper back to RawRepresentation. Providers
that return empty string MessageId (e.g. tool call responses) now get
the wrapper value recovered correctly.
Add test for empty string MessageId recovery scenario.
* Move MessageId fallback generation to AGUI layer
Move fallback MessageId generation from ChatClientAgent to
AsAGUIEventStreamAsync, addressing the architectural concern that
MessageId is nullable in the AIAgent abstraction and the requirement
for non-null values is specific to the AGUI protocol.
The AGUI layer now generates a fallback MessageId for null or
empty/whitespace values, covering all agent types (not just
ChatClientAgent) including external implementations.
Changes:
- Revert MessageId generation from ChatClientAgent.RunCoreStreamingAsync
- Add fallback MessageId generation in AsAGUIEventStreamAsync for
null/empty MessageId values (handles both null and whitespace)
- Unskip and update AGUI tests to verify fallback generation
- Update ChatClientAgent tests to reflect passthrough behavior
* Revert AsChatResponseUpdate MessageId sync-back
Remove the MessageId sync-back logic from AsChatResponseUpdate() as it
is no longer needed. With fallback generation moved to the AGUI layer,
the abstraction layer should not mutate the RawRepresentation object.
Revert to the original passthrough behavior for AsChatResponseUpdate()
and update tests accordingly.
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Persist messages during the Function Call Loop
* Revert version reset
* Fix bugs and improve sample
* Fix formatting issues
* Also updating conversation id during run
* Update based on ADR feedback
* Fix FileAgentSkillsProvider accepting SkillsInstructionPrompt without {0} placeholder (#4638)
BuildSkillsInstructionPrompt validated only format-string syntax via
string.Format(template, ""), which silently accepted templates without a
{0} placeholder. The generated skills list was then dropped from the final
instructions.
Tighten validation to format with a sentinel string and verify it appears
in the output, rejecting templates that do not reference argument 0 with
an ArgumentException.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix netstandard2.0 compat and simplify prompt template validation (#4638)
- Replace string.Contains(string, StringComparison) with IndexOf for
netstandard2.0/net472 compatibility
- Remove sentinel round-trip check; validate {0} directly on the raw
template string using IndexOf
- Add positive test verifying custom SkillsInstructionPrompt with {0}
is accepted and applied to output
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix filter combine logic for ChatHistoryMemoryProvider
* Replace var with explicit types in filter building code and test
Address PR review nit: use explicit types instead of var for better
readability in the filter-building logic and the new combined filter
compilation test.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix style issues
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
westey ·
MaciejWarchalowski ·
SergeyMenshykh ·
semenshi-m ·
Roger Barreto ·
Copilot ·
Ben Thomas ·
Teja Kusireddy ·
chetantoshniwal ·
Evan Mattson ·
Chris ·