mirror of
https://github.com/microsoft/agent-framework.git
synced 2026-06-16 21:04:09 +08:00
feature/hosted-agent-with-observability-sample
8
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  Roger BarretoandGitHub
|
fb97e93a01 |
.NET: Add dedicated Foundry.Hosting UnitTest project (#5592)
* Foundry.Hosting.UnitTests: extract project from Foundry.UnitTests Move all Hosting/* tests, three toolbox TestData JSONs, and the FakeAuthenticationTokenProvider/HttpHandlerAssert/TestDataUtil helpers (trimmed to toolbox getters) into a new Microsoft.Agents.AI.Foundry.Hosting.UnitTests project. Add it to the slnx and grant the new assembly InternalsVisibleTo from Microsoft.Agents.AI.Foundry and Microsoft.Agents.AI.Foundry.Hosting. * Foundry.Hosting.UnitTests: align namespaces to assembly name Rename namespaces from Microsoft.Agents.AI.Foundry.UnitTests(.Hosting) to Microsoft.Agents.AI.Foundry.Hosting.UnitTests across all moved tests, the duplicated helpers, and the trimmed TestDataUtil. Also fixes the prior namespace inconsistency in FoundryToolboxTests. * Foundry.Hosting.UnitTests: split WorkflowIntegrationTests by SUT Replace the WorkflowIntegrationTests file (an IT-named file inside a UT project) with two SUT-focused files plus a shared test-doubles file: - AgentFrameworkResponseHandlerWorkflowTests.cs - the 5 handler-driven tests that exercise AgentFrameworkResponseHandler with a real workflow agent. - OutputConverterWorkflowTests.cs - the 5 OutputConverter tests driven by hand-crafted update sequences mirroring real workflow patterns. - WorkflowTestAgents.cs - StreamingTextAgent and ThrowingStreamingAgent extracted as internal types used by both files. * Foundry.UnitTests: trim Hosting-related conditionals and dead testdata Now that Hosting tests live in their own project: - drop the Compile Remove guard for the Hosting subfolder, - drop the .NETCoreApp-only PackageReferences (Azure.AI.AgentServer.Responses, Microsoft.AspNetCore.TestHost, OpenTelemetry, OpenTelemetry.Exporter.InMemory), - drop the conditional ProjectReference to Microsoft.Agents.AI.Foundry.Hosting, - delete the three Toolbox JSON files and the matching Toolbox getters in TestDataUtil. * Foundry.Hosting.UnitTests: drop redundant 'using Microsoft.Agents.AI.Foundry.Hosting' The new project namespace is Microsoft.Agents.AI.Foundry.Hosting.UnitTests, which already brings the parent Microsoft.Agents.AI.Foundry.Hosting namespace into scope. The explicit using statement is therefore redundant (IDE0005). Caught by 'dotnet format --verify-no-changes' running on Linux against the .NET 10 SDK. * Foundry.Hosting: drop InternalsVisibleTo to Foundry.UnitTests The non-hosting Foundry.UnitTests project no longer holds any Hosting tests after the split, so it doesn't need access to internal types in Microsoft.Agents.AI.Foundry.Hosting. Only Microsoft.Agents.AI.Foundry.Hosting.UnitTests needs it. * Foundry.Hosting: rename DelegatingResponsesClient to UserAgentResponsesClient Address westey-m's review feedback on PR #5453: `Delegating*` is conventionally reserved for inheritable base classes (mirroring `DelegatingHandler`) where consumers override one or two members. This polyfill is sealed and only injects the User-Agent supplement, so the new name reflects its actual purpose. Renamed via `git mv` to preserve history: * `src/Microsoft.Agents.AI.Foundry.Hosting/DelegatingResponsesClient.cs` to `UserAgentResponsesClient.cs` * `tests/Microsoft.Agents.AI.Foundry.Hosting.UnitTests/DelegatingResponsesClientTests.cs` to `UserAgentResponsesClientTests.cs` Class, constructor, and all references updated across: * `src/.../UserAgentResponsesClient.cs` (class + constructor + internal log message) * `src/.../ServiceCollectionExtensions.cs` (cref + type check + instantiation) * `src/.../HostedAgentUserAgentPolicy.cs` (cref) * `tests/Foundry.UnitTests/RequestOptionsExtensionsTests.cs` (comment) * `tests/Foundry.Hosting.UnitTests/UserAgentResponsesClientTests.cs` (class + cref + instantiations) |
||
  
|
6cd81286a9 |
.NET: dotnet: Add hosted-agent User-Agent supplement to outgoing requests (#5453)
* dotnet: Add hosted-agent User-Agent supplement to outgoing requests
When an agent runs inside a Foundry Hosted Agent, the outgoing
User-Agent header now includes 'agent-framework-hosted/{version}'
alongside the existing 'MEAI/{version}' segment.
- Add HostedAgentContext with AsyncLocal<string?> property
- MeaiUserAgentPolicy reads the supplement per-call
- AgentFrameworkResponseHandler sets/restores the context
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* chore: update hosted UA format to foundry-hosting/agent-framework-dotnet/{version}
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Trying to get UA flowing, no luck yet.
* .NET: Polyfill MEAI OpenAIResponsesChatClient to add hosted-agent User-Agent supplement
When AgentFrameworkResponseHandler resolves an agent (i.e. we are running in a
hosted context), TryApplyUserAgent walks the agent's IChatClient decorator chain
to find MEAI's internal OpenAIResponsesChatClient and reflectively swaps its
inner _responseClient field with a DelegatingResponsesClient wrapper. The
wrapper overrides the public-virtual protocol methods to add a per-call
HostedAgentUserAgentPolicy to the RequestOptions and delegate to the inner
ResponsesClient. The OpenAI SDK's internal streaming overloads bottom out in
calls to the public-virtual non-streaming overloads via virtual dispatch on
this, so streaming is covered without overriding any non-virtual member.
The wrapper accepts any ResponsesClient-derived inner — both the Foundry
ProjectResponsesClient and the native OpenAI ResponsesClient — and preserves
the inner client's full pipeline (Transport, RetryPolicy, NetworkTimeout,
OrganizationId / ProjectId / UserAgentApplicationId, custom policies).
- Add DelegatingResponsesClient + HostedAgentUserAgentPolicy in Microsoft.Agents.AI.Foundry.Hosting.
- Add TryApplyUserAgent next to ApplyOpenTelemetry in FoundryHostingExtensions; wire it into AgentFrameworkResponseHandler.GetAgent for both keyed and default-agent paths.
- Drop earlier-iteration dead code: AddHostedAgentTelemetry extension, HostedUserAgentPolicy class, HostedAgentContext.cs, and the never-called ToRequestOptions helper.
- Revert RequestOptionsExtensions.MeaiUserAgentPolicy to MEAI-only (the supplement is now injected by the polyfill).
- Revert unrelated whitespace change in Agent_Step25_ToolboxServerSideTools sample.
- Tests cover streaming AND non-streaming, retry policy preservation, OrganizationId/ProjectId/UserAgentApplicationId pass-through, idempotency, native OpenAI ResponsesClient, and reflection guards for MEAI/OpenAI shape drift.
* .NET: Address review feedback on hosted-agent User-Agent polyfill
- TryApplyUserAgent: replace silent null-return with ArgumentNullException to match the codebase's convention.
- Add idempotency test (TryApplyUserAgent_CalledTwiceOnSameAgent_DoesNotDoubleWrap) — runs the polyfill twice on the same agent and asserts the wire UA contains exactly one foundry-hosting segment, proving the 'current is DelegatingResponsesClient' guard prevents nested wrapping.
- Add retry-double-append test (Polyfill_RetryWithinCall_DoesNotDuplicateSupplementInUserAgent) — exercises the HostedAgentUserAgentPolicy Contains-guard via a custom retry policy that re-runs the inner pipeline on the same message.
- Replace TryApplyUserAgent_NullAgent_ReturnsNullWithoutThrowing with TryApplyUserAgent_NullAgent_ThrowsArgumentNullException to match the new contract.
* .NET: Drop null check from TryApplyUserAgent and its now-redundant test
The two call sites in AgentFrameworkResponseHandler.GetAgent already null-check the agent before invoking TryApplyUserAgent, so the defensive ArgumentNullException is unreachable. Remove it and the corresponding test.
* .NET: Remove unused Microsoft.Shared.Diagnostics import in ServiceCollectionExtensions
The Throw.IfNull helper from this namespace was used by the now-removed null check in TryApplyUserAgent. Drop the unused import to satisfy IDE0005 in CI's full-project dotnet format run.
---------
Co-authored-by: alliscode <bentho@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Roger Barreto <19890735+rogerbarreto@users.noreply.github.com>
|
||
 
|
5fe8941ff9 |
.NET: dotnet: Add server-side Foundry Toolbox support and fix SDK beta.4 br… (#5450)
* dotnet: Add server-side Foundry Toolbox support and fix SDK beta.4 breaking changes Add FoundryToolbox and AIProjectClient extensions to Microsoft.Agents.AI.Foundry.Hosting for server-side toolbox tool integration matching Python's FoundryChatClient.get_toolbox() pattern. Tools are fetched from the Foundry project SDK and passed as server-side tools in the Responses API request. New files: - FoundryToolbox.cs: Core implementation using AgentAdministrationClient SDK - AIProjectClientToolboxExtensions.cs: Extension methods on AIProjectClient - Agent_Step25_ToolboxServerSideTools sample with create helper and combine flow - 19 unit tests covering param validation, conversion, sanitization, and extensions SDK breaking changes (Azure.AI.AgentServer.Responses beta.3 -> beta.4): - FunctionToolCallOutputResource renamed to OutputItemFunctionToolCallOutput - AzureAIAgentServerResponsesModelFactory made internal, replaced with direct constructors - ResponseUsage constructor now requires non-null token details parameters Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: reuse endpoint variable in CreateSampleToolboxAsync Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: pass endpoint through static local functions to avoid capture Static local functions cannot capture top-level variables. Thread the endpoint parameter through Main, CombineToolboxes, and CreateSampleToolboxAsync. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * refactor: remove unused projectClient param from CreateSampleToolboxAsync Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Update dotnet/samples/02-agents/AgentsWithFoundry/Agent_Step25_ToolboxServerSideTools/README.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update dotnet/samples/02-agents/AgentsWithFoundry/Agent_Step25_ToolboxServerSideTools/Program.cs Co-authored-by: westey <164392973+westey-m@users.noreply.github.com> * Update dotnet/samples/02-agents/AgentsWithFoundry/Agent_Step25_ToolboxServerSideTools/Program.cs Co-authored-by: westey <164392973+westey-m@users.noreply.github.com> * Removing GetToolbocVersion. * Removing tests for GetToolboxVersion * fix: map cached/reasoning token counts in ConvertUsage instead of hardcoding zeros Extract InputTokenDetails.CachedTokenCount and OutputTokenDetails.ReasoningTokenCount from UsageDetails.AdditionalCounts, matching the pattern in AgentResponseExtensions. Also accumulate detail counts when merging with existing usage. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: alliscode <bentho@microsoft.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: westey <164392973+westey-m@users.noreply.github.com> |
||
|
|
f2b215a2f6 |
.NET [WIP] Foundry Hosted Agents Support (#5312)
* Add Azure AI Foundry Responses hosting adapter Implement Microsoft.Agents.AI.Hosting.AzureAIResponses to host agent-framework AIAgents and workflows within Azure Foundry as hosted agents via the Azure.AI.AgentServer.Responses SDK. - AgentFrameworkResponseHandler: bridges ResponseHandler to AIAgent execution - InputConverter: converts Responses API inputs/history to MEAI ChatMessage - OutputConverter: converts agent response updates to SSE event stream - ServiceCollectionExtensions: DI registration helpers - 336 unit tests across net8.0/net9.0/net10.0 (112 per TFM) - ResponseStreamValidator: SSE protocol validation tool for samples - FoundryResponsesHosting sample app Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Bump System.ClientModel to 1.10.0 for Azure.Core 1.52.0 compat Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Clean up tests and sample formatting Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Update Azure.AI.AgentServer packages to 1.0.0-alpha.20260401.5 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add hosted package version suffix (0.9.0-hosted) to distinguish from mainline Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Move Foundry Responses hosting into Microsoft.Agents.AI.Foundry package Move source and test files from the standalone Hosting.AzureAIResponses project into the Foundry package under a Hosting/ subfolder. This consolidates the Foundry-specific hosting adapter into the main Foundry package. - Source: Microsoft.Agents.AI.Foundry.Hosting namespace - Tests: merged into Foundry.UnitTests/Hosting/ - Conditionally compiled for .NETCoreApp TFMs only (net8.0+) - Deleted standalone Hosting.AzureAIResponses project and test project - Updated sample and solution references Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Bump package version to 0.9.0-hosted.260402.2 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Bump OpenTelemetry packages to fix NU1109 downgrade errors - OpenTelemetry/Api/Exporter.Console/Exporter.InMemory: 1.13.1 -> 1.15.0 - OpenTelemetry.Exporter.OpenTelemetryProtocol: already 1.15.0 - OpenTelemetry.Extensions.Hosting: already 1.14.0 - OpenTelemetry.Instrumentation.AspNetCore/Http: already 1.14.0 - OpenTelemetry.Instrumentation.Runtime: 1.13.0 -> 1.14.0 - Azure.Monitor.OpenTelemetry.Exporter: 1.4.0 -> 1.5.0 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix CA1873: guard LogWarning with IsEnabled check Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix model override bug and add client REPL sample - InputConverter: stop propagating request.Model to ChatOptions.ModelId Hosted agents use their own model; client-provided model values like 'hosted-agent' were being passed through and causing server errors. - Add FoundryResponsesRepl sample: interactive CLI client that connects to a Foundry Responses endpoint using ResponsesClient.AsAIAgent() - Bump package version to 0.9.0-hosted.260403.1 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Catch agent errors and emit response.failed with real error message Previously, unhandled exceptions from agent execution would bubble up to the SDK orchestrator, which emits a generic 'An internal server error occurred.' message — hiding the actual cause (e.g., 401 auth failures, model not found, etc.). Now AgentFrameworkResponseHandler catches non-cancellation exceptions and emits a proper response.failed event containing the real error message, making it visible to clients and in logs. OperationCanceledException still propagates for proper cancellation handling by the SDK. Also bumps package version to 0.9.0-hosted.260403.2. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Renaming and merging hosting extensions. (#5091) * Rename AddAgentFrameworkHandler to AddFoundryResponses and add MapFoundryResponses - Rename extension methods: AddAgentFrameworkHandler -> AddFoundryResponses, MapAgentFrameworkHandler -> MapFoundryResponses - AddFoundryResponses now calls AddResponsesServer() internally - Add MapFoundryResponses() extension on IEndpointRouteBuilder - Update sample and tests to use new API names - Remove redundant AddResponsesServer() and /ready endpoint from sample Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fixing numbering in sample. --------- Co-authored-by: alliscode <bentho@microsoft.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Address breaking changes in 260408 * Bump hosted internal package version * Add UserAgent middleware tests for Foundry hosting * Hosting Samples update * Hosting Samples update * Hosting Samples update * Hosting Samples update * ChatClientAgent working * Adding SessionStorage and SessionManagement, improving samples to align Consumption vs Hosting * Using updates * Update chat client agent for contributor and devs * Foundry Agent Hosting * Address text rag sample working * Version bump * Adding LocalTools + Workflow samples * Removing extra using samples * Add Hosted-McpTools sample with dual MCP pattern Demonstrates two MCP integration layers in a single hosted agent: - Client-side MCP: McpClient connects to Microsoft Learn, agent handles tool invocations locally (docs_search, code_sample_search, docs_fetch) - Server-side MCP: HostedMcpServerTool delegates tool discovery and invocation to the LLM provider (Responses API), no local connection Includes DevTemporaryTokenCredential for Docker local debugging, Dockerfile.contributor for ProjectReference builds, and the openai/v1 route mapping for AIProjectClient compatibility in Development mode. * .NET: Bump Azure.AI.AgentServer packages to 1.0.0-beta.1/beta.21 and fix br… (#5287) * Bump Azure.AI.AgentServer packages to 1.0.0-beta.1/beta.21 and fix breaking API changes - Azure.AI.AgentServer.Core: 1.0.0-beta.11 -> 1.0.0-beta.21 - Azure.AI.AgentServer.Invocations: 1.0.0-alpha.20260408.4 -> 1.0.0-beta.1 - Azure.AI.AgentServer.Responses: 1.0.0-alpha.20260408.4 -> 1.0.0-beta.1 - Azure.Identity: 1.20.0 -> 1.21.0 (transitive requirement) - Azure.Core: 1.52.0 -> 1.53.0 (transitive requirement) - Remove azure-sdk-for-net dev feed (packages now on nuget.org) - Fix OutputConverter for new builder API (auto-tracked children, split EmitTextDone/EmitDone) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fixing small issues. --------- Co-authored-by: alliscode <bentho@microsoft.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add Azure AI Foundry Responses hosting adapter Implement Microsoft.Agents.AI.Hosting.AzureAIResponses to host agent-framework AIAgents and workflows within Azure Foundry as hosted agents via the Azure.AI.AgentServer.Responses SDK. - AgentFrameworkResponseHandler: bridges ResponseHandler to AIAgent execution - InputConverter: converts Responses API inputs/history to MEAI ChatMessage - OutputConverter: converts agent response updates to SSE event stream - ServiceCollectionExtensions: DI registration helpers - 336 unit tests across net8.0/net9.0/net10.0 (112 per TFM) - ResponseStreamValidator: SSE protocol validation tool for samples - FoundryResponsesHosting sample app Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Bump System.ClientModel to 1.10.0 for Azure.Core 1.52.0 compat Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Clean up tests and sample formatting Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Update Azure.AI.AgentServer packages to 1.0.0-alpha.20260401.5 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add hosted package version suffix (0.9.0-hosted) to distinguish from mainline Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Move Foundry Responses hosting into Microsoft.Agents.AI.Foundry package Move source and test files from the standalone Hosting.AzureAIResponses project into the Foundry package under a Hosting/ subfolder. This consolidates the Foundry-specific hosting adapter into the main Foundry package. - Source: Microsoft.Agents.AI.Foundry.Hosting namespace - Tests: merged into Foundry.UnitTests/Hosting/ - Conditionally compiled for .NETCoreApp TFMs only (net8.0+) - Deleted standalone Hosting.AzureAIResponses project and test project - Updated sample and solution references Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Bump package version to 0.9.0-hosted.260402.2 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Bump OpenTelemetry packages to fix NU1109 downgrade errors - OpenTelemetry/Api/Exporter.Console/Exporter.InMemory: 1.13.1 -> 1.15.0 - OpenTelemetry.Exporter.OpenTelemetryProtocol: already 1.15.0 - OpenTelemetry.Extensions.Hosting: already 1.14.0 - OpenTelemetry.Instrumentation.AspNetCore/Http: already 1.14.0 - OpenTelemetry.Instrumentation.Runtime: 1.13.0 -> 1.14.0 - Azure.Monitor.OpenTelemetry.Exporter: 1.4.0 -> 1.5.0 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix CA1873: guard LogWarning with IsEnabled check Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix model override bug and add client REPL sample - InputConverter: stop propagating request.Model to ChatOptions.ModelId Hosted agents use their own model; client-provided model values like 'hosted-agent' were being passed through and causing server errors. - Add FoundryResponsesRepl sample: interactive CLI client that connects to a Foundry Responses endpoint using ResponsesClient.AsAIAgent() - Bump package version to 0.9.0-hosted.260403.1 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Catch agent errors and emit response.failed with real error message Previously, unhandled exceptions from agent execution would bubble up to the SDK orchestrator, which emits a generic 'An internal server error occurred.' message — hiding the actual cause (e.g., 401 auth failures, model not found, etc.). Now AgentFrameworkResponseHandler catches non-cancellation exceptions and emits a proper response.failed event containing the real error message, making it visible to clients and in logs. OperationCanceledException still propagates for proper cancellation handling by the SDK. Also bumps package version to 0.9.0-hosted.260403.2. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Renaming and merging hosting extensions. (#5091) * Rename AddAgentFrameworkHandler to AddFoundryResponses and add MapFoundryResponses - Rename extension methods: AddAgentFrameworkHandler -> AddFoundryResponses, MapAgentFrameworkHandler -> MapFoundryResponses - AddFoundryResponses now calls AddResponsesServer() internally - Add MapFoundryResponses() extension on IEndpointRouteBuilder - Update sample and tests to use new API names - Remove redundant AddResponsesServer() and /ready endpoint from sample Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fixing numbering in sample. --------- Co-authored-by: alliscode <bentho@microsoft.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Address breaking changes in 260408 * Bump hosted internal package version * Add UserAgent middleware tests for Foundry hosting * Hosting Samples update * Hosting Samples update * Hosting Samples update * Hosting Samples update * ChatClientAgent working * Adding SessionStorage and SessionManagement, improving samples to align Consumption vs Hosting * Using updates * Update chat client agent for contributor and devs * Foundry Agent Hosting * Address text rag sample working * Version bump * Adding LocalTools + Workflow samples * Removing extra using samples * Add Hosted-McpTools sample with dual MCP pattern Demonstrates two MCP integration layers in a single hosted agent: - Client-side MCP: McpClient connects to Microsoft Learn, agent handles tool invocations locally (docs_search, code_sample_search, docs_fetch) - Server-side MCP: HostedMcpServerTool delegates tool discovery and invocation to the LLM provider (Responses API), no local connection Includes DevTemporaryTokenCredential for Docker local debugging, Dockerfile.contributor for ProjectReference builds, and the openai/v1 route mapping for AIProjectClient compatibility in Development mode. * Bump Azure.AI.AgentServer packages to 1.0.0-beta.1/beta.21 and fix breaking API changes - Azure.AI.AgentServer.Core: 1.0.0-beta.11 -> 1.0.0-beta.21 - Azure.AI.AgentServer.Invocations: 1.0.0-alpha.20260408.4 -> 1.0.0-beta.1 - Azure.AI.AgentServer.Responses: 1.0.0-alpha.20260408.4 -> 1.0.0-beta.1 - Azure.Identity: 1.20.0 -> 1.21.0 (transitive requirement) - Azure.Core: 1.52.0 -> 1.53.0 (transitive requirement) - Remove azure-sdk-for-net dev feed (packages now on nuget.org) - Fix OutputConverter for new builder API (auto-tracked children, split EmitTextDone/EmitDone) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fixing small issues. * Fix IDE0009: add 'this' qualification in DevTemporaryTokenCredential Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix IDE0009: add 'this' qualification in all HostedAgentsV2 samples Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix CHARSET: add UTF-8 BOM to Hosted-LocalTools and Hosted-Workflows Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix dotnet format: add Async suffix to test methods (IDE1006), fix encoding and style Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Register AgentSessionStore in test DI setups Add InMemoryAgentSessionStore registration to all ServiceCollection setups in AgentFrameworkResponseHandlerTests and WorkflowIntegrationTests. This is needed after the AgentSessionStore infrastructure was introduced in the responses-hosting feature. Tests still have NotImplementedException stubs for CreateSessionCoreAsync which will be fixed when the session infrastructure is fully available. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add Invocations protocol samples (hosted echo agent + client) (#5278) Add Hosted-Invocations-EchoAgent: a minimal echo agent hosted via the Invocations protocol (POST /invocations) using AddInvocationsServer and MapInvocationsServer, bridged to an Agent Framework AIAgent through a custom InvocationHandler. Add SimpleInvocationsAgent: a console REPL client that wraps HttpClient calls to the /invocations endpoint in a custom InvocationsAIAgent, demonstrating programmatic consumption of the Invocations protocol. Both samples default to port 8088 for consistency with other hosted agent samples. * Restructure FoundryHostedAgents samples into invocations/ and responses/ Align dotnet hosted agent samples with the Python side (PR #5281) by reorganizing the directory structure: - Remove HostedAgentsV1 entirely (old API pattern) - Split HostedAgentsV2 into invocations/ and responses/ based on protocol - Move Using-Samples accordingly (SimpleAgent to responses, SimpleInvocationsAgent to invocations) - Update slnx with new project paths and add previously missing invocations projects - Update README cd paths from HostedAgentsV2 to invocations or responses - Rename .env.local to .env.example to match Python naming convention - Fix format violations in newly included invocations projects * Remove launchSettings, use .env for port configuration - Delete all launchSettings.json files (port 8088 now comes from ASPNETCORE_URLS in .env) - Add DotNetEnv to Hosted-Invocations-EchoAgent so it loads .env like the responses samples - Create .env.example for EchoAgent with ASPNETCORE_URLS and ASPNETCORE_ENVIRONMENT - Add AGENT_NAME to ChatClientAgent and FoundryAgent .env.example (required by those samples) - Add AZURE_BEARER_TOKEN=DefaultAzureCredential to all .env.example files - Update DevTemporaryTokenCredential in all 6 samples to treat the sentinel value as unavailable, allowing ChainedTokenCredential to fall through to DefaultAzureCredential - Update EchoAgent README with Configuration section * Use placeholder for AGENT_NAME in Hosted-FoundryAgent .env.example * Move FoundryResponsesHosting to responses/Hosted-WorkflowHandoff, use GetResponsesClient * Rename Hosted-Workflows to Hosted-Workflow-Simple, Hosted-WorkflowHandoff to Hosted-Workflow-Handoff * Remove FoundryResponsesRepl and empty FoundryResponsesHosting directory * Add Dockerfiles, README, agent yamls and bearer token support to Hosted-Workflow-Handoff - Add Dockerfile and Dockerfile.contributor for Docker-based testing - Add agent.yaml and agent.manifest.yaml with triage-workflow as primary agent - Add README.md following sibling pattern, noting Azure OpenAI vs Foundry endpoint - Add DevTemporaryTokenCredential and ChainedTokenCredential for Docker auth - Register triage-workflow as non-keyed default so azd invoke works without model - Update .env.example with AZURE_BEARER_TOKEN sentinel - Add .gitignore to 04-hosting to suppress VS-generated launchSettings.json - Fix docker run image name in Hosted-Workflow-Simple README * Fix AgentFrameworkResponseHandlerTests: implement session methods in test mock agents * .NET: Auto-instrument resolved AIAgents with OpenTelemetry for Foundry Hosted Agents (#5316) * Auto-instrument resolved AIAgents with OpenTelemetry using Core ResponsesSourceName * Add OTel telemetry capture tests for Foundry hosted agent handler * Net: Prepare Foundry Preview Release (#5336) * Prepare Foundry preview release 1.2.0-preview.* Bump VersionPrefix to 1.2.0 and update the preview stamp date. Invert packaging opt-in so only the Foundry preview set produces NuGet packages: - Microsoft.Agents.AI.Abstractions - Microsoft.Agents.AI - Microsoft.Agents.AI.Workflows - Microsoft.Agents.AI.Workflows.Generators - Microsoft.Agents.AI.Foundry Flip IsReleased=false on the preview set so they pick up the -preview.YYMMDD.N suffix. Gate GeneratePackageOnBuild on IsPackable=true. Remove the global IsPackable=true from nuget-package.props so the repo-level default (false) applies to everything else. * Lower preview VersionPrefix to 0.0.1 Retroactive preview publish: bump VersionPrefix and GitTag from 1.2.0 to 0.0.1 so the 5 Foundry preview packages emit as 0.0.1-preview.260417.1. * Net: Publish all packages as 0.0.1-preview.260417.2 (#5341) Revises the Foundry pre-release approach to publish ALL normally packable src projects as preview packages stamped 0.0.1-preview.260417.2, including projects previously flagged IsReleased=true or with a non-default VersionSuffix (rc/alpha). nuget-package.props: - Collapse the four conditional PackageVersion expressions (IsReleaseCandidate, VersionSuffix, default preview, IsReleased stable) into a single unconditional 0.0.1-preview.260417.2. On this preview-only branch every package ships with the same pre-release stamp regardless of per-project flags. - Restore the global IsPackable=true default (offsetting the repo-wide IsPackable=false in Directory.Build.props). Projects that opt out (Mem0, Declarative) already set IsPackable=false AFTER importing this file so they remain non-packable. - Remove the IsReleased-gated EnablePackageValidation line. Package validation does not apply to a 0.0.1 preview. csproj reverts (Abstractions, Agents.AI, Workflows, Workflows.Generators, Foundry): - Revert the IsPackable=true opt-in block introduced in #5336 (now redundant since the props default is true again). - Restore IsReleased=true to its pre-PR value. The setting is now a no-op because the props no longer branches on it. * Bump preview version to 260420.1 and fix AgentServer package deps (#5367) - Bump PackageVersion to 0.0.1-preview.260420.1 - Bump Azure.AI.AgentServer.Core beta.21 -> beta.22 (required by Azure.AI.AgentServer.Responses beta.3) - Replace AgentHostTelemetry.ResponsesSourceName with local constant (type made internal in AgentServer.Core beta.22) Co-authored-by: alliscode <bentho@microsoft.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * .NET: Hosted agents toolbox support (#5368) * feat: Add Foundry Toolbox (MCP) support to AgentFrameworkResponseHandler Adds support for Foundry Toolsets MCP proxy integration in the hosted agent response handler. Toolsets connect at startup via IHostedService, gating the readiness probe per spec §3.1. MCP tools are injected into every request's ChatOptions and OAuth consent errors (-32006) are intercepted and surfaced as mcp_approval_request + incomplete SSE events. New files: - FoundryToolboxOptions.cs: configuration POCO for toolset names and API version - FoundryToolboxBearerTokenHandler.cs: DelegatingHandler with Azure Bearer token auth, Foundry-Features header injection, and 3x exponential backoff on 429/5xx - McpConsentContext.cs: AsyncLocal-based per-request consent state shared between the tool wrapper and the response handler - ConsentAwareMcpClientTool.cs: AIFunction wrapper that catches -32006 errors and signals consent via shared state and linked CancellationTokenSource - FoundryToolboxService.cs: IHostedService that creates McpClient per toolset at startup and exposes cached tools Modified files: - AgentFrameworkResponseHandler.cs: injects toolbox tools into ChatOptions, sets up linked CTS consent interception, emits mcp_approval_request on -32006 - ServiceCollectionExtensions.cs: adds AddFoundryToolboxes(params string[]) extension - Microsoft.Agents.AI.Foundry.csproj: adds ModelContextProtocol and Azure.Identity dependencies under NETCoreApp condition Sample: - Hosted-Toolbox: minimal hosted agent sample using AddFoundryToolboxes * Rename toolset to toolbox in user-facing API; rename ConsentAwareMcpClientTool to ConsentAwareMcpClientAIFunction * Add HostedMcpToolboxAITool for client-selectable Foundry toolboxes Introduces HostedMcpToolboxAITool, a marker tool subclassing HostedMcpServerTool that rides the OpenAI Responses 'mcp' wire format to let clients request a specific Foundry toolbox per request. - New FoundryAITool.CreateHostedMcpToolbox(name, version?) factory. - FoundryToolboxOptions.StrictMode (default true) rejects unregistered toolboxes; set to false to allow lazy-open on first use. - FoundryToolboxService.GetToolboxToolsAsync(name, version?) resolves cached or lazy-opened MCP tools. - AgentFrameworkResponseHandler parses request.Tools for foundry-toolbox://name[?version=v] markers and injects resolved tools per request, merging with pre-registered ones. - Unit tests for marker parsing and strict-mode resolution. * Bump Azure.AI.Projects to 2.1.0-alpha; add ToolboxRecord/ToolboxVersion factory overloads + tests * Fix PR review issues: retry off-by-one, URI encoding, docs, tests, build - Fix off-by-one in FoundryToolboxBearerTokenHandler retry loop (4 attempts → 3) - URI-encode version parameter in HostedMcpToolboxAITool.BuildAddress - Add XML doc clarifying version pinning is reserved for future use - Add comment clarifying AddHostedService deduplication safety - Fix DevTemporaryTokenCredential expiry to use DateTimeOffset.MaxValue - Fix AgentCard ambiguity in A2AServer sample with using alias - Add 18 new unit tests for retry handler and ReadMcpToolboxMarkers Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Roger Barreto <19890735+rogerbarreto@users.noreply.github.com> Co-authored-by: alliscode <bentho@microsoft.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * .NET: Hosted agent adapter (#5371) * Bump preview version to 260420.1 and fix AgentServer package deps - Bump PackageVersion to 0.0.1-preview.260420.1 - Bump Azure.AI.AgentServer.Core beta.21 -> beta.22 (required by Azure.AI.AgentServer.Responses beta.3) - Replace AgentHostTelemetry.ResponsesSourceName with local constant (type made internal in AgentServer.Core beta.22) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix CA1873: guard LogError with IsEnabled check in FoundryToolboxService Wrap the LogError call with an IsEnabled(LogLevel.Error) guard to satisfy the CA1873 analyzer rule which flags potentially expensive argument evaluation when logging is disabled. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: alliscode <bentho@microsoft.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * .NET: Hosted agent adapter (#5374) * Bump preview version to 260420.1 and fix AgentServer package deps - Bump PackageVersion to 0.0.1-preview.260420.1 - Bump Azure.AI.AgentServer.Core beta.21 -> beta.22 (required by Azure.AI.AgentServer.Responses beta.3) - Replace AgentHostTelemetry.ResponsesSourceName with local constant (type made internal in AgentServer.Core beta.22) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix CA1873: guard LogError with IsEnabled check in FoundryToolboxService Wrap the LogError call with an IsEnabled(LogLevel.Error) guard to satisfy the CA1873 analyzer rule which flags potentially expensive argument evaluation when logging is disabled. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Bumping NuGet version --------- Co-authored-by: alliscode <bentho@microsoft.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * .NET: Hosted agent adapter (#5406) * Bump preview version to 260420.1 and fix AgentServer package deps - Bump PackageVersion to 0.0.1-preview.260420.1 - Bump Azure.AI.AgentServer.Core beta.21 -> beta.22 (required by Azure.AI.AgentServer.Responses beta.3) - Replace AgentHostTelemetry.ResponsesSourceName with local constant (type made internal in AgentServer.Core beta.22) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix CA1873: guard LogError with IsEnabled check in FoundryToolboxService Wrap the LogError call with an IsEnabled(LogLevel.Error) guard to satisfy the CA1873 analyzer rule which flags potentially expensive argument evaluation when logging is disabled. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Bumping NuGet version * Restore conditional versioning, remove dev feed, bump Azure.AI.Projects to beta.1 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: alliscode <bentho@microsoft.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Hosted agent adapter (#5408) * Bump preview version to 260420.1 and fix AgentServer package deps - Bump PackageVersion to 0.0.1-preview.260420.1 - Bump Azure.AI.AgentServer.Core beta.21 -> beta.22 (required by Azure.AI.AgentServer.Responses beta.3) - Replace AgentHostTelemetry.ResponsesSourceName with local constant (type made internal in AgentServer.Core beta.22) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix CA1873: guard LogError with IsEnabled check in FoundryToolboxService Wrap the LogError call with an IsEnabled(LogLevel.Error) guard to satisfy the CA1873 analyzer rule which flags potentially expensive argument evaluation when logging is disabled. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Bumping NuGet version * Restore conditional versioning, remove dev feed, bump Azure.AI.Projects to beta.1 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Address PR #5312 review comments - Add comment explaining NU1903 suppression (Microsoft.Bcl.Memory transitive vuln) - Remove NU1903 from sample/test projects where not needed - Fix Dockerfile ENTRYPOINT mismatch in Hosted-Workflow-Simple - Align agent name to 'hosted-workflow-simple' in agent.yaml and README - Fix Hosted-McpTools README: replace GitHub PAT refs with Microsoft Learn - Fix session persistence: only persist when client provides conversation ID - Upgrade IsNullOrEmpty to IsNullOrWhiteSpace for session ID checks Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: alliscode <bentho@microsoft.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Split Foundry into stable V1 and preview Hosting package Extract hosted agent functionality from Microsoft.Agents.AI.Foundry into a new Microsoft.Agents.AI.Foundry.Hosting preview package. This resolves NU5104 build errors caused by the stable Foundry package depending on prerelease Azure SDK packages (Azure.AI.AgentServer.Responses, Azure.AI.Projects beta). Changes: - Create Microsoft.Agents.AI.Foundry.Hosting with VersionSuffix=preview, targeting .NET Core only (net8.0/9.0/10.0) - Move all Hosting/ source files to the new project - Move ToolboxRecord/ToolboxVersion overloads to FoundryAIToolExtensions - Revert Azure.AI.Projects to 2.0.0 in Directory.Packages.props; Hosting uses VersionOverride for 2.1.0-beta.1 - Clean V1 Foundry csproj: remove beta deps, ASP.NET Core ref, hosting conditionals - Update 8 hosted agent sample projects to reference Foundry.Hosting - Split unit tests: ToolboxRecord/ToolboxVersion tests moved to Hosting/ - Add Foundry.Hosting to solution file Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Address PR review comments: experimental attrs, doc fixes, token propagation - Add [Experimental(OPENAI001)] to all 7 public Hosting types per reviewer request - Fix McpConsentContext XML doc: 'Thread-static' -> 'Async-local' (AsyncLocal flows with ExecutionContext, not thread-static) - Expand UserAgentMiddleware test regex to match prerelease versions (e.g. 1.0.0-rc.4) - Propagate CancellationToken in AgentFrameworkResponseHandler session save Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Remove unnecessary MEAI001 suppression from stable Foundry package MEAI001 was a leftover from when Hosting code lived in the same project. The stable V1 Foundry package builds clean without it, and suppressing experimental diagnostics in a released package can hide unintentional exposure of experimental APIs to consumers. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add Foundry.Hosting to release solution filter Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: alliscode <bentho@microsoft.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Ben Thomas <ben.thomas@microsoft.com> |
||
|
|
aee1acbf8b |
.NET: Foundry Evals integration for .NET (#4914)
* Foundry Evals integration for .NET - Core evaluation framework: EvalItem, LocalEvaluator, FunctionEvaluator, EvalChecks - IAgentEvaluator interface with MeaiEvaluatorAdapter bridge - AgentEvaluationExtensions for agent.EvaluateAsync() overloads - FoundryEvals wrapping MEAI quality/safety evaluators - ConversationSplitters (LastTurn, Full) and IConversationSplitter - EvalItem.PerTurnItems() for multi-turn decomposition - HasImageContent for multimodal content detection - WorkflowEvaluationExtensions for per-agent workflow evaluation - 7 eval samples mirroring Python parity: 02-agents/Evaluation: SimpleEval, ExpectedOutputs, Multimodal 03-workflows/Evaluation: WorkflowEval 05-end-to-end/Evaluation: FoundryQuality, MixedProviders, ConversationSplits - Comprehensive unit tests (1958 passing) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Rewrite FoundryEvals to use real Foundry Evals API Replace MEAI evaluator shim with actual OpenAI EvaluationClient protocol methods. FoundryEvals now creates eval definitions, submits runs, polls for completion, and fetches per-item results server-side. - New constructor: FoundryEvals(AIProjectClient, model, evaluators) - Add FoundryEvalConverter for MEAI ChatMessage -> Foundry JSON format - Add EvalId, RunId, ReportUrl to AgentEvaluationResults - All 20 built-in evaluator constants now work (agent, tool, quality, safety) - Remove Microsoft.Extensions.AI.Evaluation.Quality/Safety dependencies - Update all samples for new constructor (no more ChatConfiguration) - Replace BuildEvaluators tests with ResolveEvaluator tests Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add response output to CustomEvals and ExpectedOutputs samples Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Address review: pagination, validation, error handling, tests FoundryEvals fixes: - Add pagination for output items (has_more/after cursor) - Add guard clauses for pollIntervalSeconds/timeoutSeconds <= 0 - Fix double TryGetProperty for passed field parsing - Throw on all-tool-evaluators with no tool definitions - Fix XML doc (default 300s, not 180s) New tests (30 added, 1989 total): - EvalChecks: NonEmpty, ContainsExpected (pass/fail/skip/case), HasImageContent, ToolCallsPresent - FoundryEvalConverter: ConvertMessage (text, image, function call, function results fan-out, empty fallback, mixed content), ConvertEvalItem, BuildTestingCriteria (quality/agent/tool/groundedness data mappings), BuildItemSchema Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix review: null-refs, Data.ToString() bug, ContainsExpected, add tests - Fix NullReferenceException in sample Response display (pattern matching) - Fix WorkflowEvaluationExtensions Data?.ToString() producing type names instead of message text (pattern-match ChatMessage/AgentResponse/list) - Change EvalChecks.ContainsExpected to return Passed=false when no ExpectedOutput (was silently passing, masking misconfiguration) - Add EvalItem constructor tests with LastTurn/Full/null splitters - Add FoundryEvalConverter.ConvertMessage DataContent (base64 image) test - Add ExtractAgentData tests with ChatMessage, list, and AgentResponse data Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix review: conversation fidelity, eval caching, fallback tests - WorkflowEvaluationExtensions: preserve full response messages (tool calls, intermediate) instead of synthetic 2-message conversation. Cast completed Data to AgentResponse and use Messages when available, fallback to text. - FoundryEvals: cache evalId per schema shape (hasContext, hasTools) so subsequent EvaluateAsync calls create runs under the same eval definition. - MeaiEvaluatorAdapter: code already correctly passes queryMessages (not full conversation) to IEvaluator — no change needed, verified by inspection. - Add tests: AgentResponse full messages preservation, unknown object ToString() fallback for ExtractAgentData. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Rename AzureAI→Foundry: move eval files, update references - Move FoundryEvals.cs and FoundryEvalConverter.cs from Microsoft.Agents.AI.AzureAI to Microsoft.Agents.AI.Foundry - Update namespace from AzureAI to Foundry in both files - Add explicit usings required by Foundry project (no implicit usings) - Move FoundryEvalConverter tests to Foundry.UnitTests project (avoids ReplacingRedactor type conflict from dual project refs) - Update all sample csproj references and using statements - Remove Foundry project reference from AI UnitTests Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * PR review round 4: wire up tool extraction, remove eval cache, fix null safety - BuildEvalItem: extract tools from agent via GetService<ChatOptions>() into EvalItem.Tools (Python parity) - FoundryEvals: remove eval ID cache - each call creates fresh definition (matches Python behavior) - FoundryEvals: replace null-forgiving operators with descriptive InvalidOperationException - MixedProviders sample: remove unnecessary explicit PackageReferences (transitively provided) - FoundryEvalConverter: document that tool results take precedence over text content - Add LocalEvaluator zero-checks test documenting 0 metrics = failed behavior Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Python-dotnet parity: 9 feature gaps filled New checks: - ToolCallArgsMatch() — verify tool call names + argument subset match - ToolCalledCheck(ToolCalledMode.Any, ...) — match any of the specified tools - ToolCalledMode enum (All/Any) FoundryEvals enhancements: - Default evaluators now [Relevance, Coherence, TaskAdherence] (was Relevance, Coherence) - Auto-add ToolCallAccuracy when items have tool definitions - EvaluateTracesAsync — evaluate by response_ids, trace_ids, or agent_id - EvaluateFoundryTargetAsync — evaluate deployed Foundry targets Result type enrichment: - AgentEvaluationResults: added Status, Error, PerEvaluator, DetailedItems - New EvalItemResult/EvalScoreResult/PerEvaluatorResult types - FoundryEvals populates all new fields from API responses Workflow fix: - Skip internal executors (_*, input-conversation, end-conversation, end) Tests: 8 new tests covering ToolCallArgsMatch, ToolCalledMode.Any, internal executor filtering Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add MeaiEvaluatorAdapter and PerTurnItems edge case tests - 3 tests for MeaiEvaluatorAdapter: query message forwarding, synthetic response fallback, multiple items aggregation - 3 tests for EvalItem.PerTurnItems: empty conversation, no user messages, system+assistant only - StubEvaluator and StubChatClient test helpers Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Blocking link check for outdated package in DevUI. * Replace Dictionary<string, object> payloads with typed wire models Introduce internal FoundryEvalWireModels.cs with compile-time-safe types for the OpenAI Evals API wire format. The OpenAI .NET SDK (2.9.1) only provides protocol-level methods with BinaryContent/ClientResult — no typed request models. These internal models replace scattered dictionary literals with [JsonPropertyName]-annotated classes, giving: - Compile-time safety (typos become build errors) - Single point of change when the API evolves - IntelliSense discoverability - Cleaner serialization via JsonPolymorphic for content items Models: WireContentItem hierarchy (text, image, tool_call, tool_result), WireMessage, WireEvalItemPayload, WireTestingCriterion, WireItemSchema, WireCreateEvalRequest, WireCreateRunRequest, and data source variants. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Skip metric when Foundry returns neither score nor passed When an evaluator returns no score and no passed value, the previous code created BooleanMetric(name, false), which falsely failed items via ItemPassed. Now we skip the MEAI metric entirely for indeterminate results — the raw data remains available in DetailedItems for diagnostics. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Address PR #4914 review comments: fix tool evaluator bug and add tests - Fix duplicate ToolCallAccuracy: resolve evaluator names before checking against ToolEvaluators set (Comment 2) - Make FilterToolEvaluators internal for testability; add tests for the ArgumentException edge case when all evaluators are tool-type (Comment 3) - Add CancellationToken test for LocalEvaluator (Comment 4) - Add EvaluateAsync integration test on Run with sequential workflow and per-agent SubResults verification (Comment 5) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Address Peter's review comments on PR #4914 - Add trailing newline to Evaluation_FoundryQuality.csproj (Comment 6) - Make evaluator name lookups case-insensitive: switch BuiltinEvaluators, ToolEvaluators, AgentEvaluators, and ResolveEvaluator's StartsWith check from Ordinal to OrdinalIgnoreCase (Comment 7) - Add Trace.TraceWarning when Foundry returns fewer results than submitted items, indicating expected vs actual count before padding (Comment 8) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add Microsoft.Extensions.AI.Evaluation packages to Directory.Packages.props These were removed in #5269 as unused, but are needed by the Foundry and core evaluation integration added in this PR. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: alliscode <bentho@microsoft.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |
||
|
Roger BarretoandGitHub
|
4134c74060 |
Add CreateSessionAsync(conversationId) to FoundryAgent (#5144)
Adds a public CreateSessionAsync(string conversationId, CancellationToken) method to FoundryAgent that delegates to the inner ChatClientAgent, allowing users to create sessions with existing server-side conversation IDs. Fixes #5138 |
||
|
Roger BarretoandGitHub
|
b0613a8ceb |
.NET: Bump Azure.AI.Projects to 2.0.0 GA (#5060)
* Bump Azure.AI.Projects to 2.0.0 GA - Update Azure.AI.Projects from 2.0.0-beta.2 to 2.0.0 in CPM - Update Azure.Identity from 1.19.0 to 1.20.0 (transitive dep) - Update System.ClientModel from 1.9.0 to 1.10.0 (transitive dep) - Rename types per Azure.AI.Projects.Agents 2.0.0 breaking changes: - AgentVersion -> ProjectsAgentVersion - AgentRecord -> ProjectsAgentRecord - AgentDefinition -> ProjectsAgentDefinition - AgentVersionCreationOptions -> ProjectsAgentVersionCreationOptions - PromptAgentDefinition -> DeclarativeAgentDefinition - AgentTool -> ProjectsAgentTool - AgentsClient -> AgentAdministrationClient - .Agents property -> .AgentAdministrationClient - Add using Azure.AI.Projects.Memory namespace (types moved) - Update AGENTS.md with BOM and output capture conventions * Address PR review feedback - Rename AIProjectClient parameter to aiProjectClient in AsChatClientAgent overloads - Fix XML doc: ProjectsAgentTool namespace from Azure.AI.Projects.OpenAI to Azure.AI.Projects.Agents - Rename test method to reflect DeclarativeAgentDefinition terminology |
||
|
Roger BarretoandGitHub
|
628bb1af48 |
.NET: Rename Microsoft.Agents.AI.AzureAI to Microsoft.Agents.AI.Foundry and consolidate FoundryMemory (#5042)
* Update Foundry Responses as ChatClientAgent * Migrate obsolete AzureAI integration tests to versioned agent pattern Replace obsolete CreateAIAgentAsync/GetAIAgentAsync calls with Agents.CreateAgentVersionAsync() + AsAIAgent(AgentVersion) in all AzureAI integration tests. - Rename AIProjectClient* test files to FoundryVersionedAgent* - Register AIFunction tools in PromptAgentDefinition.Tools for server-side visibility via AsOpenAIResponseTool() - Skip structured output tests (AzureAIProjectChatClient clears ResponseFormat for versioned agents) - Remove all [Obsolete] attributes and #pragma warning disable CS0618 * Merge FoundryMemory package into AzureAI under Memory/ folder Move all FoundryMemory source, unit tests, and integration tests into the Microsoft.Agents.AI.AzureAI package. Change namespace from Microsoft.Agents.AI.FoundryMemory to Microsoft.Agents.AI.AzureAI. - Add [Experimental] to FoundryMemoryProviderOptions and Scope - Rename internal AIProjectClientExtensions to MemoryStoreExtensions - Update AzureAI .csproj with Compliance.Abstractions, Redaction - Remove FoundryMemory from solution and release filter - Update sample to reference AzureAI instead of FoundryMemory - Delete old Microsoft.Agents.AI.FoundryMemory project and tests * Add EnsureMemoryStoreCreatedAsync and memory existence checks to integration tests - Ensure memory store is created before testing memory operations - Add AZURE_AI_EMBEDDING_DEPLOYMENT_NAME config setting - Assert memories exist in store via SearchMemoriesAsync before cleanup - Verify scope isolation with direct memory store queries * Fix and rename AzureAI unit tests for RAPI vs Versioned clarity - Rename AsAIAgentAsync_* to AsAIAgent_* (drop Async from method group) - Add _Rapi_ prefix to non-versioned (Responses API) tests - Add _Versioned_ prefix to versioned agent tests where needed - Fix RAPI tests: assert GetService<AIProjectClient>() is null - Fix Versioned tests: assert IsType<FoundryAgent> and GetService<AIProjectClient>() returns the client instance - Fix UserAgent header tests: proper HTTP handler routing - Fix ChatClient_UsesDefaultConversationIdAsync test setup - All 153 unit tests pass with 0 failures * Rename Microsoft.Agents.AI.AzureAI to Microsoft.Agents.AI.Foundry Rename the project, namespace, folder, and all references from Microsoft.Agents.AI.AzureAI to Microsoft.Agents.AI.Foundry. Also rename Workflows.Declarative.AzureAI to .Foundry. - Rename src, unit test, integration test, and workflow folders - Update namespaces in all source and test .cs files - Update ProjectReferences in ~47 sample and test .csproj files - Update solution files (.slnx, .slnf) - Update sample using statements - Update READMEs, SKILL.md, ADRs in docs/ - Disable package validation baseline for renamed packages - Fix UTF-8 BOM encoding on all affected .cs files - AzureAI.Persistent left completely unchanged * Fix format: remove ImplicitUsings, add explicit usings, fix BOM encoding - Remove ImplicitUsings=enable from Foundry csproj to resolve IDE0005 on shared ReplacingRedactor.cs - Add explicit System usings to all source files that relied on them - Sort usings alphabetically per editorconfig rules - Fix UTF-8 BOM on 12 sample Program.cs files - Rename Azure AI Foundry Agents to Microsoft Foundry Agents in docs |