* adds devui integration and samples
* adds unit tests for devui integration
* fix: correct formatting of copyright notice in unit test files
* fixes formatting issues
* fixes build for net8 target
* fixes formatting errors on test apphost
* adds copyright notice to multiple files and removes unnecessary using directives
* Update dotnet/aspire-integration/Aspire.Hosting.AgentFramework.DevUI/DevUIAggregatorHostedService.cs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update dotnet/aspire-integration/Aspire.Hosting.AgentFramework.DevUI/DevUIAggregatorHostedService.cs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update dotnet/tests/Aspire.Hosting.AgentFramework.DevUI.UnitTests/Aspire.Hosting.AgentFramework.DevUI.UnitTests.csproj
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update dotnet/samples/DevUIIntegration/DevUIIntegration.AppHost/DevUIIntegration.AppHost.csproj
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update dotnet/aspire-integration/Aspire.Hosting.AgentFramework.DevUI/DevUIAggregatorHostedService.cs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Refactor project files to use TargetFrameworks instead of TargetFramework for multi-targeting support; add optional port property to DevUIResource class.
* Add unit tests for DevUIAggregatorHostedService; refactor project files for TargetFrameworks support
* Refactor project files to use TargetFrameworks for multi-targeting support in DevUIIntegration samples
* Remove unnecessary using directive for Aspire.Hosting in DevUIAggregatorHostedServiceTests
* merge
* fixes Conversation routing for non-first backends
* add documentation for devui integration sample
* update project references in solution file for improved integration
* fixes package versions post merge
* move Aspire.Hosting.AgentFramework.DevUI to dotnet/src
Move the project from aspire-integration/ to src/ to be consistent
with the location of all other projects in the repo.
* move DevUI sample to samples/05-end-to-end/DevUIAspireIntegration
Move the sample from samples/DevUIIntegration/ to
samples/05-end-to-end/DevUIAspireIntegration/ to match the location
of other end-to-end samples.
* remove unnecessary net472 framework condition from sample csproj files
These projects only target net10.0, so the
Condition="'$(TargetFramework)' != 'net472'" on ItemGroup is unnecessary.
* update sample model name from gpt-4.1 to gpt-5.4
Use a more up-to-date model name in the DevUI integration samples.
* Revert "remove unnecessary net472 framework condition from sample csproj files"
This reverts commit 08cf41253b.
* fix: use TargetFrameworks to override multi-targeting from Directory.Build.props
The parent Directory.Build.props sets TargetFrameworks to net10.0;net472,
which overrides the singular TargetFramework in each csproj. Use the plural
TargetFrameworks property set to net10.0 only to properly override it, and
remove the now-unnecessary net472 condition on ItemGroup.
* fixes aspire config
* fix: update Microsoft.Extensions packages to version 10.0.1
* Address Copilot review feedback on DevUI Aspire integration
- Fix request body dropping in ProxyConversationsAsync: always read the
body when ContentLength > 0 before routing, then pass it through to
all proxy calls (previously null was passed when backend was resolved
from query param or conversation map)
- Fix resource leak: dispose aggregator on startup failure in catch block
- Fix XML docs: accurately describe embedded resource serving behavior
- Remove reflection from DevUIResourceTests (InternalsVisibleTo already set)
- Make sensitive telemetry conditional on Development environment in samples
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: update chat client version to gpt41 in both EditorAgent and WriterAgent
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Roger Barreto <19890735+rogerbarreto@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Foundry Evals integration for .NET
- Core evaluation framework: EvalItem, LocalEvaluator, FunctionEvaluator, EvalChecks
- IAgentEvaluator interface with MeaiEvaluatorAdapter bridge
- AgentEvaluationExtensions for agent.EvaluateAsync() overloads
- FoundryEvals wrapping MEAI quality/safety evaluators
- ConversationSplitters (LastTurn, Full) and IConversationSplitter
- EvalItem.PerTurnItems() for multi-turn decomposition
- HasImageContent for multimodal content detection
- WorkflowEvaluationExtensions for per-agent workflow evaluation
- 7 eval samples mirroring Python parity:
02-agents/Evaluation: SimpleEval, ExpectedOutputs, Multimodal
03-workflows/Evaluation: WorkflowEval
05-end-to-end/Evaluation: FoundryQuality, MixedProviders, ConversationSplits
- Comprehensive unit tests (1958 passing)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Rewrite FoundryEvals to use real Foundry Evals API
Replace MEAI evaluator shim with actual OpenAI EvaluationClient protocol
methods. FoundryEvals now creates eval definitions, submits runs, polls
for completion, and fetches per-item results server-side.
- New constructor: FoundryEvals(AIProjectClient, model, evaluators)
- Add FoundryEvalConverter for MEAI ChatMessage -> Foundry JSON format
- Add EvalId, RunId, ReportUrl to AgentEvaluationResults
- All 20 built-in evaluator constants now work (agent, tool, quality, safety)
- Remove Microsoft.Extensions.AI.Evaluation.Quality/Safety dependencies
- Update all samples for new constructor (no more ChatConfiguration)
- Replace BuildEvaluators tests with ResolveEvaluator tests
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add response output to CustomEvals and ExpectedOutputs samples
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address review: pagination, validation, error handling, tests
FoundryEvals fixes:
- Add pagination for output items (has_more/after cursor)
- Add guard clauses for pollIntervalSeconds/timeoutSeconds <= 0
- Fix double TryGetProperty for passed field parsing
- Throw on all-tool-evaluators with no tool definitions
- Fix XML doc (default 300s, not 180s)
New tests (30 added, 1989 total):
- EvalChecks: NonEmpty, ContainsExpected (pass/fail/skip/case),
HasImageContent, ToolCallsPresent
- FoundryEvalConverter: ConvertMessage (text, image, function call,
function results fan-out, empty fallback, mixed content),
ConvertEvalItem, BuildTestingCriteria (quality/agent/tool/groundedness
data mappings), BuildItemSchema
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix review: null-refs, Data.ToString() bug, ContainsExpected, add tests
- Fix NullReferenceException in sample Response display (pattern matching)
- Fix WorkflowEvaluationExtensions Data?.ToString() producing type names
instead of message text (pattern-match ChatMessage/AgentResponse/list)
- Change EvalChecks.ContainsExpected to return Passed=false when no
ExpectedOutput (was silently passing, masking misconfiguration)
- Add EvalItem constructor tests with LastTurn/Full/null splitters
- Add FoundryEvalConverter.ConvertMessage DataContent (base64 image) test
- Add ExtractAgentData tests with ChatMessage, list, and AgentResponse data
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix review: conversation fidelity, eval caching, fallback tests
- WorkflowEvaluationExtensions: preserve full response messages (tool calls,
intermediate) instead of synthetic 2-message conversation. Cast completed
Data to AgentResponse and use Messages when available, fallback to text.
- FoundryEvals: cache evalId per schema shape (hasContext, hasTools) so
subsequent EvaluateAsync calls create runs under the same eval definition.
- MeaiEvaluatorAdapter: code already correctly passes queryMessages (not full
conversation) to IEvaluator — no change needed, verified by inspection.
- Add tests: AgentResponse full messages preservation, unknown object
ToString() fallback for ExtractAgentData.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Rename AzureAI→Foundry: move eval files, update references
- Move FoundryEvals.cs and FoundryEvalConverter.cs from
Microsoft.Agents.AI.AzureAI to Microsoft.Agents.AI.Foundry
- Update namespace from AzureAI to Foundry in both files
- Add explicit usings required by Foundry project (no implicit usings)
- Move FoundryEvalConverter tests to Foundry.UnitTests project
(avoids ReplacingRedactor type conflict from dual project refs)
- Update all sample csproj references and using statements
- Remove Foundry project reference from AI UnitTests
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* PR review round 4: wire up tool extraction, remove eval cache, fix null safety
- BuildEvalItem: extract tools from agent via GetService<ChatOptions>() into EvalItem.Tools (Python parity)
- FoundryEvals: remove eval ID cache - each call creates fresh definition (matches Python behavior)
- FoundryEvals: replace null-forgiving operators with descriptive InvalidOperationException
- MixedProviders sample: remove unnecessary explicit PackageReferences (transitively provided)
- FoundryEvalConverter: document that tool results take precedence over text content
- Add LocalEvaluator zero-checks test documenting 0 metrics = failed behavior
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python-dotnet parity: 9 feature gaps filled
New checks:
- ToolCallArgsMatch() — verify tool call names + argument subset match
- ToolCalledCheck(ToolCalledMode.Any, ...) — match any of the specified tools
- ToolCalledMode enum (All/Any)
FoundryEvals enhancements:
- Default evaluators now [Relevance, Coherence, TaskAdherence] (was Relevance, Coherence)
- Auto-add ToolCallAccuracy when items have tool definitions
- EvaluateTracesAsync — evaluate by response_ids, trace_ids, or agent_id
- EvaluateFoundryTargetAsync — evaluate deployed Foundry targets
Result type enrichment:
- AgentEvaluationResults: added Status, Error, PerEvaluator, DetailedItems
- New EvalItemResult/EvalScoreResult/PerEvaluatorResult types
- FoundryEvals populates all new fields from API responses
Workflow fix:
- Skip internal executors (_*, input-conversation, end-conversation, end)
Tests: 8 new tests covering ToolCallArgsMatch, ToolCalledMode.Any, internal executor filtering
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add MeaiEvaluatorAdapter and PerTurnItems edge case tests
- 3 tests for MeaiEvaluatorAdapter: query message forwarding, synthetic
response fallback, multiple items aggregation
- 3 tests for EvalItem.PerTurnItems: empty conversation, no user messages,
system+assistant only
- StubEvaluator and StubChatClient test helpers
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Blocking link check for outdated package in DevUI.
* Replace Dictionary<string, object> payloads with typed wire models
Introduce internal FoundryEvalWireModels.cs with compile-time-safe types
for the OpenAI Evals API wire format. The OpenAI .NET SDK (2.9.1) only
provides protocol-level methods with BinaryContent/ClientResult — no
typed request models. These internal models replace scattered dictionary
literals with [JsonPropertyName]-annotated classes, giving:
- Compile-time safety (typos become build errors)
- Single point of change when the API evolves
- IntelliSense discoverability
- Cleaner serialization via JsonPolymorphic for content items
Models: WireContentItem hierarchy (text, image, tool_call, tool_result),
WireMessage, WireEvalItemPayload, WireTestingCriterion, WireItemSchema,
WireCreateEvalRequest, WireCreateRunRequest, and data source variants.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Skip metric when Foundry returns neither score nor passed
When an evaluator returns no score and no passed value, the previous
code created BooleanMetric(name, false), which falsely failed items
via ItemPassed. Now we skip the MEAI metric entirely for indeterminate
results — the raw data remains available in DetailedItems for diagnostics.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address PR #4914 review comments: fix tool evaluator bug and add tests
- Fix duplicate ToolCallAccuracy: resolve evaluator names before checking
against ToolEvaluators set (Comment 2)
- Make FilterToolEvaluators internal for testability; add tests for the
ArgumentException edge case when all evaluators are tool-type (Comment 3)
- Add CancellationToken test for LocalEvaluator (Comment 4)
- Add EvaluateAsync integration test on Run with sequential workflow and
per-agent SubResults verification (Comment 5)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address Peter's review comments on PR #4914
- Add trailing newline to Evaluation_FoundryQuality.csproj (Comment 6)
- Make evaluator name lookups case-insensitive: switch BuiltinEvaluators,
ToolEvaluators, AgentEvaluators, and ResolveEvaluator's StartsWith check
from Ordinal to OrdinalIgnoreCase (Comment 7)
- Add Trace.TraceWarning when Foundry returns fewer results than submitted
items, indicating expected vs actual count before padding (Comment 8)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add Microsoft.Extensions.AI.Evaluation packages to Directory.Packages.props
These were removed in #5269 as unused, but are needed by the Foundry
and core evaluation integration added in this PR.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: alliscode <bentho@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Update Anthropic from 12.11.0 to 12.13.0
- Update Anthropic.Foundry from 0.4.2 to 0.5.0
- Change Anthropic project from release candidate to preview
- Add new IBetaService members (Agents, Environments, Sessions, Vaults) to test mock
* .NET: [Feature Branch] Add basic durable workflow support (#3648)
* Add basic durable workflow support.
* PR feedback fixes
* Add conditional edge sample.
* PR feedback fixes.
* Minor cleanup.
* Minor cleanup
* Minor formatting improvements.
* Improve comments/documentation on the execution flow.
* .NET: [Feature Branch] Add Azure Functions hosting support for durable workflows (#3935)
* Adding azure functions workflow support.
* - PR feedback fixes.
- Add example to demonstrate complex Object as payload.
* rename instanceId to runId.
* Use custom ITaskOrchestrator to run orchestrator function.
* .NET: [Feature Branch] Adding support for events & shared state in durable workflows (#4020)
* Adding support for events & shared state in durable workflows.
* PR feedback fixes
* PR feedback fixes.
* Add YieldOutputAsync calls to 05_WorkflowEvents sample executors
The integration test asserts that WorkflowOutputEvent is found in the
stream, but the sample executors only used AddEventAsync for custom
events and never called YieldOutputAsync. Since WorkflowOutputEvent is
only emitted via explicit YieldOutputAsync calls, the assertion would
fail. Added YieldOutputAsync to each executor to match the test
expectation and demonstrate the API in the sample.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix deserialization to use shared serializer options.
* PR feedback updates.
* Sample cleanup
* PR feedback fixes
* Addressing PR review feedback for DurableStreamingWorkflowRun
- Use -1 instead of 0 for taskId in TaskFailedException when task ID is not relevant.
- Add [NotNullWhen(true)] to TryParseWorkflowResult out parameter following .NET TryXXX conventions.
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* .NET: [Feature Branch] Add nested sub-workflow support for durable workflows (#4190)
* .NET: [Feature Branch] Add nested sub-workflow support for durable workflows
* fix readme path
* Switch Orchestration output from string to DurableWorkflowResult.
* PR feedback fixes
* Minor cleanup based on PR feedback.
* .NET: [Feature Branch] Add Human In the Loop support for durable workflows (#4358)
* Add Azure Functions HITL workflow sample
Add 06_WorkflowHITL Azure Functions sample demonstrating Human-in-the-Loop
workflow support with HTTP endpoints for status checking and approval responses.
The sample includes:
- ExpenseReimbursement workflow with RequestPort for manager approval
- Custom HTTP endpoint to check workflow status and pending approvals
- Custom HTTP endpoint to send approval responses via RaiseEventAsync
- demo.http file with step-by-step interaction examples
* PR feedback fixes
* Minor comment cleanup
* Minor comment clReverted the `!context.IsReplaying` guards on `PendingEvents.Add`/`RemoveAll` and `SetCustomStatus` in `ExecuteRequestPortAsync`. The guards broke fan-out scenarios where parallel RequestPorts need to be discoverable after replay. `SetCustomStatus` is idempotent metadata that doesn't affect replay determinism.eanup
* fix for PR feedback
* PR feedback updates
* Improvements to samples
* Improvements to README
* Update samples to use parallel request ports.
* Unit tests
* Introduce local variables to improve readability of Workflows.Workflows access patter
* Use GitHub-style callouts and add PowerShell command variants in HITL sample README
* Add changelog entries for durable workflow support (#4436)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Bump Microsoft.DurableTask.Worker to 1.19.1 to fix version downgrade
Microsoft.Azure.Functions.Worker.Extensions.DurableTask 1.13.1 requires
Microsoft.DurableTask.Worker >= 1.19.1 via its transitive dependency on
Microsoft.DurableTask.Worker.Grpc 1.19.1.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix broken markdown links in durable workflow sample READMEs
- Create Workflow/README.md with environment setup docs
- Fix ../README.md -> ../../README.md in ConsoleApps 01, 02, 03, 08
- Fix SubWorkflows relative path (3 levels -> 4 levels up)
- Fix dead Durable Task Scheduler URL
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix build errors from main merge: Throw conflict, ExecuteAsync rename, GetNewSessionAsync rename
- Remove InjectSharedThrow from DurableTask csproj (uses Workflows' internal Throw via InternalsVisibleTo)
- Update ExecuteAsync -> ExecuteCoreAsync with WorkflowTelemetryContext.Disabled
- Update GetNewSessionAsync -> CreateSessionAsync
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Move durable workflow samples to 04-hosting/DurableWorkflows
Aligns with main branch sample reorganization where durable samples
live under 04-hosting/ (alongside DurableAgents/).
- Move samples/Durable/Workflow/ -> samples/04-hosting/DurableWorkflows/
- Add Directory.Build.props matching DurableAgents pattern
- Update slnx project paths
- Update integration test sample paths
- Update README cd paths and cross-references
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix build errors: remove duplicate base class members, update renamed APIs
- Remove duplicate OutputLog, WriteInputAsync, CreateTestTimeoutCts, etc. from
ConsoleAppSamplesValidation (already in SamplesValidationBase)
- Update AddFanInEdge -> AddFanInBarrierEdge in workflow samples
- Update GetNewSessionAsync -> CreateSessionAsync in workflow samples
- Update SourceId -> ExecutorId (obsolete) in workflow samples
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix dotnet format issues: add UTF-8 BOM and remove unused using
- Add UTF-8 BOM to 20 .cs files across DurableTask, AzureFunctions,
unit tests, and workflow samples
- Remove unnecessary using directive in 07_SubWorkflows/Executors.cs
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix typo PaymentProcesser -> PaymentProcessor and garbled arrows in README
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix GetExecutorName to handle agent names with underscores
Split on last underscore instead of first, and validate that the
suffix is a 32-char hex string (sanitized GUID) before stripping it.
This prevents truncation of agent names like 'my_agent' when the
executor ID is 'my_agent_<guid>'.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Align DurableTask.Client.AzureManaged to 1.19.1
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Bump DurableTask and Azure Functions extension package versions
- DurableTask.* packages: 1.19.1 -> 1.22.0
- Functions.Worker.Extensions.DurableTask: 1.13.1 -> 1.16.0
- Functions.Worker.Extensions.DurableTask.AzureManaged: 1.0.1 -> 1.5.0 (telemetry bug fix)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Bump DurableTask SDK packages to 1.22.0
- DurableTask.Client: 1.19.1 -> 1.22.0
- DurableTask.Client.AzureManaged: 1.19.1 -> 1.22.0
- DurableTask.Worker: 1.19.1 -> 1.22.0
- DurableTask.Worker.AzureManaged: 1.19.1 -> 1.22.0
- Azure Functions extensions kept at original versions (1.13.1/1.0.1) due to
host-side DurableTask.Core 3.7.0 incompatibility with newer extensions
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Update Microsoft.Azure.Functions.Worker.Extensions.DurableTask to "1.16.0"
* Add the local.settings.json files to the sample which were previously ignored. This aligns with our other samples.
* Increase timeout for tests as CI has them failing transiently.
* increaset timeout value for azure functions integration tests.
* Add YieldsOutput(string) to workflow shared state sample executors
ValidateOrder and EnrichOrder call YieldOutputAsync with string messages,
but only their TOutput (OrderDetails) was in the allowed yield types.
This caused TargetInvocationException in the WorkflowSharedState sample
validation integration test.
* Downgrade the durable packages to 1.18.0
* Downgrading Worker.Extensions.DurableTask to 1.12.1
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* .NET: Upgrade to XUnit 3 and Microsoft Testing Platform (#4176)
* Fix copilot studio integration tests failure (#4209)
* Fix anthropic integration tests and skip reason (#4211)
* Remove accidental add of code coverage for integration tests (#4219)
* Add solution filtered parallel test run (#4226)
* Fix build paths (#4228)
* Fix coverage settings path and trait filter (#4229)
* Add project name filter to solution (#4231)
* Increase Integration Test Parallelism (#4241)
* Increase integration tests threads to 4x (#4242)
* Separate build and test into parallel jobs (#4243)
* Filter src by framework for tests build (#4244)
* Separate build and test into parallel jobs
* Filter source projects by framework for tests build
* Pre-build samples via tests to avoid timeouts (#4245)
* Separate build from run for console sample validation (#4251)
* Address PR comments (#4255)
* Merge and move scripts (#4308)
* .NET: Add Microsoft Fabric sample #3674 (#4230)
Co-authored-by: Chris <66376200+crickman@users.noreply.github.com>
* Python: Phase 2: Embedding clients for Ollama, Bedrock, and Azure AI Inference (#4207)
* Phase 2: Embedding clients for Ollama, Bedrock, and Azure AI Inference
Add embedding client implementations to existing provider packages:
- OllamaEmbeddingClient: Text embeddings via Ollama's embed API
- BedrockEmbeddingClient: Text embeddings via Amazon Titan on Bedrock
- AzureAIInferenceEmbeddingClient: Text and image embeddings via Azure AI
Inference, supporting Content | str input with separate model IDs for
text (AZURE_AI_INFERENCE_EMBEDDING_MODEL_ID) and image
(AZURE_AI_INFERENCE_IMAGE_EMBEDDING_MODEL_ID) endpoints
Additional changes:
- Rename EmbeddingCoT -> EmbeddingT, EmbeddingOptionsCoT -> EmbeddingOptionsT
- Add otel_provider_name passthrough to all embedding clients
- Register integration pytest marker in all packages
- Add lazy-loading namespace exports for Ollama and Bedrock embeddings
- Add image embedding sample using Cohere-embed-v3-english
- Add azure-ai-inference dependency to azure-ai package
Part of #1188
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix mypy duplicate name and ruff lint issues
- Rename second 'vector' variable to 'img_vector' in image embedding loop
- Combine nested with statements in tests
- Remove unused result assignments in tests
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* updates from feedback
* Fix CI failures in embedding usage handling
- Fix Azure AI embedding mypy issues by normalizing vectors to list[float],
safely accumulating optional usage token fields, and filtering None entries
before constructing GeneratedEmbeddings
- Avoid Bandit false positive by initializing usage details as an empty dict
- Update OpenAI embedding tests to assert canonical usage keys
(input_token_count/total_token_count)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* [Purview] Mark responses as responses and fix epoch bug for python long overflow (#4225)
* .NET: Support InvokeMcpTool for declarative workflows (#4204)
* Initial implementation of InvokeMcpTool in declarative workflow
* Cleaned up sample implementation
* Updated sample comments.
* Added missing executor routing attribute
* Fix PR comments.
* Updated based on PR comments.
* Updated based on PR comments.
* Removed unnecessary using statement.
* Update Python package versions to rc2 (#4258)
- Bump core and azure-ai to 1.0.0rc2
- Bump preview packages to 1.0.0b260225
- Update dependencies to >=1.0.0rc2
- Add CHANGELOG entries for changes since rc1
- Update uv.lock
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* .NET: Fixing issue where OpenTelemetry span is never exported in .NET in-process workflow execution (#4196)
* 1. Add reproduction test for issue #4155: workflow.run Activity never stopped in streaming OffThread path
The WorkflowRunActivity_IsStopped_Streaming_OffThread test demonstrates that
the workflow.run OpenTelemetry Activity created in StreamingRunEventStream.RunLoopAsync
is started but never stopped when using the OffThread/Default streaming execution.
The background run loop keeps running after event consumption completes, so the
using Activity? declaration never disposes until explicit StopAsync() is called.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2. Fix workflow.run Activity never stopped in streaming OffThread execution (#4155)
The workflow.run OpenTelemetry Activity in StreamingRunEventStream.RunLoopAsync
was scoped to the method lifetime via 'using'. Since the run loop only exits on
cancellation, the Activity was never stopped/exported until explicit disposal.
Fix: Remove 'using' and explicitly dispose the Activity when the workflow reaches
Idle status (all supersteps complete). A safety-net disposal in the finally block
handles cancellation and error paths.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add root-level workflow.session activity spanning run loop lifetime\n\nImplements two-level telemetry hierarchy per PR feedback from lokitoth:\n- workflow.session: spans the entire run loop / stream lifetime\n- workflow_invoke: per input-to-halt cycle, nested within the session\n\nThis ensures the session activity stays open across multiple turns,\nwhile individual run activities are created and disposed per cycle.\n\nAlso fixes linkedSource CancellationTokenSource disposal leak in\nStreamingRunEventStream (added using declaration)."
* Address Copilot review: fix Activity/CTS disposal, rename activity, add error tag\n\n1. LockstepRunEventStream: Remove 'using' from Activity in async iterator\n and manually dispose in finally block (fixes#4155 pattern). Also dispose\n linkedSource CTS in finally to prevent leak.\n2. Tags.cs: Add ErrorMessage (\"error.message\") tag for runtime errors,\n distinct from BuildErrorMessage (\"build.error.message\").\n3. ActivityNames: Rename WorkflowRun from \"workflow_invoke\" to \"workflow.run\"\n for cross-language consistency.\n4. WorkflowTelemetryContext: Fix XML doc to say \"outer/parent span\" instead\n of \"root-level span\".\n5. ObservabilityTests: Assert WorkflowSession absence when DisableWorkflowRun\n is true.\n6. WorkflowRunActivityStopTests: Fix streaming test race by disposing\n StreamingRun before asserting activities are stopped.\n7. StreamingRunEventStream/LockstepRunEventStream: Use Tags.ErrorMessage\n instead of Tags.BuildErrorMessage for runtime error events."
* Review fixes: revert workflow_invoke rename, use 'using' for linkedSource, move SessionStarted earlier\n\n- Revert ActivityNames.WorkflowRun back to \"workflow_invoke\" (OTEL semantic convention contract)\n- Use 'using' declaration for linkedSource CTS in LockstepRunEventStream (no timing sensitivity)\n- Move SessionStarted event before WaitForInputAsync in StreamingRunEventStream to match Lockstep behavior"
* Improve naming and comments in WorkflowRunActivityStopTests"
* Prevent session Activity.Current leak in lockstep mode, add nesting test
Save and restore Activity.Current in LockstepRunEventStream.Start() so the
session activity doesn't leak into caller code via AsyncLocal. Re-establish
Activity.Current = sessionActivity before creating the run activity in
TakeEventStreamAsync to preserve parent-child nesting.
Add test verifying app activities after RunAsync are not parented under the
session, and that the workflow_invoke activity nests under the session."
* Fix stale XML doc: WorkflowRun -> WorkflowInvoke in ObservabilityTests
---------
Co-authored-by: alliscode <bentho@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python / .NET Samples - Restructure and Improve Samples (Feature Branc… (#4092)
* Python: .NET Samples - Restructure and Improve Samples (Feature Branch) (#4091)
* Moved by agent (#4094)
* Fix readme links
* .NET Samples - Create `04-hosting` learning path step (#4098)
* Agent move
* Agent reorderd
* Remove A2A section from README
Removed A2A section from the Getting Started README.
* Agent fixed links
* Fix broken sample links in durable-agents README (#4101)
* Initial plan
* Fix broken internal links in documentation
Co-authored-by: crickman <66376200+crickman@users.noreply.github.com>
* Revert template link changes; keep only durable-agents README fix
Co-authored-by: crickman <66376200+crickman@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: crickman <66376200+crickman@users.noreply.github.com>
* .NET Samples - Create `03-workflows` learning path step (#4102)
* Fix solution project path
* Python: Fix broken markdown links to repo resources (outside /docs) (#4105)
* Initial plan
* Fix broken markdown links to repo resources
Co-authored-by: crickman <66376200+crickman@users.noreply.github.com>
* Update README to rename .NET Workflows Samples section
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: crickman <66376200+crickman@users.noreply.github.com>
* .NET Samples - Create `02-agents` learning path step (#4107)
* .NET: Fix broken relative link in GroupChatToolApproval README (#4108)
* Initial plan
* Fix broken link in GroupChatToolApproval README
Co-authored-by: crickman <66376200+crickman@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: crickman <66376200+crickman@users.noreply.github.com>
* Update labeler configuration for workflow samples
* .NET - Reorder Agents samples to start from Step01 instead of Step04 (#4110)
* Fix solution
* Resolve new sample paths
* Move new AgentSkills and AgentWithMemory_Step04 samples
* Fix link
* Fix readme path
* fix: update stale dotnet/samples/Durable path reference in AGENTS.md
Co-authored-by: crickman <66376200+crickman@users.noreply.github.com>
* Moved new sample
* Update solution
* Resolve merge (new sample)
* Sync to new sample - FoundryAgents_Step21_BingCustomSearch
* Updated README
* .NET Samples - Configuration Naming Update (#4149)
* .NET: Restore AzureFunctions index parity with ConsoleApps under DurableAgents samples (#4221)
* Clean-up `05_host_your_agent`
* Config setting consistency
* Refine samples
* AGENTS.md
* Move new samples
* Re-order samples
* Move new project and fixup solution
* Fixup model config
* Fix up new UT project
---------
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
* Python: Fix Bedrock embedding test stub missing meta attribute (#4287)
* Fix Bedrock embedding test stub missing meta attribute
* Increase test coverage so gate passes
* Python: (ag-ui): fix approval payloads being re-processed on subsequent conversation turns (#4232)
* Fix ag-ui tool call issue
* Safe json fix
* Python: Update workflow orchestration samples to use AzureOpenAIResponsesClient (#4285)
* Update workflow orchestration samples to use AzureOpenAIResponsesClient
* Fix broken link
* Move scripts to scripts folder
---------
Co-authored-by: Roger Barreto <19890735+rogerbarreto@users.noreply.github.com>
Co-authored-by: Chris <66376200+crickman@users.noreply.github.com>
Co-authored-by: Eduard van Valkenburg <eavanvalkenburg@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Rishabh Chawla <rishabhchawla1995@gmail.com>
Co-authored-by: Peter Ibekwe <109177538+peibekwe@users.noreply.github.com>
Co-authored-by: Dmytro Struk <13853051+dmytrostruk@users.noreply.github.com>
Co-authored-by: Ben Thomas <ben.thomas@microsoft.com>
Co-authored-by: alliscode <bentho@microsoft.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Evan Mattson <35585003+moonbox3@users.noreply.github.com>
* Fix encoding (#4309)
* Disable Parallelization for WorkflowRunActivityStopTests (#4313)
* Revert parallel disable (#4324)
* .NET: Disable flakey Workflow Observability tests (#4416)
* Disable flakey OffThread test
* Disable additional OffThread test
* Disable a further test
* Disable all observability tests
---------
Co-authored-by: Roger Barreto <19890735+rogerbarreto@users.noreply.github.com>
Co-authored-by: Chris <66376200+crickman@users.noreply.github.com>
Co-authored-by: Eduard van Valkenburg <eavanvalkenburg@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Rishabh Chawla <rishabhchawla1995@gmail.com>
Co-authored-by: Peter Ibekwe <109177538+peibekwe@users.noreply.github.com>
Co-authored-by: Dmytro Struk <13853051+dmytrostruk@users.noreply.github.com>
Co-authored-by: Ben Thomas <ben.thomas@microsoft.com>
Co-authored-by: alliscode <bentho@microsoft.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Evan Mattson <35585003+moonbox3@users.noreply.github.com>
* Update Microsoft.Agents.AI.AzureAI for Azure.AI.Projects SDK 2.0.0
- Bump Azure.AI.Projects to 2.0.0-alpha.20260213.1
- Bump Azure.AI.Projects.OpenAI to 2.0.0-alpha.20260213.1
- Bump System.ClientModel to 1.9.0 (transitive dependency)
- Switch both GetAgent and CreateAgentVersion to protocol methods
with MEAI user-agent policy injection via RequestOptions
- Migrate 29 CREATE-path tests from FakeAgentClient to HttpHandlerAssert
pattern for real HTTP pipeline testing
- Fix StructuredOutputDefinition constructor (BinaryData -> IDictionary)
- Fix responses endpoint path (openai/responses -> /responses)
- Add local-packages NuGet source for pre-release nupkgs
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Update Azure.AI.Projects to 2.0.0-beta.1 from NuGet.org
- Update Azure.AI.Projects and Azure.AI.Projects.OpenAI to 2.0.0-beta.1
- Remove local-packages NuGet source (packages now on nuget.org)
- Fix MemorySearchTool -> MemorySearchPreviewTool rename
- Fix RedTeams.CreateAsync ambiguous call
- Fix CreateAgentVersion/Async signature change (BinaryData -> string)
- Suppress AAIP001 experimental warning for WorkflowAgentDefinition
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Move s_modelWriterOptionsWire field before methods that use it
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix flaky test: prevent spurious workflow_invoke Activity on timeout wake-up
The StreamingRunEventStream run loop uses a 1-second timeout on
WaitForInputAsync. When the timeout fires before the consumer calls
StopAsync, the loop would create a spurious workflow_invoke Activity
even though no actual input was provided. This caused the
WorkflowRunActivity_IsStopped_Streaming_OffThread_MultiTurnAsync test
to intermittently fail (expecting 2 activities but finding 3).
Fix: guard the loop body with a HasUnprocessedMessages check. On
timeout wake-ups with no work, the loop waits again without creating
an activity or changing the run status.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix epoch race condition causing unit tests to hang on net10.0 and net472
The HasUnprocessedMessages guard (previous commit) correctly prevents
spurious workflow_invoke Activity creation on timeout wake-ups, but
exposed a latent race in the epoch-based signal filtering.
The race: when the run loop processes messages quickly and calls
Interlocked.Increment(ref _completionEpoch) before the consumer calls
TakeEventStreamAsync, the consumer reads the already-incremented epoch
and sets myEpoch = epoch + 1. This causes the consumer to skip the
valid InternalHaltSignal (its epoch < myEpoch) and block forever
waiting for a signal that will never arrive (since the guard prevents
spurious signal generation).
Fix: read _completionEpoch without +1. The +1 was originally needed to
filter stale signals from timeout-driven spurious loop iterations, but
those no longer exist thanks to the HasUnprocessedMessages guard.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Revert "Fix epoch race condition causing unit tests to hang on net10.0 and net472"
This reverts commit 6ce7f01be8.
* Revert "Fix flaky test: prevent spurious workflow_invoke Activity on timeout wake-up"
This reverts commit 98963e17f2.
* Skip hanging multi-turn declarative integration tests
The ValidateMultiTurnAsync tests (ConfirmInput.yaml, RequestExternalInput.yaml)
hang indefinitely in CI, blocking the merge queue. The hang is SDK-independent
(reproduces with both Azure.AI.Projects 1.2.0-beta.5 and 2.0.0-beta.1) and
is a pre-existing issue in the declarative workflow multi-turn test logic.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Remove unused using directive in IntegrationTest.cs
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Restore Azure.AI.Projects 2.0.0-beta.1 version bump
The merge from main accidentally reverted the package versions back to
1.2.0-beta.5. This is the primary change of this PR.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address merge conflict
* Skip flaky WorkflowRunActivity_IsStopped_Streaming_OffThread_MultiTurnAsync test
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Skip CheckSystem test cases temporarily
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add sample demonstrating authentication and user access in agent tools
* Add fixes to enable running on windows
* Add launchsettings, add docker-compose to slnx and fix formatting
* Switch to Expenses rather than todo based sample and address PR comments
* Rename sample
* Fix formatting
* Initial Implementation of InvokeFunctionTool
* Added unit test for InvokeFunctionTool executor.
* Implemented unit and integration tests for InvokeFunctionTool.
* Add sample for InvokeFunctionTool in declarative workflows.
* Remove unused sample and updated comments.
* Updating to official OM release with InvokeFunctionTool
* Fix formatting issues.
* Updated PowerFx version
* Update test fixture
* Cleanup - Removed unused method in InvokeFunctionToolExecutor
* Update test based on PR feedback.
* Update based on PR comments
- Update ModelContextProtocol NuGet package from 0.4.0-preview.3 to 0.8.0-preview.1
- Update System.Net.ServerSentEvents from 10.0.1 to 10.0.3
- Fix OAuth config to use DynamicClientRegistration in Agent_MCP_Server_Auth
- Fix incorrect sample name references in README files
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Initial plan
* Add Foundry evaluation samples for Red Teaming and Self-Reflection
Co-authored-by: rogerbarreto <19890735+rogerbarreto@users.noreply.github.com>
* Refactor evaluation samples with real implementations in local functions
Co-authored-by: rogerbarreto <19890735+rogerbarreto@users.noreply.github.com>
* Uncomment function signatures and bodies, keep only invocations commented
Co-authored-by: rogerbarreto <19890735+rogerbarreto@users.noreply.github.com>
* Update Foundry evaluation samples with observability support
* Restructure evaluation samples to follow FoundryAgents naming convention
- Rename Evaluation/Evaluation_StepXX to FoundryAgents_Evaluations_StepXX
- Add evaluation projects to slnx
- Fix var usage, apply dotnet format, use DefaultAzureCredential
- Add try/finally for agent cleanup
- Fix evaluator deployment name separation in Step02
- Update README references
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Rewrite Step01 to use Azure.AI.Projects RedTeam API and address review comments
- Replace safety evaluator sample with actual Red Teaming using AIProjectClient.RedTeams
- Use AttackStrategy (Easy, Moderate, Jailbreak) and RiskCategory from Azure.AI.Projects
- Remove Microsoft.Extensions.AI.Evaluation.Safety dependency from Step01
- Add DefaultAzureCredential warning comments to Step02
- Remove unused bestResponse variable in Step02
- Add session isolation comments in self-reflection loop
- Fix stale directory references in READMEs
- Fix misleading evaluation overview link in main README
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add note about agent-targeted red teaming limitations in README
The .NET RedTeam API currently only supports model deployment targets
via AzureOpenAIModelConfiguration. Agent-targeted red teaming with
AzureAIAgentTarget is documented in concept docs but not yet available
in the SDK's RedTeam constructor. Results appear in classic portal view.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add classic Foundry disclaimer to red teaming sample README
Clarify that this sample uses the classic Azure AI Foundry red teaming
API (/redTeams/runs). The new Foundry portal uses a separate evaluation-
based API not yet available in the .NET SDK. AzureAIAgentTarget exists
in the SDK but is consumed by the Evaluation Taxonomy API, not RedTeam.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address PR review comments on Step02 SelfReflection
- Pass full prompt (with context) to evaluator messages instead of just
the question, so evaluator input matches what the agent received
- Include previous response text in self-reflection refinement prompt
so the LLM can meaningfully improve its answer across iterations
- Inline CreateKnowledgeAgent helper (single use, single statement)
- Add comment clarifying why RunCombinedQualityAndSafetyEvaluation
intentionally passes only the question (no context)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: rogerbarreto <19890735+rogerbarreto@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Update GitHub.Copilot.SDK to 0.1.23 and copy new session config properties
- Bump GitHub.Copilot.SDK from 0.1.18 to 0.1.23
- Add new SessionConfig properties: ReasoningEffort, Hooks, OnUserInputRequest,
WorkingDirectory, ConfigDir, InfiniteSessions
- Add missing ResumeSessionConfig properties: Model, SystemMessage,
AvailableTools, ExcludedTools, ReasoningEffort, Hooks, OnUserInputRequest,
WorkingDirectory, ConfigDir, InfiniteSessions
- Fix UserMessageDataAttachmentsItem -> UserMessageDataAttachmentsItemFile
for new polymorphic attachment API
- Add unit tests for new session config properties
* Address PR review: centralize config mapping and improve test coverage
- Extract CopySessionConfig/CopyResumeSessionConfig as internal static helpers
to eliminate duplicated mapping logic between RunCoreStreamingAsync and
CreateResumeConfig (addresses reviewer comment on drift risk)
- Add InternalsVisibleTo for unit test project
- Replace shallow constructor tests with comprehensive property-verification
tests that validate every config property is correctly copied, including
OnUserInputRequest (addresses reviewer comments on test coverage)
* Remove accidentally committed git-lfs hooks
* Builds locally and tests pass
* Fix typo
* Reverted nuget config change to remove internal feed and map to new public object model package with renames.
* Renaming Bot object model in additional sample.
---------
Co-authored-by: Peter Ibekwe <peibekwe@microsoft.com>
* Roslyn Source Generators for Workflow Executor Routing.
* Update dotnet/src/Microsoft.Agents.AI.Workflows.Generators/ExecutorRouteGenerator.cs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* WIP.
* All fixed up except dangling sends/yields attriutes, working on that next.
* Add protocol-only generation for SendsMessage/YieldsOutput attributes
* Ensuring collections that can change order are sorted to enable pipeline caching.
* Improvents per PR feedback.
---------
Co-authored-by: alliscode <bentho@microsoft.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Use GrpcEntityRunner instead of TaskEntityDispatcher
* Pin to Durable worker 1.11.0
* Set the invocation result
* Update all Durable packages
* Update changelog, rename dispatcher to encondedEntityRequest
* Update to latest Azure.AI.*, OpenAI, and M.E.AI*
Absorb breaking changes in Responses surface area
* Update dotnet/samples/AgentWebChat/AgentWebChat.AgentHost/Utilities/ChatClientExtensions.cs
* Update dotnet/samples/AgentWebChat/AgentWebChat.AgentHost/Utilities/ChatClientExtensions.cs
* Update dotnet/samples/AgentWebChat/AgentWebChat.AgentHost/Utilities/ChatClientExtensions.cs
* Update dotnet/samples/GettingStarted/AgentWithOpenAI/Agent_OpenAI_Step04_CreateFromOpenAIResponseClient/Program.cs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Using patch to remove the model is necessary, updated the response client to actually use the the ForAgent
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Roger Barreto <19890735+rogerbarreto@users.noreply.github.com>
Tommaso Stocchi ·
Ben Thomas ·
westey ·
Roger Barreto ·
dependabot[bot] ·
Jacob Alber ·
Copilot ·
Shyju Krishnankutty ·
SergeyMenshykh ·
Chris ·
Dmytro Struk ·
Peter Ibekwe ·
Chandramouleswaran ·
Stephen Toub ·
Chris Gillum ·
Jacob Viau ·