Compare commits

...
Author SHA1 Message Date
Tao Chen 20ac21c780 Fix tests 2026-06-01 13:25:54 -07:00
Tao Chen de5b4d619a Make unused tool as comment 2026-06-01 13:15:08 -07:00
Tao Chen 98fbaf2481 Resolve conflict 2026-06-01 13:08:33 -07:00
Tao Chen 0fc5600ae2 Fix toolbox consent flow in hosted agent 2026-06-01 12:59:01 -07:00
Yufeng HeandGitHub 78d175a1e2 Python: coalesce code interpreter history chunks (#5801)
* fix: coalesce code interpreter history chunks

* fix: narrow content item list types

* fix: remove redundant content list casts
2026-06-01 13:26:20 +00:00
CopilotGitHublarohracopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>Laveesh Rohra
b59a854fcd Fix integration test worker crashes in Azure Functions on Py3.13 (#4260)
* Initial plan

* Fix integration test worker crashes on Python 3.13

Three changes to prevent pytest-xdist workers from crashing during
Azure Functions integration tests:

1. Add `start_new_session=True` to subprocess on Linux so signals
   (e.g. from test-timeout) cannot propagate between the func host
   and the xdist worker process.

2. Add an overall 100-second budget to the fixture setup loop so
   the retry logic never exceeds the 120-second test timeout. When
   pytest-timeout's thread method fires during fixture setup and the
   thread doesn't respond, it calls os._exit() which kills the
   xdist worker – this is the root cause of the "Not properly
   terminated" crashes.

3. Remove the `UV_PYTHON: "3.10"` workaround from both workflow
   files so integration tests actually run on Python 3.13.

Co-authored-by: larohra <41490930+larohra@users.noreply.github.com>

* Validate integration tests on Python 3.13

Co-authored-by: larohra <41490930+larohra@users.noreply.github.com>

* Revert unintentional uv.lock dependency bumps

Co-authored-by: larohra <41490930+larohra@users.noreply.github.com>

* Use time.monotonic() instead of time.time() for fixture budget timing

Addresses review feedback: monotonic clock is immune to NTP/clock
adjustments that could skew the budget enforcement.

Co-authored-by: larohra <41490930+larohra@users.noreply.github.com>

* Fix func worker segfault on Python 3.13 by redirecting worker to Python 3.12

The Azure Functions Python worker crashes with SIGSEGV (exit code 139)
on Python 3.13 due to protobuf C extension (google._upb) compatibility
issues.  When the test runner uses Python >=3.13, the conftest now
automatically finds a compatible Python 3.10-3.12 and sets
languageWorkers__python__defaultExecutablePath so the func host uses
it for the worker process.

The CI setup action also ensures Python 3.12 is available on the
runner, falling back to uv python install if the system doesn't have
it.

Co-authored-by: larohra <41490930+larohra@users.noreply.github.com>

* Address code review: add path validation, clarify version range and config key format

Co-authored-by: larohra <41490930+larohra@users.noreply.github.com>

* Run func worker natively on Python 3.13 by disabling dependency isolation

Replace the Python 3.12 redirect workaround with the proper fix:
set PYTHON_ISOLATE_WORKER_DEPENDENCIES=0 on Python >=3.13.

The segfault (exit code 139) is caused by the Azure Functions worker's
module isolation mechanism conflicting with protobuf's C extensions
(google._upb) on Python 3.13.  Disabling isolation lets the worker
load dependencies from the app's own environment, which avoids the
crash while keeping everything running on Python 3.13.

See: https://github.com/Azure/azure-functions-python-worker/issues/1797

Co-authored-by: larohra <41490930+larohra@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: larohra <41490930+larohra@users.noreply.github.com>
Co-authored-by: Laveesh Rohra <larohra@microsoft.com>
2026-06-01 09:18:26 +00:00
Evan MattsonandGitHub 8b0db48d33 Add community PR limit workflow (#6229)
* Add community PR limit workflow

* Address PR limit workflow review feedback
2026-06-01 18:12:31 +09:00
5affc9c333 Python: Reorganize A2A samples and use package A2AExecutor (#6165)
* Reorganize A2A samples: client demos in 02-agents, use package A2AExecutor

- Move client samples (agent_with_a2a, a2a_agent_as_function_tools) to samples/02-agents/a2a/
- Add new concept samples: polling, stream reconnection, protocol selection
- Replace sample agent_executor.py with package-level A2AExecutor (stream=True)
- Update 04-hosting/a2a to focus on server-side, point to 02-agents for clients
- Add README.md for the new 02-agents/a2a/ sample collection

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix streaming artifact coalescing and address PR review feedback

A2AExecutor fix:
- Generate a stable artifact_id per stream in _run_stream so all streaming
  chunks share the same ID, enabling proper append=True coalescing per the
  A2A spec (TaskArtifactUpdateEvent with same artifactId).
- Previously, item.message_id was None for OpenAI/Foundry streaming updates,
  causing the SDK to generate a new random UUID per token (100+ separate
  artifacts instead of 1 appended artifact).

Sample improvements:
- Replace join workaround with response.text now that coalescing works
- Add background=True to stream reconnection resume call (required for
  continuation token emission on in-progress tasks)
- Fix type ignore specificity in polling sample

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-01 07:09:11 +00:00
edcc786651 .NET: Preserve and propagate CreatedAt through workflows (#3930)
* Preserve per-message CreatedAt attribute if it's available

* Add unit test

---------

Co-authored-by: Sam Chang <changsam@microsoft.com>
Co-authored-by: samchang-msft <samchang.msft@gmail.com>
2026-05-29 21:41:40 +00:00
07a1e83492 .NET: Forward Magentic participant replies to manager (#6156)
MagenticOrchestrator.TakeTurnAsync dropped the `messages` parameter
on subsequent turns, so participant replies never reached the manager's
ChatHistory. The manager kept re-dispatching the same speaker every
round until MaxRounds.

Append the incoming messages to taskContext.ChatHistory before running
the coordination round (matches Python's _handle_response).

Adds RecordingReplayAgent + regression test that asserts the worker's
reply reaches round-2's progress-ledger call.

Co-authored-by: Jacob Alber <jaalber@microsoft.com>
2026-05-29 21:41:25 +00:00
Roger BarretoandGitHub fa2a6af443 Bump Azure.AI.AgentServer.* packages and align Azure.Core/System.ClientModel (#6178)
* Bump Azure.AI.AgentServer.* package versions

* Align Azure.Core/System.ClientModel to AgentServer transitive deps

Bump Azure.Core 1.55->1.56 and System.ClientModel 1.11->1.12 to match Azure.AI.AgentServer.* requirements, and add explicit references in transitive-pinning-off Foundry consumers to avoid CS1705/MSB3277 version conflicts.
2026-05-29 19:42:07 +00:00
Peter IbekweandGitHub 11c8d89ab2 .NET: Fix InvokeMcpTool approval path for declarative workflows (#6177)
* Fix InvokeMcpTool approval path for declarative workflows

* Added more test for coverage.
2026-05-29 19:07:48 +00:00
CopilotGitHubcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>Roger Barreto
6510d6e3c8 .NET: Quarantine flaky DevUI test (#6159)
* Bump Microsoft.Extensions.AI packages to 10.6.0

* Align transitive package versions for Microsoft.Extensions.AI 10.6.0

* Initial plan

* Temporarily skip flaky DevUI keyed/default workflow test

* Revert Microsoft.Extensions.AI package bumps, keep only flaky test quarantine

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Roger Barreto <19890735+rogerbarreto@users.noreply.github.com>
2026-05-29 17:09:12 +00:00
dd9a4b6321 Python: [A2A] Set message_id on AgentResponseUpdate for message-bearing paths (#6163)
Map A2A protocol message_id to AgentResponseUpdate.message_id in two paths
where it was previously omitted, aligning with .NET behavior:

1. Standalone A2AMessage: set message_id=msg.message_id (matches .NET
   ConvertToAgentResponseUpdate(Message) which sets both ResponseId and
   MessageId to message.MessageId)

2. TaskStatusUpdateEvent (terminal/input_required): set
   message_id=message.message_id (matches .NET which sets
   MessageId=statusUpdateEvent.Status.Message?.MessageId)

Fixes #5949

Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-29 08:11:13 +00:00
e8ff541ebf Python: consolidate MCP reliability fixes (#6145)
* Python: consolidate MCP reliability fixes

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix MCP cleanup and metadata typing

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Satisfy MCP metadata mypy typing

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix Pyright metadata mapping type

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-29 07:21:14 +00:00
Daria KorenievaandGitHub d2d5384f28 Python: Add Mistral AI embedding client package (#5480)
* Python: Add Mistral AI embedding client package

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

* Address review feedback: fix dimensions check, sort embeddings by index, align docs

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

* Address review feedback: downgrade to alpha, remove integration tests - Change version to 1.0.0a260505 (alpha) - Update classifier to Development Status :: 3 - Alpha - Update PACKAGE_STATUS.md to alpha - Remove Mistral from integration test workflows (no API keys yet)

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

* Add samples directory for alpha package compliance Per python-package-management skill: alpha packages must include samples inside the package directory.

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

* Fix ruff formatting in sample file

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

---------

Signed-off-by: Daria Korenieva <daric2612@gmail.com>
2026-05-29 07:20:56 +00:00
Jacob AlberandGitHub 1fccf16f11 feat: Remove [Experimental] tag from .NET Orchestrations (#6164) 2026-05-29 00:03:19 +00:00
8ed2159c4b .NET: Workflow Outputs Overhaul: Support Tagging, Filtering Agent Outputs (#6045)
* test: reshuffle .NET Workflow tests in preparation for Outputs overhaul

Phase 1 of the .NET Workflows outputs overhaul (see
working/implementation-plan.md). Pure moves/renames in
dotnet/tests/Microsoft.Agents.AI.Workflows.UnitTests; no production code
changes, no new test cases. The split keeps each orchestration mode in
its own source file so the upcoming tag-aware and orchestration-default
test additions land on clean diffs.

Renames:
* WorkflowBuilderSmokeTests.cs -> WorkflowBuilderTests.cs (with class
  rename to match). The scope is no longer "smoke"-only once subsequent
  phases add tag-aware builder tests.
* InputWaiterAndOutputFilterTests.cs -> InputWaiterTests.cs +
  OutputFilterTests.cs. The file already declared the two test classes
  separately; this split simply gives each its own file so the
  output-filter cases have a dedicated home for tag-aware additions.

Split of AgentWorkflowBuilderTests.cs:
* AgentWorkflowBuilderTests.cs is now the outer
  `public static partial class AgentWorkflowBuilderTests` holding the
  shared test helpers (DoubleEchoAgent + session + WithBarrier variant,
  WorkflowRunResult, RunWorkflow* methods) bumped from `private` to
  `internal` so the new top-level GroupChatWorkflowBuilderTests in the
  same assembly can reach them.
* AgentWorkflowBuilder.SequentialTests.cs (nested SequentialTests):
  BuildSequential_InvalidArguments_Throws,
  BuildSequential_AgentsRunInOrderAsync.
* AgentWorkflowBuilder.ConcurrentTests.cs (nested ConcurrentTests):
  BuildConcurrent_InvalidArguments_Throws,
  BuildConcurrent_AgentsRunInParallelAsync.

Sequential and Concurrent are kept as nested classes because they're
modes of the same `AgentWorkflowBuilder` static factory and do not
produce dedicated builder types.

New file:
* GroupChatWorkflowBuilderTests.cs (top-level): the existing
  BuildGroupChat_* and GroupChatManager_* cases moved out of the old
  AgentWorkflowBuilderTests file. They exercise the
  `GroupChatWorkflowBuilder` type (returned by
  `AgentWorkflowBuilder.CreateGroupChatBuilderWith`), so a dedicated
  top-level test class - matching the convention reserved by the plan
  for HandoffWorkflowBuilderTests / MagenticWorkflowBuilderTests - is
  the right home. Cross-class helper references qualify with
  `AgentWorkflowBuilderTests.DoubleEchoAgent` and
  `AgentWorkflowBuilderTests.RunWorkflowAsync`.

The outer partial class is `static` (and nested classes carry the
instance test methods) because the outer holds only static helpers;
this satisfies CA1052 without suppressions and is invisible to xUnit
discovery, which finds tests on the nested classes as
`AgentWorkflowBuilderTests.SequentialTests.*` etc.

Validation: `dotnet build` clean on both target frameworks; all 547
tests in Microsoft.Agents.AI.Workflows.UnitTests pass on net10.0.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat: introduce OutputTag, Futures, and tag-aware WorkflowBuilder API

Phase 2 of the .NET Workflows outputs overhaul. Additive code change
only - no observable runtime behavior change. The runner still uses the
legacy bypass for AgentResponse / AgentResponseUpdate payloads, and the
new `Futures.EnableAgentResponseOutputTaggingAndFiltering` flag defaults
to false. Phase 3 will wire the flag into the runner; this commit only
introduces the types and the builder API.

New public surface:
* `OutputTag` (readonly struct): wraps a string Value with ordinal
  equality (IEquatable, GetHashCode, == / !=) so it can participate as a
  HashSet element. Internal ctor closes the set. One public singleton:
  `OutputTag.Intermediate`. Terminal / regular outputs carry no tag
  (empty Tags set). JSON-serialized as a bare string via
  [JsonConverter(typeof(OutputTagJsonConverter))], with the converter
  rehydrating to the well-known singleton on read.
* `Futures` (static class): hosts opt-in pre-GA behavior switches.
  First flag is `EnableAgentResponseOutputTaggingAndFiltering`; XML doc
  captures the v2.0.0 obsoletion / v3.0.0 removal lifecycle.
* `WorkflowOutputEvent.Tags`: `HashSet<OutputTag>` exposed directly
  (concrete collection, matches the JSON-serialization convention used
  for `WorkflowInfo.OutputExecutorIds`). Never null; empty for legacy /
  terminal events. New ctors take a single `OutputTag` or
  `IEnumerable<OutputTag>?`; the existing (data, executorId) ctor
  remains and produces an untagged event. `HasTag(OutputTag)` helper.
  `AgentResponseEvent` and `AgentResponseUpdateEvent` gain matching
  tag-accepting ctors forwarding to the base.
* `WorkflowOutputEventExtensions.IsIntermediate(this WorkflowOutputEvent)`:
  extension method returning `evt.HasTag(OutputTag.Intermediate)`. The
  preferred way to ask "is this an intermediate output?" without
  reaching into the Tags set.
* `WorkflowBuilder.WithOutputFrom(IEnumerable<ExecutorBinding>, OutputTag)`
  and `WorkflowBuilder.WithOutputFrom(ExecutorBinding, OutputTag)`:
  forward-looking tagged overloads. The IEnumerable form is the primary
  tagged surface; the single-executor form is a convenience for the
  common one-executor case. Currently usable for the
  `OutputTag.Intermediate` singleton; will become the primary surface
  once the `OutputTag` constructor is opened to user-defined tags in
  a future release. Callers in this release should prefer the
  intent-specific `WithIntermediateOutputFrom` extension for the
  intermediate case. Tags accumulate across repeated calls; same tag
  repeated dedupes via the HashSet.
* `WorkflowBuilderExtensions.WithIntermediateOutputFrom(this WorkflowBuilder, IEnumerable<ExecutorBinding>)`:
  helper that forwards to `WithOutputFrom(executors, OutputTag.Intermediate)`.
  Takes an IEnumerable (matching the tagged WithOutputFrom shape) -
  callers pass collection literals: `builder.WithIntermediateOutputFrom([a, b])`.
  XML doc remarks call out the Futures-flag interaction and the
  AIAgent-payload forwarding contract.

Internal shape changes:
* `WorkflowBuilder._outputExecutors`: HashSet<string> -> Dictionary<
  string, HashSet<OutputTag>>. The value set is empty for executors
  designated only via the untagged WithOutputFrom; contains Intermediate
  (and possibly future tags) otherwise.
* `Workflow.OutputExecutors`: HashSet<string> -> Dictionary<string,
  HashSet<OutputTag>>.
* `OutputFilter.CanOutput`: `Contains(id)` -> `ContainsKey(id)`.
* `WorkflowInfo.OutputExecutorIds`: HashSet<string> -> Dictionary<
  string, HashSet<OutputTag>>, with a custom JsonConverter that reads
  both the new map shape (`{id: ["intermediate", ...]}`) and the legacy
  array shape (`[id1, id2]`, where each id is treated as an untagged
  output). Always writes the map shape. IsMatch updated to compare
  per-id tag sets.

Tests landing in this commit (per the test-with-feature principle):
* `OutputTagTests.cs` (6 tests): KnownValues, EqualityIsOrdinalOnValue,
  DefaultStructValueIsDistinct (default(OutputTag) does not collide
  with the Intermediate singleton in a HashSet),
  GetHashCodeMatchesEquals, JsonConverter_RoundtripsValueAsString,
  ConstructorIsInternal (reflection-based assertion that the (string)
  ctor is `internal`).
* `WorkflowBuilderTests.cs` adds 7 new tests pinning the builder
  API contract: RegistersWithEmptyTagSet, AddsIntermediateTag,
  MultipleExecutorsAllUntagged, ThenIntermediate_AccumulatesTags,
  RepeatedDedupes, OnlyRegistersWithoutPriorWithOutputFrom,
  TracksExecutorBinding.
* `BackwardsCompatibility/JsonCheckpointSerializationTests.cs`
  (new folder + file, 5 tests): event-level ctor contract tests
  (single-tag, no-tag, multi-tag — the last with a custom tag);
  IsIntermediate() asserted; load-bearing JSON BC tests for
  `WorkflowInfo.OutputExecutorIds` -
  `WorkflowOutputExecutorsReadsLegacyArrayShape` (legacy ids map to
  empty tag sets) and `WorkflowOutputExecutorsWritesMapShape`.

The plan's three JSON round-trip tests for `WorkflowOutputEvent.Tags`
were dropped: `WorkflowEvent` is not currently a serialized checkpoint
shape (see the comment in WorkflowsJsonUtilities.cs about events not
being persisted), so there is no real back-compat surface to pin
through JSON. They are substituted with in-process ctor/property
round-trip tests that exercise the `Tags` / `HasTag` / `IsIntermediate`
contract.

Validation: full `Microsoft.Agents.AI.Workflows.UnitTests` suite runs
green on net10.0 (565 passing, 0 failing). Core library builds clean
on net472, netstandard2.0, net8.0, net9.0, and net10.0. Test project
builds clean on net472 + net10.0.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat: route AgentResponse(Update) through the output filter under a Futures flag

`InProcessRunnerContext.YieldOutputAsync` historically special-cased AgentResponse and
AgentResponseUpdate payloads: it built the typed event subclass and emitted it directly,
bypassing the output filter. Rewrites the method so that:

- When `Futures.EnableAgentResponseOutputTaggingAndFiltering` is `false` (the current
  default), AgentResponse(Update) keep the legacy bypass — emitted as
  AgentResponseEvent / AgentResponseUpdateEvent with no tags. Existing callers see no
  behavior change.
- When the flag is `true`, AIAgent payloads flow through the output filter just like
  every other payload type: undesignated sources are dropped, and the emitted event
  carries the source's tag set (empty for terminal `WithOutputFrom`, `{Intermediate}`
  for `WithIntermediateOutputFrom`, the set union when both designations apply).

Non-AIAgent (POCO) outputs also now carry the source's tag set on the emitted
WorkflowOutputEvent unconditionally — additive, since no existing assertion inspected
Tags. Subclass events (`AgentResponseEvent` / `AgentResponseUpdateEvent`) continue to
be emitted under both modes so `switch (evt) { case AgentResponseEvent: ... }`
consumer code keeps matching.

Adds `OutputFilter.TryGetTags` as the tag-aware lookup used by the runner.
`OutputFilter.CanOutput` is kept (still used by the existing sync tests in
`OutputFilterTests.cs`).

Tests
-----
- `Futures/Futures.AgentResponseOutputFilteringAndTaggingTests.cs` (new): the F1–F13
  matrix from the plan, covering every combination of `(flag on/off) × (designation)
  × (payload shape)`. Uses a `FuturesScope` IDisposable + a `FuturesSerial` xUnit
  collection (DisableParallelization = true) to keep the process-global flag from
  leaking across parallel tests.
- `OutputFilterTests.cs`: four new `Test_OutputFilter_…` cases for the `TryGetTags`
  surface (empty-tag-set for terminal designation, `{Intermediate}` for intermediate
  designation, union for accumulated designation, `false` for unregistered).

582/582 unit tests pass on net10.0 (565 baseline + 17 new).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat: tag-aware defaults and designation API on orchestration builders

Aligns the .NET orchestration builders with Python's output / intermediate-output
distinction. Each builder either applies a Python-aligned default designation set or
replays the user's explicit `WithOutputFrom` / `WithIntermediateOutputFrom` calls,
never both.

Static `AgentWorkflowBuilder.BuildSequential` / `BuildConcurrent` apply defaults
unconditionally (no user-facing fluent surface to take control through):

- Sequential: terminal `end` + every agent designated intermediate.
- Concurrent: terminal `end` + every agent and per-agent accumulator designated
  intermediate.

The three fluent instance builders memoize agent-typed designation calls in a
`Dictionary<AIAgent, HashSet<OutputTag>>` (empty set = terminal-only, non-empty =
intermediate tag(s)) so repeated calls dedupe naturally. They replay the entries
at `Build()` time, suppressing defaults when any call has been made:

- `HandoffWorkflowBuilder` / `HandoffWorkflowBuilderCore<TBuilder>` (also picked up
  by the obsolete `HandoffsWorkflowBuilder` via inheritance).
  Default: terminal `HandoffEnd` + every handoff agent intermediate.
  (Bug fix: legacy code relied on `WithOutputFrom(end)` to bind `HandoffEnd`. The
  new explicit-designation path bypasses that, so `Build()` now calls
  `BindExecutor(end)` unconditionally to keep validation happy.)
- `GroupChatWorkflowBuilder` — default: terminal host + every participant intermediate.
- `MagenticWorkflowBuilder` — default: terminal orchestrator + every team member
  intermediate.

Designating a non-participant agent throws `InvalidOperationException`.

The bare `WorkflowBuilder` default is unchanged — only the orchestration-style
builders gain implicit defaults, matching the plan's non-goal.

Tests
-----
- `AgentWorkflowBuilder.SequentialTests` / `.ConcurrentTests`: one default-spec
  assertion each.
- `GroupChatWorkflowBuilderTests`: defaults-match-spec, explicit-replaces-defaults,
  non-participant throws.
- `HandoffWorkflowBuilderTests` (new file): same three.
- `MagenticWorkflowBuilderTests` (new file): same three.

593/593 unit tests pass on net10.0 (582 baseline + 11 new).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat: WorkflowHostAgent forwards AgentResponseEvent unconditionally under Futures-on

Aligns the .NET Workflow-as-Agent surface with Python `as_agent`. Under
`Futures.EnableAgentResponseOutputTaggingAndFiltering = true`,
`WorkflowSession.InvokeStageAsync` now forwards `AgentResponseEvent`
unconditionally — joining `AgentResponseUpdateEvent` in ignoring the host's
`includeWorkflowOutputsInResponse` switch. That switch keeps governing the
generic `WorkflowOutputEvent` path for non-AIAgent payloads, where it is
further short-circuited by an `IsIntermediate()` check (tagged intermediate
outputs always surface).

Under Futures-off the legacy asymmetry is preserved: `AgentResponseUpdateEvent`
always forwarded, `AgentResponseEvent` gated by `includeWorkflowOutputsInResponse`.

Back-compat: with `Futures.EnableAgentResponseOutputTaggingAndFiltering` left at
its default `false`, observable behavior is identical to before.

`Futures` documentation gains a remark explaining the `Workflow.AsAIAgent()`
interaction in both flag states.

Runner fix
----------
`InProcessRunnerContext.YieldOutputAsync` now skips `Executor.CanOutput` for
AgentResponse-shaped payloads under both Futures branches. `AIAgentHostExecutor`
doesn't declare AgentResponse(Update) in its `Yields` set, so the historical
legacy bypass had silently skipped the check; Phase 3's Futures-on path was
running it and would reject AIAgent payloads. AIAgent-shaped payloads are now
always a valid output shape, matching the legacy bypass semantics.

Phase 4 follow-on
-----------------
Switched the three orchestration-builder designation-replay loops to iterate
`Dictionary.Keys` with a value lookup instead of constructing/destructuring
`KeyValuePair<,>`. Cleaner shape and avoids the netstandard2.0 / net472
`KeyValuePair<,>.Deconstruct` unavailability that surfaced when this branch
multi-TFM-built.

Tests
-----
`WorkflowHostSmokeTests.IntermediateForwarding` (new nested class, 6 tests):
- intermediate AgentResponse forwarded past the include-outputs gate (Futures on)
- terminal AgentResponse forwarded unconditionally (Futures on)
- terminal AgentResponse gated by include flag (Futures off, legacy)
- undesignated AIAgent executor emits no AgentResponseEvent under Futures-on
- legacy bypass still emits AgentResponseEvent under Futures-off
- intermediate tag is observable via `update.RawRepresentation`

The class joins the `FuturesSerial` xUnit collection so the process-global flag
is serialized against other Futures-toggling tests.

599/599 unit tests pass on net10.0 (593 baseline + 6 new).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat: SequentialWorkflowBuilder and ConcurrentWorkflowBuilder, OrchestrationBuilderBase

Promotes the Sequential and Concurrent orchestration shapes to first-class fluent
builder classes, matching Handoff / GroupChat / Magentic. Users can call
`WithOutputFrom(agents)` / `WithIntermediateOutputFrom(agents)` to control which
agents are designated output / intermediate sources; when no designation call is
made, the Python-aligned defaults apply (terminal aggregator output + every agent
intermediate; Concurrent also tags per-agent accumulators).

`AgentWorkflowBuilder.BuildSequential(...)` and `BuildConcurrent(...)` are kept
and now delegate to the new builders; observable behavior unchanged. Five static
factories now mirror each other:

- `AgentWorkflowBuilder.CreateSequentialBuilderWith(params IEnumerable<AIAgent>)`
- `AgentWorkflowBuilder.CreateConcurrentBuilderWith(params IEnumerable<AIAgent>)`
- `AgentWorkflowBuilder.CreateHandoffBuilderWith(AIAgent)`        (already existed)
- `AgentWorkflowBuilder.CreateGroupChatBuilderWith(Func<...>)`    (already existed)
- `AgentWorkflowBuilder.CreateMagenticBuilderWith(AIAgent)`       (new)

OrchestrationBuilderBase
------------------------
New abstract `OrchestrationBuilderBase<TBuilder>` unifies the shared fluent
surface across all five orchestration builders: `WithName`, `WithDescription`,
`WithOutputFrom`, `WithIntermediateOutputFrom`, and the
`ApplyOutputDesignations(builder, agentMap, kind, applyDefaults)` helper that
either replays the user's designations or invokes the orchestration-specific
defaults.

Removes ~150 LOC of duplicated designation-management code from the four
non-Handoff builders, plus the equivalent from `HandoffWorkflowBuilderCore`.

Tests
-----
- New `SequentialWorkflowBuilderTests.cs` / `ConcurrentWorkflowBuilderTests.cs`
  (replace the old `AgentWorkflowBuilder.{Sequential,Concurrent}Tests.cs`
  nested-class files). Method names normalized to
  `Test_<BuilderType>_<Scenario>[Async]`.
- Shared helpers (`DoubleEchoAgent`, `DoubleEchoAgentWithBarrier`,
  `WorkflowRunResult`, `RunWorkflow*`) moved from the old
  `AgentWorkflowBuilderTests` partial class into a new
  `OrchestrationTestHelpers` static class in `OrchestrationTestHelpers.cs`.
  Downstream test files (Group Chat, Handoff, Sequential, Concurrent) updated
  to qualify with `OrchestrationTestHelpers.*`.
- A new `AgentWorkflowBuilderTests.cs` covers the static surface directly:
  `BuildSequential` / `BuildConcurrent` invariants and aggregator wiring, plus
  null-rejection + round-trip checks for every `Create*BuilderWith` factory.
- New AsAgent intermediate-suppression tests on a nested `AsAgentForwarding`
  class for each of Sequential and Concurrent: build with only the terminal
  agent designated via `WithOutputFrom`, run via `AsAIAgent(...)`, assert via
  `AgentResponseUpdate.AuthorName` that intermediate agents do not surface.
  Both join the `FuturesSerial` collection.
- New `Test_<Builder>_WithDescriptionPropagatesToWorkflow` smoke tests on
  Sequential and Concurrent (newly available via the base class).

625/625 unit tests pass on net10.0.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: dotnet format

* fixup: encoding

* fixup: charset

* fixup: Updates for PR feedback

* fixup: format

* fixup: merge issue

* Fix intermediate filtering on .AsAgent()

* fix filter logic

* fix: Revert logic change and add comments

---------

Co-authored-by: Jacob Alber <jalber@lokitoth.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-28 21:26:31 +00:00
b000a2cf51 Python: Adding AgentFileStore and FileAccessProvider to support file access operations. (#6099)
* Adding AgentFileStore and FileAccessProvider to support file ased operations for agents.

* Address PR review feedback on FileAccessProvider

- Probe symlinks on the unresolved candidate path so in-root symlinks
  cannot silently pass and out-of-root symlinks surface the correct
  error message.
- Validate matching_lines elements in FileSearchResult.from_dict and
  raise a clean ValueError for non-mapping entries.
- Cap search regex pattern length (256 chars) via a new
  _compile_search_regex helper to mitigate ReDoS, and surface the cap
  in the file_access_search_files tool description.
- Skip non-UTF-8 files during filesystem search instead of aborting
  the entire directory walk.
- Replace the module-scope trailing string in the data-processing
  sample with comments to avoid Ruff B018.
- Remove the checked-in working/region_totals.md sample artifact so
  the save flow works from a clean checkout.
- Expand the Windows stdout reconfiguration comment in task_runner.py
  for clarity.
- Add tests for invalid/oversize regex, non-UTF-8 file search, and
  in-root symlink rejection.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix mypy redundant-cast in FileSearchResult.from_dict

Use cast(list[object], ...) instead of cast(list[Any], ...) so the
cast represents a real type change (lists are invariant) and is no
longer flagged by mypy as redundant, while still satisfying pyright's
reportUnknownVariableType. Matches the existing pattern in _memory.py.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Tighten path normalization and directory resolution in FileAccess

- _normalize_relative_path now strips surrounding whitespace up front
  so leading/trailing spaces never leak into file segments, and
  rejects trailing path separators for file paths so 'foo/' is no
  longer silently coerced to 'foo'.
- FileSystemAgentFileStore._resolve_safe_directory_path normalizes
  with is_directory=True and maps an empty normalized result to the
  root. This matches InMemoryAgentFileStore so whitespace-only
  directory inputs resolve to the root instead of raising.
- Added tests for whitespace stripping, trailing-separator rejection,
  and whitespace-only directory listing on the filesystem store.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Harden FileAccess search and atomic save in store API

- Add wall-clock timeout (10s) around regex scans so a pathological pattern (e.g. `(a+)+`) below the length cap cannot stall the event loop.
- Offload the InMemoryAgentFileStore regex scan to a worker thread, matching the filesystem store.
- Fail closed when `Path.is_symlink` raises during the safe-path probe so a permission error cannot silently bypass the symlink/reparse-point rejection.
- Add `overwrite: bool = True` to `AgentFileStore.write_file`; the in-memory store performs the check under the existing lock and the filesystem store uses `open(mode='x')` so concurrent callers cannot race past `overwrite=False`.
- `file_access_save_file` now relies on the atomic store call instead of a separate `file_exists` round-trip.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix Python 3.10 timeout handling and add directory arg to list/search tools

- Catch asyncio.TimeoutError in _run_search_with_timeout. In Python 3.10
  asyncio.wait_for raises asyncio.exceptions.TimeoutError, which is
  distinct from the builtin TimeoutError (the two were unified in 3.11).
  Catching the asyncio alias works on every supported version.
- Add an optional directory parameter to file_access_list_files and
  file_access_search_files so agents can enumerate / scope searches to
  nested folders, not just the store root.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address FileAccess review feedback: case, errors, signal, TOCTOU

- InMemoryAgentFileStore now stores (display_name, content) so list_files
  and search_files return the original-case names callers wrote, matching
  the behaviour of FileSystemAgentFileStore on case-preserving filesystems
  and removing the silent in-memory vs. on-disk contract divergence.
- FileSystemAgentFileStore.read_file raises ValueError instead of letting
  UnicodeDecodeError bubble for binary / non-UTF-8 input, restoring
  symmetry with search_files (which still skips) and giving the tool
  layer a recoverable type to translate.
- Tool wrappers now catch ValueError and OSError around every operation
  and surface them as readable strings, so 'you used ..' and 'the file
  already exists' are both reported to the model the same way instead of
  the former crashing out as an unhandled exception.
- _search_files_sync logs per skipped non-UTF-8 file at WARNING and an
  aggregate INFO summary so operators can distinguish 'no matches' from
  'half the corpus was unreadable'.
- FileSystemAgentFileStore softens its docstrings to acknowledge the
  inherent probe-then-open TOCTOU window. On POSIX both read and write
  now pass O_NOFOLLOW so the kernel refuses if the leaf segment becomes
  a symlink between the probe and the open. Windows has no equivalent
  flag; the limitation is documented.
- Tests cover: case preservation on list/search, ValueError on non-UTF-8
  read at the store and tool layer, tool-layer string responses for
  path-traversal and oversized-regex inputs, search-skip log output,
  symlink rejection on delete/search/list, and symlinked intermediate
  directory rejection.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address FileAccess nit comments: docstrings, enumerate, opt-in delete approval

- Expand FileSearchMatch/FileSearchResult.to_dict docstrings to explain why
  the override is needed (__slots__ defeats the mixin's __dict__ iteration)
  and why exclude/exclude_none are accepted-but-ignored (mixin signature
  compatibility for callers like to_json).
- Use enumerate(lines, start=1) in _search_file_content so the +1 below is
  no longer needed; rename loop variable to line_number for clarity.
- Add opt-in require_delete_approval: bool = False on FileAccessProvider.
  When True, file_access_delete_file is registered with approval_mode
  'always_require' so the host must approve every delete. Default False
  preserves current behaviour and matches the .NET reference, but
  deployments that want a safer-by-default posture can enable it.
- Add tests covering both delete approval modes.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* FileAccess: require delete approval by default

Flip the default for FileAccessProvider(require_delete_approval=...) from
False to True so destructive deletes are gated by host approval out of the
box. Callers that want the previous autonomous behaviour (which matches the
.NET reference) can pass require_delete_approval=False.

Tests updated accordingly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fixing linkinspector by installing Chrome for puppeteer first.

---------

Co-authored-by: Ben Thomas <25218250+alliscode@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-28 20:09:50 +00:00
Tao ChenandGitHub 0578f4c910 Backfill chat span request model if it's unknown and response model is avaliable (#6160) 2026-05-28 20:03:46 +00:00
e9a606344a Python A2A: Expose supported_protocol_bindings as configurable parameter (#6098)
* Expose supported_protocol_bindings as configurable parameter on A2AAgent

Add supported_protocol_bindings parameter to A2AAgent.__init__() allowing
users to configure which A2A protocol bindings (JSONRPC, GRPC, HTTP+JSON)
the client prefers when connecting to remote agents.

- Defaults to ["JSONRPC"] matching current behavior
- Passes through to ClientConfig for transport negotiation
- Replaces 4 hardcoded references with the configurable value

Closes #6057

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix empty list falsy trap and add fallback path test coverage

- Use 'is not None' check instead of 'or' to preserve explicit empty list
- Add test verifying empty list is not silently replaced with defaults
- Add test verifying fallback path uses custom bindings

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Document known protocol binding values in docstring

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Use Literal union for protocol binding type hint

Provides IDE autocomplete for known values while keeping the type
open for custom bindings (Literal is str at runtime).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-28 19:05:13 +00:00
d2f79930d5 .NET: feat: Update GroupChatManager semantics to match other Orchestration patterns (#6140)
* Refactor group chat workflow to prevent message echoing and enhance checkpointing

- Updated GroupChatWorkflowBuilder to disable forwarding incoming messages to prevent duplicates.
- Enhanced RoundRobinGroupChatManager with checkpointing support to preserve state across executions.
- Modified GroupChatHost to maintain a history of messages and track the current speaker for message broadcasting.
- Implemented broadcasting logic to ensure participants receive messages from others while excluding their own responses.
- Added comprehensive unit tests for group chat orchestration, including scenarios for tool approval and function calls.
- Introduced a new ApprovalHarness for testing tool invocation and approval workflows.

* fixup: format

* Add JSON serialization support for GroupChatManagerState and RoundRobinGroupChatManagerState

---------

Co-authored-by: Jacob Alber <jalber@lokitoth.com>
2026-05-28 18:40:48 +00:00
115 changed files with 10548 additions and 2563 deletions
+163
View File
@@ -0,0 +1,163 @@
// Copyright (c) Microsoft. All rights reserved.
function getPullRequest(context) {
const pullRequest = context.payload.pull_request;
if (!pullRequest?.number || !pullRequest.user?.login) {
throw new Error('This script must be run from a pull_request_target event.');
}
return {
author: pullRequest.user.login,
labels: pullRequest.labels?.map((label) => label.name).filter(Boolean) ?? [],
number: pullRequest.number,
};
}
async function ensureLabel({ github, owner, repo, labelName }) {
try {
await github.rest.issues.getLabel({
owner,
repo,
name: labelName,
});
} catch (error) {
if (error.status !== 404) {
throw error;
}
try {
await github.rest.issues.createLabel({
owner,
repo,
name: labelName,
color: 'd93f0b',
description: 'Community author has exceeded the open pull request limit.',
});
} catch (createError) {
if (createError.status !== 422) {
throw createError;
}
}
}
}
function hasLabel(labels, labelName) {
if (!labelName) {
return false;
}
return labels.some((label) => label.toLowerCase() === labelName.toLowerCase());
}
function buildLimitMessage({ author, exemptLabelName, maxOpenPrs, openPrCount }) {
return [
`Thank you for your contribution, @${author}.`,
'',
`To keep the review queue manageable, we currently limit community contributors to ${maxOpenPrs} `
+ `open pull requests at a time. This PR would put you at ${openPrCount} open pull requests, `
+ 'so we are closing it automatically.',
'',
'Please focus on getting your existing PRs reviewed, merged, or closed before opening another one. '
+ `If a maintainer asked you to open this PR, they can apply the \`${exemptLabelName}\` label and reopen it.`,
].join('\n');
}
async function getOpenPrCount({ github, owner, repo, author, pullRequestNumber }) {
const query = `repo:${owner}/${repo} is:pr is:open author:${author}`;
const response = await github.rest.search.issuesAndPullRequests({
q: query,
per_page: 100,
});
const indexedPrNumbers = response.data.items.map((item) => item.number);
const currentPrIsIndexed = indexedPrNumbers.includes(pullRequestNumber);
if (currentPrIsIndexed || response.data.total_count >= 100) {
return response.data.total_count;
}
return response.data.total_count + 1;
}
async function enforcePrLimit({ github, context, core, exemptLabelName, maxOpenPrs, labelName }) {
const { owner, repo } = context.repo;
const { author, labels, number } = getPullRequest(context);
if (hasLabel(labels, exemptLabelName)) {
core.info(`PR #${number} has the ${exemptLabelName} label; skipping open PR limit enforcement.`);
return {
author,
closed: false,
exempt: true,
openPrCount: null,
};
}
const openPrCount = await getOpenPrCount({
github,
owner,
repo,
author,
pullRequestNumber: number,
});
if (openPrCount <= maxOpenPrs) {
core.info(
`${author} has ${openPrCount} open pull request(s), which is within the limit of ${maxOpenPrs}.`,
);
return {
author,
closed: false,
openPrCount,
};
}
await ensureLabel({
github,
owner,
repo,
labelName,
});
await github.rest.issues.addLabels({
owner,
repo,
issue_number: number,
labels: [labelName],
});
await github.rest.issues.createComment({
owner,
repo,
issue_number: number,
body: buildLimitMessage({
author,
exemptLabelName,
maxOpenPrs,
openPrCount,
}),
});
await github.rest.pulls.update({
owner,
repo,
pull_number: number,
state: 'closed',
});
core.info(
`${author} has ${openPrCount} open pull request(s), which exceeds the limit of ${maxOpenPrs}. `
+ `Closed PR #${number}.`,
);
return {
author,
closed: true,
openPrCount,
};
}
module.exports = {
buildLimitMessage,
enforcePrLimit,
getOpenPrCount,
};
+286
View File
@@ -0,0 +1,286 @@
// Copyright (c) Microsoft. All rights reserved.
/**
* Tests for pr_limit_moderation.js.
*
* Run with: node --test .github/tests/test_pr_limit_moderation.js
*/
const { describe, it } = require('node:test');
const assert = require('node:assert/strict');
const { enforcePrLimit } = require('../scripts/pr_limit_moderation.js');
// ---------------------------------------------------------------------------
// Helpers
// ---------------------------------------------------------------------------
function createContext({ author = 'community-user', labels = [], number = 123 } = {}) {
return {
repo: {
owner: 'microsoft',
repo: 'agent-framework',
},
payload: {
pull_request: {
number,
labels: labels.map((name) => ({ name })),
user: {
login: author,
},
},
},
};
}
function createCore() {
const messages = [];
return {
messages,
info(message) {
messages.push(message);
},
};
}
function createGithub({ totalCount, itemNumbers, labelExists = true }) {
const calls = [];
return {
calls,
rest: {
search: {
async issuesAndPullRequests(params) {
calls.push({ api: 'search.issuesAndPullRequests', params });
return {
data: {
total_count: totalCount,
items: itemNumbers.map((number) => ({ number })),
},
};
},
},
issues: {
async getLabel(params) {
calls.push({ api: 'issues.getLabel', params });
if (!labelExists) {
const error = new Error('Not Found');
error.status = 404;
throw error;
}
return { data: { name: params.name } };
},
async createLabel(params) {
calls.push({ api: 'issues.createLabel', params });
return { data: { name: params.name } };
},
async addLabels(params) {
calls.push({ api: 'issues.addLabels', params });
return { data: [] };
},
async createComment(params) {
calls.push({ api: 'issues.createComment', params });
return { data: { id: 1 } };
},
},
pulls: {
async update(params) {
calls.push({ api: 'pulls.update', params });
return { data: { state: params.state } };
},
},
},
};
}
// ---------------------------------------------------------------------------
// PR limit enforcement
// ---------------------------------------------------------------------------
describe('PR limit enforcement', () => {
it('does not close the PR when the author is at the open PR limit', async () => {
const github = createGithub({
totalCount: 10,
itemNumbers: [1, 2, 3, 4, 5, 6, 7, 8, 9, 123],
});
const result = await enforcePrLimit({
github,
context: createContext(),
core: createCore(),
exemptLabelName: 'pr-limit-exempt',
maxOpenPrs: 10,
labelName: 'too-many-prs',
});
assert.equal(result.closed, false);
assert.equal(result.openPrCount, 10);
assert.deepEqual(
github.calls.map((call) => call.api),
['search.issuesAndPullRequests'],
);
});
it('counts the new PR when search has not indexed it yet', async () => {
const github = createGithub({
totalCount: 10,
itemNumbers: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
});
const result = await enforcePrLimit({
github,
context: createContext(),
core: createCore(),
exemptLabelName: 'pr-limit-exempt',
maxOpenPrs: 10,
labelName: 'too-many-prs',
});
assert.equal(result.closed, true);
assert.equal(result.openPrCount, 11);
assert.deepEqual(
github.calls.map((call) => call.api),
[
'search.issuesAndPullRequests',
'issues.getLabel',
'issues.addLabels',
'issues.createComment',
'pulls.update',
],
);
});
it('creates the label when it does not already exist', async () => {
const github = createGithub({
totalCount: 11,
itemNumbers: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 123],
labelExists: false,
});
const result = await enforcePrLimit({
github,
context: createContext(),
core: createCore(),
exemptLabelName: 'pr-limit-exempt',
maxOpenPrs: 10,
labelName: 'too-many-prs',
});
assert.equal(result.closed, true);
assert.deepEqual(
github.calls.map((call) => call.api),
[
'search.issuesAndPullRequests',
'issues.getLabel',
'issues.createLabel',
'issues.addLabels',
'issues.createComment',
'pulls.update',
],
);
assert.equal(
github.calls.find((call) => call.api === 'issues.createLabel').params.name,
'too-many-prs',
);
});
it('tolerates a 422 race when creating the label', async () => {
const github = createGithub({
totalCount: 11,
itemNumbers: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 123],
labelExists: false,
});
github.rest.issues.createLabel = async (params) => {
github.calls.push({ api: 'issues.createLabel', params });
const error = new Error('Validation Failed');
error.status = 422;
throw error;
};
const result = await enforcePrLimit({
github,
context: createContext(),
core: createCore(),
exemptLabelName: 'pr-limit-exempt',
maxOpenPrs: 10,
labelName: 'too-many-prs',
});
assert.equal(result.closed, true);
assert.deepEqual(
github.calls.map((call) => call.api),
[
'search.issuesAndPullRequests',
'issues.getLabel',
'issues.createLabel',
'issues.addLabels',
'issues.createComment',
'pulls.update',
],
);
});
it('uses a diplomatic close message with the configured limit', async () => {
const github = createGithub({
totalCount: 11,
itemNumbers: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 123],
});
await enforcePrLimit({
github,
context: createContext({ author: 'octo-contributor' }),
core: createCore(),
exemptLabelName: 'pr-limit-exempt',
maxOpenPrs: 10,
labelName: 'too-many-prs',
});
const comment = github.calls.find((call) => call.api === 'issues.createComment').params.body;
assert.match(comment, /Thank you for your contribution/);
assert.match(comment, /limit community contributors to 10 open pull requests/);
assert.match(comment, /@octo-contributor/);
assert.match(comment, /`pr-limit-exempt` label and reopen/);
});
it('does not close an exempt PR when it is reopened', async () => {
const github = createGithub({
totalCount: 11,
itemNumbers: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 123],
});
const result = await enforcePrLimit({
github,
context: createContext({ labels: ['PR-LIMIT-EXEMPT'] }),
core: createCore(),
exemptLabelName: 'pr-limit-exempt',
maxOpenPrs: 10,
labelName: 'too-many-prs',
});
assert.equal(result.closed, false);
assert.equal(result.exempt, true);
assert.equal(result.openPrCount, null);
assert.deepEqual(github.calls, []);
});
it('does not over-count when the current PR is not on the first search page', async () => {
const github = createGithub({
totalCount: 101,
itemNumbers: Array.from({ length: 100 }, (_, index) => index + 1),
});
const result = await enforcePrLimit({
github,
context: createContext({ number: 123 }),
core: createCore(),
exemptLabelName: 'pr-limit-exempt',
maxOpenPrs: 10,
labelName: 'too-many-prs',
});
assert.equal(result.closed, true);
assert.equal(result.openPrCount, 101);
});
});
+83
View File
@@ -0,0 +1,83 @@
name: Limit community pull requests
on:
pull_request_target:
types: [opened, reopened]
permissions:
contents: read
issues: write
pull-requests: write
concurrency:
group: pr-limit-${{ github.repository }}-${{ github.event.pull_request.user.login }}
cancel-in-progress: false
env:
MAX_OPEN_PULL_REQUESTS: '10'
PR_LIMIT_EXEMPT_LABEL: pr-limit-exempt
TOO_MANY_PRS_LABEL: too-many-prs
jobs:
team_check:
runs-on: ubuntu-latest
outputs:
is_team_member: ${{ steps.check.outputs.is_team_member }}
steps:
- name: Checkout scripts
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
sparse-checkout: .github/scripts
fetch-depth: 1
persist-credentials: false
- name: Check PR author team membership
id: check
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8
env:
TEAM_NAME: ${{ secrets.DEVELOPER_TEAM }}
PR_NUMBER: ${{ github.event.pull_request.number }}
with:
github-token: ${{ secrets.GH_ACTIONS_PR_WRITE }}
script: |
const checkTeamMembership = require('./.github/scripts/check_team_membership.js');
const { author, isTeamMember } = await checkTeamMembership({
github,
context,
core,
teamSlug: process.env.TEAM_NAME,
issueNumber: process.env.PR_NUMBER,
});
core.setOutput('is_team_member', isTeamMember ? 'true' : 'false');
if (isTeamMember) {
core.info(`Author ${author} is a team member; skipping open PR limit.`);
} else {
core.info(`Author ${author} is not a team member; checking open PR limit.`);
}
limit_open_prs:
runs-on: ubuntu-latest
needs: team_check
if: ${{ needs.team_check.outputs.is_team_member == 'false' }}
steps:
- name: Checkout scripts
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
sparse-checkout: .github/scripts
fetch-depth: 1
persist-credentials: false
- name: Enforce open PR limit
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8
with:
github-token: ${{ secrets.GH_ACTIONS_PR_WRITE }}
script: |
const { enforcePrLimit } = require('./.github/scripts/pr_limit_moderation.js');
await enforcePrLimit({
github,
context,
core,
exemptLabelName: process.env.PR_LIMIT_EXEMPT_LABEL,
maxOpenPrs: Number.parseInt(process.env.MAX_OPEN_PULL_REQUESTS, 10),
labelName: process.env.TOO_MANY_PRS_LABEL,
});
@@ -23,6 +23,14 @@ jobs:
with:
persist-credentials: false
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: 20
- name: Install Chrome for Puppeteer
run: npx puppeteer browsers install chrome
# Checks the status of hyperlinks in all files
- name: Run linkspector
uses: umbrelladocs/action-linkspector@963b6264d7de32c904942a70b488d3407453049e # v1
+5 -5
View File
@@ -22,14 +22,14 @@
<PackageVersion Include="Aspire.Microsoft.Azure.Cosmos" Version="$(AspireAppHostSdkVersion)" />
<PackageVersion Include="CommunityToolkit.Aspire.OllamaSharp" Version="13.0.0" />
<!-- Azure.* -->
<PackageVersion Include="Azure.AI.AgentServer.Core" Version="1.0.0-beta.23" />
<PackageVersion Include="Azure.AI.AgentServer.Invocations" Version="1.0.0-beta.3" />
<PackageVersion Include="Azure.AI.AgentServer.Responses" Version="1.0.0-beta.4" />
<PackageVersion Include="Azure.AI.AgentServer.Core" Version="1.0.0-beta.25" />
<PackageVersion Include="Azure.AI.AgentServer.Invocations" Version="1.0.0-beta.4" />
<PackageVersion Include="Azure.AI.AgentServer.Responses" Version="1.0.0-beta.5" />
<PackageVersion Include="Azure.Search.Documents" Version="12.0.0" />
<PackageVersion Include="Azure.AI.Projects" Version="2.1.0-beta.2" />
<PackageVersion Include="Azure.AI.Agents.Persistent" Version="1.2.0-beta.10" />
<PackageVersion Include="Azure.AI.OpenAI" Version="2.9.0-beta.1" />
<PackageVersion Include="Azure.Core" Version="1.55.0" />
<PackageVersion Include="Azure.Core" Version="1.56.0" />
<PackageVersion Include="Azure.Identity" Version="1.21.0" />
<PackageVersion Include="DotNetEnv" Version="3.1.1" />
<PackageVersion Include="Azure.Monitor.OpenTelemetry.Exporter" Version="1.5.0" />
@@ -44,7 +44,7 @@
<PackageVersion Include="Microsoft.Bcl.AsyncInterfaces" Version="10.0.6" />
<PackageVersion Include="Microsoft.Bcl.HashCode" Version="6.0.0" />
<PackageVersion Include="Microsoft.Bcl.Memory" Version="10.0.5" />
<PackageVersion Include="System.ClientModel" Version="1.11.0" />
<PackageVersion Include="System.ClientModel" Version="1.12.0" />
<PackageVersion Include="System.CodeDom" Version="10.0.0" />
<PackageVersion Include="System.Collections.Immutable" Version="10.0.1" />
<PackageVersion Include="System.CommandLine" Version="2.0.0-rc.2.25502.107" />
@@ -13,8 +13,10 @@
<ItemGroup>
<PackageReference Include="Azure.AI.Projects" />
<PackageReference Include="Azure.Core" />
<PackageReference Include="Azure.Identity" />
<PackageReference Include="DotNetEnv" />
<PackageReference Include="System.ClientModel" />
</ItemGroup>
<ItemGroup>
@@ -13,8 +13,10 @@
<ItemGroup>
<PackageReference Include="Azure.AI.Projects" />
<PackageReference Include="Azure.Core" />
<PackageReference Include="Azure.Identity" />
<PackageReference Include="DotNetEnv" />
<PackageReference Include="System.ClientModel" />
</ItemGroup>
<ItemGroup>
@@ -297,7 +297,7 @@ public class AgentResponse
AgentId = this.AgentId,
ResponseId = this.ResponseId,
MessageId = message.MessageId,
CreatedAt = this.CreatedAt,
CreatedAt = message.CreatedAt ?? this.CreatedAt,
};
}
@@ -75,18 +75,14 @@ internal sealed class InvokeMcpToolExecutor(
if (requireApproval)
{
// Create tool call content for approval request
// Create tool call content for approval request.
// Transport headers (e.g. Authorization) are intentionally excluded from the
// approval event: they must not cross into the externally-surfaced approval request.
McpServerToolCallContent toolCall = new(this.Id, toolName, serverLabel ?? serverUrl)
{
Arguments = arguments
};
if (headers != null)
{
toolCall.AdditionalProperties ??= [];
toolCall.AdditionalProperties.Add(headers);
}
ToolApprovalRequestContent approvalRequest = new(this.Id, toolCall);
ChatMessage requestMessage = new(ChatRole.Assistant, [approvalRequest]);
@@ -1,5 +1,6 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Collections.Generic;
using Microsoft.Shared.Diagnostics;
namespace Microsoft.Agents.AI.Workflows;
@@ -19,6 +20,28 @@ public sealed class AgentResponseEvent : WorkflowOutputEvent
this.Response = Throw.IfNull(response);
}
/// <summary>
/// Initializes a new instance of the <see cref="AgentResponseEvent"/> class with the given output tag.
/// </summary>
/// <param name="executorId">The identifier of the executor that generated this event.</param>
/// <param name="response">The agent response.</param>
/// <param name="tag">The output tag to associate with this event.</param>
public AgentResponseEvent(string executorId, AgentResponse response, OutputTag tag) : base(response, executorId, tag)
{
this.Response = Throw.IfNull(response);
}
/// <summary>
/// Initializes a new instance of the <see cref="AgentResponseEvent"/> class with the given output tags.
/// </summary>
/// <param name="executorId">The identifier of the executor that generated this event.</param>
/// <param name="response">The agent response.</param>
/// <param name="tags">The output tags to associate with this event. May be <see langword="null"/> or empty.</param>
public AgentResponseEvent(string executorId, AgentResponse response, IEnumerable<OutputTag>? tags) : base(response, executorId, tags)
{
this.Response = Throw.IfNull(response);
}
/// <summary>
/// Gets the agent response.
/// </summary>
@@ -20,6 +20,28 @@ public sealed class AgentResponseUpdateEvent : WorkflowOutputEvent
this.Update = Throw.IfNull(update);
}
/// <summary>
/// Initializes a new instance of the <see cref="AgentResponseUpdateEvent"/> class with the given output tag.
/// </summary>
/// <param name="executorId">The identifier of the executor that generated this event.</param>
/// <param name="update">The agent run response update.</param>
/// <param name="tag">The output tag to associate with this event.</param>
public AgentResponseUpdateEvent(string executorId, AgentResponseUpdate update, OutputTag tag) : base(update, executorId, tag)
{
this.Update = Throw.IfNull(update);
}
/// <summary>
/// Initializes a new instance of the <see cref="AgentResponseUpdateEvent"/> class with the given output tags.
/// </summary>
/// <param name="executorId">The identifier of the executor that generated this event.</param>
/// <param name="update">The agent run response update.</param>
/// <param name="tags">The output tags to associate with this event. May be <see langword="null"/> or empty.</param>
public AgentResponseUpdateEvent(string executorId, AgentResponseUpdate update, IEnumerable<OutputTag>? tags) : base(update, executorId, tags)
{
this.Update = Throw.IfNull(update);
}
/// <summary>
/// Gets the agent run response update.
/// </summary>
@@ -2,10 +2,6 @@
using System;
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
using System.Linq;
using System.Threading.Tasks;
using Microsoft.Agents.AI.Workflows.Specialized;
using Microsoft.Extensions.AI;
using Microsoft.Shared.Diagnostics;
@@ -37,31 +33,10 @@ public static partial class AgentWorkflowBuilder
{
Throw.IfNullOrEmpty(agents);
// Create a builder that chains the agents together in sequence. The workflow simply begins
// with the first agent in the sequence.
AIAgentHostOptions options = new()
{
ReassignOtherAgentsAsUsers = true,
ForwardIncomingMessages = true,
};
List<ExecutorBinding> agentExecutors = agents.Select(agent => agent.BindAsExecutor(options)).ToList();
ExecutorBinding previous = agentExecutors[0];
WorkflowBuilder builder = new(previous);
foreach (ExecutorBinding next in agentExecutors.Skip(1))
{
builder.AddEdge(previous, next);
previous = next;
}
OutputMessagesExecutor end = new();
builder = builder.AddEdge(previous, end).WithOutputFrom(end);
SequentialWorkflowBuilder builder = new(agents);
if (workflowName is not null)
{
builder = builder.WithName(workflowName);
builder.WithName(workflowName);
}
return builder.Build();
}
@@ -107,41 +82,14 @@ public static partial class AgentWorkflowBuilder
{
Throw.IfNull(agents);
// A workflow needs a starting executor, so we create one that forwards everything to each agent.
ChatForwardingExecutor start = new("Start");
WorkflowBuilder builder = new(start);
// For each agent, we create an executor to host it and an accumulator to batch up its output messages,
// so that the final accumulator receives a single list of messages from each agent. Otherwise, the
// accumulator would not be able to determine what came from what agent, as there's currently no
// provenance tracking exposed in the workflow context passed to a handler.
ExecutorBinding[] agentExecutors = (from agent in agents
select agent.BindAsExecutor(new AIAgentHostOptions() { ReassignOtherAgentsAsUsers = true })).ToArray();
ExecutorBinding[] accumulators = [.. from agent in agentExecutors select (ExecutorBinding)new AggregateTurnMessagesExecutor($"Batcher/{agent.Id}")];
builder.AddFanOutEdge(start, agentExecutors);
for (int i = 0; i < agentExecutors.Length; i++)
{
builder.AddEdge(agentExecutors[i], accumulators[i]);
}
// Create the accumulating executor that will gather the results from each agent, and connect
// each agent's accumulator to it. If no aggregation function was provided, we default to returning
// the last message from each agent
aggregator ??= static lists => (from list in lists where list.Count > 0 select list.Last()).ToList();
Func<string, string, ValueTask<ConcurrentEndExecutor>> endFactory =
(_, __) => new(new ConcurrentEndExecutor(agentExecutors.Length, aggregator));
ExecutorBinding end = endFactory.BindExecutor(ConcurrentEndExecutor.ExecutorId);
builder.AddFanInBarrierEdge(accumulators, end);
builder = builder.WithOutputFrom(end);
ConcurrentWorkflowBuilder builder = new(agents);
if (workflowName is not null)
{
builder = builder.WithName(workflowName);
builder.WithName(workflowName);
}
if (aggregator is not null)
{
builder.WithAggregator(aggregator);
}
return builder.Build();
}
@@ -155,7 +103,6 @@ public static partial class AgentWorkflowBuilder
/// The <see cref="AIAgent"/> must be capable of understanding those <see cref="AgentRunOptions"/> provided. If the agent
/// ignores the tools or is otherwise unable to advertize them to the underlying provider, handoffs will not occur.
/// </remarks>
[Experimental(DiagnosticConstants.ExperimentalFeatureDiagnostic)]
public static HandoffWorkflowBuilder CreateHandoffBuilderWith(AIAgent initialAgent)
{
Throw.IfNull(initialAgent);
@@ -179,4 +126,31 @@ public static partial class AgentWorkflowBuilder
Throw.IfNull(managerFactory);
return new GroupChatWorkflowBuilder(managerFactory);
}
/// <summary>Creates a new <see cref="SequentialWorkflowBuilder"/> with the given pipeline of <paramref name="agents"/>.</summary>
/// <param name="agents">The sequence of agents to compose into a sequential workflow.</param>
/// <returns>The builder for creating a sequential workflow.</returns>
public static SequentialWorkflowBuilder CreateSequentialBuilderWith(params IEnumerable<AIAgent> agents)
{
Throw.IfNull(agents);
return new SequentialWorkflowBuilder(agents);
}
/// <summary>Creates a new <see cref="ConcurrentWorkflowBuilder"/> with the given participating <paramref name="agents"/>.</summary>
/// <param name="agents">The set of agents to compose into a concurrent workflow.</param>
/// <returns>The builder for creating a concurrent workflow.</returns>
public static ConcurrentWorkflowBuilder CreateConcurrentBuilderWith(params IEnumerable<AIAgent> agents)
{
Throw.IfNull(agents);
return new ConcurrentWorkflowBuilder(agents);
}
/// <summary>Creates a new <see cref="MagenticWorkflowBuilder"/> with the given <paramref name="managerAgent"/>.</summary>
/// <param name="managerAgent">The LLM-powered manager agent that coordinates the team.</param>
/// <returns>The builder for creating a Magentic workflow.</returns>
public static MagenticWorkflowBuilder CreateMagenticBuilderWith(AIAgent managerAgent)
{
Throw.IfNull(managerAgent);
return new MagenticWorkflowBuilder(managerAgent);
}
}
@@ -1,5 +1,6 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text.Json.Serialization;
@@ -15,14 +16,14 @@ internal sealed class WorkflowInfo
Dictionary<string, List<EdgeInfo>> edges,
HashSet<RequestPortInfo> requestPorts,
string startExecutorId,
HashSet<string>? outputExecutorIds)
Dictionary<string, HashSet<OutputTag>>? outputExecutorIds)
{
this.Executors = Throw.IfNull(executors);
this.Edges = Throw.IfNull(edges);
this.RequestPorts = Throw.IfNull(requestPorts);
this.StartExecutorId = Throw.IfNullOrEmpty(startExecutorId);
this.OutputExecutorIds = outputExecutorIds ?? [];
this.OutputExecutorIds = outputExecutorIds ?? new Dictionary<string, HashSet<OutputTag>>(StringComparer.Ordinal);
}
public Dictionary<string, ExecutorInfo> Executors { get; }
@@ -32,7 +33,15 @@ internal sealed class WorkflowInfo
public TypeId? InputType { get; }
public string StartExecutorId { get; }
public HashSet<string> OutputExecutorIds { get; }
/// <summary>
/// Map of executor id to the set of <see cref="OutputTag"/>s under which the executor is registered.
/// An empty set means the executor is registered as a regular (untagged) output source.
/// JSON shape: <c>{ "executorId": ["intermediate"], ... }</c>. Legacy payloads using the
/// older <c>string[]</c> shape are read by <see cref="WorkflowInfoOutputExecutorsConverter"/> and
/// each id is treated as registered with an empty tag set.
/// </summary>
[JsonConverter(typeof(WorkflowInfoOutputExecutorsConverter))]
public Dictionary<string, HashSet<OutputTag>> OutputExecutorIds { get; }
public bool IsMatch(Workflow workflow)
{
@@ -80,9 +89,12 @@ internal sealed class WorkflowInfo
return false;
}
// Validate the outputs
// Validate the outputs (key set + tag set per id must match)
if (workflow.OutputExecutors.Count != this.OutputExecutorIds.Count ||
this.OutputExecutorIds.Any(id => !workflow.OutputExecutors.Contains(id)))
this.OutputExecutorIds.Any(kvp =>
!workflow.OutputExecutors.TryGetValue(kvp.Key, out HashSet<OutputTag>? tags) ||
tags.Count != kvp.Value.Count ||
!tags.SetEquals(kvp.Value)))
{
return false;
}
@@ -0,0 +1,122 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Text.Json;
using System.Text.Json.Serialization;
namespace Microsoft.Agents.AI.Workflows.Checkpointing;
/// <summary>
/// JSON converter for <see cref="WorkflowInfo.OutputExecutorIds"/> that supports both the new
/// map shape (<c>{ "id": ["intermediate"] }</c>) and the legacy array shape
/// (<c>["id1", "id2"]</c>). Legacy-shaped payloads are read as if every id had been registered
/// as a regular (untagged) output source; output is always written in the new map shape.
/// </summary>
internal sealed class WorkflowInfoOutputExecutorsConverter : JsonConverter<Dictionary<string, HashSet<OutputTag>>>
{
public override Dictionary<string, HashSet<OutputTag>> Read(
ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
{
Dictionary<string, HashSet<OutputTag>> result = new(StringComparer.Ordinal);
if (reader.TokenType == JsonTokenType.Null)
{
return result;
}
if (reader.TokenType == JsonTokenType.StartArray)
{
// Legacy shape: a flat array of executor ids. Treat each as a registered
// (untagged) output executor.
while (reader.Read())
{
if (reader.TokenType == JsonTokenType.EndArray)
{
return result;
}
if (reader.TokenType != JsonTokenType.String)
{
throw new JsonException($"Expected a string in legacy outputExecutorIds array, got {reader.TokenType}.");
}
string id = reader.GetString()!;
result[id] = [];
}
throw new JsonException("Unexpected end of legacy outputExecutorIds array.");
}
if (reader.TokenType != JsonTokenType.StartObject)
{
throw new JsonException($"Expected object or array for outputExecutorIds, got {reader.TokenType}.");
}
while (reader.Read())
{
if (reader.TokenType == JsonTokenType.EndObject)
{
return result;
}
if (reader.TokenType != JsonTokenType.PropertyName)
{
throw new JsonException($"Expected property name in outputExecutorIds object, got {reader.TokenType}.");
}
string id = reader.GetString()!;
reader.Read();
HashSet<OutputTag> tags = [];
if (reader.TokenType == JsonTokenType.StartArray)
{
while (reader.Read() && reader.TokenType != JsonTokenType.EndArray)
{
if (reader.TokenType != JsonTokenType.String)
{
throw new JsonException($"Expected a string tag, got {reader.TokenType}.");
}
tags.Add(ReadTag(reader.GetString()!));
}
}
else
{
throw new JsonException($"Expected array of tags for outputExecutorIds[{id}], got {reader.TokenType}.");
}
result[id] = tags;
}
throw new JsonException("Unexpected end of outputExecutorIds object.");
}
private static OutputTag ReadTag(string value)
{
if (string.Equals(value, OutputTag.Intermediate.Value, StringComparison.Ordinal))
{
return OutputTag.Intermediate;
}
return new OutputTag(value);
}
public override void Write(
Utf8JsonWriter writer,
Dictionary<string, HashSet<OutputTag>> value,
JsonSerializerOptions options)
{
writer.WriteStartObject();
foreach (KeyValuePair<string, HashSet<OutputTag>> kvp in value)
{
writer.WritePropertyName(kvp.Key);
writer.WriteStartArray();
foreach (OutputTag tag in kvp.Value)
{
writer.WriteStringValue(tag.Value);
}
writer.WriteEndArray();
}
writer.WriteEndObject();
}
}
@@ -0,0 +1,104 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using Microsoft.Agents.AI.Workflows.Specialized;
using Microsoft.Extensions.AI;
using Microsoft.Shared.Diagnostics;
namespace Microsoft.Agents.AI.Workflows;
/// <summary>
/// Fluent builder for concurrent agent workflows: a fan-out start that broadcasts the
/// incoming messages to every participating agent, a per-agent accumulator that batches
/// each agent's outgoing messages, and a fan-in aggregator that reduces them into a
/// single output list.
/// </summary>
/// <remarks>
/// When no explicit output designations are made, the default is the Python-aligned
/// shape: the terminal aggregator is the workflow output, and every participating agent
/// (plus its per-agent accumulator) is designated as an intermediate output source.
/// Calling <see cref="OrchestrationBuilderBase{TBuilder}.WithOutputFrom(IEnumerable{AIAgent})"/>
/// or <see cref="OrchestrationBuilderBase{TBuilder}.WithIntermediateOutputFrom(IEnumerable{AIAgent})"/>
/// at all suppresses these defaults.
/// </remarks>
public sealed class ConcurrentWorkflowBuilder : OrchestrationBuilderBase<ConcurrentWorkflowBuilder>
{
private readonly List<AIAgent> _agents = [];
private Func<IList<List<ChatMessage>>, List<ChatMessage>>? _aggregator;
/// <summary>
/// Initializes a new <see cref="ConcurrentWorkflowBuilder"/> with the given participating
/// <paramref name="agents"/>.
/// </summary>
public ConcurrentWorkflowBuilder(params IEnumerable<AIAgent> agents)
{
Throw.IfNull(agents);
foreach (AIAgent agent in agents)
{
Throw.IfNull(agent, nameof(agents));
this._agents.Add(agent);
}
}
/// <summary>
/// Sets the aggregator function. If not called, defaults to returning the last message
/// from each agent that produced at least one message.
/// </summary>
public ConcurrentWorkflowBuilder WithAggregator(Func<IList<List<ChatMessage>>, List<ChatMessage>> aggregator)
{
this._aggregator = Throw.IfNull(aggregator);
return this;
}
/// <summary>Builds the configured concurrent workflow.</summary>
public Workflow Build()
{
if (this._agents.Count == 0)
{
throw new ArgumentException("At least one agent must be provided to the ConcurrentWorkflowBuilder.", "agents");
}
ChatForwardingExecutor start = new("Start");
WorkflowBuilder builder = new(start);
Dictionary<AIAgent, ExecutorBinding> agentMap = new(AIAgentIDEqualityComparer.Instance);
ExecutorBinding[] agentExecutors = new ExecutorBinding[this._agents.Count];
ExecutorBinding[] accumulators = new ExecutorBinding[this._agents.Count];
AIAgentHostOptions options = new() { ReassignOtherAgentsAsUsers = true };
for (int i = 0; i < this._agents.Count; i++)
{
AIAgent agent = this._agents[i];
ExecutorBinding binding = agent.BindAsExecutor(options);
agentExecutors[i] = binding;
agentMap[agent] = binding;
accumulators[i] = new AggregateTurnMessagesExecutor($"Batcher/{binding.Id}");
}
builder.AddFanOutEdge(start, agentExecutors);
for (int i = 0; i < agentExecutors.Length; i++)
{
builder.AddEdge(agentExecutors[i], accumulators[i]);
}
Func<IList<List<ChatMessage>>, List<ChatMessage>> aggregator =
this._aggregator ?? (static lists => (from list in lists where list.Count > 0 select list.Last()).ToList());
Func<string, string, ValueTask<ConcurrentEndExecutor>> endFactory =
(_, __) => new(new ConcurrentEndExecutor(agentExecutors.Length, aggregator));
ExecutorBinding end = endFactory.BindExecutor(ConcurrentEndExecutor.ExecutorId);
builder.AddFanInBarrierEdge(accumulators, end);
this.ApplyMetadata(builder);
this.ApplyOutputDesignations(builder, agentMap, "concurrent", () =>
{
builder.WithOutputFrom(end);
builder.WithIntermediateOutputFrom([.. agentExecutors, .. accumulators]);
});
return builder.Build();
}
}
@@ -1,11 +1,17 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
namespace Microsoft.Agents.AI.Workflows.Execution;
internal sealed class OutputFilter(Workflow workflow)
{
public bool CanOutput(string sourceExecutorId, object output)
{
return workflow.OutputExecutors.Contains(sourceExecutorId);
return workflow.OutputExecutors.ContainsKey(sourceExecutorId);
}
public bool TryGetTags(string sourceExecutorId, [NotNullWhen(true)] out HashSet<OutputTag>? tags)
=> workflow.OutputExecutors.TryGetValue(sourceExecutorId, out tags);
}
@@ -0,0 +1,40 @@
// Copyright (c) Microsoft. All rights reserved.
namespace Microsoft.Agents.AI.Workflows;
/// <summary>
/// Process-wide opt-in switches for in-development behavior changes that will become
/// the default in a future major release. Each flag defaults to <see langword="false"/>
/// and should be toggled once at application startup.
/// </summary>
public static class Futures
{
/// <summary>
/// When <see langword="true"/>, <see cref="AgentResponse"/> and
/// <see cref="AgentResponseUpdate"/> payloads yielded by an executor participate
/// in the normal output-filter pipeline (i.e. they must be designated via
/// <see cref="WorkflowBuilder.WithOutputFrom(ExecutorBinding[])"/> or
/// <see cref="WorkflowBuilderExtensions.WithIntermediateOutputFrom(WorkflowBuilder, System.Collections.Generic.IEnumerable{ExecutorBinding})"/>
/// to surface), and the resulting <see cref="WorkflowOutputEvent"/>s carry
/// <see cref="WorkflowOutputEvent.Tags"/> reflecting that designation.
/// </summary>
/// <remarks>
/// <para>
/// When <see langword="false"/> (the current default), the runner emits
/// <see cref="AgentResponseEvent"/> and <see cref="AgentResponseUpdateEvent"/> unconditionally,
/// bypassing the output filter (historical behavior). Lifecycle: opt-in today, marked
/// <c>[Obsolete]</c> in v2.0.0 when the new behavior becomes default, and removed in v3.0.0.
/// </para>
/// <para>
/// <b>Interaction with <see cref="WorkflowHostingExtensions.AsAIAgent"/>.</b> When this flag
/// is <see langword="true"/>, <see cref="AgentResponseEvent"/> joins
/// <see cref="AgentResponseUpdateEvent"/> in being forwarded out of the agent surface
/// unconditionally — neither honors the host's <c>includeWorkflowOutputsInResponse</c>
/// switch. That switch only governs the generic <see cref="WorkflowOutputEvent"/> path for
/// non-AIAgent payloads. When this flag is <see langword="false"/>, the legacy asymmetry
/// is preserved: <see cref="AgentResponseUpdateEvent"/> is always forwarded but
/// <see cref="AgentResponseEvent"/> stays gated by <c>includeWorkflowOutputsInResponse</c>.
/// </para>
/// </remarks>
public static bool EnableAgentResponseOutputTaggingAndFiltering { get; set; }
}
@@ -1,6 +1,8 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.AI;
@@ -13,6 +15,16 @@ namespace Microsoft.Agents.AI.Workflows;
/// </summary>
public abstract class GroupChatManager
{
// The state key under which GroupChatManager persists its own (non-subclass) state on the
// raw IWorkflowContext supplied by the hosting GroupChatHost executor.
internal const string BaseStateKey = "GroupChatManager";
// Prefix automatically applied to every key a subclass writes through the wrapped context
// supplied to OnCheckpointingAsync / OnCheckpointRestoredAsync. Keeps subclass-defined
// state in its own namespace so it cannot collide with the host's state keys nor with
// BaseStateKey itself.
internal const string SubclassStateKeyPrefix = "GroupChatManager_";
/// <summary>
/// Initializes a new instance of the <see cref="GroupChatManager"/> class.
/// </summary>
@@ -48,12 +60,22 @@ public abstract class GroupChatManager
CancellationToken cancellationToken = default);
/// <summary>
/// Filters the chat history before it's passed to the next agent.
/// Filters the messages broadcast to participants for the current turn.
/// </summary>
/// <param name="history">The chat history to filter.</param>
/// <remarks>
/// Under the broadcast model, each participant maintains its own per-agent session (history)
/// through its <see cref="Specialized.AIAgentHostExecutor"/>. The host distributes new messages
/// (initial user input on the first turn, the most recent speaker's response on subsequent turns)
/// to every participant — except the speaker that produced them — so every participant's session
/// stays synchronized. This method lets the manager shape that broadcast payload (for example,
/// to omit certain messages or to inject orchestrator-visible annotations). The full canonical
/// conversation is still available to <see cref="SelectNextAgentAsync"/> and
/// <see cref="ShouldTerminateAsync"/>.
/// </remarks>
/// <param name="history">The new messages about to be broadcast to participants this turn.</param>
/// <param name="cancellationToken">The <see cref="CancellationToken"/> to monitor for cancellation requests.
/// The default is <see cref="CancellationToken.None"/>.</param>
/// <returns>The filtered chat history.</returns>
/// <returns>The filtered message list to broadcast.</returns>
protected internal virtual ValueTask<IEnumerable<ChatMessage>> UpdateHistoryAsync(
IReadOnlyList<ChatMessage> history,
CancellationToken cancellationToken = default) =>
@@ -78,4 +100,125 @@ public abstract class GroupChatManager
{
this.IterationCount = 0;
}
/// <summary>
/// Invoked when the hosting group chat workflow is checkpointing, giving subclasses a chance to
/// persist any additional state they maintain (e.g., a round-robin cursor or an LLM session).
/// </summary>
/// <remarks>
/// <para>
/// The default implementation is a no-op. Base-class state (currently
/// <see cref="IterationCount"/>) is persisted automatically by the hosting
/// <see cref="Specialized.GroupChatHost"/> before this method is invoked; subclasses do not
/// need to call <c>base.OnCheckpointingAsync</c>.
/// </para>
/// <para>
/// The supplied <paramref name="context"/> is a wrapper that transparently prefixes every
/// state key with <c>"GroupChatManager_"</c>, isolating subclass state from the host's own
/// state keys (and from the reserved base-state key). Implementations therefore may use any
/// human-readable key (e.g., <c>"next_index"</c>) without worrying about collisions.
/// </para>
/// </remarks>
/// <param name="context">A wrapped workflow context that scopes state keys to the
/// <see cref="GroupChatManager"/> subclass namespace.</param>
/// <param name="cancellationToken">The <see cref="CancellationToken"/> to monitor for cancellation requests.
/// The default is <see cref="CancellationToken.None"/>.</param>
protected virtual ValueTask OnCheckpointingAsync(IWorkflowContext context, CancellationToken cancellationToken = default)
=> default;
/// <summary>
/// Invoked when the hosting group chat workflow is being restored from a checkpoint, giving
/// subclasses a chance to hydrate any additional state they persisted in
/// <see cref="OnCheckpointingAsync"/>.
/// </summary>
/// <remarks>
/// The default implementation is a no-op. Base-class state (currently
/// <see cref="IterationCount"/>) is restored automatically by the hosting
/// <see cref="Specialized.GroupChatHost"/> before this method is invoked; subclasses do not
/// need to call <c>base.OnCheckpointRestoredAsync</c>. The supplied <paramref name="context"/>
/// uses the same key-prefixing wrapper as <see cref="OnCheckpointingAsync"/>.
/// </remarks>
/// <param name="context">A wrapped workflow context that scopes state keys to the
/// <see cref="GroupChatManager"/> subclass namespace.</param>
/// <param name="cancellationToken">The <see cref="CancellationToken"/> to monitor for cancellation requests.
/// The default is <see cref="CancellationToken.None"/>.</param>
protected virtual ValueTask OnCheckpointRestoredAsync(IWorkflowContext context, CancellationToken cancellationToken = default)
=> default;
// Root checkpoint entry point invoked by the hosting GroupChatHost. Persists the manager's
// own base state under the reserved BaseStateKey on the raw context, then delegates to the
// subclass-facing OnCheckpointingAsync hook with a wrapped context that prefixes every key
// with SubclassStateKeyPrefix.
internal async ValueTask CheckpointAsync(IWorkflowContext context, CancellationToken cancellationToken = default)
{
await context.QueueStateUpdateAsync(BaseStateKey, new GroupChatManagerState(this.IterationCount), cancellationToken: cancellationToken).ConfigureAwait(false);
await this.OnCheckpointingAsync(new PrefixingWorkflowContext(context, SubclassStateKeyPrefix), cancellationToken).ConfigureAwait(false);
}
// Root restore entry point invoked by the hosting GroupChatHost. Symmetric to CheckpointAsync.
internal async ValueTask RestoreCheckpointAsync(IWorkflowContext context, CancellationToken cancellationToken = default)
{
GroupChatManagerState? state = await context.ReadStateAsync<GroupChatManagerState>(BaseStateKey, cancellationToken: cancellationToken).ConfigureAwait(false);
this.IterationCount = state?.IterationCount ?? 0;
await this.OnCheckpointRestoredAsync(new PrefixingWorkflowContext(context, SubclassStateKeyPrefix), cancellationToken).ConfigureAwait(false);
}
}
internal sealed record GroupChatManagerState(int IterationCount);
// IWorkflowContext decorator that prepends a fixed prefix to every state key passed through it.
// All non-state members (events, message sending, output yielding, halt requests, trace context,
// and runtime characteristics) delegate directly to the wrapped context.
internal sealed class PrefixingWorkflowContext(IWorkflowContext inner, string prefix) : IWorkflowContext
{
private readonly IWorkflowContext _inner = Throw.IfNull(inner);
private readonly string _prefix = Throw.IfNullOrEmpty(prefix);
public IReadOnlyDictionary<string, string>? TraceContext => this._inner.TraceContext;
public bool ConcurrentRunsEnabled => this._inner.ConcurrentRunsEnabled;
public ValueTask AddEventAsync(WorkflowEvent workflowEvent, CancellationToken cancellationToken = default)
=> this._inner.AddEventAsync(workflowEvent, cancellationToken);
public ValueTask SendMessageAsync(object message, string? targetId, CancellationToken cancellationToken = default)
=> this._inner.SendMessageAsync(message, targetId, cancellationToken);
public ValueTask YieldOutputAsync(object output, CancellationToken cancellationToken = default)
=> this._inner.YieldOutputAsync(output, cancellationToken);
public ValueTask RequestHaltAsync() => this._inner.RequestHaltAsync();
public ValueTask<T?> ReadStateAsync<T>(string key, string? scopeName = null, CancellationToken cancellationToken = default)
=> this._inner.ReadStateAsync<T>(this.Wrap(key), scopeName, cancellationToken);
public ValueTask<T> ReadOrInitStateAsync<T>(string key, Func<T> initialStateFactory, string? scopeName = null, CancellationToken cancellationToken = default)
=> this._inner.ReadOrInitStateAsync(this.Wrap(key), initialStateFactory, scopeName, cancellationToken);
public async ValueTask<HashSet<string>> ReadStateKeysAsync(string? scopeName = null, CancellationToken cancellationToken = default)
{
HashSet<string> rawKeys = await this._inner.ReadStateKeysAsync(scopeName, cancellationToken).ConfigureAwait(false);
return [.. rawKeys.Where(k => k.StartsWith(this._prefix, StringComparison.Ordinal))
.Select(k => k.Substring(this._prefix.Length))];
}
public ValueTask QueueStateUpdateAsync<T>(string key, T? value, string? scopeName = null, CancellationToken cancellationToken = default)
=> this._inner.QueueStateUpdateAsync(this.Wrap(key), value, scopeName, cancellationToken);
public async ValueTask QueueClearScopeAsync(string? scopeName = null, CancellationToken cancellationToken = default)
{
// Clearing the entire underlying scope would also remove keys owned by the host and other
// subsystems sharing the executor's default scope. Restrict the clear to keys carrying
// this wrapper's prefix.
HashSet<string> rawKeys = await this._inner.ReadStateKeysAsync(scopeName, cancellationToken).ConfigureAwait(false);
foreach (string rawKey in rawKeys)
{
if (rawKey.StartsWith(this._prefix, StringComparison.Ordinal))
{
await this._inner.QueueStateUpdateAsync<object>(rawKey, null, scopeName, cancellationToken).ConfigureAwait(false);
}
}
}
private string Wrap(string key) => this._prefix + Throw.IfNullOrEmpty(key);
}
@@ -12,12 +12,10 @@ namespace Microsoft.Agents.AI.Workflows;
/// <summary>
/// Provides a builder for specifying group chat relationships between agents and building the resulting workflow.
/// </summary>
public sealed class GroupChatWorkflowBuilder
public sealed class GroupChatWorkflowBuilder : OrchestrationBuilderBase<GroupChatWorkflowBuilder>
{
private readonly Func<IReadOnlyList<AIAgent>, GroupChatManager> _managerFactory;
private readonly HashSet<AIAgent> _participants = new(AIAgentIDEqualityComparer.Instance);
private string _name = string.Empty;
private string _description = string.Empty;
internal GroupChatWorkflowBuilder(Func<IReadOnlyList<AIAgent>, GroupChatManager> managerFactory) =>
this._managerFactory = managerFactory;
@@ -44,28 +42,6 @@ public sealed class GroupChatWorkflowBuilder
return this;
}
/// <summary>
/// Sets the human-readable name for the workflow.
/// </summary>
/// <param name="name">The name of the workflow.</param>
/// <returns>This instance of the <see cref="GroupChatWorkflowBuilder"/>.</returns>
public GroupChatWorkflowBuilder WithName(string name)
{
this._name = name;
return this;
}
/// <summary>
/// Sets the description for the workflow.
/// </summary>
/// <param name="description">The description of what the workflow does.</param>
/// <returns>This instance of the <see cref="GroupChatWorkflowBuilder"/>.</returns>
public GroupChatWorkflowBuilder WithDescription(string description)
{
this._description = description;
return this;
}
/// <summary>
/// Builds a <see cref="Workflow"/> composed of agents that operate via group chat, with the next
/// agent to process messages selected by the group chat manager.
@@ -75,10 +51,14 @@ public sealed class GroupChatWorkflowBuilder
{
AIAgent[] agents = this._participants.ToArray();
// GroupChatHost owns the canonical conversation and broadcasts messages directly to every
// participant. Participants therefore must not echo their incoming messages back to the host
// (which would cause duplicates), but must still reframe other agents' assistant messages as
// user messages so each agent's own session reads coherently.
AIAgentHostOptions options = new()
{
ReassignOtherAgentsAsUsers = true,
ForwardIncomingMessages = true
ForwardIncomingMessages = false
};
Dictionary<AIAgent, ExecutorBinding> agentMap = agents.ToDictionary(a => a, a => a.BindAsExecutor(options));
@@ -89,15 +69,7 @@ public sealed class GroupChatWorkflowBuilder
ExecutorBinding host = groupChatHostFactory.BindExecutor(nameof(GroupChatHost));
WorkflowBuilder builder = new(host);
if (!string.IsNullOrEmpty(this._name))
{
builder = builder.WithName(this._name);
}
if (!string.IsNullOrEmpty(this._description))
{
builder = builder.WithDescription(this._description);
}
this.ApplyMetadata(builder);
foreach (var participant in agentMap.Values)
{
@@ -106,6 +78,15 @@ public sealed class GroupChatWorkflowBuilder
.AddEdge(participant, host);
}
return builder.WithOutputFrom(host).Build();
this.ApplyOutputDesignations(builder, agentMap, "group chat", () =>
{
builder.WithOutputFrom(host);
if (agentMap.Count > 0)
{
builder.WithIntermediateOutputFrom([.. agentMap.Values]);
}
});
return builder.Build();
}
}
@@ -15,11 +15,6 @@ using ExecutorFactoryFunc = System.Func<Microsoft.Agents.AI.Workflows.ExecutorCo
namespace Microsoft.Agents.AI.Workflows;
internal static class DiagnosticConstants
{
public const string ExperimentalFeatureDiagnostic = "MAAIW001";
}
/// <inheritdoc/>
[ExcludeFromCodeCoverage] // This is obsolete, and 1:1 equivalent to HandoffWorkflowBuilder (no "s")
[Obsolete("Prefer HandoffWorkflowBuilder (no 's') instead, which has the same API but the preferred name. This will be removed in a future release before GA.")]
@@ -30,7 +25,6 @@ public sealed class HandoffsWorkflowBuilder(AIAgent initialAgent) : HandoffWorkf
}
/// <inheritdoc/>
[Experimental(DiagnosticConstants.ExperimentalFeatureDiagnostic)]
public sealed class HandoffWorkflowBuilder(AIAgent initialAgent) : HandoffWorkflowBuilderCore<HandoffWorkflowBuilder>(initialAgent)
{
}
@@ -38,8 +32,8 @@ public sealed class HandoffWorkflowBuilder(AIAgent initialAgent) : HandoffWorkfl
/// <summary>
/// Provides a builder for specifying the handoff relationships between agents and building the resulting workflow.
/// </summary>
[Experimental(DiagnosticConstants.ExperimentalFeatureDiagnostic)]
public class HandoffWorkflowBuilderCore<TBuilder> where TBuilder : HandoffWorkflowBuilderCore<TBuilder>
public class HandoffWorkflowBuilderCore<TBuilder> : OrchestrationBuilderBase<TBuilder>
where TBuilder : HandoffWorkflowBuilderCore<TBuilder>
{
/// <summary>
/// The prefix for function calls that trigger handoffs to other agents; the full name is then `{FunctionPrefix}&lt;agent_id&gt;`,
@@ -55,8 +49,6 @@ public class HandoffWorkflowBuilderCore<TBuilder> where TBuilder : HandoffWorkfl
private bool _emitAgentResponseUpdateEvents;
private HandoffToolCallFilteringBehavior _toolCallFilteringBehavior = HandoffToolCallFilteringBehavior.HandoffOnly;
private bool _returnToPrevious;
private string? _name;
private string? _description;
// Autonomous mode configuration. When enabled, an agent's response that doesn't include a
// handoff triggers another invocation of that same agent with the continuation prompt, up to
@@ -116,20 +108,6 @@ public class HandoffWorkflowBuilderCore<TBuilder> where TBuilder : HandoffWorkfl
return (TBuilder)this;
}
/// <inheritdoc cref="WorkflowBuilder.WithName(string)"/>
public TBuilder WithName(string name)
{
this._name = name;
return (TBuilder)this;
}
/// <inheritdoc cref="WorkflowBuilder.WithDescription(string)"/>
public TBuilder WithDescription(string description)
{
this._description = description;
return (TBuilder)this;
}
/// <summary>
/// Sets a value indicating whether agent streaming update events should be emitted during execution.
/// If <see langword="null"/>, the value will be taken from the <see cref="TurnToken"/>
@@ -631,16 +609,31 @@ public class HandoffWorkflowBuilderCore<TBuilder> where TBuilder : HandoffWorkfl
});
}
if (!string.IsNullOrWhiteSpace(this._name))
// Ensure the end executor is bound regardless of whether it ends up as an output
// designation source — the user may take full control of output designations.
builder.BindExecutor(end);
// Build the AIAgent -> ExecutorBinding map the base helper expects.
Dictionary<AIAgent, ExecutorBinding> agentMap = new(AIAgentIDEqualityComparer.Instance);
foreach (AIAgent agent in this._allAgents)
{
builder.WithName(this._name);
agentMap[agent] = executors[agent.Id];
}
if (!string.IsNullOrWhiteSpace(this._description))
this.ApplyMetadata(builder);
this.ApplyOutputDesignations(builder, agentMap, "handoff", () =>
{
builder.WithDescription(this._description);
}
// Defaults (matches Python's Handoff orchestration):
// end -> terminal output
// every handoff agent -> intermediate output
builder.WithOutputFrom(end);
List<ExecutorBinding> agentBindings = [.. executors.Values];
if (agentBindings.Count > 0)
{
builder.WithIntermediateOutputFrom(agentBindings);
}
});
return builder.WithOutputFrom(end).Build();
return builder.Build();
}
}
@@ -241,30 +241,47 @@ internal sealed class InProcessRunnerContext : IRunnerContext
this.CheckEnded();
Throw.IfNull(output);
// Special-case AgentResponse and AgentResponseUpdate to create their specific event types
// and bypass the output filter (for backwards compatibility - these events were previously
// emitted directly via AddEventAsync without filtering)
if (output is AgentResponseUpdate update)
bool isAgentResponseShaped = output is AgentResponse or AgentResponseUpdate;
if (isAgentResponseShaped && !Futures.EnableAgentResponseOutputTaggingAndFiltering)
{
await this.AddEventAsync(new AgentResponseUpdateEvent(sourceId, update), cancellationToken).ConfigureAwait(false);
return;
}
else if (output is AgentResponse response)
{
await this.AddEventAsync(new AgentResponseEvent(sourceId, response), cancellationToken).ConfigureAwait(false);
// Legacy bypass: AgentResponse/AgentResponseUpdate skip the output filter and are
// emitted as their typed event subclasses with no tags. Preserved verbatim for
// back-compat; once Futures.EnableAgentResponseOutputTaggingAndFiltering becomes the
// default in v2.0.0, this branch goes away.
WorkflowEvent typedEvent = output switch
{
AgentResponseUpdate u => new AgentResponseUpdateEvent(sourceId, u),
AgentResponse r => new AgentResponseEvent(sourceId, r),
_ => throw new InvalidOperationException("Unexpected AIAgent-shaped payload type."),
};
await this.AddEventAsync(typedEvent, cancellationToken).ConfigureAwait(false);
return;
}
Executor sourceExecutor = await this.EnsureExecutorAsync(sourceId, tracer: null, cancellationToken).ConfigureAwait(false);
if (!sourceExecutor.CanOutput(output.GetType()))
if (!isAgentResponseShaped && !sourceExecutor.CanOutput(output.GetType()))
{
// AIAgent-shaped payloads bypass the per-executor declared-yield check (matching the
// legacy bypass branch above). The AIAgent host executor relays the agent's output
// without declaring AgentResponse(Update) in its Yields set, so a CanOutput probe
// here would always reject — but those payloads are always a valid output shape.
throw new InvalidOperationException($"Cannot output object of type {output.GetType().Name}. Expecting one of [{string.Join(", ", sourceExecutor.OutputTypes)}].");
}
if (this._outputFilter.CanOutput(sourceId, output))
if (!this._outputFilter.TryGetTags(sourceId, out HashSet<OutputTag>? tags))
{
await this.AddEventAsync(new WorkflowOutputEvent(output, sourceId), cancellationToken).ConfigureAwait(false);
// Not designated as an output source — drop silently.
return;
}
WorkflowOutputEvent evt = output switch
{
AgentResponseUpdate u => new AgentResponseUpdateEvent(sourceId, u, tags),
AgentResponse r => new AgentResponseEvent(sourceId, r, tags),
_ => new WorkflowOutputEvent(output, sourceId, tags),
};
await this.AddEventAsync(evt, cancellationToken).ConfigureAwait(false);
}
public IExternalRequestContext BindExternalRequestContext(string executorId)
@@ -1,7 +1,6 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
using System.Linq;
using Microsoft.Extensions.AI;
@@ -16,7 +15,6 @@ namespace Microsoft.Agents.AI.Workflows;
/// contain the latest progress ledger that determined that no progress has been made or the workflow was in
/// a loop.</param>
/// <param name="IsStalled">Whether the workflow is currently stalled.</param>
[Experimental(DiagnosticConstants.ExperimentalFeatureDiagnostic)]
public record MagenticPlanReviewRequest(ChatMessage Plan, MagenticProgressLedger? CurrentProgress, bool IsStalled)
{
/// <summary>
@@ -1,7 +1,6 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
using Microsoft.Extensions.AI;
namespace Microsoft.Agents.AI.Workflows;
@@ -13,7 +12,6 @@ namespace Microsoft.Agents.AI.Workflows;
/// <param name="Review">
/// Review feedback for a generated plan. Empty if the plan is approved as-is and changes are requested.
/// </param>
[Experimental(DiagnosticConstants.ExperimentalFeatureDiagnostic)]
public record MagenticPlanReviewResponse(List<ChatMessage> Review)
{
internal bool IsApproved => this.Review.Count == 0;
@@ -14,7 +14,6 @@ namespace Microsoft.Agents.AI.Workflows;
/// <summary>
/// Maintains a ledger of progress made by the Magentic workflow.
/// </summary>
[Experimental(DiagnosticConstants.ExperimentalFeatureDiagnostic)]
public class MagenticProgressLedger
{
internal static readonly BooleanProgressLedgerSlot IsRequestSatisfiedSlot = new("is_request_satisfied",
@@ -76,7 +75,7 @@ public class MagenticProgressLedger
this.InstructionOrQuestion = instructionOrQuestion!;
}
// TODO: To what extent do we want to enforce that the additional questions are also answered?
// TODO: To what extent do we want to enforce that the additional questions are also answered?
return requiredQuestionsAnswered;
}
@@ -2,7 +2,6 @@
using System;
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
using System.Threading.Tasks;
using Microsoft.Agents.AI.Workflows.Specialized.Magentic;
@@ -27,12 +26,9 @@ namespace Microsoft.Agents.AI.Workflows;
/// not supported on the ManagerAgent.
/// </summary>
/// <param name="managerAgent"></param>
[Experimental(DiagnosticConstants.ExperimentalFeatureDiagnostic)]
public class MagenticWorkflowBuilder(AIAgent managerAgent)
public class MagenticWorkflowBuilder(AIAgent managerAgent) : OrchestrationBuilderBase<MagenticWorkflowBuilder>
{
private readonly List<AIAgent> _team = new();
private string? _name;
private string? _description;
private int _maxStalls = TaskLimits.DefaultMaxStallCount;
private int? _maxRounds;
private int? _maxResets;
@@ -45,20 +41,6 @@ public class MagenticWorkflowBuilder(AIAgent managerAgent)
return this;
}
/// <inheritdoc cref="WorkflowBuilder.WithName(string)"/>
public MagenticWorkflowBuilder WithName(string name)
{
this._name = name;
return this;
}
/// <inheritdoc cref="WorkflowBuilder.WithDescription(string)"/>
public MagenticWorkflowBuilder WithDescription(string description)
{
this._description = description;
return this;
}
/// <summary>
/// Set the maximum number of coordination rounds. <see langword="null"/> means unlimited.
/// </summary>
@@ -115,28 +97,29 @@ public class MagenticWorkflowBuilder(AIAgent managerAgent)
ForwardIncomingMessages = false
};
Dictionary<AIAgent, ExecutorBinding> teamMap = new(AIAgentIDEqualityComparer.Instance);
List<ExecutorBinding> teamBindings = [];
foreach (AIAgent agent in team)
{
ExecutorBinding binding = agent.BindAsExecutor(options);
teamBindings.Add(binding);
teamMap[agent] = binding;
result.AddEdge(binding, orchestrator);
}
result.AddFanOutEdge(orchestrator, teamBindings)
.WithOutputFrom(orchestrator);
result.AddFanOutEdge(orchestrator, teamBindings);
if (!string.IsNullOrWhiteSpace(this._name))
this.ApplyOutputDesignations(result, teamMap, "Magentic", () =>
{
result.WithName(this._name);
}
if (!string.IsNullOrWhiteSpace(this._description))
{
result.WithDescription(this._description);
}
result.WithOutputFrom(orchestrator);
if (teamMap.Count > 0)
{
result.WithIntermediateOutputFrom([.. teamMap.Values]);
}
});
this.ApplyMetadata(result);
return result;
}
@@ -0,0 +1,154 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using Microsoft.Shared.Diagnostics;
namespace Microsoft.Agents.AI.Workflows;
/// <summary>
/// Common fluent surface shared by every orchestration-style workflow builder:
/// human-readable name + description, and the
/// <see cref="WithOutputFrom"/> / <see cref="WithIntermediateOutputFrom"/> output-designation
/// pair with memoized defaults-suppression semantics.
/// </summary>
/// <typeparam name="TBuilder">The concrete builder type, for fluent self-return.</typeparam>
public abstract class OrchestrationBuilderBase<TBuilder>
where TBuilder : OrchestrationBuilderBase<TBuilder>
{
/// <summary>Optional workflow name; applied to the inner <see cref="WorkflowBuilder"/> at <c>Build()</c>.</summary>
protected string? Name { get; private set; }
/// <summary>Optional workflow description; applied to the inner <see cref="WorkflowBuilder"/> at <c>Build()</c>.</summary>
protected string? Description { get; private set; }
/// <summary>
/// Memoized output designations. <see langword="null"/> means the user has not made any
/// explicit designation, and the orchestration-specific defaults will be applied at
/// <c>Build()</c> time. A non-<see langword="null"/> (possibly empty) map means the user took
/// control and only these designations will be replayed onto the inner
/// <see cref="WorkflowBuilder"/>. An entry's value is the set of tags requested for the
/// agent — an empty set encodes a terminal-only designation.
/// </summary>
protected Dictionary<AIAgent, HashSet<OutputTag>>? OutputDesignations { get; private set; }
/// <summary>Sets the human-readable name for the workflow.</summary>
public TBuilder WithName(string name)
{
this.Name = name;
return (TBuilder)this;
}
/// <summary>Sets the description for the workflow.</summary>
public TBuilder WithDescription(string description)
{
this.Description = description;
return (TBuilder)this;
}
/// <summary>
/// Designates the given <paramref name="agents"/> as sources of terminal workflow output.
/// Calling any output-designation method (this or <see cref="WithIntermediateOutputFrom"/>)
/// suppresses the orchestration-specific defaults: only the user-specified designations
/// reach the inner <see cref="WorkflowBuilder"/>.
/// </summary>
public TBuilder WithOutputFrom(params IEnumerable<AIAgent> agents)
{
Throw.IfNull(agents);
this.OutputDesignations ??= new(AIAgentIDEqualityComparer.Instance);
foreach (AIAgent agent in agents)
{
Throw.IfNull(agent, nameof(agents));
if (!this.OutputDesignations.ContainsKey(agent))
{
this.OutputDesignations[agent] = [];
}
}
return (TBuilder)this;
}
/// <summary>
/// Designates the given <paramref name="agents"/> as sources of <b>intermediate</b> workflow
/// output. See <see cref="WithOutputFrom"/> for the defaults-suppression semantics.
/// </summary>
public TBuilder WithIntermediateOutputFrom(IEnumerable<AIAgent> agents)
{
Throw.IfNull(agents);
this.OutputDesignations ??= new(AIAgentIDEqualityComparer.Instance);
foreach (AIAgent agent in agents)
{
Throw.IfNull(agent, nameof(agents));
if (!this.OutputDesignations.TryGetValue(agent, out HashSet<OutputTag>? tags))
{
tags = [];
this.OutputDesignations[agent] = tags;
}
tags.Add(OutputTag.Intermediate);
}
return (TBuilder)this;
}
/// <summary>
/// Applies the optional <see cref="Name"/> and <see cref="Description"/> to <paramref name="builder"/>.
/// Subclasses should call this from their <c>Build()</c> implementation.
/// </summary>
protected void ApplyMetadata(WorkflowBuilder builder)
{
Throw.IfNull(builder);
if (!string.IsNullOrWhiteSpace(this.Name))
{
builder.WithName(this.Name!);
}
if (!string.IsNullOrWhiteSpace(this.Description))
{
builder.WithDescription(this.Description!);
}
}
/// <summary>
/// Applies the user's memoized output designations to <paramref name="builder"/>, or invokes
/// <paramref name="applyDefaults"/> if the user made no explicit designation.
/// </summary>
/// <param name="builder">The inner <see cref="WorkflowBuilder"/>.</param>
/// <param name="agentMap">Map from participating <see cref="AIAgent"/> to its bound executor.</param>
/// <param name="orchestrationKind">Used in the not-a-participant error message (e.g. "sequential", "group chat").</param>
/// <param name="applyDefaults">Action invoked when no explicit designation was made.</param>
protected void ApplyOutputDesignations(
WorkflowBuilder builder,
IReadOnlyDictionary<AIAgent, ExecutorBinding> agentMap,
string orchestrationKind,
Action applyDefaults)
{
Throw.IfNull(builder);
Throw.IfNull(agentMap);
Throw.IfNull(applyDefaults);
if (this.OutputDesignations is null)
{
applyDefaults();
return;
}
foreach (AIAgent agent in this.OutputDesignations.Keys)
{
if (!agentMap.TryGetValue(agent, out ExecutorBinding? binding))
{
throw new InvalidOperationException(
$"Output designation references agent '{agent.Name ?? agent.Id}', which is not a participant in this {orchestrationKind} workflow.");
}
HashSet<OutputTag> tags = this.OutputDesignations[agent];
if (tags.Count == 0)
{
builder.WithOutputFrom(binding);
}
else
{
foreach (OutputTag tag in tags)
{
builder.WithOutputFrom(binding, tag);
}
}
}
}
}
@@ -0,0 +1,52 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Text.Json.Serialization;
using Microsoft.Shared.Diagnostics;
namespace Microsoft.Agents.AI.Workflows;
/// <summary>
/// Identifies the kind of output that a <see cref="WorkflowOutputEvent"/> represents.
/// A thin <c>ChatRole</c>-style wrapper around a normalized string <see cref="Value"/>,
/// with value equality and a closed set of well-known singletons (the constructor is
/// <see langword="internal"/> for now).
/// </summary>
[JsonConverter(typeof(OutputTagJsonConverter))]
public readonly struct OutputTag : IEquatable<OutputTag>
{
/// <summary>
/// The string identifier of the tag. Compared with ordinal equality.
/// </summary>
public string? Value { get; }
internal OutputTag(string value)
{
this.Value = Throw.IfNullOrEmpty(value);
}
/// <summary>
/// The tag denoting an intermediate workflow output &#x2014; emitted by executors
/// registered via <see cref="WorkflowBuilderExtensions.WithIntermediateOutputFrom(WorkflowBuilder, System.Collections.Generic.IEnumerable{ExecutorBinding})"/>.
/// Terminal (non-intermediate) outputs carry no tag.
/// </summary>
public static OutputTag Intermediate { get; } = new("intermediate");
/// <inheritdoc />
public bool Equals(OutputTag other) => string.Equals(this.Value, other.Value, StringComparison.Ordinal);
/// <inheritdoc />
public override bool Equals(object? obj) => obj is OutputTag other && this.Equals(other);
/// <inheritdoc />
public override int GetHashCode() => this.Value is null ? 0 : StringComparer.Ordinal.GetHashCode(this.Value);
/// <summary>Determines whether two <see cref="OutputTag"/> values are equal.</summary>
public static bool operator ==(OutputTag left, OutputTag right) => left.Equals(right);
/// <summary>Determines whether two <see cref="OutputTag"/> values are not equal.</summary>
public static bool operator !=(OutputTag left, OutputTag right) => !left.Equals(right);
/// <inheritdoc />
public override string ToString() => this.Value ?? string.Empty;
}
@@ -0,0 +1,43 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Text.Json;
using System.Text.Json.Serialization;
namespace Microsoft.Agents.AI.Workflows;
/// <summary>
/// JSON converter for <see cref="OutputTag"/> that round-trips the underlying
/// <see cref="OutputTag.Value"/> as a bare JSON string.
/// </summary>
internal sealed class OutputTagJsonConverter : JsonConverter<OutputTag>
{
public override OutputTag Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
{
string? value = reader.GetString();
if (string.IsNullOrEmpty(value))
{
return default;
}
// Reuse the well-known singleton where possible so callers can do reference
// comparisons on the common case without paying the extra allocation cost.
if (string.Equals(value, OutputTag.Intermediate.Value, StringComparison.Ordinal))
{
return OutputTag.Intermediate;
}
return new OutputTag(value!);
}
public override void Write(Utf8JsonWriter writer, OutputTag value, JsonSerializerOptions options)
{
if (value.Value is null)
{
writer.WriteNullValue();
return;
}
writer.WriteStringValue(value.Value);
}
}
@@ -69,4 +69,23 @@ public class RoundRobinGroupChatManager : GroupChatManager
base.Reset();
this._nextIndex = 0;
}
/// <inheritdoc />
protected override ValueTask OnCheckpointingAsync(IWorkflowContext context, CancellationToken cancellationToken = default)
=> context.QueueStateUpdateAsync(StateKey, new RoundRobinGroupChatManagerState(this._nextIndex), cancellationToken: cancellationToken);
/// <inheritdoc />
protected override async ValueTask OnCheckpointRestoredAsync(IWorkflowContext context, CancellationToken cancellationToken = default)
{
RoundRobinGroupChatManagerState? state = await context.ReadStateAsync<RoundRobinGroupChatManagerState>(StateKey, cancellationToken: cancellationToken).ConfigureAwait(false);
this._nextIndex = state?.NextIndex ?? 0;
if (this._nextIndex < 0 || this._nextIndex >= this._agents.Count)
{
this._nextIndex = 0;
}
}
private const string StateKey = "next_index";
}
internal sealed record RoundRobinGroupChatManagerState(int NextIndex);
@@ -0,0 +1,84 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Linq;
using Microsoft.Shared.Diagnostics;
namespace Microsoft.Agents.AI.Workflows;
/// <summary>
/// Fluent builder for sequential agent workflows: a pipeline where the output of one
/// agent is the input to the next, terminating in an aggregator that yields the
/// accumulated <see cref="Extensions.AI.ChatMessage"/>s as the workflow output.
/// </summary>
/// <remarks>
/// When no explicit output designations are made, the default is the Python-aligned
/// shape: the terminal aggregator is the workflow output, and every participating agent
/// is designated as an intermediate output source. Calling
/// <see cref="OrchestrationBuilderBase{TBuilder}.WithOutputFrom(IEnumerable{AIAgent})"/>
/// or <see cref="OrchestrationBuilderBase{TBuilder}.WithIntermediateOutputFrom(IEnumerable{AIAgent})"/>
/// at all suppresses these defaults.
/// </remarks>
public sealed class SequentialWorkflowBuilder : OrchestrationBuilderBase<SequentialWorkflowBuilder>
{
private readonly List<AIAgent> _agents = [];
/// <summary>
/// Initializes a new <see cref="SequentialWorkflowBuilder"/> with the given pipeline
/// of <paramref name="agents"/>.
/// </summary>
public SequentialWorkflowBuilder(params IEnumerable<AIAgent> agents)
{
Throw.IfNull(agents);
foreach (AIAgent agent in agents)
{
Throw.IfNull(agent, nameof(agents));
this._agents.Add(agent);
}
}
/// <summary>Builds the configured sequential workflow.</summary>
public Workflow Build()
{
if (this._agents.Count == 0)
{
throw new ArgumentException("At least one agent must be provided to the SequentialWorkflowBuilder.", "agents");
}
AIAgentHostOptions options = new()
{
ReassignOtherAgentsAsUsers = true,
ForwardIncomingMessages = true,
};
Dictionary<AIAgent, ExecutorBinding> agentMap = new(AIAgentIDEqualityComparer.Instance);
List<ExecutorBinding> agentExecutors = new(this._agents.Count);
foreach (AIAgent agent in this._agents)
{
ExecutorBinding binding = agent.BindAsExecutor(options);
agentExecutors.Add(binding);
agentMap[agent] = binding;
}
ExecutorBinding previous = agentExecutors[0];
WorkflowBuilder builder = new(previous);
foreach (ExecutorBinding next in agentExecutors.Skip(1))
{
builder.AddEdge(previous, next);
previous = next;
}
OutputMessagesExecutor end = new();
builder.AddEdge(previous, end).BindExecutor(end);
this.ApplyMetadata(builder);
this.ApplyOutputDesignations(builder, agentMap, "sequential", () =>
{
builder.WithOutputFrom(end);
builder.WithIntermediateOutputFrom(agentExecutors);
});
return builder.Build();
}
}
@@ -20,12 +20,25 @@ internal sealed class GroupChatHost(
AutoSendTurnToken = false
};
private const string HistoryStateKey = nameof(_history);
private const string CurrentSpeakerStateKey = nameof(_currentSpeakerExecutorId);
private readonly AIAgent[] _agents = agents;
private readonly Dictionary<AIAgent, ExecutorBinding> _agentMap = agentMap;
private readonly Func<IReadOnlyList<AIAgent>, GroupChatManager> _managerFactory = managerFactory;
private GroupChatManager? _manager;
// Canonical conversation accumulated across turns. Each participant maintains its own per-agent
// session/thread; the host keeps this only as the source of truth for the manager hooks
// (SelectNextAgentAsync / ShouldTerminateAsync) and for the workflow's final output.
private List<ChatMessage> _history = [];
// Executor id of the participant we most recently dispatched a TurnToken to i.e., the current
// speaker whose response is about to arrive. Used to exclude that participant from the next
// broadcast (its own session already contains the message it produced).
private string? _currentSpeakerExecutorId;
protected override ProtocolBuilder ConfigureProtocol(ProtocolBuilder protocolBuilder)
=> base.ConfigureProtocol(protocolBuilder).YieldsOutput<List<ChatMessage>>();
@@ -33,30 +46,105 @@ internal sealed class GroupChatHost(
{
this._manager ??= this._managerFactory(this._agents);
if (!await this._manager.ShouldTerminateAsync(messages, cancellationToken).ConfigureAwait(false))
// The delta arriving here is either the initial user input (turn 0) or the most recent speaker's
// response (subsequent turns) participants no longer echo incoming messages back to the host.
if (messages.Count > 0)
{
var filtered = await this._manager.UpdateHistoryAsync(messages, cancellationToken).ConfigureAwait(false);
messages = filtered is null || ReferenceEquals(filtered, messages) ? messages : [.. filtered];
this._history.AddRange(messages);
}
if (await this._manager.SelectNextAgentAsync(messages, cancellationToken).ConfigureAwait(false) is AIAgent nextAgent &&
this._agentMap.TryGetValue(nextAgent, out var executor))
if (await this._manager.ShouldTerminateAsync(this._history, cancellationToken).ConfigureAwait(false))
{
await this.CompleteAsync(context, cancellationToken).ConfigureAwait(false);
return;
}
if (messages.Count > 0)
{
IEnumerable<ChatMessage> filteredDelta = await this._manager.UpdateHistoryAsync(messages, cancellationToken).ConfigureAwait(false);
List<ChatMessage> broadcastMessages = filteredDelta is null
? messages
: (ReferenceEquals(filteredDelta, messages) ? messages : [.. filteredDelta]);
if (broadcastMessages.Count > 0)
{
this._manager.IterationCount++;
await context.SendMessageAsync(messages, executor.Id, cancellationToken).ConfigureAwait(false);
await context.SendMessageAsync(new TurnToken(emitEvents), executor.Id, cancellationToken).ConfigureAwait(false);
return;
await this.BroadcastAsync(broadcastMessages, context, cancellationToken).ConfigureAwait(false);
}
}
this._manager = null;
await context.YieldOutputAsync(messages, cancellationToken).ConfigureAwait(false);
if (await this._manager.SelectNextAgentAsync(this._history, cancellationToken).ConfigureAwait(false) is AIAgent nextAgent &&
this._agentMap.TryGetValue(nextAgent, out ExecutorBinding? executor))
{
this._manager.IterationCount++;
this._currentSpeakerExecutorId = executor.Id;
await context.SendMessageAsync(new TurnToken(emitEvents), executor.Id, cancellationToken).ConfigureAwait(false);
return;
}
await this.CompleteAsync(context, cancellationToken).ConfigureAwait(false);
}
private ValueTask BroadcastAsync(List<ChatMessage> messages, IWorkflowContext context, CancellationToken cancellationToken)
{
List<Task>? sendTasks = null;
foreach (ExecutorBinding participant in this._agentMap.Values)
{
if (string.Equals(participant.Id, this._currentSpeakerExecutorId, StringComparison.Ordinal))
{
continue;
}
(sendTasks ??= []).Add(context.SendMessageAsync(messages, participant.Id, cancellationToken).AsTask());
}
return sendTasks is null ? default : new ValueTask(Task.WhenAll(sendTasks));
}
private async ValueTask CompleteAsync(IWorkflowContext context, CancellationToken cancellationToken)
{
List<ChatMessage> output = this._history;
this._history = [];
this._currentSpeakerExecutorId = null;
this._manager = null;
await context.YieldOutputAsync(output, cancellationToken).ConfigureAwait(false);
}
protected override ValueTask ResetAsync()
{
this._manager = null;
this._history = [];
this._currentSpeakerExecutorId = null;
return base.ResetAsync();
}
ValueTask IResettableExecutor.ResetAsync() => this.ResetAsync();
protected internal override async ValueTask OnCheckpointingAsync(IWorkflowContext context, CancellationToken cancellationToken = default)
{
Task historyTask = context.QueueStateUpdateAsync(HistoryStateKey, this._history, cancellationToken: cancellationToken).AsTask();
Task currentSpeakerTask = context.QueueStateUpdateAsync(CurrentSpeakerStateKey, this._currentSpeakerExecutorId, cancellationToken: cancellationToken).AsTask();
Task baseTask = base.OnCheckpointingAsync(context, cancellationToken).AsTask();
// Eagerly materialize the manager so subclass state (e.g., the round-robin cursor) gets
// persisted on every checkpoint, even if no turn has been taken yet since the host was constructed.
this._manager ??= this._managerFactory(this._agents);
Task managerTask = this._manager.CheckpointAsync(context, cancellationToken).AsTask();
await Task.WhenAll(historyTask, currentSpeakerTask, baseTask, managerTask).ConfigureAwait(false);
}
protected internal override async ValueTask OnCheckpointRestoredAsync(IWorkflowContext context, CancellationToken cancellationToken = default)
{
this._history = await context.ReadStateAsync<List<ChatMessage>>(HistoryStateKey, cancellationToken: cancellationToken).ConfigureAwait(false) ?? [];
this._currentSpeakerExecutorId = await context.ReadStateAsync<string?>(CurrentSpeakerStateKey, cancellationToken: cancellationToken).ConfigureAwait(false);
// Instantiate the manager eagerly so its restore hook can rehydrate IterationCount and any
// subclass-defined state (e.g., RoundRobinGroupChatManager._nextIndex).
this._manager = this._managerFactory(this._agents);
await this._manager.RestoreCheckpointAsync(context, cancellationToken).ConfigureAwait(false);
await base.OnCheckpointRestoredAsync(context, cancellationToken).ConfigureAwait(false);
}
}
@@ -81,7 +81,6 @@ internal sealed record StateRef<TState>(string Key, string? ScopeName)
}
/// <summary>Executor used to represent an agent in a handoffs workflow, responding to <see cref="HandoffState"/> events.</summary>
[Experimental(DiagnosticConstants.ExperimentalFeatureDiagnostic)]
internal sealed class HandoffAgentExecutor :
StatefulExecutor<HandoffAgentHostState, HandoffState>
{
@@ -2,12 +2,10 @@
using System;
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
using Microsoft.Extensions.AI;
namespace Microsoft.Agents.AI.Workflows.Specialized;
[Experimental(DiagnosticConstants.ExperimentalFeatureDiagnostic)]
internal sealed class HandoffMessagesFilter
{
private readonly HandoffToolCallFilteringBehavior _filteringBehavior;
@@ -17,7 +15,6 @@ internal sealed class HandoffMessagesFilter
this._filteringBehavior = filteringBehavior;
}
[Experimental(DiagnosticConstants.ExperimentalFeatureDiagnostic)]
internal static bool IsHandoffFunctionName(string name)
{
return name.StartsWith(HandoffWorkflowBuilder.FunctionPrefix, StringComparison.Ordinal);
@@ -2,7 +2,6 @@
using System;
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
using System.Linq;
using System.Text.Json.Serialization;
using System.Threading;
@@ -18,7 +17,6 @@ namespace Microsoft.Agents.AI.Workflows.Specialized.Magentic;
[JsonDerivedType(typeof(MagenticPlanCreatedEvent))]
[JsonDerivedType(typeof(MagenticReplannedEvent))]
[JsonDerivedType(typeof(MagenticProgressLedgerUpdatedEvent))]
[Experimental(DiagnosticConstants.ExperimentalFeatureDiagnostic)]
public abstract class MagenticOrchestratorEvent(object? data) : WorkflowEvent(data)
{
}
@@ -27,7 +25,6 @@ public abstract class MagenticOrchestratorEvent(object? data) : WorkflowEvent(da
/// Represents the creation of the initial plan
/// </summary>
/// <param name="fullTaskLeger"></param>
[Experimental(DiagnosticConstants.ExperimentalFeatureDiagnostic)]
public sealed class MagenticPlanCreatedEvent(ChatMessage fullTaskLeger) : MagenticOrchestratorEvent(fullTaskLeger)
{
/// <summary>
@@ -40,7 +37,6 @@ public sealed class MagenticPlanCreatedEvent(ChatMessage fullTaskLeger) : Magent
/// Represents the creation of a new plan in response to a stall.
/// </summary>
/// <param name="fullTaskLeger"></param>
[Experimental(DiagnosticConstants.ExperimentalFeatureDiagnostic)]
public sealed class MagenticReplannedEvent(ChatMessage fullTaskLeger) : MagenticOrchestratorEvent(fullTaskLeger)
{
/// <summary>
@@ -53,7 +49,6 @@ public sealed class MagenticReplannedEvent(ChatMessage fullTaskLeger) : Magentic
/// Represents an update to the <see cref="MagenticProgressLedger"/> when running a coordination round.
/// </summary>
/// <param name="progressLedger"></param>
[Experimental(DiagnosticConstants.ExperimentalFeatureDiagnostic)]
public sealed class MagenticProgressLedgerUpdatedEvent(MagenticProgressLedger progressLedger) : MagenticOrchestratorEvent(progressLedger)
{
/// <summary>
@@ -138,7 +133,6 @@ internal class MagenticOrchestrator(AIAgent managerAgent, List<AIAgent> team, Ta
to the conversation and enters the inner loop.
- If revision requested, append the review comments to the chat history,
trigger replanning via the manager, emit a REPLANNED event, then run the outer loop.
*/
if (this._taskContext == null || this._taskContext.TaskLedger == null)
{
@@ -201,7 +195,12 @@ internal class MagenticOrchestrator(AIAgent managerAgent, List<AIAgent> team, Ta
}
else
{
// Subsequent turns: agent returned control, go directly to coordination (progress ledger only, no replan)
// Subsequent turns: agent returned control, go directly to coordination (progress ledger only, no replan).
// Capture the participant's reply into the manager-visible chat history so the progress ledger can see it.
if (messages is { Count: > 0 })
{
this._taskContext.ChatHistory.AddRange(messages);
}
await this.RunCoordinationRoundAsync(this._taskContext, context, cancellationToken).ConfigureAwait(false);
}
}
@@ -24,7 +24,7 @@ public class Workflow
internal Dictionary<string, ExecutorBinding> ExecutorBindings { get; init; } = [];
internal Dictionary<string, HashSet<Edge>> Edges { get; init; } = [];
internal HashSet<string> OutputExecutors { get; init; } = [];
internal Dictionary<string, HashSet<OutputTag>> OutputExecutors { get; init; } = new(StringComparer.Ordinal);
/// <summary>
/// Gets the collection of edges grouped by their source node identifier.
@@ -221,7 +221,7 @@ public class Workflow
startExecutor.AttachRequestContext(new NoOpExternalRequestContext());
ProtocolDescriptor inputProtocol = startExecutor.DescribeProtocol();
IEnumerable<Task<Executor>> outputExecutorTasks = this.OutputExecutors.Select(executorId => this.ExecutorBindings[executorId].CreateInstanceAsync(string.Empty).AsTask());
IEnumerable<Task<Executor>> outputExecutorTasks = this.OutputExecutors.Keys.Select(executorId => this.ExecutorBindings[executorId].CreateInstanceAsync(string.Empty).AsTask());
Executor[] outputExecutors = await Task.WhenAll(outputExecutorTasks).ConfigureAwait(false);
IEnumerable<Type> yieldedTypes = outputExecutors.SelectMany(executor => executor.DescribeProtocol().Yields);
@@ -33,7 +33,7 @@ public class WorkflowBuilder
private readonly HashSet<string> _unboundExecutors = [];
private readonly HashSet<EdgeConnection> _conditionlessConnections = [];
private readonly Dictionary<string, RequestPort> _requestPorts = [];
private readonly HashSet<string> _outputExecutors = [];
private readonly Dictionary<string, HashSet<OutputTag>> _outputExecutors = new(StringComparer.Ordinal);
private readonly string _startExecutorId;
private string? _name;
@@ -97,22 +97,89 @@ public class WorkflowBuilder
}
/// <summary>
/// Register executors as an output source. Executors can use <see cref="IWorkflowContext.YieldOutputAsync"/> to yield output values.
/// By default, message handlers with a non-void return type will also be yielded, unless <see cref="ExecutorOptions.AutoYieldOutputHandlerResultObject"/>
/// is set to <see langword="false"/>.
/// Register executors as a source of terminal workflow outputs. Executors can use
/// <see cref="IWorkflowContext.YieldOutputAsync"/> to yield output values; yielded values from
/// registered executors are surfaced as <see cref="WorkflowOutputEvent"/> (or one of its
/// subclasses) with an empty <see cref="WorkflowOutputEvent.Tags"/> set.
/// By default, message handlers with a non-void return type will also be yielded, unless
/// <see cref="ExecutorOptions.AutoYieldOutputHandlerResultObject"/> is set to <see langword="false"/>.
/// </summary>
/// <param name="executors"></param>
/// <returns></returns>
/// <remarks>
/// AIAgent payloads (<see cref="AgentResponse"/> / <see cref="AgentResponseUpdate"/>) only
/// participate in this designation when
/// <see cref="Futures.EnableAgentResponseOutputTaggingAndFiltering"/> is
/// <see langword="true"/>; otherwise they are emitted unconditionally and untagged.
/// </remarks>
/// <param name="executors">The executors to register as output sources.</param>
/// <returns>The current <see cref="WorkflowBuilder"/> instance, enabling fluent configuration.</returns>
public WorkflowBuilder WithOutputFrom(params ExecutorBinding[] executors)
{
foreach (ExecutorBinding executor in executors)
{
this._outputExecutors.Add(this.Track(executor).Id);
this.EnsureOutputExecutor(this.Track(executor).Id);
}
return this;
}
/// <summary>
/// Register executors as a source of workflow outputs carrying the given <paramref name="tag"/>.
/// Tags accumulate across repeated calls; the registered id always exists with the union of all
/// tags applied across all calls (and an empty set if only the untagged
/// <see cref="WithOutputFrom(ExecutorBinding[])"/> overload was used).
/// </summary>
/// <remarks>
/// Forward-looking surface for when the <see cref="OutputTag"/> constructor opens to
/// user-defined tags. Today, prefer
/// <see cref="WorkflowBuilderExtensions.WithIntermediateOutputFrom(WorkflowBuilder, IEnumerable{ExecutorBinding})"/>
/// for the <see cref="OutputTag.Intermediate"/> case.
/// </remarks>
/// <param name="executors">The executors to register.</param>
/// <param name="tag">The tag to apply to events yielded by the listed executors.</param>
/// <returns>The current <see cref="WorkflowBuilder"/> instance, enabling fluent configuration.</returns>
public WorkflowBuilder WithOutputFrom(IEnumerable<ExecutorBinding> executors, OutputTag tag)
{
Throw.IfNull(executors);
foreach (ExecutorBinding executor in executors)
{
this.EnsureOutputExecutor(this.Track(executor).Id).Add(tag);
}
return this;
}
/// <summary>
/// Register a single executor as a source of workflow outputs carrying the given <paramref name="tag"/>.
/// Convenience overload for the single-executor case; equivalent to passing a one-element sequence
/// to <see cref="WithOutputFrom(IEnumerable{ExecutorBinding}, OutputTag)"/>.
/// </summary>
/// <param name="executor">The executor to register.</param>
/// <param name="tag">The tag to apply to events yielded by the executor.</param>
/// <returns>The current <see cref="WorkflowBuilder"/> instance, enabling fluent configuration.</returns>
public WorkflowBuilder WithOutputFrom(ExecutorBinding executor, OutputTag tag)
{
Throw.IfNull(executor);
this.EnsureOutputExecutor(this.Track(executor).Id).Add(tag);
return this;
}
/// <summary>
/// Ensures the executor id is present in <see cref="_outputExecutors"/>; if newly added,
/// initializes with an empty tag set. Returns the tag set for the id (mutable).
/// </summary>
private HashSet<OutputTag> EnsureOutputExecutor(string executorId)
{
if (!this._outputExecutors.TryGetValue(executorId, out HashSet<OutputTag>? tags))
{
tags = [];
this._outputExecutors[executorId] = tags;
}
return tags;
}
/// <summary>
/// Sets the human-readable name for the workflow.
/// </summary>
@@ -211,4 +211,28 @@ public static class WorkflowBuilderExtensions
return switchBuilder.ReduceToFanOut(builder, source);
}
/// <summary>
/// Register executors as a source of <b>intermediate</b> workflow outputs. The resulting
/// <see cref="WorkflowOutputEvent"/>s carry <see cref="OutputTag.Intermediate"/> in their
/// <see cref="WorkflowOutputEvent.Tags"/> set, and
/// <see cref="WorkflowOutputEventExtensions.IsIntermediate(WorkflowOutputEvent)"/> returns
/// <see langword="true"/>. Use this for progress updates, partial results, and other
/// non-terminal emissions that downstream consumers (DevUI, logging, Workflow-as-Agent
/// surfaces) should see distinctly from the workflow's final output.
/// </summary>
/// <remarks>
/// AIAgent payloads (<see cref="AgentResponse"/> / <see cref="AgentResponseUpdate"/>) only
/// participate in this designation when
/// <see cref="Futures.EnableAgentResponseOutputTaggingAndFiltering"/> is
/// <see langword="true"/>; otherwise they bypass the filter and are emitted untagged.
/// </remarks>
/// <param name="builder">The workflow builder to register executors on.</param>
/// <param name="executors">The executors to register as intermediate output sources.</param>
/// <returns>The <paramref name="builder"/>, enabling fluent configuration.</returns>
public static WorkflowBuilder WithIntermediateOutputFrom(this WorkflowBuilder builder, IEnumerable<ExecutorBinding> executors)
{
Throw.IfNull(builder);
return builder.WithOutputFrom(executors, OutputTag.Intermediate);
}
}
@@ -1,6 +1,7 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
using System.Text.Json.Serialization;
@@ -13,14 +14,39 @@ namespace Microsoft.Agents.AI.Workflows;
[JsonDerivedType(typeof(AgentResponseUpdateEvent))]
public class WorkflowOutputEvent : WorkflowEvent
{
private readonly HashSet<OutputTag> _tags;
/// <summary>
/// Initializes a new instance of the <see cref="WorkflowOutputEvent"/> class.
/// Initializes a new instance of the <see cref="WorkflowOutputEvent"/> class with no tags.
/// </summary>
/// <param name="data">The output data.</param>
/// <param name="executorId">The identifier of the executor that yielded this output.</param>
public WorkflowOutputEvent(object data, string executorId) : base(data)
public WorkflowOutputEvent(object data, string executorId) : this(data, executorId, tags: null)
{
}
/// <summary>
/// Initializes a new instance of the <see cref="WorkflowOutputEvent"/> class carrying the
/// given output tag.
/// </summary>
/// <param name="data">The output data.</param>
/// <param name="executorId">The identifier of the executor that yielded this output.</param>
/// <param name="tag">The single output tag to associate with this event.</param>
public WorkflowOutputEvent(object data, string executorId, OutputTag tag) : this(data, executorId, tags: new[] { tag })
{
}
/// <summary>
/// Initializes a new instance of the <see cref="WorkflowOutputEvent"/> class carrying the
/// given output tags (deduplicated).
/// </summary>
/// <param name="data">The output data.</param>
/// <param name="executorId">The identifier of the executor that yielded this output.</param>
/// <param name="tags">The output tags to associate with this event. May be <see langword="null"/> or empty (the event is then untagged).</param>
public WorkflowOutputEvent(object data, string executorId, IEnumerable<OutputTag>? tags) : base(data)
{
this.ExecutorId = executorId;
this._tags = tags is null ? new HashSet<OutputTag>() : new HashSet<OutputTag>(tags);
}
/// <summary>
@@ -32,8 +58,21 @@ public class WorkflowOutputEvent : WorkflowEvent
/// The unique identifier of the executor that yielded this output.
/// </summary>
[Obsolete("Use ExecutorId instead.")]
[JsonIgnore]
public string SourceId => this.ExecutorId;
/// <summary>
/// The set of output tags associated with this event. Never <see langword="null"/>;
/// empty for terminal/regular outputs. The presence of <see cref="OutputTag.Intermediate"/>
/// marks this event as an intermediate output.
/// </summary>
public IEnumerable<OutputTag> Tags => this._tags;
/// <summary>
/// Returns <see langword="true"/> if this event carries the given tag.
/// </summary>
public bool HasTag(OutputTag tag) => this._tags.Contains(tag);
/// <summary>
/// Determines whether the underlying data is of the specified type or a derived type.
/// </summary>
@@ -0,0 +1,21 @@
// Copyright (c) Microsoft. All rights reserved.
using Microsoft.Shared.Diagnostics;
namespace Microsoft.Agents.AI.Workflows;
/// <summary>
/// Extension helpers for inspecting <see cref="WorkflowOutputEvent"/> tag membership.
/// </summary>
public static class WorkflowOutputEventExtensions
{
/// <summary>
/// Returns <see langword="true"/> if the event carries
/// <see cref="OutputTag.Intermediate"/> in its <see cref="WorkflowOutputEvent.Tags"/>.
/// </summary>
public static bool IsIntermediate(this WorkflowOutputEvent evt)
{
Throw.IfNull(evt);
return evt.HasTag(OutputTag.Intermediate);
}
}
@@ -520,11 +520,20 @@ internal sealed class WorkflowSession : AgentSession
goto default;
case AgentResponseEvent agentResponse:
if (!this._includeWorkflowOutputsInResponse)
// Under Futures.EnableAgentResponseOutputTaggingAndFiltering=true, mirror
// AgentResponseUpdateEvent's behavior: always forward, regardless of the
// _includeWorkflowOutputsInResponse host flag / "intermediate" tag. Under
// the legacy default, keep today's behavior — gated by the include flag.
if (!Futures.EnableAgentResponseOutputTaggingAndFiltering && !this._includeWorkflowOutputsInResponse)
{
goto default;
}
// Either EnableAgentResponseOutputTaggingAndFiltering -- so yield the Response
// regardless of whether it is tagged "intermediate" or whether the
// _includeWorkflowOutputInResponse flag is set. Reason being: The user specifies
// exclusion of an event by enabling filtering and then _not_ marking an Executor
// as an output executor.
foreach (ChatMessage message in agentResponse.Response.Messages)
{
yield return this.CreateUpdate(this.LastResponseId, evt, message);
@@ -539,7 +548,11 @@ internal sealed class WorkflowSession : AgentSession
_ => null
};
if (!this._includeWorkflowOutputsInResponse || updateMessages == null)
// Same assymetry as with AgentResponseEvent, but there is no EnableFiltering flag
// to consider. If this made it here (and since it is not an AgentResponse[Update]),
// it means it is already been selected as an Output() from the user. Intermediate
// is irrelevant here.
if (updateMessages == null || !this._includeWorkflowOutputsInResponse)
{
goto default;
}
@@ -80,9 +80,8 @@ internal static partial class WorkflowsJsonUtilities
[JsonSerializable(typeof(ExecutorIdentity))]
[JsonSerializable(typeof(RunnerStateData))]
// Workflow Representation Types
[JsonSerializable(typeof(WorkflowInfo))]
[JsonSerializable(typeof(EdgeConnection))]
// Workflow Output Types
[JsonSerializable(typeof(OutputTag))]
// Workflow-as-Agent
[JsonSerializable(typeof(WorkflowChatHistoryProvider.StoreState))]
@@ -101,6 +100,8 @@ internal static partial class WorkflowsJsonUtilities
[JsonSerializable(typeof(MagenticPlanReviewRequest))]
[JsonSerializable(typeof(MagenticPlanReviewResponse))]
[JsonSerializable(typeof(MagenticTaskState))]
[JsonSerializable(typeof(GroupChatManagerState))]
[JsonSerializable(typeof(RoundRobinGroupChatManagerState))]
[JsonSerializable(typeof(ResetChatSignal))]
// Event Types
@@ -222,6 +222,42 @@ public class AgentResponseTests
Assert.Equal(100, usageContent.Details.TotalTokenCount);
}
[Fact]
public void ToAgentResponseUpdatesPropagatesCreatedAt()
{
// Sets different CreatedAt values on the AgentResponse and the ChatMessage to verify that the ChatMessage.CreatedAt is the one that gets propagated to the AgentResponseUpdate
AgentResponse response = new(new ChatMessage(new ChatRole("customRole"), "Text") { MessageId = "someMessage", CreatedAt = new DateTimeOffset(2024, 11, 11, 9, 20, 0, TimeSpan.Zero) })
{
AgentId = "agentId",
ResponseId = "12345",
CreatedAt = new DateTimeOffset(2024, 11, 10, 9, 20, 0, TimeSpan.Zero),
AdditionalProperties = new() { ["key1"] = "value1", ["key2"] = 42 },
Usage = new UsageDetails
{
TotalTokenCount = 100
},
};
AgentResponseUpdate[] updates = response.ToAgentResponseUpdates();
Assert.NotNull(updates);
Assert.Equal(2, updates.Length);
AgentResponseUpdate update0 = updates[0];
Assert.Equal("agentId", update0.AgentId);
Assert.Equal("12345", update0.ResponseId);
Assert.Equal("someMessage", update0.MessageId);
Assert.Equal(new DateTimeOffset(2024, 11, 11, 9, 20, 0, TimeSpan.Zero), update0.CreatedAt);
Assert.Equal("customRole", update0.Role?.Value);
Assert.Equal("Text", update0.Text);
AgentResponseUpdate update1 = updates[1];
Assert.Equal("value1", update1.AdditionalProperties?["key1"]);
Assert.Equal(42, update1.AdditionalProperties?["key2"]);
Assert.IsType<UsageContent>(update1.Contents[0]);
UsageContent usageContent = (UsageContent)update1.Contents[0];
Assert.Equal(100, usageContent.Details.TotalTokenCount);
}
[Fact]
public void ParseAsStructuredOutputWithJSOSuccess()
{
@@ -172,7 +172,7 @@ public class DevUIIntegrationTests
Assert.Contains(discoveryResponse.Entities, e => e.Name == "workflow-three" && e.Type == "workflow");
}
[Fact]
[Fact(Skip = "Flaky in merge_group; see https://github.com/microsoft/agent-framework/issues/5845")]
public async Task TestServerWithDevUI_ResolvesWorkflows_WithKeyedAndDefaultRegistrationAsync()
{
// Arrange
@@ -8,6 +8,8 @@
<ItemGroup>
<PackageReference Include="Microsoft.Extensions.Logging" />
<PackageReference Include="Azure.AI.Projects" />
<PackageReference Include="Azure.Core" />
<PackageReference Include="System.ClientModel" />
</ItemGroup>
<ItemGroup>
@@ -1,9 +1,11 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Agents.AI.Workflows.Declarative.Events;
using Microsoft.Agents.AI.Workflows.Declarative.Interpreter;
using Microsoft.Agents.AI.Workflows.Declarative.Kit;
using Microsoft.Agents.AI.Workflows.Declarative.ObjectModel;
using Microsoft.Agents.AI.Workflows.Declarative.PowerFx;
@@ -290,6 +292,151 @@ public sealed class InvokeMcpToolExecutorTest(ITestOutputHelper output) : Workfl
await this.ExecuteTestAsync(model);
}
[Fact]
public async Task InvokeMcpToolApprovalRequestExcludesTransportHeadersAsync()
{
// Arrange
this.State.InitializeSystem();
InvokeMcpTool model = this.CreateModel(
displayName: nameof(InvokeMcpToolApprovalRequestExcludesTransportHeadersAsync),
serverUrl: TestServerUrl,
serverLabel: TestServerLabel,
toolName: TestToolName,
requireApproval: true,
headerKey: "Authorization",
headerValue: "Bearer super-secret-token");
MockMcpToolProvider mockProvider = new();
MockAgentProvider mockAgentProvider = new();
InvokeMcpToolExecutor action = new(model, mockProvider.Object, mockAgentProvider.Object, this.State);
ExternalInputRequest? capturedRequest = null;
// Act
await this.ExecuteAsync(
[
action,
new DelegateActionExecutor<ExternalInputRequest>(
InvokeMcpToolExecutor.Steps.ExternalInput(action.Id),
this.State,
CaptureRequestAsync)
],
isDiscrete: false);
// Assert - the approval event must not carry any transport headers (e.g. Authorization).
Assert.NotNull(capturedRequest);
ToolApprovalRequestContent approvalRequest =
capturedRequest!.AgentResponse.Messages
.SelectMany(message => message.Contents)
.OfType<ToolApprovalRequestContent>()
.Single();
AdditionalPropertiesDictionary? additionalProperties = approvalRequest.ToolCall.AdditionalProperties;
Assert.True(additionalProperties is null || additionalProperties.Count == 0);
// Defense in depth: the credential value must not appear anywhere in the serialized approval content.
string serializedApproval = System.Text.Json.JsonSerializer.Serialize(capturedRequest.AgentResponse);
Assert.DoesNotContain("super-secret-token", serializedApproval);
ValueTask CaptureRequestAsync(IWorkflowContext context, ExternalInputRequest request, CancellationToken cancellationToken)
{
capturedRequest = request;
return default;
}
}
[Fact]
public async Task InvokeMcpToolInvocationForwardsHeadersToTransportAsync()
{
// Arrange
this.State.InitializeSystem();
const string HeaderKey = "Authorization";
const string HeaderValue = "Bearer super-secret-token";
InvokeMcpTool model = this.CreateModel(
displayName: nameof(InvokeMcpToolInvocationForwardsHeadersToTransportAsync),
serverUrl: TestServerUrl,
serverLabel: TestServerLabel,
toolName: TestToolName,
requireApproval: false,
headerKey: HeaderKey,
headerValue: HeaderValue);
IDictionary<string, string>? capturedHeaders = null;
Mock<IMcpToolHandler> mockProvider = new();
mockProvider
.Setup(provider => provider.InvokeToolAsync(
It.IsAny<string>(),
It.IsAny<string?>(),
It.IsAny<string>(),
It.IsAny<IDictionary<string, object?>?>(),
It.IsAny<IDictionary<string, string>?>(),
It.IsAny<string?>(),
It.IsAny<CancellationToken>()))
.Callback<string, string?, string, IDictionary<string, object?>?, IDictionary<string, string>?, string?, CancellationToken>(
(_, _, _, _, headers, _, _) => capturedHeaders = headers)
.ReturnsAsync(new McpServerToolResultContent("mock-call-id") { Outputs = [new TextContent("ok")] });
MockAgentProvider mockAgentProvider = new();
InvokeMcpToolExecutor action = new(model, mockProvider.Object, mockAgentProvider.Object, this.State);
// Act
await this.ExecuteAsync(action, isDiscrete: false);
// Assert - headers remain available to the actual transport invocation.
Assert.NotNull(capturedHeaders);
Assert.True(capturedHeaders!.TryGetValue(HeaderKey, out string? forwardedValue));
Assert.Equal(HeaderValue, forwardedValue);
}
[Fact]
public async Task InvokeMcpToolApprovedCaptureResponseForwardsHeadersToTransportAsync()
{
// Arrange - exercises the post-approval CaptureResponseAsync resume path to prove the
// fix did not regress header forwarding on the path that the vulnerability actually targets.
this.State.InitializeSystem();
const string HeaderKey = "Authorization";
const string HeaderValue = "Bearer super-secret-token";
InvokeMcpTool model = this.CreateModel(
displayName: nameof(InvokeMcpToolApprovedCaptureResponseForwardsHeadersToTransportAsync),
serverUrl: TestServerUrl,
serverLabel: TestServerLabel,
toolName: TestToolName,
requireApproval: true,
headerKey: HeaderKey,
headerValue: HeaderValue);
IDictionary<string, string>? capturedHeaders = null;
Mock<IMcpToolHandler> mockProvider = new();
mockProvider
.Setup(provider => provider.InvokeToolAsync(
It.IsAny<string>(),
It.IsAny<string?>(),
It.IsAny<string>(),
It.IsAny<IDictionary<string, object?>?>(),
It.IsAny<IDictionary<string, string>?>(),
It.IsAny<string?>(),
It.IsAny<CancellationToken>()))
.Callback<string, string?, string, IDictionary<string, object?>?, IDictionary<string, string>?, string?, CancellationToken>(
(_, _, _, _, headers, _, _) => capturedHeaders = headers)
.ReturnsAsync(new McpServerToolResultContent("mock-call-id") { Outputs = [new TextContent("ok")] });
MockAgentProvider mockAgentProvider = new();
InvokeMcpToolExecutor action = new(model, mockProvider.Object, mockAgentProvider.Object, this.State);
Mock<IWorkflowContext> mockContext = new(MockBehavior.Loose);
// Build an approved response matching this action's request id.
McpServerToolCallContent toolCall = new(action.Id, TestToolName, TestServerLabel);
ToolApprovalRequestContent approvalRequest = new(action.Id, toolCall);
ToolApprovalResponseContent approvalResponse = approvalRequest.CreateResponse(approved: true);
ExternalInputResponse response = new(new ChatMessage(ChatRole.User, [approvalResponse]));
// Act - call CaptureResponseAsync directly so the post-approval branch actually executes.
await action.CaptureResponseAsync(mockContext.Object, response, CancellationToken.None);
// Assert - headers reach the transport invocation on the approved path.
Assert.NotNull(capturedHeaders);
Assert.True(capturedHeaders!.TryGetValue(HeaderKey, out string? forwardedValue));
Assert.Equal(HeaderValue, forwardedValue);
}
[Fact]
public async Task InvokeMcpToolExecuteWithEmptyHeaderValueAsync()
{
@@ -4,12 +4,9 @@ using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.CompilerServices;
using System.Text;
using System.Text.Json;
using System.Text.RegularExpressions;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Agents.AI.Workflows.InProc;
using FluentAssertions;
using Microsoft.Extensions.AI;
#pragma warning disable SYSLIB1045 // Use GeneratedRegex
@@ -17,325 +14,164 @@ using Microsoft.Extensions.AI;
namespace Microsoft.Agents.AI.Workflows.UnitTests;
/// <summary>
/// Tests targeting the static <see cref="AgentWorkflowBuilder"/> helper surface —
/// <see cref="AgentWorkflowBuilder.BuildSequential(IEnumerable{AIAgent})"/>,
/// <see cref="AgentWorkflowBuilder.BuildConcurrent(IEnumerable{AIAgent}, Func{IList{List{ChatMessage}}, List{ChatMessage}})"/>,
/// and the various <c>Create*BuilderWith</c> factories. Per-builder unit tests live in their own
/// files (<see cref="SequentialWorkflowBuilderTests"/>, <see cref="ConcurrentWorkflowBuilderTests"/>, etc.).
/// </summary>
public class AgentWorkflowBuilderTests
{
[Fact]
public void BuildSequential_InvalidArguments_Throws()
public void Test_AgentWorkflowBuilder_BuildSequential_InvalidArguments_Throws()
{
Assert.Throws<ArgumentNullException>("agents", () => AgentWorkflowBuilder.BuildSequential(workflowName: null!, null!));
Assert.Throws<ArgumentException>("agents", () => AgentWorkflowBuilder.BuildSequential());
}
[Theory]
[InlineData(1)]
[InlineData(2)]
[InlineData(3)]
public async Task Test_AgentWorkflowBuilder_BuildSequential_DelegatesToBuilderAsync(int numAgents)
{
Workflow workflow = AgentWorkflowBuilder.BuildSequential(
from i in Enumerable.Range(1, numAgents)
select new OrchestrationTestHelpers.DoubleEchoAgent($"agent{i}"));
// Smoke: end-to-end run produces a non-empty result. Detailed pipeline-ordering
// assertions live in SequentialWorkflowBuilderTests.
(string updateText, List<ChatMessage>? result, _, _) =
await OrchestrationTestHelpers.RunWorkflowAsync(workflow, [new ChatMessage(ChatRole.User, "abc")]);
Assert.NotNull(result);
Assert.Equal(numAgents + 1, result.Count);
Assert.NotEmpty(updateText);
}
[Fact]
public void BuildConcurrent_InvalidArguments_Throws()
public void Test_AgentWorkflowBuilder_BuildSequential_WithWorkflowNameSetsNameOnWorkflow()
{
Workflow workflow = AgentWorkflowBuilder.BuildSequential(
"static-sequential",
new OrchestrationTestHelpers.DoubleEchoAgent("agent1"));
workflow.Name.Should().Be("static-sequential");
}
[Fact]
public void Test_AgentWorkflowBuilder_BuildConcurrent_InvalidArguments_Throws()
{
Assert.Throws<ArgumentNullException>("agents", () => AgentWorkflowBuilder.BuildConcurrent(null!));
}
[Fact]
public void BuildGroupChat_InvalidArguments_Throws()
{
Assert.Throws<ArgumentNullException>("managerFactory", () => AgentWorkflowBuilder.CreateGroupChatBuilderWith(null!));
var groupChat = AgentWorkflowBuilder.CreateGroupChatBuilderWith(_ => new RoundRobinGroupChatManager([new DoubleEchoAgent("a1")]));
Assert.NotNull(groupChat);
Assert.Throws<ArgumentNullException>("agents", () => groupChat.AddParticipants(null!));
Assert.Throws<ArgumentNullException>("agents", () => groupChat.AddParticipants([null!]));
Assert.Throws<ArgumentNullException>("agents", () => groupChat.AddParticipants(new DoubleEchoAgent("a1"), null!));
Assert.Throws<ArgumentNullException>("agents", () => new RoundRobinGroupChatManager(null!));
}
[Fact]
public void GroupChatManager_MaximumIterationCount_Invalid_Throws()
{
var manager = new RoundRobinGroupChatManager([new DoubleEchoAgent("a1")]);
const int DefaultMaxIterations = 40;
Assert.Equal(DefaultMaxIterations, manager.MaximumIterationCount);
Assert.Throws<ArgumentOutOfRangeException>("value", void () => manager.MaximumIterationCount = 0);
Assert.Throws<ArgumentOutOfRangeException>("value", void () => manager.MaximumIterationCount = -1);
Assert.Equal(DefaultMaxIterations, manager.MaximumIterationCount);
manager.MaximumIterationCount = 30;
Assert.Equal(30, manager.MaximumIterationCount);
manager.MaximumIterationCount = 1;
Assert.Equal(1, manager.MaximumIterationCount);
manager.MaximumIterationCount = int.MaxValue;
Assert.Equal(int.MaxValue, manager.MaximumIterationCount);
}
[Fact]
public void BuildGroupChat_WithNameAndDescription_SetsWorkflowNameAndDescription()
{
const string WorkflowName = "Test Group Chat";
const string WorkflowDescription = "A test group chat workflow";
var workflow = AgentWorkflowBuilder
.CreateGroupChatBuilderWith(agents => new RoundRobinGroupChatManager(agents) { MaximumIterationCount = 2 })
.AddParticipants(new DoubleEchoAgent("agent1"), new DoubleEchoAgent("agent2"))
.WithName(WorkflowName)
.WithDescription(WorkflowDescription)
.Build();
Assert.Equal(WorkflowName, workflow.Name);
Assert.Equal(WorkflowDescription, workflow.Description);
}
[Fact]
public void BuildGroupChat_WithNameOnly_SetsWorkflowName()
{
const string WorkflowName = "Named Group Chat";
var workflow = AgentWorkflowBuilder
.CreateGroupChatBuilderWith(agents => new RoundRobinGroupChatManager(agents) { MaximumIterationCount = 2 })
.AddParticipants(new DoubleEchoAgent("agent1"))
.WithName(WorkflowName)
.Build();
Assert.Equal(WorkflowName, workflow.Name);
Assert.Null(workflow.Description);
}
[Fact]
public void BuildGroupChat_WithoutNameOrDescription_DefaultsToNull()
{
var workflow = AgentWorkflowBuilder
.CreateGroupChatBuilderWith(agents => new RoundRobinGroupChatManager(agents) { MaximumIterationCount = 2 })
.AddParticipants(new DoubleEchoAgent("agent1"))
.Build();
Assert.Null(workflow.Name);
Assert.Null(workflow.Description);
}
[Theory]
[InlineData(1)]
[InlineData(2)]
[InlineData(3)]
[InlineData(4)]
[InlineData(5)]
public async Task BuildSequential_AgentsRunInOrderAsync(int numAgents)
{
var workflow = AgentWorkflowBuilder.BuildSequential(
from i in Enumerable.Range(1, numAgents)
select new DoubleEchoAgent($"agent{i}"));
for (int iter = 0; iter < 3; iter++)
{
const string UserInput = "abc";
(string updateText, List<ChatMessage>? result, _, _) = await RunWorkflowAsync(workflow, [new ChatMessage(ChatRole.User, UserInput)]);
Assert.NotNull(result);
Assert.Equal(numAgents + 1, result.Count);
Assert.Equal(ChatRole.User, result[0].Role);
Assert.Null(result[0].AuthorName);
Assert.Equal(UserInput, result[0].Text);
string[] texts = new string[numAgents + 1];
texts[0] = UserInput;
string expectedTotal = string.Empty;
for (int i = 1; i < numAgents + 1; i++)
{
string id = $"agent{((i - 1) % numAgents) + 1}";
texts[i] = $"{id}{Double(string.Concat(texts.Take(i)))}";
Assert.Equal(ChatRole.Assistant, result[i].Role);
Assert.Equal(id, result[i].AuthorName);
Assert.Equal(texts[i], result[i].Text);
expectedTotal += texts[i];
}
Assert.Equal(expectedTotal, updateText);
Assert.Equal(UserInput + expectedTotal, string.Concat(result));
static string Double(string s) => s + s;
}
}
private class DoubleEchoAgent(string name) : AIAgent
{
public override string Name => name;
protected override ValueTask<AgentSession> CreateSessionCoreAsync(CancellationToken cancellationToken = default)
=> new(new DoubleEchoAgentSession());
protected override ValueTask<AgentSession> DeserializeSessionCoreAsync(JsonElement serializedState, JsonSerializerOptions? jsonSerializerOptions = null, CancellationToken cancellationToken = default)
=> new(new DoubleEchoAgentSession());
protected override ValueTask<JsonElement> SerializeSessionCoreAsync(AgentSession session, JsonSerializerOptions? jsonSerializerOptions = null, CancellationToken cancellationToken = default)
=> default;
protected override Task<AgentResponse> RunCoreAsync(
IEnumerable<ChatMessage> messages, AgentSession? session = null, AgentRunOptions? options = null, CancellationToken cancellationToken = default) =>
throw new NotImplementedException();
protected override async IAsyncEnumerable<AgentResponseUpdate> RunCoreStreamingAsync(
IEnumerable<ChatMessage> messages, AgentSession? session = null, AgentRunOptions? options = null, [EnumeratorCancellation] CancellationToken cancellationToken = default)
{
await Task.Yield();
var contents = messages.SelectMany(m => m.Contents).ToList();
string id = Guid.NewGuid().ToString("N");
yield return new AgentResponseUpdate(ChatRole.Assistant, this.Name) { AuthorName = this.Name, MessageId = id };
yield return new AgentResponseUpdate(ChatRole.Assistant, contents) { AuthorName = this.Name, MessageId = id };
yield return new AgentResponseUpdate(ChatRole.Assistant, contents) { AuthorName = this.Name, MessageId = id };
}
}
private sealed class DoubleEchoAgentSession() : AgentSession();
[Fact]
public async Task BuildConcurrent_AgentsRunInParallelAsync()
public async Task Test_AgentWorkflowBuilder_BuildConcurrent_DelegatesToBuilderAsync()
{
StrongBox<TaskCompletionSource<bool>> barrier = new();
StrongBox<int> remaining = new();
var workflow = AgentWorkflowBuilder.BuildConcurrent(
Workflow workflow = AgentWorkflowBuilder.BuildConcurrent(
[
new DoubleEchoAgentWithBarrier("agent1", barrier, remaining),
new DoubleEchoAgentWithBarrier("agent2", barrier, remaining),
new OrchestrationTestHelpers.DoubleEchoAgentWithBarrier("agent1", barrier, remaining),
new OrchestrationTestHelpers.DoubleEchoAgentWithBarrier("agent2", barrier, remaining),
]);
for (int iter = 0; iter < 3; iter++)
{
barrier.Value = new TaskCompletionSource<bool>(TaskCreationOptions.RunContinuationsAsynchronously);
remaining.Value = 2;
barrier.Value = new TaskCompletionSource<bool>(TaskCreationOptions.RunContinuationsAsynchronously);
remaining.Value = 2;
(string updateText, List<ChatMessage>? result, _, _) = await RunWorkflowAsync(workflow, [new ChatMessage(ChatRole.User, "abc")]);
Assert.NotEmpty(updateText);
Assert.NotNull(result);
(string updateText, List<ChatMessage>? result, _, _) =
await OrchestrationTestHelpers.RunWorkflowAsync(workflow, [new ChatMessage(ChatRole.User, "abc")]);
// TODO: https://github.com/microsoft/agent-framework/issues/784
// These asserts are flaky until we guarantee message delivery order.
Assert.Single(Regex.Matches(updateText, "agent1"));
Assert.Single(Regex.Matches(updateText, "agent2"));
Assert.Equal(4, Regex.Matches(updateText, "abc").Count);
Assert.Equal(2, result.Count);
}
Assert.NotEmpty(updateText);
Assert.NotNull(result);
Assert.Equal(2, result.Count);
Assert.Single(Regex.Matches(updateText, "agent1"));
Assert.Single(Regex.Matches(updateText, "agent2"));
}
[Theory]
[InlineData(1)]
[InlineData(2)]
[InlineData(3)]
[InlineData(4)]
[InlineData(5)]
public async Task BuildGroupChat_AgentsRunInOrderAsync(int maxIterations)
[Fact]
public void Test_AgentWorkflowBuilder_BuildConcurrent_WithWorkflowNameSetsNameOnWorkflow()
{
const int NumAgents = 3;
var workflow = AgentWorkflowBuilder.CreateGroupChatBuilderWith(agents => new RoundRobinGroupChatManager(agents) { MaximumIterationCount = maxIterations })
.AddParticipants(new DoubleEchoAgent("agent1"), new DoubleEchoAgent("agent2"))
.AddParticipants(new DoubleEchoAgent("agent3"))
.Build();
Workflow workflow = AgentWorkflowBuilder.BuildConcurrent(
"static-concurrent",
[new OrchestrationTestHelpers.DoubleEchoAgent("agent1")]);
for (int iter = 0; iter < 3; iter++)
{
const string UserInput = "abc";
(string updateText, List<ChatMessage>? result, _, _) = await RunWorkflowAsync(workflow, [new ChatMessage(ChatRole.User, UserInput)]);
Assert.NotNull(result);
Assert.Equal(maxIterations + 1, result.Count);
Assert.Equal(ChatRole.User, result[0].Role);
Assert.Null(result[0].AuthorName);
Assert.Equal(UserInput, result[0].Text);
string[] texts = new string[maxIterations + 1];
texts[0] = UserInput;
string expectedTotal = string.Empty;
for (int i = 1; i < maxIterations + 1; i++)
{
string id = $"agent{((i - 1) % NumAgents) + 1}";
texts[i] = $"{id}{Double(string.Concat(texts.Take(i)))}";
Assert.Equal(ChatRole.Assistant, result[i].Role);
Assert.Equal(id, result[i].AuthorName);
Assert.Equal(texts[i], result[i].Text);
expectedTotal += texts[i];
}
Assert.Equal(expectedTotal, updateText);
Assert.Equal(UserInput + expectedTotal, string.Concat(result));
static string Double(string s) => s + s;
}
workflow.Name.Should().Be("static-concurrent");
}
private sealed record WorkflowRunResult(string UpdateText, List<ChatMessage>? Result, CheckpointInfo? LastCheckpoint, List<RequestInfoEvent> PendingRequests);
private static async Task<WorkflowRunResult> RunWorkflowCheckpointedAsync(
Workflow workflow, List<ChatMessage> input, InProcessExecutionEnvironment environment, CheckpointInfo? fromCheckpoint = null)
[Fact]
public async Task Test_AgentWorkflowBuilder_BuildConcurrent_AggregatorIsHonoredAsync()
{
await using StreamingRun run =
fromCheckpoint != null ? await environment.ResumeStreamingAsync(workflow, fromCheckpoint)
: await environment.OpenStreamingAsync(workflow);
// Replace the default ("last message from each agent") with a custom aggregator,
// and confirm the workflow yields its result.
List<ChatMessage> sentinel = [new(ChatRole.Assistant, "custom-aggregator-result")];
await run.TrySendMessageAsync(input);
await run.TrySendMessageAsync(new TurnToken(emitEvents: true));
Workflow workflow = AgentWorkflowBuilder.BuildConcurrent(
[new OrchestrationTestHelpers.DoubleEchoAgent("agent1")],
aggregator: _ => sentinel);
return await ProcessWorkflowRunAsync(run);
(_, List<ChatMessage>? result, _, _) =
await OrchestrationTestHelpers.RunWorkflowAsync(workflow, [new ChatMessage(ChatRole.User, "abc")]);
result.Should().NotBeNull().And.ContainSingle();
result![0].Text.Should().Be("custom-aggregator-result");
}
private static async Task<WorkflowRunResult> ProcessWorkflowRunAsync(StreamingRun run)
[Fact]
public void Test_AgentWorkflowBuilder_CreateSequentialBuilderWith_RejectsNull()
{
StringBuilder sb = new();
WorkflowOutputEvent? output = null;
CheckpointInfo? lastCheckpoint = null;
List<RequestInfoEvent> pendingRequests = [];
await foreach (WorkflowEvent evt in run.WatchStreamAsync(blockOnPendingRequest: false).ConfigureAwait(false))
{
switch (evt)
{
case AgentResponseUpdateEvent responseUpdate:
sb.Append(responseUpdate.Data);
break;
case RequestInfoEvent requestInfo:
pendingRequests.Add(requestInfo);
break;
case WorkflowOutputEvent e:
output = e;
break;
case WorkflowErrorEvent errorEvent:
Assert.Fail($"Workflow execution failed with error: {errorEvent.Exception}");
break;
case SuperStepCompletedEvent stepCompleted:
lastCheckpoint = stepCompleted.CompletionInfo?.Checkpoint;
break;
}
}
return new(sb.ToString(), output?.As<List<ChatMessage>>(), lastCheckpoint, pendingRequests);
Assert.Throws<ArgumentNullException>("agents", () => AgentWorkflowBuilder.CreateSequentialBuilderWith(null!));
}
private static Task<WorkflowRunResult> RunWorkflowAsync(
Workflow workflow, List<ChatMessage> input, ExecutionEnvironment executionEnvironment = ExecutionEnvironment.InProcess_Lockstep)
=> RunWorkflowCheckpointedAsync(workflow, input, executionEnvironment.ToWorkflowExecutionEnvironment());
private sealed class DoubleEchoAgentWithBarrier(string name, StrongBox<TaskCompletionSource<bool>> barrier, StrongBox<int> remaining) : DoubleEchoAgent(name)
[Fact]
public void Test_AgentWorkflowBuilder_CreateSequentialBuilderWith_ReturnsConfigurableBuilder()
{
protected override async IAsyncEnumerable<AgentResponseUpdate> RunCoreStreamingAsync(
IEnumerable<ChatMessage> messages, AgentSession? session = null, AgentRunOptions? options = null, [EnumeratorCancellation] CancellationToken cancellationToken = default)
{
if (Interlocked.Decrement(ref remaining.Value) == 0)
{
barrier.Value!.SetResult(true);
}
OrchestrationTestHelpers.DoubleEchoAgent agent = new("agent1");
await barrier.Value!.Task.ConfigureAwait(false);
SequentialWorkflowBuilder builder = AgentWorkflowBuilder.CreateSequentialBuilderWith(agent);
Workflow workflow = builder.WithName("via-factory").Build();
await foreach (var update in base.RunCoreStreamingAsync(messages, session, options, cancellationToken))
{
await Task.Yield();
yield return update;
}
}
workflow.Name.Should().Be("via-factory");
}
[Fact]
public void Test_AgentWorkflowBuilder_CreateConcurrentBuilderWith_RejectsNull()
{
Assert.Throws<ArgumentNullException>("agents", () => AgentWorkflowBuilder.CreateConcurrentBuilderWith(null!));
}
[Fact]
public void Test_AgentWorkflowBuilder_CreateConcurrentBuilderWith_ReturnsConfigurableBuilder()
{
OrchestrationTestHelpers.DoubleEchoAgent agent = new("agent1");
ConcurrentWorkflowBuilder builder = AgentWorkflowBuilder.CreateConcurrentBuilderWith(agent);
Workflow workflow = builder.WithName("via-factory").Build();
workflow.Name.Should().Be("via-factory");
}
[Fact]
public void Test_AgentWorkflowBuilder_CreateHandoffBuilderWith_RejectsNull()
{
#pragma warning disable MAAIW001
Assert.Throws<ArgumentNullException>("initialAgent", () => AgentWorkflowBuilder.CreateHandoffBuilderWith(null!));
#pragma warning restore MAAIW001
}
[Fact]
public void Test_AgentWorkflowBuilder_CreateGroupChatBuilderWith_RejectsNull()
{
Assert.Throws<ArgumentNullException>("managerFactory", () => AgentWorkflowBuilder.CreateGroupChatBuilderWith(null!));
}
[Fact]
public void Test_AgentWorkflowBuilder_CreateMagenticBuilderWith_RejectsNull()
{
#pragma warning disable MAAIW001
Assert.Throws<ArgumentNullException>("managerAgent", () => AgentWorkflowBuilder.CreateMagenticBuilderWith(null!));
#pragma warning restore MAAIW001
}
}
@@ -0,0 +1,121 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Collections.Generic;
using System.Text.Json;
using FluentAssertions;
using Microsoft.Agents.AI.Workflows.Checkpointing;
namespace Microsoft.Agents.AI.Workflows.UnitTests.BackwardsCompatibility;
/// <summary>
/// Tests pinning the JSON shape of checkpoint-adjacent types so older payloads keep
/// deserializing correctly after the Outputs overhaul (see implementation-plan §5.7).
/// </summary>
public class JsonCheckpointSerializationTests
{
private static readonly JsonSerializerOptions s_options = WorkflowsJsonUtilities.DefaultOptions;
private static WorkflowInfo BuildInfoWithOutputExecutors(Dictionary<string, HashSet<OutputTag>> outputs)
=> new(
executors: new Dictionary<string, ExecutorInfo>(),
edges: new Dictionary<string, List<EdgeInfo>>(),
requestPorts: [],
startExecutorId: "start",
outputExecutorIds: outputs);
// ---------- WorkflowOutputEvent.Tags in-process round-trip (no JSON) ----------
[Fact]
public void Test_WorkflowOutputEvent_SingleTagCtorPopulatesTags()
{
WorkflowOutputEvent evt = new(data: "hello", executorId: "e1", tag: OutputTag.Intermediate);
evt.ExecutorId.Should().Be("e1");
evt.Tags.Should().BeEquivalentTo(new[] { OutputTag.Intermediate });
evt.HasTag(OutputTag.Intermediate).Should().BeTrue();
evt.IsIntermediate().Should().BeTrue();
}
[Fact]
public void Test_WorkflowOutputEvent_NoTagsCtorIsUntagged()
{
WorkflowOutputEvent evt = new(data: "hello", executorId: "e1");
evt.Tags.Should().BeEmpty();
evt.IsIntermediate().Should().BeFalse("an event with no tags is a terminal/regular output");
}
[Fact]
public void Test_WorkflowOutputEvent_MultiTagCtorPreservesAllTags()
{
OutputTag customTag = JsonSerializer.Deserialize<OutputTag>("\"custom\"", s_options);
WorkflowOutputEvent evt = new(data: "hello", executorId: "e1", tags: new[] { OutputTag.Intermediate, customTag });
evt.Tags.Should().HaveCount(2);
evt.HasTag(OutputTag.Intermediate).Should().BeTrue();
evt.HasTag(customTag).Should().BeTrue();
evt.IsIntermediate().Should().BeTrue();
}
// ---------- WorkflowInfo.OutputExecutorIds shape ----------
//
// Note: per the comment in WorkflowsJsonUtilities, WorkflowEvent / WorkflowOutputEvent
// is *not* currently a serialized checkpoint shape (events are not persisted into
// checkpoints today), so we do not pin a JSON round-trip for Tags on the event itself
// here. The tag JSON round-trip is exercised by OutputTagTests; the
// OutputExecutorIds map shape is the actually-load-bearing back-compat surface.
[Fact]
public void Test_JsonCheckpoint_WorkflowOutputExecutorsReadsLegacyArrayShape()
{
const string LegacyJson = """
{
"executors": {},
"edges": {},
"requestPorts": [],
"startExecutorId": "start",
"outputExecutorIds": ["a", "b"]
}
""";
WorkflowInfo? info = JsonSerializer.Deserialize<WorkflowInfo>(LegacyJson, s_options);
info.Should().NotBeNull();
info!.OutputExecutorIds.Should().HaveCount(2);
info.OutputExecutorIds["a"].Should().BeEmpty("legacy ids are untagged regular outputs");
info.OutputExecutorIds["b"].Should().BeEmpty();
}
[Fact]
public void Test_JsonCheckpoint_WorkflowOutputExecutorsWritesMapShape()
{
Dictionary<string, HashSet<OutputTag>> outputs = new()
{
["a"] = [],
["b"] = [OutputTag.Intermediate],
};
WorkflowInfo info = BuildInfoWithOutputExecutors(outputs);
string json = JsonSerializer.Serialize(info, s_options);
WorkflowInfo? back = JsonSerializer.Deserialize<WorkflowInfo>(json, s_options);
back.Should().NotBeNull();
back!.OutputExecutorIds.Should().HaveCount(2);
back.OutputExecutorIds["a"].Should().BeEmpty();
back.OutputExecutorIds["b"].Should().BeEquivalentTo(new[] { OutputTag.Intermediate });
// The map shape is detectable in the serialized JSON: the property value starts with `{`, not `[`.
int idx = json.IndexOf("\"outputExecutorIds\"", System.StringComparison.Ordinal);
idx.Should().BeGreaterThan(-1);
int colon = json.IndexOf(':', idx);
int firstNonSpace = colon + 1;
while (firstNonSpace < json.Length && char.IsWhiteSpace(json[firstNonSpace]))
{
firstNonSpace++;
}
json[firstNonSpace].Should().Be('{', "OutputExecutorIds is written in the new map shape");
}
}
@@ -0,0 +1,164 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.CompilerServices;
using System.Text.RegularExpressions;
using System.Threading.Tasks;
using FluentAssertions;
using Microsoft.Agents.AI.Workflows.UnitTests.Futures;
using Microsoft.Extensions.AI;
#pragma warning disable SYSLIB1045 // Use GeneratedRegex
#pragma warning disable RCS1186 // Use Regex instance instead of static method
namespace Microsoft.Agents.AI.Workflows.UnitTests;
public class ConcurrentWorkflowBuilderTests
{
[Fact]
public void Test_ConcurrentWorkflowBuilder_InvalidArguments_Throws()
{
Assert.Throws<ArgumentNullException>("agents", () => new ConcurrentWorkflowBuilder(null!));
Assert.Throws<ArgumentException>("agents", () => new ConcurrentWorkflowBuilder().Build());
Assert.Throws<ArgumentNullException>("agents", () => AgentWorkflowBuilder.BuildConcurrent(null!));
Assert.Throws<ArgumentNullException>("agents", () => AgentWorkflowBuilder.CreateConcurrentBuilderWith(null!));
}
[Fact]
public async Task Test_ConcurrentWorkflowBuilder_AgentsRunInParallelAsync()
{
StrongBox<TaskCompletionSource<bool>> barrier = new();
StrongBox<int> remaining = new();
var workflow = new ConcurrentWorkflowBuilder(
new OrchestrationTestHelpers.DoubleEchoAgentWithBarrier("agent1", barrier, remaining),
new OrchestrationTestHelpers.DoubleEchoAgentWithBarrier("agent2", barrier, remaining))
.Build();
for (int iter = 0; iter < 3; iter++)
{
barrier.Value = new TaskCompletionSource<bool>(TaskCreationOptions.RunContinuationsAsynchronously);
remaining.Value = 2;
(string updateText, List<ChatMessage>? result, _, _) =
await OrchestrationTestHelpers.RunWorkflowAsync(workflow, [new ChatMessage(ChatRole.User, "abc")]);
Assert.NotEmpty(updateText);
Assert.NotNull(result);
// TODO: https://github.com/microsoft/agent-framework/issues/784
// These asserts are flaky until we guarantee message delivery order.
Assert.Single(Regex.Matches(updateText, "agent1"));
Assert.Single(Regex.Matches(updateText, "agent2"));
Assert.Equal(4, Regex.Matches(updateText, "abc").Count);
Assert.Equal(2, result.Count);
}
}
[Fact]
public void Test_ConcurrentWorkflowBuilder_DefaultDesignationsMatchSpec()
{
Workflow workflow = new ConcurrentWorkflowBuilder(
new OrchestrationTestHelpers.DoubleEchoAgent("agent1"),
new OrchestrationTestHelpers.DoubleEchoAgent("agent2"),
new OrchestrationTestHelpers.DoubleEchoAgent("agent3"))
.Build();
Dictionary<string, HashSet<OutputTag>> designations = workflow.OutputExecutors;
designations.Where(kvp => kvp.Value.Count == 0)
.Should().ContainSingle("ConcurrentEndExecutor is the sole terminal output by default");
designations.Where(kvp => kvp.Value.Contains(OutputTag.Intermediate))
.Should().HaveCount(6, "every agent (3) and per-agent accumulator (3) is designated intermediate by default");
}
[Fact]
public void Test_ConcurrentWorkflowBuilder_ExplicitDesignationsReplaceDefaults()
{
OrchestrationTestHelpers.DoubleEchoAgent a1 = new("agent1");
OrchestrationTestHelpers.DoubleEchoAgent a2 = new("agent2");
OrchestrationTestHelpers.DoubleEchoAgent a3 = new("agent3");
Workflow workflow = new ConcurrentWorkflowBuilder(a1, a2, a3)
.WithOutputFrom(a1)
.WithIntermediateOutputFrom([a2])
.Build();
Dictionary<string, HashSet<OutputTag>> designations = workflow.OutputExecutors;
designations.Should().HaveCount(2,
"only the two explicitly-designated agents land on the inner builder; the end + accumulator defaults are suppressed");
designations.Values.Where(tags => tags.Count == 0)
.Should().ContainSingle("agent1 is the only terminal designation");
designations.Values.Where(tags => tags.Contains(OutputTag.Intermediate))
.Should().ContainSingle("agent2 is the only intermediate designation");
}
[Fact]
public void Test_ConcurrentWorkflowBuilder_DesignationForNonParticipantThrows()
{
OrchestrationTestHelpers.DoubleEchoAgent participant = new("p1");
OrchestrationTestHelpers.DoubleEchoAgent stranger = new("stranger");
ConcurrentWorkflowBuilder builder = new ConcurrentWorkflowBuilder(participant)
.WithIntermediateOutputFrom([stranger]);
Action build = () => builder.Build();
build.Should().Throw<InvalidOperationException>().WithMessage("*stranger*");
}
[Fact]
public void Test_ConcurrentWorkflowBuilder_WithNamePropagatesToWorkflow()
{
Workflow workflow = new ConcurrentWorkflowBuilder(new OrchestrationTestHelpers.DoubleEchoAgent("agent1"))
.WithName("named-concurrent")
.Build();
workflow.Name.Should().Be("named-concurrent");
}
[Fact]
public void Test_ConcurrentWorkflowBuilder_WithDescriptionPropagatesToWorkflow()
{
Workflow workflow = new ConcurrentWorkflowBuilder(new OrchestrationTestHelpers.DoubleEchoAgent("agent1"))
.WithDescription("describes the concurrent fan-out/fan-in")
.Build();
workflow.Description.Should().Be("describes the concurrent fan-out/fan-in");
}
[Collection(FuturesSerialCollection.Name)]
public class AsAgentForwarding
{
[Fact]
public async Task Test_ConcurrentWorkflowBuilder_AsAgent_OnlyTerminalDesignationSurfacesAsync()
{
using FuturesScope _ = new(enabled: true);
OrchestrationTestHelpers.DoubleEchoAgent agent1 = new("agent1");
OrchestrationTestHelpers.DoubleEchoAgent agent2 = new("agent2");
// Designate only agent1 as a terminal output source — agent2 and the fan-in
// aggregator default-intermediate designations are suppressed.
Workflow workflow = new ConcurrentWorkflowBuilder(agent1, agent2)
.WithOutputFrom(agent1)
.Build();
List<AgentResponseUpdate> updates = await workflow
.AsAIAgent("WorkflowAgent")
.RunStreamingAsync(new ChatMessage(ChatRole.User, "abc"))
.ToListAsync();
HashSet<string> authoredBy = updates
.Select(u => u.AuthorName)
.Where(n => !string.IsNullOrEmpty(n))
.Select(n => n!)
.ToHashSet();
authoredBy.Should().Contain("agent1", "the designated agent must surface");
authoredBy.Should().NotContain("agent2",
"the undesignated agent must not surface when only one is designated under Futures-on");
}
}
}
@@ -0,0 +1,289 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using FluentAssertions;
using Microsoft.Extensions.AI;
namespace Microsoft.Agents.AI.Workflows.UnitTests.Futures;
/// <summary>
/// Runner-level coverage for <see cref="Workflows.Futures.EnableAgentResponseOutputTaggingAndFiltering"/>.
/// Exercises every combination of (flag on/off) × (designation kind) × (payload shape) to pin the
/// runner's behavior in both the legacy bypass path and the unified filter-and-tag path.
/// </summary>
public static partial class FuturesTests
{
[Collection(FuturesSerialCollection.Name)]
public class AgentResponseOutputFilteringAndTaggingTests
{
private const string SourceId = "yielder";
private static AgentResponse SampleResponse(string text = "hi")
=> new(new ChatMessage(ChatRole.Assistant, text));
private static AgentResponseUpdate SampleUpdate(string text = "tick")
=> new(ChatRole.Assistant, text);
private static async Task<List<WorkflowEvent>> RunAsync<T>(Workflow workflow, T input) where T : notnull
{
List<WorkflowEvent> events = [];
await using StreamingRun run = await InProcessExecution.RunStreamingAsync(workflow, input).ConfigureAwait(false);
await foreach (WorkflowEvent evt in run.WatchStreamAsync().ConfigureAwait(false))
{
events.Add(evt);
}
return events;
}
private static Workflow BuildAgentResponseWorkflow(Action<WorkflowBuilder, YieldAgentResponseExecutor>? designate = null)
{
YieldAgentResponseExecutor exec = new(SourceId);
WorkflowBuilder builder = new(exec);
designate?.Invoke(builder, exec);
return builder.Build();
}
private static Workflow BuildAgentResponseUpdateWorkflow(Action<WorkflowBuilder, YieldAgentResponseUpdateExecutor>? designate = null)
{
YieldAgentResponseUpdateExecutor exec = new(SourceId);
WorkflowBuilder builder = new(exec);
designate?.Invoke(builder, exec);
return builder.Build();
}
private static Workflow BuildPocoWorkflow(Action<WorkflowBuilder, YieldPocoExecutor>? designate = null)
{
YieldPocoExecutor exec = new(SourceId);
WorkflowBuilder builder = new(exec);
designate?.Invoke(builder, exec);
return builder.Build();
}
// F1
[Fact]
public async Task Test_Runner_LegacyAgentResponseBypass_RaisesUntaggedEventAsync()
{
using FuturesScope _ = new(enabled: false);
Workflow workflow = BuildAgentResponseWorkflow(designate: null);
List<WorkflowEvent> events = await RunAsync(workflow, "go");
AgentResponseEvent emitted = events.OfType<AgentResponseEvent>().Should().ContainSingle().Subject;
emitted.ExecutorId.Should().Be(SourceId);
emitted.Tags.Should().BeEmpty("legacy bypass attaches no tags");
emitted.IsIntermediate().Should().BeFalse();
}
// F2
[Fact]
public async Task Test_Runner_LegacyAgentResponseUpdateBypass_RaisesUntaggedEventAsync()
{
using FuturesScope _ = new(enabled: false);
Workflow workflow = BuildAgentResponseUpdateWorkflow(designate: null);
List<WorkflowEvent> events = await RunAsync(workflow, "go");
AgentResponseUpdateEvent emitted = events.OfType<AgentResponseUpdateEvent>().Should().ContainSingle().Subject;
emitted.Tags.Should().BeEmpty();
}
// F3
[Fact]
public async Task Test_Runner_LegacyBypassIgnoresDesignationAsync()
{
using FuturesScope _ = new(enabled: false);
Workflow workflow = BuildAgentResponseWorkflow(static (b, e) => b.WithIntermediateOutputFrom([e]));
List<WorkflowEvent> events = await RunAsync(workflow, "go");
AgentResponseEvent emitted = events.OfType<AgentResponseEvent>().Should().ContainSingle().Subject;
emitted.Tags.Should().BeEmpty("legacy bypass ignores the designation entirely");
emitted.IsIntermediate().Should().BeFalse("legacy bypass does not propagate tags");
}
// F4
[Fact]
public async Task Test_Runner_LegacyPocoIsFilteredAsync()
{
using FuturesScope _ = new(enabled: false);
Workflow workflow = BuildPocoWorkflow(designate: null);
List<WorkflowEvent> events = await RunAsync(workflow, "go");
events.OfType<WorkflowOutputEvent>().Should().BeEmpty("POCO outputs always go through the filter; undesignated source is dropped");
}
// F5
[Fact]
public async Task Test_Runner_UndesignatedAgentResponseIsFilteredWhenFuturesOnAsync()
{
using FuturesScope _ = new(enabled: true);
Workflow workflow = BuildAgentResponseWorkflow(designate: null);
List<WorkflowEvent> events = await RunAsync(workflow, "go");
events.OfType<WorkflowOutputEvent>().Should().BeEmpty(
"with the future on, AgentResponse must be designated to surface");
}
// F6
[Fact]
public async Task Test_Runner_DesignatedTerminalAgentResponseHasEmptyTagsAsync()
{
using FuturesScope _ = new(enabled: true);
Workflow workflow = BuildAgentResponseWorkflow(static (b, e) => b.WithOutputFrom(e));
List<WorkflowEvent> events = await RunAsync(workflow, "go");
AgentResponseEvent emitted = events.OfType<AgentResponseEvent>().Should().ContainSingle().Subject;
emitted.Tags.Should().BeEmpty("terminal designation carries no tag");
emitted.IsIntermediate().Should().BeFalse();
}
// F7
[Fact]
public async Task Test_Runner_DesignatedIntermediateAgentResponseHasIntermediateTagAsync()
{
using FuturesScope _ = new(enabled: true);
Workflow workflow = BuildAgentResponseWorkflow(static (b, e) => b.WithIntermediateOutputFrom([e]));
List<WorkflowEvent> events = await RunAsync(workflow, "go");
AgentResponseEvent emitted = events.OfType<AgentResponseEvent>().Should().ContainSingle().Subject;
emitted.Tags.Should().BeEquivalentTo(new[] { OutputTag.Intermediate });
emitted.IsIntermediate().Should().BeTrue();
}
// F8
[Fact]
public async Task Test_Runner_DesignatedIntermediateAgentResponseUpdateHasIntermediateTagAsync()
{
using FuturesScope _ = new(enabled: true);
Workflow workflow = BuildAgentResponseUpdateWorkflow(static (b, e) => b.WithIntermediateOutputFrom([e]));
List<WorkflowEvent> events = await RunAsync(workflow, "go");
AgentResponseUpdateEvent emitted = events.OfType<AgentResponseUpdateEvent>().Should().ContainSingle().Subject;
emitted.Tags.Should().BeEquivalentTo(new[] { OutputTag.Intermediate });
emitted.IsIntermediate().Should().BeTrue();
}
// F9
[Fact]
public async Task Test_Runner_TagsAccumulateOutputThenIntermediateAsync()
{
using FuturesScope _ = new(enabled: true);
Workflow workflow = BuildAgentResponseWorkflow(static (b, e) =>
{
b.WithOutputFrom(e);
b.WithIntermediateOutputFrom([e]);
});
List<WorkflowEvent> events = await RunAsync(workflow, "go");
AgentResponseEvent emitted = events.OfType<AgentResponseEvent>().Should().ContainSingle().Subject;
emitted.Tags.Should().BeEquivalentTo(new[] { OutputTag.Intermediate },
"terminal+intermediate union is {{ Intermediate }} (terminal contributes the entry but no tag)");
emitted.IsIntermediate().Should().BeTrue();
}
// F10
[Fact]
public async Task Test_Runner_TagsAccumulateIntermediateThenOutputAsync()
{
using FuturesScope _ = new(enabled: true);
Workflow workflow = BuildAgentResponseWorkflow(static (b, e) =>
{
b.WithIntermediateOutputFrom([e]);
b.WithOutputFrom(e);
});
List<WorkflowEvent> events = await RunAsync(workflow, "go");
AgentResponseEvent emitted = events.OfType<AgentResponseEvent>().Should().ContainSingle().Subject;
emitted.Tags.Should().BeEquivalentTo(new[] { OutputTag.Intermediate }, "designation order is irrelevant");
emitted.IsIntermediate().Should().BeTrue();
}
// F11
[Fact]
public async Task Test_Runner_DesignatedIntermediatePocoHasIntermediateTagAsync()
{
using FuturesScope _ = new(enabled: true);
Workflow workflow = BuildPocoWorkflow(static (b, e) => b.WithIntermediateOutputFrom([e]));
List<WorkflowEvent> events = await RunAsync(workflow, "go");
WorkflowOutputEvent emitted = events.OfType<WorkflowOutputEvent>().Should().ContainSingle().Subject;
emitted.Should().NotBeOfType<AgentResponseEvent>();
emitted.Tags.Should().BeEquivalentTo(new[] { OutputTag.Intermediate });
emitted.IsIntermediate().Should().BeTrue();
}
// F12
[Fact]
public async Task Test_Runner_DesignatedTerminalPocoHasEmptyTagsAsync()
{
using FuturesScope _ = new(enabled: true);
Workflow workflow = BuildPocoWorkflow(static (b, e) => b.WithOutputFrom(e));
List<WorkflowEvent> events = await RunAsync(workflow, "go");
WorkflowOutputEvent emitted = events.OfType<WorkflowOutputEvent>().Should().ContainSingle().Subject;
emitted.Tags.Should().BeEmpty();
emitted.IsIntermediate().Should().BeFalse();
}
// F13
[Fact]
public async Task Test_Runner_RepeatedTerminalDesignationDedupesAsync()
{
using FuturesScope _ = new(enabled: true);
Workflow workflow = BuildAgentResponseWorkflow(static (b, e) =>
{
b.WithOutputFrom(e);
b.WithOutputFrom(e);
});
List<WorkflowEvent> events = await RunAsync(workflow, "go");
AgentResponseEvent emitted = events.OfType<AgentResponseEvent>().Should().ContainSingle().Subject;
emitted.Tags.Should().BeEmpty("repeated terminal designation contributes no tag");
}
// ---- Executors -----------------------------------------------------------
internal sealed class YieldAgentResponseExecutor(string id) : Executor(id)
{
protected override ProtocolBuilder ConfigureProtocol(ProtocolBuilder protocolBuilder)
=> protocolBuilder.ConfigureRoutes(rb => rb.AddHandler<string, AgentResponse>(this.HandleAsync));
private ValueTask<AgentResponse> HandleAsync(string input, IWorkflowContext context, CancellationToken cancellationToken)
=> new(SampleResponse(input));
}
internal sealed class YieldAgentResponseUpdateExecutor(string id) : Executor(id)
{
protected override ProtocolBuilder ConfigureProtocol(ProtocolBuilder protocolBuilder)
=> protocolBuilder.ConfigureRoutes(rb => rb.AddHandler<string, AgentResponseUpdate>(this.HandleAsync));
private ValueTask<AgentResponseUpdate> HandleAsync(string input, IWorkflowContext context, CancellationToken cancellationToken)
=> new(SampleUpdate(input));
}
public sealed record Poco(string Value);
internal sealed class YieldPocoExecutor(string id) : Executor(id)
{
protected override ProtocolBuilder ConfigureProtocol(ProtocolBuilder protocolBuilder)
=> protocolBuilder.ConfigureRoutes(rb => rb.AddHandler<string, Poco>(this.HandleAsync));
private ValueTask<Poco> HandleAsync(string input, IWorkflowContext context, CancellationToken cancellationToken)
=> new(new Poco(input));
}
}
}
@@ -0,0 +1,27 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
namespace Microsoft.Agents.AI.Workflows.UnitTests.Futures;
/// <summary>
/// Sets <see cref="Workflows.Futures.EnableAgentResponseOutputTaggingAndFiltering"/> for
/// the lifetime of the scope, restoring the prior value on dispose. Pair every use with
/// <c>using</c> and run inside the <c>FuturesSerial</c> xUnit collection to avoid leaking
/// state across parallel tests.
/// </summary>
internal sealed class FuturesScope : IDisposable
{
private readonly bool _previous;
public FuturesScope(bool enabled)
{
this._previous = Workflows.Futures.EnableAgentResponseOutputTaggingAndFiltering;
Workflows.Futures.EnableAgentResponseOutputTaggingAndFiltering = enabled;
}
public void Dispose()
{
Workflows.Futures.EnableAgentResponseOutputTaggingAndFiltering = this._previous;
}
}
@@ -0,0 +1,19 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Diagnostics.CodeAnalysis;
namespace Microsoft.Agents.AI.Workflows.UnitTests.Futures;
/// <summary>
/// xUnit collection marker for tests that mutate the process-global
/// <see cref="Workflows.Futures"/> switches. Membership in this collection serializes
/// the tests against each other so that <see cref="FuturesScope"/> cannot leak state
/// into a concurrently running test.
/// </summary>
[CollectionDefinition(Name, DisableParallelization = true)]
[SuppressMessage("Naming", "CA1711:Identifiers should not have incorrect suffix",
Justification = "xUnit's [CollectionDefinition] pattern names the marker type after the collection's purpose; the 'Collection' suffix is idiomatic.")]
public sealed class FuturesSerialCollection
{
public const string Name = "FuturesSerial";
}
@@ -0,0 +1,479 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Linq;
using System.Runtime.CompilerServices;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using FluentAssertions;
using Microsoft.Agents.AI.Workflows.InProc;
using Microsoft.Extensions.AI;
namespace Microsoft.Agents.AI.Workflows.UnitTests;
/// <summary>
/// Orchestration-level tests for <see cref="AgentWorkflowBuilder.CreateGroupChatBuilderWith"/> covering
/// <see cref="FunctionCallContent"/> and <see cref="ToolApprovalRequestContent"/> behavior across
/// real <see cref="ChatClientAgent"/> participants. These tests parallel the equivalents in
/// <see cref="HandoffOrchestrationTests"/> to ensure that the broadcast-based group chat host
/// (each participant maintains its own per-agent session via <see cref="Specialized.AIAgentHostExecutor"/>;
/// only the speaker receives a <see cref="TurnToken"/>; messages are broadcast to every other
/// participant) preserves the same HITL semantics as the handoff path.
/// </summary>
public class GroupChatOrchestrationTests
{
/// <summary>
/// End-to-end tool-approval checkpoint/resume scenario through a <see cref="RoundRobinGroupChatManager"/>
/// with a single participant. Mirrors the maximal repro added in PR #5952 (Track A2 in
/// <c>docs/working/issue-5350-root-cause-validation-plan.md</c>): a <see cref="ChatClientAgent"/>
/// over a mock chat client emits a <see cref="FunctionCallContent"/> for an
/// <see cref="ApprovalRequiredAIFunction"/>, the runtime surfaces a
/// <see cref="ToolApprovalRequestContent"/> as an external <see cref="RequestInfoEvent"/>, the test
/// checkpoints while the request is pending, resumes from a fresh handle, asserts that the
/// resumed <c>TARC.ToolCall</c> is still a <see cref="FunctionCallContent"/>, sends an
/// approval response, and verifies that the wrapped <see cref="AIFunction"/> is invoked
/// exactly once and the workflow completes without errors.
/// </summary>
[Fact]
public async Task GroupChat_ToolApproval_JsonCheckpointResume_PreservesFunctionCallContentAndInvokesToolAsync()
{
ApprovalHarness harness = new();
Workflow workflow = AgentWorkflowBuilder
.CreateGroupChatBuilderWith(agents => new RoundRobinGroupChatManager(agents) { MaximumIterationCount = 4 })
.AddParticipants(harness.Agent)
.Build();
await RunCheckpointedApprovalRoundTripAsync(
workflow,
harness,
CheckpointManager.CreateJson(new InMemoryJsonStore()),
scenarioName: "GroupChat (round-robin, single participant)");
}
/// <summary>
/// Round-robin group chat with two participants. The first participant exposes an
/// <see cref="ApprovalRequiredAIFunction"/> and emits a <see cref="FunctionCallContent"/> for it on
/// its first turn. The test denies the approval and asserts that the conversation continues:
/// the first agent runs once more (the FICC denial branch produces a final assistant message),
/// then the host broadcasts that message and selects the second agent, which produces its own
/// reply. This mirrors <c>Handoffs_TwoTransfers_SecondAgentUserApproval_ResponseServedByThirdAgentAsync</c>
/// but on the group-chat path.
/// </summary>
[Fact]
public async Task GroupChat_ToolApproval_DeniedResponse_ConversationContinuesAsync()
{
int approvalToolCallCount = 0;
const string ApprovalCallId = "approve_call_1";
const string ApprovalToolName = "DoSomethingPrivileged";
AIFunction approvalTool = new ApprovalRequiredAIFunction(AIFunctionFactory.Create(
() =>
{
Interlocked.Increment(ref approvalToolCallCount);
return "tool result";
},
name: ApprovalToolName,
description: "Performs a privileged action"));
int agent1CallCount = 0;
var agent1 = new ChatClientAgent(
new MockChatClient((messages, options) =>
{
int call = Interlocked.Increment(ref agent1CallCount);
return call switch
{
1 => new ChatResponse(new ChatMessage(ChatRole.Assistant,
[new FunctionCallContent(ApprovalCallId, ApprovalToolName)])),
_ => new ChatResponse(new ChatMessage(ChatRole.Assistant, "agent1 final response")),
};
}),
instructions: "You are agent1.",
name: "agent1",
tools: [approvalTool]);
int agent2CallCount = 0;
var agent2 = new ChatClientAgent(
new MockChatClient((messages, options) =>
{
Interlocked.Increment(ref agent2CallCount);
return new ChatResponse(new ChatMessage(ChatRole.Assistant, "agent2 reply"));
}),
instructions: "You are agent2.",
name: "agent2");
Workflow workflow = AgentWorkflowBuilder
.CreateGroupChatBuilderWith(agents => new RoundRobinGroupChatManager(agents) { MaximumIterationCount = 2 })
.AddParticipants(agent1, agent2)
.Build();
CheckpointManager checkpointManager = CheckpointManager.CreateInMemory();
InProcessExecutionEnvironment env = InProcessExecution.OffThread.WithCheckpointing(checkpointManager);
ExternalRequest? pendingRequest = null;
CheckpointInfo? lastCheckpoint = null;
List<WorkflowEvent> firstRunEvents = [];
await using (StreamingRun firstRun = await env.RunStreamingAsync(workflow, new List<ChatMessage> { new(ChatRole.User, "hello") }))
{
(await firstRun.TrySendMessageAsync(new TurnToken(emitEvents: false)))
.Should().BeTrue();
using CancellationTokenSource cts = new(TimeSpan.FromSeconds(30));
await foreach (WorkflowEvent evt in firstRun.WatchStreamAsync(blockOnPendingRequest: false, cts.Token))
{
firstRunEvents.Add(evt);
if (evt is RequestInfoEvent requestInfo)
{
pendingRequest ??= requestInfo.Request;
}
if (evt is SuperStepCompletedEvent step && step.CompletionInfo?.Checkpoint is { } cp)
{
lastCheckpoint = cp;
}
}
}
pendingRequest.Should().NotBeNull("agent1 should have surfaced an approval request for the privileged tool");
firstRunEvents.OfType<WorkflowErrorEvent>().Should().BeEmpty();
firstRunEvents.OfType<ExecutorFailedEvent>().Should().BeEmpty();
approvalToolCallCount.Should().Be(0, "the tool must not be invoked before approval is granted");
ToolApprovalRequestContent approvalRequest =
pendingRequest!.Data.As<ToolApprovalRequestContent>().Should().NotBeNull()
.And.Subject.As<ToolApprovalRequestContent>();
approvalRequest.ToolCall.Should().BeOfType<FunctionCallContent>();
((FunctionCallContent)approvalRequest.ToolCall).Name.Should().Be(ApprovalToolName);
// Deny the request and continue the conversation.
ExternalResponse denial = pendingRequest.CreateResponse(approvalRequest.CreateResponse(approved: false, reason: "Denied"));
List<WorkflowEvent> secondRunEvents = [];
List<ChatMessage>? finalOutput = null;
await using (StreamingRun resumed = await env.ResumeStreamingAsync(workflow, lastCheckpoint!))
{
await resumed.SendResponseAsync(denial);
using CancellationTokenSource cts = new(TimeSpan.FromSeconds(30));
await foreach (WorkflowEvent evt in resumed.WatchStreamAsync(blockOnPendingRequest: false, cts.Token))
{
secondRunEvents.Add(evt);
if (evt is WorkflowOutputEvent outputEvt)
{
finalOutput = outputEvt.As<List<ChatMessage>>();
}
}
}
secondRunEvents.OfType<WorkflowErrorEvent>().Should().BeEmpty(
"denying the approval should not surface any workflow errors");
secondRunEvents.OfType<ExecutorFailedEvent>().Should().BeEmpty(
"denying the approval should not raise executor failures (regression guard for the GroupChat duplicate-key bug pinned in PR #5952's A2 test before the broadcast refactor)");
approvalToolCallCount.Should().Be(0, "the tool must not be invoked after denial");
agent1CallCount.Should().BeGreaterThanOrEqualTo(2, "agent1 should be re-invoked by FICC after the denial to produce a final assistant message");
agent2CallCount.Should().Be(1, "agent2 should be the next round-robin speaker and produce its own reply");
finalOutput.Should().NotBeNull();
finalOutput!.Should().Contain(m => m.AuthorName == "agent1");
finalOutput.Should().Contain(m => m.AuthorName == "agent2" && m.Text == "agent2 reply");
}
/// <summary>
/// Round-robin group chat with two participants. The first participant declares a
/// non-invokable function via <c>AIFunctionFactory.CreateDeclaration</c>,
/// causing the function call to be surfaced as an external <see cref="FunctionCallContent"/>
/// (<see cref="RequestInfoEvent"/>). The test responds with a <see cref="FunctionResultContent"/>
/// and asserts that the conversation continues: the first agent's second invocation produces a
/// final assistant message, then the group chat advances to the second agent which produces
/// its own reply. This mirrors <c>Handoffs_TwoTransfers_SecondAgentToolCall_ResponseServedByThirdAgentAsync</c>
/// but on the group-chat path.
/// </summary>
[Fact]
public async Task GroupChat_FunctionCall_ExternallyResolved_ConversationContinuesAsync()
{
const string FunctionCallId = "fcc_call_1";
const string FunctionName = "FetchExternalData";
JsonElement schema = AIFunctionFactory.Create(() => true).JsonSchema;
AIFunctionDeclaration declaration = AIFunctionFactory.CreateDeclaration(FunctionName, "Fetches external data", schema);
int agent1CallCount = 0;
var agent1 = new ChatClientAgent(
new MockChatClient((messages, options) =>
{
int call = Interlocked.Increment(ref agent1CallCount);
return call switch
{
1 => new ChatResponse(new ChatMessage(ChatRole.Assistant,
[new FunctionCallContent(FunctionCallId, FunctionName)])),
_ => new ChatResponse(new ChatMessage(ChatRole.Assistant, "agent1 final response")),
};
}),
instructions: "You are agent1.",
name: "agent1",
tools: [declaration]);
int agent2CallCount = 0;
var agent2 = new ChatClientAgent(
new MockChatClient((messages, options) =>
{
Interlocked.Increment(ref agent2CallCount);
return new ChatResponse(new ChatMessage(ChatRole.Assistant, "agent2 reply"));
}),
instructions: "You are agent2.",
name: "agent2");
Workflow workflow = AgentWorkflowBuilder
.CreateGroupChatBuilderWith(agents => new RoundRobinGroupChatManager(agents) { MaximumIterationCount = 2 })
.AddParticipants(agent1, agent2)
.Build();
CheckpointManager checkpointManager = CheckpointManager.CreateInMemory();
InProcessExecutionEnvironment env = InProcessExecution.OffThread.WithCheckpointing(checkpointManager);
ExternalRequest? pendingRequest = null;
CheckpointInfo? lastCheckpoint = null;
await using (StreamingRun firstRun = await env.RunStreamingAsync(workflow, new List<ChatMessage> { new(ChatRole.User, "hello") }))
{
(await firstRun.TrySendMessageAsync(new TurnToken(emitEvents: false)))
.Should().BeTrue();
using CancellationTokenSource cts = new(TimeSpan.FromSeconds(30));
await foreach (WorkflowEvent evt in firstRun.WatchStreamAsync(blockOnPendingRequest: false, cts.Token))
{
if (evt is RequestInfoEvent requestInfo)
{
pendingRequest ??= requestInfo.Request;
}
if (evt is SuperStepCompletedEvent step && step.CompletionInfo?.Checkpoint is { } cp)
{
lastCheckpoint = cp;
}
}
}
pendingRequest.Should().NotBeNull("agent1 should have surfaced a FunctionCallContent for the declaration-only tool");
FunctionCallContent functionCall =
pendingRequest!.Data.As<FunctionCallContent>().Should().NotBeNull()
.And.Subject.As<FunctionCallContent>();
functionCall.Name.Should().Be(FunctionName);
functionCall.CallId.Should().EndWith(FunctionCallId,
"the workflow rewrites the CallId with an executor-scoped prefix, but should preserve the original tail");
// Respond with a function result and let the conversation continue.
ExternalResponse response = pendingRequest.CreateResponse(new FunctionResultContent(functionCall.CallId, "external-data-payload"));
List<WorkflowEvent> resumeEvents = [];
List<ChatMessage>? finalOutput = null;
await using (StreamingRun resumed = await env.ResumeStreamingAsync(workflow, lastCheckpoint!))
{
await resumed.SendResponseAsync(response);
using CancellationTokenSource cts = new(TimeSpan.FromSeconds(30));
await foreach (WorkflowEvent evt in resumed.WatchStreamAsync(blockOnPendingRequest: false, cts.Token))
{
resumeEvents.Add(evt);
if (evt is WorkflowOutputEvent outputEvt)
{
finalOutput = outputEvt.As<List<ChatMessage>>();
}
}
}
resumeEvents.OfType<WorkflowErrorEvent>().Should().BeEmpty();
resumeEvents.OfType<ExecutorFailedEvent>().Should().BeEmpty();
agent1CallCount.Should().BeGreaterThanOrEqualTo(2, "agent1 should be re-invoked once the externally-resolved function result is delivered");
agent2CallCount.Should().Be(1, "agent2 should be the next round-robin speaker after agent1 finishes");
finalOutput.Should().NotBeNull();
finalOutput!.Should().Contain(m => m.AuthorName == "agent1");
finalOutput.Should().Contain(m => m.AuthorName == "agent2" && m.Text == "agent2 reply");
}
/// <summary>
/// Shared end-to-end driver for the approval checkpoint/resume scenario; modelled on the
/// <c>RunReproAsync</c> helper from PR #5952. Runs the workflow until an approval request is
/// pending, captures the latest checkpoint, disposes the run, resumes from a fresh handle,
/// asserts the resumed payload still carries a <see cref="FunctionCallContent"/>, sends an
/// approval response, and asserts the wrapped tool is invoked exactly once and the workflow
/// finishes without errors.
/// </summary>
private static async Task RunCheckpointedApprovalRoundTripAsync(
Workflow workflow,
ApprovalHarness harness,
CheckpointManager checkpointManager,
string scenarioName)
{
InProcessExecutionEnvironment env = InProcessExecution.OffThread;
List<ChatMessage> inputMessages = [new(ChatRole.User, "What's the weather in Amsterdam?")];
ExternalRequest? firstRunRequest = null;
CheckpointInfo? checkpoint = null;
await using (StreamingRun firstRun = await env.WithCheckpointing(checkpointManager)
.RunStreamingAsync(workflow, inputMessages))
{
(await firstRun.TrySendMessageAsync(new TurnToken(emitEvents: false)))
.Should().BeTrue($"[{scenarioName}] the workflow should accept a TurnToken");
using CancellationTokenSource cts = new(TimeSpan.FromSeconds(30));
await foreach (WorkflowEvent evt in firstRun.WatchStreamAsync(blockOnPendingRequest: false, cts.Token))
{
if (evt is RequestInfoEvent requestInfo)
{
firstRunRequest ??= requestInfo.Request;
}
if (evt is SuperStepCompletedEvent step && step.CompletionInfo?.Checkpoint is { } cp)
{
checkpoint = cp;
}
}
}
firstRunRequest.Should().NotBeNull(
$"[{scenarioName}] the ChatClientAgent + FICC pipeline should surface the approval request as a workflow RequestInfoEvent");
checkpoint.Should().NotBeNull(
$"[{scenarioName}] a checkpoint should have been produced while the approval request was pending");
harness.ChatCallCount.Should().Be(1, $"[{scenarioName}] the mock chat client should have been called exactly once before approval was requested");
harness.InvocationCount.Should().Be(0, $"[{scenarioName}] the underlying tool must NOT have been invoked before approval was granted");
ToolApprovalRequestContent? preCheckpoint = firstRunRequest!.Data.As<ToolApprovalRequestContent>();
preCheckpoint.Should().NotBeNull($"[{scenarioName}] the pending external request should carry a ToolApprovalRequestContent payload");
preCheckpoint!.ToolCall.Should().BeOfType<FunctionCallContent>(
$"[{scenarioName}] the pre-checkpoint pending request payload must already be a FunctionCallContent");
// Resume on a fresh handle and capture the re-emitted approval request.
ExternalRequest? resumedRequest = null;
List<WorkflowEvent> postResumeEvents = [];
await using (StreamingRun resumed = await env.WithCheckpointing(checkpointManager)
.ResumeStreamingAsync(workflow, checkpoint!))
{
using CancellationTokenSource cts = new(TimeSpan.FromSeconds(30));
await foreach (WorkflowEvent evt in resumed.WatchStreamAsync(blockOnPendingRequest: false, cts.Token))
{
if (evt is RequestInfoEvent requestInfo)
{
resumedRequest ??= requestInfo.Request;
}
}
resumedRequest.Should().NotBeNull($"[{scenarioName}] the resumed workflow should re-emit the pending approval RequestInfoEvent");
ToolApprovalRequestContent? postResume = resumedRequest!.Data.As<ToolApprovalRequestContent>();
postResume.Should().NotBeNull(
$"[{scenarioName}] ExternalRequest.Data.As<ToolApprovalRequestContent>() should materialize the payload after JSON-checkpoint resume");
postResume!.ToolCall.Should().NotBeNull($"[{scenarioName}] the resumed TARC must carry its ToolCall");
postResume.ToolCall.Should().BeOfType<FunctionCallContent>(
$"[{scenarioName}] after CheckpointManager.CreateJson round-trip via ResumeStreamingAsync, " +
"ToolApprovalRequestContent.ToolCall must still be a FunctionCallContent so that " +
"FunctionInvokingChatClient's pattern match (`tarc.ToolCall is FunctionCallContent`) continues to fire.");
ToolApprovalResponseContent approvalResponse = postResume.CreateResponse(approved: true);
await resumed.SendResponseAsync(resumedRequest.CreateResponse(approvalResponse));
using CancellationTokenSource cts2 = new(TimeSpan.FromSeconds(30));
await foreach (WorkflowEvent evt in resumed.WatchStreamAsync(blockOnPendingRequest: false, cts2.Token))
{
postResumeEvents.Add(evt);
}
}
harness.InvocationCount.Should().Be(1,
$"[{scenarioName}] approving the request should cause FunctionInvokingChatClient to invoke the wrapped AIFunction exactly once");
postResumeEvents.OfType<WorkflowErrorEvent>().Should().BeEmpty(
$"[{scenarioName}] no workflow errors should be raised when responding to the resumed approval request");
postResumeEvents.OfType<ExecutorFailedEvent>().Should().BeEmpty(
$"[{scenarioName}] no executor failures should be raised when responding to the resumed approval request " +
"(regression guard: pre-broadcast-refactor this test was the `Track A2` repro in PR #5952 which surfaced a " +
"duplicate-key ArgumentException out of FunctionInvokingChatClient.ExtractAndRemoveApprovalRequestsAndResponses).");
}
/// <summary>
/// Bundles a <see cref="ChatClientAgent"/> with a counting <see cref="ApprovalRequiredAIFunction"/>
/// tool and a <see cref="MockChatClient"/> that emits a function call on the first chat turn
/// and a final assistant text on subsequent turns (after FICC has processed the approval
/// and appended a <see cref="FunctionResultContent"/>).
/// </summary>
private sealed class ApprovalHarness
{
public const string ToolName = "GetWeather";
public const string ToolResultText = "Sunny, 22°C";
public const string ToolCallId = "call-1";
public const string FinalAssistantText = "The weather in Amsterdam is sunny and 22°C.";
private int _invocationCount;
private int _chatCallIndex;
public int InvocationCount => Volatile.Read(ref this._invocationCount);
public int ChatCallCount => Volatile.Read(ref this._chatCallIndex);
public ChatClientAgent Agent { get; }
public ApprovalHarness()
{
AIFunction underlyingTool = AIFunctionFactory.Create(
([Description("City to look up")] string city) =>
{
Interlocked.Increment(ref this._invocationCount);
return ToolResultText;
},
name: ToolName,
description: "Gets the weather for the given city");
ApprovalRequiredAIFunction approvalTool = new(underlyingTool);
MockChatClient mockChatClient = new((messages, options) =>
{
int index = Interlocked.Increment(ref this._chatCallIndex) - 1;
return index switch
{
0 => new ChatResponse(new ChatMessage(ChatRole.Assistant,
[new FunctionCallContent(
callId: ToolCallId,
name: ToolName,
arguments: new Dictionary<string, object?> { ["city"] = "Amsterdam" })])),
_ => new ChatResponse(new ChatMessage(ChatRole.Assistant, FinalAssistantText)),
};
});
this.Agent = new ChatClientAgent(
mockChatClient,
instructions: "You are a weather agent.",
name: "WeatherAgent",
tools: [approvalTool]);
}
}
/// <summary>
/// Minimal <see cref="IChatClient"/> stub for orchestration tests; delegates each call to a
/// caller-supplied factory.
/// </summary>
private sealed class MockChatClient(Func<IEnumerable<ChatMessage>, ChatOptions?, ChatResponse> responseFactory) : IChatClient
{
public Task<ChatResponse> GetResponseAsync(IEnumerable<ChatMessage> messages, ChatOptions? options = null, CancellationToken cancellationToken = default)
=> Task.FromResult(responseFactory(messages, options));
public async IAsyncEnumerable<ChatResponseUpdate> GetStreamingResponseAsync(
IEnumerable<ChatMessage> messages,
ChatOptions? options = null,
[EnumeratorCancellation] CancellationToken cancellationToken = default)
{
ChatResponse response = await this.GetResponseAsync(messages, options, cancellationToken).ConfigureAwait(false);
foreach (ChatResponseUpdate update in response.ToChatResponseUpdates())
{
yield return update;
}
}
public object? GetService(Type serviceType, object? serviceKey = null) => null;
public void Dispose() { }
}
}
@@ -0,0 +1,477 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.CompilerServices;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using FluentAssertions;
using Microsoft.Agents.AI.Workflows.InProc;
using Microsoft.Extensions.AI;
namespace Microsoft.Agents.AI.Workflows.UnitTests;
public class GroupChatWorkflowBuilderTests
{
[Fact]
public void BuildGroupChat_InvalidArguments_Throws()
{
Assert.Throws<ArgumentNullException>("managerFactory", () => AgentWorkflowBuilder.CreateGroupChatBuilderWith(null!));
var groupChat = AgentWorkflowBuilder.CreateGroupChatBuilderWith(_ => new RoundRobinGroupChatManager([new OrchestrationTestHelpers.DoubleEchoAgent("a1")]));
Assert.NotNull(groupChat);
Assert.Throws<ArgumentNullException>("agents", () => groupChat.AddParticipants(null!));
Assert.Throws<ArgumentNullException>("agents", () => groupChat.AddParticipants([null!]));
Assert.Throws<ArgumentNullException>("agents", () => groupChat.AddParticipants(new OrchestrationTestHelpers.DoubleEchoAgent("a1"), null!));
Assert.Throws<ArgumentNullException>("agents", () => new RoundRobinGroupChatManager(null!));
}
[Fact]
public void GroupChatManager_MaximumIterationCount_Invalid_Throws()
{
var manager = new RoundRobinGroupChatManager([new OrchestrationTestHelpers.DoubleEchoAgent("a1")]);
const int DefaultMaxIterations = 40;
Assert.Equal(DefaultMaxIterations, manager.MaximumIterationCount);
Assert.Throws<ArgumentOutOfRangeException>("value", void () => manager.MaximumIterationCount = 0);
Assert.Throws<ArgumentOutOfRangeException>("value", void () => manager.MaximumIterationCount = -1);
Assert.Equal(DefaultMaxIterations, manager.MaximumIterationCount);
manager.MaximumIterationCount = 30;
Assert.Equal(30, manager.MaximumIterationCount);
manager.MaximumIterationCount = 1;
Assert.Equal(1, manager.MaximumIterationCount);
manager.MaximumIterationCount = int.MaxValue;
Assert.Equal(int.MaxValue, manager.MaximumIterationCount);
}
[Fact]
public void BuildGroupChat_WithNameAndDescription_SetsWorkflowNameAndDescription()
{
const string WorkflowName = "Test Group Chat";
const string WorkflowDescription = "A test group chat workflow";
var workflow = AgentWorkflowBuilder
.CreateGroupChatBuilderWith(agents => new RoundRobinGroupChatManager(agents) { MaximumIterationCount = 2 })
.AddParticipants(new OrchestrationTestHelpers.DoubleEchoAgent("agent1"), new OrchestrationTestHelpers.DoubleEchoAgent("agent2"))
.WithName(WorkflowName)
.WithDescription(WorkflowDescription)
.Build();
Assert.Equal(WorkflowName, workflow.Name);
Assert.Equal(WorkflowDescription, workflow.Description);
}
[Fact]
public void BuildGroupChat_WithNameOnly_SetsWorkflowName()
{
const string WorkflowName = "Named Group Chat";
var workflow = AgentWorkflowBuilder
.CreateGroupChatBuilderWith(agents => new RoundRobinGroupChatManager(agents) { MaximumIterationCount = 2 })
.AddParticipants(new OrchestrationTestHelpers.DoubleEchoAgent("agent1"))
.WithName(WorkflowName)
.Build();
Assert.Equal(WorkflowName, workflow.Name);
Assert.Null(workflow.Description);
}
[Fact]
public void BuildGroupChat_WithoutNameOrDescription_DefaultsToNull()
{
var workflow = AgentWorkflowBuilder
.CreateGroupChatBuilderWith(agents => new RoundRobinGroupChatManager(agents) { MaximumIterationCount = 2 })
.AddParticipants(new OrchestrationTestHelpers.DoubleEchoAgent("agent1"))
.Build();
Assert.Null(workflow.Name);
Assert.Null(workflow.Description);
}
[Theory]
[InlineData(1)]
[InlineData(2)]
[InlineData(3)]
[InlineData(4)]
[InlineData(5)]
public async Task BuildGroupChat_AgentsRunInOrderAsync(int maxIterations)
{
const int NumAgents = 3;
var workflow = AgentWorkflowBuilder.CreateGroupChatBuilderWith(agents => new RoundRobinGroupChatManager(agents) { MaximumIterationCount = maxIterations })
.AddParticipants(new OrchestrationTestHelpers.DoubleEchoAgent("agent1"), new OrchestrationTestHelpers.DoubleEchoAgent("agent2"))
.AddParticipants(new OrchestrationTestHelpers.DoubleEchoAgent("agent3"))
.Build();
for (int iter = 0; iter < 3; iter++)
{
const string UserInput = "abc";
(string updateText, List<ChatMessage>? result, _, _) = await OrchestrationTestHelpers.RunWorkflowAsync(workflow, [new ChatMessage(ChatRole.User, UserInput)]);
Assert.NotNull(result);
Assert.Equal(maxIterations + 1, result.Count);
Assert.Equal(ChatRole.User, result[0].Role);
Assert.Null(result[0].AuthorName);
Assert.Equal(UserInput, result[0].Text);
// The group-chat host broadcasts each new message (initial user input + each speaker's
// response) to every participant except the speaker that produced it. The selected
// speaker therefore sees only what's been broadcast to it since its previous turn.
string[] agentIds = ["agent1", "agent2", "agent3"];
List<string>[] buffers = new List<string>[NumAgents];
for (int a = 0; a < NumAgents; a++)
{
buffers[a] = [UserInput];
}
string[] texts = new string[maxIterations + 1];
texts[0] = UserInput;
string expectedTotal = string.Empty;
for (int i = 1; i < maxIterations + 1; i++)
{
int speakerIdx = (i - 1) % NumAgents;
string id = agentIds[speakerIdx];
string concatReceived = string.Concat(buffers[speakerIdx]);
texts[i] = $"{id}{Double(concatReceived)}";
buffers[speakerIdx].Clear();
for (int a = 0; a < NumAgents; a++)
{
if (a == speakerIdx)
{
continue;
}
buffers[a].Add(texts[i]);
}
Assert.Equal(ChatRole.Assistant, result[i].Role);
Assert.Equal(id, result[i].AuthorName);
Assert.Equal(texts[i], result[i].Text);
expectedTotal += texts[i];
}
Assert.Equal(expectedTotal, updateText);
Assert.Equal(UserInput + expectedTotal, string.Concat(result));
static string Double(string s) => s + s;
}
}
[Fact]
public void Test_GroupChatWorkflowBuilder_DefaultDesignationsMatchSpec()
{
OrchestrationTestHelpers.DoubleEchoAgent a1 = new("agent1");
OrchestrationTestHelpers.DoubleEchoAgent a2 = new("agent2");
OrchestrationTestHelpers.DoubleEchoAgent a3 = new("agent3");
Workflow workflow = AgentWorkflowBuilder
.CreateGroupChatBuilderWith(agents => new RoundRobinGroupChatManager(agents) { MaximumIterationCount = 1 })
.AddParticipants(a1, a2, a3)
.Build();
Dictionary<string, HashSet<OutputTag>> designations = workflow.OutputExecutors;
designations.Where(kvp => kvp.Value.Count == 0)
.Should().ContainSingle("group-chat host is the sole terminal output executor by default");
designations.Where(kvp => kvp.Value.Contains(OutputTag.Intermediate))
.Should().HaveCount(3, "every participant is designated intermediate by default");
}
[Fact]
public void Test_GroupChatWorkflowBuilder_ExplicitDesignationsReplaceDefaults()
{
OrchestrationTestHelpers.DoubleEchoAgent a1 = new("agent1");
OrchestrationTestHelpers.DoubleEchoAgent a2 = new("agent2");
OrchestrationTestHelpers.DoubleEchoAgent a3 = new("agent3");
Workflow workflow = AgentWorkflowBuilder
.CreateGroupChatBuilderWith(agents => new RoundRobinGroupChatManager(agents) { MaximumIterationCount = 1 })
.AddParticipants(a1, a2, a3)
.WithOutputFrom(a1)
.WithIntermediateOutputFrom([a2])
.Build();
Dictionary<string, HashSet<OutputTag>> designations = workflow.OutputExecutors;
designations.Should().HaveCount(2,
"only the two explicitly-designated agents land on the inner builder; the host default is suppressed");
designations.Values.Where(tags => tags.Count == 0)
.Should().ContainSingle("agent1 is the only terminal designation");
designations.Values.Where(tags => tags.Contains(OutputTag.Intermediate))
.Should().ContainSingle("agent2 is the only intermediate designation");
}
[Fact]
public void Test_GroupChatWorkflowBuilder_DesignationForNonParticipantThrows()
{
OrchestrationTestHelpers.DoubleEchoAgent participant = new("p1");
OrchestrationTestHelpers.DoubleEchoAgent stranger = new("stranger");
GroupChatWorkflowBuilder builder = AgentWorkflowBuilder
.CreateGroupChatBuilderWith(agents => new RoundRobinGroupChatManager(agents) { MaximumIterationCount = 1 })
.AddParticipants(participant)
.WithOutputFrom(stranger);
Action build = () => builder.Build();
build.Should().Throw<InvalidOperationException>().WithMessage("*stranger*");
}
private sealed class RecordingAgent(string name) : AIAgent
{
public List<List<string>> Invocations { get; } = [];
public override string Name => name;
protected override ValueTask<AgentSession> CreateSessionCoreAsync(CancellationToken cancellationToken = default)
=> new(new RecordingAgentSession());
protected override ValueTask<AgentSession> DeserializeSessionCoreAsync(JsonElement serializedState, JsonSerializerOptions? jsonSerializerOptions = null, CancellationToken cancellationToken = default)
=> new(new RecordingAgentSession());
protected override ValueTask<JsonElement> SerializeSessionCoreAsync(AgentSession session, JsonSerializerOptions? jsonSerializerOptions = null, CancellationToken cancellationToken = default)
=> default;
protected override Task<AgentResponse> RunCoreAsync(
IEnumerable<ChatMessage> messages, AgentSession? session = null, AgentRunOptions? options = null, CancellationToken cancellationToken = default) =>
throw new NotImplementedException();
protected override async IAsyncEnumerable<AgentResponseUpdate> RunCoreStreamingAsync(
IEnumerable<ChatMessage> messages, AgentSession? session = null, AgentRunOptions? options = null, [EnumeratorCancellation] CancellationToken cancellationToken = default)
{
await Task.Yield();
this.Invocations.Add(messages.Select(m => m.Text).ToList());
string id = Guid.NewGuid().ToString("N");
yield return new AgentResponseUpdate(ChatRole.Assistant, name) { AuthorName = name, MessageId = id };
}
}
private sealed class RecordingAgentSession() : AgentSession();
[Fact]
public async Task BuildGroupChat_BroadcastsDeltaAndTargetsTurnTokenToSpeakerOnlyAsync()
{
var agentA = new RecordingAgent("agentA");
var agentB = new RecordingAgent("agentB");
var agentC = new RecordingAgent("agentC");
var workflow = AgentWorkflowBuilder
.CreateGroupChatBuilderWith(agents => new RoundRobinGroupChatManager(agents) { MaximumIterationCount = 4 })
.AddParticipants(agentA, agentB, agentC)
.Build();
const string UserInput = "hello";
(_, List<ChatMessage>? result, _, _) = await OrchestrationTestHelpers.RunWorkflowAsync(workflow, [new ChatMessage(ChatRole.User, UserInput)]);
Assert.NotNull(result);
Assert.Equal(5, result.Count); // initial user input + 4 agent turns
Assert.Collection(
result,
m => Assert.Equal(UserInput, m.Text),
m => Assert.Equal("agentA", m.Text),
m => Assert.Equal("agentB", m.Text),
m => Assert.Equal("agentC", m.Text),
m => Assert.Equal("agentA", m.Text));
// Each agent's TurnToken fires exactly when it is the selected speaker — invocation counts
// confirm only the chosen participant receives a TurnToken on each round.
Assert.Equal(2, agentA.Invocations.Count);
Assert.Single(agentB.Invocations);
Assert.Single(agentC.Invocations);
// Turn 1: agentA is the first speaker. Initial broadcast went to every participant, so
// agentA's only buffered message is the user input.
Assert.Equal([UserInput], agentA.Invocations[0]);
// Turn 2: agentB. It received the initial broadcast (user input) plus turn-1 broadcast of
// agentA's response (agentA itself is excluded as the last speaker).
Assert.Equal([UserInput, "agentA"], agentB.Invocations[0]);
// Turn 3: agentC. It also received every broadcast so far (it has never been excluded).
Assert.Equal([UserInput, "agentA", "agentB"], agentC.Invocations[0]);
// Turn 4: agentA again. It was excluded on turn 2's broadcast (its own response), but
// received turn-3 (agentB's response) and turn-4 (agentC's response) deltas.
Assert.Equal(["agentB", "agentC"], agentA.Invocations[1]);
}
[Fact]
public async Task BuildGroupChat_UpdateHistoryAsync_FiltersBroadcastPayloadAsync()
{
var agentA = new RecordingAgent("agentA");
var agentB = new RecordingAgent("agentB");
var workflow = AgentWorkflowBuilder
.CreateGroupChatBuilderWith(agents => new PrefixingGroupChatManager(agents, "[broadcast] ") { MaximumIterationCount = 2 })
.AddParticipants(agentA, agentB)
.Build();
const string UserInput = "hello";
await OrchestrationTestHelpers.RunWorkflowAsync(workflow, [new ChatMessage(ChatRole.User, UserInput)]);
// Turn 1: agentA's buffer contains only the initial broadcast, which UpdateHistoryAsync
// prefixed.
Assert.Equal(["[broadcast] hello"], agentA.Invocations[0]);
// Turn 2: agentB received both the initial broadcast and agentA's response — both passed
// through UpdateHistoryAsync before being broadcast.
Assert.Equal(["[broadcast] hello", "[broadcast] agentA"], agentB.Invocations[0]);
}
[Fact]
public async Task BuildGroupChat_CheckpointResumeMidConversation_PreservesIterationCursorAndBroadcastExclusionAsync()
{
const string UserInput = "hello";
const int MaxIterations = 6;
// --- Baseline: run the full conversation under checkpointing and capture every checkpoint
// plus the final transcript. The same workflow + agents are reused for the resume,
// because the runner enforces workflow-shape compatibility on ResumeStreamingAsync. ---
BaselineRunResult baseline = await RunGroupChatBaselineAsync(UserInput, MaxIterations);
// We need at least one mid-conversation checkpoint to resume from. The baseline produces a
// checkpoint per superstep, which for MaxIterations=6 yields many; we pick a checkpoint
// captured roughly midway so the resumed run still has work to do.
Assert.True(baseline.Checkpoints.Count >= 5,
$"expected at least 5 checkpoints in the baseline, got {baseline.Checkpoints.Count}");
int midIndex = baseline.Checkpoints.Count / 2;
CheckpointInfo midCheckpoint = baseline.Checkpoints[midIndex];
// Snapshot per-agent invocation counts before the resume so we can isolate the invocations
// produced after the checkpoint is restored.
int aPreCount = baseline.AgentA.Invocations.Count;
int bPreCount = baseline.AgentB.Invocations.Count;
int cPreCount = baseline.AgentC.Invocations.Count;
// --- Resume the same workflow from the mid-conversation checkpoint. ---
List<ChatMessage>? resumedResult = null;
await using (StreamingRun resumed = await baseline.Environment
.ResumeStreamingAsync(baseline.Workflow, midCheckpoint))
{
await foreach (WorkflowEvent evt in resumed.WatchStreamAsync(blockOnPendingRequest: false))
{
if (evt is WorkflowOutputEvent o)
{
resumedResult = o.As<List<ChatMessage>>();
}
else if (evt is WorkflowErrorEvent err)
{
Assert.Fail($"Resumed workflow failed: {err.Exception}");
}
}
}
// (1) Iteration-count continuity: the resumed run terminates with exactly the same number
// of turns the baseline produced — proves IterationCount was rehydrated and the manager
// honored MaximumIterationCount across the boundary.
Assert.NotNull(resumedResult);
Assert.Equal(baseline.Result.Count, resumedResult!.Count);
// (2) Next-speaker consistency: the full transcript (initial input + every speaker's turn,
// in order) matches the baseline — proves the round-robin cursor was restored.
List<string?> baselineTranscript = [.. baseline.Result.Select(m => m.Text)];
List<string?> resumedTranscript = [.. resumedResult.Select(m => m.Text)];
Assert.Equal(baselineTranscript, resumedTranscript);
// (3) Broadcast exclusion holds across resume: a RecordingAgent's response text is just its
// own Name. Examine only the invocations recorded after the resume. If the host failed
// to exclude the current speaker from its post-resume broadcasts, an agent's next
// invocation buffer would contain its own previously produced response. Asserting that
// no post-resume invocation input contains the invoking agent's own name proves the
// exclusion was preserved through checkpoint+restore.
AssertPostResumeBroadcastExclusion(baseline.AgentA, aPreCount);
AssertPostResumeBroadcastExclusion(baseline.AgentB, bPreCount);
AssertPostResumeBroadcastExclusion(baseline.AgentC, cPreCount);
// Sanity: at least one agent was actually invoked after the resume; otherwise the test
// would trivially pass even if the host stopped scheduling turns after restore.
int totalPost = baseline.AgentA.Invocations.Count - aPreCount
+ (baseline.AgentB.Invocations.Count - bPreCount)
+ (baseline.AgentC.Invocations.Count - cPreCount);
Assert.True(totalPost > 0, "at least one agent should be invoked after resuming from the mid-conversation checkpoint");
static void AssertPostResumeBroadcastExclusion(RecordingAgent agent, int preCount)
{
for (int i = preCount; i < agent.Invocations.Count; i++)
{
Assert.DoesNotContain(agent.Name, agent.Invocations[i]);
}
}
}
private sealed record BaselineRunResult(
Workflow Workflow,
InProcessExecutionEnvironment Environment,
RecordingAgent AgentA,
RecordingAgent AgentB,
RecordingAgent AgentC,
List<ChatMessage> Result,
List<CheckpointInfo> Checkpoints,
CheckpointManager CheckpointManager);
private static async Task<BaselineRunResult> RunGroupChatBaselineAsync(string userInput, int maxIterations)
{
var agentA = new RecordingAgent("agentA");
var agentB = new RecordingAgent("agentB");
var agentC = new RecordingAgent("agentC");
Workflow workflow = AgentWorkflowBuilder
.CreateGroupChatBuilderWith(agents => new RoundRobinGroupChatManager(agents) { MaximumIterationCount = maxIterations })
.AddParticipants(agentA, agentB, agentC)
.Build();
CheckpointManager checkpointMgr = CheckpointManager.CreateInMemory();
InProcessExecutionEnvironment env = ExecutionEnvironment.InProcess_Lockstep
.ToWorkflowExecutionEnvironment()
.WithCheckpointing(checkpointMgr);
List<CheckpointInfo> checkpoints = [];
List<ChatMessage>? finalResult = null;
await using (StreamingRun run = await env.OpenStreamingAsync(workflow))
{
await run.TrySendMessageAsync(new List<ChatMessage> { new(ChatRole.User, userInput) });
await run.TrySendMessageAsync(new TurnToken(emitEvents: true));
await foreach (WorkflowEvent evt in run.WatchStreamAsync(blockOnPendingRequest: false))
{
switch (evt)
{
case SuperStepCompletedEvent step when step.CompletionInfo?.Checkpoint is { } cp:
checkpoints.Add(cp);
break;
case WorkflowOutputEvent o:
finalResult = o.As<List<ChatMessage>>();
break;
case WorkflowErrorEvent err:
Assert.Fail($"Baseline workflow failed: {err.Exception}");
break;
}
}
}
Assert.NotNull(finalResult);
return new BaselineRunResult(workflow, env, agentA, agentB, agentC, finalResult!, checkpoints, checkpointMgr);
}
private sealed class PrefixingGroupChatManager(IReadOnlyList<AIAgent> agents, string prefix) : RoundRobinGroupChatManager(agents)
{
protected internal override ValueTask<IEnumerable<ChatMessage>> UpdateHistoryAsync(
IReadOnlyList<ChatMessage> history,
CancellationToken cancellationToken = default)
{
IEnumerable<ChatMessage> prefixed =
history.Select(m => new ChatMessage(m.Role, $"{prefix}{m.Text}") { AuthorName = m.AuthorName });
return new(prefixed);
}
}
}
@@ -0,0 +1,77 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Linq;
using FluentAssertions;
namespace Microsoft.Agents.AI.Workflows.UnitTests;
/// <summary>
/// Tests focused on <see cref="HandoffWorkflowBuilder"/>'s output-designation surface —
/// the Python-aligned defaults applied at <see cref="HandoffWorkflowBuilderCore{TBuilder}.Build"/>
/// when the user has not made explicit designations, and the memoized
/// <c>WithOutputFrom</c> / <c>WithIntermediateOutputFrom</c> replay otherwise.
/// </summary>
#pragma warning disable MAAIW001 // Experimental: HandoffWorkflowBuilder
public class HandoffWorkflowBuilderTests
{
[Fact]
public void Test_HandoffWorkflowBuilder_DefaultDesignationsMatchSpec()
{
OrchestrationTestHelpers.DoubleEchoAgent coordinator = new("coordinator");
OrchestrationTestHelpers.DoubleEchoAgent specialist = new("specialist");
Workflow workflow = AgentWorkflowBuilder
.CreateHandoffBuilderWith(coordinator)
.WithHandoff(coordinator, specialist)
.Build();
Dictionary<string, HashSet<OutputTag>> designations = workflow.OutputExecutors;
designations.Where(kvp => kvp.Value.Count == 0)
.Should().ContainSingle("the handoff end executor is the sole terminal output by default");
designations.Where(kvp => kvp.Value.Contains(OutputTag.Intermediate))
.Should().HaveCount(2, "both the coordinator and the specialist are designated intermediate by default");
}
[Fact]
public void Test_HandoffWorkflowBuilder_ExplicitDesignationsReplaceDefaults()
{
OrchestrationTestHelpers.DoubleEchoAgent coordinator = new("coordinator");
OrchestrationTestHelpers.DoubleEchoAgent specialist = new("specialist");
Workflow workflow = AgentWorkflowBuilder
.CreateHandoffBuilderWith(coordinator)
.WithHandoff(coordinator, specialist)
.WithOutputFrom(coordinator)
.WithIntermediateOutputFrom([specialist])
.Build();
Dictionary<string, HashSet<OutputTag>> designations = workflow.OutputExecutors;
designations.Should().HaveCount(2,
"only the user-specified designations land on the inner builder; the handoff-end default is suppressed");
designations.Values.Where(tags => tags.Count == 0)
.Should().ContainSingle("coordinator is the only terminal designation");
designations.Values.Where(tags => tags.Contains(OutputTag.Intermediate))
.Should().ContainSingle("specialist is the only intermediate designation");
}
[Fact]
public void Test_HandoffWorkflowBuilder_DesignationForNonParticipantThrows()
{
OrchestrationTestHelpers.DoubleEchoAgent coordinator = new("coordinator");
OrchestrationTestHelpers.DoubleEchoAgent specialist = new("specialist");
OrchestrationTestHelpers.DoubleEchoAgent stranger = new("stranger");
HandoffWorkflowBuilder builder = AgentWorkflowBuilder
.CreateHandoffBuilderWith(coordinator)
.WithHandoff(coordinator, specialist)
.WithIntermediateOutputFrom([stranger]);
Action build = () => builder.Build();
build.Should().Throw<InvalidOperationException>().WithMessage("*stranger*");
}
}
#pragma warning restore MAAIW001
@@ -122,50 +122,3 @@ public sealed class InputWaiterTests : IDisposable
await waitTask;
}
}
public class OutputFilterTests
{
private static OutputFilter CreateFilterWithOutputFrom(string outputExecutorId)
{
NoOpExecutor start = new("start");
NoOpExecutor end = new("end");
Workflow workflow = new WorkflowBuilder("start")
.AddEdge(start, end)
.WithOutputFrom(outputExecutorId == "end" ? end : start)
.Build();
return new OutputFilter(workflow);
}
[Fact]
public void OutputFilter_CanOutput_ReturnsTrueForRegisteredExecutor()
{
OutputFilter filter = CreateFilterWithOutputFrom("end");
filter.CanOutput("end", "some output").Should().BeTrue("the executor was registered via WithOutputFrom");
}
[Fact]
public void OutputFilter_CanOutput_ReturnsFalseForUnregisteredExecutor()
{
OutputFilter filter = CreateFilterWithOutputFrom("end");
filter.CanOutput("start", "some output").Should().BeFalse("start was not registered as an output executor");
}
[Fact]
public void OutputFilter_CanOutput_ReturnsFalseForNonExistentExecutor()
{
OutputFilter filter = CreateFilterWithOutputFrom("end");
filter.CanOutput("nonexistent", "some output").Should().BeFalse("an executor not in the workflow should not be an output executor");
}
private sealed class NoOpExecutor(string id) : Executor(id)
{
protected override ProtocolBuilder ConfigureProtocol(ProtocolBuilder protocolBuilder)
=> protocolBuilder.ConfigureRoutes(routeBuilder =>
routeBuilder.AddHandler<object>((msg, ctx) => ctx.SendMessageAsync(msg)));
}
}
@@ -187,8 +187,12 @@ public class JsonSerializationTests
actual.InputType.Should().Match(prototype.InputType.CreateValidator());
actual.StartExecutorId.Should().Be(prototype.StartExecutorId);
actual.OutputExecutorIds.Should().HaveCount(prototype.OutputExecutorIds.Count)
.And.AllSatisfy(id => prototype.OutputExecutorIds.Contains(id));
actual.OutputExecutorIds.Should().HaveCount(prototype.OutputExecutorIds.Count);
foreach (KeyValuePair<string, HashSet<OutputTag>> kvp in prototype.OutputExecutorIds)
{
actual.OutputExecutorIds.Should().ContainKey(kvp.Key);
actual.OutputExecutorIds[kvp.Key].Should().BeEquivalentTo(kvp.Value);
}
void ValidateExecutorDictionary(Dictionary<string, ExecutorInfo> expected,
Dictionary<string, List<EdgeInfo>> expectedEdges,
@@ -788,6 +792,34 @@ public class JsonSerializationTests
result.IsTakingTurn.Should().Be(prototype.IsTakingTurn);
}
[Fact]
public void Test_GroupChatManagerState_JsonRoundtrip()
{
// Arrange
GroupChatManagerState prototype = new(IterationCount: 7);
// Act
GroupChatManagerState result = RunJsonRoundtrip(prototype);
// Assert
result.Should().Be(prototype);
result.IterationCount.Should().Be(prototype.IterationCount);
}
[Fact]
public void Test_RoundRobinGroupChatManagerState_JsonRoundtrip()
{
// Arrange
RoundRobinGroupChatManagerState prototype = new(NextIndex: 3);
// Act
RoundRobinGroupChatManagerState result = RunJsonRoundtrip(prototype);
// Assert
result.Should().Be(prototype);
result.NextIndex.Should().Be(prototype.NextIndex);
}
/// <summary>
/// Verifies that the default behavior (without AllowOutOfOrderMetadataProperties) fails
/// when $type metadata is not the first property, demonstrating the PostgreSQL jsonb issue.
@@ -361,6 +361,64 @@ public class MagenticOrchestrationTests
runResult.Result![0].Text.Should().Contain("Multi-round task completed!");
}
[Fact]
public async Task RunCoordinationRound_Forwards_Participant_Reply_To_ManagerAsync()
{
// Regression: MagenticOrchestrator.TakeTurnAsync used to drop the `messages`
// parameter on subsequent turns, so participant replies never reached the
// manager's ChatHistory. The manager then re-dispatched the same speaker
// every round until MaxRounds. Assert that round-2's progress-ledger call
// actually sees the worker's reply in its input.
const string TaskPrompt = "Echo back this exact magentic-regression-marker";
List<ChatMessage> factsResponse = CreatePlanResponse("Facts");
List<ChatMessage> planResponse = CreatePlanResponse("Plan");
List<ChatMessage> round1Ledger = CreateProgressLedgerResponse(
isRequestSatisfied: false,
isInLoop: false,
isProgressBeingMade: true,
nextSpeaker: "Worker",
instructionOrQuestion: TaskPrompt);
List<ChatMessage> round2Ledger = CreateProgressLedgerResponse(
isRequestSatisfied: true,
isInLoop: false,
isProgressBeingMade: true,
nextSpeaker: "Worker",
instructionOrQuestion: "Done");
List<ChatMessage> finalAnswer = CreateFinalAnswerResponse("All good");
RecordingReplayAgent manager = new(
[factsResponse, planResponse, round1Ledger, round2Ledger, finalAnswer],
name: "Manager");
TestEchoAgent worker = new(name: "Worker");
Workflow workflow = new MagenticWorkflowBuilder(manager)
.AddParticipants(worker)
.RequirePlanSignoff(false)
.Build();
WorkflowRunResult runResult = await RunMagenticWorkflowAsync(
workflow,
[new ChatMessage(ChatRole.User, TaskPrompt)]);
runResult.Result.Should().NotBeNull();
runResult.Result![0].Text.Should().Contain("All good");
// Calls in order: facts, plan, ledger1, ledger2, finalAnswer.
manager.RecordedInputs.Should().HaveCount(5);
manager.RecordedInputs[3].Should().Contain(
m => m.Role == ChatRole.Assistant
&& m.AuthorName == "Worker"
&& m.Text.Contains(TaskPrompt),
"round-2 progress ledger must see the worker's reply; without it the manager loops to MaxRounds");
manager.RecordedInputs[4].Should().Contain(
m => m.Role == ChatRole.Assistant && m.AuthorName == "Worker",
"final-answer synthesis must see what participants actually said");
}
[Fact]
public async Task PlanReview_Revised_Triggers_ReplanAsync()
{
@@ -0,0 +1,79 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Linq;
using FluentAssertions;
namespace Microsoft.Agents.AI.Workflows.UnitTests;
/// <summary>
/// Tests focused on <see cref="MagenticWorkflowBuilder"/>'s output-designation surface —
/// the Python-aligned defaults applied at <see cref="MagenticWorkflowBuilder.Build"/> when
/// the user has not made explicit designations, and the memoized
/// <c>WithOutputFrom</c> / <c>WithIntermediateOutputFrom</c> replay otherwise.
/// </summary>
#pragma warning disable MAAIW001 // Experimental: MagenticWorkflowBuilder
public class MagenticWorkflowBuilderTests
{
[Fact]
public void Test_MagenticWorkflowBuilder_DefaultDesignationsMatchSpec()
{
TestReplayAgent manager = new(name: "Manager");
TestEchoAgent member1 = new(name: "Worker1");
TestEchoAgent member2 = new(name: "Worker2");
Workflow workflow = new MagenticWorkflowBuilder(manager)
.AddParticipants(member1, member2)
.RequirePlanSignoff(false)
.Build();
Dictionary<string, HashSet<OutputTag>> designations = workflow.OutputExecutors;
designations.Where(kvp => kvp.Value.Count == 0)
.Should().ContainSingle("the Magentic orchestrator is the sole terminal output by default");
designations.Where(kvp => kvp.Value.Contains(OutputTag.Intermediate))
.Should().HaveCount(2, "every team member is designated intermediate by default");
}
[Fact]
public void Test_MagenticWorkflowBuilder_ExplicitDesignationsReplaceDefaults()
{
TestReplayAgent manager = new(name: "Manager");
TestEchoAgent member1 = new(name: "Worker1");
TestEchoAgent member2 = new(name: "Worker2");
Workflow workflow = new MagenticWorkflowBuilder(manager)
.AddParticipants(member1, member2)
.RequirePlanSignoff(false)
.WithOutputFrom(member1)
.WithIntermediateOutputFrom([member2])
.Build();
Dictionary<string, HashSet<OutputTag>> designations = workflow.OutputExecutors;
designations.Should().HaveCount(2,
"only the user-specified designations land on the inner builder; the orchestrator default is suppressed");
designations.Values.Where(tags => tags.Count == 0)
.Should().ContainSingle("member1 is the only terminal designation");
designations.Values.Where(tags => tags.Contains(OutputTag.Intermediate))
.Should().ContainSingle("member2 is the only intermediate designation");
}
[Fact]
public void Test_MagenticWorkflowBuilder_DesignationForNonParticipantThrows()
{
TestReplayAgent manager = new(name: "Manager");
TestEchoAgent member = new(name: "Worker");
TestEchoAgent stranger = new(name: "Stranger");
MagenticWorkflowBuilder builder = new MagenticWorkflowBuilder(manager)
.AddParticipants(member)
.RequirePlanSignoff(false)
.WithIntermediateOutputFrom([stranger]);
Action build = () => builder.Build();
build.Should().Throw<InvalidOperationException>().WithMessage("*Stranger*");
}
}
#pragma warning restore MAAIW001
@@ -0,0 +1,131 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.CompilerServices;
using System.Text;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Agents.AI.Workflows.InProc;
using Microsoft.Extensions.AI;
namespace Microsoft.Agents.AI.Workflows.UnitTests;
/// <summary>
/// Container for shared test helpers used by every orchestration-builder test class —
/// the <c>DoubleEchoAgent</c> family and the <c>RunWorkflow*</c> methods. The actual
/// test methods live in per-builder files (<c>SequentialWorkflowBuilderTests</c>,
/// <c>ConcurrentWorkflowBuilderTests</c>, <c>GroupChatWorkflowBuilderTests</c>, etc.).
/// </summary>
public static class OrchestrationTestHelpers
{
internal class DoubleEchoAgent(string name) : AIAgent
{
public override string Name => name;
protected override ValueTask<AgentSession> CreateSessionCoreAsync(CancellationToken cancellationToken = default)
=> new(new DoubleEchoAgentSession());
protected override ValueTask<AgentSession> DeserializeSessionCoreAsync(JsonElement serializedState, JsonSerializerOptions? jsonSerializerOptions = null, CancellationToken cancellationToken = default)
=> new(new DoubleEchoAgentSession());
protected override ValueTask<JsonElement> SerializeSessionCoreAsync(AgentSession session, JsonSerializerOptions? jsonSerializerOptions = null, CancellationToken cancellationToken = default)
=> default;
protected override Task<AgentResponse> RunCoreAsync(
IEnumerable<ChatMessage> messages, AgentSession? session = null, AgentRunOptions? options = null, CancellationToken cancellationToken = default) =>
throw new NotImplementedException();
protected override async IAsyncEnumerable<AgentResponseUpdate> RunCoreStreamingAsync(
IEnumerable<ChatMessage> messages, AgentSession? session = null, AgentRunOptions? options = null, [EnumeratorCancellation] CancellationToken cancellationToken = default)
{
await Task.Yield();
var contents = messages.SelectMany(m => m.Contents).ToList();
string id = Guid.NewGuid().ToString("N");
yield return new AgentResponseUpdate(ChatRole.Assistant, this.Name) { AuthorName = this.Name, MessageId = id };
yield return new AgentResponseUpdate(ChatRole.Assistant, contents) { AuthorName = this.Name, MessageId = id };
yield return new AgentResponseUpdate(ChatRole.Assistant, contents) { AuthorName = this.Name, MessageId = id };
}
}
internal sealed class DoubleEchoAgentSession() : AgentSession();
internal sealed class DoubleEchoAgentWithBarrier(string name, StrongBox<TaskCompletionSource<bool>> barrier, StrongBox<int> remaining) : DoubleEchoAgent(name)
{
protected override async IAsyncEnumerable<AgentResponseUpdate> RunCoreStreamingAsync(
IEnumerable<ChatMessage> messages, AgentSession? session = null, AgentRunOptions? options = null, [EnumeratorCancellation] CancellationToken cancellationToken = default)
{
if (Interlocked.Decrement(ref remaining.Value) == 0)
{
barrier.Value!.SetResult(true);
}
await barrier.Value!.Task.ConfigureAwait(false);
await foreach (var update in base.RunCoreStreamingAsync(messages, session, options, cancellationToken))
{
await Task.Yield();
yield return update;
}
}
}
internal sealed record WorkflowRunResult(string UpdateText, List<ChatMessage>? Result, CheckpointInfo? LastCheckpoint, List<RequestInfoEvent> PendingRequests);
internal static async Task<WorkflowRunResult> RunWorkflowCheckpointedAsync(
Workflow workflow, List<ChatMessage> input, InProcessExecutionEnvironment environment, CheckpointInfo? fromCheckpoint = null)
{
await using StreamingRun run =
fromCheckpoint != null ? await environment.ResumeStreamingAsync(workflow, fromCheckpoint)
: await environment.OpenStreamingAsync(workflow);
await run.TrySendMessageAsync(input);
await run.TrySendMessageAsync(new TurnToken(emitEvents: true));
return await ProcessWorkflowRunAsync(run);
}
internal static async Task<WorkflowRunResult> ProcessWorkflowRunAsync(StreamingRun run)
{
StringBuilder sb = new();
WorkflowOutputEvent? output = null;
CheckpointInfo? lastCheckpoint = null;
List<RequestInfoEvent> pendingRequests = [];
await foreach (WorkflowEvent evt in run.WatchStreamAsync(blockOnPendingRequest: false).ConfigureAwait(false))
{
switch (evt)
{
case AgentResponseUpdateEvent responseUpdate:
sb.Append(responseUpdate.Data);
break;
case RequestInfoEvent requestInfo:
pendingRequests.Add(requestInfo);
break;
case WorkflowOutputEvent e:
output = e;
break;
case WorkflowErrorEvent errorEvent:
Assert.Fail($"Workflow execution failed with error: {errorEvent.Exception}");
break;
case SuperStepCompletedEvent stepCompleted:
lastCheckpoint = stepCompleted.CompletionInfo?.Checkpoint;
break;
}
}
return new(sb.ToString(), output?.As<List<ChatMessage>>(), lastCheckpoint, pendingRequests);
}
internal static Task<WorkflowRunResult> RunWorkflowAsync(
Workflow workflow, List<ChatMessage> input, ExecutionEnvironment executionEnvironment = ExecutionEnvironment.InProcess_Lockstep)
=> RunWorkflowCheckpointedAsync(workflow, input, executionEnvironment.ToWorkflowExecutionEnvironment());
}
@@ -0,0 +1,108 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Collections.Generic;
using FluentAssertions;
using Microsoft.Agents.AI.Workflows.Execution;
namespace Microsoft.Agents.AI.Workflows.UnitTests;
public class OutputFilterTests
{
private static OutputFilter CreateFilterWithOutputFrom(string outputExecutorId)
{
NoOpExecutor start = new("start");
NoOpExecutor end = new("end");
Workflow workflow = new WorkflowBuilder("start")
.AddEdge(start, end)
.WithOutputFrom(outputExecutorId == "end" ? end : start)
.Build();
return new OutputFilter(workflow);
}
[Fact]
public void OutputFilter_CanOutput_ReturnsTrueForRegisteredExecutor()
{
OutputFilter filter = CreateFilterWithOutputFrom("end");
filter.CanOutput("end", "some output").Should().BeTrue("the executor was registered via WithOutputFrom");
}
[Fact]
public void OutputFilter_CanOutput_ReturnsFalseForUnregisteredExecutor()
{
OutputFilter filter = CreateFilterWithOutputFrom("end");
filter.CanOutput("start", "some output").Should().BeFalse("start was not registered as an output executor");
}
[Fact]
public void OutputFilter_CanOutput_ReturnsFalseForNonExistentExecutor()
{
OutputFilter filter = CreateFilterWithOutputFrom("end");
filter.CanOutput("nonexistent", "some output").Should().BeFalse("an executor not in the workflow should not be an output executor");
}
[Fact]
public void Test_OutputFilter_ReturnsEmptyTagSetWhenRegisteredViaWithOutputFrom()
{
OutputFilter filter = CreateFilterWithOutputFrom("end");
filter.TryGetTags("end", out HashSet<OutputTag>? tags).Should().BeTrue();
tags.Should().NotBeNull().And.BeEmpty("terminal designation carries no tag");
}
[Fact]
public void Test_OutputFilter_ReturnsIntermediateTagWhenRegisteredViaWithIntermediateOutputFrom()
{
NoOpExecutor start = new("start");
NoOpExecutor end = new("end");
Workflow workflow = new WorkflowBuilder("start")
.AddEdge(start, end)
.WithIntermediateOutputFrom([end])
.Build();
OutputFilter filter = new(workflow);
filter.TryGetTags("end", out HashSet<OutputTag>? tags).Should().BeTrue();
tags.Should().BeEquivalentTo(new[] { OutputTag.Intermediate });
}
[Fact]
public void Test_OutputFilter_ReturnsIntermediateTagForAccumulatedDesignation()
{
NoOpExecutor start = new("start");
NoOpExecutor end = new("end");
Workflow workflow = new WorkflowBuilder("start")
.AddEdge(start, end)
.WithOutputFrom(end)
.WithIntermediateOutputFrom([end])
.Build();
OutputFilter filter = new(workflow);
filter.TryGetTags("end", out HashSet<OutputTag>? tags).Should().BeTrue();
tags.Should().BeEquivalentTo(new[] { OutputTag.Intermediate },
"terminal designation contributes no tag; the union is the intermediate set");
}
[Fact]
public void Test_OutputFilter_TryGetTagsReturnsFalseForUnregisteredExecutor()
{
OutputFilter filter = CreateFilterWithOutputFrom("end");
filter.TryGetTags("start", out HashSet<OutputTag>? tags).Should().BeFalse();
tags.Should().BeNull();
}
private sealed class NoOpExecutor(string id) : Executor(id)
{
protected override ProtocolBuilder ConfigureProtocol(ProtocolBuilder protocolBuilder)
=> protocolBuilder.ConfigureRoutes(routeBuilder =>
routeBuilder.AddHandler<object>((msg, ctx) => ctx.SendMessageAsync(msg)));
}
}
@@ -0,0 +1,77 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Collections.Generic;
using System.Reflection;
using System.Text.Json;
using FluentAssertions;
namespace Microsoft.Agents.AI.Workflows.UnitTests;
public class OutputTagTests
{
[Fact]
public void Test_OutputTag_KnownValues()
{
OutputTag.Intermediate.Value.Should().Be("intermediate");
}
[Fact]
public void Test_OutputTag_EqualityIsOrdinalOnValue()
{
OutputTag.Intermediate.Should().Be(OutputTag.Intermediate);
(OutputTag.Intermediate == OutputTag.Intermediate).Should().BeTrue();
// Same Value via independent construction (via JSON round-trip below) is equal.
OutputTag rebuilt = JsonSerializer.Deserialize<OutputTag>("\"intermediate\"", WorkflowsJsonUtilities.DefaultOptions);
rebuilt.Should().Be(OutputTag.Intermediate);
}
[Fact]
public void Test_OutputTag_DefaultStructValueIsDistinct()
{
OutputTag def = default;
def.Value.Should().BeNull();
def.Should().NotBe(OutputTag.Intermediate);
def.GetHashCode().Should().Be(0);
HashSet<OutputTag> set = [OutputTag.Intermediate];
set.Contains(def).Should().BeFalse("default(OutputTag) must not collide with the well-known singleton in a HashSet");
}
[Fact]
public void Test_OutputTag_GetHashCodeMatchesEquals()
{
OutputTag a = OutputTag.Intermediate;
OutputTag b = JsonSerializer.Deserialize<OutputTag>("\"intermediate\"", WorkflowsJsonUtilities.DefaultOptions);
a.Equals(b).Should().BeTrue();
a.GetHashCode().Should().Be(b.GetHashCode());
}
[Fact]
public void Test_OutputTag_JsonConverter_RoundtripsValueAsString()
{
string intermediateJson = JsonSerializer.Serialize(OutputTag.Intermediate, WorkflowsJsonUtilities.DefaultOptions);
intermediateJson.Should().Be("\"intermediate\"");
OutputTag back = JsonSerializer.Deserialize<OutputTag>("\"intermediate\"", WorkflowsJsonUtilities.DefaultOptions);
back.Should().Be(OutputTag.Intermediate);
OutputTag fromUnknown = JsonSerializer.Deserialize<OutputTag>("\"custom\"", WorkflowsJsonUtilities.DefaultOptions);
fromUnknown.Value.Should().Be("custom");
}
[Fact]
public void Test_OutputTag_ConstructorIsInternal()
{
ConstructorInfo? ctor = typeof(OutputTag).GetConstructor(
BindingFlags.Instance | BindingFlags.NonPublic,
binder: null,
types: [typeof(string)],
modifiers: null);
ctor.Should().NotBeNull("OutputTag(string) must exist as an internal constructor");
ctor!.IsAssembly.Should().BeTrue("OutputTag(string) must be `internal` so external assemblies cannot synthesize tags");
ctor.IsPublic.Should().BeFalse();
}
}
@@ -0,0 +1,32 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Collections.Generic;
using System.Linq;
using System.Runtime.CompilerServices;
using System.Threading;
using Microsoft.Extensions.AI;
namespace Microsoft.Agents.AI.Workflows.UnitTests;
/// <summary>
/// A <see cref="TestReplayAgent"/> that records the input messages it receives on each call.
/// Used by tests that need to assert what context the agent was actually handed.
/// </summary>
internal sealed class RecordingReplayAgent(List<List<ChatMessage>> messages, string? id = null, string? name = null)
: TestReplayAgent(messages, id, name)
{
public List<List<ChatMessage>> RecordedInputs { get; } = [];
protected override async IAsyncEnumerable<AgentResponseUpdate> RunCoreStreamingAsync(
IEnumerable<ChatMessage> messages,
AgentSession? session = null,
AgentRunOptions? options = null,
[EnumeratorCancellation] CancellationToken cancellationToken = default)
{
this.RecordedInputs.Add(messages.ToList());
await foreach (AgentResponseUpdate update in base.RunCoreStreamingAsync(messages, session, options, cancellationToken))
{
yield return update;
}
}
}
@@ -136,4 +136,55 @@ public class RoundRobinGroupChatManagerTests
FluentActions.Invoking(() => new RoundRobinGroupChatManager([]))
.Should().Throw<System.ArgumentException>();
}
[Fact]
public async Task RoundRobinGroupChat_CheckpointRoundTrip_PreservesIterationCountAndCursorAsync()
{
TestEchoAgent agent1 = new(id: "agent1");
TestEchoAgent agent2 = new(id: "agent2");
TestEchoAgent agent3 = new(id: "agent3");
List<AIAgent> agents = [agent1, agent2, agent3];
List<ChatMessage> history = [];
TestRunState sharedState = new();
TestWorkflowContext sourceContext = new("gcm-host", sharedState);
TestWorkflowContext sinkContext = new("gcm-host", sharedState);
RoundRobinGroupChatManager source = new(agents);
await source.SelectNextAgentAsync(history); // cursor -> agent2
source.IterationCount = 7;
await source.CheckpointAsync(sourceContext);
RoundRobinGroupChatManager restored = new(agents);
restored.IterationCount.Should().Be(0, "freshly constructed manager has no iteration count");
await restored.RestoreCheckpointAsync(sinkContext);
restored.IterationCount.Should().Be(7, "the base hook must rehydrate IterationCount");
AIAgent next = await restored.SelectNextAgentAsync(history);
next.Should().BeSameAs(agent2, "the round-robin cursor should resume where the source left off");
}
[Fact]
public async Task RoundRobinGroupChat_RestoreWithoutCheckpoint_DefaultsToZeroStateAsync()
{
TestEchoAgent agent1 = new(id: "agent1");
TestEchoAgent agent2 = new(id: "agent2");
List<AIAgent> agents = [agent1, agent2];
List<ChatMessage> history = [];
TestWorkflowContext emptyContext = new("gcm-host");
RoundRobinGroupChatManager manager = new(agents);
manager.IterationCount = 3;
await manager.SelectNextAgentAsync(history); // cursor advanced
await manager.RestoreCheckpointAsync(emptyContext);
manager.IterationCount.Should().Be(0, "restore from an empty checkpoint should clear IterationCount");
AIAgent next = await manager.SelectNextAgentAsync(history);
next.Should().BeSameAs(agent1, "restore from an empty checkpoint should reset the cursor to the first agent");
}
}
@@ -0,0 +1,183 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using FluentAssertions;
using Microsoft.Agents.AI.Workflows.UnitTests.Futures;
using Microsoft.Extensions.AI;
namespace Microsoft.Agents.AI.Workflows.UnitTests;
public class SequentialWorkflowBuilderTests
{
[Fact]
public void Test_SequentialWorkflowBuilder_InvalidArguments_Throws()
{
Assert.Throws<ArgumentNullException>("agents", () => new SequentialWorkflowBuilder(null!));
Assert.Throws<ArgumentException>("agents", () => new SequentialWorkflowBuilder().Build());
Assert.Throws<ArgumentNullException>("agents", () => AgentWorkflowBuilder.BuildSequential(workflowName: null!, null!));
Assert.Throws<ArgumentException>("agents", () => AgentWorkflowBuilder.BuildSequential());
Assert.Throws<ArgumentNullException>("agents", () => AgentWorkflowBuilder.CreateSequentialBuilderWith(null!));
}
[Theory]
[InlineData(1)]
[InlineData(2)]
[InlineData(3)]
[InlineData(4)]
[InlineData(5)]
public async Task Test_SequentialWorkflowBuilder_AgentsRunInOrderAsync(int numAgents)
{
var workflow = new SequentialWorkflowBuilder(
from i in Enumerable.Range(1, numAgents)
select new OrchestrationTestHelpers.DoubleEchoAgent($"agent{i}"))
.Build();
for (int iter = 0; iter < 3; iter++)
{
const string UserInput = "abc";
(string updateText, List<ChatMessage>? result, _, _) =
await OrchestrationTestHelpers.RunWorkflowAsync(workflow, [new ChatMessage(ChatRole.User, UserInput)]);
Assert.NotNull(result);
Assert.Equal(numAgents + 1, result.Count);
Assert.Equal(ChatRole.User, result[0].Role);
Assert.Null(result[0].AuthorName);
Assert.Equal(UserInput, result[0].Text);
string[] texts = new string[numAgents + 1];
texts[0] = UserInput;
string expectedTotal = string.Empty;
for (int i = 1; i < numAgents + 1; i++)
{
string id = $"agent{((i - 1) % numAgents) + 1}";
texts[i] = $"{id}{Double(string.Concat(texts.Take(i)))}";
Assert.Equal(ChatRole.Assistant, result[i].Role);
Assert.Equal(id, result[i].AuthorName);
Assert.Equal(texts[i], result[i].Text);
expectedTotal += texts[i];
}
Assert.Equal(expectedTotal, updateText);
Assert.Equal(UserInput + expectedTotal, string.Concat(result));
static string Double(string s) => s + s;
}
}
[Fact]
public void Test_SequentialWorkflowBuilder_DefaultDesignationsMatchSpec()
{
Workflow workflow = new SequentialWorkflowBuilder(
new OrchestrationTestHelpers.DoubleEchoAgent("agent1"),
new OrchestrationTestHelpers.DoubleEchoAgent("agent2"),
new OrchestrationTestHelpers.DoubleEchoAgent("agent3"))
.Build();
Dictionary<string, HashSet<OutputTag>> designations = workflow.OutputExecutors;
designations.Where(kvp => kvp.Value.Count == 0)
.Should().ContainSingle("OutputMessagesExecutor is the sole terminal output by default");
designations.Where(kvp => kvp.Value.Contains(OutputTag.Intermediate))
.Should().HaveCount(3, "every pipeline agent is designated intermediate by default");
}
[Fact]
public void Test_SequentialWorkflowBuilder_ExplicitDesignationsReplaceDefaults()
{
OrchestrationTestHelpers.DoubleEchoAgent a1 = new("agent1");
OrchestrationTestHelpers.DoubleEchoAgent a2 = new("agent2");
OrchestrationTestHelpers.DoubleEchoAgent a3 = new("agent3");
Workflow workflow = new SequentialWorkflowBuilder(a1, a2, a3)
.WithOutputFrom(a1)
.WithIntermediateOutputFrom([a2])
.Build();
Dictionary<string, HashSet<OutputTag>> designations = workflow.OutputExecutors;
designations.Should().HaveCount(2,
"only the two explicitly-designated agents land on the inner builder; the end default is suppressed");
designations.Values.Where(tags => tags.Count == 0)
.Should().ContainSingle("agent1 is the only terminal designation");
designations.Values.Where(tags => tags.Contains(OutputTag.Intermediate))
.Should().ContainSingle("agent2 is the only intermediate designation");
}
[Fact]
public void Test_SequentialWorkflowBuilder_DesignationForNonParticipantThrows()
{
OrchestrationTestHelpers.DoubleEchoAgent participant = new("p1");
OrchestrationTestHelpers.DoubleEchoAgent stranger = new("stranger");
SequentialWorkflowBuilder builder = new SequentialWorkflowBuilder(participant)
.WithIntermediateOutputFrom([stranger]);
Action build = () => builder.Build();
build.Should().Throw<InvalidOperationException>().WithMessage("*stranger*");
}
[Fact]
public void Test_SequentialWorkflowBuilder_WithNamePropagatesToWorkflow()
{
Workflow workflow = new SequentialWorkflowBuilder(new OrchestrationTestHelpers.DoubleEchoAgent("agent1"))
.WithName("named-sequential")
.Build();
workflow.Name.Should().Be("named-sequential");
}
[Fact]
public void Test_SequentialWorkflowBuilder_WithDescriptionPropagatesToWorkflow()
{
Workflow workflow = new SequentialWorkflowBuilder(new OrchestrationTestHelpers.DoubleEchoAgent("agent1"))
.WithDescription("describes the sequential pipeline")
.Build();
workflow.Description.Should().Be("describes the sequential pipeline");
}
[Collection(FuturesSerialCollection.Name)]
public class AsAgentForwarding
{
[Fact]
public async Task Test_SequentialWorkflowBuilder_AsAgent_OnlyTerminalDesignationSurfacesAsync()
{
using FuturesScope _ = new(enabled: true);
OrchestrationTestHelpers.DoubleEchoAgent agent1 = new("agent1");
OrchestrationTestHelpers.DoubleEchoAgent agent2 = new("agent2");
OrchestrationTestHelpers.DoubleEchoAgent agent3 = new("agent3");
// Explicitly designate ONLY the last agent — defaults (which would tag every agent
// intermediate) are suppressed, so under Futures-on, agent1/agent2 produce no
// AgentResponse(Update)Events and nothing of theirs reaches the AsAgent stream.
Workflow workflow = new SequentialWorkflowBuilder(agent1, agent2, agent3)
.WithOutputFrom(agent3)
.Build();
List<AgentResponseUpdate> updates = await workflow
.AsAIAgent("WorkflowAgent")
.RunStreamingAsync(new ChatMessage(ChatRole.User, "abc"))
.ToListAsync();
// Filter by AuthorName — distinguishes which agent originated each update
// (text-content checks are unreliable because agent3 echoes earlier agents' markers
// as part of the cumulative pipeline payload).
HashSet<string> authoredBy = updates
.Select(u => u.AuthorName)
.Where(n => !string.IsNullOrEmpty(n))
.Select(n => n!)
.ToHashSet();
authoredBy.Should().Contain("agent3", "the terminal agent must surface");
authoredBy.Should().NotContain("agent1",
"the intermediate agent must not surface when only the terminal is designated");
authoredBy.Should().NotContain("agent2",
"the intermediate agent must not surface when only the terminal is designated");
}
}
}
@@ -6,7 +6,7 @@ using FluentAssertions;
namespace Microsoft.Agents.AI.Workflows.UnitTests;
public partial class WorkflowBuilderSmokeTests
public partial class WorkflowBuilderTests
{
private sealed class NoOpExecutor(string id) : Executor(id)
{
@@ -455,4 +455,112 @@ public partial class WorkflowBuilderSmokeTests
/// </summary>
private static Edge GetSingleEdge(Workflow workflow, string sourceId)
=> workflow.Edges[sourceId].Should().ContainSingle().Subject;
// --- Tag-aware WithOutputFrom / WithIntermediateOutputFrom tests ---
[Fact]
public void Test_WithOutputFrom_RegistersWithEmptyTagSet()
{
NoOpExecutor a = new("a");
NoOpExecutor b = new("b");
Workflow workflow = new WorkflowBuilder("a")
.AddEdge(a, b)
.WithOutputFrom(b)
.Build();
workflow.OutputExecutors.Should().ContainKey("b");
workflow.OutputExecutors["b"].Should().BeEmpty("regular outputs are untagged");
}
[Fact]
public void Test_WithIntermediateOutputFrom_AddsIntermediateTag()
{
NoOpExecutor a = new("a");
NoOpExecutor b = new("b");
Workflow workflow = new WorkflowBuilder("a")
.AddEdge(a, b)
.WithIntermediateOutputFrom([b])
.Build();
workflow.OutputExecutors["b"].Should().BeEquivalentTo(new[] { OutputTag.Intermediate });
}
[Fact]
public void Test_WithOutputFrom_MultipleExecutorsAllUntagged()
{
NoOpExecutor a = new("a");
NoOpExecutor b = new("b");
NoOpExecutor c = new("c");
Workflow workflow = new WorkflowBuilder("a")
.AddEdge(a, b).AddEdge(a, c)
.WithOutputFrom(b, c)
.Build();
workflow.OutputExecutors.Should().HaveCount(2);
workflow.OutputExecutors["b"].Should().BeEmpty();
workflow.OutputExecutors["c"].Should().BeEmpty();
}
[Fact]
public void Test_WithOutputFrom_ThenIntermediate_AccumulatesTags()
{
NoOpExecutor a = new("a");
NoOpExecutor b = new("b");
Workflow workflow = new WorkflowBuilder("a")
.AddEdge(a, b)
.WithOutputFrom(b)
.WithIntermediateOutputFrom([b])
.Build();
// WithOutputFrom doesn't add a tag; WithIntermediateOutputFrom adds Intermediate.
workflow.OutputExecutors["b"].Should().BeEquivalentTo(new[] { OutputTag.Intermediate });
}
[Fact]
public void Test_WithIntermediateOutputFrom_RepeatedDedupes()
{
NoOpExecutor a = new("a");
NoOpExecutor b = new("b");
Workflow workflow = new WorkflowBuilder("a")
.AddEdge(a, b)
.WithIntermediateOutputFrom([b])
.WithIntermediateOutputFrom([b])
.Build();
workflow.OutputExecutors["b"].Should().BeEquivalentTo(new[] { OutputTag.Intermediate });
}
[Fact]
public void Test_WithIntermediateOutputFrom_OnlyRegistersWithoutPriorWithOutputFrom()
{
// WithIntermediateOutputFrom on its own is sufficient to register the executor as an
// output source — the call ensures the id is in the dict with the Intermediate tag.
NoOpExecutor a = new("a");
NoOpExecutor b = new("b");
Workflow workflow = new WorkflowBuilder("a")
.AddEdge(a, b)
.WithIntermediateOutputFrom([b])
.Build();
workflow.OutputExecutors.Should().ContainKey("b");
workflow.OutputExecutors["b"].Should().BeEquivalentTo(new[] { OutputTag.Intermediate });
}
[Fact]
public void Test_WithOutputFrom_TracksExecutorBinding()
{
// A placeholder binding referenced via WithOutputFrom must end up bound by the time we Build.
NoOpExecutor a = new("a");
NoOpExecutor future = new("future");
Workflow workflow = new WorkflowBuilder("a")
.AddEdge(a, "future")
.WithIntermediateOutputFrom(["future"])
.BindExecutor(future)
.Build();
workflow.OutputExecutors.Should().ContainKey("future");
workflow.OutputExecutors["future"].Should().BeEquivalentTo(new[] { OutputTag.Intermediate });
}
}
@@ -824,4 +824,130 @@ public class WorkflowHostSmokeTests : AIAgentHostingExecutorTestsBase
Workflow handoffWorkflow = new HandoffWorkflowBuilder(agent).Build();
return this.Run_AsAgent_OutgoingMessagesInHistoryAsync(handoffWorkflow, runAsync);
}
// ----- Phase 5: Workflow-as-Agent intermediate forwarding -----------------
[Collection(Futures.FuturesSerialCollection.Name)]
public class IntermediateForwarding
{
private const string InterText = "progress";
private const string FinalText = "final";
private static async Task<List<AgentResponseUpdate>> RunStreamingAsync(
Workflow workflow,
bool includeWorkflowOutputsInResponse = false)
{
return await workflow
.AsAIAgent("WorkflowAgent", includeWorkflowOutputsInResponse: includeWorkflowOutputsInResponse)
.RunStreamingAsync(new ChatMessage(ChatRole.User, "hi"))
.ToListAsync();
}
[Fact]
public async Task Test_WorkflowHostAgent_IntermediateAgentResponseForwardedInStreamingAsync()
{
using Futures.FuturesScope _ = new(enabled: true);
TestReplayAgent agent = new(TestReplayAgent.ToChatMessages(InterText));
ExecutorBinding binding = agent.BindAsExecutor(new AIAgentHostOptions { EmitAgentResponseEvents = true });
Workflow workflow = new WorkflowBuilder(binding)
.WithIntermediateOutputFrom([binding])
.Build();
// Under Futures-on, AgentResponseEvent mirrors AgentResponseUpdateEvent: always
// forwarded regardless of the include flag. The intermediate tag is observable on
// the surfaced event for consumers that care to distinguish.
List<AgentResponseUpdate> updates = await RunStreamingAsync(workflow, includeWorkflowOutputsInResponse: false);
updates.Any(u => u.RawRepresentation is AgentResponseEvent are && are.IsIntermediate() && u.Text == InterText)
.Should().BeTrue("AgentResponseEvent is forwarded under Futures-on regardless of the include flag");
}
[Fact]
public async Task Test_WorkflowHostAgent_TerminalAgentResponseForwardedUnconditionallyWhenFuturesOnAsync()
{
using Futures.FuturesScope _ = new(enabled: true);
TestReplayAgent agent = new(TestReplayAgent.ToChatMessages(FinalText));
ExecutorBinding binding = agent.BindAsExecutor(new AIAgentHostOptions { EmitAgentResponseEvents = true });
Workflow workflow = new WorkflowBuilder(binding)
.WithOutputFrom(binding)
.Build();
// Even a terminal-only designation surfaces without the include flag — the gating
// asymmetry between AgentResponse and AgentResponseUpdate is gone under Futures-on.
List<AgentResponseUpdate> updates = await RunStreamingAsync(workflow, includeWorkflowOutputsInResponse: false);
updates.Any(u => u.RawRepresentation is AgentResponseEvent && u.Text == FinalText)
.Should().BeTrue("terminal AgentResponseEvent is forwarded under Futures-on regardless of the include flag");
}
[Fact]
public async Task Test_WorkflowHostAgent_TerminalAgentResponseGatedWhenFuturesOffAsync()
{
using Futures.FuturesScope _ = new(enabled: false);
static Workflow Build()
{
TestReplayAgent agent = new(TestReplayAgent.ToChatMessages(FinalText));
ExecutorBinding binding = agent.BindAsExecutor(new AIAgentHostOptions { EmitAgentResponseEvents = true });
return new WorkflowBuilder(binding).WithOutputFrom(binding).Build();
}
// Legacy semantics: AgentResponseEvent stays behind the include flag when Futures
// is off. Two fresh workflows because in-process runs aren't reentrant.
List<AgentResponseUpdate> gated = await RunStreamingAsync(Build(), includeWorkflowOutputsInResponse: false);
gated.Any(u => u.RawRepresentation is AgentResponseEvent && u.Text == FinalText)
.Should().BeFalse("terminal AgentResponseEvent stays gated under Futures-off");
List<AgentResponseUpdate> included = await RunStreamingAsync(Build(), includeWorkflowOutputsInResponse: true);
included.Any(u => u.RawRepresentation is AgentResponseEvent && u.Text == FinalText)
.Should().BeTrue("opting in via includeWorkflowOutputsInResponse surfaces it");
}
[Fact]
public async Task Test_WorkflowHostAgent_UndesignatedExecutorEmitsNoAgentResponseEventWhenFuturesOnAsync()
{
using Futures.FuturesScope _ = new(enabled: true);
TestReplayAgent agent = new(TestReplayAgent.ToChatMessages(InterText));
ExecutorBinding binding = agent.BindAsExecutor(new AIAgentHostOptions { EmitAgentResponseEvents = true });
// No designation — under Futures-on, the AgentResponse is dropped by the filter.
Workflow workflow = new WorkflowBuilder(binding).Build();
List<AgentResponseUpdate> updates = await RunStreamingAsync(workflow, includeWorkflowOutputsInResponse: true);
updates.Any(u => u.RawRepresentation is AgentResponseEvent)
.Should().BeFalse("an undesignated AIAgent executor produces no AgentResponseEvent under Futures-on");
}
[Fact]
public async Task Test_WorkflowHostAgent_UndesignatedAgentResponseSurfacesWhenFuturesOffAsync()
{
using Futures.FuturesScope _ = new(enabled: false);
TestReplayAgent agent = new(TestReplayAgent.ToChatMessages(InterText));
ExecutorBinding binding = agent.BindAsExecutor(new AIAgentHostOptions { EmitAgentResponseEvents = true });
Workflow workflow = new WorkflowBuilder(binding).Build();
List<AgentResponseUpdate> updates = await RunStreamingAsync(workflow, includeWorkflowOutputsInResponse: true);
updates.Any(u => u.RawRepresentation is AgentResponseEvent && u.Text == InterText)
.Should().BeTrue("legacy bypass still emits AgentResponseEvent regardless of designation");
}
[Fact]
public async Task Test_WorkflowHostAgent_IntermediateTagAvailableViaRawRepresentationAsync()
{
using Futures.FuturesScope _ = new(enabled: true);
TestReplayAgent agent = new(TestReplayAgent.ToChatMessages(InterText));
ExecutorBinding binding = agent.BindAsExecutor(new AIAgentHostOptions { EmitAgentResponseEvents = true });
Workflow workflow = new WorkflowBuilder(binding)
.WithIntermediateOutputFrom([binding])
.Build();
List<AgentResponseUpdate> updates = await RunStreamingAsync(workflow);
AgentResponseUpdate progress = updates.First(u => u.RawRepresentation is AgentResponseEvent && u.Text == InterText);
AgentResponseEvent raw = (AgentResponseEvent)progress.RawRepresentation!;
raw.IsIntermediate().Should().BeTrue();
raw.Tags.Should().BeEquivalentTo(new[] { OutputTag.Intermediate });
}
}
}
+3
View File
@@ -44,6 +44,9 @@ GEMINI_MODEL=""
# Ollama
OLLAMA_ENDPOINT=""
OLLAMA_MODEL=""
# Mistral AI
MISTRAL_API_KEY=""
MISTRAL_EMBEDDING_MODEL=""
# Observability (instrumentation is enabled by default; set "ENABLE_INSTRUMENTATION" to "false" to opt out)
ENABLE_SENSITIVE_DATA=true
OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:4317/"
+1
View File
@@ -37,6 +37,7 @@ Status is grouped into these buckets:
| `agent-framework-hyperlight` | `python/packages/hyperlight` | `beta` |
| `agent-framework-lab` | `python/packages/lab` | `beta` |
| `agent-framework-mem0` | `python/packages/mem0` | `beta` |
| `agent-framework-mistral` | `python/packages/mistral` | `alpha` |
| `agent-framework-monty` | `python/packages/monty` | `alpha` |
| `agent-framework-ollama` | `python/packages/ollama` | `beta` |
| `agent-framework-openai` | `python/packages/openai` | `released` |
@@ -2,6 +2,7 @@
import base64
import logging
import uuid
from asyncio import CancelledError
from collections.abc import Mapping
from functools import partial
@@ -181,9 +182,18 @@ class A2AExecutor(AgentExecutor):
"""Run the agent in streaming mode and publish updates to the task updater."""
response_stream = self._agent.run(query, session=session, stream=True, **self._run_kwargs)
streamed_artifact_ids: set[str] = set()
# Generate a stable artifact ID for the entire stream so all chunks share the same ID.
# This ensures clients can coalesce streaming tokens into a single artifact/message
# per the A2A spec (TaskArtifactUpdateEvent with append=True on same artifactId).
default_artifact_id = str(uuid.uuid4())
await (
response_stream.with_transform_hook(
partial(self.handle_events, updater=updater, streamed_artifact_ids=streamed_artifact_ids)
partial(
self.handle_events,
updater=updater,
streamed_artifact_ids=streamed_artifact_ids,
default_artifact_id=default_artifact_id,
)
)
).get_final_response()
@@ -199,7 +209,11 @@ class A2AExecutor(AgentExecutor):
await self.handle_events(message, updater)
async def handle_events(
self, item: Message | AgentResponseUpdate, updater: TaskUpdater, streamed_artifact_ids: set[str] | None = None
self,
item: Message | AgentResponseUpdate,
updater: TaskUpdater,
streamed_artifact_ids: set[str] | None = None,
default_artifact_id: str | None = None,
) -> None:
"""Convert agent response items (Messages or Updates) to A2A protocol events.
@@ -213,7 +227,10 @@ class A2AExecutor(AgentExecutor):
item: The agent response item (Message or AgentResponseUpdate) to process.
updater: The task updater to publish events to.
streamed_artifact_ids: A set of artifact IDs that have already been streamed.
Used to prevent duplicate updates for the same artifact.
Used to track which artifacts need append=True on subsequent chunks.
default_artifact_id: A stable artifact ID to use when the item does not provide one.
This ensures all streaming chunks for a single response share the same artifact ID,
allowing clients to coalesce them into a single message.
Example:
.. code-block:: python
@@ -224,6 +241,7 @@ class A2AExecutor(AgentExecutor):
item: Message | AgentResponseUpdate,
updater: TaskUpdater,
streamed_artifact_ids: set[str] | None = None,
default_artifact_id: str | None = None,
) -> None:
# Custom logic to transform item contents
if item.role == "assistant" and item.contents:
@@ -260,19 +278,22 @@ class A2AExecutor(AgentExecutor):
if parts:
if isinstance(item, AgentResponseUpdate):
# Resolve artifact ID: use item's message_id if available, otherwise fall back
# to the stable default_artifact_id so all streaming chunks share the same ID.
artifact_id = item.message_id or default_artifact_id
# For streaming updates, we send TaskArtifactUpdateEvent via add_artifact
await updater.add_artifact(
parts=parts,
artifact_id=item.message_id,
artifact_id=artifact_id,
metadata=metadata,
append=(
True
if streamed_artifact_ids is not None and item.message_id in (streamed_artifact_ids or set())
if streamed_artifact_ids is not None and artifact_id in streamed_artifact_ids
else None
),
)
if item.message_id and streamed_artifact_ids is not None:
streamed_artifact_ids.add(item.message_id)
if artifact_id and streamed_artifact_ids is not None:
streamed_artifact_ids.add(artifact_id)
else:
# For final messages, we send TaskStatusUpdateEvent with 'working' state
await updater.update_status(
@@ -176,6 +176,7 @@ class A2AAgent(AgentTelemetryLayer, BaseAgent):
http_client: httpx.AsyncClient | None = None,
auth_interceptor: AuthInterceptor | None = None,
timeout: float | httpx.Timeout | None = None,
supported_protocol_bindings: list[Literal["JSONRPC", "GRPC", "HTTP+JSON"] | str] | None = None,
**kwargs: Any,
) -> None:
"""Initialize the A2AAgent.
@@ -193,6 +194,9 @@ class A2AAgent(AgentTelemetryLayer, BaseAgent):
timeout: Request timeout configuration. Can be a float (applied to all timeout components),
httpx.Timeout object (for full control), or None (uses 10.0s connect, 60.0s read,
10.0s write, 5.0s pool - optimized for A2A operations).
supported_protocol_bindings: List of protocol bindings to use for transport negotiation.
Known values: "JSONRPC", "GRPC", "HTTP+JSON". Defaults to ["JSONRPC"].
The A2A spec treats this as an open-form string, so custom bindings are also accepted.
kwargs: any additional properties, passed to BaseAgent.
"""
# Default name/description from agent_card when not explicitly provided
@@ -205,6 +209,7 @@ class A2AAgent(AgentTelemetryLayer, BaseAgent):
super().__init__(id=id, name=name, description=description, **kwargs)
self._http_client: httpx.AsyncClient | None = http_client
self._timeout_config = self._create_timeout_config(timeout)
bindings = supported_protocol_bindings if supported_protocol_bindings is not None else ["JSONRPC"]
if client is not None:
self.client = client
self._non_streaming_client: Client | None = None
@@ -214,7 +219,7 @@ class A2AAgent(AgentTelemetryLayer, BaseAgent):
if url is None:
raise ValueError("Either agent_card or url must be provided")
# Create minimal agent card from URL
agent_card = minimal_agent_card(url, ["JSONRPC"])
agent_card = minimal_agent_card(url, bindings)
# Create or use provided httpx client
if http_client is None:
@@ -229,13 +234,13 @@ class A2AAgent(AgentTelemetryLayer, BaseAgent):
streaming_config = ClientConfig(
httpx_client=http_client,
streaming=True,
supported_protocol_bindings=["JSONRPC"],
supported_protocol_bindings=bindings,
)
# Create non-streaming client (single request/response for stream=False)
non_streaming_config = ClientConfig(
httpx_client=http_client,
streaming=False,
supported_protocol_bindings=["JSONRPC"],
supported_protocol_bindings=bindings,
)
streaming_factory = ClientFactory(streaming_config)
non_streaming_factory = ClientFactory(non_streaming_config)
@@ -256,7 +261,7 @@ class A2AAgent(AgentTelemetryLayer, BaseAgent):
"Provide a 'url' argument or ensure 'agent_card.supported_interfaces' "
"contains at least one interface with a URL."
) from transport_error
fallback_card = minimal_agent_card(fallback_url, ["JSONRPC"])
fallback_card = minimal_agent_card(fallback_url, bindings)
try:
self.client = streaming_factory.create(fallback_card, interceptors=interceptors) # type: ignore
self._non_streaming_client = non_streaming_factory.create(
@@ -487,6 +492,7 @@ class A2AAgent(AgentTelemetryLayer, BaseAgent):
contents=contents,
role="assistant" if msg.role == A2ARole.ROLE_AGENT else "user",
response_id=msg.message_id or str(uuid.uuid4()),
message_id=msg.message_id,
additional_properties={"a2a_metadata": metadata} if metadata else None,
raw_representation=msg,
)
@@ -727,6 +733,7 @@ class A2AAgent(AgentTelemetryLayer, BaseAgent):
contents=contents,
role="assistant" if message.role == A2ARole.ROLE_AGENT else "user",
response_id=update_event.task_id,
message_id=message.message_id,
additional_properties={"a2a_metadata": merged_metadata} if merged_metadata else None,
raw_representation=update_event,
)
+93 -3
View File
@@ -420,6 +420,7 @@ async def test_run_streaming_with_message_response(a2a_agent: A2AAgent, mock_a2a
assert content.text == "Streaming response from agent!"
assert updates[0].response_id == "msg-stream-123"
assert updates[0].message_id == "msg-stream-123"
assert mock_a2a_client.call_count == 1
@@ -703,7 +704,94 @@ def test_a2a_agent_initialization_with_timeout_parameter() -> None:
assert isinstance(timeout_arg, httpx.Timeout)
# region Continuation Token Tests
def test_a2a_agent_initialization_with_supported_protocol_bindings() -> None:
"""Test A2AAgent initialization with custom supported_protocol_bindings."""
with (
patch("agent_framework_a2a._agent.httpx.AsyncClient") as mock_async_client,
patch("agent_framework_a2a._agent.ClientConfig") as mock_config,
patch("agent_framework_a2a._agent.ClientFactory") as mock_factory,
):
mock_async_client.return_value = MagicMock()
mock_client_instance = MagicMock()
mock_factory.return_value.create.return_value = mock_client_instance
A2AAgent(
name="Test Agent",
url="https://test-agent.example.com",
supported_protocol_bindings=["GRPC", "JSONRPC"],
)
# Verify ClientConfig was called with our custom bindings for both streaming and non-streaming
assert mock_config.call_count == 2
for call in mock_config.call_args_list:
assert call.kwargs["supported_protocol_bindings"] == ["GRPC", "JSONRPC"]
def test_a2a_agent_initialization_defaults_to_jsonrpc() -> None:
"""Test A2AAgent defaults to JSONRPC when supported_protocol_bindings is not provided."""
with (
patch("agent_framework_a2a._agent.httpx.AsyncClient") as mock_async_client,
patch("agent_framework_a2a._agent.ClientConfig") as mock_config,
patch("agent_framework_a2a._agent.ClientFactory") as mock_factory,
):
mock_async_client.return_value = MagicMock()
mock_client_instance = MagicMock()
mock_factory.return_value.create.return_value = mock_client_instance
A2AAgent(name="Test Agent", url="https://test-agent.example.com")
# Verify ClientConfig was called with default JSONRPC bindings
assert mock_config.call_count == 2
for call in mock_config.call_args_list:
assert call.kwargs["supported_protocol_bindings"] == ["JSONRPC"]
def test_a2a_agent_initialization_empty_list_preserved() -> None:
"""Test that an explicit empty list is preserved and not replaced with defaults."""
with (
patch("agent_framework_a2a._agent.httpx.AsyncClient") as mock_async_client,
patch("agent_framework_a2a._agent.ClientConfig") as mock_config,
patch("agent_framework_a2a._agent.ClientFactory") as mock_factory,
):
mock_async_client.return_value = MagicMock()
mock_client_instance = MagicMock()
mock_factory.return_value.create.return_value = mock_client_instance
A2AAgent(
name="Test Agent",
url="https://test-agent.example.com",
supported_protocol_bindings=[],
)
# Verify ClientConfig was called with the explicit empty list, not the default
assert mock_config.call_count == 2
for call in mock_config.call_args_list:
assert call.kwargs["supported_protocol_bindings"] == []
def test_a2a_agent_fallback_uses_custom_bindings() -> None:
"""Test that transport fallback path uses custom bindings."""
mock_agent_card = MagicMock()
mock_agent_card.supported_interfaces = [MagicMock(url="https://fallback.example.com")]
mock_factory = MagicMock()
# First create() call fails (primary streaming), then fallback calls succeed
primary_error = Exception("no compatible transports found")
mock_factory.create.side_effect = [primary_error, MagicMock(), MagicMock()]
with (
patch("agent_framework_a2a._agent.ClientFactory", return_value=mock_factory),
patch("agent_framework_a2a._agent.minimal_agent_card") as mock_minimal_card,
patch("agent_framework_a2a._agent.httpx.AsyncClient"),
):
A2AAgent(
name="test-agent",
agent_card=mock_agent_card,
supported_protocol_bindings=["GRPC", "HTTP+JSON"],
)
# Verify minimal_agent_card was called with the custom bindings
mock_minimal_card.assert_called_once_with("https://fallback.example.com", ["GRPC", "HTTP+JSON"])
async def test_working_task_emits_continuation_token(a2a_agent: A2AAgent, mock_a2a_client: MockA2AClient) -> None:
@@ -1335,7 +1423,7 @@ async def test_streaming_status_update_event_yields_content(
status=TaskStatus(
state=TaskState.TASK_STATE_COMPLETED,
message=A2AMessage(
message_id=str(uuid4()),
message_id="msg-status-done",
role=A2ARole.ROLE_AGENT,
parts=[Part(text="Done")],
),
@@ -1350,6 +1438,7 @@ async def test_streaming_status_update_event_yields_content(
assert len(updates) == 1
assert updates[0].text == "Done"
assert updates[0].role == "assistant"
assert updates[0].message_id == "msg-status-done"
assert updates[0].raw_representation == update_event
@@ -1362,7 +1451,7 @@ async def test_streaming_input_required_emits_content(a2a_agent: A2AAgent, mock_
status=TaskStatus(
state=TaskState.TASK_STATE_INPUT_REQUIRED,
message=A2AMessage(
message_id=str(uuid4()),
message_id="msg-input-req",
role=A2ARole.ROLE_AGENT,
parts=[Part(text="What is your name?")],
),
@@ -1376,6 +1465,7 @@ async def test_streaming_input_required_emits_content(a2a_agent: A2AAgent, mock_
assert len(updates) == 1
assert updates[0].text == "What is your name?"
assert updates[0].message_id == "msg-input-req"
@mark.asyncio
@@ -365,6 +365,14 @@ def _start_function_app(sample_path: Path, port: int) -> subprocess.Popen[Any]:
# use the task hub name to separate orchestration state.
env["TASKHUB_NAME"] = f"test{uuid.uuid4().hex[:8]}"
# The Azure Functions Python worker's dependency isolation mechanism crashes
# on Python 3.13 with a SIGSEGV in the protobuf C extension (google._upb).
# Disabling isolation lets the worker load dependencies from the app's own
# environment, which avoids the crash.
# See: https://github.com/Azure/azure-functions-python-worker/issues/1797
if sys.version_info >= (3, 13):
env.setdefault("PYTHON_ISOLATE_WORKER_DEPENDENCIES", "0")
# On Windows, use CREATE_NEW_PROCESS_GROUP to allow proper termination
# shell=True only on Windows to handle PATH resolution
if sys.platform == "win32":
@@ -375,8 +383,15 @@ def _start_function_app(sample_path: Path, port: int) -> subprocess.Popen[Any]:
shell=True,
env=env,
)
# On Unix, don't use shell=True to avoid shell wrapper issues
return subprocess.Popen(["func", "start", "--port", str(port)], cwd=str(sample_path), env=env)
# On Unix, use start_new_session=True to isolate the process group from the
# pytest-xdist worker. Without this, signals (e.g. from test-timeout) can
# propagate to the func host and vice-versa, potentially killing the worker.
return subprocess.Popen(
["func", "start", "--port", str(port)],
cwd=str(sample_path),
env=env,
start_new_session=True,
)
def _wait_for_function_app_ready(func_process: subprocess.Popen[Any], port: int, max_wait: int = 60) -> None:
@@ -533,18 +548,33 @@ def function_app_for_test(request: pytest.FixtureRequest) -> Iterator[dict[str,
_load_and_validate_env(sample_path)
max_attempts = 3
# The overall budget MUST be shorter than the pytest-timeout value
# (--timeout=120 by default) so that the fixture finishes cleanly instead
# of being killed by os._exit() which crashes the xdist worker.
overall_budget = 100 # seconds leaves headroom below the 120 s test timeout
last_error: Exception | None = None
func_process: subprocess.Popen[Any] | None = None
base_url = ""
port = 0
overall_start = time.monotonic()
attempts_made = 0
for _ in range(max_attempts):
remaining = overall_budget - (time.monotonic() - overall_start)
if remaining < 10:
# Not enough time for another attempt; bail out.
break
attempts_made += 1
port = _find_available_port()
base_url = _build_base_url(port)
func_process = _start_function_app(sample_path, port)
try:
_wait_for_function_app_ready(func_process, port)
# Cap each attempt's wait to the remaining budget minus a small
# buffer for cleanup.
per_attempt_wait = min(60, int(remaining) - 5)
_wait_for_function_app_ready(func_process, port, max_wait=max(per_attempt_wait, 10))
last_error = None
break
except FunctionAppStartupError as exc:
@@ -553,7 +583,8 @@ def function_app_for_test(request: pytest.FixtureRequest) -> Iterator[dict[str,
func_process = None
if func_process is None:
error_message = f"Function app failed to start after {max_attempts} attempt(s)."
elapsed = int(time.monotonic() - overall_start)
error_message = f"Function app failed to start after {attempts_made} attempt(s) ({elapsed}s elapsed)."
if last_error is not None:
error_message += f" Last error: {last_error}"
pytest.fail(error_message)
+8
View File
@@ -76,6 +76,14 @@ agent_framework/
- **`SkillScriptRunner`** - Protocol for file-based script execution. Any callable matching `(skill, script, args) -> Any` satisfies it. Code-defined scripts do not use a runner.
- **`SkillsProvider`** - Context provider (extends `ContextProvider`) that discovers file-based skills from `SKILL.md` files and/or accepts code-defined `Skill` instances. Follows progressive disclosure: advertise → load → read resources / run scripts.
### File Access Harness (`_harness/_file_access.py`)
- **`AgentFileStore`** - Abstract async store backing the file-access harness. Implementations expose `write_file`, `read_file`, `delete_file`, `list_files`, `file_exists`, `search_files`, and `create_directory` over forward-slash relative paths.
- **`InMemoryAgentFileStore`** - Dict-backed store suitable for tests and lightweight scenarios.
- **`FileSystemAgentFileStore`** - Disk-backed store rooted under a configurable directory. Enforces relative-path normalization, root containment, and rejects symlink/reparse-point segments to prevent escape.
- **`FileSearchResult`** / **`FileSearchMatch`** - `SerializationMixin` DTOs returned by `search_files`, carrying the matching file name, a context snippet, and the matching lines with 1-based line numbers.
- **`FileAccessProvider`** - `ContextProvider` that adds shared file-access tools (`file_access_save_file`, `file_access_read_file`, `file_access_delete_file`, `file_access_list_files`, `file_access_search_files`) plus default usage instructions to each invocation. Unlike `MemoryContextProvider`, the store is intentionally shared across sessions and agents.
### Workflows (`_workflows/`)
- **`Workflow`** - Graph-based workflow definition
@@ -90,6 +90,16 @@ from ._harness._background_agents import (
BackgroundTaskInfo,
BackgroundTaskStatus,
)
from ._harness._file_access import (
DEFAULT_FILE_ACCESS_INSTRUCTIONS,
DEFAULT_FILE_ACCESS_SOURCE_ID,
AgentFileStore,
FileAccessProvider,
FileSearchMatch,
FileSearchResult,
FileSystemAgentFileStore,
InMemoryAgentFileStore,
)
from ._harness._memory import (
DEFAULT_MEMORY_SOURCE_ID,
MemoryContextProvider,
@@ -309,6 +319,8 @@ __all__ = [
"APP_INFO",
"COMPACTION_STATE_KEY",
"DEFAULT_BACKGROUND_AGENTS_SOURCE_ID",
"DEFAULT_FILE_ACCESS_INSTRUCTIONS",
"DEFAULT_FILE_ACCESS_SOURCE_ID",
"DEFAULT_HARNESS_INSTRUCTIONS",
"DEFAULT_MAX_ITERATIONS",
"DEFAULT_MEMORY_SOURCE_ID",
@@ -334,6 +346,7 @@ __all__ = [
"AgentExecutor",
"AgentExecutorRequest",
"AgentExecutorResponse",
"AgentFileStore",
"AgentFrameworkException",
"AgentMiddleware",
"AgentMiddlewareLayer",
@@ -393,11 +406,15 @@ __all__ = [
"ExperimentalFeature",
"FanInEdgeGroup",
"FanOutEdgeGroup",
"FileAccessProvider",
"FileCheckpointStorage",
"FileHistoryProvider",
"FileSearchMatch",
"FileSearchResult",
"FileSkill",
"FileSkillScript",
"FileSkillsSource",
"FileSystemAgentFileStore",
"FilteringSkillsSource",
"FinalT",
"FinishReason",
@@ -414,6 +431,7 @@ __all__ = [
"GeneratedEmbeddings",
"GraphConnectivityError",
"HistoryProvider",
"InMemoryAgentFileStore",
"InMemoryCheckpointStorage",
"InMemoryHistoryProvider",
"InMemorySkillsSource",
File diff suppressed because it is too large Load Diff
+72 -20
View File
@@ -10,7 +10,7 @@ import logging
import re
import sys
from abc import abstractmethod
from collections.abc import Callable, Collection, Coroutine, Sequence
from collections.abc import Callable, Collection, Coroutine, Mapping, Sequence
from contextlib import AsyncExitStack, _AsyncGeneratorContextManager # type: ignore
from datetime import timedelta
from functools import partial
@@ -142,6 +142,13 @@ def _inject_otel_into_mcp_meta(meta: dict[str, Any] | None = None) -> dict[str,
return meta
def _url_origin(url: Any) -> tuple[str, str, int | None]:
port = url.port
if port is None:
port = 443 if url.scheme == "https" else 80 if url.scheme == "http" else None
return (url.scheme, url.host or "", port)
def streamable_http_client(*args: Any, **kwargs: Any) -> _AsyncGeneratorContextManager[Any, None]:
"""Lazily import the MCP streamable HTTP transport."""
try:
@@ -255,6 +262,7 @@ class MCPTool:
self._exit_stack = AsyncExitStack()
self._lifecycle_lock = asyncio.Lock()
self._lifecycle_request_lock = asyncio.Lock()
self._function_load_lock = asyncio.Lock()
self._lifecycle_queue: asyncio.Queue[tuple[str, bool, bool, asyncio.Future[None]]] | None = None
self._lifecycle_owner_task: asyncio.Task[None] | None = None
self.session = session
@@ -655,6 +663,11 @@ class MCPTool:
raise
except asyncio.CancelledError:
logger.warning("Could not cleanly close MCP exit stack because the lifecycle owner task was cancelled.")
except Exception as e:
if type(e).__name__ == "ExceptionGroup":
logger.warning("Could not cleanly close MCP exit stack due to cleanup error group. Error: %s", e)
else:
raise
async def _close_and_check_cancelled(self, ex: BaseException) -> bool:
"""Close the exit stack and return True if *ex* is a genuine task cancellation.
@@ -1018,6 +1031,10 @@ class MCPTool:
Raises:
ToolExecutionException: If the MCP server is not connected.
"""
async with self._function_load_lock:
await self._load_prompts_locked()
async def _load_prompts_locked(self) -> None:
from anyio import ClosedResourceError
from mcp import types
@@ -1100,6 +1117,10 @@ class MCPTool:
Raises:
ToolExecutionException: If the MCP server is not connected.
"""
async with self._function_load_lock:
await self._load_tools_locked()
async def _load_tools_locked(self) -> None:
from anyio import ClosedResourceError
from mcp import types
@@ -1109,7 +1130,7 @@ class MCPTool:
# Track existing function names to prevent duplicates
existing_names = {func.name for func in self._functions}
self._tool_call_meta_by_name.clear()
tool_call_meta_by_name: dict[str, dict[str, Any]] = {}
params: types.PaginatedRequestParams | None = None
while True:
@@ -1145,7 +1166,7 @@ class MCPTool:
for tool in tool_list.tools:
if tool.meta is not None:
self._tool_call_meta_by_name[tool.name] = dict(tool.meta)
tool_call_meta_by_name[tool.name] = dict(tool.meta)
normalized_name = _normalize_mcp_name(tool.name)
local_name = _build_prefixed_mcp_name(normalized_name, self.tool_name_prefix)
@@ -1194,6 +1215,8 @@ class MCPTool:
break
params = types.PaginatedRequestParams(cursor=tool_list.nextCursor)
self._tool_call_meta_by_name = tool_call_meta_by_name
async def _close_on_owner(self) -> None:
# Cancel any pending reload tasks before tearing down the session.
tasks = list(self._pending_reload_tasks)
@@ -1276,7 +1299,11 @@ class MCPTool:
tool_name: The name of the tool to call.
Keyword Args:
kwargs: Arguments to pass to the tool.
_meta: Optional ``dict[str, Any]`` of MCP request metadata. This reserved key is passed as the
``meta`` parameter of the underlying ``session.call_tool`` call rather than as a tool argument.
User-supplied keys override metadata from ``tools/list``; OpenTelemetry propagation fills in
non-conflicting keys.
kwargs: Remaining arguments to pass to the tool.
Returns:
A list of Content items representing the tool output. The default
@@ -1294,6 +1321,19 @@ class MCPTool:
raise ToolExecutionException(
"Tools are not loaded for this server, please set load_tools=True in the constructor."
)
raw_user_meta: object | None = kwargs.get("_meta")
user_meta: dict[str, Any] | None = None
if raw_user_meta is not None and not isinstance(raw_user_meta, dict):
raise ToolExecutionException("MCP tool metadata provided via _meta must be a dict.")
if isinstance(raw_user_meta, dict):
raw_user_meta_dict = cast(Mapping[object, object], raw_user_meta)
user_meta = {}
for key, value in raw_user_meta_dict.items():
if not isinstance(key, str):
raise ToolExecutionException("MCP tool metadata provided via _meta must use string keys.")
user_meta[key] = value
# Filter out framework kwargs that cannot be serialized by the MCP SDK.
# These are internal objects passed through the function invocation pipeline
# that should not be forwarded to external MCP servers.
@@ -1313,12 +1353,16 @@ class MCPTool:
"conversation_id",
"options",
"response_format",
"_meta",
}
}
# Some MCP proxies require their tools/list metadata to be echoed on tools/call.
tool_meta = self._tool_call_meta_by_name.get(tool_name)
meta = _inject_otel_into_mcp_meta(dict(tool_meta) if tool_meta is not None else None)
request_meta = dict(tool_meta) if tool_meta is not None else None
if user_meta is not None:
request_meta = {**(request_meta or {}), **user_meta}
meta = _inject_otel_into_mcp_meta(request_meta)
parser = self.parse_tool_results or self._parse_tool_result_from_mcp
# Try the operation, reconnecting once if the connection is closed
@@ -1336,28 +1380,33 @@ class MCPTool:
return parser(result)
except ToolExecutionException:
raise
except ClosedResourceError as cl_ex:
except (ClosedResourceError, McpError) as call_ex:
is_session_terminated = (
isinstance(call_ex, McpError) and "session terminated" in call_ex.error.message.lower()
)
is_connection_lost = isinstance(call_ex, ClosedResourceError) or is_session_terminated
if not is_connection_lost:
error_message = call_ex.error.message if isinstance(call_ex, McpError) else str(call_ex)
raise ToolExecutionException(error_message, inner_exception=call_ex) from call_ex
if attempt == 0:
# First attempt failed, try reconnecting
logger.info("MCP connection closed unexpectedly. Reconnecting...")
# First attempt failed, try reconnecting.
logger.info("MCP connection closed or terminated unexpectedly. Reconnecting...")
try:
await self.connect(reset=True)
continue # Retry the operation
continue
except Exception as reconn_ex:
raise ToolExecutionException(
"Failed to reconnect to MCP server.",
inner_exception=reconn_ex,
) from reconn_ex
else:
# Second attempt also failed, give up
logger.error(f"MCP connection closed unexpectedly after reconnection: {cl_ex}")
raise ToolExecutionException(
f"Failed to call tool '{tool_name}' - connection lost.",
inner_exception=cl_ex,
) from cl_ex
except McpError as mcp_exc:
error_message = mcp_exc.error.message
raise ToolExecutionException(error_message, inner_exception=mcp_exc) from mcp_exc
# Second attempt also failed, give up.
logger.error("MCP connection closed unexpectedly after reconnection: %s", call_ex)
raise ToolExecutionException(
f"Failed to call tool '{tool_name}' - connection lost.",
inner_exception=call_ex,
) from call_ex
except Exception as ex:
raise ToolExecutionException(f"Failed to call tool '{tool_name}'.", inner_exception=ex) from ex
raise ToolExecutionException(f"Failed to call tool '{tool_name}' after retries.")
@@ -1718,10 +1767,11 @@ class MCPStreamableHTTPTool(MCPTool):
Returns:
An async context manager for the streamable HTTP client transport.
"""
from httpx import AsyncClient, Request, Timeout
from httpx import URL, AsyncClient, Request, Timeout
http_client = self._httpx_client
if self._header_provider is not None:
target_origin = _url_origin(URL(self.url))
if http_client is None:
http_client = AsyncClient(
follow_redirects=True,
@@ -1732,6 +1782,8 @@ class MCPStreamableHTTPTool(MCPTool):
if not hasattr(self, "_inject_headers_hook"):
async def _inject_headers(request: Request) -> None: # noqa: RUF029
if _url_origin(request.url) != target_origin:
return
headers = _mcp_call_headers.get({})
for key, value in headers.items():
request.headers[key] = value
@@ -1973,11 +1973,97 @@ def _coalesce_text_content(contents: list[Content], type_str: Literal["text", "t
contents.extend(coalesced_contents)
def _content_items_text(items: Any) -> str | None:
"""Return concatenated text when a content item list only contains text."""
if not isinstance(items, list):
return None
text_parts: list[str] = []
content_items = cast(list[object], items)
for item in content_items:
if not isinstance(item, Content) or item.type != "text":
return None
text_parts.append(item.text or "")
return "".join(text_parts)
def _merge_content_item_lists(existing: Any, incoming: Any) -> Any:
"""Merge streamed nested content lists, replacing deltas with a later full value when present."""
if incoming is None:
return existing
if existing is None:
return deepcopy(incoming)
existing_text = _content_items_text(existing)
incoming_text = _content_items_text(incoming)
if existing_text is not None and incoming_text is not None:
if incoming_text.startswith(existing_text):
return deepcopy(incoming)
if existing_text.startswith(incoming_text):
return existing
existing_items = cast(list[Content], existing)
merged = deepcopy(existing_items[0])
merged.text = existing_text + incoming_text
return [merged]
if isinstance(existing, list) and isinstance(incoming, list):
existing_list = cast(list[object], existing)
incoming_list = cast(list[object], incoming)
return [*existing_list, *deepcopy(incoming_list)]
return deepcopy(incoming)
def _merge_code_interpreter_content(existing: Content, incoming: Content) -> None:
"""Merge two code interpreter content items for the same logical call."""
existing.inputs = _merge_content_item_lists(existing.inputs, incoming.inputs)
existing.outputs = _merge_content_item_lists(existing.outputs, incoming.outputs)
existing.annotations = _combine_annotations(existing.annotations, incoming.annotations)
existing.additional_properties = {**existing.additional_properties, **incoming.additional_properties}
existing.raw_representation = _combine_raw_representations(existing.raw_representation, incoming.raw_representation)
def _code_interpreter_key(content: Content) -> tuple[str, str] | None:
"""Return the aggregation key for code interpreter call/result content."""
if content.type not in {"code_interpreter_tool_call", "code_interpreter_tool_result"}:
return None
call_id = content.call_id or content.additional_properties.get("item_id")
if not isinstance(call_id, str) or not call_id:
return None
return content.type, call_id
def _coalesce_code_interpreter_content(contents: list[Content]) -> None:
"""Coalesce streaming code interpreter chunks by call id."""
if not contents:
return
coalesced_contents: list[Content] = []
seen: dict[tuple[str, str], Content] = {}
for content in contents:
key = _code_interpreter_key(content)
if key is None:
coalesced_contents.append(content)
continue
existing = seen.get(key)
if existing is None:
copied = deepcopy(content)
seen[key] = copied
coalesced_contents.append(copied)
continue
_merge_code_interpreter_content(existing, content)
contents.clear()
contents.extend(coalesced_contents)
def _finalize_response(response: ChatResponse | AgentResponse) -> None:
"""Finalizes the response by performing any necessary post-processing."""
for msg in response.messages:
_coalesce_text_content(msg.contents, "text")
_coalesce_text_content(msg.contents, "text_reasoning")
_coalesce_code_interpreter_content(msg.contents)
# region ContinuationToken
@@ -1332,6 +1332,22 @@ class ChatTelemetryLayer(Generic[OptionsCoT]):
self.duration_histogram = _get_duration_histogram()
self.otel_provider_name = otel_provider_name or getattr(self, "OTEL_PROVIDER_NAME", "unknown")
@staticmethod
def _backfill_request_model(span: trace.Span, attributes: dict[str, Any]) -> None:
"""Backfill REQUEST_MODEL and the span name from RESPONSE_MODEL when unknown.
Chat-completion spans use REQUEST_MODEL as part of the span name. If the
request model was not known at span creation time (e.g. the client could
not resolve it before sending the request), update both the attribute and
the span name to the actual model returned in the response. Mutates
``attributes`` in place.
"""
response_model = attributes.get(OtelAttr.RESPONSE_MODEL)
if response_model and attributes.get(OtelAttr.REQUEST_MODEL, "unknown") == "unknown":
attributes[OtelAttr.REQUEST_MODEL] = response_model
operation = attributes.get(OtelAttr.OPERATION, "operation")
span.update_name(f"{operation} {response_model}")
@overload
def get_response(
self,
@@ -1480,6 +1496,7 @@ class ChatTelemetryLayer(Generic[OptionsCoT]):
response: ChatResponse[Any] = await result_stream.get_final_response()
duration = duration_state.get("duration")
response_attributes = _get_response_attributes(attributes, response)
self._backfill_request_model(span, response_attributes)
_capture_response(
span=span,
attributes=response_attributes,
@@ -1549,6 +1566,7 @@ class ChatTelemetryLayer(Generic[OptionsCoT]):
raise
duration = perf_counter() - start_time_stamp
response_attributes = _get_response_attributes(attributes, response)
self._backfill_request_model(span, response_attributes)
_capture_response(
span=span,
attributes=response_attributes,
@@ -0,0 +1,713 @@
# Copyright (c) Microsoft. All rights reserved.
from __future__ import annotations
import json
import re
import time
from pathlib import Path
import pytest
from agent_framework import (
Agent,
AgentFileStore,
AgentSession,
ExperimentalFeature,
FileAccessProvider,
FileSearchMatch,
FileSearchResult,
FileSystemAgentFileStore,
InMemoryAgentFileStore,
Message,
SupportsChatGetResponse,
)
from agent_framework._harness import _file_access as _file_access_module
from agent_framework._harness._file_access import (
DEFAULT_FILE_ACCESS_INSTRUCTIONS,
DEFAULT_FILE_ACCESS_SOURCE_ID,
_matches_glob,
_normalize_relative_path,
_run_search_with_timeout,
)
def _tool_by_name(tools: list[object], name: str) -> object:
"""Return the tool with the requested name from a prepared tool list."""
for tool in tools:
if getattr(tool, "name", None) == name:
return tool
raise AssertionError(f"Tool {name!r} was not found.")
def test_normalize_relative_path_collapses_and_validates() -> None:
"""The path normalizer should accept relative forward/backslash paths and reject unsafe ones."""
assert _normalize_relative_path("foo/bar.txt") == "foo/bar.txt"
assert _normalize_relative_path("foo\\bar.txt") == "foo/bar.txt"
assert _normalize_relative_path("foo//bar.txt") == "foo/bar.txt"
assert _normalize_relative_path(" foo/bar.txt ") == "foo/bar.txt"
assert _normalize_relative_path("", is_directory=True) == ""
assert _normalize_relative_path(" ", is_directory=True) == ""
assert _normalize_relative_path("sub/", is_directory=True) == "sub"
assert _normalize_relative_path("sub\\", is_directory=True) == "sub"
with pytest.raises(ValueError, match="must not be empty"):
_normalize_relative_path("")
with pytest.raises(ValueError, match="must not be empty"):
_normalize_relative_path(" ")
with pytest.raises(ValueError, match="must not end with a path separator"):
_normalize_relative_path("foo/")
with pytest.raises(ValueError, match="must not end with a path separator"):
_normalize_relative_path("foo\\")
with pytest.raises(ValueError, match="'..' segments"):
_normalize_relative_path("foo/../bar.txt")
with pytest.raises(ValueError, match="'..' segments"):
_normalize_relative_path("./bar.txt")
with pytest.raises(ValueError, match="must be relative"):
_normalize_relative_path("C:/abs/path")
with pytest.raises(ValueError, match="must be relative"):
_normalize_relative_path("\\rooted")
with pytest.raises(ValueError, match="must be relative"):
_normalize_relative_path("/foo/bar.txt")
def test_matches_glob_is_case_insensitive_and_optional() -> None:
"""The glob matcher should be case-insensitive and treat missing patterns as match-all."""
assert _matches_glob("notes.MD", "*.md")
assert _matches_glob("research_one.txt", "research*")
assert not _matches_glob("plan.txt", "*.md")
assert _matches_glob("anything", None)
assert _matches_glob("anything", "")
assert _matches_glob("anything", " ")
def test_file_search_match_round_trips() -> None:
"""File search match values should serialize and validate cleanly."""
raw_match = {"line_number": 3, "line": "error: boom"}
match = FileSearchMatch.from_dict(raw_match)
assert match == FileSearchMatch(line_number=3, line="error: boom")
assert match.to_dict() == raw_match
assert "FileSearchMatch(" in repr(match)
with pytest.raises(ValueError, match="positive integer"):
FileSearchMatch(line_number=0, line="oops")
with pytest.raises(ValueError, match="must be an integer"):
FileSearchMatch.from_dict({"line_number": "1", "line": "oops"})
with pytest.raises(ValueError, match="must be a string"):
FileSearchMatch.from_dict({"line_number": 1, "line": 42})
def test_file_search_result_round_trips() -> None:
"""File search result values should serialize the matching-line list correctly."""
raw_result = {
"file_name": "notes.md",
"snippet": "hello error world",
"matching_lines": [{"line_number": 2, "line": "error one"}],
}
result = FileSearchResult.from_dict(raw_result)
assert result.file_name == "notes.md"
assert result.snippet == "hello error world"
assert result.matching_lines == [FileSearchMatch(line_number=2, line="error one")]
assert result.to_dict() == raw_result
assert json.loads(result.to_json()) == raw_result
with pytest.raises(ValueError, match="matching_lines must be a list"):
FileSearchResult.from_dict({"file_name": "x", "snippet": "", "matching_lines": {}})
with pytest.raises(ValueError, match="elements must be mappings"):
FileSearchResult.from_dict({"file_name": "x", "snippet": "", "matching_lines": ["not-a-dict"]})
async def test_in_memory_store_round_trips_files() -> None:
"""The in-memory store should support write/read/exists/delete/list operations."""
store = InMemoryAgentFileStore()
await store.write_file("a.txt", "alpha")
await store.write_file("sub/b.txt", "beta")
assert await store.file_exists("a.txt")
assert not await store.file_exists("missing.txt")
assert await store.read_file("a.txt") == "alpha"
assert await store.read_file("missing.txt") is None
assert sorted(await store.list_files()) == ["a.txt"] # subdirs are not direct children
assert sorted(await store.list_files("sub")) == ["b.txt"]
assert await store.delete_file("a.txt") is True
assert await store.delete_file("a.txt") is False
assert sorted(await store.list_files()) == []
async def test_in_memory_store_search_returns_matches_with_snippets() -> None:
"""The in-memory store should search file content case-insensitively and respect glob filters."""
store = InMemoryAgentFileStore()
await store.write_file("a.md", "line one\nThis line has ERROR inside\nline three\r")
await store.write_file("b.md", "no match here")
await store.write_file("notes.txt", "ERROR but wrong extension")
results = await store.search_files("", "error", "*.md")
assert [result.file_name for result in results] == ["a.md"]
matching_lines = results[0].matching_lines
assert matching_lines == [FileSearchMatch(line_number=2, line="This line has ERROR inside")]
assert "ERROR" in results[0].snippet
# No glob -> searches every file.
results_all = await store.search_files("", "error")
assert {result.file_name for result in results_all} == {"a.md", "notes.txt"}
async def test_in_memory_store_search_rejects_invalid_and_oversize_regex() -> None:
"""``search_files`` should surface clean errors for bad regex input."""
store = InMemoryAgentFileStore()
await store.write_file("a.md", "hello")
with pytest.raises(re.error):
await store.search_files("", "[unclosed")
with pytest.raises(ValueError, match="too long"):
await store.search_files("", "a" * 257)
async def test_in_memory_store_normalizes_paths() -> None:
"""Path normalization should reject traversal in the in-memory store too."""
store = InMemoryAgentFileStore()
for bad in ("../escape.txt", "/abs/path.txt", "."):
with pytest.raises(ValueError):
await store.write_file(bad, "boom")
async def test_filesystem_store_round_trips_files(tmp_path: Path) -> None:
"""The filesystem store should round-trip files on disk and create parents on write."""
store = FileSystemAgentFileStore(tmp_path)
await store.write_file("nested/a.txt", "alpha")
assert (tmp_path / "nested" / "a.txt").read_text(encoding="utf-8") == "alpha"
assert await store.read_file("nested/a.txt") == "alpha"
assert await store.read_file("missing.txt") is None
assert await store.file_exists("nested/a.txt")
assert not await store.file_exists("missing.txt")
assert sorted(await store.list_files("nested")) == ["a.txt"]
assert sorted(await store.list_files()) == [] # root only contains the directory
assert await store.delete_file("nested/a.txt") is True
assert await store.delete_file("nested/a.txt") is False
async def test_filesystem_store_rejects_traversal_and_rooted_paths(tmp_path: Path) -> None:
"""The filesystem store should refuse paths that escape the configured root."""
store = FileSystemAgentFileStore(tmp_path)
for bad in ("../escape.txt", "/etc/passwd", "C:/Windows/System32/notepad.exe", ".", ".."):
with pytest.raises(ValueError):
await store.write_file(bad, "boom")
async def test_filesystem_store_rejects_symlinks_into_root(tmp_path: Path) -> None:
"""The filesystem store should refuse to read through a symlink target."""
target = tmp_path / "outside.txt"
target.write_text("outside", encoding="utf-8")
root = tmp_path / "root"
root.mkdir()
link = root / "link.txt"
try:
link.symlink_to(target)
except (OSError, NotImplementedError) as exc:
pytest.skip(f"Symbolic links are not supported in this environment: {exc!r}")
store = FileSystemAgentFileStore(root)
with pytest.raises(ValueError, match="symbolic link"):
await store.read_file("link.txt")
with pytest.raises(ValueError, match="symbolic link"):
await store.write_file("link.txt", "stomp")
# List operations should silently skip the symlink entry rather than raise.
assert await store.list_files() == []
async def test_filesystem_store_rejects_in_root_symlinks(tmp_path: Path) -> None:
"""Symlinks whose target lives under the root must still be rejected.
``Path.resolve`` collapses the symlink, so a naive resolved-path check
would silently follow it. The symlink probe must operate on the
unresolved candidate for this case to fail closed.
"""
root = tmp_path / "root"
root.mkdir()
real = root / "real.txt"
real.write_text("payload", encoding="utf-8")
link = root / "alias.txt"
try:
link.symlink_to(real)
except (OSError, NotImplementedError) as exc:
pytest.skip(f"Symbolic links are not supported in this environment: {exc!r}")
store = FileSystemAgentFileStore(root)
with pytest.raises(ValueError, match="symbolic link"):
await store.read_file("alias.txt")
# The non-symlinked sibling must still be readable.
assert await store.read_file("real.txt") == "payload"
async def test_filesystem_store_search_matches_lines_and_filters_globs(tmp_path: Path) -> None:
"""The filesystem store should search files on disk and apply glob filters by file name."""
store = FileSystemAgentFileStore(tmp_path)
await store.write_file("a.md", "hello\nERROR happens\nbye\r")
await store.write_file("b.txt", "ERROR happens too")
await store.write_file("c.md", "nothing here")
results = await store.search_files("", "error", "*.md")
assert [result.file_name for result in results] == ["a.md"]
assert results[0].matching_lines == [FileSearchMatch(line_number=2, line="ERROR happens")]
assert "ERROR" in results[0].snippet
results_all = await store.search_files("", "error")
assert {result.file_name for result in results_all} == {"a.md", "b.txt"}
async def test_filesystem_store_search_skips_non_utf8_files(tmp_path: Path) -> None:
"""The filesystem store should silently skip non-UTF-8 files instead of aborting the search."""
store = FileSystemAgentFileStore(tmp_path)
await store.write_file("notes.md", "ERROR happens here")
(tmp_path / "blob.bin").write_bytes(b"\x80\x81\x82\x83")
results = await store.search_files("", "error")
assert [result.file_name for result in results] == ["notes.md"]
async def test_filesystem_store_create_directory(tmp_path: Path) -> None:
"""The filesystem store should create directories under the configured root."""
store = FileSystemAgentFileStore(tmp_path)
await store.create_directory("nested/inner")
assert (tmp_path / "nested" / "inner").is_dir()
async def test_filesystem_store_list_files_accepts_blank_directory(tmp_path: Path) -> None:
"""Whitespace-only directory inputs should resolve to the root, matching the in-memory store."""
store = FileSystemAgentFileStore(tmp_path)
await store.write_file("a.txt", "alpha")
assert sorted(await store.list_files("")) == ["a.txt"]
assert sorted(await store.list_files(" ")) == ["a.txt"]
def test_filesystem_store_requires_non_empty_root() -> None:
"""The filesystem store constructor should refuse blank root paths."""
with pytest.raises(ValueError, match="must not be empty"):
FileSystemAgentFileStore("")
with pytest.raises(ValueError, match="must not be empty"):
FileSystemAgentFileStore(" ")
async def test_file_access_provider_registers_tools_and_instructions(
chat_client_base: SupportsChatGetResponse,
) -> None:
"""``FileAccessProvider.before_run`` should add the canonical instructions and five tools."""
session = AgentSession(session_id="session-1")
store = InMemoryAgentFileStore()
provider = FileAccessProvider(store=store)
agent = Agent(client=chat_client_base, context_providers=[provider])
_, options = await agent._prepare_session_and_messages( # type: ignore[reportPrivateUsage]
session=session,
input_messages=[Message(role="user", contents=["work with files"])],
)
tools = options["tools"]
assert isinstance(tools, list)
expected_names = {
"file_access_save_file",
"file_access_read_file",
"file_access_delete_file",
"file_access_list_files",
"file_access_search_files",
}
assert {getattr(tool, "name", None) for tool in tools} >= expected_names
instructions = options.get("instructions")
if isinstance(instructions, str):
assert DEFAULT_FILE_ACCESS_INSTRUCTIONS in instructions
else:
assert any(DEFAULT_FILE_ACCESS_INSTRUCTIONS in chunk for chunk in (instructions or []))
async def test_file_access_provider_delete_approval_defaults_to_always_require(
chat_client_base: SupportsChatGetResponse,
) -> None:
"""By default ``file_access_delete_file`` should require host approval."""
session = AgentSession(session_id="session-1")
provider = FileAccessProvider(store=InMemoryAgentFileStore())
agent = Agent(client=chat_client_base, context_providers=[provider])
_, options = await agent._prepare_session_and_messages( # type: ignore[reportPrivateUsage]
session=session,
input_messages=[Message(role="user", contents=["work with files"])],
)
tools = options["tools"]
assert isinstance(tools, list)
delete_file = _tool_by_name(tools, "file_access_delete_file")
assert delete_file.approval_mode == "always_require"
# The non-destructive tools should remain autonomous.
for name in (
"file_access_save_file",
"file_access_read_file",
"file_access_list_files",
"file_access_search_files",
):
assert _tool_by_name(tools, name).approval_mode == "never_require"
async def test_file_access_provider_delete_approval_opt_out(
chat_client_base: SupportsChatGetResponse,
) -> None:
"""``require_delete_approval=False`` should drop delete to ``never_require``."""
session = AgentSession(session_id="session-1")
provider = FileAccessProvider(store=InMemoryAgentFileStore(), require_delete_approval=False)
agent = Agent(client=chat_client_base, context_providers=[provider])
_, options = await agent._prepare_session_and_messages( # type: ignore[reportPrivateUsage]
session=session,
input_messages=[Message(role="user", contents=["work with files"])],
)
delete_file = _tool_by_name(options["tools"], "file_access_delete_file") # type: ignore[arg-type]
assert delete_file.approval_mode == "never_require"
async def test_file_access_provider_tools_round_trip_files(
chat_client_base: SupportsChatGetResponse,
) -> None:
"""The provider's tools should drive save/read/list/search/delete flows on an ``InMemoryAgentFileStore``."""
session = AgentSession(session_id="session-1")
store = InMemoryAgentFileStore()
provider = FileAccessProvider(store=store)
agent = Agent(client=chat_client_base, context_providers=[provider])
_, options = await agent._prepare_session_and_messages( # type: ignore[reportPrivateUsage]
session=session,
input_messages=[Message(role="user", contents=["work with files"])],
)
tools = options["tools"]
assert isinstance(tools, list)
save_file = _tool_by_name(tools, "file_access_save_file")
read_file = _tool_by_name(tools, "file_access_read_file")
delete_file = _tool_by_name(tools, "file_access_delete_file")
list_files = _tool_by_name(tools, "file_access_list_files")
search_files = _tool_by_name(tools, "file_access_search_files")
saved = await save_file.invoke(arguments={"file_name": "plan.md", "content": "step 1\nERROR step 2"})
assert "plan.md" in saved[0].text and "saved" in saved[0].text
# Default overwrite=False should refuse the second save.
refused = await save_file.invoke(arguments={"file_name": "plan.md", "content": "stomp"})
assert "already exists" in refused[0].text
# overwrite=True should succeed.
overwritten = await save_file.invoke(
arguments={"file_name": "plan.md", "content": "stomp\nERROR replaced", "overwrite": True}
)
assert "saved" in overwritten[0].text
read_back = await read_file.invoke(arguments={"file_name": "plan.md"})
assert read_back[0].text == "stomp\nERROR replaced"
listed = await list_files.invoke()
assert json.loads(listed[0].text) == ["plan.md"]
# The list tool should accept an optional directory argument so agents can
# enumerate nested folders (not only the root).
await save_file.invoke(arguments={"file_name": "reports/2024.md", "content": "annual"})
listed_nested = await list_files.invoke(arguments={"directory": "reports"})
assert json.loads(listed_nested[0].text) == ["2024.md"]
# Blank / whitespace directory should fall back to the root listing.
listed_blank = await list_files.invoke(arguments={"directory": " "})
assert sorted(json.loads(listed_blank[0].text)) == ["plan.md"]
missing = await read_file.invoke(arguments={"file_name": "missing.md"})
assert "not found" in missing[0].text
search_payload = await search_files.invoke(arguments={"regex_pattern": "error", "file_pattern": "*.md"})
parsed = json.loads(search_payload[0].text)
assert parsed[0]["file_name"] == "plan.md"
assert parsed[0]["matching_lines"][0]["line"] == "ERROR replaced"
# The search tool should likewise accept an optional directory argument so
# agents can scope a search to a subfolder.
await save_file.invoke(arguments={"file_name": "reports/issues.md", "content": "ERROR nested"})
scoped = await search_files.invoke(
arguments={"regex_pattern": "error", "file_pattern": "*.md", "directory": "reports"}
)
scoped_parsed = json.loads(scoped[0].text)
assert [entry["file_name"] for entry in scoped_parsed] == ["issues.md"]
deleted = await delete_file.invoke(arguments={"file_name": "plan.md"})
assert "deleted" in deleted[0].text
missing_delete = await delete_file.invoke(arguments={"file_name": "plan.md"})
assert "not found" in missing_delete[0].text
async def test_file_access_provider_accepts_custom_instructions() -> None:
"""Custom instructions should override the default banner."""
store = InMemoryAgentFileStore()
provider = FileAccessProvider(store=store, instructions="custom-banner")
assert provider.instructions == "custom-banner"
assert provider.source_id == DEFAULT_FILE_ACCESS_SOURCE_ID
async def test_in_memory_store_write_file_raises_when_exists_and_no_overwrite() -> None:
"""The atomic exclusive-create path should raise ``FileExistsError`` under the lock."""
store = InMemoryAgentFileStore()
await store.write_file("plan.md", "v1")
with pytest.raises(FileExistsError):
await store.write_file("plan.md", "v2", overwrite=False)
# The original content is preserved.
assert await store.read_file("plan.md") == "v1"
# Default ``overwrite=True`` still replaces.
await store.write_file("plan.md", "v3")
assert await store.read_file("plan.md") == "v3"
async def test_filesystem_store_write_file_raises_when_exists_and_no_overwrite(tmp_path: Path) -> None:
"""The filesystem store should use exclusive-create semantics when ``overwrite=False``."""
store = FileSystemAgentFileStore(tmp_path)
await store.write_file("plan.md", "v1")
with pytest.raises(FileExistsError):
await store.write_file("plan.md", "v2", overwrite=False)
assert (tmp_path / "plan.md").read_text(encoding="utf-8") == "v1"
await store.write_file("plan.md", "v3", overwrite=True)
assert (tmp_path / "plan.md").read_text(encoding="utf-8") == "v3"
async def test_run_search_with_timeout_raises_value_error(monkeypatch: pytest.MonkeyPatch) -> None:
"""A scan that exceeds the timeout should surface a clean ``ValueError``."""
monkeypatch.setattr(_file_access_module, "_SEARCH_TIMEOUT_SECONDS", 0.01)
def slow() -> list[FileSearchResult]:
time.sleep(0.5)
return []
with pytest.raises(ValueError, match="did not complete"):
await _run_search_with_timeout(slow)
async def test_filesystem_store_symlink_probe_fails_closed_on_oserror(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
"""If ``Path.is_symlink`` raises during the probe, the operation must be refused."""
store = FileSystemAgentFileStore(tmp_path)
await store.write_file("ok.txt", "content")
def boom(self: Path) -> bool:
raise PermissionError("access denied")
monkeypatch.setattr(Path, "is_symlink", boom)
with pytest.raises(ValueError, match="symbolic link or reparse point"):
await store.read_file("ok.txt")
def test_file_access_harness_classes_are_marked_experimental() -> None:
"""File-access harness public classes should expose HARNESS experimental metadata."""
assert AgentFileStore.__feature_id__ == ExperimentalFeature.HARNESS.value
assert InMemoryAgentFileStore.__feature_id__ == ExperimentalFeature.HARNESS.value
assert FileSystemAgentFileStore.__feature_id__ == ExperimentalFeature.HARNESS.value
assert FileSearchMatch.__feature_id__ == ExperimentalFeature.HARNESS.value
assert FileSearchResult.__feature_id__ == ExperimentalFeature.HARNESS.value
assert FileAccessProvider.__feature_id__ == ExperimentalFeature.HARNESS.value
assert ".. warning:: Experimental" in (FileAccessProvider.__doc__ or "")
async def test_in_memory_store_preserves_original_case_on_list_and_search() -> None:
"""``list_files`` / ``search_files`` should return original-case names, not lowercased keys.
Matches :class:`FileSystemAgentFileStore` on case-preserving filesystems so
tests written against the in-memory backend cannot encode a contract that
will diverge in production.
"""
store = InMemoryAgentFileStore()
await store.write_file("Plan.MD", "ERROR happens here\n")
await store.write_file("Reports/Q1.MD", "alpha")
# list_files keeps the original case
assert sorted(await store.list_files()) == ["Plan.MD"]
assert sorted(await store.list_files("Reports")) == ["Q1.MD"]
# case-insensitive directory lookup still works
assert sorted(await store.list_files("reports")) == ["Q1.MD"]
# search_files emits the original-case file name in FileSearchResult
results = await store.search_files("", "error", "*.MD")
assert [r.file_name for r in results] == ["Plan.MD"]
# read_file remains case-insensitive
assert await store.read_file("plan.md") == "ERROR happens here\n"
async def test_filesystem_store_read_file_raises_value_error_on_non_utf8(tmp_path: Path) -> None:
"""Binary / non-UTF-8 files should raise a clean ``ValueError`` rather than ``UnicodeDecodeError``.
The tool-layer wrapper relies on this contract to convert the failure into
a recoverable string response for the agent.
"""
store = FileSystemAgentFileStore(tmp_path)
(tmp_path / "blob.bin").write_bytes(b"\x80\x81\x82\x83")
with pytest.raises(ValueError, match="not UTF-8 text"):
await store.read_file("blob.bin")
async def test_filesystem_store_search_logs_skipped_non_utf8_files(
tmp_path: Path, caplog: pytest.LogCaptureFixture
) -> None:
"""``search_files`` skips non-UTF-8 files but logs per-file and a summary so operators have signal."""
store = FileSystemAgentFileStore(tmp_path)
await store.write_file("notes.md", "ERROR happens here")
(tmp_path / "blob.bin").write_bytes(b"\x80\x81\x82\x83")
with caplog.at_level("INFO", logger="agent_framework._harness._file_access"):
results = await store.search_files("", "error")
assert [r.file_name for r in results] == ["notes.md"]
assert any("Skipping non-UTF-8 file during search" in rec.message for rec in caplog.records)
assert any("skipped 1 non-UTF-8 file" in rec.message for rec in caplog.records)
async def test_file_access_tool_wrappers_surface_value_error_as_message(
chat_client_base: SupportsChatGetResponse,
) -> None:
"""Recoverable failures (bad path, oversized regex, non-UTF-8 read) should be returned as strings.
Without these wrappers the model sees a raw stack trace for "you used ``..``"
but a polite message for "the file already exists", which is the opposite
of what is recoverable.
"""
session = AgentSession(session_id="session-1")
store = InMemoryAgentFileStore()
provider = FileAccessProvider(store=store)
agent = Agent(client=chat_client_base, context_providers=[provider])
_, options = await agent._prepare_session_and_messages( # type: ignore[reportPrivateUsage]
session=session,
input_messages=[Message(role="user", contents=["work with files"])],
)
tools = options["tools"]
assert isinstance(tools, list)
save_file = _tool_by_name(tools, "file_access_save_file")
read_file = _tool_by_name(tools, "file_access_read_file")
delete_file = _tool_by_name(tools, "file_access_delete_file")
list_files = _tool_by_name(tools, "file_access_list_files")
search_files = _tool_by_name(tools, "file_access_search_files")
# Path-traversal attempts on each tool should return a clean string, not raise.
saved = await save_file.invoke(arguments={"file_name": "../escape.txt", "content": "x"})
assert "Could not save" in saved[0].text and "escape" in saved[0].text.lower()
read = await read_file.invoke(arguments={"file_name": "../escape.txt"})
assert "Could not read" in read[0].text
deleted = await delete_file.invoke(arguments={"file_name": "../escape.txt"})
assert "Could not delete" in deleted[0].text
listed = await list_files.invoke(arguments={"directory": "../escape"})
assert "Could not list" in listed[0].text
# Regex length cap should also be returned to the model as text.
too_long = "a" * 1024
searched = await search_files.invoke(arguments={"regex_pattern": too_long})
assert "Could not search files" in searched[0].text
async def test_file_access_tool_read_file_wrapper_surfaces_non_utf8(
tmp_path: Path, chat_client_base: SupportsChatGetResponse
) -> None:
"""The read-file tool wrapper should convert a non-UTF-8 ``ValueError`` into a readable string."""
store = FileSystemAgentFileStore(tmp_path)
(tmp_path / "blob.bin").write_bytes(b"\x80\x81\x82\x83")
session = AgentSession(session_id="session-1")
provider = FileAccessProvider(store=store)
agent = Agent(client=chat_client_base, context_providers=[provider])
_, options = await agent._prepare_session_and_messages( # type: ignore[reportPrivateUsage]
session=session,
input_messages=[Message(role="user", contents=["read it"])],
)
read_file = _tool_by_name(options["tools"], "file_access_read_file")
response = await read_file.invoke(arguments={"file_name": "blob.bin"})
assert "Could not read" in response[0].text and "UTF-8" in response[0].text
_NEEDS_SYMLINK = "Symbolic links are not supported in this environment"
async def test_filesystem_store_rejects_symlink_on_delete_search_and_list(tmp_path: Path) -> None:
"""The same symlink probe must front delete/search/list, not just read/write."""
target = tmp_path / "outside.txt"
target.write_text("outside", encoding="utf-8")
root = tmp_path / "root"
root.mkdir()
link = root / "link.txt"
try:
link.symlink_to(target)
except (OSError, NotImplementedError) as exc:
pytest.skip(f"{_NEEDS_SYMLINK}: {exc!r}")
store = FileSystemAgentFileStore(root)
with pytest.raises(ValueError, match="symbolic link"):
await store.delete_file("link.txt")
# search_files of the root never touches the symlink leaf directly, but
# search_files of a symlinked *directory* path must be rejected by the
# safe-directory resolver.
dir_link = root / "alias_dir"
other_dir = tmp_path / "outside_dir"
other_dir.mkdir()
try:
dir_link.symlink_to(other_dir)
except (OSError, NotImplementedError) as exc:
pytest.skip(f"{_NEEDS_SYMLINK}: {exc!r}")
with pytest.raises(ValueError, match="symbolic link"):
await store.search_files("alias_dir", "anything")
with pytest.raises(ValueError, match="symbolic link"):
await store.list_files("alias_dir")
async def test_filesystem_store_rejects_symlinked_intermediate_directory(tmp_path: Path) -> None:
"""A symlink used as a non-leaf path segment must still be rejected.
The classic escape vector is ``root/aliased_dir/file.txt`` where
``aliased_dir`` is a symlink to somewhere outside the root. The
``_throw_if_contains_symlink`` walk must check every segment, not only
the leaf.
"""
outside = tmp_path / "outside_dir"
outside.mkdir()
(outside / "secret.txt").write_text("payload", encoding="utf-8")
root = tmp_path / "root"
root.mkdir()
link = root / "aliased_dir"
try:
link.symlink_to(outside)
except (OSError, NotImplementedError) as exc:
pytest.skip(f"{_NEEDS_SYMLINK}: {exc!r}")
store = FileSystemAgentFileStore(root)
for op in ("read", "write", "delete"):
with pytest.raises(ValueError, match="symbolic link"):
if op == "read":
await store.read_file("aliased_dir/secret.txt")
elif op == "write":
await store.write_file("aliased_dir/secret.txt", "stomp")
else:
await store.delete_file("aliased_dir/secret.txt")
+206
View File
@@ -1161,6 +1161,43 @@ async def test_local_mcp_server_function_execution_error():
await func.invoke(param="test_value")
async def test_mcp_tool_reconnects_after_session_terminated_error():
"""Session termination errors should reconnect once and retry the tool call."""
class TestServer(MCPTool):
def __init__(self, **kwargs: Any) -> None:
super().__init__(**kwargs)
self.connect_count = 0
self.sessions: list[Any] = []
async def connect(self, *, reset: bool = False) -> None:
self.connect_count += 1
self.session = Mock(spec=ClientSession)
self.sessions.append(self.session)
if self.connect_count == 1:
self.session.call_tool = AsyncMock(
side_effect=McpError(types.ErrorData(code=-32000, message="Session terminated"))
)
else:
self.session.call_tool = AsyncMock(
return_value=types.CallToolResult(content=[types.TextContent(type="text", text="recovered")])
)
self.is_connected = True
def get_mcp_client(self) -> _AsyncGeneratorContextManager[Any, None]:
return None
server = TestServer(name="test_server")
await server.connect()
result = await server.call_tool("test_tool", param="test_value")
assert _mcp_result_to_text(result) == "recovered"
assert server.connect_count == 2
assert server.sessions[0].call_tool.await_count == 1
assert server.sessions[1].call_tool.await_count == 1
async def test_mcp_tool_call_tool_raises_on_is_error():
"""Test that call_tool raises ToolExecutionException when MCP returns isError=True."""
@@ -3260,6 +3297,68 @@ async def test_load_prompts_pagination_with_duplicates():
assert [f.name for f in tool._functions] == ["prompt_1", "prompt_2"]
async def test_load_tools_concurrent_reload_does_not_duplicate_tools_and_preserves_meta():
"""Concurrent tool reloads should not duplicate functions or lose tools/list metadata."""
tool = MCPTool(name="test_tool")
mock_session = AsyncMock()
tool.session = mock_session
tool.load_tools_flag = True
page = Mock()
page.tools = [
types.Tool(
name="tool_1",
description="First tool",
inputSchema={"type": "object", "properties": {"param": {"type": "string"}}},
_meta={"echo": "tool_1"},
),
]
page.nextCursor = None
async def mock_list_tools(params: Any = None) -> Any:
assert params is None
await asyncio.sleep(0)
return page
mock_session.list_tools = AsyncMock(side_effect=mock_list_tools)
await asyncio.wait_for(asyncio.gather(tool.load_tools(), tool.load_tools()), timeout=1)
assert mock_session.list_tools.call_count == 2
assert [f.name for f in tool._functions] == ["tool_1"]
assert tool._tool_call_meta_by_name == {"tool_1": {"echo": "tool_1"}}
async def test_load_prompts_concurrent_reload_does_not_duplicate_prompts():
"""Concurrent prompt reloads should not duplicate functions."""
tool = MCPTool(name="test_tool")
mock_session = AsyncMock()
tool.session = mock_session
tool.load_prompts_flag = True
page = Mock()
page.prompts = [
types.Prompt(
name="prompt_1",
description="First prompt",
arguments=[types.PromptArgument(name="arg1", description="Arg 1", required=True)],
),
]
page.nextCursor = None
async def mock_list_prompts(params: Any = None) -> Any:
assert params is None
await asyncio.sleep(0)
return page
mock_session.list_prompts = AsyncMock(side_effect=mock_list_prompts)
await asyncio.wait_for(asyncio.gather(tool.load_prompts(), tool.load_prompts()), timeout=1)
assert mock_session.list_prompts.call_count == 2
assert [f.name for f in tool._functions] == ["prompt_1"]
async def test_load_tools_pagination_exception_handling():
"""Test that load_tools handles exceptions during pagination gracefully."""
from unittest.mock import AsyncMock
@@ -3891,6 +3990,31 @@ async def test_mcp_tool_safe_close_handles_cancelled_error():
mock_exit_stack.aclose.assert_called_once()
async def test_mcp_tool_safe_close_handles_cleanup_exception_group():
"""Cleanup task groups should not hide the original connect failure."""
import builtins
from contextlib import AsyncExitStack
exception_group_type = getattr(builtins, "ExceptionGroup", None)
if exception_group_type is None:
pytest.skip("ExceptionGroup is not available on this Python version")
tool = MCPStreamableHTTPTool(
name="test",
url="http://example.com/mcp",
load_tools=False,
load_prompts=False,
)
mock_exit_stack = AsyncMock(spec=AsyncExitStack)
mock_exit_stack.aclose = AsyncMock(side_effect=exception_group_type("cleanup failed", [RuntimeError("reader")]))
tool._exit_stack = mock_exit_stack
await tool._safe_close_exit_stack()
mock_exit_stack.aclose.assert_called_once()
async def test_connect_sets_logging_level_when_logger_level_is_set():
"""Test that connect() sets the MCP server logging level when the logger level is not NOTSET."""
@@ -4389,6 +4513,52 @@ async def test_mcp_tool_call_tool_forwards_tool_list_meta():
assert server.session.call_tool.call_args.kwargs["meta"] == tool_meta
async def test_mcp_tool_call_tool_user_meta_merges_with_tool_list_meta():
"""User-provided _meta should be sent as MCP request metadata, not tool arguments."""
from opentelemetry import trace
tool_meta = {"from_tool": "tool-value", "shared": "tool-value"}
user_meta = {"from_user": "user-value", "shared": "user-value"}
class TestServer(MCPTool):
async def connect(self) -> None:
self.session = Mock(spec=ClientSession)
self.session.list_tools = AsyncMock(
return_value=types.ListToolsResult(
tools=[
types.Tool(
name="test_tool",
description="Test tool",
inputSchema={"type": "object", "properties": {"param": {"type": "string"}}},
_meta=tool_meta,
)
]
)
)
self.session.call_tool = AsyncMock(
return_value=types.CallToolResult(content=[types.TextContent(type="text", text="result")])
)
def get_mcp_client(self) -> _AsyncGeneratorContextManager[Any, None]:
return None
server = TestServer(name="test_server")
async with server:
await server.load_tools()
with trace.use_span(trace.NonRecordingSpan(trace.INVALID_SPAN_CONTEXT)):
await server.call_tool("test_tool", param="test_value", _meta=user_meta)
call_kwargs = server.session.call_tool.call_args.kwargs
assert call_kwargs["arguments"] == {"param": "test_value"}
assert call_kwargs["meta"] == {
"from_tool": "tool-value",
"from_user": "user-value",
"shared": "user-value",
}
assert user_meta == {"from_user": "user-value", "shared": "user-value"}
async def test_mcp_streamable_http_tool_hook_not_duplicated_on_repeated_get_mcp_client():
"""Test that calling get_mcp_client multiple times does not accumulate duplicate hooks."""
tool = MCPStreamableHTTPTool(
@@ -4641,6 +4811,42 @@ async def test_mcp_streamable_http_tool_header_provider_with_httpx_event_hook():
await tool._httpx_client.aclose()
async def test_mcp_streamable_http_tool_header_provider_skips_cross_origin_redirect():
"""The request hook must not re-add caller headers after a cross-origin redirect."""
import httpx
from agent_framework._mcp import _mcp_call_headers
tool = MCPStreamableHTTPTool(
name="test",
url="http://example.com/mcp",
header_provider=lambda kw: {"Authorization": f"Bearer {kw.get('token', '')}"},
)
try:
with patch("agent_framework._mcp.streamable_http_client"):
tool.get_mcp_client()
assert tool._httpx_client is not None
hooks = tool._httpx_client.event_hooks.get("request", [])
assert len(hooks) == 1
token = _mcp_call_headers.set({"Authorization": "Bearer secret"})
try:
same_origin = httpx.Request("POST", "http://example.com/redirected")
await hooks[0](same_origin)
assert same_origin.headers.get("Authorization") == "Bearer secret"
cross_origin = httpx.Request("POST", "http://attacker.example/capture")
await hooks[0](cross_origin)
assert "Authorization" not in cross_origin.headers
finally:
_mcp_call_headers.reset(token)
finally:
if getattr(tool, "_httpx_client", None) is not None:
await tool._httpx_client.aclose()
async def test_mcp_streamable_http_tool_header_provider_with_user_httpx_client():
"""Test that header_provider works when the user provides their own httpx client."""
import httpx
@@ -3449,6 +3449,140 @@ def test_capture_response_with_error_type(span_exporter: InMemorySpanExporter):
assert spans[0].attributes.get(OtelAttr.ERROR_TYPE) == "ValueError"
def test_backfill_request_model_when_unknown(span_exporter: InMemorySpanExporter):
"""_backfill_request_model updates the span name and REQUEST_MODEL attribute when unknown."""
from agent_framework.observability import OtelAttr, get_tracer
span_exporter.clear()
tracer = get_tracer()
attrs: dict[str, Any] = {
OtelAttr.OPERATION: "chat",
OtelAttr.REQUEST_MODEL: "unknown",
OtelAttr.RESPONSE_MODEL: "gpt-4o-mini",
}
with tracer.start_as_current_span("chat unknown") as span:
ChatTelemetryLayer._backfill_request_model(span, attrs)
assert attrs[OtelAttr.REQUEST_MODEL] == "gpt-4o-mini"
spans = span_exporter.get_finished_spans()
assert len(spans) == 1
assert spans[0].name == "chat gpt-4o-mini"
def test_backfill_request_model_noop_when_request_model_known(span_exporter: InMemorySpanExporter):
"""_backfill_request_model leaves a known REQUEST_MODEL and span name untouched."""
from agent_framework.observability import OtelAttr, get_tracer
span_exporter.clear()
tracer = get_tracer()
attrs: dict[str, Any] = {
OtelAttr.OPERATION: "chat",
OtelAttr.REQUEST_MODEL: "gpt-4o",
OtelAttr.RESPONSE_MODEL: "gpt-4o-mini",
}
with tracer.start_as_current_span("chat gpt-4o") as span:
ChatTelemetryLayer._backfill_request_model(span, attrs)
assert attrs[OtelAttr.REQUEST_MODEL] == "gpt-4o"
spans = span_exporter.get_finished_spans()
assert len(spans) == 1
assert spans[0].name == "chat gpt-4o"
def test_backfill_request_model_noop_when_response_model_missing(span_exporter: InMemorySpanExporter):
"""_backfill_request_model is a no-op when no RESPONSE_MODEL is available."""
from agent_framework.observability import OtelAttr, get_tracer
span_exporter.clear()
tracer = get_tracer()
attrs: dict[str, Any] = {
OtelAttr.OPERATION: "chat",
OtelAttr.REQUEST_MODEL: "unknown",
}
with tracer.start_as_current_span("chat unknown") as span:
ChatTelemetryLayer._backfill_request_model(span, attrs)
assert attrs[OtelAttr.REQUEST_MODEL] == "unknown"
spans = span_exporter.get_finished_spans()
assert len(spans) == 1
assert spans[0].name == "chat unknown"
async def test_chat_client_backfills_request_model_from_response(span_exporter: InMemorySpanExporter):
"""Non-streaming chat: when REQUEST_MODEL is unknown, the response model backfills it."""
class BackfillingChatClient(ChatTelemetryLayer, BaseChatClient[Any]):
def service_url(self):
return "https://test.example.com"
def _inner_get_response(
self, *, messages: MutableSequence[Message], stream: bool, options: dict[str, Any], **kwargs: Any
) -> Awaitable[ChatResponse] | ResponseStream[ChatResponseUpdate, ChatResponse]:
async def _get() -> ChatResponse:
return ChatResponse(
messages=[Message("assistant", ["Test response"])],
model="resolved-model",
)
return _get()
client = BackfillingChatClient()
span_exporter.clear()
# Note: no "model" in options, so REQUEST_MODEL starts as "unknown".
await client.get_response(messages=[Message(role="user", contents=["Hi"])], options={})
spans = span_exporter.get_finished_spans()
assert len(spans) == 1
span = spans[0]
assert span.name == "chat resolved-model"
assert span.attributes[OtelAttr.REQUEST_MODEL] == "resolved-model"
assert span.attributes[OtelAttr.RESPONSE_MODEL] == "resolved-model"
async def test_chat_client_streaming_backfills_request_model_from_response(
span_exporter: InMemorySpanExporter,
):
"""Streaming chat: when REQUEST_MODEL is unknown, the response model backfills it."""
class BackfillingStreamingChatClient(ChatTelemetryLayer, BaseChatClient[Any]):
def service_url(self):
return "https://test.example.com"
def _inner_get_response(
self, *, messages: MutableSequence[Message], stream: bool, options: dict[str, Any], **kwargs: Any
) -> Awaitable[ChatResponse] | ResponseStream[ChatResponseUpdate, ChatResponse]:
async def _stream() -> AsyncIterable[ChatResponseUpdate]:
yield ChatResponseUpdate(contents=[Content.from_text("Hello")], role="assistant")
yield ChatResponseUpdate(contents=[Content.from_text(" world")], role="assistant", finish_reason="stop")
def _finalize(updates: Sequence[ChatResponseUpdate]) -> ChatResponse:
response = ChatResponse.from_updates(updates)
response.model = "resolved-stream-model"
return response
return ResponseStream(_stream(), finalizer=_finalize)
client = BackfillingStreamingChatClient()
span_exporter.clear()
stream = client.get_response(stream=True, messages=[Message(role="user", contents=["Hi"])], options={})
async for _ in stream:
pass
await stream.get_final_response()
spans = span_exporter.get_finished_spans()
assert len(spans) == 1
span = spans[0]
assert span.name == "chat resolved-stream-model"
assert span.attributes[OtelAttr.REQUEST_MODEL] == "resolved-stream-model"
assert span.attributes[OtelAttr.RESPONSE_MODEL] == "resolved-stream-model"
def test_configure_otel_providers_with_env_file_path(monkeypatch, tmp_path):
"""Test configure_otel_providers with env_file_path creates new settings."""
import importlib
@@ -307,6 +307,63 @@ class TestHistoryProviderBase:
assert provider.stored[0].text == "hello"
assert provider.stored[1].text == "hi"
async def test_after_run_stores_coalesced_code_interpreter_chunks(self) -> None:
from agent_framework import AgentResponse, AgentResponseUpdate, Content
provider = ConcreteHistoryProvider("mem", store_inputs=False)
updates = [
AgentResponseUpdate(
role="assistant",
contents=[
Content.from_code_interpreter_tool_result(
call_id="ci_123",
outputs=[],
)
],
),
AgentResponseUpdate(
contents=[
Content.from_code_interpreter_tool_call(
call_id="ci_123",
inputs=[Content.from_text(text="import")],
additional_properties={"sequence_number": 1},
)
],
),
AgentResponseUpdate(
contents=[
Content.from_code_interpreter_tool_call(
call_id="ci_123",
inputs=[Content.from_text(text=" pandas")],
additional_properties={"sequence_number": 2},
)
],
),
AgentResponseUpdate(
contents=[
Content.from_code_interpreter_tool_call(
call_id="ci_123",
inputs=[Content.from_text(text="import pandas as pd")],
additional_properties={"sequence_number": 3},
)
],
),
]
ctx = SessionContext(session_id="s1", input_messages=[Message(role="user", contents=["make a sheet"])])
ctx._response = AgentResponse.from_updates(updates)
await provider.after_run(agent=None, session=AgentSession(), context=ctx, state={}) # type: ignore[arg-type]
assert len(provider.stored) == 1
stored_contents = provider.stored[0].contents
calls = [content for content in stored_contents if content.type == "code_interpreter_tool_call"]
results = [content for content in stored_contents if content.type == "code_interpreter_tool_result"]
assert len(calls) == 1
assert len(results) == 1
assert calls[0].inputs is not None
assert len(calls[0].inputs) == 1
assert calls[0].inputs[0].text == "import pandas as pd"
async def test_after_run_skips_inputs_when_disabled(self) -> None:
from agent_framework import AgentResponse
@@ -507,10 +507,10 @@ class _FoundryAgentChatClient( # type: ignore[misc]
.. code-block:: python
from agent_framework import Agent
from agent_framework.foundry import FoundryAgentClient
from agent_framework.foundry import FoundryAgent
from azure.identity import AzureCliCredential
client = FoundryAgentClient(
client = FoundryAgent(
project_endpoint="https://your-project.services.ai.azure.com",
agent_name="my-prompt-agent",
agent_version="1",
@@ -11,7 +11,7 @@ import tempfile
import threading
from collections.abc import AsyncIterable, AsyncIterator, Generator, Sequence
from contextlib import AbstractAsyncContextManager, AsyncExitStack, suppress
from dataclasses import asdict, is_dataclass
from dataclasses import asdict, dataclass, is_dataclass
from pathlib import Path
from typing import Protocol, cast
@@ -264,28 +264,73 @@ def _checkpoint_storage_for_context(root: str, context_id: str) -> FileCheckpoin
# Foundry Toolbox Auth integration
# Consent-URL error code returned by the Foundry MCP gateway when calling `/list`
CONSENT_ERROR_CODE = -32007
CONSENT_ERROR_CODE = -32006
def consent_url_from_error(exc: BaseException) -> str | None:
"""Return the consent URL when ``exc`` wraps a Foundry MCP gateway consent error.
@dataclass
class ConsentError:
name: str
consent_url: str
The Agent Framework MCP layer surfaces gateway consent failures by wrapping the underlying
``McpError`` inside an :class:`AgentFrameworkException` (typically a ``ToolExecutionException``
raised from ``MCPStreamableHTTPTool.__aenter__``). This helper inspects ``exc.args`` for a
wrapped ``McpError`` whose ``error.code`` is :data:`CONSENT_ERROR_CODE`; when found, the
consent link the gateway returned in ``error.message`` is returned. Returns ``None`` for
anything else, so callers can do ``if (url := consent_url_from_error(ex)) is None: raise``.
def consent_url_from_error(exc: BaseException) -> list[ConsentError] | None:
"""Return the consent URLs when ``exc`` wraps Foundry MCP gateway consent errors.
Args:
exc: The exception to inspect.
Returns:
The consent URL if ``exc`` wraps a consent ``McpError``, otherwise ``None``.
The consent URL(s) extracted from the error, or ``None`` if no consent error was found.
"""
inner_exception = next((arg for arg in exc.args if isinstance(arg, McpError)), None)
if inner_exception is not None and inner_exception.error.code == CONSENT_ERROR_CODE:
return inner_exception.error.message
# Parse the error message
# The error message is structured with the following format:
# "tools/list failed for 1 tool source(s), succeeded for 0 tool source(s) {"errors":[{"name": ..."
# where the second part is a JSON string that can be deserialized into an object with the following shape:
# ruff: disable[ERA001]
# {
# "errors" : [
# {
# "name": "Name of the MCP tool that requires consent",
# "type" : "mcp",
# "error": {
# "code": "CONSENT_REQUIRED",
# "message": consent_url,
# }
# }
# ]
# }
# ruff: enable[ERA001]
try:
consent_errors: list[ConsentError] = []
error_message_start = inner_exception.error.message.find("{")
if error_message_start == -1:
logger.warning("Consent error message does not contain JSON: %s", inner_exception.error.message)
return None
consent_details_json = inner_exception.error.message[error_message_start:]
consent_details = json.loads(consent_details_json)
if "errors" not in consent_details or not isinstance(consent_details["errors"], list):
logger.warning("Consent error message JSON does not contain 'errors' list: %s", consent_details_json)
return None
for error in consent_details["errors"]:
if (
isinstance(error, dict)
and error.get("type") == "mcp" # type: ignore
and "error" in error
and isinstance(error["error"], dict)
and error["error"].get("code") == "CONSENT_REQUIRED" # type: ignore
and "message" in error["error"]
):
consent_url = error["error"]["message"] # type: ignore
if isinstance(consent_url, str):
consent_errors.append(ConsentError(name=error.get("name", "Unknown"), consent_url=consent_url)) # type: ignore
else:
logger.warning("Consent URL in error message is not a valid URL: %s", consent_url) # type: ignore
if consent_errors:
return consent_errors
except json.JSONDecodeError:
logger.warning("Failed to parse consent details JSON: %s", inner_exception.error.message)
return None
@@ -448,18 +493,19 @@ class ResponsesHostServer(ResponsesAgentServerHost):
try:
await self._ensure_agent_ready()
except AgentFrameworkException as ex:
consent_url = consent_url_from_error(ex)
if consent_url is None:
consent_errors = consent_url_from_error(ex)
if consent_errors is None:
raise
logger.warning("OAuth consent required for Foundry MCP gateway.")
oauth_item = OAuthConsentRequestOutputItem(
id=IdGenerator.new_id("oacr"),
consent_link=consent_url,
server_label="Foundry Toolbox",
)
builder = response_event_stream.add_output_item(oauth_item.id)
yield builder.emit_added(oauth_item)
yield builder.emit_done(oauth_item)
for consent_error in consent_errors:
logger.warning("Consent URL for tool '%s': %s", consent_error.name, consent_error.consent_url)
oauth_item = OAuthConsentRequestOutputItem(
id=IdGenerator.new_id("oacr"),
consent_link=consent_error.consent_url,
server_label=consent_error.name,
)
builder = response_event_stream.add_output_item(oauth_item.id)
yield builder.emit_added(oauth_item)
yield builder.emit_done(oauth_item)
yield response_event_stream.emit_completed()
return
@@ -39,6 +39,7 @@ from agent_framework_foundry_hosting import ResponsesHostServer
from agent_framework_foundry_hosting._responses import (
_AZURE_RESPONSES_MESSAGE_ROLE_TYPE, # pyright: ignore[reportPrivateUsage]
CONSENT_ERROR_CODE,
ConsentError,
FileBasedFunctionApprovalStorage, # pyright: ignore[reportPrivateUsage]
InMemoryFunctionApprovalStorage, # pyright: ignore[reportPrivateUsage]
_item_to_message, # pyright: ignore[reportPrivateUsage]
@@ -3067,7 +3068,10 @@ class TestCheckpointContextPathValidation:
# region Agent lifecycle (lazy entry & OAuth consent surfacing)
def _make_consent_error(url: str = "https://consent.example.com/auth") -> Exception:
def _make_consent_error(
url: str = "https://consent.example.com/auth",
name: str = "Foundry Toolbox",
) -> Exception:
"""Build an exception wrapping a Foundry MCP gateway consent error.
Mirrors the real-world wrapping produced by ``MCPStreamableHTTPTool.__aenter__``,
@@ -3075,17 +3079,34 @@ def _make_consent_error(url: str = "https://consent.example.com/auth") -> Except
``ToolExecutionException`` (an ``AgentFrameworkException`` subclass) with the
original error attached via ``inner_exception``. ``consent_url_from_error``
then finds the wrapped ``McpError`` in ``exc.args``.
The McpError message uses the structured Foundry MCP gateway format:
a human-readable prefix followed by a JSON document describing each
failed tool source and its consent URL.
"""
from agent_framework.exceptions import ToolExecutionException
inner = McpError(ErrorData(code=CONSENT_ERROR_CODE, message=url))
payload = json.dumps({
"errors": [
{
"name": name,
"type": "mcp",
"error": {
"code": "CONSENT_REQUIRED",
"message": url,
},
}
]
})
message = f"tools/list failed for 1 tool source(s), succeeded for 0 tool source(s) {payload}"
inner = McpError(ErrorData(code=CONSENT_ERROR_CODE, message=message))
return ToolExecutionException("MCP consent required", inner_exception=inner)
class TestConsentUrlFromError:
def test_returns_consent_url_when_inner_arg_is_consent_mcp_error(self) -> None:
exc = _make_consent_error("https://example.com/consent")
assert consent_url_from_error(exc) == "https://example.com/consent"
exc = _make_consent_error("https://example.com/consent", name="my-tool")
assert consent_url_from_error(exc) == [ConsentError(name="my-tool", consent_url="https://example.com/consent")]
def test_returns_none_when_no_mcp_error_in_args(self) -> None:
assert consent_url_from_error(Exception("boom")) is None
@@ -3102,6 +3123,13 @@ class TestConsentUrlFromError:
bare = McpError(ErrorData(code=CONSENT_ERROR_CODE, message="https://x"))
assert consent_url_from_error(bare) is None
def test_returns_none_when_message_has_no_json(self) -> None:
from agent_framework.exceptions import ToolExecutionException
inner = McpError(ErrorData(code=CONSENT_ERROR_CODE, message="no json here"))
exc = ToolExecutionException("MCP consent required", inner_exception=inner)
assert consent_url_from_error(exc) is None
class TestAgentLifecycle:
async def test_agent_entered_lazily_on_first_request(self) -> None:
+26
View File
@@ -0,0 +1,26 @@
# Mistral Package (agent-framework-mistral)
Integration with Mistral AI for embedding generation.
## Main Classes
- **`MistralEmbeddingClient`** - Embedding client for Mistral AI models
- **`MistralEmbeddingOptions`** - Options TypedDict for Mistral-specific embedding parameters
- **`MistralEmbeddingSettings`** - TypedDict settings for Mistral configuration
## Usage
```python
from agent_framework_mistral import MistralEmbeddingClient
# Requires MISTRAL_API_KEY environment variable (or pass api_key= directly)
client = MistralEmbeddingClient(model="mistral-embed")
result = await client.get_embeddings(["Hello, world!"])
print(result[0].vector)
```
## Import Path
```python
from agent_framework_mistral import MistralEmbeddingClient
```
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) Microsoft Corporation.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE
+42
View File
@@ -0,0 +1,42 @@
# Get Started with Microsoft Agent Framework Mistral AI
Please install this package:
```bash
pip install agent-framework-mistral --pre
```
and see the [README](https://github.com/microsoft/agent-framework/tree/main/python/README.md) for more information.
## Embedding Client
The `MistralEmbeddingClient` provides embedding generation using Mistral AI models.
### Quick Start
```python
from agent_framework_mistral import MistralEmbeddingClient
# Using environment variables (MISTRAL_API_KEY, MISTRAL_EMBEDDING_MODEL)
client = MistralEmbeddingClient()
# Or passing parameters directly
client = MistralEmbeddingClient(
model="mistral-embed",
api_key="your-api-key",
)
# Generate embeddings
result = await client.get_embeddings(["Hello, world!", "How are you?"])
for embedding in result:
print(f"Dimensions: {embedding.dimensions}")
print(f"Vector: {embedding.vector[:5]}...")
```
### Configuration
| Environment Variable | Description |
|---|---|
| `MISTRAL_API_KEY` | Your Mistral AI API key |
| `MISTRAL_EMBEDDING_MODEL` | Embedding model name (e.g., `mistral-embed`) |
| `MISTRAL_SERVER_URL` | Optional server URL override |
@@ -0,0 +1,17 @@
# Copyright (c) Microsoft. All rights reserved.
import importlib.metadata
from ._embedding_client import MistralEmbeddingClient, MistralEmbeddingOptions, MistralEmbeddingSettings
try:
__version__ = importlib.metadata.version(__name__)
except importlib.metadata.PackageNotFoundError:
__version__ = "0.0.0" # Fallback for development mode
__all__ = [
"MistralEmbeddingClient",
"MistralEmbeddingOptions",
"MistralEmbeddingSettings",
"__version__",
]
@@ -0,0 +1,250 @@
# Copyright (c) Microsoft. All rights reserved.
from __future__ import annotations
import logging
import sys
from collections.abc import Sequence
from typing import Any, ClassVar, Generic, TypedDict
from agent_framework import (
BaseEmbeddingClient,
Embedding,
EmbeddingGenerationOptions,
GeneratedEmbeddings,
UsageDetails,
load_settings,
)
from agent_framework._settings import SecretString
from agent_framework.observability import EmbeddingTelemetryLayer
from mistralai.client import Mistral
if sys.version_info >= (3, 13):
from typing import TypeVar # type: ignore # pragma: no cover
else:
from typing_extensions import TypeVar # type: ignore # pragma: no cover
logger = logging.getLogger("agent_framework.mistral")
class MistralEmbeddingOptions(EmbeddingGenerationOptions, total=False):
"""Mistral AI-specific embedding options.
Extends EmbeddingGenerationOptions with Mistral-specific fields.
Examples:
.. code-block:: python
from agent_framework_mistral import MistralEmbeddingOptions
options: MistralEmbeddingOptions = {
"model": "mistral-embed",
"dimensions": 1024,
}
"""
MistralEmbeddingOptionsT = TypeVar(
"MistralEmbeddingOptionsT",
bound=TypedDict, # type: ignore[valid-type]
default="MistralEmbeddingOptions",
covariant=True,
)
class MistralEmbeddingSettings(TypedDict, total=False):
"""Mistral AI embedding settings.
Fields:
api_key: Mistral API key. Resolved from ``MISTRAL_API_KEY``.
embedding_model: Embedding model name. Resolved from ``MISTRAL_EMBEDDING_MODEL``.
server_url: Optional server URL override. Resolved from ``MISTRAL_SERVER_URL``.
"""
api_key: str | None
embedding_model: str | None
server_url: str | None
class RawMistralEmbeddingClient(
BaseEmbeddingClient[str, list[float], MistralEmbeddingOptionsT],
Generic[MistralEmbeddingOptionsT],
):
"""Raw Mistral AI embedding client without telemetry.
Keyword Args:
model: The Mistral embedding model (e.g. "mistral-embed").
Can also be set via environment variable ``MISTRAL_EMBEDDING_MODEL``.
api_key: Mistral API key. Defaults to ``MISTRAL_API_KEY`` environment variable.
server_url: Optional server URL override. Defaults to ``MISTRAL_SERVER_URL``
environment variable, or the Mistral default.
client: Optional pre-configured ``Mistral`` client instance.
additional_properties: Additional properties stored on the client instance.
env_file_path: Path to ``.env`` file for settings.
env_file_encoding: Encoding for ``.env`` file.
"""
INJECTABLE: ClassVar[set[str]] = {"client"}
def __init__(
self,
*,
model: str | None = None,
api_key: str | SecretString | None = None,
server_url: str | None = None,
client: Mistral | None = None,
additional_properties: dict[str, Any] | None = None,
env_file_path: str | None = None,
env_file_encoding: str | None = None,
) -> None:
"""Initialize a raw Mistral AI embedding client."""
mistral_settings = load_settings(
MistralEmbeddingSettings,
env_prefix="MISTRAL_",
required_fields=["embedding_model", "api_key"],
api_key=str(api_key) if isinstance(api_key, SecretString) else api_key,
embedding_model=model,
server_url=server_url,
env_file_path=env_file_path,
env_file_encoding=env_file_encoding,
)
self.model: str = mistral_settings["embedding_model"] # type: ignore[assignment]
resolved_api_key: str = mistral_settings["api_key"] # type: ignore[assignment]
resolved_server_url = mistral_settings.get("server_url")
if client is not None:
self.client = client
else:
client_kwargs: dict[str, Any] = {"api_key": resolved_api_key}
if resolved_server_url:
client_kwargs["server_url"] = resolved_server_url
self.client = Mistral(**client_kwargs)
self.server_url = resolved_server_url
super().__init__(additional_properties=additional_properties)
def service_url(self) -> str:
"""Get the URL of the service."""
return self.server_url or "https://api.mistral.ai"
async def get_embeddings(
self,
values: Sequence[str],
*,
options: MistralEmbeddingOptionsT | None = None,
) -> GeneratedEmbeddings[list[float], MistralEmbeddingOptionsT]:
"""Call the Mistral AI embeddings API.
Args:
values: The text values to generate embeddings for.
options: Optional embedding generation options.
Returns:
Generated embeddings with usage metadata.
Raises:
ValueError: If model is not provided or values is empty.
"""
if not values:
return GeneratedEmbeddings([], options=options)
opts: dict[str, Any] = options or {} # type: ignore
model = opts.get("model") or self.model
if not model:
raise ValueError("model is required")
kwargs: dict[str, Any] = {"model": model, "inputs": list(values)}
if "dimensions" in opts:
kwargs["output_dimension"] = opts["dimensions"]
response = await self.client.embeddings.create_async(**kwargs)
embeddings: list[Embedding[list[float]]] = []
if response and response.data:
items = sorted(response.data, key=lambda d: d.index if d.index is not None else 0)
for item in items:
vector = list(item.embedding) if item.embedding else []
embeddings.append(
Embedding(
vector=vector,
dimensions=len(vector),
model=response.model or model,
)
)
usage_dict: UsageDetails | None = None
if response and response.usage:
usage_dict = {
"input_token_count": response.usage.prompt_tokens,
"total_token_count": response.usage.total_tokens,
}
return GeneratedEmbeddings(embeddings, options=options, usage=usage_dict)
class MistralEmbeddingClient(
EmbeddingTelemetryLayer[str, list[float], MistralEmbeddingOptionsT],
RawMistralEmbeddingClient[MistralEmbeddingOptionsT],
Generic[MistralEmbeddingOptionsT],
):
"""Mistral AI embedding client with telemetry support.
Keyword Args:
model: The Mistral embedding model (e.g. "mistral-embed").
Can also be set via environment variable ``MISTRAL_EMBEDDING_MODEL``.
api_key: Mistral API key. Defaults to ``MISTRAL_API_KEY`` environment variable.
server_url: Optional server URL override. Defaults to ``MISTRAL_SERVER_URL``
environment variable, or the Mistral default.
client: Optional pre-configured ``Mistral`` client instance.
otel_provider_name: Optional telemetry provider name override.
env_file_path: Path to ``.env`` file for settings.
env_file_encoding: Encoding for ``.env`` file.
Examples:
.. code-block:: python
from agent_framework_mistral import MistralEmbeddingClient
# Using environment variables
# Set MISTRAL_API_KEY=your-key
# Set MISTRAL_EMBEDDING_MODEL=mistral-embed
client = MistralEmbeddingClient()
# Or passing parameters directly
client = MistralEmbeddingClient(
model="mistral-embed",
api_key="your-api-key",
)
# Generate embeddings
result = await client.get_embeddings(["Hello, world!"])
print(result[0].vector)
"""
OTEL_PROVIDER_NAME: ClassVar[str] = "mistralai"
def __init__(
self,
*,
model: str | None = None,
api_key: str | SecretString | None = None,
server_url: str | None = None,
client: Mistral | None = None,
otel_provider_name: str | None = None,
additional_properties: dict[str, Any] | None = None,
env_file_path: str | None = None,
env_file_encoding: str | None = None,
) -> None:
"""Initialize a Mistral AI embedding client."""
super().__init__(
model=model,
api_key=api_key,
server_url=server_url,
client=client,
additional_properties=additional_properties,
otel_provider_name=otel_provider_name,
env_file_path=env_file_path,
env_file_encoding=env_file_encoding,
)
@@ -0,0 +1 @@
+105
View File
@@ -0,0 +1,105 @@
[project]
name = "agent-framework-mistral"
description = "Mistral AI integration for Microsoft Agent Framework."
authors = [{ name = "Microsoft", email = "af-support@microsoft.com"}]
readme = "README.md"
requires-python = ">=3.10"
version = "1.0.0a260505"
license-files = ["LICENSE"]
urls.homepage = "https://learn.microsoft.com/en-us/agent-framework/"
urls.source = "https://github.com/microsoft/agent-framework/tree/main/python"
urls.release_notes = "https://github.com/microsoft/agent-framework/releases?q=tag%3Apython-1&expanded=true"
urls.issues = "https://github.com/microsoft/agent-framework/issues"
classifiers = [
"License :: OSI Approved :: MIT License",
"Development Status :: 3 - Alpha",
"Intended Audience :: Developers",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Framework :: Pydantic :: 2",
"Typing :: Typed",
]
dependencies = [
"agent-framework-core>=1.1.0,<2",
"mistralai>=2.0.0,<3",
]
[tool.uv]
prerelease = "if-necessary-or-explicit"
environments = [
"sys_platform == 'darwin'",
"sys_platform == 'linux'",
"sys_platform == 'win32'"
]
[tool.uv-dynamic-versioning]
fallback-version = "0.0.0"
[tool.pytest.ini_options]
testpaths = 'tests'
addopts = "-ra -q -r fEX"
asyncio_mode = "auto"
asyncio_default_fixture_loop_scope = "function"
filterwarnings = []
markers = [
"integration: marks tests as integration tests that require external services",
]
timeout = 120
[tool.ruff]
line-length = 120
[tool.ruff.lint]
select = ["E", "F", "I", "N", "W"]
[tool.coverage.run]
omit = [
"**/__init__.py"
]
[tool.pyright]
extends = "../../pyproject.toml"
include = ["agent_framework_mistral"]
exclude = ['tests']
[tool.mypy]
plugins = ['pydantic.mypy']
strict = true
python_version = "3.10"
ignore_missing_imports = true
disallow_untyped_defs = true
no_implicit_optional = true
check_untyped_defs = true
warn_return_any = true
show_error_codes = true
warn_unused_ignores = false
disallow_incomplete_defs = true
disallow_untyped_decorators = true
disallow_any_unimported = true
[tool.bandit]
targets = ["agent_framework_mistral"]
exclude_dirs = ["tests"]
[tool.poe]
executor.type = "uv"
include = "../../shared_tasks.toml"
[tool.poe.tasks.mypy]
help = "Run MyPy for this package."
cmd = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_mistral"
[tool.poe.tasks.test]
help = "Run the default unit test suite for this package."
cmd = 'pytest -m "not integration" --cov=agent_framework_mistral --cov-report=term-missing:skip-covered tests'
[tool.uv.build-backend]
module-name = "agent_framework_mistral"
module-root = ""
[build-system]
requires = ["uv_build>=0.8.2,<0.9.0"]
build-backend = "uv_build"
+15
View File
@@ -0,0 +1,15 @@
# Mistral AI Embedding Examples
This folder contains examples demonstrating how to use Mistral AI embedding models with the Agent Framework.
## Examples
| File | Description |
|------|-------------|
| [`mistral_embeddings.py`](mistral_embeddings.py) | Basic embedding generation with the Mistral AI embedding client. |
## Environment Variables
- `MISTRAL_API_KEY`: Your Mistral AI API key
- `MISTRAL_EMBEDDING_MODEL`: Embedding model name (e.g., `mistral-embed`)
- `MISTRAL_SERVER_URL` (optional): Server URL override for custom deployments
@@ -0,0 +1 @@
# Copyright (c) Microsoft. All rights reserved.
@@ -0,0 +1,77 @@
# Copyright (c) Microsoft. All rights reserved.
"""Shows how to generate embeddings using the Mistral AI embedding client.
Requires ``MISTRAL_API_KEY`` and ``MISTRAL_EMBEDDING_MODEL`` environment variables.
"""
import asyncio
from dotenv import load_dotenv
from agent_framework_mistral import MistralEmbeddingClient
load_dotenv()
async def basic_embedding_example() -> None:
"""Generate embeddings for a list of texts."""
print("=== Basic Embedding Generation ===")
# 1. Create the embedding client (uses MISTRAL_API_KEY and MISTRAL_EMBEDDING_MODEL env vars).
client = MistralEmbeddingClient()
# 2. Generate embeddings for multiple texts.
texts = ["Hello, world!", "How are you?", "Agent Framework with Mistral AI"]
result = await client.get_embeddings(texts)
# 3. Print results.
print(f"Generated {len(result)} embeddings")
for i, embedding in enumerate(result):
print(f" Text {i + 1}: dimensions={embedding.dimensions}, vector={embedding.vector[:5]}...")
if result.usage:
print(
f" Usage: {result.usage['input_token_count']} input tokens, "
f"{result.usage['total_token_count']} total tokens"
)
async def embedding_with_options_example() -> None:
"""Generate embeddings with custom dimensions."""
print("\n=== Embedding with Custom Dimensions ===")
from agent_framework_mistral import MistralEmbeddingOptions
client = MistralEmbeddingClient()
# Request a specific output dimension (model must support it).
options: MistralEmbeddingOptions = {"dimensions": 256}
result = await client.get_embeddings(["Dimensionality reduction example"], options=options)
print(f" Dimensions: {result[0].dimensions}")
print(f" Vector (first 5): {result[0].vector[:5]}...")
async def main() -> None:
"""Run embedding examples."""
await basic_embedding_example()
await embedding_with_options_example()
if __name__ == "__main__":
asyncio.run(main())
"""
Sample output:
=== Basic Embedding Generation ===
Generated 3 embeddings
Text 1: dimensions=1024, vector=[0.0123, -0.0456, 0.0789, -0.0012, 0.0345]...
Text 2: dimensions=1024, vector=[0.0234, -0.0567, 0.0891, -0.0023, 0.0456]...
Text 3: dimensions=1024, vector=[0.0345, -0.0678, 0.0912, -0.0034, 0.0567]...
Usage: 15 input tokens, 15 total tokens
=== Embedding with Custom Dimensions ===
Dimensions: 256
Vector (first 5): [0.0456, -0.0789, 0.0123, -0.0456, 0.0789]...
"""
@@ -0,0 +1,267 @@
# Copyright (c) Microsoft. All rights reserved.
import os
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from agent_framework import Embedding, GeneratedEmbeddings
from agent_framework_mistral import MistralEmbeddingClient, MistralEmbeddingOptions
# region: Unit Tests
def test_mistral_embedding_construction(monkeypatch: pytest.MonkeyPatch) -> None:
"""Test construction with environment variables."""
monkeypatch.setenv("MISTRAL_EMBEDDING_MODEL", "mistral-embed")
monkeypatch.setenv("MISTRAL_API_KEY", "test-key")
with patch("agent_framework_mistral._embedding_client.Mistral") as mock_cls:
mock_cls.return_value = MagicMock()
client = MistralEmbeddingClient()
assert client.model == "mistral-embed"
def test_mistral_embedding_construction_with_params() -> None:
"""Test construction with explicit parameters."""
with patch("agent_framework_mistral._embedding_client.Mistral") as mock_cls:
mock_cls.return_value = MagicMock()
client = MistralEmbeddingClient(
model="mistral-embed",
api_key="test-key",
)
assert client.model == "mistral-embed"
mock_cls.assert_called_once_with(api_key="test-key")
def test_mistral_embedding_construction_with_server_url() -> None:
"""Test construction with custom server URL."""
with patch("agent_framework_mistral._embedding_client.Mistral") as mock_cls:
mock_cls.return_value = MagicMock()
client = MistralEmbeddingClient(
model="mistral-embed",
api_key="test-key",
server_url="https://custom.mistral.ai",
)
assert client.model == "mistral-embed"
assert client.server_url == "https://custom.mistral.ai"
mock_cls.assert_called_once_with(
api_key="test-key",
server_url="https://custom.mistral.ai",
)
def test_mistral_embedding_construction_with_client() -> None:
"""Test construction with a pre-configured client."""
mock_client = MagicMock()
with patch("agent_framework_mistral._embedding_client.Mistral"):
client = MistralEmbeddingClient(
model="mistral-embed",
api_key="test-key",
client=mock_client,
)
assert client.client is mock_client
def test_mistral_embedding_construction_missing_model_raises(monkeypatch: pytest.MonkeyPatch) -> None:
"""Test that missing model raises an error."""
monkeypatch.delenv("MISTRAL_EMBEDDING_MODEL", raising=False)
monkeypatch.setenv("MISTRAL_API_KEY", "test-key")
from agent_framework.exceptions import SettingNotFoundError
with pytest.raises(SettingNotFoundError):
MistralEmbeddingClient()
def test_mistral_embedding_construction_missing_api_key_raises(monkeypatch: pytest.MonkeyPatch) -> None:
"""Test that missing API key raises an error."""
monkeypatch.delenv("MISTRAL_API_KEY", raising=False)
monkeypatch.setenv("MISTRAL_EMBEDDING_MODEL", "mistral-embed")
from agent_framework.exceptions import SettingNotFoundError
with pytest.raises(SettingNotFoundError):
MistralEmbeddingClient()
def test_mistral_embedding_service_url() -> None:
"""Test service_url returns the correct URL."""
with patch("agent_framework_mistral._embedding_client.Mistral") as mock_cls:
mock_cls.return_value = MagicMock()
client = MistralEmbeddingClient(
model="mistral-embed",
api_key="test-key",
)
assert client.service_url() == "https://api.mistral.ai"
def test_mistral_embedding_service_url_custom() -> None:
"""Test service_url returns custom URL when set."""
with patch("agent_framework_mistral._embedding_client.Mistral") as mock_cls:
mock_cls.return_value = MagicMock()
client = MistralEmbeddingClient(
model="mistral-embed",
api_key="test-key",
server_url="https://custom.mistral.ai",
)
assert client.service_url() == "https://custom.mistral.ai"
async def test_mistral_embedding_get_embeddings() -> None:
"""Test generating embeddings via the Mistral API."""
mock_response = MagicMock()
mock_response.data = [
MagicMock(embedding=[0.1, 0.2, 0.3], index=0, object="embedding"),
MagicMock(embedding=[0.4, 0.5, 0.6], index=1, object="embedding"),
]
mock_response.model = "mistral-embed"
mock_response.usage = MagicMock(prompt_tokens=10, total_tokens=10)
with patch("agent_framework_mistral._embedding_client.Mistral") as mock_cls:
mock_client = MagicMock()
mock_client.embeddings = MagicMock()
mock_client.embeddings.create_async = AsyncMock(return_value=mock_response)
mock_cls.return_value = mock_client
client = MistralEmbeddingClient(model="mistral-embed", api_key="test-key")
result = await client.get_embeddings(["hello", "world"])
assert isinstance(result, GeneratedEmbeddings)
assert len(result) == 2
assert result[0].vector == [0.1, 0.2, 0.3]
assert result[1].vector == [0.4, 0.5, 0.6]
assert result[0].model == "mistral-embed"
assert result.usage == {"input_token_count": 10, "total_token_count": 10}
mock_client.embeddings.create_async.assert_called_once_with(
model="mistral-embed",
inputs=["hello", "world"],
)
async def test_mistral_embedding_get_embeddings_empty_input() -> None:
"""Test generating embeddings with empty input."""
with patch("agent_framework_mistral._embedding_client.Mistral") as mock_cls:
mock_client = MagicMock()
mock_cls.return_value = mock_client
client = MistralEmbeddingClient(model="mistral-embed", api_key="test-key")
result = await client.get_embeddings([])
assert isinstance(result, GeneratedEmbeddings)
assert len(result) == 0
async def test_mistral_embedding_get_embeddings_with_dimensions() -> None:
"""Test generating embeddings with custom dimensions option."""
mock_response = MagicMock()
mock_response.data = [
MagicMock(embedding=[0.1, 0.2], index=0, object="embedding"),
]
mock_response.model = "mistral-embed"
mock_response.usage = MagicMock(prompt_tokens=5, total_tokens=5)
with patch("agent_framework_mistral._embedding_client.Mistral") as mock_cls:
mock_client = MagicMock()
mock_client.embeddings = MagicMock()
mock_client.embeddings.create_async = AsyncMock(return_value=mock_response)
mock_cls.return_value = mock_client
client = MistralEmbeddingClient(model="mistral-embed", api_key="test-key")
options: MistralEmbeddingOptions = {"dimensions": 512}
result = await client.get_embeddings(["hello"], options=options)
assert len(result) == 1
mock_client.embeddings.create_async.assert_called_once_with(
model="mistral-embed",
inputs=["hello"],
output_dimension=512,
)
async def test_mistral_embedding_get_embeddings_no_model_raises() -> None:
"""Test that missing model at call time raises ValueError."""
with patch("agent_framework_mistral._embedding_client.Mistral") as mock_cls:
mock_client = MagicMock()
mock_cls.return_value = mock_client
client = MistralEmbeddingClient(model="mistral-embed", api_key="test-key")
client.model = None # type: ignore[assignment]
with pytest.raises(ValueError, match="model is required"):
await client.get_embeddings(["hello"])
async def test_mistral_embedding_get_embeddings_model_override() -> None:
"""Test that model can be overridden via options."""
mock_response = MagicMock()
mock_response.data = [
MagicMock(embedding=[0.1, 0.2, 0.3], index=0, object="embedding"),
]
mock_response.model = "custom-embed"
mock_response.usage = MagicMock(prompt_tokens=5, total_tokens=5)
with patch("agent_framework_mistral._embedding_client.Mistral") as mock_cls:
mock_client = MagicMock()
mock_client.embeddings = MagicMock()
mock_client.embeddings.create_async = AsyncMock(return_value=mock_response)
mock_cls.return_value = mock_client
client = MistralEmbeddingClient(model="mistral-embed", api_key="test-key")
options: MistralEmbeddingOptions = {"model": "custom-embed"}
result = await client.get_embeddings(["hello"], options=options)
assert len(result) == 1
assert result[0].model == "custom-embed"
mock_client.embeddings.create_async.assert_called_once_with(
model="custom-embed",
inputs=["hello"],
)
async def test_mistral_embedding_get_embeddings_no_usage() -> None:
"""Test handling response without usage information."""
mock_response = MagicMock()
mock_response.data = [
MagicMock(embedding=[0.1, 0.2, 0.3], index=0, object="embedding"),
]
mock_response.model = "mistral-embed"
mock_response.usage = None
with patch("agent_framework_mistral._embedding_client.Mistral") as mock_cls:
mock_client = MagicMock()
mock_client.embeddings = MagicMock()
mock_client.embeddings.create_async = AsyncMock(return_value=mock_response)
mock_cls.return_value = mock_client
client = MistralEmbeddingClient(model="mistral-embed", api_key="test-key")
result = await client.get_embeddings(["hello"])
assert len(result) == 1
assert result.usage is None
# region: Integration Tests
skip_if_mistral_embedding_integration_tests_disabled = pytest.mark.skipif(
os.getenv("MISTRAL_EMBEDDING_MODEL", "") in ("", "test-model") or os.getenv("MISTRAL_API_KEY", "") == "",
reason="No real Mistral embedding model or API key provided; skipping integration tests.",
)
@pytest.mark.flaky
@pytest.mark.integration
@skip_if_mistral_embedding_integration_tests_disabled
async def test_mistral_embedding_integration() -> None:
"""Integration test for Mistral AI embedding client."""
client = MistralEmbeddingClient()
result = await client.get_embeddings(["Hello, world!", "How are you?"])
assert isinstance(result, GeneratedEmbeddings)
assert len(result) == 2
for embedding in result:
assert isinstance(embedding, Embedding)
assert isinstance(embedding.vector, list)
assert len(embedding.vector) > 0
assert all(isinstance(v, float) for v in embedding.vector)
assert result.usage is not None
assert result.usage["input_token_count"] is not None
assert result.usage["input_token_count"] > 0
+5 -1
View File
@@ -54,7 +54,9 @@ package = false
prerelease = "if-necessary-or-explicit"
# Security floors for transitive deps; overrides bypass litellm[proxy]'s strict pins.
constraint-dependencies = ["litellm>=1.83.7", "fastapi-sso>=0.19.0"]
override-dependencies = ["mcp[ws]>=1.27.0", "uvicorn[standard]>=0.34.0"]
# Allow opentelemetry-semantic-conventions 0.61b0 for mistralai compatibility
# (mistralai pins <0.61 but 0.61b0 is compatible at runtime).
override-dependencies = ["mcp[ws]>=1.27.0", "uvicorn[standard]>=0.34.0", "opentelemetry-semantic-conventions>=0.60b1"]
environments = [
"sys_platform == 'darwin'",
"sys_platform == 'linux'",
@@ -88,6 +90,7 @@ agent-framework-github-copilot = { workspace = true }
agent-framework-hyperlight = { workspace = true }
agent-framework-lab = { workspace = true }
agent-framework-mem0 = { workspace = true }
agent-framework-mistral = { workspace = true }
agent-framework-monty = { workspace = true }
agent-framework-ollama = { workspace = true }
agent-framework-openai = { workspace = true }
@@ -211,6 +214,7 @@ executionEnvironments = [
{ root = "packages/lab/lightning/tests", reportPrivateUsage = "none" },
{ root = "packages/lab/tau2/tests", reportPrivateUsage = "none" },
{ root = "packages/mem0/tests", reportPrivateUsage = "none" },
{ root = "packages/mistral/tests", reportPrivateUsage = "none" },
{ root = "packages/ollama/tests", reportPrivateUsage = "none" },
{ root = "packages/orchestrations/tests", reportPrivateUsage = "none" },
{ root = "packages/purview/tests", reportPrivateUsage = "none" },
+54
View File
@@ -0,0 +1,54 @@
# A2A Client Samples
These samples demonstrate how to **consume** remote A2A-compliant agents using the Agent Framework's `A2AAgent` class.
For hosting your own agents as A2A servers, see [`samples/04-hosting/a2a/`](../../04-hosting/a2a/).
## Samples
| Sample | Concept |
|--------|---------|
| [`agent_with_a2a.py`](agent_with_a2a.py) | Basic consumption — non-streaming and streaming |
| [`a2a_agent_as_function_tools.py`](a2a_agent_as_function_tools.py) | Expose A2A skills as function tools for a host agent |
| [`a2a_polling.py`](a2a_polling.py) | Poll a long-running task with continuation tokens |
| [`a2a_stream_reconnection.py`](a2a_stream_reconnection.py) | Resume an interrupted stream via continuation token |
| [`a2a_protocol_selection.py`](a2a_protocol_selection.py) | Configure preferred protocol bindings (JSONRPC, GRPC, HTTP+JSON) |
## Prerequisites
- A running A2A-compliant agent server (see `samples/04-hosting/a2a/` to start one)
- Set `A2A_AGENT_HOST` environment variable to the server URL
- For `a2a_agent_as_function_tools.py`: also set `FOUNDRY_PROJECT_ENDPOINT` and `FOUNDRY_MODEL`
## Running
```bash
cd python/samples/02-agents/a2a
# Start an A2A server in another terminal first:
# cd python/samples/04-hosting/a2a && uv run python a2a_server.py
export A2A_AGENT_HOST="http://localhost:5001/"
uv run python agent_with_a2a.py
```
## Key APIs
```python
from agent_framework.a2a import A2AAgent
# Connect to a remote agent
async with A2AAgent(url="http://localhost:5001/", agent_card=card) as agent:
# Non-streaming
response = await agent.run("Hello")
# Streaming
stream = agent.run("Hello", stream=True)
async for update in stream:
print(update.text)
# Background + polling
response = await agent.run("Long task", background=True)
while response.continuation_token:
response = await agent.poll_task(response.continuation_token)
```
@@ -33,7 +33,7 @@ Prerequisites:
- Set FOUNDRY_MODEL to the model deployment name (e.g. gpt-4o)
To run this sample:
cd python/samples/04-hosting/a2a
cd python/samples/02-agents/a2a
uv run python a2a_agent_as_function_tools.py
"""
@@ -0,0 +1,96 @@
# Copyright (c) Microsoft. All rights reserved.
import asyncio
import os
import httpx
from a2a.client import A2ACardResolver
from agent_framework.a2a import A2AAgent
from dotenv import load_dotenv
load_dotenv()
"""
A2A Polling for Task Completion
This sample demonstrates how to poll a long-running A2A task for completion
using continuation tokens. When `background=True`, the agent returns immediately
with a continuation token that you can use to check progress later.
Key concepts demonstrated:
- Starting a background A2A task with `background=True`
- Receiving a continuation token for in-progress tasks
- Polling with `poll_task()` until the task reaches a terminal state
This is the A2A equivalent of the .NET A2AAgent_PollingForTaskCompletion sample.
Prerequisites:
- Set A2A_AGENT_HOST to the URL of a running A2A server
To run this sample:
cd python/samples/02-agents/a2a
uv run python a2a_polling.py
"""
async def main() -> None:
"""Demonstrates polling a long-running A2A task for completion."""
a2a_agent_host = os.getenv("A2A_AGENT_HOST")
if not a2a_agent_host:
raise ValueError("A2A_AGENT_HOST environment variable is not set")
# 1. Resolve agent card and create agent.
async with httpx.AsyncClient(timeout=60.0) as http_client:
resolver = A2ACardResolver(httpx_client=http_client, base_url=a2a_agent_host)
agent_card = await resolver.get_agent_card()
async with A2AAgent(
name=agent_card.name,
agent_card=agent_card,
url=a2a_agent_host,
) as agent:
# 2. Start a background task — the agent returns immediately.
print("Starting background task...")
response = await agent.run(
"Write a detailed research report on quantum computing advances in 2025",
background=True,
)
# 3. Check if we got a continuation token (task still in progress).
if response.continuation_token is None:
# Task completed immediately — no polling needed.
print("Task completed immediately:")
print(f" {response.text}")
return
# 4. Poll until the task completes.
token = response.continuation_token
poll_count = 0
while token is not None:
poll_count += 1
print(f" Poll #{poll_count} — task still in progress, waiting 2s...")
await asyncio.sleep(2)
response = await agent.poll_task(token) # type: ignore[arg-type]
token = response.continuation_token
# 5. Task is done — print the final response.
print(f"\nTask completed after {poll_count} poll(s):")
print(f" {response.text[:200]}...")
if __name__ == "__main__":
asyncio.run(main())
"""
Sample output:
Starting background task...
Poll #1 — task still in progress, waiting 2s...
Poll #2 — task still in progress, waiting 2s...
Poll #3 — task still in progress, waiting 2s...
Task completed after 3 poll(s):
Quantum computing has seen remarkable progress in 2025, with breakthroughs in...
"""

Some files were not shown because too many files have changed in this diff Show More