* test: Split out Handoff Orchestration tests
* fix: Synthesized Handoff FunctionResult is never sent to agent
When we receive a handoff request from the agent, we need to service it outside of the Agent Loop to terminate the loop. What this means is that we take ownership of terminating the call by feeding the result back into the agent on a subsequent invocation.
When we refactored Handoff to support HITL and make use of AgentSession, we inadvertantly removed this step, causing subsequent invocations to the Handoff agent to fail (first works, but breaks the state).
The fix is to be more precise about the agent's bookmark when concatenating the result of agent invocation to the shared conversation history.
* test: Add unit tests for Handoff FunctionCall/Result matching fix
When MultiPartyConversation gets saved during checkpointing, the data for the chat history is not persisted, resulting in failures to deserialize after. The fix is to make the history visible to the source generated serialization code.
* refactor: remove dead code
* refactor: remove ignore YieldsMessageAttribute
- the correct one to use is YieldsOutputAttribute
- fixes a comment that mistakenly refers to `.YieldsMessage()` which does not exist.
* fix: ChatForwardingExecutor does not use correct role for string messages
- make ChatForwardingExecutor use its configured role for string messages rather than always use ChatRole.User
- add ChatForwardingExecutor tests
* fixup: remove unused attribute
* test: Add tests for failure when .AsAgent used on a non-ChatProtocol workflow
* test: Add FunctionExecutor tests
- also fixes Send and YieldOutput type registration for synchronous output-returning delegates
* test: Suppress CodeCoverage for obsolete names
* fix: Re-add Obsolete attributes
- avoid hard-breaking change
- properly notify users that these attributes get ignored
Some providers, e.g. Gemini, do not use the CallId mechanism to disambiguate simultaneous function calls. This can result in message lists containing multiple turn to fail to filter properly.
The fix is to take advantage of the expectation that Handoff Orchestration is a "single-speaker" flow, which only has a single active AIAgent per "turn" and an agent's turn is not finished until all outstanding function calls are finished.
This allows us to expect that any ambiguous-CallId FunctionCallContent are either in separate turns or will have had a response before the next issued call with the same Id.
* fix: Add session support for Handoff-hosted Agents
In order to better support using `Workflows` hosted as `AIAgents` inside of Handoff workflows, we need to make proper use of AgentSession. This causes potential issues around checkpointing and making sure that we properly compute only the new incoming messages for each agent invocation.
* fix: AgentSession checkpointing using AIAgent's Serialize/Deserialize methods
We cannot rely on implicit serialization through `HandoffHostState` because we are missing type information.
* fix: Thread safety issue in `MultiPartyConversation.AllMessages`
* fix: Enable unwrapping of FunctionResultContent when ExternalRequest was wrapped into FunctionCallContent
* fix: Foundry Agents without description in Handoff
Foundry Agents without a description set will return an empty string (rather than null) for the description. This was breaking the fallback logic for `handoffReason`.
* test: Add unit tests
* feat: Refactor Handoff Orchestration and add HITL support
* Change HandoffAgentExecutor to use factory-based instantiation
* Extract shared request collection logic in AIAgentUnservicedRequestsCollector
* Refactor HandoffAgentExecutor to use the "ContinueTurn" pattern as in AIAgentHostExecutor
* fix: Remove '$' from exception strings
* fix: Concurrent Workflow Sample
* Switch to using Azure AI Projects APIs
* Remove agent streaming outputs by changing emitEvents to false on TurnToken
* Disable forwarding input from agent host executors
* Make output format more legible
* refactor: Update Concurrent sample to use message delivery event callback
* fix: Remove Timeout from InputWait in StreamingRunEventStream
* fix: Race condition when the workflow executes to halt before TakeEventStream
* test: Make the OffThread Delay test more nimble
* fix: Remove slight window where runStatus could be stale
* feat: Implement return-to-previous routing in handoff workflow
- Also obsoletes HandoffsWorkflowBuilder => HandoffWorkflowBuilder (no "s")
* refactor: Remove instance-shared current agent tracking in handoffs
Because the tracker was instance-shared between the start and end executors, it would be shared between all sessions, resulting in incorrect behaviour.
The corect way to do this is to keep the data in a shared executor scope, which is per-session.
* fix: Fix test logic for Handoff to correctly use checkpointing for multiturn
* fix: HandoffAgentExecutor does not output any reponse when non-streaming
* fix: Ensure Workflow outputs persisted in chat history when hosted AsAgent
* fix: Remove duplicate history entry creation and ad test
* test: Add streaming tests for AsAgent to smoke tests
* feat: Add output configurability to Handoffs
* refactor: [BREAKING] Config => ExecutorConfig
Make the Config name less likely to collide with other classes by renaming to ExecutorConfig. Makes Configured and related classes internal as they do not need to be part of the public surface.
* fix: Make RouteBuilder explicit in SourceGen to avoid conflicts
The `sessionId`, an optional parameter when starting a new session when
running a workflow is an arbitrary string. This allows consumers to
support whatever ids are needed by other systems, but can result in
errors when an OS special or forbidden character is included.
The fix is to escape the paths, in a 1:1 manner. We rely on
EncodeDataString to do this.
* Also modifies the index file to make it easier to determine what the
name of the file on disk is for a given `sessionId`.
* refactor: Normalize Run/RunStreaming with AIAgent
* refactor: Clarify Session vs. Run -level concepts
* Rename RunId to SessionId to better match Run/Session terminology in AIAgent
* [BREAKING]: Will break existing checkpointed sessions in CosmosDb due to field rename
* refactor: Rename and simplify interface around getting typed data out of ExternalRequest/Response
* Also adds hints around using value types in PortableValue
* refactor: Rename AddFanInEdge to AddFanInBarrierEdge
This will prevent a breaking change later when we introduce a programmable FanIn edge, analogous to the FanOut edge's EdgeSelector.
The goal, in the long run is to support a number of different FanIn scenarios, with naive FanIn (no barrier) by default, similar to FanOut.
* refactor: AsAgent(this Workflow, ...) => AsAIAgent(...)
* misc - part1: SwitchBuilder internal
---------
Co-authored-by: Dmytro Struk <13853051+dmytrostruk@users.noreply.github.com>
* Fix handoff orchestration not passing user message to handoff target agent (#3161)
Filter out internal handoff function call and tool result messages before
passing conversation history to the target agent's LLM. These messages
confused the model into ignoring the original user question.
* Add handoff tool call filtering behavior and enhance workflow builder
- Introduced HandoffToolCallFilteringBehavior enum to specify filtering behavior for tool call contents in handoff workflows.
- Updated HandoffsWorkflowBuilder to support customizable handoff instructions and tool call filtering behavior.
- Enhanced HandoffAgentExecutor to utilize new filtering options for improved message handling during agent handoffs.
* Enhance handoff message filtering logic and add unit tests for filtering behaviors
* Refactor HandoffMessagesFilter to remove unused handoff function names and enhance filtering logic for non-handoff function calls
* Refactor HandoffMessagesFilter to streamline FilterCandidateState initialization and improve clarity
* Refactor HandoffMessagesFilter to improve filtering logic and add integration tests for handoff workflows
* fix: HandoffAgentExecutor tests
* [BREAKING] refactor: Decouple Checkpointing and Execution APIs
With this change, Checkpointing becomes an property of an IWorkflowExecutionEnvironment. This lets environments that are tightly-coupled to their CheckpointManager avoid needing to present APIs that would not work (e.g. taking in an InMemory CheckpointManager for Durable Tasks, for example)
* refactor: Normalize IsCheckpointingEnabled naming
Track the last CheckpointInfo in InProcessRunner so that newly created
checkpoints reference their parent. When resuming from a checkpoint,
the resumed-from checkpoint becomes the parent of the next checkpoint.
Adds tests verifying:
- First checkpoint has null parent
- Subsequent checkpoints chain parents correctly
- Checkpoint after resume references the resumed-from checkpoint
* feat: Implement Polymorphic Routing
* feat: Add support for Send/Yield annotations with basic Executor
* Adds annotations to Declarative workflow executors
* fix: Address PR Comments
* Implicit filter in collection loops
* Remove debug / usused / superfluous code
* Fix ProtocolBuilder implicit output registrations
* Fix logic error in ExecuteRouteGeneratorTests.ClassWithManualConfigureProtocol_DoesNotGenerate
* fix: Solidify type checks and send/yield type registrations
* fix: Suppress generation of TurnTokens out of AggregateTurnMessagesExecutor
* Fixes an issue where ConcurrentEndExecutor is not expecting TurnTokens.
* fix: Add ProtocolBuilder support for chained-delegation
* Updates Declarative pacakge to rely on chained-delegation Send/Yield registration
* Renames DeclarativeActionExectuor's new ExecuteAsync to ExecuteActionAsync to avoid colliding with Executor.ExecutoeAsync
* fix: Address PR Comments
* Fixes type mapping in FanInEdgeRunner
* Fixes and expalins send/yield type registration in FunctionExecutor
* fixup: build-break
* fix: Add missing SendsMesage declaration to InvokeAzureAgentExecutor
* Rename WorkflowOutputEvent.SourceId to ExecutorId for Python consistency
- Rename SourceId property to ExecutorId in WorkflowOutputEvent
- Add [Obsolete] SourceId property for backward compatibility
- Update all test usages to use ExecutorId
Resolves part of #2938
* Unify AgentResponse events with WorkflowOutputEvent (#2938)
- Change AgentResponseEvent and AgentResponseUpdateEvent to inherit from
WorkflowOutputEvent instead of ExecutorEvent
- Update AIAgentHostExecutor and HandoffAgentExecutor to use YieldOutputAsync()
instead of AddEventAsync() for agent outputs
- Add special-casing in InProcessRunnerContext.YieldOutputAsync() to create
specific event types for AgentResponse and AgentResponseUpdate, bypassing
OutputFilter for backwards compatibility
- Update TestRunContext and TestWorkflowContext with same special-casing
- Add regression tests in AgentEventsTests
* refactor: Seal AgentResponse events
Adds tests documenting current shared state behavior in subworkflows:
- State works correctly within a subworkflow
- State is isolated across parent/subworkflow boundaries
Related to #2419
* refactor: Rename AggregateTurnMessagesExecutor
* feat: Rework Agent Hosting for Configurability and HIL support
* Adds support for selecting whether updates and/or full responses are
emitted to events
* Adds support for HIL/FunctionCalls (including interception)
* Implements internal support for ExternalRequests from any executor
(not just RequestPort)
* test: Add tests for new AIAgentHostExecutor functionality
* feat: Unify non-Handoff Agent Hosting
* doc: More explicit documentation for `overwrite` in RouteBuilder
Subworkflows run into issues with Checkpointing and the Chat Protocol:
* The concurrency rework made subtle changes in behaviour that introduced a hang when using subworkflows with ChatProtocol and streaming execution.
* The ResetAsync() implementation in WorkflowHostExecutor was improperly resetting the joinContext - this was happening on restore checkpoint _after_ the join context was attached when
* Subworkflows cannot be used as the start node when hosted AsAgent due to inability to treat Catch-All as a Chat Protocol
* Subworkflow ownership issue when used in non-concurrent mode after finishing a run
Also fixes:
* When ChatMessages are output by executors that are not agents, there is no corresponding AgentResponseUpdate/AgentResponse event
Breaking Changes
* [BREAKING CHANGE] It is possible to provide the wrong RunId when resuming from CheckpointInfo (even though the data already exists on CheckpointInfo)
* fix: Expose WorkflowErrorEvent as ErrorContent
When hosted using .AsAgent(), Workflows were not exposing inner errors coming as Exceptions (through the WorkflowErrorEvent)
The fix is to convert their message to an ErrorContent on the way out, rather than rely on the default "empty update" to collect the raw event.
* feat: Add a way to show/suppress exception information
* fix: WorkflowAsAgent Sample
* Also makes ChatForwardingExecutor public
* feat: Expand ChatForwardingExecutor handled types
Make ChatForwardingExecutor match the input types of ChatProtocolExecutor.
* fix: Update for the new AgentRunResponseUpdate merge logic
AIAgent always sends out List<ChatMessage> now.
* fix: Workflow Validation
* adds orphan validation to workflow builder
* adds tests for workflow validation
* expands on the underlying reasoning why type validation is not supported
* fixup: CodeGen template
* [BREAKING] refactor: Normalize WorkflowBuilder APIs
* "partitioner" => "assigner"
* normalize ordering so sources always to the left of targets for edges
* normalize parameter ordering so sources and targets are always first arguments
* remove `params` (users should use collection expressions instead)
* refactor: Align name with Python
In #1551 we added a mechanism to open a Streaming workflow run without providing any input. This caused unintuitive behaviour when passing a string as input without providing a runId, resulting in the input being misinterpreted as the runId and the workflow not executing.
As well, the name caused confusion about why the Workflow was not running when using the input-less StreamAsync (since the Workflow cannot run without any messages to drive its execution).
* refactor: Unify ExecutorIsh and ExecutorRegistration => ExecutorBinding
* Switch to more modern Record type-tree for Sum Types
* Unify APIs for getting ExecutorBinding
* Fix an issue where workflows consisting entirely of cross-run shareable executors which are not instance-resettable do not properly clear state when running non-concurrently.
* feat: Simplify function-to-executor pattern
* refactor: Normalize API naming
Concurrent run support was recently added to workflows, but Orchestrations did not fully update to support it. A few executors were missing Cross-Run Shareable annotations, and the ConcurrentEnd executor needed to be factory-instantiated.
This also ports the fix for #1613 from #1637, to avoid waiting on that PR.
Checkpointing is used by the WorkflowHostAgent to be able to support resume from a provided thread. When a CheckpointManager is not specified, we use the InMemoryCheckpointManager and serialize its state into the thread's Serialize()ed JsonElement.
At some point InMemoryCheckpointManager became not serializable, breaking this behaviour. This change restores serializability, and adds a test.
Remove input type checking in favour of explicit `.DescribeProtocolAsync()` flow. Also removes `.AsAgentAsync()` as the validation happens at workflow run time. This makes it easier to use Workflows with DI without resorting to async-over-sync.
* refactor: remove unused internals
* feat: Execution Mode for sharing a workflow among concurrent runs
* feat: Update WorkflowHostAgent to support concurrent execution
* Also update AsAgent APIs to support injecting a CheckpointManager and an IWorkflowExecutionEnvironment
* fix: Make Read logic consistent in DeclarativeWorkflowContext
WorkflowHostAgent was initially implemented before Checkpointing was available. This meant that in order to support resuming, the WorkflowHostAgent needed to keep the runs around, which broke it when stricter rules about concurrent sharing of workflows during execution were introduced.
This change updates the hosting logic to release the underlying StreamingRun when the RunStreamingAsync or RunAsync are invoked, in favour of keeping the checkpointing information in the WorkflowThread to enable resumption.
* Updates to async run loop.
* fix: Workflow Onwership can be release by nonowner
* fix: Incorrect handling of blockOnPending in StreamingRun
Depending on whether we are running in streaming on non-streaming mode, we may be using the StreamingRun in different ways. Unfortunately, the only place we can really know what is the actual state of execution is in the RunEventStream implementations.
This resulted in blocking where blocking was unneeded and occasionally not-blocking when blocking was needed.
The fix is to move the logic of handling this blocking into RunEventStream implementations.
* fix: Fix cleanup on error and end run
This ensures we clean up the background resources correctly.
* fix: Ensure we let the run loop proceed when shutting down
* fix: Add timeout for Input Waiting
* fix: Make the samples properly clean up `Run`s and `StreamingRun`s
* fix: Simplify Declarative Workflow Run disposal pattern
* Also fixes missing .Disposal() in Integration tests
---------
Co-authored-by: Ben Thomas <ben.thomas@microsoft.com>
* feat: Add support for Workflow-as-Executor
* Fixes routing of 'object' compile-typed variables to properly take in type information
* Fixes a concurrency issue in StepTracer
* fix: Make Subworkflow ExternalRequests work properly
* fix: Threading and Concurrency fixes; prep for OffThread Mode
* refactor: Remove dead code around OffStreamRunEventStream
Currently not used, and will be replaced with a rewrite when brought back, so having it in the change is not valuable.
* ci: Work around issues with dotnet-format not properly analyzing the source
* fix: Fix the logic of AsyncCoordinator and AsyncBarrier
* Prevent individual wait cancellations from canceling the entire barrier
* Propagate information about whether the wait was completed or cancelled, and whether any waiters were present when released
* fix: Remove superfluous acces to .Keys in InProcStepTracer
* refactor: Clean up AsyncCoordinator's use of AsyncBarrier
* Disallow execution of multiple workflows at once
* Define notion of executor Reset() to allow reuse of workflows with shared executor instances
* Switch to delivering all messages to a single executor sequentially, with executors running in parallel
* feat: Make Executor id field mandatory
When checkpointing is involved, it is critical to keep executor ids consistent between runs, even when recreating a new object tree for the workflow.
The default id-setting mechanism generated a guid for part of the id, making it not work when restoring from a checkpoint.
This change prevents this situation from arising.
* feat: Enable running untyped Workflows
With the change to enable delay-instantiation of executors and support for async Executor factory methods, we must instantiate the starting executor to know what are the valid input types for the workflow.
To avoid forcing instantiation every time, and to better support workflows with multiple input types, we enable support for build and interacting with the base Workflow type without type annotations, and remove the requirement to know a valid input type when initiating a run.
* feat: Support Output from any executor and multiple outputs.
* feat: Support Checkpoint Serialization
* Implements serialization roundtripping for checkpoints.
* Adds support for JSON serialization
* Adds FileSystem-based checkpoint persistence
* fix: Executor State does not deserialize correctly
The StateManager was not properly handling delay-deserialized values.
* Fix PortableValue handling in StateManager (this makes it delegate to PortableValue the uwnrapping)
* Fix UnitTest to actually test checkpoint serialization
* Additional review comment fixes
---------
Co-authored-by: Chris <66376200+crickman@users.noreply.github.com>
* fix: MessageMerger crashes when there are no dangling messages
* refactor: Better logic for AgentId in Workflow-as-Agent
If the parent "agent" instance received an Id or Name when being instantiated, we should avoid stomping over it with the subagents' ids. But if there is no parent identifier, and only a single subagent yielded identified messages, pull that in.
When checkpointing we did not persist the set of instantiated executors. This means, in turn, when we restore from a checkpoint when using Resume(Stream) rather than restoring a checkpoint in the context of an already existing (Streaming)Run, the executors never got reinstantiated and there were no executors to notify that a state should be loaded.
The fix is to ensure we persist the list and reinstantiate the executors on rehydration.
* Also adds a rehydration restore test
* feat: Improve DevEx for simple Executors
* Add abstract types for executors that will only handle one type of message
* Add FunctionExecutor and configuration capability on delegates
* Add support for late-instantiated Executors
* refactor: Remove open-typed extension method
* refactor: Switch to TaskFactory pattern for async--from-sync
* docs: Update XML docs for publics and fix formatting
* refactor: Better naming for ExecutorIsh configuration methods
* docs: Fix typo in ExecutorIshConfigurationExtensions.ConfigureFactory
* fix: FanIn Edge does not work
We were not creating the state for FanIn edge in EdgeMap correctly, leading to crashes. After fixing that, it turns out the logic in FanInEdgeRunner was only forwarding the last message, not all of them.
* fix: Remove duplicate code and fix typo
The previous fix for the aggregation in AIAgentHostExecutor has a bug wherein we only include the first update from a ChatMessage in the outgoing "collected" message.
The fix is to ensure we put the collected updates into the message before sending it out.
The current implementation of AIAgentHostExecutor unwraps every incoming AgentRunResponseUpdate into a separate ChatMessage, amplifying the number of ChatMessages are actually generated, and yielding multiple messages with the same MessageId.
The fix is to aggregate by MesageId, with the expectation that agents do not interleave messages with differing ids, thus every new MessageId indicates a new ChatMessage and never an old one.
* feat: Make WorkflowBuilder more intutitve
Right now Executorish binding has some unintutitive behaviour. When a user adds an eecutor with an id of an executor that already exists, we silently replace it, if the user provides it inside of add_edge. When a user introduces an executor via an unbound id, the user must bind it via BindExecutor, even though the registration is created implicitly when an edge id added.
The change will remove the invisible update in favor of a "best efforts" check of type and instance equality.
* Expand errors when rebinding to disallowed
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* feat: Implement Checkpointing API
* refactor: Normalzie Namespaces and break out multi-class files
* feat: Support checkpointing in AIAgentHostExecutor
* test: Representation tests
* feat: Add Step-level Tracing and WorkflowEvents
* feat: Add Checkpointing Sample and Smoke Test
* Fixes an issue where StateManager was not properly clearing the incoming queued updates.
* Fixes order of checkpointing and in-step event publication
* Adds import of RunContext state on LoadCheckpoint
* Add re-firing of events for unserviced ExternalRequests on Checkpoint load
* docs: Add documentation to publics
* Also adds documentation to ICheckpointManager which may go public
* refactor: Fix Union Aggregators and add Tests
* fix: Fix issues raised in PR comments and remove dead code
* Allows creation of ExternalRequest objects directly to control the requestId
* Allows receiving ExternalResponse objects rather than unwrapped Data
* Normalizing naming
* feat: Host Workflow as AIAgent
* Also changes AIAgent-as-Executor to use streaming runs and streaming
events
* Also enables default setting for yielding events
* fix: Infinite loop in GenerateNewId()
* docs: Spelling
* test: Add Workflow-as-Agent sample and test
* feat: Support Executor-targeted messages
This adds support for only sending a message to a given executor. Messages will still only route through connected edges.
* feat: Support sending all valid input types after starting a run
* feat: Normalize AIAgent-as-Executor Message Protocol to use MEAI types
* feat: Define Workflow and Executor APIs
* feat: Define IExecutionContext and Events
* feat: Simple Workflow Demos
* refactor: Move Workflows classes to separate assembly
* feat: Move FanOut/In to LowLevel API with new semantics
* feat: Implement Local Execution
* refactor: Assembly name .Workflow => .Workflows
* feat: Enable Default Message Handling
* also lifts Bind in MessageHandlerInfo to better be able to direclty invoke handlers (for AOT, later)
* feat: Implement StreamingHandle APIs
This allows the user to respond to WorkflowEvents with external messages, enabling HIL.
* feat: Add checks for duplicate edges and chain cycles
* feat: Add built-in WorkflowEvents
* refactor: Pull classes into own files
* refactor: Simplify Disposal pattern in Executor
* refactor: Break EdgeRunner file into per-type files
* refactor: Use Throw.IfNull()
* refactor: Remove AddLoop()
Per https://github.com/microsoft/agent-framework/pull/272#discussion_r2241739079 we decided this was not very useful.
* refactor: Normalize use of ValueTask
* fix: Build Break from removing .AddLoop
* refactor: Explicit routing and RouteBuilder
Split out reflection from MessageRouter implemention into build phase, enabling AOT compilation to drive RouteBuilding without reflection.
* test: Add Reflection/Invocation tests
* fix: Terminate on Completion event
* refactor: Update public API surface
* feat: Add support for external requests
* feat: Support hosting AIAgent instances in Workflows
* fix: Fix routing to go through Executor.ExecuteAsync
* test: Update samples for "must SendMessage" semantics
* Add invoking samples to unit tests to avoid future breaks
* fix: ExternalRequest should block Workflow completion
* feat: Normalize API surface against Python
* Also adds xmldoc to all public APIs
* refactor: Normalize UnitTest and Sample namespaces
* fix: Formatting
* refactor: Normalize project/folder names
* feat: Remove DynamicCodeExecution from ValueTaskTypeErasure
* fix: Fix ILTrim warnings
* docs: Add missing docs and fix typos
* feat: Hosted Agents should report Run events
* fix: Fix type propagation for ILTrim changes
* refactor: Simplify DynamicallyAccessedMembers annotations
* sample: Use static-Type construction of InputPort
* feat: Support non-Streaming Run Mode
* test: Add test for non-streaming execution
* refactor: Remove unused types
* refactor: Simplify Event and EdgeData type hierarchies
* feat: Add Switch (=Conditional Edge Group) control flow
* feat: Make .NET AutoSend the MessageHandler result
* feat: Implement State APIs
* refactor: Simplify public namespaces and code organization
* refactor: Reconcile .NET and Python names
---------
Co-authored-by: Chris <66376200+crickman@users.noreply.github.com>
Jacob AlberandGitHub
