* fix: use HasSchema check in DetermineElementType to prevent empty records
When parsing JSON arrays containing objects without a predefined schema,
`DetermineElementType()` was creating a `VariableType` with an empty
(non-null) schema via `targetType.Schema?.Select(...) ?? []`. This caused
`ParseRecord` to take the schema-based parsing path, iterating over zero
schema fields and silently discarding all JSON properties.
The fix checks `targetType.HasSchema` and falls back to
`VariableType.RecordType` (which has `Schema = null`) when no schema is
defined, ensuring `ParseRecord` takes the dynamic `ParseValues()` path
that preserves all JSON properties.
Closes#4195
* test: add regression tests for schema-less JSON array-of-objects parsing (#4195)
Add two regression tests to JsonDocumentExtensionsTests:
1. ParseRecord_ObjectWithArrayOfObjects_NoSchema_PreservesNestedProperties
- Parses a JSON object containing an array of objects using
VariableType.RecordType (no schema) and verifies that nested
object properties (name, role) are preserved in each element.
- This is the exact scenario from issue #4195 where objects in
arrays were being returned as empty dictionaries.
2. ParseList_ArrayOfObjects_NoSchema_PreservesProperties
- Parses a JSON array of objects directly via ParseList with
VariableType.ListType (no schema) and verifies all properties
are preserved.
Both tests follow the existing Arrange/Act/Assert pattern and would
have failed before the DetermineElementType() fix (empty dictionaries
instead of populated ones).
When converting base AgentRunOptions to ChatClientAgentRunOptions, the middleware
now preserves AllowBackgroundResponses, ContinuationToken, and AdditionalProperties
in addition to ResponseFormat.
Added unit test verifying all properties are preserved during the conversion.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add ChatClient decorator for calling AIContextProviders
* Format new files
* Address PR comments
* Revert problematic change
* Rename Use to UseAIContextProvider
* Add Azure AI Foundry Memory Context Provider with unit tests
* Add FoundryMemory integration tests and sample application
* Fix ClearStoredMemoriesAsync to handle 404 gracefully and rename to EnsureStoredMemoriesDeletedAsync
* Refactor FoundryMemory: simplify architecture and add memory store creation
- Remove IFoundryMemoryOperations interface (was only for test mocking)
- Remove AIProjectClientMemoryOperations wrapper class
- Provider now directly uses AIProjectClient with internal extension methods
- Extension methods return actual response models instead of extracted values
- Remove WaitForUpdateCompletionAsync from provider (sample uses delay)
- Simplify EnsureMemoryStoreCreatedAsync to return Task instead of Task<bool>
- Add memory store creation with chat_model and embedding_model
- Add UpdateMemoriesResponse with SupersededBy and Error fields
- Simplify unit tests to focus on constructor validation and serialization
- Update sample to use simple delay for memory processing wait
* Add waiting operation for memory store updates
* Fix UTF-8 BOM encoding for FoundryMemory csproj files
* Update copilot instructions for UTF-8 BOM and fix sample API rename
* Fix UTF-8 BOM encoding for TestableAIProjectClient.cs
* Add missing response headers for TS
* Changing default embedding
* Using the SDK Models
* Program update
* Remove debugging code from sample
* Adapt FoundryMemoryProvider to new AIContextProvider API and add UTF-8 BOM instruction
- Override ProvideAIContextAsync/StoreAIContextAsync instead of removed virtual InvokingAsync/InvokedAsync
- Use ProviderSessionState<State> for session-scoped state management (matching Mem0Provider pattern)
- Replace constructor-based scope with stateInitializer delegate
- Remove Serialize method (no longer on base class)
- Add SearchInputMessageFilter, StorageInputMessageFilter, StateKey to options
- Update sample to use AIContextProviders list instead of AIContextProviderFactory
- Update unit and integration tests for new API
- Add UTF-8 BOM encoding and --tl:off instructions to dotnet/AGENTS.md
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Use DefaultAzureCredential in Foundry Memory sample
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address PR review comments for FoundryMemoryProvider
- Move memoryStoreName from options to required constructor parameter
- Make FoundryMemoryProviderScope require non-null/whitespace scope in constructor
- Make Scope property read-only (getter only)
- Replace ConcurrentQueue with single last update ID to fix memory leak
- Only clear pending update ID after successful completion
- Add delete success logging
- Mark FoundryMemoryProvider with [Experimental] attribute
- Update unit tests for new API signatures
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Use Throw.IfNullOrWhitespace for scope and memoryStoreName validation
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* refactor: Normalize Run/RunStreaming with AIAgent
* refactor: Clarify Session vs. Run -level concepts
* Rename RunId to SessionId to better match Run/Session terminology in AIAgent
* [BREAKING]: Will break existing checkpointed sessions in CosmosDb due to field rename
* refactor: Rename and simplify interface around getting typed data out of ExternalRequest/Response
* Also adds hints around using value types in PortableValue
* refactor: Rename AddFanInEdge to AddFanInBarrierEdge
This will prevent a breaking change later when we introduce a programmable FanIn edge, analogous to the FanOut edge's EdgeSelector.
The goal, in the long run is to support a number of different FanIn scenarios, with naive FanIn (no barrier) by default, similar to FanOut.
* refactor: AsAgent(this Workflow, ...) => AsAIAgent(...)
* misc - part1: SwitchBuilder internal
---------
Co-authored-by: Dmytro Struk <13853051+dmytrostruk@users.noreply.github.com>
* Fix handoff orchestration not passing user message to handoff target agent (#3161)
Filter out internal handoff function call and tool result messages before
passing conversation history to the target agent's LLM. These messages
confused the model into ignoring the original user question.
* Add handoff tool call filtering behavior and enhance workflow builder
- Introduced HandoffToolCallFilteringBehavior enum to specify filtering behavior for tool call contents in handoff workflows.
- Updated HandoffsWorkflowBuilder to support customizable handoff instructions and tool call filtering behavior.
- Enhanced HandoffAgentExecutor to utilize new filtering options for improved message handling during agent handoffs.
* Enhance handoff message filtering logic and add unit tests for filtering behaviors
* Refactor HandoffMessagesFilter to remove unused handoff function names and enhance filtering logic for non-handoff function calls
* Refactor HandoffMessagesFilter to streamline FilterCandidateState initialization and improve clarity
* Refactor HandoffMessagesFilter to improve filtering logic and add integration tests for handoff workflows
* fix: HandoffAgentExecutor tests
* [BREAKING] refactor: Decouple Checkpointing and Execution APIs
With this change, Checkpointing becomes an property of an IWorkflowExecutionEnvironment. This lets environments that are tightly-coupled to their CheckpointManager avoid needing to present APIs that would not work (e.g. taking in an InMemory CheckpointManager for Durable Tasks, for example)
* refactor: Normalize IsCheckpointingEnabled naming
Track the last CheckpointInfo in InProcessRunner so that newly created
checkpoints reference their parent. When resuming from a checkpoint,
the resumed-from checkpoint becomes the parent of the next checkpoint.
Adds tests verifying:
- First checkpoint has null parent
- Subsequent checkpoints chain parents correctly
- Checkpoint after resume references the resumed-from checkpoint
* feat: Implement Polymorphic Routing
* feat: Add support for Send/Yield annotations with basic Executor
* Adds annotations to Declarative workflow executors
* fix: Address PR Comments
* Implicit filter in collection loops
* Remove debug / usused / superfluous code
* Fix ProtocolBuilder implicit output registrations
* Fix logic error in ExecuteRouteGeneratorTests.ClassWithManualConfigureProtocol_DoesNotGenerate
* fix: Solidify type checks and send/yield type registrations
* fix: Suppress generation of TurnTokens out of AggregateTurnMessagesExecutor
* Fixes an issue where ConcurrentEndExecutor is not expecting TurnTokens.
* fix: Add ProtocolBuilder support for chained-delegation
* Updates Declarative pacakge to rely on chained-delegation Send/Yield registration
* Renames DeclarativeActionExectuor's new ExecuteAsync to ExecuteActionAsync to avoid colliding with Executor.ExecutoeAsync
* fix: Address PR Comments
* Fixes type mapping in FanInEdgeRunner
* Fixes and expalins send/yield type registration in FunctionExecutor
* fixup: build-break
* fix: Add missing SendsMesage declaration to InvokeAzureAgentExecutor
* Initial Implementation of InvokeFunctionTool
* Added unit test for InvokeFunctionTool executor.
* Implemented unit and integration tests for InvokeFunctionTool.
* Add sample for InvokeFunctionTool in declarative workflows.
* Remove unused sample and updated comments.
* Updating to official OM release with InvokeFunctionTool
* Fix formatting issues.
* Updated PowerFx version
* Update test fixture
* Cleanup - Removed unused method in InvokeFunctionToolExecutor
* Update test based on PR feedback.
* Update based on PR comments
* Rename WorkflowOutputEvent.SourceId to ExecutorId for Python consistency
- Rename SourceId property to ExecutorId in WorkflowOutputEvent
- Add [Obsolete] SourceId property for backward compatibility
- Update all test usages to use ExecutorId
Resolves part of #2938
* Unify AgentResponse events with WorkflowOutputEvent (#2938)
- Change AgentResponseEvent and AgentResponseUpdateEvent to inherit from
WorkflowOutputEvent instead of ExecutorEvent
- Update AIAgentHostExecutor and HandoffAgentExecutor to use YieldOutputAsync()
instead of AddEventAsync() for agent outputs
- Add special-casing in InProcessRunnerContext.YieldOutputAsync() to create
specific event types for AgentResponse and AgentResponseUpdate, bypassing
OutputFilter for backwards compatibility
- Update TestRunContext and TestWorkflowContext with same special-casing
- Add regression tests in AgentEventsTests
* refactor: Seal AgentResponse events
* .NET: [BREAKING] Add session statebag to use for state storage instead of inside providers (#3737)
* Add a StateBag to AgentSession and pass Agent and AgentSession to AIContextProvider and ChatHistoryProviders
* Convert all AIContextProviders to use the statebag
* Update InMemoryChatHistoryProvider to use StateBag
* Update Comsos and Workflow ChatHistoryProviders
* Update 3rd party chat history storage sample.
* Remove serialize method from providers
* Replacing provider factories with properties
* Remove Providers from Session and flatten state bag serialization
* Update samples to use getservice on agent
* Updated additional session types to serialize statebag
* Fix regression
* Address PR comments
* Address PR comments.
* Fix formatting
* Fix unit tests
* Remove InMemoryAgentSession since it is not required anymore.
* Address PR comments
* Convert sessions for A2AAgent, ChatClientAgent, CopilotStudioAgent and GithubCopilotAgent to use regular json serialization.
* Fix durable agent session jso usgae
* Add jso to InMemory and Workflow ChatHistoryProviders
* Update InMemoryChatHistoryProvider to use an options class for it's many optional settings.
* Apply suggestions from code review
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Address PR feedback
* Fix verification bug.
* Improve state bag thread safety
* Address PR comments and fix unit tests
* Address PR comments
* Fix unit test
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Add a public StateKey property to providers (#3810)
* .NET: [BREAKING] Update providers in such a way that they can participate in a pipeline (#3846)
* Make providers pipeline capable
* Fix unit tests
* Move source stamping to providers from base class
* Also update samples.
* Address PR comments
* Rename AsAgentRequestMessageSourcedMessage to WithAgentRequestMessageSource
* .NET: [BREAKING] Add consistent message filtering to all providers. (#3851)
* Add consistent message filtering to all providers.
* Remove old chat history filtering classes
* Fix merge issues
* Fix unit test
* Enforce non-nullable property
* Fix merging bug and make troubleshooting source info easier by adding tostring implementation
* .NET: [BREAKING] Add support for multiple AIContextProviders on a ChatClientAgent (#3863)
* Add support for multiple AIContextProviders on a ChatClientAgent
* Address PR comments and fix tests
* Address PR comments.
* .NET: [BREAKING]Delay AIContext Materialization until the end of the pipeline is reached. (#3883)
* Delay AIContext Materialization until the end of the pipeline is reached.
* Address PR comments.
* Address PR comments
* Modify InvokedContext to be immutable (#3888)
* .NET: Address Feedback on StateBag feature branch PR (#3910)
* Address Feedback on statebag feature branch PR
* Update dotnet/src/Microsoft.Agents.AI.DurableTask/CHANGELOG.md
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Address PR comments
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update GitHub.Copilot.SDK to 0.1.23 and copy new session config properties
- Bump GitHub.Copilot.SDK from 0.1.18 to 0.1.23
- Add new SessionConfig properties: ReasoningEffort, Hooks, OnUserInputRequest,
WorkingDirectory, ConfigDir, InfiniteSessions
- Add missing ResumeSessionConfig properties: Model, SystemMessage,
AvailableTools, ExcludedTools, ReasoningEffort, Hooks, OnUserInputRequest,
WorkingDirectory, ConfigDir, InfiniteSessions
- Fix UserMessageDataAttachmentsItem -> UserMessageDataAttachmentsItemFile
for new polymorphic attachment API
- Add unit tests for new session config properties
* Address PR review: centralize config mapping and improve test coverage
- Extract CopySessionConfig/CopyResumeSessionConfig as internal static helpers
to eliminate duplicated mapping logic between RunCoreStreamingAsync and
CreateResumeConfig (addresses reviewer comment on drift risk)
- Add InternalsVisibleTo for unit test project
- Replace shallow constructor tests with comprehensive property-verification
tests that validate every config property is correctly copied, including
OnUserInputRequest (addresses reviewer comments on test coverage)
* Remove accidentally committed git-lfs hooks
* Initial plan
* Add comprehensive unit tests for EditTableV2Executor
- Test AddItemOperation with record and scalar values
- Test ClearItemsOperation
- Test RemoveItemOperation
- Test TakeLastItemOperation (with items and empty table)
- Test TakeFirstItemOperation (with items and empty table)
- Test error cases (null ItemsVariable, non-table variable)
- Include ExecuteTestAsync and CreateModel helper methods
- All 10 tests passing
Co-authored-by: crickman <66376200+crickman@users.noreply.github.com>
* Add comprehensive unit tests for EditTableV2Executor - complete with 100% coverage
- Added 13 comprehensive tests covering all code paths
- Test AddItemOperation with record and scalar values
- Test ClearItemsOperation
- Test RemoveItemOperation (including non-table value case)
- Test TakeLastItemOperation (with items and empty table)
- Test TakeFirstItemOperation (with items and empty table)
- Test error cases (null ItemsVariable, non-table variable, null operation values)
- Include ExecuteTestAsync and CreateModel helper methods
- 100% line and branch coverage achieved
Co-authored-by: crickman <66376200+crickman@users.noreply.github.com>
* Update tests / refine product code
* Checkpoint
* Updated
* Update dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/ObjectModel/SetTextVariableExecutorTest.cs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Address code review feedback
- Fix typo: rename metadataExpresssion to metadataExpression
- Fix test name in AddMessageWithMetadataAsync (was using wrong test name)
- Fix test name in ClearGlobalScopeAsync (was using wrong test name)
- Remove pre-population in SetTextVariableExecutorTest that made tests ineffective
- Use explicit .Where() filter in SetMultipleVariablesExecutorTest foreach loop
Co-authored-by: crickman <66376200+crickman@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: crickman <66376200+crickman@users.noreply.github.com>
Co-authored-by: Chris Rickman <crickman@microsoft.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update message source code to match python.
* Apply suggestion from @Copilot
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Address PR comment
* Move setting of source information to extension method
* Add underscore for attribution key to indicate internal usage
* Stick to version 102 of the SDK since 103 is causing issues.
* Revert global.json change
* Fix unit test
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
RunCoreStreamingAsync was passing inputMessagesForProviders (which lacks
chat history) to GetStreamingResponseAsync instead of
inputMessagesForChatClient (which includes chat history). This caused
streaming runs to lose conversation context on subsequent calls.
The non-streaming path (RunCoreAsync) already correctly used
inputMessagesForChatClient. This aligns the streaming path to match.
Also adds a unit test that validates chat history is included in
messages sent to the chat client during streaming on subsequent calls.
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: lokitoth <6936551+lokitoth@users.noreply.github.com>
* feat(workflows): Make telemetry opt-in via WithOpenTelemetry()
- Add WorkflowTelemetryOptions class with EnableSensitiveData property
- Add WorkflowTelemetryContext to manage ActivitySource lifecycle
- Add WithOpenTelemetry() extension method on WorkflowBuilder
- Update all workflow components to use telemetry context:
- WorkflowBuilder, Workflow, Executor
- InProcessRunnerContext, InProcessRunner
- LockstepRunEventStream, StreamingRunEventStream
- All edge runners (Direct, FanIn, FanOut, Response)
- Telemetry is now disabled by default
- Users must call WithOpenTelemetry() to enable spans/activities
BREAKING CHANGE: Workflow telemetry is now opt-in. Users who relied on
automatic telemetry must add .WithOpenTelemetry() to their workflow builder.
* refactor: Pass telemetry context as parameter instead of via interface
- Remove IWorkflowContextWithTelemetry interface
- Add internal ExecuteAsync overload that accepts WorkflowTelemetryContext
- Public ExecuteAsync delegates with WorkflowTelemetryContext.Disabled
- InProcessRunner passes TelemetryContext when calling ExecuteAsync
- BoundContext now implements IWorkflowContext (not the removed interface)
* Add optional ActivitySource parameter to WithOpenTelemetry
Allow users to provide their own ActivitySource when enabling telemetry,
giving them better control over the ActivitySource lifecycle. When not
provided, the framework creates one internally (existing behavior).
Changes:
- Add optional activitySource parameter to WithOpenTelemetry() extension
- Update WorkflowTelemetryContext to accept external ActivitySource
- Add unit test for user-provided ActivitySource scenario
* Add component-level telemetry control with disable flags
Allow users to selectively disable specific activity types via
WorkflowTelemetryOptions. All activities are enabled by default.
New disable flags:
- DisableWorkflowBuild: Disables workflow.build activities
- DisableWorkflowRun: Disables workflow_invoke activities
- DisableExecutorProcess: Disables executor.process activities
- DisableEdgeGroupProcess: Disables edge_group.process activities
- DisableMessageSend: Disables message.send activities
Added helper methods to WorkflowTelemetryContext for each activity type
and updated all activity creation sites to use them.
* Implement EnableSensitiveData to log executor input/output
When EnableSensitiveData is true in WorkflowTelemetryOptions, executor
input and output are logged as JSON-serialized attributes in the
executor.process activity.
New activity tags:
- executor.input: JSON serialized input message
- executor.output: JSON serialized output result (non-void only)
Added suppression attributes for AOT/trimming warnings since this is
an opt-in feature for debugging/diagnostics.
* Refactor activity start methods to centralize tagging logic
Move tagging logic into WorkflowTelemetryContext methods:
- StartExecutorProcessActivity now accepts executorId, executorType,
messageType, and message; sets all tags including executor.input
when EnableSensitiveData is true
- Added SetExecutorOutput method to set executor.output after execution
- StartMessageSendActivity now accepts sourceId, targetId, and message;
sets all tags including message.content when EnableSensitiveData is true
Simplified Executor.cs and InProcessRunnerContext.cs by removing
inline tagging code. Added message.content tag constant.
* Revert Python changes
* Update samples and code cleanup
* Fix file formatting
* Add comment
* Add telemetry configuration to declarative workflow
* Remove delays in tests
* Address comments
* Add ability to mark the source of Agent request messages and use that for filtering
* Add support for source, in addition to source type, and add unit tests for automatic stamping
* Address PR comments.
* Add merge fixes
* Address PR comments
Peter IbekweandGitHub
Rishabh ChawlaandGitHub
Korolev DmitryandGitHub
L. Elaine DazzioandGitHub
ChandramouleswaranandGitHub
Tao ChenandGitHub