mirror of
https://github.com/microsoft/agent-framework.git
synced 2026-06-16 21:04:09 +08:00
.NET: Allow Simulating service stored ChatHistory to improve consistency (#4974)
* Allow Simulating service stored ChatHistory to improve consistency * Fixing bug in ServiceStoredSimulatingChatClient * Addressing PR comments. * Address PR comments * Apply suggestion from @SergeyMenshykh Co-authored-by: SergeyMenshykh <68852919+SergeyMenshykh@users.noreply.github.com> * Fix bug --------- Co-authored-by: SergeyMenshykh <68852919+SergeyMenshykh@users.noreply.github.com>
This commit is contained in:
westey
committed by
GitHub
Unverified
GitHub
parent
18f7ba8632
commit
401e5dc7e8
@@ -31,8 +31,6 @@ The persistence timing and `FunctionResultContent` trimming behaviors are interr
|
||||
|
||||
- **Per-run persistence**: When messages are batched and persisted at the end of the full run, trailing `FunctionResultContent` trimming becomes necessary to match the service's behavior. Without trimming, the stored history contains `FunctionResultContent` that the service would never have stored.
|
||||
|
||||
This means the trimming feature (introduced in [PR #4792](https://github.com/microsoft/agent-framework/pull/4792)) is primarily needed as a complement to per-run persistence. The `PersistChatHistoryAtEndOfRun` setting (introduced in [PR #4762](https://github.com/microsoft/agent-framework/pull/4762)) inverts the default so that per-service-call persistence is the standard behavior, and per-run persistence is opt-in.
|
||||
|
||||
## Decision Drivers
|
||||
|
||||
- **A. Consistency**: The default behavior of `ChatHistoryProvider` should produce stored history that closely matches what the underlying AI service would store, minimizing surprise when switching between framework-managed and service-managed chat history.
|
||||
@@ -43,33 +41,30 @@ This means the trimming feature (introduced in [PR #4792](https://github.com/mic
|
||||
|
||||
## Considered Options
|
||||
|
||||
- Option 1: Default to per-run persistence with `FunctionResultContent` trimming (opt-in to per-service-call)
|
||||
- Option 2: Default to per-service-call persistence (opt-in to per-run)
|
||||
- Option 1: Per-run persistence with opt-in FRC (FunctionResultContent) trimming
|
||||
- Option 2: Opt-in per-service-call persistence (via `SimulateServiceStoredChatHistory`)
|
||||
|
||||
## Pros and Cons of the Options
|
||||
|
||||
### Option 1: Default to per-run persistence with `FunctionResultContent` trimming
|
||||
### Option 1: Per-run persistence with opt-in FRC trimming
|
||||
|
||||
Keep the current default behavior of persisting chat history only at the end of the full agent run. Add `FunctionResultContent` trimming as the default to improve consistency with service storage. Provide an opt-in setting for users who want per-service-call persistence.
|
||||
|
||||
Settings:
|
||||
- `PersistChatHistoryAtEndOfRun` = `true`
|
||||
Keep the current default behavior of persisting chat history only at the end of the full agent run. Add `FunctionResultContent` trimming as an opt-in behavior to improve consistency with service storage.
|
||||
|
||||
- Good, because runs are atomic — chat history is only updated when the full run succeeds, satisfying driver B.
|
||||
- Good, because the mental model is simple: one run = one history update, satisfying driver D.
|
||||
- Good, because trimming trailing `FunctionResultContent` improves consistency with service storage, partially satisfying driver A.
|
||||
- Good, because users can opt in to per-service-call persistence for checkpointing/recovery scenarios, satisfying drivers C and E.
|
||||
- Bad, because the default persistence timing still differs from the service's behavior (per-run vs. per-service-call), only partially satisfying driver A.
|
||||
- Bad, because if the process crashes mid-loop, all intermediate progress from the current run is lost, not satisfying driver C by default.
|
||||
- Bad, because if the process crashes mid-loop, all intermediate progress from the current run is lost, not satisfying driver C.
|
||||
- Bad, because this option alone does not provide a way for users to opt into per-service-call persistence, not satisfying driver E.
|
||||
|
||||
### Option 2: Default to per-service-call persistence
|
||||
### Option 2: Opt-in per-service-call persistence (via `SimulateServiceStoredChatHistory`)
|
||||
|
||||
Change the default to persist chat history after each individual service call within the FIC loop, matching the AI service's behavior. Trailing `FunctionResultContent` trimming is unnecessary with this approach (it is naturally handled). Provide an opt-in setting for users who want per-run atomicity with trimming.
|
||||
Introduce an optional SimulateServiceStoredChatHistory setting to persist chat history after each individual service call within the FIC loop, matching the AI service's behavior. Trailing `FunctionResultContent` trimming is unnecessary with this approach (it is naturally handled).
|
||||
|
||||
Settings:
|
||||
- `PersistChatHistoryAtEndOfRun` = `false` (default)
|
||||
- `SimulateServiceStoredChatHistory` = `true`
|
||||
|
||||
- Good, because the stored history matches the service's behavior by default for both timing and content, fully satisfying driver A.
|
||||
- Good, because the stored history matches the service's behavior when opting in for both timing and content, fully satisfying driver A.
|
||||
- Good, because intermediate progress is preserved if the process is interrupted, satisfying driver C.
|
||||
- Good, because no separate `FunctionResultContent` trimming logic is needed, reducing complexity.
|
||||
- Bad, because chat history may be left in an incomplete state if the run fails mid-loop (e.g., `FunctionCallContent` stored without corresponding `FunctionResultContent`), not satisfying driver B. A subsequent run cannot proceed without manually providing the missing `FunctionResultContent`.
|
||||
@@ -78,39 +73,36 @@ Settings:
|
||||
|
||||
## Decision Outcome
|
||||
|
||||
Chosen option: **Option 2 — Default to per-service-call persistence**, because it fully satisfies the consistency driver (A), naturally handles `FunctionResultContent` trimming without additional logic, and provides better recoverability for long-running tool-calling loops. Per-run persistence remains available via the `PersistChatHistoryAtEndOfRun` setting for users who prefer atomic run semantics.
|
||||
Chosen option: **Option 2: Opt-in per-service-call persistence (via `SimulateServiceStoredChatHistory`)**. The existing per-run persistence behavior is retained as-is, requiring no changes from users. Per-service-call persistence is available as an opt-in feature via the `SimulateServiceStoredChatHistory` setting. This satisfies drivers B (atomicity) and D (simplicity) for the common case, while fully satisfying driver A (consistency) for users who opt into simulated service-stored behavior. Users who need per-service-call persistence for recoverability (driver C) can enable it explicitly.
|
||||
|
||||
### Configuration Matrix
|
||||
|
||||
The behavior depends on the combination of `UseProvidedChatClientAsIs` and `PersistChatHistoryAtEndOfRun`:
|
||||
The behavior depends on the combination of `UseProvidedChatClientAsIs` and `SimulateServiceStoredChatHistory`:
|
||||
|
||||
| `UseProvidedChatClientAsIs` | `PersistChatHistoryAtEndOfRun` | Behavior |
|
||||
| `UseProvidedChatClientAsIs` | `SimulateServiceStoredChatHistory` | Behavior |
|
||||
|---|---|---|
|
||||
| `false` (default) | `false` (default) | **Per-service-call persistence.** A `ChatHistoryPersistingChatClient` middleware is automatically injected into the chat client pipeline between `FunctionInvokingChatClient` and the leaf `IChatClient`. Messages are persisted after each service call. |
|
||||
| `true` | `false` | **User responsibility.** No middleware is injected because the user has provided a custom chat client stack. The user is responsible for ensuring correct persistence behavior (e.g., by including their own persisting middleware). |
|
||||
| `false` | `true` | **Per-run persistence with marking.** A `ChatHistoryPersistingChatClient` middleware is injected, but configured to *mark* messages with metadata rather than store them immediately. At the end of the run, marked messages are stored. Trailing `FunctionResultContent` is trimmed. |
|
||||
| `true` | `true` | **Per-run persistence with warning.** The system checks whether the custom chat client stack includes a `ChatHistoryPersistingChatClient`. If not, a warning is emitted (particularly relevant for workflow handoff scenarios where trimming cannot be guaranteed). If no `ChatHistoryPersistingChatClient` is preset, all messages are stored at the end of the run, otherwise marked messages are stored. |
|
||||
| `false` (default) | `false` (default) | **Per-run persistence.** Messages are persisted at the end of the full agent run via the `ChatHistoryProvider`. |
|
||||
| `false` | `true` | **Per-service-call persistence (simulated).** A `ServiceStoredSimulatingChatClient` middleware is automatically injected into the chat client pipeline between `FunctionInvokingChatClient` and the leaf `IChatClient`. Messages are persisted after each service call. A sentinel `ConversationId` causes FIC to treat the conversation as service-managed. |
|
||||
| `true` | `false` | **Per-run persistence.** No middleware is injected because the user has provided a custom chat client stack. Messages are persisted at the end of the run. |
|
||||
| `true` | `true` | **User responsibility.** The system checks whether the custom chat client stack includes a `ServiceStoredSimulatingChatClient`. If not, a warning is emitted — the user is expected to have added their own per-service-call persistence mechanism. End-of-run persistence is skipped. |
|
||||
|
||||
### Consequences
|
||||
|
||||
- Good, because the stored history matches the service's behavior by default for both timing and content, fully satisfying consistency (driver A).
|
||||
- Good, because intermediate progress is preserved if the process is interrupted, satisfying recoverability (driver C).
|
||||
- Good, because no separate `FunctionResultContent` trimming logic is needed in the default path, reducing complexity.
|
||||
- Good, because marking persisted messages with metadata enables deduplication and aids debugging.
|
||||
- Good, because warnings for custom chat client configurations without the persisting middleware help prevent silent failures in workflow handoff scenarios.
|
||||
- Bad, because chat history may be left in an incomplete state if the run fails mid-loop (e.g., `FunctionCallContent` stored without corresponding `FunctionResultContent`), requiring manual recovery in rare cases.
|
||||
- Bad, because the mental model is more complex for the default path: a single run may produce multiple history updates.
|
||||
- Neutral, because users who prefer atomic run semantics can opt in to per-run persistence via `PersistChatHistoryAtEndOfRun = true`.
|
||||
- Good, because per-run persistence is atomic by default — chat history is only updated when the full run succeeds, satisfying driver B.
|
||||
- Good, because the default mental model is simple: one run = one history update, satisfying driver D.
|
||||
- Good, because users who opt into `SimulateServiceStoredChatHistory` get stored history that matches the service's behavior for both timing and content, fully satisfying driver A.
|
||||
- Good, because per-service-call persistence preserves intermediate progress if the process is interrupted, satisfying driver C when opted in.
|
||||
- Good, because no separate `FunctionResultContent` trimming logic is needed when per-service-call persistence is active — it is naturally handled.
|
||||
- Good, because conflict detection (configurable via `ThrowOnChatHistoryProviderConflict`, `WarnOnChatHistoryProviderConflict`, `ClearOnChatHistoryProviderConflict`) prevents misconfiguration when a service returns a `ConversationId` alongside a configured `ChatHistoryProvider`.
|
||||
- Bad, because per-service-call persistence (when opted in) may leave chat history in an incomplete state if the run fails mid-loop (e.g., `FunctionCallContent` stored without corresponding `FunctionResultContent`), requiring manual recovery in rare cases.
|
||||
- Neutral, because users who want per-service-call consistency can opt in via `SimulateServiceStoredChatHistory = true`, satisfying driver E.
|
||||
- Neutral, because increased write frequency from per-service-call persistence may impact performance for some storage backends; this can be mitigated with a caching decorator.
|
||||
|
||||
### Implementation Notes
|
||||
|
||||
#### Conversation ID Consistency
|
||||
|
||||
The `ChatHistoryPersistingChatClient` middleware must also update the session's `ConversationId` consistently for both response-based and conversation-based service interactions, ensuring the session always reflects the latest service-provided identifier.
|
||||
We should introduce a separate `ConversationIdPersistingChatClient`, middleware which allows us to
|
||||
persist response `ConversationIds` during the FICC loop. This could be used with or without
|
||||
`ServiceStoredSimulatingChatClient`.
|
||||
|
||||
## More Information
|
||||
|
||||
- [PR #4762: Persist messages during function call loop](https://github.com/microsoft/agent-framework/pull/4762) — introduces `PersistChatHistoryAfterEachServiceCall` option and `ChatHistoryPersistingChatClient` decorator
|
||||
- [PR #4792: Trim final FRC to match service storage](https://github.com/microsoft/agent-framework/pull/4792) — introduces `StoreFinalFunctionResultContent` option and `FilterFinalFunctionResultContent` logic
|
||||
- [Issue #2889](https://github.com/microsoft/agent-framework/issues/2889) — original issue tracking chat history persistence during function call loops
|
||||
|
||||
@@ -1,15 +1,16 @@
|
||||
// Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
// This sample demonstrates how the ChatClientAgent persists chat history after each individual
|
||||
// call to the AI service.
|
||||
// call to the AI service, using the SimulateServiceStoredChatHistory option.
|
||||
// When an agent uses tools, FunctionInvokingChatClient may loop multiple times
|
||||
// (service call → tool execution → service call), and intermediate messages (tool calls and
|
||||
// results) are persisted after each service call. This allows you to inspect or recover them
|
||||
// even if the process is interrupted mid-loop, but may also result in chat history that is not
|
||||
// yet finalized (e.g., tool calls without results) being persisted, which may be undesirable in some cases.
|
||||
//
|
||||
// To opt into end-of-run persistence instead (atomic run semantics), set
|
||||
// PersistChatHistoryAtEndOfRun = true on ChatClientAgentOptions.
|
||||
// To use end-of-run persistence instead (atomic run semantics), remove the
|
||||
// SimulateServiceStoredChatHistory = true setting (or set it to false). End-of-run
|
||||
// persistence is the default behavior.
|
||||
//
|
||||
// The sample runs two multi-turn conversations: one using non-streaming (RunAsync) and one
|
||||
// using streaming (RunStreamingAsync), to demonstrate correct behavior in both modes.
|
||||
@@ -53,7 +54,7 @@ static string GetTime([Description("The city name.")] string city) =>
|
||||
_ => $"{city}: time data not available."
|
||||
};
|
||||
|
||||
// Create the agent — per-service-call persistence is the default behavior.
|
||||
// Create the agent — per-service-call persistence is enabled via SimulateServiceStoredChatHistory.
|
||||
// The in-memory ChatHistoryProvider is used by default when the service does not require service stored chat
|
||||
// history, so for those cases, we can inspect the chat history via session.TryGetInMemoryChatHistory().
|
||||
IChatClient chatClient = string.Equals(store, "TRUE", StringComparison.OrdinalIgnoreCase) ?
|
||||
@@ -63,6 +64,7 @@ AIAgent agent = chatClient.AsAIAgent(
|
||||
new ChatClientAgentOptions
|
||||
{
|
||||
Name = "WeatherAssistant",
|
||||
SimulateServiceStoredChatHistory = true,
|
||||
ChatOptions = new()
|
||||
{
|
||||
Instructions = "You are a helpful assistant. When asked about multiple cities, call the appropriate tool for each city.",
|
||||
|
||||
@@ -1,16 +1,19 @@
|
||||
# In-Function-Loop Checkpointing
|
||||
|
||||
This sample demonstrates how `ChatClientAgent` persists chat history after each individual call to the AI service by default. This per-service-call persistence ensures intermediate progress is saved during the function invocation loop.
|
||||
This sample demonstrates how `ChatClientAgent` can persist chat history after each individual call to the AI service using the `SimulateServiceStoredChatHistory` option. This per-service-call persistence ensures intermediate progress is saved during the function invocation loop.
|
||||
|
||||
## What This Sample Shows
|
||||
|
||||
When an agent uses tools, the `FunctionInvokingChatClient` loops multiple times (service call → tool execution → service call → …). By default, chat history is persisted after each service call via the `ChatHistoryPersistingChatClient` decorator:
|
||||
When an agent uses tools, the `FunctionInvokingChatClient` loops multiple times (service call → tool execution → service call → …). By enabling `SimulateServiceStoredChatHistory = true`, chat history is persisted after each service call via the `ServiceStoredSimulatingChatClient` decorator:
|
||||
|
||||
- A `ChatHistoryPersistingChatClient` decorator is automatically inserted into the chat client pipeline
|
||||
- A `ServiceStoredSimulatingChatClient` decorator is inserted into the chat client pipeline
|
||||
- Before each service call, the decorator loads history from the `ChatHistoryProvider` and prepends it to the request
|
||||
- After each service call, the decorator notifies the `ChatHistoryProvider` (and any `AIContextProvider` instances) with the new messages
|
||||
- Only **new** messages are sent to providers on each notification — messages that were already persisted in an earlier call within the same run are deduplicated automatically
|
||||
|
||||
To opt into end-of-run persistence instead (atomic run semantics), set `PersistChatHistoryAtEndOfRun = true` on `ChatClientAgentOptions`. In that mode, the decorator marks messages with metadata rather than persisting them immediately, and `ChatClientAgent` persists only the marked messages at the end of the run.
|
||||
By default (without `SimulateServiceStoredChatHistory`), chat history is persisted at the end of the full agent run instead. To use per-service-call persistence, set `SimulateServiceStoredChatHistory = true` on `ChatClientAgentOptions`.
|
||||
|
||||
With `SimulateServiceStoredChatHistory` = true, the behavior matches that of chat history stored in the underlying AI service exactly.
|
||||
|
||||
Per-service-call persistence is useful for:
|
||||
- **Crash recovery** — if the process is interrupted mid-loop, the intermediate tool calls and results are already persisted
|
||||
@@ -26,7 +29,7 @@ The sample asks the agent about the weather and time in three cities. The model
|
||||
```
|
||||
ChatClientAgent
|
||||
└─ FunctionInvokingChatClient (handles tool call loop)
|
||||
└─ ChatHistoryPersistingChatClient (persists after each service call)
|
||||
└─ ServiceStoredSimulatingChatClient (persists after each service call)
|
||||
└─ Leaf IChatClient (Azure OpenAI)
|
||||
```
|
||||
|
||||
|
||||
@@ -139,8 +139,8 @@ public sealed partial class ChatClientAgent : AIAgent
|
||||
|
||||
this._logger = (loggerFactory ?? chatClient.GetService<ILoggerFactory>() ?? NullLoggerFactory.Instance).CreateLogger<ChatClientAgent>();
|
||||
|
||||
// Warn if using a custom chat client stack with end-of-run persistence but no ChatHistoryPersistingChatClient.
|
||||
this.WarnOnMissingPersistingClient();
|
||||
// Warn if using a custom chat client stack with simulated service stored persistence but no ServiceStoredSimulatingChatClient.
|
||||
this.WarnOnMissingServiceStoredSimulatingClient();
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
@@ -454,7 +454,7 @@ public sealed partial class ChatClientAgent : AIAgent
|
||||
/// Notifies the <see cref="ChatHistoryProvider"/> and all <see cref="AIContextProviders"/> of successfully completed messages.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// This method is also called by <see cref="ChatHistoryPersistingChatClient"/> to persist messages per-service-call.
|
||||
/// This method is also called by <see cref="ServiceStoredSimulatingChatClient"/> to persist messages per-service-call.
|
||||
/// </remarks>
|
||||
internal async Task NotifyProvidersOfNewMessagesAsync(
|
||||
ChatClientAgentSession session,
|
||||
@@ -463,7 +463,7 @@ public sealed partial class ChatClientAgent : AIAgent
|
||||
ChatOptions? chatOptions,
|
||||
CancellationToken cancellationToken)
|
||||
{
|
||||
ChatHistoryProvider? chatHistoryProvider = this.ResolveChatHistoryProvider(chatOptions, session);
|
||||
ChatHistoryProvider? chatHistoryProvider = this.ResolveChatHistoryProvider(chatOptions);
|
||||
|
||||
if (chatHistoryProvider is not null)
|
||||
{
|
||||
@@ -486,7 +486,7 @@ public sealed partial class ChatClientAgent : AIAgent
|
||||
/// Notifies the <see cref="ChatHistoryProvider"/> and all <see cref="AIContextProviders"/> of a failure during a service call.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// This method is also called by <see cref="ChatHistoryPersistingChatClient"/> to report failures per-service-call.
|
||||
/// This method is also called by <see cref="ServiceStoredSimulatingChatClient"/> to report failures per-service-call.
|
||||
/// </remarks>
|
||||
internal async Task NotifyProvidersOfFailureAsync(
|
||||
ChatClientAgentSession session,
|
||||
@@ -495,7 +495,7 @@ public sealed partial class ChatClientAgent : AIAgent
|
||||
ChatOptions? chatOptions,
|
||||
CancellationToken cancellationToken)
|
||||
{
|
||||
ChatHistoryProvider? chatHistoryProvider = this.ResolveChatHistoryProvider(chatOptions, session);
|
||||
ChatHistoryProvider? chatHistoryProvider = this.ResolveChatHistoryProvider(chatOptions);
|
||||
|
||||
if (chatHistoryProvider is not null)
|
||||
{
|
||||
@@ -701,7 +701,7 @@ public sealed partial class ChatClientAgent : AIAgent
|
||||
throw new InvalidOperationException("A session must be provided when continuing a background response with a continuation token.");
|
||||
}
|
||||
|
||||
if ((continuationToken is not null || chatOptions?.AllowBackgroundResponses is true) && this.PersistsChatHistoryPerServiceCall && this._logger.IsEnabled(LogLevel.Warning))
|
||||
if ((continuationToken is not null || chatOptions?.AllowBackgroundResponses is true) && this.SimulatesServiceStoredChatHistory && this._logger.IsEnabled(LogLevel.Warning))
|
||||
{
|
||||
var warningAgentName = this.GetLoggingAgentName();
|
||||
this._logger.LogAgentChatClientBackgroundResponseFallback(this.Id, warningAgentName);
|
||||
@@ -719,57 +719,6 @@ public sealed partial class ChatClientAgent : AIAgent
|
||||
throw new InvalidOperationException("Input messages are not allowed when continuing a background response using a continuation token.");
|
||||
}
|
||||
|
||||
IEnumerable<ChatMessage> inputMessagesForChatClient = inputMessages;
|
||||
|
||||
// Populate the session messages only if we are not continuing an existing response as it's not allowed
|
||||
if (chatOptions?.ContinuationToken is null)
|
||||
{
|
||||
ChatHistoryProvider? chatHistoryProvider = this.ResolveChatHistoryProvider(chatOptions, typedSession);
|
||||
|
||||
// Add any existing messages from the session to the messages to be sent to the chat client.
|
||||
// The ChatHistoryProvider returns the merged result (history + input messages).
|
||||
if (chatHistoryProvider is not null)
|
||||
{
|
||||
var invokingContext = new ChatHistoryProvider.InvokingContext(this, typedSession, inputMessagesForChatClient);
|
||||
inputMessagesForChatClient = await chatHistoryProvider.InvokingAsync(invokingContext, cancellationToken).ConfigureAwait(false);
|
||||
}
|
||||
|
||||
// If we have an AIContextProvider, we should get context from it, and update our
|
||||
// messages and options with the additional context.
|
||||
// The AIContextProvider returns the accumulated AIContext (original + new contributions).
|
||||
if (this.AIContextProviders is { Count: > 0 } aiContextProviders)
|
||||
{
|
||||
var aiContext = new AIContext
|
||||
{
|
||||
Instructions = chatOptions?.Instructions,
|
||||
Messages = inputMessagesForChatClient,
|
||||
Tools = chatOptions?.Tools
|
||||
};
|
||||
|
||||
foreach (var aiContextProvider in aiContextProviders)
|
||||
{
|
||||
var invokingContext = new AIContextProvider.InvokingContext(this, typedSession, aiContext);
|
||||
aiContext = await aiContextProvider.InvokingAsync(invokingContext, cancellationToken).ConfigureAwait(false);
|
||||
}
|
||||
|
||||
// Materialize the accumulated messages and tools once at the end of the provider pipeline.
|
||||
inputMessagesForChatClient = aiContext.Messages ?? [];
|
||||
|
||||
var tools = aiContext.Tools as IList<AITool> ?? aiContext.Tools?.ToList();
|
||||
if (chatOptions?.Tools is { Count: > 0 } || tools is { Count: > 0 })
|
||||
{
|
||||
chatOptions ??= new();
|
||||
chatOptions.Tools = tools;
|
||||
}
|
||||
|
||||
if (chatOptions?.Instructions is not null || aiContext.Instructions is not null)
|
||||
{
|
||||
chatOptions ??= new();
|
||||
chatOptions.Instructions = aiContext.Instructions;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// If a user provided two different session ids, via the session object and options, we should throw
|
||||
// since we don't know which one to use.
|
||||
if (!string.IsNullOrWhiteSpace(typedSession.ConversationId) && !string.IsNullOrWhiteSpace(chatOptions?.ConversationId) && typedSession.ConversationId != chatOptions!.ConversationId)
|
||||
@@ -788,12 +737,53 @@ public sealed partial class ChatClientAgent : AIAgent
|
||||
chatOptions.ConversationId = typedSession.ConversationId;
|
||||
}
|
||||
|
||||
// When per-service-call persistence is active, set a sentinel conversation ID so that
|
||||
// FunctionInvokingChatClient treats locally-persisted history the same as service-managed
|
||||
// history. This prevents it from adding duplicate FunctionCallContent messages into the
|
||||
// request when processing approval responses — the loaded history already contains them.
|
||||
// ChatHistoryPersistingChatClient strips the sentinel before forwarding to the inner client.
|
||||
chatOptions = this.SetLocalHistoryConversationIdIfNeeded(chatOptions);
|
||||
IEnumerable<ChatMessage> inputMessagesForChatClient = inputMessages;
|
||||
|
||||
// Populate the session messages only if we are not continuing an existing response as it's not allowed.
|
||||
// When SimulateServiceStoredChatHistory is active, the ServiceStoredSimulatingChatClient
|
||||
// owns the chat history lifecycle — it loads history before each service call. The agent
|
||||
// must not load history itself, as that would result in duplicate messages.
|
||||
if (chatOptions?.ContinuationToken is null && !this.SimulatesServiceStoredChatHistory)
|
||||
{
|
||||
// Add any existing messages from the session to the messages to be sent to the chat client.
|
||||
// The ChatHistoryProvider returns the merged result (history + input messages).
|
||||
inputMessagesForChatClient = await this.LoadChatHistoryAsync(typedSession, inputMessagesForChatClient, chatOptions, cancellationToken).ConfigureAwait(false);
|
||||
}
|
||||
|
||||
// AIContextProviders should always be invoked (unless continuing an existing response)
|
||||
// to contribute additional messages, tools, and instructions — even when the decorator
|
||||
// handles history loading.
|
||||
if (chatOptions?.ContinuationToken is null && this.AIContextProviders is { Count: > 0 } aiContextProviders)
|
||||
{
|
||||
var aiContext = new AIContext
|
||||
{
|
||||
Instructions = chatOptions?.Instructions,
|
||||
Messages = inputMessagesForChatClient,
|
||||
Tools = chatOptions?.Tools
|
||||
};
|
||||
|
||||
foreach (var aiContextProvider in aiContextProviders)
|
||||
{
|
||||
var invokingContext = new AIContextProvider.InvokingContext(this, typedSession, aiContext);
|
||||
aiContext = await aiContextProvider.InvokingAsync(invokingContext, cancellationToken).ConfigureAwait(false);
|
||||
}
|
||||
|
||||
// Materialize the accumulated messages and tools once at the end of the provider pipeline.
|
||||
inputMessagesForChatClient = aiContext.Messages ?? [];
|
||||
|
||||
var tools = aiContext.Tools as IList<AITool> ?? aiContext.Tools?.ToList();
|
||||
if (chatOptions?.Tools is { Count: > 0 } || tools is { Count: > 0 })
|
||||
{
|
||||
chatOptions ??= new();
|
||||
chatOptions.Tools = tools;
|
||||
}
|
||||
|
||||
if (chatOptions?.Instructions is not null || aiContext.Instructions is not null)
|
||||
{
|
||||
chatOptions ??= new();
|
||||
chatOptions.Instructions = aiContext.Instructions;
|
||||
}
|
||||
}
|
||||
|
||||
// Materialize the accumulated messages once at the end of the provider pipeline, reusing the existing list if possible.
|
||||
List<ChatMessage> messagesList = inputMessagesForChatClient as List<ChatMessage> ?? inputMessagesForChatClient.ToList();
|
||||
@@ -839,8 +829,6 @@ public sealed partial class ChatClientAgent : AIAgent
|
||||
}
|
||||
}
|
||||
|
||||
// If we got a conversation id back from the chat client, it means that the service supports server side session storage
|
||||
// so we should update the session with the new id.
|
||||
session.ConversationId = responseConversationId;
|
||||
}
|
||||
}
|
||||
@@ -849,14 +837,14 @@ public sealed partial class ChatClientAgent : AIAgent
|
||||
/// Updates the session conversation ID at the end of an agent run.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// When a <see cref="ChatHistoryPersistingChatClient"/> in persist mode handles per-service-call
|
||||
/// conversation ID updates, this end-of-run update is skipped. When the decorator is in mark-only
|
||||
/// mode or absent, the update is performed here. When <paramref name="forceUpdate"/> is <see langword="true"/>
|
||||
/// When a <see cref="ServiceStoredSimulatingChatClient"/> handles per-service-call
|
||||
/// conversation ID updates, this end-of-run update is skipped. When the decorator is
|
||||
/// absent, the update is performed here. When <paramref name="forceUpdate"/> is <see langword="true"/>
|
||||
/// (continuation token scenarios), the update is always performed.
|
||||
/// </remarks>
|
||||
private void UpdateSessionConversationIdAtEndOfRun(ChatClientAgentSession session, string? responseConversationId, CancellationToken cancellationToken, bool forceUpdate = false)
|
||||
{
|
||||
if (!forceUpdate && this.PersistsChatHistoryPerServiceCall)
|
||||
if (!forceUpdate && this.SimulatesServiceStoredChatHistory)
|
||||
{
|
||||
return;
|
||||
}
|
||||
@@ -868,10 +856,9 @@ public sealed partial class ChatClientAgent : AIAgent
|
||||
/// Notifies providers of successfully completed messages at the end of an agent run.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// When a <see cref="ChatHistoryPersistingChatClient"/> in persist mode handles per-service-call
|
||||
/// notification, this end-of-run notification is skipped. When the decorator is in mark-only mode,
|
||||
/// only the marked messages are persisted. When no decorator is present (custom stack with
|
||||
/// <see cref="ChatClientAgentOptions.PersistChatHistoryAtEndOfRun"/>), all messages are persisted.
|
||||
/// When a <see cref="ServiceStoredSimulatingChatClient"/> handles per-service-call
|
||||
/// notification, this end-of-run notification is skipped. When no decorator is present,
|
||||
/// all messages are persisted.
|
||||
/// When <paramref name="forceNotify"/> is <see langword="true"/> (continuation token or
|
||||
/// background response scenarios), notification is always performed with all messages because
|
||||
/// per-service-call persistence is unreliable in these scenarios.
|
||||
@@ -884,19 +871,11 @@ public sealed partial class ChatClientAgent : AIAgent
|
||||
CancellationToken cancellationToken,
|
||||
bool forceNotify = false)
|
||||
{
|
||||
if (!forceNotify && this.PersistsChatHistoryPerServiceCall)
|
||||
if (!forceNotify && this.SimulatesServiceStoredChatHistory)
|
||||
{
|
||||
return Task.CompletedTask;
|
||||
}
|
||||
|
||||
if (!forceNotify && this.HasMarkOnlyChatHistoryPersistingClient)
|
||||
{
|
||||
// In mark-only mode, persist only messages that were marked by the decorator.
|
||||
var markedRequestMessages = GetMarkedMessages(requestMessages);
|
||||
var markedResponseMessages = GetMarkedMessages(responseMessages);
|
||||
return this.NotifyProvidersOfNewMessagesAsync(session, markedRequestMessages, markedResponseMessages, chatOptions, cancellationToken);
|
||||
}
|
||||
|
||||
return this.NotifyProvidersOfNewMessagesAsync(session, requestMessages, responseMessages, chatOptions, cancellationToken);
|
||||
}
|
||||
|
||||
@@ -904,7 +883,7 @@ public sealed partial class ChatClientAgent : AIAgent
|
||||
/// Notifies providers of a failure at the end of an agent run.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// When a <see cref="ChatHistoryPersistingChatClient"/> in persist mode handles per-service-call
|
||||
/// When a <see cref="ServiceStoredSimulatingChatClient"/> handles per-service-call
|
||||
/// notification (including failure), this end-of-run notification is skipped to avoid
|
||||
/// duplicate notification. In all other cases, failure is reported at the end of the run.
|
||||
/// </remarks>
|
||||
@@ -915,7 +894,7 @@ public sealed partial class ChatClientAgent : AIAgent
|
||||
ChatOptions? chatOptions,
|
||||
CancellationToken cancellationToken)
|
||||
{
|
||||
if (this.PersistsChatHistoryPerServiceCall)
|
||||
if (this.SimulatesServiceStoredChatHistory)
|
||||
{
|
||||
return Task.CompletedTask;
|
||||
}
|
||||
@@ -924,60 +903,19 @@ public sealed partial class ChatClientAgent : AIAgent
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Gets a value indicating whether the agent has a <see cref="ChatHistoryPersistingChatClient"/>
|
||||
/// decorator in persist mode (not mark-only), which handles per-service-call persistence.
|
||||
/// Gets a value indicating whether the agent is configured to simulate service-stored chat history.
|
||||
/// When <see langword="true"/>, end-of-run persistence and history loading are skipped because a
|
||||
/// per-service-call decorator (such as <see cref="ServiceStoredSimulatingChatClient"/> or a
|
||||
/// user-supplied equivalent) is expected to handle the history lifecycle.
|
||||
/// </summary>
|
||||
private bool PersistsChatHistoryPerServiceCall
|
||||
private bool SimulatesServiceStoredChatHistory
|
||||
{
|
||||
get
|
||||
{
|
||||
var persistingClient = this.ChatClient.GetService<ChatHistoryPersistingChatClient>();
|
||||
return persistingClient?.MarkOnly == false;
|
||||
return this._agentOptions?.SimulateServiceStoredChatHistory is true;
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Sets the <see cref="ChatHistoryPersistingChatClient.LocalHistoryConversationId"/> sentinel on
|
||||
/// <paramref name="chatOptions"/> when per-service-call persistence is active and no real
|
||||
/// conversation ID is present.
|
||||
/// </summary>
|
||||
/// <returns>
|
||||
/// The (possibly new) <see cref="ChatOptions"/> with the sentinel set, or the original
|
||||
/// <paramref name="chatOptions"/> if no sentinel is needed.
|
||||
/// </returns>
|
||||
private ChatOptions? SetLocalHistoryConversationIdIfNeeded(ChatOptions? chatOptions)
|
||||
{
|
||||
if (this.PersistsChatHistoryPerServiceCall && string.IsNullOrWhiteSpace(chatOptions?.ConversationId))
|
||||
{
|
||||
chatOptions ??= new ChatOptions();
|
||||
chatOptions.ConversationId = ChatHistoryPersistingChatClient.LocalHistoryConversationId;
|
||||
}
|
||||
|
||||
return chatOptions;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Gets a value indicating whether the agent has a <see cref="ChatHistoryPersistingChatClient"/>
|
||||
/// decorator in mark-only mode, which marks messages for later persistence at the end of the run.
|
||||
/// </summary>
|
||||
private bool HasMarkOnlyChatHistoryPersistingClient
|
||||
{
|
||||
get
|
||||
{
|
||||
var persistingClient = this.ChatClient.GetService<ChatHistoryPersistingChatClient>();
|
||||
return persistingClient?.MarkOnly == true;
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Returns only the messages that have been marked as persisted by a <see cref="ChatHistoryPersistingChatClient"/> in mark-only mode.
|
||||
/// </summary>
|
||||
private static List<ChatMessage> GetMarkedMessages(IEnumerable<ChatMessage> messages)
|
||||
{
|
||||
return messages.Where(m =>
|
||||
m.AdditionalProperties?.TryGetValue(ChatHistoryPersistingChatClient.PersistedMarkerKey, out var value) == true && value is true).ToList();
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Ensures that <see cref="AIAgent.CurrentRunContext"/> contains the resolved session.
|
||||
/// </summary>
|
||||
@@ -985,7 +923,7 @@ public sealed partial class ChatClientAgent : AIAgent
|
||||
/// The base class sets <see cref="AIAgent.CurrentRunContext"/> with the raw session parameter
|
||||
/// (which may be null) and restores it after each yield in streaming scenarios. After
|
||||
/// <see cref="PrepareSessionAndMessagesAsync"/> resolves or creates a session, we update the
|
||||
/// context so the <see cref="ChatHistoryPersistingChatClient"/> decorator always has a valid session.
|
||||
/// context so the <see cref="ServiceStoredSimulatingChatClient"/> decorator always has a valid session.
|
||||
/// The original agent from the context is preserved to maintain the top-of-stack agent in
|
||||
/// decorated agent scenarios.
|
||||
/// </remarks>
|
||||
@@ -1001,19 +939,19 @@ public sealed partial class ChatClientAgent : AIAgent
|
||||
/// <summary>
|
||||
/// Checks for potential misconfiguration when using a custom chat client stack and logs warnings.
|
||||
/// </summary>
|
||||
private void WarnOnMissingPersistingClient()
|
||||
private void WarnOnMissingServiceStoredSimulatingClient()
|
||||
{
|
||||
if (this._agentOptions?.UseProvidedChatClientAsIs is not true)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
if (this._agentOptions?.PersistChatHistoryAtEndOfRun is not true)
|
||||
if (this._agentOptions?.SimulateServiceStoredChatHistory is not true)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
var persistingClient = this.ChatClient.GetService<ChatHistoryPersistingChatClient>();
|
||||
var persistingClient = this.ChatClient.GetService<ServiceStoredSimulatingChatClient>();
|
||||
if (persistingClient is null && this._logger.IsEnabled(LogLevel.Warning))
|
||||
{
|
||||
var loggingAgentName = this.GetLoggingAgentName();
|
||||
@@ -1023,14 +961,14 @@ public sealed partial class ChatClientAgent : AIAgent
|
||||
}
|
||||
}
|
||||
|
||||
private ChatHistoryProvider? ResolveChatHistoryProvider(ChatOptions? chatOptions, ChatClientAgentSession session)
|
||||
private ChatHistoryProvider? ResolveChatHistoryProvider(ChatOptions? chatOptions)
|
||||
{
|
||||
ChatHistoryProvider? provider = session.ConversationId is null ? this.ChatHistoryProvider : null;
|
||||
ChatHistoryProvider? provider = chatOptions?.ConversationId is null ? this.ChatHistoryProvider : null;
|
||||
|
||||
// If someone provided an override ChatHistoryProvider via AdditionalProperties, we should use that instead.
|
||||
if (chatOptions?.AdditionalProperties?.TryGetValue(out ChatHistoryProvider? overrideProvider) is true)
|
||||
{
|
||||
if (session.ConversationId is not null && overrideProvider is not null)
|
||||
if (this._agentOptions?.ThrowOnChatHistoryProviderConflict is true && string.IsNullOrWhiteSpace(chatOptions?.ConversationId) is false)
|
||||
{
|
||||
throw new InvalidOperationException(
|
||||
$"Only {nameof(ChatClientAgentSession.ConversationId)} or {nameof(this.ChatHistoryProvider)} may be used, but not both. The current {nameof(ChatClientAgentSession)} has a {nameof(ChatClientAgentSession.ConversationId)} indicating server-side chat history management, but an override {nameof(this.ChatHistoryProvider)} was provided via {nameof(AgentRunOptions.AdditionalProperties)}.");
|
||||
@@ -1055,6 +993,29 @@ public sealed partial class ChatClientAgent : AIAgent
|
||||
return provider;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Loads chat history from the resolved <see cref="ChatHistoryProvider"/> and prepends it to the given messages.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// This method is used by both the agent (during <see cref="PrepareSessionAndMessagesAsync"/>) and by
|
||||
/// <see cref="ServiceStoredSimulatingChatClient"/> to load history before each service call.
|
||||
/// </remarks>
|
||||
internal async Task<IEnumerable<ChatMessage>> LoadChatHistoryAsync(
|
||||
ChatClientAgentSession session,
|
||||
IEnumerable<ChatMessage> messages,
|
||||
ChatOptions? chatOptions,
|
||||
CancellationToken cancellationToken)
|
||||
{
|
||||
var chatHistoryProvider = this.ResolveChatHistoryProvider(chatOptions);
|
||||
if (chatHistoryProvider is null)
|
||||
{
|
||||
return messages;
|
||||
}
|
||||
|
||||
var invokingContext = new ChatHistoryProvider.InvokingContext(this, session, messages);
|
||||
return await chatHistoryProvider.InvokingAsync(invokingContext, cancellationToken).ConfigureAwait(false);
|
||||
}
|
||||
|
||||
private static ChatClientAgentContinuationToken? WrapContinuationToken(ResponseContinuationToken? continuationToken, IEnumerable<ChatMessage>? inputMessages = null, List<ChatResponseUpdate>? responseUpdates = null)
|
||||
{
|
||||
if (continuationToken is null)
|
||||
|
||||
@@ -72,12 +72,12 @@ internal static partial class ChatClientAgentLogMessages
|
||||
|
||||
/// <summary>
|
||||
/// Logs a warning when <see cref="ChatClientAgentOptions.UseProvidedChatClientAsIs"/> is <see langword="true"/>
|
||||
/// and <see cref="ChatClientAgentOptions.PersistChatHistoryAtEndOfRun"/> is <see langword="true"/>,
|
||||
/// but no <see cref="ChatHistoryPersistingChatClient"/> is found in the custom chat client stack.
|
||||
/// and <see cref="ChatClientAgentOptions.SimulateServiceStoredChatHistory"/> is <see langword="true"/>,
|
||||
/// but no <see cref="ServiceStoredSimulatingChatClient"/> is found in the custom chat client stack.
|
||||
/// </summary>
|
||||
[LoggerMessage(
|
||||
Level = LogLevel.Warning,
|
||||
Message = "Agent {AgentId}/{AgentName}: PersistChatHistoryAtEndOfRun is enabled with a custom chat client stack (UseProvidedChatClientAsIs), but no ChatHistoryPersistingChatClient was found in the pipeline. All messages will be persisted at the end of the run without marking. This setup is not supported with some other features, e.g. handoffs. Consider adding a ChatHistoryPersistingChatClient to the pipeline using the UseChatHistoryPersisting extension method.")]
|
||||
Message = "Agent {AgentId}/{AgentName}: SimulateServiceStoredChatHistory is enabled with a custom chat client stack (UseProvidedChatClientAsIs), but no ServiceStoredSimulatingChatClient was found in the pipeline. Chat history will not be persisted by ChatClientAgent. Consider adding a ServiceStoredSimulatingChatClient to the pipeline using the UseServiceStoredChatHistorySimulation extension method if you have not added your own persistence mechanism.")]
|
||||
public static partial void LogAgentChatClientMissingPersistingClient(
|
||||
this ILogger logger,
|
||||
string agentId,
|
||||
@@ -92,7 +92,7 @@ internal static partial class ChatClientAgentLogMessages
|
||||
/// </summary>
|
||||
[LoggerMessage(
|
||||
Level = LogLevel.Warning,
|
||||
Message = "Agent {AgentId}/{AgentName}: Per-service-call persistence is falling back to end-of-run persistence because the run involves background responses. Messages will be marked during the run and persisted at the end.")]
|
||||
Message = "Agent {AgentId}/{AgentName}: SimulateServiceStoredChatHistory is enabled but we have to fall back to end-of-run persistence because the run involves background responses.")]
|
||||
public static partial void LogAgentChatClientBackgroundResponseFallback(
|
||||
this ILogger logger,
|
||||
string agentId,
|
||||
|
||||
@@ -92,54 +92,46 @@ public sealed class ChatClientAgentOptions
|
||||
public bool ThrowOnChatHistoryProviderConflict { get; set; } = true;
|
||||
|
||||
/// <summary>
|
||||
/// Gets or sets a value indicating whether to persist chat history only at the end of the full agent run
|
||||
/// rather than after each individual service call.
|
||||
/// Gets or sets a value indicating whether the <see cref="ChatClientAgent"/> should simulate
|
||||
/// service-stored chat history behavior using its configured <see cref="ChatHistoryProvider"/>.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// <para>
|
||||
/// By default, <see cref="ChatClientAgent"/> persists request and response messages either via
|
||||
/// a <see cref="ChatHistoryProvider"/>, or the underlying AI service's chat history storage.
|
||||
/// Persistence is done immediately after each call to the AI service within the function invocation loop.
|
||||
/// When storing in the underlying AI service, the session's <see cref="ChatClientAgentSession.ConversationId"/>
|
||||
/// is also updated after each service call, keeping it in sync with the service-side conversation state.
|
||||
/// When set to <see langword="true"/>, a <see cref="ServiceStoredSimulatingChatClient"/> decorator is
|
||||
/// injected between the <see cref="FunctionInvokingChatClient"/> and the leaf <see cref="IChatClient"/>
|
||||
/// in the chat client pipeline. This decorator takes full ownership of the chat history lifecycle:
|
||||
/// it loads history from the <see cref="ChatHistoryProvider"/> before each service call and persists
|
||||
/// new messages after each service call. It also returns a sentinel <see cref="ChatOptions.ConversationId"/>
|
||||
/// on the response, causing the <see cref="FunctionInvokingChatClient"/> to treat the conversation
|
||||
/// as service-managed — clearing accumulated history and not injecting duplicate
|
||||
/// <see cref="FunctionCallContent"/> during approval-response processing.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// Setting this property to <see langword="true"/> causes messages to be marked during the function
|
||||
/// invocation loop but persisted only at the end of the full agent run, providing atomic run semantics.
|
||||
/// Updating the <see cref="ChatClientAgentSession.ConversationId"/> is likewise deferred and
|
||||
/// updated only at the end of the run, consistent with atomic run semantics.
|
||||
/// A <see cref="ChatHistoryPersistingChatClient"/> decorator is inserted into the chat client pipeline
|
||||
/// in mark-only mode, and the <see cref="ChatClientAgent"/> persists only the marked messages at the
|
||||
/// end of the run.
|
||||
/// This mode aligns the behavior of framework-managed chat history with service-stored chat history,
|
||||
/// ensuring consistency in how messages are stored and loaded, including during function calling loops
|
||||
/// and tool-call termination scenarios.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// When this option is <see langword="false"/> (the default), the <see cref="ChatHistoryPersistingChatClient"/>
|
||||
/// decorator persists messages and updates the <see cref="ChatClientAgentSession.ConversationId"/>
|
||||
/// immediately after each service call. This may leave chat history in a state where
|
||||
/// <see cref="FunctionResultContent"/> is required to start a new run if the last successful service
|
||||
/// call returned <see cref="FunctionCallContent"/>.
|
||||
/// When set to <see langword="false"/> (the default), the <see cref="ChatClientAgent"/> handles
|
||||
/// chat history persistence at the end of the full agent run via the <see cref="ChatHistoryProvider"/>
|
||||
/// pipeline.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// This option has no effect when <see cref="UseProvidedChatClientAsIs"/> is <see langword="true"/>.
|
||||
/// When using a custom chat client stack, you can add a <see cref="ChatHistoryPersistingChatClient"/>
|
||||
/// manually via the <see cref="ChatClientBuilderExtensions.UseChatHistoryPersisting"/>
|
||||
/// When setting the <see cref="UseProvidedChatClientAsIs"/> setting to <see langword="true"/> and
|
||||
/// <see cref="SimulateServiceStoredChatHistory"/> to <see langword="true"/>, ensure that your custom chat client stack includes a
|
||||
/// <see cref="ServiceStoredSimulatingChatClient"/> to enable per-service-call persistence.
|
||||
/// If no <see cref="ServiceStoredSimulatingChatClient"/> is provided, and you are not storing chat history via other means,
|
||||
/// no chat history may be stored.
|
||||
/// When using a custom chat client stack, you can add a <see cref="ServiceStoredSimulatingChatClient"/>
|
||||
/// manually via the <see cref="ChatClientBuilderExtensions.UseServiceStoredChatHistorySimulation"/>
|
||||
/// extension method.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// Note that when using single threaded service stored chat history, like OpenAI Conversations,
|
||||
/// there is only one id, so even if the conversation id is not updated after each service call,
|
||||
/// the chat history will still contain intermediate messages. Setting this property to <see langword="true"/>
|
||||
/// in this case will therefore have no real effect. Setting this property to <see langword="true"/> when using
|
||||
/// OpenAI Responses with response ids on the other hand, allows atomic run semantics, since
|
||||
/// each service request produces a new response id, and if the run fails mid-loop, the session will
|
||||
/// still contain the pre-run respnose id, allowing the next run to start with a clean slate.
|
||||
/// </para>
|
||||
/// </remarks>
|
||||
/// <value>
|
||||
/// Default is <see langword="false"/>.
|
||||
/// </value>
|
||||
[Experimental(DiagnosticIds.Experiments.AgentsAIExperiments)]
|
||||
public bool PersistChatHistoryAtEndOfRun { get; set; }
|
||||
public bool SimulateServiceStoredChatHistory { get; set; }
|
||||
|
||||
/// <summary>
|
||||
/// Creates a new instance of <see cref="ChatClientAgentOptions"/> with the same values as this instance.
|
||||
@@ -157,6 +149,6 @@ public sealed class ChatClientAgentOptions
|
||||
ClearOnChatHistoryProviderConflict = this.ClearOnChatHistoryProviderConflict,
|
||||
WarnOnChatHistoryProviderConflict = this.WarnOnChatHistoryProviderConflict,
|
||||
ThrowOnChatHistoryProviderConflict = this.ThrowOnChatHistoryProviderConflict,
|
||||
PersistChatHistoryAtEndOfRun = this.PersistChatHistoryAtEndOfRun,
|
||||
SimulateServiceStoredChatHistory = this.SimulateServiceStoredChatHistory,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -86,25 +86,21 @@ public static class ChatClientBuilderExtensions
|
||||
services: services);
|
||||
|
||||
/// <summary>
|
||||
/// Adds a <see cref="ChatHistoryPersistingChatClient"/> to the chat client pipeline.
|
||||
/// Adds a <see cref="ServiceStoredSimulatingChatClient"/> to the chat client pipeline.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// <para>
|
||||
/// This decorator should be positioned between the <see cref="FunctionInvokingChatClient"/> and the leaf
|
||||
/// <see cref="IChatClient"/> in the pipeline. It intercepts service calls to either persist messages
|
||||
/// immediately or mark them for later persistence, depending on the <paramref name="markOnly"/> parameter.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// If <paramref name="markOnly"/> is set to <see langword="true"/>, the <see cref="ChatClientAgent"/>
|
||||
/// should be configured with <see cref="ChatClientAgentOptions.PersistChatHistoryAtEndOfRun"/> set to <see langword="true"/>
|
||||
/// as without this combination, messages will never be persisted when using a <see cref="ChatHistoryProvider"/> for
|
||||
/// chat history persistence.
|
||||
/// <see cref="IChatClient"/> in the pipeline. It simulates service-stored chat history behavior by
|
||||
/// loading history before each service call, persisting after each call, and returning a sentinel
|
||||
/// <see cref="ChatOptions.ConversationId"/> on the response.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// This extension method is intended for use with custom chat client stacks when
|
||||
/// <see cref="ChatClientAgentOptions.UseProvidedChatClientAsIs"/> is <see langword="true"/>.
|
||||
/// When <see cref="ChatClientAgentOptions.UseProvidedChatClientAsIs"/> is <see langword="false"/> (the default),
|
||||
/// the <see cref="ChatClientAgent"/> automatically injects this decorator.
|
||||
/// the <see cref="ChatClientAgent"/> automatically injects this decorator when
|
||||
/// <see cref="ChatClientAgentOptions.SimulateServiceStoredChatHistory"/> is <see langword="true"/>.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// This decorator only works within the context of a running <see cref="ChatClientAgent"/> and will throw an
|
||||
@@ -112,18 +108,10 @@ public static class ChatClientBuilderExtensions
|
||||
/// </para>
|
||||
/// </remarks>
|
||||
/// <param name="builder">The <see cref="ChatClientBuilder"/> to add the decorator to.</param>
|
||||
/// <param name="markOnly">
|
||||
/// When <see langword="true"/>, messages are marked with metadata but not persisted immediately,
|
||||
/// and the session's <see cref="ChatClientAgentSession.ConversationId"/> is not updated.
|
||||
/// The <see cref="ChatClientAgent"/> will persist only the marked messages and update the
|
||||
/// conversation ID at the end of the run.
|
||||
/// When <see langword="false"/> (the default), messages are persisted and the conversation ID
|
||||
/// is updated immediately after each service call.
|
||||
/// </param>
|
||||
/// <returns>The <paramref name="builder"/> for chaining.</returns>
|
||||
[Experimental(DiagnosticIds.Experiments.AgentsAIExperiments)]
|
||||
public static ChatClientBuilder UseChatHistoryPersisting(this ChatClientBuilder builder, bool markOnly = false)
|
||||
public static ChatClientBuilder UseServiceStoredChatHistorySimulation(this ChatClientBuilder builder)
|
||||
{
|
||||
return builder.Use(innerClient => new ChatHistoryPersistingChatClient(innerClient, markOnly));
|
||||
return builder.Use(innerClient => new ServiceStoredSimulatingChatClient(innerClient));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -63,14 +63,17 @@ public static class ChatClientExtensions
|
||||
});
|
||||
}
|
||||
|
||||
// ChatHistoryPersistingChatClient is registered after FunctionInvokingChatClient so that it sits
|
||||
// between FIC and the leaf client. ChatClientBuilder.Build applies factories in reverse order,
|
||||
// making the first Use() call outermost. By adding our decorator second, the resulting pipeline is:
|
||||
// FunctionInvokingChatClient → ChatHistoryPersistingChatClient → leaf IChatClient
|
||||
// This allows the decorator to persist messages after each individual service call within
|
||||
// FIC's function invocation loop, or to mark them for later persistence at the end of the run.
|
||||
bool markOnly = options?.PersistChatHistoryAtEndOfRun is true;
|
||||
chatBuilder.Use(innerClient => new ChatHistoryPersistingChatClient(innerClient, markOnly));
|
||||
// ServiceStoredSimulatingChatClient is only injected when SimulateServiceStoredChatHistory is enabled.
|
||||
// It is registered after FunctionInvokingChatClient so that it sits between FIC and the leaf client.
|
||||
// ChatClientBuilder.Build applies factories in reverse order, making the first Use() call outermost.
|
||||
// By adding our decorator second, the resulting pipeline is:
|
||||
// FunctionInvokingChatClient → ServiceStoredSimulatingChatClient → leaf IChatClient
|
||||
// This allows the decorator to simulate service-stored chat history by loading history before
|
||||
// each service call, persisting after each call, and returning a sentinel ConversationId.
|
||||
if (options?.SimulateServiceStoredChatHistory is true)
|
||||
{
|
||||
chatBuilder.Use(innerClient => new ServiceStoredSimulatingChatClient(innerClient));
|
||||
}
|
||||
|
||||
var agentChatClient = chatBuilder.Build(services);
|
||||
|
||||
|
||||
@@ -1,351 +0,0 @@
|
||||
// Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Linq;
|
||||
using System.Runtime.CompilerServices;
|
||||
using System.Threading;
|
||||
using System.Threading.Tasks;
|
||||
using Microsoft.Extensions.AI;
|
||||
|
||||
namespace Microsoft.Agents.AI;
|
||||
|
||||
/// <summary>
|
||||
/// A delegating chat client that notifies <see cref="ChatHistoryProvider"/> and <see cref="AIContextProvider"/>
|
||||
/// instances of request and response messages after each individual call to the inner chat client,
|
||||
/// or marks messages for later persistence depending on the configured mode.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// <para>
|
||||
/// This decorator is intended to operate between the <see cref="FunctionInvokingChatClient"/> and the leaf
|
||||
/// <see cref="IChatClient"/> in a <see cref="ChatClientAgent"/> pipeline.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// In persist mode (the default), it ensures that providers are notified and the session's
|
||||
/// <see cref="ChatClientAgentSession.ConversationId"/> is updated after each service call, so that
|
||||
/// intermediate messages (e.g., tool calls and results) are saved even if the process is interrupted
|
||||
/// mid-loop.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// In mark-only mode (<see cref="MarkOnly"/> is <see langword="true"/>), it marks messages with metadata
|
||||
/// but does not notify providers or update the <see cref="ChatClientAgentSession.ConversationId"/>.
|
||||
/// Both are deferred to the <see cref="ChatClientAgent"/> at the end of the run, providing atomic
|
||||
/// run semantics.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// This chat client must be used within the context of a running <see cref="ChatClientAgent"/>. It retrieves the
|
||||
/// current agent and session from <see cref="AIAgent.CurrentRunContext"/>, which is set automatically when an agent's
|
||||
/// <see cref="AIAgent.RunAsync(IEnumerable{ChatMessage}, AgentSession?, AgentRunOptions?, CancellationToken)"/> or
|
||||
/// <see cref="AIAgent.RunStreamingAsync(IEnumerable{ChatMessage}, AgentSession?, AgentRunOptions?, CancellationToken)"/>
|
||||
/// method is called. The <see cref="ChatClientAgent"/> ensures the run context always contains a resolved session,
|
||||
/// even when the caller passes null. An <see cref="InvalidOperationException"/> is thrown if no run context is
|
||||
/// available or if the agent is not a <see cref="ChatClientAgent"/>.
|
||||
/// </para>
|
||||
/// </remarks>
|
||||
internal sealed class ChatHistoryPersistingChatClient : DelegatingChatClient
|
||||
{
|
||||
/// <summary>
|
||||
/// The key used in <see cref="ChatMessage.AdditionalProperties"/> and <see cref="AIContent.AdditionalProperties"/>
|
||||
/// to mark messages and their content as already persisted to chat history.
|
||||
/// </summary>
|
||||
internal const string PersistedMarkerKey = "_chatHistoryPersisted";
|
||||
|
||||
/// <summary>
|
||||
/// A sentinel value set on <see cref="ChatOptions.ConversationId"/> by <see cref="ChatClientAgent"/>
|
||||
/// when per-service-call persistence is active and no real conversation ID exists.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// <para>
|
||||
/// This signals to <see cref="FunctionInvokingChatClient"/> that the chat history is being managed
|
||||
/// externally (by this decorator), which prevents it from adding duplicate <see cref="FunctionCallContent"/>
|
||||
/// messages into the request during approval-response processing. Without this sentinel,
|
||||
/// <see cref="FunctionInvokingChatClient"/> would reconstruct function-call messages from approval
|
||||
/// responses and append them to the original messages — but the loaded history already contains
|
||||
/// those same function calls, causing duplicate tool-call entries that the model rejects.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// This decorator strips the sentinel before forwarding requests to the inner client, so the
|
||||
/// underlying model never sees it.
|
||||
/// </para>
|
||||
/// </remarks>
|
||||
internal const string LocalHistoryConversationId = "_agent_local_history";
|
||||
|
||||
/// <summary>
|
||||
/// Initializes a new instance of the <see cref="ChatHistoryPersistingChatClient"/> class.
|
||||
/// </summary>
|
||||
/// <param name="innerClient">The underlying chat client that will handle the core operations.</param>
|
||||
/// <param name="markOnly">
|
||||
/// When <see langword="true"/>, messages are marked with metadata but not persisted immediately,
|
||||
/// and the session's <see cref="ChatClientAgentSession.ConversationId"/> is not updated.
|
||||
/// The <see cref="ChatClientAgent"/> will persist only the marked messages and update the
|
||||
/// conversation ID at the end of the run.
|
||||
/// When <see langword="false"/> (the default), messages are persisted and the conversation ID
|
||||
/// is updated immediately after each service call.
|
||||
/// </param>
|
||||
public ChatHistoryPersistingChatClient(IChatClient innerClient, bool markOnly = false)
|
||||
: base(innerClient)
|
||||
{
|
||||
this.MarkOnly = markOnly;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Gets a value indicating whether this decorator is in mark-only mode.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// When <see langword="true"/>, messages are marked with metadata but not persisted immediately,
|
||||
/// and the session's <see cref="ChatClientAgentSession.ConversationId"/> is not updated.
|
||||
/// Both are deferred to the <see cref="ChatClientAgent"/> at the end of the run.
|
||||
/// When <see langword="false"/>, messages are persisted and the conversation ID is updated
|
||||
/// after each service call.
|
||||
/// </remarks>
|
||||
public bool MarkOnly { get; }
|
||||
|
||||
/// <inheritdoc/>
|
||||
public override async Task<ChatResponse> GetResponseAsync(
|
||||
IEnumerable<ChatMessage> messages,
|
||||
ChatOptions? options = null,
|
||||
CancellationToken cancellationToken = default)
|
||||
{
|
||||
var (agent, session) = GetRequiredAgentAndSession();
|
||||
options = StripLocalHistoryConversationId(options);
|
||||
|
||||
ChatResponse response;
|
||||
try
|
||||
{
|
||||
response = await base.GetResponseAsync(messages, options, cancellationToken).ConfigureAwait(false);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
var newRequestMessagesOnFailure = GetNewRequestMessages(messages);
|
||||
await agent.NotifyProvidersOfFailureAsync(session, ex, newRequestMessagesOnFailure, options, cancellationToken).ConfigureAwait(false);
|
||||
throw;
|
||||
}
|
||||
|
||||
var newRequestMessages = GetNewRequestMessages(messages);
|
||||
|
||||
if (this.ShouldDeferPersistence(options))
|
||||
{
|
||||
// In mark-only mode or when resuming from a continuation token, just mark messages
|
||||
// for later persistence by ChatClientAgent. Conversation ID and provider notification
|
||||
// are deferred to end-of-run. For continuation tokens, the end-of-run handler needs
|
||||
// to send the combined data from both the previous and current runs.
|
||||
MarkAsPersisted(newRequestMessages);
|
||||
MarkAsPersisted(response.Messages);
|
||||
}
|
||||
else
|
||||
{
|
||||
// In persist mode, persist immediately and update conversation ID.
|
||||
agent.UpdateSessionConversationId(session, response.ConversationId, cancellationToken);
|
||||
await agent.NotifyProvidersOfNewMessagesAsync(session, newRequestMessages, response.Messages, options, cancellationToken).ConfigureAwait(false);
|
||||
MarkAsPersisted(newRequestMessages);
|
||||
MarkAsPersisted(response.Messages);
|
||||
}
|
||||
|
||||
return response;
|
||||
}
|
||||
|
||||
/// <inheritdoc/>
|
||||
public override async IAsyncEnumerable<ChatResponseUpdate> GetStreamingResponseAsync(
|
||||
IEnumerable<ChatMessage> messages,
|
||||
ChatOptions? options = null,
|
||||
[EnumeratorCancellation] CancellationToken cancellationToken = default)
|
||||
{
|
||||
var (agent, session) = GetRequiredAgentAndSession();
|
||||
options = StripLocalHistoryConversationId(options);
|
||||
|
||||
List<ChatResponseUpdate> responseUpdates = [];
|
||||
|
||||
IAsyncEnumerator<ChatResponseUpdate> enumerator;
|
||||
try
|
||||
{
|
||||
enumerator = base.GetStreamingResponseAsync(messages, options, cancellationToken).GetAsyncEnumerator(cancellationToken);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
var newRequestMessagesOnFailure = GetNewRequestMessages(messages);
|
||||
await agent.NotifyProvidersOfFailureAsync(session, ex, newRequestMessagesOnFailure, options, cancellationToken).ConfigureAwait(false);
|
||||
throw;
|
||||
}
|
||||
|
||||
bool hasUpdates;
|
||||
try
|
||||
{
|
||||
hasUpdates = await enumerator.MoveNextAsync().ConfigureAwait(false);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
var newRequestMessagesOnFailure = GetNewRequestMessages(messages);
|
||||
await agent.NotifyProvidersOfFailureAsync(session, ex, newRequestMessagesOnFailure, options, cancellationToken).ConfigureAwait(false);
|
||||
throw;
|
||||
}
|
||||
|
||||
while (hasUpdates)
|
||||
{
|
||||
var update = enumerator.Current;
|
||||
responseUpdates.Add(update);
|
||||
yield return update;
|
||||
|
||||
try
|
||||
{
|
||||
hasUpdates = await enumerator.MoveNextAsync().ConfigureAwait(false);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
var newRequestMessagesOnFailure = GetNewRequestMessages(messages);
|
||||
await agent.NotifyProvidersOfFailureAsync(session, ex, newRequestMessagesOnFailure, options, cancellationToken).ConfigureAwait(false);
|
||||
throw;
|
||||
}
|
||||
}
|
||||
|
||||
var chatResponse = responseUpdates.ToChatResponse();
|
||||
var newRequestMessages = GetNewRequestMessages(messages);
|
||||
|
||||
if (this.ShouldDeferPersistence(options))
|
||||
{
|
||||
// In mark-only mode or when resuming from a continuation token, just mark messages
|
||||
// for later persistence by ChatClientAgent. Conversation ID and provider notification
|
||||
// are deferred to end-of-run. For continuation tokens, the end-of-run handler needs
|
||||
// to send the combined data from both the previous and current runs.
|
||||
MarkAsPersisted(newRequestMessages);
|
||||
MarkAsPersisted(chatResponse.Messages);
|
||||
}
|
||||
else
|
||||
{
|
||||
// In persist mode, persist immediately and update conversation ID.
|
||||
agent.UpdateSessionConversationId(session, chatResponse.ConversationId, cancellationToken);
|
||||
await agent.NotifyProvidersOfNewMessagesAsync(session, newRequestMessages, chatResponse.Messages, options, cancellationToken).ConfigureAwait(false);
|
||||
MarkAsPersisted(newRequestMessages);
|
||||
MarkAsPersisted(chatResponse.Messages);
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Gets the current <see cref="ChatClientAgent"/> and <see cref="ChatClientAgentSession"/> from the run context.
|
||||
/// </summary>
|
||||
private static (ChatClientAgent Agent, ChatClientAgentSession Session) GetRequiredAgentAndSession()
|
||||
{
|
||||
var runContext = AIAgent.CurrentRunContext
|
||||
?? throw new InvalidOperationException(
|
||||
$"{nameof(ChatHistoryPersistingChatClient)} can only be used within the context of a running AIAgent. " +
|
||||
"Ensure that the chat client is being invoked as part of an AIAgent.RunAsync or AIAgent.RunStreamingAsync call.");
|
||||
|
||||
var chatClientAgent = runContext.Agent.GetService<ChatClientAgent>()
|
||||
?? throw new InvalidOperationException(
|
||||
$"{nameof(ChatHistoryPersistingChatClient)} can only be used with a {nameof(ChatClientAgent)}. " +
|
||||
$"The current agent is of type '{runContext.Agent.GetType().Name}'.");
|
||||
|
||||
if (runContext.Session is not ChatClientAgentSession chatClientAgentSession)
|
||||
{
|
||||
throw new InvalidOperationException(
|
||||
$"{nameof(ChatHistoryPersistingChatClient)} requires a {nameof(ChatClientAgentSession)}. " +
|
||||
$"The current session is of type '{runContext.Session?.GetType().Name ?? "null"}'.");
|
||||
}
|
||||
|
||||
return (chatClientAgent, chatClientAgentSession);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Determines whether persistence should be deferred to end-of-run instead of happening immediately.
|
||||
/// </summary>
|
||||
/// <returns>
|
||||
/// <see langword="true"/> when in <see cref="MarkOnly"/> mode, when the call is resuming from
|
||||
/// a continuation token (since the end-of-run handler needs to combine data from the previous
|
||||
/// and current runs), or when background responses are allowed (since the caller may stop
|
||||
/// consuming the stream mid-run, preventing the post-stream persistence code from executing).
|
||||
/// </returns>
|
||||
private bool ShouldDeferPersistence(ChatOptions? options)
|
||||
{
|
||||
return this.MarkOnly || options?.ContinuationToken is not null || options?.AllowBackgroundResponses is true;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Returns only the request messages that have not yet been persisted to chat history.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// A message is considered already persisted if any of the following is true:
|
||||
/// <list type="bullet">
|
||||
/// <item>It has the <see cref="PersistedMarkerKey"/> in its <see cref="ChatMessage.AdditionalProperties"/>.</item>
|
||||
/// <item>It has an <see cref="AgentRequestMessageSourceType"/> of <see cref="AgentRequestMessageSourceType.ChatHistory"/>
|
||||
/// (indicating it was loaded from chat history and does not need to be re-persisted).</item>
|
||||
/// <item>It has <see cref="ChatMessage.Contents"/> and all of its <see cref="AIContent"/> items have the
|
||||
/// <see cref="PersistedMarkerKey"/> in their <see cref="AIContent.AdditionalProperties"/>. This handles the
|
||||
/// streaming case where <see cref="FunctionInvokingChatClient"/> reconstructs <see cref="ChatMessage"/> objects
|
||||
/// independently via <c>ToChatResponse()</c>, producing different object references that share the same
|
||||
/// underlying <see cref="AIContent"/> instances.</item>
|
||||
/// </list>
|
||||
/// </remarks>
|
||||
/// <returns>A list of request messages that have not yet been persisted.</returns>
|
||||
/// <param name="messages">The full set of request messages to filter.</param>
|
||||
private static List<ChatMessage> GetNewRequestMessages(IEnumerable<ChatMessage> messages)
|
||||
{
|
||||
return messages.Where(m => !IsAlreadyPersisted(m)).ToList();
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Determines whether a message has already been persisted to chat history by this decorator.
|
||||
/// </summary>
|
||||
private static bool IsAlreadyPersisted(ChatMessage message)
|
||||
{
|
||||
if (message.AdditionalProperties?.TryGetValue(PersistedMarkerKey, out var value) == true && value is true)
|
||||
{
|
||||
return true;
|
||||
}
|
||||
|
||||
if (message.GetAgentRequestMessageSourceType() == AgentRequestMessageSourceType.ChatHistory)
|
||||
{
|
||||
return true;
|
||||
}
|
||||
|
||||
// In streaming mode, FunctionInvokingChatClient reconstructs ChatMessage objects via ToChatResponse()
|
||||
// independently, producing different ChatMessage instances. However, the underlying AIContent objects
|
||||
// (e.g., FunctionCallContent, FunctionResultContent) are shared references. Checking for markers on
|
||||
// AIContent handles dedup in this case.
|
||||
if (message.Contents.Count > 0 && message.Contents.All(c => c.AdditionalProperties?.TryGetValue(PersistedMarkerKey, out var value) == true && value is true))
|
||||
{
|
||||
return true;
|
||||
}
|
||||
|
||||
return false;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Marks the given messages as persisted by setting a marker on both the <see cref="ChatMessage"/>
|
||||
/// and each of its <see cref="AIContent"/> items.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// Both levels are marked because <see cref="FunctionInvokingChatClient"/> may reconstruct
|
||||
/// <see cref="ChatMessage"/> objects in streaming mode (losing the message-level marker),
|
||||
/// but the <see cref="AIContent"/> references are shared and retain their markers.
|
||||
/// </remarks>
|
||||
/// <param name="messages">The messages to mark as persisted.</param>
|
||||
private static void MarkAsPersisted(IEnumerable<ChatMessage> messages)
|
||||
{
|
||||
foreach (var message in messages)
|
||||
{
|
||||
message.AdditionalProperties ??= new();
|
||||
message.AdditionalProperties[PersistedMarkerKey] = true;
|
||||
|
||||
foreach (var content in message.Contents)
|
||||
{
|
||||
content.AdditionalProperties ??= new();
|
||||
content.AdditionalProperties[PersistedMarkerKey] = true;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// If the <paramref name="options"/> carry the <see cref="LocalHistoryConversationId"/> sentinel,
|
||||
/// returns a clone with the conversation ID cleared so the inner client never sees it.
|
||||
/// Otherwise returns the original <paramref name="options"/> unchanged.
|
||||
/// </summary>
|
||||
private static ChatOptions? StripLocalHistoryConversationId(ChatOptions? options)
|
||||
{
|
||||
if (options?.ConversationId == LocalHistoryConversationId)
|
||||
{
|
||||
options = options.Clone();
|
||||
options.ConversationId = null;
|
||||
}
|
||||
|
||||
return options;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,273 @@
|
||||
// Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Linq;
|
||||
using System.Runtime.CompilerServices;
|
||||
using System.Threading;
|
||||
using System.Threading.Tasks;
|
||||
using Microsoft.Extensions.AI;
|
||||
|
||||
namespace Microsoft.Agents.AI;
|
||||
|
||||
/// <summary>
|
||||
/// A delegating chat client that simulates service-stored chat history behavior using
|
||||
/// framework-managed <see cref="ChatHistoryProvider"/> instances.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// <para>
|
||||
/// This decorator is intended to operate between the <see cref="FunctionInvokingChatClient"/> and the leaf
|
||||
/// <see cref="IChatClient"/> in a <see cref="ChatClientAgent"/> pipeline.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// Before each service call, it loads chat history from the agent's <see cref="ChatHistoryProvider"/>
|
||||
/// and prepends it to the request messages. After each successful service call, it persists
|
||||
/// new request and response messages to the provider. It also returns a sentinel
|
||||
/// <see cref="ChatOptions.ConversationId"/> on the response so that the
|
||||
/// <see cref="FunctionInvokingChatClient"/> treats the conversation as service-managed —
|
||||
/// clearing accumulated history between iterations and not injecting duplicate
|
||||
/// <see cref="FunctionCallContent"/> during approval-response processing.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// This chat client must be used within the context of a running <see cref="ChatClientAgent"/>. It retrieves the
|
||||
/// current agent and session from <see cref="AIAgent.CurrentRunContext"/>, which is set automatically when an agent's
|
||||
/// <see cref="AIAgent.RunAsync(IEnumerable{ChatMessage}, AgentSession?, AgentRunOptions?, CancellationToken)"/> or
|
||||
/// <see cref="AIAgent.RunStreamingAsync(IEnumerable{ChatMessage}, AgentSession?, AgentRunOptions?, CancellationToken)"/>
|
||||
/// method is called. The <see cref="ChatClientAgent"/> ensures the run context always contains a resolved session,
|
||||
/// even when the caller passes null. An <see cref="InvalidOperationException"/> is thrown if no run context is
|
||||
/// available or if the agent is not a <see cref="ChatClientAgent"/>.
|
||||
/// </para>
|
||||
/// </remarks>
|
||||
internal sealed class ServiceStoredSimulatingChatClient : DelegatingChatClient
|
||||
{
|
||||
/// <summary>
|
||||
/// A sentinel value returned on <see cref="ChatResponse.ConversationId"/> to signal
|
||||
/// <see cref="FunctionInvokingChatClient"/> that chat history is being managed downstream.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// <para>
|
||||
/// When <see cref="FunctionInvokingChatClient"/> sees a non-null <see cref="ChatResponse.ConversationId"/>,
|
||||
/// it treats the conversation as service-managed: it clears accumulated history between
|
||||
/// iterations (via <c>FixupHistories</c>) and does not inject <see cref="FunctionCallContent"/>
|
||||
/// into the request during approval-response processing (via <c>ProcessFunctionApprovalResponses</c>).
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// This decorator strips the sentinel from <see cref="ChatOptions.ConversationId"/> on incoming
|
||||
/// requests before forwarding to the inner client, so the underlying model never sees it.
|
||||
/// </para>
|
||||
/// </remarks>
|
||||
internal const string LocalHistoryConversationId = "_agent_local_chat_history";
|
||||
|
||||
/// <summary>
|
||||
/// Initializes a new instance of the <see cref="ServiceStoredSimulatingChatClient"/> class.
|
||||
/// </summary>
|
||||
/// <param name="innerClient">The underlying chat client that will handle the core operations.</param>
|
||||
public ServiceStoredSimulatingChatClient(IChatClient innerClient)
|
||||
: base(innerClient)
|
||||
{
|
||||
}
|
||||
|
||||
/// <inheritdoc/>
|
||||
public override async Task<ChatResponse> GetResponseAsync(
|
||||
IEnumerable<ChatMessage> messages,
|
||||
ChatOptions? options = null,
|
||||
CancellationToken cancellationToken = default)
|
||||
{
|
||||
var (agent, session) = GetRequiredAgentAndSession();
|
||||
options = StripLocalHistoryConversationId(options);
|
||||
|
||||
bool isServiceManaged = !string.IsNullOrEmpty(options?.ConversationId);
|
||||
bool isContinuationOrBackground = options?.ContinuationToken is not null
|
||||
|| options?.AllowBackgroundResponses is true;
|
||||
bool skipSimulation = isServiceManaged || isContinuationOrBackground;
|
||||
|
||||
var newMessages = messages as IList<ChatMessage> ?? messages.ToList();
|
||||
|
||||
// When simulating, load history and prepend it. When the service manages
|
||||
// history (real ConversationId) or this is a continuation/background run,
|
||||
// just forward the input messages as-is.
|
||||
var messagesForService = skipSimulation
|
||||
? newMessages
|
||||
: await agent.LoadChatHistoryAsync(session, newMessages, options, cancellationToken).ConfigureAwait(false);
|
||||
|
||||
ChatResponse response;
|
||||
try
|
||||
{
|
||||
response = await base.GetResponseAsync(messagesForService, options, cancellationToken).ConfigureAwait(false);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
await agent.NotifyProvidersOfFailureAsync(session, ex, newMessages, options, cancellationToken).ConfigureAwait(false);
|
||||
throw;
|
||||
}
|
||||
|
||||
await agent.NotifyProvidersOfNewMessagesAsync(session, newMessages, response.Messages, options, cancellationToken).ConfigureAwait(false);
|
||||
|
||||
if (isContinuationOrBackground)
|
||||
{
|
||||
// Continuation/background run — the agent's forced end-of-run handles
|
||||
// session ConversationId and persistence; the decorator is a no-op.
|
||||
}
|
||||
else if (isServiceManaged || !string.IsNullOrEmpty(response.ConversationId))
|
||||
{
|
||||
// Service manages history — update session with the real ConversationId.
|
||||
agent.UpdateSessionConversationId(session, response.ConversationId, cancellationToken);
|
||||
}
|
||||
else
|
||||
{
|
||||
// Normal simulated path — set sentinel so FICC treats this as service-managed.
|
||||
SetSentinelConversationId(response, session);
|
||||
}
|
||||
|
||||
return response;
|
||||
}
|
||||
|
||||
/// <inheritdoc/>
|
||||
public override async IAsyncEnumerable<ChatResponseUpdate> GetStreamingResponseAsync(
|
||||
IEnumerable<ChatMessage> messages,
|
||||
ChatOptions? options = null,
|
||||
[EnumeratorCancellation] CancellationToken cancellationToken = default)
|
||||
{
|
||||
var (agent, session) = GetRequiredAgentAndSession();
|
||||
options = StripLocalHistoryConversationId(options);
|
||||
|
||||
bool isServiceManaged = !string.IsNullOrEmpty(options?.ConversationId);
|
||||
bool isContinuationOrBackground = options?.ContinuationToken is not null
|
||||
|| options?.AllowBackgroundResponses is true;
|
||||
bool skipSimulation = isServiceManaged || isContinuationOrBackground;
|
||||
|
||||
var newMessages = messages as IList<ChatMessage> ?? messages.ToList();
|
||||
|
||||
// When simulating, load history and prepend it. When the service manages
|
||||
// history (real ConversationId) or this is a continuation/background run,
|
||||
// just forward the input messages as-is.
|
||||
var messagesForService = skipSimulation
|
||||
? newMessages
|
||||
: await agent.LoadChatHistoryAsync(session, newMessages, options, cancellationToken).ConfigureAwait(false);
|
||||
|
||||
List<ChatResponseUpdate> responseUpdates = [];
|
||||
|
||||
IAsyncEnumerator<ChatResponseUpdate> enumerator;
|
||||
try
|
||||
{
|
||||
enumerator = base.GetStreamingResponseAsync(messagesForService, options, cancellationToken).GetAsyncEnumerator(cancellationToken);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
await agent.NotifyProvidersOfFailureAsync(session, ex, newMessages, options, cancellationToken).ConfigureAwait(false);
|
||||
throw;
|
||||
}
|
||||
|
||||
bool hasUpdates;
|
||||
try
|
||||
{
|
||||
hasUpdates = await enumerator.MoveNextAsync().ConfigureAwait(false);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
await agent.NotifyProvidersOfFailureAsync(session, ex, newMessages, options, cancellationToken).ConfigureAwait(false);
|
||||
throw;
|
||||
}
|
||||
|
||||
while (hasUpdates)
|
||||
{
|
||||
var update = enumerator.Current;
|
||||
responseUpdates.Add(update);
|
||||
|
||||
// If the service returned a real ConversationId on any update, remember that.
|
||||
// Otherwise stamp our sentinel so FICC treats this as service-managed —
|
||||
// unless this is a continuation/background run where the agent handles everything.
|
||||
if (!string.IsNullOrEmpty(update.ConversationId))
|
||||
{
|
||||
isServiceManaged = true;
|
||||
}
|
||||
else if (!skipSimulation)
|
||||
{
|
||||
update.ConversationId = LocalHistoryConversationId;
|
||||
}
|
||||
|
||||
yield return update;
|
||||
|
||||
try
|
||||
{
|
||||
hasUpdates = await enumerator.MoveNextAsync().ConfigureAwait(false);
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
await agent.NotifyProvidersOfFailureAsync(session, ex, newMessages, options, cancellationToken).ConfigureAwait(false);
|
||||
throw;
|
||||
}
|
||||
}
|
||||
|
||||
var chatResponse = responseUpdates.ToChatResponse();
|
||||
|
||||
await agent.NotifyProvidersOfNewMessagesAsync(session, newMessages, chatResponse.Messages, options, cancellationToken).ConfigureAwait(false);
|
||||
|
||||
if (isContinuationOrBackground)
|
||||
{
|
||||
// Continuation/background run — the agent's forced end-of-run handles
|
||||
// session ConversationId and persistence; the decorator is a no-op.
|
||||
}
|
||||
else if (isServiceManaged)
|
||||
{
|
||||
// Service manages history — update session with the real ConversationId.
|
||||
agent.UpdateSessionConversationId(session, chatResponse.ConversationId, cancellationToken);
|
||||
}
|
||||
else
|
||||
{
|
||||
// Normal simulated path — set sentinel on session.
|
||||
session.ConversationId = LocalHistoryConversationId;
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Sets the sentinel <see cref="LocalHistoryConversationId"/> on the response and session
|
||||
/// so that <see cref="FunctionInvokingChatClient"/> treats the conversation as service-managed.
|
||||
/// </summary>
|
||||
private static void SetSentinelConversationId(ChatResponse response, ChatClientAgentSession session)
|
||||
{
|
||||
response.ConversationId = LocalHistoryConversationId;
|
||||
session.ConversationId = LocalHistoryConversationId;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Gets the current <see cref="ChatClientAgent"/> and <see cref="ChatClientAgentSession"/> from the run context.
|
||||
/// </summary>
|
||||
private static (ChatClientAgent Agent, ChatClientAgentSession Session) GetRequiredAgentAndSession()
|
||||
{
|
||||
var runContext = AIAgent.CurrentRunContext
|
||||
?? throw new InvalidOperationException(
|
||||
$"{nameof(ServiceStoredSimulatingChatClient)} can only be used within the context of a running AIAgent. " +
|
||||
"Ensure that the chat client is being invoked as part of an AIAgent.RunAsync or AIAgent.RunStreamingAsync call.");
|
||||
|
||||
var chatClientAgent = runContext.Agent.GetService<ChatClientAgent>()
|
||||
?? throw new InvalidOperationException(
|
||||
$"{nameof(ServiceStoredSimulatingChatClient)} can only be used with a {nameof(ChatClientAgent)}. " +
|
||||
$"The current agent is of type '{runContext.Agent.GetType().Name}'.");
|
||||
|
||||
if (runContext.Session is not ChatClientAgentSession chatClientAgentSession)
|
||||
{
|
||||
throw new InvalidOperationException(
|
||||
$"{nameof(ServiceStoredSimulatingChatClient)} requires a {nameof(ChatClientAgentSession)}. " +
|
||||
$"The current session is of type '{runContext.Session?.GetType().Name ?? "null"}'.");
|
||||
}
|
||||
|
||||
return (chatClientAgent, chatClientAgentSession);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// If the <paramref name="options"/> carry the <see cref="LocalHistoryConversationId"/> sentinel,
|
||||
/// returns a clone with the conversation ID cleared so the inner client never sees it.
|
||||
/// Otherwise returns the original <paramref name="options"/> unchanged.
|
||||
/// </summary>
|
||||
private static ChatOptions? StripLocalHistoryConversationId(ChatOptions? options)
|
||||
{
|
||||
if (options?.ConversationId == LocalHistoryConversationId)
|
||||
{
|
||||
options = options.Clone();
|
||||
options.ConversationId = null;
|
||||
}
|
||||
|
||||
return options;
|
||||
}
|
||||
}
|
||||
@@ -14,7 +14,7 @@ namespace Microsoft.Agents.AI.UnitTests;
|
||||
|
||||
/// <summary>
|
||||
/// Shared test helper for <see cref="ChatClientAgent"/> integration tests that verify
|
||||
/// end-to-end behavior with <see cref="ChatHistoryPersistingChatClient"/> and
|
||||
/// end-to-end behavior with <see cref="ServiceStoredSimulatingChatClient"/> and
|
||||
/// <see cref="FunctionInvokingChatClient"/>.
|
||||
/// </summary>
|
||||
internal static class ChatClientAgentTestHelper
|
||||
|
||||
@@ -379,12 +379,10 @@ public partial class ChatClientAgentTests
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Verify that RunAsync passes ChatOptions with null ConversationId when using regular AgentRunOptions.
|
||||
/// When per-service-call persistence is active (default), the sentinel conversation ID is set on ChatOptions
|
||||
/// and then stripped by ChatHistoryPersistingChatClient before reaching the inner client.
|
||||
/// Verify that RunAsync passes null ChatOptions when using regular AgentRunOptions.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public async Task RunAsyncPassesChatOptionsWithNullConversationIdWhenUsingRegularAgentRunOptionsAsync()
|
||||
public async Task RunAsyncPassesNullChatOptionsWhenUsingRegularAgentRunOptionsAsync()
|
||||
{
|
||||
// Arrange
|
||||
ChatOptions? capturedOptions = null;
|
||||
@@ -403,9 +401,8 @@ public partial class ChatClientAgentTests
|
||||
// Act
|
||||
await agent.RunAsync([new(ChatRole.User, "test")], options: runOptions);
|
||||
|
||||
// Assert — the inner client receives ChatOptions with null ConversationId (sentinel was stripped)
|
||||
Assert.NotNull(capturedOptions);
|
||||
Assert.Null(capturedOptions!.ConversationId);
|
||||
// Assert
|
||||
Assert.Null(capturedOptions);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
|
||||
+5
-5
@@ -9,7 +9,7 @@ namespace Microsoft.Agents.AI.UnitTests;
|
||||
|
||||
/// <summary>
|
||||
/// Contains unit tests that verify the end-to-end approval flow behavior of the
|
||||
/// <see cref="ChatClientAgent"/> class with <see cref="ChatHistoryPersistingChatClient"/>,
|
||||
/// <see cref="ChatClientAgent"/> class with <see cref="ServiceStoredSimulatingChatClient"/>,
|
||||
/// ensuring that chat history is correctly persisted across multi-turn approval interactions.
|
||||
/// </summary>
|
||||
public class ChatClientAgent_ApprovalsTests
|
||||
@@ -48,7 +48,7 @@ public class ChatClientAgent_ApprovalsTests
|
||||
agentOptions: new()
|
||||
{
|
||||
ChatOptions = new() { Tools = [approvalTool] },
|
||||
PersistChatHistoryAtEndOfRun = false,
|
||||
SimulateServiceStoredChatHistory = true,
|
||||
},
|
||||
callIndex: callIndex,
|
||||
capturedInputs: capturedInputs);
|
||||
@@ -123,7 +123,6 @@ public class ChatClientAgent_ApprovalsTests
|
||||
agentOptions: new()
|
||||
{
|
||||
ChatOptions = new() { Tools = [approvalTool] },
|
||||
PersistChatHistoryAtEndOfRun = true,
|
||||
},
|
||||
callIndex: callIndex,
|
||||
capturedInputs: capturedInputs);
|
||||
@@ -150,8 +149,10 @@ public class ChatClientAgent_ApprovalsTests
|
||||
expectedHistory:
|
||||
[
|
||||
// End-of-run persistence retains the approval request from Turn 1
|
||||
// and the approval response from Turn 2
|
||||
new(ChatRole.User, TextContains: "What's the weather?"),
|
||||
new(ChatRole.Assistant, ContentTypes: [typeof(ToolApprovalRequestContent)]),
|
||||
new(ChatRole.User, ContentTypes: [typeof(ToolApprovalResponseContent)]),
|
||||
new(ChatRole.Assistant, ContentTypes: [typeof(FunctionCallContent)]),
|
||||
new(ChatRole.Tool, ContentTypes: [typeof(FunctionResultContent)]),
|
||||
new(ChatRole.Assistant, TextContains: "sunny and 22°C"),
|
||||
@@ -196,7 +197,6 @@ public class ChatClientAgent_ApprovalsTests
|
||||
agentOptions: new()
|
||||
{
|
||||
ChatOptions = new() { Tools = [approvalTool] },
|
||||
PersistChatHistoryAtEndOfRun = false,
|
||||
},
|
||||
callIndex: callIndex,
|
||||
capturedInputs: capturedInputs);
|
||||
@@ -260,7 +260,7 @@ public class ChatClientAgent_ApprovalsTests
|
||||
agentOptions: new()
|
||||
{
|
||||
ChatOptions = new() { Tools = [approvalTool] },
|
||||
PersistChatHistoryAtEndOfRun = false,
|
||||
SimulateServiceStoredChatHistory = true,
|
||||
},
|
||||
callIndex: callIndex,
|
||||
capturedInputs: capturedInputs);
|
||||
|
||||
+2
-5
@@ -520,7 +520,7 @@ public class ChatClientAgent_ChatHistoryManagementTests
|
||||
agentOptions: new()
|
||||
{
|
||||
ChatOptions = new() { Instructions = "Be helpful" },
|
||||
PersistChatHistoryAtEndOfRun = false,
|
||||
SimulateServiceStoredChatHistory = true,
|
||||
},
|
||||
expectedServiceCallCount: 1,
|
||||
expectedHistory:
|
||||
@@ -554,7 +554,7 @@ public class ChatClientAgent_ChatHistoryManagementTests
|
||||
agentOptions: new()
|
||||
{
|
||||
ChatOptions = new() { Tools = [tool] },
|
||||
PersistChatHistoryAtEndOfRun = false,
|
||||
SimulateServiceStoredChatHistory = true,
|
||||
},
|
||||
expectedServiceCallCount: 2,
|
||||
expectedHistory:
|
||||
@@ -583,7 +583,6 @@ public class ChatClientAgent_ChatHistoryManagementTests
|
||||
agentOptions: new()
|
||||
{
|
||||
ChatOptions = new() { Instructions = "Be helpful" },
|
||||
PersistChatHistoryAtEndOfRun = true,
|
||||
},
|
||||
expectedServiceCallCount: 1,
|
||||
expectedHistory:
|
||||
@@ -615,7 +614,6 @@ public class ChatClientAgent_ChatHistoryManagementTests
|
||||
agentOptions: new()
|
||||
{
|
||||
ChatOptions = new() { Tools = [tool] },
|
||||
PersistChatHistoryAtEndOfRun = true,
|
||||
},
|
||||
expectedServiceCallCount: 2,
|
||||
expectedHistory:
|
||||
@@ -644,7 +642,6 @@ public class ChatClientAgent_ChatHistoryManagementTests
|
||||
agentOptions: new()
|
||||
{
|
||||
ChatOptions = new() { Instructions = "Be helpful" },
|
||||
PersistChatHistoryAtEndOfRun = false,
|
||||
},
|
||||
expectedServiceCallCount: 1);
|
||||
|
||||
|
||||
+5
-7
@@ -176,12 +176,11 @@ public class ChatClientAgent_ChatOptionsMergingTests
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Verify that ChatOptions merging returns a non-null ChatOptions instance with null ConversationId
|
||||
/// when both agent and request have no ChatOptions. The sentinel conversation ID is set for
|
||||
/// per-service-call persistence and stripped before reaching the inner client.
|
||||
/// Verify that when both agent and request have no ChatOptions, the inner client
|
||||
/// receives null options.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public async Task ChatOptionsMergingReturnsChatOptionsWithNullConversationIdWhenBothAgentAndRequestHaveNoneAsync()
|
||||
public async Task ChatOptionsMergingReturnsNullChatOptionsWhenBothAgentAndRequestHaveNoneAsync()
|
||||
{
|
||||
// Arrange
|
||||
Mock<IChatClient> mockService = new();
|
||||
@@ -201,9 +200,8 @@ public class ChatClientAgent_ChatOptionsMergingTests
|
||||
// Act
|
||||
await agent.RunAsync(messages);
|
||||
|
||||
// Assert — ChatOptions is non-null because the sentinel was set, but ConversationId is null (stripped)
|
||||
Assert.NotNull(capturedChatOptions);
|
||||
Assert.Null(capturedChatOptions!.ConversationId);
|
||||
// Assert
|
||||
Assert.Null(capturedChatOptions);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
|
||||
+470
-89
@@ -13,15 +13,15 @@ using Moq.Protected;
|
||||
namespace Microsoft.Agents.AI.UnitTests;
|
||||
|
||||
/// <summary>
|
||||
/// Contains unit tests for the <see cref="ChatHistoryPersistingChatClient"/> decorator,
|
||||
/// Contains unit tests for the <see cref="ServiceStoredSimulatingChatClient"/> decorator,
|
||||
/// verifying that it persists messages via the <see cref="ChatHistoryProvider"/> after each
|
||||
/// individual service call by default, or marks messages for end-of-run persistence when the
|
||||
/// <see cref="ChatClientAgentOptions.PersistChatHistoryAtEndOfRun"/> option is enabled.
|
||||
/// <see cref="ChatClientAgentOptions.SimulateServiceStoredChatHistory"/> option is enabled.
|
||||
/// </summary>
|
||||
public class ChatHistoryPersistingChatClientTests
|
||||
public class ServiceStoredSimulatingChatClientTests
|
||||
{
|
||||
/// <summary>
|
||||
/// Verifies that by default (PersistChatHistoryAtEndOfRun is false),
|
||||
/// Verifies that by default (SimulateServiceStoredChatHistory is false),
|
||||
/// the ChatHistoryProvider receives messages after a successful non-streaming call.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
@@ -50,7 +50,7 @@ public class ChatHistoryPersistingChatClientTests
|
||||
ChatClientAgent agent = new(mockService.Object, options: new()
|
||||
{
|
||||
ChatHistoryProvider = mockChatHistoryProvider.Object,
|
||||
PersistChatHistoryAtEndOfRun = false,
|
||||
SimulateServiceStoredChatHistory = true,
|
||||
});
|
||||
|
||||
// Act
|
||||
@@ -97,7 +97,7 @@ public class ChatHistoryPersistingChatClientTests
|
||||
ChatClientAgent agent = new(mockService.Object, options: new()
|
||||
{
|
||||
ChatHistoryProvider = mockChatHistoryProvider.Object,
|
||||
PersistChatHistoryAtEndOfRun = true,
|
||||
SimulateServiceStoredChatHistory = true,
|
||||
});
|
||||
|
||||
// Act
|
||||
@@ -145,7 +145,7 @@ public class ChatHistoryPersistingChatClientTests
|
||||
ChatClientAgent agent = new(mockService.Object, options: new()
|
||||
{
|
||||
ChatHistoryProvider = mockChatHistoryProvider.Object,
|
||||
PersistChatHistoryAtEndOfRun = false,
|
||||
SimulateServiceStoredChatHistory = true,
|
||||
});
|
||||
|
||||
// Act
|
||||
@@ -163,11 +163,10 @@ public class ChatHistoryPersistingChatClientTests
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Verifies that the decorator is injected in persist mode by default
|
||||
/// and can be discovered via GetService.
|
||||
/// Verifies that the decorator is NOT injected by default (SimulateServiceStoredChatHistory is false).
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public void ChatClient_ContainsDecorator_InPersistMode_ByDefault()
|
||||
public void ChatClient_DoesNotContainDecorator_ByDefault()
|
||||
{
|
||||
// Arrange
|
||||
Mock<IChatClient> mockService = new();
|
||||
@@ -176,16 +175,15 @@ public class ChatHistoryPersistingChatClientTests
|
||||
ChatClientAgent agent = new(mockService.Object, options: new());
|
||||
|
||||
// Assert
|
||||
var decorator = agent.ChatClient.GetService<ChatHistoryPersistingChatClient>();
|
||||
Assert.NotNull(decorator);
|
||||
Assert.False(decorator.MarkOnly);
|
||||
var decorator = agent.ChatClient.GetService<ServiceStoredSimulatingChatClient>();
|
||||
Assert.Null(decorator);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Verifies that the decorator is injected in mark-only mode when PersistChatHistoryAtEndOfRun is true.
|
||||
/// Verifies that the decorator is injected when SimulateServiceStoredChatHistory is true.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public void ChatClient_ContainsDecorator_InMarkOnlyMode_WhenPersistAtEndOfRun()
|
||||
public void ChatClient_ContainsDecorator_WhenSimulateServiceStoredChatHistory()
|
||||
{
|
||||
// Arrange
|
||||
Mock<IChatClient> mockService = new();
|
||||
@@ -193,13 +191,12 @@ public class ChatHistoryPersistingChatClientTests
|
||||
// Act
|
||||
ChatClientAgent agent = new(mockService.Object, options: new()
|
||||
{
|
||||
PersistChatHistoryAtEndOfRun = true,
|
||||
SimulateServiceStoredChatHistory = true,
|
||||
});
|
||||
|
||||
// Assert
|
||||
var decorator = agent.ChatClient.GetService<ChatHistoryPersistingChatClient>();
|
||||
var decorator = agent.ChatClient.GetService<ServiceStoredSimulatingChatClient>();
|
||||
Assert.NotNull(decorator);
|
||||
Assert.True(decorator.MarkOnly);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
@@ -218,27 +215,27 @@ public class ChatHistoryPersistingChatClientTests
|
||||
});
|
||||
|
||||
// Assert
|
||||
var decorator = agent.ChatClient.GetService<ChatHistoryPersistingChatClient>();
|
||||
var decorator = agent.ChatClient.GetService<ServiceStoredSimulatingChatClient>();
|
||||
Assert.Null(decorator);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Verifies that the PersistChatHistoryAtEndOfRun option is included in Clone().
|
||||
/// Verifies that the SimulateServiceStoredChatHistory option is included in Clone().
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public void ChatClientAgentOptions_Clone_IncludesPersistChatHistoryAtEndOfRun()
|
||||
public void ChatClientAgentOptions_Clone_IncludesSimulateServiceStoredChatHistory()
|
||||
{
|
||||
// Arrange
|
||||
var options = new ChatClientAgentOptions
|
||||
{
|
||||
PersistChatHistoryAtEndOfRun = true,
|
||||
SimulateServiceStoredChatHistory = true,
|
||||
};
|
||||
|
||||
// Act
|
||||
var cloned = options.Clone();
|
||||
|
||||
// Assert
|
||||
Assert.True(cloned.PersistChatHistoryAtEndOfRun);
|
||||
Assert.True(cloned.SimulateServiceStoredChatHistory);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
@@ -292,7 +289,7 @@ public class ChatHistoryPersistingChatClientTests
|
||||
{
|
||||
ChatOptions = new() { Tools = [tool] },
|
||||
ChatHistoryProvider = mockChatHistoryProvider.Object,
|
||||
PersistChatHistoryAtEndOfRun = false,
|
||||
SimulateServiceStoredChatHistory = true,
|
||||
}, services: new ServiceCollection().BuildServiceProvider());
|
||||
|
||||
// Act
|
||||
@@ -361,7 +358,7 @@ public class ChatHistoryPersistingChatClientTests
|
||||
ChatClientAgent agent = new(mockService.Object, options: new()
|
||||
{
|
||||
ChatHistoryProvider = mockChatHistoryProvider.Object,
|
||||
PersistChatHistoryAtEndOfRun = false,
|
||||
SimulateServiceStoredChatHistory = true,
|
||||
});
|
||||
|
||||
// Act
|
||||
@@ -410,7 +407,7 @@ public class ChatHistoryPersistingChatClientTests
|
||||
ChatClientAgent agent = new(mockService.Object, options: new()
|
||||
{
|
||||
AIContextProviders = [mockContextProvider.Object],
|
||||
PersistChatHistoryAtEndOfRun = false,
|
||||
SimulateServiceStoredChatHistory = true,
|
||||
});
|
||||
|
||||
// Act
|
||||
@@ -457,7 +454,7 @@ public class ChatHistoryPersistingChatClientTests
|
||||
ChatClientAgent agent = new(mockService.Object, options: new()
|
||||
{
|
||||
AIContextProviders = [mockContextProvider.Object],
|
||||
PersistChatHistoryAtEndOfRun = false,
|
||||
SimulateServiceStoredChatHistory = true,
|
||||
});
|
||||
|
||||
// Act
|
||||
@@ -516,7 +513,7 @@ public class ChatHistoryPersistingChatClientTests
|
||||
{
|
||||
ChatHistoryProvider = mockChatHistoryProvider.Object,
|
||||
AIContextProviders = [mockContextProvider.Object],
|
||||
PersistChatHistoryAtEndOfRun = false,
|
||||
SimulateServiceStoredChatHistory = true,
|
||||
});
|
||||
|
||||
// Act
|
||||
@@ -590,7 +587,7 @@ public class ChatHistoryPersistingChatClientTests
|
||||
{
|
||||
ChatOptions = new() { Tools = [tool] },
|
||||
ChatHistoryProvider = mockChatHistoryProvider.Object,
|
||||
PersistChatHistoryAtEndOfRun = false,
|
||||
SimulateServiceStoredChatHistory = true,
|
||||
}, services: new ServiceCollection().BuildServiceProvider());
|
||||
|
||||
// Act
|
||||
@@ -655,7 +652,7 @@ public class ChatHistoryPersistingChatClientTests
|
||||
{
|
||||
ChatOptions = new() { Tools = [tool] },
|
||||
ChatHistoryProvider = mockChatHistoryProvider.Object,
|
||||
PersistChatHistoryAtEndOfRun = false,
|
||||
SimulateServiceStoredChatHistory = true,
|
||||
}, services: new ServiceCollection().BuildServiceProvider());
|
||||
|
||||
// Act
|
||||
@@ -680,52 +677,12 @@ public class ChatHistoryPersistingChatClientTests
|
||||
/// Verifies that after a successful run with per-service-call persistence, the notified
|
||||
/// messages are stamped with the persisted marker so they are not re-notified.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public async Task RunAsync_MarksNotifiedMessages_WithPersistedMarkerAsync()
|
||||
{
|
||||
// Arrange
|
||||
Mock<IChatClient> mockService = new();
|
||||
mockService.Setup(
|
||||
s => s.GetResponseAsync(
|
||||
It.IsAny<IEnumerable<ChatMessage>>(),
|
||||
It.IsAny<ChatOptions>(),
|
||||
It.IsAny<CancellationToken>())).ReturnsAsync(new ChatResponse([new(ChatRole.Assistant, "response")]));
|
||||
|
||||
Mock<ChatHistoryProvider> mockChatHistoryProvider = new(null, null, null);
|
||||
mockChatHistoryProvider.SetupGet(p => p.StateKeys).Returns(["TestChatHistoryProvider"]);
|
||||
mockChatHistoryProvider
|
||||
.Protected()
|
||||
.Setup<ValueTask<IEnumerable<ChatMessage>>>("InvokingCoreAsync", ItExpr.IsAny<ChatHistoryProvider.InvokingContext>(), ItExpr.IsAny<CancellationToken>())
|
||||
.Returns((ChatHistoryProvider.InvokingContext ctx, CancellationToken _) =>
|
||||
new ValueTask<IEnumerable<ChatMessage>>(ctx.RequestMessages.ToList()));
|
||||
mockChatHistoryProvider
|
||||
.Protected()
|
||||
.Setup<ValueTask>("InvokedCoreAsync", ItExpr.IsAny<ChatHistoryProvider.InvokedContext>(), ItExpr.IsAny<CancellationToken>())
|
||||
.Returns(() => new ValueTask());
|
||||
|
||||
ChatClientAgent agent = new(mockService.Object, options: new()
|
||||
{
|
||||
ChatHistoryProvider = mockChatHistoryProvider.Object,
|
||||
PersistChatHistoryAtEndOfRun = false,
|
||||
});
|
||||
|
||||
// Act
|
||||
var inputMessage = new ChatMessage(ChatRole.User, "test");
|
||||
var session = await agent.CreateSessionAsync() as ChatClientAgentSession;
|
||||
await agent.RunAsync([inputMessage], session);
|
||||
|
||||
// Assert — input message should be marked as persisted
|
||||
Assert.True(
|
||||
inputMessage.AdditionalProperties?.ContainsKey(ChatHistoryPersistingChatClient.PersistedMarkerKey) == true,
|
||||
"Input message should be marked as persisted after a successful run.");
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Verifies that when per-service-call persistence is enabled and the inner client returns a
|
||||
/// conversation ID, the session's ConversationId is updated after the service call.
|
||||
/// Verifies that when the inner client returns a real conversation ID,
|
||||
/// the session's ConversationId is updated after the run.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public async Task RunAsync_UpdatesSessionConversationId_WhenPerServiceCallPersistenceEnabledAsync()
|
||||
public async Task RunAsync_UpdatesSessionConversationId_WhenServiceReturnsOneAsync()
|
||||
{
|
||||
// Arrange
|
||||
const string ExpectedConversationId = "conv-123";
|
||||
@@ -741,10 +698,7 @@ public class ChatHistoryPersistingChatClientTests
|
||||
ConversationId = ExpectedConversationId,
|
||||
});
|
||||
|
||||
ChatClientAgent agent = new(mockService.Object, options: new()
|
||||
{
|
||||
PersistChatHistoryAtEndOfRun = false,
|
||||
});
|
||||
ChatClientAgent agent = new(mockService.Object);
|
||||
|
||||
// Act
|
||||
var session = await agent.CreateSessionAsync() as ChatClientAgentSession;
|
||||
@@ -766,8 +720,8 @@ public class ChatHistoryPersistingChatClientTests
|
||||
|
||||
/// <summary>
|
||||
/// Verifies that when per-service-call persistence is active and no real conversation ID exists,
|
||||
/// <see cref="ChatClientAgent"/> sets the <see cref="ChatHistoryPersistingChatClient.LocalHistoryConversationId"/>
|
||||
/// sentinel on the chat options and <see cref="ChatHistoryPersistingChatClient"/> strips it before
|
||||
/// <see cref="ChatClientAgent"/> sets the <see cref="ServiceStoredSimulatingChatClient.LocalHistoryConversationId"/>
|
||||
/// sentinel on the chat options and <see cref="ServiceStoredSimulatingChatClient"/> strips it before
|
||||
/// forwarding to the inner client.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
@@ -787,7 +741,7 @@ public class ChatHistoryPersistingChatClientTests
|
||||
ChatClientAgent agent = new(mockService.Object, options: new()
|
||||
{
|
||||
ChatOptions = new() { Instructions = "test" },
|
||||
PersistChatHistoryAtEndOfRun = false,
|
||||
SimulateServiceStoredChatHistory = true,
|
||||
});
|
||||
|
||||
// Act
|
||||
@@ -819,7 +773,7 @@ public class ChatHistoryPersistingChatClientTests
|
||||
ChatClientAgent agent = new(mockService.Object, options: new()
|
||||
{
|
||||
ChatOptions = new() { Instructions = "test" },
|
||||
PersistChatHistoryAtEndOfRun = true,
|
||||
SimulateServiceStoredChatHistory = true,
|
||||
});
|
||||
|
||||
// Act
|
||||
@@ -854,7 +808,7 @@ public class ChatHistoryPersistingChatClientTests
|
||||
|
||||
ChatClientAgent agent = new(mockService.Object, options: new()
|
||||
{
|
||||
PersistChatHistoryAtEndOfRun = false,
|
||||
SimulateServiceStoredChatHistory = true,
|
||||
});
|
||||
|
||||
// Create a session with a real conversation ID.
|
||||
@@ -888,7 +842,7 @@ public class ChatHistoryPersistingChatClientTests
|
||||
ChatClientAgent agent = new(mockService.Object, options: new()
|
||||
{
|
||||
ChatOptions = new() { Instructions = "test" },
|
||||
PersistChatHistoryAtEndOfRun = false,
|
||||
SimulateServiceStoredChatHistory = true,
|
||||
});
|
||||
|
||||
// Act
|
||||
@@ -903,11 +857,12 @@ public class ChatHistoryPersistingChatClientTests
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Verifies that the session's conversation ID is NOT set to the sentinel after the run.
|
||||
/// The sentinel should only exist transiently on the ChatOptions for the pipeline.
|
||||
/// Verifies that the session's conversation ID IS set to the sentinel after the run
|
||||
/// when simulating service-stored chat history. This allows subsequent runs to
|
||||
/// skip provider resolution in the agent (the decorator handles it).
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public async Task RunAsync_SentinelDoesNotLeakToSession_WhenPerServiceCallPersistenceActiveAsync()
|
||||
public async Task RunAsync_SetsSentinelOnSession_WhenSimulateServiceStoredChatHistoryActiveAsync()
|
||||
{
|
||||
// Arrange
|
||||
Mock<IChatClient> mockService = new();
|
||||
@@ -920,14 +875,440 @@ public class ChatHistoryPersistingChatClientTests
|
||||
|
||||
ChatClientAgent agent = new(mockService.Object, options: new()
|
||||
{
|
||||
PersistChatHistoryAtEndOfRun = false,
|
||||
SimulateServiceStoredChatHistory = true,
|
||||
});
|
||||
|
||||
// Act
|
||||
var session = await agent.CreateSessionAsync() as ChatClientAgentSession;
|
||||
await agent.RunAsync([new(ChatRole.User, "test")], session);
|
||||
|
||||
// Assert — session should NOT have the sentinel conversation ID
|
||||
Assert.Null(session!.ConversationId);
|
||||
// Assert — session should have the sentinel conversation ID
|
||||
Assert.Equal(ServiceStoredSimulatingChatClient.LocalHistoryConversationId, session!.ConversationId);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Verifies that when simulating service-stored chat history and the service returns
|
||||
/// a real <see cref="ChatResponse.ConversationId"/>, the conflict detection in
|
||||
/// <see cref="ChatClientAgent.UpdateSessionConversationId"/> throws because both a
|
||||
/// <see cref="ChatHistoryProvider"/> and a service-managed ConversationId are present.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public async Task RunAsync_Throws_WhenServiceReturnsRealConversationIdWithChatHistoryProviderAsync()
|
||||
{
|
||||
// Arrange
|
||||
const string RealConversationId = "service-conv-456";
|
||||
|
||||
Mock<ChatHistoryProvider> mockChatHistoryProvider = new(null, null, null);
|
||||
mockChatHistoryProvider.SetupGet(p => p.StateKeys).Returns(["TestChatHistoryProvider"]);
|
||||
mockChatHistoryProvider
|
||||
.Protected()
|
||||
.Setup<ValueTask<IEnumerable<ChatMessage>>>("InvokingCoreAsync", ItExpr.IsAny<ChatHistoryProvider.InvokingContext>(), ItExpr.IsAny<CancellationToken>())
|
||||
.Returns((ChatHistoryProvider.InvokingContext ctx, CancellationToken _) =>
|
||||
new ValueTask<IEnumerable<ChatMessage>>(ctx.RequestMessages.ToList()));
|
||||
mockChatHistoryProvider
|
||||
.Protected()
|
||||
.Setup<ValueTask>("InvokedCoreAsync", ItExpr.IsAny<ChatHistoryProvider.InvokedContext>(), ItExpr.IsAny<CancellationToken>())
|
||||
.Returns(new ValueTask());
|
||||
|
||||
Mock<IChatClient> mockService = new();
|
||||
mockService.Setup(
|
||||
s => s.GetResponseAsync(
|
||||
It.IsAny<IEnumerable<ChatMessage>>(),
|
||||
It.IsAny<ChatOptions>(),
|
||||
It.IsAny<CancellationToken>()))
|
||||
.ReturnsAsync(new ChatResponse([new(ChatRole.Assistant, "response")])
|
||||
{
|
||||
ConversationId = RealConversationId,
|
||||
});
|
||||
|
||||
ChatClientAgent agent = new(mockService.Object, options: new()
|
||||
{
|
||||
ChatHistoryProvider = mockChatHistoryProvider.Object,
|
||||
SimulateServiceStoredChatHistory = true,
|
||||
});
|
||||
|
||||
// Act & Assert — conflict detection should throw
|
||||
var session = await agent.CreateSessionAsync() as ChatClientAgentSession;
|
||||
await Assert.ThrowsAsync<InvalidOperationException>(() => agent.RunAsync([new(ChatRole.User, "test")], session));
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Verifies that when simulating service-stored chat history and the request carries a real
|
||||
/// <see cref="ChatOptions.ConversationId"/>, the decorator skips history loading but still
|
||||
/// notifies <see cref="AIContextProvider"/>s on success and updates the session ConversationId.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public async Task RunAsync_NotifiesProvidersAndUpdatesSession_WhenRequestHasRealConversationIdAsync()
|
||||
{
|
||||
// Arrange
|
||||
const string RealConversationId = "real-conv-request";
|
||||
const string ServiceConversationId = "real-conv-response";
|
||||
|
||||
Mock<AIContextProvider> mockContextProvider = new(null, null, null);
|
||||
mockContextProvider.SetupGet(p => p.StateKeys).Returns(["TestContextProvider"]);
|
||||
mockContextProvider
|
||||
.Protected()
|
||||
.Setup<ValueTask<AIContext>>("InvokingCoreAsync", ItExpr.IsAny<AIContextProvider.InvokingContext>(), ItExpr.IsAny<CancellationToken>())
|
||||
.Returns((AIContextProvider.InvokingContext ctx, CancellationToken _) =>
|
||||
new ValueTask<AIContext>(new AIContext { Messages = ctx.AIContext.Messages }));
|
||||
mockContextProvider
|
||||
.Protected()
|
||||
.Setup<ValueTask>("InvokedCoreAsync", ItExpr.IsAny<AIContextProvider.InvokedContext>(), ItExpr.IsAny<CancellationToken>())
|
||||
.Returns(new ValueTask());
|
||||
|
||||
Mock<IChatClient> mockService = new();
|
||||
mockService.Setup(
|
||||
s => s.GetResponseAsync(
|
||||
It.IsAny<IEnumerable<ChatMessage>>(),
|
||||
It.IsAny<ChatOptions>(),
|
||||
It.IsAny<CancellationToken>()))
|
||||
.ReturnsAsync(new ChatResponse([new(ChatRole.Assistant, "response")])
|
||||
{
|
||||
ConversationId = ServiceConversationId,
|
||||
});
|
||||
|
||||
ChatClientAgent agent = new(mockService.Object, options: new()
|
||||
{
|
||||
SimulateServiceStoredChatHistory = true,
|
||||
AIContextProviders = [mockContextProvider.Object],
|
||||
});
|
||||
|
||||
// Create a session with a real conversation ID so it's on chatOptions.
|
||||
var session = await agent.CreateSessionAsync(RealConversationId);
|
||||
|
||||
// Act
|
||||
await agent.RunAsync([new(ChatRole.User, "test")], session);
|
||||
|
||||
// Assert — AIContextProvider.InvokedAsync should have been called
|
||||
mockContextProvider
|
||||
.Protected()
|
||||
.Verify<ValueTask>("InvokedCoreAsync", Times.Once(),
|
||||
ItExpr.Is<AIContextProvider.InvokedContext>(x =>
|
||||
x.RequestMessages.Any(m => m.Text == "test") &&
|
||||
x.ResponseMessages!.Any(m => m.Text == "response")),
|
||||
ItExpr.IsAny<CancellationToken>());
|
||||
|
||||
// Assert — session should have the service-returned ConversationId
|
||||
Assert.Equal(ServiceConversationId, (session as ChatClientAgentSession)!.ConversationId);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Verifies that when simulating service-stored chat history and the request carries a real
|
||||
/// <see cref="ChatOptions.ConversationId"/>, the decorator notifies providers of failure
|
||||
/// when the inner client throws.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public async Task RunAsync_NotifiesProvidersOfFailure_WhenRequestHasRealConversationIdAsync()
|
||||
{
|
||||
// Arrange
|
||||
const string RealConversationId = "real-conv-failure";
|
||||
|
||||
Mock<AIContextProvider> mockContextProvider = new(null, null, null);
|
||||
mockContextProvider.SetupGet(p => p.StateKeys).Returns(["TestContextProvider"]);
|
||||
mockContextProvider
|
||||
.Protected()
|
||||
.Setup<ValueTask<AIContext>>("InvokingCoreAsync", ItExpr.IsAny<AIContextProvider.InvokingContext>(), ItExpr.IsAny<CancellationToken>())
|
||||
.Returns((AIContextProvider.InvokingContext ctx, CancellationToken _) =>
|
||||
new ValueTask<AIContext>(new AIContext { Messages = ctx.AIContext.Messages }));
|
||||
mockContextProvider
|
||||
.Protected()
|
||||
.Setup<ValueTask>("InvokedCoreAsync", ItExpr.IsAny<AIContextProvider.InvokedContext>(), ItExpr.IsAny<CancellationToken>())
|
||||
.Returns(new ValueTask());
|
||||
|
||||
Mock<IChatClient> mockService = new();
|
||||
mockService.Setup(
|
||||
s => s.GetResponseAsync(
|
||||
It.IsAny<IEnumerable<ChatMessage>>(),
|
||||
It.IsAny<ChatOptions>(),
|
||||
It.IsAny<CancellationToken>()))
|
||||
.ThrowsAsync(new InvalidOperationException("Service error"));
|
||||
|
||||
ChatClientAgent agent = new(mockService.Object, options: new()
|
||||
{
|
||||
SimulateServiceStoredChatHistory = true,
|
||||
AIContextProviders = [mockContextProvider.Object],
|
||||
});
|
||||
|
||||
var session = await agent.CreateSessionAsync(RealConversationId);
|
||||
|
||||
// Act & Assert — should throw
|
||||
await Assert.ThrowsAsync<InvalidOperationException>(() => agent.RunAsync([new(ChatRole.User, "test")], session));
|
||||
|
||||
// Assert — AIContextProvider.InvokedAsync should have been called with the failure
|
||||
mockContextProvider
|
||||
.Protected()
|
||||
.Verify<ValueTask>("InvokedCoreAsync", Times.Once(),
|
||||
ItExpr.Is<AIContextProvider.InvokedContext>(x => x.InvokeException != null),
|
||||
ItExpr.IsAny<CancellationToken>());
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Verifies that in the streaming path, when the request carries a real
|
||||
/// <see cref="ChatOptions.ConversationId"/>, the decorator skips history loading but still
|
||||
/// notifies providers and updates the session ConversationId.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public async Task RunStreamingAsync_NotifiesProvidersAndUpdatesSession_WhenRequestHasRealConversationIdAsync()
|
||||
{
|
||||
// Arrange
|
||||
const string RealConversationId = "real-conv-streaming";
|
||||
const string ServiceConversationId = "service-conv-streaming";
|
||||
|
||||
Mock<AIContextProvider> mockContextProvider = new(null, null, null);
|
||||
mockContextProvider.SetupGet(p => p.StateKeys).Returns(["TestContextProvider"]);
|
||||
mockContextProvider
|
||||
.Protected()
|
||||
.Setup<ValueTask<AIContext>>("InvokingCoreAsync", ItExpr.IsAny<AIContextProvider.InvokingContext>(), ItExpr.IsAny<CancellationToken>())
|
||||
.Returns((AIContextProvider.InvokingContext ctx, CancellationToken _) =>
|
||||
new ValueTask<AIContext>(new AIContext { Messages = ctx.AIContext.Messages }));
|
||||
mockContextProvider
|
||||
.Protected()
|
||||
.Setup<ValueTask>("InvokedCoreAsync", ItExpr.IsAny<AIContextProvider.InvokedContext>(), ItExpr.IsAny<CancellationToken>())
|
||||
.Returns(new ValueTask());
|
||||
|
||||
Mock<IChatClient> mockService = new();
|
||||
mockService.Setup(
|
||||
s => s.GetStreamingResponseAsync(
|
||||
It.IsAny<IEnumerable<ChatMessage>>(),
|
||||
It.IsAny<ChatOptions>(),
|
||||
It.IsAny<CancellationToken>()))
|
||||
.Returns(CreateAsyncEnumerableAsync(
|
||||
new ChatResponseUpdate(ChatRole.Assistant, "streamed") { ConversationId = ServiceConversationId }));
|
||||
|
||||
ChatClientAgent agent = new(mockService.Object, options: new()
|
||||
{
|
||||
SimulateServiceStoredChatHistory = true,
|
||||
AIContextProviders = [mockContextProvider.Object],
|
||||
});
|
||||
|
||||
var session = await agent.CreateSessionAsync(RealConversationId);
|
||||
|
||||
// Act
|
||||
await foreach (var _ in agent.RunStreamingAsync([new(ChatRole.User, "test")], session))
|
||||
{
|
||||
// Consume all updates.
|
||||
}
|
||||
|
||||
// Assert — AIContextProvider.InvokedAsync should have been called
|
||||
mockContextProvider
|
||||
.Protected()
|
||||
.Verify<ValueTask>("InvokedCoreAsync", Times.Once(),
|
||||
ItExpr.IsAny<AIContextProvider.InvokedContext>(),
|
||||
ItExpr.IsAny<CancellationToken>());
|
||||
|
||||
// Assert — session should have the service-returned ConversationId
|
||||
Assert.Equal(ServiceConversationId, (session as ChatClientAgentSession)!.ConversationId);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Verifies that when simulating and the service unexpectedly returns a real
|
||||
/// <see cref="ChatResponse.ConversationId"/> (no ConversationId on the request), the decorator
|
||||
/// notifies providers and updates the session ConversationId without setting the sentinel.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public async Task RunAsync_NotifiesProvidersAndUpdatesSession_WhenServiceReturnsUnexpectedConversationIdAsync()
|
||||
{
|
||||
// Arrange
|
||||
const string ServiceConversationId = "unexpected-conv-id";
|
||||
|
||||
Mock<AIContextProvider> mockContextProvider = new(null, null, null);
|
||||
mockContextProvider.SetupGet(p => p.StateKeys).Returns(["TestContextProvider"]);
|
||||
mockContextProvider
|
||||
.Protected()
|
||||
.Setup<ValueTask<AIContext>>("InvokingCoreAsync", ItExpr.IsAny<AIContextProvider.InvokingContext>(), ItExpr.IsAny<CancellationToken>())
|
||||
.Returns((AIContextProvider.InvokingContext ctx, CancellationToken _) =>
|
||||
new ValueTask<AIContext>(new AIContext { Messages = ctx.AIContext.Messages }));
|
||||
mockContextProvider
|
||||
.Protected()
|
||||
.Setup<ValueTask>("InvokedCoreAsync", ItExpr.IsAny<AIContextProvider.InvokedContext>(), ItExpr.IsAny<CancellationToken>())
|
||||
.Returns(new ValueTask());
|
||||
|
||||
Mock<IChatClient> mockService = new();
|
||||
mockService.Setup(
|
||||
s => s.GetResponseAsync(
|
||||
It.IsAny<IEnumerable<ChatMessage>>(),
|
||||
It.IsAny<ChatOptions>(),
|
||||
It.IsAny<CancellationToken>()))
|
||||
.ReturnsAsync(new ChatResponse([new(ChatRole.Assistant, "response")])
|
||||
{
|
||||
ConversationId = ServiceConversationId,
|
||||
});
|
||||
|
||||
// No ChatHistoryProvider — so conflict detection won't throw.
|
||||
ChatClientAgent agent = new(mockService.Object, options: new()
|
||||
{
|
||||
SimulateServiceStoredChatHistory = true,
|
||||
AIContextProviders = [mockContextProvider.Object],
|
||||
});
|
||||
|
||||
// Act
|
||||
var session = await agent.CreateSessionAsync() as ChatClientAgentSession;
|
||||
await agent.RunAsync([new(ChatRole.User, "test")], session);
|
||||
|
||||
// Assert — AIContextProvider.InvokedAsync should have been called
|
||||
mockContextProvider
|
||||
.Protected()
|
||||
.Verify<ValueTask>("InvokedCoreAsync", Times.Once(),
|
||||
ItExpr.Is<AIContextProvider.InvokedContext>(x =>
|
||||
x.ResponseMessages!.Any(m => m.Text == "response")),
|
||||
ItExpr.IsAny<CancellationToken>());
|
||||
|
||||
// Assert — session should have the service ConversationId, not the sentinel
|
||||
Assert.Equal(ServiceConversationId, session!.ConversationId);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Verifies that in the streaming path, when the service returns a real ConversationId mid-stream
|
||||
/// (no ConversationId on the request), the decorator notifies providers and updates the session.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public async Task RunStreamingAsync_NotifiesProvidersAndUpdatesSession_WhenServiceReturnsUnexpectedConversationIdAsync()
|
||||
{
|
||||
// Arrange
|
||||
const string ServiceConversationId = "unexpected-stream-conv";
|
||||
|
||||
Mock<AIContextProvider> mockContextProvider = new(null, null, null);
|
||||
mockContextProvider.SetupGet(p => p.StateKeys).Returns(["TestContextProvider"]);
|
||||
mockContextProvider
|
||||
.Protected()
|
||||
.Setup<ValueTask<AIContext>>("InvokingCoreAsync", ItExpr.IsAny<AIContextProvider.InvokingContext>(), ItExpr.IsAny<CancellationToken>())
|
||||
.Returns((AIContextProvider.InvokingContext ctx, CancellationToken _) =>
|
||||
new ValueTask<AIContext>(new AIContext { Messages = ctx.AIContext.Messages }));
|
||||
mockContextProvider
|
||||
.Protected()
|
||||
.Setup<ValueTask>("InvokedCoreAsync", ItExpr.IsAny<AIContextProvider.InvokedContext>(), ItExpr.IsAny<CancellationToken>())
|
||||
.Returns(new ValueTask());
|
||||
|
||||
Mock<IChatClient> mockService = new();
|
||||
mockService.Setup(
|
||||
s => s.GetStreamingResponseAsync(
|
||||
It.IsAny<IEnumerable<ChatMessage>>(),
|
||||
It.IsAny<ChatOptions>(),
|
||||
It.IsAny<CancellationToken>()))
|
||||
.Returns(CreateAsyncEnumerableAsync(
|
||||
new ChatResponseUpdate(ChatRole.Assistant, "part1"),
|
||||
new ChatResponseUpdate(null, "part2") { ConversationId = ServiceConversationId }));
|
||||
|
||||
// No ChatHistoryProvider — so conflict detection won't throw.
|
||||
ChatClientAgent agent = new(mockService.Object, options: new()
|
||||
{
|
||||
SimulateServiceStoredChatHistory = true,
|
||||
AIContextProviders = [mockContextProvider.Object],
|
||||
});
|
||||
|
||||
// Act
|
||||
var session = await agent.CreateSessionAsync() as ChatClientAgentSession;
|
||||
await foreach (var _ in agent.RunStreamingAsync([new(ChatRole.User, "test")], session))
|
||||
{
|
||||
// Consume all updates.
|
||||
}
|
||||
|
||||
// Assert — AIContextProvider.InvokedAsync should have been called
|
||||
mockContextProvider
|
||||
.Protected()
|
||||
.Verify<ValueTask>("InvokedCoreAsync", Times.Once(),
|
||||
ItExpr.IsAny<AIContextProvider.InvokedContext>(),
|
||||
ItExpr.IsAny<CancellationToken>());
|
||||
|
||||
// Assert — session should have the service ConversationId, not the sentinel
|
||||
Assert.Equal(ServiceConversationId, session!.ConversationId);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Verifies that when <see cref="ChatOptions.AllowBackgroundResponses"/> is true,
|
||||
/// the decorator skips history loading and sentinel setting, letting the agent's
|
||||
/// forced end-of-run path handle persistence.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public async Task RunAsync_SkipsSimulation_WhenAllowBackgroundResponsesAsync()
|
||||
{
|
||||
// Arrange
|
||||
IEnumerable<ChatMessage>? capturedMessages = null;
|
||||
Mock<IChatClient> mockService = new();
|
||||
mockService.Setup(
|
||||
s => s.GetResponseAsync(
|
||||
It.IsAny<IEnumerable<ChatMessage>>(),
|
||||
It.IsAny<ChatOptions>(),
|
||||
It.IsAny<CancellationToken>()))
|
||||
.Callback<IEnumerable<ChatMessage>, ChatOptions?, CancellationToken>((msgs, _, _) => capturedMessages = msgs)
|
||||
.ReturnsAsync(new ChatResponse([new(ChatRole.Assistant, "response")]));
|
||||
|
||||
Mock<ChatHistoryProvider> mockChatHistoryProvider = new(null, null, null);
|
||||
mockChatHistoryProvider.SetupGet(p => p.StateKeys).Returns(["TestChatHistoryProvider"]);
|
||||
mockChatHistoryProvider
|
||||
.Protected()
|
||||
.Setup<ValueTask<IEnumerable<ChatMessage>>>("InvokingCoreAsync", ItExpr.IsAny<ChatHistoryProvider.InvokingContext>(), ItExpr.IsAny<CancellationToken>())
|
||||
.Returns((ChatHistoryProvider.InvokingContext ctx, CancellationToken _) =>
|
||||
{
|
||||
// Add a history message to verify it's NOT prepended in this scenario.
|
||||
var result = ctx.RequestMessages.ToList();
|
||||
result.Insert(0, new ChatMessage(ChatRole.Assistant, "history"));
|
||||
return new ValueTask<IEnumerable<ChatMessage>>(result);
|
||||
});
|
||||
mockChatHistoryProvider
|
||||
.Protected()
|
||||
.Setup<ValueTask>("InvokedCoreAsync", ItExpr.IsAny<ChatHistoryProvider.InvokedContext>(), ItExpr.IsAny<CancellationToken>())
|
||||
.Returns(new ValueTask());
|
||||
|
||||
ChatClientAgent agent = new(mockService.Object, options: new()
|
||||
{
|
||||
ChatHistoryProvider = mockChatHistoryProvider.Object,
|
||||
SimulateServiceStoredChatHistory = true,
|
||||
});
|
||||
|
||||
// Act
|
||||
var session = await agent.CreateSessionAsync() as ChatClientAgentSession;
|
||||
await agent.RunAsync(
|
||||
[new(ChatRole.User, "test")],
|
||||
session,
|
||||
new AgentRunOptions { AllowBackgroundResponses = true });
|
||||
|
||||
// Assert — the inner client should NOT have received history messages
|
||||
Assert.NotNull(capturedMessages);
|
||||
var messageList = capturedMessages!.ToList();
|
||||
Assert.Single(messageList);
|
||||
Assert.Equal("test", messageList[0].Text);
|
||||
|
||||
// Assert — session should NOT have the sentinel (agent handles ConversationId at end-of-run)
|
||||
Assert.NotEqual(ServiceStoredSimulatingChatClient.LocalHistoryConversationId, session!.ConversationId);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Verifies that in the streaming path, when <see cref="ChatOptions.AllowBackgroundResponses"/> is true,
|
||||
/// the decorator skips history loading and sentinel setting.
|
||||
/// </summary>
|
||||
[Fact]
|
||||
public async Task RunStreamingAsync_SkipsSimulation_WhenAllowBackgroundResponsesAsync()
|
||||
{
|
||||
// Arrange
|
||||
Mock<IChatClient> mockService = new();
|
||||
mockService.Setup(
|
||||
s => s.GetStreamingResponseAsync(
|
||||
It.IsAny<IEnumerable<ChatMessage>>(),
|
||||
It.IsAny<ChatOptions>(),
|
||||
It.IsAny<CancellationToken>()))
|
||||
.Returns(CreateAsyncEnumerableAsync(new ChatResponseUpdate(ChatRole.Assistant, "response")));
|
||||
|
||||
ChatClientAgent agent = new(mockService.Object, options: new()
|
||||
{
|
||||
SimulateServiceStoredChatHistory = true,
|
||||
});
|
||||
|
||||
// Act
|
||||
var session = await agent.CreateSessionAsync() as ChatClientAgentSession;
|
||||
List<AgentResponseUpdate> updates = [];
|
||||
await foreach (var update in agent.RunStreamingAsync(
|
||||
[new(ChatRole.User, "test")],
|
||||
session,
|
||||
new AgentRunOptions { AllowBackgroundResponses = true }))
|
||||
{
|
||||
updates.Add(update);
|
||||
}
|
||||
|
||||
// Assert — updates should NOT carry the sentinel ConversationId
|
||||
Assert.NotEmpty(updates);
|
||||
|
||||
// Assert — session should NOT have the sentinel
|
||||
Assert.NotEqual(ServiceStoredSimulatingChatClient.LocalHistoryConversationId, session!.ConversationId);
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user