diff --git a/docs/decisions/0001-agent-run-response.md b/docs/decisions/0001-agent-run-response.md new file mode 100644 index 0000000000..bf589d7fe0 --- /dev/null +++ b/docs/decisions/0001-agent-run-response.md @@ -0,0 +1,515 @@ +--- +# These are optional elements. Feel free to remove any of them. +status: accepted +contact: westey-m +date: 2025-07-10 {YYYY-MM-DD when the decision was last updated} +deciders: sergeymenshykh, markwallace, rbarreto, dmytrostruk, westey-m, eavanvalkenburg, stephentoub +consulted: +informed: +--- + +# Agent Run Responses Design + +## Context and Problem Statement + +Agents may produce lots of output during a run including + +1. **[Primary]** General response messages to the caller (this may be in the form of text, including structured output, images, sound, etc.) +2. **[Primary]** Structured confirmation requests to the caller +3. **[Secondary]** Tool invocation activities executed (both local and remote). For information only. +4. Reasoning/Thinking output. + 1. **[Primary]** In some cases an LLM may return reasoning output intermixed with as part of the answer to the caller, since the caller's prompt asked for this detail in some way. This should be considered a specialization of 1. + 1. **[Secondary]** Reasonining models optionally produce reasoning output separate from the answer to the caller's question, and this should be considered secondary content. +5. **[Secondary]** Handoffs / transitions from agent to agent where an agent contains sub agents. +6. **[Secondary]** An indication that the agent is responding (i.e. typing) as if it's a real human. +7. Complete messages in addition to updates, when streaming +8. Id for long running process that is launched +9. and more + +We need to ensure that with this diverse list of output, we are able to + +- Support all with abstractions where needed +- Provide a simple getting started experience that doesn't overwhelm developers + +### Agent response data types + +When comparing various agent SDKs and protocols, agent output is often divided into two categories: + +1. **Result**: A response from the agent that communicates the result of the agent's work to the caller in natural language (or images/sound/etc.). Let's call this **Primary** output. + 1. Includes cases where the agent finished because it requires more input from the user. +2. **Progress**: Updates while the agent is running, which are informational only, typically showing what the agent is doing, and does not allow any actions to be taken by the caller that modify the behavior of the agent before completing the run. Let's call this **Secondary** output. + +A potential third category is: + +3. **Long Running**: A response that does not contain a Primary response or Secondary updates, but rather a reference to a long running task. + +### Different use cases for Primary and Secondary output + +To solve complex problems, many agents must be used together. These agents typically have their own capabilities and responsibilities and communicate via input messages and final responses/handoff calls, while the internal workings of each agent is not of interest to the other agents participating in solving the problem. + +When an agent is in conversation with one or more humans, the information that may be displayed to the user(s) can vary. E.g. When an agent is part of a conversation with multiple humans it may be asked to perform tasks by the humans, and they may not want a stream of distracting updates posted to the conversation, but rather just a final response. On the other hand, if an agent is being used by a single human to perform a task, the human may be waiting for the agent to complete the task. Therefore, they may be interested in getting updates of what the agent is doing. + +Where agents are nested, consumers would also likely want to constrain the amount of data from an agent that bubbles up into higher level conversations to avoid exceeding the context window, therefore limiting it to the Primary response only. + +### Comparison with other SDKs / Protocols + +Approaches observed from the compared SDKs: + +1. Response object with separate properties for Primary and Secondary +2. Response stream that contains Primary and Secondary entries and callers need to filter. +3. Response containing just Primary. + +| SDK | Non-Streaming | Streaming | +|-|-|-| +| AutoGen | **Approach 1** Separates messages into Agent-Agent (maps to Primary) and Internal (maps to Secondary) and these are returned as separate properties on the agent response object. See [types of messages](https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/tutorial/messages.html#types-of-messages) and [Response](https://microsoft.github.io/autogen/stable/reference/python/autogen_agentchat.base.html#autogen_agentchat.base.Response) | **Approach 2** Returns a stream of internal events and the last item is a Response object. See [ChatAgent.on_messages_stream](https://microsoft.github.io/autogen/stable/reference/python/autogen_agentchat.base.html#autogen_agentchat.base.ChatAgent.on_messages_stream) | +| OpenAI Agent SDK | **Approach 1** Separates new_items (Primary+Secondary) from final output (Primary) as separate properties on the [RunResult](https://github.com/openai/openai-agents-python/blob/main/src/agents/result.py#L39) | **Approach 1** Similar to non-streaming, has a way of streaming updates via a method on the response object which includes all data, and then a separate final output property on the response object which is populated only when the run is complete. See [RunResultStreaming](https://github.com/openai/openai-agents-python/blob/main/src/agents/result.py#L136) | +| Google ADK | **Approach 2** [Emits events](https://google.github.io/adk-docs/runtime/#step-by-step-breakdown) with [FinalResponse](https://github.com/google/adk-java/blob/main/core/src/main/java/com/google/adk/events/Event.java#L232) true (Primary) / false (Secondary) and callers have to filter out those with false to get just the final response message | **Approach 2** Similar to non-streaming except [events](https://google.github.io/adk-docs/runtime/#streaming-vs-non-streaming-output-partialtrue) are emitted with [Partial](https://github.com/google/adk-java/blob/main/core/src/main/java/com/google/adk/events/Event.java#L133) true to indicate that they are streaming messages. A final non partial event is also emitted. | +| AWS (Strands) | **Approach 3** Returns an [AgentResult](https://strandsagents.com/latest/api-reference/agent/#strands.agent.agent_result.AgentResult) (Primary) with messages and a reason for the run's completion. | **Approach 2** [Streams events](https://strandsagents.com/latest/api-reference/agent/#strands.agent.agent.Agent.stream_async) (Primary+Secondary) including, response text, current_tool_use, even data from "callbacks" (strands plugins) | +| LangGraph | **Approach 2** A mixed list of all [messages](https://langchain-ai.github.io/langgraph/agents/run_agents/#output-format) | **Approach 2** A mixed list of all [messages](https://langchain-ai.github.io/langgraph/agents/run_agents/#output-format) | +| Agno | **Combination of various approaches** Returns a [RunResponse](https://docs.agno.com/reference/agents/run-response) object with text content, messages (essentially chat history including inputs and instructions), reasoning and thinking text properties. Secondary events could potentially be extracted from messages. | **Approach 2** Returns [RunResponseEvent](https://docs.agno.com/reference/agents/run-response#runresponseevent-types-and-attributes) objects including tool call, memory update, etc, information, where the [RunResponseCompletedEvent](https://docs.agno.com/reference/agents/run-response#runresponsecompletedevent) has similar properties to RunResponse| +| A2A | **Approach 3** Returns a [Task or Message](https://a2aproject.github.io/A2A/latest/specification/#71-messagesend) where the message is the final result (Primary) and task is a reference to a long running process. | **Approach 2** Returns a [stream](https://a2aproject.github.io/A2A/latest/specification/#72-messagestream) that contains task updates (Secondary) and a final message (Primary) | +| Protocol Activity | **Approach 2** Single stream of responses including secondary events and final response messages (Primary). | No separate behavior for streaming. | + +## Decision Drivers + +- Solutions provides an easy to use experience for users who are getting started and just want the answer to a question. +- Solution must be extensible to future requirements, e.g. long running agent processes. +- Experience is in line or better than the best in class experience from other SDKs + +## Response Type Options + +- **Option 1** Run: Messages List contains mix of Primary and Secondary content, RunStreaming: Stream of Primary + Secondary + - **Option 1.1** Secondary content do not use `TextContent` + - **Option 1.2** Presence of Secondary Content is determined by a runtime parameter + - **Option 1.3** Use ChatClient response types + - **Option 1.4** Return derived ChatClient response types +- **Option 2** Run: Container with Primary and Secondary Properties, RunStreaming: Stream of Primary + Secondary + - **Option 2.1** Response types extend MEAI types + - **Option 2.2** New Response types +- **Option 3** Run: Primary-only, RunStreaming: Stream of Primary + Secondary +- **Option 4** Remove Run API and retain RunStreaming API only, which returns a Stream of Primary + Secondary. + +Since the suggested options vary only for the non-streaming case, the following detailed explanations for each +focuses on the non-streaming case. + +### Option 1 Run: Messages List contains mix of Primary and Secondary content, RunStreaming: Stream of Primary + Secondary + +Run returns a `Task` and RunStreaming returns a `IAsyncEnumerable`. +For Run, the returned `ChatResponse.Messages` contains an ordered list of messages that contain both the Primary and Secondary content. + +`ChatResponse.Text` automatically aggregates all text from any `TextContent` items in all `ChatMessage` items in the response. +If we can ensure that no updates ever contain `TextContent`, this will mean that `ChatResponse.Text` will always contain +the Primary response text. See option 1.1. +If we cannot ensure this, either the solution or usage becomes more complex, see 1.3 and 1.4. + +#### Option 1.1 `TextContent`, `DataContent` and `UriContent` means Primary content + +`ChatResponse.Text` aggregates all `TextContent` values, and no secondary updates use `TextContent` +so `ChatResponse.Text` will always contain the Primary content. + +```csharp +// Since the Text property contains the primary content, it's a simple getting started experience. +var response = await agent.RunAsync("Do Something"); +Console.WriteLine(response.Text); + +// Callers can still get access to all updates too. +foreach (var update in response.Messages) +{ + Console.WriteLine(update.Contents.FirstOrDefault()?.GetType().Name); +} + +// For streaming, it's possible to output the primary content by also using the Text property on each update. +await foreach (var update in agent.RunStreamingAsync("Do Something")) +{ + Console.Writeline(update.Text) +} +``` + +- **PROS**: Easy and familiar user experience, reuse response types from IChatClient. Similar experience for both streaming and non streaming. +- **CONS**: The agent response types cannot evolve separately from MEAI if needed. + +#### Option 1.1a `TextContent`, `DataContent` and `UriContent` means Primary content, with custom Agent response types + +Same as 1.1 but with custom Agent Framework response types. +The response types should preferably resemble ChatResponse types closely, to ensure user's have a fimilar experience when moving between the two. +Therefore something like `AgentResponse.Text` which also aggregates all `TextContent` values similar to 1.1 makes sense. + +- **PROS**: Easy getting started experience, and response types can be customized for the Agent Framework where needed. +- **CONS**: More work to define custom response types. + +#### Option 1.2 Presence of Secondary Content is determined by a runtime parameter + +We can allow callers to choose whether to include secondary content in the list of reponse messages. +Open Question: Do we allow secondary content to use `TextContent` types? + +```csharp +// By default the response only has the primary content, so text +// contains the primary content, and it's a good starting experience. +var response = await agent.RunAsync("Do Something"); +Console.WriteLine(response.Text); + +// we can also optionally include updates via an option. +var response = await agent.RunAsync("Do Something", options: new() { IncludeUpdates = true }); +// Callers can now access all updates. +foreach (var update in response.Messages) +{ + Console.WriteLine(update.Contents.FirstOrDefault()?.GetType().Name); +} +``` + +- **PROS**: Easy getting started experience, reuse response types from IChatClient. +- **CONS**: Since the basic experience is the same as 1.1, and when you look at individual messages, you most likely want all anyway, it seems arbitrarily limiting compared to 1.1. + +### Option 2 Run: Container with Primary and Secondary Properties, RunStreaming: Stream of Primary + Secondary + +Run returns a new response type that has separate properties for the Primary Content and the Secondary Updates leading up to it. +The Primary content is available in the `AgentRunResponse.Messages` property while Secondary updates are in a new `AgentRunResponse.Updates` property. +`AgentRunResponse.Text` returns the Primary content text. + +Since streaming would still need to return an `IAsyncEnumerable` of updates, the design would differ from non-streaming. +With non-streaming Primary and Secondary content is split into separate lists, while with streaming it's combined in one stream. + +```csharp +// Since text contains the primary content, it's a good getting started experience. +var response = await agent.RunAsync("Do Something"); +Console.WriteLine(response.Text); + +// Callers can still get access to all updates too. +foreach (var update in response.Updates) +{ + Console.WriteLine(update.Contents.FirstOrDefault()?.GetType().Name); +} +``` + +- **PROS**: Primary content and Secondary Updates are categorised for non-streaming and therefore easy to distinguish and this design matches popular SDKs like AutoGen and OpenAI SDK. +- **CONS**: Requires custom response types and design would differ between streaming and non-streaming. + +### Option 3 Run: Primary-only, RunStreaming: Stream of Primary + Secondary + +Run returns a `Task` and RunStreaming returns a `IAsyncEnumerable`. +For Run, the returned `ChatResponse.Messages` contains only the Primary content messages. +`ChatResponse.Text` will contain the aggregate text of `ChatResponse.Messages` and therefore the primary content messages text. + +```csharp +// Since text contains the primary content response, it's a good getting started experience. +var response = await agent.RunAsync("Do Something"); +Console.WriteLine(response.Text); + +// Callers cannot get access to all updates, since only the primary content is in messages. +var primaryContentOnly = response.Messages.FirstOrDefault(); +``` + +- **PROS**: Simple getting started experience, Reusing IChatClient response types. +- **CONS**: Intermediate updates are only availble in streaming mode. + +### Option 4: Remove Run API and retain RunStreaming API only, which returns a Stream of Primary + Secondary + +With this option, we remove the `RunAsync` method and only retain the `RunStreamingAsync` method, but +we add helpers to process the streaming responses and extract information from it. + +```csharp +// User can get the primary content through an extension method on the async enumerable stream. +var responses = agent.RunStreamingAsync("Do Something"); +// E.g. an extension method that builds the primary content text. +Console.WriteLine(await responses.AggregateFinalResult()); +// Or an extention method that builds complete messages from the updates. +Console.WriteLine(await responses.BuildMessage().Text); + +// Callers can also iterate through all updates if needed +await foreach (var update in responses) +{ + Console.WriteLine(update.Contents.FirstOrDefault()?.GetType().Name); +} +``` + +- **PROS**: Single API for streaming/non-streaming +- **CONS**: More complex to for inexperienced users. + +## Custom Response Type Design Options + +### Option 1 Response types extend MEAI types + +```csharp +class Agent +{ + public abstract Task RunAsync( + IReadOnlyCollection messages, + AgentThread? thread = null, + AgentRunOptions? options = null, + CancellationToken cancellationToken = default); + + public abstract IAsyncEnumerable RunStreamingAsync( + IReadOnlyCollection messages, + AgentThread? thread = null, + AgentRunOptions? options = null, + CancellationToken cancellationToken = default); +} + +class AgentRunResponse : ChatResponse +{ +} + +public class AgentRunResponseUpdate : ChatResponseUpdate +{ +} +``` + +- **PROS**: Fimilar response types for anyone already using MEAI. +- **CONS**: Agent response types cannot evolve separately. + +### Option 2 New Response types + +We could create new response types for Agents. +The new types could also exclude properties that make less sense for agents, like ConversationId, which is abstracted away by AgentThread, or ModelId, where an agent might use multiple models. + +```csharp +class Agent +{ + public abstract Task RunAsync( + IReadOnlyCollection messages, + AgentThread? thread = null, + AgentRunOptions? options = null, + CancellationToken cancellationToken = default); + + public abstract IAsyncEnumerable RunStreamingAsync( + IReadOnlyCollection messages, + AgentThread? thread = null, + AgentRunOptions? options = null, + CancellationToken cancellationToken = default); +} + +class AgentRunResponse // Compare with ChatResponse +{ + public string Text { get; } // Aggregation of TextContent from messages. + + public IList Messages { get; set; } + + public string? ResponseId { get; set; } + + // Metadata + public string? AuthorName { get; set; } + public DateTimeOffset? CreatedAt { get; set; } + public object? RawRepresentation { get; set; } + public UsageDetails? Usage { get; set; } + public AdditionalPropertiesDictionary? AdditionalProperties { get; set; } +} + +// Not Included in AgentRunResponse compared to ChatResponse +public ChatFinishReason? FinishReason { get; set; } +public string? ConversationId { get; set; } +public string? ModelId { get; set; } + +public class AgentRunResponseUpdate // Compare with ChatResponseUpdate +{ + public string Text { get; } // Aggregation of TextContent from Contents. + + public IList Contents { get; set; } + + public string? ResponseId { get; set; } + public string? MessageId { get; set; } + + // Metadata + public ChatRole? Role { get; set; } + public string? AuthorName { get; set; } + public DateTimeOffset? CreatedAt { get; set; } + public UsageDetails? Usage { get; set; } + public object? RawRepresentation { get; set; } + public AdditionalPropertiesDictionary? AdditionalProperties { get; set; } +} + +// Not Included in AgentRunResponseUpdate compared to ChatResponseUpdate +public ChatFinishReason? FinishReason { get; set; } +public string? ConversationId { get; set; } +public string? ModelId { get; set; } +``` + +- **PROS**: Agent response types can evolve separately. Types can still resemble MEAI response types to ensure a fimilar experience for developers. +- **CONS**: No automatic inheritence of new properties from MEAI. (this might also be a pro) + +## Long Running Processes Options + +Some agent protocols, like A2A, support long running agentic processes. When invoking the agent +in the non-streaming case, the agent may respond with an id of a process that was launched. + +The caller is then expected to poll the service to get status updates using the id. +The caller may also subscribe to updates from the process using the id. + +We therefore need to be able to support providing this type of response to agent callers. + +- **Option 1** Add a new `AIContent` type and `ChatFinishReason` for long running processes. +- **Option 2** Add another property on a custom response type. + +### Option 1: Add another AIContent type and ChatFinishReason for long running processes + +```csharp +public class AgentRunContent : AIContent +{ + public string AgentRunId { get; set; } +} + +// Add a new long running chat finish reason. +public class ChatFinishReason +{ + public static ChatFinishReason LongRunning { get; } = new ChatFinishReason("long_running"); +} +``` + +- **PROS**: Fits well into existing `ChatResponse` design. +- **CONS**: More complex for users to extract the required long running result (can be mitigated with extenion methods) + +### Option 2: Add another property on responses for AgentRun + +```csharp +class AgentRunResponse +{ + ... + public AgentRun RunReference { get; set; } // Reference to long running process + ... +} + + +public class AgentRunResponseUpdate +{ + ... + public AgentRun RunReference { get; set; } // Reference to long running process + ... +} + +// Add a new long running chat finish reason. +public class ChatFinishReason +{ + ... + public static ChatFinishReason LongRunning { get; } = new ChatFinishReason("long_running"); + ... +} + +// Can be added in future: Class representing long running processing by the agent +// that can be used to check for updates and status of the processing. +public class AgentRun +{ + public string AgentRunId { get; set; } +} +``` + +- **PROS**: Easy access to long running result values +- **CONS**: Requires custom response types. + +## Structured user input options (Work in progress) + +Some agent services may ask end users a question while also providing a list of options that the user can pick from or a template for the input required. +We need to decide whether to maintain an abstraction for these, so that similar types of structured input from different agents can be used by callers without +needing to break out of the abstraction. + +## Tool result options (Work in progress) + +We need to consider abstractions for `AIContent` derived types for tool call results for common tool types beyond Function calls, e.g. CodeInterpreter, WebSearch, etc. + +## StructuredOutputs + +Structured outputs is a valueable aspect of any Agent system, since it forces an Agent to produce output in a required format, and may include required fields. This allows turning unstructured data into structured data easily using a general purpose language model. + +Not all agent types necessarily support this or necessarily support this in the same way. +Requesting a specific output schema at invocation time is widely supported by inference services though, and therefore inference based agents would support this well. +Custom agents on the other hand may not necessarily want to support this, and forcing all custom Agent implementations to have a final structured output step to produce this complicates implementations. +Custom agents may also have a built in output schema, that they always produce. + +Options: + +1. Support configuring the preferred structured output schema at agent construction time for those agents that support structured outputs. +2. Support configuring the preferred structured output schema at invocation time, and ignore/throw if not supported (similar to IChatClient) +3. Support both options with the invocation time schema overriding the construction time (or built in) schema if both are supported. + +Note that where an agent doesn't support structured output, it may also be possible to use a decorator to produce structured output from the agent's unstructured response, thereby turning an agent that doesn't support this into one that does. + +See [Structured Outputs Support](#structured-outputs-support) for a comparison on what other agent frameworks and protocols support. + +To support a good user experience for structured outputs, I'm proposing that we follow the pattern used by MEAI. +We would add a generic version of `AgentRunResponse`, that allows us to get the agent result already deserialized into our preferred type. +This would be coupled with generic overload extension methods for Run that automatically builds a schema from the supplied type and updates +the run options. + +If we support requesting a schema at invocation time the following would be the preferred approach: + +```csharp +class Movie +{ + public string Title { get; set; } + public string DirectorFullName { get; set; } + public int ReleaseYear { get; set; } +} + +AgentRunResponse response = agent.RunAsync("What are the top 3 children's movies of the 80s."); +Movie[] movies = response.Result +``` + +If we only support requesting a schema at agent creation time or where an agent has a built in schema, the following would be the preferred approach: + +```csharp +AgentRunResponse response = agent.RunAsync("What are the top 3 children's movies of the 80s."); +Movie[] movies = response.TryParseStructuredOutput(); +``` + +## Decision Outcome + +### Response Type Options Decision + +Option 1.1 with the caveate that we cannot control the output of all agents. However, as far as possible we should have appropriate AIContext derived types for +progress updates so that TextContent is not used for these. + +### Custom Response Type Design Options Decision + +Option 2 chosen so that we can vary Agent responses independently of Chat Client. + +### StructuredOutputs Decision + +We will not support structured output per run request, but individual agents are free to allow this on the concrete implementation or at construction time. +We will however add support for easily extracting a structured output type from the `AgentRunResponse`. + +## Addendum 1: AIContext Derived Types for different response types / Gap Analysis (Work in progress) + +We need to decide what AIContent types, each agent response type will be mapped to. + +| Number | DataType | AIContent Type | +|-|-|-| +| 1. | General response messages to the user | TextContent + DataContent + UriContent | +| 2. | Structured confirmation requests to the user | ? | +| 3. | Function invocation activities executed (both local and remote). For information only. | FunctionCallContent + FunctionResultContent | +| 4. | Tool invocation activities executed (both local and remote). For information only. | FunctionCallContent/FunctionResultContent/Custom ? | +| 5. | Reasoning/Thinking output. For information only. | TextReasoningContent | +| 6. | Handoffs / transitions from agent to agent. | ? | +| 7. | An indication that the agent is responding (i.e. typing) as if it's a real human. | ? | +| 8. | Complete messages in addition to updates, when streaming | TextContent | +| 9. | Id for long running process that is launched | ? | +| 10. | Memory storage / lookups (are these just traces?) | ? | +| 11. | RAG indexing / lookups (are these just traces?) | ? | +| 12. | General status updates for human consumption / Tracing | ? | +| 13. | Unknown Type | AIContent | + +## Addendum 2: Other SDK feature comparison + +### Structured Outputs Support + +1. Configure Schema on Agent at Agent construction +2. Pass schema at Agent invocation + +| SDK | Structured Outputs support | +|-|-| +| AutoGen | **Approach 1** Supports [configuring an agent](https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/tutorial/agents.html#structured-output) at agent creation. | +| Google ADK | **Approach 1** Both [input and output shemas can be specified for LLM Agents](https://google.github.io/adk-docs/agents/llm-agents/#structuring-data-input_schema-output_schema-output_key) at construction time. This option is specific to this agent type and other agent types do not necessarily support | +| AWS (Strands) | **Approach 2** Supports a special invocation method called [structured_output](https://strandsagents.com/latest/api-reference/agent/#strands.agent.agent.Agent.structured_output) | +| LangGraph | **Approach 1** Supports [configuring an agent](https://langchain-ai.github.io/langgraph/agents/agents/?h=structured#6-configure-structured-output) at agent construction time, and a [structured response](https://langchain-ai.github.io/langgraph/agents/run_agents/#output-format) can be retrieved as a special property on the agent response | +| Agno | **Approach 1** Supports [configuring an agent](https://docs.agno.com/examples/getting-started/structured-output) at agent construction time | +| A2A | **Informal Approach 2** Doesn't formally support schema negotiation, but [hints can be provided via metadata](https://a2aproject.github.io/A2A/v0.2.5/specification/#97-structured-data-exchange-requesting-and-providing-json) at invocation time | +| Protocol Activity | Supports returning [Complex types](https://github.com/microsoft/Agents/blob/main/specs/activity/protocol-activity.md#complex-types) but no support for requesting a type | + +### Response Reason Support + +| SDK | Response Reason support | +|-|-| +| AutoGen | Supports a [stop reason](https://microsoft.github.io/autogen/stable/reference/python/autogen_agentchat.base.html#autogen_agentchat.base.TaskResult.stop_reason) which is a freeform text string | +| Google ADK | [No equivalent present](https://github.com/google/adk-python/blob/main/src/google/adk/events/event.py) | +| AWS (Strands) | Exposes a [stop_reason](https://strandsagents.com/latest/api-reference/types/#strands.types.event_loop.StopReason) property on the [AgentResult](https://strandsagents.com/latest/api-reference/agent/#strands.agent.agent_result.AgentResult) class with options that are tied closely to LLM operations. | +| LangGraph | No equivalent present, output contains only [messages](https://langchain-ai.github.io/langgraph/agents/run_agents/#output-format) | +| Agno | [No equivalent present](https://docs.agno.com/reference/agents/run-response) | +| A2A | No equivalent present, response only contains a [message](https://a2aproject.github.io/A2A/v0.2.5/specification/#64-message-object) or [task](https://a2aproject.github.io/A2A/v0.2.5/specification/#61-task-object). | +| Protocol Activity | [No equivalent present.](https://github.com/microsoft/Agents/blob/main/specs/activity/protocol-activity.md) | diff --git a/dotnet/src/Microsoft.Agents.Orchestration/AgentActor.cs b/dotnet/src/Microsoft.Agents.Orchestration/AgentActor.cs index e78638d959..925221ed78 100644 --- a/dotnet/src/Microsoft.Agents.Orchestration/AgentActor.cs +++ b/dotnet/src/Microsoft.Agents.Orchestration/AgentActor.cs @@ -64,7 +64,7 @@ public abstract class AgentActor : OrchestrationActor /// /// Override this method to customize the invocation of the agent. /// - protected virtual Task InvokeAsync( + protected virtual Task InvokeAsync( IReadOnlyCollection messages, AgentRunOptions options, CancellationToken cancellationToken = default) => @@ -84,7 +84,7 @@ public abstract class AgentActor : OrchestrationActor /// /// Override this method to customize the invocation of the agent. /// - protected virtual IAsyncEnumerable InvokeStreamingAsync(IReadOnlyCollection messages, AgentRunOptions options, CancellationToken cancellationToken) => + protected virtual IAsyncEnumerable InvokeStreamingAsync(IReadOnlyCollection messages, AgentRunOptions options, CancellationToken cancellationToken) => this.Agent.RunStreamingAsync( messages, this.Thread, @@ -111,54 +111,46 @@ public abstract class AgentActor : OrchestrationActor { this.Context.Cancellation.ThrowIfCancellationRequested(); - List? responseMessages = []; - ChatResponse response = new(responseMessages); - - AgentRunOptions options = - new() - { - OnIntermediateMessages = HandleMessage, - }; + AgentRunOptions options = new(); if (this.Context.StreamingResponseCallback == null) { // No need to utilize streaming if no callback is provided - await this.InvokeAsync([.. input], options, cancellationToken).ConfigureAwait(false); - } - else - { - IAsyncEnumerable streamedResponses = this.InvokeStreamingAsync([.. input], options, cancellationToken); - ChatResponseUpdate? lastStreamedResponse = null; - await foreach (ChatResponseUpdate streamedResponse in streamedResponses.ConfigureAwait(false)) - { - this.Context.Cancellation.ThrowIfCancellationRequested(); - - await HandleStreamedMessage(lastStreamedResponse, isFinal: false).ConfigureAwait(false); - - lastStreamedResponse = streamedResponse; - } - - await HandleStreamedMessage(lastStreamedResponse, isFinal: true).ConfigureAwait(false); - } - - return response.Messages.Last(); - - async Task HandleMessage(IReadOnlyCollection messages) - { - responseMessages?.AddRange(messages); + AgentRunResponse response = await this.InvokeAsync([.. input], options, cancellationToken).ConfigureAwait(false); if (this.Context.ResponseCallback is not null) { - await this.Context.ResponseCallback.Invoke(messages).ConfigureAwait(false); + await this.Context.ResponseCallback.Invoke(response.Messages).ConfigureAwait(false); } + + return response.Messages.Last(); } - async ValueTask HandleStreamedMessage(ChatResponseUpdate? streamedResponse, bool isFinal) + IAsyncEnumerable streamedResponses = this.InvokeStreamingAsync([.. input], options, cancellationToken); + AgentRunResponseUpdate? lastStreamedResponse = null; + List updates = []; + await foreach (AgentRunResponseUpdate streamedResponse in streamedResponses.ConfigureAwait(false)) + { + this.Context.Cancellation.ThrowIfCancellationRequested(); + + await HandleStreamedMessage(lastStreamedResponse, isFinal: false).ConfigureAwait(false); + + lastStreamedResponse = streamedResponse; + } + + return updates.ToAgentRunResponse().Messages.Last(); + + async ValueTask HandleStreamedMessage(AgentRunResponseUpdate? streamedResponse, bool isFinal) { if (this.Context.StreamingResponseCallback != null && streamedResponse != null) { await this.Context.StreamingResponseCallback.Invoke(streamedResponse, isFinal).ConfigureAwait(false); } + + if (streamedResponse != null) + { + updates.Add(streamedResponse); + } } } } diff --git a/dotnet/src/Microsoft.Agents.Orchestration/AgentOrchestration.cs b/dotnet/src/Microsoft.Agents.Orchestration/AgentOrchestration.cs index c95961469c..8f32787357 100644 --- a/dotnet/src/Microsoft.Agents.Orchestration/AgentOrchestration.cs +++ b/dotnet/src/Microsoft.Agents.Orchestration/AgentOrchestration.cs @@ -27,7 +27,7 @@ public delegate ValueTask OrchestrationResponseCallback(IEnumerable /// /// The agent response /// Indicates if streamed content is final chunk of the message. -public delegate ValueTask OrchestrationStreamingCallback(ChatResponseUpdate response, bool isFinal); +public delegate ValueTask OrchestrationStreamingCallback(AgentRunResponseUpdate response, bool isFinal); /// /// Called when human interaction is requested. diff --git a/dotnet/src/Microsoft.Agents.Orchestration/Handoff/HandoffActor.cs b/dotnet/src/Microsoft.Agents.Orchestration/Handoff/HandoffActor.cs index cbd57e6f29..8d699e9bda 100644 --- a/dotnet/src/Microsoft.Agents.Orchestration/Handoff/HandoffActor.cs +++ b/dotnet/src/Microsoft.Agents.Orchestration/Handoff/HandoffActor.cs @@ -60,7 +60,7 @@ internal sealed class HandoffActor : AgentActor } /// - protected override Task InvokeAsync( + protected override Task InvokeAsync( IReadOnlyCollection messages, AgentRunOptions options, CancellationToken cancellationToken = default) => @@ -72,7 +72,7 @@ internal sealed class HandoffActor : AgentActor cancellationToken); /// - protected override IAsyncEnumerable InvokeStreamingAsync(IReadOnlyCollection messages, AgentRunOptions options, CancellationToken cancellationToken) => + protected override IAsyncEnumerable InvokeStreamingAsync(IReadOnlyCollection messages, AgentRunOptions options, CancellationToken cancellationToken) => this._chatAgent.RunStreamingAsync( messages, this.Thread, diff --git a/dotnet/src/Microsoft.Extensions.AI.Agents.Abstractions/AIContentExtensions.cs b/dotnet/src/Microsoft.Extensions.AI.Agents.Abstractions/AIContentExtensions.cs new file mode 100644 index 0000000000..22e0193e8b --- /dev/null +++ b/dotnet/src/Microsoft.Extensions.AI.Agents.Abstractions/AIContentExtensions.cs @@ -0,0 +1,118 @@ +// Copyright (c) Microsoft. All rights reserved. + +#if NET +using System; +#endif +using System.Collections.Generic; +using System.Linq; +#if NET +using System.Runtime.CompilerServices; +#else +using System.Text; +#endif + +namespace Microsoft.Extensions.AI.Agents; + +// TODO: Consolidate with same internal class in Microsoft.Extensions.AI.Abstractions when both are available in the same repository. +/// Internal extensions for working with . +internal static class AIContentExtensions +{ + /// Concatenates the text of all instances in the list. + public static string ConcatText(this IEnumerable contents) + { + if (contents is IList list) + { + int count = list.Count; + switch (count) + { + case 0: + return string.Empty; + + case 1: + return (list[0] as TextContent)?.Text ?? string.Empty; + + default: +#if NET + DefaultInterpolatedStringHandler builder = new(count, 0, null, stackalloc char[512]); + for (int i = 0; i < count; i++) + { + if (list[i] is TextContent text) + { + builder.AppendLiteral(text.Text); + } + } + + return builder.ToStringAndClear(); +#else + StringBuilder builder = new(); + for (int i = 0; i < count; i++) + { + if (list[i] is TextContent text) + { + builder.Append(text.Text); + } + } + + return builder.ToString(); +#endif + } + } + + return string.Concat(contents.OfType()); + } + + /// Concatenates the of all instances in the list. + /// A newline separator is added between each non-empty piece of text. + public static string ConcatText(this IList messages) + { + int count = messages.Count; + switch (count) + { + case 0: + return string.Empty; + + case 1: + return messages[0].Text; + + default: +#if NET + DefaultInterpolatedStringHandler builder = new(count, 0, null, stackalloc char[512]); + bool needsSeparator = false; + for (int i = 0; i < count; i++) + { + string text = messages[i].Text; + if (text.Length > 0) + { + if (needsSeparator) + { + builder.AppendLiteral(Environment.NewLine); + } + + builder.AppendLiteral(text); + + needsSeparator = true; + } + } + + return builder.ToStringAndClear(); +#else + StringBuilder builder = new(); + for (int i = 0; i < count; i++) + { + string text = messages[i].Text; + if (text.Length > 0) + { + if (builder.Length > 0) + { + builder.AppendLine(); + } + + builder.Append(text); + } + } + + return builder.ToString(); +#endif + } + } +} diff --git a/dotnet/src/Microsoft.Extensions.AI.Agents.Abstractions/Agent.cs b/dotnet/src/Microsoft.Extensions.AI.Agents.Abstractions/Agent.cs index 10e516708d..611d983be2 100644 --- a/dotnet/src/Microsoft.Extensions.AI.Agents.Abstractions/Agent.cs +++ b/dotnet/src/Microsoft.Extensions.AI.Agents.Abstractions/Agent.cs @@ -53,8 +53,8 @@ public abstract class Agent /// The conversation thread to continue with this invocation. If not provided, creates a new thread. The thread will be mutated with the provided messages and agent response. /// Optional parameters for agent invocation. /// The to monitor for cancellation requests. The default is . - /// A containing the list of items. - public Task RunAsync( + /// A containing the list of items. + public Task RunAsync( AgentThread? thread = null, AgentRunOptions? options = null, CancellationToken cancellationToken = default) @@ -69,11 +69,11 @@ public abstract class Agent /// The conversation thread to continue with this invocation. If not provided, creates a new thread. The thread will be mutated with the provided messages and agent response. /// Optional parameters for agent invocation. /// The to monitor for cancellation requests. The default is . - /// A containing the list of items. + /// A containing the list of items. /// /// The provided message string will be treated as a user message. /// - public Task RunAsync( + public Task RunAsync( string message, AgentThread? thread = null, AgentRunOptions? options = null, @@ -91,8 +91,8 @@ public abstract class Agent /// The conversation thread to continue with this invocation. If not provided, creates a new thread. The thread will be mutated with the provided messages and agent response. /// Optional parameters for agent invocation. /// The to monitor for cancellation requests. The default is . - /// A containing the list of items. - public Task RunAsync( + /// A containing the list of items. + public Task RunAsync( ChatMessage message, AgentThread? thread = null, AgentRunOptions? options = null, @@ -110,8 +110,8 @@ public abstract class Agent /// The conversation thread to continue with this invocation. If not provided, creates a new thread. The thread will be mutated with the provided messages and agent response. /// Optional parameters for agent invocation. /// The to monitor for cancellation requests. The default is . - /// A containing the list of items. - public abstract Task RunAsync( + /// A containing the list of items. + public abstract Task RunAsync( IReadOnlyCollection messages, AgentThread? thread = null, AgentRunOptions? options = null, @@ -123,8 +123,8 @@ public abstract class Agent /// The conversation thread to continue with this invocation. If not provided, creates a new thread. The thread will be mutated with the provided messages and agent response. /// Optional parameters for agent invocation. /// The to monitor for cancellation requests. The default is . - /// An async list of response items that each contain a . - public IAsyncEnumerable RunStreamingAsync( + /// An async list of response items that each contain a . + public IAsyncEnumerable RunStreamingAsync( AgentThread? thread = null, AgentRunOptions? options = null, CancellationToken cancellationToken = default) @@ -139,11 +139,11 @@ public abstract class Agent /// The conversation thread to continue with this invocation. If not provided, creates a new thread. The thread will be mutated with the provided messages and agent response. /// Optional parameters for agent invocation. /// The to monitor for cancellation requests. The default is . - /// An async list of response items that each contain a . + /// An async list of response items that each contain a . /// /// The provided message string will be treated as a user message. /// - public IAsyncEnumerable RunStreamingAsync( + public IAsyncEnumerable RunStreamingAsync( string message, AgentThread? thread = null, AgentRunOptions? options = null, @@ -161,8 +161,8 @@ public abstract class Agent /// The conversation thread to continue with this invocation. If not provided, creates a new thread. The thread will be mutated with the provided messages and agent response. /// Optional parameters for agent invocation. /// The to monitor for cancellation requests. The default is . - /// An async list of response items that each contain a . - public IAsyncEnumerable RunStreamingAsync( + /// An async list of response items that each contain a . + public IAsyncEnumerable RunStreamingAsync( ChatMessage message, AgentThread? thread = null, AgentRunOptions? options = null, @@ -180,8 +180,8 @@ public abstract class Agent /// The conversation thread to continue with this invocation. If not provided, creates a new thread. The thread will be mutated with the provided messages and agent response. /// Optional parameters for agent invocation. /// The to monitor for cancellation requests. The default is . - /// An async list of response items that each contain a . - public abstract IAsyncEnumerable RunStreamingAsync( + /// An async list of response items that each contain a . + public abstract IAsyncEnumerable RunStreamingAsync( IReadOnlyCollection messages, AgentThread? thread = null, AgentRunOptions? options = null, diff --git a/dotnet/src/Microsoft.Extensions.AI.Agents.Abstractions/AgentRunOptions.cs b/dotnet/src/Microsoft.Extensions.AI.Agents.Abstractions/AgentRunOptions.cs index 6e0fb2a81a..83a98ffd4e 100644 --- a/dotnet/src/Microsoft.Extensions.AI.Agents.Abstractions/AgentRunOptions.cs +++ b/dotnet/src/Microsoft.Extensions.AI.Agents.Abstractions/AgentRunOptions.cs @@ -1,8 +1,5 @@ // Copyright (c) Microsoft. All rights reserved. -using System; -using System.Collections.Generic; -using System.Threading.Tasks; using Microsoft.Shared.Diagnostics; namespace Microsoft.Extensions.AI.Agents; @@ -26,17 +23,5 @@ public class AgentRunOptions public AgentRunOptions(AgentRunOptions options) { Throw.IfNull(options); - this.OnIntermediateMessages = options.OnIntermediateMessages; } - - /// - /// Gets or sets a function to be called when a complete new message is generated by the agent. - /// - /// - /// - /// This callback is particularly useful in cases where the caller wants to receive complete messages - /// when invoking the agent with streaming. - /// - /// - public Func, Task>? OnIntermediateMessages { get; set; } = null; } diff --git a/dotnet/src/Microsoft.Extensions.AI.Agents.Abstractions/AgentRunResponse.cs b/dotnet/src/Microsoft.Extensions.AI.Agents.Abstractions/AgentRunResponse.cs new file mode 100644 index 0000000000..2dab486bf8 --- /dev/null +++ b/dotnet/src/Microsoft.Extensions.AI.Agents.Abstractions/AgentRunResponse.cs @@ -0,0 +1,244 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Buffers; +using System.Collections.Generic; +using System.Diagnostics.CodeAnalysis; +using System.Text; +using System.Text.Json; +using System.Text.Json.Serialization; +using System.Text.Json.Serialization.Metadata; +using Microsoft.Shared.Diagnostics; + +namespace Microsoft.Extensions.AI.Agents; + +/// Represents the response to an Agent run request. +/// +/// provides one or more response messages and metadata about the response. +/// A typical response will contain a single message, however a response may contain multiple messages +/// in a variety of scenarios. For example, if the agent internally invokes functions or tools, performs +/// RAG retrievals or has other complex logic, a single run by the agent may produce many messages showing +/// the intermediate progress that the agent made towards producing the agent result. +/// +public class AgentRunResponse +{ + private static readonly JsonReaderOptions s_allowMultipleValuesJsonReaderOptions = new() + { +#if NET9_0_OR_GREATER + AllowMultipleValues = true +#endif + }; + + /// The response messages. + private IList? _messages; + + /// Initializes a new instance of the class. + public AgentRunResponse() + { + } + + /// Initializes a new instance of the class. + /// The response message. + /// is . + public AgentRunResponse(ChatMessage message) + { + _ = Throw.IfNull(message); + + this.Messages.Add(message); + } + + /// Initializes a new instance of the class. + /// The response messages. + public AgentRunResponse(IList? messages) + { + this._messages = messages; + } + + /// Gets or sets the agent response messages. + [AllowNull] + public IList Messages + { + get => this._messages ??= new List(1); + set => this._messages = value; + } + + /// Gets the text of the response. + /// + /// This property concatenates the of all + /// instances in . + /// + [JsonIgnore] + public string Text => this._messages?.ConcatText() ?? string.Empty; + + /// Gets or sets the ID of the agent that produced the response. + public string? AgentId { get; set; } + + /// Gets or sets the ID of the agent response. + public string? ResponseId { get; set; } + + /// Gets or sets a timestamp for the run response. + public DateTimeOffset? CreatedAt { get; set; } + + /// Gets or sets usage details for the run response. + /// + /// Where the agent run response is produced via many model invocations, this + /// usage is an aggregation of the usage for all these model invocations. + /// + public UsageDetails? Usage { get; set; } + + /// Gets or sets the raw representation of the run response from an underlying implementation. + /// + /// If a is created to represent some underlying object from another object + /// model, this property can be used to store that original object. This can be useful for debugging or + /// for enabling a consumer to access the underlying object model if needed. + /// + [JsonIgnore] + public object? RawRepresentation { get; set; } + + /// Gets or sets any additional properties associated with the run response. + public AdditionalPropertiesDictionary? AdditionalProperties { get; set; } + + /// + public override string ToString() => this.Text; + + /// Creates an array of instances that represent this . + /// An array of instances that may be used to represent this . + public AgentRunResponseUpdate[] ToAgentRunResponseUpdates() + { + AgentRunResponseUpdate? extra = null; + if (this.AdditionalProperties is not null || this.Usage is not null) + { + extra = new AgentRunResponseUpdate + { + AdditionalProperties = this.AdditionalProperties + }; + + if (this.Usage is { } usage) + { + extra.Contents.Add(new UsageContent(usage)); + } + } + + int messageCount = this._messages?.Count ?? 0; + var updates = new AgentRunResponseUpdate[messageCount + (extra is not null ? 1 : 0)]; + + int i; + for (i = 0; i < messageCount; i++) + { + ChatMessage message = this._messages![i]; + updates[i] = new AgentRunResponseUpdate + { + AdditionalProperties = message.AdditionalProperties, + AuthorName = message.AuthorName, + Contents = message.Contents, + RawRepresentation = message.RawRepresentation, + Role = message.Role, + + AgentId = this.AgentId, + ResponseId = this.ResponseId, + MessageId = message.MessageId, + CreatedAt = this.CreatedAt, + }; + } + + if (extra is not null) + { + updates[i] = extra; + } + + return updates; + } + + // TODO: Add overloads without serializer options. + /// + /// Deserializes the response text into the given type using the specified serializer options. + /// + /// The output type to deserialize into. + /// The JSON serialization options to use. + /// The result as the requested type. + /// The result is not parsable into the requested type. + public T Deserialize(JsonSerializerOptions serializerOptions) + { + var structuredOutput = this.GetResultCore(serializerOptions, out var failureReason); + return failureReason switch + { + FailureReason.ResultDidNotContainJson => throw new InvalidOperationException("The response did not contain JSON to be deserialized."), + FailureReason.DeserializationProducedNull => throw new InvalidOperationException("The deserialized response is null."), + _ => structuredOutput!, + }; + } + + /// + /// Tries to deserialize response text into the given type using the specified serializer options. + /// + /// The output type to deserialize into. + /// The JSON serialization options to use. + /// The parsed structured output. + /// if parsing was successful; otherwise, . + public bool TryDeserialize(JsonSerializerOptions serializerOptions, [NotNullWhen(true)] out T? structuredOutput) + { + try + { + structuredOutput = this.GetResultCore(serializerOptions, out var failureReason); + return failureReason is null; + } +#pragma warning disable CA1031 // Do not catch general exception types + catch + { + structuredOutput = default; + return false; + } +#pragma warning restore CA1031 // Do not catch general exception types + } + + private static T? DeserializeFirstTopLevelObject(string json, JsonTypeInfo typeInfo) + { + // We need to deserialize only the first top-level object as a workaround for a common LLM backend + // issue. GPT 3.5 Turbo commonly returns multiple top-level objects after doing a function call. + // See https://community.openai.com/t/2-json-objects-returned-when-using-function-calling-and-json-mode/574348 + var utf8ByteLength = Encoding.UTF8.GetByteCount(json); + var buffer = ArrayPool.Shared.Rent(utf8ByteLength); + try + { + var utf8SpanLength = Encoding.UTF8.GetBytes(json, 0, json.Length, buffer, 0); + var utf8Span = new ReadOnlySpan(buffer, 0, utf8SpanLength); + var reader = new Utf8JsonReader(utf8Span, s_allowMultipleValuesJsonReaderOptions); + return JsonSerializer.Deserialize(ref reader, typeInfo); + } + finally + { + ArrayPool.Shared.Return(buffer); + } + } + + private T? GetResultCore(JsonSerializerOptions serializerOptions, out FailureReason? failureReason) + { + var json = this.Text; + if (string.IsNullOrEmpty(json)) + { + failureReason = FailureReason.ResultDidNotContainJson; + return default; + } + + T? deserialized = default; + + // If there's an exception here, we want it to propagate, since the Result property is meant to throw directly + + deserialized = DeserializeFirstTopLevelObject(json!, (JsonTypeInfo)serializerOptions.GetTypeInfo(typeof(T))); + + if (deserialized is null) + { + failureReason = FailureReason.DeserializationProducedNull; + return default; + } + + failureReason = default; + return deserialized; + } + + private enum FailureReason + { + ResultDidNotContainJson, + DeserializationProducedNull + } +} diff --git a/dotnet/src/Microsoft.Extensions.AI.Agents.Abstractions/AgentRunResponseUpdate.cs b/dotnet/src/Microsoft.Extensions.AI.Agents.Abstractions/AgentRunResponseUpdate.cs new file mode 100644 index 0000000000..aa8dd1246c --- /dev/null +++ b/dotnet/src/Microsoft.Extensions.AI.Agents.Abstractions/AgentRunResponseUpdate.cs @@ -0,0 +1,132 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Collections.Generic; +using System.Diagnostics; +using System.Diagnostics.CodeAnalysis; +using System.Text.Json.Serialization; + +namespace Microsoft.Extensions.AI.Agents; + +/// +/// Represents a single streaming response chunk from an . +/// +/// +/// +/// is so named because it represents updates +/// that layer on each other to form a single agent response. Conceptually, this combines the roles of +/// and in streaming output. +/// +/// +/// The relationship between and is +/// codified in the and +/// , which enable bidirectional conversions +/// between the two. Note, however, that the provided conversions may be lossy, for example if multiple +/// updates all have different objects whereas there's only one slot for +/// such an object available in . +/// +/// +[DebuggerDisplay("[{Role}] {ContentForDebuggerDisplay}{EllipsesForDebuggerDisplay,nq}")] +public class AgentRunResponseUpdate +{ + /// The response update content items. + private IList? _contents; + + /// The name of the author of the update. + private string? _authorName; + + /// Initializes a new instance of the class. + [JsonConstructor] + public AgentRunResponseUpdate() + { + } + + /// Initializes a new instance of the class. + /// The role of the author of the update. + /// The text content of the update. + public AgentRunResponseUpdate(ChatRole? role, string? content) + : this(role, content is null ? null : [new TextContent(content)]) + { + } + + /// Initializes a new instance of the class. + /// The role of the author of the update. + /// The contents of the update. + public AgentRunResponseUpdate(ChatRole? role, IList? contents) + { + this.Role = role; + this._contents = contents; + } + + /// Gets or sets the name of the author of the response update. + public string? AuthorName + { + get => this._authorName; + set => this._authorName = string.IsNullOrWhiteSpace(value) ? null : value; + } + + /// Gets or sets the role of the author of the response update. + public ChatRole? Role { get; set; } + + /// Gets the text of this update. + /// + /// This property concatenates the text of all objects in . + /// + [JsonIgnore] + public string Text => this._contents is not null ? this._contents.ConcatText() : string.Empty; + + /// Gets or sets the agent run response update content items. + [AllowNull] + public IList Contents + { + get => this._contents ??= []; + set => this._contents = value; + } + + /// Gets or sets the raw representation of the response update from an underlying implementation. + /// + /// If a is created to represent some underlying object from another object + /// model, this property can be used to store that original object. This can be useful for debugging or + /// for enabling a consumer to access the underlying object model if needed. + /// + [JsonIgnore] + public object? RawRepresentation { get; set; } + + /// Gets or sets additional properties for the update. + public AdditionalPropertiesDictionary? AdditionalProperties { get; set; } + + /// Gets or sets the ID of the agent that produced the response. + public string? AgentId { get; set; } + + /// Gets or sets the ID of the response of which this update is a part. + public string? ResponseId { get; set; } + + /// Gets or sets the ID of the message of which this update is a part. + /// + /// A single streaming response may be composed of multiple messages, each of which may be represented + /// by multiple updates. This property is used to group those updates together into messages. + /// + /// Some providers may consider streaming responses to be a single message, and in that case + /// the value of this property may be the same as the response ID. + /// + /// This value is used when + /// groups instances into instances. + /// The value must be unique to each call to the underlying provider, and must be shared by + /// all updates that are part of the same logical message within a streaming response. + /// + public string? MessageId { get; set; } + + /// Gets or sets a timestamp for the response update. + public DateTimeOffset? CreatedAt { get; set; } + + /// + public override string ToString() => this.Text; + + /// Gets a object to display in the debugger display. + [DebuggerBrowsable(DebuggerBrowsableState.Never)] + private AIContent? ContentForDebuggerDisplay => this._contents is { Count: > 0 } ? this._contents[0] : null; + + /// Gets an indication for the debugger display of whether there's more content. + [DebuggerBrowsable(DebuggerBrowsableState.Never)] + private string EllipsesForDebuggerDisplay => this._contents is { Count: > 1 } ? ", ..." : string.Empty; +} diff --git a/dotnet/src/Microsoft.Extensions.AI.Agents.Abstractions/AgentRunResponseUpdateExtensions.cs b/dotnet/src/Microsoft.Extensions.AI.Agents.Abstractions/AgentRunResponseUpdateExtensions.cs new file mode 100644 index 0000000000..4078ce5939 --- /dev/null +++ b/dotnet/src/Microsoft.Extensions.AI.Agents.Abstractions/AgentRunResponseUpdateExtensions.cs @@ -0,0 +1,246 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Collections.Generic; +using System.Linq; +using System.Text; +using System.Threading; +using System.Threading.Tasks; +using Microsoft.Shared.Diagnostics; + +namespace Microsoft.Extensions.AI.Agents; + +/// +/// Provides extension methods for working with instances. +/// +public static class AgentRunResponseUpdateExtensions +{ + /// Combines instances into a single . + /// The updates to be combined. + /// The combined . + /// is . + /// + /// As part of combining into a single , the method will attempt to reconstruct + /// instances. This includes using to determine + /// message boundaries, as well as coalescing contiguous items where applicable, e.g. multiple + /// instances in a row may be combined into a single . + /// + public static AgentRunResponse ToAgentRunResponse( + this IEnumerable updates) + { + _ = Throw.IfNull(updates); + + AgentRunResponse response = new(); + + foreach (var update in updates) + { + ProcessUpdate(update, response); + } + + FinalizeResponse(response); + + return response; + } + + /// Combines instances into a single . + /// The updates to be combined. + /// The to monitor for cancellation requests. The default is . + /// The combined . + /// is . + /// + /// As part of combining into a single , the method will attempt to reconstruct + /// instances. This includes using to determine + /// message boundaries, as well as coalescing contiguous items where applicable, e.g. multiple + /// instances in a row may be combined into a single . + /// + public static Task ToAgentRunResponseAsync( + this IAsyncEnumerable updates, + CancellationToken cancellationToken = default) + { + _ = Throw.IfNull(updates); + + return ToAgentRunResponseAsync(updates, cancellationToken); + + static async Task ToAgentRunResponseAsync( + IAsyncEnumerable updates, + CancellationToken cancellationToken) + { + AgentRunResponse response = new(); + + await foreach (var update in updates.WithCancellation(cancellationToken).ConfigureAwait(false)) + { + ProcessUpdate(update, response); + } + + FinalizeResponse(response); + + return response; + } + } + + /// Coalesces sequential content elements. + internal static void CoalesceTextContent(List contents) + { + Coalesce(contents, static text => new(text)); + Coalesce(contents, static text => new(text)); + + // This implementation relies on TContent's ToString returning its exact text. + static void Coalesce(List contents, Func fromText) + where TContent : AIContent + { + StringBuilder? coalescedText = null; + + // Iterate through all of the items in the list looking for contiguous items that can be coalesced. + int start = 0; + while (start < contents.Count - 1) + { + // We need at least two TextContents in a row to be able to coalesce. + if (contents[start] is not TContent firstText) + { + start++; + continue; + } + + if (contents[start + 1] is not TContent secondText) + { + start += 2; + continue; + } + + // Append the text from those nodes and continue appending subsequent TextContents until we run out. + // We null out nodes as their text is appended so that we can later remove them all in one O(N) operation. + coalescedText ??= new(); + _ = coalescedText.Clear().Append(firstText).Append(secondText); + contents[start + 1] = null!; + int i = start + 2; + for (; i < contents.Count && contents[i] is TContent next; i++) + { + _ = coalescedText.Append(next); + contents[i] = null!; + } + + // Store the replacement node. We inherit the properties of the first text node. We don't + // currently propagate additional properties from the subsequent nodes. If we ever need to, + // we can add that here. + var newContent = fromText(coalescedText.ToString()); + contents[start] = newContent; + newContent.AdditionalProperties = firstText.AdditionalProperties?.Clone(); + + start = i; + } + + // Remove all of the null slots left over from the coalescing process. + _ = contents.RemoveAll(u => u is null); + } + } + + /// Finalizes the object. + private static void FinalizeResponse(AgentRunResponse response) + { + int count = response.Messages.Count; + for (int i = 0; i < count; i++) + { + CoalesceTextContent((List)response.Messages[i].Contents); + } + } + + /// Processes the , incorporating its contents into . + /// The update to process. + /// The object that should be updated based on . + private static void ProcessUpdate(AgentRunResponseUpdate update, AgentRunResponse response) + { + // If there is no message created yet, or if the last update we saw had a different + // message ID than the newest update, create a new message. + ChatMessage message; + var isNewMessage = false; + if (response.Messages.Count == 0) + { + isNewMessage = true; + } + else if (update.MessageId is { Length: > 0 } updateMessageId + && response.Messages[response.Messages.Count - 1].MessageId is string lastMessageId + && updateMessageId != lastMessageId) + { + isNewMessage = true; + } + + if (isNewMessage) + { + message = new ChatMessage(ChatRole.Assistant, []); + response.Messages.Add(message); + } + else + { + message = response.Messages[response.Messages.Count - 1]; + } + + // Some members on AgentRunResponseUpdate map to members of ChatMessage. + // Incorporate those into the latest message; in cases where the message + // stores a single value, prefer the latest update's value over anything + // stored in the message. + if (update.AuthorName is not null) + { + message.AuthorName = update.AuthorName; + } + + if (update.Role is ChatRole role) + { + message.Role = role; + } + + if (update.MessageId is { Length: > 0 }) + { + // Note that this must come after the message checks earlier, as they depend + // on this value for change detection. + message.MessageId = update.MessageId; + } + + foreach (var content in update.Contents) + { + switch (content) + { + // Usage content is treated specially and propagated to the response's Usage. + case UsageContent usage: + (response.Usage ??= new()).Add(usage.Details); + break; + + default: + message.Contents.Add(content); + break; + } + } + + // Other members on a AgentRunResponseUpdate map to members of the AgentRunResponse. + // Update the response object with those, preferring the values from later updates. + + if (update.AgentId is { Length: > 0 }) + { + response.AgentId = update.AgentId; + } + + if (update.ResponseId is { Length: > 0 }) + { + response.ResponseId = update.ResponseId; + } + + if (update.CreatedAt is not null) + { + response.CreatedAt = update.CreatedAt; + } + + if (update.AdditionalProperties is not null) + { + if (response.AdditionalProperties is null) + { + response.AdditionalProperties = new(update.AdditionalProperties); + } + else + { + foreach (var item in update.AdditionalProperties) + { + response.AdditionalProperties[item.Key] = item.Value; + } + } + } + } +} diff --git a/dotnet/src/Microsoft.Extensions.AI.Agents.CopilotStudio/ActivityProcessor.cs b/dotnet/src/Microsoft.Extensions.AI.Agents.CopilotStudio/ActivityProcessor.cs index 7b10f52d4e..e36c57ea78 100644 --- a/dotnet/src/Microsoft.Extensions.AI.Agents.CopilotStudio/ActivityProcessor.cs +++ b/dotnet/src/Microsoft.Extensions.AI.Agents.CopilotStudio/ActivityProcessor.cs @@ -12,7 +12,7 @@ namespace Microsoft.Extensions.AI.Agents.CopilotStudio; /// internal static class ActivityProcessor { - public static async IAsyncEnumerable<(ChatMessage message, bool reasoning)> ProcessActivityAsync(IAsyncEnumerable activities, bool streaming, ILogger logger) + public static async IAsyncEnumerable ProcessActivityAsync(IAsyncEnumerable activities, bool streaming, ILogger logger) { await foreach (IActivity activity in activities.ConfigureAwait(false)) { @@ -27,13 +27,13 @@ internal static class ActivityProcessor // pick from a list of actions. // The activity text doesn't make sense without the actions, as the message // is often instructing the user to pick from the provided list of actions. - yield return (CreateChatMessageFromActivity(activity, [new TextContent(activity.Text)]), false); + yield return CreateChatMessageFromActivity(activity, [new TextContent(activity.Text)]); break; case "typing": case "event": // TODO: Revisit usage of TextReasoningContent here, to evaluate whether all are really reasoning // or whether simply an AIContent base type would be more appropriate. - yield return (CreateChatMessageFromActivity(activity, [new TextReasoningContent(activity.Text)]), true); + yield return CreateChatMessageFromActivity(activity, [new TextReasoningContent(activity.Text)]); break; default: logger.LogWarning("Unknown activity type '{ActivityType}' received.", activity.Type); diff --git a/dotnet/src/Microsoft.Extensions.AI.Agents.CopilotStudio/CopilotStudioAgent.cs b/dotnet/src/Microsoft.Extensions.AI.Agents.CopilotStudio/CopilotStudioAgent.cs index 3c605224a1..c972cf8db7 100644 --- a/dotnet/src/Microsoft.Extensions.AI.Agents.CopilotStudio/CopilotStudioAgent.cs +++ b/dotnet/src/Microsoft.Extensions.AI.Agents.CopilotStudio/CopilotStudioAgent.cs @@ -44,7 +44,7 @@ public class CopilotStudioAgent : Agent } /// - public override async Task RunAsync( + public override async Task RunAsync( IReadOnlyCollection messages, AgentThread? thread = null, AgentRunOptions? options = null, @@ -63,39 +63,24 @@ public class CopilotStudioAgent : Agent // Invoke the Copilot Studio agent with the provided messages. string question = string.Join("\n", messages.Select(m => m.Text)); var responseMessages = ActivityProcessor.ProcessActivityAsync(this.Client.AskQuestionAsync(question, copilotStudioAgentThread.Id, cancellationToken), streaming: false, this._logger); - - // Enumerate the response messages var responseMessagesList = new List(); - await foreach ((ChatMessage message, bool reasoning) in responseMessages.ConfigureAwait(false)) + await foreach (var message in responseMessages.ConfigureAwait(false)) { - // If the message is a reasoning message, return it as part of the intermediate messages - // instead of the final response. - if (reasoning) - { - if (options?.OnIntermediateMessages is not null) - { - await options.OnIntermediateMessages.Invoke([message]).ConfigureAwait(false); - } - - continue; - } - - // Add the message to the list responseMessagesList.Add(message); } // TODO: Review list of ChatResponse properties to ensure we set all availble values. // Setting ResponseId and MessageId end up being particularly important for streaming consumers // so that they can tell things like response boundaries. - return new ChatResponse(responseMessagesList) + return new AgentRunResponse(responseMessagesList) { + AgentId = this.Id, ResponseId = responseMessagesList.LastOrDefault()?.MessageId, - ConversationId = copilotStudioAgentThread.Id, }; } /// - public override async IAsyncEnumerable RunStreamingAsync( + public override async IAsyncEnumerable RunStreamingAsync( IReadOnlyCollection messages, AgentThread? thread = null, AgentRunOptions? options = null, @@ -116,25 +101,19 @@ public class CopilotStudioAgent : Agent var responseMessages = ActivityProcessor.ProcessActivityAsync(this.Client.AskQuestionAsync(question, copilotStudioAgentThread.Id, cancellationToken), streaming: true, this._logger); // Enumerate the response messages - await foreach ((ChatMessage message, bool reasoning) in responseMessages.ConfigureAwait(false)) + await foreach (ChatMessage message in responseMessages.ConfigureAwait(false)) { - // If the message is a reasoning message, return it as part of the intermediate messages. - if (reasoning && options?.OnIntermediateMessages is not null) - { - await options.OnIntermediateMessages.Invoke([message]).ConfigureAwait(false); - } - // TODO: Review list of ChatResponse properties to ensure we set all availble values. // Setting ResponseId and MessageId end up being particularly important for streaming consumers // so that they can tell things like response boundaries. - yield return new ChatResponseUpdate(message.Role, message.Contents) + yield return new AgentRunResponseUpdate(message.Role, message.Contents) { + AgentId = this.Id, AdditionalProperties = message.AdditionalProperties, AuthorName = message.AuthorName, RawRepresentation = message.RawRepresentation, ResponseId = message.MessageId, MessageId = message.MessageId, - ConversationId = copilotStudioAgentThread.Id, }; } } diff --git a/dotnet/src/Microsoft.Extensions.AI.Agents/ChatCompletion/ChatClientAgent.cs b/dotnet/src/Microsoft.Extensions.AI.Agents/ChatCompletion/ChatClientAgent.cs index d530f27cf2..feaf07969c 100644 --- a/dotnet/src/Microsoft.Extensions.AI.Agents/ChatCompletion/ChatClientAgent.cs +++ b/dotnet/src/Microsoft.Extensions.AI.Agents/ChatCompletion/ChatClientAgent.cs @@ -67,7 +67,7 @@ public sealed class ChatClientAgent : Agent internal ChatOptions? ChatOptions => this._agentOptions?.ChatOptions; /// - public override async Task RunAsync( + public override async Task RunAsync( IReadOnlyCollection messages, AgentThread? thread = null, AgentRunOptions? options = null, @@ -103,16 +103,12 @@ public sealed class ChatClientAgent : Agent var chatResponseMessages = chatResponse.Messages as IReadOnlyCollection ?? chatResponse.Messages.ToArray(); await this.NotifyThreadOfNewMessagesAsync(chatClientThread, chatResponseMessages, cancellationToken).ConfigureAwait(false); - if (options?.OnIntermediateMessages is not null) - { - await options.OnIntermediateMessages(chatResponseMessages).ConfigureAwait(false); - } - return chatResponse; + return chatResponse.ToAgentRunResponse(this.Id); } /// - public override async IAsyncEnumerable RunStreamingAsync( + public override async IAsyncEnumerable RunStreamingAsync( IReadOnlyCollection messages, AgentThread? thread = null, AgentRunOptions? options = null, @@ -145,7 +141,7 @@ public sealed class ChatClientAgent : Agent { responseUpdates.Add(update); update.AuthorName ??= agentName; - yield return update; + yield return update.ToAgentRunResponseUpdate(this.Id); } hasUpdates = await responseUpdatesEnumerator.MoveNextAsync().ConfigureAwait(false); @@ -162,10 +158,6 @@ public sealed class ChatClientAgent : Agent await this.NotifyThreadOfNewMessagesAsync(chatClientThread, inputMessages, cancellationToken).ConfigureAwait(false); await this.NotifyThreadOfNewMessagesAsync(chatClientThread, chatResponseMessages, cancellationToken).ConfigureAwait(false); - if (options?.OnIntermediateMessages is not null) - { - await options.OnIntermediateMessages(chatResponseMessages).ConfigureAwait(false); - } } /// diff --git a/dotnet/src/Microsoft.Extensions.AI.Agents/ChatCompletion/ChatClientAgentExtensions.cs b/dotnet/src/Microsoft.Extensions.AI.Agents/ChatCompletion/ChatClientAgentExtensions.cs index 4c1691540f..d66a34f546 100644 --- a/dotnet/src/Microsoft.Extensions.AI.Agents/ChatCompletion/ChatClientAgentExtensions.cs +++ b/dotnet/src/Microsoft.Extensions.AI.Agents/ChatCompletion/ChatClientAgentExtensions.cs @@ -21,8 +21,8 @@ public static class ChatClientAgentExtensions /// Optional parameters for agent invocation. /// Optional chat options. /// The to monitor for cancellation requests. The default is . - /// A containing the list of items. - public static Task RunAsync( + /// A containing the list of items. + public static Task RunAsync( this ChatClientAgent agent, IReadOnlyCollection messages, AgentThread? thread = null, @@ -45,8 +45,8 @@ public static class ChatClientAgentExtensions /// Optional parameters for agent invocation. /// Optional chat options. /// The to monitor for cancellation requests. The default is . - /// A containing the list of items. - public static Task RunAsync( + /// A containing the list of items. + public static Task RunAsync( this ChatClientAgent agent, string prompt, AgentThread? thread = null, @@ -69,7 +69,7 @@ public static class ChatClientAgentExtensions /// Optional parameters for agent invocation. /// Optional chat options. /// The to monitor for cancellation requests. The default is . - public static IAsyncEnumerable RunStreamingAsync( + public static IAsyncEnumerable RunStreamingAsync( this ChatClientAgent agent, IReadOnlyCollection messages, AgentThread? thread = null, @@ -92,8 +92,8 @@ public static class ChatClientAgentExtensions /// Optional parameters for agent invocation. /// Optional chat options. /// The to monitor for cancellation requests. The default is . - /// An async enumerable of items for streaming the response. - public static IAsyncEnumerable RunStreamingAsync( + /// An async enumerable of items for streaming the response. + public static IAsyncEnumerable RunStreamingAsync( this ChatClientAgent agent, string prompt, AgentThread? thread = null, diff --git a/dotnet/src/Microsoft.Extensions.AI.Agents/ChatCompletion/ChatClientAgentRunOptions.cs b/dotnet/src/Microsoft.Extensions.AI.Agents/ChatCompletion/ChatClientAgentRunOptions.cs index 110a8febc3..f3043a35f8 100644 --- a/dotnet/src/Microsoft.Extensions.AI.Agents/ChatCompletion/ChatClientAgentRunOptions.cs +++ b/dotnet/src/Microsoft.Extensions.AI.Agents/ChatCompletion/ChatClientAgentRunOptions.cs @@ -14,7 +14,6 @@ internal sealed class ChatClientAgentRunOptions : AgentRunOptions /// Optional chat options to pass to the agent's invocation. internal ChatClientAgentRunOptions(AgentRunOptions? source = null, ChatOptions? chatOptions = null) { - this.OnIntermediateMessages = source?.OnIntermediateMessages; this.ChatOptions = chatOptions; } diff --git a/dotnet/src/Microsoft.Extensions.AI.Agents/ChatCompletion/ChatResponseExtensions.cs b/dotnet/src/Microsoft.Extensions.AI.Agents/ChatCompletion/ChatResponseExtensions.cs new file mode 100644 index 0000000000..02c7d3950c --- /dev/null +++ b/dotnet/src/Microsoft.Extensions.AI.Agents/ChatCompletion/ChatResponseExtensions.cs @@ -0,0 +1,60 @@ +// Copyright (c) Microsoft. All rights reserved. + +using Microsoft.Shared.Diagnostics; + +namespace Microsoft.Extensions.AI.Agents; + +/// +/// Contains extension methods for and . +/// +internal static class ChatResponseExtensions +{ + /// + /// Converts a instance to an . + /// + /// The to convert. Cannot be . + /// The ID of the agent that generated the response. Cannot be . + /// + /// An containing the messages, metadata, and additional properties from the + /// specified . + /// + public static AgentRunResponse ToAgentRunResponse(this ChatResponse chatResponse, string agentId) + { + _ = Throw.IfNull(chatResponse); + _ = Throw.IfNullOrWhitespace(agentId); + + return new AgentRunResponse(chatResponse.Messages) + { + AgentId = agentId, + ResponseId = chatResponse.ResponseId, + CreatedAt = chatResponse.CreatedAt, + Usage = chatResponse.Usage, + RawRepresentation = chatResponse, + AdditionalProperties = chatResponse.AdditionalProperties + }; + } + + /// + /// Converts a instance to an . + /// + /// The to convert. Cannot be . + /// The ID of the agent that generated the response. Cannot be . + /// An containing the properties from the specified . + public static AgentRunResponseUpdate ToAgentRunResponseUpdate(this ChatResponseUpdate chatResponseUpdate, string agentId) + { + _ = Throw.IfNull(chatResponseUpdate); + + return new() + { + AgentId = agentId, + Role = chatResponseUpdate.Role, + AuthorName = chatResponseUpdate.AuthorName, + Contents = chatResponseUpdate.Contents, + MessageId = chatResponseUpdate.MessageId, + ResponseId = chatResponseUpdate.ResponseId, + CreatedAt = chatResponseUpdate.CreatedAt, + RawRepresentation = chatResponseUpdate, + AdditionalProperties = chatResponseUpdate.AdditionalProperties + }; + } +} diff --git a/dotnet/src/Shared/Samples/BaseSample.cs b/dotnet/src/Shared/Samples/BaseSample.cs index ed3c80a014..020e98070c 100644 --- a/dotnet/src/Shared/Samples/BaseSample.cs +++ b/dotnet/src/Shared/Samples/BaseSample.cs @@ -4,6 +4,7 @@ using System.Reflection; using System.Text; using System.Text.Json; using Microsoft.Extensions.AI; +using Microsoft.Extensions.AI.Agents; using Microsoft.Extensions.Configuration; using Microsoft.Extensions.Logging; using Microsoft.Shared.Samples; @@ -87,29 +88,29 @@ public abstract class BaseSample : TextWriter /// Processes and writes the latest agent chat response to the console, including metadata and content details. /// /// This method formats and outputs the most recent message from the provided object. It includes the message role, author name (if available), text content, and + /// cref="AgentRunResponse"/> object. It includes the message role, author name (if available), text content, and /// additional content such as images, function calls, and function results. Usage statistics, including token /// counts, are also displayed. - /// The object containing the chat messages and usage data. + /// The object containing the chat messages and usage data. /// The flag to indicate whether to print usage information. Defaults to . - protected void WriteResponseOutput(ChatResponse chatResponse, bool? printUsage = true) + protected void WriteResponseOutput(AgentRunResponse response, bool? printUsage = true) { - if (chatResponse.Messages.Count == 0) + if (response.Messages.Count == 0) { // If there are no messages, we can skip writing the message. return; } - var message = chatResponse.Messages.Last(); + var message = response.Messages.Last(); this.WriteMessageOutput(message); WriteUsage(); void WriteUsage() { - if (!(printUsage ?? true) || chatResponse.Usage is null) { return; } + if (!(printUsage ?? true) || response.Usage is null) { return; } - UsageDetails usageDetails = chatResponse.Usage; + UsageDetails usageDetails = response.Usage; Console.WriteLine($" [Usage] Tokens: {usageDetails.TotalTokenCount}, Input: {usageDetails.InputTokenCount}, Output: {usageDetails.OutputTokenCount}"); } @@ -151,11 +152,11 @@ public abstract class BaseSample : TextWriter /// Writes the streaming agent response updates to the console. /// /// This method formats and outputs the most recent message from the provided object. It includes the message role, author name (if available), text content, and + /// cref="AgentRunResponseUpdate"/> object. It includes the message role, author name (if available), text content, and /// additional content such as images, function calls, and function results. Usage statistics, including token /// counts, are also displayed. - /// The object containing the chat messages and usage data. - protected void WriteAgentOutput(ChatResponseUpdate update) + /// The object containing the chat messages and usage data. + protected void WriteAgentOutput(AgentRunResponseUpdate update) { if (update.Contents.Count == 0) { diff --git a/dotnet/src/Shared/Samples/OrchestrationSample.cs b/dotnet/src/Shared/Samples/OrchestrationSample.cs index 3f9fa69ebc..1d5570ef77 100644 --- a/dotnet/src/Shared/Samples/OrchestrationSample.cs +++ b/dotnet/src/Shared/Samples/OrchestrationSample.cs @@ -74,7 +74,7 @@ public abstract class OrchestrationSample : BaseSample } /// - /// Writes the provided chat response messages to the console or test output, including role and author information. + /// Writes the provided messages to the console or test output, including role and author information. /// /// An enumerable of objects to write. protected static void WriteResponse(IEnumerable response) @@ -89,15 +89,15 @@ public abstract class OrchestrationSample : BaseSample } /// - /// Writes the streamed chat response updates to the console or test output, including role and author information. + /// Writes the streamed agent run response updates to the console or test output, including role and author information. /// - /// An enumerable of objects representing streamed responses. - protected static void WriteStreamedResponse(IEnumerable streamedResponses) + /// An enumerable of objects representing streamed responses. + protected static void WriteStreamedResponse(IEnumerable streamedResponses) { string? authorName = null; ChatRole? authorRole = null; StringBuilder builder = new(); - foreach (ChatResponseUpdate response in streamedResponses) + foreach (AgentRunResponseUpdate response in streamedResponses) { authorName ??= response.AuthorName; authorRole ??= response.Role; @@ -122,7 +122,7 @@ public abstract class OrchestrationSample : BaseSample /// /// Gets the list of streamed response updates received so far. /// - public List StreamedResponses { get; } = []; + public List StreamedResponses { get; } = []; /// /// Gets the list of chat messages representing the conversation history. @@ -142,12 +142,12 @@ public abstract class OrchestrationSample : BaseSample } /// - /// Callback to handle a streamed chat response update, adding it to the list and writing output if final. + /// Callback to handle a streamed agent run response update, adding it to the list and writing output if final. /// - /// The to process. + /// The to process. /// Indicates whether this is the final update in the stream. /// A representing the asynchronous operation. - public ValueTask StreamingResultCallback(ChatResponseUpdate streamedResponse, bool isFinal) + public ValueTask StreamingResultCallback(AgentRunResponseUpdate streamedResponse, bool isFinal) { this.StreamedResponses.Add(streamedResponse); diff --git a/dotnet/tests/AgentConformance.IntegrationTests/ChatClientAgentRunStreamingTests.cs b/dotnet/tests/AgentConformance.IntegrationTests/ChatClientAgentRunStreamingTests.cs index 52d12a1109..b94977ecd7 100644 --- a/dotnet/tests/AgentConformance.IntegrationTests/ChatClientAgentRunStreamingTests.cs +++ b/dotnet/tests/AgentConformance.IntegrationTests/ChatClientAgentRunStreamingTests.cs @@ -27,10 +27,10 @@ public abstract class ChatClientAgentRunStreamingTests(Func x.Text)); + var chatResponseText = string.Concat(responseUpdates.Select(x => x.Text)); Assert.Contains("Computer says no", chatResponseText, StringComparison.OrdinalIgnoreCase); } @@ -58,12 +58,12 @@ public abstract class ChatClientAgentRunStreamingTests(Func x.Text)); + var chatResponseText = string.Concat(responseUpdates.Select(x => x.Text)); Assert.Contains(questionAndAnswer.ExpectedAnswer, chatResponseText, StringComparison.OrdinalIgnoreCase); } } diff --git a/dotnet/tests/AgentConformance.IntegrationTests/ChatClientAgentRunTests.cs b/dotnet/tests/AgentConformance.IntegrationTests/ChatClientAgentRunTests.cs index b454adbac0..60d73fa1fd 100644 --- a/dotnet/tests/AgentConformance.IntegrationTests/ChatClientAgentRunTests.cs +++ b/dotnet/tests/AgentConformance.IntegrationTests/ChatClientAgentRunTests.cs @@ -26,12 +26,12 @@ public abstract class ChatClientAgentRunTests(Func await using var threadCleanup = new ThreadCleanup(thread, this.Fixture); // Act - var chatResponse = await agent.RunAsync(thread); + var response = await agent.RunAsync(thread); // Assert - Assert.NotNull(chatResponse); - Assert.Single(chatResponse.Messages); - Assert.Contains("Computer says no", chatResponse.Text, StringComparison.OrdinalIgnoreCase); + Assert.NotNull(response); + Assert.Single(response.Messages); + Assert.Contains("Computer says no", response.Text, StringComparison.OrdinalIgnoreCase); } [RetryFact(Constants.RetryCount, Constants.RetryDelay)] diff --git a/dotnet/tests/AgentConformance.IntegrationTests/RunStreamingTests.cs b/dotnet/tests/AgentConformance.IntegrationTests/RunStreamingTests.cs index 4f922e4bb4..4e3bc65dcd 100644 --- a/dotnet/tests/AgentConformance.IntegrationTests/RunStreamingTests.cs +++ b/dotnet/tests/AgentConformance.IntegrationTests/RunStreamingTests.cs @@ -37,10 +37,10 @@ public abstract class RunStreamingTests(Func creat await using var cleanup = new ThreadCleanup(thread, this.Fixture); // Act - var chatResponses = await agent.RunStreamingAsync("What is the capital of France.", thread).ToListAsync(); + var responseUpdates = await agent.RunStreamingAsync("What is the capital of France.", thread).ToListAsync(); // Assert - var chatResponseText = string.Concat(chatResponses.Select(x => x.Text)); + var chatResponseText = string.Concat(responseUpdates.Select(x => x.Text)); Assert.Contains("Paris", chatResponseText); } @@ -53,10 +53,10 @@ public abstract class RunStreamingTests(Func creat await using var cleanup = new ThreadCleanup(thread, this.Fixture); // Act - var chatResponses = await agent.RunStreamingAsync(new ChatMessage(ChatRole.User, "What is the capital of France."), thread).ToListAsync(); + var responseUpdates = await agent.RunStreamingAsync(new ChatMessage(ChatRole.User, "What is the capital of France."), thread).ToListAsync(); // Assert - var chatResponseText = string.Concat(chatResponses.Select(x => x.Text)); + var chatResponseText = string.Concat(responseUpdates.Select(x => x.Text)); Assert.Contains("Paris", chatResponseText); } @@ -69,7 +69,7 @@ public abstract class RunStreamingTests(Func creat await using var cleanup = new ThreadCleanup(thread, this.Fixture); // Act - var chatResponses = await agent.RunStreamingAsync( + var responseUpdates = await agent.RunStreamingAsync( [ new ChatMessage(ChatRole.User, "Hello."), new ChatMessage(ChatRole.User, "What is the capital of France.") @@ -77,7 +77,7 @@ public abstract class RunStreamingTests(Func creat thread).ToListAsync(); // Assert - var chatResponseText = string.Concat(chatResponses.Select(x => x.Text)); + var chatResponseText = string.Concat(responseUpdates.Select(x => x.Text)); Assert.Contains("Paris", chatResponseText); } @@ -92,14 +92,14 @@ public abstract class RunStreamingTests(Func creat await using var cleanup = new ThreadCleanup(thread, this.Fixture); // Act - var chatResponses1 = await agent.RunStreamingAsync(q1, thread).ToListAsync(); - var chatResponses2 = await agent.RunStreamingAsync(q2, thread).ToListAsync(); + var responseUpdates1 = await agent.RunStreamingAsync(q1, thread).ToListAsync(); + var responseUpdates2 = await agent.RunStreamingAsync(q2, thread).ToListAsync(); // Assert - var chatResponse1Text = string.Concat(chatResponses1.Select(x => x.Text)); - var chatResponse2Text = string.Concat(chatResponses2.Select(x => x.Text)); - Assert.Contains("Paris", chatResponse1Text); - Assert.Contains("Vienna", chatResponse2Text); + var response1Text = string.Concat(responseUpdates1.Select(x => x.Text)); + var response2Text = string.Concat(responseUpdates2.Select(x => x.Text)); + Assert.Contains("Paris", response1Text); + Assert.Contains("Vienna", response2Text); var chatHistory = await this.Fixture.GetChatHistoryAsync(thread); Assert.Equal(4, chatHistory.Count); diff --git a/dotnet/tests/AgentConformance.IntegrationTests/RunTests.cs b/dotnet/tests/AgentConformance.IntegrationTests/RunTests.cs index f8e4915742..d2b34eb419 100644 --- a/dotnet/tests/AgentConformance.IntegrationTests/RunTests.cs +++ b/dotnet/tests/AgentConformance.IntegrationTests/RunTests.cs @@ -40,12 +40,13 @@ public abstract class RunTests(Func createAgentFix await using var cleanup = new ThreadCleanup(thread, this.Fixture); // Act - var chatResponse = await agent.RunAsync("What is the capital of France.", thread); + var response = await agent.RunAsync("What is the capital of France.", thread); // Assert - Assert.NotNull(chatResponse); - Assert.Single(chatResponse.Messages); - Assert.Contains("Paris", chatResponse.Text); + Assert.NotNull(response); + Assert.Single(response.Messages); + Assert.Contains("Paris", response.Text); + Assert.Equal(agent.Id, response.AgentId); } [RetryFact(Constants.RetryCount, Constants.RetryDelay)] @@ -57,12 +58,12 @@ public abstract class RunTests(Func createAgentFix await using var cleanup = new ThreadCleanup(thread, this.Fixture); // Act - var chatResponse = await agent.RunAsync(new ChatMessage(ChatRole.User, "What is the capital of France."), thread); + var response = await agent.RunAsync(new ChatMessage(ChatRole.User, "What is the capital of France."), thread); // Assert - Assert.NotNull(chatResponse); - Assert.Single(chatResponse.Messages); - Assert.Contains("Paris", chatResponse.Text); + Assert.NotNull(response); + Assert.Single(response.Messages); + Assert.Contains("Paris", response.Text); } [RetryFact(Constants.RetryCount, Constants.RetryDelay)] @@ -74,7 +75,7 @@ public abstract class RunTests(Func createAgentFix await using var cleanup = new ThreadCleanup(thread, this.Fixture); // Act - var chatResponse = await agent.RunAsync( + var response = await agent.RunAsync( [ new ChatMessage(ChatRole.User, "Hello."), new ChatMessage(ChatRole.User, "What is the capital of France.") @@ -82,9 +83,9 @@ public abstract class RunTests(Func createAgentFix thread); // Assert - Assert.NotNull(chatResponse); - Assert.Single(chatResponse.Messages); - Assert.Contains("Paris", chatResponse.Text); + Assert.NotNull(response); + Assert.Single(response.Messages); + Assert.Contains("Paris", response.Text); } [RetryFact(Constants.RetryCount, Constants.RetryDelay)] diff --git a/dotnet/tests/Microsoft.Agents.Orchestration.UnitTests/MockAgent.cs b/dotnet/tests/Microsoft.Agents.Orchestration.UnitTests/MockAgent.cs index 671a5a2c4b..9f734694d2 100644 --- a/dotnet/tests/Microsoft.Agents.Orchestration.UnitTests/MockAgent.cs +++ b/dotnet/tests/Microsoft.Agents.Orchestration.UnitTests/MockAgent.cs @@ -1,7 +1,7 @@ // Copyright (c) Microsoft. All rights reserved. using System; using System.Collections.Generic; -using System.Runtime.CompilerServices; +using System.Linq; using System.Threading; using System.Threading.Tasks; using Microsoft.Extensions.AI; @@ -36,7 +36,7 @@ internal sealed class MockAgent(int index) : Agent return new AgentThread() { Id = Guid.NewGuid().ToString() }; } - public override async Task RunAsync(IReadOnlyCollection messages, AgentThread? thread = null, AgentRunOptions? options = null, CancellationToken cancellationToken = default) + public override Task RunAsync(IReadOnlyCollection messages, AgentThread? thread = null, AgentRunOptions? options = null, CancellationToken cancellationToken = default) { this.InvokeCount++; if (thread == null) @@ -45,20 +45,13 @@ internal sealed class MockAgent(int index) : Agent thread = mockThread.Object; } - await (options?.OnIntermediateMessages?.Invoke(this.Response) ?? Task.CompletedTask); - - return new ChatResponse(messages: [.. this.Response]); + return Task.FromResult(new AgentRunResponse(messages: [.. this.Response])); } - public override async IAsyncEnumerable RunStreamingAsync(IReadOnlyCollection messages, AgentThread? thread = null, AgentRunOptions? options = null, [EnumeratorCancellation] CancellationToken cancellationToken = default) + public override IAsyncEnumerable RunStreamingAsync(IReadOnlyCollection messages, AgentThread? thread = null, AgentRunOptions? options = null, CancellationToken cancellationToken = default) { this.InvokeCount++; - await (options?.OnIntermediateMessages?.Invoke(this.Response) ?? Task.CompletedTask); - - foreach (ChatMessage message in this.Response) - { - yield return new ChatResponseUpdate(message.Role, message.Text); - } + return this.Response.Select(message => new AgentRunResponseUpdate(message.Role, message.Text)).ToAsyncEnumerable(); } } diff --git a/dotnet/tests/Microsoft.Extensions.AI.Agents.Abstractions.UnitTests/AgentRunOptionsTests.cs b/dotnet/tests/Microsoft.Extensions.AI.Agents.Abstractions.UnitTests/AgentRunOptionsTests.cs index a37d288ed6..c602af78f0 100644 --- a/dotnet/tests/Microsoft.Extensions.AI.Agents.Abstractions.UnitTests/AgentRunOptionsTests.cs +++ b/dotnet/tests/Microsoft.Extensions.AI.Agents.Abstractions.UnitTests/AgentRunOptionsTests.cs @@ -1,7 +1,6 @@ // Copyright (c) Microsoft. All rights reserved. using System; -using System.Threading.Tasks; namespace Microsoft.Extensions.AI.Agents.Abstractions.UnitTests; @@ -16,14 +15,10 @@ public class AgentRunOptionsTests // Arrange var options = new AgentRunOptions { - OnIntermediateMessages = msg => Task.CompletedTask }; // Act var clone = new AgentRunOptions(options); - - // Assert - Assert.Equal(options.OnIntermediateMessages, clone.OnIntermediateMessages); } [Fact] diff --git a/dotnet/tests/Microsoft.Extensions.AI.Agents.Abstractions.UnitTests/AgentRunResponseTests.cs b/dotnet/tests/Microsoft.Extensions.AI.Agents.Abstractions.UnitTests/AgentRunResponseTests.cs new file mode 100644 index 0000000000..a1b674a64b --- /dev/null +++ b/dotnet/tests/Microsoft.Extensions.AI.Agents.Abstractions.UnitTests/AgentRunResponseTests.cs @@ -0,0 +1,268 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Collections.Generic; +using System.Linq; +using System.Text.Json; +using Microsoft.Extensions.AI.Agents.Abstractions.UnitTests.Models; + +namespace Microsoft.Extensions.AI.Agents.Abstractions.UnitTests; + +public class AgentRunResponseTests +{ + [Fact] + public void ConstructorWithNullEmptyArgsIsValid() + { + AgentRunResponse response; + + response = new(); + Assert.Empty(response.Messages); + Assert.Empty(response.Text); + + response = new((IList?)null); + Assert.Empty(response.Messages); + Assert.Empty(response.Text); + + Assert.Throws("message", () => new AgentRunResponse((ChatMessage)null!)); + } + + [Fact] + public void ConstructorWithMessagesRoundtrips() + { + AgentRunResponse response = new(); + Assert.NotNull(response.Messages); + Assert.Same(response.Messages, response.Messages); + + List messages = []; + response = new(messages); + Assert.Same(messages, response.Messages); + + messages = []; + Assert.NotSame(messages, response.Messages); + response.Messages = messages; + Assert.Same(messages, response.Messages); + } + + [Fact] + public void PropertiesRoundtrip() + { + AgentRunResponse response = new(); + + Assert.Null(response.AgentId); + response.AgentId = "agentId"; + Assert.Equal("agentId", response.AgentId); + + Assert.Null(response.ResponseId); + response.ResponseId = "id"; + Assert.Equal("id", response.ResponseId); + + Assert.Null(response.CreatedAt); + response.CreatedAt = new DateTimeOffset(2022, 1, 1, 0, 0, 0, TimeSpan.Zero); + Assert.Equal(new DateTimeOffset(2022, 1, 1, 0, 0, 0, TimeSpan.Zero), response.CreatedAt); + + Assert.Null(response.Usage); + UsageDetails usage = new(); + response.Usage = usage; + Assert.Same(usage, response.Usage); + + Assert.Null(response.RawRepresentation); + object raw = new(); + response.RawRepresentation = raw; + Assert.Same(raw, response.RawRepresentation); + + Assert.Null(response.AdditionalProperties); + AdditionalPropertiesDictionary additionalProps = []; + response.AdditionalProperties = additionalProps; + Assert.Same(additionalProps, response.AdditionalProperties); + } + + [Fact] + public void JsonSerializationRoundtrips() + { + AgentRunResponse original = new(new ChatMessage(ChatRole.Assistant, "the message")) + { + AgentId = "agentId", + ResponseId = "id", + CreatedAt = new DateTimeOffset(2022, 1, 1, 0, 0, 0, TimeSpan.Zero), + Usage = new UsageDetails(), + RawRepresentation = new(), + AdditionalProperties = new() { ["key"] = "value" }, + }; + + string json = JsonSerializer.Serialize(original, TestJsonSerializerContext.Default.AgentRunResponse); + + AgentRunResponse? result = JsonSerializer.Deserialize(json, TestJsonSerializerContext.Default.AgentRunResponse); + + Assert.NotNull(result); + Assert.Equal(ChatRole.Assistant, result.Messages.Single().Role); + Assert.Equal("the message", result.Messages.Single().Text); + + Assert.Equal("agentId", result.AgentId); + Assert.Equal("id", result.ResponseId); + Assert.Equal(new DateTimeOffset(2022, 1, 1, 0, 0, 0, TimeSpan.Zero), result.CreatedAt); + Assert.NotNull(result.Usage); + + Assert.NotNull(result.AdditionalProperties); + Assert.Single(result.AdditionalProperties); + Assert.True(result.AdditionalProperties.TryGetValue("key", out object? value)); + Assert.IsType(value); + Assert.Equal("value", ((JsonElement)value!).GetString()); + } + + [Fact] + public void ToStringOutputsText() + { + AgentRunResponse response = new(new ChatMessage(ChatRole.Assistant, $"This is a test.{Environment.NewLine}It's multiple lines.")); + + Assert.Equal(response.Text, response.ToString()); + } + + [Fact] + public void TextGetConcatenatesAllTextContent() + { + AgentRunResponse response = new( + [ + new ChatMessage( + ChatRole.Assistant, + [ + new DataContent("data:image/audio;base64,aGVsbG8="), + new DataContent("data:image/image;base64,aGVsbG8="), + new FunctionCallContent("callId1", "fc1"), + new TextContent("message1-text-1"), + new TextContent("message1-text-2"), + new FunctionResultContent("callId1", "result"), + ]), + new ChatMessage(ChatRole.Assistant, "message2") + ]); + + Assert.Equal($"message1-text-1message1-text-2{Environment.NewLine}message2", response.Text); + } + + [Fact] + public void TextGetReturnsEmptyStringWithNoMessages() + { + AgentRunResponse response = new(); + + Assert.Equal(string.Empty, response.Text); + } + + [Fact] + public void ToAgentRunResponseUpdatesProducesUpdates() + { + AgentRunResponse response = new(new ChatMessage(new ChatRole("customRole"), "Text") { MessageId = "someMessage" }) + { + AgentId = "agentId", + ResponseId = "12345", + CreatedAt = new DateTimeOffset(2024, 11, 10, 9, 20, 0, TimeSpan.Zero), + AdditionalProperties = new() { ["key1"] = "value1", ["key2"] = 42 }, + Usage = new UsageDetails + { + TotalTokenCount = 100 + }, + }; + + AgentRunResponseUpdate[] updates = response.ToAgentRunResponseUpdates(); + Assert.NotNull(updates); + Assert.Equal(2, updates.Length); + + AgentRunResponseUpdate update0 = updates[0]; + Assert.Equal("agentId", update0.AgentId); + Assert.Equal("12345", update0.ResponseId); + Assert.Equal("someMessage", update0.MessageId); + Assert.Equal(new DateTimeOffset(2024, 11, 10, 9, 20, 0, TimeSpan.Zero), update0.CreatedAt); + Assert.Equal("customRole", update0.Role?.Value); + Assert.Equal("Text", update0.Text); + + AgentRunResponseUpdate update1 = updates[1]; + Assert.Equal("value1", update1.AdditionalProperties?["key1"]); + Assert.Equal(42, update1.AdditionalProperties?["key2"]); + Assert.IsType(update1.Contents[0]); + UsageContent usageContent = (UsageContent)update1.Contents[0]; + Assert.Equal(100, usageContent.Details.TotalTokenCount); + } + + [Fact] + public void ParseAsStructuredOutputSuccess() + { + // Arrange. + var expectedResult = new Animal { Id = 1, FullName = "Tigger", Species = Species.Tiger }; + var response = new AgentRunResponse(new ChatMessage(ChatRole.Assistant, JsonSerializer.Serialize(expectedResult, TestJsonSerializerContext.Default.Animal))); + + // Act. + var animal = response.Deserialize(TestJsonSerializerContext.Default.Options); + + // Assert. + Assert.NotNull(animal); + Assert.Equal(expectedResult.Id, animal.Id); + Assert.Equal(expectedResult.FullName, animal.FullName); + Assert.Equal(expectedResult.Species, animal.Species); + } + + [Fact] + public void ParseAsStructuredOutputFailsWithEmptyString() + { + // Arrange. + var response = new AgentRunResponse(new ChatMessage(ChatRole.Assistant, string.Empty)); + + // Act & Assert. + var exception = Assert.Throws(() => response.Deserialize(TestJsonSerializerContext.Default.Options)); + Assert.Equal("The response did not contain JSON to be deserialized.", exception.Message); + } + + [Fact] + public void ParseAsStructuredOutputFailsWithInvalidJson() + { + // Arrange. + var response = new AgentRunResponse(new ChatMessage(ChatRole.Assistant, "invalid json")); + + // Act & Assert. + Assert.Throws(() => response.Deserialize(TestJsonSerializerContext.Default.Options)); + } + + [Fact] + public void ParseAsStructuredOutputFailsWithIncorrectTypedJson() + { + // Arrange. + var response = new AgentRunResponse(new ChatMessage(ChatRole.Assistant, "[]")); + + // Act & Assert. + Assert.Throws(() => response.Deserialize(TestJsonSerializerContext.Default.Options)); + } + + [Fact] + public void TryParseAsStructuredOutputSuccess() + { + // Arrange. + var expectedResult = new Animal { Id = 1, FullName = "Tigger", Species = Species.Tiger }; + var response = new AgentRunResponse(new ChatMessage(ChatRole.Assistant, JsonSerializer.Serialize(expectedResult, TestJsonSerializerContext.Default.Animal))); + + // Act. + var animal = response.Deserialize(TestJsonSerializerContext.Default.Options); + + // Assert. + Assert.NotNull(animal); + Assert.Equal(expectedResult.Id, animal.Id); + Assert.Equal(expectedResult.FullName, animal.FullName); + Assert.Equal(expectedResult.Species, animal.Species); + } + + [Fact] + public void TryParseAsStructuredOutputFailsWithEmptyText() + { + // Arrange. + var response = new AgentRunResponse(new ChatMessage(ChatRole.Assistant, string.Empty)); + + // Act & Assert. + Assert.False(response.TryDeserialize(TestJsonSerializerContext.Default.Options, out _)); + } + + [Fact] + public void TryParseAsStructuredOutputFailsWithIncorrectTypedJson() + { + // Arrange. + var response = new AgentRunResponse(new ChatMessage(ChatRole.Assistant, "[]")); + + // Act & Assert. + Assert.False(response.TryDeserialize(TestJsonSerializerContext.Default.Options, out _)); + } +} diff --git a/dotnet/tests/Microsoft.Extensions.AI.Agents.Abstractions.UnitTests/AgentRunResponseUpdateExtensionsTests.cs b/dotnet/tests/Microsoft.Extensions.AI.Agents.Abstractions.UnitTests/AgentRunResponseUpdateExtensionsTests.cs new file mode 100644 index 0000000000..7f91394417 --- /dev/null +++ b/dotnet/tests/Microsoft.Extensions.AI.Agents.Abstractions.UnitTests/AgentRunResponseUpdateExtensionsTests.cs @@ -0,0 +1,208 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Collections.Generic; +using System.Linq; +using System.Text; +using System.Threading.Tasks; + +namespace Microsoft.Extensions.AI.Agents.Abstractions.UnitTests; + +public class AgentRunResponseUpdateExtensionsTests +{ + [Fact] + public void ToAgentRunResponseWithInvalidArgsThrows() + { + Assert.Throws("updates", () => ((List)null!).ToAgentRunResponse()); + } + + [Theory] + [InlineData(false)] + [InlineData(true)] + public async Task ToAgentRunResponseSuccessfullyCreatesResponseAsync(bool useAsync) + { + AgentRunResponseUpdate[] updates = + [ + new(ChatRole.Assistant, "Hello") { ResponseId = "someResponse", MessageId = "12345", CreatedAt = new DateTimeOffset(1, 2, 3, 4, 5, 6, TimeSpan.Zero), AgentId = "agentId" }, + new(new("human"), ", ") { AuthorName = "Someone", AdditionalProperties = new() { ["a"] = "b" } }, + new(null, "world!") { CreatedAt = new DateTimeOffset(2, 2, 3, 4, 5, 6, TimeSpan.Zero), AdditionalProperties = new() { ["c"] = "d" } }, + + new() { Contents = [new UsageContent(new() { InputTokenCount = 1, OutputTokenCount = 2 })] }, + new() { Contents = [new UsageContent(new() { InputTokenCount = 4, OutputTokenCount = 5 })] }, + ]; + + AgentRunResponse response = useAsync ? + updates.ToAgentRunResponse() : + await YieldAsync(updates).ToAgentRunResponseAsync(); + Assert.NotNull(response); + + Assert.Equal("agentId", response.AgentId); + + Assert.NotNull(response.Usage); + Assert.Equal(5, response.Usage.InputTokenCount); + Assert.Equal(7, response.Usage.OutputTokenCount); + + Assert.Equal("someResponse", response.ResponseId); + Assert.Equal(new DateTimeOffset(2, 2, 3, 4, 5, 6, TimeSpan.Zero), response.CreatedAt); + + ChatMessage message = response.Messages.Single(); + Assert.Equal("12345", message.MessageId); + Assert.Equal(new ChatRole("human"), message.Role); + Assert.Equal("Someone", message.AuthorName); + Assert.Null(message.AdditionalProperties); + + Assert.NotNull(response.AdditionalProperties); + Assert.Equal(2, response.AdditionalProperties.Count); + Assert.Equal("b", response.AdditionalProperties["a"]); + Assert.Equal("d", response.AdditionalProperties["c"]); + + Assert.Equal("Hello, world!", response.Text); + } + + public static IEnumerable ToAgentRunResponseCoalescesVariousSequenceAndGapLengthsMemberData() + { + foreach (bool useAsync in new[] { false, true }) + { + for (int numSequences = 1; numSequences <= 3; numSequences++) + { + for (int sequenceLength = 1; sequenceLength <= 3; sequenceLength++) + { + for (int gapLength = 1; gapLength <= 3; gapLength++) + { + foreach (bool gapBeginningEnd in new[] { false, true }) + { + yield return new object[] { useAsync, numSequences, sequenceLength, gapLength, false }; + } + } + } + } + } + } + + [Theory] + [MemberData(nameof(ToAgentRunResponseCoalescesVariousSequenceAndGapLengthsMemberData))] + public async Task ToAgentRunResponseCoalescesVariousSequenceAndGapLengthsAsync(bool useAsync, int numSequences, int sequenceLength, int gapLength, bool gapBeginningEnd) + { + List updates = []; + + List expected = []; + + if (gapBeginningEnd) + { + AddGap(); + } + + for (int sequenceNum = 0; sequenceNum < numSequences; sequenceNum++) + { + StringBuilder sb = new(); + for (int i = 0; i < sequenceLength; i++) + { + string text = $"{(char)('A' + sequenceNum)}{i}"; + updates.Add(new(null, text)); + sb.Append(text); + } + + expected.Add(sb.ToString()); + + if (sequenceNum < numSequences - 1) + { + AddGap(); + } + } + + if (gapBeginningEnd) + { + AddGap(); + } + + void AddGap() + { + for (int i = 0; i < gapLength; i++) + { + updates.Add(new() { Contents = [new DataContent("data:image/png;base64,aGVsbG8=")] }); + } + } + + AgentRunResponse response = useAsync ? await YieldAsync(updates).ToAgentRunResponseAsync() : updates.ToAgentRunResponse(); + Assert.NotNull(response); + + ChatMessage message = response.Messages.Single(); + Assert.NotNull(message); + + Assert.Equal(expected.Count + (gapLength * ((numSequences - 1) + (gapBeginningEnd ? 2 : 0))), message.Contents.Count); + + TextContent[] contents = message.Contents.OfType().ToArray(); + Assert.Equal(expected.Count, contents.Length); + for (int i = 0; i < expected.Count; i++) + { + Assert.Equal(expected[i], contents[i].Text); + } + } + + [Theory] + [InlineData(false)] + [InlineData(true)] + public async Task ToAgentRunResponseCoalescesTextContentAndTextReasoningContentSeparatelyAsync(bool useAsync) + { + AgentRunResponseUpdate[] updates = + { + new(null, "A"), + new(null, "B"), + new(null, "C"), + new() { Contents = [new TextReasoningContent("D")] }, + new() { Contents = [new TextReasoningContent("E")] }, + new() { Contents = [new TextReasoningContent("F")] }, + new(null, "G"), + new(null, "H"), + new() { Contents = [new TextReasoningContent("I")] }, + new() { Contents = [new TextReasoningContent("J")] }, + new(null, "K"), + new() { Contents = [new TextReasoningContent("L")] }, + new(null, "M"), + new(null, "N"), + new() { Contents = [new TextReasoningContent("O")] }, + new() { Contents = [new TextReasoningContent("P")] }, + }; + + AgentRunResponse response = useAsync ? await YieldAsync(updates).ToAgentRunResponseAsync() : updates.ToAgentRunResponse(); + ChatMessage message = Assert.Single(response.Messages); + Assert.Equal(8, message.Contents.Count); + Assert.Equal("ABC", Assert.IsType(message.Contents[0]).Text); + Assert.Equal("DEF", Assert.IsType(message.Contents[1]).Text); + Assert.Equal("GH", Assert.IsType(message.Contents[2]).Text); + Assert.Equal("IJ", Assert.IsType(message.Contents[3]).Text); + Assert.Equal("K", Assert.IsType(message.Contents[4]).Text); + Assert.Equal("L", Assert.IsType(message.Contents[5]).Text); + Assert.Equal("MN", Assert.IsType(message.Contents[6]).Text); + Assert.Equal("OP", Assert.IsType(message.Contents[7]).Text); + } + + [Fact] + public async Task ToAgentRunResponseUsesContentExtractedFromContentsAsync() + { + AgentRunResponseUpdate[] updates = + { + new(null, "Hello, "), + new(null, "world!"), + new() { Contents = [new UsageContent(new() { TotalTokenCount = 42 })] }, + }; + + AgentRunResponse response = await YieldAsync(updates).ToAgentRunResponseAsync(); + + Assert.NotNull(response); + + Assert.NotNull(response.Usage); + Assert.Equal(42, response.Usage.TotalTokenCount); + + Assert.Equal("Hello, world!", Assert.IsType(Assert.Single(Assert.Single(response.Messages).Contents)).Text); + } + + private static async IAsyncEnumerable YieldAsync(IEnumerable updates) + { + foreach (AgentRunResponseUpdate update in updates) + { + await Task.Yield(); + yield return update; + } + } +} diff --git a/dotnet/tests/Microsoft.Extensions.AI.Agents.Abstractions.UnitTests/AgentRunResponseUpdatesTests.cs b/dotnet/tests/Microsoft.Extensions.AI.Agents.Abstractions.UnitTests/AgentRunResponseUpdatesTests.cs new file mode 100644 index 0000000000..cc6096a59a --- /dev/null +++ b/dotnet/tests/Microsoft.Extensions.AI.Agents.Abstractions.UnitTests/AgentRunResponseUpdatesTests.cs @@ -0,0 +1,161 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System; +using System.Collections.Generic; +using System.Text.Json; + +namespace Microsoft.Extensions.AI.Agents.Abstractions.UnitTests; + +public class AgentRunResponseUpdateTests +{ + [Fact] + public void ConstructorPropsDefaulted() + { + AgentRunResponseUpdate update = new(); + Assert.Null(update.AuthorName); + Assert.Null(update.Role); + Assert.Empty(update.Text); + Assert.Empty(update.Contents); + Assert.Null(update.RawRepresentation); + Assert.Null(update.AdditionalProperties); + Assert.Null(update.ResponseId); + Assert.Null(update.MessageId); + Assert.Null(update.CreatedAt); + Assert.Equal(string.Empty, update.ToString()); + } + + [Fact] + public void PropertiesRoundtrip() + { + AgentRunResponseUpdate update = new(); + + Assert.Null(update.AuthorName); + update.AuthorName = "author"; + Assert.Equal("author", update.AuthorName); + + Assert.Null(update.Role); + update.Role = ChatRole.Assistant; + Assert.Equal(ChatRole.Assistant, update.Role); + + Assert.Empty(update.Contents); + update.Contents.Add(new TextContent("text")); + Assert.Single(update.Contents); + Assert.Equal("text", update.Text); + Assert.Same(update.Contents, update.Contents); + IList newList = [new TextContent("text")]; + update.Contents = newList; + Assert.Same(newList, update.Contents); + update.Contents = null; + Assert.NotNull(update.Contents); + Assert.Empty(update.Contents); + + Assert.Empty(update.Text); + + Assert.Null(update.RawRepresentation); + object raw = new(); + update.RawRepresentation = raw; + Assert.Same(raw, update.RawRepresentation); + + Assert.Null(update.AdditionalProperties); + AdditionalPropertiesDictionary props = new() { ["key"] = "value" }; + update.AdditionalProperties = props; + Assert.Same(props, update.AdditionalProperties); + + Assert.Null(update.ResponseId); + update.ResponseId = "id"; + Assert.Equal("id", update.ResponseId); + + Assert.Null(update.MessageId); + update.MessageId = "messageid"; + Assert.Equal("messageid", update.MessageId); + + Assert.Null(update.CreatedAt); + update.CreatedAt = new DateTimeOffset(2022, 1, 1, 0, 0, 0, TimeSpan.Zero); + Assert.Equal(new DateTimeOffset(2022, 1, 1, 0, 0, 0, TimeSpan.Zero), update.CreatedAt); + } + + [Fact] + public void TextGetUsesAllTextContent() + { + AgentRunResponseUpdate update = new() + { + Role = ChatRole.User, + Contents = + [ + new DataContent("data:image/audio;base64,aGVsbG8="), + new DataContent("data:image/image;base64,aGVsbG8="), + new FunctionCallContent("callId1", "fc1"), + new TextContent("text-1"), + new TextContent("text-2"), + new FunctionResultContent("callId1", "result"), + ], + }; + + TextContent textContent = Assert.IsType(update.Contents[3]); + Assert.Equal("text-1", textContent.Text); + Assert.Equal("text-1text-2", update.Text); + Assert.Equal("text-1text-2", update.ToString()); + + ((TextContent)update.Contents[3]).Text = "text-3"; + Assert.Equal("text-3text-2", update.Text); + Assert.Same(textContent, update.Contents[3]); + Assert.Equal("text-3text-2", update.ToString()); + } + + [Fact] + public void JsonSerializationRoundtrips() + { + AgentRunResponseUpdate original = new() + { + AuthorName = "author", + Role = ChatRole.Assistant, + Contents = + [ + new TextContent("text-1"), + new DataContent("data:image/png;base64,aGVsbG8="), + new FunctionCallContent("callId1", "fc1"), + new DataContent("data"u8.ToArray(), "text/plain"), + new TextContent("text-2"), + ], + RawRepresentation = new object(), + ResponseId = "id", + MessageId = "messageid", + CreatedAt = new DateTimeOffset(2022, 1, 1, 0, 0, 0, TimeSpan.Zero), + AdditionalProperties = new() { ["key"] = "value" }, + }; + + string json = JsonSerializer.Serialize(original, TestJsonSerializerContext.Default.AgentRunResponseUpdate); + + AgentRunResponseUpdate? result = JsonSerializer.Deserialize(json, TestJsonSerializerContext.Default.AgentRunResponseUpdate); + + Assert.NotNull(result); + Assert.Equal(5, result.Contents.Count); + + Assert.IsType(result.Contents[0]); + Assert.Equal("text-1", ((TextContent)result.Contents[0]).Text); + + Assert.IsType(result.Contents[1]); + Assert.Equal("data:image/png;base64,aGVsbG8=", ((DataContent)result.Contents[1]).Uri); + + Assert.IsType(result.Contents[2]); + Assert.Equal("fc1", ((FunctionCallContent)result.Contents[2]).Name); + + Assert.IsType(result.Contents[3]); + Assert.Equal("data"u8.ToArray(), ((DataContent)result.Contents[3]).Data.ToArray()); + + Assert.IsType(result.Contents[4]); + Assert.Equal("text-2", ((TextContent)result.Contents[4]).Text); + + Assert.Equal("author", result.AuthorName); + Assert.Equal(ChatRole.Assistant, result.Role); + Assert.Equal("id", result.ResponseId); + Assert.Equal("messageid", result.MessageId); + Assert.Equal(new DateTimeOffset(2022, 1, 1, 0, 0, 0, TimeSpan.Zero), result.CreatedAt); + + Assert.NotNull(result.AdditionalProperties); + Assert.Single(result.AdditionalProperties); + Assert.True(result.AdditionalProperties.TryGetValue("key", out object? value)); + Assert.IsType(value); + Assert.Equal("value", ((JsonElement)value!).GetString()); + } +} diff --git a/dotnet/tests/Microsoft.Extensions.AI.Agents.Abstractions.UnitTests/AgentTests.cs b/dotnet/tests/Microsoft.Extensions.AI.Agents.Abstractions.UnitTests/AgentTests.cs index c676ebca56..3d80335938 100644 --- a/dotnet/tests/Microsoft.Extensions.AI.Agents.Abstractions.UnitTests/AgentTests.cs +++ b/dotnet/tests/Microsoft.Extensions.AI.Agents.Abstractions.UnitTests/AgentTests.cs @@ -17,8 +17,8 @@ public class AgentTests { private readonly Mock _agentMock; private readonly Mock _agentThreadMock; - private readonly ChatResponse _invokeResponse = new(); - private readonly List _invokeStreamingResponses = []; + private readonly AgentRunResponse _invokeResponse = new(); + private readonly List _invokeStreamingResponses = []; /// /// Initializes a new instance of the class. @@ -27,8 +27,8 @@ public class AgentTests { this._agentThreadMock = new Mock(MockBehavior.Strict); - this._invokeResponse = new ChatResponse(new ChatMessage(ChatRole.Assistant, "Hi")); - this._invokeStreamingResponses.Add(new ChatResponseUpdate(ChatRole.Assistant, "Hi")); + this._invokeResponse = new AgentRunResponse(new ChatMessage(ChatRole.Assistant, "Hi")); + this._invokeStreamingResponses.Add(new AgentRunResponseUpdate(ChatRole.Assistant, "Hi")); this._agentMock = new Mock() { CallBase = true }; this._agentMock @@ -282,12 +282,12 @@ public class AgentTests throw new NotImplementedException(); } - public override Task RunAsync(IReadOnlyCollection messages, AgentThread? thread = null, AgentRunOptions? options = null, CancellationToken cancellationToken = default) + public override Task RunAsync(IReadOnlyCollection messages, AgentThread? thread = null, AgentRunOptions? options = null, CancellationToken cancellationToken = default) { throw new System.NotImplementedException(); } - public override IAsyncEnumerable RunStreamingAsync(IReadOnlyCollection messages, AgentThread? thread = null, AgentRunOptions? options = null, CancellationToken cancellationToken = default) + public override IAsyncEnumerable RunStreamingAsync(IReadOnlyCollection messages, AgentThread? thread = null, AgentRunOptions? options = null, CancellationToken cancellationToken = default) { throw new System.NotImplementedException(); } diff --git a/dotnet/tests/Microsoft.Extensions.AI.Agents.Abstractions.UnitTests/Microsoft.Extensions.AI.Agents.Abstractions.UnitTests.csproj b/dotnet/tests/Microsoft.Extensions.AI.Agents.Abstractions.UnitTests/Microsoft.Extensions.AI.Agents.Abstractions.UnitTests.csproj index c1ff571b03..51859bfd8d 100644 --- a/dotnet/tests/Microsoft.Extensions.AI.Agents.Abstractions.UnitTests/Microsoft.Extensions.AI.Agents.Abstractions.UnitTests.csproj +++ b/dotnet/tests/Microsoft.Extensions.AI.Agents.Abstractions.UnitTests/Microsoft.Extensions.AI.Agents.Abstractions.UnitTests.csproj @@ -9,4 +9,8 @@ + + + + diff --git a/dotnet/tests/Microsoft.Extensions.AI.Agents.Abstractions.UnitTests/Models/Animal.cs b/dotnet/tests/Microsoft.Extensions.AI.Agents.Abstractions.UnitTests/Models/Animal.cs new file mode 100644 index 0000000000..1c061c4720 --- /dev/null +++ b/dotnet/tests/Microsoft.Extensions.AI.Agents.Abstractions.UnitTests/Models/Animal.cs @@ -0,0 +1,13 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System.ComponentModel; + +namespace Microsoft.Extensions.AI.Agents.Abstractions.UnitTests.Models; + +[Description("Some test description")] +internal sealed class Animal +{ + public int Id { get; set; } + public string? FullName { get; set; } + public Species Species { get; set; } +} diff --git a/dotnet/tests/Microsoft.Extensions.AI.Agents.Abstractions.UnitTests/Models/Species.cs b/dotnet/tests/Microsoft.Extensions.AI.Agents.Abstractions.UnitTests/Models/Species.cs new file mode 100644 index 0000000000..5c9a24ae8a --- /dev/null +++ b/dotnet/tests/Microsoft.Extensions.AI.Agents.Abstractions.UnitTests/Models/Species.cs @@ -0,0 +1,10 @@ +// Copyright (c) Microsoft. All rights reserved. + +namespace Microsoft.Extensions.AI.Agents.Abstractions.UnitTests.Models; + +internal enum Species +{ + Bear, + Tiger, + Walrus, +} diff --git a/dotnet/tests/Microsoft.Extensions.AI.Agents.Abstractions.UnitTests/TestJsonSerializerContext.cs b/dotnet/tests/Microsoft.Extensions.AI.Agents.Abstractions.UnitTests/TestJsonSerializerContext.cs new file mode 100644 index 0000000000..bf0a0ee62e --- /dev/null +++ b/dotnet/tests/Microsoft.Extensions.AI.Agents.Abstractions.UnitTests/TestJsonSerializerContext.cs @@ -0,0 +1,16 @@ +// Copyright (c) Microsoft. All rights reserved. + +using System.Text.Json.Serialization; +using Microsoft.Extensions.AI.Agents.Abstractions.UnitTests.Models; + +namespace Microsoft.Extensions.AI.Agents.Abstractions.UnitTests; + +[JsonSourceGenerationOptions( + PropertyNamingPolicy = JsonKnownNamingPolicy.CamelCase, + DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull, + UseStringEnumConverter = true)] +[JsonSerializable(typeof(AgentRunResponse))] +[JsonSerializable(typeof(AgentRunResponseUpdate))] +[JsonSerializable(typeof(AgentRunOptions))] +[JsonSerializable(typeof(Animal))] +internal sealed partial class TestJsonSerializerContext : JsonSerializerContext; diff --git a/dotnet/tests/Microsoft.Extensions.AI.Agents.UnitTests/ChatCompletion/ChatClientAgentExtensionsTests.cs b/dotnet/tests/Microsoft.Extensions.AI.Agents.UnitTests/ChatCompletion/ChatClientAgentExtensionsTests.cs index 8635958fa6..f98e5d14fc 100644 --- a/dotnet/tests/Microsoft.Extensions.AI.Agents.UnitTests/ChatCompletion/ChatClientAgentExtensionsTests.cs +++ b/dotnet/tests/Microsoft.Extensions.AI.Agents.UnitTests/ChatCompletion/ChatClientAgentExtensionsTests.cs @@ -441,7 +441,7 @@ public class ChatClientAgentExtensionsTests var messages = new List { new(ChatRole.User, "test message") }; // Act - var updates = new List(); + var updates = new List(); await foreach (var update in ChatClientAgentExtensions.RunStreamingAsync(agent, messages)) { updates.Add(update); @@ -522,7 +522,7 @@ public class ChatClientAgentExtensionsTests var messages = new List { new(ChatRole.User, "test") }; // Act - var updates = new List(); + var updates = new List(); await foreach (var update in ChatClientAgentExtensions.RunStreamingAsync(agent, messages, chatOptions: chatOptions)) { updates.Add(update); @@ -559,7 +559,7 @@ public class ChatClientAgentExtensionsTests var thread = agent.GetNewThread(); // Act - var updates = new List(); + var updates = new List(); await foreach (var update in ChatClientAgentExtensions.RunStreamingAsync(agent, messages, thread: thread)) { updates.Add(update); @@ -633,7 +633,7 @@ public class ChatClientAgentExtensionsTests const string TestPrompt = "test prompt"; // Act - var updates = new List(); + var updates = new List(); await foreach (var update in ChatClientAgentExtensions.RunStreamingAsync(agent, TestPrompt)) { updates.Add(update); @@ -738,7 +738,7 @@ public class ChatClientAgentExtensionsTests const string TestPrompt = "test prompt"; // Act - var updates = new List(); + var updates = new List(); await foreach (var update in agent.RunStreamingAsync(TestPrompt)) { updates.Add(update); @@ -771,7 +771,7 @@ public class ChatClientAgentExtensionsTests const string TestPrompt = "test prompt"; // Act - var updates = new List(); + var updates = new List(); await foreach (var update in agent.RunStreamingAsync(TestPrompt, chatOptions: chatOptions)) { updates.Add(update); @@ -808,7 +808,7 @@ public class ChatClientAgentExtensionsTests var thread = agent.GetNewThread(); // Act - var updates = new List(); + var updates = new List(); await foreach (var update in agent.RunStreamingAsync(TestPrompt, thread: thread)) { updates.Add(update); diff --git a/dotnet/tests/Microsoft.Extensions.AI.Agents.UnitTests/ChatCompletion/ChatClientAgentRunOptionsTests.cs b/dotnet/tests/Microsoft.Extensions.AI.Agents.UnitTests/ChatCompletion/ChatClientAgentRunOptionsTests.cs index daec315bc8..6639e05341 100644 --- a/dotnet/tests/Microsoft.Extensions.AI.Agents.UnitTests/ChatCompletion/ChatClientAgentRunOptionsTests.cs +++ b/dotnet/tests/Microsoft.Extensions.AI.Agents.UnitTests/ChatCompletion/ChatClientAgentRunOptionsTests.cs @@ -1,7 +1,5 @@ // Copyright (c) Microsoft. All rights reserved. -using System.Threading.Tasks; - namespace Microsoft.Extensions.AI.Agents.UnitTests.ChatCompletion; public class ChatClientAgentRunOptionsTests @@ -16,7 +14,6 @@ public class ChatClientAgentRunOptionsTests var runOptions = new ChatClientAgentRunOptions(); // Assert - Assert.Null(runOptions.OnIntermediateMessages); Assert.Null(runOptions.ChatOptions); } @@ -33,7 +30,6 @@ public class ChatClientAgentRunOptionsTests var runOptions = new ChatClientAgentRunOptions(null, chatOptions); // Assert - Assert.Null(runOptions.OnIntermediateMessages); Assert.Same(chatOptions, runOptions.ChatOptions); } @@ -46,7 +42,6 @@ public class ChatClientAgentRunOptionsTests // Arrange var sourceRunOptions = new AgentRunOptions { - OnIntermediateMessages = messages => Task.CompletedTask }; var chatOptions = new ChatOptions { MaxOutputTokens = 200 }; @@ -54,7 +49,6 @@ public class ChatClientAgentRunOptionsTests var runOptions = new ChatClientAgentRunOptions(sourceRunOptions, chatOptions); // Assert - Assert.Same(sourceRunOptions.OnIntermediateMessages, runOptions.OnIntermediateMessages); Assert.Same(chatOptions, runOptions.ChatOptions); } @@ -67,14 +61,12 @@ public class ChatClientAgentRunOptionsTests // Arrange var sourceRunOptions = new AgentRunOptions { - OnIntermediateMessages = messages => Task.CompletedTask }; // Act var runOptions = new ChatClientAgentRunOptions(sourceRunOptions, null); // Assert - Assert.Same(sourceRunOptions.OnIntermediateMessages, runOptions.OnIntermediateMessages); Assert.Null(runOptions.ChatOptions); } diff --git a/dotnet/tests/Microsoft.Extensions.AI.Agents.UnitTests/ChatCompletion/ChatClientAgentTests.cs b/dotnet/tests/Microsoft.Extensions.AI.Agents.UnitTests/ChatCompletion/ChatClientAgentTests.cs index 485338c72e..de0ce57fd0 100644 --- a/dotnet/tests/Microsoft.Extensions.AI.Agents.UnitTests/ChatCompletion/ChatClientAgentTests.cs +++ b/dotnet/tests/Microsoft.Extensions.AI.Agents.UnitTests/ChatCompletion/ChatClientAgentTests.cs @@ -60,7 +60,7 @@ public class ChatClientAgentTests }); // Act - ChatResponse result = await agent.RunAsync([new(ChatRole.User, "Where are you?")]); + var result = await agent.RunAsync([new(ChatRole.User, "Where are you?")]); // Assert Assert.Single(result.Messages); @@ -183,47 +183,6 @@ public class ChatClientAgentTests Assert.Contains(capturedMessages, m => m.Text == "test" && m.Role == ChatRole.User); } - /// - /// Verify that RunAsync calls OnIntermediateMessage callback for each response message. - /// - [Fact] - public async Task RunAsyncCallsOnIntermediateMessageForEachResponseMessageAsync() - { - // Arrange - Mock mockService = new(); - var responseMessages = new[] - { - new ChatMessage(ChatRole.Assistant, "first response"), - new ChatMessage(ChatRole.Assistant, "second response") - }; - mockService.Setup( - s => s.GetResponseAsync( - It.IsAny>(), - It.IsAny(), - It.IsAny())).ReturnsAsync(new ChatResponse(responseMessages)); - - ChatClientAgent agent = new(mockService.Object, new() { Instructions = "test instructions", Name = "TestAgent" }); - - var callbackMessages = new List(); - var runOptions = new AgentRunOptions - { - OnIntermediateMessages = messages => - { - callbackMessages.AddRange(messages); - return Task.CompletedTask; - } - }; - - // Act - await agent.RunAsync([new(ChatRole.User, "test")], options: runOptions); - - // Assert - Assert.Equal(2, callbackMessages.Count); - Assert.Equal("first response", callbackMessages[0].Text); - Assert.Equal("second response", callbackMessages[1].Text); - Assert.All(callbackMessages, msg => Assert.Equal("TestAgent", msg.AuthorName)); - } - /// /// Verify that RunAsync sets AuthorName on all response messages. /// @@ -1054,7 +1013,7 @@ public class ChatClientAgentTests }); // Act - ChatResponseUpdate[] result = await agent.RunStreamingAsync([new ChatMessage(ChatRole.User, "Hello")]).ToArrayAsync(); + var result = await agent.RunStreamingAsync([new ChatMessage(ChatRole.User, "Hello")]).ToArrayAsync(); // Assert Assert.Equal(2, result.Length); diff --git a/dotnet/tests/Microsoft.Extensions.AI.Agents.UnitTests/ChatCompletion/ChatClientAgentThreadTests.cs b/dotnet/tests/Microsoft.Extensions.AI.Agents.UnitTests/ChatCompletion/ChatClientAgentThreadTests.cs index c01547fa58..e59311f83e 100644 --- a/dotnet/tests/Microsoft.Extensions.AI.Agents.UnitTests/ChatCompletion/ChatClientAgentThreadTests.cs +++ b/dotnet/tests/Microsoft.Extensions.AI.Agents.UnitTests/ChatCompletion/ChatClientAgentThreadTests.cs @@ -358,7 +358,7 @@ public class ChatClientAgentThreadTests var thread = agent.GetNewThread(); // Act - Run the agent with streaming to populate the thread with messages - var streamingResults = new List(); + var streamingResults = new List(); await foreach (var update in agent.RunStreamingAsync([userMessage], thread)) { streamingResults.Add(update); @@ -419,13 +419,13 @@ public class ChatClientAgentThreadTests var thread = agent.GetNewThread(); // Act - Make two streaming calls - var firstStreamingResults = new List(); + var firstStreamingResults = new List(); await foreach (var update in agent.RunStreamingAsync([firstUserMessage], thread)) { firstStreamingResults.Add(update); } - var secondStreamingResults = new List(); + var secondStreamingResults = new List(); await foreach (var update in agent.RunStreamingAsync([secondUserMessage], thread)) { secondStreamingResults.Add(update); @@ -493,7 +493,7 @@ public class ChatClientAgentThreadTests await agent.RunAsync([initialUserMessage], thread); // Then make a streaming call - var streamingResults = new List(); + var streamingResults = new List(); await foreach (var update in agent.RunStreamingAsync([newUserMessage], thread)) { streamingResults.Add(update); @@ -542,7 +542,7 @@ public class ChatClientAgentThreadTests var thread = agent.GetNewThread(); // Act - Run the agent with streaming that returns no updates - var streamingResults = new List(); + var streamingResults = new List(); await foreach (var update in agent.RunStreamingAsync([userMessage], thread)) { streamingResults.Add(update); @@ -597,7 +597,7 @@ public class ChatClientAgentThreadTests var thread = agent.GetNewThread(); // Act - Run the agent with streaming that returns multiple updates - var streamingResults = new List(); + var streamingResults = new List(); await foreach (var update in agent.RunStreamingAsync([userMessage], thread)) { streamingResults.Add(update); @@ -692,7 +692,7 @@ public class ChatClientAgentThreadTests var thread = agent.GetNewThread(); // Act & Assert - Verify that streaming throws an exception after some updates - var streamingResults = new List(); + var streamingResults = new List(); await Assert.ThrowsAsync(async () => { await foreach (var update in agent.RunStreamingAsync([userMessage], thread))