.NET Workflows - WIP Declarative action update (#1761)

* WIP

* Fixed build errors (#1638)

Comment and nullable type alignment

* Sync to SDK update

* Checkpoint

* Checkpoint: Tests passing

* Checkpoint: EndWorkflow

* Add trace

* .NET: Azure.AI.Agents Package Split + Initial Extensions (#1657)

* Move packages

* Update nuget.config

* Address Xmldoc

* Remove format from branches checks

* Address Xmldocs

* Add more details to the implementation

* Moving Agent logic to ChatClient

* Adding Name and Id overrides to AzureAIAgent

* Updating extensions

* Add GetAiAgent extensions

* Adding support for version as name can conflict 409 using the Agents API with same name

* Addressing more updates to the extensions

* More improvements

* Remove debugging code from sample

* Address copilot feedback

* Apply suggestions from co-pilot code review

* Checkpoint

* Update Directory.Packages.props

Fix package version rollback:

Azure.AI.Agents.Persistent (beta-6 => beta-7)

* Add project reference

* .NET: Add comprehensive unit tests for Microsoft.Agents.AI.AzureAIAgents extension methods (#1786)

* Initial plan

* Add comprehensive unit test project for Microsoft.Agents.AI.AzureAIAgents

Co-authored-by: rogerbarreto <19890735+rogerbarreto@users.noreply.github.com>

* Add README documenting test project and package dependency requirements

Co-authored-by: rogerbarreto <19890735+rogerbarreto@users.noreply.github.com>

* Fix documentation URL to use learn.microsoft.com

Co-authored-by: rogerbarreto <19890735+rogerbarreto@users.noreply.github.com>

* Bump back AAAP 1.2.0-beta.7

* Address AI generated UT's

* Remove UT Readme

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: rogerbarreto <19890735+rogerbarreto@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* .NET: Change model to be required just for prompt agent definition specific extensions (#1812)

* Remove unneeded model from extensions

* Add noop justification

* Update Package Nameing: V1 -> AzureAI.Persistent / V2 -> AzureAI (#1829)

* Checkpoint for merge

* No build errors

* .NET: Update Extensions for Strict Agent Definitions + Improvements (#1892)

* Update Package Nameing: V1 -> AzureAI.Persistent / V2 -> AzureAI

* Update agents and extensions to comply with strict agent definitions

* More static updates

* Address UT, and ResponseTool support

* Improving reusability extensions

* Addressing ResponseTools Unit Tests and extension setup

* Adapted workaround on breaking AAA with OpenAI 2.6.0

* Small updates

* Remove strictness when retrieving agents, improved XmlDocs

* Improve sample comments

* Update dotnet/tests/Microsoft.Agents.AI.AzureAI.UnitTests/AgentsClientExtensionsTests.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Apply suggestion from @Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Apply suggestion from @Copilot

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Address PR comments

* Address UT failing

* Address Copilot feedback

* Address Copilot feedback

* Address comment typo

* Address PR feedback

* Address typo

* Add missing Extensions with ChatClientAgentOptions

* Address comments

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Updated package version (#1897)

* Version update (#1901)

* Checkpoint

* Updated package version (#1906)

* Checkpoint

* Checkpoint

* Checkpoint

* Align with azure ai agent

* Update dotnet/samples/GettingStarted/Workflows/Declarative/StudentTeacher/Program.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update dotnet/samples/GettingStarted/Workflows/Declarative/MCPToolApproval/Program.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update dotnet/samples/GettingStarted/Workflows/Declarative/DeepResearch/Program.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Refactored external input

* Update dotnet/samples/GettingStarted/Workflows/Declarative/MCPToolApproval/Program.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Agent tools patch

* Demos validated

* Checkpoint

* Hygiene

* Checkpoint - Samples

* Update dotnet/samples/GettingStarted/Workflows/Declarative/StudentTeacher/Program.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update dotnet/samples/GettingStarted/Workflows/Declarative/StudentTeacher/Program.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Checkpoint

* Checkpoint - Deep Research

* Update baseline

* Update

* Typo

* Checkpoint

* Typos

* Sample cleanup

* Update dotnet/src/Microsoft.Agents.AI.Workflows.Declarative/AzureAgentProvider.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update dotnet/src/Microsoft.Agents.AI.AzureAI/AgentsClientExtensions.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update dotnet/samples/GettingStarted/Workflows/Declarative/FunctionTools/Program.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update dotnet/samples/GettingStarted/Workflows/Declarative/StudentTeacher/Program.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update dotnet/samples/GettingStarted/Workflows/Declarative/ToolApproval/Program.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update dotnet/samples/GettingStarted/Workflows/Declarative/DeepResearch/Program.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Typo

* Typo

* Fix input loop

* Sample - Function Calling / External Input

* Typo

* Finessed

* Checkpoint

* Fix feed

* Checkpoint - so close

* Ding dong!

* "there" ***

* Fixup comments

* Fix sample

* Code analysis

* Header

* Typo (variableName)

* Remove dead code

* Skip test (agent api ratchet)

* Comment

* Update dotnet/samples/GettingStarted/Workflows/Declarative/StudentTeacher/Program.cs

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Typo

---------

Co-authored-by: Roger Barreto <19890735+rogerbarreto@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Dmytro Struk <13853051+dmytrostruk@users.noreply.github.com>
This commit is contained in:
Chris
2025-11-07 11:37:21 -08:00
committed by GitHub
Unverified
parent 7c3d4fcf30
commit 2b869c2396
125 changed files with 3192 additions and 3712 deletions
@@ -0,0 +1,52 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Threading.Tasks;
using Azure.AI.Agents;
using Microsoft.Extensions.Configuration;
namespace Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests.Agents;
internal abstract class AgentProvider(IConfiguration configuration)
{
public static class Names
{
public const string FunctionTool = "FUNCTIONTOOL";
public const string Marketing = "MARKETING";
public const string MathChat = "MATHCHAT";
}
public static class Settings
{
public const string FoundryEndpoint = "FOUNDRY_PROJECT_ENDPOINT";
public const string FoundryModelMini = "FOUNDRY_MODEL_DEPLOYMENT_NAME";
public const string FoundryModelFull = "FOUNDRY_MEDIA_DEPLOYMENT_NAME";
public const string FoundryGroundingTool = "FOUNDRY_CONNECTION_GROUNDING_TOOL";
}
public static AgentProvider Create(IConfiguration configuration, string providerType) =>
providerType.ToUpperInvariant() switch
{
Names.FunctionTool => new FunctionToolAgentProvider(configuration),
Names.Marketing => new MarketingAgentProvider(configuration),
Names.MathChat => new MathChatAgentProvider(configuration),
_ => new TestAgentProvider(configuration),
};
public async ValueTask CreateAgentsAsync()
{
Uri foundryEndpoint = new(this.GetSetting(Settings.FoundryEndpoint));
await foreach (AgentVersion agent in this.CreateAgentsAsync(foundryEndpoint))
{
Console.WriteLine($"Created agent: {agent.Name}:{agent.Version})");
}
}
protected abstract IAsyncEnumerable<AgentVersion> CreateAgentsAsync(Uri foundryEndpoint);
protected string GetSetting(string settingName) =>
configuration[settingName] ??
throw new InvalidOperationException($"Undefined configuration setting: {settingName}");
}
@@ -0,0 +1,55 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using Azure.AI.Agents;
using Azure.Identity;
using Microsoft.Extensions.AI;
using Microsoft.Extensions.Configuration;
using OpenAI.Responses;
using Shared.Foundry;
namespace Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests.Agents;
internal sealed class FunctionToolAgentProvider(IConfiguration configuration) : AgentProvider(configuration)
{
protected override async IAsyncEnumerable<AgentVersion> CreateAgentsAsync(Uri foundryEndpoint)
{
MenuPlugin menuPlugin = new();
AIFunction[] functions =
[
AIFunctionFactory.Create(menuPlugin.GetMenu),
AIFunctionFactory.Create(menuPlugin.GetSpecials),
AIFunctionFactory.Create(menuPlugin.GetItemPrice),
];
AgentsClient agentsClient = new(foundryEndpoint, new AzureCliCredential());
yield return
await agentsClient.CreateAgentAsync(
agentName: "MenuAgent",
agentDefinition: this.DefineMenuAgent(functions),
agentDescription: "Provides information about the restaurant menu");
}
private PromptAgentDefinition DefineMenuAgent(AIFunction[] functions)
{
PromptAgentDefinition agentDefinition =
new(this.GetSetting(Settings.FoundryModelMini))
{
Instructions =
"""
Answer the users questions on the menu.
For questions or input that do not require searching the documentation, inform the
user that you can only answer questions what's on the menu.
"""
};
foreach (AIFunction function in functions)
{
agentDefinition.Tools.Add(function.AsOpenAIResponseTool());
}
return agentDefinition;
}
}
@@ -0,0 +1,75 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using Azure.AI.Agents;
using Azure.Identity;
using Microsoft.Extensions.Configuration;
using Shared.Foundry;
namespace Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests.Agents;
internal sealed class MarketingAgentProvider(IConfiguration configuration) : AgentProvider(configuration)
{
protected override async IAsyncEnumerable<AgentVersion> CreateAgentsAsync(Uri foundryEndpoint)
{
AgentsClient agentsClient = new(foundryEndpoint, new AzureCliCredential());
yield return
await agentsClient.CreateAgentAsync(
agentName: "AnalystAgent",
agentDefinition: this.DefineAnalystAgent(),
agentDescription: "Analyst agent for Marketing workflow");
yield return
await agentsClient.CreateAgentAsync(
agentName: "WriterAgent",
agentDefinition: this.DefineWriterAgent(),
agentDescription: "Writer agent for Marketing workflow");
yield return
await agentsClient.CreateAgentAsync(
agentName: "EditorAgent",
agentDefinition: this.DefineEditorAgent(),
agentDescription: "Editor agent for Marketing workflow");
}
private PromptAgentDefinition DefineAnalystAgent() =>
new(this.GetSetting(Settings.FoundryModelFull))
{
Instructions =
"""
You are a marketing analyst. Given a product description, identify:
- Key features
- Target audience
- Unique selling points
""",
Tools =
{
//AgentTool.CreateBingGroundingTool( // TODO: Use Bing Grounding when available
// new BingGroundingSearchToolParameters(
// [new BingGroundingSearchConfiguration(this.GetSetting(Settings.FoundryGroundingTool))]))
}
};
private PromptAgentDefinition DefineWriterAgent() =>
new(this.GetSetting(Settings.FoundryModelFull))
{
Instructions =
"""
You are a marketing copywriter. Given a block of text describing features, audience, and USPs,
compose a compelling marketing copy (like a newsletter section) that highlights these points.
Output should be short (around 150 words), output just the copy as a single text block.
"""
};
private PromptAgentDefinition DefineEditorAgent() =>
new(this.GetSetting(Settings.FoundryModelFull))
{
Instructions =
"""
You are an editor. Given the draft copy, correct grammar, improve clarity, ensure consistent tone,
give format and make it polished. Output the final improved copy as a single text block.
"""
};
}
@@ -0,0 +1,54 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using Azure.AI.Agents;
using Azure.Identity;
using Microsoft.Extensions.Configuration;
using Shared.Foundry;
namespace Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests.Agents;
internal sealed class MathChatAgentProvider(IConfiguration configuration) : AgentProvider(configuration)
{
protected override async IAsyncEnumerable<AgentVersion> CreateAgentsAsync(Uri foundryEndpoint)
{
AgentsClient agentsClient = new(foundryEndpoint, new AzureCliCredential());
yield return
await agentsClient.CreateAgentAsync(
agentName: "StudentAgent",
agentDefinition: this.DefineStudentAgent(),
agentDescription: "Student agent for MathChat workflow");
yield return
await agentsClient.CreateAgentAsync(
agentName: "TeacherAgent",
agentDefinition: this.DefineTeacherAgent(),
agentDescription: "Teacher agent for MathChat workflow");
}
private PromptAgentDefinition DefineStudentAgent() =>
new(this.GetSetting(Settings.FoundryModelMini))
{
Instructions =
"""
Your job is help a math teacher practice teaching by making intentional mistakes.
You attempt to solve the given math problem, but with intentional mistakes so the teacher can help.
Always incorporate the teacher's advice to fix your next response.
You have the math-skills of a 6th grader.
"""
};
private PromptAgentDefinition DefineTeacherAgent() =>
new(this.GetSetting(Settings.FoundryModelMini))
{
Instructions =
"""
Review and coach the student's approach to solving the given math problem.
Don't repeat the solution or try and solve it.
If the student has demonstrated comprehension and responded to all of your feedback,
give the student your congratulations by using the word "congratulations".
"""
};
}
@@ -5,32 +5,33 @@ using System.Collections.Generic;
using System.ComponentModel;
using System.Linq;
using Microsoft.Extensions.AI;
using Microsoft.SemanticKernel;
namespace Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests.Agents;
#pragma warning disable CA1822
public sealed class MenuPlugin
{
public IEnumerable<AIFunction> GetTools()
{
yield return AIFunctionFactory.Create(this.GetMenu, name: $"{nameof(MenuPlugin)}_{nameof(GetMenu)}");
yield return AIFunctionFactory.Create(this.GetSpecials, name: $"{nameof(MenuPlugin)}_{nameof(GetSpecials)}");
yield return AIFunctionFactory.Create(this.GetItemPrice, name: $"{nameof(MenuPlugin)}_{nameof(GetItemPrice)}");
yield return AIFunctionFactory.Create(this.GetMenu);
yield return AIFunctionFactory.Create(this.GetSpecials);
yield return AIFunctionFactory.Create(this.GetItemPrice);
}
[KernelFunction, Description("Provides a list items on the menu.")]
[Description("Provides a list items on the menu.")]
public MenuItem[] GetMenu()
{
return s_menuItems;
}
[KernelFunction, Description("Provides a list of specials from the menu.")]
[Description("Provides a list of specials from the menu.")]
public MenuItem[] GetSpecials()
{
return [.. s_menuItems.Where(i => i.IsSpecial)];
}
[KernelFunction, Description("Provides the price of the requested menu item.")]
[Description("Provides the price of the requested menu item.")]
public float? GetItemPrice(
[Description("The name of the menu item.")]
string name)
@@ -1,5 +0,0 @@
type: foundry_agent
name: BasicAgent
description: Basic agent for integration tests
model:
id: ${FOUNDRY_MEDIA_DEPLOYMENT_NAME}
@@ -0,0 +1,27 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using Azure.AI.Agents;
using Azure.Identity;
using Microsoft.Extensions.Configuration;
using Shared.Foundry;
namespace Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests.Agents;
internal sealed class TestAgentProvider(IConfiguration configuration) : AgentProvider(configuration)
{
protected override async IAsyncEnumerable<AgentVersion> CreateAgentsAsync(Uri foundryEndpoint)
{
AgentsClient agentsClient = new(foundryEndpoint, new AzureCliCredential());
yield return
await agentsClient.CreateAgentAsync(
agentName: "TestAgent",
agentDefinition: this.DefineMenuAgent(),
agentDescription: "Provides information about the restaurant menu");
}
private PromptAgentDefinition DefineMenuAgent() =>
new(this.GetSetting(Settings.FoundryModelFull));
}
@@ -1,15 +0,0 @@
type: foundry_agent
name: ToolAgent
description: Agent with a function tool defined.
model:
id: ${FOUNDRY_MODEL_DEPLOYMENT_NAME}
tools:
- id: MenuPlugin_GetMenu
type: function
description: Provides a list items on the menu.
- id: MenuPlugin_GetSpecials
type: function
description: Provides a list of specials from the menu.
- id: MenuPlugin_GetItemPrice
type: function
description: Provides the price of the requested menu item.
@@ -1,10 +1,7 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Linq;
using System.Threading.Tasks;
using Azure;
using Azure.AI.Agents.Persistent;
using Azure.Identity;
using Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests.Framework;
using Microsoft.Extensions.AI;
@@ -18,7 +15,7 @@ public sealed class AzureAgentProviderTest(ITestOutputHelper output) : Integrati
public async Task ConversationTestAsync()
{
// Arrange
AzureAgentProvider provider = new(this.FoundryConfiguration.Endpoint, new AzureCliCredential());
AzureAgentProvider provider = new(this.TestEndpoint, new AzureCliCredential());
// Act
string conversationId = await provider.CreateConversationAsync();
// Assert
@@ -43,35 +40,4 @@ public sealed class AzureAgentProviderTest(ITestOutputHelper output) : Integrati
Assert.NotNull(message);
Assert.Equal(messages[3].Text, message.Text);
}
[Fact]
public async Task GetAgentTestAsync()
{
// Arrange
AzureAgentProvider provider = new(this.FoundryConfiguration.Endpoint, new AzureCliCredential());
string agentName = $"TestAgent-{DateTime.UtcNow:yyMMdd-HHmmss-fff}";
string agent1Id = await this.CreateAgentAsync();
string agent2Id = await this.CreateAgentAsync(agentName);
// Act
AIAgent agent1 = await provider.GetAgentAsync(agent1Id);
// Assert
Assert.Equal(agent1Id, agent1.Id);
// Act
AIAgent agent2 = await provider.GetAgentAsync(agent2Id);
// Assert
Assert.Equal(agent2Id, agent2.Id);
// Act & Assert
await Assert.ThrowsAsync<RequestFailedException>(() => provider.GetAgentAsync(agentName));
}
private async ValueTask<string> CreateAgentAsync(string? name = null)
{
PersistentAgentsClient client = new(this.FoundryConfiguration.Endpoint, new AzureCliCredential());
PersistentAgent agent = await client.Administration.CreateAgentAsync(this.FoundryConfiguration.DeploymentName, name: name);
return agent.Id;
}
}
@@ -4,6 +4,7 @@ using System;
using System.IO;
using System.Linq;
using System.Threading.Tasks;
using Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests.Agents;
using Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests.Framework;
using Xunit.Abstractions;
@@ -22,7 +23,7 @@ public sealed class DeclarativeWorkflowTest(ITestOutputHelper output) : Workflow
[InlineData("ConversationMessages.yaml", "ConversationMessages.json")]
[InlineData("ConversationMessages.yaml", "ConversationMessages.json", true)]
public Task ValidateCaseAsync(string workflowFileName, string testcaseFileName, bool externalConveration = false) =>
this.RunWorkflowAsync(Path.Combine(Environment.CurrentDirectory, "Workflows", workflowFileName), testcaseFileName, externalConveration);
this.RunWorkflowAsync(GetWorkflowPath(workflowFileName, isSample: false), testcaseFileName, externalConveration);
[Theory]
[InlineData("Marketing.yaml", "Marketing.json")]
@@ -30,14 +31,24 @@ public sealed class DeclarativeWorkflowTest(ITestOutputHelper output) : Workflow
[InlineData("MathChat.yaml", "MathChat.json", true)]
[InlineData("DeepResearch.yaml", "DeepResearch.json", Skip = "Long running")]
public Task ValidateScenarioAsync(string workflowFileName, string testcaseFileName, bool externalConveration = false) =>
this.RunWorkflowAsync(Path.Combine(GetRepoFolder(), "workflow-samples", workflowFileName), testcaseFileName, externalConveration);
this.RunWorkflowAsync(GetWorkflowPath(workflowFileName, isSample: true), testcaseFileName, externalConveration);
[Fact]
public Task ValidateMultiTurnAsync() =>
this.RunWorkflowAsync(Path.Combine(GetRepoFolder(), "workflow-samples", "HumanInLoop.yaml"), "HumanInLoop.json", useJsonCheckpoint: true);
[Theory]
[InlineData("ConfirmInput.yaml", "ConfirmInput.json", true)]
[InlineData("RequestExternalInput.yaml", "RequestExternalInput.json", false)]
public Task ValidateMultiTurnAsync(string workflowFileName, string testcaseFileName, bool isSample) =>
this.RunWorkflowAsync(GetWorkflowPath(workflowFileName, isSample), testcaseFileName, useJsonCheckpoint: true);
private static string GetWorkflowPath(string workflowFileName, bool isSample) =>
isSample
? Path.Combine(GetRepoFolder(), "workflow-samples", workflowFileName)
: Path.Combine(Environment.CurrentDirectory, "Workflows", workflowFileName);
protected override async Task RunAndVerifyAsync<TInput>(Testcase testcase, string workflowPath, DeclarativeWorkflowOptions workflowOptions, TInput input, bool useJsonCheckpoint)
{
AgentProvider agentProvider = AgentProvider.Create(this.Configuration, Path.GetFileNameWithoutExtension(workflowPath));
await agentProvider.CreateAgentsAsync().ConfigureAwait(false);
Workflow workflow = DeclarativeWorkflowBuilder.Build<TInput>(workflowPath, workflowOptions);
WorkflowHarness harness = new(workflow, runId: Path.GetFileNameWithoutExtension(workflowPath));
@@ -1,90 +0,0 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Frozen;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Azure.AI.Agents.Persistent;
using Azure.AI.Projects;
using Azure.Identity;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.Agents;
using Microsoft.SemanticKernel.Agents.AzureAI;
using Shared.IntegrationTests;
namespace Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests.Framework;
internal static class AgentFactory
{
private static readonly Dictionary<string, string> s_agentDefinitions =
new()
{
["FOUNDRY_AGENT_TEST"] = "TestAgent.yaml",
["FOUNDRY_AGENT_TOOL"] = "ToolAgent.yaml",
["FOUNDRY_AGENT_ANSWER"] = "QuestionAgent.yaml",
["FOUNDRY_AGENT_STUDENT"] = "StudentAgent.yaml",
["FOUNDRY_AGENT_TEACHER"] = "TeacherAgent.yaml",
["FOUNDRY_AGENT_RESEARCHANALYST"] = "AnalystAgent.yaml",
["FOUNDRY_AGENT_RESEARCHCODER"] = "CoderAgent.yaml",
["FOUNDRY_AGENT_RESEARCHMANAGER"] = "ManagerAgent.yaml",
["FOUNDRY_AGENT_RESEARCHWEATHER"] = "WeatherAgent.yaml",
["FOUNDRY_AGENT_RESEARCHWEB"] = "WebAgent.yaml",
};
private static FrozenDictionary<string, string?>? s_agentMap;
public static async Task<FrozenDictionary<string, string?>> GetAgentsAsync(AzureAIConfiguration config, IConfiguration configuration, CancellationToken cancellationToken = default)
{
if (s_agentMap is not null)
{
return s_agentMap;
}
PersistentAgentsClient clientAgents = new(config.Endpoint, new AzureCliCredential());
AIProjectClient clientProjects = new(new Uri(config.Endpoint), new AzureCliCredential());
IKernelBuilder kernelBuilder = Kernel.CreateBuilder();
kernelBuilder.Services.AddSingleton(clientAgents);
kernelBuilder.Services.AddSingleton(clientProjects);
kernelBuilder.Plugins.AddFromType<Agents.MenuPlugin>();
AgentCreationOptions creationOptions = new() { Kernel = kernelBuilder.Build() };
AzureAIAgentFactory factory = new();
string repoRoot = WorkflowTest.GetRepoFolder();
return s_agentMap = (await Task.WhenAll(s_agentDefinitions.Select(kvp => CreateAgentAsync(kvp.Key, kvp.Value, cancellationToken)))).ToFrozenDictionary(t => t.Name, t => t.Id);
async Task<(string Name, string? Id)> CreateAgentAsync(string id, string file, CancellationToken cancellationToken)
{
try
{
string filePath = Path.Combine(Environment.CurrentDirectory, "Agents", file);
if (!File.Exists(filePath))
{
filePath = Path.Combine(repoRoot, "workflow-samples", "setup", file);
}
Assert.True(File.Exists(filePath), $"Agent definition file not found: {file}");
Debug.WriteLine($"TEST AGENT: Creating - {file}");
string agentText = File.ReadAllText(filePath);
Agent? agent = await factory.CreateAgentFromYamlAsync(agentText, creationOptions, configuration, cancellationToken);
Assert.NotNull(agent?.Name);
Debug.WriteLine($"TEST AGENT: {agent.Name} => {agent.Id} [{id}]");
return (id, agent.Id);
}
catch (Exception exception)
{
Console.WriteLine($"FAILURE: Error creating agent {id}: {exception.Message}");
throw;
}
}
}
}
@@ -1,16 +1,15 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Frozen;
using System.Collections.Generic;
using System.Reflection;
using System.Threading.Tasks;
using Azure.Identity;
using Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests.Agents;
using Microsoft.Agents.AI.Workflows.Declarative.PowerFx;
using Microsoft.Bot.ObjectModel;
using Microsoft.Extensions.AI;
using Microsoft.Extensions.Configuration;
using Shared.IntegrationTests;
using Xunit.Abstractions;
namespace Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests.Framework;
@@ -21,25 +20,20 @@ namespace Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests.Framework;
public abstract class IntegrationTest : IDisposable
{
private IConfigurationRoot? _configuration;
private AzureAIConfiguration? _foundryConfiguration;
protected IConfigurationRoot Configuration => this._configuration ??= InitializeConfig();
internal AzureAIConfiguration FoundryConfiguration
{
get
{
this._foundryConfiguration ??= this.Configuration.GetSection("AzureAI").Get<AzureAIConfiguration>();
Assert.NotNull(this._foundryConfiguration);
return this._foundryConfiguration;
}
}
public Uri TestEndpoint { get; }
public TestOutputAdapter Output { get; }
protected IntegrationTest(ITestOutputHelper output)
{
this.Output = new TestOutputAdapter(output);
this.TestEndpoint =
new Uri(
this.Configuration[AgentProvider.Settings.FoundryEndpoint] ??
throw new InvalidOperationException($"Undefined configuration setting: {AgentProvider.Settings.FoundryEndpoint}"));
Console.SetOut(this.Output);
SetProduct();
}
@@ -70,15 +64,8 @@ public abstract class IntegrationTest : IDisposable
protected async ValueTask<DeclarativeWorkflowOptions> CreateOptionsAsync(bool externalConversation = false, params IEnumerable<AIFunction> functionTools)
{
FrozenDictionary<string, string?> agentMap = await AgentFactory.GetAgentsAsync(this.FoundryConfiguration, this.Configuration);
IConfiguration workflowConfig =
new ConfigurationBuilder()
.AddInMemoryCollection(agentMap)
.Build();
AzureAgentProvider agentProvider =
new(this.FoundryConfiguration.Endpoint, new AzureCliCredential())
new(this.TestEndpoint, new AzureCliCredential())
{
Functions = functionTools,
};
@@ -92,7 +79,6 @@ public abstract class IntegrationTest : IDisposable
return
new DeclarativeWorkflowOptions(agentProvider)
{
Configuration = workflowConfig,
ConversationId = conversationId,
LoggerFactory = this.Output
};
@@ -100,7 +86,6 @@ public abstract class IntegrationTest : IDisposable
private static IConfigurationRoot InitializeConfig() =>
new ConfigurationBuilder()
.AddJsonFile("appsettings.Development.json", true)
.AddEnvironmentVariables()
.AddUserSecrets(Assembly.GetExecutingAssembly())
.Build();
@@ -17,23 +17,26 @@ namespace Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests.Framework;
internal sealed class WorkflowHarness(Workflow workflow, string runId)
{
private CheckpointManager? _checkpointManager;
private CheckpointInfo? LastCheckpoint { get; set; }
private CheckpointInfo? _lastCheckpoint;
public async Task<WorkflowEvents> RunTestcaseAsync<TInput>(Testcase testcase, TInput input, bool useJson = false) where TInput : notnull
{
WorkflowEvents workflowEvents = await this.RunWorkflowAsync(input, useJson);
int requestCount = (workflowEvents.InputEvents.Count + 1) / 2;
int requestCount = workflowEvents.InputEvents.Count;
int responseCount = 0;
while (requestCount > responseCount)
{
ExternalRequest request = workflowEvents.InputEvents[workflowEvents.InputEvents.Count - 1].Request;
Assert.NotNull(testcase.Setup.Responses);
Assert.NotEmpty(testcase.Setup.Responses);
string inputText = testcase.Setup.Responses[responseCount].Value;
Console.WriteLine($"ID: {request.RequestId}");
Console.WriteLine($"INPUT: {inputText}");
++responseCount;
WorkflowEvents runEvents = await this.ResumeAsync(new AnswerResponse(inputText)).ConfigureAwait(false);
ExternalResponse response = request.CreateResponse(new ExternalInputResponse(new ChatMessage(ChatRole.User, inputText)));
WorkflowEvents runEvents = await this.ResumeAsync(response).ConfigureAwait(false);
workflowEvents = new WorkflowEvents([.. workflowEvents.Events, .. runEvents.Events]);
requestCount = (workflowEvents.InputEvents.Count + 1) / 2;
requestCount = workflowEvents.InputEvents.Count;
}
return workflowEvents;
@@ -44,15 +47,15 @@ internal sealed class WorkflowHarness(Workflow workflow, string runId)
Console.WriteLine("RUNNING WORKFLOW...");
Checkpointed<StreamingRun> run = await InProcessExecution.StreamAsync(workflow, input, this.GetCheckpointManager(useJson), runId);
IReadOnlyList<WorkflowEvent> workflowEvents = await MonitorAndDisposeWorkflowRunAsync(run).ToArrayAsync();
this.LastCheckpoint = workflowEvents.OfType<SuperStepCompletedEvent>().LastOrDefault()?.CompletionInfo?.Checkpoint;
this._lastCheckpoint = workflowEvents.OfType<SuperStepCompletedEvent>().LastOrDefault()?.CompletionInfo?.Checkpoint;
return new WorkflowEvents(workflowEvents);
}
public async Task<WorkflowEvents> ResumeAsync(object response)
public async Task<WorkflowEvents> ResumeAsync(ExternalResponse response)
{
Console.WriteLine("\nRESUMING WORKFLOW...");
Assert.NotNull(this.LastCheckpoint);
Checkpointed<StreamingRun> run = await InProcessExecution.ResumeStreamAsync(workflow, this.LastCheckpoint, this.GetCheckpointManager(), runId);
Assert.NotNull(this._lastCheckpoint);
Checkpointed<StreamingRun> run = await InProcessExecution.ResumeStreamAsync(workflow, this._lastCheckpoint, this.GetCheckpointManager(), runId);
IReadOnlyList<WorkflowEvent> workflowEvents = await MonitorAndDisposeWorkflowRunAsync(run, response).ToArrayAsync();
return new WorkflowEvents(workflowEvents);
}
@@ -93,29 +96,32 @@ internal sealed class WorkflowHarness(Workflow workflow, string runId)
return this._checkpointManager;
}
private static async IAsyncEnumerable<WorkflowEvent> MonitorAndDisposeWorkflowRunAsync(Checkpointed<StreamingRun> run, object? response = null)
private static async IAsyncEnumerable<WorkflowEvent> MonitorAndDisposeWorkflowRunAsync(Checkpointed<StreamingRun> run, ExternalResponse? response = null)
{
await using IAsyncDisposable disposeRun = run;
if (response is not null)
{
await run.Run.SendResponseAsync(response).ConfigureAwait(false);
}
bool exitLoop = false;
bool hasRequest = false;
await foreach (WorkflowEvent workflowEvent in run.Run.WatchStreamAsync().ConfigureAwait(false))
{
bool exitLoop = false;
switch (workflowEvent)
{
case RequestInfoEvent requestInfo:
Console.WriteLine($"REQUEST #{requestInfo.Request.RequestId}");
if (response is not null)
{
ExternalResponse requestResponse = requestInfo.Request.CreateResponse(response);
await run.Run.SendResponseAsync(requestResponse).ConfigureAwait(false);
response = null;
}
else
case SuperStepCompletedEvent:
if (hasRequest)
{
exitLoop = true;
}
break;
case RequestInfoEvent requestInfo:
Console.WriteLine($"REQUEST #{requestInfo.Request.RequestId}");
hasRequest = true;
break;
case ConversationUpdateEvent conversationEvent:
Console.WriteLine($"CONVERSATION: {conversationEvent.ConversationId}");
@@ -33,6 +33,9 @@ public sealed class FunctionCallingWorkflowTest(ITestOutputHelper output) : Inte
private async Task RunWorkflowAsync(bool autoInvoke, params IEnumerable<AIFunction> functionTools)
{
AgentProvider agentProvider = AgentProvider.Create(this.Configuration, AgentProvider.Names.FunctionTool);
await agentProvider.CreateAgentsAsync().ConfigureAwait(false);
string workflowPath = GetWorkflowPath("FunctionTool.yaml");
Dictionary<string, AIFunction> functionMap = autoInvoke ? [] : functionTools.ToDictionary(tool => tool.Name, tool => tool);
DeclarativeWorkflowOptions workflowOptions = await this.CreateOptionsAsync(externalConversation: false, autoInvoke ? functionTools : []);
@@ -47,11 +50,11 @@ public sealed class FunctionCallingWorkflowTest(ITestOutputHelper output) : Inte
Assert.False(autoInvoke);
RequestInfoEvent inputEvent = workflowEvents.InputEvents[workflowEvents.InputEvents.Count - 1];
AgentFunctionToolRequest? toolRequest = inputEvent.Request.Data.As<AgentFunctionToolRequest>();
ExternalInputRequest? toolRequest = inputEvent.Request.Data.As<ExternalInputRequest>();
Assert.NotNull(toolRequest);
List<(FunctionCallContent, AIFunction)> functionCalls = [];
foreach (FunctionCallContent functionCall in toolRequest.FunctionCalls)
foreach (FunctionCallContent functionCall in toolRequest.AgentResponse.Messages.SelectMany(message => message.Contents).OfType<FunctionCallContent>())
{
this.Output.WriteLine($"TOOL REQUEST: {functionCall.Name}");
if (!functionMap.TryGetValue(functionCall.Name, out AIFunction? functionTool))
@@ -62,11 +65,12 @@ public sealed class FunctionCallingWorkflowTest(ITestOutputHelper output) : Inte
functionCalls.Add((functionCall, functionTool));
}
IList<FunctionResultContent> functionResults = await InvokeToolsAsync(functionCalls);
IList<AIContent> functionResults = await InvokeToolsAsync(functionCalls);
++responseCount;
WorkflowEvents runEvents = await harness.ResumeAsync(AgentFunctionToolResponse.Create(toolRequest, functionResults)).ConfigureAwait(false);
ChatMessage resultMessage = new(ChatRole.Tool, functionResults);
WorkflowEvents runEvents = await harness.ResumeAsync(inputEvent.Request.CreateResponse(new ExternalInputResponse(resultMessage))).ConfigureAwait(false);
workflowEvents = new WorkflowEvents([.. workflowEvents.Events, .. runEvents.Events]);
}
@@ -79,13 +83,13 @@ public sealed class FunctionCallingWorkflowTest(ITestOutputHelper output) : Inte
Assert.NotEmpty(workflowEvents.InputEvents);
}
Assert.Equal(autoInvoke ? 3 : 5, workflowEvents.AgentResponseEvents.Count);
Assert.Equal(autoInvoke ? 3 : 4, workflowEvents.AgentResponseEvents.Count);
Assert.All(workflowEvents.AgentResponseEvents, response => response.Response.Text.Contains("4.95"));
}
private static async ValueTask<IList<FunctionResultContent>> InvokeToolsAsync(IEnumerable<(FunctionCallContent, AIFunction)> functionCalls)
private static async ValueTask<IList<AIContent>> InvokeToolsAsync(IEnumerable<(FunctionCallContent, AIFunction)> functionCalls)
{
List<FunctionResultContent> results = [];
List<AIContent> results = [];
foreach ((FunctionCallContent functionCall, AIFunction functionTool) in functionCalls)
{
@@ -4,10 +4,11 @@ using System;
using System.IO;
using System.Net.Http;
using System.Threading.Tasks;
using Azure.AI.Agents.Persistent;
using Azure.AI.Agents;
using Azure.Identity;
using Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests.Framework;
using Microsoft.Extensions.AI;
using OpenAI.Files;
using Xunit.Abstractions;
namespace Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests;
@@ -18,9 +19,9 @@ namespace Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests;
public sealed class MediaInputTest(ITestOutputHelper output) : IntegrationTest(output)
{
private const string WorkflowFileName = "MediaInput.yaml";
private const string ImageReference = "https://upload.wikimedia.org/wikipedia/commons/5/56/White_shark.jpg";
private const string ImageReference = "https://sample-files.com/downloads/documents/pdf/basic-text.pdf";
[Fact(Skip = "Service issue prevents this simple case")]
[Fact]
public async Task ValidateImageUrlAsync()
{
this.Output.WriteLine($"Image: {ImageReference}");
@@ -30,20 +31,21 @@ public sealed class MediaInputTest(ITestOutputHelper output) : IntegrationTest(o
[Fact]
public async Task ValidateImageDataAsync()
{
byte[] imageData = await DownloadImageAsync();
byte[] imageData = await DownloadFileAsync();
string encodedData = Convert.ToBase64String(imageData);
string imageUrl = $"data:image/png;base64,{encodedData}";
this.Output.WriteLine($"Image: {imageUrl.Substring(0, 112)}...");
await this.ValidateImageAsync(new DataContent(imageUrl));
}
[Fact]
[Fact(Skip = "Not behaving will in git-hub build pipeline")]
public async Task ValidateImageUploadAsync()
{
byte[] imageData = await DownloadImageAsync();
PersistentAgentsClient client = new(this.FoundryConfiguration.Endpoint, new AzureCliCredential());
byte[] imageData = await DownloadFileAsync();
AgentsClient client = new(this.TestEndpoint, new AzureCliCredential());
using MemoryStream contentStream = new(imageData);
PersistentAgentFileInfo fileInfo = await client.Files.UploadFileAsync(contentStream, PersistentAgentFilePurpose.Agents, "image.jpg");
OpenAIFileClient fileClient = client.GetOpenAIClient().GetOpenAIFileClient();
OpenAIFile fileInfo = await fileClient.UploadFileAsync(contentStream, "basic-text.pdf", FileUploadPurpose.Assistants);
try
{
this.Output.WriteLine($"Image: {fileInfo.Id}");
@@ -51,11 +53,11 @@ public sealed class MediaInputTest(ITestOutputHelper output) : IntegrationTest(o
}
finally
{
await client.Files.DeleteFileAsync(fileInfo.Id);
await fileClient.DeleteFileAsync(fileInfo.Id);
}
}
private static async Task<byte[]> DownloadImageAsync()
private static async Task<byte[]> DownloadFileAsync()
{
using HttpClient client = new();
client.DefaultRequestHeaders.UserAgent.ParseAdd("Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/110.0");
@@ -5,8 +5,9 @@
</PropertyGroup>
<PropertyGroup>
<InjectSharedIntegrationTestCode>true</InjectSharedIntegrationTestCode>
<InjectSharedBuildTestCode>true</InjectSharedBuildTestCode>
<InjectSharedFoundryAgents>true</InjectSharedFoundryAgents>
<InjectSharedIntegrationTestCode>true</InjectSharedIntegrationTestCode>
</PropertyGroup>
<ItemGroup>
@@ -21,9 +22,6 @@
<PackageReference Include="Microsoft.Extensions.Configuration.UserSecrets" />
<PackageReference Include="Microsoft.Extensions.Configuration.EnvironmentVariables" />
<PackageReference Include="Microsoft.Extensions.AI.OpenAI" />
<PackageReference Include="Microsoft.SemanticKernel.Agents.Abstractions" />
<PackageReference Include="Microsoft.SemanticKernel.Agents.AzureAI" />
<PackageReference Include="Microsoft.SemanticKernel.Agents.Yaml" />
<PackageReference Include="System.Linq.Async" />
</ItemGroup>
@@ -0,0 +1,29 @@
{
"description": "Human in the loop sample - RequestExternalInput.yaml.",
"setup": {
"input": {
"type": "String",
"value": "1234"
},
"responses": [
{
"type": "String",
"value": "1234"
}
]
},
"validation": {
"conversation_count": 1,
"min_action_count": 4,
"max_action_count": -1,
"min_response_count": 0,
"actions": {
"start": [
"set_project"
],
"final": [
"sendActivity_confirmed"
]
}
}
}
@@ -9,7 +9,8 @@
"validation": {
"conversation_count": 3,
"min_action_count": 3,
"min_response_count": 2,
"min_response_count": 3,
"min_message_count": 4,
"actions": {
"start": [
"invoke_inner1",
@@ -10,6 +10,7 @@
"conversation_count": 1,
"min_action_count": 3,
"min_response_count": 3,
"min_message_count": 6,
"actions": {
"start": [
"invoke_analyst",
@@ -12,6 +12,8 @@
"max_action_count": -1,
"min_response_count": 2,
"max_response_count": 8,
"min_message_count": 4,
"max_message_count": 17,
"actions": {
"start": [
],
@@ -0,0 +1,29 @@
{
"description": "Human in the loop sample - RequestExternalInput.yaml.",
"setup": {
"input": {
"type": "String",
"value": "n/a"
},
"responses": [
{
"type": "String",
"value": "This is external input"
}
]
},
"validation": {
"conversation_count": 1,
"min_action_count": 2,
"min_response_count": 0,
"min_message_count": 1,
"actions": {
"start": [
"get_input"
],
"final": [
"show_input"
]
}
}
}
@@ -12,43 +12,43 @@ trigger:
- condition: =IsBlank(System.Conversation)
id: conversation_check
actions:
- kind: EndDialog
- kind: EndWorkflow
id: conversation_bad
- condition: =IsBlank(System.Conversation.Id)
id: conversation_id_check1
actions:
- kind: EndDialog
- kind: EndWorkflow
id: conversation_id_bad1
- condition: =IsBlank(System.ConversationId)
id: conversation_id_check2
actions:
- kind: EndDialog
- kind: EndWorkflow
id: conversation_id_bad2
- condition: =IsBlank(System.LastMessage)
id: message_check
actions:
- kind: EndDialog
- kind: EndWorkflow
id: message_bad
- condition: =IsBlank(System.LastMessage.Id)
id: message_id_check1
actions:
- kind: EndDialog
- kind: EndWorkflow
id: message_id_bad1
- condition: =IsBlank(System.LastMessageId)
id: message_id_check2
actions:
- kind: EndDialog
- kind: EndWorkflow
id: message_id_bad2
- condition: =IsBlank(System.LastMessageText)
id: message_text_check
actions:
- kind: EndDialog
- kind: EndWorkflow
id: message_text_bad
elseActions:
@@ -9,13 +9,13 @@ trigger:
id: invoke_greet
conversationId: =System.ConversationId
agent:
name: =Env.FOUNDRY_AGENT_TOOL
name: MenuAgent
- kind: InvokeAzureAgent
id: invoke_menu
conversationId: =System.ConversationId
agent:
name: =Env.FOUNDRY_AGENT_TOOL
name: MenuAgent
input:
messages: =UserMessage("What's on today's menu?")
@@ -23,6 +23,6 @@ trigger:
id: invoke_item
conversationId: =System.ConversationId
agent:
name: =Env.FOUNDRY_AGENT_TOOL
name: MenuAgent
input:
messages: =UserMessage("How much is the clam chowder?")
@@ -8,27 +8,25 @@ trigger:
- kind: InvokeAzureAgent
id: invoke_inner1
agent:
name: =Env.FOUNDRY_AGENT_TEST
name: TestAgent
input:
messages: =UserMessage("Can an LLM think of funny jokes?")
output:
autoSend: false
- kind: InvokeAzureAgent
id: invoke_inner2
agent:
name: =Env.FOUNDRY_AGENT_TEST
name: TestAgent
input:
messages: =UserMessage("Do you know the joke about the chicken crossing the road? Tell me an improved version of that joke.")
output:
autoSend: true
- kind: InvokeAzureAgent
id: invoke_external
conversationId: =System.ConversationId
agent:
name: =Env.FOUNDRY_AGENT_TEST
name: TestAgent
input:
additionalInstructions: |-
Rate the originality of this well known joke that is being re-told on a scale of 1 to 10.
Take note on where improvements or changes were made.
messages: =UserMessage("Rate the originality of this well known joke that is being re-told on a scale of 1 to 10. Take note on where improvements or changes were made.")
output:
messages: Local.RatingResponse
@@ -9,7 +9,7 @@ trigger:
id: invoke_vision
conversationId: =System.ConversationId
agent:
name: =Env.FOUNDRY_AGENT_TEST
name: TestAgent
input:
additionalInstructions: |-
Describe the image contained in the user request, if any;
@@ -0,0 +1,14 @@
kind: Workflow
trigger:
kind: OnConversationStart
id: workflow_test
actions:
- kind: RequestExternalInput
id: get_input
variable: Local.MyInput
- kind: SendMessage
id: show_input
message: "You provided: {Local.MyInput}"
@@ -15,7 +15,7 @@ trigger:
- kind: SetVariable
id: set_user_name
variable: Global.UserName
value: =Env.USERNAME
value: TestAgent
# Respond with input
- kind: SendActivity
@@ -15,6 +15,7 @@ public sealed class DeclarativeEjectionTest(ITestOutputHelper output) : Workflow
{
[Theory]
[InlineData("AddConversationMessage.yaml")]
[InlineData("CancelWorkflow.yaml")]
[InlineData("ClearAllVariables.yaml")]
[InlineData("CopyConversationMessages.yaml")]
[InlineData("Condition.yaml")]
@@ -23,7 +24,7 @@ public sealed class DeclarativeEjectionTest(ITestOutputHelper output) : Workflow
[InlineData("EditTable.yaml")]
[InlineData("EditTableV2.yaml")]
[InlineData("EndConversation.yaml")]
[InlineData("EndDialog.yaml")]
[InlineData("EndWorkflow.yaml")]
[InlineData("Goto.yaml")]
[InlineData("InvokeAgent.yaml")]
[InlineData("LoopBreak.yaml")]
@@ -44,18 +44,6 @@ public class InvokeAzureAgentTemplateTest(ITestOutputHelper output) : WorkflowAc
BoolExpression.Expression("1 < 2"));
}
[Fact]
public void AdditionalInstructions()
{
// Act, Assert
this.ExecuteTest(
nameof(VariableConversation),
StringExpression.Literal("asst_123abc"),
StringExpression.Variable(PropertyPath.TopicVariable("TestConversation")),
"MyMessages",
additionalInstructions: "Test instructions...");
}
[Fact]
public void InputMessagesVariable()
{
@@ -86,7 +74,6 @@ public class InvokeAzureAgentTemplateTest(ITestOutputHelper output) : WorkflowAc
StringExpression.Builder conversation,
string? messagesVariable = null,
BoolExpression.Builder? autoSend = null,
string? additionalInstructions = null,
ValueExpression.Builder? messages = null)
{
// Arrange
@@ -97,7 +84,6 @@ public class InvokeAzureAgentTemplateTest(ITestOutputHelper output) : WorkflowAc
conversation,
messagesVariable,
autoSend,
additionalInstructions is null ? null : (TemplateLine.Builder)TemplateLine.Parse(additionalInstructions),
messages);
// Act
@@ -117,7 +103,6 @@ public class InvokeAzureAgentTemplateTest(ITestOutputHelper output) : WorkflowAc
StringExpression.Builder conversation,
string? messagesVariable = null,
BoolExpression.Builder? autoSend = null,
TemplateLine.Builder? additionalInstructions = null,
ValueExpression.Builder? messages = null)
{
InitializablePropertyPath? outputMessages = null;
@@ -141,7 +126,6 @@ public class InvokeAzureAgentTemplateTest(ITestOutputHelper output) : WorkflowAc
new AzureAgentInput.Builder
{
Messages = messages,
AdditionalInstructions = additionalInstructions,
},
Output =
new AzureAgentOutput.Builder
@@ -145,8 +145,9 @@ public sealed class DeclarativeWorkflowTest(ITestOutputHelper output) : Workflow
}
[Theory]
[InlineData("CancelWorkflow.yaml", 1, "end_all")]
[InlineData("EndConversation.yaml", 1, "end_all")]
[InlineData("EndDialog.yaml", 1, "end_all")]
[InlineData("EndWorkflow.yaml", 1, "end_all")]
[InlineData("EditTable.yaml", 2, "edit_var")]
[InlineData("EditTableV2.yaml", 2, "edit_var")]
[InlineData("ParseValue.yaml", 2, "parse_var")]
@@ -170,8 +171,6 @@ public sealed class DeclarativeWorkflowTest(ITestOutputHelper output) : Workflow
[InlineData(typeof(AdaptiveCardPrompt.Builder))]
[InlineData(typeof(BeginDialog.Builder))]
[InlineData(typeof(CSATQuestion.Builder))]
[InlineData(typeof(CancelAllDialogs.Builder))]
[InlineData(typeof(CancelDialog.Builder))]
[InlineData(typeof(CreateSearchQuery.Builder))]
[InlineData(typeof(DeleteActivity.Builder))]
[InlineData(typeof(DisableTrigger.Builder))]
@@ -231,7 +230,7 @@ public sealed class DeclarativeWorkflowTest(ITestOutputHelper output) : Workflow
[InlineData("Condition.yaml", "setVariable_test")]
[InlineData("ConditionElse.yaml", "setVariable_test")]
[InlineData("EndConversation.yaml", "end_all")]
[InlineData("EndDialog.yaml", "end_all")]
[InlineData("EndWorkflow.yaml", "end_all")]
[InlineData("EditTable.yaml", "edit_var")]
[InlineData("EditTableV2.yaml", "edit_var")]
[InlineData("Goto.yaml", "goto_end")]
@@ -1,48 +0,0 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Collections.Generic;
using Microsoft.Agents.AI.Workflows.Declarative.Events;
using Microsoft.Extensions.AI;
using Xunit.Abstractions;
namespace Microsoft.Agents.AI.Workflows.Declarative.UnitTests.Events;
/// <summary>
/// Verify <see cref="AgentFunctionToolRequest"/> class
/// </summary>
public sealed class AgentFunctionToolRequestTest(ITestOutputHelper output) : EventTest(output)
{
[Fact]
public void VerifySerializationEmpty()
{
// Arrange & Act
AgentFunctionToolRequest copy = VerifyEventSerialization(new AgentFunctionToolRequest("testagent", []));
// Assert
Assert.Equal("testagent", copy.AgentName);
Assert.Empty(copy.FunctionCalls);
}
[Fact]
public void VerifySerializationWithRequests()
{
// Arrange & Act
AgentFunctionToolRequest copy =
VerifyEventSerialization(
new AgentFunctionToolRequest(
"agent",
[
new FunctionCallContent("call1", "result1"),
new FunctionCallContent("call2", "result2", new Dictionary<string, object?>() { { "name", "Clam Chowder" } }),
]));
// Assert
Assert.Equal("agent", copy.AgentName);
Assert.Equal(2, copy.FunctionCalls.Count);
Assert.IsType<FunctionCallContent>(copy.FunctionCalls[0]);
Assert.Null(copy.FunctionCalls[0].Arguments);
Assert.IsType<FunctionCallContent>(copy.FunctionCalls[1]);
Assert.NotNull(copy.FunctionCalls[1].Arguments);
Assert.NotEmpty(copy.FunctionCalls[1].Arguments!);
}
}
@@ -1,46 +0,0 @@
// Copyright (c) Microsoft. All rights reserved.
using Microsoft.Agents.AI.Workflows.Declarative.Events;
using Microsoft.Extensions.AI;
using Xunit.Abstractions;
namespace Microsoft.Agents.AI.Workflows.Declarative.UnitTests.Events;
/// <summary>
/// Verify <see cref="AgentFunctionToolResponse"/> class
/// </summary>
public sealed class AgentFunctionToolResponseTest(ITestOutputHelper output) : EventTest(output)
{
[Fact]
public void VerifySerializationEmpty()
{
// Arrange & Act
AgentFunctionToolResponse copy = VerifyEventSerialization(new AgentFunctionToolResponse("testagent", []));
// Assert
Assert.Equal("testagent", copy.AgentName);
Assert.Empty(copy.FunctionResults);
}
[Fact]
public void VerifySerializationWithResults()
{
// Arrange & Act
AgentFunctionToolResponse copy =
VerifyEventSerialization(
new AgentFunctionToolResponse(
"agent",
[
new FunctionResultContent("call1", "result1"),
new FunctionResultContent("call2", "result2"),
]));
// Assert
Assert.Equal("agent", copy.AgentName);
Assert.Equal(2, copy.FunctionResults.Count);
Assert.IsType<FunctionResultContent>(copy.FunctionResults[0]);
Assert.Equal("call1", copy.FunctionResults[0].CallId);
Assert.IsType<FunctionResultContent>(copy.FunctionResults[1]);
Assert.Equal("call2", copy.FunctionResults[1].CallId);
}
}
@@ -1,5 +1,6 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Linq;
using System.Text.Json;
using Microsoft.Extensions.AI;
using Xunit.Abstractions;
@@ -19,4 +20,17 @@ public abstract class EventTest(ITestOutputHelper output) : WorkflowTest(output)
Assert.NotNull(copy);
return copy;
}
protected static void AssertMessage(ChatMessage source, ChatMessage copy)
{
Assert.Equal(source.Role, copy.Role);
Assert.Equal(source.Text, copy.Text);
Assert.Equal(source.Contents.Count, copy.Contents.Count);
}
protected static TContent AssertContent<TContent>(ChatMessage message) where TContent : AIContent
{
TContent[] contents = message.Contents.OfType<TContent>().ToArray();
return Assert.Single(contents);
}
}
@@ -0,0 +1,62 @@
// Copyright (c) Microsoft. All rights reserved.
using Microsoft.Agents.AI.Workflows.Declarative.Events;
using Microsoft.Extensions.AI;
using Xunit.Abstractions;
namespace Microsoft.Agents.AI.Workflows.Declarative.UnitTests.Events;
/// <summary>
/// Verify <see cref="ExternalInputRequest"/> class
/// </summary>
public sealed class ExternalInputRequestTest(ITestOutputHelper output) : EventTest(output)
{
[Fact]
public void VerifySerializationWithText()
{
// Arrange
ExternalInputRequest source = new(new AgentRunResponse(new ChatMessage(ChatRole.User, "Wassup?")));
// Act
ExternalInputRequest copy = VerifyEventSerialization(source);
// Assert
ChatMessage messageCopy = Assert.Single(source.AgentResponse.Messages);
AssertMessage(messageCopy, copy.AgentResponse.Messages[0]);
}
[Fact]
public void VerifySerializationWithRequests()
{
// Arrange
ExternalInputRequest source =
new(new AgentRunResponse(
new ChatMessage(
ChatRole.Assistant,
[
new McpServerToolApprovalRequestContent("call1", new McpServerToolCallContent("call1", "testmcp", "server-name")),
new FunctionApprovalRequestContent("call2", new FunctionCallContent("call2", "result1")),
new FunctionCallContent("call3", "myfunc"),
new TextContent("Heya"),
])));
// Act
ExternalInputRequest copy = VerifyEventSerialization(source);
// Assert
ChatMessage messageCopy = Assert.Single(source.AgentResponse.Messages);
Assert.Equal(messageCopy.Contents.Count, copy.AgentResponse.Messages[0].Contents.Count);
McpServerToolApprovalRequestContent mcpRequest = AssertContent<McpServerToolApprovalRequestContent>(messageCopy);
Assert.Equal("call1", mcpRequest.Id);
FunctionApprovalRequestContent functionRequest = AssertContent<FunctionApprovalRequestContent>(messageCopy);
Assert.Equal("call2", functionRequest.Id);
FunctionCallContent functionCall = AssertContent<FunctionCallContent>(messageCopy);
Assert.Equal("call3", functionCall.CallId);
TextContent textContent = AssertContent<TextContent>(messageCopy);
Assert.Equal("Heya", textContent.Text);
}
}
@@ -0,0 +1,61 @@
// Copyright (c) Microsoft. All rights reserved.
using Microsoft.Agents.AI.Workflows.Declarative.Events;
using Microsoft.Extensions.AI;
using Xunit.Abstractions;
namespace Microsoft.Agents.AI.Workflows.Declarative.UnitTests.Events;
/// <summary>
/// Verify <see cref="ExternalInputResponse"/> class
/// </summary>
public sealed class ExternalInputResponseTest(ITestOutputHelper output) : EventTest(output)
{
[Fact]
public void VerifySerializationEmpty()
{
// Arrange
ExternalInputResponse source = new(new ChatMessage(ChatRole.User, "Wassup?"));
// Act
ExternalInputResponse copy = VerifyEventSerialization(source);
// Assert
ChatMessage messageCopy = Assert.Single(source.Messages);
AssertMessage(messageCopy, copy.Messages[0]);
}
[Fact]
public void VerifySerializationWithResponses()
{
// Arrange
ExternalInputResponse source =
new(new ChatMessage(
ChatRole.Assistant,
[
new McpServerToolApprovalRequestContent("call1", new McpServerToolCallContent("call1", "testmcp", "server-name")).CreateResponse(approved: true),
new FunctionApprovalRequestContent("call2", new FunctionCallContent("call2", "result1")).CreateResponse(approved: true),
new FunctionResultContent("call3", 33),
new TextContent("Heya"),
]));
// Act
ExternalInputResponse copy = VerifyEventSerialization(source);
// Assert
ChatMessage responseMessage = Assert.Single(source.Messages);
Assert.Equal(responseMessage.Contents.Count, copy.Messages[0].Contents.Count);
McpServerToolApprovalResponseContent mcpApproval = AssertContent<McpServerToolApprovalResponseContent>(responseMessage);
Assert.Equal("call1", mcpApproval.Id);
FunctionApprovalResponseContent functionApproval = AssertContent<FunctionApprovalResponseContent>(responseMessage);
Assert.Equal("call2", functionApproval.Id);
FunctionResultContent functionResult = AssertContent<FunctionResultContent>(responseMessage);
Assert.Equal("call3", functionResult.CallId);
TextContent textContent = AssertContent<TextContent>(responseMessage);
Assert.Equal("Heya", textContent.Text);
}
}
@@ -1,46 +0,0 @@
// Copyright (c) Microsoft. All rights reserved.
using Microsoft.Agents.AI.Workflows.Declarative.Events;
using Microsoft.Extensions.AI;
using Xunit.Abstractions;
namespace Microsoft.Agents.AI.Workflows.Declarative.UnitTests.Events;
/// <summary>
/// Verify <see cref="UserInputRequest"/> class
/// </summary>
public sealed class UserInputRequestTest(ITestOutputHelper output) : EventTest(output)
{
[Fact]
public void VerifySerializationEmpty()
{
// Arrange & Act
UserInputRequest copy = VerifyEventSerialization(new UserInputRequest("test agent", []));
// Assert
Assert.Equal("test agent", copy.AgentName);
Assert.Empty(copy.InputRequests);
}
[Fact]
public void VerifySerializationWithRequests()
{
// Arrange & Act
UserInputRequest copy =
VerifyEventSerialization(
new UserInputRequest(
"agent",
[
new McpServerToolApprovalRequestContent("call1", new McpServerToolCallContent("call1", "testmcp", "server-name")),
new FunctionApprovalRequestContent("call2", new FunctionCallContent("call2", "result1")),
]));
// Assert
Assert.Equal("agent", copy.AgentName);
Assert.Equal(2, copy.InputRequests.Count);
McpServerToolApprovalRequestContent mcpRequest = Assert.IsType<McpServerToolApprovalRequestContent>(copy.InputRequests[0]);
Assert.Equal("call1", mcpRequest.Id);
FunctionApprovalRequestContent functionRequest = Assert.IsType<FunctionApprovalRequestContent>(copy.InputRequests[1]);
Assert.Equal("call2", functionRequest.Id);
}
}
@@ -1,46 +0,0 @@
// Copyright (c) Microsoft. All rights reserved.
using Microsoft.Agents.AI.Workflows.Declarative.Events;
using Microsoft.Extensions.AI;
using Xunit.Abstractions;
namespace Microsoft.Agents.AI.Workflows.Declarative.UnitTests.Events;
/// <summary>
/// Verify <see cref="UserInputResponse"/> class
/// </summary>
public sealed class UserInputResponseTest(ITestOutputHelper output) : EventTest(output)
{
[Fact]
public void VerifySerializationEmpty()
{
// Arrange & Act
UserInputResponse copy = VerifyEventSerialization(new UserInputResponse("testagent", []));
// Assert
Assert.Equal("testagent", copy.AgentName);
Assert.Empty(copy.InputResponses);
}
[Fact]
public void VerifySerializationWithResponses()
{
// Arrange & Act
UserInputResponse copy =
VerifyEventSerialization(
new UserInputResponse(
"agent",
[
new McpServerToolApprovalRequestContent("call1", new McpServerToolCallContent("call1", "testmcp", "server-name")).CreateResponse(approved: true),
new FunctionApprovalRequestContent("call2", new FunctionCallContent("call2", "result1")).CreateResponse(approved: true),
]));
// Assert
Assert.Equal("agent", copy.AgentName);
Assert.Equal(2, copy.InputResponses.Count);
McpServerToolApprovalResponseContent mcpResponse = Assert.IsType<McpServerToolApprovalResponseContent>(copy.InputResponses[0]);
Assert.Equal("call1", mcpResponse.Id);
FunctionApprovalResponseContent functionResponse = Assert.IsType<FunctionApprovalResponseContent>(copy.InputResponses[1]);
Assert.Equal("call2", functionResponse.Id);
}
}
@@ -1,22 +0,0 @@
// Copyright (c) Microsoft. All rights reserved.
using Microsoft.Agents.AI.Workflows.Declarative.Events;
using Xunit.Abstractions;
namespace Microsoft.Agents.AI.Workflows.Declarative.UnitTests.Events;
/// <summary>
/// Verify <see cref="AnswerRequest"/> class
/// </summary>
public sealed class UserMessageRequestTest(ITestOutputHelper output) : EventTest(output)
{
[Fact]
public void VerifySerialization()
{
// Arrange & Act
AnswerRequest copy = VerifyEventSerialization(new AnswerRequest("wassup"));
// Assert
Assert.Equal("wassup", copy.Prompt);
}
}
@@ -1,33 +0,0 @@
// Copyright (c) Microsoft. All rights reserved.
using Microsoft.Agents.AI.Workflows.Declarative.Events;
using Microsoft.Extensions.AI;
using Xunit.Abstractions;
namespace Microsoft.Agents.AI.Workflows.Declarative.UnitTests.Events;
/// <summary>
/// Verify <see cref="AnswerResponse"/> class
/// </summary>
public sealed class UserMessageResponseTest(ITestOutputHelper output) : EventTest(output)
{
[Fact]
public void VerifySerializationText()
{
// Arrange & Act
AnswerResponse copy = VerifyEventSerialization(new AnswerResponse("test response"));
// Assert
Assert.Equal("test response", copy.Value.Text);
}
[Fact]
public void VerifySerializationMessage()
{
// Arrange & Act
AnswerResponse copy = VerifyEventSerialization(new AnswerResponse(new ChatMessage(ChatRole.User, "test response")));
// Assert
Assert.Equal("test response", copy.Value.Text);
}
}
@@ -58,6 +58,38 @@ public sealed class JsonDocumentExtensionsTests
Assert.Equal(expectedTimeSpan, result["time"]);
}
[Fact]
public void ParseRecord_Object_NoSchema_Succeeds()
{
// Arrange
JsonDocument document = JsonDocument.Parse(
"""
{
"text": "hello",
"numberInt": 7,
"numberLong": 9223372036854775807,
"numberDecimal": 12.5,
"numberDouble": 3.99E99,
"flag": true,
"date": "2024-10-01T12:34:56Z",
"time": "12:34:56"
}
""");
// Act
Dictionary<string, object?> result = document.ParseRecord(VariableType.RecordType);
// Assert
Assert.Equal("hello", result["text"]);
Assert.Equal(7, result["numberInt"]);
Assert.Equal(9223372036854775807L, result["numberLong"]);
Assert.Equal(12.5m, result["numberDecimal"]);
Assert.Equal(3.99E99, result["numberDouble"]);
Assert.Equal(true, result["flag"]);
Assert.Equal("2024-10-01T12:34:56Z", result["date"]);
Assert.Equal("12:34:56", result["time"]);
}
[Fact]
public void ParseRecord_Object_NestedRecord_Succeeds()
{
@@ -14,8 +14,8 @@ public sealed class TemplateExtensionsTests
{
// Arrange
RecalcEngine engine = new();
IEnumerable<TemplateLine> template = new List<TemplateLine>
{
IEnumerable<TemplateLine> template =
[
new TemplateLine.Builder
{
Segments =
@@ -24,7 +24,7 @@ public sealed class TemplateExtensionsTests
new TextSegment.Builder { Value = "World" }
}
}.Build()
};
];
// Act
string result = engine.Format(template);
@@ -38,8 +38,8 @@ public sealed class TemplateExtensionsTests
{
// Arrange
RecalcEngine engine = new();
IEnumerable<TemplateLine> template = new List<TemplateLine>
{
IEnumerable<TemplateLine> template =
[
new TemplateLine.Builder
{
Segments =
@@ -54,7 +54,7 @@ public sealed class TemplateExtensionsTests
new TextSegment.Builder { Value = "Line 2" }
}
}.Build()
};
];
// Act
string result = engine.Format(template);
@@ -0,0 +1,13 @@
kind: Workflow
trigger:
kind: OnConversationStart
id: my_workflow
actions:
- kind: CancelWorkflow
id: end_all
- kind: SendActivity
id: send_activity_1
activity: NEVER 1!
@@ -21,7 +21,7 @@ trigger:
- id: condition_match
condition: =Local.TestValue1 + Local.TestValue2 = 7
actions:
- kind: EndDialog
- kind: EndWorkflow
id: end_when_match
- kind: SendActivity
@@ -0,0 +1,94 @@
// ------------------------------------------------------------------------------
// <auto-generated>
// This code was generated by a tool.
// </auto-generated>
// ------------------------------------------------------------------------------
#nullable enable
#pragma warning disable IDE0005 // Extra using directive is ok.
using System;
using System.Collections;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Agents.AI;
using Microsoft.Agents.AI.Workflows;
using Microsoft.Agents.AI.Workflows.Declarative;
using Microsoft.Agents.AI.Workflows.Declarative.Kit;
using Microsoft.Extensions.AI;
namespace Test.WorkflowProviders;
/// <summary>
/// This class provides a factory method to create a <see cref="Workflow" /> instance.
/// </summary>
/// <remarks>
/// The workflow defined here was generated from a declarative workflow definition.
/// Declarative workflows utilize Power FX for defining conditions and expressions.
/// To learn more about Power FX, see:
/// https://learn.microsoft.com/power-platform/power-fx/formula-reference-copilot-studio
/// </remarks>
public static class WorkflowProvider
{
/// <summary>
/// The root executor for a declarative workflow.
/// </summary>
internal sealed class MyWorkflowRootExecutor<TInput>(
DeclarativeWorkflowOptions options,
Func<TInput, ChatMessage> inputTransform) :
RootExecutor<TInput>("my_workflow_Root", options, inputTransform)
where TInput : notnull
{
protected override async ValueTask ExecuteAsync(TInput message, IWorkflowContext context, CancellationToken cancellationToken)
{
}
}
/// <summary>
/// Formats a message template and sends an activity event.
/// </summary>
internal sealed class SendActivity1Executor(FormulaSession session) : ActionExecutor(id: "send_activity_1", session)
{
// <inheritdoc />
protected override async ValueTask<object?> ExecuteAsync(IWorkflowContext context, CancellationToken cancellationToken)
{
string activityText =
await context.FormatTemplateAsync(
"""
NEVER 1!
"""
);
AgentRunResponse response = new([new ChatMessage(ChatRole.Assistant, activityText)]);
await context.AddEventAsync(new AgentRunResponseEvent(this.Id, response)).ConfigureAwait(false);
return default;
}
}
public static Workflow CreateWorkflow<TInput>(
DeclarativeWorkflowOptions options,
Func<TInput, ChatMessage>? inputTransform = null)
where TInput : notnull
{
// Create root executor to initialize the workflow.
inputTransform ??= (message) => DeclarativeWorkflowBuilder.DefaultTransform(message);
MyWorkflowRootExecutor<TInput> myWorkflowRoot = new(options, inputTransform);
DelegateExecutor myWorkflow = new(id: "my_workflow", myWorkflowRoot.Session);
DelegateExecutor endAll = new(id: "end_all", myWorkflowRoot.Session);
DelegateExecutor endAllRestart = new(id: "end_all_Restart", myWorkflowRoot.Session);
SendActivity1Executor sendActivity1 = new(myWorkflowRoot.Session);
// Define the workflow builder
WorkflowBuilder builder = new(myWorkflowRoot);
// Connect executors
builder.AddEdge(myWorkflowRoot, myWorkflow);
builder.AddEdge(myWorkflow, endAll);
builder.AddEdge(endAllRestart, sendActivity1);
// Build the workflow
return builder.Build(validateOrphans: false);
}
}
@@ -5,7 +5,7 @@ trigger:
id: my_workflow
actions:
- kind: EndDialog
- kind: EndWorkflow
id: end_all
- kind: SendActivity
@@ -68,7 +68,6 @@ public static class WorkflowProvider
string? conversationId = await context.ReadStateAsync<string>(key: "ConversationId", scopeName: "System").ConfigureAwait(false);
bool autoSend = true;
string? additionalInstructions = null;
IList<ChatMessage>? inputMessages = await context.EvaluateListAsync<ChatMessage>("[UserMessage(System.LastMessageText)]").ConfigureAwait(false);
AgentRunResponse agentResponse =
@@ -77,7 +76,6 @@ public static class WorkflowProvider
agentName,
conversationId,
autoSend,
additionalInstructions,
inputMessages,
cancellationToken).ConfigureAwait(false);
@@ -66,7 +66,7 @@ public class InProcessExecutionTests
messageSent.Should().BeTrue("TurnToken should be accepted");
// Collect events
List<WorkflowEvent> events = new();
List<WorkflowEvent> events = [];
await foreach (WorkflowEvent evt in run.WatchStreamAsync())
{
events.Add(evt);