mirror of
https://github.com/microsoft/agent-framework.git
synced 2026-06-16 21:04:09 +08:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
b5e4553f05 | ||
|
|
310d1b8e10 | ||
|
|
e8d0bd9051 |
@@ -1,9 +1,12 @@
|
||||
{
|
||||
"name": "C# (.NET)",
|
||||
"image": "mcr.microsoft.com/devcontainers/dotnet",
|
||||
//"image": "mcr.microsoft.com/devcontainers/dotnet",
|
||||
// Workaround for https://github.com/devcontainers/images/issues/1752
|
||||
"build": {
|
||||
"dockerfile": "dotnet.Dockerfile"
|
||||
},
|
||||
"features": {
|
||||
"ghcr.io/devcontainers/features/azure-cli:1.2.9": {},
|
||||
"ghcr.io/devcontainers/features/docker-in-docker:2": {},
|
||||
"ghcr.io/devcontainers/features/github-cli:1": {
|
||||
"version": "2"
|
||||
},
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
FROM mcr.microsoft.com/devcontainers/universal:latest
|
||||
|
||||
# Remove Yarn repository with expired GPG key to prevent apt-get update failures
|
||||
# Tracking issue: https://github.com/devcontainers/images/issues/1752
|
||||
RUN rm -f /etc/apt/sources.list.d/yarn.list
|
||||
@@ -20,7 +20,6 @@ ignorePatterns:
|
||||
- pattern: "https://your-resource.openai.azure.com/"
|
||||
- pattern: "http://host.docker.internal"
|
||||
- pattern: "https://openai.github.io/openai-agents-js/openai/agents/classes/"
|
||||
- pattern: "https:\/\/dotnet.microsoft.com\/download"
|
||||
# excludedDirs:
|
||||
# Folders which include links to localhost, since it's not ignored with regular expressions
|
||||
baseUrl: https://github.com/microsoft/agent-framework/
|
||||
|
||||
@@ -8,10 +8,6 @@ inputs:
|
||||
os:
|
||||
description: The operating system to set up
|
||||
required: true
|
||||
exclude-packages:
|
||||
description: Space-separated list of packages to exclude from uv sync
|
||||
required: false
|
||||
default: ''
|
||||
|
||||
runs:
|
||||
using: "composite"
|
||||
@@ -23,20 +19,6 @@ runs:
|
||||
enable-cache: true
|
||||
cache-suffix: ${{ inputs.os }}-${{ inputs.python-version }}
|
||||
cache-dependency-glob: "**/uv.lock"
|
||||
- name: Exclude incompatible workspace packages
|
||||
if: ${{ inputs.exclude-packages != '' }}
|
||||
shell: bash
|
||||
run: |
|
||||
for pkg in ${{ inputs.exclude-packages }}; do
|
||||
for f in python/packages/*/pyproject.toml; do
|
||||
if grep -q "name = \"$pkg\"" "$f"; then
|
||||
pkg_dir=$(dirname "$f" | sed 's|python/||')
|
||||
echo "Excluding workspace package: $pkg ($pkg_dir)"
|
||||
sed -i.bak '/\[tool\.uv\.workspace\]/a\exclude = ["'"$pkg_dir"'"]' python/pyproject.toml
|
||||
sed -i.bak '/'"$pkg"' = { workspace = true }/d' python/pyproject.toml
|
||||
fi
|
||||
done
|
||||
done
|
||||
- name: Install the project
|
||||
shell: bash
|
||||
run: |
|
||||
|
||||
@@ -1,50 +0,0 @@
|
||||
name: Sample Validation Setup
|
||||
description: Sets up the environment for sample validation (checkout, Node.js, Copilot CLI, Azure login, Python)
|
||||
|
||||
inputs:
|
||||
azure-client-id:
|
||||
description: Azure Client ID for OIDC login
|
||||
required: true
|
||||
azure-tenant-id:
|
||||
description: Azure Tenant ID for OIDC login
|
||||
required: true
|
||||
azure-subscription-id:
|
||||
description: Azure Subscription ID for OIDC login
|
||||
required: true
|
||||
python-version:
|
||||
description: The Python version to set up
|
||||
required: false
|
||||
default: "3.12"
|
||||
os:
|
||||
description: The operating system to set up
|
||||
required: false
|
||||
default: "Linux"
|
||||
|
||||
runs:
|
||||
using: "composite"
|
||||
steps:
|
||||
- name: Set up Node.js environment
|
||||
uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: 22
|
||||
|
||||
- name: Install Copilot CLI
|
||||
shell: bash
|
||||
run: npm install -g @github/copilot
|
||||
|
||||
- name: Test Copilot CLI
|
||||
shell: bash
|
||||
run: copilot -p "What can you do in one sentence?"
|
||||
|
||||
- name: Azure CLI Login
|
||||
uses: azure/login@v2
|
||||
with:
|
||||
client-id: ${{ inputs.azure-client-id }}
|
||||
tenant-id: ${{ inputs.azure-tenant-id }}
|
||||
subscription-id: ${{ inputs.azure-subscription-id }}
|
||||
|
||||
- name: Set up python and install the project
|
||||
uses: ./.github/actions/python-setup
|
||||
with:
|
||||
python-version: ${{ inputs.python-version }}
|
||||
os: ${{ inputs.os }}
|
||||
@@ -1,19 +1,69 @@
|
||||
# GitHub Copilot Instructions
|
||||
|
||||
Microsoft Agent Framework - a multi-language framework for building, orchestrating, and deploying AI agents.
|
||||
This repository contains both Python and C# code.
|
||||
All python code resides under the `python/` directory.
|
||||
All C# code resides under the `dotnet/` directory.
|
||||
|
||||
## Repository Structure
|
||||
The purpose of the code is to provide a framework for building AI agents.
|
||||
|
||||
- `python/` - Python implementation → see [python/AGENTS.md](../python/AGENTS.md)
|
||||
- `dotnet/` - C#/.NET implementation → see [dotnet/AGENTS.md](../dotnet/AGENTS.md)
|
||||
- `docs/` - Design documents and architectural decision records
|
||||
When contributing to this repository, please follow these guidelines:
|
||||
|
||||
## Architectural Decision Records (ADRs)
|
||||
## C# Code Guidelines
|
||||
|
||||
ADRs in `docs/decisions/` capture significant design decisions and their rationale. They document considered alternatives, trade-offs, and the reasoning behind choices.
|
||||
Here are some general guidelines that apply to all code.
|
||||
|
||||
**Templates:**
|
||||
- `adr-template.md` - Full template with detailed sections
|
||||
- `adr-short-template.md` - Abbreviated template for simpler decisions
|
||||
- The top of all *.cs files should have a copyright notice: `// Copyright (c) Microsoft. All rights reserved.`
|
||||
- All public methods and classes should have XML documentation comments.
|
||||
- After adding, modifying or deleting code, run `dotnet build`, and then fix any reported build errors.
|
||||
- After adding or modifying code, run `dotnet format` to automatically fix any formatting errors.
|
||||
|
||||
When proposing architectural changes, create an ADR to capture options considered and the decision rationale. See [docs/decisions/README.md](../docs/decisions/README.md) for the full process.
|
||||
### C# Sample Code Guidelines
|
||||
|
||||
Sample code is located in the `dotnet/samples` directory.
|
||||
|
||||
When adding a new sample, follow these steps:
|
||||
|
||||
- The sample should be a standalone .net project in one of the subdirectories of the samples directory.
|
||||
- The directory name should be the same as the project name.
|
||||
- The directory should contain a README.md file that explains what the sample does and how to run it.
|
||||
- The README.md file should follow the same format as other samples.
|
||||
- The csproj file should match the directory name.
|
||||
- The csproj file should be configured in the same way as other samples.
|
||||
- The project should preferably contain a single Program.cs file that contains all the sample code.
|
||||
- The sample should be added to the solution file in the samples directory.
|
||||
- The sample should be tested to ensure it works as expected.
|
||||
- A reference to the new samples should be added to the README.md file in the parent directory of the new sample.
|
||||
|
||||
The sample code should follow these guidelines:
|
||||
|
||||
- Configuration settings should be read from environment variables, e.g. `var endpoint = Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT") ?? throw new InvalidOperationException("AZURE_OPENAI_ENDPOINT is not set.");`.
|
||||
- Environment variables should use upper snake_case naming convention.
|
||||
- Secrets should not be hardcoded in the code or committed to the repository.
|
||||
- The code should be well-documented with comments explaining the purpose of each step.
|
||||
- The code should be simple and to the point, avoiding unnecessary complexity.
|
||||
- Prefer inline literals over constants for values that are not reused. For example, use `new ChatClientAgent(chatClient, instructions: "You are a helpful assistant.")` instead of defining a constant for "instructions".
|
||||
- Ensure that all private classes are sealed
|
||||
- Use the Async suffix on the name of all async methods that return a Task or ValueTask.
|
||||
- Prefer defining variables using types rather than var, to help users understand the types involved.
|
||||
- Follow the patterns in the samples in the same directories where new samples are being added.
|
||||
- The structure of the sample should be as follows:
|
||||
- The top of the Program.cs should have a copyright notice: `// Copyright (c) Microsoft. All rights reserved.`
|
||||
- Then add a comment describing what the sample is demonstrating.
|
||||
- Then add the necessary using statements.
|
||||
- Then add the main code logic.
|
||||
- Finally, add any helper methods or classes at the bottom of the file.
|
||||
|
||||
### C# Unit Test Guidelines
|
||||
|
||||
Unit tests are located in the `dotnet/tests` directory in projects with a `.UnitTests.csproj` suffix.
|
||||
|
||||
Unit tests should follow these guidelines:
|
||||
|
||||
- Use `this.` for accessing class members
|
||||
- Add Arrange, Act and Assert comments for each test
|
||||
- Ensure that all private classes, that are not subclassed, are sealed
|
||||
- Use the Async suffix on the name of all async methods
|
||||
- Use the Moq library for mocking objects where possible
|
||||
- Validate that each test actually tests the target behavior, e.g. we should not have tests that creates a mock, calls the mock and then verifies that the mock was called, without the target code being involved. We also shouldn't have tests that test language features, e.g. something that the compiler would catch anyway.
|
||||
- Avoid adding excessive comments to tests. Instead favour clear easy to understand code.
|
||||
- Follow the patterns in the unit tests in the same project or classes to which new tests are being added
|
||||
|
||||
+1
-1
@@ -23,7 +23,7 @@ workflows:
|
||||
- any-glob-to-any-file:
|
||||
- dotnet/src/Microsoft.Agents.AI.Workflows/**
|
||||
- dotnet/src/Microsoft.Agents.AI.Workflows.Declarative/**
|
||||
- dotnet/samples/03-workflows/**
|
||||
- dotnet/samples/GettingStarted/Workflow/**
|
||||
- python/packages/main/agent_framework/_workflow/**
|
||||
- python/samples/getting_started/workflow/**
|
||||
|
||||
|
||||
@@ -1,216 +0,0 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
"""Scan open issues and PRs labeled 'waiting-for-author' for stale follow-ups.
|
||||
|
||||
Team members manually add the 'waiting-for-author' label when they need a
|
||||
response from the external author. If the author hasn't replied within
|
||||
DAYS_THRESHOLD days of the last team comment, post a reminder and add the
|
||||
'requested-info' label to prevent duplicate pings.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from github import Auth, Github, GithubException
|
||||
from github.Issue import Issue
|
||||
from github.IssueComment import IssueComment
|
||||
|
||||
|
||||
PING_COMMENT = (
|
||||
"@{author}, friendly reminder — this issue is waiting on your response. "
|
||||
"Please share any updates when you get a chance. (This is an automated message.)"
|
||||
)
|
||||
TRIGGER_LABEL = "waiting-for-author"
|
||||
PINGED_LABEL = "requested-info"
|
||||
|
||||
|
||||
def get_team_members(g: Github, org: str, team_slug: str) -> set[str]:
|
||||
"""Fetch active team member usernames."""
|
||||
try:
|
||||
org_obj = g.get_organization(org)
|
||||
team = org_obj.get_team_by_slug(team_slug)
|
||||
return {m.login for m in team.get_members()}
|
||||
except GithubException as exc:
|
||||
if exc.status in (403, 404):
|
||||
print(
|
||||
f"ERROR: Failed to fetch team members for {org}/{team_slug} "
|
||||
f"(HTTP {exc.status}). Check that the token has the 'read:org' "
|
||||
f"scope and that the team slug '{team_slug}' is correct."
|
||||
)
|
||||
else:
|
||||
print(f"ERROR: Failed to fetch team members for {org}/{team_slug}: {exc}")
|
||||
sys.exit(1)
|
||||
except Exception as exc:
|
||||
print(f"ERROR: Failed to fetch team members for {org}/{team_slug}: {exc}")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def find_last_team_comment(
|
||||
comments: list[IssueComment], team_members: set[str]
|
||||
) -> IssueComment | None:
|
||||
"""Return the most recent comment from a team member, or None."""
|
||||
for comment in reversed(comments):
|
||||
if comment.user and comment.user.login in team_members:
|
||||
return comment
|
||||
return None
|
||||
|
||||
|
||||
def author_replied_after(
|
||||
comments: list[IssueComment], author: str, after: datetime
|
||||
) -> bool:
|
||||
"""Check if the issue author commented after the given timestamp."""
|
||||
for comment in comments:
|
||||
if (
|
||||
comment.user
|
||||
and comment.user.login == author
|
||||
and comment.created_at > after
|
||||
):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def should_ping(
|
||||
issue: Issue,
|
||||
team_members: set[str],
|
||||
days_threshold: int,
|
||||
now: datetime,
|
||||
) -> bool:
|
||||
"""Determine whether this issue/PR should be pinged.
|
||||
|
||||
Only issues/PRs carrying the 'waiting-for-author' label are candidates.
|
||||
"""
|
||||
author = issue.user.login
|
||||
|
||||
# Skip if the trigger label is not present
|
||||
if not any(label.name == TRIGGER_LABEL for label in issue.labels):
|
||||
return False
|
||||
# Skip if author is a team member
|
||||
if author in team_members:
|
||||
return False
|
||||
|
||||
# Skip if already pinged
|
||||
if any(label.name == PINGED_LABEL for label in issue.labels):
|
||||
return False
|
||||
|
||||
# Skip if no comments at all
|
||||
if issue.comments == 0:
|
||||
return False
|
||||
|
||||
# Fetch comments once for both lookups
|
||||
comments = list(issue.get_comments())
|
||||
|
||||
# Find last team member comment
|
||||
last_team_comment = find_last_team_comment(comments, team_members)
|
||||
if last_team_comment is None:
|
||||
return False
|
||||
|
||||
# Skip if author replied after the last team comment
|
||||
if author_replied_after(comments, author, last_team_comment.created_at):
|
||||
return False
|
||||
|
||||
# Check if enough days have passed
|
||||
days_since = (now - last_team_comment.created_at.astimezone(timezone.utc)).days
|
||||
if days_since < days_threshold:
|
||||
return False
|
||||
|
||||
return True
|
||||
|
||||
|
||||
def ping(issue: Issue, dry_run: bool) -> bool:
|
||||
"""Post a reminder comment and add the 'requested-info' label. Returns True on success."""
|
||||
author = issue.user.login
|
||||
kind = "PR" if issue.pull_request else "Issue"
|
||||
|
||||
if dry_run:
|
||||
print(f" [DRY RUN] Would ping {kind} #{issue.number} (@{author})")
|
||||
return True
|
||||
|
||||
max_retries = 3
|
||||
commented = False
|
||||
labeled = False
|
||||
for attempt in range(1, max_retries + 1):
|
||||
try:
|
||||
if not commented:
|
||||
issue.create_comment(PING_COMMENT.format(author=author))
|
||||
commented = True
|
||||
if not labeled:
|
||||
issue.add_to_labels(PINGED_LABEL)
|
||||
labeled = True
|
||||
print(f" Pinged {kind} #{issue.number} (@{author})")
|
||||
return True
|
||||
except Exception as exc:
|
||||
if attempt < max_retries:
|
||||
wait = 2 ** attempt # 2s, 4s
|
||||
print(f" WARN: Attempt {attempt}/{max_retries} failed for {kind} #{issue.number}: {exc}. Retrying in {wait}s...")
|
||||
time.sleep(wait)
|
||||
else:
|
||||
print(f" ERROR: Failed to ping {kind} #{issue.number} after {max_retries} attempts: {exc}")
|
||||
return False
|
||||
|
||||
|
||||
def main() -> None:
|
||||
token = os.environ.get("GITHUB_TOKEN")
|
||||
if not token:
|
||||
print("ERROR: GITHUB_TOKEN environment variable is required")
|
||||
sys.exit(1)
|
||||
|
||||
repository = os.environ.get("GITHUB_REPOSITORY")
|
||||
if not repository:
|
||||
print("ERROR: GITHUB_REPOSITORY environment variable is required")
|
||||
sys.exit(1)
|
||||
|
||||
team_slug = os.environ.get("TEAM_SLUG")
|
||||
if not team_slug:
|
||||
print("ERROR: TEAM_SLUG environment variable is required")
|
||||
sys.exit(1)
|
||||
|
||||
days_threshold_raw = os.environ.get("DAYS_THRESHOLD", "4")
|
||||
try:
|
||||
days_threshold = int(days_threshold_raw)
|
||||
except ValueError:
|
||||
print(f"ERROR: DAYS_THRESHOLD must be a numeric value, got '{days_threshold_raw}'")
|
||||
sys.exit(1)
|
||||
dry_run = os.environ.get("DRY_RUN", "false").lower() == "true"
|
||||
|
||||
org = repository.split("/")[0]
|
||||
|
||||
if dry_run:
|
||||
print("Running in DRY RUN mode — no comments or labels will be applied.\n")
|
||||
|
||||
g = Github(auth=Auth.Token(token))
|
||||
repo = g.get_repo(repository)
|
||||
|
||||
print(f"Fetching team members for {org}/{team_slug}...")
|
||||
team_members = get_team_members(g, org, team_slug)
|
||||
print(f"Found {len(team_members)} team members.\n")
|
||||
|
||||
now = datetime.now(timezone.utc)
|
||||
pinged = []
|
||||
failed = []
|
||||
scanned = 0
|
||||
|
||||
print(f"Scanning open issues and PRs labeled '{TRIGGER_LABEL}' (threshold: {days_threshold} days)...\n")
|
||||
|
||||
for issue in repo.get_issues(state="open", labels=[TRIGGER_LABEL]):
|
||||
scanned += 1
|
||||
|
||||
if should_ping(issue, team_members, days_threshold, now):
|
||||
if ping(issue, dry_run):
|
||||
pinged.append(issue.number)
|
||||
else:
|
||||
failed.append(issue.number)
|
||||
|
||||
print(f"\nDone. Scanned {scanned} items, pinged {len(pinged)}, failed {len(failed)}.")
|
||||
if pinged:
|
||||
print(f"Pinged: {', '.join(f'#{n}' for n in pinged)}")
|
||||
if failed:
|
||||
print(f"Failed: {', '.join(f'#{n}' for n in failed)}")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,297 +0,0 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
"""Tests for stale_issue_pr_ping.py."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import sys
|
||||
from datetime import datetime, timezone, timedelta
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
# Ensure the script directory is importable
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "scripts"))
|
||||
|
||||
from stale_issue_pr_ping import (
|
||||
PINGED_LABEL,
|
||||
PING_COMMENT,
|
||||
TRIGGER_LABEL,
|
||||
author_replied_after,
|
||||
find_last_team_comment,
|
||||
get_team_members,
|
||||
main,
|
||||
ping,
|
||||
should_ping,
|
||||
)
|
||||
|
||||
TEAM = {"alice", "bob"}
|
||||
NOW = datetime(2026, 3, 15, 12, 0, 0, tzinfo=timezone.utc)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _make_comment(login: str | None, created_at: datetime) -> MagicMock:
|
||||
"""Create a mock IssueComment."""
|
||||
c = MagicMock()
|
||||
if login is None:
|
||||
c.user = None
|
||||
else:
|
||||
c.user = MagicMock()
|
||||
c.user.login = login
|
||||
c.created_at = created_at
|
||||
return c
|
||||
|
||||
|
||||
def _make_label(name: str) -> MagicMock:
|
||||
lbl = MagicMock()
|
||||
lbl.name = name
|
||||
return lbl
|
||||
|
||||
|
||||
def _make_issue(
|
||||
author: str = "external",
|
||||
labels: list[str] | None = None,
|
||||
comment_count: int = 1,
|
||||
comments: list[MagicMock] | None = None,
|
||||
pull_request: bool = False,
|
||||
number: int = 42,
|
||||
) -> MagicMock:
|
||||
issue = MagicMock()
|
||||
issue.user = MagicMock()
|
||||
issue.user.login = author
|
||||
issue.number = number
|
||||
# Default to having the trigger label, since the API query pre-filters.
|
||||
if labels is None:
|
||||
labels = [TRIGGER_LABEL]
|
||||
issue.labels = [_make_label(n) for n in labels]
|
||||
issue.comments = comment_count
|
||||
issue.pull_request = MagicMock() if pull_request else None
|
||||
if comments is not None:
|
||||
issue.get_comments.return_value = comments
|
||||
return issue
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# find_last_team_comment
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestFindLastTeamComment:
|
||||
def test_returns_last_team_comment(self):
|
||||
c1 = _make_comment("alice", datetime(2026, 3, 1, tzinfo=timezone.utc))
|
||||
c2 = _make_comment("external", datetime(2026, 3, 2, tzinfo=timezone.utc))
|
||||
c3 = _make_comment("bob", datetime(2026, 3, 3, tzinfo=timezone.utc))
|
||||
assert find_last_team_comment([c1, c2, c3], TEAM) is c3
|
||||
|
||||
def test_returns_none_when_no_team_comments(self):
|
||||
c1 = _make_comment("external", datetime(2026, 3, 1, tzinfo=timezone.utc))
|
||||
assert find_last_team_comment([c1], TEAM) is None
|
||||
|
||||
def test_returns_none_for_empty_list(self):
|
||||
assert find_last_team_comment([], TEAM) is None
|
||||
|
||||
def test_skips_deleted_user(self):
|
||||
c1 = _make_comment(None, datetime(2026, 3, 1, tzinfo=timezone.utc))
|
||||
c2 = _make_comment("alice", datetime(2026, 3, 2, tzinfo=timezone.utc))
|
||||
assert find_last_team_comment([c1, c2], TEAM) is c2
|
||||
|
||||
def test_only_deleted_users(self):
|
||||
c1 = _make_comment(None, datetime(2026, 3, 1, tzinfo=timezone.utc))
|
||||
assert find_last_team_comment([c1], TEAM) is None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# author_replied_after
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestAuthorRepliedAfter:
|
||||
def test_author_replied(self):
|
||||
after = datetime(2026, 3, 1, tzinfo=timezone.utc)
|
||||
c1 = _make_comment("external", datetime(2026, 3, 2, tzinfo=timezone.utc))
|
||||
assert author_replied_after([c1], "external", after) is True
|
||||
|
||||
def test_author_not_replied(self):
|
||||
after = datetime(2026, 3, 5, tzinfo=timezone.utc)
|
||||
c1 = _make_comment("external", datetime(2026, 3, 2, tzinfo=timezone.utc))
|
||||
assert author_replied_after([c1], "external", after) is False
|
||||
|
||||
def test_different_user_replied(self):
|
||||
after = datetime(2026, 3, 1, tzinfo=timezone.utc)
|
||||
c1 = _make_comment("someone_else", datetime(2026, 3, 2, tzinfo=timezone.utc))
|
||||
assert author_replied_after([c1], "external", after) is False
|
||||
|
||||
def test_deleted_user_comment(self):
|
||||
after = datetime(2026, 3, 1, tzinfo=timezone.utc)
|
||||
c1 = _make_comment(None, datetime(2026, 3, 2, tzinfo=timezone.utc))
|
||||
assert author_replied_after([c1], "external", after) is False
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# should_ping
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestShouldPing:
|
||||
def test_should_ping_stale_issue(self):
|
||||
team_comment = _make_comment("alice", NOW - timedelta(days=5))
|
||||
issue = _make_issue(comments=[team_comment], comment_count=1)
|
||||
assert should_ping(issue, TEAM, 4, NOW) is True
|
||||
|
||||
def test_skip_team_member_author(self):
|
||||
issue = _make_issue(author="alice", labels=[TRIGGER_LABEL], comment_count=1)
|
||||
assert should_ping(issue, TEAM, 4, NOW) is False
|
||||
|
||||
def test_skip_already_pinged(self):
|
||||
issue = _make_issue(labels=[TRIGGER_LABEL, PINGED_LABEL], comment_count=1)
|
||||
assert should_ping(issue, TEAM, 4, NOW) is False
|
||||
|
||||
def test_skip_no_comments(self):
|
||||
issue = _make_issue(comment_count=0)
|
||||
assert should_ping(issue, TEAM, 4, NOW) is False
|
||||
|
||||
def test_skip_no_team_comment(self):
|
||||
c = _make_comment("external", NOW - timedelta(days=5))
|
||||
issue = _make_issue(comments=[c], comment_count=1)
|
||||
assert should_ping(issue, TEAM, 4, NOW) is False
|
||||
|
||||
def test_skip_author_replied(self):
|
||||
team_c = _make_comment("alice", NOW - timedelta(days=5))
|
||||
author_c = _make_comment("external", NOW - timedelta(days=3))
|
||||
issue = _make_issue(comments=[team_c, author_c], comment_count=2)
|
||||
assert should_ping(issue, TEAM, 4, NOW) is False
|
||||
|
||||
def test_skip_not_enough_days(self):
|
||||
team_comment = _make_comment("alice", NOW - timedelta(days=2))
|
||||
issue = _make_issue(comments=[team_comment], comment_count=1)
|
||||
assert should_ping(issue, TEAM, 4, NOW) is False
|
||||
|
||||
def test_aware_datetime_handled(self):
|
||||
"""Timezone-aware datetimes should not be mangled by astimezone."""
|
||||
aware_dt = (NOW - timedelta(days=5)).replace(tzinfo=timezone.utc)
|
||||
team_comment = _make_comment("alice", aware_dt)
|
||||
issue = _make_issue(comments=[team_comment], comment_count=1)
|
||||
assert should_ping(issue, TEAM, 4, NOW) is True
|
||||
|
||||
def test_naive_datetime_handled(self):
|
||||
"""Naive datetimes (pre-PyGithub 2.x) should be handled by astimezone."""
|
||||
naive_dt = (NOW - timedelta(days=5)).replace(tzinfo=None)
|
||||
team_comment = _make_comment("alice", naive_dt)
|
||||
issue = _make_issue(comments=[team_comment], comment_count=1)
|
||||
# astimezone on naive datetime treats it as local time; just verify no crash
|
||||
should_ping(issue, TEAM, 4, NOW)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# ping
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestPing:
|
||||
def test_dry_run(self, capsys):
|
||||
issue = _make_issue()
|
||||
assert ping(issue, dry_run=True) is True
|
||||
issue.create_comment.assert_not_called()
|
||||
assert "DRY RUN" in capsys.readouterr().out
|
||||
|
||||
def test_success(self, capsys):
|
||||
issue = _make_issue()
|
||||
assert ping(issue, dry_run=False) is True
|
||||
issue.create_comment.assert_called_once()
|
||||
issue.add_to_labels.assert_called_once_with(PINGED_LABEL)
|
||||
|
||||
@patch("stale_issue_pr_ping.time.sleep")
|
||||
def test_retry_on_failure(self, mock_sleep):
|
||||
issue = _make_issue()
|
||||
issue.create_comment.side_effect = [Exception("net error"), None]
|
||||
assert ping(issue, dry_run=False) is True
|
||||
assert issue.create_comment.call_count == 2
|
||||
mock_sleep.assert_called_once()
|
||||
|
||||
@patch("stale_issue_pr_ping.time.sleep")
|
||||
def test_idempotent_retry_skips_comment_on_label_failure(self, mock_sleep):
|
||||
"""If create_comment succeeds but add_to_labels fails, retry should not re-comment."""
|
||||
issue = _make_issue()
|
||||
issue.add_to_labels.side_effect = [Exception("label error"), None]
|
||||
assert ping(issue, dry_run=False) is True
|
||||
# Comment should only be created once even though there were 2 attempts
|
||||
assert issue.create_comment.call_count == 1
|
||||
assert issue.add_to_labels.call_count == 2
|
||||
|
||||
@patch("stale_issue_pr_ping.time.sleep")
|
||||
def test_all_retries_fail(self, mock_sleep):
|
||||
issue = _make_issue()
|
||||
issue.create_comment.side_effect = Exception("permanent error")
|
||||
assert ping(issue, dry_run=False) is False
|
||||
assert issue.create_comment.call_count == 3
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# get_team_members
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestGetTeamMembers:
|
||||
def test_success(self):
|
||||
g = MagicMock()
|
||||
member = MagicMock()
|
||||
member.login = "alice"
|
||||
g.get_organization.return_value.get_team_by_slug.return_value.get_members.return_value = [member]
|
||||
assert get_team_members(g, "org", "my-team") == {"alice"}
|
||||
|
||||
def test_403_error_message(self, capsys):
|
||||
from github import GithubException
|
||||
|
||||
g = MagicMock()
|
||||
g.get_organization.return_value.get_team_by_slug.side_effect = GithubException(
|
||||
403, {"message": "Forbidden"}, None
|
||||
)
|
||||
with pytest.raises(SystemExit):
|
||||
get_team_members(g, "org", "my-team")
|
||||
out = capsys.readouterr().out
|
||||
assert "read:org" in out
|
||||
assert "403" in out
|
||||
|
||||
def test_404_error_message(self, capsys):
|
||||
from github import GithubException
|
||||
|
||||
g = MagicMock()
|
||||
g.get_organization.return_value.get_team_by_slug.side_effect = GithubException(
|
||||
404, {"message": "Not Found"}, None
|
||||
)
|
||||
with pytest.raises(SystemExit):
|
||||
get_team_members(g, "org", "bad-slug")
|
||||
out = capsys.readouterr().out
|
||||
assert "read:org" in out
|
||||
assert "bad-slug" in out
|
||||
|
||||
def test_generic_error(self, capsys):
|
||||
g = MagicMock()
|
||||
g.get_organization.side_effect = RuntimeError("boom")
|
||||
with pytest.raises(SystemExit):
|
||||
get_team_members(g, "org", "team")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# main – env var validation
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestMain:
|
||||
@patch.dict(os.environ, {
|
||||
"GITHUB_TOKEN": "tok",
|
||||
"GITHUB_REPOSITORY": "org/repo",
|
||||
"TEAM_SLUG": "my-team",
|
||||
"DAYS_THRESHOLD": "abc",
|
||||
}, clear=True)
|
||||
def test_invalid_days_threshold(self, capsys):
|
||||
with pytest.raises(SystemExit):
|
||||
main()
|
||||
assert "numeric" in capsys.readouterr().out
|
||||
|
||||
@patch.dict(os.environ, {
|
||||
"GITHUB_TOKEN": "tok",
|
||||
"GITHUB_REPOSITORY": "org/repo",
|
||||
}, clear=True)
|
||||
def test_missing_team_slug(self, capsys):
|
||||
with pytest.raises(SystemExit):
|
||||
main()
|
||||
assert "TEAM_SLUG" in capsys.readouterr().out
|
||||
@@ -59,20 +59,20 @@ jobs:
|
||||
if: steps.filter.outputs.dotnet != 'true'
|
||||
run: echo "NOT dotnet file"
|
||||
|
||||
# Build the full solution (including samples) on all TFMs. No tests.
|
||||
dotnet-build:
|
||||
dotnet-build-and-test:
|
||||
needs: paths-filter
|
||||
if: needs.paths-filter.outputs.dotnetChanges == 'true'
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
- { targetFramework: "net10.0", os: "ubuntu-latest", configuration: Release }
|
||||
- { targetFramework: "net10.0", os: "ubuntu-latest", configuration: Release, integration-tests: true, environment: "integration" }
|
||||
- { targetFramework: "net9.0", os: "windows-latest", configuration: Debug }
|
||||
- { targetFramework: "net8.0", os: "ubuntu-latest", configuration: Release }
|
||||
- { targetFramework: "net472", os: "windows-latest", configuration: Release }
|
||||
- { targetFramework: "net472", os: "windows-latest", configuration: Release, integration-tests: true, environment: "integration" }
|
||||
|
||||
runs-on: ${{ matrix.os }}
|
||||
environment: ${{ matrix.environment }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
@@ -84,8 +84,18 @@ jobs:
|
||||
python
|
||||
workflow-samples
|
||||
|
||||
# Start Cosmos DB Emulator for all integration tests and only for unit tests when CosmosDB changes happened)
|
||||
- name: Start Azure Cosmos DB Emulator
|
||||
if: ${{ runner.os == 'Windows' && (needs.paths-filter.outputs.cosmosDbChanges == 'true' || (github.event_name != 'pull_request' && matrix.integration-tests)) }}
|
||||
shell: pwsh
|
||||
run: |
|
||||
Write-Host "Launching Azure Cosmos DB Emulator"
|
||||
Import-Module "$env:ProgramFiles\Azure Cosmos DB Emulator\PSModules\Microsoft.Azure.CosmosDB.Emulator"
|
||||
Start-CosmosDbEmulator -NoUI -Key "C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw=="
|
||||
echo "COSMOS_EMULATOR_AVAILABLE=true" >> $env:GITHUB_ENV
|
||||
|
||||
- name: Setup dotnet
|
||||
uses: actions/setup-dotnet@v5.2.0
|
||||
uses: actions/setup-dotnet@v5.1.0
|
||||
with:
|
||||
global-json-file: ${{ github.workspace }}/dotnet/global.json
|
||||
- name: Build dotnet solutions
|
||||
@@ -130,98 +140,25 @@ jobs:
|
||||
popd
|
||||
rm -rf "$TEMP_DIR"
|
||||
|
||||
# Build src+tests only (no samples) for a single TFM and run tests.
|
||||
dotnet-test:
|
||||
needs: paths-filter
|
||||
if: needs.paths-filter.outputs.dotnetChanges == 'true'
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
- { targetFramework: "net10.0", os: "ubuntu-latest", configuration: Release, integration-tests: true, environment: "integration" }
|
||||
- { targetFramework: "net472", os: "windows-latest", configuration: Release, integration-tests: true, environment: "integration" }
|
||||
|
||||
runs-on: ${{ matrix.os }}
|
||||
environment: ${{ matrix.environment }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
persist-credentials: false
|
||||
sparse-checkout: |
|
||||
.
|
||||
.github
|
||||
dotnet
|
||||
python
|
||||
workflow-samples
|
||||
|
||||
# Start Cosmos DB Emulator for all integration tests and only for unit tests when CosmosDB changes happened)
|
||||
- name: Start Azure Cosmos DB Emulator
|
||||
if: ${{ runner.os == 'Windows' && (needs.paths-filter.outputs.cosmosDbChanges == 'true' || (github.event_name != 'pull_request' && matrix.integration-tests)) }}
|
||||
shell: pwsh
|
||||
run: |
|
||||
Write-Host "Launching Azure Cosmos DB Emulator"
|
||||
Import-Module "$env:ProgramFiles\Azure Cosmos DB Emulator\PSModules\Microsoft.Azure.CosmosDB.Emulator"
|
||||
Start-CosmosDbEmulator -NoUI -Key "C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw=="
|
||||
echo "COSMOSDB_EMULATOR_AVAILABLE=true" >> $env:GITHUB_ENV
|
||||
|
||||
- name: Setup dotnet
|
||||
uses: actions/setup-dotnet@v5.2.0
|
||||
with:
|
||||
global-json-file: ${{ github.workspace }}/dotnet/global.json
|
||||
|
||||
- name: Generate test solution (no samples)
|
||||
shell: pwsh
|
||||
run: |
|
||||
./dotnet/eng/scripts/New-FilteredSolution.ps1 `
|
||||
-Solution dotnet/agent-framework-dotnet.slnx `
|
||||
-TargetFramework ${{ matrix.targetFramework }} `
|
||||
-Configuration ${{ matrix.configuration }} `
|
||||
-ExcludeSamples `
|
||||
-OutputPath dotnet/filtered.slnx `
|
||||
-Verbose
|
||||
|
||||
- name: Build src and tests
|
||||
shell: bash
|
||||
run: dotnet build dotnet/filtered.slnx -c ${{ matrix.configuration }} -f ${{ matrix.targetFramework }} --warnaserror
|
||||
|
||||
- name: Generate test-type filtered solutions
|
||||
shell: pwsh
|
||||
run: |
|
||||
$commonArgs = @{
|
||||
Solution = "dotnet/filtered.slnx"
|
||||
TargetFramework = "${{ matrix.targetFramework }}"
|
||||
Configuration = "${{ matrix.configuration }}"
|
||||
Verbose = $true
|
||||
}
|
||||
./dotnet/eng/scripts/New-FilteredSolution.ps1 @commonArgs `
|
||||
-TestProjectNameFilter "*UnitTests*" `
|
||||
-OutputPath dotnet/filtered-unit.slnx
|
||||
./dotnet/eng/scripts/New-FilteredSolution.ps1 @commonArgs `
|
||||
-TestProjectNameFilter "*IntegrationTests*" `
|
||||
-OutputPath dotnet/filtered-integration.slnx
|
||||
|
||||
- name: Run Unit Tests
|
||||
shell: pwsh
|
||||
working-directory: dotnet
|
||||
shell: bash
|
||||
run: |
|
||||
$coverageSettings = Join-Path $PWD "tests/coverage.runsettings"
|
||||
$coverageArgs = @()
|
||||
if ("${{ matrix.targetFramework }}" -eq "${{ env.COVERAGE_FRAMEWORK }}") {
|
||||
$coverageArgs = @(
|
||||
"--coverage",
|
||||
"--coverage-output-format", "cobertura",
|
||||
"--coverage-settings", $coverageSettings,
|
||||
"--results-directory", "../TestResults/Coverage/"
|
||||
)
|
||||
}
|
||||
export UT_PROJECTS=$(find ./dotnet -type f -name "*.UnitTests.csproj" | tr '\n' ' ')
|
||||
for project in $UT_PROJECTS; do
|
||||
# Query the project's target frameworks using MSBuild with the current configuration
|
||||
target_frameworks=$(dotnet msbuild $project -getProperty:TargetFrameworks -p:Configuration=${{ matrix.configuration }} -nologo 2>/dev/null | tr -d '\r')
|
||||
|
||||
dotnet test --solution ./filtered-unit.slnx `
|
||||
-f ${{ matrix.targetFramework }} `
|
||||
-c ${{ matrix.configuration }} `
|
||||
--no-build -v Normal `
|
||||
--report-xunit-trx `
|
||||
--ignore-exit-code 8 `
|
||||
@coverageArgs
|
||||
# Check if the project supports the target framework
|
||||
if [[ "$target_frameworks" == *"${{ matrix.targetFramework }}"* ]]; then
|
||||
if [[ "${{ matrix.targetFramework }}" == "${{ env.COVERAGE_FRAMEWORK }}" ]]; then
|
||||
dotnet test -f ${{ matrix.targetFramework }} -c ${{ matrix.configuration }} $project --no-build -v Normal --logger trx --collect:"XPlat Code Coverage" --results-directory:"TestResults/Coverage/" -- DataCollectionRunSettings.DataCollectors.DataCollector.Configuration.ExcludeByAttribute=GeneratedCodeAttribute,CompilerGeneratedAttribute,ExcludeFromCodeCoverageAttribute
|
||||
else
|
||||
dotnet test -f ${{ matrix.targetFramework }} -c ${{ matrix.configuration }} $project --no-build -v Normal --logger trx
|
||||
fi
|
||||
else
|
||||
echo "Skipping $project - does not support target framework ${{ matrix.targetFramework }} (supports: $target_frameworks)"
|
||||
fi
|
||||
done
|
||||
env:
|
||||
# Cosmos DB Emulator connection settings
|
||||
COSMOSDB_ENDPOINT: https://localhost:8081
|
||||
@@ -248,48 +185,53 @@ jobs:
|
||||
id: azure-functions-setup
|
||||
|
||||
- name: Run Integration Tests
|
||||
shell: pwsh
|
||||
working-directory: dotnet
|
||||
shell: bash
|
||||
if: github.event_name != 'pull_request' && matrix.integration-tests
|
||||
run: |
|
||||
dotnet test --solution ./filtered-integration.slnx `
|
||||
-f ${{ matrix.targetFramework }} `
|
||||
-c ${{ matrix.configuration }} `
|
||||
--no-build -v Normal `
|
||||
--report-xunit-trx `
|
||||
--ignore-exit-code 8 `
|
||||
--filter-not-trait "Category=IntegrationDisabled" `
|
||||
--parallel-algorithm aggressive `
|
||||
--max-threads 2.0x
|
||||
export INTEGRATION_TEST_PROJECTS=$(find ./dotnet -type f -name "*IntegrationTests.csproj" | tr '\n' ' ')
|
||||
for project in $INTEGRATION_TEST_PROJECTS; do
|
||||
# Query the project's target frameworks using MSBuild with the current configuration
|
||||
target_frameworks=$(dotnet msbuild $project -getProperty:TargetFrameworks -p:Configuration=${{ matrix.configuration }} -nologo 2>/dev/null | tr -d '\r')
|
||||
|
||||
# Check if the project supports the target framework
|
||||
if [[ "$target_frameworks" == *"${{ matrix.targetFramework }}"* ]]; then
|
||||
dotnet test -f ${{ matrix.targetFramework }} -c ${{ matrix.configuration }} $project --no-build -v Normal --logger trx --filter "Category!=IntegrationDisabled"
|
||||
else
|
||||
echo "Skipping $project - does not support target framework ${{ matrix.targetFramework }} (supports: $target_frameworks)"
|
||||
fi
|
||||
done
|
||||
env:
|
||||
# Cosmos DB Emulator connection settings
|
||||
COSMOSDB_ENDPOINT: https://localhost:8081
|
||||
COSMOSDB_KEY: C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==
|
||||
# OpenAI Models
|
||||
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
|
||||
OPENAI_CHAT_MODEL_NAME: ${{ vars.OPENAI_CHAT_MODEL_NAME }}
|
||||
OPENAI_REASONING_MODEL_NAME: ${{ vars.OPENAI_REASONING_MODEL_NAME }}
|
||||
OpenAI__ApiKey: ${{ secrets.OPENAI__APIKEY }}
|
||||
OpenAI__ChatModelId: ${{ vars.OPENAI__CHATMODELID }}
|
||||
OpenAI__ChatReasoningModelId: ${{ vars.OPENAI__CHATREASONINGMODELID }}
|
||||
# Azure OpenAI Models
|
||||
AZURE_OPENAI_DEPLOYMENT_NAME: ${{ vars.AZURE_OPENAI_DEPLOYMENT_NAME }}
|
||||
AZURE_OPENAI_CHAT_DEPLOYMENT_NAME: ${{ vars.AZURE_OPENAI_DEPLOYMENT_NAME }}
|
||||
AZURE_OPENAI_ENDPOINT: ${{ vars.AZURE_OPENAI_ENDPOINT }}
|
||||
AZURE_OPENAI_CHAT_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__CHATDEPLOYMENTNAME }}
|
||||
AZURE_OPENAI_ENDPOINT: ${{ vars.AZUREOPENAI__ENDPOINT }}
|
||||
# Azure AI Foundry
|
||||
AZURE_AI_PROJECT_ENDPOINT: ${{ vars.AZURE_AI_PROJECT_ENDPOINT }}
|
||||
AZURE_AI_MODEL_DEPLOYMENT_NAME: ${{ vars.AZURE_AI_MODEL_DEPLOYMENT_NAME }}
|
||||
AZURE_AI_BING_CONNECTION_ID: ${{ vars.AZURE_AI_BING_CONNECTION_ID }}
|
||||
AzureAI__Endpoint: ${{ secrets.AZUREAI__ENDPOINT }}
|
||||
AzureAI__DeploymentName: ${{ vars.AZUREAI__DEPLOYMENTNAME }}
|
||||
AzureAI__BingConnectionId: ${{ vars.AZUREAI__BINGCONECTIONID }}
|
||||
FOUNDRY_PROJECT_ENDPOINT: ${{ vars.FOUNDRY_PROJECT_ENDPOINT }}
|
||||
FOUNDRY_MEDIA_DEPLOYMENT_NAME: ${{ vars.FOUNDRY_MEDIA_DEPLOYMENT_NAME }}
|
||||
FOUNDRY_MODEL_DEPLOYMENT_NAME: ${{ vars.FOUNDRY_MODEL_DEPLOYMENT_NAME }}
|
||||
FOUNDRY_CONNECTION_GROUNDING_TOOL: ${{ vars.FOUNDRY_CONNECTION_GROUNDING_TOOL }}
|
||||
|
||||
# Generate test reports and check coverage
|
||||
- name: Generate test reports
|
||||
if: matrix.targetFramework == env.COVERAGE_FRAMEWORK
|
||||
uses: danielpalme/ReportGenerator-GitHub-Action@5.5.3
|
||||
uses: danielpalme/ReportGenerator-GitHub-Action@5.5.1
|
||||
with:
|
||||
reports: "./TestResults/Coverage/**/*.cobertura.xml"
|
||||
reports: "./TestResults/Coverage/**/coverage.cobertura.xml"
|
||||
targetdir: "./TestResults/Reports"
|
||||
reporttypes: "HtmlInline;JsonSummary"
|
||||
|
||||
- name: Upload coverage report artifact
|
||||
if: matrix.targetFramework == env.COVERAGE_FRAMEWORK
|
||||
uses: actions/upload-artifact@v7
|
||||
uses: actions/upload-artifact@v6
|
||||
with:
|
||||
name: CoverageReport-${{ matrix.os }}-${{ matrix.targetFramework }}-${{ matrix.configuration }} # Artifact name
|
||||
path: ./TestResults/Reports # Directory containing files to upload
|
||||
@@ -297,13 +239,13 @@ jobs:
|
||||
- name: Check coverage
|
||||
if: matrix.targetFramework == env.COVERAGE_FRAMEWORK
|
||||
shell: pwsh
|
||||
run: ./dotnet/eng/scripts/dotnet-check-coverage.ps1 -JsonReportPath "TestResults/Reports/Summary.json" -CoverageThreshold $env:COVERAGE_THRESHOLD
|
||||
run: .github/workflows/dotnet-check-coverage.ps1 -JsonReportPath "TestResults/Reports/Summary.json" -CoverageThreshold $env:COVERAGE_THRESHOLD
|
||||
|
||||
# This final job is required to satisfy the merge queue. It must only run (or succeed) if no tests failed
|
||||
dotnet-build-and-test-check:
|
||||
if: always()
|
||||
runs-on: ubuntu-latest
|
||||
needs: [dotnet-build, dotnet-test]
|
||||
needs: [dotnet-build-and-test]
|
||||
steps:
|
||||
- name: Get Date
|
||||
shell: bash
|
||||
|
||||
@@ -86,10 +86,11 @@ jobs:
|
||||
run: docker pull mcr.microsoft.com/dotnet/sdk:${{ matrix.dotnet }}
|
||||
|
||||
# This step will run dotnet format on each of the unique csproj files and fail if any changes are made
|
||||
# exclude-diagnostics should be removed after fixes for IL2026 and IL3050 are out: https://github.com/dotnet/sdk/issues/51136
|
||||
- name: Run dotnet format
|
||||
if: steps.find-csproj.outputs.csproj_files != ''
|
||||
run: |
|
||||
for csproj in ${{ steps.find-csproj.outputs.csproj_files }}; do
|
||||
echo "Running dotnet format on $csproj"
|
||||
docker run --rm -v $(pwd):/app -w /app mcr.microsoft.com/dotnet/sdk:${{ matrix.dotnet }} /bin/sh -c "dotnet format $csproj --verify-no-changes --verbosity diagnostic"
|
||||
docker run --rm -v $(pwd):/app -w /app mcr.microsoft.com/dotnet/sdk:${{ matrix.dotnet }} /bin/sh -c "dotnet format $csproj --verify-no-changes --verbosity diagnostic --exclude-diagnostics IL2026 IL3050"
|
||||
done
|
||||
|
||||
@@ -1,102 +0,0 @@
|
||||
#
|
||||
# Dedicated .NET integration tests workflow, called from the manual integration test orchestrator.
|
||||
# Only runs integration test matrix entries (net10.0 and net472).
|
||||
#
|
||||
|
||||
name: dotnet-integration-tests
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
checkout-ref:
|
||||
description: "Git ref to checkout (e.g., refs/pull/123/head)"
|
||||
required: true
|
||||
type: string
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
|
||||
jobs:
|
||||
dotnet-integration-tests:
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
- { targetFramework: "net10.0", os: "ubuntu-latest", configuration: Release }
|
||||
- { targetFramework: "net472", os: "windows-latest", configuration: Release }
|
||||
runs-on: ${{ matrix.os }}
|
||||
environment: integration
|
||||
timeout-minutes: 60
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
ref: ${{ inputs.checkout-ref }}
|
||||
persist-credentials: false
|
||||
sparse-checkout: |
|
||||
.
|
||||
.github
|
||||
dotnet
|
||||
python
|
||||
workflow-samples
|
||||
|
||||
- name: Start Azure Cosmos DB Emulator
|
||||
if: runner.os == 'Windows'
|
||||
shell: pwsh
|
||||
run: |
|
||||
Write-Host "Launching Azure Cosmos DB Emulator"
|
||||
Import-Module "$env:ProgramFiles\Azure Cosmos DB Emulator\PSModules\Microsoft.Azure.CosmosDB.Emulator"
|
||||
Start-CosmosDbEmulator -NoUI -Key "C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw=="
|
||||
echo "COSMOS_EMULATOR_AVAILABLE=true" >> $env:GITHUB_ENV
|
||||
|
||||
- name: Setup dotnet
|
||||
uses: actions/setup-dotnet@v5.2.0
|
||||
with:
|
||||
global-json-file: ${{ github.workspace }}/dotnet/global.json
|
||||
|
||||
- name: Build dotnet solutions
|
||||
shell: bash
|
||||
run: |
|
||||
export SOLUTIONS=$(find ./dotnet/ -type f -name "*.slnx" | tr '\n' ' ')
|
||||
for solution in $SOLUTIONS; do
|
||||
dotnet build $solution -c ${{ matrix.configuration }} --warnaserror
|
||||
done
|
||||
|
||||
- name: Azure CLI Login
|
||||
uses: azure/login@v2
|
||||
with:
|
||||
client-id: ${{ secrets.AZURE_CLIENT_ID }}
|
||||
tenant-id: ${{ secrets.AZURE_TENANT_ID }}
|
||||
subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
|
||||
|
||||
- name: Set up Durable Task and Azure Functions Integration Test Emulators
|
||||
if: matrix.os == 'ubuntu-latest'
|
||||
uses: ./.github/actions/azure-functions-integration-setup
|
||||
|
||||
- name: Run Integration Tests
|
||||
shell: bash
|
||||
run: |
|
||||
export INTEGRATION_TEST_PROJECTS=$(find ./dotnet -type f -name "*IntegrationTests.csproj" | tr '\n' ' ')
|
||||
for project in $INTEGRATION_TEST_PROJECTS; do
|
||||
target_frameworks=$(dotnet msbuild $project -getProperty:TargetFrameworks -p:Configuration=${{ matrix.configuration }} -nologo 2>/dev/null | tr -d '\r')
|
||||
if [[ "$target_frameworks" == *"${{ matrix.targetFramework }}"* ]]; then
|
||||
dotnet test -f ${{ matrix.targetFramework }} -c ${{ matrix.configuration }} $project --no-build -v Normal --logger trx --filter "Category!=IntegrationDisabled"
|
||||
else
|
||||
echo "Skipping $project - does not support target framework ${{ matrix.targetFramework }} (supports: $target_frameworks)"
|
||||
fi
|
||||
done
|
||||
env:
|
||||
COSMOSDB_ENDPOINT: https://localhost:8081
|
||||
COSMOSDB_KEY: C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==
|
||||
OpenAI__ApiKey: ${{ secrets.OPENAI__APIKEY }}
|
||||
OpenAI__ChatModelId: ${{ vars.OPENAI__CHATMODELID }}
|
||||
OpenAI__ChatReasoningModelId: ${{ vars.OPENAI__CHATREASONINGMODELID }}
|
||||
AZURE_OPENAI_CHAT_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__CHATDEPLOYMENTNAME }}
|
||||
AZURE_OPENAI_ENDPOINT: ${{ vars.AZUREOPENAI__ENDPOINT }}
|
||||
AzureAI__Endpoint: ${{ secrets.AZUREAI__ENDPOINT }}
|
||||
AzureAI__DeploymentName: ${{ vars.AZUREAI__DEPLOYMENTNAME }}
|
||||
AzureAI__BingConnectionId: ${{ vars.AZUREAI__BINGCONECTIONID }}
|
||||
FOUNDRY_PROJECT_ENDPOINT: ${{ vars.FOUNDRY_PROJECT_ENDPOINT }}
|
||||
FOUNDRY_MEDIA_DEPLOYMENT_NAME: ${{ vars.FOUNDRY_MEDIA_DEPLOYMENT_NAME }}
|
||||
FOUNDRY_MODEL_DEPLOYMENT_NAME: ${{ vars.FOUNDRY_MODEL_DEPLOYMENT_NAME }}
|
||||
FOUNDRY_CONNECTION_GROUNDING_TOOL: ${{ vars.FOUNDRY_CONNECTION_GROUNDING_TOOL }}
|
||||
@@ -1,134 +0,0 @@
|
||||
#
|
||||
# This workflow allows manually running integration tests against an open PR or a branch.
|
||||
# Go to Actions → "Integration Tests (Manual)" → Run workflow → enter a PR number or branch name.
|
||||
#
|
||||
# It calls dedicated integration-only workflows (dotnet-integration-tests and python-integration-tests),
|
||||
# passing a ref so they check out and test the correct code.
|
||||
# Changed paths are detected here so only the relevant test suites run.
|
||||
#
|
||||
|
||||
name: Integration Tests (Manual)
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
pr-number:
|
||||
description: "PR number to run integration tests against (leave empty if using branch)"
|
||||
required: false
|
||||
type: string
|
||||
default: ""
|
||||
branch:
|
||||
description: "Branch name to run integration tests against (leave empty if using PR number)"
|
||||
required: false
|
||||
type: string
|
||||
default: ""
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: read
|
||||
id-token: write
|
||||
|
||||
concurrency:
|
||||
group: integration-tests-manual-${{ github.event.inputs.pr-number || github.event.inputs.branch }}
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
resolve-ref:
|
||||
name: Resolve ref
|
||||
runs-on: ubuntu-latest
|
||||
outputs:
|
||||
checkout-ref: ${{ steps.resolve.outputs.checkout-ref }}
|
||||
dotnet-changes: ${{ steps.detect-changes.outputs.dotnet }}
|
||||
python-changes: ${{ steps.detect-changes.outputs.python }}
|
||||
steps:
|
||||
- name: Resolve checkout ref
|
||||
id: resolve
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
PR_NUMBER: ${{ github.event.inputs.pr-number }}
|
||||
BRANCH: ${{ github.event.inputs.branch }}
|
||||
REPO: ${{ github.repository }}
|
||||
run: |
|
||||
if [ -n "$PR_NUMBER" ] && [ -n "$BRANCH" ]; then
|
||||
echo "::error::Please provide either a PR number or a branch name, not both."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ -z "$PR_NUMBER" ] && [ -z "$BRANCH" ]; then
|
||||
echo "::error::Please provide either a PR number or a branch name."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ -n "$PR_NUMBER" ]; then
|
||||
if ! echo "$PR_NUMBER" | grep -Eq '^[0-9]+$'; then
|
||||
echo "::error::Invalid PR number. Only numeric values are allowed."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
PR_DATA=$(gh pr view "$PR_NUMBER" --repo "$REPO" --json state)
|
||||
PR_STATE=$(echo "$PR_DATA" | jq -r '.state')
|
||||
|
||||
if [ "$PR_STATE" != "OPEN" ]; then
|
||||
echo "::error::PR #$PR_NUMBER is not open (state: $PR_STATE)"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "checkout-ref=refs/pull/$PR_NUMBER/head" >> "$GITHUB_OUTPUT"
|
||||
echo "Running integration tests for PR #$PR_NUMBER"
|
||||
else
|
||||
if ! echo "$BRANCH" | grep -Eq '^[a-zA-Z0-9_./-]+$'; then
|
||||
echo "::error::Invalid branch name. Only alphanumeric characters, hyphens, underscores, dots, and slashes are allowed."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "checkout-ref=$BRANCH" >> "$GITHUB_OUTPUT"
|
||||
echo "Running integration tests for branch $BRANCH"
|
||||
fi
|
||||
|
||||
- name: Detect changed paths
|
||||
id: detect-changes
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
PR_NUMBER: ${{ github.event.inputs.pr-number }}
|
||||
BRANCH: ${{ github.event.inputs.branch }}
|
||||
REPO: ${{ github.repository }}
|
||||
run: |
|
||||
if [ -n "$PR_NUMBER" ]; then
|
||||
CHANGED_FILES=$(gh pr diff "$PR_NUMBER" --repo "$REPO" --name-only)
|
||||
else
|
||||
# For branches, compare against main using the GitHub API
|
||||
CHANGED_FILES=$(gh api "repos/$REPO/compare/main...$BRANCH" --jq '.files[].filename')
|
||||
fi
|
||||
|
||||
DOTNET_CHANGES=false
|
||||
PYTHON_CHANGES=false
|
||||
|
||||
if echo "$CHANGED_FILES" | grep -q '^dotnet/'; then
|
||||
DOTNET_CHANGES=true
|
||||
fi
|
||||
|
||||
if echo "$CHANGED_FILES" | grep -q '^python/'; then
|
||||
PYTHON_CHANGES=true
|
||||
fi
|
||||
|
||||
echo "dotnet=$DOTNET_CHANGES" >> "$GITHUB_OUTPUT"
|
||||
echo "python=$PYTHON_CHANGES" >> "$GITHUB_OUTPUT"
|
||||
echo "Detected changes — dotnet: $DOTNET_CHANGES, python: $PYTHON_CHANGES"
|
||||
|
||||
dotnet-integration-tests:
|
||||
name: .NET Integration Tests
|
||||
needs: resolve-ref
|
||||
if: needs.resolve-ref.outputs.dotnet-changes == 'true'
|
||||
uses: ./.github/workflows/dotnet-integration-tests.yml
|
||||
with:
|
||||
checkout-ref: ${{ needs.resolve-ref.outputs.checkout-ref }}
|
||||
secrets: inherit
|
||||
|
||||
python-integration-tests:
|
||||
name: Python Integration Tests
|
||||
needs: resolve-ref
|
||||
if: needs.resolve-ref.outputs.python-changes == 'true'
|
||||
uses: ./.github/workflows/python-integration-tests.yml
|
||||
with:
|
||||
checkout-ref: ${{ needs.resolve-ref.outputs.checkout-ref }}
|
||||
secrets: inherit
|
||||
@@ -29,7 +29,4 @@ jobs:
|
||||
token: ${{ secrets.GITHUB_TOKEN }}
|
||||
timeout: 3600
|
||||
interval: 30
|
||||
# "Cleanup artifacts", "Agent", "Prepare", and "Upload results" are check runs
|
||||
# created by an org-level GitHub App (MSDO), not by any workflow in this repo.
|
||||
# They are outside our control and their transient failures should not block merges.
|
||||
ignored: CodeQL,CodeQL analysis (csharp),Cleanup artifacts,Agent,Prepare,Upload results
|
||||
ignored: CodeQL,CodeQL analysis (csharp)
|
||||
|
||||
@@ -1,13 +1,10 @@
|
||||
#!/usr/bin/env python3
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
"""Check Python test coverage against threshold for enforced targets.
|
||||
"""Check Python test coverage against threshold for enforced modules.
|
||||
|
||||
This script parses a Cobertura XML coverage report and enforces a minimum
|
||||
coverage threshold on specific targets. Targets can be package names
|
||||
(e.g., "packages.core.agent_framework") or individual Python file paths
|
||||
(e.g., "packages/core/agent_framework/observability.py").
|
||||
|
||||
Non-enforced targets are reported for visibility but don't block the build.
|
||||
coverage threshold on specific modules. Non-enforced modules are reported
|
||||
for visibility but don't block the build.
|
||||
|
||||
Usage:
|
||||
python python-check-coverage.py <coverage-xml-path> <threshold>
|
||||
@@ -21,31 +18,22 @@ import xml.etree.ElementTree as ET
|
||||
from dataclasses import dataclass
|
||||
|
||||
# =============================================================================
|
||||
# ENFORCED TARGETS CONFIGURATION
|
||||
# ENFORCED MODULES CONFIGURATION
|
||||
# =============================================================================
|
||||
# Add or remove entries from this set to control which targets must meet
|
||||
# the coverage threshold. Only these targets will fail the build if below
|
||||
# threshold. Other targets are reported for visibility only.
|
||||
# Add or remove modules from this set to control which packages must meet
|
||||
# the coverage threshold. Only these modules will fail the build if below
|
||||
# threshold. Other modules are reported for visibility only.
|
||||
#
|
||||
# Target values can be:
|
||||
# - Package paths as they appear in the coverage report
|
||||
# (e.g., "packages.azure-ai.agent_framework_azure_ai")
|
||||
# - Python source file paths as they appear in the coverage report
|
||||
# (e.g., "packages/core/agent_framework/observability.py")
|
||||
# Module paths should match the package paths as they appear in the coverage
|
||||
# report (e.g., "packages.azure-ai.agent_framework_azure_ai" for packages/azure-ai).
|
||||
# Sub-modules can be included by specifying their full path.
|
||||
# =============================================================================
|
||||
ENFORCED_TARGETS: set[str] = {
|
||||
# Packages
|
||||
ENFORCED_MODULES: set[str] = {
|
||||
"packages.azure-ai.agent_framework_azure_ai",
|
||||
"packages.core.agent_framework",
|
||||
"packages.core.agent_framework._workflows",
|
||||
"packages.purview.agent_framework_purview",
|
||||
"packages.anthropic.agent_framework_anthropic",
|
||||
"packages.azure-ai-search.agent_framework_azure_ai_search",
|
||||
"packages.core.agent_framework.azure",
|
||||
"packages.core.agent_framework.openai",
|
||||
# Individual files (if you want to enforce specific files instead of whole packages)
|
||||
"packages/core/agent_framework/observability.py",
|
||||
# Add more targets here as coverage improves
|
||||
# Add more modules here as coverage improves:
|
||||
# "packages.core.agent_framework",
|
||||
# "packages.core.agent_framework._workflows",
|
||||
# "packages.anthropic.agent_framework_anthropic",
|
||||
}
|
||||
|
||||
|
||||
@@ -72,21 +60,14 @@ class PackageCoverage:
|
||||
return self.branch_rate * 100
|
||||
|
||||
|
||||
def normalize_coverage_path(path: str) -> str:
|
||||
"""Normalize coverage paths for reliable matching."""
|
||||
return path.replace("\\", "/").lstrip("./")
|
||||
|
||||
|
||||
def parse_coverage_xml(
|
||||
xml_path: str,
|
||||
) -> tuple[dict[str, PackageCoverage], dict[str, PackageCoverage], float, float]:
|
||||
def parse_coverage_xml(xml_path: str) -> tuple[dict[str, PackageCoverage], float, float]:
|
||||
"""Parse Cobertura XML and extract per-package coverage data.
|
||||
|
||||
Args:
|
||||
xml_path: Path to the Cobertura XML coverage report.
|
||||
|
||||
Returns:
|
||||
A tuple of (packages_dict, files_dict, overall_line_rate, overall_branch_rate).
|
||||
A tuple of (packages_dict, overall_line_rate, overall_branch_rate).
|
||||
"""
|
||||
tree = ET.parse(xml_path)
|
||||
root = tree.getroot()
|
||||
@@ -96,7 +77,6 @@ def parse_coverage_xml(
|
||||
overall_branch_rate = float(root.get("branch-rate", 0))
|
||||
|
||||
packages: dict[str, PackageCoverage] = {}
|
||||
file_stats: dict[str, dict[str, int]] = {}
|
||||
|
||||
for package in root.findall(".//package"):
|
||||
package_path = package.get("name", "unknown")
|
||||
@@ -111,43 +91,19 @@ def parse_coverage_xml(
|
||||
branches_covered = 0
|
||||
|
||||
for class_elem in package.findall(".//class"):
|
||||
file_path = normalize_coverage_path(class_elem.get("filename", ""))
|
||||
if file_path and file_path not in file_stats:
|
||||
file_stats[file_path] = {
|
||||
"lines_valid": 0,
|
||||
"lines_covered": 0,
|
||||
"branches_valid": 0,
|
||||
"branches_covered": 0,
|
||||
}
|
||||
|
||||
for line in class_elem.findall(".//line"):
|
||||
lines_valid += 1
|
||||
if int(line.get("hits", 0)) > 0:
|
||||
lines_covered += 1
|
||||
|
||||
if file_path:
|
||||
file_stats[file_path]["lines_valid"] += 1
|
||||
if int(line.get("hits", 0)) > 0:
|
||||
file_stats[file_path]["lines_covered"] += 1
|
||||
|
||||
# Branch coverage from line elements
|
||||
if line.get("branch") == "true":
|
||||
condition_coverage = line.get("condition-coverage", "")
|
||||
if condition_coverage:
|
||||
# Parse "X% (covered/total)" format
|
||||
try:
|
||||
coverage_parts = (
|
||||
condition_coverage.split("(")[1].rstrip(")").split("/")
|
||||
)
|
||||
coverage_parts = condition_coverage.split("(")[1].rstrip(")").split("/")
|
||||
branches_covered += int(coverage_parts[0])
|
||||
branches_valid += int(coverage_parts[1])
|
||||
if file_path:
|
||||
file_stats[file_path]["branches_covered"] += int(
|
||||
coverage_parts[0]
|
||||
)
|
||||
file_stats[file_path]["branches_valid"] += int(
|
||||
coverage_parts[1]
|
||||
)
|
||||
except (IndexError, ValueError):
|
||||
# Ignore malformed condition-coverage strings; treat this line as having no branch data.
|
||||
pass
|
||||
@@ -156,33 +112,14 @@ def parse_coverage_xml(
|
||||
packages[package_path] = PackageCoverage(
|
||||
name=package_path,
|
||||
line_rate=line_rate if lines_valid == 0 else lines_covered / lines_valid,
|
||||
branch_rate=branch_rate
|
||||
if branches_valid == 0
|
||||
else branches_covered / branches_valid,
|
||||
branch_rate=branch_rate if branches_valid == 0 else branches_covered / branches_valid,
|
||||
lines_valid=lines_valid,
|
||||
lines_covered=lines_covered,
|
||||
branches_valid=branches_valid,
|
||||
branches_covered=branches_covered,
|
||||
)
|
||||
|
||||
files: dict[str, PackageCoverage] = {}
|
||||
for file_path, stats in file_stats.items():
|
||||
lines_valid = stats["lines_valid"]
|
||||
lines_covered = stats["lines_covered"]
|
||||
branches_valid = stats["branches_valid"]
|
||||
branches_covered = stats["branches_covered"]
|
||||
|
||||
files[file_path] = PackageCoverage(
|
||||
name=file_path,
|
||||
line_rate=0 if lines_valid == 0 else lines_covered / lines_valid,
|
||||
branch_rate=0 if branches_valid == 0 else branches_covered / branches_valid,
|
||||
lines_valid=lines_valid,
|
||||
lines_covered=lines_covered,
|
||||
branches_valid=branches_valid,
|
||||
branches_covered=branches_covered,
|
||||
)
|
||||
|
||||
return packages, files, overall_line_rate, overall_branch_rate
|
||||
return packages, overall_line_rate, overall_branch_rate
|
||||
|
||||
|
||||
def format_coverage_value(coverage: float, threshold: float, is_enforced: bool) -> str:
|
||||
@@ -191,7 +128,7 @@ def format_coverage_value(coverage: float, threshold: float, is_enforced: bool)
|
||||
Args:
|
||||
coverage: Coverage percentage (0-100).
|
||||
threshold: Minimum required coverage percentage.
|
||||
is_enforced: Whether this target is enforced.
|
||||
is_enforced: Whether this module is enforced.
|
||||
|
||||
Returns:
|
||||
Formatted string like "85.5%" or "85.5% ✅" or "75.0% ❌".
|
||||
@@ -205,7 +142,6 @@ def format_coverage_value(coverage: float, threshold: float, is_enforced: bool)
|
||||
|
||||
def print_coverage_table(
|
||||
packages: dict[str, PackageCoverage],
|
||||
files: dict[str, PackageCoverage],
|
||||
threshold: float,
|
||||
overall_line_rate: float,
|
||||
overall_branch_rate: float,
|
||||
@@ -214,7 +150,6 @@ def print_coverage_table(
|
||||
|
||||
Args:
|
||||
packages: Dictionary of package name to coverage data.
|
||||
files: Dictionary of file path to coverage data, used for per-file enforcement.
|
||||
threshold: Minimum required coverage percentage.
|
||||
overall_line_rate: Overall line coverage rate (0-1).
|
||||
overall_branch_rate: Overall branch coverage rate (0-1).
|
||||
@@ -228,25 +163,21 @@ def print_coverage_table(
|
||||
print(f"Overall Branch Coverage: {overall_branch_rate * 100:.1f}%")
|
||||
print(f"Threshold: {threshold}%")
|
||||
|
||||
enforced_targets = {normalize_coverage_path(t) for t in ENFORCED_TARGETS}
|
||||
|
||||
# Package table
|
||||
print("\n" + "-" * 110)
|
||||
print(f"{'Package':<80} {'Lines':<15} {'Line Cov':<15}")
|
||||
print("-" * 110)
|
||||
|
||||
# Sort: enforced package targets first, then alphabetically
|
||||
# Sort: enforced modules first, then alphabetically
|
||||
sorted_packages = sorted(
|
||||
packages.values(),
|
||||
key=lambda p: (p.name not in ENFORCED_TARGETS, p.name),
|
||||
key=lambda p: (p.name not in ENFORCED_MODULES, p.name),
|
||||
)
|
||||
|
||||
for pkg in sorted_packages:
|
||||
is_enforced = normalize_coverage_path(pkg.name) in enforced_targets
|
||||
is_enforced = pkg.name in ENFORCED_MODULES
|
||||
enforced_marker = "[ENFORCED] " if is_enforced else ""
|
||||
line_cov = format_coverage_value(
|
||||
pkg.line_coverage_percent, threshold, is_enforced
|
||||
)
|
||||
line_cov = format_coverage_value(pkg.line_coverage_percent, threshold, is_enforced)
|
||||
lines_info = f"{pkg.lines_covered}/{pkg.lines_valid}"
|
||||
package_label = f"{enforced_marker}{pkg.name}"
|
||||
|
||||
@@ -254,98 +185,50 @@ def print_coverage_table(
|
||||
|
||||
print("-" * 110)
|
||||
|
||||
# Enforced file/model entries (if configured)
|
||||
enforced_files = [
|
||||
files[target]
|
||||
for target in sorted(enforced_targets)
|
||||
if target in files and target.endswith(".py")
|
||||
]
|
||||
|
||||
if enforced_files:
|
||||
print("\nEnforced Files/Models")
|
||||
print("-" * 110)
|
||||
print(f"{'File':<80} {'Lines':<15} {'Line Cov':<15}")
|
||||
print("-" * 110)
|
||||
|
||||
for file_cov in enforced_files:
|
||||
line_cov = format_coverage_value(
|
||||
file_cov.line_coverage_percent, threshold, True
|
||||
)
|
||||
lines_info = f"{file_cov.lines_covered}/{file_cov.lines_valid}"
|
||||
print(f"[ENFORCED] {file_cov.name:<69} {lines_info:<15} {line_cov:<15}")
|
||||
|
||||
print("-" * 110)
|
||||
|
||||
|
||||
def check_coverage(xml_path: str, threshold: float) -> bool:
|
||||
"""Check if all enforced targets meet the coverage threshold.
|
||||
"""Check if all enforced modules meet the coverage threshold.
|
||||
|
||||
Args:
|
||||
xml_path: Path to the Cobertura XML coverage report.
|
||||
threshold: Minimum required coverage percentage.
|
||||
|
||||
Returns:
|
||||
True if all enforced targets pass, False otherwise.
|
||||
True if all enforced modules pass, False otherwise.
|
||||
"""
|
||||
packages, files, overall_line_rate, overall_branch_rate = parse_coverage_xml(
|
||||
xml_path
|
||||
)
|
||||
packages, overall_line_rate, overall_branch_rate = parse_coverage_xml(xml_path)
|
||||
|
||||
print_coverage_table(
|
||||
packages, files, threshold, overall_line_rate, overall_branch_rate
|
||||
)
|
||||
print_coverage_table(packages, threshold, overall_line_rate, overall_branch_rate)
|
||||
|
||||
# Check enforced targets
|
||||
failed_targets: list[str] = []
|
||||
missing_targets: list[str] = []
|
||||
# Check enforced modules
|
||||
failed_modules: list[str] = []
|
||||
missing_modules: list[str] = []
|
||||
|
||||
for target_name in ENFORCED_TARGETS:
|
||||
normalized_target = normalize_coverage_path(target_name)
|
||||
package_alias = normalized_target.replace("/", ".")
|
||||
|
||||
target_coverage = None
|
||||
if target_name in packages:
|
||||
target_coverage = packages[target_name]
|
||||
elif normalized_target in files:
|
||||
target_coverage = files[normalized_target]
|
||||
elif package_alias in packages:
|
||||
target_coverage = packages[package_alias]
|
||||
|
||||
if target_coverage is None:
|
||||
missing_targets.append(target_name)
|
||||
for module_name in ENFORCED_MODULES:
|
||||
if module_name not in packages:
|
||||
missing_modules.append(module_name)
|
||||
continue
|
||||
|
||||
if target_coverage.line_coverage_percent < threshold:
|
||||
failed_targets.append(
|
||||
f"{target_name} ({target_coverage.line_coverage_percent:.1f}%)"
|
||||
)
|
||||
pkg = packages[module_name]
|
||||
if pkg.line_coverage_percent < threshold:
|
||||
failed_modules.append(f"{module_name} ({pkg.line_coverage_percent:.1f}%)")
|
||||
|
||||
# Report results
|
||||
if missing_targets:
|
||||
print(
|
||||
f"\n❌ FAILED: Enforced targets not found in coverage report: {', '.join(missing_targets)}"
|
||||
)
|
||||
if missing_modules:
|
||||
print(f"\n❌ FAILED: Enforced modules not found in coverage report: {', '.join(missing_modules)}")
|
||||
return False
|
||||
|
||||
if failed_targets:
|
||||
print(
|
||||
f"\n❌ FAILED: The following enforced targets are below {threshold}% coverage threshold:"
|
||||
)
|
||||
for target in failed_targets:
|
||||
print(f" - {target}")
|
||||
print("\nTo fix: Add more tests to improve coverage for the failing targets.")
|
||||
if failed_modules:
|
||||
print(f"\n❌ FAILED: The following enforced modules are below {threshold}% coverage threshold:")
|
||||
for module in failed_modules:
|
||||
print(f" - {module}")
|
||||
print("\nTo fix: Add more tests to improve coverage for the failing modules.")
|
||||
return False
|
||||
|
||||
if ENFORCED_TARGETS:
|
||||
found_enforced = [
|
||||
target
|
||||
for target in ENFORCED_TARGETS
|
||||
if target in packages or normalize_coverage_path(target) in files
|
||||
]
|
||||
if ENFORCED_MODULES:
|
||||
found_enforced = [m for m in ENFORCED_MODULES if m in packages]
|
||||
if found_enforced:
|
||||
print(
|
||||
f"\n✅ PASSED: All enforced targets meet the {threshold}% coverage threshold."
|
||||
)
|
||||
print(f"\n✅ PASSED: All enforced modules meet the {threshold}% coverage threshold.")
|
||||
|
||||
return True
|
||||
|
||||
|
||||
@@ -12,13 +12,13 @@ env:
|
||||
UV_CACHE_DIR: /tmp/.uv-cache
|
||||
|
||||
jobs:
|
||||
pre-commit-hooks:
|
||||
name: Pre-commit Hooks
|
||||
pre-commit:
|
||||
name: Checks
|
||||
if: "!cancelled()"
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
python-version: ["3.11"]
|
||||
python-version: ["3.10", "3.14"]
|
||||
runs-on: ubuntu-latest
|
||||
continue-on-error: true
|
||||
defaults:
|
||||
@@ -37,105 +37,17 @@ jobs:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
os: ${{ runner.os }}
|
||||
env:
|
||||
# Configure a constant location for the uv cache
|
||||
UV_CACHE_DIR: /tmp/.uv-cache
|
||||
- uses: actions/cache@v5
|
||||
with:
|
||||
path: ~/.cache/prek
|
||||
key: prek|${{ matrix.python-version }}|${{ hashFiles('python/.pre-commit-config.yaml') }}
|
||||
- uses: j178/prek-action@v1
|
||||
name: Run Pre-commit Hooks (excluding poe-check)
|
||||
env:
|
||||
SKIP: poe-check
|
||||
path: ~/.cache/pre-commit
|
||||
key: pre-commit|${{ matrix.python-version }}|${{ hashFiles('python/.pre-commit-config.yaml') }}
|
||||
- uses: pre-commit/action@v3.0.1
|
||||
name: Run Pre-Commit Hooks
|
||||
with:
|
||||
extra-args: --cd python --all-files
|
||||
|
||||
package-checks:
|
||||
name: Package Checks
|
||||
if: "!cancelled()"
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
python-version: ["3.11"]
|
||||
runs-on: ubuntu-latest
|
||||
continue-on-error: true
|
||||
defaults:
|
||||
run:
|
||||
working-directory: ./python
|
||||
env:
|
||||
UV_PYTHON: ${{ matrix.python-version }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
fetch-depth: 0
|
||||
- name: Set up python and install the project
|
||||
id: python-setup
|
||||
uses: ./.github/actions/python-setup
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
os: ${{ runner.os }}
|
||||
env:
|
||||
UV_CACHE_DIR: /tmp/.uv-cache
|
||||
- name: Run syntax and pyright across packages
|
||||
run: uv run poe check-packages
|
||||
|
||||
samples-markdown:
|
||||
name: Samples & Markdown
|
||||
if: "!cancelled()"
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
python-version: ["3.11"]
|
||||
runs-on: ubuntu-latest
|
||||
continue-on-error: true
|
||||
defaults:
|
||||
run:
|
||||
working-directory: ./python
|
||||
env:
|
||||
UV_PYTHON: ${{ matrix.python-version }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
fetch-depth: 0
|
||||
- name: Set up python and install the project
|
||||
id: python-setup
|
||||
uses: ./.github/actions/python-setup
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
os: ${{ runner.os }}
|
||||
env:
|
||||
UV_CACHE_DIR: /tmp/.uv-cache
|
||||
- name: Run samples checks
|
||||
run: uv run poe check -S
|
||||
- name: Run markdown code lint
|
||||
run: uv run poe markdown-code-lint
|
||||
|
||||
mypy:
|
||||
name: Mypy Checks
|
||||
if: "!cancelled()"
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
python-version: ["3.11"]
|
||||
runs-on: ubuntu-latest
|
||||
continue-on-error: true
|
||||
defaults:
|
||||
run:
|
||||
working-directory: ./python
|
||||
env:
|
||||
UV_PYTHON: ${{ matrix.python-version }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
fetch-depth: 0
|
||||
- name: Set up python and install the project
|
||||
id: python-setup
|
||||
uses: ./.github/actions/python-setup
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
os: ${{ runner.os }}
|
||||
env:
|
||||
UV_CACHE_DIR: /tmp/.uv-cache
|
||||
extra_args: --config python/.pre-commit-config.yaml --all-files
|
||||
- name: Run Mypy
|
||||
env:
|
||||
GITHUB_BASE_REF: ${{ github.event.pull_request.base.ref || github.base_ref || 'main' }}
|
||||
run: uv run python scripts/workspace_poe_tasks.py ci-mypy
|
||||
run: uv run poe ci-mypy
|
||||
|
||||
@@ -1,216 +0,0 @@
|
||||
# Probe the highest allowed dependency versions, then open issues/PRs from the passing updates.
|
||||
name: Python - Dependency Range Validation
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: write
|
||||
issues: write
|
||||
pull-requests: write
|
||||
|
||||
env:
|
||||
UV_CACHE_DIR: /tmp/.uv-cache
|
||||
|
||||
jobs:
|
||||
dependency-range-validation:
|
||||
name: Dependency Range Validation
|
||||
runs-on: ubuntu-latest
|
||||
env:
|
||||
# For now only run 3.13, if we do encounter situations where there are mismatches between packages and python versions (other then 3.10 and 3.14 which are known to not be able to install everything)
|
||||
# then we will have to reevaluate.
|
||||
UV_PYTHON: "3.13"
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Set up python and install the project
|
||||
uses: ./.github/actions/python-setup
|
||||
with:
|
||||
python-version: ${{ env.UV_PYTHON }}
|
||||
os: ${{ runner.os }}
|
||||
env:
|
||||
UV_CACHE_DIR: /tmp/.uv-cache
|
||||
|
||||
- name: Run dependency range validation
|
||||
id: validate_ranges
|
||||
# Keep workflow running so we can still publish diagnostics from this run.
|
||||
continue-on-error: true
|
||||
run: uv run poe validate-dependency-bounds-project --mode upper --package "*"
|
||||
working-directory: ./python
|
||||
|
||||
- name: Upload dependency range report
|
||||
# Always publish the report so failures are inspectable even when validation fails.
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v7
|
||||
with:
|
||||
name: dependency-range-results
|
||||
path: python/scripts/dependencies/dependency-range-results.json
|
||||
if-no-files-found: warn
|
||||
|
||||
- name: Create issues for failed dependency candidates
|
||||
# Always process the report so failed candidates create actionable tracking issues.
|
||||
if: always()
|
||||
uses: actions/github-script@v8
|
||||
with:
|
||||
script: |
|
||||
const fs = require("fs")
|
||||
const reportPath = "python/scripts/dependencies/dependency-range-results.json"
|
||||
|
||||
if (!fs.existsSync(reportPath)) {
|
||||
core.warning(`No dependency range report found at ${reportPath}`)
|
||||
return
|
||||
}
|
||||
|
||||
const report = JSON.parse(fs.readFileSync(reportPath, "utf8"))
|
||||
const dependencyFailures = []
|
||||
|
||||
for (const packageResult of report.packages ?? []) {
|
||||
for (const dependency of packageResult.dependencies ?? []) {
|
||||
const candidateVersions = new Set(dependency.candidate_versions ?? [])
|
||||
const failedAttempts = (dependency.attempts ?? []).filter(
|
||||
(attempt) => attempt.status === "failed" && candidateVersions.has(attempt.trial_upper)
|
||||
)
|
||||
if (!failedAttempts.length) {
|
||||
continue
|
||||
}
|
||||
|
||||
const failuresByVersion = new Map()
|
||||
for (const attempt of failedAttempts) {
|
||||
const version = attempt.trial_upper || "unknown"
|
||||
if (!failuresByVersion.has(version)) {
|
||||
failuresByVersion.set(version, attempt.error || "No error output captured.")
|
||||
}
|
||||
}
|
||||
|
||||
dependencyFailures.push({
|
||||
packageName: packageResult.package_name,
|
||||
projectPath: packageResult.project_path,
|
||||
dependencyName: dependency.name,
|
||||
originalRequirements: dependency.original_requirements ?? [],
|
||||
finalRequirements: dependency.final_requirements ?? [],
|
||||
failedVersions: [...failuresByVersion.entries()].map(([version, error]) => ({ version, error })),
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
if (!dependencyFailures.length) {
|
||||
core.info("No failing dependency candidates found.")
|
||||
return
|
||||
}
|
||||
|
||||
const owner = context.repo.owner
|
||||
const repo = context.repo.repo
|
||||
const openIssues = await github.paginate(github.rest.issues.listForRepo, {
|
||||
owner,
|
||||
repo,
|
||||
state: "open",
|
||||
per_page: 100,
|
||||
})
|
||||
const openIssueTitles = new Set(
|
||||
openIssues.filter((issue) => !issue.pull_request).map((issue) => issue.title)
|
||||
)
|
||||
|
||||
const formatError = (message) => String(message || "No error output captured.").replace(/```/g, "'''")
|
||||
|
||||
for (const failure of dependencyFailures) {
|
||||
const title = `Dependency validation failed: ${failure.dependencyName} (${failure.packageName})`
|
||||
if (openIssueTitles.has(title)) {
|
||||
core.info(`Issue already exists: ${title}`)
|
||||
continue
|
||||
}
|
||||
|
||||
const visibleFailures = failure.failedVersions.slice(0, 5)
|
||||
const omittedCount = failure.failedVersions.length - visibleFailures.length
|
||||
const failureDetails = visibleFailures
|
||||
.map(
|
||||
(entry) =>
|
||||
`- \`${entry.version}\`\n\n\`\`\`\n${formatError(entry.error).slice(0, 3500)}\n\`\`\``
|
||||
)
|
||||
.join("\n\n")
|
||||
|
||||
const body = [
|
||||
"Automated dependency range validation found candidate versions that failed checks.",
|
||||
"",
|
||||
`- Package: \`${failure.packageName}\``,
|
||||
`- Project path: \`${failure.projectPath}\``,
|
||||
`- Dependency: \`${failure.dependencyName}\``,
|
||||
`- Original requirements: ${
|
||||
failure.originalRequirements.length
|
||||
? failure.originalRequirements.map((value) => `\`${value}\``).join(", ")
|
||||
: "_none_"
|
||||
}`,
|
||||
`- Final requirements after run: ${
|
||||
failure.finalRequirements.length
|
||||
? failure.finalRequirements.map((value) => `\`${value}\``).join(", ")
|
||||
: "_none_"
|
||||
}`,
|
||||
"",
|
||||
"### Failed versions and errors",
|
||||
failureDetails,
|
||||
omittedCount > 0 ? `\n_Additional failed versions omitted: ${omittedCount}_` : "",
|
||||
"",
|
||||
`Workflow run: ${context.serverUrl}/${owner}/${repo}/actions/runs/${context.runId}`,
|
||||
].join("\n")
|
||||
|
||||
await github.rest.issues.create({
|
||||
owner,
|
||||
repo,
|
||||
title,
|
||||
body,
|
||||
})
|
||||
openIssueTitles.add(title)
|
||||
core.info(`Created issue: ${title}`)
|
||||
}
|
||||
|
||||
- name: Refresh lockfile
|
||||
# Only refresh lockfile after a clean validation to avoid committing known-bad ranges.
|
||||
if: steps.validate_ranges.outcome == 'success'
|
||||
run: uv lock --upgrade
|
||||
working-directory: ./python
|
||||
|
||||
- name: Commit and push dependency updates
|
||||
id: commit_updates
|
||||
if: steps.validate_ranges.outcome == 'success'
|
||||
run: |
|
||||
BRANCH="automation/python-dependency-range-updates"
|
||||
|
||||
git config user.name "github-actions[bot]"
|
||||
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
|
||||
git checkout -B "${BRANCH}"
|
||||
|
||||
git add python/packages/*/pyproject.toml python/uv.lock
|
||||
if git diff --cached --quiet; then
|
||||
echo "has_changes=false" >> "$GITHUB_OUTPUT"
|
||||
echo "No dependency updates to commit."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
git commit -m "chore: update dependency ranges"
|
||||
git push --force-with-lease --set-upstream origin "${BRANCH}"
|
||||
echo "has_changes=true" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Create or update pull request with GitHub CLI
|
||||
# Only open/update PRs for validated updates to keep automation branches trustworthy.
|
||||
if: steps.validate_ranges.outcome == 'success' && steps.commit_updates.outputs.has_changes == 'true'
|
||||
run: |
|
||||
BRANCH="automation/python-dependency-range-updates"
|
||||
PR_TITLE="Python: chore: update dependency ranges"
|
||||
PR_BODY_FILE="$(mktemp)"
|
||||
|
||||
cat > "${PR_BODY_FILE}" <<'EOF'
|
||||
This PR was generated by the dependency range validation workflow.
|
||||
|
||||
- Ran `uv run poe validate-dependency-bounds-project --mode upper --package "*"`
|
||||
- Updated package dependency bounds
|
||||
- Refreshed `python/uv.lock` with `uv lock --upgrade`
|
||||
EOF
|
||||
|
||||
PR_NUMBER="$(gh pr list --head "${BRANCH}" --base main --state open --json number --jq '.[0].number')"
|
||||
if [ -n "${PR_NUMBER}" ]; then
|
||||
gh pr edit "${PR_NUMBER}" --title "${PR_TITLE}" --body-file "${PR_BODY_FILE}"
|
||||
else
|
||||
gh pr create --base main --head "${BRANCH}" --title "${PR_TITLE}" --body-file "${PR_BODY_FILE}"
|
||||
fi
|
||||
@@ -1,91 +0,0 @@
|
||||
name: Python - Dev Dependency Upgrade
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
|
||||
env:
|
||||
UV_CACHE_DIR: /tmp/.uv-cache
|
||||
|
||||
jobs:
|
||||
upgrade-dev-dependencies:
|
||||
name: Upgrade Dev Dependencies
|
||||
runs-on: ubuntu-latest
|
||||
env:
|
||||
UV_PYTHON: "3.13"
|
||||
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Set up python and install the project
|
||||
uses: ./.github/actions/python-setup
|
||||
with:
|
||||
python-version: ${{ env.UV_PYTHON }}
|
||||
os: ${{ runner.os }}
|
||||
env:
|
||||
UV_CACHE_DIR: /tmp/.uv-cache
|
||||
|
||||
- name: Upgrade dev dependencies and validate workspace
|
||||
run: uv run poe upgrade-dev-dependencies
|
||||
working-directory: ./python
|
||||
|
||||
- name: Commit and push dev dependency updates
|
||||
id: commit_updates
|
||||
run: |
|
||||
BRANCH="automation/python-dev-dependency-updates"
|
||||
|
||||
git config user.name "github-actions[bot]"
|
||||
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
|
||||
git checkout -B "${BRANCH}"
|
||||
|
||||
git add python/pyproject.toml python/packages/*/pyproject.toml python/uv.lock
|
||||
if git diff --cached --quiet; then
|
||||
echo "has_changes=false" >> "$GITHUB_OUTPUT"
|
||||
echo "No dev dependency updates to commit."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
git commit -F- <<'EOF'
|
||||
Python: chore: upgrade dev dependencies
|
||||
EOF
|
||||
git push --force-with-lease --set-upstream origin "${BRANCH}"
|
||||
echo "has_changes=true" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Create or update pull request with GitHub CLI
|
||||
if: steps.commit_updates.outputs.has_changes == 'true'
|
||||
run: |
|
||||
BRANCH="automation/python-dev-dependency-updates"
|
||||
PR_TITLE="Python: chore: upgrade dev dependencies"
|
||||
PR_BODY_FILE="$(mktemp)"
|
||||
|
||||
cat > "${PR_BODY_FILE}" <<'EOF'
|
||||
### Motivation and Context
|
||||
|
||||
This automated update refreshes Python dev dependency pins across the workspace and reruns the repo validation gates before opening a pull request.
|
||||
|
||||
### Description
|
||||
|
||||
- Ran `uv run poe upgrade-dev-dependencies`
|
||||
- Refreshed dev dependency pins in workspace `pyproject.toml` files
|
||||
- Refreshed `python/uv.lock` with `uv lock --upgrade`
|
||||
- Reinstalled from the frozen lockfile and reran `check`, `typing`, and `test`
|
||||
|
||||
### Contribution Checklist
|
||||
|
||||
- [x] The code builds clean without any errors or warnings
|
||||
- [x] The PR follows the [Contribution Guidelines](https://github.com/microsoft/agent-framework/blob/main/CONTRIBUTING.md)
|
||||
- [x] All unit tests pass, and I have added new tests where possible
|
||||
- [ ] **Is this a breaking change?** If yes, add "[BREAKING]" prefix to the title of the PR.
|
||||
EOF
|
||||
|
||||
PR_NUMBER="$(gh pr list --head "${BRANCH}" --base main --state open --json number --jq '.[0].number')"
|
||||
if [ -n "${PR_NUMBER}" ]; then
|
||||
gh pr edit "${PR_NUMBER}" --title "${PR_TITLE}" --body-file "${PR_BODY_FILE}"
|
||||
else
|
||||
gh pr create --base main --head "${BRANCH}" --title "${PR_TITLE}" --body-file "${PR_BODY_FILE}"
|
||||
fi
|
||||
@@ -1,318 +0,0 @@
|
||||
#
|
||||
# Dedicated Python integration tests workflow, called from the manual integration test orchestrator.
|
||||
# Runs all tests (unit + integration) split into parallel jobs by provider.
|
||||
#
|
||||
# NOTE: This workflow and python-merge-tests.yml share the same set of parallel
|
||||
# test jobs. Keep them in sync — when adding, removing, or modifying a job here,
|
||||
# apply the same change to python-merge-tests.yml.
|
||||
#
|
||||
|
||||
name: python-integration-tests
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
checkout-ref:
|
||||
description: "Git ref to checkout (e.g., refs/pull/123/head)"
|
||||
required: true
|
||||
type: string
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
|
||||
env:
|
||||
UV_CACHE_DIR: /tmp/.uv-cache
|
||||
UV_PYTHON: "3.13"
|
||||
|
||||
jobs:
|
||||
# Unit tests: all non-integration tests across all packages
|
||||
python-tests-unit:
|
||||
name: Python Integration Tests - Unit
|
||||
runs-on: ubuntu-latest
|
||||
environment: integration
|
||||
timeout-minutes: 60
|
||||
defaults:
|
||||
run:
|
||||
working-directory: python
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
ref: ${{ inputs.checkout-ref }}
|
||||
persist-credentials: false
|
||||
- name: Set up python and install the project
|
||||
id: python-setup
|
||||
uses: ./.github/actions/python-setup
|
||||
with:
|
||||
python-version: ${{ env.UV_PYTHON }}
|
||||
os: ${{ runner.os }}
|
||||
- name: Test with pytest (unit tests only)
|
||||
run: >
|
||||
uv run poe test -A
|
||||
-m "not integration"
|
||||
--timeout=120 --session-timeout=900 --timeout_method thread
|
||||
--retries 2 --retry-delay 5
|
||||
|
||||
# OpenAI integration tests
|
||||
python-tests-openai:
|
||||
name: Python Integration Tests - OpenAI
|
||||
runs-on: ubuntu-latest
|
||||
environment: integration
|
||||
timeout-minutes: 60
|
||||
env:
|
||||
OPENAI_CHAT_MODEL_ID: ${{ vars.OPENAI__CHATMODELID }}
|
||||
OPENAI_RESPONSES_MODEL_ID: ${{ vars.OPENAI__RESPONSESMODELID }}
|
||||
OPENAI_EMBEDDINGS_MODEL_ID: ${{ vars.OPENAI_EMBEDDING_MODEL_ID }}
|
||||
OPENAI_API_KEY: ${{ secrets.OPENAI__APIKEY }}
|
||||
defaults:
|
||||
run:
|
||||
working-directory: python
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
ref: ${{ inputs.checkout-ref }}
|
||||
persist-credentials: false
|
||||
- name: Set up python and install the project
|
||||
id: python-setup
|
||||
uses: ./.github/actions/python-setup
|
||||
with:
|
||||
python-version: ${{ env.UV_PYTHON }}
|
||||
os: ${{ runner.os }}
|
||||
- name: Test with pytest (OpenAI integration)
|
||||
run: >
|
||||
uv run pytest --import-mode=importlib
|
||||
packages/core/tests/openai
|
||||
-m integration
|
||||
-n logical --dist worksteal
|
||||
--timeout=120 --session-timeout=900 --timeout_method thread
|
||||
--retries 2 --retry-delay 5
|
||||
|
||||
# Azure OpenAI integration tests
|
||||
python-tests-azure-openai:
|
||||
name: Python Integration Tests - Azure OpenAI
|
||||
runs-on: ubuntu-latest
|
||||
environment: integration
|
||||
timeout-minutes: 60
|
||||
env:
|
||||
AZURE_OPENAI_CHAT_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__CHATDEPLOYMENTNAME }}
|
||||
AZURE_OPENAI_RESPONSES_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
|
||||
AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__EMBEDDINGDEPLOYMENTNAME }}
|
||||
AZURE_OPENAI_ENDPOINT: ${{ vars.AZUREOPENAI__ENDPOINT }}
|
||||
defaults:
|
||||
run:
|
||||
working-directory: python
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
ref: ${{ inputs.checkout-ref }}
|
||||
persist-credentials: false
|
||||
- name: Set up python and install the project
|
||||
id: python-setup
|
||||
uses: ./.github/actions/python-setup
|
||||
with:
|
||||
python-version: ${{ env.UV_PYTHON }}
|
||||
os: ${{ runner.os }}
|
||||
- name: Azure CLI Login
|
||||
uses: azure/login@v2
|
||||
with:
|
||||
client-id: ${{ secrets.AZURE_CLIENT_ID }}
|
||||
tenant-id: ${{ secrets.AZURE_TENANT_ID }}
|
||||
subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
|
||||
- name: Test with pytest (Azure OpenAI integration)
|
||||
run: >
|
||||
uv run pytest --import-mode=importlib
|
||||
packages/core/tests/azure
|
||||
-m integration
|
||||
-n logical --dist worksteal
|
||||
--timeout=120 --session-timeout=900 --timeout_method thread
|
||||
--retries 2 --retry-delay 5
|
||||
|
||||
# Misc integration tests (Anthropic, Ollama, MCP)
|
||||
python-tests-misc-integration:
|
||||
name: Python Integration Tests - Misc
|
||||
runs-on: ubuntu-latest
|
||||
environment: integration
|
||||
timeout-minutes: 60
|
||||
env:
|
||||
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
|
||||
ANTHROPIC_CHAT_MODEL_ID: ${{ vars.ANTHROPIC_CHAT_MODEL_ID }}
|
||||
LOCAL_MCP_URL: ${{ vars.LOCAL_MCP__URL }}
|
||||
defaults:
|
||||
run:
|
||||
working-directory: python
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
ref: ${{ inputs.checkout-ref }}
|
||||
persist-credentials: false
|
||||
- name: Set up python and install the project
|
||||
id: python-setup
|
||||
uses: ./.github/actions/python-setup
|
||||
with:
|
||||
python-version: ${{ env.UV_PYTHON }}
|
||||
os: ${{ runner.os }}
|
||||
- name: Test with pytest (Anthropic, Ollama, MCP integration)
|
||||
run: >
|
||||
uv run pytest --import-mode=importlib
|
||||
packages/anthropic/tests
|
||||
packages/ollama/tests
|
||||
packages/core/tests/core/test_mcp.py
|
||||
-m integration
|
||||
-n logical --dist worksteal
|
||||
--timeout=120 --session-timeout=900 --timeout_method thread
|
||||
--retries 2 --retry-delay 5
|
||||
|
||||
# Azure Functions + Durable Task integration tests
|
||||
python-tests-functions:
|
||||
name: Python Integration Tests - Functions
|
||||
runs-on: ubuntu-latest
|
||||
environment: integration
|
||||
timeout-minutes: 60
|
||||
env:
|
||||
UV_PYTHON: "3.11"
|
||||
OPENAI_CHAT_MODEL_ID: ${{ vars.OPENAI__CHATMODELID }}
|
||||
OPENAI_RESPONSES_MODEL_ID: ${{ vars.OPENAI__RESPONSESMODELID }}
|
||||
OPENAI_API_KEY: ${{ secrets.OPENAI__APIKEY }}
|
||||
AZURE_OPENAI_CHAT_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__CHATDEPLOYMENTNAME }}
|
||||
AZURE_OPENAI_RESPONSES_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
|
||||
AZURE_OPENAI_ENDPOINT: ${{ vars.AZUREOPENAI__ENDPOINT }}
|
||||
FUNCTIONS_WORKER_RUNTIME: "python"
|
||||
DURABLE_TASK_SCHEDULER_CONNECTION_STRING: "Endpoint=http://localhost:8080;TaskHub=default;Authentication=None"
|
||||
AzureWebJobsStorage: "UseDevelopmentStorage=true"
|
||||
defaults:
|
||||
run:
|
||||
working-directory: python
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
ref: ${{ inputs.checkout-ref }}
|
||||
persist-credentials: false
|
||||
- name: Set up python and install the project
|
||||
id: python-setup
|
||||
uses: ./.github/actions/python-setup
|
||||
with:
|
||||
python-version: ${{ env.UV_PYTHON }}
|
||||
os: ${{ runner.os }}
|
||||
- name: Azure CLI Login
|
||||
uses: azure/login@v2
|
||||
with:
|
||||
client-id: ${{ secrets.AZURE_CLIENT_ID }}
|
||||
tenant-id: ${{ secrets.AZURE_TENANT_ID }}
|
||||
subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
|
||||
- name: Set up Azure Functions Integration Test Emulators
|
||||
uses: ./.github/actions/azure-functions-integration-setup
|
||||
id: azure-functions-setup
|
||||
- name: Test with pytest (Functions + Durable Task integration)
|
||||
run: >
|
||||
uv run pytest --import-mode=importlib
|
||||
packages/azurefunctions/tests/integration_tests
|
||||
packages/durabletask/tests/integration_tests
|
||||
-m integration
|
||||
-n logical --dist worksteal
|
||||
--timeout=120 --session-timeout=900 --timeout_method thread
|
||||
--retries 2 --retry-delay 5
|
||||
|
||||
# Azure AI integration tests
|
||||
python-tests-azure-ai:
|
||||
name: Python Integration Tests - Azure AI
|
||||
runs-on: ubuntu-latest
|
||||
environment: integration
|
||||
timeout-minutes: 60
|
||||
env:
|
||||
AZURE_AI_PROJECT_ENDPOINT: ${{ secrets.AZUREAI__ENDPOINT }}
|
||||
AZURE_AI_MODEL_DEPLOYMENT_NAME: ${{ vars.AZUREAI__DEPLOYMENTNAME }}
|
||||
LOCAL_MCP_URL: ${{ vars.LOCAL_MCP__URL }}
|
||||
defaults:
|
||||
run:
|
||||
working-directory: python
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
ref: ${{ inputs.checkout-ref }}
|
||||
persist-credentials: false
|
||||
- name: Set up python and install the project
|
||||
id: python-setup
|
||||
uses: ./.github/actions/python-setup
|
||||
with:
|
||||
python-version: ${{ env.UV_PYTHON }}
|
||||
os: ${{ runner.os }}
|
||||
- name: Azure CLI Login
|
||||
uses: azure/login@v2
|
||||
with:
|
||||
client-id: ${{ secrets.AZURE_CLIENT_ID }}
|
||||
tenant-id: ${{ secrets.AZURE_TENANT_ID }}
|
||||
subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
|
||||
- name: Test with pytest
|
||||
timeout-minutes: 15
|
||||
run: uv run --directory packages/azure-ai poe integration-tests -n logical --dist worksteal --timeout=120 --session-timeout=900 --timeout_method thread --retries 2 --retry-delay 5
|
||||
|
||||
# Azure Cosmos integration tests
|
||||
python-tests-cosmos:
|
||||
name: Python Integration Tests - Cosmos
|
||||
runs-on: ubuntu-latest
|
||||
environment: integration
|
||||
timeout-minutes: 60
|
||||
services:
|
||||
cosmosdb:
|
||||
image: mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:vnext-preview
|
||||
ports:
|
||||
- 8081:8081
|
||||
env:
|
||||
AZURE_COSMOS_ENDPOINT: "http://localhost:8081/"
|
||||
# Static Azure Cosmos DB emulator key (documented): https://learn.microsoft.com/en-us/azure/cosmos-db/emulator
|
||||
AZURE_COSMOS_KEY: "C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw=="
|
||||
AZURE_COSMOS_DATABASE_NAME: "agent-framework-cosmos-it-db"
|
||||
AZURE_COSMOS_CONTAINER_NAME: "agent-framework-cosmos-it-container"
|
||||
defaults:
|
||||
run:
|
||||
working-directory: python
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
ref: ${{ inputs.checkout-ref }}
|
||||
persist-credentials: false
|
||||
- name: Set up python and install the project
|
||||
id: python-setup
|
||||
uses: ./.github/actions/python-setup
|
||||
with:
|
||||
python-version: ${{ env.UV_PYTHON }}
|
||||
os: ${{ runner.os }}
|
||||
- name: Wait for Cosmos DB emulator
|
||||
run: |
|
||||
for i in {1..60}; do
|
||||
if curl --silent --show-error http://localhost:8081/ > /dev/null; then
|
||||
echo "Cosmos DB emulator is ready."
|
||||
exit 0
|
||||
fi
|
||||
sleep 2
|
||||
done
|
||||
echo "Cosmos DB emulator did not become ready in time." >&2
|
||||
exit 1
|
||||
- name: Test with pytest (Cosmos integration)
|
||||
run: uv run --directory packages/azure-cosmos poe integration-tests -n logical --dist worksteal --timeout=120 --session-timeout=900 --timeout_method thread --retries 2 --retry-delay 5
|
||||
|
||||
python-integration-tests-check:
|
||||
if: always()
|
||||
runs-on: ubuntu-latest
|
||||
needs:
|
||||
[
|
||||
python-tests-unit,
|
||||
python-tests-openai,
|
||||
python-tests-azure-openai,
|
||||
python-tests-misc-integration,
|
||||
python-tests-functions,
|
||||
python-tests-azure-ai,
|
||||
python-tests-cosmos
|
||||
]
|
||||
steps:
|
||||
- name: Fail workflow if tests failed
|
||||
if: contains(join(needs.*.result, ','), 'failure')
|
||||
uses: actions/github-script@v8
|
||||
with:
|
||||
script: core.setFailed('Integration Tests Failed!')
|
||||
|
||||
- name: Fail workflow if tests cancelled
|
||||
if: contains(join(needs.*.result, ','), 'cancelled')
|
||||
uses: actions/github-script@v8
|
||||
with:
|
||||
script: core.setFailed('Integration Tests Cancelled!')
|
||||
@@ -67,7 +67,6 @@ jobs:
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
os: ${{ runner.os }}
|
||||
exclude-packages: ${{ matrix.python-version == '3.10' && 'agent-framework-github-copilot' || '' }}
|
||||
env:
|
||||
# Configure a constant location for the uv cache
|
||||
UV_CACHE_DIR: /tmp/.uv-cache
|
||||
@@ -76,9 +75,6 @@ jobs:
|
||||
- name: Run lab tests
|
||||
run: cd packages/lab && uv run poe test
|
||||
|
||||
- name: Run resource-intensive lab tests
|
||||
run: cd packages/lab && uv run pytest -m "resource_intensive and not integration" --junitxml=test-results-resource-intensive.xml
|
||||
|
||||
- name: Run lab lint
|
||||
run: cd packages/lab && uv run poe lint
|
||||
|
||||
|
||||
@@ -1,9 +1,4 @@
|
||||
name: Python - Merge - Tests
|
||||
#
|
||||
# NOTE: This workflow and python-integration-tests.yml share the same set of
|
||||
# parallel test jobs. Keep them in sync — when adding, removing, or modifying a
|
||||
# job here, apply the same change to python-integration-tests.yml.
|
||||
#
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
@@ -15,13 +10,13 @@ on:
|
||||
- cron: "0 0 * * *" # Run at midnight UTC daily
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
contents: write
|
||||
id-token: write
|
||||
|
||||
env:
|
||||
# Configure a constant location for the uv cache
|
||||
UV_CACHE_DIR: /tmp/.uv-cache
|
||||
UV_PYTHON: "3.13"
|
||||
RUN_INTEGRATION_TESTS: "true"
|
||||
RUN_SAMPLES_TESTS: ${{ vars.RUN_SAMPLES_TESTS }}
|
||||
|
||||
jobs:
|
||||
@@ -31,14 +26,7 @@ jobs:
|
||||
contents: read
|
||||
pull-requests: read
|
||||
outputs:
|
||||
pythonChanges: ${{ steps.filter.outputs.python }}
|
||||
coreChanged: ${{ steps.filter.outputs.core }}
|
||||
openaiChanged: ${{ steps.filter.outputs.openai }}
|
||||
azureChanged: ${{ steps.filter.outputs.azure }}
|
||||
miscChanged: ${{ steps.filter.outputs.misc }}
|
||||
functionsChanged: ${{ steps.filter.outputs.functions }}
|
||||
azureAiChanged: ${{ steps.filter.outputs.azure-ai }}
|
||||
cosmosChanged: ${{ steps.filter.outputs.cosmos }}
|
||||
pythonChanges: ${{ steps.filter.outputs.python}}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- uses: dorny/paths-filter@v3
|
||||
@@ -47,29 +35,6 @@ jobs:
|
||||
filters: |
|
||||
python:
|
||||
- 'python/**'
|
||||
core:
|
||||
- 'python/packages/core/agent_framework/_*.py'
|
||||
- 'python/packages/core/agent_framework/_workflows/**'
|
||||
- 'python/packages/core/agent_framework/exceptions.py'
|
||||
- 'python/packages/core/agent_framework/observability.py'
|
||||
openai:
|
||||
- 'python/packages/core/agent_framework/openai/**'
|
||||
- 'python/packages/core/tests/openai/**'
|
||||
azure:
|
||||
- 'python/packages/core/agent_framework/azure/**'
|
||||
- 'python/packages/core/tests/azure/**'
|
||||
misc:
|
||||
- 'python/packages/anthropic/**'
|
||||
- 'python/packages/ollama/**'
|
||||
- 'python/packages/core/agent_framework/_mcp.py'
|
||||
- 'python/packages/core/tests/core/test_mcp.py'
|
||||
functions:
|
||||
- 'python/packages/azurefunctions/**'
|
||||
- 'python/packages/durabletask/**'
|
||||
azure-ai:
|
||||
- 'python/packages/azure-ai/**'
|
||||
cosmos:
|
||||
- 'python/packages/azure-cosmos/**'
|
||||
# run only if 'python' files were changed
|
||||
- name: python tests
|
||||
if: steps.filter.outputs.python == 'true'
|
||||
@@ -78,225 +43,34 @@ jobs:
|
||||
- name: not python tests
|
||||
if: steps.filter.outputs.python != 'true'
|
||||
run: echo "NOT python file"
|
||||
# Unit tests: always run all non-integration tests across all packages
|
||||
python-tests-unit:
|
||||
name: Python Tests - Unit
|
||||
python-tests-core:
|
||||
name: Python Tests - Core
|
||||
needs: paths-filter
|
||||
if: >
|
||||
github.event_name != 'pull_request' &&
|
||||
needs.paths-filter.outputs.pythonChanges == 'true'
|
||||
runs-on: ubuntu-latest
|
||||
environment: integration
|
||||
defaults:
|
||||
run:
|
||||
working-directory: python
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- name: Set up python and install the project
|
||||
id: python-setup
|
||||
uses: ./.github/actions/python-setup
|
||||
with:
|
||||
python-version: ${{ env.UV_PYTHON }}
|
||||
os: ${{ runner.os }}
|
||||
- name: Test with pytest (unit tests only)
|
||||
run: >
|
||||
uv run poe test -A
|
||||
-m "not integration"
|
||||
--timeout=120 --session-timeout=900 --timeout_method thread
|
||||
--retries 2 --retry-delay 5
|
||||
working-directory: ./python
|
||||
- name: Surface failing tests
|
||||
if: always()
|
||||
uses: pmeier/pytest-results-action@v0.7.2
|
||||
with:
|
||||
path: ./python/**.xml
|
||||
summary: true
|
||||
display-options: fEX
|
||||
fail-on-empty: false
|
||||
title: Unit test results
|
||||
|
||||
# OpenAI integration tests
|
||||
python-tests-openai:
|
||||
name: Python Tests - OpenAI Integration
|
||||
needs: paths-filter
|
||||
if: >
|
||||
github.event_name != 'pull_request' &&
|
||||
needs.paths-filter.outputs.pythonChanges == 'true' &&
|
||||
(github.event_name != 'merge_group' ||
|
||||
needs.paths-filter.outputs.openaiChanged == 'true' ||
|
||||
needs.paths-filter.outputs.coreChanged == 'true')
|
||||
runs-on: ubuntu-latest
|
||||
environment: integration
|
||||
if: github.event_name != 'pull_request' && needs.paths-filter.outputs.pythonChanges == 'true'
|
||||
runs-on: ${{ matrix.os }}
|
||||
environment: ${{ matrix.environment }}
|
||||
strategy:
|
||||
fail-fast: true
|
||||
matrix:
|
||||
python-version: ["3.10"]
|
||||
os: [ubuntu-latest]
|
||||
environment: ["integration"]
|
||||
env:
|
||||
UV_PYTHON: ${{ matrix.python-version }}
|
||||
OPENAI_CHAT_MODEL_ID: ${{ vars.OPENAI__CHATMODELID }}
|
||||
OPENAI_RESPONSES_MODEL_ID: ${{ vars.OPENAI__RESPONSESMODELID }}
|
||||
OPENAI_EMBEDDINGS_MODEL_ID: ${{ vars.OPENAI_EMBEDDING_MODEL_ID }}
|
||||
OPENAI_API_KEY: ${{ secrets.OPENAI__APIKEY }}
|
||||
defaults:
|
||||
run:
|
||||
working-directory: python
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- name: Set up python and install the project
|
||||
id: python-setup
|
||||
uses: ./.github/actions/python-setup
|
||||
with:
|
||||
python-version: ${{ env.UV_PYTHON }}
|
||||
os: ${{ runner.os }}
|
||||
- name: Test with pytest (OpenAI integration)
|
||||
run: >
|
||||
uv run pytest --import-mode=importlib
|
||||
packages/core/tests/openai
|
||||
-m integration
|
||||
-n logical --dist worksteal
|
||||
--timeout=120 --session-timeout=900 --timeout_method thread
|
||||
--retries 2 --retry-delay 5
|
||||
working-directory: ./python
|
||||
- name: Test OpenAI samples
|
||||
timeout-minutes: 10
|
||||
if: env.RUN_SAMPLES_TESTS == 'true'
|
||||
run: uv run pytest tests/samples/ -m "openai"
|
||||
working-directory: ./python
|
||||
- name: Surface failing tests
|
||||
if: always()
|
||||
uses: pmeier/pytest-results-action@v0.7.2
|
||||
with:
|
||||
path: ./python/**.xml
|
||||
summary: true
|
||||
display-options: fEX
|
||||
fail-on-empty: false
|
||||
title: OpenAI integration test results
|
||||
|
||||
# Azure OpenAI integration tests
|
||||
python-tests-azure-openai:
|
||||
name: Python Tests - Azure OpenAI Integration
|
||||
needs: paths-filter
|
||||
if: >
|
||||
github.event_name != 'pull_request' &&
|
||||
needs.paths-filter.outputs.pythonChanges == 'true' &&
|
||||
(github.event_name != 'merge_group' ||
|
||||
needs.paths-filter.outputs.azureChanged == 'true' ||
|
||||
needs.paths-filter.outputs.coreChanged == 'true')
|
||||
runs-on: ubuntu-latest
|
||||
environment: integration
|
||||
env:
|
||||
AZURE_OPENAI_CHAT_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__CHATDEPLOYMENTNAME }}
|
||||
AZURE_OPENAI_RESPONSES_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
|
||||
AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__EMBEDDINGDEPLOYMENTNAME }}
|
||||
AZURE_OPENAI_ENDPOINT: ${{ vars.AZUREOPENAI__ENDPOINT }}
|
||||
defaults:
|
||||
run:
|
||||
working-directory: python
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- name: Set up python and install the project
|
||||
id: python-setup
|
||||
uses: ./.github/actions/python-setup
|
||||
with:
|
||||
python-version: ${{ env.UV_PYTHON }}
|
||||
os: ${{ runner.os }}
|
||||
- name: Azure CLI Login
|
||||
if: github.event_name != 'pull_request'
|
||||
uses: azure/login@v2
|
||||
with:
|
||||
client-id: ${{ secrets.AZURE_CLIENT_ID }}
|
||||
tenant-id: ${{ secrets.AZURE_TENANT_ID }}
|
||||
subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
|
||||
- name: Test with pytest (Azure OpenAI integration)
|
||||
run: >
|
||||
uv run pytest --import-mode=importlib
|
||||
packages/core/tests/azure
|
||||
-m integration
|
||||
-n logical --dist worksteal
|
||||
--timeout=120 --session-timeout=900 --timeout_method thread
|
||||
--retries 2 --retry-delay 5
|
||||
working-directory: ./python
|
||||
- name: Test Azure samples
|
||||
timeout-minutes: 10
|
||||
if: env.RUN_SAMPLES_TESTS == 'true'
|
||||
run: uv run pytest tests/samples/ -m "azure"
|
||||
working-directory: ./python
|
||||
- name: Surface failing tests
|
||||
if: always()
|
||||
uses: pmeier/pytest-results-action@v0.7.2
|
||||
with:
|
||||
path: ./python/**.xml
|
||||
summary: true
|
||||
display-options: fEX
|
||||
fail-on-empty: false
|
||||
title: Azure OpenAI integration test results
|
||||
|
||||
# Misc integration tests (Anthropic, Ollama, MCP)
|
||||
python-tests-misc-integration:
|
||||
name: Python Tests - Misc Integration
|
||||
needs: paths-filter
|
||||
if: >
|
||||
github.event_name != 'pull_request' &&
|
||||
needs.paths-filter.outputs.pythonChanges == 'true' &&
|
||||
(github.event_name != 'merge_group' ||
|
||||
needs.paths-filter.outputs.miscChanged == 'true' ||
|
||||
needs.paths-filter.outputs.coreChanged == 'true')
|
||||
runs-on: ubuntu-latest
|
||||
environment: integration
|
||||
env:
|
||||
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
|
||||
ANTHROPIC_CHAT_MODEL_ID: ${{ vars.ANTHROPIC_CHAT_MODEL_ID }}
|
||||
LOCAL_MCP_URL: ${{ vars.LOCAL_MCP__URL }}
|
||||
defaults:
|
||||
run:
|
||||
working-directory: python
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- name: Set up python and install the project
|
||||
id: python-setup
|
||||
uses: ./.github/actions/python-setup
|
||||
with:
|
||||
python-version: ${{ env.UV_PYTHON }}
|
||||
os: ${{ runner.os }}
|
||||
- name: Test with pytest (Anthropic, Ollama, MCP integration)
|
||||
run: >
|
||||
uv run pytest --import-mode=importlib
|
||||
packages/anthropic/tests
|
||||
packages/ollama/tests
|
||||
packages/core/tests/core/test_mcp.py
|
||||
-m integration
|
||||
-n logical --dist worksteal
|
||||
--timeout=120 --session-timeout=900 --timeout_method thread
|
||||
--retries 2 --retry-delay 5
|
||||
working-directory: ./python
|
||||
- name: Surface failing tests
|
||||
if: always()
|
||||
uses: pmeier/pytest-results-action@v0.7.2
|
||||
with:
|
||||
path: ./python/**.xml
|
||||
summary: true
|
||||
display-options: fEX
|
||||
fail-on-empty: false
|
||||
title: Misc integration test results
|
||||
|
||||
# Azure Functions + Durable Task integration tests
|
||||
python-tests-functions:
|
||||
name: Python Tests - Functions Integration
|
||||
needs: paths-filter
|
||||
if: >
|
||||
github.event_name != 'pull_request' &&
|
||||
needs.paths-filter.outputs.pythonChanges == 'true' &&
|
||||
(github.event_name != 'merge_group' ||
|
||||
needs.paths-filter.outputs.functionsChanged == 'true' ||
|
||||
needs.paths-filter.outputs.coreChanged == 'true')
|
||||
runs-on: ubuntu-latest
|
||||
environment: integration
|
||||
env:
|
||||
UV_PYTHON: "3.11"
|
||||
OPENAI_CHAT_MODEL_ID: ${{ vars.OPENAI__CHATMODELID }}
|
||||
OPENAI_RESPONSES_MODEL_ID: ${{ vars.OPENAI__RESPONSESMODELID }}
|
||||
OPENAI_API_KEY: ${{ secrets.OPENAI__APIKEY }}
|
||||
AZURE_OPENAI_CHAT_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__CHATDEPLOYMENTNAME }}
|
||||
AZURE_OPENAI_RESPONSES_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
|
||||
AZURE_OPENAI_ENDPOINT: ${{ vars.AZUREOPENAI__ENDPOINT }}
|
||||
LOCAL_MCP_URL: ${{ vars.LOCAL_MCP__URL }}
|
||||
# For Azure Functions integration tests
|
||||
FUNCTIONS_WORKER_RUNTIME: "python"
|
||||
DURABLE_TASK_SCHEDULER_CONNECTION_STRING: "Endpoint=http://localhost:8080;TaskHub=default;Authentication=None"
|
||||
AzureWebJobsStorage: "UseDevelopmentStorage=true"
|
||||
|
||||
defaults:
|
||||
run:
|
||||
working-directory: python
|
||||
@@ -306,8 +80,11 @@ jobs:
|
||||
id: python-setup
|
||||
uses: ./.github/actions/python-setup
|
||||
with:
|
||||
python-version: ${{ env.UV_PYTHON }}
|
||||
python-version: ${{ matrix.python-version }}
|
||||
os: ${{ runner.os }}
|
||||
env:
|
||||
# Configure a constant location for the uv cache
|
||||
UV_CACHE_DIR: /tmp/.uv-cache
|
||||
- name: Azure CLI Login
|
||||
if: github.event_name != 'pull_request'
|
||||
uses: azure/login@v2
|
||||
@@ -318,15 +95,14 @@ jobs:
|
||||
- name: Set up Azure Functions Integration Test Emulators
|
||||
uses: ./.github/actions/azure-functions-integration-setup
|
||||
id: azure-functions-setup
|
||||
- name: Test with pytest (Functions + Durable Task integration)
|
||||
run: >
|
||||
uv run pytest --import-mode=importlib
|
||||
packages/azurefunctions/tests/integration_tests
|
||||
packages/durabletask/tests/integration_tests
|
||||
-m integration
|
||||
-n logical --dist worksteal
|
||||
--timeout=120 --session-timeout=900 --timeout_method thread
|
||||
--retries 2 --retry-delay 5
|
||||
- name: Test with pytest
|
||||
timeout-minutes: 10
|
||||
run: uv run poe all-tests -n logical --dist loadfile --dist worksteal --timeout 900 --retries 3 --retry-delay 10
|
||||
working-directory: ./python
|
||||
- name: Test core samples
|
||||
timeout-minutes: 10
|
||||
if: env.RUN_SAMPLES_TESTS == 'true'
|
||||
run: uv run pytest tests/samples/ -m "openai" -m "azure"
|
||||
working-directory: ./python
|
||||
- name: Surface failing tests
|
||||
if: always()
|
||||
@@ -336,20 +112,22 @@ jobs:
|
||||
summary: true
|
||||
display-options: fEX
|
||||
fail-on-empty: false
|
||||
title: Functions integration test results
|
||||
title: Test results
|
||||
|
||||
python-tests-azure-ai:
|
||||
name: Python Tests - Azure AI
|
||||
needs: paths-filter
|
||||
if: >
|
||||
github.event_name != 'pull_request' &&
|
||||
needs.paths-filter.outputs.pythonChanges == 'true' &&
|
||||
(github.event_name != 'merge_group' ||
|
||||
needs.paths-filter.outputs.azureAiChanged == 'true' ||
|
||||
needs.paths-filter.outputs.coreChanged == 'true')
|
||||
runs-on: ubuntu-latest
|
||||
environment: integration
|
||||
if: github.event_name != 'pull_request' && needs.paths-filter.outputs.pythonChanges == 'true'
|
||||
runs-on: ${{ matrix.os }}
|
||||
environment: ${{ matrix.environment }}
|
||||
strategy:
|
||||
fail-fast: true
|
||||
matrix:
|
||||
python-version: ["3.10"]
|
||||
os: [ubuntu-latest]
|
||||
environment: ["integration"]
|
||||
env:
|
||||
UV_PYTHON: ${{ matrix.python-version }}
|
||||
AZURE_AI_PROJECT_ENDPOINT: ${{ secrets.AZUREAI__ENDPOINT }}
|
||||
AZURE_AI_MODEL_DEPLOYMENT_NAME: ${{ vars.AZUREAI__DEPLOYMENTNAME }}
|
||||
LOCAL_MCP_URL: ${{ vars.LOCAL_MCP__URL }}
|
||||
@@ -362,8 +140,11 @@ jobs:
|
||||
id: python-setup
|
||||
uses: ./.github/actions/python-setup
|
||||
with:
|
||||
python-version: ${{ env.UV_PYTHON }}
|
||||
python-version: ${{ matrix.python-version }}
|
||||
os: ${{ runner.os }}
|
||||
env:
|
||||
# Configure a constant location for the uv cache
|
||||
UV_CACHE_DIR: /tmp/.uv-cache
|
||||
- name: Azure CLI Login
|
||||
if: github.event_name != 'pull_request'
|
||||
uses: azure/login@v2
|
||||
@@ -372,8 +153,8 @@ jobs:
|
||||
tenant-id: ${{ secrets.AZURE_TENANT_ID }}
|
||||
subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
|
||||
- name: Test with pytest
|
||||
timeout-minutes: 15
|
||||
run: uv run --directory packages/azure-ai poe integration-tests -n logical --dist worksteal --timeout=120 --session-timeout=900 --timeout_method thread --retries 2 --retry-delay 5
|
||||
timeout-minutes: 10
|
||||
run: uv run --directory packages/azure-ai poe integration-tests -n logical --dist loadfile --dist worksteal --timeout 300 --retries 3 --retry-delay 10
|
||||
working-directory: ./python
|
||||
- name: Test Azure AI samples
|
||||
timeout-minutes: 10
|
||||
@@ -392,78 +173,16 @@ jobs:
|
||||
|
||||
# TODO: Add python-tests-lab
|
||||
|
||||
# Azure Cosmos integration tests
|
||||
python-tests-cosmos:
|
||||
name: Python Tests - Cosmos Integration
|
||||
needs: paths-filter
|
||||
if: >
|
||||
github.event_name != 'pull_request' &&
|
||||
needs.paths-filter.outputs.pythonChanges == 'true' &&
|
||||
(github.event_name != 'merge_group' ||
|
||||
needs.paths-filter.outputs.cosmosChanged == 'true' ||
|
||||
needs.paths-filter.outputs.coreChanged == 'true')
|
||||
runs-on: ubuntu-latest
|
||||
environment: integration
|
||||
services:
|
||||
cosmosdb:
|
||||
image: mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:vnext-preview
|
||||
ports:
|
||||
- 8081:8081
|
||||
env:
|
||||
AZURE_COSMOS_ENDPOINT: "http://localhost:8081/"
|
||||
# Static Azure Cosmos DB emulator key (documented): https://learn.microsoft.com/en-us/azure/cosmos-db/emulator
|
||||
AZURE_COSMOS_KEY: "C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw=="
|
||||
AZURE_COSMOS_DATABASE_NAME: "agent-framework-cosmos-it-db"
|
||||
AZURE_COSMOS_CONTAINER_NAME: "agent-framework-cosmos-it-container"
|
||||
defaults:
|
||||
run:
|
||||
working-directory: python
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- name: Set up python and install the project
|
||||
id: python-setup
|
||||
uses: ./.github/actions/python-setup
|
||||
with:
|
||||
python-version: ${{ env.UV_PYTHON }}
|
||||
os: ${{ runner.os }}
|
||||
- name: Wait for Cosmos DB emulator
|
||||
run: |
|
||||
for i in {1..60}; do
|
||||
if curl --silent --show-error http://localhost:8081/ > /dev/null; then
|
||||
echo "Cosmos DB emulator is ready."
|
||||
exit 0
|
||||
fi
|
||||
sleep 2
|
||||
done
|
||||
echo "Cosmos DB emulator did not become ready in time." >&2
|
||||
exit 1
|
||||
- name: Test with pytest (Cosmos integration)
|
||||
run: uv run --directory packages/azure-cosmos poe integration-tests -n logical --dist worksteal --timeout=120 --session-timeout=900 --timeout_method thread --retries 2 --retry-delay 5
|
||||
working-directory: ./python
|
||||
- name: Surface failing tests
|
||||
if: always()
|
||||
uses: pmeier/pytest-results-action@v0.7.2
|
||||
with:
|
||||
path: ./python/**.xml
|
||||
summary: true
|
||||
display-options: fEX
|
||||
fail-on-empty: false
|
||||
title: Cosmos integration test results
|
||||
|
||||
python-integration-tests-check:
|
||||
if: always()
|
||||
runs-on: ubuntu-latest
|
||||
needs:
|
||||
[
|
||||
python-tests-unit,
|
||||
python-tests-openai,
|
||||
python-tests-azure-openai,
|
||||
python-tests-misc-integration,
|
||||
python-tests-functions,
|
||||
python-tests-azure-ai,
|
||||
python-tests-cosmos,
|
||||
python-tests-core,
|
||||
python-tests-azure-ai
|
||||
]
|
||||
steps:
|
||||
|
||||
- name: Fail workflow if tests failed
|
||||
id: check_tests_failed
|
||||
if: contains(join(needs.*.result, ','), 'failure')
|
||||
|
||||
@@ -1,814 +0,0 @@
|
||||
name: Python - Sample Validation
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
schedule:
|
||||
- cron: "0 0 * * *" # Run at midnight UTC daily
|
||||
|
||||
env:
|
||||
# Configure a constant location for the uv cache
|
||||
UV_CACHE_DIR: /tmp/.uv-cache
|
||||
# GitHub Copilot configuration
|
||||
GITHUB_COPILOT_MODEL: claude-opus-4.6
|
||||
COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }}
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
|
||||
jobs:
|
||||
validate-01-get-started:
|
||||
name: Validate 01-get-started
|
||||
runs-on: ubuntu-latest
|
||||
environment: integration
|
||||
env:
|
||||
# Required configuration for get-started samples
|
||||
AZURE_AI_PROJECT_ENDPOINT: ${{ vars.AZURE_AI_PROJECT_ENDPOINT }}
|
||||
AZURE_OPENAI_RESPONSES_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
|
||||
AZURE_OPENAI_ENDPOINT: ${{ vars.AZUREOPENAI__ENDPOINT }}
|
||||
AZURE_OPENAI_CHAT_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__CHATDEPLOYMENTNAME }}
|
||||
defaults:
|
||||
run:
|
||||
working-directory: python
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- name: Setup environment
|
||||
uses: ./.github/actions/sample-validation-setup
|
||||
with:
|
||||
azure-client-id: ${{ secrets.AZURE_CLIENT_ID }}
|
||||
azure-tenant-id: ${{ secrets.AZURE_TENANT_ID }}
|
||||
azure-subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
|
||||
os: ${{ runner.os }}
|
||||
|
||||
- name: Create .env for samples
|
||||
run: |
|
||||
echo "AZURE_AI_PROJECT_ENDPOINT=$AZURE_AI_PROJECT_ENDPOINT" >> .env
|
||||
echo "AZURE_OPENAI_RESPONSES_DEPLOYMENT_NAME=$AZURE_OPENAI_RESPONSES_DEPLOYMENT_NAME" >> .env
|
||||
echo "AZURE_OPENAI_ENDPOINT=$AZURE_OPENAI_ENDPOINT" >> .env
|
||||
echo "AZURE_OPENAI_CHAT_DEPLOYMENT_NAME=$AZURE_OPENAI_CHAT_DEPLOYMENT_NAME" >> .env
|
||||
|
||||
- name: Run sample validation
|
||||
run: |
|
||||
cd scripts && uv run python -m sample_validation --subdir 01-get-started --save-report --report-name 01-get-started
|
||||
|
||||
- name: Upload validation report
|
||||
uses: actions/upload-artifact@v7
|
||||
if: always()
|
||||
with:
|
||||
name: validation-report-01-get-started
|
||||
path: python/samples/sample_validation/reports/
|
||||
|
||||
validate-02-agents:
|
||||
name: Validate 02-agents
|
||||
runs-on: ubuntu-latest
|
||||
environment: integration
|
||||
env:
|
||||
# Azure AI configuration
|
||||
AZURE_AI_PROJECT_ENDPOINT: ${{ vars.AZURE_AI_PROJECT_ENDPOINT }}
|
||||
AZURE_AI_MODEL_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
|
||||
# Azure OpenAI configuration
|
||||
AZURE_OPENAI_ENDPOINT: ${{ vars.AZUREOPENAI__ENDPOINT }}
|
||||
AZURE_OPENAI_CHAT_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__CHATDEPLOYMENTNAME }}
|
||||
AZURE_OPENAI_RESPONSES_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
|
||||
AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__EMBEDDINGDEPLOYMENTNAME }}
|
||||
# OpenAI configuration
|
||||
OPENAI_API_KEY: ${{ secrets.OPENAI__APIKEY }}
|
||||
OPENAI_CHAT_MODEL_ID: ${{ vars.OPENAI__CHATMODELID }}
|
||||
OPENAI_RESPONSES_MODEL_ID: ${{ vars.OPENAI__RESPONSESMODELID }}
|
||||
# GitHub MCP
|
||||
GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
|
||||
# Observability
|
||||
ENABLE_INSTRUMENTATION: "true"
|
||||
defaults:
|
||||
run:
|
||||
working-directory: python
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- name: Setup environment
|
||||
uses: ./.github/actions/sample-validation-setup
|
||||
with:
|
||||
azure-client-id: ${{ secrets.AZURE_CLIENT_ID }}
|
||||
azure-tenant-id: ${{ secrets.AZURE_TENANT_ID }}
|
||||
azure-subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
|
||||
os: ${{ runner.os }}
|
||||
|
||||
- name: Create .env for samples
|
||||
run: |
|
||||
echo "AZURE_AI_PROJECT_ENDPOINT=$AZURE_AI_PROJECT_ENDPOINT" >> .env
|
||||
echo "AZURE_AI_MODEL_DEPLOYMENT_NAME=$AZURE_AI_MODEL_DEPLOYMENT_NAME" >> .env
|
||||
echo "AZURE_OPENAI_ENDPOINT=$AZURE_OPENAI_ENDPOINT" >> .env
|
||||
echo "AZURE_OPENAI_CHAT_DEPLOYMENT_NAME=$AZURE_OPENAI_CHAT_DEPLOYMENT_NAME" >> .env
|
||||
echo "AZURE_OPENAI_RESPONSES_DEPLOYMENT_NAME=$AZURE_OPENAI_RESPONSES_DEPLOYMENT_NAME" >> .env
|
||||
echo "AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME=$AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME" >> .env
|
||||
echo "OPENAI_API_KEY=$OPENAI_API_KEY" >> .env
|
||||
echo "OPENAI_CHAT_MODEL_ID=$OPENAI_CHAT_MODEL_ID" >> .env
|
||||
echo "OPENAI_RESPONSES_MODEL_ID=$OPENAI_RESPONSES_MODEL_ID" >> .env
|
||||
echo "GITHUB_PAT=$GITHUB_PAT" >> .env
|
||||
|
||||
- name: Run sample validation
|
||||
run: |
|
||||
cd scripts && uv run python -m sample_validation --subdir 02-agents --exclude providers --save-report --report-name 02-agents
|
||||
|
||||
- name: Upload validation report
|
||||
uses: actions/upload-artifact@v7
|
||||
if: always()
|
||||
with:
|
||||
name: validation-report-02-agents
|
||||
path: python/samples/sample_validation/reports/
|
||||
|
||||
validate-02-agents-openai:
|
||||
name: Validate 02-agents/providers/openai
|
||||
runs-on: ubuntu-latest
|
||||
environment: integration
|
||||
env:
|
||||
OPENAI_API_KEY: ${{ secrets.OPENAI__APIKEY }}
|
||||
OPENAI_CHAT_MODEL_ID: ${{ vars.OPENAI__CHATMODELID }}
|
||||
OPENAI_RESPONSES_MODEL_ID: ${{ vars.OPENAI__RESPONSESMODELID }}
|
||||
defaults:
|
||||
run:
|
||||
working-directory: python
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- name: Setup environment
|
||||
uses: ./.github/actions/sample-validation-setup
|
||||
with:
|
||||
azure-client-id: ${{ secrets.AZURE_CLIENT_ID }}
|
||||
azure-tenant-id: ${{ secrets.AZURE_TENANT_ID }}
|
||||
azure-subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
|
||||
os: ${{ runner.os }}
|
||||
|
||||
- name: Create .env for samples
|
||||
run: |
|
||||
echo "OPENAI_API_KEY=$OPENAI_API_KEY" >> .env
|
||||
echo "OPENAI_CHAT_MODEL_ID=$OPENAI_CHAT_MODEL_ID" >> .env
|
||||
echo "OPENAI_RESPONSES_MODEL_ID=$OPENAI_RESPONSES_MODEL_ID" >> .env
|
||||
|
||||
- name: Run sample validation
|
||||
run: |
|
||||
cd scripts && uv run python -m sample_validation --subdir 02-agents/providers/openai --save-report --report-name 02-agents-openai
|
||||
|
||||
- name: Upload validation report
|
||||
uses: actions/upload-artifact@v7
|
||||
if: always()
|
||||
with:
|
||||
name: validation-report-02-agents-openai
|
||||
path: python/samples/sample_validation/reports/
|
||||
|
||||
validate-02-agents-azure-openai:
|
||||
name: Validate 02-agents/providers/azure_openai
|
||||
runs-on: ubuntu-latest
|
||||
environment: integration
|
||||
env:
|
||||
AZURE_AI_PROJECT_ENDPOINT: ${{ vars.AZURE_AI_PROJECT_ENDPOINT }}
|
||||
AZURE_OPENAI_ENDPOINT: ${{ vars.AZUREOPENAI__ENDPOINT }}
|
||||
AZURE_OPENAI_CHAT_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__CHATDEPLOYMENTNAME }}
|
||||
AZURE_OPENAI_RESPONSES_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
|
||||
defaults:
|
||||
run:
|
||||
working-directory: python
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- name: Setup environment
|
||||
uses: ./.github/actions/sample-validation-setup
|
||||
with:
|
||||
azure-client-id: ${{ secrets.AZURE_CLIENT_ID }}
|
||||
azure-tenant-id: ${{ secrets.AZURE_TENANT_ID }}
|
||||
azure-subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
|
||||
os: ${{ runner.os }}
|
||||
|
||||
- name: Create .env for samples
|
||||
run: |
|
||||
echo "AZURE_AI_PROJECT_ENDPOINT=$AZURE_AI_PROJECT_ENDPOINT" >> .env
|
||||
echo "AZURE_OPENAI_ENDPOINT=$AZURE_OPENAI_ENDPOINT" >> .env
|
||||
echo "AZURE_OPENAI_CHAT_DEPLOYMENT_NAME=$AZURE_OPENAI_CHAT_DEPLOYMENT_NAME" >> .env
|
||||
echo "AZURE_OPENAI_RESPONSES_DEPLOYMENT_NAME=$AZURE_OPENAI_RESPONSES_DEPLOYMENT_NAME" >> .env
|
||||
|
||||
- name: Run sample validation
|
||||
run: |
|
||||
cd scripts && uv run python -m sample_validation --subdir 02-agents/providers/azure_openai --save-report --report-name 02-agents-azure-openai
|
||||
|
||||
- name: Upload validation report
|
||||
uses: actions/upload-artifact@v7
|
||||
if: always()
|
||||
with:
|
||||
name: validation-report-02-agents-azure-openai
|
||||
path: python/samples/sample_validation/reports/
|
||||
|
||||
validate-02-agents-azure-ai:
|
||||
name: Validate 02-agents/providers/azure_ai
|
||||
runs-on: ubuntu-latest
|
||||
environment: integration
|
||||
env:
|
||||
AZURE_AI_PROJECT_ENDPOINT: ${{ vars.AZURE_AI_PROJECT_ENDPOINT }}
|
||||
AZURE_AI_MODEL_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
|
||||
AZURE_AI_CHAT_MODEL_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__CHATDEPLOYMENTNAME }}
|
||||
AZURE_AI_EMBEDDING_MODEL_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__EMBEDDINGDEPLOYMENTNAME }}
|
||||
BING_CONNECTION_ID: ${{ secrets.BING_CONNECTION_ID }}
|
||||
defaults:
|
||||
run:
|
||||
working-directory: python
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- name: Setup environment
|
||||
uses: ./.github/actions/sample-validation-setup
|
||||
with:
|
||||
azure-client-id: ${{ secrets.AZURE_CLIENT_ID }}
|
||||
azure-tenant-id: ${{ secrets.AZURE_TENANT_ID }}
|
||||
azure-subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
|
||||
os: ${{ runner.os }}
|
||||
|
||||
- name: Create .env for samples
|
||||
run: |
|
||||
echo "AZURE_AI_PROJECT_ENDPOINT=$AZURE_AI_PROJECT_ENDPOINT" >> .env
|
||||
echo "AZURE_AI_MODEL_DEPLOYMENT_NAME=$AZURE_AI_MODEL_DEPLOYMENT_NAME" >> .env
|
||||
echo "AZURE_AI_CHAT_MODEL_DEPLOYMENT_NAME=$AZURE_AI_CHAT_MODEL_DEPLOYMENT_NAME" >> .env
|
||||
echo "AZURE_AI_EMBEDDING_MODEL_DEPLOYMENT_NAME=$AZURE_AI_EMBEDDING_MODEL_DEPLOYMENT_NAME" >> .env
|
||||
echo "BING_CONNECTION_ID=$BING_CONNECTION_ID" >> .env
|
||||
|
||||
- name: Run sample validation
|
||||
run: |
|
||||
cd scripts && uv run python -m sample_validation --subdir 02-agents/providers/azure_ai --save-report --report-name 02-agents-azure-ai
|
||||
|
||||
- name: Upload validation report
|
||||
uses: actions/upload-artifact@v7
|
||||
if: always()
|
||||
with:
|
||||
name: validation-report-02-agents-azure-ai
|
||||
path: python/samples/sample_validation/reports/
|
||||
|
||||
validate-02-agents-azure-ai-agent:
|
||||
name: Validate 02-agents/providers/azure_ai_agent
|
||||
runs-on: ubuntu-latest
|
||||
environment: integration
|
||||
env:
|
||||
AZURE_AI_PROJECT_ENDPOINT: ${{ vars.AZURE_AI_PROJECT_ENDPOINT }}
|
||||
AZURE_AI_MODEL_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
|
||||
defaults:
|
||||
run:
|
||||
working-directory: python
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- name: Setup environment
|
||||
uses: ./.github/actions/sample-validation-setup
|
||||
with:
|
||||
azure-client-id: ${{ secrets.AZURE_CLIENT_ID }}
|
||||
azure-tenant-id: ${{ secrets.AZURE_TENANT_ID }}
|
||||
azure-subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
|
||||
os: ${{ runner.os }}
|
||||
|
||||
- name: Create .env for samples
|
||||
run: |
|
||||
echo "AZURE_AI_PROJECT_ENDPOINT=$AZURE_AI_PROJECT_ENDPOINT" >> .env
|
||||
echo "AZURE_AI_MODEL_DEPLOYMENT_NAME=$AZURE_AI_MODEL_DEPLOYMENT_NAME" >> .env
|
||||
|
||||
- name: Run sample validation
|
||||
run: |
|
||||
cd scripts && uv run python -m sample_validation --subdir 02-agents/providers/azure_ai_agent --save-report --report-name 02-agents-azure-ai-agent
|
||||
|
||||
- name: Upload validation report
|
||||
uses: actions/upload-artifact@v7
|
||||
if: always()
|
||||
with:
|
||||
name: validation-report-02-agents-azure-ai-agent
|
||||
path: python/samples/sample_validation/reports/
|
||||
|
||||
validate-02-agents-anthropic:
|
||||
name: Validate 02-agents/providers/anthropic
|
||||
runs-on: ubuntu-latest
|
||||
environment: integration
|
||||
env:
|
||||
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
|
||||
ANTHROPIC_CHAT_MODEL_ID: ${{ vars.ANTHROPIC_CHAT_MODEL_ID }}
|
||||
defaults:
|
||||
run:
|
||||
working-directory: python
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- name: Setup environment
|
||||
uses: ./.github/actions/sample-validation-setup
|
||||
with:
|
||||
azure-client-id: ${{ secrets.AZURE_CLIENT_ID }}
|
||||
azure-tenant-id: ${{ secrets.AZURE_TENANT_ID }}
|
||||
azure-subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
|
||||
os: ${{ runner.os }}
|
||||
|
||||
- name: Create .env for samples
|
||||
run: |
|
||||
echo "ANTHROPIC_API_KEY=$ANTHROPIC_API_KEY" >> .env
|
||||
echo "ANTHROPIC_CHAT_MODEL_ID=$ANTHROPIC_CHAT_MODEL_ID" >> .env
|
||||
|
||||
- name: Run sample validation
|
||||
run: |
|
||||
cd scripts && uv run python -m sample_validation --subdir 02-agents/providers/anthropic --save-report --report-name 02-agents-anthropic
|
||||
|
||||
- name: Upload validation report
|
||||
uses: actions/upload-artifact@v7
|
||||
if: always()
|
||||
with:
|
||||
name: validation-report-02-agents-anthropic
|
||||
path: python/samples/sample_validation/reports/
|
||||
|
||||
validate-02-agents-github-copilot:
|
||||
name: Validate 02-agents/providers/github_copilot
|
||||
runs-on: ubuntu-latest
|
||||
environment: integration
|
||||
defaults:
|
||||
run:
|
||||
working-directory: python
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- name: Setup environment
|
||||
uses: ./.github/actions/sample-validation-setup
|
||||
with:
|
||||
azure-client-id: ${{ secrets.AZURE_CLIENT_ID }}
|
||||
azure-tenant-id: ${{ secrets.AZURE_TENANT_ID }}
|
||||
azure-subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
|
||||
os: ${{ runner.os }}
|
||||
|
||||
- name: Run sample validation
|
||||
run: |
|
||||
cd scripts && uv run python -m sample_validation --subdir 02-agents/providers/github_copilot --save-report --report-name 02-agents-github-copilot
|
||||
|
||||
- name: Upload validation report
|
||||
uses: actions/upload-artifact@v7
|
||||
if: always()
|
||||
with:
|
||||
name: validation-report-02-agents-github-copilot
|
||||
path: python/samples/sample_validation/reports/
|
||||
|
||||
validate-02-agents-amazon:
|
||||
name: Validate 02-agents/providers/amazon
|
||||
if: false # Temporarily disabled - requires AWS credentials
|
||||
runs-on: ubuntu-latest
|
||||
environment: integration
|
||||
env:
|
||||
BEDROCK_CHAT_MODEL_ID: ${{ vars.BEDROCK__CHATMODELID }}
|
||||
defaults:
|
||||
run:
|
||||
working-directory: python
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- name: Setup environment
|
||||
uses: ./.github/actions/sample-validation-setup
|
||||
with:
|
||||
azure-client-id: ${{ secrets.AZURE_CLIENT_ID }}
|
||||
azure-tenant-id: ${{ secrets.AZURE_TENANT_ID }}
|
||||
azure-subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
|
||||
os: ${{ runner.os }}
|
||||
|
||||
- name: Run sample validation
|
||||
run: |
|
||||
cd scripts && uv run python -m sample_validation --subdir 02-agents/providers/amazon --save-report --report-name 02-agents-amazon
|
||||
|
||||
- name: Upload validation report
|
||||
uses: actions/upload-artifact@v7
|
||||
if: always()
|
||||
with:
|
||||
name: validation-report-02-agents-amazon
|
||||
path: python/samples/sample_validation/reports/
|
||||
|
||||
validate-02-agents-ollama:
|
||||
name: Validate 02-agents/providers/ollama
|
||||
if: false # Temporarily disabled - requires local Ollama server
|
||||
runs-on: ubuntu-latest
|
||||
environment: integration
|
||||
env:
|
||||
OLLAMA_MODEL: ${{ vars.OLLAMA__MODEL }}
|
||||
defaults:
|
||||
run:
|
||||
working-directory: python
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- name: Setup environment
|
||||
uses: ./.github/actions/sample-validation-setup
|
||||
with:
|
||||
azure-client-id: ${{ secrets.AZURE_CLIENT_ID }}
|
||||
azure-tenant-id: ${{ secrets.AZURE_TENANT_ID }}
|
||||
azure-subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
|
||||
os: ${{ runner.os }}
|
||||
|
||||
- name: Run sample validation
|
||||
run: |
|
||||
cd scripts && uv run python -m sample_validation --subdir 02-agents/providers/ollama --save-report --report-name 02-agents-ollama
|
||||
|
||||
- name: Upload validation report
|
||||
uses: actions/upload-artifact@v7
|
||||
if: always()
|
||||
with:
|
||||
name: validation-report-02-agents-ollama
|
||||
path: python/samples/sample_validation/reports/
|
||||
|
||||
validate-02-agents-foundry-local:
|
||||
name: Validate 02-agents/providers/foundry_local
|
||||
if: false # Temporarily disabled - requires local Foundry setup
|
||||
runs-on: ubuntu-latest
|
||||
environment: integration
|
||||
defaults:
|
||||
run:
|
||||
working-directory: python
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- name: Setup environment
|
||||
uses: ./.github/actions/sample-validation-setup
|
||||
with:
|
||||
azure-client-id: ${{ secrets.AZURE_CLIENT_ID }}
|
||||
azure-tenant-id: ${{ secrets.AZURE_TENANT_ID }}
|
||||
azure-subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
|
||||
os: ${{ runner.os }}
|
||||
|
||||
- name: Run sample validation
|
||||
run: |
|
||||
cd scripts && uv run python -m sample_validation --subdir 02-agents/providers/foundry_local --save-report --report-name 02-agents-foundry-local
|
||||
|
||||
- name: Upload validation report
|
||||
uses: actions/upload-artifact@v7
|
||||
if: always()
|
||||
with:
|
||||
name: validation-report-02-agents-foundry-local
|
||||
path: python/samples/sample_validation/reports/
|
||||
|
||||
validate-02-agents-copilotstudio:
|
||||
name: Validate 02-agents/providers/copilotstudio
|
||||
if: false # Temporarily disabled - requires Copilot Studio setup
|
||||
runs-on: ubuntu-latest
|
||||
environment: integration
|
||||
env:
|
||||
COPILOTSTUDIOAGENT__ENVIRONMENTID: ${{ secrets.COPILOTSTUDIOAGENT__ENVIRONMENTID }}
|
||||
COPILOTSTUDIOAGENT__SCHEMANAME: ${{ secrets.COPILOTSTUDIOAGENT__SCHEMANAME }}
|
||||
COPILOTSTUDIOAGENT__TENANTID: ${{ secrets.COPILOTSTUDIOAGENT__TENANTID }}
|
||||
COPILOTSTUDIOAGENT__AGENTAPPID: ${{ secrets.COPILOTSTUDIOAGENT__AGENTAPPID }}
|
||||
defaults:
|
||||
run:
|
||||
working-directory: python
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- name: Setup environment
|
||||
uses: ./.github/actions/sample-validation-setup
|
||||
with:
|
||||
azure-client-id: ${{ secrets.AZURE_CLIENT_ID }}
|
||||
azure-tenant-id: ${{ secrets.AZURE_TENANT_ID }}
|
||||
azure-subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
|
||||
os: ${{ runner.os }}
|
||||
|
||||
- name: Create .env for samples
|
||||
run: |
|
||||
echo "COPILOTSTUDIOAGENT__ENVIRONMENTID=$COPILOTSTUDIOAGENT__ENVIRONMENTID" >> .env
|
||||
echo "COPILOTSTUDIOAGENT__SCHEMANAME=$COPILOTSTUDIOAGENT__SCHEMANAME" >> .env
|
||||
echo "COPILOTSTUDIOAGENT__TENANTID=$COPILOTSTUDIOAGENT__TENANTID" >> .env
|
||||
echo "COPILOTSTUDIOAGENT__AGENTAPPID=$COPILOTSTUDIOAGENT__AGENTAPPID" >> .env
|
||||
|
||||
- name: Run sample validation
|
||||
run: |
|
||||
cd scripts && uv run python -m sample_validation --subdir 02-agents/providers/copilotstudio --save-report --report-name 02-agents-copilotstudio
|
||||
|
||||
- name: Upload validation report
|
||||
uses: actions/upload-artifact@v7
|
||||
if: always()
|
||||
with:
|
||||
name: validation-report-02-agents-copilotstudio
|
||||
path: python/samples/sample_validation/reports/
|
||||
|
||||
validate-02-agents-custom:
|
||||
name: Validate 02-agents/providers/custom
|
||||
runs-on: ubuntu-latest
|
||||
environment: integration
|
||||
defaults:
|
||||
run:
|
||||
working-directory: python
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- name: Setup environment
|
||||
uses: ./.github/actions/sample-validation-setup
|
||||
with:
|
||||
azure-client-id: ${{ secrets.AZURE_CLIENT_ID }}
|
||||
azure-tenant-id: ${{ secrets.AZURE_TENANT_ID }}
|
||||
azure-subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
|
||||
os: ${{ runner.os }}
|
||||
|
||||
- name: Run sample validation
|
||||
run: |
|
||||
cd scripts && uv run python -m sample_validation --subdir 02-agents/providers/custom --save-report --report-name 02-agents-custom
|
||||
|
||||
- name: Upload validation report
|
||||
uses: actions/upload-artifact@v7
|
||||
if: always()
|
||||
with:
|
||||
name: validation-report-02-agents-custom
|
||||
path: python/samples/sample_validation/reports/
|
||||
|
||||
validate-03-workflows:
|
||||
name: Validate 03-workflows
|
||||
runs-on: ubuntu-latest
|
||||
environment: integration
|
||||
env:
|
||||
# Azure AI configuration
|
||||
AZURE_AI_PROJECT_ENDPOINT: ${{ vars.AZURE_AI_PROJECT_ENDPOINT }}
|
||||
AZURE_AI_MODEL_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
|
||||
# Azure OpenAI configuration
|
||||
AZURE_OPENAI_ENDPOINT: ${{ vars.AZUREOPENAI__ENDPOINT }}
|
||||
AZURE_OPENAI_CHAT_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__CHATDEPLOYMENTNAME }}
|
||||
AZURE_OPENAI_RESPONSES_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
|
||||
defaults:
|
||||
run:
|
||||
working-directory: python
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- name: Setup environment
|
||||
uses: ./.github/actions/sample-validation-setup
|
||||
with:
|
||||
azure-client-id: ${{ secrets.AZURE_CLIENT_ID }}
|
||||
azure-tenant-id: ${{ secrets.AZURE_TENANT_ID }}
|
||||
azure-subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
|
||||
os: ${{ runner.os }}
|
||||
|
||||
- name: Create .env for samples
|
||||
run: |
|
||||
echo "AZURE_AI_PROJECT_ENDPOINT=$AZURE_AI_PROJECT_ENDPOINT" >> .env
|
||||
echo "AZURE_AI_MODEL_DEPLOYMENT_NAME=$AZURE_AI_MODEL_DEPLOYMENT_NAME" >> .env
|
||||
echo "AZURE_OPENAI_ENDPOINT=$AZURE_OPENAI_ENDPOINT" >> .env
|
||||
echo "AZURE_OPENAI_CHAT_DEPLOYMENT_NAME=$AZURE_OPENAI_CHAT_DEPLOYMENT_NAME" >> .env
|
||||
echo "AZURE_OPENAI_RESPONSES_DEPLOYMENT_NAME=$AZURE_OPENAI_RESPONSES_DEPLOYMENT_NAME" >> .env
|
||||
|
||||
- name: Run sample validation
|
||||
run: |
|
||||
cd scripts && uv run python -m sample_validation --subdir 03-workflows --save-report --report-name 03-workflows
|
||||
|
||||
- name: Upload validation report
|
||||
uses: actions/upload-artifact@v7
|
||||
if: always()
|
||||
with:
|
||||
name: validation-report-03-workflows
|
||||
path: python/samples/sample_validation/reports/
|
||||
|
||||
validate-04-hosting:
|
||||
name: Validate 04-hosting
|
||||
if: false # Temporarily disabled because of sample complexity
|
||||
runs-on: ubuntu-latest
|
||||
environment: integration
|
||||
env:
|
||||
# Azure AI configuration
|
||||
AZURE_AI_PROJECT_ENDPOINT: ${{ vars.AZURE_AI_PROJECT_ENDPOINT }}
|
||||
AZURE_AI_MODEL_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
|
||||
# Azure OpenAI configuration
|
||||
AZURE_OPENAI_ENDPOINT: ${{ vars.AZUREOPENAI__ENDPOINT }}
|
||||
AZURE_OPENAI_RESPONSES_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
|
||||
# A2A configuration
|
||||
A2A_AGENT_HOST: http://localhost:5001/
|
||||
defaults:
|
||||
run:
|
||||
working-directory: python
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- name: Setup environment
|
||||
uses: ./.github/actions/sample-validation-setup
|
||||
with:
|
||||
azure-client-id: ${{ secrets.AZURE_CLIENT_ID }}
|
||||
azure-tenant-id: ${{ secrets.AZURE_TENANT_ID }}
|
||||
azure-subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
|
||||
os: ${{ runner.os }}
|
||||
|
||||
- name: Run sample validation
|
||||
run: |
|
||||
cd scripts && uv run python -m sample_validation --subdir 04-hosting --save-report --report-name 04-hosting
|
||||
|
||||
- name: Upload validation report
|
||||
uses: actions/upload-artifact@v7
|
||||
if: always()
|
||||
with:
|
||||
name: validation-report-04-hosting
|
||||
path: python/samples/sample_validation/reports/
|
||||
|
||||
validate-05-end-to-end:
|
||||
name: Validate 05-end-to-end
|
||||
if: false # Temporarily disabled because of sample complexity
|
||||
runs-on: ubuntu-latest
|
||||
environment: integration
|
||||
env:
|
||||
# Azure AI configuration
|
||||
AZURE_AI_PROJECT_ENDPOINT: ${{ vars.AZURE_AI_PROJECT_ENDPOINT }}
|
||||
AZURE_AI_MODEL_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
|
||||
# Azure OpenAI configuration
|
||||
AZURE_OPENAI_ENDPOINT: ${{ vars.AZUREOPENAI__ENDPOINT }}
|
||||
AZURE_OPENAI_CHAT_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__CHATDEPLOYMENTNAME }}
|
||||
AZURE_OPENAI_RESPONSES_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
|
||||
# Azure AI Search (for evaluation samples)
|
||||
AZURE_SEARCH_ENDPOINT: ${{ secrets.AZURE_SEARCH_ENDPOINT }}
|
||||
AZURE_SEARCH_API_KEY: ${{ secrets.AZURE_SEARCH_API_KEY }}
|
||||
AZURE_SEARCH_INDEX_NAME: ${{ secrets.AZURE_SEARCH_INDEX_NAME }}
|
||||
# Evaluation sample
|
||||
AZURE_AI_MODEL_DEPLOYMENT_NAME_WORKFLOW: ${{ vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
|
||||
defaults:
|
||||
run:
|
||||
working-directory: python
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- name: Setup environment
|
||||
uses: ./.github/actions/sample-validation-setup
|
||||
with:
|
||||
azure-client-id: ${{ secrets.AZURE_CLIENT_ID }}
|
||||
azure-tenant-id: ${{ secrets.AZURE_TENANT_ID }}
|
||||
azure-subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
|
||||
os: ${{ runner.os }}
|
||||
|
||||
- name: Run sample validation
|
||||
run: |
|
||||
cd scripts && uv run python -m sample_validation --subdir 05-end-to-end --save-report --report-name 05-end-to-end
|
||||
|
||||
- name: Upload validation report
|
||||
uses: actions/upload-artifact@v7
|
||||
if: always()
|
||||
with:
|
||||
name: validation-report-05-end-to-end
|
||||
path: python/samples/sample_validation/reports/
|
||||
|
||||
validate-autogen-migration:
|
||||
name: Validate autogen-migration
|
||||
runs-on: ubuntu-latest
|
||||
environment: integration
|
||||
env:
|
||||
# Azure AI configuration
|
||||
AZURE_AI_PROJECT_ENDPOINT: ${{ vars.AZURE_AI_PROJECT_ENDPOINT }}
|
||||
AZURE_AI_MODEL_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
|
||||
# Azure OpenAI configuration
|
||||
AZURE_OPENAI_ENDPOINT: ${{ vars.AZUREOPENAI__ENDPOINT }}
|
||||
AZURE_OPENAI_CHAT_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__CHATDEPLOYMENTNAME }}
|
||||
# OpenAI configuration
|
||||
OPENAI_API_KEY: ${{ secrets.OPENAI__APIKEY }}
|
||||
OPENAI_CHAT_MODEL_ID: ${{ vars.OPENAI__CHATMODELID }}
|
||||
OPENAI_RESPONSES_MODEL_ID: ${{ vars.OPENAI__RESPONSESMODELID }}
|
||||
defaults:
|
||||
run:
|
||||
working-directory: python
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- name: Setup environment
|
||||
uses: ./.github/actions/sample-validation-setup
|
||||
with:
|
||||
azure-client-id: ${{ secrets.AZURE_CLIENT_ID }}
|
||||
azure-tenant-id: ${{ secrets.AZURE_TENANT_ID }}
|
||||
azure-subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
|
||||
os: ${{ runner.os }}
|
||||
|
||||
- name: Create .env for samples
|
||||
run: |
|
||||
echo "AZURE_AI_PROJECT_ENDPOINT=$AZURE_AI_PROJECT_ENDPOINT" >> .env
|
||||
echo "AZURE_AI_MODEL_DEPLOYMENT_NAME=$AZURE_AI_MODEL_DEPLOYMENT_NAME" >> .env
|
||||
echo "AZURE_OPENAI_ENDPOINT=$AZURE_OPENAI_ENDPOINT" >> .env
|
||||
echo "AZURE_OPENAI_CHAT_DEPLOYMENT_NAME=$AZURE_OPENAI_CHAT_DEPLOYMENT_NAME" >> .env
|
||||
echo "OPENAI_API_KEY=$OPENAI_API_KEY" >> .env
|
||||
echo "OPENAI_CHAT_MODEL_ID=$OPENAI_CHAT_MODEL_ID" >> .env
|
||||
echo "OPENAI_RESPONSES_MODEL_ID=$OPENAI_RESPONSES_MODEL_ID" >> .env
|
||||
|
||||
- name: Run sample validation
|
||||
run: |
|
||||
cd scripts && uv run python -m sample_validation --subdir autogen-migration --save-report --report-name autogen-migration
|
||||
|
||||
- name: Upload validation report
|
||||
uses: actions/upload-artifact@v7
|
||||
if: always()
|
||||
with:
|
||||
name: validation-report-autogen-migration
|
||||
path: python/samples/sample_validation/reports/
|
||||
|
||||
validate-semantic-kernel-migration:
|
||||
name: Validate semantic-kernel-migration
|
||||
runs-on: ubuntu-latest
|
||||
environment: integration
|
||||
env:
|
||||
# Azure AI configuration
|
||||
AZURE_AI_PROJECT_ENDPOINT: ${{ vars.AZURE_AI_PROJECT_ENDPOINT }}
|
||||
AZURE_AI_MODEL_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
|
||||
# Azure OpenAI configuration
|
||||
AZURE_OPENAI_ENDPOINT: ${{ vars.AZUREOPENAI__ENDPOINT }}
|
||||
AZURE_OPENAI_CHAT_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__CHATDEPLOYMENTNAME }}
|
||||
AZURE_OPENAI_RESPONSES_DEPLOYMENT_NAME: ${{ vars.AZUREOPENAI__RESPONSESDEPLOYMENTNAME }}
|
||||
# OpenAI configuration
|
||||
OPENAI_API_KEY: ${{ secrets.OPENAI__APIKEY }}
|
||||
OPENAI_CHAT_MODEL_ID: ${{ vars.OPENAI__CHATMODELID }}
|
||||
OPENAI_RESPONSES_MODEL_ID: ${{ vars.OPENAI__RESPONSESMODELID }}
|
||||
# Copilot Studio
|
||||
COPILOTSTUDIOAGENT__ENVIRONMENTID: ${{ secrets.COPILOTSTUDIOAGENT__ENVIRONMENTID }}
|
||||
COPILOTSTUDIOAGENT__SCHEMANAME: ${{ secrets.COPILOTSTUDIOAGENT__SCHEMANAME }}
|
||||
COPILOTSTUDIOAGENT__TENANTID: ${{ secrets.COPILOTSTUDIOAGENT__TENANTID }}
|
||||
COPILOTSTUDIOAGENT__AGENTAPPID: ${{ secrets.COPILOTSTUDIOAGENT__AGENTAPPID }}
|
||||
defaults:
|
||||
run:
|
||||
working-directory: python
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- name: Setup environment
|
||||
uses: ./.github/actions/sample-validation-setup
|
||||
with:
|
||||
azure-client-id: ${{ secrets.AZURE_CLIENT_ID }}
|
||||
azure-tenant-id: ${{ secrets.AZURE_TENANT_ID }}
|
||||
azure-subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
|
||||
os: ${{ runner.os }}
|
||||
|
||||
- name: Create .env for samples
|
||||
run: |
|
||||
echo "AZURE_AI_PROJECT_ENDPOINT=$AZURE_AI_PROJECT_ENDPOINT" >> .env
|
||||
echo "AZURE_AI_MODEL_DEPLOYMENT_NAME=$AZURE_AI_MODEL_DEPLOYMENT_NAME" >> .env
|
||||
echo "AZURE_OPENAI_ENDPOINT=$AZURE_OPENAI_ENDPOINT" >> .env
|
||||
echo "AZURE_OPENAI_CHAT_DEPLOYMENT_NAME=$AZURE_OPENAI_CHAT_DEPLOYMENT_NAME" >> .env
|
||||
echo "AZURE_OPENAI_RESPONSES_DEPLOYMENT_NAME=$AZURE_OPENAI_RESPONSES_DEPLOYMENT_NAME" >> .env
|
||||
echo "OPENAI_API_KEY=$OPENAI_API_KEY" >> .env
|
||||
echo "OPENAI_CHAT_MODEL_ID=$OPENAI_CHAT_MODEL_ID" >> .env
|
||||
echo "OPENAI_RESPONSES_MODEL_ID=$OPENAI_RESPONSES_MODEL_ID" >> .env
|
||||
echo "COPILOTSTUDIOAGENT__ENVIRONMENTID=$COPILOTSTUDIOAGENT__ENVIRONMENTID" >> .env
|
||||
echo "COPILOTSTUDIOAGENT__SCHEMANAME=$COPILOTSTUDIOAGENT__SCHEMANAME" >> .env
|
||||
echo "COPILOTSTUDIOAGENT__TENANTID=$COPILOTSTUDIOAGENT__TENANTID" >> .env
|
||||
echo "COPILOTSTUDIOAGENT__AGENTAPPID=$COPILOTSTUDIOAGENT__AGENTAPPID" >> .env
|
||||
|
||||
- name: Run sample validation
|
||||
run: |
|
||||
cd scripts && uv run python -m sample_validation --subdir semantic-kernel-migration --save-report --report-name semantic-kernel-migration
|
||||
|
||||
- name: Upload validation report
|
||||
uses: actions/upload-artifact@v7
|
||||
if: always()
|
||||
with:
|
||||
name: validation-report-semantic-kernel-migration
|
||||
path: python/samples/sample_validation/reports/
|
||||
|
||||
aggregate-results:
|
||||
name: Aggregate Results
|
||||
runs-on: ubuntu-latest
|
||||
if: always()
|
||||
needs:
|
||||
- validate-01-get-started
|
||||
- validate-02-agents
|
||||
- validate-02-agents-openai
|
||||
- validate-02-agents-azure-openai
|
||||
- validate-02-agents-azure-ai
|
||||
- validate-02-agents-azure-ai-agent
|
||||
- validate-02-agents-anthropic
|
||||
- validate-02-agents-github-copilot
|
||||
- validate-02-agents-amazon
|
||||
- validate-02-agents-ollama
|
||||
- validate-02-agents-foundry-local
|
||||
- validate-02-agents-copilotstudio
|
||||
- validate-02-agents-custom
|
||||
- validate-03-workflows
|
||||
- validate-04-hosting
|
||||
- validate-05-end-to-end
|
||||
- validate-autogen-migration
|
||||
- validate-semantic-kernel-migration
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- name: Download all validation reports
|
||||
uses: actions/download-artifact@v7
|
||||
with:
|
||||
pattern: validation-report-*
|
||||
path: reports/
|
||||
merge-multiple: true
|
||||
|
||||
- name: Restore validation history
|
||||
id: cache-restore
|
||||
uses: actions/cache/restore@v4
|
||||
with:
|
||||
path: validation-history/
|
||||
key: validation-history-${{ github.run_id }}
|
||||
restore-keys: |
|
||||
validation-history-
|
||||
|
||||
- name: Aggregate results and generate trend report
|
||||
run: |
|
||||
python3 python/scripts/sample_validation/aggregate.py \
|
||||
reports/ \
|
||||
validation-history/history.json \
|
||||
trend-report.md
|
||||
|
||||
- name: Write trend report to job summary
|
||||
run: cat trend-report.md >> "$GITHUB_STEP_SUMMARY"
|
||||
|
||||
- name: Save validation history
|
||||
uses: actions/cache/save@v4
|
||||
with:
|
||||
path: validation-history/
|
||||
key: validation-history-${{ github.run_id }}
|
||||
|
||||
- name: Upload trend report
|
||||
uses: actions/upload-artifact@v7
|
||||
if: always()
|
||||
with:
|
||||
name: validation-trend-report
|
||||
path: trend-report.md
|
||||
@@ -21,7 +21,7 @@ jobs:
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- name: Download coverage report
|
||||
uses: actions/download-artifact@v8
|
||||
uses: actions/download-artifact@v7
|
||||
with:
|
||||
github-token: ${{ secrets.GH_ACTIONS_PR_WRITE }}
|
||||
run-id: ${{ github.event.workflow_run.id }}
|
||||
@@ -46,7 +46,7 @@ jobs:
|
||||
echo "PR_NUMBER=$PR_NUMBER" >> "$GITHUB_ENV"
|
||||
- name: Pytest coverage comment
|
||||
id: coverageComment
|
||||
uses: MishaKav/pytest-coverage-comment@v1.6.0
|
||||
uses: MishaKav/pytest-coverage-comment@v1.2.0
|
||||
with:
|
||||
github-token: ${{ secrets.GH_ACTIONS_PR_WRITE }}
|
||||
issue-number: ${{ env.PR_NUMBER }}
|
||||
|
||||
@@ -20,7 +20,7 @@ jobs:
|
||||
run:
|
||||
working-directory: python
|
||||
env:
|
||||
UV_PYTHON: "3.11"
|
||||
UV_PYTHON: "3.10"
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
# Save the PR number to a file since the workflow_run event
|
||||
@@ -32,17 +32,17 @@ jobs:
|
||||
id: python-setup
|
||||
uses: ./.github/actions/python-setup
|
||||
with:
|
||||
python-version: ${{ env.UV_PYTHON }}
|
||||
python-version: ${{ matrix.python-version }}
|
||||
os: ${{ runner.os }}
|
||||
env:
|
||||
# Configure a constant location for the uv cache
|
||||
UV_CACHE_DIR: /tmp/.uv-cache
|
||||
- name: Run all tests with coverage report
|
||||
run: uv run poe test -A -C --cov-report=xml:python-coverage.xml -q --junitxml=pytest.xml
|
||||
run: uv run poe all-tests-cov --cov-report=xml:python-coverage.xml -q --junitxml=pytest.xml
|
||||
- name: Check coverage threshold
|
||||
run: python ${{ github.workspace }}/.github/workflows/python-check-coverage.py python-coverage.xml ${{ env.COVERAGE_THRESHOLD }}
|
||||
- name: Upload coverage report
|
||||
uses: actions/upload-artifact@v7
|
||||
uses: actions/upload-artifact@v6
|
||||
with:
|
||||
path: |
|
||||
python/python-coverage.xml
|
||||
|
||||
@@ -34,13 +34,12 @@ jobs:
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
os: ${{ runner.os }}
|
||||
exclude-packages: ${{ matrix.python-version == '3.10' && 'agent-framework-github-copilot' || '' }}
|
||||
env:
|
||||
# Configure a constant location for the uv cache
|
||||
UV_CACHE_DIR: /tmp/.uv-cache
|
||||
# Unit tests
|
||||
- name: Run all tests
|
||||
run: uv run poe test -A
|
||||
run: uv run poe all-tests
|
||||
working-directory: ./python
|
||||
|
||||
# Surface failing tests
|
||||
|
||||
@@ -1,49 +0,0 @@
|
||||
name: Stale issue and PR ping
|
||||
|
||||
on:
|
||||
schedule:
|
||||
- cron: '0 0 * * *' # Midnight UTC daily
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
days_threshold:
|
||||
description: 'Days of silence before pinging the author'
|
||||
required: false
|
||||
default: '4'
|
||||
dry_run:
|
||||
description: 'Log what would be pinged without taking action'
|
||||
required: false
|
||||
default: 'false'
|
||||
type: choice
|
||||
options:
|
||||
- 'false'
|
||||
- 'true'
|
||||
|
||||
concurrency:
|
||||
group: stale-issue-pr-ping
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
ping_stale:
|
||||
name: "Ping stale issues and PRs"
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: read
|
||||
issues: write
|
||||
pull-requests: write
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: '3.13'
|
||||
|
||||
- name: Install dependencies
|
||||
run: pip install PyGithub==2.6.0
|
||||
|
||||
- name: Run stale issue/PR ping
|
||||
run: python .github/scripts/stale_issue_pr_ping.py
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GH_ACTIONS_PR_WRITE }}
|
||||
TEAM_SLUG: ${{ secrets.DEVELOPER_TEAM }}
|
||||
DAYS_THRESHOLD: ${{ github.event.inputs.days_threshold || '4' }}
|
||||
DRY_RUN: ${{ github.event.inputs.dry_run || 'false' }}
|
||||
+2
-3
@@ -199,15 +199,14 @@ temp*/
|
||||
.tmp/
|
||||
.temp/
|
||||
|
||||
agents.md
|
||||
|
||||
# AI
|
||||
.claude/
|
||||
WARP.md
|
||||
**/memory-bank/
|
||||
**/projectBrief.md
|
||||
**/tmpclaude*
|
||||
# Dependency-bound validation reports
|
||||
python/scripts/dependency-*-results.json
|
||||
python/scripts/dependencies/dependency-*-results.json
|
||||
|
||||
# Azurite storage emulator files
|
||||
*/__azurite_db_blob__.json*
|
||||
|
||||
@@ -53,7 +53,7 @@ Still have questions? Join our [weekly office hours](./COMMUNITY.md#public-commu
|
||||
### ✨ **Highlights**
|
||||
|
||||
- **Graph-based Workflows**: Connect agents and deterministic functions using data flows with streaming, checkpointing, human-in-the-loop, and time-travel capabilities
|
||||
- [Python workflows](./python/samples/03-workflows/) | [.NET workflows](./dotnet/samples/03-workflows/)
|
||||
- [Python workflows](./python/samples/getting_started/workflows/) | [.NET workflows](./dotnet/samples/GettingStarted/Workflows/)
|
||||
- **AF Labs**: Experimental packages for cutting-edge features including benchmarking, reinforcement learning, and research initiatives
|
||||
- [Labs directory](./python/packages/lab/)
|
||||
- **DevUI**: Interactive developer UI for agent development, testing, and debugging workflows
|
||||
@@ -73,11 +73,11 @@ Still have questions? Join our [weekly office hours](./COMMUNITY.md#public-commu
|
||||
- **Python and C#/.NET Support**: Full framework support for both Python and C#/.NET implementations with consistent APIs
|
||||
- [Python packages](./python/packages/) | [.NET source](./dotnet/src/)
|
||||
- **Observability**: Built-in OpenTelemetry integration for distributed tracing, monitoring, and debugging
|
||||
- [Python observability](./python/samples/02-agents/observability/) | [.NET telemetry](./dotnet/samples/02-agents/AgentOpenTelemetry/)
|
||||
- [Python observability](./python/samples/getting_started/observability/) | [.NET telemetry](./dotnet/samples/GettingStarted/AgentOpenTelemetry/)
|
||||
- **Multiple Agent Provider Support**: Support for various LLM providers with more being added continuously
|
||||
- [Python examples](./python/samples/02-agents/providers/) | [.NET examples](./dotnet/samples/02-agents/AgentProviders/)
|
||||
- [Python examples](./python/samples/getting_started/agents/) | [.NET examples](./dotnet/samples/GettingStarted/AgentProviders/)
|
||||
- **Middleware**: Flexible middleware system for request/response processing, exception handling, and custom pipelines
|
||||
- [Python middleware](./python/samples/02-agents/middleware/) | [.NET middleware](./dotnet/samples/02-agents/Agents/Agent_Step11_Middleware/)
|
||||
- [Python middleware](./python/samples/getting_started/middleware/) | [.NET middleware](./dotnet/samples/GettingStarted/Agents/Agent_Step14_Middleware/)
|
||||
|
||||
### 💬 **We want your feedback!**
|
||||
|
||||
@@ -125,13 +125,12 @@ Create a simple Agent, using OpenAI Responses, that writes a haiku about the Mic
|
||||
|
||||
```c#
|
||||
// dotnet add package Microsoft.Agents.AI.OpenAI --prerelease
|
||||
using Microsoft.Agents.AI;
|
||||
using System;
|
||||
using OpenAI;
|
||||
using OpenAI.Responses;
|
||||
|
||||
// Replace the <apikey> with your OpenAI API key.
|
||||
var agent = new OpenAIClient("<apikey>")
|
||||
.GetResponsesClient("gpt-4o-mini")
|
||||
.GetOpenAIResponseClient("gpt-4o-mini")
|
||||
.AsAIAgent(name: "HaikuBot", instructions: "You are an upbeat assistant that writes beautifully.");
|
||||
|
||||
Console.WriteLine(await agent.RunAsync("Write a haiku about Microsoft Agent Framework."));
|
||||
@@ -143,17 +142,14 @@ Create a simple Agent, using Azure OpenAI Responses with token based auth, that
|
||||
// dotnet add package Microsoft.Agents.AI.OpenAI --prerelease
|
||||
// dotnet add package Azure.Identity
|
||||
// Use `az login` to authenticate with Azure CLI
|
||||
using System.ClientModel.Primitives;
|
||||
using Azure.Identity;
|
||||
using Microsoft.Agents.AI;
|
||||
using System;
|
||||
using OpenAI;
|
||||
using OpenAI.Responses;
|
||||
|
||||
// Replace <resource> and gpt-4o-mini with your Azure OpenAI resource name and deployment name.
|
||||
var agent = new OpenAIClient(
|
||||
new BearerTokenPolicy(new AzureCliCredential(), "https://ai.azure.com/.default"),
|
||||
new OpenAIClientOptions() { Endpoint = new Uri("https://<resource>.openai.azure.com/openai/v1") })
|
||||
.GetResponsesClient("gpt-4o-mini")
|
||||
.GetOpenAIResponseClient("gpt-4o-mini")
|
||||
.AsAIAgent(name: "HaikuBot", instructions: "You are an upbeat assistant that writes beautifully.");
|
||||
|
||||
Console.WriteLine(await agent.RunAsync("Write a haiku about Microsoft Agent Framework."));
|
||||
@@ -163,15 +159,15 @@ Console.WriteLine(await agent.RunAsync("Write a haiku about Microsoft Agent Fram
|
||||
|
||||
### Python
|
||||
|
||||
- [Getting Started with Agents](./python/samples/01-get-started): progressive tutorial from hello-world to hosting
|
||||
- [Agent Concepts](./python/samples/02-agents): deep-dive samples by topic (tools, middleware, providers, etc.)
|
||||
- [Getting Started with Workflows](./python/samples/03-workflows): workflow creation and integration with agents
|
||||
- [Getting Started with Agents](./python/samples/getting_started/agents): basic agent creation and tool usage
|
||||
- [Chat Client Examples](./python/samples/getting_started/chat_client): direct chat client usage patterns
|
||||
- [Getting Started with Workflows](./python/samples/getting_started/workflows): basic workflow creation and integration with agents
|
||||
|
||||
### .NET
|
||||
|
||||
- [Getting Started with Agents](./dotnet/samples/02-agents/Agents): basic agent creation and tool usage
|
||||
- [Agent Provider Samples](./dotnet/samples/02-agents/AgentProviders): samples showing different agent providers
|
||||
- [Workflow Samples](./dotnet/samples/03-workflows): advanced multi-agent patterns and workflow orchestration
|
||||
- [Getting Started with Agents](./dotnet/samples/GettingStarted/Agents): basic agent creation and tool usage
|
||||
- [Agent Provider Samples](./dotnet/samples/GettingStarted/AgentProviders): samples showing different agent providers
|
||||
- [Workflow Samples](./dotnet/samples/GettingStarted/Workflows): advanced multi-agent patterns and workflow orchestration
|
||||
|
||||
## Contributor Resources
|
||||
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
# Declarative Agents
|
||||
|
||||
This folder contains sample agent definitions that can be run using the declarative agent support, for python see the [declarative agent python sample folder](../python/samples/02-agents/declarative/).
|
||||
This folder contains sample agent definitions that can be run using the declarative agent support, for python see the [declarative agent python sample folder](../python/samples/getting_started/declarative/).
|
||||
|
||||
@@ -11,7 +11,7 @@ model:
|
||||
topP: 0.95
|
||||
connection:
|
||||
kind: key
|
||||
apiKey: =Env.OPENAI_API_KEY
|
||||
apiKey: =Env.OPENAI_APIKEY
|
||||
outputSchema:
|
||||
properties:
|
||||
language:
|
||||
|
||||
@@ -4,8 +4,8 @@ status: accepted
|
||||
contact: westey-m
|
||||
date: 2025-07-10 {YYYY-MM-DD when the decision was last updated}
|
||||
deciders: sergeymenshykh, markwallace, rbarreto, dmytrostruk, westey-m, eavanvalkenburg, stephentoub
|
||||
consulted:
|
||||
informed:
|
||||
consulted:
|
||||
informed:
|
||||
---
|
||||
|
||||
# Agent Run Responses Design
|
||||
@@ -64,7 +64,7 @@ Approaches observed from the compared SDKs:
|
||||
| AutoGen | **Approach 1** Separates messages into Agent-Agent (maps to Primary) and Internal (maps to Secondary) and these are returned as separate properties on the agent response object. See [types of messages](https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/tutorial/messages.html#types-of-messages) and [Response](https://microsoft.github.io/autogen/stable/reference/python/autogen_agentchat.base.html#autogen_agentchat.base.Response) | **Approach 2** Returns a stream of internal events and the last item is a Response object. See [ChatAgent.on_messages_stream](https://microsoft.github.io/autogen/stable/reference/python/autogen_agentchat.base.html#autogen_agentchat.base.ChatAgent.on_messages_stream) |
|
||||
| OpenAI Agent SDK | **Approach 1** Separates new_items (Primary+Secondary) from final output (Primary) as separate properties on the [RunResult](https://github.com/openai/openai-agents-python/blob/main/src/agents/result.py#L39) | **Approach 1** Similar to non-streaming, has a way of streaming updates via a method on the response object which includes all data, and then a separate final output property on the response object which is populated only when the run is complete. See [RunResultStreaming](https://github.com/openai/openai-agents-python/blob/main/src/agents/result.py#L136) |
|
||||
| Google ADK | **Approach 2** [Emits events](https://google.github.io/adk-docs/runtime/#step-by-step-breakdown) with [FinalResponse](https://github.com/google/adk-java/blob/main/core/src/main/java/com/google/adk/events/Event.java#L232) true (Primary) / false (Secondary) and callers have to filter out those with false to get just the final response message | **Approach 2** Similar to non-streaming except [events](https://google.github.io/adk-docs/runtime/#streaming-vs-non-streaming-output-partialtrue) are emitted with [Partial](https://github.com/google/adk-java/blob/main/core/src/main/java/com/google/adk/events/Event.java#L133) true to indicate that they are streaming messages. A final non partial event is also emitted. |
|
||||
| AWS (Strands) | **Approach 3** Returns an [AgentResult](https://strandsagents.com/docs/api/python/strands.agent.agent_result/) (Primary) with messages and a reason for the run's completion. | **Approach 2** [Streams events](https://strandsagents.com/docs/api/python/strands.agent.agent/) (Primary+Secondary) including, response text, current_tool_use, even data from "callbacks" (strands plugins) |
|
||||
| AWS (Strands) | **Approach 3** Returns an [AgentResult](https://strandsagents.com/latest/documentation/docs/api-reference/python/agent/agent_result/) (Primary) with messages and a reason for the run's completion. | **Approach 2** [Streams events](https://strandsagents.com/latest/documentation/docs/api-reference/python/agent/agent/#strands.agent.agent.Agent.stream_async) (Primary+Secondary) including, response text, current_tool_use, even data from "callbacks" (strands plugins) |
|
||||
| LangGraph | **Approach 2** A mixed list of all [messages](https://langchain-ai.github.io/langgraph/agents/run_agents/#output-format) | **Approach 2** A mixed list of all [messages](https://langchain-ai.github.io/langgraph/agents/run_agents/#output-format) |
|
||||
| Agno | **Combination of various approaches** Returns a [RunResponse](https://docs.agno.com/reference/agents/run-response) object with text content, messages (essentially chat history including inputs and instructions), reasoning and thinking text properties. Secondary events could potentially be extracted from messages. | **Approach 2** Returns [RunResponseEvent](https://docs.agno.com/reference/agents/run-response#runresponseevent-types-and-attributes) objects including tool call, memory update, etc, information, where the [RunResponseCompletedEvent](https://docs.agno.com/reference/agents/run-response#runresponsecompletedevent) has similar properties to RunResponse|
|
||||
| A2A | **Approach 3** Returns a [Task or Message](https://a2aproject.github.io/A2A/latest/specification/#71-messagesend) where the message is the final result (Primary) and task is a reference to a long running process. | **Approach 2** Returns a [stream](https://a2aproject.github.io/A2A/latest/specification/#72-messagestream) that contains task updates (Secondary) and a final message (Primary) |
|
||||
@@ -496,9 +496,9 @@ We need to decide what AIContent types, each agent response type will be mapped
|
||||
|-|-|
|
||||
| AutoGen | **Approach 1** Supports [configuring an agent](https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/tutorial/agents.html#structured-output) at agent creation. |
|
||||
| Google ADK | **Approach 1** Both [input and output schemas can be specified for LLM Agents](https://google.github.io/adk-docs/agents/llm-agents/#structuring-data-input_schema-output_schema-output_key) at construction time. This option is specific to this agent type and other agent types do not necessarily support |
|
||||
| AWS (Strands) | **Approach 2** Supports a special invocation method called [structured_output](https://strandsagents.com/docs/api/python/strands.agent.agent/) |
|
||||
| AWS (Strands) | **Approach 2** Supports a special invocation method called [structured_output](https://strandsagents.com/latest/documentation/docs/api-reference/python/agent/agent/#strands.agent.agent.Agent.structured_output) |
|
||||
| LangGraph | **Approach 1** Supports [configuring an agent](https://langchain-ai.github.io/langgraph/agents/agents/?h=structured#6-configure-structured-output) at agent construction time, and a [structured response](https://langchain-ai.github.io/langgraph/agents/run_agents/#output-format) can be retrieved as a special property on the agent response |
|
||||
| Agno | **Approach 1** Supports [configuring an agent](https://docs.agno.com/input-output/structured-output/agent) at agent construction time |
|
||||
| Agno | **Approach 1** Supports [configuring an agent](https://docs.agno.com/examples/getting-started/structured-output) at agent construction time |
|
||||
| A2A | **Informal Approach 2** Doesn't formally support schema negotiation, but [hints can be provided via metadata](https://a2a-protocol.org/latest/specification/#97-structured-data-exchange-requesting-and-providing-json) at invocation time |
|
||||
| Protocol Activity | Supports returning [Complex types](https://github.com/microsoft/Agents/blob/main/specs/activity/protocol-activity.md#complex-types) but no support for requesting a type |
|
||||
|
||||
@@ -508,7 +508,7 @@ We need to decide what AIContent types, each agent response type will be mapped
|
||||
|-|-|
|
||||
| AutoGen | Supports a [stop reason](https://microsoft.github.io/autogen/stable/reference/python/autogen_agentchat.base.html#autogen_agentchat.base.TaskResult.stop_reason) which is a freeform text string |
|
||||
| Google ADK | [No equivalent present](https://github.com/google/adk-python/blob/main/src/google/adk/events/event.py) |
|
||||
| AWS (Strands) | Exposes a [stop_reason](https://strandsagents.com/docs/api/python/strands.types.event_loop/) property on the [AgentResult](https://strandsagents.com/docs/api/python/strands.agent.agent_result/) class with options that are tied closely to LLM operations. |
|
||||
| AWS (Strands) | Exposes a [stop_reason](https://strandsagents.com/latest/documentation/docs/api-reference/python/types/event_loop/#strands.types.event_loop.StopReason) property on the [AgentResult](https://strandsagents.com/latest/documentation/docs/api-reference/python/agent/agent_result/) class with options that are tied closely to LLM operations. |
|
||||
| LangGraph | No equivalent present, output contains only [messages](https://langchain-ai.github.io/langgraph/agents/run_agents/#output-format) |
|
||||
| Agno | [No equivalent present](https://docs.agno.com/reference/agents/run-response) |
|
||||
| A2A | No equivalent present, response only contains a [message](https://a2a-protocol.org/latest/specification/#64-message-object) or [task](https://a2a-protocol.org/latest/specification/#61-task-object). |
|
||||
|
||||
@@ -126,4 +126,4 @@ response = await client.get_response(
|
||||
|
||||
Chosen option: **"Option 2: TypedDict with Generic Type Parameters"**, because it provides full type safety, excellent IDE support with autocompletion, and allows users to extend provider-specific options for their use cases. Extended this Generic to ChatAgents in order to also properly type the options used in agent construction and run methods.
|
||||
|
||||
See [typed_options.py](../../python/samples/02-agents/typed_options.py) for a complete example demonstrating the usage of typed options with custom extensions.
|
||||
See [typed_options.py](../../python/samples/getting_started/chat_client/typed_options.py) for a complete example demonstrating the usage of typed options with custom extensions.
|
||||
|
||||
@@ -1,147 +0,0 @@
|
||||
---
|
||||
status: proposed
|
||||
contact: westey-m
|
||||
date: 2026-01-27
|
||||
deciders: sergeymenshykh, markwallace, rbarreto, dmytrostruk, westey-m, eavanvalkenburg, stephentoub, lokitoth, alliscode, taochenosu, moonbox3
|
||||
consulted:
|
||||
informed:
|
||||
---
|
||||
|
||||
# AgentRunContext for Agent Run
|
||||
|
||||
## Context and Problem Statement
|
||||
|
||||
During an agent run, various components involved in the execution (middleware, filters, tools, nested agents, etc.) may need access to contextual information about the current run, such as:
|
||||
|
||||
1. The agent that is executing the run
|
||||
2. The session associated with the run
|
||||
3. The request messages passed to the agent
|
||||
4. The run options controlling the agent's behavior
|
||||
|
||||
Additionally, some components may need to modify this context during execution, for example:
|
||||
|
||||
- Replacing the session with a different one
|
||||
- Modifying the request messages before they reach the agent core
|
||||
- Updating or replacing the run options entirely
|
||||
|
||||
Currently, there is no standardized way to access or modify this context from arbitrary code that executes during an agent run, especially from deeply nested call stacks where the context is not explicitly passed.
|
||||
|
||||
## Sample Scenario
|
||||
|
||||
When using an Agent as an AIFunction developers may want to pass context from the parent agent run to the child agent run. For example, the developer may want to copy chat history to the child agent, or share the same session across both agents.
|
||||
|
||||
To enable these scenarios, we need a way to access the parent agent run context, including e.g. the parent agent itself, the parent agent session, and the parent run options from function tool calls.
|
||||
|
||||
```csharp
|
||||
public static AIFunction AsAIFunctionWithSessionPropagation(this ChatClientAgent agent, AIFunctionFactoryOptions? options = null)
|
||||
{
|
||||
Throw.IfNull(agent);
|
||||
|
||||
[Description("Invoke an agent to retrieve some information.")]
|
||||
async Task<string> InvokeAgentAsync(
|
||||
[Description("Input query to invoke the agent.")] string query,
|
||||
CancellationToken cancellationToken)
|
||||
{
|
||||
// Get the session from the parent agent and pass it to the child agent.
|
||||
var session = AIAgent.CurrentRunContext?.Session;
|
||||
|
||||
// Alternatively, the developer may want to create a new session but copy over the chat history from the parent agent.
|
||||
// var parentChatHistory = AIAgent.CurrentRunContext?.Session?.GetService<IList<ChatMessage>>();
|
||||
// if (parentChatHistory != null)
|
||||
// {
|
||||
// var chp = new InMemoryChatHistoryProvider();
|
||||
// foreach (var message in parentChatHistory)
|
||||
// {
|
||||
// chp.Add(message);
|
||||
// }
|
||||
// session = agent.GetNewSession(chp);
|
||||
// }
|
||||
|
||||
var response = await agent.RunAsync(query, session: session, cancellationToken: cancellationToken).ConfigureAwait(false);
|
||||
return response.Text;
|
||||
}
|
||||
|
||||
options ??= new();
|
||||
options.Name ??= SanitizeAgentName(agent.Name);
|
||||
options.Description ??= agent.Description;
|
||||
|
||||
return AIFunctionFactory.Create(InvokeAgentAsync, options);
|
||||
}
|
||||
```
|
||||
|
||||
## Decision Drivers
|
||||
|
||||
- Components executing during an agent run need access to run context without explicit parameter passing through every layer
|
||||
- Context should flow naturally across async calls without manual propagation
|
||||
- The design should allow modification of context properties by agent decorators (e.g., replacing options or session)
|
||||
- Solution should be consistent with patterns used in similar frameworks (e.g., `FunctionInvokingChatClient.CurrentContext` `HttpContext.Current`, `Activity.Current`)
|
||||
|
||||
## Considered Options
|
||||
|
||||
- **Option 1**: Pass context explicitly through all method signatures
|
||||
- **Option 2**: Use `AsyncLocal<T>` to provide ambient context accessible anywhere during the run
|
||||
- **Option 3**: Use a combination of explicit parameters for `RunCoreAsync` and `AsyncLocal<T>` for ambient access
|
||||
|
||||
## Decision Outcome
|
||||
|
||||
Chosen option: **Option 3** - Combination of explicit parameters and AsyncLocal ambient access.
|
||||
|
||||
This approach provides the best of both worlds:
|
||||
|
||||
1. **Explicit parameters are passed to `RunCoreAsync`**: The core agent implementation receives the parameters explicitly, making it clear what data is available and enabling easy unit testing. Any modification of these in a decorator will require calling `RunAsync` on the inner agent with the updated parameters, which would result in the inner agent creating a new `AgentRunContext` instance.
|
||||
|
||||
```csharp
|
||||
public async Task<AgentResponse> RunAsync(
|
||||
IEnumerable<ChatMessage> messages,
|
||||
AgentSession? session = null,
|
||||
AgentRunOptions? options = null,
|
||||
CancellationToken cancellationToken = default)
|
||||
{
|
||||
|
||||
CurrentRunContext = new(this, session, messages as IReadOnlyCollection<ChatMessage> ?? messages.ToList(), options);
|
||||
return await this.RunCoreAsync(messages, session, options, cancellationToken).ConfigureAwait(false);
|
||||
}
|
||||
```
|
||||
|
||||
2. **`AsyncLocal<AgentRunContext?>` for ambient access**: The context is stored in an `AsyncLocal<T>` field, making it accessible from any code executing during the agent run via a static property.
|
||||
|
||||
The main scenario for this is to allow deeply nested components (e.g., tools, chat client middleware) to access the context without needing to pass it through every method signature. These are external components that cannot easily be modified to accept additional parameters. For internal components, we prefer passing any parameters explicitly.
|
||||
|
||||
```csharp
|
||||
public static AgentRunContext? CurrentRunContext
|
||||
{
|
||||
get => s_currentContext.Value;
|
||||
protected set => s_currentContext.Value = value;
|
||||
}
|
||||
```
|
||||
|
||||
### AgentRunContext Design
|
||||
|
||||
The `AgentRunContext` class encapsulates all run-related state:
|
||||
|
||||
```csharp
|
||||
public class AgentRunContext
|
||||
{
|
||||
public AgentRunContext(
|
||||
AIAgent agent,
|
||||
AgentSession? session,
|
||||
IReadOnlyCollection<ChatMessage> requestMessages,
|
||||
AgentRunOptions? agentRunOptions)
|
||||
|
||||
public AIAgent Agent { get; }
|
||||
public AgentSession? Session { get; }
|
||||
public IReadOnlyCollection<ChatMessage> RequestMessages { get; }
|
||||
public AgentRunOptions? RunOptions { get; }
|
||||
}
|
||||
```
|
||||
|
||||
Key design decisions:
|
||||
|
||||
- **All properties are read-only**: While some of the sub-properties on the provided properties (like `AgentRunOptions.AllowBackgroundResponses`) may be mutable, the `AgentRunContext` itself is immutable and we want to discourage anyone modifying the values in the context. Modifying the context is unlikely to result in the desired behavior, as the values will typically already have been used by the time any custom code accesses them.
|
||||
|
||||
### Benefits
|
||||
|
||||
1. **Ambient Access**: Any code executing during the run can access context via `AIAgent.CurrentRunContext` without needing explicit parameters
|
||||
2. **Async Flow**: `AsyncLocal<T>` automatically flows across async/await boundaries
|
||||
3. **Modifiability**: Components can modify or replace session, messages, or options as needed
|
||||
4. **Testability**: The explicit parameter to `RunCoreAsync` makes unit testing straightforward
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,658 +0,0 @@
|
||||
---
|
||||
status: proposed
|
||||
contact: sergeymenshykh
|
||||
date: 2026-01-22
|
||||
deciders: rbarreto, westey-m, stephentoub
|
||||
informed: {}
|
||||
---
|
||||
|
||||
# Structured Output
|
||||
|
||||
Structured output is a valuable aspect of any agent system, since it forces an agent to produce output in a required format that may include required fields.
|
||||
This allows easily turning unstructured data into structured data using a general-purpose language model.
|
||||
|
||||
## Context and Problem Statement
|
||||
|
||||
Structured output is currently supported only by `ChatClientAgent` and can be configured in two ways:
|
||||
|
||||
**Approach 1: ResponseFormat + Deserialize**
|
||||
|
||||
Specify the SO type schema via the `ChatClientAgent{Run}Options.ChatOptions.ResponseFormat` property at agent creation or invocation time, then use `JsonSerializer.Deserialize<T>` to extract the structured data from the response text.
|
||||
|
||||
```csharp
|
||||
// SO type can be provided at agent creation time
|
||||
ChatClientAgent agent = chatClient.AsAIAgent(new ChatClientAgentOptions()
|
||||
{
|
||||
Name = "...",
|
||||
ChatOptions = new() { ResponseFormat = ChatResponseFormat.ForJsonSchema<PersonInfo>() }
|
||||
});
|
||||
|
||||
AgentResponse response = await agent.RunAsync("...");
|
||||
|
||||
PersonInfo personInfo = response.Deserialize<PersonInfo>(JsonSerializerOptions.Web);
|
||||
|
||||
Console.WriteLine($"Name: {personInfo.Name}");
|
||||
Console.WriteLine($"Age: {personInfo.Age}");
|
||||
Console.WriteLine($"Occupation: {personInfo.Occupation}");
|
||||
|
||||
// Alternatively, SO type can be provided at agent invocation time
|
||||
response = await agent.RunAsync("...", new ChatClientAgentRunOptions()
|
||||
{
|
||||
ChatOptions = new() { ResponseFormat = ChatResponseFormat.ForJsonSchema<PersonInfo>() }
|
||||
});
|
||||
|
||||
personInfo = response.Deserialize<PersonInfo>(JsonSerializerOptions.Web);
|
||||
|
||||
Console.WriteLine($"Name: {personInfo.Name}");
|
||||
Console.WriteLine($"Age: {personInfo.Age}");
|
||||
Console.WriteLine($"Occupation: {personInfo.Occupation}");
|
||||
```
|
||||
|
||||
**Approach 2: Generic RunAsync<T>**
|
||||
|
||||
Supply the SO type as a generic parameter to `RunAsync<T>` and access the parsed result directly via the `Result` property.
|
||||
|
||||
```csharp
|
||||
ChatClientAgent agent = ...;
|
||||
|
||||
AgentResponse<PersonInfo> response = await agent.RunAsync<PersonInfo>("...");
|
||||
|
||||
Console.WriteLine($"Name: {response.Result.Name}");
|
||||
Console.WriteLine($"Age: {response.Result.Age}");
|
||||
Console.WriteLine($"Occupation: {response.Result.Occupation}");
|
||||
```
|
||||
Note: `RunAsync<T>` is an instance method of `ChatClientAgent` and not part of the `AIAgent` base class since not all agents support structured output.
|
||||
|
||||
Approach 1 is perceived as cumbersome by the community, as it requires additional effort when using primitive or collection types - the SO schema may need to be wrapped in an artificial JSON object. Otherwise, the caller will encounter an error like _Invalid schema for response_format 'Movie': schema must be a JSON Schema of 'type: "object"', got 'type: "array"'_.
|
||||
This occurs because OpenAI and compatible APIs require a JSON object as the root schema.
|
||||
|
||||
Approach 1 is also necessary in scenarios where (a) agents can only be configured with SO at creation time (such as with `AIProjectClient`), (b) the SO type is not known at compile time, or (c) the JSON schema is represented as text (for declarative agents) or as a `JsonElement`.
|
||||
|
||||
Approach 2 is more convenient and works seamlessly with primitives and collections. However, it requires the SO type to be known at compile time, making it less flexible.
|
||||
|
||||
Additionally, since the `RunAsync<T>` methods are instance methods of `ChatClientAgent` and are not part of the `AIAgent` base class, applying decorators like `OpenTelemetryAgent` on top of `ChatClientAgent` prevents users from accessing `RunAsync<T>`, meaning structured output is not available with decorated agents.
|
||||
|
||||
Given the different scenarios above in which structured output can be used, there is no one-size-fits-all solution. Each approach has its own advantages and limitations,
|
||||
and the two can complement each other to provide a comprehensive structured output experience across various use cases.
|
||||
|
||||
## Approaches Overview
|
||||
|
||||
1. SO usage via `ResponseFormat` property
|
||||
2. SO usage via `RunAsync<T>` generic method
|
||||
|
||||
## 1. SO usage via `ResponseFormat` property
|
||||
|
||||
This approach should be used in the following scenarios:
|
||||
- 1.1 SO result as text is sufficient as is, and deserialization is not required
|
||||
- 1.2 SO for inter-agent collaboration
|
||||
- 1.3 SO can only be configured at agent creation time (such as with `AIProjectClient`)
|
||||
- 1.4 SO type is not known at compile time and represented by System.Type
|
||||
- 1.5 SO is represented by JSON schema and there's no corresponding .NET type either at compile time or at runtime
|
||||
- 1.6 SO in streaming scenarios, where the SO response is produced in parts
|
||||
|
||||
**Note: Primitives and arrays are not supported by this approach.**
|
||||
|
||||
When a caller provides a schema via `ResponseFormat`, they are explicitly telling the framework what schema to use. The framework passes that schema through as-is and
|
||||
is not responsible for transforming it. Because the framework does not own the schema, it cannot wrap primitives or arrays into a JSON object to satisfy API requirements,
|
||||
nor can it unwrap the response afterward - the caller controls the schema and is responsible for ensuring it is compatible with the underlying API.
|
||||
|
||||
This is in contrast to the `RunAsync<T>` approach (section 2), where the caller provides a type `T` and says "make it work." In that case, the caller does not
|
||||
dictate the schema - the framework infers the schema from `T`, owns the end-to-end pipeline (schema generation, API invocation, and deserialization), and can
|
||||
therefore wrap and unwrap primitives and arrays transparently.
|
||||
|
||||
Additionally, in streaming scenarios (1.6), the framework cannot reliably unwrap a response it did not wrap, since it has no way of knowing whether the caller wrapped the schema.Wrapping and unwrapping can only be done safely when the framework owns the entire lifecycle - from schema creation through deserialization — which is only the case with `RunAsync<T>`.
|
||||
|
||||
If a caller needs to work with primitives or arrays via the `ResponseFormat` approach, they can easily create a wrapper type around them:
|
||||
|
||||
```csharp
|
||||
public class MovieListWrapper
|
||||
{
|
||||
public List<string> Movies { get; set; }
|
||||
}
|
||||
```
|
||||
|
||||
### 1.1 SO result as text is sufficient as is, and deserialization is not required
|
||||
|
||||
In this scenario, the caller only needs the raw JSON text returned by the model and does not need to deserialize it into a .NET type.
|
||||
The SO schema is specified via `ResponseFormat` at agent creation or invocation time, and the response text is consumed directly from the `AgentResponse`.
|
||||
|
||||
```csharp
|
||||
AIAgent agent = chatClient.AsAIAgent();
|
||||
|
||||
AgentRunOptions runOptions = new()
|
||||
{
|
||||
ResponseFormat = ChatResponseFormat.ForJsonSchema<PersonInfo>()
|
||||
};
|
||||
|
||||
AgentResponse response = await agent.RunAsync("...", options: runOptions);
|
||||
|
||||
Console.WriteLine(response.Text);
|
||||
```
|
||||
|
||||
### 1.2 SO for inter-agent collaboration
|
||||
|
||||
This scenario assumes a multi-agent setup where agents collaborate by passing messages to each other.
|
||||
One agent produces structured output as text that is then passed directly as input to the next agent, without intermediate deserialization.
|
||||
|
||||
```csharp
|
||||
// First agent extracts structured data from unstructured input
|
||||
AIAgent extractionAgent = chatClient.AsAIAgent(new ChatClientAgentOptions()
|
||||
{
|
||||
Name = "ExtractionAgent",
|
||||
ChatOptions = new()
|
||||
{
|
||||
Instructions = "Extract person information from the provided text.",
|
||||
ResponseFormat = ChatResponseFormat.ForJsonSchema<PersonInfo>()
|
||||
}
|
||||
});
|
||||
|
||||
AgentResponse extractionResponse = await extractionAgent.RunAsync("John Smith is a 35-year-old software engineer.");
|
||||
|
||||
// Pass the message with structured output text directly to the next agent
|
||||
ChatMessage soMessage = extractionResponse.Messages.Last();
|
||||
|
||||
AIAgent summaryAgent = chatClient.AsAIAgent(new ChatClientAgentOptions()
|
||||
{
|
||||
Name = "SummaryAgent",
|
||||
ChatOptions = new() { Instructions = "Given the following structured person data, write a short professional bio." }
|
||||
});
|
||||
|
||||
AgentResponse summaryResponse = await summaryAgent.RunAsync(soMessage);
|
||||
|
||||
Console.WriteLine(summaryResponse);
|
||||
```
|
||||
|
||||
### 1.3 SO configured at agent creation time
|
||||
|
||||
In this scenario, the SO schema can only be configured at agent creation time (such as with `AIProjectClient`) and cannot be changed on a per-run basis.
|
||||
The caller specifies the `ResponseFormat` when creating the agent, and all subsequent invocations use the same schema.
|
||||
|
||||
```csharp
|
||||
AIProjectClient client = ...;
|
||||
|
||||
AIAgent agent = await client.CreateAIAgentAsync(model: "<model>", new ChatClientAgentOptions()
|
||||
{
|
||||
Name = "...",
|
||||
ChatOptions = new() { ResponseFormat = ChatResponseFormat.ForJsonSchema<PersonInfo>() }
|
||||
});
|
||||
|
||||
AgentResponse response = await agent.RunAsync("Please provide information about John Smith.");
|
||||
|
||||
PersonInfo personInfo = JsonSerializer.Deserialize<PersonInfo>(response.Text, JsonSerializerOptions.Web)!;
|
||||
|
||||
Console.WriteLine($"Name: {personInfo.Name}");
|
||||
Console.WriteLine($"Age: {personInfo.Age}");
|
||||
Console.WriteLine($"Occupation: {personInfo.Occupation}");
|
||||
```
|
||||
|
||||
### 1.4 SO type not known at compile time and represented by System.Type
|
||||
|
||||
In this scenario, the SO type is not known at compile time and is provided as a `System.Type` at runtime. This is useful for dynamic scenarios where the schema is determined programmatically,
|
||||
such as when building tooling or frameworks that work with user-defined types.
|
||||
|
||||
```csharp
|
||||
Type soType = GetStructuredOutputTypeFromConfiguration(); // e.g., typeof(PersonInfo)
|
||||
|
||||
ChatResponseFormat responseFormat = ChatResponseFormat.ForJsonSchema(soType);
|
||||
|
||||
AgentResponse response = await agent.RunAsync("...", new ChatClientAgentRunOptions()
|
||||
{
|
||||
ChatOptions = new() { ResponseFormat = responseFormat }
|
||||
});
|
||||
|
||||
PersonInfo personInfo = (PersonInfo)JsonSerializer.Deserialize(response.Text, soType, JsonSerializerOptions.Web)!;
|
||||
```
|
||||
|
||||
### 1.5 SO represented by JSON schema with no corresponding .NET type
|
||||
|
||||
In this scenario, the SO schema is represented as raw JSON schema text or a `JsonElement`, and there is no corresponding .NET type available at compile time or runtime.
|
||||
This is typical for declarative agents or scenarios where schemas are loaded from external configuration.
|
||||
|
||||
```csharp
|
||||
// JSON schema provided as a string, e.g., loaded from a configuration file
|
||||
string jsonSchema = """
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"name": { "type": "string" },
|
||||
"age": { "type": "integer" },
|
||||
"occupation": { "type": "string" }
|
||||
},
|
||||
"required": ["name", "age", "occupation"]
|
||||
}
|
||||
""";
|
||||
|
||||
ChatResponseFormat responseFormat = ChatResponseFormat.ForJsonSchema(
|
||||
jsonSchemaName: "PersonInfo",
|
||||
jsonSchema: BinaryData.FromString(jsonSchema));
|
||||
|
||||
AgentResponse response = await agent.RunAsync("...", new ChatClientAgentRunOptions()
|
||||
{
|
||||
ChatOptions = new() { ResponseFormat = responseFormat }
|
||||
});
|
||||
|
||||
// Consume the SO result as text since there's no .NET type to deserialize into
|
||||
Console.WriteLine(response.Text);
|
||||
```
|
||||
|
||||
### 1.6 SO in streaming scenarios
|
||||
|
||||
In this scenario, the SO response is produced incrementally in parts via streaming. The caller specifies the `ResponseFormat` and consumes the response chunks as they arrive.
|
||||
Deserialization is performed after all chunks have been received.
|
||||
|
||||
```csharp
|
||||
AIAgent agent = chatClient.AsAIAgent(new ChatClientAgentOptions()
|
||||
{
|
||||
Name = "HelpfulAssistant",
|
||||
ChatOptions = new()
|
||||
{
|
||||
Instructions = "You are a helpful assistant.",
|
||||
ResponseFormat = ChatResponseFormat.ForJsonSchema<PersonInfo>()
|
||||
}
|
||||
});
|
||||
|
||||
IAsyncEnumerable<AgentResponseUpdate> updates = agent.RunStreamingAsync("Please provide information about John Smith, who is a 35-year-old software engineer.");
|
||||
|
||||
AgentResponse response = await updates.ToAgentResponseAsync();
|
||||
|
||||
// Deserialize the complete SO result after streaming is finished
|
||||
PersonInfo personInfo = JsonSerializer.Deserialize<PersonInfo>(response.Text)!;
|
||||
```
|
||||
|
||||
## 2. SO usage via `RunAsync<T>` generic method
|
||||
|
||||
This approach provides a convenient way to work with structured output on a per-run basis when the target type is known at compile time and a typed instance of the result
|
||||
is required.
|
||||
|
||||
### Decision Drivers
|
||||
|
||||
1. Support arrays and primitives as SO types
|
||||
2. Support complex types as SO types
|
||||
3. Work with `AIAgent` decorators (e.g., `OpenTelemetryAgent`)
|
||||
4. Enable SO for all AI agents, regardless of whether they natively support it
|
||||
|
||||
### Considered Options
|
||||
|
||||
1. `RunAsync<T>` as an instance method of `AIAgent` class delegating to virtual `RunCoreAsync<T>`
|
||||
2. `RunAsync<T>` as an extension method using feature collection
|
||||
3. `RunAsync<T>` as a method of the new `ITypedAIAgent` interface
|
||||
4. `RunAsync<T>` as an instance method of `AIAgent` class working via the new `AgentRunOptions.ResponseFormat` property
|
||||
|
||||
### 1. `RunAsync<T>` as an instance method of `AIAgent` class delegating to virtual `RunCoreAsync<T>`
|
||||
|
||||
This option adds the `RunAsync<T>` method directly to the `AIAgent` base class.
|
||||
|
||||
```csharp
|
||||
public abstract class AIAgent
|
||||
{
|
||||
public Task<AgentResponse<T>> RunAsync<T>(
|
||||
IEnumerable<ChatMessage> messages,
|
||||
AgentSession? session = null,
|
||||
JsonSerializerOptions? serializerOptions = null,
|
||||
AgentRunOptions? options = null,
|
||||
CancellationToken cancellationToken = default)
|
||||
=> this.RunCoreAsync<T>(messages, session, serializerOptions, options, cancellationToken);
|
||||
|
||||
protected virtual Task<AgentResponse<T>> RunCoreAsync<T>(
|
||||
IEnumerable<ChatMessage> messages,
|
||||
AgentSession? session = null,
|
||||
JsonSerializerOptions? serializerOptions = null,
|
||||
AgentRunOptions? options = null,
|
||||
CancellationToken cancellationToken = default)
|
||||
{
|
||||
throw new NotSupportedException($"The agent of type '{this.GetType().FullName}' does not support typed responses.");
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Agents with native SO support override the `RunCoreAsync<T>` method to provide their implementation. If not overridden, the method throws a `NotSupportedException`.
|
||||
|
||||
Users will call the generic `RunAsync<T>` method directly on the agent:
|
||||
|
||||
```csharp
|
||||
AIAgent agent = chatClient.AsAIAgent(name: "HelpfulAssistant", instructions: "You are a helpful assistant.");
|
||||
|
||||
AgentResponse<PersonInfo> response = await agent.RunAsync<PersonInfo>("Please provide information about John Smith, who is a 35-year-old software engineer.");
|
||||
```
|
||||
|
||||
Decision drivers satisfied:
|
||||
1. Support arrays and primitives as SO types
|
||||
2. Support complex types as SO types
|
||||
3. Work with `AIAgent` decorators (e.g., `OpenTelemetryAgent`)
|
||||
4. Enable SO for all AI agents, regardless of whether they natively support it
|
||||
|
||||
Pros:
|
||||
- The `AIAgent.RunAsync<T>` method is easily discoverable.
|
||||
- Both the SO decorator and `ChatClientAgent` have compile-time access to the type `T`, allowing them to use the native `IChatClient.GetResponseAsync<T>` API, which handles primitives and collections seamlessly.
|
||||
|
||||
Cons:
|
||||
- Agents without native SO support will still expose `RunAsync<T>`, which may be misleading.
|
||||
- `ChatClientAgent` exposing `RunAsync<T>` may be misleading when the underlying chat client does not support SO.
|
||||
- All `AIAgent` decorators must override `RunCoreAsync<T>` to properly handle `RunAsync<T>` calls.
|
||||
|
||||
### 2. `RunAsync<T>` as an extension method using feature collection
|
||||
|
||||
This option uses the Agent Framework feature collection (implemented via `AgentRunOptions.AdditionalProperties`) to pass a `StructuredOutputFeature` to agents, signaling that SO is requested.
|
||||
|
||||
Agents with native SO support check for this feature. If present, they read the target type, build the schema, invoke the underlying API, and store the response back in the feature.
|
||||
```csharp
|
||||
public class StructuredOutputFeature
|
||||
{
|
||||
public StructuredOutputFeature(Type outputType)
|
||||
{
|
||||
this.OutputType = outputType;
|
||||
}
|
||||
|
||||
[JsonIgnore]
|
||||
public Type OutputType { get; set; }
|
||||
|
||||
public JsonSerializerOptions? SerializerOptions { get; set; }
|
||||
|
||||
public AgentResponse? Response { get; set; }
|
||||
}
|
||||
```
|
||||
|
||||
The `RunAsync<T>` extension method for `AIAgent` adds this feature to the collection.
|
||||
```csharp
|
||||
public static async Task<AgentResponse<T>> RunAsync<T>(
|
||||
this AIAgent agent,
|
||||
IEnumerable<ChatMessage> messages,
|
||||
AgentSession? session = null,
|
||||
JsonSerializerOptions? serializerOptions = null,
|
||||
AgentRunOptions? options = null,
|
||||
CancellationToken cancellationToken = default)
|
||||
{
|
||||
// Create the structured output feature.
|
||||
StructuredOutputFeature structuredOutputFeature = new(typeof(T))
|
||||
{
|
||||
SerializerOptions = serializerOptions,
|
||||
};
|
||||
|
||||
// Register it in the feature collection.
|
||||
((options ??= new AgentRunOptions()).AdditionalProperties ??= []).Add(typeof(StructuredOutputFeature).FullName!, structuredOutputFeature);
|
||||
|
||||
var response = await agent.RunAsync(messages, session, options, cancellationToken).ConfigureAwait(false);
|
||||
|
||||
if (structuredOutputFeature.Response is not null)
|
||||
{
|
||||
return new StructuredOutputResponse<T>(structuredOutputFeature.Response, response, serializerOptions);
|
||||
}
|
||||
|
||||
throw new InvalidOperationException("No structured output response was generated by the agent.");
|
||||
}
|
||||
```
|
||||
|
||||
Users will call the `RunAsync<T>` extension method directly on the agent:
|
||||
|
||||
```csharp
|
||||
AIAgent agent = chatClient.AsAIAgent(name: "HelpfulAssistant", instructions: "You are a helpful assistant.");
|
||||
|
||||
AgentResponse<PersonInfo> response = await agent.RunAsync<PersonInfo>("Please provide information about John Smith, who is a 35-year-old software engineer.");
|
||||
```
|
||||
|
||||
Decision drivers satisfied:
|
||||
1. Support arrays and primitives as SO types
|
||||
2. Support complex types as SO types
|
||||
3. Work with `AIAgent` decorators (e.g., `OpenTelemetryAgent`)
|
||||
4. Enable SO for all AI agents, regardless of whether they natively support it
|
||||
|
||||
Pros:
|
||||
- The `RunAsync<T>` extension method is easily discoverable.
|
||||
- The `AIAgent` public API surface remains unchanged.
|
||||
- No changes required to `AIAgent` decorators.
|
||||
|
||||
Cons:
|
||||
- Agents without native SO support will still expose `RunAsync<T>`, which may be misleading.
|
||||
- `ChatClientAgent` exposing `RunAsync<T>` may be misleading when the underlying chat client does not support SO.
|
||||
|
||||
### 3. `RunAsync<T>` as a method of the new `ITypedAIAgent` interface
|
||||
|
||||
This option defines a new `ITypedAIAgent` interface that agents with SO support implement. Agents without SO support do not implement it, allowing users to check for SO capability via interface detection.
|
||||
|
||||
The interface:
|
||||
```csharp
|
||||
public interface ITypedAIAgent
|
||||
{
|
||||
Task<AgentResponse<T>> RunAsync<T>(
|
||||
IEnumerable<ChatMessage> messages,
|
||||
AgentSession? session = null,
|
||||
JsonSerializerOptions? serializerOptions = null,
|
||||
AgentRunOptions? options = null,
|
||||
CancellationToken cancellationToken = default);
|
||||
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
Agents with SO support implement this interface:
|
||||
```csharp
|
||||
public sealed partial class ChatClientAgent : AIAgent, ITypedAIAgent
|
||||
{
|
||||
public async Task<AgentResponse<T>> RunAsync<T>(
|
||||
IEnumerable<ChatMessage> messages,
|
||||
AgentSession? session = null,
|
||||
JsonSerializerOptions? serializerOptions = null,
|
||||
AgentRunOptions? options = null,
|
||||
CancellationToken cancellationToken = default)
|
||||
{
|
||||
...
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
However, `ChatClientAgent` presents a challenge: it can work with chat clients that either support or do not support SO. Implementing the interface does not guarantee
|
||||
the underlying chat client supports SO, which undermines the core idea of using interface detection to determine SO capability.
|
||||
|
||||
Additionally, to allow users to access interface methods on decorated agents, all decorators must implement `ITypedAIAgent`. This makes it difficult for users to
|
||||
determine whether the underlying agent actually supports SO, further weakening the purpose of this approach.
|
||||
|
||||
Furthermore, users would have to probe the agent type to check if it implements the `ITypedAIAgent` interface and cast it accordingly to access the `RunAsync<T>` methods.
|
||||
This adds friction to the user experience. A `RunAsync<T>` extension method for `AIAgent` could be provided to alleviate that.
|
||||
|
||||
Given these drawbacks, this option is more complex to implement than the others without providing clear benefits.
|
||||
|
||||
Decision drivers satisfied:
|
||||
1. Support arrays and primitives as SO types
|
||||
2. Support complex types as SO types
|
||||
3. Work with `AIAgent` decorators (e.g., `OpenTelemetryAgent`)
|
||||
4. Enable SO for all AI agents, regardless of whether they natively support it
|
||||
|
||||
Pros:
|
||||
- Both the SO decorator and `ChatClientAgent` have compile-time access to the type `T`, allowing them to use the native `IChatClient.GetResponseAsync<T>` API, which handles primitives and collections seamlessly.
|
||||
|
||||
Cons:
|
||||
- `ChatClientAgent` implementing `ITypedAIAgent` may be misleading when the underlying chat client does not support SO.
|
||||
- All `AIAgent` decorators must implement `ITypedAIAgent` to handle `RunAsync<T>` calls.
|
||||
- Decorators implementing the interface may mislead users into thinking the underlying agent natively supports SO.
|
||||
- Agents must implement all members of `ITypedAIAgent`, not just a core method.
|
||||
- Users must check the agent type and cast to `ITypedAIAgent` to access `RunAsync<T>`.
|
||||
|
||||
### 4. `RunAsync<T>` as an instance method of `AIAgent` class working via the new `AgentRunOptions.ResponseFormat` property
|
||||
|
||||
This option adds a `ResponseFormat` property of type `ChatResponseFormat` to `AgentRunOptions`. Agents that support SO check for the presence of
|
||||
this property in the options passed to `RunAsync` to determine whether structured output is requested. If present, they use the schema from `ResponseFormat`
|
||||
to invoke the underlying API and obtain the SO response.
|
||||
|
||||
```csharp
|
||||
public class AgentRunOptions
|
||||
{
|
||||
public ChatResponseFormat? ResponseFormat { get; set; }
|
||||
}
|
||||
```
|
||||
|
||||
Additionally, a generic `RunAsync<T>` method is added to `AIAgent` that initializes the `ResponseFormat` based on the type `T` and delegates to the non-generic `RunAsync`.
|
||||
|
||||
```csharp
|
||||
public abstract class AIAgent
|
||||
{
|
||||
public async Task<AgentResponse<T>> RunAsync<T>(
|
||||
IEnumerable<ChatMessage> messages,
|
||||
AgentSession? session = null,
|
||||
JsonSerializerOptions? serializerOptions = null,
|
||||
AgentRunOptions? options = null,
|
||||
CancellationToken cancellationToken = default)
|
||||
{
|
||||
serializerOptions ??= AgentAbstractionsJsonUtilities.DefaultOptions;
|
||||
|
||||
var responseFormat = ChatResponseFormat.ForJsonSchema<T>(serializerOptions);
|
||||
|
||||
options = options?.Clone() ?? new AgentRunOptions();
|
||||
options.ResponseFormat = responseFormat;
|
||||
|
||||
AgentResponse response = await this.RunAsync(messages, session, options, cancellationToken).ConfigureAwait(false);
|
||||
|
||||
return new AgentResponse<T>(response, serializerOptions);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Users call the generic `RunAsync<T>` method directly on the agent:
|
||||
|
||||
```csharp
|
||||
AIAgent agent = chatClient.AsAIAgent(name: "HelpfulAssistant", instructions: "You are a helpful assistant.");
|
||||
|
||||
AgentResponse<PersonInfo> response = await agent.RunAsync<PersonInfo>("Please provide information about John Smith, who is a 35-year-old software engineer.");
|
||||
```
|
||||
|
||||
Decision drivers satisfied:
|
||||
1. Support arrays and primitives as SO types
|
||||
2. Support complex types as SO types
|
||||
3. Work with `AIAgent` decorators (e.g., `OpenTelemetryAgent`)
|
||||
4. Enable SO for all AI agents, regardless of whether they natively support it
|
||||
|
||||
Pros:
|
||||
- The `AIAgent.RunAsync<T>` method is easily discoverable.
|
||||
- No changes required to `AIAgent` decorators
|
||||
|
||||
Cons:
|
||||
- Agents without native SO support will still expose `RunAsync<T>`, which may be misleading.
|
||||
- `ChatClientAgent` exposing `RunAsync<T>` may be misleading when the underlying chat client does not support SO.
|
||||
|
||||
### Decision Table
|
||||
|
||||
| | Option 1: Instance method + RunCoreAsync<T> | Option 2: Extension method + feature collection | Option 3: ITypedAIAgent Interface | Option 4: Instance method + AgentRunOptions.ResponseFormat |
|
||||
|---|---|---|---|---|
|
||||
| Discoverability | ✅ `RunAsync<T>` easily discoverable | ✅ `RunAsync<T>` easily discoverable | ❌ Requires type check and cast | ✅ `RunAsync<T>` easily discoverable |
|
||||
| Decorator changes | ❌ All decorators must override `RunCoreAsync<T>` | ✅ No changes required | ❌ All decorators must implement `ITypedAIAgent` | ✅ No changes required to decorators |
|
||||
| Primitives/collections handling | ✅ Native support via `IChatClient.GetResponseAsync<T>` | ❌ Must wrap/unwrap internally | ✅ Native support via `IChatClient.GetResponseAsync<T>` | ❌ Must wrap/unwrap internally |
|
||||
| Misleading API exposure | ❌ Agents without SO still expose `RunAsync<T>` | ❌ Agents without SO still expose `RunAsync<T>` | ❌ Interface on `ChatClientAgent` may be misleading | ❌ Agents without SO still expose `RunAsync<T>` |
|
||||
| Implementation burden | ❌ Decorators must override method | ❌ Must handle schema wrapping | ❌ Agents must implement all interface members | ✅ Delegates to existing `RunAsync` via `ResponseFormat` |
|
||||
|
||||
## Cross-Cutting Aspects
|
||||
|
||||
1. **The `useJsonSchemaResponseFormat` parameter**: The `ChatClientAgent.RunAsync<T>` method has this parameter to enable structured output on LLMs that do not natively support it.
|
||||
It works by adding a user message like "Respond with a JSON value conforming to the following schema:" along with the JSON schema. However, this approach has not been reliable historically. The recommendation is not to carry this parameter forward, regardless of which option is chosen.
|
||||
|
||||
2. **Primitives and array types handling**: There are a few options for how primitive and array types can be handled in the Agent Framework:
|
||||
|
||||
1. **Never wrap**, regardless of whether the schema is provided via `ResponseFormat` or `RunAsync<T>`.
|
||||
- Pro: No changes needed; user has full control.
|
||||
- Pro: No issues with unwrapping in streaming scenarios.
|
||||
- Con: User must wrap manually.
|
||||
|
||||
2. **Always wrap**, regardless of whether the schema is provided via `ResponseFormat` or `RunAsync<T>`.
|
||||
- Pro: Consistent wrapping behavior; no manual wrapping needed.
|
||||
- Con: Inconsistent unwrapping behavior; it may be unexpected to have SO result wrapped when schema is provided via `ResponseFormat`.
|
||||
- Con: Impossible to know if SO result is wrapped to unwrap it in streaming scenarios.
|
||||
|
||||
3. **Wrap only for `RunAsync<T>`** and do not wrap the schema provided via `ResponseFormat`.
|
||||
- Pro: No unexpectedly wrapped result when schema is provided via `ResponseFormat`.
|
||||
- Pro: Solves the problem with unwrapping in streaming scenarios.
|
||||
|
||||
4. **User decides** whether to wrap schema provided via `ResponseFormat` using a new `wrapPrimitivesAndArrays` property of `ChatResponseFormatJson`. For SO provided via `RunAsync<T>`, AF always wraps.
|
||||
- Pro: No manual wrapping needed; just flip a switch.
|
||||
- Pro: Solves the problem with unwrapping in streaming scenarios.
|
||||
- Con: Extends the public API surface.
|
||||
|
||||
3. **Structured output for agents without native SO support**: Some AI agents in AF do not support structured output natively. This is either because it is not part of the protocol (e.g., A2A agent) or because the agents use LLMs without structured output capabilities.
|
||||
To address this gap, AF can provide the `StructuredOutputAgent` decorator. This decorator wraps any `AIAgent` and adds structured output support by obtaining the text response from the decorated agent and delegating it to a configured chat client for JSON transformation.
|
||||
|
||||
```csharp
|
||||
public class StructuredOutputAgent : DelegatingAIAgent
|
||||
{
|
||||
private readonly IChatClient _chatClient;
|
||||
|
||||
public StructuredOutputAgent(AIAgent innerAgent, IChatClient chatClient)
|
||||
: base(innerAgent)
|
||||
{
|
||||
this._chatClient = Throw.IfNull(chatClient);
|
||||
}
|
||||
|
||||
protected override async Task<AgentResponse<T>> RunCoreAsync<T>(
|
||||
IEnumerable<ChatMessage> messages,
|
||||
AgentSession? session = null,
|
||||
JsonSerializerOptions? serializerOptions = null,
|
||||
AgentRunOptions? options = null,
|
||||
CancellationToken cancellationToken = default)
|
||||
{
|
||||
// Run the inner agent first, to get back the text response we want to convert.
|
||||
var textResponse = await this.InnerAgent.RunAsync(messages, session, options, cancellationToken).ConfigureAwait(false);
|
||||
|
||||
// Invoke the chat client to transform the text output into structured data.
|
||||
ChatResponse<T> soResponse = await this._chatClient.GetResponseAsync<T>(
|
||||
messages:
|
||||
[
|
||||
new ChatMessage(ChatRole.System, "You are a json expert and when provided with any text, will convert it to the requested json format."),
|
||||
new ChatMessage(ChatRole.User, textResponse.Text)
|
||||
],
|
||||
serializerOptions: serializerOptions ?? AgentJsonUtilities.DefaultOptions,
|
||||
cancellationToken: cancellationToken).ConfigureAwait(false);
|
||||
|
||||
return new StructuredOutputAgentResponse(soResponse, textResponse);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The decorator preserves the original response from the decorated agent and surfaces it via the `OriginalResponse` property on the returned `StructuredOutputAgentResponse`.
|
||||
This allows users to access both the original unstructured response and the new structured response when using this decorator.
|
||||
```csharp
|
||||
public class StructuredOutputAgentResponse : AgentResponse
|
||||
{
|
||||
internal StructuredOutputAgentResponse(ChatResponse chatResponse, AgentResponse agentResponse) : base(chatResponse)
|
||||
{
|
||||
this.OriginalResponse = agentResponse;
|
||||
}
|
||||
|
||||
public AgentResponse OriginalResponse { get; }
|
||||
}
|
||||
```
|
||||
|
||||
The decorator can be registered during the agent configuration step using the `UseStructuredOutput` extension method on `AIAgentBuilder`.
|
||||
|
||||
```csharp
|
||||
IChatClient meaiChatClient = chatClient.AsIChatClient();
|
||||
|
||||
AIAgent baseAgent = meaiChatClient.AsAIAgent(name: "HelpfulAssistant", instructions: "You are a helpful assistant.");
|
||||
|
||||
// Register the StructuredOutputAgent decorator during agent building
|
||||
AIAgent agent = baseAgent
|
||||
.AsBuilder()
|
||||
.UseStructuredOutput(meaiChatClient)
|
||||
.Build();
|
||||
|
||||
AgentResponse<PersonInfo> response = await agent.RunAsync<PersonInfo>("Please provide information about John Smith, who is a 35-year-old software engineer.");
|
||||
|
||||
Console.WriteLine($"Name: {response.Result.Name}");
|
||||
Console.WriteLine($"Age: {response.Result.Age}");
|
||||
Console.WriteLine($"Occupation: {response.Result.Occupation}");
|
||||
|
||||
var originalResponse = ((StructuredOutputAgentResponse)response.RawRepresentation!).OriginalResponse;
|
||||
Console.WriteLine($"Original unstructured response: {originalResponse.Text}");
|
||||
|
||||
```
|
||||
|
||||
## Decision Outcome
|
||||
|
||||
It was decided to keep both approaches for structured output - via `ResponseFormat` and via `RunAsync<T>` since they serve different scenarios and use cases.
|
||||
|
||||
For the `RunAsync<T>` approach, option 4 was selected, which adds a generic `RunAsync<T>` method to `AIAgent` that works via the new `AgentRunOptions.ResponseFormat` property.
|
||||
This was chosen for its simplicity and because no changes are required to existing `AIAgent` decorators.
|
||||
|
||||
For cross-cutting aspects, the `useJsonSchemaResponseFormat` parameter will not be carried forward due to reliability issues.
|
||||
|
||||
For handling primitives and array types, option 3 was selected: wrap only for `RunAsync<T>` and do not wrap the schema provided via `ResponseFormat`.
|
||||
This avoids the issues described in the Approach 1 section note.
|
||||
|
||||
Finally, it was decided not to include the `StructuredOutputAgent` decorator in the framework, since the reliability of producing structured output via an additional
|
||||
LLM call may not be sufficient for all scenarios. Instead, this pattern is provided as a sample to demonstrate how structured output can be achieved for agents without native support,
|
||||
giving users a reference implementation they can adapt to their own requirements.
|
||||
@@ -1,211 +0,0 @@
|
||||
---
|
||||
status: accepted
|
||||
contact: westey-m
|
||||
date: 2026-02-24
|
||||
deciders: sergeymenshykh, markwallace, rbarreto, dmytrostruk, westey-m, eavanvalkenburg, stephentoub, lokitoth, alliscode, taochenosu, moonbox3
|
||||
consulted:
|
||||
informed:
|
||||
---
|
||||
|
||||
# AdditionalProperties for AIAgent and AgentSession
|
||||
|
||||
## Context and Problem Statement
|
||||
|
||||
The `AIAgent` base class currently exposes `Id`, `Name`, and `Description` as its core metadata properties, and `AgentSession` exposes only a `StateBag` property.
|
||||
Neither type has a mechanism for attaching arbitrary metadata, such as protocol-specific descriptors (e.g., A2A agent cards), hosting attributes, session-level tags, or custom user-defined metadata for discovery and routing.
|
||||
|
||||
Other types in the framework already carry `AdditionalProperties` — notably `AgentRunOptions`, `AgentResponse`, and `AgentResponseUpdate` — all using `AdditionalPropertiesDictionary` from `Microsoft.Extensions.AI`.
|
||||
Adding a similar property to `AIAgent` and `AgentSession` would give both types a consistent, extensible metadata surface.
|
||||
|
||||
Related: [Work Item #2133](https://github.com/microsoft/agent-framework/issues/2133)
|
||||
|
||||
## Decision Drivers
|
||||
|
||||
- **Consistency**: Other core types (`AgentRunOptions`, `AgentResponse`, `AgentResponseUpdate`) already expose `AdditionalProperties`. `AIAgent` and `AgentSession` are the major abstractions that lack this.
|
||||
- **Extensibility**: Hosting libraries, protocol adapters (A2A, AG-UI), and discovery mechanisms need a place to attach agent-level and session-level metadata without subclassing.
|
||||
- **Simplicity**: The solution should be easy to understand and use; avoid over-engineering.
|
||||
- **Minimal breaking change**: The addition should not require changes to existing agent implementations.
|
||||
- **Clear semantics**: Users should understand what `AdditionalProperties` on an agent or session means and how it differs from `AdditionalProperties` on `AgentRunOptions`.
|
||||
|
||||
## Considered Options
|
||||
|
||||
### Surface Area
|
||||
|
||||
- **Option A**: Public get-only property, auto-initialized (`AdditionalPropertiesDictionary AdditionalProperties { get; } = new()`) on both `AIAgent` and `AgentSession`
|
||||
- **Option B**: Public get/set nullable property (`AdditionalPropertiesDictionary? AdditionalProperties { get; set; }`) on both `AIAgent` and `AgentSession`
|
||||
- **Option C**: Constructor-injected dictionary with public get-only accessor on both `AIAgent` and `AgentSession`
|
||||
- **Option D**: External container/wrapper object — metadata lives outside `AIAgent` and `AgentSession`; no changes to the base classes
|
||||
|
||||
### Semantics
|
||||
|
||||
- **Option 1**: Metadata only — describes the agent or session; not propagated when calling `IChatClient`
|
||||
- **Option 2**: Passed down the stack — merged into `ChatOptions.AdditionalProperties` during `ChatClientAgent` runs
|
||||
|
||||
## Decision Outcome
|
||||
|
||||
The chosen option is **Option D + Option 1**: an external container/wrapper object, used purely as metadata.
|
||||
|
||||
### Consequences
|
||||
|
||||
- Good, because `AIAgent` and `AgentSession` remain unchanged, avoiding any increase to the core framework surface area while still enabling extensible metadata.
|
||||
- Good, because an external wrapper (owned by hosting/protocol libraries or user code, not the `AIAgent` / `AgentSession` base classes) can internally use `AdditionalPropertiesDictionary` to stay consistent with existing patterns on `AgentRunOptions`, `AgentResponse`, and `AgentResponseUpdate`.
|
||||
- Good, because metadata-only semantics keep a clean separation from per-run extensibility (`AgentRunOptions.AdditionalProperties`) and avoid unexpected side effects during agent execution.
|
||||
- Good, because no additional allocation occurs on `AIAgent` or `AgentSession` when no metadata is needed; external wrappers can be created only when metadata is required.
|
||||
- Bad, because callers and libraries must manage and pass around both the agent/session instance and its associated metadata wrapper, keeping them correctly associated.
|
||||
- Bad, because different hosting or protocol layers may define their own wrapper types, which can fragment the ecosystem unless conventions are agreed upon.
|
||||
|
||||
## Pros and Cons of the Options
|
||||
|
||||
### Option A — Public get-only property, auto-initialized
|
||||
|
||||
The property is always non-null and ready to use. Users add metadata after construction.
|
||||
|
||||
```csharp
|
||||
public abstract partial class AIAgent
|
||||
{
|
||||
public AdditionalPropertiesDictionary AdditionalProperties { get; } = new();
|
||||
}
|
||||
|
||||
public abstract partial class AgentSession
|
||||
{
|
||||
public AdditionalPropertiesDictionary AdditionalProperties { get; } = new();
|
||||
}
|
||||
|
||||
// Usage
|
||||
agent.AdditionalProperties["protocol"] = "A2A";
|
||||
agent.AdditionalProperties.Add<MyAgentCardInfo>(cardInfo);
|
||||
session.AdditionalProperties["tenant"] = tenantId;
|
||||
```
|
||||
|
||||
- Good, because users never encounter `null` — no defensive null checks needed.
|
||||
- Good, because the dictionary reference cannot be replaced, preventing accidental data loss.
|
||||
- Good, because it is the simplest API surface to use.
|
||||
- Neutral, because it always allocates, even when no metadata is needed. The allocation cost is negligible.
|
||||
- Bad, because it cannot be set at construction time as a single object (users must populate it post-construction).
|
||||
|
||||
### Option B — Public get/set nullable property
|
||||
|
||||
Matches the existing pattern on `AgentRunOptions`, `AgentResponse`, and `AgentResponseUpdate`.
|
||||
|
||||
```csharp
|
||||
public abstract partial class AIAgent
|
||||
{
|
||||
public AdditionalPropertiesDictionary? AdditionalProperties { get; set; }
|
||||
}
|
||||
|
||||
public abstract partial class AgentSession
|
||||
{
|
||||
public AdditionalPropertiesDictionary? AdditionalProperties { get; set; }
|
||||
}
|
||||
|
||||
// Usage
|
||||
agent.AdditionalProperties ??= new();
|
||||
agent.AdditionalProperties["protocol"] = "A2A";
|
||||
session.AdditionalProperties ??= new();
|
||||
session.AdditionalProperties["tenant"] = tenantId;
|
||||
```
|
||||
|
||||
- Good, because it is consistent with the existing `AdditionalProperties` pattern on `AgentRunOptions` and `AgentResponse`.
|
||||
- Good, because it avoids allocation when no metadata is needed.
|
||||
- Bad, because every consumer must null-check before reading or writing.
|
||||
- Bad, because the entire dictionary can be replaced, risking accidental loss of metadata set by other components (e.g., a hosting library sets metadata, then user code replaces the dictionary).
|
||||
|
||||
### Option C — Constructor-injected with public get
|
||||
|
||||
The dictionary is provided at construction time and exposed as get-only.
|
||||
|
||||
```csharp
|
||||
public abstract partial class AIAgent
|
||||
{
|
||||
public AdditionalPropertiesDictionary AdditionalProperties { get; }
|
||||
|
||||
protected AIAgent(AdditionalPropertiesDictionary? additionalProperties = null)
|
||||
{
|
||||
this.AdditionalProperties = additionalProperties ?? new();
|
||||
}
|
||||
}
|
||||
|
||||
public abstract partial class AgentSession
|
||||
{
|
||||
public AdditionalPropertiesDictionary AdditionalProperties { get; }
|
||||
|
||||
protected AgentSession(AdditionalPropertiesDictionary? additionalProperties = null)
|
||||
{
|
||||
this.AdditionalProperties = additionalProperties ?? new();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- Good, because an agent's metadata can be established before any code runs against it.
|
||||
- Bad, because `AdditionalPropertiesDictionary` has no read-only variant, so the constructor-injection pattern gives a false sense of immutability — callers can still mutate the dictionary contents after construction.
|
||||
- Bad, because it requires adding a constructor parameter to the abstract base classes, which is a source-breaking change for all existing `AIAgent` and `AgentSession` subclasses (even with a default value, it changes the constructor signature that derived classes chain to).
|
||||
- Bad, because it is more complex with little practical benefit over Option A, since post-construction mutation is equally possible.
|
||||
|
||||
### Option D — External container/wrapper object
|
||||
|
||||
Rather than adding `AdditionalProperties` to `AIAgent` or `AgentSession`, users wrap the agent or session in a container object that carries both the instance and any associated metadata. No changes to the base classes are required.
|
||||
|
||||
```csharp
|
||||
public class AgentWithMetadata
|
||||
{
|
||||
public required AIAgent Agent { get; init; }
|
||||
public AdditionalPropertiesDictionary? AdditionalProperties { get; set; }
|
||||
}
|
||||
|
||||
public class SessionWithMetadata
|
||||
{
|
||||
public required AgentSession Session { get; init; }
|
||||
public AdditionalPropertiesDictionary? AdditionalProperties { get; set; }
|
||||
}
|
||||
|
||||
// Usage
|
||||
var wrapper = new AgentWithMetadata
|
||||
{
|
||||
Agent = myAgent,
|
||||
AdditionalProperties = new() { ["protocol"] = "A2A" }
|
||||
};
|
||||
```
|
||||
|
||||
- Good, because it requires no changes to `AIAgent` or `AgentSession`, avoiding any risk of breaking existing implementations.
|
||||
- Good, because metadata is clearly external to the agent and session, eliminating any ambiguity about whether it might be passed down the execution stack.
|
||||
- Good, because the container pattern gives the user full control over the metadata lifecycle and serialization.
|
||||
- Bad, because it is not discoverable — users must know about the container convention; there is no built-in API surface guiding them.
|
||||
|
||||
### Option 1 — Metadata only
|
||||
|
||||
`AdditionalProperties` on `AIAgent` and `AgentSession` is descriptive metadata. It is **not** automatically propagated when the agent calls downstream services such as `IChatClient`.
|
||||
|
||||
- Good, because it keeps a clean separation of concerns: agent/session-level metadata vs. per-run options.
|
||||
- Good, because it avoids unintended side effects — metadata added for discovery or hosting won't leak into LLM requests.
|
||||
- Good, because per-run extensibility is already served by `AgentRunOptions.AdditionalProperties` (see [ADR 0014](0014-feature-collections.md)), so there is no gap.
|
||||
- Neutral, because users who want to pass agent metadata to the chat client can still do so manually via `AgentRunOptions`.
|
||||
|
||||
### Option 2 — Passed down the stack
|
||||
|
||||
`AdditionalProperties` on `AIAgent` and `AgentSession` are automatically merged into `ChatOptions.AdditionalProperties` (or similar) when `ChatClientAgent` invokes the underlying `IChatClient`.
|
||||
|
||||
- Good, because it provides an automatic way to send agent-level configuration to the LLM provider.
|
||||
- Bad, because it conflates metadata (describing the agent) with operational parameters (controlling LLM behavior), leading to potential confusion.
|
||||
- Bad, because it risks leaking unrelated metadata into LLM calls (e.g., hosting tags, discovery URLs).
|
||||
- Bad, because it would be `ChatClientAgent`-specific behavior on a base-class property, creating inconsistency for non-`ChatClientAgent` implementations.
|
||||
- Bad, because it duplicates the purpose of `AgentRunOptions.AdditionalProperties`, which already serves as the per-run extensibility point for passing data down the stack.
|
||||
|
||||
## Serialization Considerations
|
||||
|
||||
`AIAgent` instances are not typically serialized, so `AdditionalProperties` on `AIAgent` does not raise serialization concerns.
|
||||
|
||||
`AgentSession` instances, however, are routinely serialized and deserialized — for example, to persist conversation state across application restarts. Adding `AdditionalProperties` to `AgentSession` introduces a serialization challenge: `AdditionalPropertiesDictionary` is a `Dictionary<string, object?>`, and `object?` values do not carry enough type information for the JSON deserializer to reconstruct the original CLR types.
|
||||
|
||||
### Default behavior — JsonElement round-tripping
|
||||
|
||||
By default, when an `AgentSession` with `AdditionalProperties` is serialized and later deserialized, any complex objects stored as values in the dictionary will be deserialized as `JsonElement` rather than their original types. This is the same behavior exhibited by `ChatMessage.AdditionalProperties` and other `AdditionalPropertiesDictionary` usages in `Microsoft.Extensions.AI`, and is the approach we will follow.
|
||||
|
||||
### Custom serialization via JsonSerializerOptions
|
||||
|
||||
`AIAgent.SerializeSessionAsync` and `AIAgent.DeserializeSessionAsync` already accept an optional `JsonSerializerOptions` parameter. Users who need strongly-typed round-tripping of `AdditionalProperties` values can supply custom options with appropriate converters or type info resolvers. This is non-trivial to implement but provides full control over deserialization behavior when needed.
|
||||
|
||||
## More Information
|
||||
|
||||
- [ADR 0014 — Feature Collections](0014-feature-collections.md) established that `AdditionalProperties` on `AgentRunOptions` serves as the per-run extensibility mechanism. The proposed agent-level and session-level properties serve a complementary, distinct purpose: static metadata describing the agent or session itself.
|
||||
- `AdditionalPropertiesDictionary` is defined in `Microsoft.Extensions.AI` and is already a dependency of `Microsoft.Agents.AI.Abstractions`. No new package references are needed.
|
||||
- Type-safe access is available via the existing `AdditionalPropertiesExtensions` helper methods (`Add<T>`, `TryGetValue<T>`, `Contains<T>`, `Remove<T>`), which use `typeof(T).FullName` as the dictionary key.
|
||||
@@ -1,163 +0,0 @@
|
||||
---
|
||||
# These are optional elements. Feel free to remove any of them.
|
||||
status: accepted
|
||||
contact: westey-m
|
||||
date: 2026-02-25
|
||||
deciders: sergeymenshykh, markwallace, rbarreto, dmytrostruk, westey-m, eavanvalkenburg, stephentoub
|
||||
consulted:
|
||||
informed:
|
||||
---
|
||||
|
||||
# AgentSession serialization
|
||||
|
||||
## Context and Problem Statement
|
||||
|
||||
Serializing AgentSessions is done today by calling SerializeSession on the AIAgent instance and deserialization
|
||||
is done via the DeserializeSession method on the AIAgent instance.
|
||||
|
||||
This approach has some drawbacks:
|
||||
|
||||
1. It requires each AgentSession implementation to implement its own serialization logic. This can lead to inconsistencies and errors if not done correctly.
|
||||
1. It means that only one serialization format can be supported at a time. If we want to support multiple formats (e.g., JSON, XML, binary), we would need to implement separate serialization logic for each format.
|
||||
1. It is not possible to serialize and deserialize lists of AgentSessions, since each need to be handled individually.
|
||||
1. Users may not realise that they need to call these specific methods to serialize/deserialize AgentSessions.
|
||||
|
||||
The reason why this approach was chosen initially is that AgentSessions may have behaviors that are attached to them and only the agent knows what behaviors to attach.
|
||||
These behaviors also have their own state that are attached to the AgentSession.
|
||||
The behaviors may have references to SDKs or other resources that cannot be created via standard deserialization mechanisms.
|
||||
E.g. an AgentSession may have a custom ChatMessageStore that knows how to store chat history in a specific storage backend and has a reference to the SDK client for that backend.
|
||||
When deserializing the AgentSession, we need to make sure that the ChatMessageStore is created with the correct SDK client.
|
||||
|
||||
## Decision Drivers
|
||||
|
||||
- A. Ability to continue to support custom behaviors (AIContextProviders / ChatHistoryProviders).
|
||||
- B. Ability to serialize and deserialize AgentSessions via standard serialization mechanisms, e.g. JsonSerializer.Serialize and JsonSerializer.Deserialize.
|
||||
- C. Ability for the caller to access custom providers.
|
||||
|
||||
## Considered Options
|
||||
|
||||
- Option 1: Separate state from behavior, serialize state only and re-attach behavior on first usage
|
||||
- Option 2: Separate state from behavior, and only have state on AgentSession
|
||||
- Option 3: Keep the current approach of custom Serialize/Deserialize methods
|
||||
|
||||
### Option 1: Separate state from behavior, serialize state only and re-attach behavior on first usage
|
||||
|
||||
Decision Drivers satisfied: A, B and C (C only partially)
|
||||
|
||||
Have separate properties on the AgentSession for state and behavior and mark the behavior property with [JsonIgnore].
|
||||
After deserializing the AgentSession, the behavior is null and when the AgentSession is first used by the Agent, the behavior is created and attached to the AgentSession.
|
||||
|
||||
This requires polymorphic deserialization to be supported, so that the correct AgentSession subclass and the correct behavior state is created during deserialization.
|
||||
Since the implementations for AgentSessions and their behaviors are not all known at compile time, we need a way to register custom AgentSession types and their corresponding behavior types for serialization with System.Text.Json on our JsonUtilities helpers.
|
||||
|
||||
A drawback of this approach is that the AgentSession is in an incomplete state after deserialization until it is first used,
|
||||
so if a user was to call `GetService<MyBehavior>()` on the AgentSession before it is used by the Agent, it would return null.
|
||||
|
||||
Behaviors like ChatMessageStore and AIContextProviders would need to change to support taking state as input and exposing state publicly.
|
||||
|
||||
```csharp
|
||||
public class ChatClientAgentSession
|
||||
{
|
||||
...
|
||||
public ChatMessageStoreState ChatMessageStoreState { get; }
|
||||
public ChatMessageStore? ChatMessageStore { get; }
|
||||
...
|
||||
}
|
||||
|
||||
[JsonPolymorphic(TypeDiscriminatorPropertyName = "$type")]
|
||||
[JsonDerivedType(typeof(InMemoryChatMessageStoreState), nameof(InMemoryChatMessageStoreState))]
|
||||
public abstract class ChatMessageStoreState
|
||||
{
|
||||
}
|
||||
public class InMemoryChatMessageStoreState : ChatMessageStoreState
|
||||
{
|
||||
public IList<ChatMessage> Messages { get; set; } = [];
|
||||
}
|
||||
|
||||
public abstract class ChatMessageStore<TState>
|
||||
where TState : ChatMessageStoreState
|
||||
{
|
||||
...
|
||||
public abstract TState State { get; }
|
||||
...
|
||||
}
|
||||
|
||||
public sealed class InMemoryChatMessageStore : ChatMessageStore<InMemoryChatMessageStoreState>, IList<ChatMessage>
|
||||
{
|
||||
private readonly InMemoryChatMessageStoreState _state;
|
||||
|
||||
public InMemoryChatMessageStore(InMemoryChatMessageStoreState? state)
|
||||
{
|
||||
this._state = state ?? new InMemoryChatMessageStoreState();
|
||||
}
|
||||
|
||||
public override InMemoryChatMessageStoreState State => this._state;
|
||||
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
ChatClientAgent factories would need to change to support creating behaviors based on state:
|
||||
|
||||
```csharp
|
||||
public Func<ChatMessageStoreFactoryContext, ChatMessageStore>? ChatMessageStoreFactory { get; set; }
|
||||
|
||||
public class ChatMessageStoreFactoryContext
|
||||
{
|
||||
public ChatMessageStoreState? State { get; set; }
|
||||
}
|
||||
```
|
||||
|
||||
The run behavior of the ChatClientAgent would be as follows:
|
||||
|
||||
1. If an AgentSession is provided, check if the ChatMessageStore property is null.
|
||||
1. If it is, check if the ChatMessageStoreState property is null.
|
||||
1. If ChatMessageStoreState is null, check if there is a provided ChatMessageStoreFactory.
|
||||
1. If there is, call it with a ChatMessageStoreFactoryContext containing null State to create a default ChatMessageStore behavior, and update the AgentSession with the created behavior and its state.
|
||||
2. If there is not, create a default InMemoryChatMessageStore behavior, and update the AgentSession with the created behavior and its state.
|
||||
1. If ChatMessageStoreState is not null, check if there is a provided ChatMessageStoreFactory.
|
||||
1. If there is, call it with a ChatMessageStoreFactoryContext containing the State to create a ChatMessageStore behavior based on the state.
|
||||
2. If there is not, create an InMemoryChatMessageStore behavior based on the State.
|
||||
|
||||
### Option 2: Separate state from behavior, and only have state on AgentSession
|
||||
|
||||
Decision Drivers satisfied: A, B and C.
|
||||
|
||||
This is similar to Option 1 but instead of having a behavior property on the AgentSession, we only have a StateBag property on the AgentSession.
|
||||
Behaviors really make more sense to live with the agent rather than the Session, but state should live on the session.
|
||||
When the AgentSession is used by the Agent, the Agent runs the behaviors against the Session, and the behavior stores it's state on the Session StateBag.
|
||||
|
||||
This means that users are unable to access the behavior from the AgentSession, e.g. via `AgentSession.GetService<TBehavior>()`.
|
||||
|
||||
However, the behaviors can be public properties on the Agent or can be retrieved from the agent via `AIAgent.GetService<MyAIContextProvider>()`.
|
||||
|
||||
```csharp
|
||||
public class AgentSession
|
||||
{
|
||||
...
|
||||
public AgentSessionStateBag StateBag { get; protected set; } = new();
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
### Option 3: Keep the current approach of custom Serialize/Deserialize methods
|
||||
|
||||
Decision Drivers satisfied: A and C
|
||||
|
||||
This option keeps the current approach of having custom Serialize/Deserialize methods on the AgentSession and AIAgent.
|
||||
|
||||
## Decision Outcome
|
||||
|
||||
Chosen option:
|
||||
|
||||
**Option 2** — separate state from behavior, with only state on the AgentSession — because it satisfies all decision drivers and provides the cleanest separation of concerns. Since not all AgentSession implementations have yet been cleanly separated from their behaviors, AIAgent.SerializeSession and AIAgent.DeserializeSession is kept for the time being, but most session types can be serialized and deserialized directly using JsonSerializer.
|
||||
|
||||
### Consequences
|
||||
|
||||
- Good, because providers are fully stateless — the same provider instance works correctly across any number of concurrent sessions without risk of state leakage.
|
||||
- Good, because `AgentSession` can be serialized and deserialized with standard `System.Text.Json` mechanisms, satisfying decision driver B.
|
||||
- Good, because the generic `StateBag` is extensible — new providers can store arbitrary state without requiring changes to the session class.
|
||||
- Good, because users can access providers via the agent (e.g. `agent.GetService<InMemoryChatHistoryProvider>()`) satisfying decision driver C.
|
||||
- Good, because sessions are always in a complete and valid state after deserialization — there is no "incomplete until first use" problem as in Option 1.
|
||||
- Neutral, because providers cannot be accessed directly from the session; callers must go through the agent. This is a minor usability trade-off but keeps the session focused on state only.
|
||||
- Bad, because each provider must be disciplined about using `ProviderSessionState<T>` and not storing session-specific data in instance fields. This is a correctness concern for custom provider implementers.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,815 +0,0 @@
|
||||
---
|
||||
status: accepted
|
||||
contact: bentho
|
||||
date: 2026-02-27
|
||||
deciders: bentho, markwallace-microsoft, westey-m
|
||||
consulted: Pratyush Mishra, Shivam Shrivastava, Manni Arora (Centrica eval scenario)
|
||||
informed: Agent Framework team, Foundry Evals team
|
||||
---
|
||||
|
||||
# Agent Evaluation Architecture with Azure AI Foundry Integration
|
||||
|
||||
## Context and Problem Statement
|
||||
|
||||
Azure AI Foundry provides a rich evaluation service for AI agents — built-in evaluators for agent behavior (task adherence, intent resolution), tool usage (tool call accuracy, tool selection), quality (coherence, fluency, relevance), and safety (violence, self-harm, prohibited actions). Results are viewable in the Foundry portal with dashboards and comparison views.
|
||||
|
||||
However, using Foundry Evals with an agent-framework agent today requires significant manual effort. Developers must:
|
||||
|
||||
1. Transform agent-framework's `Message`/`Content` types into the OpenAI-style agent message schema that Foundry evaluators expect
|
||||
2. Map tool definitions from agent-framework's `FunctionTool` format to evaluator-compatible schemas
|
||||
3. Manually wire up the correct Foundry data source type (`azure_ai_traces`, `jsonl`, `azure_ai_target_completions`, etc.) depending on their scenario
|
||||
4. Handle App Insights trace ID queries, response ID collection, and eval polling
|
||||
|
||||
Additionally, evaluation is a concern that extends beyond any single provider. Developers may want to use local evaluators (LLM-as-judge, regex, keyword matching), third-party evaluation libraries, or multiple providers in combination. The architecture must support this without creating a Foundry-specific lock-in at the API level.
|
||||
|
||||
### Functional Requirements for Agent Evaluation
|
||||
|
||||
- **Single agents and workflows.** Evaluate both individual agent responses and multi-agent workflow results, with per-agent breakdown to pinpoint underperformance.
|
||||
- **One-shot and multi-turn conversations.** Capture full conversation trajectories — including tool calls and results — not just final query/response pairs.
|
||||
- **Conversation factoring.** Support splitting conversations into query/response in multiple ways (last turn, full trajectory, per-turn) because different factorings measure different things.
|
||||
- **Multiple providers, mix and match.** Run Foundry LLM-as-judge evaluators alongside fast local checks and custom evaluators on the same data, without restructuring code.
|
||||
- **Third-party extensibility.** Any evaluation library can participate by implementing the `Evaluator` protocol (Python) or `IAgentEvaluator` interface (.NET). No predetermined list of supported libraries — the protocol is intentionally simple (`evaluate(items) → results`) so that wrappers for libraries like DeepEval, RAGAS, or Promptfoo are straightforward to write.
|
||||
- **Bring your own evaluator.** Creating a custom evaluator should be as simple as writing a function.
|
||||
- **Evaluate without re-running.** Evaluate existing responses from logs or previous runs without invoking the agent again.
|
||||
|
||||
## Decision Drivers
|
||||
|
||||
- **Zero-friction evaluation**: Developers should go from "I have an agent" to "I have eval results" with minimal code.
|
||||
- **Provider-agnostic API**: Core evaluation capabilities must not be tied to any specific provider. Provider configuration should be separate from the evaluation call.
|
||||
- **Lowest concept count**: Introduce the fewest possible new types, abstractions, and APIs for developers to learn.
|
||||
- **Leverage existing knowledge**: The framework already knows which agents exist, what tools they have, and what conversations occurred. Evals should use this automatically rather than requiring the developer to re-specify it.
|
||||
- **Foundry-native results**: When using Foundry, results should be viewable in the Foundry portal with dashboards and comparison views.
|
||||
- **Progressive disclosure**: Simple scenarios should be near-zero code. Advanced scenarios should build on the same primitives.
|
||||
- **Cross-language parity**: Design must be implementable in both Python and .NET.
|
||||
|
||||
## Considered Options
|
||||
|
||||
1. **Provider-specific functions** — Build Foundry-specific helper functions (`evaluate_agent()`, etc.) directly in the Azure package. All eval functions take Foundry connection parameters.
|
||||
2. **Evaluator protocol with shared orchestration** — Define a provider-agnostic `Evaluator` protocol in the base agent library (`agent_framework` in Python, `Microsoft.Agents.AI` in .NET). Orchestration functions live alongside it. Providers implement the protocol.
|
||||
3. **Full eval framework** — Build comprehensive eval infrastructure including custom evaluator definitions, scoring profiles, and reporting inside agent-framework.
|
||||
|
||||
## Decision Outcome
|
||||
|
||||
Proposed option: "Evaluator protocol with shared orchestration", because it delivers the low-friction developer experience, supports multiple providers without API changes, and keeps the concept count low.
|
||||
|
||||
### Usage Examples
|
||||
|
||||
#### Evaluate an agent
|
||||
|
||||
The agent is invoked once per query by default. For statistically meaningful evaluation, provide multiple diverse queries. For measuring **consistency** (does the same query produce reliable results?), use `num_repetitions` to run each query N times independently:
|
||||
|
||||
**Python:**
|
||||
|
||||
```python
|
||||
evals = FoundryEvals(
|
||||
project_client=client,
|
||||
model_deployment="gpt-4o",
|
||||
evaluators=[FoundryEvals.RELEVANCE, FoundryEvals.COHERENCE],
|
||||
)
|
||||
|
||||
results = await evaluate_agent(
|
||||
agent=my_agent,
|
||||
queries=[
|
||||
"What's the weather in Seattle?",
|
||||
"Plan a weekend trip to Portland",
|
||||
"What restaurants are near Pike Place?",
|
||||
],
|
||||
evaluators=evals,
|
||||
)
|
||||
for r in results:
|
||||
r.assert_passed()
|
||||
```
|
||||
|
||||
**C#:**
|
||||
|
||||
```csharp
|
||||
var evals = new FoundryEvals(chatConfiguration, FoundryEvals.Relevance, FoundryEvals.Coherence);
|
||||
|
||||
AgentEvaluationResults results = await agent.EvaluateAsync(
|
||||
new[] {
|
||||
"What's the weather in Seattle?",
|
||||
"Plan a weekend trip to Portland",
|
||||
"What restaurants are near Pike Place?",
|
||||
},
|
||||
evals);
|
||||
|
||||
results.AssertAllPassed();
|
||||
```
|
||||
|
||||
`evaluate_agent` returns one `EvalResults` per evaluator. Each result contains per-item scores with the evaluated response for auditing:
|
||||
|
||||
```
|
||||
# results[0] (FoundryEvals)
|
||||
EvalResults(status="completed", passed=3, failed=0, total=3)
|
||||
items[0]: EvalItemResult(
|
||||
query="What's the weather in Seattle?",
|
||||
response="It's currently 72°F and sunny in Seattle.",
|
||||
scores={"relevance": 5, "coherence": 5})
|
||||
items[1]: EvalItemResult(
|
||||
query="Plan a weekend trip to Portland",
|
||||
response="Here's a 2-day Portland itinerary...",
|
||||
scores={"relevance": 4, "coherence": 5})
|
||||
items[2]: EvalItemResult(
|
||||
query="What restaurants are near Pike Place?",
|
||||
response="Top restaurants near Pike Place Market: ...",
|
||||
scores={"relevance": 5, "coherence": 4})
|
||||
```
|
||||
|
||||
#### Measure consistency with repetitions
|
||||
|
||||
Run each query multiple times to detect non-deterministic behavior:
|
||||
|
||||
**Python:**
|
||||
|
||||
```python
|
||||
results = await evaluate_agent(
|
||||
agent=my_agent,
|
||||
queries=["What's the weather in Seattle?"],
|
||||
evaluators=evals,
|
||||
num_repetitions=3, # each query runs 3 times independently
|
||||
)
|
||||
# results contain 3 items (1 query × 3 repetitions)
|
||||
```
|
||||
|
||||
**C#:**
|
||||
|
||||
```csharp
|
||||
AgentEvaluationResults results = await agent.EvaluateAsync(
|
||||
new[] { "What's the weather in Seattle?" },
|
||||
evals,
|
||||
numRepetitions: 3); // each query runs 3 times independently
|
||||
// results contain 3 items (1 query × 3 repetitions)
|
||||
```
|
||||
|
||||
#### Evaluate a response you already have
|
||||
|
||||
When you already have agent responses, pass them directly to skip re-running the agent. Each query is paired with its corresponding response:
|
||||
|
||||
**Python:**
|
||||
|
||||
```python
|
||||
queries = ["What's the weather?", "What's the capital of France?"]
|
||||
responses = [await agent.run([Message("user", [q])]) for q in queries]
|
||||
|
||||
results = await evaluate_agent(
|
||||
responses=responses,
|
||||
evaluators=evals,
|
||||
)
|
||||
```
|
||||
|
||||
**C#:**
|
||||
|
||||
```csharp
|
||||
var queries = new[] { "What's the weather?" };
|
||||
var responses = new List<AgentResponse>();
|
||||
foreach (var q in queries)
|
||||
responses.Add(await agent.RunAsync(new[] { new ChatMessage(ChatRole.User, q) }));
|
||||
|
||||
AgentEvaluationResults results = await agent.EvaluateAsync(
|
||||
responses: responses,
|
||||
evals);
|
||||
```
|
||||
|
||||
Each `AgentResponse` already contains the conversation (query + response), so the evaluator extracts query/response from the conversation. When you pass `responses` without `queries`, the conversation is the source of truth.
|
||||
|
||||
#### Evaluate with conversation split strategies
|
||||
|
||||
By default, evaluators see only the last turn (final user message → final assistant response). For multi-turn conversations, you can control how the conversation is factored for evaluation:
|
||||
|
||||
**Python:**
|
||||
|
||||
```python
|
||||
results = await evaluate_agent(
|
||||
agent=agent,
|
||||
queries=["Plan a 3-day trip to Paris"],
|
||||
evaluators=evals,
|
||||
conversation_split=ConversationSplit.FULL, # evaluate entire trajectory
|
||||
)
|
||||
|
||||
# Or per-turn: each user→assistant exchange scored independently
|
||||
results = await evaluate_agent(
|
||||
agent=agent,
|
||||
queries=["Plan a 3-day trip to Paris"],
|
||||
evaluators=evals,
|
||||
conversation_split=ConversationSplit.PER_TURN,
|
||||
)
|
||||
```
|
||||
|
||||
**C#:**
|
||||
|
||||
```csharp
|
||||
// Full conversation as context
|
||||
AgentEvaluationResults results = await agent.EvaluateAsync(
|
||||
new[] { "Plan a 3-day trip to Paris" },
|
||||
evals,
|
||||
splitter: ConversationSplitters.Full);
|
||||
|
||||
// Per-turn splitting
|
||||
var items = EvalItem.PerTurnItems(conversation); // one EvalItem per user turn
|
||||
var results = await evals.EvaluateAsync(items);
|
||||
```
|
||||
|
||||
With `PER_TURN`, a 3-turn conversation produces 3 scored items:
|
||||
|
||||
```
|
||||
EvalResults(status="completed", passed=3, failed=0, total=3)
|
||||
items[0]: query="Plan a 3-day trip to Paris" scores={"relevance": 5}
|
||||
items[1]: query="What about restaurants?" scores={"relevance": 4}
|
||||
items[2]: query="Make it budget-friendly" scores={"relevance": 5}
|
||||
```
|
||||
|
||||
#### Evaluate a multi-agent workflow
|
||||
|
||||
**Python:**
|
||||
|
||||
```python
|
||||
result = await workflow.run("Plan a trip to Paris")
|
||||
eval_results = await evaluate_workflow(
|
||||
workflow=workflow,
|
||||
workflow_result=result,
|
||||
evaluators=evals,
|
||||
)
|
||||
|
||||
for r in eval_results:
|
||||
print(f" overall: {r.passed}/{r.total}")
|
||||
for name, sub in r.sub_results.items():
|
||||
print(f" {name}: {sub.passed}/{sub.total}")
|
||||
```
|
||||
|
||||
**C#:**
|
||||
|
||||
```csharp
|
||||
WorkflowRunResult result = await workflow.RunAsync("Plan a trip to Paris");
|
||||
|
||||
IReadOnlyList<AgentEvaluationResults> evalResults = await result.EvaluateAsync(evals);
|
||||
|
||||
foreach (var r in evalResults)
|
||||
{
|
||||
Console.WriteLine($" overall: {r.Passed}/{r.Total}");
|
||||
foreach (var (name, sub) in r.SubResults)
|
||||
Console.WriteLine($" {name}: {sub.Passed}/{sub.Total}");
|
||||
}
|
||||
```
|
||||
|
||||
Workflows return one result per evaluator, with sub-results per agent in the workflow:
|
||||
|
||||
```
|
||||
EvalResults(status="completed", passed=2, failed=0, total=2)
|
||||
sub_results:
|
||||
"planner": EvalResults(passed=1, total=1)
|
||||
"researcher": EvalResults(passed=1, total=1)
|
||||
```
|
||||
|
||||
#### Mix multiple providers
|
||||
|
||||
**Python:**
|
||||
|
||||
```python
|
||||
@evaluator
|
||||
def is_helpful(response: str) -> bool:
|
||||
return len(response.split()) > 10
|
||||
|
||||
foundry = FoundryEvals(
|
||||
project_client=client,
|
||||
model_deployment="gpt-4o",
|
||||
evaluators=[FoundryEvals.RELEVANCE, FoundryEvals.COHERENCE],
|
||||
)
|
||||
|
||||
results = await evaluate_agent(
|
||||
agent=agent,
|
||||
queries=queries,
|
||||
evaluators=[is_helpful, keyword_check("weather"), foundry],
|
||||
)
|
||||
```
|
||||
|
||||
**C#:**
|
||||
|
||||
```csharp
|
||||
IReadOnlyList<AgentEvaluationResults> results = await agent.EvaluateAsync(
|
||||
queries,
|
||||
evaluators: new IAgentEvaluator[]
|
||||
{
|
||||
new LocalEvaluator(
|
||||
EvalChecks.KeywordCheck("weather"),
|
||||
FunctionEvaluator.Create("is_helpful", (string r) => r.Split(' ').Length > 10)),
|
||||
new FoundryEvals(chatConfiguration, FoundryEvals.Relevance, FoundryEvals.Coherence),
|
||||
});
|
||||
```
|
||||
|
||||
Multiple evaluators return one result each — `results[0]` is the local evaluator, `results[1]` is Foundry.
|
||||
|
||||
#### Custom function evaluators
|
||||
|
||||
**Python:**
|
||||
|
||||
```python
|
||||
@evaluator
|
||||
def mentions_city(response: str, expected_output: str) -> bool:
|
||||
return expected_output.lower() in response.lower()
|
||||
|
||||
@evaluator
|
||||
def used_tools(conversation: list, tools: list) -> float:
|
||||
# ... scoring logic
|
||||
return score
|
||||
|
||||
local = LocalEvaluator(mentions_city, used_tools)
|
||||
```
|
||||
|
||||
`@evaluator` uses **parameter name injection** — the function's parameter names determine what data it receives from the `EvalItem`. Supported names: `query`, `response`, `expected`, `expected_tool_calls`, `conversation`, `tools`, `context`. Any combination is valid.
|
||||
|
||||
**C#:**
|
||||
|
||||
```csharp
|
||||
var local = new LocalEvaluator(
|
||||
FunctionEvaluator.Create("mentions_city",
|
||||
(EvalItem item) => item.ExpectedOutput != null
|
||||
&& item.Response.Contains(item.ExpectedOutput, StringComparison.OrdinalIgnoreCase)),
|
||||
FunctionEvaluator.Create("is_concise",
|
||||
(string response) => response.Split(' ').Length < 500));
|
||||
```
|
||||
|
||||
## What To Build
|
||||
|
||||
### Core: Evaluator Protocol
|
||||
|
||||
A runtime-checkable protocol that any evaluation provider implements:
|
||||
|
||||
```python
|
||||
@runtime_checkable
|
||||
class Evaluator(Protocol):
|
||||
name: str
|
||||
|
||||
async def evaluate(
|
||||
self, items: Sequence[EvalItem], *, eval_name: str = "Agent Framework Eval"
|
||||
) -> EvalResults: ...
|
||||
```
|
||||
|
||||
The protocol is minimal — just `name` and `evaluate()`.
|
||||
|
||||
### Core: EvalItem
|
||||
|
||||
Provider-agnostic data format for items to evaluate:
|
||||
|
||||
```python
|
||||
@dataclass
|
||||
class ExpectedToolCall:
|
||||
name: str # Tool/function name
|
||||
arguments: dict[str, Any] | None = None # None = don't check args
|
||||
|
||||
@dataclass
|
||||
class EvalItem:
|
||||
conversation: list[Message] # Single source of truth
|
||||
tools: list[FunctionTool] | None = None # Agent's available tools
|
||||
context: str | None = None
|
||||
expected_output: str | None = None # Ground-truth for comparison
|
||||
expected_tool_calls: list[ExpectedToolCall] | None = None
|
||||
split_strategy: ConversationSplitter | None = None
|
||||
|
||||
query: str # property — derived from conversation split
|
||||
response: str # property — derived from conversation split
|
||||
```
|
||||
|
||||
`conversation` is the single source of truth. `query` and `response` are derived properties — splitting the conversation at the last user message (default) and extracting text from each side. Changing the `split_strategy` consistently changes all derived values.
|
||||
|
||||
`tools` provides typed `FunctionTool` objects — including MCP tools, which are automatically extracted after agent runs.
|
||||
|
||||
### Internal: AgentEvalConverter
|
||||
|
||||
Internal class that converts agent-framework types to `EvalItem`. Used by `evaluate_agent()` and `evaluate_workflow()` — not part of the public API:
|
||||
|
||||
| Agent Framework | Eval Format |
|
||||
|---|---|
|
||||
| `Content.function_call` | `tool_call` in OpenAI chat format |
|
||||
| `Content.function_result` | `tool_result` in OpenAI chat format |
|
||||
| `FunctionTool` | `{name, description, parameters}` schema |
|
||||
| `Message` history | `conversation` list + `query`/`response` extraction |
|
||||
|
||||
### Core: EvalResults
|
||||
|
||||
Rich result type with convenience properties for CI integration:
|
||||
|
||||
```python
|
||||
results.all_passed # bool: no failures or errors (recursive for workflow)
|
||||
results.passed # int: passing count
|
||||
results.failed # int: failure count
|
||||
results.total # int: total = passed + failed + errored
|
||||
results.items # list[EvalItemResult]: per-item detail with query, response, and scores
|
||||
results.error # str | None: error details on failure
|
||||
results.sub_results # dict: per-agent breakdown (workflow evals)
|
||||
results.report_url # str | None: portal link (Foundry)
|
||||
results.assert_passed() # raises AssertionError with details
|
||||
```
|
||||
|
||||
### Core: Orchestration Functions
|
||||
|
||||
Provider-agnostic functions that extract data and delegate to evaluators:
|
||||
|
||||
| Function | What it does |
|
||||
|---|---|
|
||||
| `evaluate_agent()` | Runs agent against test queries (or evaluates pre-existing `responses=`), converts to `EvalItem`s, passes to evaluator. Accepts optional `expected_output=` for ground-truth comparison, `expected_tool_calls=` for tool-correctness evaluation, and `num_repetitions=` for consistency measurement |
|
||||
| `evaluate_workflow()` | Extracts per-agent data from `WorkflowRunResult`, evaluates each agent and overall output. Per-agent breakdown in `sub_results`. Also accepts `num_repetitions=` |
|
||||
|
||||
### Core: Conversation Split Strategies
|
||||
|
||||
Multi-turn conversations must be split into query (input) and response (output) halves for evaluation. How you split determines *what you're evaluating*:
|
||||
|
||||
**Last-turn split** — split at the last user message. Everything up to and including it is the query context; the agent's subsequent actions are the response:
|
||||
|
||||
```
|
||||
conversation: user1 → assistant1 → user2 → assistant2(tool) → tool_result → assistant3
|
||||
query_messages: [user1, assistant1, user2]
|
||||
response_messages: [assistant2(tool), tool_result, assistant3]
|
||||
```
|
||||
|
||||
This evaluates: "Given all the context so far, did the agent answer the latest question well?" Best for response quality at a specific point in the conversation.
|
||||
|
||||
**Full-conversation split** — the first user message is the query; everything after is the response:
|
||||
|
||||
```
|
||||
query_messages: [user1]
|
||||
response_messages: [assistant1, user2, assistant2(tool), tool_result, assistant3]
|
||||
```
|
||||
|
||||
This evaluates: "Given the original request, did the entire conversation trajectory serve the user?" Best for task completion and overall conversation quality.
|
||||
|
||||
**Per-turn split** — produces N eval items from an N-turn conversation. Each turn is evaluated with its cumulative context:
|
||||
|
||||
```
|
||||
item 1: query = [user1], response = [assistant1]
|
||||
item 2: query = [user1, assistant1, user2], response = [assistant2(tool), tool_result, assistant3]
|
||||
```
|
||||
|
||||
This evaluates each response independently. Best for fine-grained analysis and pinpointing where a conversation goes wrong.
|
||||
|
||||
These factorings produce different scores for the same conversation. The framework ships all three as built-in strategies, defaulting to last-turn. Developers can also provide a custom splitter — a function (Python) or `IConversationSplitter` implementation (.NET) — and override the strategy at the call site or per evaluator.
|
||||
|
||||
### Azure AI: FoundryEvals
|
||||
|
||||
`Evaluator` implementation backed by Azure AI Foundry:
|
||||
|
||||
```python
|
||||
class FoundryEvals:
|
||||
def __init__(self, *, project_client=None, openai_client=None,
|
||||
model_deployment: str, evaluators=None, ...)
|
||||
async def evaluate(self, items, *, eval_name) -> EvalResults
|
||||
```
|
||||
|
||||
**Smart auto-detection in `evaluate()`:**
|
||||
- Default evaluators: relevance, coherence, task_adherence
|
||||
- Auto-adds `tool_call_accuracy` when items have tools/`tool_definitions`
|
||||
- Filters out tool evaluators for items without tools
|
||||
|
||||
### Azure AI: FoundryEvals Constants
|
||||
|
||||
```python
|
||||
from agent_framework_azure_ai import FoundryEvals
|
||||
|
||||
evaluators = [FoundryEvals.RELEVANCE, FoundryEvals.TOOL_CALL_ACCURACY]
|
||||
```
|
||||
|
||||
Categories: Agent behavior, Tool usage, Quality, Safety.
|
||||
|
||||
### Azure AI: Foundry-Specific Functions
|
||||
|
||||
| Function | What it does |
|
||||
|---|---|
|
||||
| `evaluate_traces()` | Evaluate from stored response IDs or OTel traces |
|
||||
| `evaluate_foundry_target()` | Evaluate a Foundry-registered agent or deployment |
|
||||
|
||||
### Core: LocalEvaluator and Function Evaluators
|
||||
|
||||
`LocalEvaluator` implements the `Evaluator` protocol for fast, API-free evaluation. It runs check functions locally — useful for inner-loop development, CI smoke tests, and combining with cloud-based evaluators.
|
||||
|
||||
Built-in checks:
|
||||
- `keyword_check(*keywords)` — response must contain specified keywords
|
||||
- `tool_called_check(*tool_names)` — agent must have called specified tools
|
||||
- `tool_calls_present` — all `expected_tool_calls` names appear in conversation (unordered, extras OK)
|
||||
- `tool_call_args_match` — expected tool calls match on name + arguments (subset match on args)
|
||||
|
||||
Custom function evaluators use `@evaluator` to wrap plain Python functions. The function's **parameter names** determine what data it receives from the `EvalItem`:
|
||||
|
||||
```python
|
||||
from agent_framework import evaluator, LocalEvaluator
|
||||
|
||||
# Tier 1: Simple check — just query + response
|
||||
@evaluator
|
||||
def is_concise(response: str) -> bool:
|
||||
return len(response.split()) < 500
|
||||
|
||||
# Tier 2: Ground truth — compare against expected output
|
||||
@evaluator
|
||||
def mentions_city(response: str, expected_output: str) -> bool:
|
||||
return expected_output.lower() in response.lower()
|
||||
|
||||
# Tier 3: Full context — inspect conversation and tools
|
||||
@evaluator
|
||||
def used_tools(conversation: list, tools: list) -> float:
|
||||
# ... scoring logic
|
||||
return score
|
||||
|
||||
local = LocalEvaluator(is_concise, mentions_city, used_tools)
|
||||
```
|
||||
|
||||
Supported parameters: `query`, `response`, `expected`, `expected_tool_calls`, `conversation`, `tools`, `context`.
|
||||
Return types: `bool`, `float` (≥0.5 = pass), `dict` with `score` or `passed` key, or `CheckResult`.
|
||||
|
||||
Async functions are handled automatically — `@evaluator` detects `async def` and produces the right wrapper.
|
||||
|
||||
### Example: GAIA Benchmark
|
||||
|
||||
[GAIA](https://huggingface.co/gaia-benchmark) tests real-world multi-step tasks with known expected answers. Each task has a question and a ground-truth answer, with optional file attachments. The framework accommodates GAIA's knobs (difficulty levels, file inputs, multi-step tool use) through the existing `EvalItem` fields:
|
||||
|
||||
```python
|
||||
from datasets import load_dataset
|
||||
from agent_framework import evaluate_agent, evaluator, LocalEvaluator
|
||||
|
||||
gaia = load_dataset("gaia-benchmark/GAIA", "2023_level1", split="test")
|
||||
|
||||
@evaluator
|
||||
def exact_match(response: str, expected_output: str) -> bool:
|
||||
return expected_output.strip().lower() in response.strip().lower()
|
||||
|
||||
# Simple path — evaluate_agent handles running + expected_output stamping
|
||||
results = await evaluate_agent(
|
||||
agent=agent,
|
||||
queries=[task["Question"] for task in gaia],
|
||||
expected_output=[task["Final answer"] for task in gaia],
|
||||
evaluators=LocalEvaluator(exact_match),
|
||||
)
|
||||
```
|
||||
|
||||
### Package Location
|
||||
|
||||
- Core types and orchestration: `agent_framework._eval`, `agent_framework._local_eval` (Python), `Microsoft.Agents.AI` (.NET)
|
||||
- Foundry provider: `agent_framework_azure_ai._foundry_evals` (Python), `Microsoft.Agents.AI.AzureAI` (.NET)
|
||||
- Azure-AI re-exports core types for convenience (Python)
|
||||
|
||||
## Known Limitations
|
||||
|
||||
1. **Tool evaluators require query + agent**: Tool evaluators need tool definition schemas. When using these evaluators with `evaluate_agent(responses=...)`, provide `queries=` and pass an agent with tool definitions.
|
||||
2. **`model_deployment` always required**: Could potentially be inferred from the Foundry project configuration.
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. **Red teaming non-registered agents**: Requires Foundry API support for callback-based flows.
|
||||
2. **Datasets with expected outputs**: A dataset abstraction for pre-populating `expected_output` values across eval runs is a natural next step but not yet designed.
|
||||
3. **Multi-modal evaluation**: The `conversation` field on `EvalItem` already stores full `Message`/`Content` (Python) and `ChatMessage` (.NET) objects, which can represent multi-modal content (images, audio, structured data). Evaluators that accept the full `EvalItem` or `conversation` parameter can access this content today. However, the convenience shortcuts — `query`/`response` string projections and the `FunctionEvaluator` string overloads — are text-only. Multi-modal-aware evaluators should use the full-item path (`Func<EvalItem, CheckResult>` in .NET, `conversation: list` parameter in Python).
|
||||
|
||||
## .NET Implementation Design
|
||||
|
||||
### Key Difference: MEAI Ecosystem
|
||||
|
||||
Unlike Python, the .NET ecosystem already has `Microsoft.Extensions.AI.Evaluation` (v10.3.0) providing:
|
||||
|
||||
- `IEvaluator` — per-item evaluation of `(messages, chatResponse) → EvaluationResult`
|
||||
- `CompositeEvaluator` — combines multiple evaluators
|
||||
- Quality evaluators — `RelevanceEvaluator`, `CoherenceEvaluator`, `GroundednessEvaluator`
|
||||
- Safety evaluators — `ContentHarmEvaluator`, `ProtectedMaterialEvaluator`
|
||||
- Metric types — `NumericMetric`, `BooleanMetric`, `StringMetric`
|
||||
|
||||
The .NET integration uses MEAI's `IEvaluator` directly — no new evaluator interface. Our contribution is the **orchestration layer**: extension methods that run agents, extract data, call `IEvaluator` per item, and aggregate results.
|
||||
|
||||
### Architecture
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────────┐
|
||||
│ Developer Code │
|
||||
│ agent.EvaluateAsync(queries, evaluator) │
|
||||
│ run.EvaluateAsync(evaluator) │
|
||||
└────────────────┬─────────────────────────────────────────────┘
|
||||
│
|
||||
┌────────────────▼─────────────────────────────────────────────┐
|
||||
│ Orchestration Layer (Microsoft.Agents.AI) │
|
||||
│ AgentEvaluationExtensions — runs agents, extracts data, │
|
||||
│ calls IEvaluator per item, aggregates into │
|
||||
│ AgentEvaluationResults │
|
||||
└────────────────┬─────────────────────────────────────────────┘
|
||||
│ IEvaluator (MEAI)
|
||||
│
|
||||
┌───────────┼────────────┐
|
||||
│ │ │
|
||||
┌───▼───-┐ ┌───▼────┐ ┌────▼──────────┐
|
||||
│ MEAI │ │ Local │ │ Foundry │
|
||||
│ Quality│ │ Checks │ │ (cloud batch) │
|
||||
│ Safety │ │ Lambdas│ │ │
|
||||
└────────┘ └────────┘ └───────────────┘
|
||||
```
|
||||
|
||||
All evaluators implement MEAI's `IEvaluator`. The orchestration layer doesn't need to know which kind — it calls `EvaluateAsync(messages, chatResponse)` per item on all of them. `FoundryEvals` handles batching internally (buffers items, submits once, returns per-item results).
|
||||
|
||||
### .NET Core Types
|
||||
|
||||
**No new evaluator interface.** Use MEAI's `IEvaluator` directly.
|
||||
|
||||
**`AgentEvaluationResults`** — The only new type. Aggregates per-item MEAI `EvaluationResult`s across a batch of queries:
|
||||
|
||||
```csharp
|
||||
public class AgentEvaluationResults
|
||||
{
|
||||
public string Provider { get; init; }
|
||||
public string? ReportUrl { get; init; }
|
||||
|
||||
// Per-item — standard MEAI EvaluationResult, unchanged
|
||||
public IReadOnlyList<EvaluationResult> Items { get; init; }
|
||||
|
||||
// Aggregate pass/fail derived from metric interpretations
|
||||
public int Passed { get; }
|
||||
public int Failed { get; }
|
||||
public int Total { get; }
|
||||
public bool AllPassed { get; }
|
||||
|
||||
// Workflow: per-agent breakdown
|
||||
public IReadOnlyDictionary<string, AgentEvaluationResults>? SubResults { get; init; }
|
||||
|
||||
public void AssertAllPassed(string? message = null);
|
||||
}
|
||||
```
|
||||
|
||||
### .NET Evaluator Implementations
|
||||
|
||||
All implement MEAI's `IEvaluator`:
|
||||
|
||||
**`LocalEvaluator`** — Runs lambda checks locally, returns `BooleanMetric` per check:
|
||||
|
||||
```csharp
|
||||
var local = new LocalEvaluator(
|
||||
FunctionEvaluator.Create("is_concise",
|
||||
(string response) => response.Split().Length < 500),
|
||||
EvalChecks.KeywordCheck("weather"),
|
||||
EvalChecks.ToolCalledCheck("get_weather"));
|
||||
```
|
||||
|
||||
**MEAI evaluators** — Used directly, no adapter needed:
|
||||
|
||||
```csharp
|
||||
var quality = new CompositeEvaluator(
|
||||
new RelevanceEvaluator(),
|
||||
new CoherenceEvaluator());
|
||||
```
|
||||
|
||||
**`FoundryEvals`** — Implements `IEvaluator` but batches internally. On first call, buffers the item. On the last item (or when explicitly flushed), submits the batch to Foundry and distributes per-item results:
|
||||
|
||||
```csharp
|
||||
var foundry = new FoundryEvals(projectClient, "gpt-4o");
|
||||
```
|
||||
|
||||
### .NET Orchestration: Extension Methods
|
||||
|
||||
```csharp
|
||||
public static class AgentEvaluationExtensions
|
||||
{
|
||||
// Evaluate an agent against test queries
|
||||
public static Task<AgentEvaluationResults> EvaluateAsync(
|
||||
this AIAgent agent,
|
||||
IEnumerable<string> queries,
|
||||
IEvaluator evaluator,
|
||||
ChatConfiguration? chatConfiguration = null,
|
||||
IEnumerable<string>? expectedOutput = null,
|
||||
CancellationToken cancellationToken = default);
|
||||
|
||||
// Evaluate pre-existing responses (without re-running the agent)
|
||||
public static Task<AgentEvaluationResults> EvaluateAsync(
|
||||
this AIAgent agent,
|
||||
AgentResponse responses,
|
||||
IEvaluator evaluator,
|
||||
IEnumerable<string>? queries = null,
|
||||
ChatConfiguration? chatConfiguration = null,
|
||||
IEnumerable<string>? expectedOutput = null,
|
||||
CancellationToken cancellationToken = default);
|
||||
|
||||
// Evaluate with multiple evaluators (one result per evaluator)
|
||||
public static Task<IReadOnlyList<AgentEvaluationResults>> EvaluateAsync(
|
||||
this AIAgent agent,
|
||||
IEnumerable<string> queries,
|
||||
IEnumerable<IEvaluator> evaluators,
|
||||
ChatConfiguration? chatConfiguration = null,
|
||||
IEnumerable<string>? expectedOutput = null,
|
||||
CancellationToken cancellationToken = default);
|
||||
|
||||
// Evaluate a workflow run with per-agent breakdown
|
||||
public static Task<AgentEvaluationResults> EvaluateAsync(
|
||||
this Run run,
|
||||
IEvaluator evaluator,
|
||||
ChatConfiguration? chatConfiguration = null,
|
||||
bool includeOverall = true,
|
||||
bool includePerAgent = true,
|
||||
CancellationToken cancellationToken = default);
|
||||
}
|
||||
```
|
||||
|
||||
**Usage:**
|
||||
|
||||
```csharp
|
||||
// MEAI evaluators — just works
|
||||
var results = await agent.EvaluateAsync(
|
||||
queries: ["What's the weather?"],
|
||||
evaluator: new RelevanceEvaluator(),
|
||||
chatConfiguration: new ChatConfiguration(evalClient));
|
||||
|
||||
// Local checks
|
||||
var results = await agent.EvaluateAsync(
|
||||
queries: ["What's the weather?"],
|
||||
evaluator: new LocalEvaluator(
|
||||
EvalChecks.KeywordCheck("weather")));
|
||||
|
||||
// Foundry cloud
|
||||
var results = await agent.EvaluateAsync(
|
||||
queries: ["What's the weather?"],
|
||||
evaluator: new FoundryEvals(projectClient, "gpt-4o"));
|
||||
|
||||
// Evaluate existing response (without re-running the agent)
|
||||
var response = await agent.RunAsync("What's the weather?");
|
||||
var results = await agent.EvaluateAsync(
|
||||
responses: response,
|
||||
queries: ["What's the weather?"],
|
||||
evaluator: new FoundryEvals(projectClient, "gpt-4o"));
|
||||
|
||||
// Mixed — one result per evaluator
|
||||
var results = await agent.EvaluateAsync(
|
||||
queries: ["What's the weather?"],
|
||||
evaluators: [
|
||||
new LocalEvaluator(EvalChecks.KeywordCheck("weather")),
|
||||
new RelevanceEvaluator(),
|
||||
new FoundryEvals(projectClient, "gpt-4o")
|
||||
],
|
||||
chatConfiguration: new ChatConfiguration(evalClient));
|
||||
|
||||
// Workflow with per-agent breakdown
|
||||
Run run = await workflowRunner.RunAsync(workflow, "Plan a trip");
|
||||
var results = await run.EvaluateAsync(
|
||||
evaluator: new FoundryEvals(projectClient, "gpt-4o"));
|
||||
```
|
||||
|
||||
### .NET Function Evaluators
|
||||
|
||||
Typed factory overloads (C# equivalent of Python's `@evaluator`):
|
||||
|
||||
```csharp
|
||||
public static class FunctionEvaluator
|
||||
{
|
||||
public static EvalCheck Create(string name, Func<string, bool> check); // response only
|
||||
public static EvalCheck Create(string name, Func<string, string?, bool> check); // expectedOutput
|
||||
public static EvalCheck Create(string name, Func<EvalItem, bool> check); // full item
|
||||
public static EvalCheck Create(string name, Func<EvalItem, CheckResult> check); // full control
|
||||
public static EvalCheck Create(string name, Func<string, Task<bool>> check); // async
|
||||
}
|
||||
```
|
||||
|
||||
`EvalItem` is a lightweight record used only by `FunctionEvaluator` and `LocalEvaluator` to pass context to check functions. It is not part of the `IEvaluator` interface:
|
||||
|
||||
```csharp
|
||||
public record ExpectedToolCall(string Name, IReadOnlyDictionary<string, object>? Arguments = null);
|
||||
|
||||
public sealed class EvalItem
|
||||
{
|
||||
public EvalItem(string query, string response, IReadOnlyList<ChatMessage> conversation);
|
||||
|
||||
public string Query { get; }
|
||||
public string Response { get; }
|
||||
public IReadOnlyList<ChatMessage> Conversation { get; }
|
||||
public IReadOnlyList<AITool>? Tools { get; set; }
|
||||
public string? ExpectedOutput { get; set; }
|
||||
public IReadOnlyList<ExpectedToolCall>? ExpectedToolCalls { get; set; }
|
||||
public string? Context { get; set; }
|
||||
public IConversationSplitter? Splitter { get; set; }
|
||||
}
|
||||
```
|
||||
|
||||
### Workflow Data Extraction (.NET)
|
||||
|
||||
`run.EvaluateAsync()` walks `Run.OutgoingEvents` via LINQ:
|
||||
|
||||
1. Pair `ExecutorInvokedEvent` / `ExecutorCompletedEvent` by `ExecutorId`
|
||||
2. Extract `AgentResponseEvent` for per-agent `ChatResponse`
|
||||
3. Call `evaluator.EvaluateAsync()` per invocation
|
||||
4. Group by `ExecutorId` for per-agent `SubResults`
|
||||
5. Use final workflow output for overall eval
|
||||
|
||||
### .NET Package Structure
|
||||
|
||||
| Package | Contents |
|
||||
|---------|----------|
|
||||
| `Microsoft.Agents.AI` | `IAgentEvaluator`, `AgentEvaluationResults`, `LocalEvaluator`, `FunctionEvaluator`, `EvalChecks`, `EvalItem`, `ExpectedToolCall`, `AgentEvaluationExtensions` |
|
||||
| `Microsoft.Agents.AI.AzureAI` | `FoundryEvals` (provider + constants) |
|
||||
|
||||
### Python ↔ .NET Mapping
|
||||
|
||||
| Python | .NET |
|
||||
|--------|------|
|
||||
| `Evaluator` protocol | `IAgentEvaluator` (our interface; MEAI provides `IEvaluator` for per-item scoring) |
|
||||
| `EvalItem` dataclass | `EvalItem` class |
|
||||
| `EvalResults` | `AgentEvaluationResults` |
|
||||
| `EvalItemResult` / `EvalScoreResult` | MEAI `EvaluationResult` / `EvaluationMetric` (reused) |
|
||||
| `LocalEvaluator` | `LocalEvaluator` (implements `IAgentEvaluator`) |
|
||||
| `@evaluator` | `FunctionEvaluator.Create()` overloads |
|
||||
| `keyword_check()` / `tool_called_check()` | `EvalChecks.KeywordCheck()` / `EvalChecks.ToolCalledCheck()` |
|
||||
| `tool_calls_present` / `tool_call_args_match` | (custom `FunctionEvaluator` — same pattern) |
|
||||
| `ExpectedToolCall` dataclass | `ExpectedToolCall` record |
|
||||
| `FoundryEvals` | `FoundryEvals` (implements `IAgentEvaluator`, includes evaluator name constants) |
|
||||
| `evaluate_agent()` | `agent.EvaluateAsync(queries, evaluator)` extension method |
|
||||
| `evaluate_agent(responses=)` | `agent.EvaluateAsync(responses, evaluator)` extension method |
|
||||
| `evaluate_workflow()` | `run.EvaluateAsync()` extension method |
|
||||
|
||||
## More Information
|
||||
|
||||
- [Foundry Evals documentation](https://learn.microsoft.com/azure/ai-foundry/concepts/evaluation-approach-gen-ai) — Azure AI Foundry evaluation overview
|
||||
@@ -1,48 +0,0 @@
|
||||
# AGENTS.md
|
||||
|
||||
Instructions for AI coding agents working on durable agents documentation.
|
||||
|
||||
## Scope
|
||||
|
||||
This directory contains feature documentation for the durable agents integration. The source code and samples live elsewhere:
|
||||
|
||||
- .NET implementation: `dotnet/src/Microsoft.Agents.AI.DurableTask/` and `dotnet/src/Microsoft.Agents.AI.Hosting.AzureFunctions/`
|
||||
- Python implementation: `python/packages/durabletask/` and `python/packages/azurefunctions/` (package `agent-framework-azurefunctions`)
|
||||
- .NET samples: `dotnet/samples/04-hosting/DurableAgents/`
|
||||
- Python samples: `python/samples/04-hosting/durabletask/`
|
||||
- Official docs (Microsoft Learn): <https://learn.microsoft.com/agent-framework/integrations/azure-functions>
|
||||
|
||||
## Document structure
|
||||
|
||||
| File | Purpose |
|
||||
| --- | --- |
|
||||
| `README.md` | Main technical overview: architecture, hosting models, orchestration patterns, and links to samples. |
|
||||
| `durable-agents-ttl.md` | Deep-dive on session Time-To-Live (TTL) configuration and behavior. |
|
||||
|
||||
Add new sibling documents when a topic is too detailed for the README (e.g., a new feature like reliable streaming or MCP tool exposure). Keep the README focused on orientation and link out to siblings for depth.
|
||||
|
||||
## Writing guidelines
|
||||
|
||||
- **Audience**: Developers already familiar with the Microsoft Agent Framework who want to understand what durability adds and how to use it.
|
||||
- **Host-agnostic first**: Durable agents work in console apps, Azure Functions, and any Durable Task–compatible host. Show host-agnostic patterns (plain orchestration functions, `IServiceCollection` registration) before Azure Functions–specific patterns. Avoid giving the impression that Azure Functions is the only hosting option.
|
||||
- **Both languages**: Always include C# and Python examples side by side. Keep them equivalent in functionality.
|
||||
- **Callout syntax**: Use GitHub-flavored callouts (`> [!NOTE]`, `> [!IMPORTANT]`, `> [!WARNING]`) rather than bold-text callouts (`> **Note:** ...`).
|
||||
- **Line length**: Do not wrap long lines. Rely on text viewers / renderers for line wrapping.
|
||||
- **Tables**: Use spaces around pipes in separator rows (`| --- |` not `|---|`).
|
||||
- **Code snippets**: Keep them minimal and self-contained. Omit boilerplate (using statements, environment variable reads) unless the snippet is specifically about setup.
|
||||
- **Cross-references**: Link to Microsoft Learn for conceptual background (Durable Entities, Durable Task Scheduler, Azure Functions). Link to sibling docs within this directory for feature deep-dives.
|
||||
|
||||
## Linting
|
||||
|
||||
Run markdownlint on all documents before committing, with line-length checks disabled:
|
||||
|
||||
```bash
|
||||
markdownlint docs/features/durable-agents/ --disable MD013
|
||||
```
|
||||
|
||||
## When to update these docs
|
||||
|
||||
- A new durable agent feature is added (e.g., a new orchestration pattern, hosting model, or configuration option).
|
||||
- The public API surface changes in a way that affects how developers use durable agents.
|
||||
- New sample directories are added — update the sample links in README.md.
|
||||
- The official Microsoft Learn documentation is restructured — update external links.
|
||||
@@ -1,239 +0,0 @@
|
||||
# Durable agents
|
||||
|
||||
## Overview
|
||||
|
||||
Durable agents extend the standard Microsoft Agent Framework with **durable state management** powered by the Durable Task framework. An ordinary Agent Framework agent runs in-process: its conversation history lives in memory and is lost when the process ends. A durable agent persists conversation history and execution state in external storage so that sessions survive process restarts, failures, and scale-out events.
|
||||
|
||||
| Capability | Ordinary agent | Durable agent |
|
||||
| --- | --- | --- |
|
||||
| Conversation history | In-memory only | Durably persisted |
|
||||
| Failure recovery | State lost on crash | Automatically resumed |
|
||||
| Multi-instance scale-out | Not supported | Any worker can resume a session |
|
||||
| Multi-agent orchestrations | Manual coordination | Deterministic, checkpointed workflows |
|
||||
| Human-in-the-loop | Must keep process alive | Can wait days/weeks with zero compute |
|
||||
| Hosting | Any process | Console app, Azure Functions, or any Durable Task–compatible host |
|
||||
|
||||
> [!NOTE]
|
||||
> For a step-by-step tutorial and deployment guidance, see [Azure Functions (Durable)](https://learn.microsoft.com/agent-framework/integrations/azure-functions) on Microsoft Learn.
|
||||
|
||||
## How durable agents work
|
||||
|
||||
Durable agents are implemented on top of [Durable Entities](https://learn.microsoft.com/azure/azure-functions/durable/durable-functions-entities) (also called "virtual actors"). Each **agent session** maps to one entity instance whose state contains the full conversation history. When you send a message to a durable agent, the following happens:
|
||||
|
||||
1. The message is dispatched to the entity identified by an `AgentSessionId` (a composite of the agent name and a unique session key).
|
||||
2. The entity loads its persisted `DurableAgentState`, which includes the complete conversation history.
|
||||
3. The entity invokes the underlying `AIAgent` with the full conversation history, collects the response, and appends both the request and the response to the state.
|
||||
4. The updated state is persisted back to durable storage automatically.
|
||||
|
||||
Because the entity framework serializes access to each entity instance, concurrent messages to the same session are processed one at a time, eliminating race conditions.
|
||||
|
||||
### Agent session identity
|
||||
|
||||
Every durable agent session is identified by an `AgentSessionId`, which has two components:
|
||||
|
||||
- **Name** – the registered name of the agent (case-insensitive).
|
||||
- **Key** – a unique session key (case-sensitive), typically a GUID.
|
||||
|
||||
The session ID is mapped to an underlying Durable Task entity ID with a `dafx-` prefix (e.g., `dafx-joker`). This naming convention is consistent across both .NET and Python implementations.
|
||||
|
||||
## Architecture
|
||||
|
||||
### .NET
|
||||
|
||||
The .NET implementation consists of two NuGet packages:
|
||||
|
||||
| Package | Purpose |
|
||||
| --- | --- |
|
||||
| `Microsoft.Agents.AI.DurableTask` | Core durable agent types: `DurableAIAgent`, `AgentEntity`, `DurableAgentSession`, `AgentSessionId`, `DurableAgentsOptions`, and the state model. |
|
||||
| `Microsoft.Agents.AI.Hosting.AzureFunctions` | Azure Functions hosting integration: auto-generated HTTP endpoints, MCP tool triggers, entity function triggers, and the `ConfigureDurableAgents` extension method on `FunctionsApplicationBuilder`. |
|
||||
|
||||
Key types:
|
||||
|
||||
- **`DurableAIAgent`** – A subclass of `AIAgent` used *inside orchestrations*. Obtained via `context.GetAgent("agentName")`, it routes `RunAsync` calls through the orchestration's entity APIs so that each call is checkpointed.
|
||||
- **`DurableAIAgentProxy`** – A subclass of `AIAgent` used *outside orchestrations* (e.g., from HTTP triggers or console apps). It signals the entity via `DurableTaskClient` and polls for the response.
|
||||
- **`AgentEntity`** – The `TaskEntity<DurableAgentState>` that hosts the real agent. It loads the registered `AIAgent` by name, wraps it in an `EntityAgentWrapper`, feeds it the full conversation history, and persists the result.
|
||||
- **`DurableAgentSession`** – An `AgentSession` subclass that carries the `AgentSessionId`.
|
||||
- **`DurableAgentsOptions`** – Builder for registering agents and configuring TTL.
|
||||
|
||||
### Python
|
||||
|
||||
The core Python implementation is in the `agent-framework-durabletask` package (`python/packages/durabletask`). Azure Functions hosting (including `AgentFunctionApp`) is in the separate `agent-framework-azurefunctions` package (`python/packages/azurefunctions`).
|
||||
|
||||
Key types:
|
||||
|
||||
- **`DurableAIAgent`** – A generic proxy (`DurableAIAgent[TaskT]`) implementing `SupportsAgentRun`. Returns a `TaskT` from `run()` — either an `AgentResponse` (client context) or a `DurableAgentTask` (orchestration context, must be `yield`ed).
|
||||
- **`DurableAIAgentWorker`** – Wraps a `TaskHubGrpcWorker` and registers agents as durable entities via `add_agent()`.
|
||||
- **`DurableAIAgentClient`** – Wraps a `TaskHubGrpcClient` for external callers. `get_agent()` returns a `DurableAIAgent[AgentResponse]`.
|
||||
- **`DurableAIAgentOrchestrationContext`** – Wraps an `OrchestrationContext` for use inside orchestrations. `get_agent()` returns a `DurableAIAgent[DurableAgentTask]`.
|
||||
- **`AgentEntity`** – Platform-agnostic agent execution logic that manages state, invokes the agent, handles streaming, and calls response callbacks.
|
||||
|
||||
## Hosting models
|
||||
|
||||
### Azure Functions
|
||||
|
||||
The recommended production hosting model. A single call to `ConfigureDurableAgents` (C#) or `AgentFunctionApp` (Python) automatically:
|
||||
|
||||
- Registers agent entities with the Durable Task worker.
|
||||
- Generates HTTP endpoints at `/api/agents/{agentName}/run` for each registered agent.
|
||||
- Supports `thread_id` query parameter / JSON field and the `x-ms-thread-id` response header for session continuity.
|
||||
- Supports fire-and-forget via the `x-ms-wait-for-response: false` header (returns HTTP 202).
|
||||
- Optionally exposes agents as MCP tools.
|
||||
|
||||
**C# example:**
|
||||
|
||||
```csharp
|
||||
using IHost app = FunctionsApplication
|
||||
.CreateBuilder(args)
|
||||
.ConfigureFunctionsWebApplication()
|
||||
.ConfigureDurableAgents(options => options.AddAIAgent(agent))
|
||||
.Build();
|
||||
app.Run();
|
||||
```
|
||||
|
||||
**Python example:**
|
||||
|
||||
```python
|
||||
app = AgentFunctionApp(agents=[agent])
|
||||
```
|
||||
|
||||
### Console apps / generic hosts
|
||||
|
||||
For self-hosted or non-serverless scenarios, register durable agents via `IServiceCollection.ConfigureDurableAgents` (.NET) or `DurableAIAgentWorker` (Python) with explicit Durable Task worker and client configuration.
|
||||
|
||||
**C# example:**
|
||||
|
||||
```csharp
|
||||
IHost host = Host.CreateDefaultBuilder(args)
|
||||
.ConfigureServices(services =>
|
||||
{
|
||||
services.ConfigureDurableAgents(
|
||||
options => options.AddAIAgent(agent),
|
||||
workerBuilder: b => b.UseDurableTaskScheduler(connectionString),
|
||||
clientBuilder: b => b.UseDurableTaskScheduler(connectionString));
|
||||
})
|
||||
.Build();
|
||||
```
|
||||
|
||||
**Python example:**
|
||||
|
||||
```python
|
||||
worker = DurableAIAgentWorker(TaskHubGrpcWorker(host_address="localhost:4001"))
|
||||
worker.add_agent(agent)
|
||||
worker.start()
|
||||
```
|
||||
|
||||
## Deterministic multi-agent orchestrations
|
||||
|
||||
Durable agents can be composed into deterministic, checkpointed workflows using Durable Task orchestrations. The orchestration framework replays orchestrator code on failure, so completed agent calls are not re-executed.
|
||||
|
||||
### Patterns
|
||||
|
||||
| Pattern | Description |
|
||||
| --- | --- |
|
||||
| **Sequential (chaining)** | Call agents one after another, passing outputs forward. |
|
||||
| **Parallel (fan-out/fan-in)** | Run multiple agents concurrently and aggregate results. |
|
||||
| **Conditional** | Branch orchestration logic based on structured agent output. |
|
||||
| **Human-in-the-loop** | Pause for external events (approvals, feedback) with optional timeouts. |
|
||||
|
||||
### Using agents in orchestrations
|
||||
|
||||
Inside an orchestration function, obtain a `DurableAIAgent` via the orchestration context. Each agent gets its own session (created with `CreateSessionAsync` / `create_session`), and you can call the same agent multiple times on the same session to maintain conversation context across sequential invocations.
|
||||
|
||||
**C#:**
|
||||
|
||||
```csharp
|
||||
static async Task<string> WritingOrchestration(TaskOrchestrationContext context)
|
||||
{
|
||||
// Get a durable agent reference — works in any host (console app, Azure Functions, etc.)
|
||||
DurableAIAgent writer = context.GetAgent("WriterAgent");
|
||||
|
||||
// Create a session to maintain conversation context across multiple calls
|
||||
AgentSession session = await writer.CreateSessionAsync();
|
||||
|
||||
// First call: generate an initial draft
|
||||
AgentResponse<TextResponse> draft = await writer.RunAsync<TextResponse>(
|
||||
message: "Write a concise inspirational sentence about learning.",
|
||||
session: session);
|
||||
|
||||
// Second call: refine the draft — the agent sees the full conversation history
|
||||
AgentResponse<TextResponse> refined = await writer.RunAsync<TextResponse>(
|
||||
message: $"Improve this further while keeping it under 25 words: {draft.Result.Text}",
|
||||
session: session);
|
||||
|
||||
return refined.Result.Text;
|
||||
}
|
||||
```
|
||||
|
||||
**Python:**
|
||||
|
||||
```python
|
||||
def writing_orchestration(context, _):
|
||||
agent_ctx = DurableAIAgentOrchestrationContext(context)
|
||||
|
||||
# Get a durable agent reference — works in any host (standalone worker, Azure Functions, etc.)
|
||||
writer = agent_ctx.get_agent("WriterAgent")
|
||||
|
||||
# Create a session to maintain conversation context across multiple calls
|
||||
session = writer.create_session()
|
||||
|
||||
# First call: generate an initial draft
|
||||
draft = yield writer.run(
|
||||
messages="Write a concise inspirational sentence about learning.",
|
||||
session=session,
|
||||
)
|
||||
|
||||
# Second call: refine the draft — the agent sees the full conversation history
|
||||
refined = yield writer.run(
|
||||
messages=f"Improve this further while keeping it under 25 words: {draft.text}",
|
||||
session=session,
|
||||
)
|
||||
|
||||
return refined.text
|
||||
```
|
||||
|
||||
> [!IMPORTANT]
|
||||
> In .NET, `DurableAIAgent.RunAsync<T>` deliberately avoids `ConfigureAwait(false)` because the Durable Task Framework uses a custom synchronization context — all continuations must run on the orchestration thread.
|
||||
|
||||
## Streaming and response callbacks
|
||||
|
||||
Durable agents do not support true end-to-end streaming because entity operations are request/response. However, **reliable streaming** is supported via response callbacks:
|
||||
|
||||
- **`IAgentResponseHandler`** (.NET) or **`AgentResponseCallbackProtocol`** (Python) – Implement this interface to receive streaming updates as the underlying agent generates them (e.g., push tokens to a Redis Stream for client consumption).
|
||||
- The entity still returns the complete `AgentResponse` after the stream is fully consumed.
|
||||
- Clients can reconnect and resume reading from a cursor-based stream (e.g., Redis Streams) without losing messages.
|
||||
|
||||
See the **Reliable Streaming** samples for a complete implementation using Redis Streams.
|
||||
|
||||
## Session TTL (Time-To-Live)
|
||||
|
||||
Durable agent sessions support automatic cleanup via configurable TTL. See [Session TTL](durable-agents-ttl.md) for details on configuration, behavior, and best practices.
|
||||
|
||||
## Observability
|
||||
|
||||
When using the [Durable Task Scheduler](https://learn.microsoft.com/azure/azure-functions/durable/durable-task-scheduler/durable-task-scheduler) as the durable backend, you get built-in observability through its dashboard:
|
||||
|
||||
- **Conversation history** – View complete chat history for each agent session.
|
||||
- **Orchestration visualization** – See multi-agent execution flows, including parallel branches and conditional logic.
|
||||
- **Performance metrics** – Monitor agent response times, token usage, and orchestration duration.
|
||||
- **Debugging** – Trace tool invocations and external event handling.
|
||||
|
||||
## Samples
|
||||
|
||||
- **.NET** – [Console app samples](../../../dotnet/samples/04-hosting/DurableAgents/ConsoleApps/) and [Azure Functions samples](../../../dotnet/samples/04-hosting/DurableAgents/AzureFunctions/) covering single-agent, chaining, concurrency, conditionals, human-in-the-loop, long-running tools, MCP tool exposure, and reliable streaming.
|
||||
- **Python** – [Durable Task samples](../../../python/samples/04-hosting/durabletask/) covering single-agent, multi-agent, streaming, chaining, concurrency, conditionals, and human-in-the-loop.
|
||||
|
||||
## Packages
|
||||
|
||||
| Language | Package | Source |
|
||||
| --- | --- | --- |
|
||||
| .NET | `Microsoft.Agents.AI.DurableTask` | [`dotnet/src/Microsoft.Agents.AI.DurableTask`](../../../dotnet/src/Microsoft.Agents.AI.DurableTask) |
|
||||
| .NET | `Microsoft.Agents.AI.Hosting.AzureFunctions` | [`dotnet/src/Microsoft.Agents.AI.Hosting.AzureFunctions`](../../../dotnet/src/Microsoft.Agents.AI.Hosting.AzureFunctions) |
|
||||
| Python | `agent-framework-durabletask` | [`python/packages/durabletask`](../../../python/packages/durabletask) |
|
||||
| Python | `agent-framework-azurefunctions` | [`python/packages/azurefunctions`](../../../python/packages/azurefunctions) |
|
||||
|
||||
## Further reading
|
||||
|
||||
- [Azure Functions (Durable) — Microsoft Learn](https://learn.microsoft.com/agent-framework/integrations/azure-functions)
|
||||
- [Durable Task Scheduler](https://learn.microsoft.com/azure/azure-functions/durable/durable-task-scheduler/durable-task-scheduler)
|
||||
- [Durable Entities](https://learn.microsoft.com/azure/azure-functions/durable/durable-functions-entities)
|
||||
- [Session TTL](durable-agents-ttl.md)
|
||||
@@ -1,390 +0,0 @@
|
||||
# Vector Stores and Embeddings
|
||||
|
||||
## Overview
|
||||
|
||||
This feature ports the vector store abstractions, embedding generator abstractions, and their implementations from Semantic Kernel into Agent Framework. The ported code follows AF's coding standards, feels native to AF, and is structured to allow data models/schemas to be reusable across both frameworks. The embedding abstraction combines the best of SK's `EmbeddingGeneratorBase` and MEAI's `IEmbeddingGenerator<TInput, TEmbedding>`.
|
||||
|
||||
| Capability | Description |
|
||||
| --- | --- |
|
||||
| Embedding generation | Generic embedding client abstraction supporting text, image, and audio inputs |
|
||||
| Vector store collections | CRUD operations on vector store collections (upsert, get, delete) |
|
||||
| Vector search | Unified search interface with `search_type` parameter (`"vector"`, `"keyword_hybrid"`) |
|
||||
| Data model decorator | `@vectorstoremodel` decorator for defining vector store data models (supports Pydantic, dataclasses, plain classes, dicts) |
|
||||
| Agent tools | `create_search_tool`, `create_upsert_tool`, `create_get_tool`, `create_delete_tool` for agent-usable vector store operations |
|
||||
| In-memory store | Zero-dependency vector store for testing and development |
|
||||
| 13+ connectors | Azure AI Search, Qdrant, Redis, PostgreSQL, MongoDB, Cosmos DB, Pinecone, Chroma, Weaviate, Oracle, SQL Server, FAISS |
|
||||
|
||||
## Key Design Decisions
|
||||
|
||||
### Embedding Abstractions (combining SK + MEAI)
|
||||
- **Both Protocol and Base class** (matching AF's `SupportsChatGetResponse` + `BaseChatClient` pattern):
|
||||
- `SupportsGetEmbeddings` — Protocol for duck-typing
|
||||
- `BaseEmbeddingClient` — ABC base class for implementations (similar to `BaseChatClient`)
|
||||
- **Generic input type** (`EmbeddingInputT`, default `str`) from MEAI — allows image/audio embeddings in the future
|
||||
- **Generic output type** (`EmbeddingT`, default `list[float]`) from MEAI — supports `list[float]`, `list[int]`, `bytes`, etc.
|
||||
- **Generic order**: `[EmbeddingInputT, EmbeddingT, EmbeddingOptionsT]` — options last, matching MEAI's `IEmbeddingGenerator<TInput, TEmbedding>` with options appended
|
||||
- **TypeVar naming convention**: Use `SuffixT` per AF standard (e.g., `EmbeddingInputT`, `EmbeddingT`, `ModelT`, `KeyT`)
|
||||
- `EmbeddingGenerationOptions` TypedDict (inspired by MEAI, matching AF's `ChatOptions` pattern) — `total=False`, includes `dimensions`, `model_id`. No `additional_properties` since each implementation extends with its own fields.
|
||||
- Protocol and base class are generic over input, output, and options: `SupportsGetEmbeddings[EmbeddingInputT, EmbeddingT, OptionsContraT]`, `BaseEmbeddingClient[EmbeddingInputT, EmbeddingT, OptionsCoT]`
|
||||
- **`Embedding[EmbeddingT]` type** in `_types.py` — a lightweight generic class (not Pydantic) with `vector: EmbeddingT`, `model_id: str | None`, `dimensions: int | None` (explicit or computed from vector), `created_at: datetime | None`, `additional_properties: dict[str, Any]`
|
||||
- **`GeneratedEmbeddings[EmbeddingT, EmbeddingOptionsT]` type** — a list-like container of `Embedding[EmbeddingT]` objects with `options: EmbeddingOptionsT | None` (stores the options used to generate), `usage: dict[str, Any] | None`, `additional_properties: dict[str, Any]`
|
||||
- **No numpy dependency** — return `list[float]` by default; users cast as needed
|
||||
|
||||
### Vector Store Abstractions
|
||||
- **Port core abstractions without Pydantic for internal classes** — use plain classes
|
||||
- **Both Protocol and Base class** for vector store operations (matching AF pattern):
|
||||
- `SupportsVectorUpsert` / `SupportsVectorSearch` — Protocols for duck-typing (follows `Supports<Capability>` naming convention)
|
||||
- `BaseVectorCollection` / `BaseVectorSearch` — ABC base classes for implementations
|
||||
- `BaseVectorStore` — ABC base class for store operations (factory for collections, no protocol needed)
|
||||
- **TypeVar naming convention**: `ModelT`, `KeyT`, `FilterT` (suffix T, per AF standard)
|
||||
- **Support Pydantic for user-facing data models** — the `@vectorstoremodel` decorator and `VectorStoreCollectionDefinition` should work with Pydantic models, dataclasses, plain classes, and dicts
|
||||
- **Remove SK-specific dependencies** — no `KernelBaseModel`, `KernelFunction`, `KernelParameterMetadata`, `kernel_function`, `PromptExecutionSettings`
|
||||
- **Embedding types in `_types.py`**, embedding protocol/base class in `_clients.py`
|
||||
- **All vector store specific types, enums, protocols, base classes** in `_vectors.py`
|
||||
- **Error handling** uses AF's exception hierarchy (e.g., `IntegrationException` variants)
|
||||
|
||||
### Package Structure
|
||||
- **Embedding types** (`Embedding`, `GeneratedEmbeddings`, `EmbeddingGenerationOptions`) in `agent_framework/_types.py`
|
||||
- **Embedding protocol + base class** (`SupportsGetEmbeddings`, `BaseEmbeddingClient`) in `agent_framework/_clients.py`
|
||||
- **All vector store specific code** in a new `agent_framework/_vectors.py` module — this includes:
|
||||
- Enums: `FieldTypes`, `IndexKind`, `DistanceFunction`
|
||||
- `VectorStoreField`, `VectorStoreCollectionDefinition`
|
||||
- `SearchOptions`, `SearchResponse`, `RecordFilterOptions`
|
||||
- `@vectorstoremodel` decorator
|
||||
- Serialization/deserialization protocols
|
||||
- `VectorStoreRecordHandler`, `BaseVectorCollection`, `BaseVectorStore`, `BaseVectorSearch`
|
||||
- `SupportsVectorUpsert`, `SupportsVectorSearch` protocols
|
||||
- **OpenAI embeddings** in `agent_framework/openai/` (built into core, like OpenAI chat)
|
||||
- **Azure OpenAI embeddings** in `agent_framework/azure/` (built into core, follows `AzureOpenAIChatClient` pattern)
|
||||
- **Each vector store connector** in its own AF package under `packages/`
|
||||
- **In-memory store** in core (no external deps)
|
||||
- **TextSearch and its implementations** (Brave, Google) — last phase, separate work
|
||||
|
||||
## Naming: SK → AF
|
||||
|
||||
### Names that change
|
||||
|
||||
| SK Name | AF Name | Rationale |
|
||||
|---------|---------|-----------|
|
||||
| `VectorStoreCollection` | `BaseVectorCollection` | Drop redundant `Store`, add `Base` prefix per AF pattern |
|
||||
| `VectorStore` | `BaseVectorStore` | Add `Base` prefix per AF pattern |
|
||||
| `VectorSearch` | `BaseVectorSearch` | Add `Base` prefix per AF pattern |
|
||||
| `VectorSearchOptions` | `SearchOptions` | Shorter — context is already vector search |
|
||||
| `VectorSearchResult` | `SearchResponse` | Align with `ChatResponse`/`AgentResponse` |
|
||||
| `GetFilteredRecordOptions` | `RecordFilterOptions` | Shorter, more natural |
|
||||
| `EmbeddingGeneratorBase` | `BaseEmbeddingClient` | Matches AF `BaseChatClient` pattern |
|
||||
| `VectorStoreCollectionProtocol` | `SupportsVectorUpsert` | AF `Supports*` naming convention |
|
||||
| `VectorSearchProtocol` | `SupportsVectorSearch` | AF `Supports*` naming convention |
|
||||
| `__kernel_vectorstoremodel__` | `__vectorstoremodel__` | Drop SK `kernel` prefix |
|
||||
| `__kernel_vectorstoremodel_definition__` | `__vectorstoremodel_definition__` | Drop SK `kernel` prefix |
|
||||
| `search()` + `hybrid_search()` | `search(search_type=...)` | Single method with `Literal` parameter |
|
||||
| `SearchType` enum | `Literal["vector", "keyword_hybrid"]` | No enum, just a literal |
|
||||
| `KernelSearchResults` | `SearchResults` | Drop SK `Kernel` prefix (plural — container of `SearchResponse` items) |
|
||||
|
||||
### Names that stay the same
|
||||
|
||||
| Name | Location |
|
||||
|------|----------|
|
||||
| `@vectorstoremodel` | `_vectors.py` |
|
||||
| `VectorStoreField` | `_vectors.py` |
|
||||
| `VectorStoreCollectionDefinition` | `_vectors.py` |
|
||||
| `VectorStoreRecordHandler` | `_vectors.py` |
|
||||
| `FieldTypes` | `_vectors.py` |
|
||||
| `IndexKind` | `_vectors.py` |
|
||||
| `DistanceFunction` | `_vectors.py` |
|
||||
| `DISTANCE_FUNCTION_DIRECTION_HELPER` | `_vectors.py` |
|
||||
| `Embedding` | `_types.py` |
|
||||
| `GeneratedEmbeddings` | `_types.py` |
|
||||
| `EmbeddingGenerationOptions` | `_types.py` |
|
||||
| `SupportsGetEmbeddings` | `_clients.py` |
|
||||
|
||||
### New AF-only names (no SK equivalent)
|
||||
|
||||
| Name | Location | Purpose |
|
||||
|------|----------|---------|
|
||||
| `BaseEmbeddingClient` | `_clients.py` | ABC base for embedding implementations |
|
||||
| `EmbeddingInputT` | `_types.py` | TypeVar for generic embedding input (default `str`) |
|
||||
| `EmbeddingTelemetryLayer` | `observability.py` | MRO-based OTel tracing for embeddings |
|
||||
| `SupportsVectorUpsert` | `_vectors.py` | Protocol for collection CRUD |
|
||||
| `SupportsVectorSearch` | `_vectors.py` | Protocol for vector search |
|
||||
| `create_search_tool` | `_vectors.py` | Creates AF `FunctionTool` from vector search |
|
||||
|
||||
## Source Files Reference (SK → AF mapping)
|
||||
|
||||
### SK Source Files
|
||||
| SK File | Lines | Content |
|
||||
|---------|-------|---------|
|
||||
| `data/vector.py` | 2369 | All vector store abstractions, enums, decorator, search |
|
||||
| `data/_shared.py` | 184 | SearchOptions, KernelSearchResults, shared search types |
|
||||
| `data/text_search.py` | 349 | TextSearch base, TextSearchResult |
|
||||
| `connectors/ai/embedding_generator_base.py` | 50 | EmbeddingGeneratorBase ABC |
|
||||
| `connectors/in_memory.py` | 520 | InMemoryCollection, InMemoryStore |
|
||||
| `connectors/azure_ai_search.py` | 793 | Azure AI Search collection + store |
|
||||
| `connectors/azure_cosmos_db.py` | 1104 | Cosmos DB (Mongo + NoSQL) |
|
||||
| `connectors/redis.py` | 845 | Redis (Hashset + JSON) |
|
||||
| `connectors/qdrant.py` | 653 | Qdrant collection + store |
|
||||
| `connectors/postgres.py` | 987 | PostgreSQL collection + store |
|
||||
| `connectors/mongodb.py` | 633 | MongoDB Atlas collection + store |
|
||||
| `connectors/pinecone.py` | 691 | Pinecone collection + store |
|
||||
| `connectors/chroma.py` | 484 | Chroma collection + store |
|
||||
| `connectors/faiss.py` | 278 | FAISS (extends InMemory) |
|
||||
| `connectors/weaviate.py` | 804 | Weaviate collection + store |
|
||||
| `connectors/oracle.py` | 1267 | Oracle collection + store |
|
||||
| `connectors/sql_server.py` | 1132 | SQL Server collection + store |
|
||||
| `connectors/ai/open_ai/services/open_ai_text_embedding.py` | 91 | OpenAI embedding impl |
|
||||
| `connectors/ai/open_ai/services/open_ai_text_embedding_base.py` | 78 | OpenAI embedding base |
|
||||
| `connectors/brave.py` | ~200 | Brave TextSearch impl |
|
||||
| `connectors/google_search.py` | ~200 | Google TextSearch impl |
|
||||
|
||||
---
|
||||
|
||||
## Implementation Phases
|
||||
|
||||
### Phase 1: Core Embedding Abstractions & OpenAI Implementation ✅ DONE
|
||||
**Goal:** Establish the embedding generator abstraction and ship one working implementation.
|
||||
**Mergeable:** Yes — adds new types/protocols, no breaking changes.
|
||||
**Status:** Merged via PR #4153. Closes sub-issue #4163.
|
||||
|
||||
#### 1.1 — Embedding types in `_types.py`
|
||||
- `EmbeddingInputT` TypeVar (default `str`) — generic input type for embedding generation
|
||||
- `EmbeddingT` TypeVar (default `list[float]`) — generic output embedding vector type
|
||||
- `Embedding[EmbeddingT]` generic class: `vector: EmbeddingT`, `model_id: str | None`, `dimensions: int | None` (explicit param or computed from vector length), `created_at: datetime | None`, `additional_properties: dict[str, Any]`
|
||||
- `GeneratedEmbeddings[EmbeddingT, EmbeddingOptionsT]` generic class: list-like container of `Embedding[EmbeddingT]` objects with `options: EmbeddingOptionsT | None` (the options used to generate), `usage: dict[str, Any] | None`, `additional_properties: dict[str, Any]`
|
||||
- `EmbeddingGenerationOptions` TypedDict (`total=False`): `dimensions: int`, `model_id: str` — follows the same pattern as `ChatOptions`. No `additional_properties` needed since it's a TypedDict and each implementation can extend with its own fields.
|
||||
|
||||
#### 1.2 — Embedding generator protocol + base class in `_clients.py`
|
||||
- `SupportsGetEmbeddings(Protocol[EmbeddingInputT, EmbeddingT, OptionsContraT])`: generic over input, output, and options (all with defaults), `get_embeddings(values: Sequence[EmbeddingInputT], *, options: OptionsContraT | None = None) -> Awaitable[GeneratedEmbeddings[EmbeddingT]]`
|
||||
- `BaseEmbeddingClient(ABC, Generic[EmbeddingInputT, EmbeddingT, OptionsCoT])`: ABC base class mirroring `BaseChatClient` pattern
|
||||
- `__init__` with `additional_properties`, etc.
|
||||
- Abstract `get_embeddings(...)` for subclasses to implement directly (no `_inner_*` indirection — simpler than chat, no middleware needed)
|
||||
- `EmbeddingTelemetryLayer` in `observability.py` — MRO-based telemetry (no closure), `gen_ai.operation.name = "embeddings"`
|
||||
|
||||
#### 1.3 — OpenAI embedding generator in `agent_framework/openai/` and `agent_framework/azure/`
|
||||
- `RawOpenAIEmbeddingClient` — implements `get_embeddings` via `_ensure_client()` factory
|
||||
- `OpenAIEmbeddingClient(OpenAIConfigMixin, EmbeddingTelemetryLayer[str, list[float], OptionsT], RawOpenAIEmbeddingClient[OptionsT])` — full client with config + telemetry layers
|
||||
- `OpenAIEmbeddingOptions(EmbeddingGenerationOptions)` — extends with `encoding_format`, `user`
|
||||
- `AzureOpenAIEmbeddingClient` in `agent_framework/azure/` — follows `AzureOpenAIChatClient` pattern with `AzureOpenAIConfigMixin`, `load_settings`, Entra ID credential support
|
||||
- `AzureOpenAISettings` extended with `embedding_deployment_name` (env var: `AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME`)
|
||||
|
||||
#### 1.4 — Tests and samples
|
||||
- Unit tests for types, protocol, base class, OpenAI client, Azure OpenAI client
|
||||
- Integration tests for OpenAI and Azure OpenAI (gated behind credentials check, `@pytest.mark.flaky`)
|
||||
- Samples in `samples/02-agents/embeddings/` — `openai_embeddings.py`, `azure_openai_embeddings.py`
|
||||
|
||||
---
|
||||
|
||||
### Phase 2: Embedding Generators for Existing Providers
|
||||
**Goal:** Add embedding generators to all existing AF provider packages that have chat clients.
|
||||
**Mergeable:** Yes — each is independent, added to existing provider packages.
|
||||
|
||||
#### 2.1 — Azure AI Inference embedding (in `packages/azure-ai/`)
|
||||
#### 2.2 — Ollama embedding (in `packages/ollama/`)
|
||||
#### 2.3 — Anthropic embedding (in `packages/anthropic/`)
|
||||
#### 2.4 — Bedrock embedding (in `packages/bedrock/`)
|
||||
|
||||
---
|
||||
|
||||
### Phase 3: Core Vector Store Abstractions
|
||||
**Goal:** Establish all vector store types, enums, the decorator, collection definition, and base classes.
|
||||
**Mergeable:** Yes — adds new abstractions, no breaking changes.
|
||||
|
||||
#### 3.1 — Vector store enums and field types in `_vectors.py`
|
||||
- `FieldTypes` enum: `KEY`, `VECTOR`, `DATA`
|
||||
- `IndexKind` enum: `HNSW`, `FLAT`, `IVF_FLAT`, `DISK_ANN`, `QUANTIZED_FLAT`, `DYNAMIC`, `DEFAULT`
|
||||
- `DistanceFunction` enum: `COSINE_SIMILARITY`, `COSINE_DISTANCE`, `DOT_PROD`, `EUCLIDEAN_DISTANCE`, `EUCLIDEAN_SQUARED_DISTANCE`, `MANHATTAN`, `HAMMING`, `DEFAULT`
|
||||
- No `SearchType` enum — use `Literal["vector", "keyword_hybrid"]` instead, per AF convention of avoiding unnecessary imports
|
||||
- `VectorStoreField` plain class (not Pydantic)
|
||||
- `VectorStoreCollectionDefinition` class (not Pydantic internally, but supports Pydantic models as input)
|
||||
- `SearchOptions` plain class — includes `score_threshold: float | None` for filtering results by score (see note below)
|
||||
- `SearchResponse` generic class
|
||||
- `RecordFilterOptions` plain class
|
||||
- `DISTANCE_FUNCTION_DIRECTION_HELPER` dict
|
||||
|
||||
#### 3.2 — `@vectorstoremodel` decorator
|
||||
- Port from SK, works with dataclasses, Pydantic models, plain classes, and dicts
|
||||
- Sets `__vectorstoremodel__` and `__vectorstoremodel_definition__` on the class
|
||||
- Remove SK-specific `kernel` prefix (`__kernel_vectorstoremodel__` → `__vectorstoremodel__`)
|
||||
|
||||
#### 3.3 — Serialization/deserialization protocols
|
||||
- `SerializeMethodProtocol`, `ToDictFunctionProtocol`, `FromDictFunctionProtocol`, etc.
|
||||
- Port the record handler logic but without Pydantic base class — use plain class or ABC
|
||||
|
||||
#### 3.4 — Vector store base classes in `_vectors.py`
|
||||
- `VectorStoreRecordHandler` — internal base class that handles serialization/deserialization between user data models and store-specific formats, plus embedding generation for vector fields. Both `BaseVectorCollection` and `BaseVectorSearch` extend this.
|
||||
- `BaseVectorCollection(VectorStoreRecordHandler)` — base for collections
|
||||
- Uses `SupportsGetEmbeddings` instead of `EmbeddingGeneratorBase`
|
||||
- Not a Pydantic model — use `__init__` with explicit params
|
||||
- `upsert`, `get`, `delete`, `ensure_collection_exists`, `collection_exists`, `ensure_collection_deleted`
|
||||
- Async context manager support
|
||||
- `BaseVectorStore` — base for stores
|
||||
- `get_collection`, `list_collection_names`, `collection_exists`, `ensure_collection_deleted`
|
||||
- Async context manager support
|
||||
|
||||
#### 3.5 — Vector search base class
|
||||
- `BaseVectorSearch(VectorStoreRecordHandler)` — base for vector search
|
||||
- Single `search(search_type=...)` method with `search_type: Literal["vector", "keyword_hybrid"]` parameter — no enum, just a literal
|
||||
- `_inner_search` abstract method for implementations
|
||||
- Filter building with lambda parser (AST-based)
|
||||
- Vector generation from values using embedding generator
|
||||
|
||||
#### 3.6 — Protocols for type checking
|
||||
- `SupportsVectorUpsert` — Protocol for upsert/get/delete operations
|
||||
- `SupportsVectorSearch` — Protocol for vector search (single `search()` with `search_type` parameter)
|
||||
- No separate `SupportsVectorHybridSearch` — search type is a parameter, not a separate capability
|
||||
- No protocol for `VectorStore` — it's a factory for collections, not a capability to duck-type against
|
||||
|
||||
#### 3.7 — Exception types
|
||||
- Add vector store exceptions under `IntegrationException` or create new branch
|
||||
- `VectorStoreException`, `VectorStoreOperationException`, `VectorSearchException`, `VectorStoreModelException`, etc.
|
||||
|
||||
#### 3.8 — `create_search_tool` on `BaseVectorSearch`
|
||||
- Method on `BaseVectorSearch` that creates an AF `FunctionTool` from the vector search
|
||||
- Wraps the single `search()` method, passing `search_type` parameter
|
||||
- Accepts: `name`, `description`, `search_type`, `top`, `skip`, `filter`, `string_mapper`
|
||||
- The tool takes a query string, vectorizes it, searches, and returns results as strings
|
||||
- Can also be a standalone factory function in `_vectors.py`
|
||||
|
||||
#### 3.9 — Tests for all vector store abstractions
|
||||
- Unit tests for enums, field types, collection definition
|
||||
- Unit tests for decorator
|
||||
- Unit tests for serialization/deserialization
|
||||
- Unit tests for record handler
|
||||
|
||||
---
|
||||
|
||||
### Phase 4: In-Memory Vector Store
|
||||
**Goal:** Provide a zero-dependency vector store for testing and development.
|
||||
**Mergeable:** Yes — first usable vector store.
|
||||
|
||||
#### 4.1 — Port `InMemoryCollection` and `InMemoryStore` into core
|
||||
- Place in `agent_framework/_vectors.py` (alongside the abstractions)
|
||||
- Supports vector search (cosine similarity, etc.)
|
||||
- No external dependencies
|
||||
|
||||
#### 4.2 — Port FAISS extension (optional, can be separate package)
|
||||
- Extends InMemory with FAISS indexing
|
||||
|
||||
#### 4.3 — Tests and sample code
|
||||
|
||||
---
|
||||
|
||||
### Phase 5: Vector Store Connectors — Tier 1 (High Priority)
|
||||
**Goal:** Ship the most commonly used vector store connectors.
|
||||
**Mergeable:** Yes — each connector is independent.
|
||||
|
||||
Each connector follows the AF package structure:
|
||||
- New package under `packages/`
|
||||
- Own `pyproject.toml`, `tests/`, lazy loading in core
|
||||
|
||||
#### 5.1 — Azure AI Search (`packages/azure-ai-search/`)
|
||||
- May extend existing package or be new
|
||||
- `AzureAISearchCollection`, `AzureAISearchStore`
|
||||
|
||||
#### 5.2 — Qdrant (`packages/qdrant/`)
|
||||
- New package
|
||||
- `QdrantCollection`, `QdrantStore`
|
||||
|
||||
#### 5.3 — Redis (`packages/redis/`)
|
||||
- May extend existing redis package
|
||||
- `RedisCollection` (JSON + Hashset variants), `RedisStore`
|
||||
|
||||
#### 5.4 — PostgreSQL/pgvector (`packages/postgres/`)
|
||||
- New package
|
||||
- `PostgresCollection`, `PostgresStore`
|
||||
|
||||
---
|
||||
|
||||
### Phase 6: Vector Store Connectors — Tier 2
|
||||
**Goal:** Ship remaining vector store connectors.
|
||||
**Mergeable:** Yes — each connector is independent.
|
||||
|
||||
#### 6.1 — MongoDB Atlas (`packages/mongodb/`)
|
||||
#### 6.2 — Azure Cosmos DB (`packages/azure-cosmos-db/`)
|
||||
- Cosmos Mongo + Cosmos NoSQL
|
||||
#### 6.3 — Pinecone (`packages/pinecone/`)
|
||||
#### 6.4 — Chroma (`packages/chroma/`)
|
||||
#### 6.5 — Weaviate (`packages/weaviate/`)
|
||||
|
||||
---
|
||||
|
||||
### Phase 7: Vector Store Connectors — Tier 3
|
||||
**Goal:** Ship niche or less common connectors.
|
||||
**Mergeable:** Yes — each connector is independent.
|
||||
|
||||
#### 7.1 — Oracle (`packages/oracle/`)
|
||||
#### 7.2 — SQL Server (`packages/sql-server/`)
|
||||
#### 7.3 — FAISS (`packages/faiss/` or in core extending InMemory)
|
||||
|
||||
> **Note:** When implementing any SQL-based connector (PostgreSQL, SQL Server, SQLite, Cosmos DB), review the .NET MEVD changes made by @roji (Shay Rojansky) in SK for design patterns, query building, filter translation, and feature parity: https://github.com/microsoft/semantic-kernel/pulls?q=is%3Apr+author%3Aroji+is%3Aclosed
|
||||
|
||||
---
|
||||
|
||||
### Phase 8: Vector Store CRUD Tools
|
||||
**Goal:** Provide a full set of agent-usable tools for CRUD operations on vector store collections.
|
||||
**Mergeable:** Yes — adds tools without changing existing APIs.
|
||||
|
||||
#### 8.1 — `create_upsert_tool` — tool for upserting records into a collection
|
||||
#### 8.2 — `create_get_tool` — tool for retrieving records by key
|
||||
- Key-based lookup only (by primary key), not a search tool
|
||||
- Documentation must clearly distinguish this from `create_search_tool`: get_tool retrieves specific records by their known key, while search_tool performs similarity/filtered search across the collection
|
||||
- Consider if this overlaps with filtered search and document when to use which
|
||||
#### 8.3 — `create_delete_tool` — tool for deleting records by key
|
||||
#### 8.4 — Tests and samples for CRUD tools
|
||||
|
||||
---
|
||||
|
||||
### Phase 9: Additional Embedding Implementations (New Providers)
|
||||
**Goal:** Provide embedding generators for providers that don't yet have AF packages.
|
||||
**Mergeable:** Yes — each is independent, new packages.
|
||||
|
||||
#### 9.1 — HuggingFace/ONNX embedding (new package or lab)
|
||||
#### 9.2 — Mistral AI embedding (new package)
|
||||
#### 9.3 — Google AI / Vertex AI embedding (new package)
|
||||
#### 9.4 — Nvidia embedding (new package)
|
||||
|
||||
---
|
||||
|
||||
### Phase 10: TextSearch Abstractions & Implementations (Separate Work)
|
||||
**Goal:** Port text search (non-vector) abstractions and implementations.
|
||||
**Mergeable:** Yes — independent of vector stores.
|
||||
|
||||
#### 10.1 — TextSearch base class and types
|
||||
- `SearchOptions`, `SearchResponse`, `TextSearchResult`
|
||||
- `TextSearch` base class with `search()` method
|
||||
- `create_search_function()` for kernel integration (may need AF equivalent)
|
||||
|
||||
#### 10.2 — Brave Search implementation
|
||||
#### 10.3 — Google Search implementation
|
||||
#### 10.4 — Vector store text search bridge (connecting VectorSearch to TextSearch interface)
|
||||
|
||||
---
|
||||
|
||||
## Key Considerations
|
||||
|
||||
1. **No Pydantic for internal classes**: All AF internal classes should use plain classes. Pydantic is only used for user-facing input validation (e.g., vector store data models).
|
||||
|
||||
2. **Protocol + Base class**: Follow AF's pattern of both a `Protocol` for duck-typing and a `Base` ABC for implementation, matching how `SupportsChatGetResponse` + `BaseChatClient` works.
|
||||
|
||||
3. **Exception hierarchy**: Use AF's `IntegrationException` branch for vector store operations, since vector stores are external dependencies.
|
||||
|
||||
4. **`from __future__ import annotations`**: Required in all files per AF coding standard.
|
||||
|
||||
5. **No `**kwargs` escape hatches in public APIs**: For user-facing interfaces, use explicit named parameters per AF coding standard. Internal implementation details (e.g., cooperative multiple inheritance / MRO patterns) may use `**kwargs` where necessary, as long as they are not exposed in public signatures.
|
||||
|
||||
6. **Lazy loading**: Connector packages use `__getattr__` lazy loading in core provider folders.
|
||||
|
||||
7. **Reusable data models**: The `@vectorstoremodel` decorator and `VectorStoreCollectionDefinition` should be agnostic enough to work with both SK and AF. The core types (`FieldTypes`, `IndexKind`, `DistanceFunction`, `VectorStoreField`) should be identical or easily mapped.
|
||||
|
||||
8. **`create_search_tool`**: The AF-native equivalent of SK's `create_search_function`. Instead of creating a `KernelFunction`, this creates an AF `FunctionTool` (via the `@tool` decorator pattern) from a vector search. This allows agents to use vector search as a tool during conversations. Design:
|
||||
- `create_search_tool(name, description, search_type, ...)` → returns a `FunctionTool` that wraps `VectorSearch.search(search_type=...)`
|
||||
- The tool accepts a query string, performs embedding + vector search, and returns results as strings
|
||||
- Supports configurable string mappers, filter functions, top/skip defaults
|
||||
- Lives in `_vectors.py` as a method on `BaseVectorSearch` and/or as a standalone factory function
|
||||
|
||||
9. **CRUD tools**: A full set of create/read/update/delete tools for vector store collections, allowing agents to manage data in vector stores. Design:
|
||||
- `create_upsert_tool(...)` → tool for upserting records
|
||||
- `create_get_tool(...)` → tool for retrieving records by key
|
||||
- `create_delete_tool(...)` → tool for deleting records
|
||||
- These are separate from search and are placed in a later phase
|
||||
|
||||
10. **Score threshold filtering**: `SearchOptions` includes `score_threshold: float | None` to filter search results by relevance score (ref: [SK .NET PR #13501](https://github.com/microsoft/semantic-kernel/pull/13501)). The semantics depend on the distance function: for similarity functions (cosine similarity, dot product), results *below* the threshold are filtered out; for distance functions (cosine distance, euclidean), results *above* the threshold are filtered out. Use `DISTANCE_FUNCTION_DIRECTION_HELPER` to determine direction. Connectors should implement this natively where the database supports it, falling back to client-side post-filtering otherwise.
|
||||
-130
@@ -1,130 +0,0 @@
|
||||
---
|
||||
name: build-and-test
|
||||
description: How to build and test .NET projects in the Agent Framework repository. Use this when verifying or testing changes.
|
||||
---
|
||||
|
||||
- Only **UnitTest** projects need to be run locally; IntegrationTests require external dependencies.
|
||||
- See `../project-structure/SKILL.md` for project structure details.
|
||||
|
||||
## Build, Test, and Lint Commands
|
||||
|
||||
```bash
|
||||
# From dotnet/ directory
|
||||
dotnet restore --tl:off # Restore dependencies for all projects
|
||||
dotnet build --tl:off # Build all projects
|
||||
dotnet test # Run all tests
|
||||
dotnet format # Auto-fix formatting for all projects
|
||||
|
||||
# Build/test/format a specific project (preferred for isolated/internal changes)
|
||||
dotnet build src/Microsoft.Agents.AI.<Package> --tl:off
|
||||
dotnet test --project tests/Microsoft.Agents.AI.<Package>.UnitTests
|
||||
dotnet format src/Microsoft.Agents.AI.<Package>
|
||||
|
||||
# Run a single test
|
||||
# Replace the filter values with the appropriate assembly, namespace, class, and method names for the test you want to run and use * as a wildcard elsewhere, e.g. "/*/*/HttpClientTests/GetAsync_ReturnsSuccessStatusCode"
|
||||
# Use `--ignore-exit-code 8` to avoid failing the build when no tests are found for some projects
|
||||
dotnet test --filter-query "/<assemblyFilter>/<namespaceFilter>/<classFilter>/<methodFilter>" --ignore-exit-code 8
|
||||
|
||||
# Run unit tests only
|
||||
# Use `--ignore-exit-code 8` to avoid failing the build when no tests are found for integration test projects
|
||||
dotnet test --filter-query "/*UnitTests*/*/*/*" --ignore-exit-code 8
|
||||
```
|
||||
|
||||
Use `--tl:off` when building to avoid flickering when running commands in the agent.
|
||||
|
||||
## Speeding Up Builds and Testing
|
||||
|
||||
The full solution is large. Use these shortcuts:
|
||||
|
||||
| Change type | What to do |
|
||||
|-------------|------------|
|
||||
| Isolated/Internal logic | Build only the affected project and its `*.UnitTests` project. Fix issues, then build the full solution and run all unit tests. |
|
||||
| Public API surface | Build the full solution and run all unit tests immediately. |
|
||||
|
||||
Example: Building a single code project for all target frameworks
|
||||
|
||||
```bash
|
||||
# From dotnet/ directory
|
||||
dotnet build ./src/Microsoft.Agents.AI.Abstractions
|
||||
```
|
||||
|
||||
Example: Building a single code project for just .NET 10.
|
||||
|
||||
```bash
|
||||
# From dotnet/ directory
|
||||
dotnet build ./src/Microsoft.Agents.AI.Abstractions -f net10.0
|
||||
```
|
||||
|
||||
Example: Running tests for a single project using .NET 10.
|
||||
|
||||
```bash
|
||||
# From dotnet/ directory
|
||||
dotnet test --project ./tests/Microsoft.Agents.AI.Abstractions.UnitTests -f net10.0
|
||||
```
|
||||
|
||||
Example: Running a single test in a specific project using .NET 10.
|
||||
Provide the full namespace, class name, and method name for the test you want to run:
|
||||
|
||||
```bash
|
||||
# From dotnet/ directory
|
||||
dotnet test --project ./tests/Microsoft.Agents.AI.Abstractions.UnitTests -f net10.0 --filter-query "/*/Microsoft.Agents.AI.Abstractions.UnitTests/AgentRunOptionsTests/CloningConstructorCopiesProperties"
|
||||
```
|
||||
|
||||
### Multi-target framework tip
|
||||
|
||||
Most projects target multiple .NET frameworks. If the affected code does **not** use `#if` directives for framework-specific logic, pass `-f net10.0` to speed up building and testing.
|
||||
|
||||
### Package Restore tip
|
||||
|
||||
`dotnet build` will try and restore packages for all projects on each build, which can be slow.
|
||||
Unless packages have been changed, or it's the first time building the solution, add `--no-restore` to the build command to skip this step and speed up builds.
|
||||
|
||||
Just remember to run `dotnet restore` after pulling changes, making changes to project references, or when building for the first time.
|
||||
|
||||
### Testing on Linux tip
|
||||
|
||||
Unit tests target both .NET Framework as well as .NET Core. When running on Linux, only the .NET Core tests can be run, as .NET Framework is not supported on Linux.
|
||||
|
||||
To run only the .NET Core tests, use the `-f net10.0` option with `dotnet test`.
|
||||
|
||||
### Microsoft Testing Platform (MTP)
|
||||
|
||||
Tests use the [Microsoft Testing Platform](https://learn.microsoft.com/dotnet/core/testing/unit-testing-platform-intro) via xUnit v3. Key differences from the legacy VSTest runner:
|
||||
|
||||
- **`dotnet test` requires `--project`** to specify a test project directly (positional arguments are no longer supported).
|
||||
- **Test output** uses the MTP format (e.g., `[✓112/x0/↓0]` progress and `Test run summary: Passed!`).
|
||||
- **TRX reports** use `--report-xunit-trx` instead of `--logger trx`.
|
||||
- **Code coverage** uses `Microsoft.Testing.Extensions.CodeCoverage` with `--coverage --coverage-output-format cobertura`.
|
||||
- **Running a test project directly** is supported via `dotnet run --project <test-project>`. This bypasses the `dotnet test` infrastructure and runs the test executable directly with the MTP command line.
|
||||
|
||||
- **Running tests across the solution** with a filter may cause some projects to match zero tests, which MTP treats as a failure (exit code 8). Use `--ignore-exit-code 8` to suppress this:
|
||||
|
||||
```bash
|
||||
# Run all unit tests across the solution, ignoring projects with no matching tests
|
||||
dotnet test --solution ./agent-framework-dotnet.slnx --no-build -f net10.0 --ignore-exit-code 8
|
||||
```
|
||||
|
||||
- **Running tests with `--solution` for a specific TFM** requires all projects in the solution to support that TFM. Not all projects target every framework (e.g., some are `net10.0`-only). Use `./dotnet/eng/scripts/New-FilteredSolution.ps1` to generate a filtered solution:
|
||||
|
||||
```powershell
|
||||
# Generate a filtered solution for net472 and run tests
|
||||
$filtered = ./dotnet/eng/scripts/New-FilteredSolution.ps1 -Solution dotnet/agent-framework-dotnet.slnx -TargetFramework net472
|
||||
dotnet test --solution $filtered --no-build -f net472 --ignore-exit-code 8
|
||||
|
||||
# Exclude samples and keep only unit test projects
|
||||
./dotnet/eng/scripts/New-FilteredSolution.ps1 -Solution dotnet/agent-framework-dotnet.slnx -TargetFramework net10.0 -ExcludeSamples -TestProjectNameFilter "*UnitTests*" -OutputPath dotnet/filtered-unit.slnx
|
||||
```
|
||||
|
||||
```bash
|
||||
# Run tests via dotnet test (uses MTP under the hood)
|
||||
dotnet test --project ./tests/Microsoft.Agents.AI.UnitTests -f net10.0
|
||||
|
||||
# Run tests with code coverage (Cobertura format)
|
||||
dotnet test --project ./tests/Microsoft.Agents.AI.UnitTests -f net10.0 --coverage --coverage-output-format cobertura --coverage-settings ./tests/coverage.runsettings
|
||||
|
||||
# Run tests directly via dotnet run (MTP native command line)
|
||||
dotnet run --project ./tests/Microsoft.Agents.AI.UnitTests -f net10.0
|
||||
|
||||
# Show MTP command line help
|
||||
dotnet run --project ./tests/Microsoft.Agents.AI.UnitTests -f net10.0 -- -?
|
||||
```
|
||||
@@ -1,31 +0,0 @@
|
||||
---
|
||||
name: project-structure
|
||||
description: Explains the project structure of the agent-framework .NET solution
|
||||
---
|
||||
|
||||
# Agent Framework .NET Project Structure
|
||||
|
||||
```
|
||||
dotnet/
|
||||
├── src/
|
||||
│ ├── Microsoft.Agents.AI/ # Core AI agent implementations
|
||||
│ ├── Microsoft.Agents.AI.Abstractions/ # Core AI agent abstractions
|
||||
│ ├── Microsoft.Agents.AI.A2A/ # Agent-to-Agent (A2A) provider
|
||||
│ ├── Microsoft.Agents.AI.OpenAI/ # OpenAI provider
|
||||
│ ├── Microsoft.Agents.AI.AzureAI/ # Azure AI Foundry Agents (v2) provider
|
||||
│ ├── Microsoft.Agents.AI.AzureAI.Persistent/ # Legacy Azure AI Foundry Agents (v1) provider
|
||||
│ ├── Microsoft.Agents.AI.Anthropic/ # Anthropic provider
|
||||
│ ├── Microsoft.Agents.AI.Workflows/ # Workflow orchestration
|
||||
│ └── ... # Other packages
|
||||
├── samples/ # Sample applications
|
||||
└── tests/ # Unit and integration tests
|
||||
```
|
||||
|
||||
## Main Folders
|
||||
|
||||
| Folder | Contents |
|
||||
|--------|----------|
|
||||
| `src/` | Source code projects |
|
||||
| `tests/` | Test projects — named `<Source-Code-Project>.UnitTests` or `<Source-Code-Project>.IntegrationTests` |
|
||||
| `samples/` | Sample projects |
|
||||
| `src/Shared`, `src/LegacySupport` | Shared code files included by multiple source code projects (see README.md files in these folders or their subdirectories for instructions on how to include them in a project) |
|
||||
@@ -1,82 +0,0 @@
|
||||
---
|
||||
name: verify-dotnet-samples
|
||||
description: How to build, run and verify the .NET sample projects in the Agent Framework repository. Use this when a user wants to verify that the samples still function as expected.
|
||||
---
|
||||
|
||||
# Verifying .NET Sample Projects
|
||||
|
||||
## Sample Pre-requisites
|
||||
|
||||
We should only support verifying samples that:
|
||||
1. Use environment variables for configuration.
|
||||
2. Have no complex setup requirements, e.g., where multiple applications need to be run together, or where we need to launch a browser, etc.
|
||||
|
||||
Always report to the user which samples were run and which were not, and why.
|
||||
|
||||
## Verifying a sample
|
||||
|
||||
Samples should be verified to ensure that they actually work as intended and that their output matches what is expected.
|
||||
For each sample that is run, output should be produced that shows the result and explains the reasoning about what output
|
||||
was expected, what was produced, and why it didn't match what the sample was expected to produce.
|
||||
|
||||
Steps to verify a sample:
|
||||
1. Read the code for the sample
|
||||
1. Check what environment variables are required for the sample
|
||||
1. Check if each environment variable has been set
|
||||
1. If there are any missing, give the user a list of missing environment variables to set and terminate
|
||||
1. Summarize what the expected output of the sample should be
|
||||
1. Run the sample
|
||||
1. Show the user any output from the sample run as it gets produced, so that they can see the run progress
|
||||
1. Check the output of the run against expectations
|
||||
1. After running all requested samples, produce output for each sample that was verified:
|
||||
1. If expectations were matched, output the following:
|
||||
```text
|
||||
[Sample Name] Succeeded
|
||||
```
|
||||
1. If expectations were not matched, output the following:
|
||||
```text
|
||||
[Sample Name] Failed
|
||||
Actual Output:
|
||||
[What the sample produced]
|
||||
Expected Output:
|
||||
[Explanation of what was expected and why the actual output didn't match expectations]
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
Most samples use environment variables to configure settings.
|
||||
|
||||
```csharp
|
||||
var endpoint = Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT") ?? throw new InvalidOperationException("AZURE_OPENAI_ENDPOINT is not set.");
|
||||
var deploymentName = Environment.GetEnvironmentVariable("AZURE_OPENAI_DEPLOYMENT_NAME") ?? "gpt-4o-mini";
|
||||
```
|
||||
|
||||
To run a sample, the environment variables should be set first.
|
||||
Before running a sample, check whether each environment variable in the sample has a value and
|
||||
then give the user a list of environment variables to set.
|
||||
|
||||
You can provide the user some examples of how to set the variables like this:
|
||||
|
||||
```bash
|
||||
export AZURE_OPENAI_ENDPOINT="https://my-openai-instance.openai.azure.com/"
|
||||
export AZURE_OPENAI_DEPLOYMENT_NAME="gpt-4o-mini"
|
||||
```
|
||||
|
||||
To check if a variable has a value use e.g.:
|
||||
|
||||
```bash
|
||||
echo $AZURE_OPENAI_ENDPOINT
|
||||
```
|
||||
|
||||
## How to Run a Sample (General Pattern)
|
||||
|
||||
```bash
|
||||
cd dotnet/samples/<category>/<sample-dir>
|
||||
dotnet run
|
||||
```
|
||||
|
||||
For multi-targeted projects (e.g., Durable console apps), specify the framework:
|
||||
|
||||
```bash
|
||||
dotnet run --framework net10.0
|
||||
```
|
||||
Vendored
+1
-2
@@ -1,6 +1,5 @@
|
||||
{
|
||||
"dotnet.defaultSolution": "agent-framework-dotnet.slnx",
|
||||
"git.openRepositoryInParentFolders": "always",
|
||||
"chat.agent.enabled": true,
|
||||
"dotnet.automaticallySyncWithActiveItem": true
|
||||
"chat.agent.enabled": true
|
||||
}
|
||||
|
||||
@@ -1,66 +0,0 @@
|
||||
# AGENTS.md
|
||||
|
||||
Instructions for AI coding agents working in the .NET codebase.
|
||||
|
||||
## Build, Test, and Lint Commands
|
||||
|
||||
See `./.github/skills/build-and-test/SKILL.md` for detailed instructions on building, testing, and linting projects.
|
||||
|
||||
## Project Structure
|
||||
|
||||
See `./.github/skills/project-structure/SKILL.md` for an overview of the project structure.
|
||||
|
||||
### Core types
|
||||
|
||||
- `AIAgent`: The abstract base class that all agents derive from, providing common methods for interacting with an agent.
|
||||
- `AgentSession`: The abstract base class that all agent sessions derive from, representing a conversation with an agent.
|
||||
- `ChatClientAgent`: An `AIAgent` implementation that uses an `IChatClient` to send messages to an AI provider and receive responses.
|
||||
- `IChatClient`: Interface for sending messages to an AI provider and receiving responses. Used by `ChatClientAgent` and implemented by provider-specific packages.
|
||||
- `FunctionInvokingChatClient`: Decorator for `IChatClient` that adds function invocation capabilities.
|
||||
- `AITool`: Represents a tool that an agent/AI provider can use, with metadata and an execution delegate.
|
||||
- `AIFunction`: A specific type of `AITool` that represents a local function the agent/AI provider can call, with parameters and return types defined.
|
||||
- `ChatMessage`: Represents a message in a conversation.
|
||||
- `AIContent`: Represents content in a message, which can be text, a function call, tool output and more.
|
||||
|
||||
### External Dependencies
|
||||
|
||||
The framework integrates with `Microsoft.Extensions.AI` and `Microsoft.Extensions.AI.Abstractions` (external NuGet packages)
|
||||
using types like `IChatClient`, `FunctionInvokingChatClient`, `AITool`, `AIFunction`, `ChatMessage`, and `AIContent`.
|
||||
|
||||
## Key Conventions
|
||||
|
||||
- **Encoding**: All new files must be saved with UTF-8 encoding with BOM (Byte Order Mark). This is required for `dotnet format` to work correctly.
|
||||
- **Copyright header**: `// Copyright (c) Microsoft. All rights reserved.` at top of all `.cs` files
|
||||
- **XML docs**: Required for all public methods and classes
|
||||
- **Async**: Use `Async` suffix for methods returning `Task`/`ValueTask`
|
||||
- **Private classes**: Should be `sealed` unless subclassed
|
||||
- **Config**: Read from environment variables with `UPPER_SNAKE_CASE` naming
|
||||
- **Tests**: Add Arrange/Act/Assert comments; use Moq for mocking
|
||||
|
||||
## Key Design Principles
|
||||
|
||||
When developing or reviewing code, verify adherence to these key design principles:
|
||||
|
||||
- **DRY**: Avoid code duplication by moving common logic into helper methods or helper classes.
|
||||
- **Single Responsibility**: Each class should have one clear responsibility.
|
||||
- **Encapsulation**: Keep implementation details private and expose only necessary public APIs.
|
||||
- **Strong Typing**: Use strong typing to ensure that code is self-documenting and to catch errors at compile time.
|
||||
|
||||
## Sample Structure
|
||||
|
||||
Samples (in `./samples/` folder) should follow this structure:
|
||||
|
||||
1. Copyright header: `// Copyright (c) Microsoft. All rights reserved.`
|
||||
2. Description comment explaining what the sample demonstrates
|
||||
3. Using statements
|
||||
4. Main code logic
|
||||
5. Helper methods at bottom
|
||||
|
||||
Configuration via environment variables (never hardcode secrets). Keep samples simple and focused.
|
||||
|
||||
When adding a new sample:
|
||||
|
||||
- Create a standalone project in `samples/` with matching directory and project names
|
||||
- Include a README.md explaining what the sample does and how to run it
|
||||
- Add the project to the solution file
|
||||
- Reference the sample in the parent directory's README.md
|
||||
@@ -11,18 +11,19 @@
|
||||
</PropertyGroup>
|
||||
<ItemGroup>
|
||||
<!-- Aspire.* -->
|
||||
<PackageVersion Include="Anthropic" Version="12.8.0" />
|
||||
<PackageVersion Include="Anthropic.Foundry" Version="0.4.2" />
|
||||
<PackageVersion Include="Anthropic" Version="12.3.0" />
|
||||
<PackageVersion Include="Anthropic.Foundry" Version="0.4.1" />
|
||||
<PackageVersion Include="Aspire.Azure.AI.OpenAI" Version="13.0.0-preview.1.25560.3" />
|
||||
<PackageVersion Include="Aspire.Hosting.AppHost" Version="$(AspireAppHostSdkVersion)" />
|
||||
<PackageVersion Include="Aspire.Hosting.Azure.CognitiveServices" Version="$(AspireAppHostSdkVersion)" />
|
||||
<PackageVersion Include="Aspire.Microsoft.Azure.Cosmos" Version="$(AspireAppHostSdkVersion)" />
|
||||
<PackageVersion Include="CommunityToolkit.Aspire.OllamaSharp" Version="13.0.0" />
|
||||
<!-- Azure.* -->
|
||||
<PackageVersion Include="Azure.AI.Projects" Version="2.0.0-beta.2" />
|
||||
<PackageVersion Include="Azure.AI.Agents.Persistent" Version="1.2.0-beta.10" />
|
||||
<PackageVersion Include="Azure.AI.OpenAI" Version="2.9.0-beta.1" />
|
||||
<PackageVersion Include="Azure.Identity" Version="1.19.0" />
|
||||
<PackageVersion Include="Azure.AI.Projects" Version="1.2.0-beta.5" />
|
||||
<PackageVersion Include="Azure.AI.Projects.OpenAI" Version="1.0.0-beta.5" />
|
||||
<PackageVersion Include="Azure.AI.Agents.Persistent" Version="1.2.0-beta.8" />
|
||||
<PackageVersion Include="Azure.AI.OpenAI" Version="2.8.0-beta.1" />
|
||||
<PackageVersion Include="Azure.Identity" Version="1.17.1" />
|
||||
<PackageVersion Include="Azure.Monitor.OpenTelemetry.Exporter" Version="1.4.0" />
|
||||
<!-- Google Gemini -->
|
||||
<PackageVersion Include="Google.GenAI" Version="0.11.0" />
|
||||
@@ -32,19 +33,18 @@
|
||||
<!-- Newtonsoft.Json -->
|
||||
<PackageVersion Include="Newtonsoft.Json" Version="13.0.4" />
|
||||
<!-- System.* -->
|
||||
<PackageVersion Include="Microsoft.Bcl.AsyncInterfaces" Version="10.0.4" />
|
||||
<PackageVersion Include="Microsoft.Bcl.AsyncInterfaces" Version="10.0.2" />
|
||||
<PackageVersion Include="Microsoft.Bcl.HashCode" Version="6.0.0" />
|
||||
<PackageVersion Include="Microsoft.Bcl.Memory" Version="10.0.4" />
|
||||
<PackageVersion Include="System.ClientModel" Version="1.9.0" />
|
||||
<PackageVersion Include="System.ClientModel" Version="1.8.1" />
|
||||
<PackageVersion Include="System.CodeDom" Version="10.0.0" />
|
||||
<PackageVersion Include="System.Collections.Immutable" Version="10.0.1" />
|
||||
<PackageVersion Include="System.CommandLine" Version="2.0.0-rc.2.25502.107" />
|
||||
<PackageVersion Include="System.Diagnostics.DiagnosticSource" Version="10.0.4" />
|
||||
<PackageVersion Include="System.Linq.AsyncEnumerable" Version="10.0.4" />
|
||||
<PackageVersion Include="System.Diagnostics.DiagnosticSource" Version="10.0.2" />
|
||||
<PackageVersion Include="System.Linq.AsyncEnumerable" Version="10.0.0" />
|
||||
<PackageVersion Include="System.Net.Http.Json" Version="10.0.0" />
|
||||
<PackageVersion Include="System.Net.ServerSentEvents" Version="10.0.4" />
|
||||
<PackageVersion Include="System.Text.Json" Version="10.0.4" />
|
||||
<PackageVersion Include="System.Threading.Channels" Version="10.0.4" />
|
||||
<PackageVersion Include="System.Net.ServerSentEvents" Version="10.0.1" />
|
||||
<PackageVersion Include="System.Text.Json" Version="10.0.2" />
|
||||
<PackageVersion Include="System.Threading.Channels" Version="10.0.2" />
|
||||
<PackageVersion Include="System.Threading.Tasks.Extensions" Version="4.6.3" />
|
||||
<PackageVersion Include="System.Net.Security" Version="4.3.2" />
|
||||
<!-- OpenTelemetry -->
|
||||
@@ -58,30 +58,24 @@
|
||||
<PackageVersion Include="OpenTelemetry.Instrumentation.Http" Version="1.13.0" />
|
||||
<PackageVersion Include="OpenTelemetry.Instrumentation.Runtime" Version="1.13.0" />
|
||||
<!-- Microsoft.AspNetCore.* -->
|
||||
<PackageVersion Include="Microsoft.AspNetCore.Authentication.JwtBearer" Version="10.0.0" />
|
||||
<PackageVersion Include="Microsoft.AspNetCore.Authentication.OpenIdConnect" Version="10.0.0" />
|
||||
<PackageVersion Include="Microsoft.AspNetCore.OpenApi" Version="10.0.0" />
|
||||
<PackageVersion Include="Swashbuckle.AspNetCore.SwaggerUI" Version="10.0.0" />
|
||||
<!-- Microsoft.Extensions.* -->
|
||||
<PackageVersion Include="Microsoft.Extensions.AI" Version="10.4.0" />
|
||||
<PackageVersion Include="Microsoft.Extensions.AI.Abstractions" Version="10.4.0" />
|
||||
<PackageVersion Include="Microsoft.Extensions.AI.Evaluation" Version="10.4.0" />
|
||||
<PackageVersion Include="Microsoft.Extensions.AI.Evaluation.Quality" Version="10.4.0" />
|
||||
<PackageVersion Include="Microsoft.Extensions.AI.Evaluation.Safety" Version="10.3.0-preview.1.26109.11" />
|
||||
<PackageVersion Include="Microsoft.Extensions.AI.OpenAI" Version="10.4.0" />
|
||||
<PackageVersion Include="Microsoft.Extensions.AI" Version="10.2.0" />
|
||||
<PackageVersion Include="Microsoft.Extensions.AI.Abstractions" Version="10.2.0" />
|
||||
<PackageVersion Include="Microsoft.Extensions.AI.OpenAI" Version="10.2.0-preview.1.26063.2" />
|
||||
<PackageVersion Include="Microsoft.Extensions.Caching.Memory" Version="10.0.0" />
|
||||
<PackageVersion Include="Microsoft.Extensions.Compliance.Abstractions" Version="10.4.0" />
|
||||
<PackageVersion Include="Microsoft.Extensions.Configuration" Version="10.0.0" />
|
||||
<PackageVersion Include="Microsoft.Extensions.Configuration.Binder" Version="10.0.0" />
|
||||
<PackageVersion Include="Microsoft.Extensions.Configuration.EnvironmentVariables" Version="10.0.0" />
|
||||
<PackageVersion Include="Microsoft.Extensions.Configuration.Json" Version="10.0.0" />
|
||||
<PackageVersion Include="Microsoft.Extensions.Configuration.UserSecrets" Version="10.0.0" />
|
||||
<PackageVersion Include="Microsoft.Extensions.DependencyInjection" Version="10.0.0" />
|
||||
<PackageVersion Include="Microsoft.Extensions.DependencyInjection.Abstractions" Version="10.0.4" />
|
||||
<PackageVersion Include="Microsoft.Extensions.DependencyInjection.Abstractions" Version="10.0.2" />
|
||||
<PackageVersion Include="Microsoft.Extensions.Hosting" Version="10.0.0" />
|
||||
<PackageVersion Include="Microsoft.Extensions.Http.Resilience" Version="10.0.0" />
|
||||
<PackageVersion Include="Microsoft.Extensions.Logging" Version="10.0.0" />
|
||||
<PackageVersion Include="Microsoft.Extensions.Logging.Abstractions" Version="10.0.4" />
|
||||
<PackageVersion Include="Microsoft.Extensions.Logging.Abstractions" Version="10.0.2" />
|
||||
<PackageVersion Include="Microsoft.Extensions.Logging.Console" Version="10.0.0" />
|
||||
<PackageVersion Include="Microsoft.Extensions.ServiceDiscovery" Version="10.0.0" />
|
||||
<PackageVersion Include="Microsoft.Extensions.VectorData.Abstractions" Version="9.7.0" />
|
||||
@@ -95,39 +89,38 @@
|
||||
<PackageVersion Include="Microsoft.SemanticKernel.Agents.AzureAI" Version="1.67.0-preview" />
|
||||
<PackageVersion Include="Microsoft.SemanticKernel.Plugins.OpenApi" Version="1.67.0" />
|
||||
<!-- Agent SDKs -->
|
||||
<PackageVersion Include="GitHub.Copilot.SDK" Version="0.1.29" />
|
||||
<PackageVersion Include="GitHub.Copilot.SDK" Version="0.1.18" />
|
||||
<PackageVersion Include="Microsoft.Agents.CopilotStudio.Client" Version="1.3.171-beta" />
|
||||
<!-- M365 Agents SDK -->
|
||||
<PackageVersion Include="AdaptiveCards" Version="3.1.0" />
|
||||
<PackageVersion Include="Microsoft.Agents.Authentication.Msal" Version="1.3.171-beta" />
|
||||
<PackageVersion Include="Microsoft.Agents.Hosting.AspNetCore" Version="1.3.171-beta" />
|
||||
<!-- A2A -->
|
||||
<PackageVersion Include="A2A" Version="0.3.4-preview" />
|
||||
<PackageVersion Include="A2A.AspNetCore" Version="0.3.4-preview" />
|
||||
<PackageVersion Include="A2A" Version="0.3.3-preview" />
|
||||
<PackageVersion Include="A2A.AspNetCore" Version="0.3.3-preview" />
|
||||
<!-- MCP -->
|
||||
<PackageVersion Include="ModelContextProtocol" Version="1.1.0" />
|
||||
<PackageVersion Include="ModelContextProtocol" Version="0.4.0-preview.3" />
|
||||
<!-- Inference SDKs -->
|
||||
<PackageVersion Include="AWSSDK.Extensions.Bedrock.MEAI" Version="4.0.5.1" />
|
||||
<PackageVersion Include="Microsoft.ML.OnnxRuntimeGenAI" Version="0.10.0" />
|
||||
<PackageVersion Include="Microsoft.ML.Tokenizers" Version="2.0.0" />
|
||||
<PackageVersion Include="OllamaSharp" Version="5.4.8" />
|
||||
<PackageVersion Include="OpenAI" Version="2.9.1" />
|
||||
<PackageVersion Include="OpenAI" Version="2.8.0" />
|
||||
<!-- Identity -->
|
||||
<PackageVersion Include="Microsoft.Identity.Client.Extensions.Msal" Version="4.83.1" />
|
||||
<PackageVersion Include="Microsoft.Identity.Client.Extensions.Msal" Version="4.78.0" />
|
||||
<!-- Workflows -->
|
||||
<PackageVersion Include="Microsoft.Agents.ObjectModel" Version="2026.2.4.1" />
|
||||
<PackageVersion Include="Microsoft.Agents.ObjectModel.Json" Version="2026.2.4.1" />
|
||||
<PackageVersion Include="Microsoft.Agents.ObjectModel.PowerFx" Version="2026.2.4.1" />
|
||||
<PackageVersion Include="Microsoft.PowerFx.Interpreter" Version="1.8.1" />
|
||||
<PackageVersion Include="Microsoft.Agents.ObjectModel" Version="2026.1.2.3" />
|
||||
<PackageVersion Include="Microsoft.Agents.ObjectModel.Json" Version="2026.1.2.3" />
|
||||
<PackageVersion Include="Microsoft.Agents.ObjectModel.PowerFx" Version="2026.1.2.3" />
|
||||
<PackageVersion Include="Microsoft.PowerFx.Interpreter" Version="1.5.0-build.20251008-1002" />
|
||||
<!-- Durable Task -->
|
||||
<PackageVersion Include="Microsoft.DurableTask.Client" Version="1.18.0" />
|
||||
<PackageVersion Include="Microsoft.DurableTask.Client.AzureManaged" Version="1.18.0" />
|
||||
<PackageVersion Include="Microsoft.DurableTask.Worker" Version="1.18.0" />
|
||||
<PackageVersion Include="Microsoft.DurableTask.Worker.AzureManaged" Version="1.18.0" />
|
||||
<PackageVersion Include="Microsoft.DurableTask.Client" Version="1.19.1" />
|
||||
<PackageVersion Include="Microsoft.DurableTask.Client.AzureManaged" Version="1.19.0" />
|
||||
<PackageVersion Include="Microsoft.DurableTask.Worker" Version="1.19.0" />
|
||||
<PackageVersion Include="Microsoft.DurableTask.Worker.AzureManaged" Version="1.19.0" />
|
||||
<!-- Azure Functions -->
|
||||
<PackageVersion Include="Microsoft.Azure.Functions.Worker" Version="2.50.0" />
|
||||
<PackageVersion Include="Microsoft.Azure.Functions.Worker.ApplicationInsights" Version="2.50.0" />
|
||||
<PackageVersion Include="Microsoft.Azure.Functions.Worker.Extensions.DurableTask" Version="1.12.1" />
|
||||
<PackageVersion Include="Microsoft.Azure.Functions.Worker.Extensions.DurableTask" Version="1.13.1" />
|
||||
<PackageVersion Include="Microsoft.Azure.Functions.Worker.Extensions.DurableTask.AzureManaged" Version="1.0.1" />
|
||||
<PackageVersion Include="Microsoft.Azure.Functions.Worker.Extensions.Http" Version="3.3.0" />
|
||||
<PackageVersion Include="Microsoft.Azure.Functions.Worker.Extensions.Http.AspNetCore" Version="2.1.0" />
|
||||
@@ -142,14 +135,15 @@
|
||||
<PackageVersion Include="Microsoft.AspNetCore.TestHost" Condition="'$(TargetFramework)' == 'net10.0'" Version="10.0.0" />
|
||||
<PackageVersion Include="Microsoft.NET.Test.Sdk" Version="18.0.0" />
|
||||
<PackageVersion Include="Moq" Version="[4.18.4]" />
|
||||
<PackageVersion Include="xunit.v3.mtp-v2" Version="3.2.2" />
|
||||
<PackageVersion Include="xunit.runner.visualstudio" Version="3.1.5" />
|
||||
<PackageVersion Include="xRetry.v3" Version="1.0.0-rc3" />
|
||||
<PackageVersion Include="Microsoft.Testing.Extensions.CodeCoverage" Version="18.4.1" />
|
||||
<PackageVersion Include="xunit" Version="2.9.3" />
|
||||
<PackageVersion Include="xunit.abstractions" Version="2.0.3" />
|
||||
<PackageVersion Include="xunit.runner.visualstudio" Version="3.1.3" />
|
||||
<PackageVersion Include="Xunit.SkippableFact" Version="1.5.23" />
|
||||
<PackageVersion Include="xretry" Version="1.9.0" />
|
||||
<PackageVersion Include="coverlet.collector" Version="6.0.4" />
|
||||
<!-- Symbols -->
|
||||
<PackageVersion Include="Microsoft.SourceLink.GitHub" Version="8.0.0" />
|
||||
<!-- Toolset -->
|
||||
<PackageVersion Include="ReferenceTrimmer" Version="3.4.5" />
|
||||
<PackageVersion Include="Microsoft.CodeAnalysis.Analyzers" Version="3.11.0" />
|
||||
<PackageVersion Include="Microsoft.CodeAnalysis.CSharp" Version="4.14.0" />
|
||||
<PackageVersion Include="Microsoft.CodeAnalysis.NetAnalyzers" Version="10.0.100" />
|
||||
@@ -188,4 +182,4 @@
|
||||
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
|
||||
</PackageReference>
|
||||
</ItemGroup>
|
||||
</Project>
|
||||
</Project>
|
||||
+11
-5
@@ -1,20 +1,26 @@
|
||||
# Get Started with Microsoft Agent Framework for C# Developers
|
||||
|
||||
## Samples
|
||||
|
||||
- [Getting Started with Agents](./samples/GettingStarted/Agents): basic agent creation and tool usage
|
||||
- [Agent Provider Samples](./samples/GettingStarted/AgentProviders): samples showing different agent providers
|
||||
- [Workflow Samples](./samples/GettingStarted/Workflows): advanced multi-agent patterns and workflow orchestration
|
||||
|
||||
## Quickstart
|
||||
|
||||
### Basic Agent - .NET
|
||||
|
||||
```c#
|
||||
using System;
|
||||
using Azure.AI.OpenAI;
|
||||
using Azure.Identity;
|
||||
using Microsoft.Agents.AI;
|
||||
using OpenAI.Responses;
|
||||
|
||||
var endpoint = Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT")!;
|
||||
var deploymentName = Environment.GetEnvironmentVariable("AZURE_OPENAI_DEPLOYMENT_NAME")!;
|
||||
|
||||
var agent = new AzureOpenAIClient(new Uri(endpoint), new AzureCliCredential())
|
||||
.GetResponsesClient(deploymentName)
|
||||
.GetOpenAIResponseClient(deploymentName)
|
||||
.AsAIAgent(name: "HaikuBot", instructions: "You are an upbeat assistant that writes beautifully.");
|
||||
|
||||
Console.WriteLine(await agent.RunAsync("Write a haiku about Microsoft Agent Framework."));
|
||||
@@ -22,9 +28,9 @@ Console.WriteLine(await agent.RunAsync("Write a haiku about Microsoft Agent Fram
|
||||
|
||||
## Examples & Samples
|
||||
|
||||
- [Getting Started with Agents](./samples/02-agents/Agents): basic agent creation and tool usage
|
||||
- [Agent Provider Samples](./samples/02-agents/AgentProviders): samples showing different agent providers
|
||||
- [Workflow Samples](./samples/03-workflows): advanced multi-agent patterns and workflow orchestration
|
||||
- [Getting Started with Agents](./samples/GettingStarted/Agents): basic agent creation and tool usage
|
||||
- [Agent Provider Samples](./samples/GettingStarted/AgentProviders): samples showing different agent providers
|
||||
- [Workflow Samples](./samples/GettingStarted/Workflows): advanced multi-agent patterns and workflow orchestration
|
||||
|
||||
## Agent Framework Documentation
|
||||
|
||||
|
||||
+230
-296
@@ -5,210 +5,229 @@
|
||||
<BuildType Name="Release" />
|
||||
</Configurations>
|
||||
<Folder Name="/Samples/">
|
||||
<File Path="samples/AGENTS.md" />
|
||||
<File Path="samples/README.md" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/01-get-started/">
|
||||
<Project Path="samples/01-get-started/01_hello_agent/01_hello_agent.csproj" />
|
||||
<Project Path="samples/01-get-started/02_add_tools/02_add_tools.csproj" />
|
||||
<Project Path="samples/01-get-started/03_multi_turn/03_multi_turn.csproj" />
|
||||
<Project Path="samples/01-get-started/04_memory/04_memory.csproj" />
|
||||
<Project Path="samples/01-get-started/05_first_workflow/05_first_workflow.csproj" />
|
||||
<Project Path="samples/01-get-started/06_host_your_agent/06_host_your_agent.csproj" />
|
||||
<Folder Name="/Samples/A2AClientServer/">
|
||||
<File Path="samples/A2AClientServer/README.md" />
|
||||
<Project Path="samples/A2AClientServer/A2AClient/A2AClient.csproj" />
|
||||
<Project Path="samples/A2AClientServer/A2AServer/A2AServer.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/02-agents/">
|
||||
<File Path="samples/02-agents/README.md" />
|
||||
<Folder Name="/Samples/AgentWebChat/">
|
||||
<Project Path="samples/AgentWebChat/AgentWebChat.AgentHost/AgentWebChat.AgentHost.csproj" />
|
||||
<Project Path="samples/AgentWebChat/AgentWebChat.AppHost/AgentWebChat.AppHost.csproj" />
|
||||
<Project Path="samples/AgentWebChat/AgentWebChat.ServiceDefaults/AgentWebChat.ServiceDefaults.csproj" />
|
||||
<Project Path="samples/AgentWebChat/AgentWebChat.Web/AgentWebChat.Web.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/02-agents/AgentProviders/">
|
||||
<File Path="samples/02-agents/AgentProviders/README.md" />
|
||||
<Project Path="samples/02-agents/AgentProviders/Agent_With_A2A/Agent_With_A2A.csproj" />
|
||||
<Project Path="samples/02-agents/AgentProviders/Agent_With_Anthropic/Agent_With_Anthropic.csproj" />
|
||||
<Project Path="samples/02-agents/AgentProviders/Agent_With_AzureAIAgentsPersistent/Agent_With_AzureAIAgentsPersistent.csproj" />
|
||||
<Project Path="samples/02-agents/AgentProviders/Agent_With_AzureAIProject/Agent_With_AzureAIProject.csproj" />
|
||||
<Project Path="samples/02-agents/AgentProviders/Agent_With_AzureFoundryModel/Agent_With_AzureFoundryModel.csproj" />
|
||||
<Project Path="samples/02-agents/AgentProviders/Agent_With_AzureOpenAIChatCompletion/Agent_With_AzureOpenAIChatCompletion.csproj" />
|
||||
<Project Path="samples/02-agents/AgentProviders/Agent_With_AzureOpenAIResponses/Agent_With_AzureOpenAIResponses.csproj" />
|
||||
<Project Path="samples/02-agents/AgentProviders/Agent_With_CustomImplementation/Agent_With_CustomImplementation.csproj" />
|
||||
<Project Path="samples/02-agents/AgentProviders/Agent_With_GitHubCopilot/Agent_With_GitHubCopilot.csproj" />
|
||||
<Project Path="samples/02-agents/AgentProviders/Agent_With_GoogleGemini/Agent_With_GoogleGemini.csproj" />
|
||||
<Project Path="samples/02-agents/AgentProviders/Agent_With_Ollama/Agent_With_Ollama.csproj" />
|
||||
<Project Path="samples/02-agents/AgentProviders/Agent_With_ONNX/Agent_With_ONNX.csproj" />
|
||||
<Project Path="samples/02-agents/AgentProviders/Agent_With_OpenAIAssistants/Agent_With_OpenAIAssistants.csproj" />
|
||||
<Project Path="samples/02-agents/AgentProviders/Agent_With_OpenAIChatCompletion/Agent_With_OpenAIChatCompletion.csproj" />
|
||||
<Project Path="samples/02-agents/AgentProviders/Agent_With_OpenAIResponses/Agent_With_OpenAIResponses.csproj" />
|
||||
<Folder Name="/Samples/AGUIClientServer/">
|
||||
<Project Path="samples/AGUIClientServer/AGUIClient/AGUIClient.csproj" />
|
||||
<Project Path="samples/AGUIClientServer/AGUIDojoServer/AGUIDojoServer.csproj" />
|
||||
<Project Path="samples/AGUIClientServer/AGUIServer/AGUIServer.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/02-agents/Agents/">
|
||||
<File Path="samples/02-agents/Agents/README.md" />
|
||||
<Project Path="samples/02-agents/Agents/Agent_Step01_UsingFunctionToolsWithApprovals/Agent_Step01_UsingFunctionToolsWithApprovals.csproj" />
|
||||
<Project Path="samples/02-agents/Agents/Agent_Step02_StructuredOutput/Agent_Step02_StructuredOutput.csproj" />
|
||||
<Project Path="samples/02-agents/Agents/Agent_Step03_PersistedConversations/Agent_Step03_PersistedConversations.csproj" />
|
||||
<Project Path="samples/02-agents/Agents/Agent_Step04_3rdPartyChatHistoryStorage/Agent_Step04_3rdPartyChatHistoryStorage.csproj" />
|
||||
<Project Path="samples/02-agents/Agents/Agent_Step05_Observability/Agent_Step05_Observability.csproj" />
|
||||
<Project Path="samples/02-agents/Agents/Agent_Step06_DependencyInjection/Agent_Step06_DependencyInjection.csproj" />
|
||||
<Project Path="samples/02-agents/Agents/Agent_Step07_AsMcpTool/Agent_Step07_AsMcpTool.csproj" />
|
||||
<Project Path="samples/02-agents/Agents/Agent_Step08_UsingImages/Agent_Step08_UsingImages.csproj" />
|
||||
<Project Path="samples/02-agents/Agents/Agent_Step09_AsFunctionTool/Agent_Step09_AsFunctionTool.csproj" />
|
||||
<Project Path="samples/02-agents/Agents/Agent_Step10_BackgroundResponsesWithToolsAndPersistence/Agent_Step10_BackgroundResponsesWithToolsAndPersistence.csproj" />
|
||||
<Project Path="samples/02-agents/Agents/Agent_Step11_Middleware/Agent_Step11_Middleware.csproj" />
|
||||
<Project Path="samples/02-agents/Agents/Agent_Step12_Plugins/Agent_Step12_Plugins.csproj" />
|
||||
<Project Path="samples/02-agents/Agents/Agent_Step13_ChatReduction/Agent_Step13_ChatReduction.csproj" />
|
||||
<Project Path="samples/02-agents/Agents/Agent_Step14_BackgroundResponses/Agent_Step14_BackgroundResponses.csproj" />
|
||||
<Project Path="samples/02-agents/Agents/Agent_Step15_DeepResearch/Agent_Step15_DeepResearch.csproj" />
|
||||
<Project Path="samples/02-agents/Agents/Agent_Step16_Declarative/Agent_Step16_Declarative.csproj" />
|
||||
<Project Path="samples/02-agents/Agents/Agent_Step17_AdditionalAIContext/Agent_Step17_AdditionalAIContext.csproj" />
|
||||
<Project Path="samples/02-agents/Agents/Agent_Step18_CompactionPipeline/Agent_Step18_CompactionPipeline.csproj" />
|
||||
<Folder Name="/Samples/Durable/" />
|
||||
<Folder Name="/Samples/Durable/Agents/" />
|
||||
<Folder Name="/Samples/Durable/Agents/AzureFunctions/">
|
||||
<File Path="samples/Durable/Agents/AzureFunctions/.editorconfig" />
|
||||
<File Path="samples/Durable/Agents/AzureFunctions/README.md" />
|
||||
<Project Path="samples/Durable/Agents/AzureFunctions/01_SingleAgent/01_SingleAgent.csproj" />
|
||||
<Project Path="samples/Durable/Agents/AzureFunctions/02_AgentOrchestration_Chaining/02_AgentOrchestration_Chaining.csproj" />
|
||||
<Project Path="samples/Durable/Agents/AzureFunctions/03_AgentOrchestration_Concurrency/03_AgentOrchestration_Concurrency.csproj" />
|
||||
<Project Path="samples/Durable/Agents/AzureFunctions/04_AgentOrchestration_Conditionals/04_AgentOrchestration_Conditionals.csproj" />
|
||||
<Project Path="samples/Durable/Agents/AzureFunctions/05_AgentOrchestration_HITL/05_AgentOrchestration_HITL.csproj" />
|
||||
<Project Path="samples/Durable/Agents/AzureFunctions/06_LongRunningTools/06_LongRunningTools.csproj" />
|
||||
<Project Path="samples/Durable/Agents/AzureFunctions/07_AgentAsMcpTool/07_AgentAsMcpTool.csproj" />
|
||||
<Project Path="samples/Durable/Agents/AzureFunctions/08_ReliableStreaming/08_ReliableStreaming.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/02-agents/DeclarativeAgents/">
|
||||
<Project Path="samples/02-agents/DeclarativeAgents/ChatClient/DeclarativeChatClientAgents.csproj" />
|
||||
<Folder Name="/Samples/Durable/Agents/ConsoleApps/">
|
||||
<File Path="samples/Durable/Agents/ConsoleApps/README.md" />
|
||||
<Project Path="samples/Durable/Agents/ConsoleApps/01_SingleAgent/01_SingleAgent.csproj" />
|
||||
<Project Path="samples/Durable/Agents/ConsoleApps/02_AgentOrchestration_Chaining/02_AgentOrchestration_Chaining.csproj" />
|
||||
<Project Path="samples/Durable/Agents/ConsoleApps/03_AgentOrchestration_Concurrency/03_AgentOrchestration_Concurrency.csproj" />
|
||||
<Project Path="samples/Durable/Agents/ConsoleApps/04_AgentOrchestration_Conditionals/04_AgentOrchestration_Conditionals.csproj" />
|
||||
<Project Path="samples/Durable/Agents/ConsoleApps/05_AgentOrchestration_HITL/05_AgentOrchestration_HITL.csproj" />
|
||||
<Project Path="samples/Durable/Agents/ConsoleApps/06_LongRunningTools/06_LongRunningTools.csproj" />
|
||||
<Project Path="samples/Durable/Agents/ConsoleApps/07_ReliableStreaming/07_ReliableStreaming.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/04-hosting/DurableWorkflows/" />
|
||||
<Folder Name="/Samples/04-hosting/DurableWorkflows/ConsoleApps/">
|
||||
<Project Path="samples/04-hosting/DurableWorkflows/ConsoleApps/01_SequentialWorkflow/01_SequentialWorkflow.csproj" />
|
||||
<Project Path="samples/04-hosting/DurableWorkflows/ConsoleApps/02_ConcurrentWorkflow/02_ConcurrentWorkflow.csproj" />
|
||||
<Project Path="samples/04-hosting/DurableWorkflows/ConsoleApps/03_ConditionalEdges/03_ConditionalEdges.csproj" />
|
||||
<Project Path="samples/04-hosting/DurableWorkflows/ConsoleApps/04_WorkflowAndAgents/04_WorkflowAndAgents.csproj" />
|
||||
<Project Path="samples/04-hosting/DurableWorkflows/ConsoleApps/05_WorkflowEvents/05_WorkflowEvents.csproj" />
|
||||
<Project Path="samples/04-hosting/DurableWorkflows/ConsoleApps/06_WorkflowSharedState/06_WorkflowSharedState.csproj" />
|
||||
<Project Path="samples/04-hosting/DurableWorkflows/ConsoleApps/07_SubWorkflows/07_SubWorkflows.csproj" />
|
||||
<Project Path="samples/04-hosting/DurableWorkflows/ConsoleApps/08_WorkflowHITL/08_WorkflowHITL.csproj" />
|
||||
<Folder Name="/Samples/Durable/Workflows/">
|
||||
<Project Path="samples/Durable/Workflow/ConsoleApps/01_SequentialWorkflow/01_SequentialWorkflow.csproj" />
|
||||
<Project Path="samples/Durable/Workflow/ConsoleApps/02_ConcurrentWorkflow/02_ConcurrentWorkflow.csproj" />
|
||||
<Project Path="samples/Durable/Workflow/ConsoleApps/03_ConditionalEdges/03_ConditionalEdges.csproj" />
|
||||
<Project Path="samples/Durable/Workflow/ConsoleApps/04_WorkflowAndAgents/04_WorkflowAndAgents.csproj" />
|
||||
<Project Path="samples/Durable/Workflow/ConsoleApps/05_NestedWorkflows/05_NestedWorkflows.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/04-hosting/DurableWorkflows/AzureFunctions/">
|
||||
<Project Path="samples/04-hosting/DurableWorkflows/AzureFunctions/01_SequentialWorkflow/01_SequentialWorkflow.csproj" />
|
||||
<Project Path="samples/04-hosting/DurableWorkflows/AzureFunctions/02_ConcurrentWorkflow/02_ConcurrentWorkflow.csproj" />
|
||||
<Project Path="samples/04-hosting/DurableWorkflows/AzureFunctions/03_WorkflowHITL/03_WorkflowHITL.csproj" />
|
||||
<Folder Name="/Samples/Durable/Workflows/AzureFunctions/">
|
||||
<Project Path="samples/Durable/Workflow/AzureFunctions/01_SequentialWorkflow/01_SequentialWorkflow.csproj" />
|
||||
<Project Path="samples/Durable/Workflow/AzureFunctions/02_ConcurrentWorkflow/02_ConcurrentWorkflow.csproj" />
|
||||
<Project Path="samples/Durable/Workflow/AzureFunctions/03_ConditionalEdges/03_ConditionalEdges.csproj" />
|
||||
<Project Path="samples/Durable/Workflow/AzureFunctions/04_NestedWorkflows/04_NestedWorkflows.csproj" />
|
||||
<Project Path="samples/Durable/Workflow/AzureFunctions/05_WorkflowAndAgents/05_WorkflowAndAgents.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/GettingStarted/">
|
||||
<File Path="samples/GettingStarted/README.md" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/02-agents/AGUI/">
|
||||
<File Path="samples/02-agents/AGUI/README.md" />
|
||||
<Folder Name="/Samples/GettingStarted/A2A/">
|
||||
<File Path="samples/GettingStarted/A2A/README.md" />
|
||||
<Project Path="samples/GettingStarted/A2A/A2AAgent_AsFunctionTools/A2AAgent_AsFunctionTools.csproj" />
|
||||
<Project Path="samples/GettingStarted/A2A/A2AAgent_PollingForTaskCompletion/A2AAgent_PollingForTaskCompletion.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/02-agents/AGUI/Step01_GettingStarted/">
|
||||
<Project Path="samples/02-agents/AGUI/Step01_GettingStarted/Client/Client.csproj" />
|
||||
<Project Path="samples/02-agents/AGUI/Step01_GettingStarted/Server/Server.csproj" />
|
||||
<Folder Name="/Samples/GettingStarted/AgentProviders/">
|
||||
<File Path="samples/GettingStarted/AgentProviders/README.md" />
|
||||
<Project Path="samples/GettingStarted/AgentProviders/Agent_With_A2A/Agent_With_A2A.csproj" />
|
||||
<Project Path="samples/GettingStarted/AgentProviders/Agent_With_Anthropic/Agent_With_Anthropic.csproj" />
|
||||
<Project Path="samples/GettingStarted/AgentProviders/Agent_With_AzureAIAgentsPersistent/Agent_With_AzureAIAgentsPersistent.csproj" />
|
||||
<Project Path="samples/GettingStarted/AgentProviders/Agent_With_AzureAIProject/Agent_With_AzureAIProject.csproj" />
|
||||
<Project Path="samples/GettingStarted/AgentProviders/Agent_With_AzureFoundryModel/Agent_With_AzureFoundryModel.csproj" />
|
||||
<Project Path="samples/GettingStarted/AgentProviders/Agent_With_AzureOpenAIChatCompletion/Agent_With_AzureOpenAIChatCompletion.csproj" />
|
||||
<Project Path="samples/GettingStarted/AgentProviders/Agent_With_AzureOpenAIResponses/Agent_With_AzureOpenAIResponses.csproj" />
|
||||
<Project Path="samples/GettingStarted/AgentProviders/Agent_With_CustomImplementation/Agent_With_CustomImplementation.csproj" />
|
||||
<Project Path="samples/GettingStarted/AgentProviders/Agent_With_GitHubCopilot/Agent_With_GitHubCopilot.csproj" />
|
||||
<Project Path="samples/GettingStarted/AgentProviders/Agent_With_GoogleGemini/Agent_With_GoogleGemini.csproj" />
|
||||
<Project Path="samples/GettingStarted/AgentProviders/Agent_With_Ollama/Agent_With_Ollama.csproj" />
|
||||
<Project Path="samples/GettingStarted/AgentProviders/Agent_With_ONNX/Agent_With_ONNX.csproj" />
|
||||
<Project Path="samples/GettingStarted/AgentProviders/Agent_With_OpenAIAssistants/Agent_With_OpenAIAssistants.csproj" />
|
||||
<Project Path="samples/GettingStarted/AgentProviders/Agent_With_OpenAIChatCompletion/Agent_With_OpenAIChatCompletion.csproj" />
|
||||
<Project Path="samples/GettingStarted/AgentProviders/Agent_With_OpenAIResponses/Agent_With_OpenAIResponses.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/02-agents/AGUI/Step02_BackendTools/">
|
||||
<Project Path="samples/02-agents/AGUI/Step02_BackendTools/Client/Client.csproj" />
|
||||
<Project Path="samples/02-agents/AGUI/Step02_BackendTools/Server/Server.csproj" />
|
||||
<Folder Name="/Samples/GettingStarted/Agents/">
|
||||
<File Path="samples/GettingStarted/Agents/README.md" />
|
||||
<Project Path="samples/GettingStarted/Agents/Agent_Step01_Running/Agent_Step01_Running.csproj" />
|
||||
<Project Path="samples/GettingStarted/Agents/Agent_Step02_MultiturnConversation/Agent_Step02_MultiturnConversation.csproj" />
|
||||
<Project Path="samples/GettingStarted/Agents/Agent_Step03_UsingFunctionTools/Agent_Step03_UsingFunctionTools.csproj" />
|
||||
<Project Path="samples/GettingStarted/Agents/Agent_Step04_UsingFunctionToolsWithApprovals/Agent_Step04_UsingFunctionToolsWithApprovals.csproj" />
|
||||
<Project Path="samples/GettingStarted/Agents/Agent_Step05_StructuredOutput/Agent_Step05_StructuredOutput.csproj" />
|
||||
<Project Path="samples/GettingStarted/Agents/Agent_Step06_PersistedConversations/Agent_Step06_PersistedConversations.csproj" />
|
||||
<Project Path="samples/GettingStarted/Agents/Agent_Step07_3rdPartyThreadStorage/Agent_Step07_3rdPartyThreadStorage.csproj" />
|
||||
<Project Path="samples/GettingStarted/Agents/Agent_Step08_Observability/Agent_Step08_Observability.csproj" />
|
||||
<Project Path="samples/GettingStarted/Agents/Agent_Step09_DependencyInjection/Agent_Step09_DependencyInjection.csproj" />
|
||||
<Project Path="samples/GettingStarted/Agents/Agent_Step10_AsMcpTool/Agent_Step10_AsMcpTool.csproj" />
|
||||
<Project Path="samples/GettingStarted/Agents/Agent_Step11_UsingImages/Agent_Step11_UsingImages.csproj" />
|
||||
<Project Path="samples/GettingStarted/Agents/Agent_Step12_AsFunctionTool/Agent_Step12_AsFunctionTool.csproj" />
|
||||
<Project Path="samples/GettingStarted/Agents/Agent_Step13_BackgroundResponsesWithToolsAndPersistence/Agent_Step13_BackgroundResponsesWithToolsAndPersistence.csproj" />
|
||||
<Project Path="samples/GettingStarted/Agents/Agent_Step14_Middleware/Agent_Step14_Middleware.csproj" />
|
||||
<Project Path="samples/GettingStarted/Agents/Agent_Step15_Plugins/Agent_Step15_Plugins.csproj" />
|
||||
<Project Path="samples/GettingStarted/Agents/Agent_Step16_ChatReduction/Agent_Step16_ChatReduction.csproj" />
|
||||
<Project Path="samples/GettingStarted/Agents/Agent_Step17_BackgroundResponses/Agent_Step17_BackgroundResponses.csproj" />
|
||||
<Project Path="samples/GettingStarted/Agents/Agent_Step18_DeepResearch/Agent_Step18_DeepResearch.csproj" />
|
||||
<Project Path="samples/GettingStarted/Agents/Agent_Step19_Declarative/Agent_Step19_Declarative.csproj" />
|
||||
<Project Path="samples/GettingStarted/Agents/Agent_Step20_AdditionalAIContext/Agent_Step20_AdditionalAIContext.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/02-agents/AGUI/Step03_FrontendTools/">
|
||||
<Project Path="samples/02-agents/AGUI/Step03_FrontendTools/Client/Client.csproj" />
|
||||
<Project Path="samples/02-agents/AGUI/Step03_FrontendTools/Server/Server.csproj" />
|
||||
<Folder Name="/Samples/GettingStarted/DeclarativeAgents/">
|
||||
<Project Path="samples/GettingStarted/DeclarativeAgents/ChatClient/DeclarativeChatClientAgents.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/02-agents/AGUI/Step04_HumanInLoop/">
|
||||
<Project Path="samples/02-agents/AGUI/Step04_HumanInLoop/Client/Client.csproj" />
|
||||
<Project Path="samples/02-agents/AGUI/Step04_HumanInLoop/Server/Server.csproj" />
|
||||
<Folder Name="/Samples/GettingStarted/AGUI/">
|
||||
<File Path="samples/GettingStarted/AGUI/README.md" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/02-agents/AgentSkills/">
|
||||
<File Path="samples/02-agents/AgentSkills/README.md" />
|
||||
<Project Path="samples/02-agents/AgentSkills/Agent_Step01_BasicSkills/Agent_Step01_BasicSkills.csproj" />
|
||||
<Folder Name="/Samples/GettingStarted/AGUI/Step01_GettingStarted/">
|
||||
<Project Path="samples/GettingStarted/AGUI/Step01_GettingStarted/Client/Client.csproj" />
|
||||
<Project Path="samples/GettingStarted/AGUI/Step01_GettingStarted/Server/Server.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/02-agents/AGUI/Step05_StateManagement/">
|
||||
<Project Path="samples/02-agents/AGUI/Step05_StateManagement/Client/Client.csproj" />
|
||||
<Project Path="samples/02-agents/AGUI/Step05_StateManagement/Server/Server.csproj" />
|
||||
<Folder Name="/Samples/GettingStarted/AGUI/Step02_BackendTools/">
|
||||
<Project Path="samples/GettingStarted/AGUI/Step02_BackendTools/Client/Client.csproj" />
|
||||
<Project Path="samples/GettingStarted/AGUI/Step02_BackendTools/Server/Server.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/02-agents/DevUI/">
|
||||
<File Path="samples/02-agents/DevUI/README.md" />
|
||||
<Project Path="samples/02-agents/DevUI/DevUI_Step01_BasicUsage/DevUI_Step01_BasicUsage.csproj" />
|
||||
<Folder Name="/Samples/GettingStarted/AGUI/Step03_FrontendTools/">
|
||||
<Project Path="samples/GettingStarted/AGUI/Step03_FrontendTools/Client/Client.csproj" />
|
||||
<Project Path="samples/GettingStarted/AGUI/Step03_FrontendTools/Server/Server.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/02-agents/AgentWithAnthropic/">
|
||||
<File Path="samples/02-agents/AgentWithAnthropic/README.md" />
|
||||
<Project Path="samples/02-agents/AgentWithAnthropic/Agent_Anthropic_Step01_Running/Agent_Anthropic_Step01_Running.csproj" />
|
||||
<Project Path="samples/02-agents/AgentWithAnthropic/Agent_Anthropic_Step02_Reasoning/Agent_Anthropic_Step02_Reasoning.csproj" />
|
||||
<Project Path="samples/02-agents/AgentWithAnthropic/Agent_Anthropic_Step03_UsingFunctionTools/Agent_Anthropic_Step03_UsingFunctionTools.csproj" />
|
||||
<Project Path="samples/02-agents/AgentWithAnthropic/Agent_Anthropic_Step04_UsingSkills/Agent_Anthropic_Step04_UsingSkills.csproj" />
|
||||
<Folder Name="/Samples/GettingStarted/AGUI/Step04_HumanInLoop/">
|
||||
<Project Path="samples/GettingStarted/AGUI/Step04_HumanInLoop/Client/Client.csproj" />
|
||||
<Project Path="samples/GettingStarted/AGUI/Step04_HumanInLoop/Server/Server.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/02-agents/AgentWithMemory/">
|
||||
<File Path="samples/02-agents/AgentWithMemory/README.md" />
|
||||
<Project Path="samples/02-agents/AgentWithMemory/AgentWithMemory_Step01_ChatHistoryMemory/AgentWithMemory_Step01_ChatHistoryMemory.csproj" />
|
||||
<Project Path="samples/02-agents/AgentWithMemory/AgentWithMemory_Step02_MemoryUsingMem0/AgentWithMemory_Step02_MemoryUsingMem0.csproj" />
|
||||
<Project Path="samples/02-agents/AgentWithMemory/AgentWithMemory_Step04_MemoryUsingFoundry/AgentWithMemory_Step04_MemoryUsingFoundry.csproj" />
|
||||
<Project Path="samples/02-agents/AgentWithMemory/AgentWithMemory_Step05_BoundedChatHistory/AgentWithMemory_Step05_BoundedChatHistory.csproj" />
|
||||
<Folder Name="/Samples/GettingStarted/AGUI/Step05_StateManagement/">
|
||||
<Project Path="samples/GettingStarted/AGUI/Step05_StateManagement/Client/Client.csproj" />
|
||||
<Project Path="samples/GettingStarted/AGUI/Step05_StateManagement/Server/Server.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/02-agents/AgentWithOpenAI/">
|
||||
<File Path="samples/02-agents/AgentWithOpenAI/README.md" />
|
||||
<Project Path="samples/02-agents/AgentWithOpenAI/Agent_OpenAI_Step01_Running/Agent_OpenAI_Step01_Running.csproj" />
|
||||
<Project Path="samples/02-agents/AgentWithOpenAI/Agent_OpenAI_Step02_Reasoning/Agent_OpenAI_Step02_Reasoning.csproj" />
|
||||
<Project Path="samples/02-agents/AgentWithOpenAI/Agent_OpenAI_Step03_CreateFromChatClient/Agent_OpenAI_Step03_CreateFromChatClient.csproj" />
|
||||
<Project Path="samples/02-agents/AgentWithOpenAI/Agent_OpenAI_Step04_CreateFromOpenAIResponseClient/Agent_OpenAI_Step04_CreateFromOpenAIResponseClient.csproj" />
|
||||
<Project Path="samples/02-agents/AgentWithOpenAI/Agent_OpenAI_Step05_Conversation/Agent_OpenAI_Step05_Conversation.csproj" />
|
||||
<Folder Name="/Samples/GettingStarted/DevUI/">
|
||||
<File Path="samples/GettingStarted/DevUI/README.md" />
|
||||
<Project Path="samples/GettingStarted/DevUI/DevUI_Step01_BasicUsage/DevUI_Step01_BasicUsage.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/02-agents/AgentWithRAG/">
|
||||
<File Path="samples/02-agents/AgentWithRAG/README.md" />
|
||||
<Project Path="samples/02-agents/AgentWithRAG/AgentWithRAG_Step01_BasicTextRAG/AgentWithRAG_Step01_BasicTextRAG.csproj" />
|
||||
<Project Path="samples/02-agents/AgentWithRAG/AgentWithRAG_Step02_CustomVectorStoreRAG/AgentWithRAG_Step02_CustomVectorStoreRAG.csproj" />
|
||||
<Project Path="samples/02-agents/AgentWithRAG/AgentWithRAG_Step03_CustomRAGDataSource/AgentWithRAG_Step03_CustomRAGDataSource.csproj" />
|
||||
<Project Path="samples/02-agents/AgentWithRAG/AgentWithRAG_Step04_FoundryServiceRAG/AgentWithRAG_Step04_FoundryServiceRAG.csproj" />
|
||||
<Folder Name="/Samples/GettingStarted/AgentWithAnthropic/">
|
||||
<File Path="samples/GettingStarted/AgentWithAnthropic/README.md" />
|
||||
<Project Path="samples/GettingStarted/AgentWithAnthropic/Agent_Anthropic_Step01_Running/Agent_Anthropic_Step01_Running.csproj" />
|
||||
<Project Path="samples/GettingStarted/AgentWithAnthropic/Agent_Anthropic_Step02_Reasoning/Agent_Anthropic_Step02_Reasoning.csproj" />
|
||||
<Project Path="samples/GettingStarted/AgentWithAnthropic/Agent_Anthropic_Step03_UsingFunctionTools/Agent_Anthropic_Step03_UsingFunctionTools.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/02-agents/FoundryAgents/">
|
||||
<File Path="samples/02-agents/FoundryAgents/README.md" />
|
||||
<Project Path="samples/02-agents/FoundryAgents/FoundryAgents_Evaluations_Step01_RedTeaming/FoundryAgents_Evaluations_Step01_RedTeaming.csproj" />
|
||||
<Project Path="samples/02-agents/FoundryAgents/FoundryAgents_Evaluations_Step02_SelfReflection/FoundryAgents_Evaluations_Step02_SelfReflection.csproj" />
|
||||
<Project Path="samples/02-agents/FoundryAgents/FoundryAgents_Step01.1_Basics/FoundryAgents_Step01.1_Basics.csproj" />
|
||||
<Project Path="samples/02-agents/FoundryAgents/FoundryAgents_Step01.2_Running/FoundryAgents_Step01.2_Running.csproj" />
|
||||
<Project Path="samples/02-agents/FoundryAgents/FoundryAgents_Step02_MultiturnConversation/FoundryAgents_Step02_MultiturnConversation.csproj" />
|
||||
<Project Path="samples/02-agents/FoundryAgents/FoundryAgents_Step03_UsingFunctionTools/FoundryAgents_Step03_UsingFunctionTools.csproj" />
|
||||
<Project Path="samples/02-agents/FoundryAgents/FoundryAgents_Step04_UsingFunctionToolsWithApprovals/FoundryAgents_Step04_UsingFunctionToolsWithApprovals.csproj" />
|
||||
<Project Path="samples/02-agents/FoundryAgents/FoundryAgents_Step05_StructuredOutput/FoundryAgents_Step05_StructuredOutput.csproj" />
|
||||
<Project Path="samples/02-agents/FoundryAgents/FoundryAgents_Step06_PersistedConversations/FoundryAgents_Step06_PersistedConversations.csproj" />
|
||||
<Project Path="samples/02-agents/FoundryAgents/FoundryAgents_Step07_Observability/FoundryAgents_Step07_Observability.csproj" />
|
||||
<Project Path="samples/02-agents/FoundryAgents/FoundryAgents_Step08_DependencyInjection/FoundryAgents_Step08_DependencyInjection.csproj" />
|
||||
<Project Path="samples/02-agents/FoundryAgents/FoundryAgents_Step09_UsingMcpClientAsTools/FoundryAgents_Step09_UsingMcpClientAsTools.csproj" />
|
||||
<Project Path="samples/02-agents/FoundryAgents/FoundryAgents_Step10_UsingImages/FoundryAgents_Step10_UsingImages.csproj" />
|
||||
<Project Path="samples/02-agents/FoundryAgents/FoundryAgents_Step11_AsFunctionTool/FoundryAgents_Step11_AsFunctionTool.csproj" />
|
||||
<Project Path="samples/02-agents/FoundryAgents/FoundryAgents_Step12_Middleware/FoundryAgents_Step12_Middleware.csproj" />
|
||||
<Project Path="samples/02-agents/FoundryAgents/FoundryAgents_Step13_Plugins/FoundryAgents_Step13_Plugins.csproj" />
|
||||
<Project Path="samples/02-agents/FoundryAgents/FoundryAgents_Step14_CodeInterpreter/FoundryAgents_Step14_CodeInterpreter.csproj" />
|
||||
<Project Path="samples/02-agents/FoundryAgents/FoundryAgents_Step15_ComputerUse/FoundryAgents_Step15_ComputerUse.csproj" />
|
||||
<Project Path="samples/02-agents/FoundryAgents/FoundryAgents_Step16_FileSearch/FoundryAgents_Step16_FileSearch.csproj" />
|
||||
<Project Path="samples/02-agents/FoundryAgents/FoundryAgents_Step17_OpenAPITools/FoundryAgents_Step17_OpenAPITools.csproj" />
|
||||
<Project Path="samples/02-agents/FoundryAgents/FoundryAgents_Step18_BingCustomSearch/FoundryAgents_Step18_BingCustomSearch.csproj" />
|
||||
<Project Path="samples/02-agents/FoundryAgents/FoundryAgents_Step19_SharePoint/FoundryAgents_Step19_SharePoint.csproj" />
|
||||
<Project Path="samples/02-agents/FoundryAgents/FoundryAgents_Step20_MicrosoftFabric/FoundryAgents_Step20_MicrosoftFabric.csproj" />
|
||||
<Project Path="samples/02-agents/FoundryAgents/FoundryAgents_Step21_WebSearch/FoundryAgents_Step21_WebSearch.csproj" />
|
||||
<Project Path="samples/02-agents/FoundryAgents/FoundryAgents_Step22_MemorySearch/FoundryAgents_Step22_MemorySearch.csproj" />
|
||||
<Project Path="samples/02-agents/FoundryAgents/FoundryAgents_Step23_LocalMCP/FoundryAgents_Step23_LocalMCP.csproj" />
|
||||
<Folder Name="/Samples/GettingStarted/AgentWithMemory/">
|
||||
<File Path="samples/GettingStarted/AgentWithMemory/README.md" />
|
||||
<Project Path="samples/GettingStarted/AgentWithMemory/AgentWithMemory_Step01_ChatHistoryMemory/AgentWithMemory_Step01_ChatHistoryMemory.csproj" />
|
||||
<Project Path="samples/GettingStarted/AgentWithMemory/AgentWithMemory_Step02_MemoryUsingMem0/AgentWithMemory_Step02_MemoryUsingMem0.csproj" />
|
||||
<Project Path="samples/GettingStarted/AgentWithMemory/AgentWithMemory_Step03_CustomMemory/AgentWithMemory_Step03_CustomMemory.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/02-agents/ModelContextProtocol/">
|
||||
<File Path="samples/02-agents/ModelContextProtocol/README.md" />
|
||||
<Project Path="samples/02-agents/ModelContextProtocol/Agent_MCP_Server/Agent_MCP_Server.csproj" />
|
||||
<Project Path="samples/02-agents/ModelContextProtocol/Agent_MCP_Server_Auth/Agent_MCP_Server_Auth.csproj" />
|
||||
<Project Path="samples/02-agents/ModelContextProtocol/FoundryAgent_Hosted_MCP/FoundryAgent_Hosted_MCP.csproj" />
|
||||
<Project Path="samples/02-agents/ModelContextProtocol/ResponseAgent_Hosted_MCP/ResponseAgent_Hosted_MCP.csproj" />
|
||||
<Folder Name="/Samples/GettingStarted/AgentWithOpenAI/">
|
||||
<File Path="samples/GettingStarted/AgentWithOpenAI/README.md" />
|
||||
<Project Path="samples/GettingStarted/AgentWithOpenAI/Agent_OpenAI_Step01_Running/Agent_OpenAI_Step01_Running.csproj" />
|
||||
<Project Path="samples/GettingStarted/AgentWithOpenAI/Agent_OpenAI_Step02_Reasoning/Agent_OpenAI_Step02_Reasoning.csproj" />
|
||||
<Project Path="samples/GettingStarted/AgentWithOpenAI/Agent_OpenAI_Step03_CreateFromChatClient/Agent_OpenAI_Step03_CreateFromChatClient.csproj" />
|
||||
<Project Path="samples/GettingStarted/AgentWithOpenAI/Agent_OpenAI_Step04_CreateFromOpenAIResponseClient/Agent_OpenAI_Step04_CreateFromOpenAIResponseClient.csproj" />
|
||||
<Project Path="samples/GettingStarted/AgentWithOpenAI/Agent_OpenAI_Step05_Conversation/Agent_OpenAI_Step05_Conversation.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/02-agents/Observability/">
|
||||
<Project Path="samples/02-agents/AgentOpenTelemetry/AgentOpenTelemetry.csproj" />
|
||||
<Folder Name="/Samples/Purview/" />
|
||||
<Folder Name="/Samples/Purview/AgentWithPurview/">
|
||||
<Project Path="samples/Purview/AgentWithPurview/AgentWithPurview.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/03-workflows/">
|
||||
<File Path="samples/03-workflows/README.md" />
|
||||
<Folder Name="/Samples/GettingStarted/AgentWithRAG/">
|
||||
<File Path="samples/GettingStarted/AgentWithRAG/README.md" />
|
||||
<Project Path="samples/GettingStarted/AgentWithRAG/AgentWithRAG_Step01_BasicTextRAG/AgentWithRAG_Step01_BasicTextRAG.csproj" />
|
||||
<Project Path="samples/GettingStarted/AgentWithRAG/AgentWithRAG_Step02_CustomVectorStoreRAG/AgentWithRAG_Step02_CustomVectorStoreRAG.csproj" />
|
||||
<Project Path="samples/GettingStarted/AgentWithRAG/AgentWithRAG_Step03_CustomRAGDataSource/AgentWithRAG_Step03_CustomRAGDataSource.csproj" />
|
||||
<Project Path="samples/GettingStarted/AgentWithRAG/AgentWithRAG_Step04_FoundryServiceRAG/AgentWithRAG_Step04_FoundryServiceRAG.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/03-workflows/Concurrent/">
|
||||
<Project Path="samples/03-workflows/Concurrent/Concurrent/Concurrent.csproj" />
|
||||
<Project Path="samples/03-workflows/Concurrent/MapReduce/MapReduce.csproj" />
|
||||
<Folder Name="/Samples/GettingStarted/FoundryAgents/">
|
||||
<File Path="samples/GettingStarted/FoundryAgents/README.md" />
|
||||
<Project Path="samples/GettingStarted/FoundryAgents/FoundryAgents_Step01.1_Basics/FoundryAgents_Step01.1_Basics.csproj" />
|
||||
<Project Path="samples/GettingStarted/FoundryAgents/FoundryAgents_Step01.2_Running/FoundryAgents_Step01.2_Running.csproj" />
|
||||
<Project Path="samples/GettingStarted/FoundryAgents/FoundryAgents_Step02_MultiturnConversation/FoundryAgents_Step02_MultiturnConversation.csproj" />
|
||||
<Project Path="samples/GettingStarted/FoundryAgents/FoundryAgents_Step03_UsingFunctionTools/FoundryAgents_Step03_UsingFunctionTools.csproj" />
|
||||
<Project Path="samples/GettingStarted/FoundryAgents/FoundryAgents_Step04_UsingFunctionToolsWithApprovals/FoundryAgents_Step04_UsingFunctionToolsWithApprovals.csproj" />
|
||||
<Project Path="samples/GettingStarted/FoundryAgents/FoundryAgents_Step05_StructuredOutput/FoundryAgents_Step05_StructuredOutput.csproj" />
|
||||
<Project Path="samples/GettingStarted/FoundryAgents/FoundryAgents_Step06_PersistedConversations/FoundryAgents_Step06_PersistedConversations.csproj" />
|
||||
<Project Path="samples/GettingStarted/FoundryAgents/FoundryAgents_Step07_Observability/FoundryAgents_Step07_Observability.csproj" />
|
||||
<Project Path="samples/GettingStarted/FoundryAgents/FoundryAgents_Step08_DependencyInjection/FoundryAgents_Step08_DependencyInjection.csproj" />
|
||||
<Project Path="samples/GettingStarted/FoundryAgents/FoundryAgents_Step09_UsingMcpClientAsTools/FoundryAgents_Step09_UsingMcpClientAsTools.csproj" />
|
||||
<Project Path="samples/GettingStarted/FoundryAgents/FoundryAgents_Step10_UsingImages/FoundryAgents_Step10_UsingImages.csproj" />
|
||||
<Project Path="samples/GettingStarted/FoundryAgents/FoundryAgents_Step11_AsFunctionTool/FoundryAgents_Step11_AsFunctionTool.csproj" />
|
||||
<Project Path="samples/GettingStarted/FoundryAgents/FoundryAgents_Step12_Middleware/FoundryAgents_Step12_Middleware.csproj" />
|
||||
<Project Path="samples/GettingStarted/FoundryAgents/FoundryAgents_Step13_Plugins/FoundryAgents_Step13_Plugins.csproj" />
|
||||
<Project Path="samples/GettingStarted/FoundryAgents/FoundryAgents_Step14_CodeInterpreter/FoundryAgents_Step14_CodeInterpreter.csproj" />
|
||||
<Project Path="samples/GettingStarted/FoundryAgents/FoundryAgents_Step15_ComputerUse/FoundryAgents_Step15_ComputerUse.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/03-workflows/ConditionalEdges/">
|
||||
<Project Path="samples/03-workflows/ConditionalEdges/01_EdgeCondition/01_EdgeCondition.csproj" />
|
||||
<Project Path="samples/03-workflows/ConditionalEdges/02_SwitchCase/02_SwitchCase.csproj" />
|
||||
<Project Path="samples/03-workflows/ConditionalEdges/03_MultiSelection/03_MultiSelection.csproj" />
|
||||
<Folder Name="/Samples/GettingStarted/ModelContextProtocol/">
|
||||
<File Path="samples/GettingStarted/ModelContextProtocol/README.md" />
|
||||
<Project Path="samples/GettingStarted/ModelContextProtocol/Agent_MCP_Server/Agent_MCP_Server.csproj" />
|
||||
<Project Path="samples/GettingStarted/ModelContextProtocol/Agent_MCP_Server_Auth/Agent_MCP_Server_Auth.csproj" />
|
||||
<Project Path="samples/GettingStarted/ModelContextProtocol/FoundryAgent_Hosted_MCP/FoundryAgent_Hosted_MCP.csproj" />
|
||||
<Project Path="samples/GettingStarted/ModelContextProtocol/ResponseAgent_Hosted_MCP/ResponseAgent_Hosted_MCP.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/03-workflows/Declarative/">
|
||||
<File Path="samples/03-workflows/Declarative/README.md" />
|
||||
<Project Path="samples/03-workflows/Declarative/ConfirmInput/ConfirmInput.csproj" />
|
||||
<Project Path="samples/03-workflows/Declarative/CustomerSupport/CustomerSupport.csproj" />
|
||||
<Project Path="samples/03-workflows/Declarative/DeepResearch/DeepResearch.csproj" />
|
||||
<Project Path="samples/03-workflows/Declarative/ExecuteCode/ExecuteCode.csproj" />
|
||||
<Project Path="samples/03-workflows/Declarative/ExecuteWorkflow/ExecuteWorkflow.csproj" />
|
||||
<Project Path="samples/03-workflows/Declarative/FunctionTools/FunctionTools.csproj" />
|
||||
<Project Path="samples/03-workflows/Declarative/GenerateCode/GenerateCode.csproj" />
|
||||
<Project Path="samples/03-workflows/Declarative/HostedWorkflow/HostedWorkflow.csproj" />
|
||||
<Project Path="samples/03-workflows/Declarative/InputArguments/InputArguments.csproj" />
|
||||
<Project Path="samples/03-workflows/Declarative/InvokeFunctionTool/InvokeFunctionTool.csproj" />
|
||||
<Project Path="samples/03-workflows/Declarative/InvokeMcpTool/InvokeMcpTool.csproj" />
|
||||
<Project Path="samples/03-workflows/Declarative/Marketing/Marketing.csproj" />
|
||||
<Project Path="samples/03-workflows/Declarative/StudentTeacher/StudentTeacher.csproj" />
|
||||
<Project Path="samples/03-workflows/Declarative/ToolApproval/ToolApproval.csproj" />
|
||||
<Folder Name="/Samples/GettingStarted/Observability/">
|
||||
<Project Path="samples/GettingStarted/AgentOpenTelemetry/AgentOpenTelemetry.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/03-workflows/Declarative/Examples/">
|
||||
<Folder Name="/Samples/GettingStarted/Workflows/">
|
||||
<File Path="samples/GettingStarted/Workflows/README.md" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/GettingStarted/Workflows/Concurrent/">
|
||||
<Project Path="samples/GettingStarted/Workflows/Concurrent/Concurrent/Concurrent.csproj" />
|
||||
<Project Path="samples/GettingStarted/Workflows/Concurrent/MapReduce/MapReduce.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/GettingStarted/Workflows/ConditionalEdges/">
|
||||
<Project Path="samples/GettingStarted/Workflows/ConditionalEdges/01_EdgeCondition/01_EdgeCondition.csproj" />
|
||||
<Project Path="samples/GettingStarted/Workflows/ConditionalEdges/02_SwitchCase/02_SwitchCase.csproj" />
|
||||
<Project Path="samples/GettingStarted/Workflows/ConditionalEdges/03_MultiSelection/03_MultiSelection.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/GettingStarted/Workflows/Declarative/">
|
||||
<File Path="samples/GettingStarted/Workflows/Declarative/README.md" />
|
||||
<Project Path="samples/GettingStarted/Workflows/Declarative/ConfirmInput/ConfirmInput.csproj" />
|
||||
<Project Path="samples/GettingStarted/Workflows/Declarative/CustomerSupport/CustomerSupport.csproj" />
|
||||
<Project Path="samples/GettingStarted/Workflows/Declarative/DeepResearch/DeepResearch.csproj" />
|
||||
<Project Path="samples/GettingStarted/Workflows/Declarative/ExecuteCode/ExecuteCode.csproj" />
|
||||
<Project Path="samples/GettingStarted/Workflows/Declarative/ExecuteWorkflow/ExecuteWorkflow.csproj" />
|
||||
<Project Path="samples/GettingStarted/Workflows/Declarative/FunctionTools/FunctionTools.csproj" />
|
||||
<Project Path="samples/GettingStarted/Workflows/Declarative/GenerateCode/GenerateCode.csproj" />
|
||||
<Project Path="samples/GettingStarted/Workflows/Declarative/HostedWorkflow/HostedWorkflow.csproj" />
|
||||
<Project Path="samples/GettingStarted/Workflows/Declarative/InputArguments/InputArguments.csproj" />
|
||||
<Project Path="samples/GettingStarted/Workflows/Declarative/Marketing/Marketing.csproj" />
|
||||
<Project Path="samples/GettingStarted/Workflows/Declarative/StudentTeacher/StudentTeacher.csproj" />
|
||||
<Project Path="samples/GettingStarted/Workflows/Declarative/ToolApproval/ToolApproval.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/GettingStarted/Workflows/Declarative/Examples/">
|
||||
<File Path="../workflow-samples/CustomerSupport.yaml" />
|
||||
<File Path="../workflow-samples/DeepResearch.yaml" />
|
||||
<File Path="../workflow-samples/Marketing.yaml" />
|
||||
@@ -216,118 +235,60 @@
|
||||
<File Path="../workflow-samples/README.md" />
|
||||
<File Path="../workflow-samples/wttr.json" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/03-workflows/SharedStates/">
|
||||
<Project Path="samples/03-workflows/SharedStates/SharedStates.csproj" />
|
||||
<Folder Name="/Samples/GettingStarted/Workflows/SharedStates/">
|
||||
<Project Path="samples/GettingStarted/Workflows/SharedStates/SharedStates.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/03-workflows/Loop/">
|
||||
<Project Path="samples/03-workflows/Loop/Loop.csproj" />
|
||||
<Folder Name="/Samples/GettingStarted/Workflows/Loop/">
|
||||
<Project Path="samples/GettingStarted/Workflows/Loop/Loop.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/03-workflows/Agents/">
|
||||
<Project Path="samples/03-workflows/Agents/CustomAgentExecutors/CustomAgentExecutors.csproj" />
|
||||
<Project Path="samples/03-workflows/Agents/FoundryAgent/FoundryAgent.csproj" />
|
||||
<Project Path="samples/03-workflows/Agents/GroupChatToolApproval/GroupChatToolApproval.csproj" />
|
||||
<Project Path="samples/03-workflows/Agents/WorkflowAsAnAgent/WorkflowAsAnAgent.csproj" />
|
||||
<Folder Name="/Samples/GettingStarted/Workflows/Agents/">
|
||||
<Project Path="samples/GettingStarted/Workflows/Agents/CustomAgentExecutors/CustomAgentExecutors.csproj" />
|
||||
<Project Path="samples/GettingStarted/Workflows/Agents/FoundryAgent/FoundryAgent.csproj" />
|
||||
<Project Path="samples/GettingStarted/Workflows/Agents/GroupChatToolApproval/GroupChatToolApproval.csproj" />
|
||||
<Project Path="samples/GettingStarted/Workflows/Agents/WorkflowAsAnAgent/WorkflowAsAnAgent.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/03-workflows/Checkpoint/">
|
||||
<Project Path="samples/03-workflows/Checkpoint/CheckpointAndRehydrate/CheckpointAndRehydrate.csproj" />
|
||||
<Project Path="samples/03-workflows/Checkpoint/CheckpointAndResume/CheckpointAndResume.csproj" />
|
||||
<Project Path="samples/03-workflows/Checkpoint/CheckpointWithHumanInTheLoop/CheckpointWithHumanInTheLoop.csproj" />
|
||||
<Folder Name="/Samples/GettingStarted/Workflows/Checkpoint/">
|
||||
<Project Path="samples/GettingStarted/Workflows/Checkpoint/CheckpointAndRehydrate/CheckpointAndRehydrate.csproj" />
|
||||
<Project Path="samples/GettingStarted/Workflows/Checkpoint/CheckpointAndResume/CheckpointAndResume.csproj" />
|
||||
<Project Path="samples/GettingStarted/Workflows/Checkpoint/CheckpointWithHumanInTheLoop/CheckpointWithHumanInTheLoop.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/03-workflows/HumanInTheLoop/">
|
||||
<Project Path="samples/03-workflows/HumanInTheLoop/HumanInTheLoopBasic/HumanInTheLoopBasic.csproj" />
|
||||
<Folder Name="/Samples/GettingStarted/Workflows/HumanInTheLoop/">
|
||||
<Project Path="samples/GettingStarted/Workflows/HumanInTheLoop/HumanInTheLoopBasic/HumanInTheLoopBasic.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/03-workflows/Observability/">
|
||||
<Project Path="samples/03-workflows/Observability/ApplicationInsights/ApplicationInsights.csproj" />
|
||||
<Project Path="samples/03-workflows/Observability/AspireDashboard/AspireDashboard.csproj" />
|
||||
<Project Path="samples/03-workflows/Observability/WorkflowAsAnAgent/WorkflowAsAnAgentObservability.csproj" />
|
||||
<Folder Name="/Samples/GettingStarted/Workflows/Observability/">
|
||||
<Project Path="samples/GettingStarted/Workflows/Observability/ApplicationInsights/ApplicationInsights.csproj" />
|
||||
<Project Path="samples/GettingStarted/Workflows/Observability/AspireDashboard/AspireDashboard.csproj" />
|
||||
<Project Path="samples/GettingStarted/Workflows/Observability/WorkflowAsAnAgent/WorkflowAsAnAgentObservability.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/03-workflows/Visualization/">
|
||||
<Project Path="samples/03-workflows/Visualization/Visualization.csproj" />
|
||||
<Folder Name="/Samples/GettingStarted/Workflows/Visualization/">
|
||||
<Project Path="samples/GettingStarted/Workflows/Visualization/Visualization.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/03-workflows/_StartHere/">
|
||||
<Project Path="samples/03-workflows/_StartHere/01_Streaming/01_Streaming.csproj" />
|
||||
<Project Path="samples/03-workflows/_StartHere/02_AgentsInWorkflows/02_AgentsInWorkflows.csproj" />
|
||||
<Project Path="samples/03-workflows/_StartHere/03_AgentWorkflowPatterns/03_AgentWorkflowPatterns.csproj" />
|
||||
<Project Path="samples/03-workflows/_StartHere/04_MultiModelService/04_MultiModelService.csproj" />
|
||||
<Project Path="samples/03-workflows/_StartHere/05_SubWorkflows/05_SubWorkflows.csproj" />
|
||||
<Project Path="samples/03-workflows/_StartHere/06_MixedWorkflowAgentsAndExecutors/06_MixedWorkflowAgentsAndExecutors.csproj" />
|
||||
<Project Path="samples/03-workflows/_StartHere/07_WriterCriticWorkflow/07_WriterCriticWorkflow.csproj" />
|
||||
<Folder Name="/Samples/GettingStarted/Workflows/_Foundational/">
|
||||
<Project Path="samples/GettingStarted/Workflows/_Foundational/01_ExecutorsAndEdges/01_ExecutorsAndEdges.csproj" />
|
||||
<Project Path="samples/GettingStarted/Workflows/_Foundational/02_Streaming/02_Streaming.csproj" />
|
||||
<Project Path="samples/GettingStarted/Workflows/_Foundational/03_AgentsInWorkflows/03_AgentsInWorkflows.csproj" />
|
||||
<Project Path="samples/GettingStarted/Workflows/_Foundational/04_AgentWorkflowPatterns/04_AgentWorkflowPatterns.csproj" />
|
||||
<Project Path="samples/GettingStarted/Workflows/_Foundational/05_MultiModelService/05_MultiModelService.csproj" />
|
||||
<Project Path="samples/GettingStarted/Workflows/_Foundational/06_SubWorkflows/06_SubWorkflows.csproj" />
|
||||
<Project Path="samples/GettingStarted/Workflows/_Foundational/07_MixedWorkflowAgentsAndExecutors/07_MixedWorkflowAgentsAndExecutors.csproj" />
|
||||
<Project Path="samples/GettingStarted/Workflows/_Foundational/08_WriterCriticWorkflow/08_WriterCriticWorkflow.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/04-hosting/" />
|
||||
<Folder Name="/Samples/04-hosting/DurableAgents/" />
|
||||
<Folder Name="/Samples/04-hosting/DurableAgents/AzureFunctions/">
|
||||
<File Path="samples/04-hosting/DurableAgents/AzureFunctions/.editorconfig" />
|
||||
<File Path="samples/04-hosting/DurableAgents/AzureFunctions/README.md" />
|
||||
<Project Path="samples/04-hosting/DurableAgents/AzureFunctions/01_SingleAgent/01_SingleAgent.csproj" />
|
||||
<Project Path="samples/04-hosting/DurableAgents/AzureFunctions/02_AgentOrchestration_Chaining/02_AgentOrchestration_Chaining.csproj" />
|
||||
<Project Path="samples/04-hosting/DurableAgents/AzureFunctions/03_AgentOrchestration_Concurrency/03_AgentOrchestration_Concurrency.csproj" />
|
||||
<Project Path="samples/04-hosting/DurableAgents/AzureFunctions/04_AgentOrchestration_Conditionals/04_AgentOrchestration_Conditionals.csproj" />
|
||||
<Project Path="samples/04-hosting/DurableAgents/AzureFunctions/05_AgentOrchestration_HITL/05_AgentOrchestration_HITL.csproj" />
|
||||
<Project Path="samples/04-hosting/DurableAgents/AzureFunctions/06_LongRunningTools/06_LongRunningTools.csproj" />
|
||||
<Project Path="samples/04-hosting/DurableAgents/AzureFunctions/07_AgentAsMcpTool/07_AgentAsMcpTool.csproj" />
|
||||
<Project Path="samples/04-hosting/DurableAgents/AzureFunctions/08_ReliableStreaming/08_ReliableStreaming.csproj" />
|
||||
<Folder Name="/Samples/HostedAgents/">
|
||||
<Project Path="samples/HostedAgents/AgentsInWorkflows/AgentsInWorkflows.csproj" />
|
||||
<Project Path="samples/HostedAgents/AgentWithHostedMCP/AgentWithHostedMCP.csproj" />
|
||||
<Project Path="samples/HostedAgents/AgentWithTextSearchRag/AgentWithTextSearchRag.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/04-hosting/DurableAgents/ConsoleApps/">
|
||||
<File Path="samples/04-hosting/DurableAgents/ConsoleApps/README.md" />
|
||||
<Project Path="samples/04-hosting/DurableAgents/ConsoleApps/01_SingleAgent/01_SingleAgent.csproj" />
|
||||
<Project Path="samples/04-hosting/DurableAgents/ConsoleApps/02_AgentOrchestration_Chaining/02_AgentOrchestration_Chaining.csproj" />
|
||||
<Project Path="samples/04-hosting/DurableAgents/ConsoleApps/03_AgentOrchestration_Concurrency/03_AgentOrchestration_Concurrency.csproj" />
|
||||
<Project Path="samples/04-hosting/DurableAgents/ConsoleApps/04_AgentOrchestration_Conditionals/04_AgentOrchestration_Conditionals.csproj" />
|
||||
<Project Path="samples/04-hosting/DurableAgents/ConsoleApps/05_AgentOrchestration_HITL/05_AgentOrchestration_HITL.csproj" />
|
||||
<Project Path="samples/04-hosting/DurableAgents/ConsoleApps/06_LongRunningTools/06_LongRunningTools.csproj" />
|
||||
<Project Path="samples/04-hosting/DurableAgents/ConsoleApps/07_ReliableStreaming/07_ReliableStreaming.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/04-hosting/A2A/">
|
||||
<File Path="samples/04-hosting/A2A/README.md" />
|
||||
<Project Path="samples/04-hosting/A2A/A2AAgent_AsFunctionTools/A2AAgent_AsFunctionTools.csproj" />
|
||||
<Project Path="samples/04-hosting/A2A/A2AAgent_PollingForTaskCompletion/A2AAgent_PollingForTaskCompletion.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/05-end-to-end/">
|
||||
<Project Path="samples/05-end-to-end/AgentWithPurview/AgentWithPurview.csproj" />
|
||||
<Project Path="samples/05-end-to-end/M365Agent/M365Agent.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/05-end-to-end/A2AClientServer/">
|
||||
<File Path="samples/05-end-to-end/A2AClientServer/README.md" />
|
||||
<Project Path="samples/05-end-to-end/A2AClientServer/A2AClient/A2AClient.csproj" />
|
||||
<Project Path="samples/05-end-to-end/A2AClientServer/A2AServer/A2AServer.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/05-end-to-end/AgentWebChat/">
|
||||
<Project Path="samples/05-end-to-end/AgentWebChat/AgentWebChat.AgentHost/AgentWebChat.AgentHost.csproj" />
|
||||
<Project Path="samples/05-end-to-end/AgentWebChat/AgentWebChat.AppHost/AgentWebChat.AppHost.csproj" />
|
||||
<Project Path="samples/05-end-to-end/AgentWebChat/AgentWebChat.ServiceDefaults/AgentWebChat.ServiceDefaults.csproj" />
|
||||
<Project Path="samples/05-end-to-end/AgentWebChat/AgentWebChat.Web/AgentWebChat.Web.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/05-end-to-end/AGUIClientServer/">
|
||||
<File Path="samples/05-end-to-end/AGUIClientServer/README.md" />
|
||||
<Project Path="samples/05-end-to-end/AGUIClientServer/AGUIClient/AGUIClient.csproj" />
|
||||
<Project Path="samples/05-end-to-end/AGUIClientServer/AGUIDojoServer/AGUIDojoServer.csproj" />
|
||||
<Project Path="samples/05-end-to-end/AGUIClientServer/AGUIServer/AGUIServer.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/05-end-to-end/HostedAgents/">
|
||||
<Project Path="samples/05-end-to-end/HostedAgents/AgentsInWorkflows/AgentsInWorkflows.csproj" />
|
||||
<Project Path="samples/05-end-to-end/HostedAgents/AgentThreadAndHITL/AgentThreadAndHITL.csproj" />
|
||||
<Project Path="samples/05-end-to-end/HostedAgents/AgentWithHostedMCP/AgentWithHostedMCP.csproj" />
|
||||
<Project Path="samples/05-end-to-end/HostedAgents/AgentWithLocalTools/AgentWithLocalTools.csproj" />
|
||||
<Project Path="samples/05-end-to-end/HostedAgents/AgentWithTextSearchRag/AgentWithTextSearchRag.csproj" />
|
||||
<Project Path="samples/05-end-to-end/HostedAgents/FoundryMultiAgent/FoundryMultiAgent.csproj" />
|
||||
<Project Path="samples/05-end-to-end/HostedAgents/FoundrySingleAgent/FoundrySingleAgent.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Samples/05-end-to-end/AspNetAgentAuthorization/">
|
||||
<File Path="samples/05-end-to-end/AspNetAgentAuthorization/docker-compose.yml" />
|
||||
<File Path="samples/05-end-to-end/AspNetAgentAuthorization/README.md" />
|
||||
<Project Path="samples/05-end-to-end/AspNetAgentAuthorization/Service/Service.csproj" />
|
||||
<Project Path="samples/05-end-to-end/AspNetAgentAuthorization/RazorWebClient/RazorWebClient.csproj" />
|
||||
<Folder Name="/Samples/M365Agent/">
|
||||
<Project Path="samples/M365Agent/M365Agent.csproj" />
|
||||
</Folder>
|
||||
<Folder Name="/Solution Items/">
|
||||
<File Path=".editorconfig" />
|
||||
<File Path=".gitignore" />
|
||||
<File Path="AGENTS.md" />
|
||||
<File Path="Directory.Build.props" />
|
||||
<File Path="Directory.Build.targets" />
|
||||
<File Path="Directory.Packages.props" />
|
||||
<File Path="global.json" />
|
||||
<File Path="nuget.config" />
|
||||
<File Path="README.md" />
|
||||
</Folder>
|
||||
<Folder Name="/Solution Items/.github/" />
|
||||
<Folder Name="/Solution Items/.github/upgrades/" />
|
||||
@@ -336,6 +297,7 @@
|
||||
</Folder>
|
||||
<Folder Name="/Solution Items/.github/workflows/">
|
||||
<File Path="../.github/workflows/dotnet-build-and-test.yml" />
|
||||
<File Path="../.github/workflows/dotnet-check-coverage.ps1" />
|
||||
<File Path="../.github/workflows/dotnet-format.yml" />
|
||||
</Folder>
|
||||
<Folder Name="/Solution Items/demos/">
|
||||
@@ -358,10 +320,6 @@
|
||||
<File Path="../docs/decisions/0012-python-typeddict-options.md" />
|
||||
<File Path="../docs/decisions/0013-python-get-response-simplification.md" />
|
||||
<File Path="../docs/decisions/0014-feature-collections.md" />
|
||||
<File Path="../docs/decisions/0015-agent-run-context.md" />
|
||||
<File Path="../docs/decisions/0016-python-context-middleware.md" />
|
||||
<File Path="../docs/decisions/0017-agent-additional-properties.md" />
|
||||
<File Path="../docs/decisions/0018-agentthread-serialization.md" />
|
||||
<File Path="../docs/decisions/adr-short-template.md" />
|
||||
<File Path="../docs/decisions/adr-template.md" />
|
||||
<File Path="../docs/decisions/README.md" />
|
||||
@@ -372,10 +330,6 @@
|
||||
<File Path="eng/MSBuild/Shared.props" />
|
||||
<File Path="eng/MSBuild/Shared.targets" />
|
||||
</Folder>
|
||||
<Folder Name="/Solution Items/eng/scripts/">
|
||||
<File Path="eng/scripts/dotnet-check-coverage.ps1" />
|
||||
<File Path="eng/scripts/New-FilteredSolution.ps1" />
|
||||
</Folder>
|
||||
<Folder Name="/Solution Items/nuget/">
|
||||
<File Path="nuget/icon.png" />
|
||||
<File Path="nuget/nuget-package.props" />
|
||||
@@ -430,10 +384,6 @@
|
||||
<File Path="src/Shared/Demos/README.md" />
|
||||
<File Path="src/Shared/Demos/SampleEnvironment.cs" />
|
||||
</Folder>
|
||||
<Folder Name="/Solution Items/src/Shared/DiagnosticIds/">
|
||||
<File Path="src/Shared/DiagnosticIds/DiagnosticsIds.cs" />
|
||||
<File Path="src/Shared/DiagnosticIds/README.md" />
|
||||
</Folder>
|
||||
<Folder Name="/Solution Items/src/Shared/IntegrationTests/">
|
||||
<File Path="src/Shared/IntegrationTests/AnthropicConfiguration.cs" />
|
||||
<File Path="src/Shared/IntegrationTests/AzureAIConfiguration.cs" />
|
||||
@@ -441,10 +391,6 @@
|
||||
<File Path="src/Shared/IntegrationTests/OpenAIConfiguration.cs" />
|
||||
<File Path="src/Shared/IntegrationTests/README.md" />
|
||||
</Folder>
|
||||
<Folder Name="/Solution Items/src/Shared/IntegrationTestsAzureCredentials/">
|
||||
<File Path="src/Shared/IntegrationTestsAzureCredentials/README.md" />
|
||||
<File Path="src/Shared/IntegrationTestsAzureCredentials/TestAzureCliCredentials.cs" />
|
||||
</Folder>
|
||||
<Folder Name="/Solution Items/src/Shared/Samples/">
|
||||
<File Path="src/Shared/Samples/BaseSample.cs" />
|
||||
<File Path="src/Shared/Samples/README.md" />
|
||||
@@ -452,17 +398,10 @@
|
||||
<File Path="src/Shared/Samples/TextOutputHelperExtensions.cs" />
|
||||
<File Path="src/Shared/Samples/XunitLogger.cs" />
|
||||
</Folder>
|
||||
<Folder Name="/Solution Items/src/Shared/Redaction/">
|
||||
<File Path="src/Shared/Redaction/README.md" />
|
||||
<File Path="src/Shared/Redaction/ReplacingRedactor.cs" />
|
||||
</Folder>
|
||||
<Folder Name="/Solution Items/src/Shared/Throw/">
|
||||
<File Path="src/Shared/Throw/README.md" />
|
||||
<File Path="src/Shared/Throw/Throw.cs" />
|
||||
</Folder>
|
||||
<Folder Name="/Solution Items/src/Shared/StructuredOutput/">
|
||||
<File Path="src/Shared/StructuredOutput/StructuredOutputSchemaUtilities.cs" />
|
||||
</Folder>
|
||||
<Folder Name="/Solution Items/tests/">
|
||||
<File Path="tests/.editorconfig" />
|
||||
<File Path="tests/Directory.Build.props" />
|
||||
@@ -479,7 +418,6 @@
|
||||
<Project Path="src/Microsoft.Agents.AI.Declarative/Microsoft.Agents.AI.Declarative.csproj" />
|
||||
<Project Path="src/Microsoft.Agents.AI.DevUI/Microsoft.Agents.AI.DevUI.csproj" />
|
||||
<Project Path="src/Microsoft.Agents.AI.DurableTask/Microsoft.Agents.AI.DurableTask.csproj" />
|
||||
<Project Path="src/Microsoft.Agents.AI.FoundryMemory/Microsoft.Agents.AI.FoundryMemory.csproj" />
|
||||
<Project Path="src/Microsoft.Agents.AI.GitHub.Copilot/Microsoft.Agents.AI.GitHub.Copilot.csproj" />
|
||||
<Project Path="src/Microsoft.Agents.AI.Hosting.A2A.AspNetCore/Microsoft.Agents.AI.Hosting.A2A.AspNetCore.csproj" />
|
||||
<Project Path="src/Microsoft.Agents.AI.Hosting.A2A/Microsoft.Agents.AI.Hosting.A2A.csproj" />
|
||||
@@ -491,7 +429,6 @@
|
||||
<Project Path="src/Microsoft.Agents.AI.OpenAI/Microsoft.Agents.AI.OpenAI.csproj" />
|
||||
<Project Path="src/Microsoft.Agents.AI.Purview/Microsoft.Agents.AI.Purview.csproj" />
|
||||
<Project Path="src/Microsoft.Agents.AI.Workflows.Declarative.AzureAI/Microsoft.Agents.AI.Workflows.Declarative.AzureAI.csproj" />
|
||||
<Project Path="src/Microsoft.Agents.AI.Workflows.Declarative.Mcp/Microsoft.Agents.AI.Workflows.Declarative.Mcp.csproj" />
|
||||
<Project Path="src/Microsoft.Agents.AI.Workflows.Declarative/Microsoft.Agents.AI.Workflows.Declarative.csproj" />
|
||||
<Project Path="src/Microsoft.Agents.AI.Workflows.Generators/Microsoft.Agents.AI.Workflows.Generators.csproj" />
|
||||
<Project Path="src/Microsoft.Agents.AI.Workflows/Microsoft.Agents.AI.Workflows.csproj" />
|
||||
@@ -505,7 +442,6 @@
|
||||
<Project Path="tests/AzureAIAgentsPersistent.IntegrationTests/AzureAIAgentsPersistent.IntegrationTests.csproj" />
|
||||
<Project Path="tests/CopilotStudio.IntegrationTests/CopilotStudio.IntegrationTests.csproj" />
|
||||
<Project Path="tests/Microsoft.Agents.AI.DurableTask.IntegrationTests/Microsoft.Agents.AI.DurableTask.IntegrationTests.csproj" />
|
||||
<Project Path="tests/Microsoft.Agents.AI.FoundryMemory.IntegrationTests/Microsoft.Agents.AI.FoundryMemory.IntegrationTests.csproj" />
|
||||
<Project Path="tests/Microsoft.Agents.AI.GitHub.Copilot.IntegrationTests/Microsoft.Agents.AI.GitHub.Copilot.IntegrationTests.csproj" />
|
||||
<Project Path="tests/Microsoft.Agents.AI.Hosting.AGUI.AspNetCore.IntegrationTests/Microsoft.Agents.AI.Hosting.AGUI.AspNetCore.IntegrationTests.csproj" />
|
||||
<Project Path="tests/Microsoft.Agents.AI.Hosting.AzureFunctions.IntegrationTests/Microsoft.Agents.AI.Hosting.AzureFunctions.IntegrationTests.csproj" />
|
||||
@@ -526,7 +462,6 @@
|
||||
<Project Path="tests/Microsoft.Agents.AI.Declarative.UnitTests/Microsoft.Agents.AI.Declarative.UnitTests.csproj" />
|
||||
<Project Path="tests/Microsoft.Agents.AI.DevUI.UnitTests/Microsoft.Agents.AI.DevUI.UnitTests.csproj" />
|
||||
<Project Path="tests/Microsoft.Agents.AI.DurableTask.UnitTests/Microsoft.Agents.AI.DurableTask.UnitTests.csproj" />
|
||||
<Project Path="tests/Microsoft.Agents.AI.FoundryMemory.UnitTests/Microsoft.Agents.AI.FoundryMemory.UnitTests.csproj" />
|
||||
<Project Path="tests/Microsoft.Agents.AI.GitHub.Copilot.UnitTests/Microsoft.Agents.AI.GitHub.Copilot.UnitTests.csproj" />
|
||||
<Project Path="tests/Microsoft.Agents.AI.Hosting.A2A.UnitTests/Microsoft.Agents.AI.Hosting.A2A.UnitTests.csproj" />
|
||||
<Project Path="tests/Microsoft.Agents.AI.Hosting.AGUI.AspNetCore.UnitTests/Microsoft.Agents.AI.Hosting.AGUI.AspNetCore.UnitTests.csproj" />
|
||||
@@ -537,9 +472,8 @@
|
||||
<Project Path="tests/Microsoft.Agents.AI.OpenAI.UnitTests/Microsoft.Agents.AI.OpenAI.UnitTests.csproj" />
|
||||
<Project Path="tests/Microsoft.Agents.AI.Purview.UnitTests/Microsoft.Agents.AI.Purview.UnitTests.csproj" />
|
||||
<Project Path="tests/Microsoft.Agents.AI.UnitTests/Microsoft.Agents.AI.UnitTests.csproj" />
|
||||
<Project Path="tests/Microsoft.Agents.AI.Workflows.Declarative.Mcp.UnitTests/Microsoft.Agents.AI.Workflows.Declarative.Mcp.UnitTests.csproj" />
|
||||
<Project Path="tests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests/Microsoft.Agents.AI.Workflows.Declarative.UnitTests.csproj" />
|
||||
<Project Path="tests/Microsoft.Agents.AI.Workflows.Generators.UnitTests/Microsoft.Agents.AI.Workflows.Generators.UnitTests.csproj" />
|
||||
<Project Path="tests/Microsoft.Agents.AI.Workflows.UnitTests/Microsoft.Agents.AI.Workflows.UnitTests.csproj" />
|
||||
</Folder>
|
||||
</Solution>
|
||||
</Solution>
|
||||
@@ -14,7 +14,6 @@
|
||||
"src\\Microsoft.Agents.AI.Declarative\\Microsoft.Agents.AI.Declarative.csproj",
|
||||
"src\\Microsoft.Agents.AI.DevUI\\Microsoft.Agents.AI.DevUI.csproj",
|
||||
"src\\Microsoft.Agents.AI.DurableTask\\Microsoft.Agents.AI.DurableTask.csproj",
|
||||
"src\\Microsoft.Agents.AI.FoundryMemory\\Microsoft.Agents.AI.FoundryMemory.csproj",
|
||||
"src\\Microsoft.Agents.AI.Hosting.A2A.AspNetCore\\Microsoft.Agents.AI.Hosting.A2A.AspNetCore.csproj",
|
||||
"src\\Microsoft.Agents.AI.Hosting.A2A\\Microsoft.Agents.AI.Hosting.A2A.csproj",
|
||||
"src\\Microsoft.Agents.AI.Hosting.AGUI.AspNetCore\\Microsoft.Agents.AI.Hosting.AGUI.AspNetCore.csproj",
|
||||
@@ -26,7 +25,6 @@
|
||||
"src\\Microsoft.Agents.AI.Purview\\Microsoft.Agents.AI.Purview.csproj",
|
||||
"src\\Microsoft.Agents.AI.Workflows.Declarative.AzureAI\\Microsoft.Agents.AI.Workflows.Declarative.AzureAI.csproj",
|
||||
"src\\Microsoft.Agents.AI.Workflows.Declarative\\Microsoft.Agents.AI.Workflows.Declarative.csproj",
|
||||
"src\\Microsoft.Agents.AI.Workflows.Generators\\Microsoft.Agents.AI.Workflows.Generators.csproj",
|
||||
"src\\Microsoft.Agents.AI.Workflows\\Microsoft.Agents.AI.Workflows.csproj",
|
||||
"src\\Microsoft.Agents.AI\\Microsoft.Agents.AI.csproj"
|
||||
]
|
||||
|
||||
@@ -8,9 +8,6 @@
|
||||
<ItemGroup Condition="'$(InjectSharedIntegrationTestCode)' == 'true'">
|
||||
<Compile Include="$(MSBuildThisFileDirectory)\..\..\src\Shared\IntegrationTests\*.cs" LinkBase="Shared\IntegrationTests" />
|
||||
</ItemGroup>
|
||||
<ItemGroup Condition="'$(InjectSharedIntegrationTestAzureCredentialsCode)' == 'true'">
|
||||
<Compile Include="$(MSBuildThisFileDirectory)\..\..\src\Shared\IntegrationTestsAzureCredentials\*.cs" LinkBase="Shared\IntegrationTestsAzureCredentials" />
|
||||
</ItemGroup>
|
||||
<ItemGroup Condition="'$(InjectSharedBuildTestCode)' == 'true'">
|
||||
<Compile Include="$(MSBuildThisFileDirectory)\..\..\src\Shared\CodeTests\*.cs" LinkBase="Shared\CodeTests" />
|
||||
</ItemGroup>
|
||||
@@ -23,13 +20,4 @@
|
||||
<ItemGroup Condition="'$(InjectSharedFoundryAgents)' == 'true'">
|
||||
<Compile Include="$(MSBuildThisFileDirectory)\..\..\src\Shared\Foundry\Agents\*.cs" LinkBase="Shared\Foundry" />
|
||||
</ItemGroup>
|
||||
<ItemGroup Condition="'$(InjectSharedStructuredOutput)' == 'true'">
|
||||
<Compile Include="$(MSBuildThisFileDirectory)\..\..\src\Shared\StructuredOutput\*.cs" LinkBase="Shared\StructuredOutput" />
|
||||
</ItemGroup>
|
||||
<ItemGroup Condition="'$(InjectSharedDiagnosticIds)' == 'true'">
|
||||
<Compile Include="$(MSBuildThisFileDirectory)\..\..\src\Shared\DiagnosticIds\*.cs" LinkBase="Shared\DiagnosticIds" />
|
||||
</ItemGroup>
|
||||
<ItemGroup Condition="'$(InjectSharedRedaction)' == 'true'">
|
||||
<Compile Include="$(MSBuildThisFileDirectory)\..\..\src\Shared\Redaction\*.cs" LinkBase="Shared\Redaction" />
|
||||
</ItemGroup>
|
||||
</Project>
|
||||
|
||||
@@ -1,145 +0,0 @@
|
||||
#!/usr/bin/env pwsh
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Generates a filtered .slnx solution file by removing projects that don't match the specified criteria.
|
||||
|
||||
.DESCRIPTION
|
||||
Parses a .slnx solution file and applies one or more filters:
|
||||
- Removes projects that don't support the specified target framework (via MSBuild query).
|
||||
- Optionally removes all sample projects (under samples/).
|
||||
- Optionally filters test projects by name pattern (e.g., only *UnitTests*).
|
||||
Writes the filtered solution to the specified output path and prints the path.
|
||||
|
||||
.PARAMETER Solution
|
||||
Path to the source .slnx solution file.
|
||||
|
||||
.PARAMETER TargetFramework
|
||||
The target framework to filter by (e.g., net10.0, net472).
|
||||
|
||||
.PARAMETER Configuration
|
||||
Optional MSBuild configuration used when querying TargetFrameworks. Defaults to Debug.
|
||||
|
||||
.PARAMETER TestProjectNameFilter
|
||||
Optional wildcard pattern to filter test project names (e.g., *UnitTests*, *IntegrationTests*).
|
||||
When specified, only test projects whose filename matches this pattern are kept.
|
||||
|
||||
.PARAMETER ExcludeSamples
|
||||
When specified, removes all projects under the samples/ directory from the solution.
|
||||
|
||||
.PARAMETER OutputPath
|
||||
Optional output path for the filtered .slnx file. If not specified, a temp file is created.
|
||||
|
||||
.EXAMPLE
|
||||
# Generate a filtered solution and run tests
|
||||
$filtered = ./dotnet/eng/scripts/New-FilteredSolution.ps1 -Solution dotnet/agent-framework-dotnet.slnx -TargetFramework net472
|
||||
dotnet test --solution $filtered --no-build -f net472
|
||||
|
||||
.EXAMPLE
|
||||
# Generate a solution with only unit test projects
|
||||
./dotnet/eng/scripts/New-FilteredSolution.ps1 -Solution dotnet/agent-framework-dotnet.slnx -TargetFramework net10.0 -TestProjectNameFilter "*UnitTests*" -OutputPath filtered-unit.slnx
|
||||
|
||||
.EXAMPLE
|
||||
# Inline usage with dotnet test (PowerShell)
|
||||
dotnet test --solution (./dotnet/eng/scripts/New-FilteredSolution.ps1 -Solution dotnet/agent-framework-dotnet.slnx -TargetFramework net472) --no-build -f net472
|
||||
#>
|
||||
|
||||
[CmdletBinding()]
|
||||
param(
|
||||
[Parameter(Mandatory)]
|
||||
[string]$Solution,
|
||||
|
||||
[Parameter(Mandatory)]
|
||||
[string]$TargetFramework,
|
||||
|
||||
[string]$Configuration = "Debug",
|
||||
|
||||
[string]$TestProjectNameFilter,
|
||||
|
||||
[switch]$ExcludeSamples,
|
||||
|
||||
[string]$OutputPath
|
||||
)
|
||||
|
||||
$ErrorActionPreference = "Stop"
|
||||
|
||||
# Resolve the solution path
|
||||
$solutionPath = Resolve-Path $Solution
|
||||
$solutionDir = Split-Path $solutionPath -Parent
|
||||
|
||||
if (-not $OutputPath) {
|
||||
$OutputPath = [System.IO.Path]::Combine([System.IO.Path]::GetTempPath(), "filtered-$(Split-Path $solutionPath -Leaf)")
|
||||
}
|
||||
|
||||
# Parse the .slnx XML
|
||||
[xml]$slnx = Get-Content $solutionPath -Raw
|
||||
|
||||
$removed = @()
|
||||
$kept = @()
|
||||
|
||||
# Remove sample projects if requested
|
||||
if ($ExcludeSamples) {
|
||||
$sampleProjects = $slnx.SelectNodes("//Project[contains(@Path, 'samples/')]")
|
||||
foreach ($proj in $sampleProjects) {
|
||||
$projRelPath = $proj.GetAttribute("Path")
|
||||
Write-Verbose "Removing (sample): $projRelPath"
|
||||
$removed += $projRelPath
|
||||
$proj.ParentNode.RemoveChild($proj) | Out-Null
|
||||
}
|
||||
Write-Host "Removed $($sampleProjects.Count) sample project(s)." -ForegroundColor Yellow
|
||||
}
|
||||
|
||||
# Filter all remaining projects by target framework
|
||||
$allProjects = $slnx.SelectNodes("//Project")
|
||||
|
||||
foreach ($proj in $allProjects) {
|
||||
$projRelPath = $proj.GetAttribute("Path")
|
||||
$projFullPath = Join-Path $solutionDir $projRelPath
|
||||
$projFileName = Split-Path $projRelPath -Leaf
|
||||
$isTestProject = $projRelPath -like "*tests/*"
|
||||
|
||||
# Filter test projects by name pattern if specified
|
||||
if ($isTestProject -and $TestProjectNameFilter -and ($projFileName -notlike $TestProjectNameFilter)) {
|
||||
Write-Verbose "Removing (name filter): $projRelPath"
|
||||
$removed += $projRelPath
|
||||
$proj.ParentNode.RemoveChild($proj) | Out-Null
|
||||
continue
|
||||
}
|
||||
|
||||
if (-not (Test-Path $projFullPath)) {
|
||||
Write-Verbose "Project not found, keeping in solution: $projRelPath"
|
||||
$kept += $projRelPath
|
||||
continue
|
||||
}
|
||||
|
||||
# Query the project's target frameworks using MSBuild
|
||||
$targetFrameworks = & dotnet msbuild $projFullPath -getProperty:TargetFrameworks -p:Configuration=$Configuration -nologo 2>$null
|
||||
$targetFrameworks = $targetFrameworks.Trim()
|
||||
|
||||
if ($targetFrameworks -like "*$TargetFramework*") {
|
||||
Write-Verbose "Keeping: $projRelPath (targets: $targetFrameworks)"
|
||||
$kept += $projRelPath
|
||||
}
|
||||
else {
|
||||
Write-Verbose "Removing: $projRelPath (targets: $targetFrameworks, missing: $TargetFramework)"
|
||||
$removed += $projRelPath
|
||||
$proj.ParentNode.RemoveChild($proj) | Out-Null
|
||||
}
|
||||
}
|
||||
|
||||
# Write the filtered solution
|
||||
$slnx.Save($OutputPath)
|
||||
|
||||
# Report results to stderr so stdout is clean for piping
|
||||
Write-Host "Filtered solution written to: $OutputPath" -ForegroundColor Green
|
||||
if ($removed.Count -gt 0) {
|
||||
Write-Host "Removed $($removed.Count) project(s):" -ForegroundColor Yellow
|
||||
foreach ($r in $removed) {
|
||||
Write-Host " - $r" -ForegroundColor Yellow
|
||||
}
|
||||
}
|
||||
Write-Host "Kept $($kept.Count) project(s)." -ForegroundColor Green
|
||||
|
||||
# Output the path for piping
|
||||
Write-Output $OutputPath
|
||||
+1
-4
@@ -1,10 +1,7 @@
|
||||
{
|
||||
"sdk": {
|
||||
"version": "10.0.200",
|
||||
"version": "10.0.100",
|
||||
"rollForward": "minor",
|
||||
"allowPrerelease": false
|
||||
},
|
||||
"test": {
|
||||
"runner": "Microsoft.Testing.Platform"
|
||||
}
|
||||
}
|
||||
@@ -2,11 +2,9 @@
|
||||
<PropertyGroup>
|
||||
<!-- Central version prefix - applies to all nuget packages. -->
|
||||
<VersionPrefix>1.0.0</VersionPrefix>
|
||||
<RCNumber>4</RCNumber>
|
||||
<PackageVersion Condition="'$(IsReleaseCandidate)' == 'true'">$(VersionPrefix)-rc$(RCNumber)</PackageVersion>
|
||||
<PackageVersion Condition="'$(IsReleaseCandidate)' != 'true' AND '$(VersionSuffix)' != ''">$(VersionPrefix)-$(VersionSuffix).260311.1</PackageVersion>
|
||||
<PackageVersion Condition="'$(IsReleaseCandidate)' != 'true' AND '$(VersionSuffix)' == ''">$(VersionPrefix)-preview.260311.1</PackageVersion>
|
||||
<GitTag>1.0.0-rc4</GitTag>
|
||||
<PackageVersion Condition="'$(VersionSuffix)' != ''">$(VersionPrefix)-$(VersionSuffix).260128.1</PackageVersion>
|
||||
<PackageVersion Condition="'$(VersionSuffix)' == ''">$(VersionPrefix)-preview.260128.1</PackageVersion>
|
||||
<GitTag>1.0.0-preview.260128.1</GitTag>
|
||||
|
||||
<Configurations>Debug;Release;Publish</Configurations>
|
||||
<IsPackable>true</IsPackable>
|
||||
|
||||
@@ -1,21 +0,0 @@
|
||||
<Project Sdk="Microsoft.NET.Sdk">
|
||||
|
||||
<PropertyGroup>
|
||||
<OutputType>Exe</OutputType>
|
||||
<TargetFrameworks>net10.0</TargetFrameworks>
|
||||
|
||||
<Nullable>enable</Nullable>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="Azure.AI.OpenAI" />
|
||||
<PackageReference Include="Azure.Identity" />
|
||||
<PackageReference Include="Microsoft.Extensions.AI.OpenAI" />
|
||||
</ItemGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<ProjectReference Include="..\..\..\src\Microsoft.Agents.AI.OpenAI\Microsoft.Agents.AI.OpenAI.csproj" />
|
||||
</ItemGroup>
|
||||
|
||||
</Project>
|
||||
@@ -1,14 +0,0 @@
|
||||
<Project Sdk="Microsoft.NET.Sdk">
|
||||
|
||||
<PropertyGroup>
|
||||
<OutputType>Exe</OutputType>
|
||||
<TargetFrameworks>net10.0</TargetFrameworks>
|
||||
<Nullable>enable</Nullable>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<ProjectReference Include="..\..\..\src\Microsoft.Agents.AI.Workflows\Microsoft.Agents.AI.Workflows.csproj" />
|
||||
</ItemGroup>
|
||||
|
||||
</Project>
|
||||
@@ -1,45 +0,0 @@
|
||||
// Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
// This sample shows how to host an AI agent with Azure Functions (DurableAgents).
|
||||
//
|
||||
// Prerequisites:
|
||||
// - Azure Functions Core Tools
|
||||
// - Azure OpenAI resource
|
||||
//
|
||||
// Environment variables:
|
||||
// AZURE_OPENAI_ENDPOINT
|
||||
// AZURE_OPENAI_DEPLOYMENT_NAME (defaults to "gpt-4o-mini")
|
||||
//
|
||||
// Run with: func start
|
||||
// Then call: POST http://localhost:7071/api/agents/HostedAgent/run
|
||||
|
||||
using Azure.AI.OpenAI;
|
||||
using Azure.Identity;
|
||||
using Microsoft.Agents.AI;
|
||||
using Microsoft.Agents.AI.Hosting.AzureFunctions;
|
||||
using Microsoft.Azure.Functions.Worker.Builder;
|
||||
using Microsoft.Extensions.Hosting;
|
||||
using OpenAI.Chat;
|
||||
|
||||
var endpoint = Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT")
|
||||
?? throw new InvalidOperationException("AZURE_OPENAI_ENDPOINT is not set.");
|
||||
var deploymentName = Environment.GetEnvironmentVariable("AZURE_OPENAI_DEPLOYMENT_NAME") ?? "gpt-4o-mini";
|
||||
|
||||
// Set up an AI agent following the standard Microsoft Agent Framework pattern.
|
||||
// WARNING: DefaultAzureCredential is convenient for development but requires careful consideration in production.
|
||||
// In production, consider using a specific credential (e.g., ManagedIdentityCredential) to avoid
|
||||
// latency issues, unintended credential probing, and potential security risks from fallback mechanisms.
|
||||
AIAgent agent = new AzureOpenAIClient(new Uri(endpoint), new DefaultAzureCredential())
|
||||
.GetChatClient(deploymentName)
|
||||
.AsAIAgent(
|
||||
instructions: "You are a helpful assistant hosted in Azure Functions.",
|
||||
name: "HostedAgent");
|
||||
|
||||
// Configure the function app to host the AI agent.
|
||||
// This will automatically generate HTTP API endpoints for the agent.
|
||||
using IHost app = FunctionsApplication
|
||||
.CreateBuilder(args)
|
||||
.ConfigureFunctionsWebApplication()
|
||||
.ConfigureDurableAgents(options => options.AddAIAgent(agent, timeToLive: TimeSpan.FromHours(1)))
|
||||
.Build();
|
||||
app.Run();
|
||||
@@ -1,20 +0,0 @@
|
||||
<Project Sdk="Microsoft.NET.Sdk.Web">
|
||||
|
||||
<PropertyGroup>
|
||||
<OutputType>Exe</OutputType>
|
||||
<TargetFrameworks>net10.0</TargetFrameworks>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
<Nullable>enable</Nullable>
|
||||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="Azure.AI.OpenAI" />
|
||||
<PackageReference Include="Azure.Identity" />
|
||||
</ItemGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<ProjectReference Include="..\..\..\..\..\src\Microsoft.Agents.AI.Hosting.AGUI.AspNetCore\Microsoft.Agents.AI.Hosting.AGUI.AspNetCore.csproj" />
|
||||
<ProjectReference Include="..\..\..\..\..\src\Microsoft.Agents.AI.OpenAI\Microsoft.Agents.AI.OpenAI.csproj" />
|
||||
</ItemGroup>
|
||||
|
||||
</Project>
|
||||
@@ -1,37 +0,0 @@
|
||||
// Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
// This sample shows how to create and use a simple AI agent with Azure OpenAI Responses as the backend.
|
||||
|
||||
using Azure.AI.OpenAI;
|
||||
using Azure.Identity;
|
||||
using Microsoft.Agents.AI;
|
||||
using Microsoft.Extensions.AI;
|
||||
using OpenAI.Responses;
|
||||
|
||||
var endpoint = Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT") ?? throw new InvalidOperationException("AZURE_OPENAI_ENDPOINT is not set.");
|
||||
var deploymentName = Environment.GetEnvironmentVariable("AZURE_OPENAI_DEPLOYMENT_NAME") ?? "gpt-4o-mini";
|
||||
|
||||
// WARNING: DefaultAzureCredential is convenient for development but requires careful consideration in production.
|
||||
// In production, consider using a specific credential (e.g., ManagedIdentityCredential) to avoid
|
||||
// latency issues, unintended credential probing, and potential security risks from fallback mechanisms.
|
||||
AIAgent agent = new AzureOpenAIClient(
|
||||
new Uri(endpoint),
|
||||
new DefaultAzureCredential())
|
||||
.GetResponsesClient()
|
||||
.AsAIAgent(model: deploymentName, instructions: "You are good at telling jokes.", name: "Joker");
|
||||
|
||||
// Invoke the agent and output the text result.
|
||||
Console.WriteLine(await agent.RunAsync("Tell me a joke about a pirate."));
|
||||
|
||||
// Create a responses based agent with "store"=false.
|
||||
// This means that chat history is managed locally by Agent Framework
|
||||
// instead of being stored in the service (default).
|
||||
AIAgent agentStoreFalse = new AzureOpenAIClient(
|
||||
new Uri(endpoint),
|
||||
new DefaultAzureCredential())
|
||||
.GetResponsesClient()
|
||||
.AsIChatClientWithStoredOutputDisabled(model: deploymentName)
|
||||
.AsAIAgent(instructions: "You are good at telling jokes.", name: "Joker");
|
||||
|
||||
// Invoke the agent and output the text result.
|
||||
Console.WriteLine(await agentStoreFalse.RunAsync("Tell me a joke about a pirate."));
|
||||
-28
@@ -1,28 +0,0 @@
|
||||
<Project Sdk="Microsoft.NET.Sdk">
|
||||
|
||||
<PropertyGroup>
|
||||
<OutputType>Exe</OutputType>
|
||||
<TargetFrameworks>net10.0</TargetFrameworks>
|
||||
|
||||
<Nullable>enable</Nullable>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
<NoWarn>$(NoWarn);MAAI001</NoWarn>
|
||||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="Azure.AI.OpenAI" />
|
||||
<PackageReference Include="Azure.Identity" />
|
||||
</ItemGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<ProjectReference Include="..\..\..\..\src\Microsoft.Agents.AI.OpenAI\Microsoft.Agents.AI.OpenAI.csproj" />
|
||||
</ItemGroup>
|
||||
|
||||
<!-- Copy skills directory to output -->
|
||||
<ItemGroup>
|
||||
<None Include="skills\**\*.*">
|
||||
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
|
||||
</None>
|
||||
</ItemGroup>
|
||||
|
||||
</Project>
|
||||
@@ -1,50 +0,0 @@
|
||||
// Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
// This sample demonstrates how to use Agent Skills with a ChatClientAgent.
|
||||
// Agent Skills are modular packages of instructions and resources that extend an agent's capabilities.
|
||||
// Skills follow the progressive disclosure pattern: advertise -> load -> read resources.
|
||||
//
|
||||
// This sample includes the expense-report skill:
|
||||
// - Policy-based expense filing with references and assets
|
||||
|
||||
using Azure.AI.OpenAI;
|
||||
using Azure.Identity;
|
||||
using Microsoft.Agents.AI;
|
||||
using OpenAI.Responses;
|
||||
|
||||
// --- Configuration ---
|
||||
string endpoint = Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT")
|
||||
?? throw new InvalidOperationException("AZURE_OPENAI_ENDPOINT is not set.");
|
||||
string deploymentName = Environment.GetEnvironmentVariable("AZURE_OPENAI_DEPLOYMENT_NAME") ?? "gpt-4o-mini";
|
||||
|
||||
// --- Skills Provider ---
|
||||
// Discovers skills from the 'skills' directory and makes them available to the agent
|
||||
var skillsProvider = new FileAgentSkillsProvider(skillPath: Path.Combine(AppContext.BaseDirectory, "skills"));
|
||||
|
||||
// --- Agent Setup ---
|
||||
AIAgent agent = new AzureOpenAIClient(new Uri(endpoint), new DefaultAzureCredential())
|
||||
.GetResponsesClient()
|
||||
.AsAIAgent(new ChatClientAgentOptions
|
||||
{
|
||||
Name = "SkillsAgent",
|
||||
ChatOptions = new()
|
||||
{
|
||||
Instructions = "You are a helpful assistant.",
|
||||
},
|
||||
AIContextProviders = [skillsProvider],
|
||||
},
|
||||
model: deploymentName);
|
||||
|
||||
// --- Example 1: Expense policy question (loads FAQ resource) ---
|
||||
Console.WriteLine("Example 1: Checking expense policy FAQ");
|
||||
Console.WriteLine("---------------------------------------");
|
||||
AgentResponse response1 = await agent.RunAsync("Are tips reimbursable? I left a 25% tip on a taxi ride and want to know if that's covered.");
|
||||
Console.WriteLine($"Agent: {response1.Text}\n");
|
||||
|
||||
// --- Example 2: Filing an expense report (multi-turn with template asset) ---
|
||||
Console.WriteLine("Example 2: Filing an expense report");
|
||||
Console.WriteLine("---------------------------------------");
|
||||
AgentSession session = await agent.CreateSessionAsync();
|
||||
AgentResponse response2 = await agent.RunAsync("I had 3 client dinners and a $1,200 flight last week. Return a draft expense report and ask about any missing details.",
|
||||
session);
|
||||
Console.WriteLine($"Agent: {response2.Text}\n");
|
||||
@@ -1,63 +0,0 @@
|
||||
# Agent Skills Sample
|
||||
|
||||
This sample demonstrates how to use **Agent Skills** with a `ChatClientAgent` in the Microsoft Agent Framework.
|
||||
|
||||
## What are Agent Skills?
|
||||
|
||||
Agent Skills are modular packages of instructions and resources that enable AI agents to perform specialized tasks. They follow the [Agent Skills specification](https://agentskills.io/) and implement the progressive disclosure pattern:
|
||||
|
||||
1. **Advertise**: Skills are advertised with name + description (~100 tokens per skill)
|
||||
2. **Load**: Full instructions are loaded on-demand via `load_skill` tool
|
||||
3. **Resources**: References and other files loaded via `read_skill_resource` tool
|
||||
|
||||
## Skills Included
|
||||
|
||||
### expense-report
|
||||
Policy-based expense filing with spending limits, receipt requirements, and approval workflows.
|
||||
- `references/POLICY_FAQ.md` — Detailed expense policy Q&A
|
||||
- `assets/expense-report-template.md` — Submission template
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
Agent_Step01_BasicSkills/
|
||||
├── Program.cs
|
||||
├── Agent_Step01_BasicSkills.csproj
|
||||
└── skills/
|
||||
└── expense-report/
|
||||
├── SKILL.md
|
||||
├── references/
|
||||
│ └── POLICY_FAQ.md
|
||||
└── assets/
|
||||
└── expense-report-template.md
|
||||
```
|
||||
|
||||
## Running the Sample
|
||||
|
||||
### Prerequisites
|
||||
- .NET 10.0 SDK
|
||||
- Azure OpenAI endpoint with a deployed model
|
||||
|
||||
### Setup
|
||||
1. Set environment variables:
|
||||
```bash
|
||||
export AZURE_OPENAI_ENDPOINT="https://your-endpoint.openai.azure.com/"
|
||||
export AZURE_OPENAI_DEPLOYMENT_NAME="gpt-4o-mini"
|
||||
```
|
||||
|
||||
2. Run the sample:
|
||||
```bash
|
||||
dotnet run
|
||||
```
|
||||
|
||||
### Examples
|
||||
|
||||
The sample runs two examples:
|
||||
|
||||
1. **Expense policy FAQ** — Asks about tip reimbursement; the agent loads the expense-report skill and reads the FAQ resource
|
||||
2. **Filing an expense report** — Multi-turn conversation to draft an expense report using the template asset
|
||||
|
||||
## Learn More
|
||||
|
||||
- [Agent Skills Specification](https://agentskills.io/)
|
||||
- [Microsoft Agent Framework Documentation](../../../../../docs/)
|
||||
-40
@@ -1,40 +0,0 @@
|
||||
---
|
||||
name: expense-report
|
||||
description: File and validate employee expense reports according to Contoso company policy. Use when asked about expense submissions, reimbursement rules, receipt requirements, spending limits, or expense categories.
|
||||
metadata:
|
||||
author: contoso-finance
|
||||
version: "2.1"
|
||||
---
|
||||
|
||||
# Expense Report
|
||||
|
||||
## Categories and Limits
|
||||
|
||||
| Category | Limit | Receipt | Approval |
|
||||
|---|---|---|---|
|
||||
| Meals — solo | $50/day | >$25 | No |
|
||||
| Meals — team/client | $75/person | Always | Manager if >$200 total |
|
||||
| Lodging | $250/night | Always | Manager if >3 nights |
|
||||
| Ground transport | $100/day | >$15 | No |
|
||||
| Airfare | Economy | Always | Manager; VP if >$1,500 |
|
||||
| Conference/training | $2,000/event | Always | Manager + L&D |
|
||||
| Office supplies | $100 | Yes | No |
|
||||
| Software/subscriptions | $50/month | Yes | Manager if >$200/year |
|
||||
|
||||
## Filing Process
|
||||
|
||||
1. Collect receipts — must show vendor, date, amount, payment method.
|
||||
2. Categorize per table above.
|
||||
3. Use template: [assets/expense-report-template.md](assets/expense-report-template.md).
|
||||
4. For client/team meals: list attendee names and business purpose.
|
||||
5. Submit — auto-approved if <$500; manager if $500–$2,000; VP if >$2,000.
|
||||
6. Reimbursement: 10 business days via direct deposit.
|
||||
|
||||
## Policy Rules
|
||||
|
||||
- Submit within 30 days of transaction.
|
||||
- Alcohol is never reimbursable.
|
||||
- Foreign currency: convert to USD at transaction-date rate; note original currency and amount.
|
||||
- Mixed personal/business travel: only business portion reimbursable; provide comparison quotes.
|
||||
- Lost receipts (>$25): file Lost Receipt Affidavit from Finance. Max 2 per quarter.
|
||||
- For policy questions not covered above, consult the FAQ: [references/POLICY_FAQ.md](references/POLICY_FAQ.md). Answers should be based on what this document and the FAQ state.
|
||||
-5
@@ -1,5 +0,0 @@
|
||||
# Expense Report Template
|
||||
|
||||
| Date | Category | Vendor | Description | Amount (USD) | Original Currency | Original Amount | Attendees | Business Purpose | Receipt Attached |
|
||||
|------|----------|--------|-------------|--------------|-------------------|-----------------|-----------|------------------|------------------|
|
||||
| | | | | | | | | | Yes or No |
|
||||
-55
@@ -1,55 +0,0 @@
|
||||
# Expense Policy — Frequently Asked Questions
|
||||
|
||||
## Meals
|
||||
|
||||
**Q: Can I expense coffee or snacks during the workday?**
|
||||
A: Daily coffee/snacks under $10 are not reimbursable (considered personal). Coffee purchased during a client meeting or team working session is reimbursable as a team meal.
|
||||
|
||||
**Q: What if a team dinner exceeds the per-person limit?**
|
||||
A: The $75/person limit applies as a guideline. Overages up to 20% are accepted with a written justification (e.g., "client dinner at venue chosen by client"). Overages beyond 20% require pre-approval from your VP.
|
||||
|
||||
**Q: Do I need to list every attendee?**
|
||||
A: Yes. For client meals, list the client's name and company. For team meals, list all employee names. For groups over 10, you may attach a separate attendee list.
|
||||
|
||||
## Travel
|
||||
|
||||
**Q: Can I book a premium economy or business class flight?**
|
||||
A: Economy class is the standard. Premium economy is allowed for flights over 6 hours. Business class requires VP pre-approval and is generally reserved for flights over 10 hours or medical accommodation.
|
||||
|
||||
**Q: What about ride-sharing (Uber/Lyft) vs. rental cars?**
|
||||
A: Use ride-sharing for trips under 30 miles round-trip. Rent a car for multi-day travel or when ride-sharing would exceed $100/day. Always choose the compact/standard category unless traveling with 3+ people.
|
||||
|
||||
**Q: Are tips reimbursable?**
|
||||
A: Tips up to 20% are reimbursable for meals, taxi/ride-share, and hotel housekeeping. Tips above 20% require justification.
|
||||
|
||||
## Lodging
|
||||
|
||||
**Q: What if the $250/night limit isn't enough for the city I'm visiting?**
|
||||
A: For high-cost cities (New York, San Francisco, London, Tokyo, Sydney), the limit is automatically increased to $350/night. No additional approval is needed. For other locations where rates are unusually high (e.g., during a major conference), request a per-trip exception from your manager before booking.
|
||||
|
||||
**Q: Can I stay with friends/family instead and get a per-diem?**
|
||||
A: No. Contoso reimburses actual lodging costs only, not per-diems.
|
||||
|
||||
## Subscriptions and Software
|
||||
|
||||
**Q: Can I expense a personal productivity tool?**
|
||||
A: Software must be directly related to your job function. Tools like IDE licenses, design software, or project management apps are reimbursable. General productivity apps (note-taking, personal calendar) are not, unless your manager confirms a business need in writing.
|
||||
|
||||
**Q: What about annual subscriptions?**
|
||||
A: Annual subscriptions over $200 require manager approval before purchase. Submit the approval email with your expense report.
|
||||
|
||||
## Receipts and Documentation
|
||||
|
||||
**Q: My receipt is faded/damaged. What do I do?**
|
||||
A: Try to obtain a duplicate from the vendor. If not possible, submit a Lost Receipt Affidavit (available from the Finance SharePoint site). You're limited to 2 affidavits per quarter.
|
||||
|
||||
**Q: Do I need a receipt for parking meters or tolls?**
|
||||
A: For amounts under $15, no receipt is required — just note the date, location, and amount. For $15 and above, a receipt or bank/credit card statement excerpt is required.
|
||||
|
||||
## Approval and Reimbursement
|
||||
|
||||
**Q: My manager is on leave. Who approves my report?**
|
||||
A: Expense reports can be approved by your skip-level manager or any manager designated as an alternate approver in the expense system.
|
||||
|
||||
**Q: Can I submit expenses from a previous quarter?**
|
||||
A: The standard 30-day window applies. Expenses older than 30 days require a written explanation and VP approval. Expenses older than 90 days are not reimbursable except in extraordinary circumstances (extended leave, medical emergency) with CFO approval.
|
||||
@@ -1,7 +0,0 @@
|
||||
# AgentSkills Samples
|
||||
|
||||
Samples demonstrating Agent Skills capabilities.
|
||||
|
||||
| Sample | Description |
|
||||
|--------|-------------|
|
||||
| [Agent_Step01_BasicSkills](Agent_Step01_BasicSkills/) | Using Agent Skills with a ChatClientAgent, including progressive disclosure and skill resources |
|
||||
-15
@@ -1,15 +0,0 @@
|
||||
<Project Sdk="Microsoft.NET.Sdk">
|
||||
|
||||
<PropertyGroup>
|
||||
<OutputType>Exe</OutputType>
|
||||
<TargetFramework>net10.0</TargetFramework>
|
||||
|
||||
<Nullable>enable</Nullable>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<ProjectReference Include="..\..\..\..\src\Microsoft.Agents.AI.Anthropic\Microsoft.Agents.AI.Anthropic.csproj" />
|
||||
</ItemGroup>
|
||||
|
||||
</Project>
|
||||
-127
@@ -1,127 +0,0 @@
|
||||
// Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
// This sample demonstrates how to use Anthropic-managed Skills with an AI agent.
|
||||
// Skills are pre-built capabilities provided by Anthropic that can be used with the Claude API.
|
||||
// This sample shows how to:
|
||||
// 1. List available Anthropic-managed skills
|
||||
// 2. Use the pptx skill to create PowerPoint presentations
|
||||
// 3. Download and save generated files
|
||||
|
||||
using Anthropic;
|
||||
using Anthropic.Core;
|
||||
using Anthropic.Models.Beta;
|
||||
using Anthropic.Models.Beta.Files;
|
||||
using Anthropic.Models.Beta.Messages;
|
||||
using Anthropic.Models.Beta.Skills;
|
||||
using Anthropic.Services;
|
||||
using Microsoft.Agents.AI;
|
||||
using Microsoft.Extensions.AI;
|
||||
|
||||
string apiKey = Environment.GetEnvironmentVariable("ANTHROPIC_API_KEY") ?? throw new InvalidOperationException("ANTHROPIC_API_KEY is not set.");
|
||||
// Skills require Claude 4.5 models (Sonnet 4.5, Haiku 4.5, or Opus 4.5)
|
||||
string model = Environment.GetEnvironmentVariable("ANTHROPIC_CHAT_MODEL_NAME") ?? "claude-sonnet-4-5-20250929";
|
||||
|
||||
// Create the Anthropic client
|
||||
AnthropicClient anthropicClient = new() { ApiKey = apiKey };
|
||||
|
||||
// List available Anthropic-managed skills (optional - API may not be available in all regions)
|
||||
Console.WriteLine("Available Anthropic-managed skills:");
|
||||
try
|
||||
{
|
||||
SkillListPage skills = await anthropicClient.Beta.Skills.List(
|
||||
new SkillListParams { Source = "anthropic", Betas = [AnthropicBeta.Skills2025_10_02] });
|
||||
|
||||
foreach (var skill in skills.Items)
|
||||
{
|
||||
Console.WriteLine($" {skill.Source}: {skill.ID} (version: {skill.LatestVersion})");
|
||||
}
|
||||
}
|
||||
catch (Exception ex)
|
||||
{
|
||||
Console.WriteLine($" (Skills listing not available: {ex.Message})");
|
||||
}
|
||||
|
||||
Console.WriteLine();
|
||||
|
||||
// Define the pptx skill - the SDK handles all beta flags and container configuration automatically
|
||||
// when using AsAITool(), so no manual RawRepresentationFactory configuration is needed.
|
||||
BetaSkillParams pptxSkill = new()
|
||||
{
|
||||
Type = BetaSkillParamsType.Anthropic,
|
||||
SkillID = "pptx",
|
||||
Version = "latest"
|
||||
};
|
||||
|
||||
// Create an agent with the pptx skill enabled.
|
||||
// Skills require extended thinking and higher max tokens for complex file generation.
|
||||
// The SDK's AsAITool() handles beta flags and container config automatically.
|
||||
ChatClientAgent agent = anthropicClient.Beta.AsAIAgent(
|
||||
model: model,
|
||||
instructions: "You are a helpful agent for creating PowerPoint presentations.",
|
||||
tools: [pptxSkill.AsAITool()],
|
||||
clientFactory: (chatClient) => chatClient
|
||||
.AsBuilder()
|
||||
.ConfigureOptions(options =>
|
||||
{
|
||||
options.RawRepresentationFactory = (_) => new MessageCreateParams()
|
||||
{
|
||||
Model = model,
|
||||
MaxTokens = 20000,
|
||||
Messages = [],
|
||||
Thinking = new BetaThinkingConfigParam(
|
||||
new BetaThinkingConfigEnabled(budgetTokens: 10000))
|
||||
};
|
||||
})
|
||||
.Build());
|
||||
|
||||
Console.WriteLine("Creating a presentation about renewable energy...\n");
|
||||
|
||||
// Run the agent with a request to create a presentation
|
||||
AgentResponse response = await agent.RunAsync("Create a simple 3-slide presentation about renewable energy sources. Include a title slide, a slide about solar energy, and a slide about wind energy.");
|
||||
|
||||
Console.WriteLine("#### Agent Response ####");
|
||||
Console.WriteLine(response.Text);
|
||||
|
||||
// Display any reasoning/thinking content
|
||||
List<TextReasoningContent> reasoningContents = response.Messages.SelectMany(m => m.Contents.OfType<TextReasoningContent>()).ToList();
|
||||
if (reasoningContents.Count > 0)
|
||||
{
|
||||
Console.WriteLine("\n#### Agent Reasoning ####");
|
||||
Console.WriteLine($"\e[92m{string.Join("\n", reasoningContents.Select(c => c.Text))}\e[0m");
|
||||
}
|
||||
|
||||
// Collect generated files from CodeInterpreterToolResultContent outputs
|
||||
List<HostedFileContent> hostedFiles = response.Messages
|
||||
.SelectMany(m => m.Contents.OfType<CodeInterpreterToolResultContent>())
|
||||
.Where(c => c.Outputs is not null)
|
||||
.SelectMany(c => c.Outputs!.OfType<HostedFileContent>())
|
||||
.ToList();
|
||||
|
||||
if (hostedFiles.Count > 0)
|
||||
{
|
||||
Console.WriteLine("\n#### Generated Files ####");
|
||||
foreach (HostedFileContent file in hostedFiles)
|
||||
{
|
||||
Console.WriteLine($" FileId: {file.FileId}");
|
||||
|
||||
// Download the file using the Anthropic Files API
|
||||
using HttpResponse fileResponse = await anthropicClient.Beta.Files.Download(
|
||||
file.FileId,
|
||||
new FileDownloadParams { Betas = ["files-api-2025-04-14"] });
|
||||
|
||||
// Save the file to disk
|
||||
string fileName = $"presentation_{file.FileId.Substring(0, 8)}.pptx";
|
||||
using FileStream fileStream = File.Create(fileName);
|
||||
Stream contentStream = await fileResponse.ReadAsStream();
|
||||
await contentStream.CopyToAsync(fileStream);
|
||||
|
||||
Console.WriteLine($" Saved to: {fileName}");
|
||||
}
|
||||
}
|
||||
|
||||
Console.WriteLine("\nToken usage:");
|
||||
Console.WriteLine($"Input: {response.Usage?.InputTokenCount}, Output: {response.Usage?.OutputTokenCount}");
|
||||
if (response.Usage?.AdditionalCounts is not null)
|
||||
{
|
||||
Console.WriteLine($"Additional: {string.Join(", ", response.Usage.AdditionalCounts)}");
|
||||
}
|
||||
-119
@@ -1,119 +0,0 @@
|
||||
# Using Anthropic Skills with agents
|
||||
|
||||
This sample demonstrates how to use Anthropic-managed Skills with AI agents. Skills are pre-built capabilities provided by Anthropic that can be used with the Claude API.
|
||||
|
||||
## What this sample demonstrates
|
||||
|
||||
- Listing available Anthropic-managed skills
|
||||
- Creating an AI agent with Anthropic Claude Skills support using the simplified `AsAITool()` approach
|
||||
- Using the pptx skill to create PowerPoint presentations
|
||||
- Downloading and saving generated files to disk
|
||||
- Handling agent responses with generated content
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Before you begin, ensure you have the following prerequisites:
|
||||
|
||||
- .NET 10.0 SDK or later
|
||||
- Anthropic API key configured
|
||||
- Access to Anthropic Claude models with Skills support
|
||||
|
||||
**Note**: This sample uses Anthropic Claude models with Skills. Skills are a beta feature. For more information, see [Anthropic documentation](https://docs.anthropic.com/).
|
||||
|
||||
Set the following environment variables:
|
||||
|
||||
```powershell
|
||||
$env:ANTHROPIC_API_KEY="your-anthropic-api-key" # Replace with your Anthropic API key
|
||||
$env:ANTHROPIC_CHAT_MODEL_NAME="your-anthropic-model" # Replace with your Anthropic model (e.g., claude-sonnet-4-5-20250929)
|
||||
```
|
||||
|
||||
## Run the sample
|
||||
|
||||
Navigate to the AgentWithAnthropic sample directory and run:
|
||||
|
||||
```powershell
|
||||
cd dotnet\samples\02-agents\AgentWithAnthropic
|
||||
dotnet run --project .\Agent_Anthropic_Step04_UsingSkills
|
||||
```
|
||||
|
||||
## Available Anthropic Skills
|
||||
|
||||
Anthropic provides several managed skills that can be used with the Claude API:
|
||||
|
||||
- `pptx` - Create PowerPoint presentations
|
||||
- `xlsx` - Create Excel spreadsheets
|
||||
- `docx` - Create Word documents
|
||||
- `pdf` - Create and analyze PDF documents
|
||||
|
||||
You can list available skills using the Anthropic SDK:
|
||||
|
||||
```csharp
|
||||
SkillListPage skills = await anthropicClient.Beta.Skills.List(
|
||||
new SkillListParams { Source = "anthropic", Betas = [AnthropicBeta.Skills2025_10_02] });
|
||||
|
||||
foreach (var skill in skills.Items)
|
||||
{
|
||||
Console.WriteLine($"{skill.Source}: {skill.ID} (version: {skill.LatestVersion})");
|
||||
}
|
||||
```
|
||||
|
||||
## Expected behavior
|
||||
|
||||
The sample will:
|
||||
|
||||
1. List all available Anthropic-managed skills
|
||||
2. Create an agent with the pptx skill enabled
|
||||
3. Run the agent with a request to create a presentation
|
||||
4. Display the agent's response text
|
||||
5. Download any generated files and save them to disk
|
||||
6. Display token usage statistics
|
||||
|
||||
## Code highlights
|
||||
|
||||
### Simplified skill configuration
|
||||
|
||||
The Anthropic SDK handles all beta flags and container configuration automatically when using `AsAITool()`:
|
||||
|
||||
```csharp
|
||||
// Define the pptx skill
|
||||
BetaSkillParams pptxSkill = new()
|
||||
{
|
||||
Type = BetaSkillParamsType.Anthropic,
|
||||
SkillID = "pptx",
|
||||
Version = "latest"
|
||||
};
|
||||
|
||||
// Create an agent - the SDK handles beta flags automatically!
|
||||
ChatClientAgent agent = anthropicClient.Beta.AsAIAgent(
|
||||
model: model,
|
||||
instructions: "You are a helpful agent for creating PowerPoint presentations.",
|
||||
tools: [pptxSkill.AsAITool()]);
|
||||
```
|
||||
|
||||
**Note**: No manual `RawRepresentationFactory`, `Betas`, or `Container` configuration is needed. The SDK automatically adds the required beta headers (`skills-2025-10-02`, `code-execution-2025-08-25`) and configures the container with the skill.
|
||||
|
||||
### Handling generated files
|
||||
|
||||
Generated files are returned as `HostedFileContent` within `CodeInterpreterToolResultContent`:
|
||||
|
||||
```csharp
|
||||
// Collect generated files from response
|
||||
List<HostedFileContent> hostedFiles = response.Messages
|
||||
.SelectMany(m => m.Contents.OfType<CodeInterpreterToolResultContent>())
|
||||
.Where(c => c.Outputs is not null)
|
||||
.SelectMany(c => c.Outputs!.OfType<HostedFileContent>())
|
||||
.ToList();
|
||||
|
||||
// Download and save each file
|
||||
foreach (HostedFileContent file in hostedFiles)
|
||||
{
|
||||
using HttpResponse fileResponse = await anthropicClient.Beta.Files.Download(
|
||||
file.FileId,
|
||||
new FileDownloadParams { Betas = ["files-api-2025-04-14"] });
|
||||
|
||||
string fileName = $"presentation_{file.FileId.Substring(0, 8)}.pptx";
|
||||
await using FileStream fileStream = File.Create(fileName);
|
||||
Stream contentStream = await fileResponse.ReadAsStream();
|
||||
await contentStream.CopyToAsync(fileStream);
|
||||
}
|
||||
```
|
||||
-21
@@ -1,21 +0,0 @@
|
||||
<Project Sdk="Microsoft.NET.Sdk">
|
||||
|
||||
<PropertyGroup>
|
||||
<OutputType>Exe</OutputType>
|
||||
<TargetFrameworks>net10.0</TargetFrameworks>
|
||||
|
||||
<Nullable>enable</Nullable>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="Azure.AI.Projects" />
|
||||
<PackageReference Include="Azure.Identity" />
|
||||
</ItemGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<ProjectReference Include="..\..\..\..\src\Microsoft.Agents.AI.AzureAI\Microsoft.Agents.AI.AzureAI.csproj" />
|
||||
<ProjectReference Include="..\..\..\..\src\Microsoft.Agents.AI.FoundryMemory\Microsoft.Agents.AI.FoundryMemory.csproj" />
|
||||
</ItemGroup>
|
||||
|
||||
</Project>
|
||||
-77
@@ -1,77 +0,0 @@
|
||||
// Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
// This sample shows how to use the FoundryMemoryProvider to persist and recall memories for an agent.
|
||||
// The sample stores conversation messages in an Azure AI Foundry memory store and retrieves relevant
|
||||
// memories for subsequent invocations, even across new sessions.
|
||||
//
|
||||
// Note: Memory extraction in Azure AI Foundry is asynchronous and takes time. This sample demonstrates
|
||||
// a simple polling approach to wait for memory updates to complete before querying.
|
||||
|
||||
using System.Text.Json;
|
||||
using Azure.AI.Projects;
|
||||
using Azure.Identity;
|
||||
using Microsoft.Agents.AI;
|
||||
using Microsoft.Agents.AI.FoundryMemory;
|
||||
|
||||
string foundryEndpoint = Environment.GetEnvironmentVariable("AZURE_AI_PROJECT_ENDPOINT") ?? throw new InvalidOperationException("AZURE_AI_PROJECT_ENDPOINT is not set.");
|
||||
string memoryStoreName = Environment.GetEnvironmentVariable("AZURE_AI_MEMORY_STORE_ID") ?? "memory-store-sample";
|
||||
string deploymentName = Environment.GetEnvironmentVariable("AZURE_AI_MODEL_DEPLOYMENT_NAME") ?? "gpt-4o-mini";
|
||||
string embeddingModelName = Environment.GetEnvironmentVariable("AZURE_AI_EMBEDDING_DEPLOYMENT_NAME") ?? "text-embedding-ada-002";
|
||||
|
||||
// Create an AIProjectClient for Foundry with Azure Identity authentication.
|
||||
DefaultAzureCredential credential = new();
|
||||
AIProjectClient projectClient = new(new Uri(foundryEndpoint), credential);
|
||||
|
||||
// Get the ChatClient from the AIProjectClient's OpenAI property using the deployment name.
|
||||
// The stateInitializer can be used to customize the Foundry Memory scope per session and it will be called each time a session
|
||||
// is encountered by the FoundryMemoryProvider that does not already have state stored on the session.
|
||||
// If each session should have its own scope, you can create a new id per session via the stateInitializer, e.g.:
|
||||
// new FoundryMemoryProvider(projectClient, memoryStoreName, stateInitializer: _ => new(new FoundryMemoryProviderScope(Guid.NewGuid().ToString())), ...)
|
||||
// In our case we are storing memories scoped by user so that memories are retained across sessions.
|
||||
FoundryMemoryProvider memoryProvider = new(
|
||||
projectClient,
|
||||
memoryStoreName,
|
||||
stateInitializer: _ => new(new FoundryMemoryProviderScope("sample-user-123")));
|
||||
|
||||
AIAgent agent = await projectClient.CreateAIAgentAsync(deploymentName,
|
||||
options: new ChatClientAgentOptions()
|
||||
{
|
||||
Name = "TravelAssistantWithFoundryMemory",
|
||||
ChatOptions = new() { Instructions = "You are a friendly travel assistant. Use known memories about the user when responding, and do not invent details." },
|
||||
AIContextProviders = [memoryProvider]
|
||||
});
|
||||
|
||||
AgentSession session = await agent.CreateSessionAsync();
|
||||
|
||||
Console.WriteLine("\n>> Setting up Foundry Memory Store\n");
|
||||
|
||||
// Ensure the memory store exists (creates it with the specified models if needed).
|
||||
await memoryProvider.EnsureMemoryStoreCreatedAsync(deploymentName, embeddingModelName, "Sample memory store for travel assistant");
|
||||
|
||||
// Clear any existing memories for this scope to demonstrate fresh behavior.
|
||||
await memoryProvider.EnsureStoredMemoriesDeletedAsync(session);
|
||||
|
||||
Console.WriteLine(await agent.RunAsync("Hi there! My name is Taylor and I'm planning a hiking trip to Patagonia in November.", session));
|
||||
Console.WriteLine(await agent.RunAsync("I'm travelling with my sister and we love finding scenic viewpoints.", session));
|
||||
|
||||
// Memory extraction in Azure AI Foundry is asynchronous and takes time to process.
|
||||
// WhenUpdatesCompletedAsync polls all pending updates and waits for them to complete.
|
||||
Console.WriteLine("\nWaiting for Foundry Memory to process updates...");
|
||||
await memoryProvider.WhenUpdatesCompletedAsync();
|
||||
|
||||
Console.WriteLine("Updates completed.\n");
|
||||
|
||||
Console.WriteLine(await agent.RunAsync("What do you already know about my upcoming trip?", session));
|
||||
|
||||
Console.WriteLine("\n>> Serialize and deserialize the session to demonstrate persisted state\n");
|
||||
JsonElement serializedSession = await agent.SerializeSessionAsync(session);
|
||||
AgentSession restoredSession = await agent.DeserializeSessionAsync(serializedSession);
|
||||
Console.WriteLine(await agent.RunAsync("Can you recap the personal details you remember?", restoredSession));
|
||||
|
||||
Console.WriteLine("\n>> Start a new session that shares the same Foundry Memory scope\n");
|
||||
|
||||
Console.WriteLine("\nWaiting for Foundry Memory to process updates...");
|
||||
await memoryProvider.WhenUpdatesCompletedAsync();
|
||||
|
||||
AgentSession newSession = await agent.CreateSessionAsync();
|
||||
Console.WriteLine(await agent.RunAsync("Summarize what you already know about me.", newSession));
|
||||
-57
@@ -1,57 +0,0 @@
|
||||
# Agent with Memory Using Azure AI Foundry
|
||||
|
||||
This sample demonstrates how to create and run an agent that uses Azure AI Foundry's managed memory service to extract and retrieve individual memories across sessions.
|
||||
|
||||
## Features Demonstrated
|
||||
|
||||
- Creating a `FoundryMemoryProvider` with Azure Identity authentication
|
||||
- Automatic memory store creation if it doesn't exist
|
||||
- Multi-turn conversations with automatic memory extraction
|
||||
- Memory retrieval to inform agent responses
|
||||
- Session serialization and deserialization
|
||||
- Memory persistence across completely new sessions
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. Azure subscription with Azure AI Foundry project
|
||||
2. Azure OpenAI resource with a chat model deployment (e.g., gpt-4o-mini) and an embedding model deployment (e.g., text-embedding-ada-002)
|
||||
3. .NET 10.0 SDK
|
||||
4. Azure CLI logged in (`az login`)
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
# Azure AI Foundry project endpoint and memory store name
|
||||
export AZURE_AI_PROJECT_ENDPOINT="https://your-account.services.ai.azure.com/api/projects/your-project"
|
||||
export AZURE_AI_MEMORY_STORE_ID="my_memory_store"
|
||||
|
||||
# Model deployment names (models deployed in your Foundry project)
|
||||
export AZURE_AI_MODEL_DEPLOYMENT_NAME="gpt-4o-mini"
|
||||
export AZURE_AI_EMBEDDING_DEPLOYMENT_NAME="text-embedding-ada-002"
|
||||
```
|
||||
|
||||
## Run the Sample
|
||||
|
||||
```bash
|
||||
dotnet run
|
||||
```
|
||||
|
||||
## Expected Output
|
||||
|
||||
The agent will:
|
||||
1. Create the memory store if it doesn't exist (using the specified chat and embedding models)
|
||||
2. Learn your name (Taylor), travel destination (Patagonia), timing (November), companions (sister), and interests (scenic viewpoints)
|
||||
3. Wait for Foundry Memory to index the memories
|
||||
4. Recall those details when asked about the trip
|
||||
5. Demonstrate memory persistence across session serialization/deserialization
|
||||
6. Show that a brand new session can still access the same memories
|
||||
|
||||
## Key Differences from Mem0
|
||||
|
||||
| Aspect | Mem0 | Azure AI Foundry Memory |
|
||||
|--------|------|------------------------|
|
||||
| Authentication | API Key | Azure Identity (DefaultAzureCredential) |
|
||||
| Scope | ApplicationId, UserId, AgentId, ThreadId | Single `Scope` string |
|
||||
| Memory Types | Single memory store | User Profile + Chat Summary |
|
||||
| Hosting | Mem0 cloud or self-hosted | Azure AI Foundry managed service |
|
||||
| Store Creation | N/A (automatic) | Explicit via `EnsureMemoryStoreCreatedAsync` |
|
||||
-133
@@ -1,133 +0,0 @@
|
||||
// Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
using Microsoft.Agents.AI;
|
||||
using Microsoft.Extensions.AI;
|
||||
using Microsoft.Extensions.VectorData;
|
||||
|
||||
namespace SampleApp;
|
||||
|
||||
/// <summary>
|
||||
/// A <see cref="ChatHistoryProvider"/> that keeps a bounded window of recent messages in session state
|
||||
/// (via <see cref="InMemoryChatHistoryProvider"/>) and overflows older messages to a vector store
|
||||
/// (via <see cref="ChatHistoryMemoryProvider"/>). When providing chat history, it searches the vector
|
||||
/// store for relevant older messages and prepends them as a memory context message.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// Only non-system messages are counted towards the session state limit and overflow mechanism. System messages are always retained in session state and are not included in the vector store.
|
||||
/// Function calls and function results are also dropped when truncation happens, both from in-memory state, and they are also not persisted to the vector store.
|
||||
/// </remarks>
|
||||
internal sealed class BoundedChatHistoryProvider : ChatHistoryProvider, IDisposable
|
||||
{
|
||||
private readonly InMemoryChatHistoryProvider _chatHistoryProvider;
|
||||
private readonly ChatHistoryMemoryProvider _memoryProvider;
|
||||
private readonly TruncatingChatReducer _reducer;
|
||||
private readonly string _contextPrompt;
|
||||
private IReadOnlyList<string>? _stateKeys;
|
||||
|
||||
/// <summary>
|
||||
/// Initializes a new instance of the <see cref="BoundedChatHistoryProvider"/> class.
|
||||
/// </summary>
|
||||
/// <param name="maxSessionMessages">The maximum number of non-system messages to keep in session state before overflowing to the vector store.</param>
|
||||
/// <param name="vectorStore">The vector store to use for storing and retrieving overflow chat history.</param>
|
||||
/// <param name="collectionName">The name of the collection for storing overflow chat history in the vector store.</param>
|
||||
/// <param name="vectorDimensions">The number of dimensions to use for the chat history vector store embeddings.</param>
|
||||
/// <param name="stateInitializer">A delegate that initializes the memory provider state, providing the storage and search scopes.</param>
|
||||
/// <param name="contextPrompt">Optional prompt to prefix memory search results. Defaults to a standard memory context prompt.</param>
|
||||
public BoundedChatHistoryProvider(
|
||||
int maxSessionMessages,
|
||||
VectorStore vectorStore,
|
||||
string collectionName,
|
||||
int vectorDimensions,
|
||||
Func<AgentSession?, ChatHistoryMemoryProvider.State> stateInitializer,
|
||||
string? contextPrompt = null)
|
||||
{
|
||||
if (maxSessionMessages < 0)
|
||||
{
|
||||
throw new ArgumentOutOfRangeException(nameof(maxSessionMessages), "maxSessionMessages must be non-negative.");
|
||||
}
|
||||
|
||||
this._reducer = new TruncatingChatReducer(maxSessionMessages);
|
||||
this._chatHistoryProvider = new InMemoryChatHistoryProvider(new InMemoryChatHistoryProviderOptions
|
||||
{
|
||||
ChatReducer = this._reducer,
|
||||
ReducerTriggerEvent = InMemoryChatHistoryProviderOptions.ChatReducerTriggerEvent.AfterMessageAdded,
|
||||
StorageInputRequestMessageFilter = msgs => msgs,
|
||||
});
|
||||
this._memoryProvider = new ChatHistoryMemoryProvider(
|
||||
vectorStore,
|
||||
collectionName,
|
||||
vectorDimensions,
|
||||
stateInitializer,
|
||||
options: new ChatHistoryMemoryProviderOptions
|
||||
{
|
||||
SearchInputMessageFilter = msgs => msgs,
|
||||
StorageInputRequestMessageFilter = msgs => msgs,
|
||||
});
|
||||
this._contextPrompt = contextPrompt
|
||||
?? "The following are memories from earlier in this conversation. Use them to inform your responses:";
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
public override IReadOnlyList<string> StateKeys => this._stateKeys ??= this._chatHistoryProvider.StateKeys.Concat(this._memoryProvider.StateKeys).ToArray();
|
||||
|
||||
/// <inheritdoc />
|
||||
protected override async ValueTask<IEnumerable<ChatMessage>> ProvideChatHistoryAsync(
|
||||
InvokingContext context,
|
||||
CancellationToken cancellationToken = default)
|
||||
{
|
||||
// Delegate to the inner provider's full lifecycle (retrieve, filter, stamp, merge with request messages).
|
||||
var chatHistoryProviderInputContext = new InvokingContext(context.Agent, context.Session, []);
|
||||
var allMessages = await this._chatHistoryProvider.InvokingAsync(chatHistoryProviderInputContext, cancellationToken).ConfigureAwait(false);
|
||||
|
||||
// Search the vector store for relevant older messages.
|
||||
var aiContext = new AIContext { Messages = context.RequestMessages.ToList() };
|
||||
var invokingContext = new AIContextProvider.InvokingContext(
|
||||
context.Agent, context.Session, aiContext);
|
||||
|
||||
var result = await this._memoryProvider.InvokingAsync(invokingContext, cancellationToken).ConfigureAwait(false);
|
||||
|
||||
// Extract only the messages added by the memory provider (stamped with AIContextProvider source type).
|
||||
var memoryMessages = result.Messages?
|
||||
.Where(m => m.GetAgentRequestMessageSourceType() == AgentRequestMessageSourceType.AIContextProvider)
|
||||
.ToList();
|
||||
|
||||
if (memoryMessages is { Count: > 0 })
|
||||
{
|
||||
var memoryText = string.Join("\n", memoryMessages.Select(m => m.Text).Where(t => !string.IsNullOrWhiteSpace(t)));
|
||||
|
||||
if (!string.IsNullOrWhiteSpace(memoryText))
|
||||
{
|
||||
var contextMessage = new ChatMessage(ChatRole.User, $"{this._contextPrompt}\n{memoryText}");
|
||||
return new[] { contextMessage }.Concat(allMessages);
|
||||
}
|
||||
}
|
||||
|
||||
return allMessages;
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
protected override async ValueTask StoreChatHistoryAsync(
|
||||
InvokedContext context,
|
||||
CancellationToken cancellationToken = default)
|
||||
{
|
||||
// Delegate storage to the in-memory provider. Its TruncatingChatReducer (AfterMessageAdded trigger)
|
||||
// will automatically truncate to the configured maximum and expose any removed messages.
|
||||
var innerContext = new InvokedContext(
|
||||
context.Agent, context.Session, context.RequestMessages, context.ResponseMessages!);
|
||||
await this._chatHistoryProvider.InvokedAsync(innerContext, cancellationToken).ConfigureAwait(false);
|
||||
|
||||
// Archive any messages that the reducer removed to the vector store.
|
||||
if (this._reducer.RemovedMessages is { Count: > 0 })
|
||||
{
|
||||
var overflowContext = new AIContextProvider.InvokedContext(
|
||||
context.Agent, context.Session, this._reducer.RemovedMessages, []);
|
||||
await this._memoryProvider.InvokedAsync(overflowContext, cancellationToken).ConfigureAwait(false);
|
||||
}
|
||||
}
|
||||
|
||||
/// <inheritdoc/>
|
||||
public void Dispose()
|
||||
{
|
||||
this._memoryProvider.Dispose();
|
||||
}
|
||||
}
|
||||
-79
@@ -1,79 +0,0 @@
|
||||
// Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
// This sample shows how to create a bounded chat history provider that keeps a configurable number of
|
||||
// recent messages in session state and automatically overflows older messages to a vector store.
|
||||
// When the agent is invoked, it searches the vector store for relevant older messages and
|
||||
// prepends them as a "memory" context message before the recent session history.
|
||||
|
||||
using Azure.AI.OpenAI;
|
||||
using Azure.Identity;
|
||||
using Microsoft.Agents.AI;
|
||||
using Microsoft.Extensions.AI;
|
||||
using Microsoft.Extensions.VectorData;
|
||||
using Microsoft.SemanticKernel.Connectors.InMemory;
|
||||
using OpenAI.Chat;
|
||||
using SampleApp;
|
||||
|
||||
var endpoint = Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT") ?? throw new InvalidOperationException("AZURE_OPENAI_ENDPOINT is not set.");
|
||||
var deploymentName = Environment.GetEnvironmentVariable("AZURE_OPENAI_DEPLOYMENT_NAME") ?? "gpt-4o-mini";
|
||||
var embeddingDeploymentName = Environment.GetEnvironmentVariable("AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME") ?? "text-embedding-3-large";
|
||||
|
||||
// WARNING: DefaultAzureCredential is convenient for development but requires careful consideration in production.
|
||||
// In production, consider using a specific credential (e.g., ManagedIdentityCredential) to avoid
|
||||
// latency issues, unintended credential probing, and potential security risks from fallback mechanisms.
|
||||
var credential = new DefaultAzureCredential();
|
||||
|
||||
// Create a vector store to store overflow chat messages.
|
||||
// For demonstration purposes, we are using an in-memory vector store.
|
||||
// Replace this with a persistent vector store implementation for production scenarios.
|
||||
VectorStore vectorStore = new InMemoryVectorStore(new InMemoryVectorStoreOptions()
|
||||
{
|
||||
EmbeddingGenerator = new AzureOpenAIClient(new Uri(endpoint), credential)
|
||||
.GetEmbeddingClient(embeddingDeploymentName)
|
||||
.AsIEmbeddingGenerator()
|
||||
});
|
||||
|
||||
var sessionId = Guid.NewGuid().ToString();
|
||||
|
||||
// Create the BoundedChatHistoryProvider with a maximum of 4 non-system messages in session state.
|
||||
// It internally creates an InMemoryChatHistoryProvider with a TruncatingChatReducer and a
|
||||
// ChatHistoryMemoryProvider with the correct configuration to ensure overflow messages are
|
||||
// automatically archived to the vector store and recalled via semantic search.
|
||||
var boundedProvider = new BoundedChatHistoryProvider(
|
||||
maxSessionMessages: 4,
|
||||
vectorStore,
|
||||
collectionName: "chathistory-overflow",
|
||||
vectorDimensions: 3072,
|
||||
session => new ChatHistoryMemoryProvider.State(
|
||||
storageScope: new() { UserId = "UID1", SessionId = sessionId },
|
||||
searchScope: new() { UserId = "UID1" }));
|
||||
|
||||
// Create the agent with the bounded chat history provider.
|
||||
AIAgent agent = new AzureOpenAIClient(new Uri(endpoint), credential)
|
||||
.GetChatClient(deploymentName)
|
||||
.AsAIAgent(new ChatClientAgentOptions
|
||||
{
|
||||
ChatOptions = new() { Instructions = "You are a helpful assistant. Answer questions concisely." },
|
||||
Name = "Assistant",
|
||||
ChatHistoryProvider = boundedProvider,
|
||||
});
|
||||
|
||||
// Start a conversation. The first several exchanges will fill up the session state window.
|
||||
AgentSession session = await agent.CreateSessionAsync();
|
||||
|
||||
Console.WriteLine("--- Filling the session window (4 messages max) ---\n");
|
||||
|
||||
Console.WriteLine(await agent.RunAsync("My favorite color is blue.", session));
|
||||
Console.WriteLine(await agent.RunAsync("I have a dog named Max.", session));
|
||||
|
||||
// At this point the session state holds 4 messages (2 user + 2 assistant).
|
||||
// The next exchange will push the oldest messages into the vector store.
|
||||
Console.WriteLine("\n--- Next exchange will trigger overflow to vector store ---\n");
|
||||
|
||||
Console.WriteLine(await agent.RunAsync("What is the capital of France?", session));
|
||||
|
||||
// The oldest messages about favorite color have now been archived to the vector store.
|
||||
// Ask the agent something that requires recalling the overflowed information.
|
||||
Console.WriteLine("\n--- Asking about overflowed information (should recall from vector store) ---\n");
|
||||
|
||||
Console.WriteLine(await agent.RunAsync("What is my favorite color?", session));
|
||||
-40
@@ -1,40 +0,0 @@
|
||||
# Bounded Chat History with Vector Store Overflow
|
||||
|
||||
This sample demonstrates how to create a custom `ChatHistoryProvider` that keeps a bounded window of recent messages in session state and automatically overflows older messages to a vector store. When the agent is invoked, it searches the vector store for relevant older messages and prepends them as memory context.
|
||||
|
||||
## Concepts
|
||||
|
||||
- **`TruncatingChatReducer`**: A custom `IChatReducer` that keeps the most recent N messages and exposes removed messages via a `RemovedMessages` property.
|
||||
- **`BoundedChatHistoryProvider`**: A custom `ChatHistoryProvider` that composes:
|
||||
- `InMemoryChatHistoryProvider` for fast session-state storage (bounded by the reducer)
|
||||
- `ChatHistoryMemoryProvider` for vector-store overflow and semantic search of older messages
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- [.NET 10 SDK](https://dotnet.microsoft.com/download/dotnet/10.0)
|
||||
- An Azure OpenAI resource with:
|
||||
- A chat deployment (e.g., `gpt-4o-mini`)
|
||||
- An embedding deployment (e.g., `text-embedding-3-large`)
|
||||
|
||||
## Configuration
|
||||
|
||||
Set the following environment variables:
|
||||
|
||||
| Variable | Description | Default |
|
||||
|---|---|---|
|
||||
| `AZURE_OPENAI_ENDPOINT` | Your Azure OpenAI endpoint URL | *(required)* |
|
||||
| `AZURE_OPENAI_DEPLOYMENT_NAME` | Chat model deployment name | `gpt-4o-mini` |
|
||||
| `AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME` | Embedding model deployment name | `text-embedding-3-large` |
|
||||
|
||||
## Running the Sample
|
||||
|
||||
```bash
|
||||
dotnet run
|
||||
```
|
||||
|
||||
## How it Works
|
||||
|
||||
1. The agent starts a conversation with a bounded session window of 4 non-system, non-function messages (i.e., user/assistant turns). System messages are always preserved, and function call/result messages are truncated and not preserved.
|
||||
2. As messages accumulate beyond the limit, the `TruncatingChatReducer` removes the oldest messages.
|
||||
3. The `BoundedChatHistoryProvider` detects the removed messages and stores them in a vector store via `ChatHistoryMemoryProvider`.
|
||||
4. On subsequent invocations, the provider searches the vector store for relevant older messages and prepends them as memory context, allowing the agent to recall information from earlier in the conversation.
|
||||
-65
@@ -1,65 +0,0 @@
|
||||
// Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
using Microsoft.Extensions.AI;
|
||||
|
||||
namespace SampleApp;
|
||||
|
||||
/// <summary>
|
||||
/// A truncating chat reducer that keeps the most recent messages up to a configured maximum,
|
||||
/// preserving any leading system message. Removed messages are exposed via <see cref="RemovedMessages"/>
|
||||
/// so that a caller can archive them (e.g. to a vector store).
|
||||
/// </summary>
|
||||
internal sealed class TruncatingChatReducer : IChatReducer
|
||||
{
|
||||
private readonly int _maxMessages;
|
||||
|
||||
/// <summary>
|
||||
/// Initializes a new instance of the <see cref="TruncatingChatReducer"/> class.
|
||||
/// </summary>
|
||||
/// <param name="maxMessages">The maximum number of non-system messages to retain.</param>
|
||||
public TruncatingChatReducer(int maxMessages)
|
||||
{
|
||||
this._maxMessages = maxMessages > 0 ? maxMessages : throw new ArgumentOutOfRangeException(nameof(maxMessages));
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Gets the messages that were removed during the most recent call to <see cref="ReduceAsync"/>.
|
||||
/// </summary>
|
||||
public IReadOnlyList<ChatMessage> RemovedMessages { get; private set; } = [];
|
||||
|
||||
/// <inheritdoc />
|
||||
public Task<IEnumerable<ChatMessage>> ReduceAsync(IEnumerable<ChatMessage> messages, CancellationToken cancellationToken)
|
||||
{
|
||||
_ = messages ?? throw new ArgumentNullException(nameof(messages));
|
||||
|
||||
ChatMessage? systemMessage = null;
|
||||
Queue<ChatMessage> retained = new(capacity: this._maxMessages);
|
||||
List<ChatMessage> removed = [];
|
||||
|
||||
foreach (var message in messages)
|
||||
{
|
||||
if (message.Role == ChatRole.System)
|
||||
{
|
||||
// Preserve the first system message outside the counting window.
|
||||
systemMessage ??= message;
|
||||
}
|
||||
else if (!message.Contents.Any(c => c is FunctionCallContent or FunctionResultContent))
|
||||
{
|
||||
if (retained.Count >= this._maxMessages)
|
||||
{
|
||||
removed.Add(retained.Dequeue());
|
||||
}
|
||||
|
||||
retained.Enqueue(message);
|
||||
}
|
||||
}
|
||||
|
||||
this.RemovedMessages = removed;
|
||||
|
||||
IEnumerable<ChatMessage> result = systemMessage is not null
|
||||
? new[] { systemMessage }.Concat(retained)
|
||||
: retained;
|
||||
|
||||
return Task.FromResult(result);
|
||||
}
|
||||
}
|
||||
@@ -1,13 +0,0 @@
|
||||
# Agent Framework Retrieval Augmented Generation (RAG)
|
||||
|
||||
These samples show how to create an agent with the Agent Framework that uses Memory to remember previous conversations or facts from previous conversations.
|
||||
|
||||
|Sample|Description|
|
||||
|---|---|
|
||||
|[Chat History memory](./AgentWithMemory_Step01_ChatHistoryMemory/)|This sample demonstrates how to enable an agent to remember messages from previous conversations.|
|
||||
|[Memory with MemoryStore](./AgentWithMemory_Step02_MemoryUsingMem0/)|This sample demonstrates how to create and run an agent that uses the Mem0 service to extract and retrieve individual memories.|
|
||||
|[Custom Memory Implementation](../../01-get-started/04_memory/)|This sample demonstrates how to create a custom memory component and attach it to an agent.|
|
||||
|[Memory with Azure AI Foundry](./AgentWithMemory_Step04_MemoryUsingFoundry/)|This sample demonstrates how to create and run an agent that uses Azure AI Foundry's managed memory service to extract and retrieve individual memories.|
|
||||
|[Bounded Chat History with Overflow](./AgentWithMemory_Step05_BoundedChatHistory/)|This sample demonstrates how to create a bounded chat history provider that overflows older messages to a vector store and recalls them as memories.|
|
||||
|
||||
> **See also**: [Memory Search with Foundry Agents](../FoundryAgents/FoundryAgents_Step22_MemorySearch/) - demonstrates using the built-in Memory Search tool with Azure Foundry Agents.
|
||||
-49
@@ -1,49 +0,0 @@
|
||||
// Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
using Microsoft.Agents.AI;
|
||||
using Microsoft.Extensions.AI;
|
||||
using Microsoft.Extensions.DependencyInjection;
|
||||
|
||||
namespace SampleApp;
|
||||
|
||||
/// <summary>
|
||||
/// Provides extension methods for adding structured output capabilities to <see cref="AIAgentBuilder"/> instances.
|
||||
/// </summary>
|
||||
internal static class AIAgentBuilderExtensions
|
||||
{
|
||||
/// <summary>
|
||||
/// Adds structured output capabilities to the agent pipeline, enabling conversion of text responses to structured JSON format.
|
||||
/// </summary>
|
||||
/// <param name="builder">The <see cref="AIAgentBuilder"/> to which structured output support will be added.</param>
|
||||
/// <param name="chatClient">
|
||||
/// The chat client used to transform text responses into structured JSON format.
|
||||
/// If <see langword="null"/>, the chat client will be resolved from the service provider.
|
||||
/// </param>
|
||||
/// <param name="optionsFactory">
|
||||
/// An optional factory function that returns the <see cref="StructuredOutputAgentOptions"/> instance to use.
|
||||
/// This allows for fine-tuning the structured output behavior such as setting the response format or system message.
|
||||
/// </param>
|
||||
/// <returns>The <see cref="AIAgentBuilder"/> with structured output capabilities added, enabling method chaining.</returns>
|
||||
/// <remarks>
|
||||
/// <para>
|
||||
/// A <see cref="ChatResponseFormatJson"/> must be specified either through the
|
||||
/// <see cref="AgentRunOptions.ResponseFormat"/> at runtime or the <see cref="StructuredOutputAgentOptions.ChatOptions"/>
|
||||
/// provided during configuration.
|
||||
/// </para>
|
||||
/// </remarks>
|
||||
public static AIAgentBuilder UseStructuredOutput(
|
||||
this AIAgentBuilder builder,
|
||||
IChatClient? chatClient = null,
|
||||
Func<StructuredOutputAgentOptions>? optionsFactory = null)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(builder);
|
||||
|
||||
return builder.Use((innerAgent, services) =>
|
||||
{
|
||||
chatClient ??= services?.GetService<IChatClient>()
|
||||
?? throw new InvalidOperationException($"No {nameof(IChatClient)} was provided and none could be resolved from the service provider. Either provide an {nameof(IChatClient)} explicitly or register one in the dependency injection container.");
|
||||
|
||||
return new StructuredOutputAgent(innerAgent, chatClient, optionsFactory?.Invoke());
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,183 +0,0 @@
|
||||
// Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
// This sample shows how to configure ChatClientAgent to produce structured output.
|
||||
|
||||
using System.ComponentModel;
|
||||
using System.Text.Json;
|
||||
using System.Text.Json.Serialization;
|
||||
using Azure.AI.OpenAI;
|
||||
using Azure.Identity;
|
||||
using Microsoft.Agents.AI;
|
||||
using Microsoft.Extensions.AI;
|
||||
using OpenAI.Chat;
|
||||
using SampleApp;
|
||||
using ChatMessage = Microsoft.Extensions.AI.ChatMessage;
|
||||
|
||||
string endpoint = Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT") ?? throw new InvalidOperationException("AZURE_OPENAI_ENDPOINT is not set.");
|
||||
string deploymentName = Environment.GetEnvironmentVariable("AZURE_OPENAI_DEPLOYMENT_NAME") ?? "gpt-4o-mini";
|
||||
|
||||
// Create chat client to be used by chat client agents.
|
||||
// WARNING: DefaultAzureCredential is convenient for development but requires careful consideration in production.
|
||||
// In production, consider using a specific credential (e.g., ManagedIdentityCredential) to avoid
|
||||
// latency issues, unintended credential probing, and potential security risks from fallback mechanisms.
|
||||
ChatClient chatClient = new AzureOpenAIClient(
|
||||
new Uri(endpoint),
|
||||
new DefaultAzureCredential())
|
||||
.GetChatClient(deploymentName);
|
||||
|
||||
// Demonstrates how to work with structured output via ResponseFormat with the non-generic RunAsync method.
|
||||
// This approach is useful when:
|
||||
// a. Structured output is used for inter-agent communication, where one agent produces structured output
|
||||
// and passes it as text to another agent as input, without the need for the caller to directly work with the structured output.
|
||||
// b. The type of the structured output is not known at compile time, so the generic RunAsync<T> method cannot be used.
|
||||
// c. The type of the structured output is represented by JSON schema only, without a corresponding class or type in the code.
|
||||
await UseStructuredOutputWithResponseFormatAsync(chatClient);
|
||||
|
||||
// Demonstrates how to work with structured output via the generic RunAsync<T> method.
|
||||
// This approach is useful when the caller needs to directly work with the structured output in the code
|
||||
// via an instance of the corresponding class or type and the type is known at compile time.
|
||||
await UseStructuredOutputWithRunAsync(chatClient);
|
||||
|
||||
// Demonstrates how to work with structured output when streaming using the RunStreamingAsync method.
|
||||
await UseStructuredOutputWithRunStreamingAsync(chatClient);
|
||||
|
||||
// Demonstrates how to add structured output support to agents that don't natively support it using the structured output middleware.
|
||||
// This approach is useful when working with agents that don't support structured output natively, or agents using models
|
||||
// that don't have the capability to produce structured output, allowing you to still leverage structured output features by transforming
|
||||
// the text output from the agent into structured data using a chat client.
|
||||
await UseStructuredOutputWithMiddlewareAsync(chatClient);
|
||||
|
||||
static async Task UseStructuredOutputWithResponseFormatAsync(ChatClient chatClient)
|
||||
{
|
||||
Console.WriteLine("=== Structured Output with ResponseFormat ===");
|
||||
|
||||
// Create the agent
|
||||
AIAgent agent = chatClient.AsAIAgent(new ChatClientAgentOptions()
|
||||
{
|
||||
Name = "HelpfulAssistant",
|
||||
ChatOptions = new()
|
||||
{
|
||||
Instructions = "You are a helpful assistant.",
|
||||
// Specify CityInfo as the type parameter of ForJsonSchema to indicate the expected structured output from the agent.
|
||||
ResponseFormat = Microsoft.Extensions.AI.ChatResponseFormat.ForJsonSchema<CityInfo>()
|
||||
}
|
||||
});
|
||||
|
||||
// Invoke the agent with some unstructured input to extract the structured information from.
|
||||
AgentResponse response = await agent.RunAsync("Provide information about the capital of France.");
|
||||
|
||||
// Access the structured output via the Text property of the agent response as JSON in scenarios when JSON as text is required
|
||||
// and no object instance is needed (e.g., for logging, forwarding to another service, or storing in a database).
|
||||
Console.WriteLine("Assistant Output (JSON):");
|
||||
Console.WriteLine(response.Text);
|
||||
Console.WriteLine();
|
||||
|
||||
// Deserialize the JSON text to work with the structured object in scenarios when you need to access properties,
|
||||
// perform operations, or pass the data to methods that require the typed object instance.
|
||||
CityInfo cityInfo = JsonSerializer.Deserialize<CityInfo>(response.Text)!;
|
||||
|
||||
Console.WriteLine("Assistant Output (Deserialized):");
|
||||
Console.WriteLine($"Name: {cityInfo.Name}");
|
||||
Console.WriteLine();
|
||||
}
|
||||
|
||||
static async Task UseStructuredOutputWithRunAsync(ChatClient chatClient)
|
||||
{
|
||||
Console.WriteLine("=== Structured Output with RunAsync<T> ===");
|
||||
|
||||
// Create the agent
|
||||
AIAgent agent = chatClient.AsAIAgent(name: "HelpfulAssistant", instructions: "You are a helpful assistant.");
|
||||
|
||||
// Set CityInfo as the type parameter of RunAsync method to specify the expected structured output from the agent and invoke it with some unstructured input.
|
||||
AgentResponse<CityInfo> response = await agent.RunAsync<CityInfo>("Provide information about the capital of France.");
|
||||
|
||||
// Access the structured output via the Result property of the agent response.
|
||||
CityInfo cityInfo = response.Result;
|
||||
|
||||
Console.WriteLine("Assistant Output:");
|
||||
Console.WriteLine($"Name: {cityInfo.Name}");
|
||||
Console.WriteLine();
|
||||
}
|
||||
|
||||
static async Task UseStructuredOutputWithRunStreamingAsync(ChatClient chatClient)
|
||||
{
|
||||
Console.WriteLine("=== Structured Output with RunStreamingAsync ===");
|
||||
|
||||
// Create the agent
|
||||
AIAgent agent = chatClient.AsAIAgent(new ChatClientAgentOptions()
|
||||
{
|
||||
Name = "HelpfulAssistant",
|
||||
ChatOptions = new()
|
||||
{
|
||||
Instructions = "You are a helpful assistant.",
|
||||
// Specify CityInfo as the type parameter of ForJsonSchema to indicate the expected structured output from the agent.
|
||||
ResponseFormat = Microsoft.Extensions.AI.ChatResponseFormat.ForJsonSchema<CityInfo>()
|
||||
}
|
||||
});
|
||||
|
||||
// Invoke the agent with some unstructured input while streaming, to extract the structured information from.
|
||||
IAsyncEnumerable<AgentResponseUpdate> updates = agent.RunStreamingAsync("Provide information about the capital of France.");
|
||||
|
||||
// Assemble all the parts of the streamed output.
|
||||
AgentResponse nonGenericResponse = await updates.ToAgentResponseAsync();
|
||||
|
||||
// Access the structured output by deserializing JSON in the Text property.
|
||||
CityInfo cityInfo = JsonSerializer.Deserialize<CityInfo>(nonGenericResponse.Text)!;
|
||||
|
||||
Console.WriteLine("Assistant Output:");
|
||||
Console.WriteLine($"Name: {cityInfo.Name}");
|
||||
Console.WriteLine();
|
||||
}
|
||||
|
||||
static async Task UseStructuredOutputWithMiddlewareAsync(ChatClient chatClient)
|
||||
{
|
||||
Console.WriteLine("=== Structured Output with UseStructuredOutput Middleware ===");
|
||||
|
||||
// Create chat client that will transform the agent text response into structured output.
|
||||
IChatClient meaiChatClient = chatClient.AsIChatClient();
|
||||
|
||||
// Create the agent
|
||||
AIAgent agent = meaiChatClient.AsAIAgent(name: "HelpfulAssistant", instructions: "You are a helpful assistant.");
|
||||
|
||||
// Add structured output middleware via UseStructuredOutput method to add structured output support to the agent.
|
||||
// This middleware transforms the agent's text response into structured data using a chat client.
|
||||
// Since our agent does support structured output natively, we will add a middleware that removes ResponseFormat
|
||||
// from the AgentRunOptions to emulate an agent that doesn't support structured output natively
|
||||
agent = agent
|
||||
.AsBuilder()
|
||||
.UseStructuredOutput(meaiChatClient)
|
||||
.Use(ResponseFormatRemovalMiddleware, null)
|
||||
.Build();
|
||||
|
||||
// Set CityInfo as the type parameter of RunAsync method to specify the expected structured output from the agent and invoke it with some unstructured input.
|
||||
AgentResponse<CityInfo> response = await agent.RunAsync<CityInfo>("Provide information about the capital of France.");
|
||||
|
||||
// Access the structured output via the Result property of the agent response.
|
||||
CityInfo cityInfo = response.Result;
|
||||
|
||||
Console.WriteLine("Assistant Output:");
|
||||
Console.WriteLine($"Name: {cityInfo.Name}");
|
||||
Console.WriteLine();
|
||||
}
|
||||
|
||||
static Task<AgentResponse> ResponseFormatRemovalMiddleware(IEnumerable<ChatMessage> messages, AgentSession? session, AgentRunOptions? options, AIAgent innerAgent, CancellationToken cancellationToken)
|
||||
{
|
||||
// Remove any ResponseFormat from the options to emulate an agent that doesn't support structured output natively.
|
||||
options = options?.Clone();
|
||||
options?.ResponseFormat = null;
|
||||
|
||||
return innerAgent.RunAsync(messages, session, options, cancellationToken);
|
||||
}
|
||||
|
||||
namespace SampleApp
|
||||
{
|
||||
/// <summary>
|
||||
/// Represents information about a city, including its name.
|
||||
/// </summary>
|
||||
[Description("Information about a city")]
|
||||
public sealed class CityInfo
|
||||
{
|
||||
[JsonPropertyName("name")]
|
||||
public string? Name { get; set; }
|
||||
}
|
||||
}
|
||||
@@ -1,52 +0,0 @@
|
||||
# Structured Output with ChatClientAgent
|
||||
|
||||
This sample demonstrates how to configure ChatClientAgent to produce structured output in JSON format using various approaches.
|
||||
|
||||
## What this sample demonstrates
|
||||
|
||||
- **ResponseFormat approach**: Configuring agents with JSON schema response format via `ChatResponseFormat.ForJsonSchema<T>()` for inter-agent communication or when the type is not known at compile time
|
||||
- **Generic RunAsync<T> method**: Using the generic `RunAsync<T>` method for structured output when the caller needs to work directly with typed objects
|
||||
- **Structured output with Streaming**: Using `RunStreamingAsync` to stream responses while still obtaining structured output by assembling and deserializing the streamed content
|
||||
- **StructuredOutput middleware**: Adding structured output support to agents that don't natively support it (like A2A agents or models without structured output capability) by transforming text output into structured data using a chat client
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Before you begin, ensure you have the following prerequisites:
|
||||
|
||||
- .NET 10 SDK or later
|
||||
- Azure OpenAI service endpoint and deployment configured
|
||||
- Azure CLI installed and authenticated (for Azure credential authentication)
|
||||
- User has the `Cognitive Services OpenAI Contributor` role for the Azure OpenAI resource
|
||||
|
||||
**Note**: This sample uses Azure OpenAI models. For more information, see [how to deploy Azure OpenAI models with Azure AI Foundry](https://learn.microsoft.com/en-us/azure/ai-foundry/how-to/deploy-models-openai).
|
||||
|
||||
**Note**: This demo uses Azure CLI credentials for authentication. Make sure you're logged in with `az login` and have access to the Azure OpenAI resource and have the `Cognitive Services OpenAI Contributor` role. For more information, see the [Azure CLI documentation](https://learn.microsoft.com/cli/azure/authenticate-azure-cli-interactively).
|
||||
|
||||
## Environment Variables
|
||||
|
||||
Set the following environment variables:
|
||||
|
||||
```powershell
|
||||
$env:AZURE_OPENAI_ENDPOINT="https://your-resource.openai.azure.com/" # Replace with your Azure OpenAI resource endpoint
|
||||
$env:AZURE_OPENAI_DEPLOYMENT_NAME="gpt-4o-mini" # Optional, defaults to gpt-4o-mini
|
||||
```
|
||||
|
||||
## Run the sample
|
||||
|
||||
Navigate to the sample directory and run:
|
||||
|
||||
```powershell
|
||||
cd dotnet/samples/02-agents/Agents/Agent_Step02_StructuredOutput
|
||||
dotnet run
|
||||
```
|
||||
|
||||
## Expected behavior
|
||||
|
||||
The sample will demonstrate four different approaches to structured output:
|
||||
|
||||
1. **Structured Output with ResponseFormat**: Creates an agent with `ResponseFormat` set to `ForJsonSchema<CityInfo>()`, invokes it with unstructured input, and accesses the structured output via the `Text` property
|
||||
2. **Structured Output with RunAsync<T>**: Creates an agent and uses the generic `RunAsync<CityInfo>()` method to get a typed `AgentResponse<CityInfo>` with the result accessible via the `Result` property
|
||||
3. **Structured Output with RunStreamingAsync**: Creates an agent with JSON schema response format, streams the response using `RunStreamingAsync`, assembles the updates using `ToAgentResponseAsync()`, and deserializes the JSON text into a typed object
|
||||
4. **Structured Output with StructuredOutput Middleware**: Uses the `UseStructuredOutput` method on `AIAgentBuilder` to add structured output support to agents that don't natively support it
|
||||
|
||||
Each approach will output information about the capital of France (Paris) in a structured format.
|
||||
-88
@@ -1,88 +0,0 @@
|
||||
// Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
using Microsoft.Agents.AI;
|
||||
using Microsoft.Extensions.AI;
|
||||
|
||||
namespace SampleApp;
|
||||
|
||||
/// <summary>
|
||||
/// A delegating AI agent that converts text responses from an inner AI agent into structured output using a chat client.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// <para>
|
||||
/// The <see cref="StructuredOutputAgent"/> wraps an inner agent and uses a chat client to transform
|
||||
/// the inner agent's text response into a structured JSON format based on the specified response format.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// This agent requires a <see cref="ChatResponseFormatJson"/> to be specified either through the
|
||||
/// <see cref="AgentRunOptions.ResponseFormat"/> or the <see cref="StructuredOutputAgentOptions.ChatOptions"/>
|
||||
/// provided during construction.
|
||||
/// </para>
|
||||
/// </remarks>
|
||||
internal sealed class StructuredOutputAgent : DelegatingAIAgent
|
||||
{
|
||||
private readonly IChatClient _chatClient;
|
||||
private readonly StructuredOutputAgentOptions? _agentOptions;
|
||||
|
||||
/// <summary>
|
||||
/// Initializes a new instance of the <see cref="StructuredOutputAgent"/> class.
|
||||
/// </summary>
|
||||
/// <param name="innerAgent">The underlying agent that generates text responses to be converted to structured output.</param>
|
||||
/// <param name="chatClient">The chat client used to transform text responses into structured JSON format.</param>
|
||||
/// <param name="options">Optional configuration options for the structured output agent.</param>
|
||||
public StructuredOutputAgent(AIAgent innerAgent, IChatClient chatClient, StructuredOutputAgentOptions? options = null)
|
||||
: base(innerAgent)
|
||||
{
|
||||
this._chatClient = chatClient ?? throw new ArgumentNullException(nameof(chatClient));
|
||||
this._agentOptions = options;
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
protected override async Task<AgentResponse> RunCoreAsync(
|
||||
IEnumerable<ChatMessage> messages,
|
||||
AgentSession? session = null,
|
||||
AgentRunOptions? options = null,
|
||||
CancellationToken cancellationToken = default)
|
||||
{
|
||||
// Run the inner agent first, to get back the text response we want to convert.
|
||||
var textResponse = await this.InnerAgent.RunAsync(messages, session, options, cancellationToken).ConfigureAwait(false);
|
||||
|
||||
// Invoke the chat client to transform the text output into structured data.
|
||||
ChatResponse soResponse = await this._chatClient.GetResponseAsync(
|
||||
messages: this.GetChatMessages(textResponse.Text),
|
||||
options: this.GetChatOptions(options),
|
||||
cancellationToken: cancellationToken).ConfigureAwait(false);
|
||||
|
||||
return new StructuredOutputAgentResponse(soResponse, textResponse);
|
||||
}
|
||||
|
||||
private List<ChatMessage> GetChatMessages(string? textResponseText)
|
||||
{
|
||||
List<ChatMessage> chatMessages = [];
|
||||
|
||||
if (this._agentOptions?.ChatClientSystemMessage is not null)
|
||||
{
|
||||
chatMessages.Add(new ChatMessage(ChatRole.System, this._agentOptions.ChatClientSystemMessage));
|
||||
}
|
||||
|
||||
chatMessages.Add(new ChatMessage(ChatRole.User, textResponseText));
|
||||
|
||||
return chatMessages;
|
||||
}
|
||||
|
||||
private ChatOptions GetChatOptions(AgentRunOptions? options)
|
||||
{
|
||||
ChatResponseFormat responseFormat = options?.ResponseFormat
|
||||
?? this._agentOptions?.ChatOptions?.ResponseFormat
|
||||
?? throw new InvalidOperationException($"A response format of type '{nameof(ChatResponseFormatJson)}' must be specified, but none was specified.");
|
||||
|
||||
if (responseFormat is not ChatResponseFormatJson jsonResponseFormat)
|
||||
{
|
||||
throw new NotSupportedException($"A response format of type '{nameof(ChatResponseFormatJson)}' must be specified, but was '{responseFormat.GetType().Name}'.");
|
||||
}
|
||||
|
||||
var chatOptions = this._agentOptions?.ChatOptions?.Clone() ?? new ChatOptions();
|
||||
chatOptions.ResponseFormat = jsonResponseFormat;
|
||||
return chatOptions;
|
||||
}
|
||||
}
|
||||
-31
@@ -1,31 +0,0 @@
|
||||
// Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
using Microsoft.Agents.AI;
|
||||
using Microsoft.Extensions.AI;
|
||||
|
||||
namespace SampleApp;
|
||||
|
||||
/// <summary>
|
||||
/// Represents configuration options for a <see cref="StructuredOutputAgent"/>.
|
||||
/// </summary>
|
||||
#pragma warning disable CA1812 // Instantiated via AIAgentBuilderExtensions.UseStructuredOutput optionsFactory parameter
|
||||
internal sealed class StructuredOutputAgentOptions
|
||||
#pragma warning restore CA1812
|
||||
{
|
||||
/// <summary>
|
||||
/// Gets or sets the system message to use when invoking the chat client for structured output conversion.
|
||||
/// </summary>
|
||||
public string? ChatClientSystemMessage { get; set; }
|
||||
|
||||
/// <summary>
|
||||
/// Gets or sets the chat options to use for the structured output conversion by the chat client
|
||||
/// used by the agent.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// This property is optional. The <see cref="ChatOptions.ResponseFormat"/> should be set to a
|
||||
/// <see cref="ChatResponseFormatJson"/> instance to specify the expected JSON schema for the structured output.
|
||||
/// Note that if <see cref="AgentRunOptions.ResponseFormat"/> is provided when running the agent,
|
||||
/// it will take precedence and override the <see cref="ChatOptions.ResponseFormat"/> specified here.
|
||||
/// </remarks>
|
||||
public ChatOptions? ChatOptions { get; set; }
|
||||
}
|
||||
-28
@@ -1,28 +0,0 @@
|
||||
// Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
using Microsoft.Agents.AI;
|
||||
using Microsoft.Extensions.AI;
|
||||
|
||||
namespace SampleApp;
|
||||
|
||||
/// <summary>
|
||||
/// Represents an agent response that contains structured output and
|
||||
/// the original agent response from which the structured output was generated.
|
||||
/// </summary>
|
||||
internal sealed class StructuredOutputAgentResponse : AgentResponse
|
||||
{
|
||||
/// <summary>
|
||||
/// Initializes a new instance of the <see cref="StructuredOutputAgentResponse"/> class.
|
||||
/// </summary>
|
||||
/// <param name="chatResponse">The <see cref="ChatResponse"/> containing the structured output.</param>
|
||||
/// <param name="agentResponse">The original <see cref="AgentResponse"/> from the inner agent.</param>
|
||||
public StructuredOutputAgentResponse(ChatResponse chatResponse, AgentResponse agentResponse) : base(chatResponse)
|
||||
{
|
||||
this.OriginalResponse = agentResponse;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Gets the original non-structured response from the inner agent used by chat client to produce the structured output.
|
||||
/// </summary>
|
||||
public AgentResponse OriginalResponse { get; }
|
||||
}
|
||||
@@ -1,72 +0,0 @@
|
||||
// Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
// This sample shows how to use a chat history reducer to keep the context within model size limits.
|
||||
// Any implementation of Microsoft.Extensions.AI.IChatReducer can be used to customize how the chat history is reduced.
|
||||
// NOTE: this feature is only supported where the chat history is stored locally, such as with OpenAI Chat Completion.
|
||||
// Where the chat history is stored server side, such as with Azure Foundry Agents, the service must manage the chat history size.
|
||||
|
||||
using Azure.AI.OpenAI;
|
||||
using Azure.Identity;
|
||||
using Microsoft.Agents.AI;
|
||||
using Microsoft.Extensions.AI;
|
||||
using OpenAI.Chat;
|
||||
|
||||
var endpoint = Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT") ?? throw new InvalidOperationException("AZURE_OPENAI_ENDPOINT is not set.");
|
||||
var deploymentName = Environment.GetEnvironmentVariable("AZURE_OPENAI_DEPLOYMENT_NAME") ?? "gpt-4o-mini";
|
||||
|
||||
// Construct the agent, and provide a factory to create an in-memory chat message store with a reducer that keeps only the last 2 non-system messages.
|
||||
// WARNING: DefaultAzureCredential is convenient for development but requires careful consideration in production.
|
||||
// In production, consider using a specific credential (e.g., ManagedIdentityCredential) to avoid
|
||||
// latency issues, unintended credential probing, and potential security risks from fallback mechanisms.
|
||||
AIAgent agent = new AzureOpenAIClient(
|
||||
new Uri(endpoint),
|
||||
new DefaultAzureCredential())
|
||||
.GetChatClient(deploymentName)
|
||||
.AsAIAgent(new ChatClientAgentOptions
|
||||
{
|
||||
ChatOptions = new() { Instructions = "You are good at telling jokes." },
|
||||
Name = "Joker",
|
||||
ChatHistoryProvider = new InMemoryChatHistoryProvider(new() { ChatReducer = new MessageCountingChatReducer(2) })
|
||||
});
|
||||
|
||||
AgentSession session = await agent.CreateSessionAsync();
|
||||
|
||||
// Invoke the agent and output the text result.
|
||||
Console.WriteLine(await agent.RunAsync("Tell me a joke about a pirate.", session));
|
||||
|
||||
// Get the chat history to see how many messages are stored.
|
||||
// We can use the ChatHistoryProvider, that is also used by the agent, to read the
|
||||
// chat history from the session state, and see how the reducer is affecting the stored messages.
|
||||
// Here we expect to see 2 messages, the original user message and the agent response message.
|
||||
if (session.TryGetInMemoryChatHistory(out var chatHistory))
|
||||
{
|
||||
Console.WriteLine($"\nChat history has {chatHistory.Count} messages.\n");
|
||||
}
|
||||
|
||||
// Invoke the agent a few more times.
|
||||
Console.WriteLine(await agent.RunAsync("Tell me a joke about a robot.", session));
|
||||
|
||||
// Now we expect to see 4 messages in the chat history, 2 input and 2 output.
|
||||
// While the target number of messages is 2, the default time for the InMemoryChatHistoryProvider
|
||||
// to trigger the reducer is just before messages are contributed to a new agent run.
|
||||
// So at this time, we have not yet triggered the reducer for the most recently added messages,
|
||||
// and they are still in the chat history.
|
||||
if (session.TryGetInMemoryChatHistory(out chatHistory))
|
||||
{
|
||||
Console.WriteLine($"\nChat history has {chatHistory.Count} messages.\n");
|
||||
}
|
||||
|
||||
Console.WriteLine(await agent.RunAsync("Tell me a joke about a lemur.", session));
|
||||
if (session.TryGetInMemoryChatHistory(out chatHistory))
|
||||
{
|
||||
Console.WriteLine($"\nChat history has {chatHistory.Count} messages.\n");
|
||||
}
|
||||
|
||||
// At this point, the chat history has exceeded the limit and the original message will not exist anymore,
|
||||
// so asking a follow up question about it may not work as expected.
|
||||
Console.WriteLine(await agent.RunAsync("What was the first joke I asked you to tell again?", session));
|
||||
|
||||
if (session.TryGetInMemoryChatHistory(out chatHistory))
|
||||
{
|
||||
Console.WriteLine($"\nChat history has {chatHistory.Count} messages.\n");
|
||||
}
|
||||
@@ -1,167 +0,0 @@
|
||||
// Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
// This sample shows how to inject additional AI context into a ChatClientAgent using custom AIContextProvider components that are attached to the agent.
|
||||
// Multiple providers can be attached to an agent, and they will be called in sequence, each receiving the accumulated context from the previous one.
|
||||
// This mechanism can be used for various purposes, such as injecting RAG search results or memories into the agent's context.
|
||||
// Also note that Agent Framework already provides built-in AIContextProviders for many of these scenarios.
|
||||
|
||||
#pragma warning disable CA1869 // Cache and reuse 'JsonSerializerOptions' instances
|
||||
|
||||
using System.Text;
|
||||
using System.Text.Json;
|
||||
using Azure.AI.OpenAI;
|
||||
using Azure.Identity;
|
||||
using Microsoft.Agents.AI;
|
||||
using Microsoft.Extensions.AI;
|
||||
using OpenAI.Chat;
|
||||
using SampleApp;
|
||||
using MEAI = Microsoft.Extensions.AI;
|
||||
|
||||
var endpoint = Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT") ?? throw new InvalidOperationException("AZURE_OPENAI_ENDPOINT is not set.");
|
||||
var deploymentName = Environment.GetEnvironmentVariable("AZURE_OPENAI_DEPLOYMENT_NAME") ?? "gpt-5-mini";
|
||||
|
||||
// A sample function to load the next three calendar events for the user.
|
||||
Func<Task<string[]>> loadNextThreeCalendarEvents = async () =>
|
||||
{
|
||||
// In a real implementation, this method would connect to a calendar service
|
||||
return new string[]
|
||||
{
|
||||
"Doctor's appointment today at 15:00",
|
||||
"Team meeting today at 17:00",
|
||||
"Birthday party today at 20:00"
|
||||
};
|
||||
};
|
||||
|
||||
// Create an agent with an AI context provider attached that aggregates two other providers:
|
||||
// WARNING: DefaultAzureCredential is convenient for development but requires careful consideration in production.
|
||||
// In production, consider using a specific credential (e.g., ManagedIdentityCredential) to avoid
|
||||
// latency issues, unintended credential probing, and potential security risks from fallback mechanisms.
|
||||
AIAgent agent = new AzureOpenAIClient(
|
||||
new Uri(endpoint),
|
||||
new DefaultAzureCredential())
|
||||
.GetChatClient(deploymentName)
|
||||
.AsAIAgent(new ChatClientAgentOptions()
|
||||
{
|
||||
ChatOptions = new() { Instructions = """
|
||||
You are a helpful personal assistant.
|
||||
You manage a TODO list for the user. When the user has completed one of the tasks it can be removed from the TODO list. Only provide the list of TODO items if asked.
|
||||
You remind users of upcoming calendar events when the user interacts with you.
|
||||
""" },
|
||||
ChatHistoryProvider = new InMemoryChatHistoryProvider(new InMemoryChatHistoryProviderOptions
|
||||
{
|
||||
// Use StorageInputRequestMessageFilter to provide a custom filter for request messages stored in chat history.
|
||||
// By default the chat history provider will store all messages, except for those that came from chat history in the first place.
|
||||
// In this case, we want to also exclude messages that came from AI context providers.
|
||||
// You may want to store these messages, depending on their content and your requirements.
|
||||
StorageInputRequestMessageFilter = messages => messages.Where(m => m.GetAgentRequestMessageSourceType() != AgentRequestMessageSourceType.AIContextProvider && m.GetAgentRequestMessageSourceType() != AgentRequestMessageSourceType.ChatHistory)
|
||||
}),
|
||||
// Add multiple AI context providers: one that maintains a todo list and one that provides upcoming calendar entries.
|
||||
// The agent will call each provider in sequence, accumulating context from each.
|
||||
AIContextProviders = [
|
||||
new TodoListAIContextProvider(),
|
||||
new CalendarSearchAIContextProvider(loadNextThreeCalendarEvents)
|
||||
],
|
||||
});
|
||||
|
||||
// Invoke the agent and output the text result.
|
||||
AgentSession session = await agent.CreateSessionAsync();
|
||||
Console.WriteLine(await agent.RunAsync("I need to pick up milk from the supermarket.", session) + "\n");
|
||||
Console.WriteLine(await agent.RunAsync("I need to take Sally for soccer practice.", session) + "\n");
|
||||
Console.WriteLine(await agent.RunAsync("I need to make a dentist appointment for Jimmy.", session) + "\n");
|
||||
Console.WriteLine(await agent.RunAsync("I've taken Sally to soccer practice.", session) + "\n");
|
||||
|
||||
// We can serialize the session, and it will contain both the chat history and the data that each AI context provider serialized.
|
||||
JsonElement serializedSession = await agent.SerializeSessionAsync(session);
|
||||
// Let's print it to console to show the contents.
|
||||
Console.WriteLine(JsonSerializer.Serialize(serializedSession, options: new JsonSerializerOptions() { WriteIndented = true, IndentSize = 2 }) + "\n");
|
||||
// The serialized session can be stored long term in a persistent store, but in this case we will just deserialize again and continue the conversation.
|
||||
session = await agent.DeserializeSessionAsync(serializedSession);
|
||||
|
||||
Console.WriteLine(await agent.RunAsync("Considering my appointments, can you create a plan for my day that plans out when I should complete the items on my todo list?", session) + "\n");
|
||||
|
||||
namespace SampleApp
|
||||
{
|
||||
/// <summary>
|
||||
/// An <see cref="AIContextProvider"/>, which maintains a todo list for the agent.
|
||||
/// </summary>
|
||||
internal sealed class TodoListAIContextProvider : AIContextProvider
|
||||
{
|
||||
private static List<string> GetTodoItems(AgentSession? session)
|
||||
=> session?.StateBag.GetValue<List<string>>(nameof(TodoListAIContextProvider)) ?? new List<string>();
|
||||
|
||||
private static void SetTodoItems(AgentSession? session, List<string> items)
|
||||
=> session?.StateBag.SetValue(nameof(TodoListAIContextProvider), items);
|
||||
|
||||
protected override ValueTask<AIContext> ProvideAIContextAsync(InvokingContext context, CancellationToken cancellationToken = default)
|
||||
{
|
||||
var todoItems = GetTodoItems(context.Session);
|
||||
|
||||
StringBuilder outputMessageBuilder = new();
|
||||
outputMessageBuilder.AppendLine("Your todo list contains the following items:");
|
||||
|
||||
if (todoItems.Count == 0)
|
||||
{
|
||||
outputMessageBuilder.AppendLine(" (no items)");
|
||||
}
|
||||
else
|
||||
{
|
||||
for (int i = 0; i < todoItems.Count; i++)
|
||||
{
|
||||
outputMessageBuilder.AppendLine($"{i}. {todoItems[i]}");
|
||||
}
|
||||
}
|
||||
|
||||
return new ValueTask<AIContext>(new AIContext
|
||||
{
|
||||
Tools =
|
||||
[
|
||||
AIFunctionFactory.Create((string item) => AddTodoItem(context.Session, item), "AddTodoItem", "Adds an item to the todo list."),
|
||||
AIFunctionFactory.Create((int index) => RemoveTodoItem(context.Session, index), "RemoveTodoItem", "Removes an item from the todo list. Index is zero based.")
|
||||
],
|
||||
Messages =
|
||||
[
|
||||
new MEAI.ChatMessage(ChatRole.User, outputMessageBuilder.ToString())
|
||||
]
|
||||
});
|
||||
}
|
||||
|
||||
private static void RemoveTodoItem(AgentSession? session, int index)
|
||||
{
|
||||
var items = GetTodoItems(session);
|
||||
items.RemoveAt(index);
|
||||
SetTodoItems(session, items);
|
||||
}
|
||||
|
||||
private static void AddTodoItem(AgentSession? session, string item)
|
||||
{
|
||||
if (string.IsNullOrWhiteSpace(item))
|
||||
{
|
||||
throw new ArgumentException("Item must have a value");
|
||||
}
|
||||
|
||||
var items = GetTodoItems(session);
|
||||
items.Add(item);
|
||||
SetTodoItems(session, items);
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// A <see cref="MessageAIContextProvider"/> which searches for upcoming calendar events and adds them to the AI context.
|
||||
/// </summary>
|
||||
internal sealed class CalendarSearchAIContextProvider(Func<Task<string[]>> loadNextThreeCalendarEvents) : MessageAIContextProvider
|
||||
{
|
||||
protected override async ValueTask<IEnumerable<MEAI.ChatMessage>> ProvideMessagesAsync(InvokingContext context, CancellationToken cancellationToken = default)
|
||||
{
|
||||
var events = await loadNextThreeCalendarEvents();
|
||||
|
||||
StringBuilder outputMessageBuilder = new();
|
||||
outputMessageBuilder.AppendLine("You have the following upcoming calendar events:");
|
||||
foreach (var calendarEvent in events)
|
||||
{
|
||||
outputMessageBuilder.AppendLine($" - {calendarEvent}");
|
||||
}
|
||||
|
||||
return [new MEAI.ChatMessage(ChatRole.User, outputMessageBuilder.ToString())];
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,120 +0,0 @@
|
||||
// Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
// This sample demonstrates how to use a CompactionProvider with a compaction pipeline
|
||||
// as an AIContextProvider for an agent's in-run context management. The pipeline chains multiple
|
||||
// compaction strategies from gentle to aggressive:
|
||||
// 1. ToolResultCompactionStrategy - Collapses old tool-call groups into concise summaries
|
||||
// 2. SummarizationCompactionStrategy - LLM-compresses older conversation spans
|
||||
// 3. SlidingWindowCompactionStrategy - Keeps only the most recent N user turns
|
||||
// 4. TruncationCompactionStrategy - Emergency token-budget backstop
|
||||
|
||||
using System.ComponentModel;
|
||||
using Azure.AI.OpenAI;
|
||||
using Azure.Identity;
|
||||
using Microsoft.Agents.AI;
|
||||
using Microsoft.Agents.AI.Compaction;
|
||||
using Microsoft.Extensions.AI;
|
||||
|
||||
var endpoint = Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT") ?? throw new InvalidOperationException("AZURE_OPENAI_ENDPOINT is not set.");
|
||||
var deploymentName = Environment.GetEnvironmentVariable("AZURE_OPENAI_DEPLOYMENT_NAME") ?? "gpt-4o-mini";
|
||||
|
||||
// WARNING: DefaultAzureCredential is convenient for development but requires careful consideration in production.
|
||||
// In production, consider using a specific credential (e.g., ManagedIdentityCredential) to avoid
|
||||
// latency issues, unintended credential probing, and potential security risks from fallback mechanisms.
|
||||
AzureOpenAIClient openAIClient = new(new Uri(endpoint), new DefaultAzureCredential());
|
||||
|
||||
// Create a chat client for the agent and a separate one for the summarization strategy.
|
||||
// Using the same model for simplicity; in production, use a smaller/cheaper model for summarization.
|
||||
IChatClient agentChatClient = openAIClient.GetChatClient(deploymentName).AsIChatClient();
|
||||
IChatClient summarizerChatClient = openAIClient.GetChatClient(deploymentName).AsIChatClient();
|
||||
|
||||
// Define a tool the agent can use, so we can see tool-result compaction in action.
|
||||
[Description("Look up the current price of a product by name.")]
|
||||
static string LookupPrice([Description("The product name to look up.")] string productName) =>
|
||||
productName.ToUpperInvariant() switch
|
||||
{
|
||||
"LAPTOP" => "The laptop costs $999.99.",
|
||||
"KEYBOARD" => "The keyboard costs $79.99.",
|
||||
"MOUSE" => "The mouse costs $29.99.",
|
||||
_ => $"Sorry, I don't have pricing for '{productName}'."
|
||||
};
|
||||
|
||||
// Configure the compaction pipeline with one of each strategy, ordered least to most aggressive.
|
||||
PipelineCompactionStrategy compactionPipeline =
|
||||
new(// 1. Gentle: collapse old tool-call groups into short summaries
|
||||
new ToolResultCompactionStrategy(CompactionTriggers.MessagesExceed(7)),
|
||||
|
||||
// 2. Moderate: use an LLM to summarize older conversation spans into a concise message
|
||||
new SummarizationCompactionStrategy(summarizerChatClient, CompactionTriggers.TokensExceed(0x500)),
|
||||
|
||||
// 3. Aggressive: keep only the last N user turns and their responses
|
||||
new SlidingWindowCompactionStrategy(CompactionTriggers.TurnsExceed(4)),
|
||||
|
||||
// 4. Emergency: drop oldest groups until under the token budget
|
||||
new TruncationCompactionStrategy(CompactionTriggers.TokensExceed(0x8000)));
|
||||
|
||||
// Create the agent with a CompactionProvider that uses the compaction pipeline.
|
||||
AIAgent agent =
|
||||
agentChatClient
|
||||
.AsBuilder()
|
||||
// Note: Adding the CompactionProvider at the builder level means it will be applied to all agents
|
||||
// built from this builder and will manage context for both agent messages and tool calls.
|
||||
.UseAIContextProviders(new CompactionProvider(compactionPipeline))
|
||||
.BuildAIAgent(
|
||||
new ChatClientAgentOptions
|
||||
{
|
||||
Name = "ShoppingAssistant",
|
||||
ChatOptions = new()
|
||||
{
|
||||
Instructions =
|
||||
"""
|
||||
You are a helpful, but long winded, shopping assistant.
|
||||
Help the user look up prices and compare products.
|
||||
When responding, Be sure to be extra descriptive and use as
|
||||
many words as possible without sounding ridiculous.
|
||||
""",
|
||||
Tools = [AIFunctionFactory.Create(LookupPrice)]
|
||||
},
|
||||
// Note: AIContextProviders may be specified here instead of ChatClientBuilder.UseAIContextProviders.
|
||||
// Specifying compaction at the agent level skips compaction in the function calling loop.
|
||||
//AIContextProviders = [new CompactionProvider(compactionPipeline)]
|
||||
});
|
||||
|
||||
AgentSession session = await agent.CreateSessionAsync();
|
||||
|
||||
// Helper to print chat history size
|
||||
void PrintChatHistory()
|
||||
{
|
||||
if (session.TryGetInMemoryChatHistory(out var history))
|
||||
{
|
||||
Console.ForegroundColor = ConsoleColor.Cyan;
|
||||
Console.WriteLine($"\n[Messages: #{history.Count}]\n");
|
||||
Console.ResetColor();
|
||||
}
|
||||
}
|
||||
|
||||
// Run a multi-turn conversation with tool calls to exercise the pipeline.
|
||||
string[] prompts =
|
||||
[
|
||||
"What's the price of a laptop?",
|
||||
"How about a keyboard?",
|
||||
"And a mouse?",
|
||||
"Which product is the cheapest?",
|
||||
"Can you compare the laptop and the keyboard for me?",
|
||||
"What was the first product I asked about?",
|
||||
"Thank you!",
|
||||
];
|
||||
|
||||
foreach (string prompt in prompts)
|
||||
{
|
||||
Console.ForegroundColor = ConsoleColor.Cyan;
|
||||
Console.Write("\n[User] ");
|
||||
Console.ResetColor();
|
||||
Console.WriteLine(prompt);
|
||||
Console.ForegroundColor = ConsoleColor.Cyan;
|
||||
Console.Write("\n[Agent] ");
|
||||
Console.ResetColor();
|
||||
Console.WriteLine(await agent.RunAsync(prompt, session));
|
||||
|
||||
PrintChatHistory();
|
||||
}
|
||||
@@ -1,132 +0,0 @@
|
||||
# Compaction Pipeline
|
||||
|
||||
This sample demonstrates how to use a `CompactionProvider` with a `PipelineCompactionStrategy` to manage long conversation histories in a token-efficient way. The pipeline chains four compaction strategies, ordered from gentle to aggressive, so that the least disruptive strategy runs first and more aggressive strategies only activate when necessary.
|
||||
|
||||
## What This Sample Shows
|
||||
|
||||
- **`CompactionProvider`** — an `AIContextProvider` that applies a compaction strategy before each agent invocation, keeping only the most relevant messages within the model's context window
|
||||
- **`PipelineCompactionStrategy`** — chains multiple compaction strategies into an ordered pipeline; each strategy evaluates its own trigger independently and operates on the output of the previous one
|
||||
- **`ToolResultCompactionStrategy`** — collapses older tool-call groups into concise inline summaries, activated by a message-count trigger
|
||||
- **`SummarizationCompactionStrategy`** — uses an LLM to compress older conversation spans into a single summary message, activated by a token-count trigger
|
||||
- **`SlidingWindowCompactionStrategy`** — retains only the most recent N user turns and their responses, activated by a turn-count trigger
|
||||
- **`TruncationCompactionStrategy`** — emergency backstop that drops the oldest groups until the conversation fits within a hard token budget
|
||||
- **`CompactionTriggers`** — factory methods (`MessagesExceed`, `TokensExceed`, `TurnsExceed`, `GroupsExceed`, `HasToolCalls`, `All`, `Any`) that control when each strategy activates
|
||||
|
||||
## Concepts
|
||||
|
||||
### Message groups
|
||||
|
||||
The compaction engine organizes messages into atomic *groups* that are treated as indivisible units during compaction. A group is either:
|
||||
|
||||
| Group kind | Contents |
|
||||
|---|---|
|
||||
| `System` | System prompt message(s) |
|
||||
| `User` | A single user message |
|
||||
| `ToolCall` | One assistant message with tool calls + the matching tool result messages |
|
||||
| `AssistantText` | A single assistant text-only message |
|
||||
| `Summary` | One or more messages summarizing earlier conversation spans, produced by compaction strategies |
|
||||
|
||||
`Summary` groups (`CompactionGroupKind.Summary`) are created by compaction strategies (for example, `SummarizationCompactionStrategy`) and do not originate directly from user or assistant messages.
|
||||
Strategies exclude entire groups rather than individual messages, preserving the tool-call/result pairing required by most model APIs.
|
||||
|
||||
### Compaction triggers
|
||||
|
||||
A `CompactionTrigger` is a predicate evaluated against the current `MessageIndex`. When the trigger fires, the strategy performs compaction; when it does not fire, the strategy is skipped. Available triggers are:
|
||||
|
||||
| Trigger | Activates when… |
|
||||
|---|---|
|
||||
| `CompactionTriggers.Always` | Always (unconditional) |
|
||||
| `CompactionTriggers.Never` | Never (disabled) |
|
||||
| `CompactionTriggers.MessagesExceed(n)` | Included message count > n |
|
||||
| `CompactionTriggers.TokensExceed(n)` | Included token count > n |
|
||||
| `CompactionTriggers.TurnsExceed(n)` | Included user-turn count > n |
|
||||
| `CompactionTriggers.GroupsExceed(n)` | Included group count > n |
|
||||
| `CompactionTriggers.HasToolCalls()` | At least one included tool-call group exists |
|
||||
| `CompactionTriggers.All(...)` | All supplied triggers fire (logical AND) |
|
||||
| `CompactionTriggers.Any(...)` | Any supplied trigger fires (logical OR) |
|
||||
|
||||
### Pipeline ordering
|
||||
|
||||
Order strategies from **least aggressive** to **most aggressive**. The pipeline runs every strategy whose trigger is met. Earlier strategies reduce the conversation gently so that later, more destructive strategies may not need to activate at all.
|
||||
|
||||
```
|
||||
1. ToolResultCompactionStrategy – gentle: replaces verbose tool results with a short label
|
||||
2. SummarizationCompactionStrategy – moderate: LLM-summarizes older turns
|
||||
3. SlidingWindowCompactionStrategy – aggressive: drops turns beyond the window
|
||||
4. TruncationCompactionStrategy – emergency: hard token-budget enforcement
|
||||
```
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- .NET 10 SDK or later
|
||||
- Azure OpenAI service endpoint and model deployment
|
||||
- Azure CLI installed and authenticated
|
||||
|
||||
**Note**: This sample uses `DefaultAzureCredential`. Sign in with `az login` before running. For production, prefer a specific credential such as `ManagedIdentityCredential`. For more information, see the [Azure CLI authentication documentation](https://learn.microsoft.com/cli/azure/authenticate-azure-cli-interactively).
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```powershell
|
||||
$env:AZURE_OPENAI_ENDPOINT="https://your-resource.openai.azure.com/" # Required
|
||||
$env:AZURE_OPENAI_DEPLOYMENT_NAME="gpt-4o-mini" # Optional, defaults to gpt-4o-mini
|
||||
```
|
||||
|
||||
## Running the Sample
|
||||
|
||||
```powershell
|
||||
cd dotnet/samples/02-agents/Agents/Agent_Step18_CompactionPipeline
|
||||
dotnet run
|
||||
```
|
||||
|
||||
## Expected Behavior
|
||||
|
||||
The sample runs a seven-turn shopping-assistant conversation with tool calls. After each turn it prints the full message count so you can observe the pipeline compaction doesn't alter the source conversation.
|
||||
|
||||
Each of the four compaction strategies has a deliberately low threshold so that it activates during the short demonstration conversation. In a production scenario you would raise the thresholds to match your model's context window and cost requirements.
|
||||
|
||||
## Customizing the Pipeline
|
||||
|
||||
### Using a single strategy
|
||||
|
||||
If you only need one compaction strategy, pass it directly to `CompactionProvider` without wrapping it in a pipeline:
|
||||
|
||||
```csharp
|
||||
CompactionProvider provider =
|
||||
new(new SlidingWindowCompactionStrategy(CompactionTriggers.TurnsExceed(20)));
|
||||
```
|
||||
|
||||
### Ad-hoc compaction outside the provider pipeline
|
||||
|
||||
`CompactionProvider.CompactAsync` applies a strategy to an arbitrary list of messages without an active agent session:
|
||||
|
||||
```csharp
|
||||
IEnumerable<ChatMessage> compacted = await CompactionProvider.CompactAsync(
|
||||
new TruncationCompactionStrategy(CompactionTriggers.TokensExceed(8000)),
|
||||
existingMessages);
|
||||
```
|
||||
|
||||
### Using a different model for summarization
|
||||
|
||||
The `SummarizationCompactionStrategy` accepts any `IChatClient`. Use a smaller, cheaper model to reduce summarization cost:
|
||||
|
||||
```csharp
|
||||
IChatClient summarizerChatClient = openAIClient.GetChatClient("gpt-4o-mini").AsIChatClient();
|
||||
new SummarizationCompactionStrategy(summarizerChatClient, CompactionTriggers.TokensExceed(4000))
|
||||
```
|
||||
|
||||
### Registering through `ChatClientAgentOptions`
|
||||
|
||||
`CompactionProvider` can also be specified directly on `ChatClientAgentOptions` instead of calling `UseAIContextProviders` on the `ChatClientBuilder`:
|
||||
|
||||
```csharp
|
||||
AIAgent agent = agentChatClient
|
||||
.AsBuilder()
|
||||
.BuildAIAgent(new ChatClientAgentOptions
|
||||
{
|
||||
AIContextProviders = [new CompactionProvider(compactionPipeline)]
|
||||
});
|
||||
```
|
||||
|
||||
This places the compaction provider at the agent level instead of the chat client level, which allows you to use different compaction strategies for different agents that share the same chat client.
|
||||
|
||||
> Note: In this mode the `CompactionProvider` is not engaged during the tool calling loop. Agent-level `AIContextProviders` run before chat history is stored, so any synthetic summary messages produced by `CompactionProvider` can become part of the persisted history when using `ChatHistoryProvider`. If you want to compact only the request context while preserving the original stored history, register `CompactionProvider` on the `ChatClientBuilder` via `UseAIContextProviders(...)` instead of on `ChatClientAgentOptions`.
|
||||
-100
@@ -1,100 +0,0 @@
|
||||
// Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
// This sample demonstrates how to use Azure AI Foundry's Red Teaming service to assess
|
||||
// the safety and resilience of an AI model against adversarial attacks.
|
||||
//
|
||||
// It uses the RedTeam API from Azure.AI.Projects to run automated attack simulations
|
||||
// with various attack strategies (encoding, obfuscation, jailbreaks) across multiple
|
||||
// risk categories (Violence, HateUnfairness, Sexual, SelfHarm).
|
||||
//
|
||||
// For more details, see:
|
||||
// https://learn.microsoft.com/azure/ai-foundry/concepts/ai-red-teaming-agent
|
||||
|
||||
using Azure.AI.Projects;
|
||||
using Azure.Identity;
|
||||
|
||||
string endpoint = Environment.GetEnvironmentVariable("AZURE_AI_PROJECT_ENDPOINT") ?? throw new InvalidOperationException("AZURE_AI_PROJECT_ENDPOINT is not set.");
|
||||
string deploymentName = Environment.GetEnvironmentVariable("AZURE_AI_MODEL_DEPLOYMENT_NAME") ?? "gpt-4o-mini";
|
||||
|
||||
Console.WriteLine("=" + new string('=', 79));
|
||||
Console.WriteLine("RED TEAMING EVALUATION SAMPLE");
|
||||
Console.WriteLine("=" + new string('=', 79));
|
||||
Console.WriteLine();
|
||||
|
||||
// Initialize Azure credentials and clients
|
||||
// WARNING: DefaultAzureCredential is convenient for development but requires careful consideration in production.
|
||||
// In production, consider using a specific credential (e.g., ManagedIdentityCredential) to avoid
|
||||
// latency issues, unintended credential probing, and potential security risks from fallback mechanisms.
|
||||
DefaultAzureCredential credential = new();
|
||||
AIProjectClient aiProjectClient = new(new Uri(endpoint), credential);
|
||||
|
||||
// Configure the target model for red teaming
|
||||
AzureOpenAIModelConfiguration targetConfig = new(deploymentName);
|
||||
|
||||
// Create the red team run configuration
|
||||
RedTeam redTeamConfig = new(targetConfig)
|
||||
{
|
||||
DisplayName = "FinancialAdvisor-RedTeam",
|
||||
ApplicationScenario = "A financial advisor assistant that provides general financial advice and information.",
|
||||
NumTurns = 3,
|
||||
RiskCategories =
|
||||
{
|
||||
RiskCategory.Violence,
|
||||
RiskCategory.HateUnfairness,
|
||||
RiskCategory.Sexual,
|
||||
RiskCategory.SelfHarm,
|
||||
},
|
||||
AttackStrategies =
|
||||
{
|
||||
AttackStrategy.Easy,
|
||||
AttackStrategy.Moderate,
|
||||
AttackStrategy.Jailbreak,
|
||||
},
|
||||
};
|
||||
|
||||
Console.WriteLine($"Target model: {deploymentName}");
|
||||
Console.WriteLine("Risk categories: Violence, HateUnfairness, Sexual, SelfHarm");
|
||||
Console.WriteLine("Attack strategies: Easy, Moderate, Jailbreak");
|
||||
Console.WriteLine($"Simulation turns: {redTeamConfig.NumTurns}");
|
||||
Console.WriteLine();
|
||||
|
||||
// Submit the red team run to the service
|
||||
Console.WriteLine("Submitting red team run...");
|
||||
RedTeam redTeamRun = await aiProjectClient.RedTeams.CreateAsync(redTeamConfig, options: null);
|
||||
|
||||
Console.WriteLine($"Red team run created: {redTeamRun.Name}");
|
||||
Console.WriteLine($"Status: {redTeamRun.Status}");
|
||||
Console.WriteLine();
|
||||
|
||||
// Poll for completion
|
||||
Console.WriteLine("Waiting for red team run to complete (this may take several minutes)...");
|
||||
while (redTeamRun.Status != "Completed" && redTeamRun.Status != "Failed" && redTeamRun.Status != "Canceled")
|
||||
{
|
||||
await Task.Delay(TimeSpan.FromSeconds(15));
|
||||
redTeamRun = await aiProjectClient.RedTeams.GetAsync(redTeamRun.Name);
|
||||
Console.WriteLine($" Status: {redTeamRun.Status}");
|
||||
}
|
||||
|
||||
Console.WriteLine();
|
||||
|
||||
if (redTeamRun.Status == "Completed")
|
||||
{
|
||||
Console.WriteLine("Red team run completed successfully!");
|
||||
Console.WriteLine();
|
||||
Console.WriteLine("Results:");
|
||||
Console.WriteLine(new string('-', 80));
|
||||
Console.WriteLine($" Run name: {redTeamRun.Name}");
|
||||
Console.WriteLine($" Display name: {redTeamRun.DisplayName}");
|
||||
Console.WriteLine($" Status: {redTeamRun.Status}");
|
||||
|
||||
Console.WriteLine();
|
||||
Console.WriteLine("Review the detailed results in the Azure AI Foundry portal:");
|
||||
Console.WriteLine($" {endpoint}");
|
||||
}
|
||||
else
|
||||
{
|
||||
Console.WriteLine($"Red team run ended with status: {redTeamRun.Status}");
|
||||
}
|
||||
|
||||
Console.WriteLine();
|
||||
Console.WriteLine(new string('=', 80));
|
||||
-101
@@ -1,101 +0,0 @@
|
||||
# Red Teaming with Azure AI Foundry (Classic)
|
||||
|
||||
> [!IMPORTANT]
|
||||
> This sample uses the **classic Azure AI Foundry** red teaming API (`/redTeams/runs`) via `Azure.AI.Projects`. Results are viewable in the classic Foundry portal experience. The **new Foundry** portal's red teaming feature uses a different evaluation-based API that is not yet available in the .NET SDK.
|
||||
|
||||
This sample demonstrates how to use Azure AI Foundry's Red Teaming service to assess the safety and resilience of an AI model against adversarial attacks.
|
||||
|
||||
## What this sample demonstrates
|
||||
|
||||
- Configuring a red team run targeting an Azure OpenAI model deployment
|
||||
- Using multiple `AttackStrategy` options (Easy, Moderate, Jailbreak)
|
||||
- Evaluating across `RiskCategory` categories (Violence, HateUnfairness, Sexual, SelfHarm)
|
||||
- Submitting a red team scan and polling for completion
|
||||
- Reviewing results in the Azure AI Foundry portal
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Before you begin, ensure you have the following prerequisites:
|
||||
|
||||
- .NET 10 SDK or later
|
||||
- Azure AI Foundry project (hub and project created)
|
||||
- Azure OpenAI deployment (e.g., gpt-4o or gpt-4o-mini)
|
||||
- Azure CLI installed and authenticated (for Azure credential authentication)
|
||||
|
||||
### Regional Requirements
|
||||
|
||||
Red teaming is only available in regions that support risk and safety evaluators:
|
||||
- **East US 2**, **Sweden Central**, **US North Central**, **France Central**, **Switzerland West**
|
||||
|
||||
### Environment Variables
|
||||
|
||||
Set the following environment variables:
|
||||
|
||||
```powershell
|
||||
$env:AZURE_AI_PROJECT_ENDPOINT="https://your-project.services.ai.azure.com/api/projects/your-project" # Replace with your Azure Foundry project endpoint
|
||||
$env:AZURE_AI_MODEL_DEPLOYMENT_NAME="gpt-4o-mini" # Optional, defaults to gpt-4o-mini
|
||||
```
|
||||
|
||||
## Run the sample
|
||||
|
||||
Navigate to the sample directory and run:
|
||||
|
||||
```powershell
|
||||
cd dotnet/samples/02-agents/FoundryAgents/FoundryAgents_Evaluations_Step01_RedTeaming
|
||||
dotnet run
|
||||
```
|
||||
|
||||
## Expected behavior
|
||||
|
||||
The sample will:
|
||||
|
||||
1. Configure a `RedTeam` run targeting the specified model deployment
|
||||
2. Define risk categories and attack strategies
|
||||
3. Submit the scan to Azure AI Foundry's Red Teaming service
|
||||
4. Poll for completion (this may take several minutes)
|
||||
5. Display the run status and direct you to the Azure AI Foundry portal for detailed results
|
||||
|
||||
## Understanding Red Teaming
|
||||
|
||||
### Attack Strategies
|
||||
|
||||
| Strategy | Description |
|
||||
|----------|-------------|
|
||||
| Easy | Simple encoding/obfuscation attacks (ROT13, Leetspeak, etc.) |
|
||||
| Moderate | Moderate complexity attacks requiring an LLM for orchestration |
|
||||
| Jailbreak | Crafted prompts designed to bypass AI safeguards (UPIA) |
|
||||
|
||||
### Risk Categories
|
||||
|
||||
| Category | Description |
|
||||
|----------|-------------|
|
||||
| Violence | Content related to violence |
|
||||
| HateUnfairness | Hate speech or unfair content |
|
||||
| Sexual | Sexual content |
|
||||
| SelfHarm | Self-harm related content |
|
||||
|
||||
### Interpreting Results
|
||||
|
||||
- Results are available in the Azure AI Foundry portal (**classic view** — toggle at top-right) under the red teaming section
|
||||
- Lower Attack Success Rate (ASR) is better — target ASR < 5% for production
|
||||
- Review individual attack conversations to understand vulnerabilities
|
||||
|
||||
### Current Limitations
|
||||
|
||||
> [!NOTE]
|
||||
> - The .NET Red Teaming API (`Azure.AI.Projects`) currently supports targeting **model deployments only** via `AzureOpenAIModelConfiguration`. The `AzureAIAgentTarget` type exists in the SDK but is consumed by the **Evaluation Taxonomy** API (`/evaluationtaxonomies`), not by the Red Teaming API (`/redTeams/runs`).
|
||||
> - Agent-targeted red teaming with agent-specific risk categories (Prohibited actions, Sensitive data leakage, Task adherence) is documented in the [concept docs](https://learn.microsoft.com/azure/ai-foundry/concepts/ai-red-teaming-agent) but is not yet available via the public REST API or .NET SDK.
|
||||
> - Results from this API appear in the **classic** Azure AI Foundry portal view. The new Foundry portal uses a separate evaluation-based system with `eval_*` identifiers.
|
||||
|
||||
## Related Resources
|
||||
|
||||
- [Azure AI Red Teaming Agent](https://learn.microsoft.com/azure/ai-foundry/concepts/ai-red-teaming-agent)
|
||||
- [RedTeam .NET API Reference](https://learn.microsoft.com/dotnet/api/azure.ai.projects.redteam?view=azure-dotnet-preview)
|
||||
- [Risk and Safety Evaluations](https://learn.microsoft.com/azure/ai-foundry/concepts/evaluation-metrics-built-in#risk-and-safety-evaluators)
|
||||
|
||||
## Next Steps
|
||||
|
||||
After running red teaming:
|
||||
1. Review attack results and strengthen agent guardrails
|
||||
2. Explore the Self-Reflection sample (FoundryAgents_Evaluations_Step02_SelfReflection) for quality assessment
|
||||
3. Set up continuous red teaming in your CI/CD pipeline
|
||||
-25
@@ -1,25 +0,0 @@
|
||||
<Project Sdk="Microsoft.NET.Sdk">
|
||||
|
||||
<PropertyGroup>
|
||||
<OutputType>Exe</OutputType>
|
||||
<TargetFrameworks>net10.0</TargetFrameworks>
|
||||
|
||||
<Nullable>enable</Nullable>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="Azure.AI.OpenAI" />
|
||||
<PackageReference Include="Azure.AI.Projects" />
|
||||
<PackageReference Include="Azure.Identity" />
|
||||
<PackageReference Include="Microsoft.Extensions.AI.Evaluation" />
|
||||
<PackageReference Include="Microsoft.Extensions.AI.Evaluation.Quality" />
|
||||
<PackageReference Include="Microsoft.Extensions.AI.Evaluation.Safety" />
|
||||
<PackageReference Include="Microsoft.Extensions.AI.OpenAI" />
|
||||
</ItemGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<ProjectReference Include="..\..\..\..\src\Microsoft.Agents.AI.AzureAI\Microsoft.Agents.AI.AzureAI.csproj" />
|
||||
</ItemGroup>
|
||||
|
||||
</Project>
|
||||
-292
@@ -1,292 +0,0 @@
|
||||
// Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
// This sample demonstrates how to use Microsoft.Extensions.AI.Evaluation.Quality to evaluate
|
||||
// an Agent Framework agent's response quality with a self-reflection loop.
|
||||
//
|
||||
// It uses GroundednessEvaluator, RelevanceEvaluator, and CoherenceEvaluator to score responses,
|
||||
// then iteratively asks the agent to improve based on evaluation feedback.
|
||||
//
|
||||
// Based on: Reflexion: Language Agents with Verbal Reinforcement Learning (NeurIPS 2023)
|
||||
// Reference: https://arxiv.org/abs/2303.11366
|
||||
//
|
||||
// For more details, see:
|
||||
// https://learn.microsoft.com/dotnet/ai/evaluation/libraries
|
||||
|
||||
using Azure.AI.OpenAI;
|
||||
using Azure.AI.Projects;
|
||||
using Azure.Identity;
|
||||
using Microsoft.Agents.AI;
|
||||
using Microsoft.Extensions.AI;
|
||||
using Microsoft.Extensions.AI.Evaluation;
|
||||
using Microsoft.Extensions.AI.Evaluation.Quality;
|
||||
using Microsoft.Extensions.AI.Evaluation.Safety;
|
||||
|
||||
using ChatMessage = Microsoft.Extensions.AI.ChatMessage;
|
||||
using ChatRole = Microsoft.Extensions.AI.ChatRole;
|
||||
|
||||
string endpoint = Environment.GetEnvironmentVariable("AZURE_AI_PROJECT_ENDPOINT") ?? throw new InvalidOperationException("AZURE_AI_PROJECT_ENDPOINT is not set.");
|
||||
string deploymentName = Environment.GetEnvironmentVariable("AZURE_AI_MODEL_DEPLOYMENT_NAME") ?? "gpt-4o-mini";
|
||||
string openAiEndpoint = Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT") ?? throw new InvalidOperationException("AZURE_OPENAI_ENDPOINT is not set.");
|
||||
string evaluatorDeploymentName = Environment.GetEnvironmentVariable("AZURE_OPENAI_DEPLOYMENT_NAME") ?? deploymentName;
|
||||
|
||||
Console.WriteLine("=" + new string('=', 79));
|
||||
Console.WriteLine("SELF-REFLECTION EVALUATION SAMPLE");
|
||||
Console.WriteLine("=" + new string('=', 79));
|
||||
Console.WriteLine();
|
||||
|
||||
// Initialize Azure credentials and client
|
||||
// WARNING: DefaultAzureCredential is convenient for development but requires careful consideration in production.
|
||||
// In production, consider using a specific credential (e.g., ManagedIdentityCredential) to avoid
|
||||
// latency issues, unintended credential probing, and potential security risks from fallback mechanisms.
|
||||
DefaultAzureCredential credential = new();
|
||||
AIProjectClient aiProjectClient = new(new Uri(endpoint), credential);
|
||||
|
||||
// Set up the LLM-based chat client for quality evaluators
|
||||
IChatClient chatClient = new AzureOpenAIClient(new Uri(openAiEndpoint), credential)
|
||||
.GetChatClient(evaluatorDeploymentName)
|
||||
.AsIChatClient();
|
||||
|
||||
// Configure evaluation: quality evaluators use the LLM, safety evaluators use Azure AI Foundry
|
||||
ContentSafetyServiceConfiguration safetyConfig = new(
|
||||
credential: credential,
|
||||
endpoint: new Uri(endpoint));
|
||||
|
||||
ChatConfiguration chatConfiguration = safetyConfig.ToChatConfiguration(
|
||||
originalChatConfiguration: new ChatConfiguration(chatClient));
|
||||
|
||||
// Create a test agent
|
||||
AIAgent agent = await aiProjectClient.CreateAIAgentAsync(
|
||||
name: "KnowledgeAgent",
|
||||
model: deploymentName,
|
||||
instructions: "You are a helpful assistant. Answer questions accurately based on the provided context.");
|
||||
Console.WriteLine($"Created agent: {agent.Name}");
|
||||
Console.WriteLine();
|
||||
|
||||
// Example question and grounding context
|
||||
const string Question = """
|
||||
What are the main benefits of using Azure AI Foundry for building AI applications?
|
||||
""";
|
||||
|
||||
const string Context = """
|
||||
Azure AI Foundry is a comprehensive platform for building, deploying, and managing AI applications.
|
||||
Key benefits include:
|
||||
1. Unified development environment with support for multiple AI frameworks and models
|
||||
2. Built-in safety and security features including content filtering and red teaming tools
|
||||
3. Scalable infrastructure that handles deployment and monitoring automatically
|
||||
4. Integration with Azure services like Azure OpenAI, Cognitive Services, and Machine Learning
|
||||
5. Evaluation tools for assessing model quality, safety, and performance
|
||||
6. Support for RAG (Retrieval-Augmented Generation) patterns with vector search
|
||||
7. Enterprise-grade compliance and governance features
|
||||
""";
|
||||
|
||||
Console.WriteLine("Question:");
|
||||
Console.WriteLine(Question);
|
||||
Console.WriteLine();
|
||||
|
||||
// Run evaluations
|
||||
try
|
||||
{
|
||||
await RunSelfReflectionWithGroundedness(agent, Question, Context, chatConfiguration);
|
||||
await RunQualityEvaluation(agent, Question, Context, chatConfiguration);
|
||||
await RunCombinedQualityAndSafetyEvaluation(agent, Question, chatConfiguration);
|
||||
}
|
||||
finally
|
||||
{
|
||||
// Cleanup
|
||||
await aiProjectClient.Agents.DeleteAgentAsync(agent.Name);
|
||||
Console.WriteLine();
|
||||
Console.WriteLine("Cleanup: Agent deleted.");
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Implementation Functions
|
||||
// ============================================================================
|
||||
|
||||
static async Task RunSelfReflectionWithGroundedness(
|
||||
AIAgent agent, string question, string context, ChatConfiguration chatConfiguration)
|
||||
{
|
||||
Console.WriteLine("Running Self-Reflection with Groundedness Evaluation...");
|
||||
Console.WriteLine();
|
||||
|
||||
GroundednessEvaluator groundednessEvaluator = new();
|
||||
GroundednessEvaluatorContext groundingContext = new(context);
|
||||
|
||||
const int MaxReflections = 3;
|
||||
double bestScore = 0;
|
||||
|
||||
string currentPrompt = $"Context: {context}\n\nQuestion: {question}";
|
||||
|
||||
for (int i = 0; i < MaxReflections; i++)
|
||||
{
|
||||
Console.WriteLine($"Iteration {i + 1}/{MaxReflections}:");
|
||||
Console.WriteLine(new string('-', 40));
|
||||
|
||||
// Create a new session for each reflection iteration so that
|
||||
// conversation context does not carry over between runs. This keeps
|
||||
// each evaluation independent and avoids biasing groundedness scores.
|
||||
AgentSession session = await agent.CreateSessionAsync();
|
||||
AgentResponse agentResponse = await agent.RunAsync(currentPrompt, session);
|
||||
string responseText = agentResponse.Text;
|
||||
|
||||
Console.WriteLine($"Response: {responseText[..Math.Min(150, responseText.Length)]}...");
|
||||
|
||||
List<ChatMessage> messages =
|
||||
[
|
||||
new(ChatRole.User, currentPrompt),
|
||||
];
|
||||
ChatResponse chatResponse = new(new ChatMessage(ChatRole.Assistant, responseText));
|
||||
|
||||
EvaluationResult result = await groundednessEvaluator.EvaluateAsync(
|
||||
messages,
|
||||
chatResponse,
|
||||
chatConfiguration,
|
||||
additionalContext: [groundingContext]);
|
||||
|
||||
NumericMetric groundedness = result.Get<NumericMetric>(GroundednessEvaluator.GroundednessMetricName);
|
||||
double score = groundedness.Value ?? 0;
|
||||
string rating = groundedness.Interpretation?.Rating.ToString() ?? "N/A";
|
||||
|
||||
Console.WriteLine($"Groundedness score: {score:F1}/5 (Rating: {rating})");
|
||||
Console.WriteLine();
|
||||
|
||||
if (score > bestScore)
|
||||
{
|
||||
bestScore = score;
|
||||
}
|
||||
|
||||
if (score >= 4.0 || i == MaxReflections - 1)
|
||||
{
|
||||
if (score >= 4.0)
|
||||
{
|
||||
Console.WriteLine("Good groundedness achieved!");
|
||||
}
|
||||
|
||||
break;
|
||||
}
|
||||
|
||||
// Ask for improvement in the next iteration, including the previous response
|
||||
// so the LLM knows what to improve on (each iteration uses a new session).
|
||||
currentPrompt = $"""
|
||||
Context: {context}
|
||||
|
||||
Your previous answer scored {score}/5 on groundedness.
|
||||
Your previous answer was:
|
||||
{responseText}
|
||||
|
||||
Please improve your answer to be more grounded in the provided context.
|
||||
Only include information that is directly supported by the context.
|
||||
|
||||
Question: {question}
|
||||
""";
|
||||
Console.WriteLine("Requesting improvement...");
|
||||
Console.WriteLine();
|
||||
}
|
||||
|
||||
Console.WriteLine($"Best groundedness score: {bestScore:F1}/5");
|
||||
Console.WriteLine(new string('=', 80));
|
||||
Console.WriteLine();
|
||||
}
|
||||
|
||||
static async Task RunQualityEvaluation(
|
||||
AIAgent agent, string question, string context, ChatConfiguration chatConfiguration)
|
||||
{
|
||||
Console.WriteLine("Running Quality Evaluation (Relevance, Coherence, Groundedness)...");
|
||||
Console.WriteLine();
|
||||
|
||||
IEvaluator[] evaluators =
|
||||
[
|
||||
new RelevanceEvaluator(),
|
||||
new CoherenceEvaluator(),
|
||||
new GroundednessEvaluator(),
|
||||
];
|
||||
|
||||
CompositeEvaluator compositeEvaluator = new(evaluators);
|
||||
GroundednessEvaluatorContext groundingContext = new(context);
|
||||
|
||||
string prompt = $"Context: {context}\n\nQuestion: {question}";
|
||||
|
||||
AgentSession session = await agent.CreateSessionAsync();
|
||||
AgentResponse agentResponse = await agent.RunAsync(prompt, session);
|
||||
string responseText = agentResponse.Text;
|
||||
|
||||
Console.WriteLine($"Response: {responseText[..Math.Min(150, responseText.Length)]}...");
|
||||
Console.WriteLine();
|
||||
|
||||
List<ChatMessage> messages =
|
||||
[
|
||||
new(ChatRole.User, prompt),
|
||||
];
|
||||
ChatResponse chatResponse = new(new ChatMessage(ChatRole.Assistant, responseText));
|
||||
|
||||
EvaluationResult result = await compositeEvaluator.EvaluateAsync(
|
||||
messages,
|
||||
chatResponse,
|
||||
chatConfiguration,
|
||||
additionalContext: [groundingContext]);
|
||||
|
||||
foreach (EvaluationMetric metric in result.Metrics.Values)
|
||||
{
|
||||
if (metric is NumericMetric n)
|
||||
{
|
||||
string rating = n.Interpretation?.Rating.ToString() ?? "N/A";
|
||||
Console.WriteLine($" {n.Name,-20} Score: {n.Value:F1}/5 Rating: {rating}");
|
||||
}
|
||||
}
|
||||
|
||||
Console.WriteLine(new string('=', 80));
|
||||
Console.WriteLine();
|
||||
}
|
||||
|
||||
static async Task RunCombinedQualityAndSafetyEvaluation(
|
||||
AIAgent agent, string question, ChatConfiguration chatConfiguration)
|
||||
{
|
||||
Console.WriteLine("Running Combined Quality + Safety Evaluation...");
|
||||
Console.WriteLine();
|
||||
|
||||
IEvaluator[] evaluators =
|
||||
[
|
||||
new RelevanceEvaluator(),
|
||||
new CoherenceEvaluator(),
|
||||
new ContentHarmEvaluator(),
|
||||
new ProtectedMaterialEvaluator(),
|
||||
];
|
||||
|
||||
CompositeEvaluator compositeEvaluator = new(evaluators);
|
||||
|
||||
AgentSession session = await agent.CreateSessionAsync();
|
||||
AgentResponse agentResponse = await agent.RunAsync(question, session);
|
||||
string responseText = agentResponse.Text;
|
||||
|
||||
Console.WriteLine($"Response: {responseText[..Math.Min(150, responseText.Length)]}...");
|
||||
Console.WriteLine();
|
||||
|
||||
List<ChatMessage> messages =
|
||||
[
|
||||
new(ChatRole.User, question), // No context in this evaluation — testing quality and safety on raw question
|
||||
];
|
||||
ChatResponse chatResponse = new(new ChatMessage(ChatRole.Assistant, responseText));
|
||||
|
||||
EvaluationResult result = await compositeEvaluator.EvaluateAsync(
|
||||
messages,
|
||||
chatResponse,
|
||||
chatConfiguration);
|
||||
|
||||
Console.WriteLine("Quality Metrics:");
|
||||
foreach (EvaluationMetric metric in result.Metrics.Values)
|
||||
{
|
||||
if (metric is NumericMetric n)
|
||||
{
|
||||
string rating = n.Interpretation?.Rating.ToString() ?? "N/A";
|
||||
bool failed = n.Interpretation?.Failed ?? false;
|
||||
Console.WriteLine($" {n.Name,-25} Score: {n.Value:F1,-6} Rating: {rating,-15} Failed: {failed}");
|
||||
}
|
||||
else if (metric is BooleanMetric b)
|
||||
{
|
||||
string rating = b.Interpretation?.Rating.ToString() ?? "N/A";
|
||||
bool failed = b.Interpretation?.Failed ?? false;
|
||||
Console.WriteLine($" {b.Name,-25} Value: {b.Value,-6} Rating: {rating,-15} Failed: {failed}");
|
||||
}
|
||||
}
|
||||
|
||||
Console.WriteLine(new string('=', 80));
|
||||
}
|
||||
-118
@@ -1,118 +0,0 @@
|
||||
# Self-Reflection Evaluation with Groundedness Assessment
|
||||
|
||||
This sample demonstrates the self-reflection pattern using Agent Framework with `Microsoft.Extensions.AI.Evaluation.Quality` evaluators. The agent iteratively improves its responses based on real groundedness evaluation scores.
|
||||
|
||||
For details on the self-reflection approach, see [Reflexion: Language Agents with Verbal Reinforcement Learning](https://arxiv.org/abs/2303.11366) (NeurIPS 2023).
|
||||
|
||||
## What this sample demonstrates
|
||||
|
||||
- Self-reflection loop that improves responses using real `GroundednessEvaluator` scores
|
||||
- Using `RelevanceEvaluator` and `CoherenceEvaluator` for multi-metric quality assessment
|
||||
- Combining quality and safety evaluators with `CompositeEvaluator`
|
||||
- Configuring `ContentSafetyServiceConfiguration` for safety evaluators alongside LLM-based quality evaluators
|
||||
- Tracking improvement across iterations
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Before you begin, ensure you have the following prerequisites:
|
||||
|
||||
- .NET 10 SDK or later
|
||||
- Azure AI Foundry project (hub and project created)
|
||||
- Azure OpenAI deployment (e.g., gpt-4o or gpt-4o-mini)
|
||||
- Azure CLI installed and authenticated (for Azure credential authentication)
|
||||
|
||||
**Note**: This demo uses Azure CLI credentials for authentication. Make sure you're logged in with `az login` and have access to the Azure Foundry resource. For more information, see the [Azure CLI documentation](https://learn.microsoft.com/cli/azure/authenticate-azure-cli-interactively).
|
||||
|
||||
### Azure Resources Required
|
||||
|
||||
1. **Azure AI Hub and Project**: Create these in the Azure Portal
|
||||
- Follow: https://learn.microsoft.com/azure/ai-foundry/how-to/create-projects
|
||||
2. **Azure OpenAI Deployment**: Deploy a model (e.g., gpt-4o or gpt-4o-mini)
|
||||
- Agent model: Used to generate responses
|
||||
- Evaluator model: Quality evaluators use an LLM; best results with GPT-4o
|
||||
3. **Azure CLI**: Install and authenticate with `az login`
|
||||
|
||||
### Environment Variables
|
||||
|
||||
Set the following environment variables:
|
||||
|
||||
```powershell
|
||||
$env:AZURE_AI_PROJECT_ENDPOINT="https://your-project.api.azureml.ms" # Azure Foundry project endpoint
|
||||
$env:AZURE_OPENAI_ENDPOINT="https://your-openai.openai.azure.com/" # Azure OpenAI endpoint (for quality evaluators)
|
||||
$env:AZURE_AI_MODEL_DEPLOYMENT_NAME="gpt-4o-mini" # Model deployment name
|
||||
```
|
||||
|
||||
**Note**: For best evaluation results, use GPT-4o or GPT-4o-mini as the evaluator model. The groundedness evaluator has been tested and tuned for these models.
|
||||
|
||||
## Run the sample
|
||||
|
||||
Navigate to the sample directory and run:
|
||||
|
||||
```powershell
|
||||
cd dotnet/samples/02-agents/FoundryAgents/FoundryAgents_Evaluations_Step02_SelfReflection
|
||||
dotnet run
|
||||
```
|
||||
|
||||
## Expected behavior
|
||||
|
||||
The sample runs three evaluation scenarios:
|
||||
|
||||
### 1. Self-Reflection with Groundedness
|
||||
- Asks a question with grounding context
|
||||
- Evaluates response groundedness using `GroundednessEvaluator`
|
||||
- If score is below 4/5, asks the agent to improve with feedback
|
||||
- Repeats up to 3 iterations
|
||||
- Tracks and reports the best score achieved
|
||||
|
||||
### 2. Quality Evaluation
|
||||
- Evaluates a single response with multiple quality evaluators:
|
||||
- `RelevanceEvaluator` — is the response relevant to the question?
|
||||
- `CoherenceEvaluator` — is the response logically coherent?
|
||||
- `GroundednessEvaluator` — is the response grounded in the provided context?
|
||||
|
||||
### 3. Combined Quality + Safety Evaluation
|
||||
- Runs both quality and safety evaluators together:
|
||||
- `RelevanceEvaluator`, `CoherenceEvaluator` (quality)
|
||||
- `ContentHarmEvaluator` (safety — violence, hate, sexual, self-harm)
|
||||
- `ProtectedMaterialEvaluator` (safety — copyrighted content detection)
|
||||
|
||||
## Understanding the Evaluation
|
||||
|
||||
### Groundedness Score (1-5 scale)
|
||||
|
||||
The `GroundednessEvaluator` measures how well the agent's response is grounded in the provided context:
|
||||
|
||||
- **5** = Excellent - Response is fully grounded in context
|
||||
- **4** = Good - Mostly grounded with minor deviations
|
||||
- **3** = Fair - Partially grounded but includes unsupported claims
|
||||
- **2** = Poor - Significant amount of ungrounded content
|
||||
- **1** = Very Poor - Response is largely unsupported by context
|
||||
|
||||
### Self-Reflection Process
|
||||
|
||||
1. **Initial Response**: Agent generates answer based on question + context
|
||||
2. **Evaluation**: `GroundednessEvaluator` scores the response (1-5)
|
||||
3. **Feedback**: If score < 4, agent receives the score and is asked to improve
|
||||
4. **Iteration**: Process repeats until good score or max iterations
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Provide Complete Context**: Ensure grounding context contains all information needed to answer the question
|
||||
2. **Clear Instructions**: Give the agent clear instructions about staying grounded in context
|
||||
3. **Use Quality Models**: GPT-4o recommended for evaluation tasks
|
||||
4. **Multiple Evaluators**: Use combination of evaluators (groundedness + relevance + coherence)
|
||||
5. **Batch Processing**: For production, process multiple questions in batch
|
||||
|
||||
## Related Resources
|
||||
|
||||
- [Reflexion Paper (NeurIPS 2023)](https://arxiv.org/abs/2303.11366)
|
||||
- [Microsoft.Extensions.AI.Evaluation Libraries](https://learn.microsoft.com/dotnet/ai/evaluation/libraries)
|
||||
- [GroundednessEvaluator API Reference](https://learn.microsoft.com/dotnet/api/microsoft.extensions.ai.evaluation.quality.groundednessevaluator)
|
||||
- [Azure AI Foundry Evaluation Service](https://learn.microsoft.com/azure/ai-foundry/how-to/develop/evaluate-sdk)
|
||||
|
||||
## Next Steps
|
||||
|
||||
After running self-reflection evaluation:
|
||||
1. Implement similar patterns for other quality metrics (relevance, coherence, fluency)
|
||||
2. Integrate into CI/CD pipeline for continuous quality assurance
|
||||
3. Explore the Safety Evaluation sample (FoundryAgents_Evaluations_Step01_RedTeaming) for content safety assessment
|
||||
-22
@@ -1,22 +0,0 @@
|
||||
<Project Sdk="Microsoft.NET.Sdk">
|
||||
|
||||
<PropertyGroup>
|
||||
<OutputType>Exe</OutputType>
|
||||
<TargetFrameworks>net10.0</TargetFrameworks>
|
||||
|
||||
<Nullable>enable</Nullable>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
<NoWarn>$(NoWarn);CA1812</NoWarn>
|
||||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="Microsoft.Extensions.Logging.Console" />
|
||||
<PackageReference Include="Azure.Identity" />
|
||||
<PackageReference Include="Azure.AI.Projects" />
|
||||
</ItemGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<ProjectReference Include="..\..\..\..\src\Microsoft.Agents.AI.AzureAI\Microsoft.Agents.AI.AzureAI.csproj" />
|
||||
</ItemGroup>
|
||||
|
||||
</Project>
|
||||
@@ -1,111 +0,0 @@
|
||||
// Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
// This sample shows how to use File Search Tool with AI Agents.
|
||||
|
||||
using Azure.AI.Projects;
|
||||
using Azure.AI.Projects.Agents;
|
||||
using Azure.Identity;
|
||||
using Microsoft.Agents.AI;
|
||||
using Microsoft.Extensions.AI;
|
||||
using OpenAI.Assistants;
|
||||
using OpenAI.Files;
|
||||
using OpenAI.Responses;
|
||||
|
||||
string endpoint = Environment.GetEnvironmentVariable("AZURE_AI_PROJECT_ENDPOINT") ?? throw new InvalidOperationException("AZURE_AI_PROJECT_ENDPOINT is not set.");
|
||||
string deploymentName = Environment.GetEnvironmentVariable("AZURE_AI_MODEL_DEPLOYMENT_NAME") ?? "gpt-4o-mini";
|
||||
|
||||
const string AgentInstructions = "You are a helpful assistant that can search through uploaded files to answer questions.";
|
||||
|
||||
// Get a client to create/retrieve/delete server side agents with Azure Foundry Agents.
|
||||
// WARNING: DefaultAzureCredential is convenient for development but requires careful consideration in production.
|
||||
// In production, consider using a specific credential (e.g., ManagedIdentityCredential) to avoid
|
||||
// latency issues, unintended credential probing, and potential security risks from fallback mechanisms.
|
||||
AIProjectClient aiProjectClient = new(new Uri(endpoint), new DefaultAzureCredential());
|
||||
var projectOpenAIClient = aiProjectClient.GetProjectOpenAIClient();
|
||||
var filesClient = projectOpenAIClient.GetProjectFilesClient();
|
||||
var vectorStoresClient = projectOpenAIClient.GetProjectVectorStoresClient();
|
||||
|
||||
// 1. Create a temp file with test content and upload it.
|
||||
string searchFilePath = Path.Combine(Path.GetTempPath(), Path.GetRandomFileName() + "_lookup.txt");
|
||||
File.WriteAllText(
|
||||
path: searchFilePath,
|
||||
contents: """
|
||||
Employee Directory:
|
||||
- Alice Johnson, 28 years old, Software Engineer, Engineering Department
|
||||
- Bob Smith, 35 years old, Sales Manager, Sales Department
|
||||
- Carol Williams, 42 years old, HR Director, Human Resources Department
|
||||
- David Brown, 31 years old, Customer Support Lead, Support Department
|
||||
"""
|
||||
);
|
||||
|
||||
Console.WriteLine($"Uploading file: {searchFilePath}");
|
||||
OpenAIFile uploadedFile = filesClient.UploadFile(
|
||||
filePath: searchFilePath,
|
||||
purpose: FileUploadPurpose.Assistants
|
||||
);
|
||||
Console.WriteLine($"Uploaded file, file ID: {uploadedFile.Id}");
|
||||
|
||||
// 2. Create a vector store with the uploaded file.
|
||||
var vectorStoreResult = await vectorStoresClient.CreateVectorStoreAsync(
|
||||
options: new() { FileIds = { uploadedFile.Id }, Name = "EmployeeDirectory_VectorStore" }
|
||||
);
|
||||
string vectorStoreId = vectorStoreResult.Value.Id;
|
||||
Console.WriteLine($"Created vector store, vector store ID: {vectorStoreId}");
|
||||
|
||||
AIAgent agent = await CreateAgentWithMEAI();
|
||||
// AIAgent agent = await CreateAgentWithNativeSDK();
|
||||
|
||||
// Run the agent
|
||||
Console.WriteLine("\n--- Running File Search Agent ---");
|
||||
AgentResponse response = await agent.RunAsync("Who is the youngest employee?");
|
||||
Console.WriteLine($"Response: {response}");
|
||||
|
||||
// Getting any file citation annotations generated by the tool
|
||||
foreach (AIAnnotation annotation in response.Messages.SelectMany(m => m.Contents).SelectMany(c => c.Annotations ?? []))
|
||||
{
|
||||
if (annotation.RawRepresentation is TextAnnotationUpdate citationAnnotation)
|
||||
{
|
||||
Console.WriteLine($$"""
|
||||
File Citation:
|
||||
File Id: {{citationAnnotation.OutputFileId}}
|
||||
Text to Replace: {{citationAnnotation.TextToReplace}}
|
||||
""");
|
||||
}
|
||||
}
|
||||
|
||||
// Cleanup.
|
||||
Console.WriteLine("\n--- Cleanup ---");
|
||||
await aiProjectClient.Agents.DeleteAgentAsync(agent.Name);
|
||||
await vectorStoresClient.DeleteVectorStoreAsync(vectorStoreId);
|
||||
await filesClient.DeleteFileAsync(uploadedFile.Id);
|
||||
File.Delete(searchFilePath);
|
||||
Console.WriteLine("Cleanup completed successfully.");
|
||||
|
||||
// --- Agent Creation Options ---
|
||||
|
||||
#pragma warning disable CS8321 // Local function is declared but never used
|
||||
// Option 1 - Using HostedFileSearchTool (MEAI + AgentFramework)
|
||||
async Task<AIAgent> CreateAgentWithMEAI()
|
||||
{
|
||||
return await aiProjectClient.CreateAIAgentAsync(
|
||||
model: deploymentName,
|
||||
name: "FileSearchAgent-MEAI",
|
||||
instructions: AgentInstructions,
|
||||
tools: [new HostedFileSearchTool() { Inputs = [new HostedVectorStoreContent(vectorStoreId)] }]);
|
||||
}
|
||||
|
||||
// Option 2 - Using PromptAgentDefinition with ResponseTool.CreateFileSearchTool (Native SDK)
|
||||
async Task<AIAgent> CreateAgentWithNativeSDK()
|
||||
{
|
||||
return await aiProjectClient.CreateAIAgentAsync(
|
||||
name: "FileSearchAgent-NATIVE",
|
||||
creationOptions: new AgentVersionCreationOptions(
|
||||
new PromptAgentDefinition(model: deploymentName)
|
||||
{
|
||||
Instructions = AgentInstructions,
|
||||
Tools = {
|
||||
ResponseTool.CreateFileSearchTool(vectorStoreIds: [vectorStoreId])
|
||||
}
|
||||
})
|
||||
);
|
||||
}
|
||||
@@ -1,52 +0,0 @@
|
||||
# Using File Search with AI Agents
|
||||
|
||||
This sample demonstrates how to use the file search tool with AI agents. The file search tool allows agents to search through uploaded files stored in vector stores to answer user questions.
|
||||
|
||||
## What this sample demonstrates
|
||||
|
||||
- Uploading files and creating vector stores
|
||||
- Creating agents with file search capabilities
|
||||
- Using HostedFileSearchTool (MEAI abstraction)
|
||||
- Using native SDK file search tools (ResponseTool.CreateFileSearchTool)
|
||||
- Handling file citation annotations
|
||||
- Managing agent and resource lifecycle (creation and deletion)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Before you begin, ensure you have the following prerequisites:
|
||||
|
||||
- .NET 10 SDK or later
|
||||
- Azure Foundry service endpoint and deployment configured
|
||||
- Azure CLI installed and authenticated (for Azure credential authentication)
|
||||
|
||||
**Note**: This demo uses `DefaultAzureCredential` for authentication. For local development, make sure you're logged in with `az login` and have access to the Azure Foundry resource. For more information, see the [Azure Identity documentation](https://learn.microsoft.com/dotnet/api/azure.identity.defaultazurecredential).
|
||||
|
||||
Set the following environment variables:
|
||||
|
||||
```powershell
|
||||
$env:AZURE_AI_PROJECT_ENDPOINT="https://your-foundry-service.services.ai.azure.com/api/projects/your-foundry-project" # Replace with your Azure Foundry resource endpoint
|
||||
$env:AZURE_AI_MODEL_DEPLOYMENT_NAME="gpt-4o-mini" # Optional, defaults to gpt-4o-mini
|
||||
```
|
||||
|
||||
## Run the sample
|
||||
|
||||
Navigate to the FoundryAgents sample directory and run:
|
||||
|
||||
```powershell
|
||||
cd dotnet/samples/02-agents/FoundryAgents
|
||||
dotnet run --project .\FoundryAgents_Step16_FileSearch
|
||||
```
|
||||
|
||||
## Expected behavior
|
||||
|
||||
The sample will:
|
||||
|
||||
1. Create a temporary text file with employee directory information
|
||||
2. Upload the file to Azure Foundry
|
||||
3. Create a vector store with the uploaded file
|
||||
4. Create an agent with file search capabilities using one of:
|
||||
- Option 1: Using HostedFileSearchTool (MEAI abstraction)
|
||||
- Option 2: Using native SDK file search tools
|
||||
5. Run a query against the agent to search through the uploaded file
|
||||
6. Display file citation annotations from responses
|
||||
7. Clean up resources (agent, vector store, and uploaded file)
|
||||
-22
@@ -1,22 +0,0 @@
|
||||
<Project Sdk="Microsoft.NET.Sdk">
|
||||
|
||||
<PropertyGroup>
|
||||
<OutputType>Exe</OutputType>
|
||||
<TargetFrameworks>net10.0</TargetFrameworks>
|
||||
|
||||
<Nullable>enable</Nullable>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
<NoWarn>$(NoWarn);CA1812;CS8321</NoWarn>
|
||||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="Microsoft.Extensions.Logging.Console" />
|
||||
<PackageReference Include="Azure.Identity" />
|
||||
<PackageReference Include="Azure.AI.Projects" />
|
||||
</ItemGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<ProjectReference Include="..\..\..\..\src\Microsoft.Agents.AI.AzureAI\Microsoft.Agents.AI.AzureAI.csproj" />
|
||||
</ItemGroup>
|
||||
|
||||
</Project>
|
||||
@@ -1,116 +0,0 @@
|
||||
// Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
// This sample shows how to use OpenAPI Tools with AI Agents.
|
||||
|
||||
using Azure.AI.Projects;
|
||||
using Azure.AI.Projects.Agents;
|
||||
using Azure.Identity;
|
||||
using Microsoft.Agents.AI;
|
||||
using OpenAI.Responses;
|
||||
|
||||
// Warning: DefaultAzureCredential is intended for simplicity in development. For production scenarios, consider using a more specific credential.
|
||||
string endpoint = Environment.GetEnvironmentVariable("AZURE_AI_PROJECT_ENDPOINT") ?? throw new InvalidOperationException("AZURE_AI_PROJECT_ENDPOINT is not set.");
|
||||
string deploymentName = Environment.GetEnvironmentVariable("AZURE_AI_MODEL_DEPLOYMENT_NAME") ?? "gpt-4o-mini";
|
||||
|
||||
const string AgentInstructions = "You are a helpful assistant that can use the countries API to retrieve information about countries by their currency code.";
|
||||
|
||||
// A simple OpenAPI specification for the REST Countries API
|
||||
const string CountriesOpenApiSpec = """
|
||||
{
|
||||
"openapi": "3.1.0",
|
||||
"info": {
|
||||
"title": "REST Countries API",
|
||||
"description": "Retrieve information about countries by currency code",
|
||||
"version": "v3.1"
|
||||
},
|
||||
"servers": [
|
||||
{
|
||||
"url": "https://restcountries.com/v3.1"
|
||||
}
|
||||
],
|
||||
"paths": {
|
||||
"/currency/{currency}": {
|
||||
"get": {
|
||||
"description": "Get countries that use a specific currency code (e.g., USD, EUR, GBP)",
|
||||
"operationId": "GetCountriesByCurrency",
|
||||
"parameters": [
|
||||
{
|
||||
"name": "currency",
|
||||
"in": "path",
|
||||
"description": "Currency code (e.g., USD, EUR, GBP)",
|
||||
"required": true,
|
||||
"schema": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
],
|
||||
"responses": {
|
||||
"200": {
|
||||
"description": "Successful response with list of countries",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "object"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"404": {
|
||||
"description": "No countries found for the currency"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
""";
|
||||
|
||||
// Get a client to create/retrieve/delete server side agents with Azure Foundry Agents.
|
||||
AIProjectClient aiProjectClient = new(new Uri(endpoint), new DefaultAzureCredential());
|
||||
|
||||
// Create the OpenAPI function definition
|
||||
var openApiFunction = new OpenApiFunctionDefinition(
|
||||
"get_countries",
|
||||
BinaryData.FromString(CountriesOpenApiSpec),
|
||||
new OpenAPIAnonymousAuthenticationDetails())
|
||||
{
|
||||
Description = "Retrieve information about countries by currency code"
|
||||
};
|
||||
|
||||
AIAgent agent = await CreateAgentWithMEAI();
|
||||
// AIAgent agent = await CreateAgentWithNativeSDK();
|
||||
|
||||
// Run the agent with a question about countries
|
||||
Console.WriteLine(await agent.RunAsync("What countries use the Euro (EUR) as their currency? Please list them."));
|
||||
|
||||
// Cleanup by deleting the agent
|
||||
await aiProjectClient.Agents.DeleteAgentAsync(agent.Name);
|
||||
|
||||
// --- Agent Creation Options ---
|
||||
|
||||
// Option 1 - Using AsAITool wrapping for OpenApiTool (MEAI + AgentFramework)
|
||||
async Task<AIAgent> CreateAgentWithMEAI()
|
||||
{
|
||||
return await aiProjectClient.CreateAIAgentAsync(
|
||||
model: deploymentName,
|
||||
name: "OpenAPIToolsAgent-MEAI",
|
||||
instructions: AgentInstructions,
|
||||
tools: [((ResponseTool)AgentTool.CreateOpenApiTool(openApiFunction)).AsAITool()]);
|
||||
}
|
||||
|
||||
// Option 2 - Using PromptAgentDefinition with AgentTool.CreateOpenApiTool (Native SDK)
|
||||
async Task<AIAgent> CreateAgentWithNativeSDK()
|
||||
{
|
||||
return await aiProjectClient.CreateAIAgentAsync(
|
||||
name: "OpenAPIToolsAgent-NATIVE",
|
||||
creationOptions: new AgentVersionCreationOptions(
|
||||
new PromptAgentDefinition(model: deploymentName)
|
||||
{
|
||||
Instructions = AgentInstructions,
|
||||
Tools = { (ResponseTool)AgentTool.CreateOpenApiTool(openApiFunction) }
|
||||
})
|
||||
);
|
||||
}
|
||||
@@ -1,47 +0,0 @@
|
||||
# Using OpenAPI Tools with AI Agents
|
||||
|
||||
This sample demonstrates how to use OpenAPI tools with AI agents. OpenAPI tools allow agents to call external REST APIs defined by OpenAPI specifications.
|
||||
|
||||
## What this sample demonstrates
|
||||
|
||||
- Creating agents with OpenAPI tool capabilities
|
||||
- Using AgentTool.CreateOpenApiTool with an embedded OpenAPI specification
|
||||
- Anonymous authentication for public APIs
|
||||
- Running an agent that can call external REST APIs
|
||||
- Managing agent lifecycle (creation and deletion)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Before you begin, ensure you have the following prerequisites:
|
||||
|
||||
- .NET 10 SDK or later
|
||||
- Azure Foundry service endpoint and deployment configured
|
||||
- Azure CLI installed and authenticated (for Azure credential authentication)
|
||||
|
||||
**Note**: This demo uses `DefaultAzureCredential` for authentication, which supports multiple authentication methods including Azure CLI, managed identity, and more. Make sure you're logged in with `az login` and have access to the Azure Foundry resource. For more information, see the [Azure Identity documentation](https://learn.microsoft.com/dotnet/api/azure.identity.defaultazurecredential).
|
||||
|
||||
Set the following environment variables:
|
||||
|
||||
```powershell
|
||||
$env:AZURE_AI_PROJECT_ENDPOINT="https://your-foundry-service.services.ai.azure.com/api/projects/your-foundry-project" # Replace with your Azure Foundry resource endpoint
|
||||
$env:AZURE_AI_MODEL_DEPLOYMENT_NAME="gpt-4o-mini" # Optional, defaults to gpt-4o-mini
|
||||
```
|
||||
|
||||
## Run the sample
|
||||
|
||||
Navigate to the FoundryAgents sample directory and run:
|
||||
|
||||
```powershell
|
||||
cd dotnet/samples/02-agents/FoundryAgents
|
||||
dotnet run --project .\FoundryAgents_Step17_OpenAPITools
|
||||
```
|
||||
|
||||
## Expected behavior
|
||||
|
||||
The sample will:
|
||||
|
||||
1. Create an agent with an OpenAPI tool configured to call the REST Countries API
|
||||
2. Ask the agent: "What countries use the Euro (EUR) as their currency?"
|
||||
3. The agent will use the OpenAPI tool to call the REST Countries API
|
||||
4. Display the response containing the list of countries that use EUR
|
||||
5. Clean up resources by deleting the agent
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user