agent/agent-framework
1
0
Fork 0
mirror of https://github.com/microsoft/agent-framework.git synced 2026-06-16 21:04:09 +08:00
Code Issues Packages Projects Releases Wiki Activity

Python: [Feature Branch] Merge from main to Azure AI branch (#2111)

Browse Source 
* Do not build DevUI assets during .NET project build (#2010)

* .NET: Add unit tests for declarative executor SetMultipleVariables (#2016)

* Add unit tests for create conversation executor

* Update indentation and comment typo.

* Added unit tests for declarative executor SetMultipleVariablesExecutor

* Updated comments and syntactic sugar

* Python: DevUI: Use metadata.entity_id instead of model field (#1984)

* DevUI: Use metadata.entity_id for agent/workflow name instead of model field

* OpenAI Responses: add explicit request validation

* Review feedback

* .NET: DevUI - Do not automatically add/map OpenAI services/endpoints (#2014)

* Don't add OpenAIResponses as part of Dev UI

You should be able to add and remove Dev UI without impacting your other production endpoints.

* Remove `AddDevUI()` and do not map OpenAI endpoints from `MapDevUI()`

* Fix comment wording

* Revise documentation

---------

Co-authored-by: Daniel Roth <daroth@microsoft.com>

* Python: DevUI: Add OpenAI Responses API proxy support  + HIL for Workflows (#1737)

* DevUI: Add OpenAI Responses API proxy support with enhanced UI features

This commit adds support for proxying requests to OpenAI's Responses API,
allowing DevUI to route conversations to OpenAI models when configured to enable testing.

Backend changes:
- Add OpenAI proxy executor with conversation routing logic
- Enhance event mapper to support OpenAI Responses API format
- Extend server endpoints to handle OpenAI proxy mode
- Update models with OpenAI-specific response types
- Remove emojis from logging and CLI output for cleaner text

Frontend changes:
- Add settings modal with OpenAI proxy configuration UI
- Enhance agent and workflow views with improved state management
- Add new UI components (separator, switch) for settings
- Update debug panel with better event filtering
- Improve message renderers for OpenAI content types
- Update types and API client for OpenAI integration

* update ui, settings modal and workflow input form, add register cleanup hooks.

* add workflow HIL support, user mode, other fixes

* feat(devui): add human-in-the-loop (HIL) support with dynamic response schemas

Implement  HIL workflow support allowing workflows to pause for user input
with dynamically generated JSON schemas based on response handler type hints.

Key Features:
- Automatic response schema extraction from @response_handler decorators
- Dynamic form generation in UI based on Pydantic/dataclass response types
- Checkpoint-based conversation storage for HIL requests/responses
- Resume workflow execution after user provides HIL response

Backend Changes:
- Add extract_response_type_from_executor() to introspect response handlers
- Enrich RequestInfoEvent with response_schema via _enrich_request_info_event_with_response_schema()
- Map RequestInfoEvent to response.input.requested OpenAI event format
- Store HIL responses in conversation history and restore checkpoints

Frontend Changes:
- Add HILInputModal component with SchemaFormRenderer for dynamic forms
- Support Pydantic BaseModel and dataclass response types
- Render enum fields as dropdowns, strings as text/textarea, numbers, booleans, arrays, objects
- Display original request context alongside response form

Testing:
- Add  tests for checkpoint storage (test_checkpoints.py)
- Add schema generation tests for all input types (test_schema_generation.py)
- Validate end-to-end HIL flow with spam workflow sample

This enables workflows to seamlessly pause execution and request structured user input
with type-safe, validated forms generated automatically from response type annotations.

* improve HIL support, improve workflow execution view

* ui updates

* ui updates

* improve HIL for workflows, add auth and view modes

* update workflow

* security improvements , ui fixes

* fix mypy error

* update loading spinner in ui

---------

Co-authored-by: Mark Wallace <127216156+markwallace-microsoft@users.noreply.github.com>

* .NET: Remove launchSettings.json from .gitignore in dotnet/samples (#2006)

* Remove launchSettings.json from .gitignore in dotnet/samples

* Update dotnet/samples/GettingStarted/DevUI/DevUI_Step01_BasicUsage/Properties/launchSettings.json

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update dotnet/samples/AGUIClientServer/AGUIServer/Properties/launchSettings.json

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* DevUI: Serialize workflow input as string to maintain conformance with OpenAI Responses format (#2021)

Co-authored-by: Victor Dibia <chuvidi2003@gmail.com>

* Add Microsoft Agent Framework logo to assets (#2007)

* Updated package versions (#2027)

* DevUI: Prevent line breaks within words in the agent view (#2024)

Co-authored-by: Victor Dibia <chuvidi2003@gmail.com>

* .NET [AG-UI]: Adds support for shared state. (#1996)

* Product changes

* Tests

* Dojo project

* Cleanups

* Python: Fix underlying tool choice bug and all for return to previous Handoff subagent (#2037)

* Fix tool_choice override bug and add enable_return_to_previous support

* Add unit test for handoff checkpointing

* Handle tools when we have them

* added missing chatAgent params (#2044)

* .NET: fix ChatCompletions Tools serialization (#2043)

* fix serialization in chat completions on tools

* nit

* .NET: assign AgentCard's URL to mapped-endpoint if not defined explicitly (#2047)

* fix serialization in chat completions on tools

* nit

* write e2e test for agent card resolve + adjust behavior

* nit

* Version 1.0.0-preview.251110.1 (#2048)

* .NET: Remove moved OpenAPI sample and point to SK one. (#1997)

* Remove moved OpenAPI sample and point to SK one.

* Update dotnet/samples/GettingStarted/Agents/README.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Bump AWSSDK.Extensions.Bedrock.MEAI from 4.0.4.2 to 4.0.4.6 (#2031)

---
updated-dependencies:
- dependency-name: AWSSDK.Extensions.Bedrock.MEAI
  dependency-version: 4.0.4.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* .NET: Separate all memory and rag samples into their own folders (#2000)

* Separate all memory and rag samples into their own folders

* Fix broken link.

* Python: .Net: Dotnet devui compatibility fixes (#2026)

* DevUI: Add OpenAI Responses API proxy support with enhanced UI features

This commit adds support for proxying requests to OpenAI's Responses API,
allowing DevUI to route conversations to OpenAI models when configured to enable testing.

Backend changes:
- Add OpenAI proxy executor with conversation routing logic
- Enhance event mapper to support OpenAI Responses API format
- Extend server endpoints to handle OpenAI proxy mode
- Update models with OpenAI-specific response types
- Remove emojis from logging and CLI output for cleaner text

Frontend changes:
- Add settings modal with OpenAI proxy configuration UI
- Enhance agent and workflow views with improved state management
- Add new UI components (separator, switch) for settings
- Update debug panel with better event filtering
- Improve message renderers for OpenAI content types
- Update types and API client for OpenAI integration

* update ui, settings modal and workflow input form, add register cleanup hooks.

* add workflow HIL support, user mode, other fixes

* feat(devui): add human-in-the-loop (HIL) support with dynamic response schemas

Implement  HIL workflow support allowing workflows to pause for user input
with dynamically generated JSON schemas based on response handler type hints.

Key Features:
- Automatic response schema extraction from @response_handler decorators
- Dynamic form generation in UI based on Pydantic/dataclass response types
- Checkpoint-based conversation storage for HIL requests/responses
- Resume workflow execution after user provides HIL response

Backend Changes:
- Add extract_response_type_from_executor() to introspect response handlers
- Enrich RequestInfoEvent with response_schema via _enrich_request_info_event_with_response_schema()
- Map RequestInfoEvent to response.input.requested OpenAI event format
- Store HIL responses in conversation history and restore checkpoints

Frontend Changes:
- Add HILInputModal component with SchemaFormRenderer for dynamic forms
- Support Pydantic BaseModel and dataclass response types
- Render enum fields as dropdowns, strings as text/textarea, numbers, booleans, arrays, objects
- Display original request context alongside response form

Testing:
- Add  tests for checkpoint storage (test_checkpoints.py)
- Add schema generation tests for all input types (test_schema_generation.py)
- Validate end-to-end HIL flow with spam workflow sample

This enables workflows to seamlessly pause execution and request structured user input
with type-safe, validated forms generated automatically from response type annotations.

* improve HIL support, improve workflow execution view

* ui updates

* ui updates

* improve HIL for workflows, add auth and view modes

* update workflow

* security improvements , ui fixes

* fix mypy error

* update loading spinner in ui

* DevUI: Serialize workflow input as string to maintain conformance with OpenAI Responses format

* Phase 1: Add /meta endpoint and fix workflow event naming for .NET DevUI compatibility

* additional fixes for .NET DevUI workflow visualization item ID tracking

**Problem:**
.NET DevUI was generating different item IDs for ExecutorInvokedEvent and
ExecutorCompletedEvent, causing only the first executor to highlight in the
workflow graph. Long executor names and error messages also broke UI layout.

**Changes:**
- Add ExecutorActionItemResource to match Python DevUI implementation
- Track item IDs per executor using dictionary in AgentRunResponseUpdateExtensions
- Reuse same item ID across invoked/completed/failed events for proper pairing
- Add truncateText() utility to workflow-utils.ts
- Truncate executor names to 35 chars in execution timeline
- Truncate error messages to 150 chars in workflow graph nodes

** Details:**
- ExecutorActionItemResource registered with JSON source generation context
- Dictionary cleaned up after executor completion/failure to prevent memory leaks
- Frontend item tracking by unique item.id supports multiple executor runs
- All changes follow existing codebase patterns and conventions

Tested with review-workflow showing correct executor highlighting and state
transitions for sequential and concurrent executors.

* format fixes, remove cors tests

* remove unecessary attributes

---------

Co-authored-by: Mark Wallace <127216156+markwallace-microsoft@users.noreply.github.com>
Co-authored-by: Reuben Bond <reuben.bond@gmail.com>

* DevUI: support having both an agent and a workflow with the same id in discovery (#2023)

* Python: Fix Model ID attribute not showing up in `invoke_agent` span (#2061)

* Best effort to surface the model id to invoke agent span

* Fix tests

* Fix tests

* Version 1.0.0-preview.251107.2 (#2065)

* Version 1.0.0-preview.251110.2 (#2067)

* Update README.md to change Grafana links to Azure portal links for dashboard access (#1983)

* .NET - Enable build & test on branch `feature-foundry-agents` (#2068)

* Tests good, mkay

* Update .github/workflows/dotnet-build-and-test.yml

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Enable feature build pipelines

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Roger Barreto <19890735+rogerbarreto@users.noreply.github.com>

* Python: Add concrete AGUIChatClient (#2072)

* Add concrete AGUIChatClient

* Update logging docstrings and conventions

* PR feedback

* Updates to support client-side tool calls

* .NET: Move catalog samples to the HostedAgents folder (#2090)

* move catalog samples to the HostedAgents folder

* move the catalog samples' projects to the HostedAgents folder

* Bump OpenTelemetry.Instrumentation.Runtime from 1.12.0 to 1.13.0 (#1856)

---
updated-dependencies:
- dependency-name: OpenTelemetry.Instrumentation.Runtime
  dependency-version: 1.13.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* .NET: Bump Microsoft.SemanticKernel.Agents.Abstractions from 1.66.0 to 1.67.0 (#1962)

* Bump Microsoft.SemanticKernel.Agents.Abstractions from 1.66.0 to 1.67.0

---
updated-dependencies:
- dependency-name: Microsoft.SemanticKernel.Agents.Abstractions
  dependency-version: 1.67.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* .NET: Bump all Microsoft.SemanticKernel packages from 1.66.* to 1.67.* (#1969)

* Initial plan

* Update all Microsoft.SemanticKernel packages to 1.67.*

Co-authored-by: rogerbarreto <19890735+rogerbarreto@users.noreply.github.com>

* Remove unrelated changes to package-lock.json and yarn.lock

Co-authored-by: markwallace-microsoft <127216156+markwallace-microsoft@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: rogerbarreto <19890735+rogerbarreto@users.noreply.github.com>
Co-authored-by: markwallace-microsoft <127216156+markwallace-microsoft@users.noreply.github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: rogerbarreto <19890735+rogerbarreto@users.noreply.github.com>
Co-authored-by: markwallace-microsoft <127216156+markwallace-microsoft@users.noreply.github.com>

* .NET: fix: WorkflowAsAgent Sample (#1787)

* fix: WorkflowAsAgent Sample

* Also makes ChatForwardingExecutor public

* feat: Expand ChatForwardingExecutor handled types

Make ChatForwardingExecutor match the input types of ChatProtocolExecutor.

* fix: Update for the new AgentRunResponseUpdate merge logic

AIAgent always sends out List<ChatMessage> now.

* Updated (#2076)

* Bump vite in /python/samples/demos/chatkit-integration/frontend (#1918)

Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) from 7.1.9 to 7.1.12.
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/v7.1.12/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v7.1.12/packages/vite)

---
updated-dependencies:
- dependency-name: vite
  dependency-version: 7.1.12
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump Roslynator.Analyzers from 4.14.0 to 4.14.1 (#1857)

---
updated-dependencies:
- dependency-name: Roslynator.Analyzers
  dependency-version: 4.14.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump MishaKav/pytest-coverage-comment from 1.1.57 to 1.1.59 (#2034)

Bumps [MishaKav/pytest-coverage-comment](https://github.com/mishakav/pytest-coverage-comment) from 1.1.57 to 1.1.59.
- [Release notes](https://github.com/mishakav/pytest-coverage-comment/releases)
- [Changelog](https://github.com/MishaKav/pytest-coverage-comment/blob/main/CHANGELOG.md)
- [Commits](https://github.com/mishakav/pytest-coverage-comment/compare/v1.1.57...v1.1.59)

---
updated-dependencies:
- dependency-name: MishaKav/pytest-coverage-comment
  dependency-version: 1.1.59
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris <66376200+crickman@users.noreply.github.com>

* Python: Handle agent user input request in AgentExecutor (#2022)

* Handle agent user input request in AgentExecutor

* fix test

* Address comments

* Fix tests

* Fix tests

* Address comments

* Address comments

* Python: OpenAI Responses Image Generation Stream Support, Sample and Unit Tests (#1853)

* support for image gen streaming

* small fixes

* fixes

* added comment

* Python: Fix MCP Tool Parameter Descriptions Not Propagated to LLMs (#1978)

* mcp tool description fix

* small fix

* .NET: Allow extending agent run options via additional properties (#1872)

* Allow extending agent run options via additional properties

This mirrors the M.E.AI model in ChatOptions.AdditionalProperties which is very useful when building functionality pipelines.

Fixes https://github.com/microsoft/agent-framework/issues/1815

* Expand XML documentation

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Add AdditionalProperties tests to AgentRunOptions

Co-authored-by: kzu <169707+kzu@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: kzu <169707+kzu@users.noreply.github.com>

* Python: Use the last entry in the task history to avoid empty responses (#2101)

* Use the last entry in the task history to avoid empty responses

* History only contains Messages

* Updated package versions (#2104)

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Reuben Bond <203839+ReubenBond@users.noreply.github.com>
Co-authored-by: Peter Ibekwe <109177538+peibekwe@users.noreply.github.com>
Co-authored-by: Jeff Handley <jeffhandley@users.noreply.github.com>
Co-authored-by: Daniel Roth <daroth@microsoft.com>
Co-authored-by: Victor Dibia <chuvidi2003@gmail.com>
Co-authored-by: Mark Wallace <127216156+markwallace-microsoft@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Shawn Henry <sphenry@gmail.com>
Co-authored-by: Javier Calvarro Nelson <jacalvar@microsoft.com>
Co-authored-by: Evan Mattson <35585003+moonbox3@users.noreply.github.com>
Co-authored-by: Eduard van Valkenburg <eavanvalkenburg@users.noreply.github.com>
Co-authored-by: Korolev Dmitry <deagle.gross@gmail.com>
Co-authored-by: westey <164392973+westey-m@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Reuben Bond <reuben.bond@gmail.com>
Co-authored-by: Tao Chen <taochen@microsoft.com>
Co-authored-by: wuweng <wuweng@microsoft.com>
Co-authored-by: Chris <66376200+crickman@users.noreply.github.com>
Co-authored-by: Roger Barreto <19890735+rogerbarreto@users.noreply.github.com>
Co-authored-by: SergeyMenshykh <68852919+SergeyMenshykh@users.noreply.github.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Jacob Alber <jaalber@microsoft.com>
Co-authored-by: Giles Odigwe <79032838+giles17@users.noreply.github.com>
Co-authored-by: Daniel Cazzulino <daniel@cazzulino.com>
Co-authored-by: kzu <169707+kzu@users.noreply.github.com>
This commit is contained in:
Dmytro Struk
2025-11-11 23:12:09 -08:00
committed by GitHub
Unverified
parent 85fcd230bf
commit 361c47f30f
231 changed files with 19659 additions and 4143 deletions
Download Patch File Download Diff File Expand all files Collapse all files
python/packages/ag-ui/getting_started/README.md
+185 -429
View File
@@ -2,6 +2,135 @@
The AG-UI (Agent UI) protocol provides a standardized way for client applications to interact with AI agents over HTTP. This tutorial demonstrates how to build both server and client applications using the AG-UI protocol with Python.
## Quick Start - Client Examples
If you want to quickly try out the AG-UI client, we provide three ready-to-use examples:
### Basic Interactive Client (`client.py`)
A simple command-line chat client that demonstrates:
- Streaming responses in real-time
- Automatic thread management for conversation continuity
- Direct `AGUIChatClient` usage (caller manages message history)
**Run:**
```bash
python client.py
```
**Note:** This example sends only the current message to the server. The server is responsible for maintaining conversation history using the thread_id.
### Advanced Features Client (`client_advanced.py`)
Demonstrates advanced capabilities:
- Tool/function calling
- Both streaming and non-streaming responses
- Multi-turn conversations
- Error handling patterns
**Run:**
```bash
python client_advanced.py
```
**Note:** This example shows direct `AGUIChatClient` usage. Tool execution and conversation continuity depend on server-side configuration and capabilities.
### ChatAgent Integration (`client_with_agent.py`)
Best practice example using `ChatAgent` wrapper with **AgentThread**
- **AgentThread** maintains conversation state
- Client-side conversation history management via `thread.message_store`
- **Hybrid tool execution**: client-side + server-side tools simultaneously
- Full conversation history sent on each request
- Tool calling with conversation context
**To demonstrate hybrid tools:**
1. **Start server with server-side tool** (Terminal 1):
```bash
# Server has get_time_zone tool
python server.py
```
2. **Run client with client-side tool** (Terminal 2):
```bash
# Client has get_weather tool
python client_with_agent.py
```
All examples require a running AG-UI server (see Step 1 below for setup).
## Understanding AG-UI Architecture
### Thread Management
The AG-UI protocol supports two approaches to conversation history:
1. **Server-Managed Threads** (client.py, client_advanced.py)
- Client sends only the current message + thread_id
- Server maintains full conversation history
- Requires server to support stateful thread storage
- Lighter network payload
2. **Client-Managed History** (client_with_agent.py)
- Client maintains full conversation history locally
- Full message history sent with each request
- Works with any AG-UI server (stateful or stateless)
The `ChatAgent` wrapper (used in client_with_agent.py) collects messages from local storage and sends the full history to `AGUIChatClient`, which then forwards everything to the server.
### Tool/Function Calling
The AG-UI protocol supports **hybrid tool execution** - both client-side AND server-side tools can coexist in the same conversation.
**The Hybrid Pattern** (client_with_agent.py):
```
Client defines: Server defines:
- get_weather() - get_current_time()
- read_sensors() - get_server_forecast()
User: "What's the weather in SF and what time is it?"
↓
ChatAgent sends: full history + tool definitions for get_weather, read_sensors
↓
Server LLM decides: "I need get_weather('SF') and get_current_time()"
↓
Server executes get_current_time() → "2025-11-11 14:30:00 UTC"
Server sends function call request → get_weather('SF')
↓
ChatAgent intercepts get_weather call → executes locally
↓
Client sends result → "Sunny, 72°F"
↓
Server combines both results → "It's sunny and 72°F in SF, and the current time is 2:30 PM UTC"
↓
Client receives final response
```
**How it works:**
1. **Client-Side Tools** (`client_with_agent.py`):
- Tools defined in ChatAgent's `tools` parameter execute locally
- Tool metadata (name, description, schema) sent to server for planning
- When server requests client tool → client intercepts → executes locally → sends result
2. **Server-Side Tools**:
- Defined in server agent's configuration
- Server executes directly without client involvement
- Results included in server's response
3. **Hybrid Pattern (Both Together)**:
- Server LLM sees ALL tool definitions (client + server)
- Decides which to use based on task
- Server tools execute server-side
- Client tools execute client-side
**Direct AGUIChatClient Usage** (client_advanced.py):
Even without ChatAgent wrapper, client-side tools work:
- Tools passed in ChatOptions execute locally
- Server can also have its own tools
- Hybrid execution works automatically
## What is AG-UI?
AG-UI is a protocol that enables:
@@ -35,13 +164,13 @@ The AG-UI server hosts your AI agent and exposes it via HTTP endpoints using Fas
### Install Required Packages
```bash
pip install agent-framework-ag-ui agent-framework-core fastapi uvicorn
pip install agent-framework-ag-ui
```
Or using uv:
```bash
uv pip install agent-framework-ag-ui agent-framework-core fastapi uvicorn
uv pip install agent-framework-ag-ui
```
### Server Code
@@ -57,17 +186,20 @@ import os
from agent_framework import ChatAgent
from agent_framework.azure import AzureOpenAIChatClient
from agent_framework_ag_ui import add_agent_framework_fastapi_endpoint
from agent_framework.ag_ui import add_agent_framework_fastapi_endpoint
from fastapi import FastAPI
# Read required configuration
endpoint = os.environ.get("AZURE_OPENAI_ENDPOINT")
deployment_name = os.environ.get("AZURE_OPENAI_DEPLOYMENT_NAME")
api_key = os.environ.get("AZURE_OPENAI_API_KEY")
if not endpoint:
raise ValueError("AZURE_OPENAI_ENDPOINT environment variable is required")
if not deployment_name:
raise ValueError("AZURE_OPENAI_DEPLOYMENT_NAME environment variable is required")
if not api_key:
raise ValueError("AZURE_OPENAI_API_KEY environment variable is required")
# Create the AI agent
agent = ChatAgent(
@@ -76,6 +208,7 @@ agent = ChatAgent(
chat_client=AzureOpenAIChatClient(
endpoint=endpoint,
deployment_name=deployment_name,
api_key=api_key,
),
)
@@ -137,12 +270,14 @@ The server will start listening on `http://127.0.0.1:5100`.
## Step 2: Creating an AG-UI Client
The AG-UI client connects to the remote server and displays streaming responses.
The AG-UI client connects to the remote server and displays streaming responses. The `AGUIChatClient` is a built-in implementation that integrates with the Agent Framework's standard chat interface.
### Install Required Packages
The `AGUIChatClient` is included in the `agent-framework-ag-ui` package (already installed if you installed the server packages).
```bash
pip install httpx
pip install agent-framework-ag-ui
```
### Client Code
@@ -152,122 +287,61 @@ Create a file named `client.py`:
```python
# Copyright (c) Microsoft. All rights reserved.
"""AG-UI client example."""
"""AG-UI client example using AGUIChatClient."""
import asyncio
import json
import os
from typing import AsyncIterator
import httpx
class AGUIClient:
"""Simple AG-UI protocol client."""
def __init__(self, server_url: str):
"""Initialize the client.
Args:
server_url: The AG-UI server endpoint URL
"""
self.server_url = server_url
self.thread_id: str | None = None
async def send_message(self, message: str) -> AsyncIterator[dict]:
"""Send a message and stream the response.
Args:
message: The user message to send
Yields:
AG-UI events from the server
"""
# Prepare the request
request_data = {
"messages": [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": message},
]
}
# Include thread_id if we have one (for conversation continuity)
if self.thread_id:
request_data["thread_id"] = self.thread_id
# Stream the response
async with httpx.AsyncClient(timeout=60.0) as client:
async with client.stream(
"POST",
self.server_url,
json=request_data,
headers={"Accept": "text/event-stream"},
) as response:
response.raise_for_status()
async for line in response.aiter_lines():
# Parse Server-Sent Events format
if line.startswith("data: "):
data = line[6:] # Remove "data: " prefix
try:
event = json.loads(data)
yield event
# Capture thread_id from RUN_STARTED event
if event.get("type") == "RUN_STARTED" and not self.thread_id:
self.thread_id = event.get("threadId")
except json.JSONDecodeError:
continue
from agent_framework import TextContent
from agent_framework.ag_ui import AGUIChatClient
async def main():
"""Main client loop."""
"""Main client loop demonstrating AGUIChatClient usage."""
# Get server URL from environment or use default
server_url = os.environ.get("AGUI_SERVER_URL", "http://127.0.0.1:5100/")
print(f"Connecting to AG-UI server at: {server_url}\n")
client = AGUIClient(server_url)
# Create client with context manager for automatic cleanup
async with AGUIChatClient(endpoint=server_url) as client:
thread_id: str | None = None
try:
while True:
# Get user input
message = input("\nUser (:q or quit to exit): ")
if not message.strip():
print("Request cannot be empty.")
continue
try:
while True:
# Get user input
message = input("\nUser (:q or quit to exit): ")
if not message.strip():
print("Request cannot be empty.")
continue
if message.lower() in (":q", "quit"):
break
if message.lower() in (":q", "quit"):
break
# Send message and display streaming response
print("\n", end="")
async for event in client.send_message(message):
event_type = event.get("type", "")
# Send message and stream the response
print("\nAssistant: ", end="", flush=True)
if event_type == "RUN_STARTED":
thread_id = event.get("threadId", "")
run_id = event.get("runId", "")
print(f"\033[93m[Run Started - Thread: {thread_id}, Run: {run_id}]\033[0m")
# Use metadata to maintain conversation continuity
metadata = {"thread_id": thread_id} if thread_id else None
elif event_type == "TEXT_MESSAGE_CONTENT":
# Stream text content in cyan
print(f"\033[96m{event.get('delta', '')}\033[0m", end="", flush=True)
async for update in client.get_streaming_response(message, metadata=metadata):
# Extract thread ID from first update
if not thread_id and update.additional_properties:
thread_id = update.additional_properties.get("thread_id")
if thread_id:
print(f"\n[Thread: {thread_id}]")
print("Assistant: ", end="", flush=True)
elif event_type == "RUN_FINISHED":
thread_id = event.get("threadId", "")
run_id = event.get("runId", "")
print(f"\n\033[92m[Run Finished - Thread: {thread_id}, Run: {run_id}]\033[0m")
# Stream text content as it arrives
for content in update.contents:
if isinstance(content, TextContent) and content.text:
print(content.text, end="", flush=True)
elif event_type == "RUN_ERROR":
error_message = event.get("message", "Unknown error")
print(f"\n\033[91m[Run Error - Message: {error_message}]\033[0m")
print() # New line after response
print()
except KeyboardInterrupt:
print("\n\nExiting...")
except Exception as e:
print(f"\n\033[91mAn error occurred: {e}\033[0m")
except KeyboardInterrupt:
print("\n\nExiting...")
except Exception as e:
print(f"\nAn error occurred: {e}")
if __name__ == "__main__":
@@ -276,17 +350,13 @@ if __name__ == "__main__":
### Key Concepts
- **Server-Sent Events (SSE)**: The protocol uses SSE format (`data: {json}\n\n`)
- **Event Types**: Different events provide metadata and content (all event types use UPPERCASE with underscores):
- `RUN_STARTED`: Signals the agent has started processing
- `TEXT_MESSAGE_START`: Signals the start of a text message from the agent
- `TEXT_MESSAGE_CONTENT`: Incremental text streamed from the agent (with `delta` field)
- `TEXT_MESSAGE_END`: Signals the end of a text message
- `RUN_FINISHED`: Signals successful completion
- `RUN_ERROR`: Error information if something goes wrong
- **Field Naming**: Event fields use camelCase (e.g., `threadId`, `runId`, `messageId`) when accessing JSON events
- **Thread Management**: The `threadId` maintains conversation context across requests
- **Client-Side Instructions**: System messages are sent from the client
- **`AGUIChatClient`**: Built-in client that implements the Agent Framework's `BaseChatClient` interface
- **Automatic Event Handling**: The client automatically converts AG-UI events to Agent Framework types
- **Thread Management**: Pass `thread_id` in metadata to maintain conversation context across requests
- **Streaming Responses**: Use `get_streaming_response()` for real-time streaming or `get_response()` for non-streaming
- **Context Manager**: Use `async with` for automatic cleanup of HTTP connections
- **Standard Interface**: Works with all Agent Framework patterns (ChatAgent, tools, etc.)
- **Hybrid Tool Execution**: Supports both client-side and server-side tools executing together in the same conversation
### Configure and Run the Client
@@ -312,327 +382,13 @@ Connecting to AG-UI server at: http://127.0.0.1:5100/
User (:q or quit to exit): What is the capital of France?
[Run Started - Thread: abc123, Run: xyz789]
The capital of France is Paris. It is known for its rich history, culture,
[Thread: abc123]
Assistant: The capital of France is Paris. It is known for its rich history, culture,
and iconic landmarks such as the Eiffel Tower and the Louvre Museum.
[Run Finished - Thread: abc123, Run: xyz789]
User (:q or quit to exit): Tell me a fun fact about space
[Run Started - Thread: abc123, Run: def456]
Here's a fun fact: A day on Venus is longer than its year! Venus takes
about 243 Earth days to rotate once on its axis, but only about 225 Earth
days to orbit the Sun.
[Run Finished - Thread: abc123, Run: def456]
User (:q or quit to exit): :q
```
### Color-Coded Output
The client displays different content types with distinct colors:
- **Yellow**: Run started notifications
- **Cyan**: Agent text responses (streamed in real-time)
- **Green**: Run completion notifications
- **Red**: Error messages
## Testing with curl (Optional)
Before running the client, you can test the server manually using curl:
```bash
curl -N http://127.0.0.1:5100/ \
-H "Content-Type: application/json" \
-H "Accept: text/event-stream" \
-d '{
"messages": [
{"role": "user", "content": "What is the capital of France?"}
]
}'
```
You should see Server-Sent Events streaming back:
```
data: {"type":"RUN_STARTED","threadId":"...","runId":"..."}
data: {"type":"TEXT_MESSAGE_START","messageId":"...","role":"assistant"}
data: {"type":"TEXT_MESSAGE_CONTENT","messageId":"...","delta":"The"}
data: {"type":"TEXT_MESSAGE_CONTENT","messageId":"...","delta":" capital"}
...
data: {"type":"TEXT_MESSAGE_END","messageId":"..."}
data: {"type":"RUN_FINISHED","threadId":"...","runId":"..."}
```
## How It Works
### Server-Side Flow
1. Client sends HTTP POST request with messages
2. FastAPI endpoint receives the request
3. `AgentFrameworkAgent` wrapper orchestrates the execution
4. Agent processes the messages using Agent Framework
5. `AgentFrameworkEventBridge` converts agent updates to AG-UI events
6. Responses are streamed back as Server-Sent Events (SSE)
7. Connection closes when the run completes
### Client-Side Flow
1. Client sends HTTP POST request to server endpoint
2. Server responds with SSE stream
3. Client parses incoming `data:` lines as JSON events
4. Each event is displayed based on its type
5. `threadId` is captured for conversation continuity
6. Stream completes when `RUN_FINISHED` event arrives
### Protocol Details
The AG-UI protocol uses:
- **HTTP POST** for sending requests
- **Server-Sent Events (SSE)** for streaming responses
- **JSON** for event serialization
- **Thread IDs** for maintaining conversation context
- **Run IDs** for tracking individual executions
- **Event type naming**: UPPERCASE with underscores (e.g., `RUN_STARTED`, `TEXT_MESSAGE_CONTENT`)
- **Field naming**: camelCase (e.g., `threadId`, `runId`, `messageId`)
## Advanced Features
The Python AG-UI implementation supports all 7 AG-UI features:
### 1. Backend Tool Rendering
Add tools to your agent for backend execution:
```python
from typing import Any
from agent_framework import ChatAgent, ai_function
from agent_framework.azure import AzureOpenAIChatClient
@ai_function
def get_weather(location: str) -> dict[str, Any]:
"""Get weather for a location."""
return {"temperature": 72, "conditions": "sunny"}
agent = ChatAgent(
name="weather_agent",
instructions="Use tools to help users.",
chat_client=AzureOpenAIChatClient(
endpoint="https://your-resource.openai.azure.com/",
deployment_name="gpt-4o-mini",
),
tools=[get_weather],
)
```
The client will receive `TOOL_CALL_START`, `TOOL_CALL_ARGS`, `TOOL_CALL_END`, and `TOOL_CALL_RESULT` events.
### 2. Human in the Loop
Request user confirmation before executing tools:
```python
from fastapi import FastAPI
from agent_framework import ChatAgent
from agent_framework.azure import AzureOpenAIChatClient
from agent_framework_ag_ui import AgentFrameworkAgent, add_agent_framework_fastapi_endpoint
agent = ChatAgent(
name="my_agent",
instructions="You are a helpful assistant.",
chat_client=AzureOpenAIChatClient(
endpoint="https://your-resource.openai.azure.com/",
deployment_name="gpt-4o-mini",
),
)
wrapped_agent = AgentFrameworkAgent(
agent=agent,
require_confirmation=True, # Enable human-in-the-loop
)
app = FastAPI()
add_agent_framework_fastapi_endpoint(app, wrapped_agent, "/")
```
The client receives tool approval request events and can send approval responses.
### 3. State Management
Share state between client and server:
```python
wrapped_agent = AgentFrameworkAgent(
agent=agent,
state_schema={
"location": {"type": "string"},
"preferences": {"type": "object"},
},
)
```
Events include `STATE_SNAPSHOT` and `STATE_DELTA` for bidirectional sync.
### 4. Predictive State Updates
Stream tool arguments as optimistic state updates:
```python
wrapped_agent = AgentFrameworkAgent(
agent=agent,
predict_state_config={
"location": {"tool": "get_weather", "tool_argument": "location"}
},
require_confirmation=False, # Auto-update without confirmation
)
```
State updates stream in real-time as the LLM generates tool arguments.
## Common Patterns
### Custom Server Configuration
```python
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
app = FastAPI()
# Add CORS for web clients
app.add_middleware(
CORSMiddleware,
allow_origins=["*"],
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
add_agent_framework_fastapi_endpoint(app, agent, "/agent")
```
### Multiple Agents
```python
app = FastAPI()
weather_agent = ChatAgent(name="weather", ...)
finance_agent = ChatAgent(name="finance", ...)
add_agent_framework_fastapi_endpoint(app, weather_agent, "/weather")
add_agent_framework_fastapi_endpoint(app, finance_agent, "/finance")
```
### Custom Client Timeout
```python
async with httpx.AsyncClient(timeout=300.0) as client:
async with client.stream("POST", server_url, ...) as response:
async for line in response.aiter_lines():
# Process events
pass
```
### Error Handling
```python
try:
async for event in client.send_message(message):
if event.get("type") == "RUN_ERROR":
error_msg = event.get("message", "Unknown error")
print(f"Error: {error_msg}")
# Handle error appropriately
except httpx.HTTPError as e:
print(f"HTTP error: {e}")
except Exception as e:
print(f"Unexpected error: {e}")
```
### Conversation Continuity
The client automatically maintains `threadId` across requests:
```python
client = AGUIClient(server_url)
# First message
async for event in client.send_message("Hello"):
# Client captures threadId from RUN_STARTED
pass
# Second message - uses same threadId
async for event in client.send_message("Continue our conversation"):
# Conversation context is maintained
pass
```
## AG-UI Event Reference
### Core Events
| Event Type | Description | Key Fields |
|------------|-------------|------------|
| `RUN_STARTED` | Agent execution started | `threadId`, `runId` |
| `RUN_FINISHED` | Agent execution completed | `threadId`, `runId` |
| `RUN_ERROR` | Agent execution error | `message` |
### Text Message Events
| Event Type | Description | Key Fields |
|------------|-------------|------------|
| `TEXT_MESSAGE_START` | Start of agent text message | `messageId`, `role` |
| `TEXT_MESSAGE_CONTENT` | Streaming text content | `messageId`, `delta` |
| `TEXT_MESSAGE_END` | End of agent text message | `messageId` |
### Tool Events
| Event Type | Description | Key Fields |
|------------|-------------|------------|
| `TOOL_CALL_START` | Tool call initiated | `toolCallId`, `toolCallName` |
| `TOOL_CALL_ARGS` | Tool arguments streaming | `toolCallId`, `delta` |
| `TOOL_CALL_END` | Tool call complete | `toolCallId` |
| `TOOL_CALL_RESULT` | Tool execution result | `toolCallId`, `content` |
### State Events
| Event Type | Description | Key Fields |
|------------|-------------|------------|
| `STATE_SNAPSHOT` | Complete state | `snapshot` |
| `STATE_DELTA` | State changes (JSON Patch) | `delta` |
### Other Events
| Event Type | Description | Key Fields |
|------------|-------------|------------|
| `MESSAGES_SNAPSHOT` | Conversation history | `messages` |
| `CUSTOM` | Custom event data | `name`, `value` |
## Next Steps
Now that you understand the basics of AG-UI, you can:
- **Add Tools**: Create custom `@ai_function` tools for your domain
- **Web Integration**: Build React/Vue frontends using the AG-UI protocol
- **State Management**: Implement shared state for generative UI applications
- **Human-in-the-Loop**: Add approval workflows for sensitive operations
- **Deployment**: Deploy to Azure Container Apps or Azure App Service
- **Multi-Agent Systems**: Coordinate multiple specialized agents
- **Monitoring**: Add logging and OpenTelemetry for observability
## Additional Resources
- [AG-UI Examples](../agent_framework_ag_ui_examples/README.md): Complete working examples for all 7 features
- [Agent Framework Documentation](../../core/README.md): Learn more about creating agents
- [AG-UI Protocol Spec](https://docs.ag-ui.com/): Official protocol documentation
## Troubleshooting
### Connection Refused