Python: restructure: Python samples into progressive 01-05 layout (#3862)

* restructure: Python samples into progressive 01-05 layout

- 01-get-started/: 6 numbered steps (hello agent → hosting)
- 02-agents/: all agent concept samples (tools, middleware, providers, etc.)
- 03-workflows/: ALL existing workflow samples preserved as-is
- 04-hosting/: azure-functions, durabletask, a2a
- 05-end-to-end/: demos, evaluation, hosted agents
- Old files moved to _to_delete/ for review
- Added AGENTS.md with structure documentation
- autogen-migration/ and semantic-kernel-migration/ preserved at root

* fix: switch to AzureOpenAI Foundry, fix CI failures

- Switch all 01-get-started samples to AzureOpenAIResponsesClient with
  Azure AI Foundry project endpoint (AZURE_AI_PROJECT_ENDPOINT +
  AZURE_OPENAI_RESPONSES_DEPLOYMENT_NAME + AzureCliCredential)
- Add _to_delete/ and 05-end-to-end/ to pyrightconfig.samples.json excludes
- Fix test paths in packages/ that referenced old getting_started/ dirs:
  durabletask conftest + streaming test, azurefunctions conftest,
  devui conftest + capture_messages + openai_sdk_integration
- Fix workflow_as_agent_human_in_the_loop.py import (sibling import)
- Update hosting READMEs and tool comment paths
- Replace root README.md with new structure overview
- Update AGENTS.md to document Azure OpenAI Foundry as default provider

* cleanup: remove _to_delete folder, copy resource files to active dirs

All files in _to_delete/ were either:
- Exact duplicates of files in the new structure (240 files)
- Same file with only comment path updates (100 files)
- One import-fix diff (workflow_as_agent_human_in_the_loop.py)
- One superseded minimal_sample.py

Resource files (sample.pdf, countries.json, employees.pdf, weather.json)
copied to 02-agents/sample_assets/ and 02-agents/resources/ since active
samples reference them.

* fix: address PR review comments, centralize resources, remove root duplicates

- Fix type annotation in 04_memory.py (string union -> proper types)
- Fix old sample paths in observability files
- Fix grammar/spelling in observability samples
- Move sample_assets/ and resources/ to shared/ folder
- Remove 8 duplicate observability files from 02-agents root
- Update resource path references in multimodal_input and provider samples

* fix: update broken links from old getting_started paths to new structure

- Update relative paths in READMEs: getting_started/ → 01-get-started/,
  02-agents/, 03-workflows/, 04-hosting/, 05-end-to-end/
- Fix absolute GitHub URLs in package READMEs
- Fix broken link in ollama package README

* fix: convert absolute GitHub URLs to relative paths for link checker

Absolute URLs to python/samples/ on main branch 404 until PR merges.
Converted to relative paths that linkspector can verify locally.

* fix: update link for handoff sample moved to orchestrations/

* fix: update chatkit-integration README path from demos/ to 05-end-to-end/

* fix: update broken links in orchestrations README to match flat directory structure
This commit is contained in:
Eduard van Valkenburg
2026-02-12 18:36:36 +01:00
committed by GitHub
Unverified
parent 69dcfe31ee
commit a2856d3b92
536 changed files with 3816 additions and 1632 deletions
@@ -1,4 +0,0 @@
*.db
*.db-shm
*.db-wal
uploads/
@@ -1,318 +0,0 @@
# ChatKit Integration Sample with Weather Agent and Image Analysis
This sample demonstrates how to integrate Microsoft Agent Framework with OpenAI ChatKit. It provides a complete implementation of a weather assistant with interactive widget visualization, image analysis, and file upload support.
**Features:**
- Weather information with interactive widgets
- Image analysis using vision models
- Current time queries
- File upload with attachment storage
- Chat interface with streaming responses
- City selector widget with one-click weather
## Architecture
```mermaid
graph TB
subgraph Frontend["React Frontend (ChatKit UI)"]
UI[ChatKit Components]
Upload[File Upload]
end
subgraph Backend["FastAPI Server"]
FastAPI[FastAPI Endpoints]
subgraph ChatKit["WeatherChatKitServer"]
Respond[respond method]
Action[action method]
end
subgraph Stores["Data & Storage Layer"]
SQLite[SQLiteStore<br/>Store Protocol]
AttStore[FileBasedAttachmentStore<br/>AttachmentStore Protocol]
DB[(SQLite DB<br/>chatkit_demo.db)]
Files[/uploads directory/]
end
subgraph Integration["Agent Framework Integration"]
Converter[ThreadItemConverter]
Streamer[stream_agent_response]
Agent[Agent]
end
Widgets[Widget Rendering<br/>render_weather_widget<br/>render_city_selector_widget]
end
subgraph Azure["Azure AI"]
Foundry[GPT-5<br/>with Vision]
end
UI -->|HTTP POST /chatkit| FastAPI
Upload -->|HTTP POST /upload/id| FastAPI
FastAPI --> ChatKit
ChatKit -->|save/load threads| SQLite
ChatKit -->|save/load attachments| AttStore
ChatKit -->|convert messages| Converter
SQLite -.->|persist| DB
AttStore -.->|save files| Files
AttStore -.->|save metadata| SQLite
Converter -->|Message array| Agent
Agent -->|AgentResponseUpdate| Streamer
Streamer -->|ThreadStreamEvent| ChatKit
ChatKit --> Widgets
Widgets -->|WidgetItem| ChatKit
Agent <-->|Chat Completions API| Foundry
ChatKit -->|ThreadStreamEvent| FastAPI
FastAPI -->|SSE Stream| UI
style ChatKit fill:#e1f5ff
style Stores fill:#fff4e1
style Integration fill:#f0e1ff
style Azure fill:#e1ffe1
```
### Server Implementation
The sample implements a ChatKit server using the `ChatKitServer` base class from the `chatkit` package:
**Core Components:**
- **`WeatherChatKitServer`**: Custom ChatKit server implementation that:
- Extends `ChatKitServer[dict[str, Any]]`
- Uses Agent Framework's `Agent` with Azure OpenAI
- Converts ChatKit messages to Agent Framework format using `ThreadItemConverter`
- Streams responses back to ChatKit using `stream_agent_response`
- Creates and streams interactive widgets after agent responses
- **`SQLiteStore`**: Data persistence layer that:
- Implements the `Store[dict[str, Any]]` protocol from ChatKit
- Persists threads, messages, and attachment metadata in SQLite
- Provides thread management and item history
- Stores attachment metadata for the upload lifecycle
- **`FileBasedAttachmentStore`**: File storage implementation that:
- Implements the `AttachmentStore[dict[str, Any]]` protocol from ChatKit
- Stores uploaded files on the local filesystem (in `./uploads` directory)
- Generates upload URLs for two-phase file upload
- Saves attachment metadata to the data store for upload tracking
- Provides preview URLs for images
**Key Integration Points:**
```python
# Converting ChatKit messages to Agent Framework
converter = ThreadItemConverter(
attachment_data_fetcher=self._fetch_attachment_data
)
agent_messages = await converter.to_agent_input(user_message_item)
# Running agent and streaming back to ChatKit
async for event in stream_agent_response(
self.weather_agent.run(agent_messages, stream=True),
thread_id=thread.id,
):
yield event
# Streaming widgets
widget = render_weather_widget(weather_data)
async for event in stream_widget(thread_id=thread.id, widget=widget):
yield event
```
## Installation and Setup
### Prerequisites
- Python 3.10+
- Node.js 18.18+ and npm 9+
- Azure OpenAI service configured
- Azure CLI for authentication (`az login`)
### Network Requirements
> **Important:** This sample uses the OpenAI ChatKit frontend, which requires internet connectivity to OpenAI services.
The frontend makes outbound requests to:
- `cdn.platform.openai.com` - ChatKit UI library (required)
- `chatgpt.com` - Configuration endpoint
- `api-js.mixpanel.com` - Telemetry
**This sample is not suitable for air-gapped or network-restricted environments.** The ChatKit frontend library cannot be self-hosted. See [Limitations](#limitations) for details.
### Domain Key Configuration
For **local development**, the sample uses a default domain key (`domain_pk_localhost_dev`).
For **production deployment**:
1. Register your domain at [platform.openai.com](https://platform.openai.com/settings/organization/security/domain-allowlist)
2. Create a `.env` file in the `frontend` directory:
```
VITE_CHATKIT_API_DOMAIN_KEY=your_domain_key_here
```
### Backend Setup
1. **Install Python packages:**
```bash
cd python/samples/demos/chatkit-integration
pip install agent-framework-chatkit fastapi uvicorn azure-identity
```
2. **Configure Azure OpenAI:**
```bash
export AZURE_OPENAI_ENDPOINT="https://your-resource.openai.azure.com/"
export AZURE_OPENAI_API_VERSION="2024-06-01"
export AZURE_OPENAI_CHAT_DEPLOYMENT_NAME="gpt-4o"
```
3. **Authenticate with Azure:**
```bash
az login
```
### Frontend Setup
Install the Node.js dependencies:
```bash
cd frontend
npm install
```
## How to Run
### Start the Backend Server
From the `chatkit-integration` directory:
```bash
python app.py
```
Or with auto-reload for development:
```bash
uvicorn app:app --host 127.0.0.1 --port 8001 --reload
```
The backend will start on `http://localhost:8001`
### Start the Frontend Development Server
In a new terminal, from the `frontend` directory:
```bash
npm run dev
```
The frontend will start on `http://localhost:5171`
### Access the Application
Open your browser and navigate to:
```
http://localhost:5171
```
You can now:
- Ask about weather in any location (weather widgets display automatically)
- Upload images for analysis using the attachment button
- Get the current time
- Ask to see available cities and click city buttons for instant weather
### Project Structure
```
chatkit-integration/
├── app.py # FastAPI backend with ChatKitServer implementation
├── store.py # SQLiteStore implementation
├── attachment_store.py # FileBasedAttachmentStore implementation
├── weather_widget.py # Widget rendering functions
├── chatkit_demo.db # SQLite database (auto-created)
├── uploads/ # Uploaded files directory (auto-created)
└── frontend/
├── package.json
├── vite.config.ts
├── index.html
└── src/
├── main.tsx
└── App.tsx # ChatKit UI integration
```
### Configuration
You can customize the application by editing constants at the top of `app.py`:
```python
# Server configuration
SERVER_HOST = "127.0.0.1" # Bind to localhost only for security (local dev)
SERVER_PORT = 8001
SERVER_BASE_URL = f"http://localhost:{SERVER_PORT}"
# Database configuration
DATABASE_PATH = "chatkit_demo.db"
# File storage configuration
UPLOADS_DIRECTORY = "./uploads"
# User context
DEFAULT_USER_ID = "demo_user"
```
### Sample Conversations
Try these example queries:
- "What's the weather like in Tokyo?"
- "Show me available cities" (displays interactive city selector)
- "What's the current time?"
- Upload an image and ask "What do you see in this image?"
## Limitations
### Air-Gapped / Regulated Environments
The ChatKit frontend (`chatkit.js`) is loaded from OpenAI's CDN and cannot be self-hosted. This means:
- **Not suitable for air-gapped environments** where `*.openai.com` is blocked
- **Not suitable for regulated environments** that prohibit external telemetry
- **Requires domain registration** with OpenAI for production use
**What you CAN self-host:**
- The Python backend (FastAPI server, `ChatKitServer`, stores)
- The `agent-framework-chatkit` integration layer
- Your LLM infrastructure (Azure OpenAI, local models, etc.)
**What you CANNOT self-host:**
- The ChatKit frontend UI library
For more details, see:
- [openai/chatkit-js#57](https://github.com/openai/chatkit-js/issues/57) - Self-hosting feature request
- [openai/chatkit-js#76](https://github.com/openai/chatkit-js/issues/76) - Domain key requirements
## Learn More
- [Agent Framework Documentation](https://aka.ms/agent-framework)
- [ChatKit Documentation](https://platform.openai.com/docs/guides/chatkit)
- [Azure OpenAI Documentation](https://learn.microsoft.com/en-us/azure/ai-foundry/)
@@ -1 +0,0 @@
# Copyright (c) Microsoft. All rights reserved.
@@ -1,645 +0,0 @@
# /// script
# requires-python = ">=3.10"
# dependencies = [
# "fastapi",
# "uvicorn",
# ]
# ///
# Run with any PEP 723 compatible runner, e.g.:
# uv run samples/demos/chatkit-integration/app.py
# Copyright (c) Microsoft. All rights reserved.
"""
ChatKit Integration Sample with Weather Agent and Image Analysis
This sample demonstrates how to integrate Microsoft Agent Framework with OpenAI ChatKit
using a weather tool with widget visualization, image analysis, and Azure OpenAI. It shows
a complete ChatKit server implementation using Agent Framework agents with proper FastAPI
setup, interactive weather widgets, and vision capabilities for analyzing uploaded images.
"""
import logging
from collections.abc import AsyncIterator, Callable
from datetime import datetime, timezone
from random import randint
from typing import Annotated, Any
import uvicorn
# Agent Framework imports
from agent_framework import Agent, AgentResponseUpdate, FunctionResultContent, Message, Role, tool
from agent_framework.azure import AzureOpenAIChatClient
# Agent Framework ChatKit integration
from agent_framework_chatkit import ThreadItemConverter, stream_agent_response
# Local imports
from attachment_store import FileBasedAttachmentStore
from azure.identity import AzureCliCredential
# ChatKit imports
from chatkit.actions import Action
from chatkit.server import ChatKitServer
from chatkit.store import StoreItemType, default_generate_id
from chatkit.types import (
ThreadItem,
ThreadItemDoneEvent,
ThreadMetadata,
ThreadStreamEvent,
UserMessageItem,
WidgetItem,
)
from chatkit.widgets import WidgetRoot
from fastapi import FastAPI, File, Request, UploadFile
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import FileResponse, JSONResponse, Response, StreamingResponse
from pydantic import Field
from store import SQLiteStore
from weather_widget import (
WeatherData,
city_selector_copy_text,
render_city_selector_widget,
render_weather_widget,
weather_widget_copy_text,
)
# ============================================================================
# Configuration Constants
# ============================================================================
# Server configuration
SERVER_HOST = "127.0.0.1" # Bind to localhost only for security (local dev)
SERVER_PORT = 8001
SERVER_BASE_URL = f"http://localhost:{SERVER_PORT}"
# Database configuration
DATABASE_PATH = "chatkit_demo.db"
# File storage configuration
UPLOADS_DIRECTORY = "./uploads"
# User context
DEFAULT_USER_ID = "demo_user"
# Logging configuration
LOG_LEVEL = logging.INFO
LOG_FORMAT = "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
LOG_DATE_FORMAT = "%Y-%m-%d %H:%M:%S"
# ============================================================================
# Logging Setup
# ============================================================================
logging.basicConfig(
level=LOG_LEVEL,
format=LOG_FORMAT,
datefmt=LOG_DATE_FORMAT,
)
logger = logging.getLogger(__name__)
class WeatherResponse(str):
"""A string response that also carries WeatherData for widget creation."""
def __new__(cls, text: str, weather_data: WeatherData):
instance = super().__new__(cls, text)
instance.weather_data = weather_data # type: ignore
return instance
async def stream_widget(
thread_id: str,
widget: WidgetRoot,
copy_text: str | None = None,
generate_id: Callable[[StoreItemType], str] = default_generate_id,
) -> AsyncIterator[ThreadStreamEvent]:
"""Stream a ChatKit widget as a ThreadStreamEvent.
This helper function creates a ChatKit widget item and yields it as a
ThreadItemDoneEvent that can be consumed by the ChatKit UI.
Args:
thread_id: The ChatKit thread ID for the conversation.
widget: The ChatKit widget to display.
copy_text: Optional text representation of the widget for copy/paste.
generate_id: Optional function to generate IDs for ChatKit items.
Yields:
ThreadStreamEvent: ChatKit event containing the widget.
"""
item_id = generate_id("message")
widget_item = WidgetItem(
id=item_id,
thread_id=thread_id,
created_at=datetime.now(),
widget=widget,
copy_text=copy_text,
)
yield ThreadItemDoneEvent(type="thread.item.done", item=widget_item)
# NOTE: approval_mode="never_require" is for sample brevity. Use "always_require" in production; see samples/getting_started/tools/function_tool_with_approval.py and samples/getting_started/tools/function_tool_with_approval_and_threads.py.
@tool(approval_mode="never_require")
def get_weather(
location: Annotated[str, Field(description="The location to get the weather for.")],
) -> str:
"""Get the weather for a given location.
Returns a string description with embedded WeatherData for widget creation.
"""
logger.info(f"Fetching weather for location: {location}")
conditions = ["sunny", "cloudy", "rainy", "stormy", "snowy", "foggy"]
temperature = randint(-5, 35)
condition = conditions[randint(0, len(conditions) - 1)]
# Add some realistic details
humidity = randint(30, 90)
wind_speed = randint(5, 25)
weather_data = WeatherData(
location=location,
condition=condition,
temperature=temperature,
humidity=humidity,
wind_speed=wind_speed,
)
logger.debug(f"Weather data generated: {condition}, {temperature}°C, {humidity}% humidity, {wind_speed} km/h wind")
# Return a WeatherResponse that is both a string (for the LLM) and carries structured data
text = (
f"Weather in {location}:\n"
f"• Condition: {condition.title()}\n"
f"• Temperature: {temperature}°C\n"
f"• Humidity: {humidity}%\n"
f"• Wind: {wind_speed} km/h"
)
return WeatherResponse(text, weather_data)
@tool(approval_mode="never_require")
def get_time() -> str:
"""Get the current UTC time."""
current_time = datetime.now(timezone.utc)
logger.info("Getting current UTC time")
return f"Current UTC time: {current_time.strftime('%Y-%m-%d %H:%M:%S')} UTC"
@tool(approval_mode="never_require")
def show_city_selector() -> str:
"""Show an interactive city selector widget to the user.
This function triggers the display of a widget that allows users
to select from popular cities to get weather information.
Returns a special marker string that will be detected to show the widget.
"""
logger.info("Activating city selector widget")
return "__SHOW_CITY_SELECTOR__"
class WeatherChatKitServer(ChatKitServer[dict[str, Any]]):
"""ChatKit server implementation using Agent Framework.
This server integrates Agent Framework agents with ChatKit's server protocol,
providing weather information with interactive widgets and time queries through Azure OpenAI.
"""
def __init__(self, data_store: SQLiteStore, attachment_store: FileBasedAttachmentStore):
super().__init__(data_store, attachment_store)
logger.info("Initializing WeatherChatKitServer")
# Create Agent Framework agent with Azure OpenAI
# For authentication, run `az login` command in terminal
try:
self.weather_agent = Agent(
client=AzureOpenAIChatClient(credential=AzureCliCredential()),
instructions=(
"You are a helpful weather assistant with image analysis capabilities. "
"You can provide weather information for any location, tell the current time, "
"and analyze images that users upload. Be friendly and informative in your responses.\n\n"
"If a user asks to see a list of cities or wants to choose from available cities, "
"use the show_city_selector tool to display an interactive city selector.\n\n"
"When users upload images, you will automatically receive them and can analyze their content. "
"Describe what you see in detail and be helpful in answering questions about the images."
),
tools=[get_weather, get_time, show_city_selector],
)
logger.info("Weather agent initialized successfully with Azure OpenAI")
except Exception as e:
logger.error(f"Failed to initialize weather agent: {e}")
raise
# Create ThreadItemConverter with attachment data fetcher
self.converter = ThreadItemConverter(
attachment_data_fetcher=self._fetch_attachment_data,
)
logger.info("WeatherChatKitServer initialized")
async def _fetch_attachment_data(self, attachment_id: str) -> bytes:
"""Fetch attachment binary data for the converter.
Args:
attachment_id: The ID of the attachment to fetch.
Returns:
The binary data of the attachment.
"""
return await attachment_store.read_attachment_bytes(attachment_id)
async def _update_thread_title(
self, thread: ThreadMetadata, thread_items: list[ThreadItem], context: dict[str, Any]
) -> None:
"""Update thread title using LLM to generate a concise summary.
Args:
thread: The thread metadata to update.
thread_items: All items in the thread.
context: The context dictionary.
"""
logger.info(f"Attempting to update thread title for thread: {thread.id}")
if not thread_items:
logger.debug("No thread items available for title generation")
return
# Collect user messages to understand the conversation topic
user_messages: list[str] = []
for item in thread_items:
if isinstance(item, UserMessageItem) and item.content:
for content_part in item.content:
if hasattr(content_part, "text") and isinstance(content_part.text, str):
user_messages.append(content_part.text)
break
if not user_messages:
logger.debug("No user messages found for title generation")
return
logger.debug(f"Found {len(user_messages)} user message(s) for title generation")
try:
# Use the agent's chat client to generate a concise title
# Combine first few messages to capture the conversation topic
conversation_context = "\n".join(user_messages[:3])
title_prompt = [
Message(
role=Role.USER,
text=(
f"Generate a very short, concise title (max 40 characters) for a conversation "
f"that starts with:\n\n{conversation_context}\n\n"
"Respond with ONLY the title, nothing else."
),
)
]
# Use the chat client directly for a quick, lightweight call
response = await self.weather_agent.client.get_response(
messages=title_prompt,
options={
"temperature": 0.3,
"max_tokens": 20,
},
)
if response.messages and response.messages[-1].text:
title = response.messages[-1].text.strip().strip('"').strip("'")
# Ensure it's not too long
if len(title) > 50:
title = title[:47] + "..."
thread.title = title
await self.store.save_thread(thread, context)
logger.info(f"Updated thread {thread.id} title to: {title}")
except Exception as e:
logger.warning(f"Failed to generate thread title, using fallback: {e}")
# Fallback to simple truncation
first_message: str = user_messages[0]
title: str = first_message[:50].strip()
if len(first_message) > 50:
title += "..."
thread.title = title
await self.store.save_thread(thread, context)
logger.info(f"Updated thread {thread.id} title to (fallback): {title}")
async def respond(
self,
thread: ThreadMetadata,
input_user_message: UserMessageItem | None,
context: dict[str, Any],
) -> AsyncIterator[ThreadStreamEvent]:
"""Handle incoming user messages and generate responses.
This method converts ChatKit messages to Agent Framework format using ThreadItemConverter,
runs the agent, converts the response back to ChatKit events using stream_agent_response,
and creates interactive weather widgets when weather data is queried.
"""
from agent_framework import FunctionResultContent
if input_user_message is None:
logger.debug("Received None user message, skipping")
return
logger.info(f"Processing message for thread: {thread.id}")
try:
# Track weather data and city selector flag for this request
weather_data: WeatherData | None = None
show_city_selector = False
# Load full thread history from the store
thread_items_page = await self.store.load_thread_items(
thread_id=thread.id,
after=None,
limit=1000,
order="asc",
context=context,
)
thread_items = thread_items_page.data
# Convert ALL thread items to Agent Framework ChatMessages using ThreadItemConverter
# This ensures the agent has the full conversation context
agent_messages = await self.converter.to_agent_input(thread_items)
if not agent_messages:
logger.warning("No messages after conversion")
return
logger.info(f"Running agent with {len(agent_messages)} message(s)")
# Run the Agent Framework agent with streaming
agent_stream = self.weather_agent.run(agent_messages, stream=True)
# Create an intercepting stream that extracts function results while passing through updates
async def intercept_stream() -> AsyncIterator[AgentResponseUpdate]:
nonlocal weather_data, show_city_selector
async for update in agent_stream:
# Check for function results in the update
if update.contents:
for content in update.contents:
if isinstance(content, FunctionResultContent):
result = content.result
# Check if it's a WeatherResponse (string subclass with weather_data attribute)
if isinstance(result, str) and hasattr(result, "weather_data"):
extracted_data = getattr(result, "weather_data", None)
if isinstance(extracted_data, WeatherData):
weather_data = extracted_data
logger.info(f"Weather data extracted: {weather_data.location}")
# Check if it's the city selector marker
elif isinstance(result, str) and result == "__SHOW_CITY_SELECTOR__":
show_city_selector = True
logger.info("City selector flag detected")
yield update
# Stream updates as ChatKit events with interception
async for event in stream_agent_response(
intercept_stream(),
thread_id=thread.id,
):
yield event
# If weather data was collected during the tool call, create a widget
if weather_data is not None and isinstance(weather_data, WeatherData):
logger.info(f"Creating weather widget for location: {weather_data.location}")
# Create weather widget
widget = render_weather_widget(weather_data)
copy_text = weather_widget_copy_text(weather_data)
# Stream the widget
async for widget_event in stream_widget(thread_id=thread.id, widget=widget, copy_text=copy_text):
yield widget_event
logger.debug("Weather widget streamed successfully")
# If city selector should be shown, create and stream that widget
if show_city_selector:
logger.info("Creating city selector widget")
# Create city selector widget
selector_widget = render_city_selector_widget()
selector_copy_text = city_selector_copy_text()
# Stream the widget
async for widget_event in stream_widget(
thread_id=thread.id, widget=selector_widget, copy_text=selector_copy_text
):
yield widget_event
logger.debug("City selector widget streamed successfully")
# Update thread title based on first user message if not already set
if not thread.title or thread.title == "New thread":
await self._update_thread_title(thread, thread_items, context)
logger.info(f"Completed processing message for thread: {thread.id}")
except Exception as e:
logger.error(f"Error processing message for thread {thread.id}: {e}", exc_info=True)
async def action(
self,
thread: ThreadMetadata,
action: Action[str, Any],
sender: WidgetItem | None,
context: dict[str, Any],
) -> AsyncIterator[ThreadStreamEvent]:
"""Handle widget actions from the frontend.
This method processes actions triggered by interactive widgets,
such as city selection from the city selector widget.
"""
logger.info(f"Received action: {action.type} for thread: {thread.id}")
if action.type == "city_selected":
# Extract city information from the action payload
city_label = action.payload.get("city_label", "Unknown")
logger.info(f"City selected: {city_label}")
logger.debug(f"Action payload: {action.payload}")
# Track weather data for this request
weather_data: WeatherData | None = None
# Create an agent message asking about the weather
agent_messages = [Message(role=Role.USER, text=f"What's the weather in {city_label}?")]
logger.debug(f"Processing weather query: {agent_messages[0].text}")
# Run the Agent Framework agent with streaming
agent_stream = self.weather_agent.run(agent_messages, stream=True)
# Create an intercepting stream that extracts function results while passing through updates
async def intercept_stream() -> AsyncIterator[AgentResponseUpdate]:
nonlocal weather_data
async for update in agent_stream:
# Check for function results in the update
if update.contents:
for content in update.contents:
if isinstance(content, FunctionResultContent):
result = content.result
# Check if it's a WeatherResponse (string subclass with weather_data attribute)
if isinstance(result, str) and hasattr(result, "weather_data"):
extracted_data = getattr(result, "weather_data", None)
if isinstance(extracted_data, WeatherData):
weather_data = extracted_data
logger.info(f"Weather data extracted: {weather_data.location}")
yield update
# Stream updates as ChatKit events with interception
async for event in stream_agent_response(
intercept_stream(),
thread_id=thread.id,
):
yield event
# If weather data was collected during the tool call, create a widget
if weather_data is not None and isinstance(weather_data, WeatherData):
logger.info(f"Creating weather widget for: {weather_data.location}")
# Create weather widget
widget = render_weather_widget(weather_data)
copy_text = weather_widget_copy_text(weather_data)
# Stream the widget
async for widget_event in stream_widget(thread_id=thread.id, widget=widget, copy_text=copy_text):
yield widget_event
logger.debug("Weather widget created successfully from action")
else:
logger.warning("No weather data available to create widget after action")
# FastAPI application setup
app = FastAPI(
title="ChatKit Weather & Vision Agent",
description="Weather and image analysis assistant powered by Agent Framework and Azure OpenAI",
version="1.0.0",
)
# Add CORS middleware to allow frontend connections
app.add_middleware(
CORSMiddleware,
allow_origins=["*"], # In production, specify exact origins
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
# Initialize data store and ChatKit server
logger.info("Initializing application components")
data_store = SQLiteStore(db_path=DATABASE_PATH)
attachment_store = FileBasedAttachmentStore(
uploads_dir=UPLOADS_DIRECTORY,
base_url=SERVER_BASE_URL,
data_store=data_store,
)
chatkit_server = WeatherChatKitServer(data_store, attachment_store)
logger.info("Application initialization complete")
@app.post("/chatkit")
async def chatkit_endpoint(request: Request):
"""Main ChatKit endpoint that handles all ChatKit requests.
This endpoint follows the ChatKit server protocol and handles both
streaming and non-streaming responses.
"""
logger.debug(f"Received ChatKit request from {request.client}")
request_body = await request.body()
# Create context following the working examples pattern
context = {"request": request}
try:
# Process the request using ChatKit server
result = await chatkit_server.process(request_body, context)
# Return appropriate response type
if hasattr(result, "__aiter__"): # StreamingResult
logger.debug("Returning streaming response")
return StreamingResponse(result, media_type="text/event-stream") # type: ignore[arg-type]
# NonStreamingResult
logger.debug("Returning non-streaming response")
return Response(content=result.json, media_type="application/json") # type: ignore[union-attr]
except Exception as e:
logger.error(f"Error processing ChatKit request: {e}", exc_info=True)
raise
@app.post("/upload/{attachment_id}")
async def upload_file(attachment_id: str, file: UploadFile = File(...)): # noqa: B008
"""Handle file upload for two-phase upload.
The client POSTs the file bytes here after creating the attachment
via the ChatKit attachments.create endpoint.
"""
logger.info(f"Receiving file upload for attachment: {attachment_id}")
try:
# Read file contents
contents = await file.read()
# Save to disk
file_path = attachment_store.get_file_path(attachment_id)
file_path.write_bytes(contents)
logger.info(f"Saved {len(contents)} bytes to {file_path}")
# Load the attachment metadata from the data store
attachment = await data_store.load_attachment(attachment_id, {"user_id": DEFAULT_USER_ID})
# Clear the upload_url since upload is complete
attachment.upload_url = None
# Save the updated attachment back to the store
await data_store.save_attachment(attachment, {"user_id": DEFAULT_USER_ID})
# Return the attachment metadata as JSON
return JSONResponse(content=attachment.model_dump(mode="json"))
except Exception as e:
logger.error(f"Error uploading file for attachment {attachment_id}: {e}", exc_info=True)
return JSONResponse(status_code=500, content={"error": "Failed to upload file."})
@app.get("/preview/{attachment_id}")
async def preview_image(attachment_id: str):
"""Serve image preview/thumbnail.
For simplicity, this serves the full image. In production, you should
generate and cache thumbnails.
"""
logger.debug(f"Serving preview for attachment: {attachment_id}")
try:
file_path = attachment_store.get_file_path(attachment_id)
if not file_path.exists():
return JSONResponse(status_code=404, content={"error": "File not found"})
# Determine media type from file extension or attachment metadata
# For simplicity, we'll try to load from the store
try:
attachment = await data_store.load_attachment(attachment_id, {"user_id": DEFAULT_USER_ID})
media_type = attachment.mime_type
except Exception:
# Default to binary if we can't determine
media_type = "application/octet-stream"
return FileResponse(file_path, media_type=media_type)
except Exception as e:
logger.error(f"Error serving preview for attachment {attachment_id}: {e}", exc_info=True)
return JSONResponse(status_code=500, content={"error": "Error serving preview for attachment."})
if __name__ == "__main__":
# Run the server
logger.info(f"Starting ChatKit Weather Agent server on {SERVER_HOST}:{SERVER_PORT}")
uvicorn.run(app, host=SERVER_HOST, port=SERVER_PORT, log_level="info")
@@ -1,119 +0,0 @@
# Copyright (c) Microsoft. All rights reserved.
"""File-based AttachmentStore implementation for ChatKit.
This module provides a simple AttachmentStore implementation that stores
uploaded files on the local filesystem. In production, you should use
cloud storage like S3, Azure Blob Storage, or Google Cloud Storage.
"""
from pathlib import Path
from typing import TYPE_CHECKING, Any
from chatkit.store import AttachmentStore
from chatkit.types import Attachment, AttachmentCreateParams, FileAttachment, ImageAttachment
from pydantic import AnyUrl
if TYPE_CHECKING:
from store import SQLiteStore
class FileBasedAttachmentStore(AttachmentStore[dict[str, Any]]):
"""File-based AttachmentStore that stores files on local disk.
This implementation stores uploaded files in a local directory and provides
upload URLs that point to the FastAPI upload endpoint. It supports both
image and file attachments.
Features:
- Stores files in a local uploads directory
- Generates upload URLs for two-phase upload
- Generates preview URLs for images
- Proper cleanup on deletion
Note: This is for demonstration purposes. In production, use cloud storage
with signed URLs for better security and scalability.
"""
def __init__(
self,
uploads_dir: str = "./uploads",
base_url: str = "http://localhost:8001",
data_store: "SQLiteStore | None" = None,
):
"""Initialize the file-based attachment store.
Args:
uploads_dir: Directory where uploaded files will be stored
base_url: Base URL for generating upload and preview URLs
data_store: Optional data store to persist attachment metadata
"""
self.uploads_dir = Path(uploads_dir)
self.base_url = base_url.rstrip("/")
self.data_store = data_store
# Create uploads directory if it doesn't exist
self.uploads_dir.mkdir(parents=True, exist_ok=True)
def get_file_path(self, attachment_id: str) -> Path:
"""Get the filesystem path for an attachment."""
return self.uploads_dir / attachment_id
async def delete_attachment(self, attachment_id: str, context: dict[str, Any]) -> None:
"""Delete an attachment and its file from disk."""
file_path = self.get_file_path(attachment_id)
if file_path.exists():
file_path.unlink()
async def create_attachment(self, input: AttachmentCreateParams, context: dict[str, Any]) -> Attachment:
"""Create an attachment with upload URL for two-phase upload.
This creates the attachment metadata and returns upload URLs that
the client will use to POST the actual file bytes.
"""
# Generate unique ID for this attachment
attachment_id = self.generate_attachment_id(input.mime_type, context)
# Generate upload URL that points to our FastAPI upload endpoint
upload_url = f"{self.base_url}/upload/{attachment_id}"
# Create appropriate attachment type based on MIME type
if input.mime_type.startswith("image/"):
# For images, also provide a preview URL
preview_url = f"{self.base_url}/preview/{attachment_id}"
attachment = ImageAttachment(
id=attachment_id,
type="image",
mime_type=input.mime_type,
name=input.name,
upload_url=AnyUrl(upload_url),
preview_url=AnyUrl(preview_url),
)
else:
# For files, just provide upload URL
attachment = FileAttachment(
id=attachment_id,
type="file",
mime_type=input.mime_type,
name=input.name,
upload_url=AnyUrl(upload_url),
)
# Save attachment metadata to data store so it's available during upload
if self.data_store is not None:
await self.data_store.save_attachment(attachment, context)
return attachment
async def read_attachment_bytes(self, attachment_id: str) -> bytes:
"""Read the raw bytes of an uploaded attachment.
This is used by the ThreadItemConverter to create base64-encoded
content for sending to the Agent Framework.
"""
file_path = self.get_file_path(attachment_id)
if not file_path.exists():
raise FileNotFoundError(f"Attachment {attachment_id} not found on disk")
return file_path.read_bytes()
@@ -1,57 +0,0 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>ChatKit + Agent Framework Demo</title>
<!--
IMPORTANT: The ChatKit UI library is loaded from OpenAI's CDN and cannot be self-hosted.
This requires internet connectivity and is not suitable for air-gapped environments.
See: https://github.com/openai/chatkit-js/issues/57
-->
<script src="https://cdn.platform.openai.com/deployments/chatkit/chatkit.js"></script>
<style>
* {
margin: 0;
padding: 0;
box-sizing: border-box;
}
body {
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, 'Helvetica Neue', Arial, sans-serif;
height: 100vh;
display: flex;
flex-direction: column;
}
header {
padding: 1rem;
background: #f5f5f5;
border-bottom: 1px solid #ddd;
}
h1 {
font-size: 1.5rem;
margin-bottom: 0.5rem;
}
p {
color: #666;
font-size: 0.9rem;
}
#root {
flex: 1;
overflow: hidden;
}
</style>
</head>
<body>
<header>
<h1>ChatKit + Agent Framework Demo</h1>
<p>Simple weather assistant powered by Agent Framework and ChatKit</p>
</header>
<div id="root"></div>
<script type="module" src="/src/main.tsx"></script>
</body>
</html>
File diff suppressed because it is too large Load Diff
@@ -1,27 +0,0 @@
{
"name": "chatkit-agent-framework-demo",
"version": "0.1.0",
"private": true,
"type": "module",
"scripts": {
"dev": "vite",
"build": "vite build",
"preview": "vite preview"
},
"engines": {
"node": ">=18.18",
"npm": ">=9"
},
"dependencies": {
"@openai/chatkit-react": "^0",
"react": "^19.2.0",
"react-dom": "^19.2.0"
},
"devDependencies": {
"@types/react": "^19.2.0",
"@types/react-dom": "^19.2.0",
"@vitejs/plugin-react-swc": "^3.5.0",
"typescript": "^5.4.0",
"vite": "^7.1.12"
}
}
@@ -1,39 +0,0 @@
import { ChatKit, useChatKit } from "@openai/chatkit-react";
const CHATKIT_API_URL = "/chatkit";
// Domain key for ChatKit integration
// - Local development: Uses default "domain_pk_localhost_dev"
// - Production: Register your domain at https://platform.openai.com/settings/organization/security/domain-allowlist
// and set VITE_CHATKIT_API_DOMAIN_KEY in your .env file
// See: https://github.com/openai/chatkit-js/issues/76
const CHATKIT_API_DOMAIN_KEY =
import.meta.env.VITE_CHATKIT_API_DOMAIN_KEY ?? "domain_pk_localhost_dev";
export default function App() {
const chatkit = useChatKit({
api: {
url: CHATKIT_API_URL,
domainKey: CHATKIT_API_DOMAIN_KEY,
uploadStrategy: { type: "two_phase" },
},
startScreen: {
greeting: "Hello! I'm your weather and image analysis assistant. Ask me about the weather in any location or upload images for me to analyze.",
prompts: [
{ label: "Weather in New York", prompt: "What's the weather in New York?" },
{ label: "Select City to Get Weather", prompt: "Show me the city selector for weather" },
{ label: "Current Time", prompt: "What time is it?" },
{ label: "Analyze an Image", prompt: "I'll upload an image for you to analyze" },
],
},
composer: {
placeholder: "Ask about weather or upload an image...",
attachments: {
enabled: true,
accept: { "image/*": [".png", ".jpg", ".jpeg", ".gif", ".webp"] },
},
},
});
return <ChatKit control={chatkit.control} style={{ height: "100%" }} />;
}
@@ -1,15 +0,0 @@
import { StrictMode } from "react";
import { createRoot } from "react-dom/client";
import App from "./App";
const container = document.getElementById("root");
if (!container) {
throw new Error("Root element with id 'root' not found");
}
createRoot(container).render(
<StrictMode>
<App />
</StrictMode>,
);
@@ -1 +0,0 @@
/// <reference types="vite/client" />
@@ -1,21 +0,0 @@
{
"compilerOptions": {
"target": "ES2020",
"useDefineForClassFields": true,
"lib": ["ES2020", "DOM", "DOM.Iterable"],
"module": "ESNext",
"skipLibCheck": true,
"moduleResolution": "bundler",
"allowImportingTsExtensions": true,
"resolveJsonModule": true,
"isolatedModules": true,
"noEmit": true,
"jsx": "react-jsx",
"strict": true,
"noUnusedLocals": true,
"noUnusedParameters": true,
"noFallthroughCasesInSwitch": true
},
"include": ["src"],
"references": [{ "path": "./tsconfig.node.json" }]
}
@@ -1,10 +0,0 @@
{
"compilerOptions": {
"composite": true,
"skipLibCheck": true,
"module": "ESNext",
"moduleResolution": "bundler",
"allowSyntheticDefaultImports": true
},
"include": ["vite.config.ts"]
}
@@ -1,24 +0,0 @@
import { defineConfig } from "vite";
import react from "@vitejs/plugin-react-swc";
const backendTarget = process.env.BACKEND_URL ?? "http://127.0.0.1:8001";
export default defineConfig({
plugins: [react()],
server: {
host: "0.0.0.0",
port: 5171,
proxy: {
"/chatkit": {
target: backendTarget,
changeOrigin: true,
},
},
// For production deployments, you need to add your public domains to this list
allowedHosts: [
// You can remove these examples added just to demonstrate how to configure the allowlist
".ngrok.io",
".trycloudflare.com",
],
},
});
@@ -1,348 +0,0 @@
# Copyright (c) Microsoft. All rights reserved.
"""SQLite-based store implementation for ChatKit data persistence.
This module provides a complete Store implementation using SQLite for data persistence.
It includes proper thread safety, user isolation, and follows the ChatKit Store protocol.
"""
import sqlite3
import uuid
from typing import Any
from chatkit.store import NotFoundError, Store
from chatkit.types import (
Attachment,
Page,
ThreadItem,
ThreadMetadata,
)
from pydantic import BaseModel
class ThreadData(BaseModel):
"""Model for serializing thread data to SQLite."""
thread: ThreadMetadata
class ItemData(BaseModel):
"""Model for serializing thread item data to SQLite."""
item: ThreadItem
class AttachmentData(BaseModel):
"""Model for serializing attachment data to SQLite."""
attachment: Attachment
class SQLiteStore(Store[dict[str, Any]]):
"""SQLite-based store implementation for ChatKit data.
This implementation follows the pattern from the ChatKit Python tests
and provides persistent storage for threads, messages, and attachments.
Features:
- Thread-safe SQLite connections with WAL mode
- User isolation for multi-tenant support
- Proper error handling and transaction management
- Complete Store protocol implementation
Note: This is for demonstration purposes. In production, you should
implement proper error handling, connection pooling, and migration strategies.
"""
def __init__(self, db_path: str | None = None):
self.db_path = db_path or "chatkit_demo.db" # Use file-based DB for demo
self._create_tables()
def _create_connection(self):
# Enable thread safety and WAL mode for better concurrent access
conn = sqlite3.connect(self.db_path, check_same_thread=False)
conn.execute("PRAGMA journal_mode=WAL")
return conn
def _create_tables(self):
with self._create_connection() as conn:
# Create threads table
conn.execute(
"""CREATE TABLE IF NOT EXISTS threads (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL,
created_at TEXT NOT NULL,
data TEXT NOT NULL
)"""
)
# Create items table
conn.execute(
"""CREATE TABLE IF NOT EXISTS items (
id TEXT PRIMARY KEY,
thread_id TEXT NOT NULL,
user_id TEXT NOT NULL,
created_at TEXT NOT NULL,
data TEXT NOT NULL
)"""
)
# Create attachments table
conn.execute(
"""CREATE TABLE IF NOT EXISTS attachments (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL,
data TEXT NOT NULL
)"""
)
conn.commit()
def generate_thread_id(self, context: dict[str, Any]) -> str:
return f"thr_{uuid.uuid4().hex[:8]}"
def generate_item_id(
self,
item_type: str,
thread: ThreadMetadata,
context: dict[str, Any],
) -> str:
prefix_map = {
"message": "msg",
"tool_call": "tc",
"task": "tsk",
"workflow": "wf",
"attachment": "atc",
}
prefix = prefix_map.get(item_type, "itm")
return f"{prefix}_{uuid.uuid4().hex[:8]}"
async def load_thread(self, thread_id: str, context: dict[str, Any]) -> ThreadMetadata:
user_id = context.get("user_id", "demo_user")
with self._create_connection() as conn:
cursor = conn.execute(
"SELECT data FROM threads WHERE id = ? AND user_id = ?",
(thread_id, user_id),
).fetchone()
if cursor is None:
raise NotFoundError(f"Thread {thread_id} not found")
thread_data = ThreadData.model_validate_json(cursor[0])
return thread_data.thread
async def save_thread(self, thread: ThreadMetadata, context: dict[str, Any]) -> None:
user_id = context.get("user_id", "demo_user")
with self._create_connection() as conn:
thread_data = ThreadData(thread=thread)
# Replace existing thread data
conn.execute(
"DELETE FROM threads WHERE id = ? AND user_id = ?",
(thread.id, user_id),
)
conn.execute(
"INSERT INTO threads (id, user_id, created_at, data) VALUES (?, ?, ?, ?)",
(
thread.id,
user_id,
thread.created_at.isoformat(),
thread_data.model_dump_json(),
),
)
conn.commit()
async def load_thread_items(
self,
thread_id: str,
after: str | None,
limit: int,
order: str,
context: dict[str, Any],
) -> Page[ThreadItem]:
user_id = context.get("user_id", "demo_user")
with self._create_connection() as conn:
created_after: str | None = None
if after:
after_cursor = conn.execute(
"SELECT created_at FROM items WHERE id = ? AND user_id = ?",
(after, user_id),
).fetchone()
if after_cursor is None:
raise NotFoundError(f"Item {after} not found")
created_after = after_cursor[0]
query = """
SELECT data FROM items
WHERE thread_id = ? AND user_id = ?
"""
params: list[Any] = [thread_id, user_id]
if created_after:
query += " AND created_at > ?" if order == "asc" else " AND created_at < ?"
params.append(created_after)
query += f" ORDER BY created_at {order} LIMIT ?"
params.append(limit + 1)
items_cursor = conn.execute(query, params).fetchall()
items = [ItemData.model_validate_json(row[0]).item for row in items_cursor]
has_more = len(items) > limit
if has_more:
items = items[:limit]
return Page[ThreadItem](data=items, has_more=has_more, after=items[-1].id if items else None)
async def save_attachment(self, attachment: Attachment, context: dict[str, Any]) -> None:
user_id = context.get("user_id", "demo_user")
with self._create_connection() as conn:
attachment_data = AttachmentData(attachment=attachment)
conn.execute(
"INSERT OR REPLACE INTO attachments (id, user_id, data) VALUES (?, ?, ?)",
(
attachment.id,
user_id,
attachment_data.model_dump_json(),
),
)
conn.commit()
async def load_attachment(self, attachment_id: str, context: dict[str, Any]) -> Attachment:
user_id = context.get("user_id", "demo_user")
with self._create_connection() as conn:
cursor = conn.execute(
"SELECT data FROM attachments WHERE id = ? AND user_id = ?",
(attachment_id, user_id),
).fetchone()
if cursor is None:
raise NotFoundError(f"Attachment {attachment_id} not found")
attachment_data = AttachmentData.model_validate_json(cursor[0])
return attachment_data.attachment
async def delete_attachment(self, attachment_id: str, context: dict[str, Any]) -> None:
user_id = context.get("user_id", "demo_user")
with self._create_connection() as conn:
conn.execute(
"DELETE FROM attachments WHERE id = ? AND user_id = ?",
(attachment_id, user_id),
)
conn.commit()
async def load_threads(
self,
limit: int,
after: str | None,
order: str,
context: dict[str, Any],
) -> Page[ThreadMetadata]:
user_id = context.get("user_id", "demo_user")
with self._create_connection() as conn:
created_after: str | None = None
if after:
after_cursor = conn.execute(
"SELECT created_at FROM threads WHERE id = ? AND user_id = ?",
(after, user_id),
).fetchone()
if after_cursor is None:
raise NotFoundError(f"Thread {after} not found")
created_after = after_cursor[0]
query = "SELECT data FROM threads WHERE user_id = ?"
params: list[Any] = [user_id]
if created_after:
query += " AND created_at > ?" if order == "asc" else " AND created_at < ?"
params.append(created_after)
query += f" ORDER BY created_at {order} LIMIT ?"
params.append(limit + 1)
threads_cursor = conn.execute(query, params).fetchall()
threads = [ThreadData.model_validate_json(row[0]).thread for row in threads_cursor]
has_more = len(threads) > limit
if has_more:
threads = threads[:limit]
return Page[ThreadMetadata](data=threads, has_more=has_more, after=threads[-1].id if threads else None)
async def add_thread_item(self, thread_id: str, item: ThreadItem, context: dict[str, Any]) -> None:
user_id = context.get("user_id", "demo_user")
with self._create_connection() as conn:
item_data = ItemData(item=item)
conn.execute(
"INSERT INTO items (id, thread_id, user_id, created_at, data) VALUES (?, ?, ?, ?, ?)",
(
item.id,
thread_id,
user_id,
item.created_at.isoformat(),
item_data.model_dump_json(),
),
)
conn.commit()
async def save_item(self, thread_id: str, item: ThreadItem, context: dict[str, Any]) -> None:
user_id = context.get("user_id", "demo_user")
with self._create_connection() as conn:
item_data = ItemData(item=item)
conn.execute(
"UPDATE items SET data = ? WHERE id = ? AND thread_id = ? AND user_id = ?",
(
item_data.model_dump_json(),
item.id,
thread_id,
user_id,
),
)
conn.commit()
async def load_item(self, thread_id: str, item_id: str, context: dict[str, Any]) -> ThreadItem:
user_id = context.get("user_id", "demo_user")
with self._create_connection() as conn:
cursor = conn.execute(
"SELECT data FROM items WHERE id = ? AND thread_id = ? AND user_id = ?",
(item_id, thread_id, user_id),
).fetchone()
if cursor is None:
raise NotFoundError(f"Item {item_id} not found in thread {thread_id}")
item_data = ItemData.model_validate_json(cursor[0])
return item_data.item
async def delete_thread(self, thread_id: str, context: dict[str, Any]) -> None:
user_id = context.get("user_id", "demo_user")
with self._create_connection() as conn:
conn.execute(
"DELETE FROM threads WHERE id = ? AND user_id = ?",
(thread_id, user_id),
)
conn.execute(
"DELETE FROM items WHERE thread_id = ? AND user_id = ?",
(thread_id, user_id),
)
conn.commit()
async def delete_thread_item(self, thread_id: str, item_id: str, context: dict[str, Any]) -> None:
user_id = context.get("user_id", "demo_user")
with self._create_connection() as conn:
conn.execute(
"DELETE FROM items WHERE id = ? AND thread_id = ? AND user_id = ?",
(item_id, thread_id, user_id),
)
conn.commit()
@@ -1,436 +0,0 @@
# Copyright (c) Microsoft. All rights reserved.
"""Weather widget rendering for ChatKit integration sample."""
import base64
from dataclasses import dataclass
from chatkit.actions import ActionConfig
from chatkit.widgets import Box, Button, Card, Col, Image, Row, Text, Title, WidgetRoot
WEATHER_ICON_COLOR = "#1D4ED8"
WEATHER_ICON_ACCENT = "#DBEAFE"
# Popular cities for the selector
POPULAR_CITIES = [
{"value": "seattle", "label": "Seattle, WA", "description": "Pacific Northwest"},
{"value": "new_york", "label": "New York, NY", "description": "East Coast"},
{"value": "san_francisco", "label": "San Francisco, CA", "description": "Bay Area"},
{"value": "chicago", "label": "Chicago, IL", "description": "Midwest"},
{"value": "miami", "label": "Miami, FL", "description": "Southeast"},
{"value": "austin", "label": "Austin, TX", "description": "Southwest"},
{"value": "boston", "label": "Boston, MA", "description": "New England"},
{"value": "denver", "label": "Denver, CO", "description": "Mountain West"},
{"value": "portland", "label": "Portland, OR", "description": "Pacific Northwest"},
{"value": "atlanta", "label": "Atlanta, GA", "description": "Southeast"},
]
# Mapping from city values to display names for weather queries
CITY_VALUE_TO_NAME = {city["value"]: city["label"] for city in POPULAR_CITIES}
def _sun_svg() -> str:
"""Generate SVG for sunny weather icon."""
color = WEATHER_ICON_COLOR
accent = WEATHER_ICON_ACCENT
return (
'<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64" fill="none">'
f'<circle cx="32" cy="32" r="13" fill="{accent}" stroke="{color}" stroke-width="3"/>'
f'<g stroke="{color}" stroke-width="3" stroke-linecap="round">'
'<line x1="32" y1="8" x2="32" y2="16"/>'
'<line x1="32" y1="48" x2="32" y2="56"/>'
'<line x1="8" y1="32" x2="16" y2="32"/>'
'<line x1="48" y1="32" x2="56" y2="32"/>'
'<line x1="14.93" y1="14.93" x2="20.55" y2="20.55"/>'
'<line x1="43.45" y1="43.45" x2="49.07" y2="49.07"/>'
'<line x1="14.93" y1="49.07" x2="20.55" y2="43.45"/>'
'<line x1="43.45" y1="20.55" x2="49.07" y2="14.93"/>'
"</g>"
"</svg>"
)
def _cloud_svg() -> str:
"""Generate SVG for cloudy weather icon."""
color = WEATHER_ICON_COLOR
accent = WEATHER_ICON_ACCENT
return (
'<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64" fill="none">'
f'<path d="M22 46H44C50.075 46 55 41.075 55 35S50.075 24 44 24H42.7C41.2 16.2 34.7 10 26.5 10 18 10 11.6 16.1 11 24.3 6.5 25.6 3 29.8 3 35s4.925 11 11 11h8Z" '
f'fill="{accent}" stroke="{color}" stroke-width="3" stroke-linejoin="round"/>'
"</svg>"
)
def _rain_svg() -> str:
"""Generate SVG for rainy weather icon."""
color = WEATHER_ICON_COLOR
accent = WEATHER_ICON_ACCENT
return (
'<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64" fill="none">'
f'<path d="M22 40H44C50.075 40 55 35.075 55 29S50.075 18 44 18H42.7C41.2 10.2 34.7 4 26.5 4 18 4 11.6 10.1 11 18.3 6.5 19.6 3 23.8 3 29s4.925 11 11 11h8Z" '
f'fill="{accent}" stroke="{color}" stroke-width="3" stroke-linejoin="round"/>'
f'<g stroke="{color}" stroke-width="3" stroke-linecap="round">'
'<line x1="20" y1="48" x2="24" y2="56"/>'
'<line x1="30" y1="50" x2="34" y2="58"/>'
'<line x1="40" y1="48" x2="44" y2="56"/>'
"</g>"
"</svg>"
)
def _storm_svg() -> str:
"""Generate SVG for stormy weather icon."""
color = WEATHER_ICON_COLOR
accent = WEATHER_ICON_ACCENT
return (
'<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64" fill="none">'
f'<path d="M22 40H44C50.075 40 55 35.075 55 29S50.075 18 44 18H42.7C41.2 10.2 34.7 4 26.5 4 18 4 11.6 10.1 11 18.3 6.5 19.6 3 23.8 3 29s4.925 11 11 11h8Z" '
f'fill="{accent}" stroke="{color}" stroke-width="3" stroke-linejoin="round"/>'
f'<path d="M34 46L28 56H34L30 64L42 50H36L40 46Z" '
f'fill="{color}" stroke="{color}" stroke-width="2" stroke-linejoin="round"/>'
"</svg>"
)
def _snow_svg() -> str:
"""Generate SVG for snowy weather icon."""
color = WEATHER_ICON_COLOR
accent = WEATHER_ICON_ACCENT
return (
'<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64" fill="none">'
f'<path d="M22 40H44C50.075 40 55 35.075 55 29S50.075 18 44 18H42.7C41.2 10.2 34.7 4 26.5 4 18 4 11.6 10.1 11 18.3 6.5 19.6 3 23.8 3 29s4.925 11 11 11h8Z" '
f'fill="{accent}" stroke="{color}" stroke-width="3" stroke-linejoin="round"/>'
f'<g stroke="{color}" stroke-width="2" stroke-linecap="round">'
'<line x1="20" y1="48" x2="20" y2="56"/>'
'<line x1="17" y1="51" x2="23" y2="53"/>'
'<line x1="17" y1="53" x2="23" y2="51"/>'
'<line x1="36" y1="48" x2="36" y2="56"/>'
'<line x1="33" y1="51" x2="39" y2="53"/>'
'<line x1="33" y1="53" x2="39" y2="51"/>'
"</g>"
"</svg>"
)
def _fog_svg() -> str:
"""Generate SVG for foggy weather icon."""
color = WEATHER_ICON_COLOR
accent = WEATHER_ICON_ACCENT
return (
'<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64" fill="none">'
f'<path d="M22 40H44C50.075 40 55 35.075 55 29S50.075 18 44 18H42.7C41.2 10.2 34.7 4 26.5 4 18 4 11.6 10.1 11 18.3 6.5 19.6 3 23.8 3 29s4.925 11 11 11h8Z" '
f'fill="{accent}" stroke="{color}" stroke-width="3" stroke-linejoin="round"/>'
f'<g stroke="{color}" stroke-width="3" stroke-linecap="round">'
'<line x1="18" y1="50" x2="42" y2="50"/>'
'<line x1="24" y1="56" x2="48" y2="56"/>'
"</g>"
"</svg>"
)
def _encode_svg(svg: str) -> str:
"""Encode SVG as base64 data URI."""
encoded = base64.b64encode(svg.encode("utf-8")).decode("ascii")
return f"data:image/svg+xml;base64,{encoded}"
# Weather condition to icon mapping
WEATHER_ICONS = {
"sunny": _encode_svg(_sun_svg()),
"cloudy": _encode_svg(_cloud_svg()),
"rainy": _encode_svg(_rain_svg()),
"stormy": _encode_svg(_storm_svg()),
"snowy": _encode_svg(_snow_svg()),
"foggy": _encode_svg(_fog_svg()),
}
DEFAULT_WEATHER_ICON = _encode_svg(_cloud_svg())
@dataclass
class WeatherData:
"""Weather data container."""
location: str
condition: str
temperature: int
humidity: int
wind_speed: int
def render_weather_widget(data: WeatherData) -> WidgetRoot:
"""Render a weather widget from weather data.
Args:
data: WeatherData containing weather information
Returns:
A ChatKit WidgetRoot (Card) displaying the weather information
"""
# Get weather icon
weather_icon_src = WEATHER_ICONS.get(data.condition.lower(), DEFAULT_WEATHER_ICON)
# Build the widget
header = Box(
padding=5,
background="surface-tertiary",
children=[
Row(
justify="between",
align="center",
children=[
Col(
align="start",
gap=1,
children=[
Text(
value=data.location,
size="lg",
weight="semibold",
),
Text(
value="Current conditions",
color="tertiary",
size="xs",
),
],
),
Box(
padding=3,
radius="full",
background="blue-100",
children=[
Image(
src=weather_icon_src,
alt=data.condition,
size=28,
fit="contain",
)
],
),
],
),
Row(
align="start",
gap=4,
children=[
Title(
value=f"{data.temperature}°C",
size="lg",
weight="semibold",
),
Col(
align="start",
gap=1,
children=[
Text(
value=data.condition.title(),
color="secondary",
size="sm",
weight="medium",
),
],
),
],
),
],
)
# Details section
details = Box(
padding=5,
gap=4,
children=[
Text(value="Weather details", weight="semibold", size="sm"),
Row(
gap=3,
wrap="wrap",
children=[
_detail_chip("Humidity", f"{data.humidity}%"),
_detail_chip("Wind", f"{data.wind_speed} km/h"),
],
),
],
)
return Card(
key="weather",
padding=0,
children=[header, details],
)
def _detail_chip(label: str, value: str) -> Box:
"""Create a detail chip widget component."""
return Box(
padding=3,
radius="xl",
background="surface-tertiary",
width=150,
minWidth=150,
maxWidth=150,
minHeight=80,
maxHeight=80,
flex="0 0 auto",
children=[
Col(
align="stretch",
gap=2,
children=[
Text(value=label, size="xs", weight="medium", color="tertiary"),
Row(
justify="center",
margin={"top": 2},
children=[Text(value=value, weight="semibold", size="lg")],
),
],
)
],
)
def weather_widget_copy_text(data: WeatherData) -> str:
"""Generate plain text representation of weather data.
Args:
data: WeatherData containing weather information
Returns:
Plain text description for copy/paste functionality
"""
return (
f"Weather in {data.location}:\n"
f"• Condition: {data.condition.title()}\n"
f"• Temperature: {data.temperature}°C\n"
f"• Humidity: {data.humidity}%\n"
f"• Wind: {data.wind_speed} km/h"
)
def render_city_selector_widget() -> WidgetRoot:
"""Render an interactive city selector widget.
This widget displays popular cities as a visual selection interface.
Users can click or ask about any city to get weather information.
Returns:
A ChatKit WidgetRoot (Card) with city selection display
"""
# Create location icon SVG
location_icon = _encode_svg(
'<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64" fill="none">'
f'<path d="M32 8c-8.837 0-16 7.163-16 16 0 12 16 32 16 32s16-20 16-32c0-8.837-7.163-16-16-16z" '
f'fill="{WEATHER_ICON_ACCENT}" stroke="{WEATHER_ICON_COLOR}" stroke-width="3" stroke-linejoin="round"/>'
f'<circle cx="32" cy="24" r="6" fill="{WEATHER_ICON_COLOR}"/>'
"</svg>"
)
# Header section
header = Box(
padding=5,
background="surface-tertiary",
children=[
Row(
gap=3,
align="center",
children=[
Box(
padding=3,
radius="full",
background="blue-100",
children=[
Image(
src=location_icon,
alt="Location",
size=28,
fit="contain",
)
],
),
Col(
align="start",
gap=1,
children=[
Title(
value="Popular Cities",
size="md",
weight="semibold",
),
Text(
value="Select a city or ask about any location",
color="tertiary",
size="xs",
),
],
),
],
),
],
)
# Create city chips in a grid layout
city_chips: list[Button] = []
for city in POPULAR_CITIES:
# Create a button that sends an action to query weather for the selected city
chip = Button(
label=city["label"],
variant="outline",
size="md",
onClickAction=ActionConfig(
type="city_selected",
payload={"city_value": city["value"], "city_label": city["label"]},
handler="server", # Handle on server-side
),
)
city_chips.append(chip)
# Arrange in rows of 3
city_rows: list[Row] = []
for i in range(0, len(city_chips), 3):
row_chips: list[Button] = city_chips[i : i + 3]
city_rows.append(
Row(
gap=3,
wrap="wrap",
justify="start",
children=list(row_chips), # Convert to generic list
)
)
# Cities display section
cities_section = Box(
padding=5,
gap=3,
children=[
*city_rows,
Box(
padding=3,
radius="md",
background="blue-50",
children=[
Text(
value="💡 Click any city to get its weather, or ask about any other location!",
size="xs",
color="secondary",
),
],
),
],
)
return Card(
key="city_selector",
padding=0,
children=[header, cities_section],
)
def city_selector_copy_text() -> str:
"""Generate plain text representation of city selector.
Returns:
Plain text description for copy/paste functionality
"""
cities_list = "\n".join([f"{city['label']}" for city in POPULAR_CITIES])
return f"Popular cities (click to get weather):\n{cities_list}\n\nYou can also ask about weather in any other location!"
@@ -1,16 +0,0 @@
FROM python:3.12-slim
WORKDIR /app
COPY . user_agent/
WORKDIR /app/user_agent
RUN if [ -f requirements.txt ]; then \
pip install -r requirements.txt; \
else \
echo "No requirements.txt found"; \
fi
EXPOSE 8088
CMD ["python", "main.py"]
@@ -1,30 +0,0 @@
# Unique identifier/name for this agent
name: agent-with-hosted-mcp
# Brief description of what this agent does
description: >
An AI agent that uses Azure OpenAI with a Hosted Model Context Protocol (MCP) server.
The agent answers questions by searching Microsoft Learn documentation using MCP tools.
metadata:
# Categorization tags for organizing and discovering agents
authors:
- Microsoft Agent Framework Team
tags:
- Azure AI AgentServer
- Microsoft Agent Framework
- Model Context Protocol
- MCP
template:
name: agent-with-hosted-mcp
# The type of agent - "hosted" for HOBO, "container" for COBO
kind: hosted
protocols:
- protocol: responses
environment_variables:
- name: AZURE_OPENAI_ENDPOINT
value: ${AZURE_OPENAI_ENDPOINT}
- name: AZURE_OPENAI_CHAT_DEPLOYMENT_NAME
value: "{{chat}}"
resources:
- kind: model
id: gpt-4o-mini
name: chat
@@ -1,28 +0,0 @@
# Copyright (c) Microsoft. All rights reserved.
from agent_framework.azure import AzureOpenAIChatClient
from azure.ai.agentserver.agentframework import from_agent_framework # pyright: ignore[reportUnknownVariableType]
from azure.identity import DefaultAzureCredential
def main():
# Create MCP tool configuration as dict
mcp_tool = {
"type": "mcp",
"server_label": "Microsoft_Learn_MCP",
"server_url": "https://learn.microsoft.com/api/mcp",
}
# Create an Agent using the Azure OpenAI Chat Client with a MCP Tool that connects to Microsoft Learn MCP
agent = AzureOpenAIChatClient(credential=DefaultAzureCredential()).as_agent(
name="DocsAgent",
instructions="You are a helpful assistant that can help with microsoft documentation questions.",
tools=mcp_tool,
)
# Run the agent as a hosted agent
from_agent_framework(agent).run()
if __name__ == "__main__":
main()
@@ -1,2 +0,0 @@
azure-ai-agentserver-agentframework==1.0.0b3
agent-framework
@@ -1,16 +0,0 @@
FROM python:3.12-slim
WORKDIR /app
COPY . user_agent/
WORKDIR /app/user_agent
RUN if [ -f requirements.txt ]; then \
pip install -r requirements.txt; \
else \
echo "No requirements.txt found"; \
fi
EXPOSE 8088
CMD ["python", "main.py"]
@@ -1,33 +0,0 @@
# Unique identifier/name for this agent
name: agent-with-text-search-rag
# Brief description of what this agent does
description: >
An AI agent that uses a ContextProvider for retrieval augmented generation (RAG) capabilities.
The agent runs searches against an external knowledge base before each model invocation and
injects the results into the model context. It can answer questions about Contoso Outdoors
policies and products, including return policies, refunds, shipping options, and product care
instructions such as tent maintenance.
metadata:
# Categorization tags for organizing and discovering agents
authors:
- Microsoft Agent Framework Team
tags:
- Azure AI AgentServer
- Microsoft Agent Framework
- Retrieval-Augmented Generation
- RAG
template:
name: agent-with-text-search-rag
# The type of agent - "hosted" for HOBO, "container" for COBO
kind: hosted
protocols:
- protocol: responses
environment_variables:
- name: AZURE_OPENAI_ENDPOINT
value: ${AZURE_OPENAI_ENDPOINT}
- name: AZURE_OPENAI_CHAT_DEPLOYMENT_NAME
value: "{{chat}}"
resources:
- kind: model
id: gpt-4o-mini
name: chat
@@ -1,110 +0,0 @@
# Copyright (c) Microsoft. All rights reserved.
import json
import sys
from collections.abc import MutableSequence
from dataclasses import dataclass
from typing import Any
from agent_framework import Context, ContextProvider, Message
from agent_framework.azure import AzureOpenAIChatClient
from azure.ai.agentserver.agentframework import from_agent_framework # pyright: ignore[reportUnknownVariableType]
from azure.identity import DefaultAzureCredential
if sys.version_info >= (3, 12):
from typing import override
else:
from typing_extensions import override
@dataclass
class TextSearchResult:
source_name: str
source_link: str
text: str
class TextSearchContextProvider(ContextProvider):
"""A simple context provider that simulates text search results based on keywords in the user's message."""
def _get_most_recent_message(self, messages: Message | MutableSequence[Message]) -> Message:
"""Helper method to extract the most recent message from the input."""
if isinstance(messages, Message):
return messages
if messages:
return messages[-1]
raise ValueError("No messages provided")
@override
async def invoking(self, messages: Message | MutableSequence[Message], **kwargs: Any) -> Context:
message = self._get_most_recent_message(messages)
query = message.text.lower()
results: list[TextSearchResult] = []
if "return" in query and "refund" in query:
results.append(
TextSearchResult(
source_name="Contoso Outdoors Return Policy",
source_link="https://contoso.com/policies/returns",
text=(
"Customers may return any item within 30 days of delivery. "
"Items should be unused and include original packaging. "
"Refunds are issued to the original payment method within 5 business days of inspection."
),
)
)
if "shipping" in query:
results.append(
TextSearchResult(
source_name="Contoso Outdoors Shipping Guide",
source_link="https://contoso.com/help/shipping",
text=(
"Standard shipping is free on orders over $50 and typically arrives in 3-5 business days "
"within the continental United States. Expedited options are available at checkout."
),
)
)
if "tent" in query or "fabric" in query:
results.append(
TextSearchResult(
source_name="TrailRunner Tent Care Instructions",
source_link="https://contoso.com/manuals/trailrunner-tent",
text=(
"Clean the tent fabric with lukewarm water and a non-detergent soap. "
"Allow it to air dry completely before storage and avoid prolonged UV "
"exposure to extend the lifespan of the waterproof coating."
),
)
)
if not results:
return Context()
return Context(
messages=[
Message(
role="user", text="\n\n".join(json.dumps(result.__dict__, indent=2) for result in results)
)
]
)
def main():
# Create an Agent using the Azure OpenAI Chat Client
agent = AzureOpenAIChatClient(credential=DefaultAzureCredential()).as_agent(
name="SupportSpecialist",
instructions=(
"You are a helpful support specialist for Contoso Outdoors. "
"Answer questions using the provided context and cite the source document when available."
),
context_provider=TextSearchContextProvider(),
)
# Run the agent as a hosted agent
from_agent_framework(agent).run()
if __name__ == "__main__":
main()
@@ -1,2 +0,0 @@
azure-ai-agentserver-agentframework==1.0.0b3
agent-framework
@@ -1,16 +0,0 @@
FROM python:3.12-slim
WORKDIR /app
COPY . user_agent/
WORKDIR /app/user_agent
RUN if [ -f requirements.txt ]; then \
pip install -r requirements.txt; \
else \
echo "No requirements.txt found"; \
fi
EXPOSE 8088
CMD ["python", "main.py"]
@@ -1,28 +0,0 @@
# Unique identifier/name for this agent
name: agents-in-workflow
# Brief description of what this agent does
description: >
A workflow agent that responds to product launch strategy inquiries by concurrently leveraging insights from three specialized agents.
metadata:
# Categorization tags for organizing and discovering agents
authors:
- Microsoft Agent Framework Team
tags:
- Azure AI AgentServer
- Microsoft Agent Framework
- Workflows
template:
name: agents-in-workflow
# The type of agent - "hosted" for HOBO, "container" for COBO
kind: hosted
protocols:
- protocol: responses
environment_variables:
- name: AZURE_OPENAI_ENDPOINT
value: ${AZURE_OPENAI_ENDPOINT}
- name: AZURE_OPENAI_CHAT_DEPLOYMENT_NAME
value: "{{chat}}"
resources:
- kind: model
id: gpt-4o-mini
name: chat
@@ -1,44 +0,0 @@
# Copyright (c) Microsoft. All rights reserved.
from agent_framework.azure import AzureOpenAIChatClient
from agent_framework_orchestrations import ConcurrentBuilder
from azure.ai.agentserver.agentframework import from_agent_framework
from azure.identity import DefaultAzureCredential # pyright: ignore[reportUnknownVariableType]
def main():
# Create agents
researcher = AzureOpenAIChatClient(credential=DefaultAzureCredential()).as_agent(
instructions=(
"You're an expert market and product researcher. "
"Given a prompt, provide concise, factual insights, opportunities, and risks."
),
name="researcher",
)
marketer = AzureOpenAIChatClient(credential=DefaultAzureCredential()).as_agent(
instructions=(
"You're a creative marketing strategist. "
"Craft compelling value propositions and target messaging aligned to the prompt."
),
name="marketer",
)
legal = AzureOpenAIChatClient(credential=DefaultAzureCredential()).as_agent(
instructions=(
"You're a cautious legal/compliance reviewer. "
"Highlight constraints, disclaimers, and policy concerns based on the prompt."
),
name="legal",
)
# Build a concurrent workflow
workflow = ConcurrentBuilder(participants=[researcher, marketer, legal]).build()
# Convert the workflow to an agent
workflow_agent = workflow.as_agent()
# Run the agent as a hosted agent
from_agent_framework(workflow_agent).run()
if __name__ == "__main__":
main()
@@ -1,2 +0,0 @@
azure-ai-agentserver-agentframework==1.0.0b3
agent-framework
@@ -1,17 +0,0 @@
# OpenAI Configuration
OPENAI_API_KEY=
OPENAI_CHAT_MODEL_ID=
# Agent 365 Agentic Authentication Configuration
USE_ANONYMOUS_MODE=
CONNECTIONS__SERVICE_CONNECTION__SETTINGS__CLIENTID=
CONNECTIONS__SERVICE_CONNECTION__SETTINGS__CLIENTSECRET=
CONNECTIONS__SERVICE_CONNECTION__SETTINGS__TENANTID=
CONNECTIONS__SERVICE_CONNECTION__SETTINGS__SCOPES=
AGENTAPPLICATION__USERAUTHORIZATION__HANDLERS__AGENTIC__SETTINGS__TYPE=AgenticUserAuthorization
AGENTAPPLICATION__USERAUTHORIZATION__HANDLERS__AGENTIC__SETTINGS__SCOPES=https://graph.microsoft.com/.default
AGENTAPPLICATION__USERAUTHORIZATION__HANDLERS__AGENTIC__SETTINGS__ALTERNATEBLUEPRINTCONNECTIONNAME=https://graph.microsoft.com/.default
CONNECTIONSMAP_0_SERVICEURL=*
CONNECTIONSMAP_0_CONNECTION=SERVICE_CONNECTION
-100
View File
@@ -1,100 +0,0 @@
# Microsoft Agent Framework Python Weather Agent sample (M365 Agents SDK)
This sample demonstrates a simple Weather Forecast Agent built with the Python Microsoft Agent Framework, exposed through the Microsoft 365 Agents SDK compatible endpoints. The agent accepts natural language requests for a weather forecast and responds with a textual answer. It supports multi-turn conversations to gather required information.
## Prerequisites
- Python 3.11+
- [uv](https://github.com/astral-sh/uv) for fast dependency management
- [devtunnel](https://learn.microsoft.com/azure/developer/dev-tunnels/get-started?tabs=windows)
- [Microsoft 365 Agents Toolkit](https://github.com/OfficeDev/microsoft-365-agents-toolkit) for playground/testing
- Access to OpenAI or Azure OpenAI with a model like `gpt-4o-mini`
## Configuration
Set the following environment variables:
```bash
# Common
export PORT=3978
export USE_ANONYMOUS_MODE=True # set to false if using auth
# OpenAI
export OPENAI_API_KEY="..."
export OPENAI_CHAT_MODEL_ID="..."
```
## Installing Dependencies
From the repository root or the sample folder:
```bash
uv sync
```
## Running the Agent Locally
```bash
# Activate environment first if not already
source .venv/bin/activate # (Windows PowerShell: .venv\Scripts\Activate.ps1)
# Run the weather agent demo
python m365_agent_demo/app.py
```
The agent starts on `http://localhost:3978`. Health check: `GET /api/health`.
## QuickStart using Agents Playground
1. Install (if not already):
```bash
winget install agentsplayground
```
2. Start the Python agent locally: `python m365_agent_demo/app.py`
3. Start the playground: `agentsplayground`
4. Chat with the Weather Agent.
## QuickStart using WebChat (Azure Bot)
To test via WebChat you can provision an Azure Bot and point its messaging endpoint to your agent.
1. Create an Azure Bot (choose Client Secret auth for local tunneling).
2. Create a `.env` file in this sample folder with the following (replace placeholders):
```bash
# Authentication / Agentic configuration
USE_ANONYMOUS_MODE=False
CONNECTIONS__SERVICE_CONNECTION__SETTINGS__CLIENTID="<client-id>"
CONNECTIONS__SERVICE_CONNECTION__SETTINGS__CLIENTSECRET="<client-secret>"
CONNECTIONS__SERVICE_CONNECTION__SETTINGS__TENANTID="<tenant-id>"
CONNECTIONS__SERVICE_CONNECTION__SETTINGS__SCOPES=https://graph.microsoft.com/.default
AGENTAPPLICATION__USERAUTHORIZATION__HANDLERS__AGENTIC__SETTINGS__TYPE=AgenticUserAuthorization
AGENTAPPLICATION__USERAUTHORIZATION__HANDLERS__AGENTIC__SETTINGS__SCOPES=https://graph.microsoft.com/.default
AGENTAPPLICATION__USERAUTHORIZATION__HANDLERS__AGENTIC__SETTINGS__ALTERNATEBLUEPRINTCONNECTIONNAME=https://graph.microsoft.com/.default
```
3. Host dev tunnel:
```bash
devtunnel host -p 3978 --allow-anonymous
```
4. Set the bot Messaging endpoint to: `https://<tunnel-host>/api/messages`
5. Run your local agent: `python m365_agent_demo/app.py`
6. Use "Test in WebChat" in Azure Portal.
> Federated Credentials or Managed Identity auth types typically require deployment to Azure App Service instead of tunneling.
## Troubleshooting
- 404 on `/api/messages`: Ensure you are POSTing and using the correct tunnel URL.
- Empty responses: Check model key / quota and ensure environment variables are set.
- Auth errors when anonymous disabled: Validate MSAL config matches your Azure Bot registration.
## Further Reading
- [Microsoft 365 Agents SDK](https://learn.microsoft.com/microsoft-365/agents-sdk/)
- [Devtunnel docs](https://learn.microsoft.com/azure/developer/dev-tunnels/)
@@ -1,242 +0,0 @@
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "microsoft-agents-hosting-aiohttp",
# "microsoft-agents-hosting-core",
# "microsoft-agents-authentication-msal",
# "microsoft-agents-activity",
# "agent-framework-core",
# "aiohttp"
# ]
# ///
# Copyright (c) Microsoft. All rights reserved.
# Run with any PEP 723 compatible runner, e.g.:
# uv run samples/demos/m365-agent/m365_agent_demo/app.py
import os
from dataclasses import dataclass
from random import randint
from typing import Annotated
from agent_framework import Agent, tool
from agent_framework.openai import OpenAIChatClient
from aiohttp import web
from aiohttp.web_middlewares import middleware
from microsoft_agents.activity import load_configuration_from_env
from microsoft_agents.authentication.msal import MsalConnectionManager
from microsoft_agents.hosting.aiohttp import CloudAdapter, start_agent_process
from microsoft_agents.hosting.core import (
AgentApplication,
AuthenticationConstants,
Authorization,
ClaimsIdentity,
MemoryStorage,
TurnContext,
TurnState,
)
from pydantic import Field
"""
Demo application using Microsoft Agent 365 SDK.
This sample demonstrates how to build an AI agent using the Agent Framework,
integrating with Microsoft 365 authentication and hosting components.
The agent provides a simple weather tool and can be run in either anonymous mode
(no authentication required) or authenticated mode using MSAL and Azure AD.
Key features:
- Loads configuration from environment variables.
- Demonstrates agent creation and tool registration.
- Supports both anonymous and authenticated scenarios.
- Uses aiohttp for web hosting.
To run, set the appropriate environment variables (check .env.example file) for authentication or use
anonymous mode for local testing.
"""
@dataclass
class AppConfig:
use_anonymous_mode: bool
port: int
agents_sdk_config: dict
def load_app_config() -> AppConfig:
"""Load application configuration from environment variables.
Returns:
AppConfig: Consolidated configuration including anonymous mode flag, port, and SDK config.
"""
agents_sdk_config = load_configuration_from_env(os.environ)
use_anonymous_mode = os.environ.get("USE_ANONYMOUS_MODE", "true").lower() == "true"
port_str = os.getenv("PORT", "3978")
try:
port = int(port_str)
except ValueError:
port = 3978
return AppConfig(use_anonymous_mode=use_anonymous_mode, port=port, agents_sdk_config=agents_sdk_config)
# NOTE: approval_mode="never_require" is for sample brevity. Use "always_require" in production; see samples/getting_started/tools/function_tool_with_approval.py and samples/getting_started/tools/function_tool_with_approval_and_threads.py.
@tool(approval_mode="never_require")
def get_weather(
location: Annotated[str, Field(description="The location to get the weather for.")],
) -> str:
"""Generate a mock weather report for the provided location.
Args:
location: The geographic location name.
Returns:
str: Human-readable weather summary.
"""
conditions = ["sunny", "cloudy", "rainy", "stormy"]
return f"The weather in {location} is {conditions[randint(0, 3)]} with a high of {randint(10, 30)}°C."
def build_agent() -> Agent:
"""Create and return the chat agent instance with weather tool registered."""
return OpenAIChatClient().as_agent(
name="WeatherAgent", instructions="You are a helpful weather agent.", tools=get_weather
)
def build_connection_manager(config: AppConfig) -> MsalConnectionManager | None:
"""Build the connection manager unless running in anonymous mode.
Args:
config: Application configuration.
Returns:
MsalConnectionManager | None: Connection manager when authenticated mode is enabled.
"""
if config.use_anonymous_mode:
return None
return MsalConnectionManager(**config.agents_sdk_config)
def build_adapter(connection_manager: MsalConnectionManager | None) -> CloudAdapter:
"""Instantiate the CloudAdapter with the optional connection manager."""
return CloudAdapter(connection_manager=connection_manager)
def build_authorization(
storage: MemoryStorage, connection_manager: MsalConnectionManager | None, config: AppConfig
) -> Authorization | None:
"""Create Authorization component if not in anonymous mode.
Args:
storage: State storage backend.
connection_manager: Optional connection manager.
config: Application configuration.
Returns:
Authorization | None: Authorization component when enabled.
"""
if config.use_anonymous_mode:
return None
return Authorization(storage, connection_manager, **config.agents_sdk_config)
def build_agent_application(
storage: MemoryStorage,
adapter: CloudAdapter,
authorization: Authorization | None,
config: AppConfig,
) -> AgentApplication[TurnState]:
"""Compose and return the AgentApplication instance.
Args:
storage: Storage implementation.
adapter: CloudAdapter handling requests.
authorization: Optional authorization component.
config: App configuration.
Returns:
AgentApplication[TurnState]: Configured agent application.
"""
return AgentApplication[TurnState](
storage=storage, adapter=adapter, authorization=authorization, **config.agents_sdk_config
)
def build_anonymous_claims_middleware(use_anonymous_mode: bool):
"""Return a middleware that injects anonymous claims when enabled.
Args:
use_anonymous_mode: Whether to apply anonymous identity for each request.
Returns:
Callable: Aiohttp middleware function.
"""
@middleware
async def anonymous_claims_middleware(request, handler):
"""Inject claims for anonymous users if anonymous mode is active."""
if use_anonymous_mode:
request["claims_identity"] = ClaimsIdentity(
{
AuthenticationConstants.AUDIENCE_CLAIM: "anonymous",
AuthenticationConstants.APP_ID_CLAIM: "anonymous-app",
},
False,
"Anonymous",
)
return await handler(request)
return anonymous_claims_middleware
def create_app(config: AppConfig) -> web.Application:
"""Create and configure the aiohttp web application.
Args:
config: Loaded application configuration.
Returns:
web.Application: Fully initialized web application.
"""
middleware_fn = build_anonymous_claims_middleware(config.use_anonymous_mode)
app = web.Application(middleware=[middleware_fn])
storage = MemoryStorage()
agent = build_agent()
connection_manager = build_connection_manager(config)
adapter = build_adapter(connection_manager)
authorization = build_authorization(storage, connection_manager, config)
agent_app = build_agent_application(storage, adapter, authorization, config)
@agent_app.activity("message")
async def on_message(context: TurnContext, _: TurnState):
user_message = context.activity.text or ""
if not user_message.strip():
return
response = await agent.run(user_message)
response_text = response.text
await context.send_activity(response_text)
async def health(request: web.Request) -> web.Response:
return web.json_response({"status": "ok"})
async def entry_point(req: web.Request) -> web.Response:
return await start_agent_process(req, req.app["agent_app"], req.app["adapter"])
app.add_routes([
web.get("/api/health", health),
web.get("/api/messages", lambda _: web.Response(status=200)),
web.post("/api/messages", entry_point),
])
app["agent_app"] = agent_app
app["adapter"] = adapter
return app
def main() -> None:
"""Entry point: load configuration, build app, and start server."""
config = load_app_config()
app = create_app(config)
web.run_app(app, host="localhost", port=config.port)
if __name__ == "__main__":
main()
@@ -1,2 +0,0 @@
AZURE_AI_PROJECT_ENDPOINT="<your-project-endpoint>"
AZURE_AI_MODEL_DEPLOYMENT_NAME="<your-model-deployment>"
@@ -1,30 +0,0 @@
# Multi-Agent Travel Planning Workflow Evaluation
This sample demonstrates evaluating a multi-agent workflow using Azure AI's built-in evaluators. The workflow processes travel planning requests through seven specialized agents in a fan-out/fan-in pattern: travel request handler, hotel/flight/activity search agents, booking aggregator, booking confirmation, and payment processing.
## Evaluation Metrics
The evaluation uses four Azure AI built-in evaluators:
- **Relevance** - How well responses address the user query
- **Groundedness** - Whether responses are grounded in available context
- **Tool Call Accuracy** - Correct tool selection and parameter usage
- **Tool Output Utilization** - Effective use of tool outputs in responses
## Setup
Create a `.env` file with configuration as in the `.env.example` file in this folder.
## Running the Evaluation
Execute the complete workflow and evaluation:
```bash
python run_evaluation.py
```
The script will:
1. Execute the multi-agent travel planning workflow
2. Display response summary for each agent
3. Create and run evaluation on hotel, flight, and activity search agents
4. Monitor progress and display the evaluation report URL
@@ -1,750 +0,0 @@
# Copyright (c) Microsoft. All rights reserved.
import json
from datetime import datetime
from typing import Annotated
from agent_framework import tool
from pydantic import Field
# --- Travel Planning Tools ---
# Note: These are mock tools for demonstration purposes. They return simulated data
# and do not make real API calls or bookings.
# Mock hotel search tool
@tool(name="search_hotels", description="Search for available hotels based on location and dates.")
def search_hotels(
location: Annotated[str, Field(description="City or region to search for hotels.")],
check_in: Annotated[str, Field(description="Check-in date (e.g., 'December 15, 2025').")],
check_out: Annotated[str, Field(description="Check-out date (e.g., 'December 18, 2025').")],
guests: Annotated[int, Field(description="Number of guests.")] = 2,
) -> str:
"""Search for available hotels based on location and dates.
Returns:
JSON string containing search results with hotel details including name, rating,
price, distance to landmarks, amenities, and availability.
"""
# Specific mock data for Paris December 15-18, 2025
if "paris" in location.lower():
mock_hotels = [
{
"name": "Hotel Eiffel Trocadéro",
"rating": 4.6,
"price_per_night": "$185",
"total_price": "$555 for 3 nights",
"distance_to_eiffel_tower": "0.3 miles",
"amenities": ["WiFi", "Breakfast", "Eiffel Tower View", "Concierge"],
"availability": "Available",
"address": "35 Rue Benjamin Franklin, 16th arr., Paris"
},
{
"name": "Mercure Paris Centre Tour Eiffel",
"rating": 4.4,
"price_per_night": "$220",
"total_price": "$660 for 3 nights",
"distance_to_eiffel_tower": "0.5 miles",
"amenities": ["WiFi", "Restaurant", "Bar", "Gym", "Air Conditioning"],
"availability": "Available",
"address": "20 Rue Jean Rey, 15th arr., Paris"
},
{
"name": "Pullman Paris Tour Eiffel",
"rating": 4.7,
"price_per_night": "$280",
"total_price": "$840 for 3 nights",
"distance_to_eiffel_tower": "0.2 miles",
"amenities": ["WiFi", "Spa", "Gym", "Restaurant", "Rooftop Bar", "Concierge"],
"availability": "Limited",
"address": "18 Avenue de Suffren, 15th arr., Paris"
}
]
else:
mock_hotels = [
{
"name": "Grand Plaza Hotel",
"rating": 4.5,
"price_per_night": "$150",
"amenities": ["WiFi", "Pool", "Gym", "Restaurant"],
"availability": "Available"
}
]
return json.dumps({
"location": location,
"check_in": check_in,
"check_out": check_out,
"guests": guests,
"hotels_found": len(mock_hotels),
"hotels": mock_hotels,
"note": "Hotel search results matching your query"
})
# Mock hotel details tool
@tool(name="get_hotel_details", description="Get detailed information about a specific hotel.")
def get_hotel_details(
hotel_name: Annotated[str, Field(description="Name of the hotel to get details for.")],
) -> str:
"""Get detailed information about a specific hotel.
Returns:
JSON string containing detailed hotel information including description,
check-in/out times, cancellation policy, reviews, and nearby attractions.
"""
hotel_details = {
"Hotel Eiffel Trocadéro": {
"description": "Charming boutique hotel with stunning Eiffel Tower views from select rooms. Perfect for couples and families.",
"check_in_time": "3:00 PM",
"check_out_time": "11:00 AM",
"cancellation_policy": "Free cancellation up to 24 hours before check-in",
"reviews": {
"total": 1247,
"recent_comments": [
"Amazing location! Walked to Eiffel Tower in 5 minutes.",
"Staff was incredibly helpful with restaurant recommendations.",
"Rooms are cozy and clean with great views."
]
},
"nearby_attractions": ["Eiffel Tower (0.3 mi)", "Trocadéro Gardens (0.2 mi)", "Seine River (0.4 mi)"]
},
"Mercure Paris Centre Tour Eiffel": {
"description": "Modern hotel with contemporary rooms and excellent dining options. Close to metro stations.",
"check_in_time": "2:00 PM",
"check_out_time": "12:00 PM",
"cancellation_policy": "Free cancellation up to 48 hours before check-in",
"reviews": {
"total": 2156,
"recent_comments": [
"Great value for money, clean and comfortable.",
"Restaurant had excellent French cuisine.",
"Easy access to public transportation."
]
},
"nearby_attractions": ["Eiffel Tower (0.5 mi)", "Champ de Mars (0.4 mi)", "Les Invalides (0.8 mi)"]
},
"Pullman Paris Tour Eiffel": {
"description": "Luxury hotel offering panoramic views, upscale amenities, and exceptional service. Ideal for a premium experience.",
"check_in_time": "3:00 PM",
"check_out_time": "12:00 PM",
"cancellation_policy": "Free cancellation up to 72 hours before check-in",
"reviews": {
"total": 3421,
"recent_comments": [
"Rooftop bar has the best Eiffel Tower views in Paris!",
"Luxurious rooms with every amenity you could want.",
"Worth the price for the location and service."
]
},
"nearby_attractions": ["Eiffel Tower (0.2 mi)", "Seine River Cruise Dock (0.3 mi)", "Trocadéro (0.5 mi)"]
}
}
details = hotel_details.get(hotel_name, {
"name": hotel_name,
"description": "Comfortable hotel with modern amenities",
"check_in_time": "3:00 PM",
"check_out_time": "11:00 AM",
"cancellation_policy": "Standard cancellation policy applies",
"reviews": {"total": 0, "recent_comments": []},
"nearby_attractions": []
})
return json.dumps({
"hotel_name": hotel_name,
"details": details
})
# Mock flight search tool
@tool(name="search_flights", description="Search for available flights between two locations.")
def search_flights(
origin: Annotated[str, Field(description="Departure airport or city (e.g., 'JFK' or 'New York').")],
destination: Annotated[str, Field(description="Arrival airport or city (e.g., 'CDG' or 'Paris').")],
departure_date: Annotated[str, Field(description="Departure date (e.g., 'December 15, 2025').")],
return_date: Annotated[str | None, Field(description="Return date (e.g., 'December 18, 2025').")] = None,
passengers: Annotated[int, Field(description="Number of passengers.")] = 1,
) -> str:
"""Search for available flights between two locations.
Returns:
JSON string containing flight search results with details including flight numbers,
airlines, departure/arrival times, prices, durations, and baggage allowances.
"""
# Specific mock data for JFK to Paris December 15-18, 2025
if "jfk" in origin.lower() or "new york" in origin.lower():
if "paris" in destination.lower() or "cdg" in destination.lower():
mock_flights = [
{
"outbound": {
"flight_number": "AF007",
"airline": "Air France",
"departure": "December 15, 2025 at 6:30 PM",
"arrival": "December 16, 2025 at 8:15 AM",
"duration": "7h 45m",
"aircraft": "Boeing 777-300ER",
"class": "Economy",
"price": "$520"
},
"return": {
"flight_number": "AF008",
"airline": "Air France",
"departure": "December 18, 2025 at 11:00 AM",
"arrival": "December 18, 2025 at 2:15 PM",
"duration": "8h 15m",
"aircraft": "Airbus A350-900",
"class": "Economy",
"price": "Included"
},
"total_price": "$520",
"stops": "Nonstop",
"baggage": "1 checked bag included"
},
{
"outbound": {
"flight_number": "DL264",
"airline": "Delta",
"departure": "December 15, 2025 at 10:15 PM",
"arrival": "December 16, 2025 at 12:05 PM",
"duration": "7h 50m",
"aircraft": "Airbus A330-900neo",
"class": "Economy",
"price": "$485"
},
"return": {
"flight_number": "DL265",
"airline": "Delta",
"departure": "December 18, 2025 at 1:45 PM",
"arrival": "December 18, 2025 at 5:00 PM",
"duration": "8h 15m",
"aircraft": "Airbus A330-900neo",
"class": "Economy",
"price": "Included"
},
"total_price": "$485",
"stops": "Nonstop",
"baggage": "1 checked bag included"
},
{
"outbound": {
"flight_number": "UA57",
"airline": "United Airlines",
"departure": "December 15, 2025 at 5:00 PM",
"arrival": "December 16, 2025 at 6:50 AM",
"duration": "7h 50m",
"aircraft": "Boeing 767-400ER",
"class": "Economy",
"price": "$560"
},
"return": {
"flight_number": "UA58",
"airline": "United Airlines",
"departure": "December 18, 2025 at 9:30 AM",
"arrival": "December 18, 2025 at 12:45 PM",
"duration": "8h 15m",
"aircraft": "Boeing 787-10",
"class": "Economy",
"price": "Included"
},
"total_price": "$560",
"stops": "Nonstop",
"baggage": "1 checked bag included"
}
]
else:
mock_flights = [{"flight_number": "XX123", "airline": "Generic Air", "price": "$400", "note": "Generic route"}]
else:
mock_flights = [
{
"outbound": {
"flight_number": "AA123",
"airline": "Generic Airlines",
"departure": f"{departure_date} at 9:00 AM",
"arrival": f"{departure_date} at 2:30 PM",
"duration": "5h 30m",
"class": "Economy",
"price": "$350"
},
"total_price": "$350",
"stops": "Nonstop"
}
]
return json.dumps({
"origin": origin,
"destination": destination,
"departure_date": departure_date,
"return_date": return_date,
"passengers": passengers,
"flights_found": len(mock_flights),
"flights": mock_flights,
"note": "Flight search results for JFK to Paris CDG"
})
# Mock flight details tool
@tool(name="get_flight_details", description="Get detailed information about a specific flight.")
def get_flight_details(
flight_number: Annotated[str, Field(description="Flight number (e.g., 'AF007' or 'DL264').")],
) -> str:
"""Get detailed information about a specific flight.
Returns:
JSON string containing detailed flight information including airline, aircraft type,
departure/arrival airports and times, gates, terminals, duration, and amenities.
"""
mock_details = {
"flight_number": flight_number,
"airline": "Sky Airways",
"aircraft": "Boeing 737-800",
"departure": {
"airport": "JFK International Airport",
"terminal": "Terminal 4",
"gate": "B23",
"time": "08:00 AM"
},
"arrival": {
"airport": "Charles de Gaulle Airport",
"terminal": "Terminal 2E",
"gate": "K15",
"time": "11:30 AM local time"
},
"duration": "3h 30m",
"baggage_allowance": {
"carry_on": "1 bag (10kg)",
"checked": "1 bag (23kg)"
},
"amenities": ["WiFi", "In-flight entertainment", "Meals included"]
}
return json.dumps({
"flight_details": mock_details
})
# Mock activity search tool
@tool(name="search_activities", description="Search for available activities and attractions at a destination.")
def search_activities(
location: Annotated[str, Field(description="City or region to search for activities.")],
date: Annotated[str | None, Field(description="Date for the activity (e.g., 'December 16, 2025').")] = None,
category: Annotated[str | None, Field(description="Activity category (e.g., 'Sightseeing', 'Culture', 'Culinary').")] = None,
) -> str:
"""Search for available activities and attractions at a destination.
Returns:
JSON string containing activity search results with details including name, category,
duration, price, rating, description, availability, and booking requirements.
"""
# Specific mock data for Paris activities
if "paris" in location.lower():
all_activities = [
{
"name": "Eiffel Tower Summit Access",
"category": "Sightseeing",
"duration": "2-3 hours",
"price": "$35",
"rating": 4.8,
"description": "Skip-the-line access to all three levels including the summit. Best views of Paris!",
"availability": "Daily 9:30 AM - 11:00 PM",
"best_time": "Early morning or sunset",
"booking_required": True
},
{
"name": "Louvre Museum Guided Tour",
"category": "Sightseeing",
"duration": "3 hours",
"price": "$55",
"rating": 4.7,
"description": "Expert-guided tour covering masterpieces including Mona Lisa and Venus de Milo.",
"availability": "Daily except Tuesdays, 9:00 AM entry",
"best_time": "Morning entry recommended",
"booking_required": True
},
{
"name": "Seine River Cruise",
"category": "Sightseeing",
"duration": "1 hour",
"price": "$18",
"rating": 4.6,
"description": "Scenic cruise past Notre-Dame, Eiffel Tower, and historic bridges.",
"availability": "Every 30 minutes, 10:00 AM - 10:00 PM",
"best_time": "Evening for illuminated monuments",
"booking_required": False
},
{
"name": "Musée d'Orsay Visit",
"category": "Culture",
"duration": "2-3 hours",
"price": "$16",
"rating": 4.7,
"description": "Impressionist masterpieces in a stunning Beaux-Arts railway station.",
"availability": "Tuesday-Sunday 9:30 AM - 6:00 PM",
"best_time": "Weekday mornings",
"booking_required": True
},
{
"name": "Versailles Palace Day Trip",
"category": "Culture",
"duration": "5-6 hours",
"price": "$75",
"rating": 4.9,
"description": "Explore the opulent palace and stunning gardens of Louis XIV (includes transport).",
"availability": "Daily except Mondays, 8:00 AM departure",
"best_time": "Full day trip",
"booking_required": True
},
{
"name": "Montmartre Walking Tour",
"category": "Culture",
"duration": "2.5 hours",
"price": "$25",
"rating": 4.6,
"description": "Discover the artistic heart of Paris, including Sacré-Cœur and artists' square.",
"availability": "Daily at 10:00 AM and 2:00 PM",
"best_time": "Morning or late afternoon",
"booking_required": False
},
{
"name": "French Cooking Class",
"category": "Culinary",
"duration": "3 hours",
"price": "$120",
"rating": 4.9,
"description": "Learn to make classic French dishes like coq au vin and crème brûlée, then enjoy your creations.",
"availability": "Tuesday-Saturday, 10:00 AM and 6:00 PM sessions",
"best_time": "Morning or evening sessions",
"booking_required": True
},
{
"name": "Wine & Cheese Tasting",
"category": "Culinary",
"duration": "1.5 hours",
"price": "$65",
"rating": 4.7,
"description": "Sample French wines and artisanal cheeses with expert sommelier guidance.",
"availability": "Daily at 5:00 PM and 7:30 PM",
"best_time": "Evening sessions",
"booking_required": True
},
{
"name": "Food Market Tour",
"category": "Culinary",
"duration": "2 hours",
"price": "$45",
"rating": 4.6,
"description": "Explore authentic Parisian markets and taste local specialties like cheeses, pastries, and charcuterie.",
"availability": "Tuesday, Thursday, Saturday mornings",
"best_time": "Morning (markets are freshest)",
"booking_required": False
}
]
activities = [act for act in all_activities if act["category"] == category] if category else all_activities
else:
activities = [
{
"name": "City Walking Tour",
"category": "Sightseeing",
"duration": "3 hours",
"price": "$45",
"rating": 4.7,
"description": "Explore the historic downtown area with an expert guide",
"availability": "Daily at 10:00 AM and 2:00 PM"
}
]
return json.dumps({
"location": location,
"date": date,
"category": category,
"activities_found": len(activities),
"activities": activities,
"note": "Activity search results for Paris with sightseeing, culture, and culinary options"
})
# Mock activity details tool
@tool(name="get_activity_details", description="Get detailed information about a specific activity.")
def get_activity_details(
activity_name: Annotated[str, Field(description="Name of the activity to get details for.")],
) -> str:
"""Get detailed information about a specific activity.
Returns:
JSON string containing detailed activity information including description, duration,
price, included items, meeting point, what to bring, cancellation policy, and reviews.
"""
# Paris-specific activity details
activity_details_map = {
"Eiffel Tower Summit Access": {
"name": "Eiffel Tower Summit Access",
"description": "Skip-the-line access to all three levels of the Eiffel Tower, including the summit. Enjoy panoramic views of Paris from 276 meters high.",
"duration": "2-3 hours (self-guided)",
"price": "$35 per person",
"included": ["Skip-the-line ticket", "Access to all 3 levels", "Summit access", "Audio guide app"],
"meeting_point": "Eiffel Tower South Pillar entrance, look for priority access line",
"what_to_bring": ["Photo ID", "Comfortable shoes", "Camera", "Light jacket (summit can be windy)"],
"cancellation_policy": "Free cancellation up to 24 hours in advance",
"languages": ["English", "French", "Spanish", "German", "Italian"],
"max_group_size": "No limit",
"rating": 4.8,
"reviews_count": 15234
},
"Louvre Museum Guided Tour": {
"name": "Louvre Museum Guided Tour",
"description": "Expert-guided tour of the world's largest art museum, focusing on must-see masterpieces including Mona Lisa, Venus de Milo, and Winged Victory.",
"duration": "3 hours",
"price": "$55 per person",
"included": ["Skip-the-line entry", "Expert art historian guide", "Headsets for groups over 6", "Museum highlights map"],
"meeting_point": "Glass Pyramid main entrance, look for guide with 'Louvre Tours' sign",
"what_to_bring": ["Photo ID", "Comfortable shoes", "Camera (no flash)", "Water bottle"],
"cancellation_policy": "Free cancellation up to 48 hours in advance",
"languages": ["English", "French", "Spanish"],
"max_group_size": 20,
"rating": 4.7,
"reviews_count": 8921
},
"French Cooking Class": {
"name": "French Cooking Class",
"description": "Hands-on cooking experience where you'll learn to prepare classic French dishes like coq au vin, ratatouille, and crème brûlée under expert chef guidance.",
"duration": "3 hours",
"price": "$120 per person",
"included": ["All ingredients", "Chef instruction", "Apron and recipe booklet", "Wine pairing", "Lunch/dinner of your creations"],
"meeting_point": "Le Chef Cooking Studio, 15 Rue du Bac, 7th arrondissement",
"what_to_bring": ["Appetite", "Camera for food photos"],
"cancellation_policy": "Free cancellation up to 72 hours in advance",
"languages": ["English", "French"],
"max_group_size": 12,
"rating": 4.9,
"reviews_count": 2341
}
}
details = activity_details_map.get(activity_name, {
"name": activity_name,
"description": "An immersive experience that showcases the best of local culture and attractions.",
"duration": "3 hours",
"price": "$45 per person",
"included": ["Professional guide", "Entry fees"],
"meeting_point": "Central meeting location",
"what_to_bring": ["Comfortable shoes", "Camera"],
"cancellation_policy": "Free cancellation up to 24 hours in advance",
"languages": ["English"],
"max_group_size": 15,
"rating": 4.5,
"reviews_count": 100
})
return json.dumps({
"activity_details": details
})
# Mock booking confirmation tool
@tool(name="confirm_booking", description="Confirm a booking reservation.")
def confirm_booking(
booking_type: Annotated[str, Field(description="Type of booking (e.g., 'hotel', 'flight', 'activity').")],
booking_id: Annotated[str, Field(description="Unique booking identifier.")],
customer_info: Annotated[dict, Field(description="Customer information including name and email.")],
) -> str:
"""Confirm a booking reservation.
Returns:
JSON string containing confirmation details including confirmation number,
booking status, customer information, and next steps.
"""
confirmation_number = f"CONF-{booking_type.upper()}-{booking_id}"
confirmation_data = {
"confirmation_number": confirmation_number,
"booking_type": booking_type,
"status": "Confirmed",
"customer_name": customer_info.get("name", "Guest"),
"email": customer_info.get("email", "guest@example.com"),
"confirmation_sent": True,
"next_steps": [
"Check your email for booking details",
"Arrive 30 minutes before scheduled time",
"Bring confirmation number and valid ID"
]
}
return json.dumps({
"confirmation": confirmation_data
})
# Mock hotel availability check tool
@tool(name="check_hotel_availability", description="Check availability for hotel rooms.")
def check_hotel_availability(
hotel_name: Annotated[str, Field(description="Name of the hotel to check availability for.")],
check_in: Annotated[str, Field(description="Check-in date (e.g., 'December 15, 2025').")],
check_out: Annotated[str, Field(description="Check-out date (e.g., 'December 18, 2025').")],
rooms: Annotated[int, Field(description="Number of rooms needed.")] = 1,
) -> str:
"""Check availability for hotel rooms.
Sample Date format: "December 15, 2025"
Returns:
JSON string containing availability status, available rooms count, price per night,
and last checked timestamp.
"""
availability_status = "Available"
availability_data = {
"service_type": "hotel",
"hotel_name": hotel_name,
"check_in": check_in,
"check_out": check_out,
"rooms_requested": rooms,
"status": availability_status,
"available_rooms": 8,
"price_per_night": "$185",
"last_checked": datetime.now().isoformat()
}
return json.dumps({
"availability": availability_data
})
# Mock flight availability check tool
@tool(name="check_flight_availability", description="Check availability for flight seats.")
def check_flight_availability(
flight_number: Annotated[str, Field(description="Flight number to check availability for.")],
date: Annotated[str, Field(description="Flight date (e.g., 'December 15, 2025').")],
passengers: Annotated[int, Field(description="Number of passengers.")] = 1,
) -> str:
"""Check availability for flight seats.
Sample Date format: "December 15, 2025"
Returns:
JSON string containing availability status, available seats count, price per passenger,
and last checked timestamp.
"""
availability_status = "Available"
availability_data = {
"service_type": "flight",
"flight_number": flight_number,
"date": date,
"passengers_requested": passengers,
"status": availability_status,
"available_seats": 45,
"price_per_passenger": "$520",
"last_checked": datetime.now().isoformat()
}
return json.dumps({
"availability": availability_data
})
# Mock activity availability check tool
@tool(name="check_activity_availability", description="Check availability for activity bookings.")
def check_activity_availability(
activity_name: Annotated[str, Field(description="Name of the activity to check availability for.")],
date: Annotated[str, Field(description="Activity date (e.g., 'December 16, 2025').")],
participants: Annotated[int, Field(description="Number of participants.")] = 1,
) -> str:
"""Check availability for activity bookings.
Sample Date format: "December 16, 2025"
Returns:
JSON string containing availability status, available spots count, price per person,
and last checked timestamp.
"""
availability_status = "Available"
availability_data = {
"service_type": "activity",
"activity_name": activity_name,
"date": date,
"participants_requested": participants,
"status": availability_status,
"available_spots": 15,
"price_per_person": "$45",
"last_checked": datetime.now().isoformat()
}
return json.dumps({
"availability": availability_data
})
# Mock payment processing tool
@tool(name="process_payment", description="Process payment for a booking.")
def process_payment(
amount: Annotated[float, Field(description="Payment amount.")],
currency: Annotated[str, Field(description="Currency code (e.g., 'USD', 'EUR').")],
payment_method: Annotated[dict, Field(description="Payment method details (type, card info).")],
booking_reference: Annotated[str, Field(description="Booking reference number for the payment.")],
) -> str:
"""Process payment for a booking.
Returns:
JSON string containing payment result with transaction ID, status, amount, currency,
payment method details, and receipt URL.
"""
transaction_id = f"TXN-{datetime.now().strftime('%Y%m%d%H%M%S')}"
payment_result = {
"transaction_id": transaction_id,
"amount": amount,
"currency": currency,
"status": "Success",
"payment_method": payment_method.get("type", "Credit Card"),
"last_4_digits": payment_method.get("last_4", "****"),
"booking_reference": booking_reference,
"timestamp": datetime.now().isoformat(),
"receipt_url": f"https://payments.travelagency.com/receipt/{transaction_id}"
}
return json.dumps({
"payment_result": payment_result
})
# Mock payment validation tool
@tool(name="validate_payment_method", description="Validate a payment method before processing.")
def validate_payment_method(
payment_method: Annotated[dict, Field(description="Payment method to validate (type, number, expiry, cvv).")],
) -> str:
"""Validate payment method details.
Returns:
JSON string containing validation result with is_valid flag, payment method type,
validation messages, supported currencies, and processing fee information.
"""
method_type = payment_method.get("type", "credit_card")
# Validation logic
is_valid = True
validation_messages = []
if method_type == "credit_card":
if not payment_method.get("number"):
is_valid = False
validation_messages.append("Card number is required")
if not payment_method.get("expiry"):
is_valid = False
validation_messages.append("Expiry date is required")
if not payment_method.get("cvv"):
is_valid = False
validation_messages.append("CVV is required")
validation_result = {
"is_valid": is_valid,
"payment_method_type": method_type,
"validation_messages": validation_messages if not is_valid else ["Payment method is valid"],
"supported_currencies": ["USD", "EUR", "GBP", "JPY"],
"processing_fee": "2.5%"
}
return json.dumps({
"validation_result": validation_result
})
@@ -1,445 +0,0 @@
# Copyright (c) Microsoft. All rights reserved.
"""
Multi-Agent Travel Planning Workflow Evaluation with Multiple Response Tracking
This sample demonstrates a multi-agent travel planning workflow using the Azure AI Client that:
1. Processes travel queries through 7 specialized agents
2. Tracks MULTIPLE response and conversation IDs per agent for evaluation
3. Uses the new Prompt Agents API (V2)
4. Captures complete interaction sequences including multiple invocations
5. Aggregates findings through a travel planning coordinator
WORKFLOW STRUCTURE (7 agents):
- Travel Agent Executor → Hotel Search, Flight Search, Activity Search (fan-out)
- Hotel Search Executor → Booking Information Aggregation Executor
- Flight Search Executor → Booking Information Aggregation Executor
- Booking Information Aggregation Executor → Booking Confirmation Executor
- Booking Confirmation Executor → Booking Payment Executor
- Booking Information Aggregation, Booking Payment, Activity Search → Travel Planning Coordinator (ResearchLead) for final aggregation (fan-in)
Agents:
1. Travel Agent - Main coordinator (no tools to avoid thread conflicts)
2. Hotel Search - Searches hotels with tools
3. Flight Search - Searches flights with tools
4. Activity Search - Searches activities with tools
5. Booking Information Aggregation - Aggregates hotel & flight booking info
6. Booking Confirmation - Confirms bookings with tools
7. Booking Payment - Processes payments with tools
"""
import asyncio
import os
from collections import defaultdict
from _tools import (
check_flight_availability,
check_hotel_availability,
confirm_booking,
get_flight_details,
get_hotel_details,
process_payment,
search_activities,
search_flights,
# Travel planning tools
search_hotels,
validate_payment_method,
)
from agent_framework import (
AgentExecutorResponse,
AgentResponseUpdate,
Executor,
Message,
WorkflowBuilder,
WorkflowContext,
executor,
handler,
)
from agent_framework.azure import AzureAIClient
from azure.ai.projects.aio import AIProjectClient
from azure.identity.aio import DefaultAzureCredential
from dotenv import load_dotenv
from typing_extensions import Never
load_dotenv()
@executor(id="start_executor")
async def start_executor(input: str, ctx: WorkflowContext[list[Message]]) -> None:
"""Initiates the workflow by sending the user query to all specialized agents."""
await ctx.send_message([Message("user", [input])])
class ResearchLead(Executor):
"""Aggregates and summarizes travel planning findings from all specialized agents."""
def __init__(self, client: AzureAIClient, id: str = "travel-planning-coordinator"):
# store=True to preserve conversation history for evaluation
self.agent = client.as_agent(
id="travel-planning-coordinator",
instructions=(
"You are the final coordinator. You will receive responses from multiple agents: "
"booking-info-aggregation-agent (hotel/flight options), booking-payment-agent (payment confirmation), "
"and activity-search-agent (activities). "
"Review each agent's response, then create a comprehensive travel itinerary organized by: "
"1. Flights 2. Hotels 3. Activities 4. Booking confirmations 5. Payment details. "
"Clearly indicate which information came from which agent. Do not use tools."
),
name="travel-planning-coordinator",
store=True,
)
super().__init__(id=id)
@handler
async def fan_in_handle(self, responses: list[AgentExecutorResponse], ctx: WorkflowContext[Never, str]) -> None:
user_query = responses[0].full_conversation[0].text
# Extract findings from all agent responses
agent_findings = self._extract_agent_findings(responses)
summary_text = (
"\n".join(agent_findings) if agent_findings else "No specific findings were provided by the agents."
)
# Generate comprehensive travel plan summary
messages = [
Message(
role="system",
text="You are a travel planning coordinator. Summarize findings from multiple specialized travel agents and provide a clear, comprehensive travel plan based on the user's query.",
),
Message(
role="user",
text=f"Original query: {user_query}\n\nFindings from specialized travel agents:\n{summary_text}\n\nPlease provide a comprehensive travel plan based on these findings.",
),
]
try:
final_response = await self.agent.run(messages)
output_text = (
final_response.messages[-1].text
if final_response.messages and final_response.messages[-1].text
else f"Based on the available findings, here's your travel plan for '{user_query}': {summary_text}"
)
except Exception:
output_text = f"Based on the available findings, here's your travel plan for '{user_query}': {summary_text}"
await ctx.yield_output(output_text)
def _extract_agent_findings(self, responses: list[AgentExecutorResponse]) -> list[str]:
"""Extract findings from agent responses."""
agent_findings = []
for response in responses:
findings = []
if response.agent_response and response.agent_response.messages:
for msg in response.agent_response.messages:
if msg.role == "assistant" and msg.text and msg.text.strip():
findings.append(msg.text.strip())
if findings:
combined_findings = " ".join(findings)
agent_findings.append(f"[{response.executor_id}]: {combined_findings}")
return agent_findings
async def run_workflow_with_response_tracking(query: str, client: AzureAIClient | None = None) -> dict:
"""Run multi-agent workflow and track conversation IDs, response IDs, and interaction sequence.
Args:
query: The user query to process through the multi-agent workflow
client: Optional AzureAIClient instance
Returns:
Dictionary containing interaction sequence, conversation/response IDs, and conversation analysis
"""
if client is None:
try:
async with DefaultAzureCredential() as credential:
# Create AIProjectClient with the correct API version for V2 prompt agents
project_client = AIProjectClient(
endpoint=os.environ["AZURE_AI_PROJECT_ENDPOINT"],
credential=credential,
api_version="2025-11-15-preview",
)
async with (
project_client,
AzureAIClient(project_client=project_client, credential=credential) as client,
):
return await _run_workflow_with_client(query, client)
except Exception as e:
print(f"Error during workflow execution: {e}")
raise
else:
return await _run_workflow_with_client(query, client)
async def _run_workflow_with_client(query: str, client: AzureAIClient) -> dict:
"""Execute workflow with given client and track all interactions."""
# Initialize tracking variables - use lists to track multiple responses per agent
conversation_ids = defaultdict(list)
response_ids = defaultdict(list)
workflow_output = None
# Create workflow components and keep agent references
# Pass project_client and credential to create separate client instances per agent
workflow, agent_map = await _create_workflow(client.project_client, client.credential)
# Process workflow events
events = workflow.run(query, stream=True)
workflow_output = await _process_workflow_events(events, conversation_ids, response_ids)
return {
"conversation_ids": dict(conversation_ids),
"response_ids": dict(response_ids),
"output": workflow_output,
"query": query,
}
async def _create_workflow(project_client, credential):
"""Create the multi-agent travel planning workflow with specialized agents.
IMPORTANT: Each agent needs its own client instance because the V2 client stores
agent_name and agent_version as instance variables, causing all agents to share
the same agent identity if they share a client.
"""
# Create separate client for Final Coordinator
final_coordinator_client = AzureAIClient(
project_client=project_client, credential=credential, agent_name="final-coordinator"
)
final_coordinator = ResearchLead(client=final_coordinator_client, id="final-coordinator")
# Agent 1: Travel Request Handler (initial coordinator)
# Create separate client with unique agent_name
travel_request_handler_client = AzureAIClient(
project_client=project_client, credential=credential, agent_name="travel-request-handler"
)
travel_request_handler = travel_request_handler_client.as_agent(
id="travel-request-handler",
instructions=(
"You receive user travel queries and relay them to specialized agents. Extract key information: destination, dates, budget, and preferences. Pass this information forward clearly to the next agents."
),
name="travel-request-handler",
store=True,
)
# Agent 2: Hotel Search Executor
hotel_search_client = AzureAIClient(
project_client=project_client, credential=credential, agent_name="hotel-search-agent"
)
hotel_search_agent = hotel_search_client.as_agent(
id="hotel-search-agent",
instructions=(
"You are a hotel search specialist. Your task is ONLY to search for and provide hotel information. Use search_hotels to find options, get_hotel_details for specifics, and check_availability to verify rooms. Output format: List hotel names, prices per night, total cost for the stay, locations, ratings, amenities, and addresses. IMPORTANT: Only provide hotel information without additional commentary."
),
name="hotel-search-agent",
tools=[search_hotels, get_hotel_details, check_hotel_availability],
store=True,
)
# Agent 3: Flight Search Executor
flight_search_client = AzureAIClient(
project_client=project_client, credential=credential, agent_name="flight-search-agent"
)
flight_search_agent = flight_search_client.as_agent(
id="flight-search-agent",
instructions=(
"You are a flight search specialist. Your task is ONLY to search for and provide flight information. Use search_flights to find options, get_flight_details for specifics, and check_availability for seats. Output format: List flight numbers, airlines, departure/arrival times, prices, durations, and cabin class. IMPORTANT: Only provide flight information without additional commentary."
),
name="flight-search-agent",
tools=[search_flights, get_flight_details, check_flight_availability],
store=True,
)
# Agent 4: Activity Search Executor
activity_search_client = AzureAIClient(
project_client=project_client, credential=credential, agent_name="activity-search-agent"
)
activity_search_agent = activity_search_client.as_agent(
id="activity-search-agent",
instructions=(
"You are an activities specialist. Your task is ONLY to search for and provide activity information. Use search_activities to find options for activities. Output format: List activity names, descriptions, prices, durations, ratings, and categories. IMPORTANT: Only provide activity information without additional commentary."
),
name="activity-search-agent",
tools=[search_activities],
store=True,
)
# Agent 5: Booking Confirmation Executor
booking_confirmation_client = AzureAIClient(
project_client=project_client, credential=credential, agent_name="booking-confirmation-agent"
)
booking_confirmation_agent = booking_confirmation_client.as_agent(
id="booking-confirmation-agent",
instructions=(
"You confirm bookings. Use check_hotel_availability and check_flight_availability to verify slots, then confirm_booking to finalize. Provide ONLY: confirmation numbers, booking references, and confirmation status."
),
name="booking-confirmation-agent",
tools=[confirm_booking, check_hotel_availability, check_flight_availability],
store=True,
)
# Agent 6: Booking Payment Executor
booking_payment_client = AzureAIClient(
project_client=project_client, credential=credential, agent_name="booking-payment-agent"
)
booking_payment_agent = booking_payment_client.as_agent(
id="booking-payment-agent",
instructions=(
"You process payments. Use validate_payment_method to verify payment, then process_payment to complete transactions. Provide ONLY: payment confirmation status, transaction IDs, and payment amounts."
),
name="booking-payment-agent",
tools=[process_payment, validate_payment_method],
store=True,
)
# Agent 7: Booking Information Aggregation Executor
booking_info_client = AzureAIClient(
project_client=project_client, credential=credential, agent_name="booking-info-aggregation-agent"
)
booking_info_aggregation_agent = booking_info_client.as_agent(
id="booking-info-aggregation-agent",
instructions=(
"You aggregate hotel and flight search results. Receive options from search agents and organize them. Provide: top 2-3 hotel options with prices and top 2-3 flight options with prices in a structured format."
),
name="booking-info-aggregation-agent",
store=True,
)
# Build workflow with logical booking flow:
# 1. start_executor → travel_request_handler
# 2. travel_request_handler → hotel_search, flight_search, activity_search (fan-out)
# 3. hotel_search → booking_info_aggregation
# 4. flight_search → booking_info_aggregation
# 5. booking_info_aggregation → booking_confirmation
# 6. booking_confirmation → booking_payment
# 7. booking_info_aggregation, booking_payment, activity_search → final_coordinator (final aggregation, fan-in)
workflow = (
WorkflowBuilder(name="Travel Planning Workflow", start_executor=start_executor)
.add_edge(start_executor, travel_request_handler)
.add_fan_out_edges(travel_request_handler, [hotel_search_agent, flight_search_agent, activity_search_agent])
.add_edge(hotel_search_agent, booking_info_aggregation_agent)
.add_edge(flight_search_agent, booking_info_aggregation_agent)
.add_edge(booking_info_aggregation_agent, booking_confirmation_agent)
.add_edge(booking_confirmation_agent, booking_payment_agent)
.add_fan_in_edges(
[booking_info_aggregation_agent, booking_payment_agent, activity_search_agent], final_coordinator
)
.build()
)
# Return workflow and agent map for thread ID extraction
agent_map = {
"travel_request_handler": travel_request_handler,
"hotel-search-agent": hotel_search_agent,
"flight-search-agent": flight_search_agent,
"activity-search-agent": activity_search_agent,
"booking-confirmation-agent": booking_confirmation_agent,
"booking-payment-agent": booking_payment_agent,
"booking-info-aggregation-agent": booking_info_aggregation_agent,
"final-coordinator": final_coordinator.agent,
}
return workflow, agent_map
async def _process_workflow_events(events, conversation_ids, response_ids):
"""Process workflow events and track interactions."""
workflow_output = None
async for event in events:
if event.type == "output":
workflow_output = event.data
# Handle Unicode characters that may not be displayable in Windows console
try:
print(f"\nWorkflow Output: {event.data}\n")
except UnicodeEncodeError:
output_str = str(event.data).encode("ascii", "replace").decode("ascii")
print(f"\nWorkflow Output: {output_str}\n")
elif event.type == "output" and isinstance(event.data, AgentResponseUpdate):
_track_agent_ids(event, event.executor_id, response_ids, conversation_ids)
return workflow_output
def _track_agent_ids(event, agent, response_ids, conversation_ids):
"""Track agent response and conversation IDs - supporting multiple responses per agent."""
if (
isinstance(event.data, AgentResponseUpdate)
and hasattr(event.data, "raw_representation")
and event.data.raw_representation
):
# Check for conversation_id and response_id from raw_representation
# V2 API stores conversation_id directly on raw_representation (ChatResponseUpdate)
raw = event.data.raw_representation
# Try conversation_id directly on raw representation
if (
hasattr(raw, "conversation_id")
and raw.conversation_id # type: ignore[union-attr]
and raw.conversation_id not in conversation_ids[agent] # type: ignore[union-attr]
):
# Only add if not already in the list
conversation_ids[agent].append(raw.conversation_id) # type: ignore[union-attr]
# Extract response_id from the OpenAI event (available from first event)
if hasattr(raw, "raw_representation") and raw.raw_representation: # type: ignore[union-attr]
openai_event = raw.raw_representation # type: ignore[union-attr]
# Check if event has response object with id
if (
hasattr(openai_event, "response")
and hasattr(openai_event.response, "id")
and openai_event.response.id not in response_ids[agent]
):
# Only add if not already in the list
response_ids[agent].append(openai_event.response.id)
async def create_and_run_workflow():
"""Run the workflow evaluation and display results.
Returns:
Dictionary containing agents data with conversation IDs, response IDs, and query information
"""
example_queries = [
"Plan a 3-day trip to Paris from December 15-18, 2025. Budget is $2000. Need hotel near Eiffel Tower, round-trip flights from New York JFK, and recommend 2-3 activities per day.",
"Find a budget hotel in Tokyo for January 5-10, 2026 under $150/night near Shibuya station, book activities including a sushi making class",
"Search for round-trip flights from Los Angeles to London departing March 20, 2026, returning March 27, 2026. Economy class, 2 passengers. Recommend tourist attractions and museums.",
]
query = example_queries[0]
print(f"Query: {query}\n")
result = await run_workflow_with_response_tracking(query)
# Create output data structure
output_data = {"agents": {}, "query": result["query"], "output": result.get("output", "")}
# Create agent-specific mappings - now with lists of IDs
all_agents = set(result["conversation_ids"].keys()) | set(result["response_ids"].keys())
for agent_name in all_agents:
output_data["agents"][agent_name] = {
"conversation_ids": result["conversation_ids"].get(agent_name, []),
"response_ids": result["response_ids"].get(agent_name, []),
"response_count": len(result["response_ids"].get(agent_name, [])),
}
print(f"\nTotal agents tracked: {len(output_data['agents'])}")
# Print summary of multiple responses
print("\n=== Multi-Response Summary ===")
for agent_name, agent_data in output_data["agents"].items():
response_count = agent_data["response_count"]
print(f"{agent_name}: {response_count} response(s)")
return output_data
if __name__ == "__main__":
asyncio.run(create_and_run_workflow())
@@ -1,219 +0,0 @@
# Copyright (c) Microsoft. All rights reserved.
"""
Script to run multi-agent travel planning workflow and evaluate agent responses.
This script:
1. Executes the multi-agent workflow
2. Displays response data summary
3. Creates and runs evaluation with multiple evaluators
4. Monitors evaluation progress and displays results
"""
import asyncio
import os
import time
from azure.ai.projects import AIProjectClient
from azure.identity import DefaultAzureCredential
from create_workflow import create_and_run_workflow
from dotenv import load_dotenv
def print_section(title: str):
"""Print a formatted section header."""
print(f"\n{'=' * 80}")
print(f"{title}")
print(f"{'=' * 80}")
async def run_workflow():
"""Execute the multi-agent travel planning workflow.
Returns:
Dictionary containing workflow data with agent response IDs
"""
print_section("Step 1: Running Workflow")
print("Executing multi-agent travel planning workflow...")
print("This may take a few minutes...")
workflow_data = await create_and_run_workflow()
print("Workflow execution completed")
return workflow_data
def display_response_summary(workflow_data: dict):
"""Display summary of response data."""
print_section("Step 2: Response Data Summary")
print(f"Query: {workflow_data['query']}")
print(f"\nAgents tracked: {len(workflow_data['agents'])}")
for agent_name, agent_data in workflow_data["agents"].items():
response_count = agent_data["response_count"]
print(f" {agent_name}: {response_count} response(s)")
def fetch_agent_responses(openai_client, workflow_data: dict, agent_names: list):
"""Fetch and display final responses from specified agents."""
print_section("Step 3: Fetching Agent Responses")
for agent_name in agent_names:
if agent_name not in workflow_data["agents"]:
continue
agent_data = workflow_data["agents"][agent_name]
if not agent_data["response_ids"]:
continue
final_response_id = agent_data["response_ids"][-1]
print(f"\n{agent_name}")
print(f" Response ID: {final_response_id}")
try:
response = openai_client.responses.retrieve(response_id=final_response_id)
content = response.output[-1].content[-1].text
truncated = content[:300] + "..." if len(content) > 300 else content
print(f" Content preview: {truncated}")
except Exception as e:
print(f" Error: {e}")
def create_evaluation(openai_client, model_deployment: str):
"""Create evaluation with multiple evaluators."""
print_section("Step 4: Creating Evaluation")
data_source_config = {"type": "azure_ai_source", "scenario": "responses"}
testing_criteria = [
{
"type": "azure_ai_evaluator",
"name": "relevance",
"evaluator_name": "builtin.relevance",
"initialization_parameters": {"deployment_name": model_deployment}
},
{
"type": "azure_ai_evaluator",
"name": "groundedness",
"evaluator_name": "builtin.groundedness",
"initialization_parameters": {"deployment_name": model_deployment}
},
{
"type": "azure_ai_evaluator",
"name": "tool_call_accuracy",
"evaluator_name": "builtin.tool_call_accuracy",
"initialization_parameters": {"deployment_name": model_deployment}
},
{
"type": "azure_ai_evaluator",
"name": "tool_output_utilization",
"evaluator_name": "builtin.tool_output_utilization",
"initialization_parameters": {"deployment_name": model_deployment}
},
]
eval_object = openai_client.evals.create(
name="Travel Workflow Multi-Evaluator Assessment",
data_source_config=data_source_config,
testing_criteria=testing_criteria,
)
evaluator_names = [criterion["name"] for criterion in testing_criteria]
print(f"Evaluation created: {eval_object.id}")
print(f"Evaluators ({len(evaluator_names)}): {', '.join(evaluator_names)}")
return eval_object
def run_evaluation(openai_client, eval_object, workflow_data: dict, agent_names: list):
"""Run evaluation on selected agent responses."""
print_section("Step 5: Running Evaluation")
selected_response_ids = []
for agent_name in agent_names:
if agent_name in workflow_data["agents"]:
agent_data = workflow_data["agents"][agent_name]
if agent_data["response_ids"]:
selected_response_ids.append(agent_data["response_ids"][-1])
print(f"Selected {len(selected_response_ids)} responses for evaluation")
data_source = {
"type": "azure_ai_responses",
"item_generation_params": {
"type": "response_retrieval",
"data_mapping": {"response_id": "{{item.resp_id}}"},
"source": {
"type": "file_content",
"content": [{"item": {"resp_id": resp_id}} for resp_id in selected_response_ids]
},
},
}
eval_run = openai_client.evals.runs.create(
eval_id=eval_object.id,
name="Multi-Agent Response Evaluation",
data_source=data_source
)
print(f"Evaluation run created: {eval_run.id}")
return eval_run
def monitor_evaluation(openai_client, eval_object, eval_run):
"""Monitor evaluation progress and display results."""
print_section("Step 6: Monitoring Evaluation")
print("Waiting for evaluation to complete...")
while eval_run.status not in ["completed", "failed"]:
eval_run = openai_client.evals.runs.retrieve(
run_id=eval_run.id,
eval_id=eval_object.id
)
print(f"Status: {eval_run.status}")
time.sleep(5)
if eval_run.status == "completed":
print("\nEvaluation completed successfully")
print(f"Result counts: {eval_run.result_counts}")
print(f"\nReport URL: {eval_run.report_url}")
else:
print("\nEvaluation failed")
async def main():
"""Main execution flow."""
load_dotenv()
print("Travel Planning Workflow Evaluation")
workflow_data = await run_workflow()
display_response_summary(workflow_data)
project_client = AIProjectClient(
endpoint=os.environ["AZURE_AI_PROJECT_ENDPOINT"],
credential=DefaultAzureCredential(),
api_version="2025-11-15-preview"
)
openai_client = project_client.get_openai_client()
agents_to_evaluate = ["hotel-search-agent", "flight-search-agent", "activity-search-agent"]
fetch_agent_responses(openai_client, workflow_data, agents_to_evaluate)
model_deployment = os.environ.get("AZURE_AI_MODEL_DEPLOYMENT_NAME", "gpt-4o-mini")
eval_object = create_evaluation(openai_client, model_deployment)
eval_run = run_evaluation(openai_client, eval_object, workflow_data, agents_to_evaluate)
monitor_evaluation(openai_client, eval_object, eval_run)
print_section("Complete")
if __name__ == "__main__":
asyncio.run(main())