Merge branch 'main' into feature-python-foundry-agents

This commit is contained in:
Dmytro Struk
2025-11-05 08:04:26 -08:00
Unverified
214 changed files with 22261 additions and 1154 deletions
+8 -3
View File
@@ -218,9 +218,14 @@ This directory contains samples demonstrating the capabilities of Microsoft Agen
| File | Description |
|------|-------------|
| [`getting_started/tools/ai_tool_with_approval.py`](./getting_started/tools/ai_tool_with_approval.py) | Demonstration of a tool with approvals |
| [`getting_started/tools/ai_tool_with_approval_and_threads.py`](./getting_started/tools/ai_tool_with_approval_and_threads.py) | Tool Approvals with Threads |
| [`getting_started/tools/failing_tools.py`](./getting_started/tools/failing_tools.py) | Tool exceptions handled by returning the error for the agent to recover from |
| [`getting_started/tools/ai_function_declaration_only.py`](./getting_started/tools/ai_function_declaration_only.py) | Function declarations without implementations for testing agent reasoning |
| [`getting_started/tools/ai_function_from_dict_with_dependency_injection.py`](./getting_started/tools/ai_function_from_dict_with_dependency_injection.py) | Creating AI functions from dictionary definitions using dependency injection |
| [`getting_started/tools/ai_function_recover_from_failures.py`](./getting_started/tools/ai_function_recover_from_failures.py) | Graceful error handling when tools raise exceptions |
| [`getting_started/tools/ai_function_with_approval.py`](./getting_started/tools/ai_function_with_approval.py) | User approval workflows for function calls without threads |
| [`getting_started/tools/ai_function_with_approval_and_threads.py`](./getting_started/tools/ai_function_with_approval_and_threads.py) | Tool approval workflows using threads for conversation history management |
| [`getting_started/tools/ai_function_with_max_exceptions.py`](./getting_started/tools/ai_function_with_max_exceptions.py) | Limiting tool failure exceptions using max_invocation_exceptions |
| [`getting_started/tools/ai_function_with_max_invocations.py`](./getting_started/tools/ai_function_with_max_invocations.py) | Limiting total tool invocations using max_invocations |
| [`getting_started/tools/ai_functions_in_class.py`](./getting_started/tools/ai_functions_in_class.py) | Using ai_function decorator with class methods for stateful tools |
## Workflows
@@ -0,0 +1,4 @@
*.db
*.db-shm
*.db-wal
uploads/
@@ -0,0 +1,268 @@
# ChatKit Integration Sample with Weather Agent and Image Analysis
This sample demonstrates how to integrate Microsoft Agent Framework with OpenAI ChatKit. It provides a complete implementation of a weather assistant with interactive widget visualization, image analysis, and file upload support.
**Features:**
- Weather information with interactive widgets
- Image analysis using vision models
- Current time queries
- File upload with attachment storage
- Chat interface with streaming responses
- City selector widget with one-click weather
## Architecture
```mermaid
graph TB
subgraph Frontend["React Frontend (ChatKit UI)"]
UI[ChatKit Components]
Upload[File Upload]
end
subgraph Backend["FastAPI Server"]
FastAPI[FastAPI Endpoints]
subgraph ChatKit["WeatherChatKitServer"]
Respond[respond method]
Action[action method]
end
subgraph Stores["Data & Storage Layer"]
SQLite[SQLiteStore<br/>Store Protocol]
AttStore[FileBasedAttachmentStore<br/>AttachmentStore Protocol]
DB[(SQLite DB<br/>chatkit_demo.db)]
Files[/uploads directory/]
end
subgraph Integration["Agent Framework Integration"]
Converter[ThreadItemConverter]
Streamer[stream_agent_response]
Agent[ChatAgent]
end
Widgets[Widget Rendering<br/>render_weather_widget<br/>render_city_selector_widget]
end
subgraph Azure["Azure AI"]
Foundry[GPT-5<br/>with Vision]
end
UI -->|HTTP POST /chatkit| FastAPI
Upload -->|HTTP POST /upload/id| FastAPI
FastAPI --> ChatKit
ChatKit -->|save/load threads| SQLite
ChatKit -->|save/load attachments| AttStore
ChatKit -->|convert messages| Converter
SQLite -.->|persist| DB
AttStore -.->|save files| Files
AttStore -.->|save metadata| SQLite
Converter -->|ChatMessage array| Agent
Agent -->|AgentRunResponseUpdate| Streamer
Streamer -->|ThreadStreamEvent| ChatKit
ChatKit --> Widgets
Widgets -->|WidgetItem| ChatKit
Agent <-->|Chat Completions API| Foundry
ChatKit -->|ThreadStreamEvent| FastAPI
FastAPI -->|SSE Stream| UI
style ChatKit fill:#e1f5ff
style Stores fill:#fff4e1
style Integration fill:#f0e1ff
style Azure fill:#e1ffe1
```
### Server Implementation
The sample implements a ChatKit server using the `ChatKitServer` base class from the `chatkit` package:
**Core Components:**
- **`WeatherChatKitServer`**: Custom ChatKit server implementation that:
- Extends `ChatKitServer[dict[str, Any]]`
- Uses Agent Framework's `ChatAgent` with Azure OpenAI
- Converts ChatKit messages to Agent Framework format using `ThreadItemConverter`
- Streams responses back to ChatKit using `stream_agent_response`
- Creates and streams interactive widgets after agent responses
- **`SQLiteStore`**: Data persistence layer that:
- Implements the `Store[dict[str, Any]]` protocol from ChatKit
- Persists threads, messages, and attachment metadata in SQLite
- Provides thread management and item history
- Stores attachment metadata for the upload lifecycle
- **`FileBasedAttachmentStore`**: File storage implementation that:
- Implements the `AttachmentStore[dict[str, Any]]` protocol from ChatKit
- Stores uploaded files on the local filesystem (in `./uploads` directory)
- Generates upload URLs for two-phase file upload
- Saves attachment metadata to the data store for upload tracking
- Provides preview URLs for images
**Key Integration Points:**
```python
# Converting ChatKit messages to Agent Framework
converter = ThreadItemConverter(
attachment_data_fetcher=self._fetch_attachment_data
)
agent_messages = await converter.to_agent_input(user_message_item)
# Running agent and streaming back to ChatKit
async for event in stream_agent_response(
self.weather_agent.run_stream(agent_messages),
thread_id=thread.id,
):
yield event
# Streaming widgets
widget = render_weather_widget(weather_data)
async for event in stream_widget(thread_id=thread.id, widget=widget):
yield event
```
## Installation and Setup
### Prerequisites
- Python 3.10+
- Node.js 18.18+ and npm 9+
- Azure OpenAI service configured
- Azure CLI for authentication (`az login`)
### Backend Setup
1. **Install Python packages:**
```bash
cd python/samples/demos/chatkit-integration
pip install agent-framework-chatkit fastapi uvicorn azure-identity
```
2. **Configure Azure OpenAI:**
```bash
export AZURE_OPENAI_ENDPOINT="https://your-resource.openai.azure.com/"
export AZURE_OPENAI_API_VERSION="2024-06-01"
export AZURE_OPENAI_CHAT_DEPLOYMENT_NAME="gpt-4o"
```
3. **Authenticate with Azure:**
```bash
az login
```
### Frontend Setup
Install the Node.js dependencies:
```bash
cd frontend
npm install
```
## How to Run
### Start the Backend Server
From the `chatkit-integration` directory:
```bash
python app.py
```
Or with auto-reload for development:
```bash
uvicorn app:app --host 127.0.0.1 --port 8001 --reload
```
The backend will start on `http://localhost:8001`
### Start the Frontend Development Server
In a new terminal, from the `frontend` directory:
```bash
npm run dev
```
The frontend will start on `http://localhost:5171`
### Access the Application
Open your browser and navigate to:
```
http://localhost:5171
```
You can now:
- Ask about weather in any location (weather widgets display automatically)
- Upload images for analysis using the attachment button
- Get the current time
- Ask to see available cities and click city buttons for instant weather
### Project Structure
```
chatkit-integration/
├── app.py # FastAPI backend with ChatKitServer implementation
├── store.py # SQLiteStore implementation
├── attachment_store.py # FileBasedAttachmentStore implementation
├── weather_widget.py # Widget rendering functions
├── chatkit_demo.db # SQLite database (auto-created)
├── uploads/ # Uploaded files directory (auto-created)
└── frontend/
├── package.json
├── vite.config.ts
├── index.html
└── src/
├── main.tsx
└── App.tsx # ChatKit UI integration
```
### Configuration
You can customize the application by editing constants at the top of `app.py`:
```python
# Server configuration
SERVER_HOST = "127.0.0.1" # Bind to localhost only for security (local dev)
SERVER_PORT = 8001
SERVER_BASE_URL = f"http://localhost:{SERVER_PORT}"
# Database configuration
DATABASE_PATH = "chatkit_demo.db"
# File storage configuration
UPLOADS_DIRECTORY = "./uploads"
# User context
DEFAULT_USER_ID = "demo_user"
```
### Sample Conversations
Try these example queries:
- "What's the weather like in Tokyo?"
- "Show me available cities" (displays interactive city selector)
- "What's the current time?"
- Upload an image and ask "What do you see in this image?"
## Learn More
- [Agent Framework Documentation](https://aka.ms/agent-framework)
- [ChatKit Documentation](https://platform.openai.com/docs/guides/chatkit)
- [Azure OpenAI Documentation](https://learn.microsoft.com/en-us/azure/ai-foundry/)
@@ -0,0 +1 @@
# Copyright (c) Microsoft. All rights reserved.
@@ -0,0 +1,538 @@
# Copyright (c) Microsoft. All rights reserved.
"""
ChatKit Integration Sample with Weather Agent and Image Analysis
This sample demonstrates how to integrate Microsoft Agent Framework with OpenAI ChatKit
using a weather tool with widget visualization, image analysis, and Azure OpenAI. It shows
a complete ChatKit server implementation using Agent Framework agents with proper FastAPI
setup, interactive weather widgets, and vision capabilities for analyzing uploaded images.
"""
import logging
from collections.abc import AsyncIterator, Callable
from datetime import datetime, timezone
from random import randint
from typing import Annotated, Any
import uvicorn
from azure.identity import AzureCliCredential
from fastapi import FastAPI, File, Request, UploadFile
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import FileResponse, JSONResponse, Response, StreamingResponse
from pydantic import Field
# ============================================================================
# Configuration Constants
# ============================================================================
# Server configuration
SERVER_HOST = "127.0.0.1" # Bind to localhost only for security (local dev)
SERVER_PORT = 8001
SERVER_BASE_URL = f"http://localhost:{SERVER_PORT}"
# Database configuration
DATABASE_PATH = "chatkit_demo.db"
# File storage configuration
UPLOADS_DIRECTORY = "./uploads"
# User context
DEFAULT_USER_ID = "demo_user"
# Logging configuration
LOG_LEVEL = logging.INFO
LOG_FORMAT = "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
LOG_DATE_FORMAT = "%Y-%m-%d %H:%M:%S"
# ============================================================================
# Logging Setup
# ============================================================================
logging.basicConfig(
level=LOG_LEVEL,
format=LOG_FORMAT,
datefmt=LOG_DATE_FORMAT,
)
logger = logging.getLogger(__name__)
# Agent Framework imports
from agent_framework import AgentRunResponseUpdate, ChatAgent, ChatMessage, FunctionResultContent, Role
from agent_framework.azure import AzureOpenAIChatClient
# Agent Framework ChatKit integration
from agent_framework_chatkit import ThreadItemConverter, stream_agent_response
# Local imports
from attachment_store import FileBasedAttachmentStore
# ChatKit imports
from chatkit.actions import Action
from chatkit.server import ChatKitServer
from chatkit.store import StoreItemType, default_generate_id
from chatkit.types import (
ThreadItemDoneEvent,
ThreadMetadata,
ThreadStreamEvent,
UserMessageItem,
WidgetItem,
)
from chatkit.widgets import WidgetRoot
from store import SQLiteStore
from weather_widget import (
WeatherData,
city_selector_copy_text,
render_city_selector_widget,
render_weather_widget,
weather_widget_copy_text,
)
class WeatherResponse(str):
"""A string response that also carries WeatherData for widget creation."""
def __new__(cls, text: str, weather_data: WeatherData):
instance = super().__new__(cls, text)
instance.weather_data = weather_data # type: ignore
return instance
async def stream_widget(
thread_id: str,
widget: WidgetRoot,
copy_text: str | None = None,
generate_id: Callable[[StoreItemType], str] = default_generate_id,
) -> AsyncIterator[ThreadStreamEvent]:
"""Stream a ChatKit widget as a ThreadStreamEvent.
This helper function creates a ChatKit widget item and yields it as a
ThreadItemDoneEvent that can be consumed by the ChatKit UI.
Args:
thread_id: The ChatKit thread ID for the conversation.
widget: The ChatKit widget to display.
copy_text: Optional text representation of the widget for copy/paste.
generate_id: Optional function to generate IDs for ChatKit items.
Yields:
ThreadStreamEvent: ChatKit event containing the widget.
"""
item_id = generate_id("message")
widget_item = WidgetItem(
id=item_id,
thread_id=thread_id,
created_at=datetime.now(),
widget=widget,
copy_text=copy_text,
)
yield ThreadItemDoneEvent(type="thread.item.done", item=widget_item)
def get_weather(
location: Annotated[str, Field(description="The location to get the weather for.")],
) -> str:
"""Get the weather for a given location.
Returns a string description with embedded WeatherData for widget creation.
"""
logger.info(f"Fetching weather for location: {location}")
conditions = ["sunny", "cloudy", "rainy", "stormy", "snowy", "foggy"]
temperature = randint(-5, 35)
condition = conditions[randint(0, len(conditions) - 1)]
# Add some realistic details
humidity = randint(30, 90)
wind_speed = randint(5, 25)
weather_data = WeatherData(
location=location,
condition=condition,
temperature=temperature,
humidity=humidity,
wind_speed=wind_speed,
)
logger.debug(f"Weather data generated: {condition}, {temperature}°C, {humidity}% humidity, {wind_speed} km/h wind")
# Return a WeatherResponse that is both a string (for the LLM) and carries structured data
text = (
f"Weather in {location}:\n"
f"• Condition: {condition.title()}\n"
f"• Temperature: {temperature}°C\n"
f"• Humidity: {humidity}%\n"
f"• Wind: {wind_speed} km/h"
)
return WeatherResponse(text, weather_data)
def get_time() -> str:
"""Get the current UTC time."""
current_time = datetime.now(timezone.utc)
logger.info("Getting current UTC time")
return f"Current UTC time: {current_time.strftime('%Y-%m-%d %H:%M:%S')} UTC"
def show_city_selector() -> str:
"""Show an interactive city selector widget to the user.
This function triggers the display of a widget that allows users
to select from popular cities to get weather information.
Returns a special marker string that will be detected to show the widget.
"""
logger.info("Activating city selector widget")
return "__SHOW_CITY_SELECTOR__"
class WeatherChatKitServer(ChatKitServer[dict[str, Any]]):
"""ChatKit server implementation using Agent Framework.
This server integrates Agent Framework agents with ChatKit's server protocol,
providing weather information with interactive widgets and time queries through Azure OpenAI.
"""
def __init__(self, data_store: SQLiteStore, attachment_store: FileBasedAttachmentStore):
super().__init__(data_store, attachment_store)
logger.info("Initializing WeatherChatKitServer")
# Create Agent Framework agent with Azure OpenAI
# For authentication, run `az login` command in terminal
try:
self.weather_agent = ChatAgent(
chat_client=AzureOpenAIChatClient(credential=AzureCliCredential()),
instructions=(
"You are a helpful weather assistant with image analysis capabilities. "
"You can provide weather information for any location, tell the current time, "
"and analyze images that users upload. Be friendly and informative in your responses.\n\n"
"If a user asks to see a list of cities or wants to choose from available cities, "
"use the show_city_selector tool to display an interactive city selector.\n\n"
"When users upload images, you will automatically receive them and can analyze their content. "
"Describe what you see in detail and be helpful in answering questions about the images."
),
tools=[get_weather, get_time, show_city_selector],
)
logger.info("Weather agent initialized successfully with Azure OpenAI")
except Exception as e:
logger.error(f"Failed to initialize weather agent: {e}")
raise
# Create ThreadItemConverter with attachment data fetcher
self.converter = ThreadItemConverter(
attachment_data_fetcher=self._fetch_attachment_data,
)
logger.info("WeatherChatKitServer initialized")
async def _fetch_attachment_data(self, attachment_id: str) -> bytes:
"""Fetch attachment binary data for the converter.
Args:
attachment_id: The ID of the attachment to fetch.
Returns:
The binary data of the attachment.
"""
return await attachment_store.read_attachment_bytes(attachment_id)
async def respond(
self,
thread: ThreadMetadata,
input_user_message: UserMessageItem | None,
context: dict[str, Any],
) -> AsyncIterator[ThreadStreamEvent]:
"""Handle incoming user messages and generate responses.
This method converts ChatKit messages to Agent Framework format using ThreadItemConverter,
runs the agent, converts the response back to ChatKit events using stream_agent_response,
and creates interactive weather widgets when weather data is queried.
"""
from agent_framework import FunctionResultContent
if input_user_message is None:
logger.debug("Received None user message, skipping")
return
logger.info(f"Processing message for thread: {thread.id}")
try:
# Track weather data and city selector flag for this request
weather_data: WeatherData | None = None
show_city_selector = False
# Convert ChatKit user message to Agent Framework ChatMessage using ThreadItemConverter
agent_messages = await self.converter.to_agent_input(input_user_message)
if not agent_messages:
logger.warning("No messages after conversion")
return
logger.info(f"Running agent with {len(agent_messages)} message(s)")
# Run the Agent Framework agent with streaming
agent_stream = self.weather_agent.run_stream(agent_messages)
# Create an intercepting stream that extracts function results while passing through updates
async def intercept_stream() -> AsyncIterator[AgentRunResponseUpdate]:
nonlocal weather_data, show_city_selector
async for update in agent_stream:
# Check for function results in the update
if update.contents:
for content in update.contents:
if isinstance(content, FunctionResultContent):
result = content.result
# Check if it's a WeatherResponse (string subclass with weather_data attribute)
if isinstance(result, str) and hasattr(result, "weather_data"):
extracted_data = getattr(result, "weather_data", None)
if isinstance(extracted_data, WeatherData):
weather_data = extracted_data
logger.info(f"Weather data extracted: {weather_data.location}")
# Check if it's the city selector marker
elif isinstance(result, str) and result == "__SHOW_CITY_SELECTOR__":
show_city_selector = True
logger.info("City selector flag detected")
yield update
# Stream updates as ChatKit events with interception
async for event in stream_agent_response(
intercept_stream(),
thread_id=thread.id,
):
yield event
# If weather data was collected during the tool call, create a widget
if weather_data is not None and isinstance(weather_data, WeatherData):
logger.info(f"Creating weather widget for location: {weather_data.location}")
# Create weather widget
widget = render_weather_widget(weather_data)
copy_text = weather_widget_copy_text(weather_data)
# Stream the widget
async for widget_event in stream_widget(thread_id=thread.id, widget=widget, copy_text=copy_text):
yield widget_event
logger.debug("Weather widget streamed successfully")
# If city selector should be shown, create and stream that widget
if show_city_selector:
logger.info("Creating city selector widget")
# Create city selector widget
selector_widget = render_city_selector_widget()
selector_copy_text = city_selector_copy_text()
# Stream the widget
async for widget_event in stream_widget(
thread_id=thread.id, widget=selector_widget, copy_text=selector_copy_text
):
yield widget_event
logger.debug("City selector widget streamed successfully")
logger.info(f"Completed processing message for thread: {thread.id}")
except Exception as e:
logger.error(f"Error processing message for thread {thread.id}: {e}", exc_info=True)
async def action(
self,
thread: ThreadMetadata,
action: Action[str, Any],
sender: WidgetItem | None,
context: dict[str, Any],
) -> AsyncIterator[ThreadStreamEvent]:
"""Handle widget actions from the frontend.
This method processes actions triggered by interactive widgets,
such as city selection from the city selector widget.
"""
logger.info(f"Received action: {action.type} for thread: {thread.id}")
if action.type == "city_selected":
# Extract city information from the action payload
city_label = action.payload.get("city_label", "Unknown")
logger.info(f"City selected: {city_label}")
logger.debug(f"Action payload: {action.payload}")
# Track weather data for this request
weather_data: WeatherData | None = None
# Create an agent message asking about the weather
agent_messages = [ChatMessage(role=Role.USER, text=f"What's the weather in {city_label}?")]
logger.debug(f"Processing weather query: {agent_messages[0].text}")
# Run the Agent Framework agent with streaming
agent_stream = self.weather_agent.run_stream(agent_messages)
# Create an intercepting stream that extracts function results while passing through updates
async def intercept_stream() -> AsyncIterator[AgentRunResponseUpdate]:
nonlocal weather_data
async for update in agent_stream:
# Check for function results in the update
if update.contents:
for content in update.contents:
if isinstance(content, FunctionResultContent):
result = content.result
# Check if it's a WeatherResponse (string subclass with weather_data attribute)
if isinstance(result, str) and hasattr(result, "weather_data"):
extracted_data = getattr(result, "weather_data", None)
if isinstance(extracted_data, WeatherData):
weather_data = extracted_data
logger.info(f"Weather data extracted: {weather_data.location}")
yield update
# Stream updates as ChatKit events with interception
async for event in stream_agent_response(
intercept_stream(),
thread_id=thread.id,
):
yield event
# If weather data was collected during the tool call, create a widget
if weather_data is not None and isinstance(weather_data, WeatherData):
logger.info(f"Creating weather widget for: {weather_data.location}")
# Create weather widget
widget = render_weather_widget(weather_data)
copy_text = weather_widget_copy_text(weather_data)
# Stream the widget
async for widget_event in stream_widget(thread_id=thread.id, widget=widget, copy_text=copy_text):
yield widget_event
logger.debug("Weather widget created successfully from action")
else:
logger.warning("No weather data available to create widget after action")
# FastAPI application setup
app = FastAPI(
title="ChatKit Weather & Vision Agent",
description="Weather and image analysis assistant powered by Agent Framework and Azure OpenAI",
version="1.0.0",
)
# Add CORS middleware to allow frontend connections
app.add_middleware(
CORSMiddleware,
allow_origins=["*"], # In production, specify exact origins
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
# Initialize data store and ChatKit server
logger.info("Initializing application components")
data_store = SQLiteStore(db_path=DATABASE_PATH)
attachment_store = FileBasedAttachmentStore(
uploads_dir=UPLOADS_DIRECTORY,
base_url=SERVER_BASE_URL,
data_store=data_store,
)
chatkit_server = WeatherChatKitServer(data_store, attachment_store)
logger.info("Application initialization complete")
@app.post("/chatkit")
async def chatkit_endpoint(request: Request):
"""Main ChatKit endpoint that handles all ChatKit requests.
This endpoint follows the ChatKit server protocol and handles both
streaming and non-streaming responses.
"""
logger.debug(f"Received ChatKit request from {request.client}")
request_body = await request.body()
# Create context following the working examples pattern
context = {"request": request}
try:
# Process the request using ChatKit server
result = await chatkit_server.process(request_body, context)
# Return appropriate response type
if hasattr(result, "__aiter__"): # StreamingResult
logger.debug("Returning streaming response")
return StreamingResponse(result, media_type="text/event-stream") # type: ignore[arg-type]
# NonStreamingResult
logger.debug("Returning non-streaming response")
return Response(content=result.json, media_type="application/json") # type: ignore[union-attr]
except Exception as e:
logger.error(f"Error processing ChatKit request: {e}", exc_info=True)
raise
@app.post("/upload/{attachment_id}")
async def upload_file(attachment_id: str, file: UploadFile = File(...)):
"""Handle file upload for two-phase upload.
The client POSTs the file bytes here after creating the attachment
via the ChatKit attachments.create endpoint.
"""
logger.info(f"Receiving file upload for attachment: {attachment_id}")
try:
# Read file contents
contents = await file.read()
# Save to disk
file_path = attachment_store.get_file_path(attachment_id)
file_path.write_bytes(contents)
logger.info(f"Saved {len(contents)} bytes to {file_path}")
# Load the attachment metadata from the data store
attachment = await data_store.load_attachment(attachment_id, {"user_id": DEFAULT_USER_ID})
# Clear the upload_url since upload is complete
attachment.upload_url = None
# Save the updated attachment back to the store
await data_store.save_attachment(attachment, {"user_id": DEFAULT_USER_ID})
# Return the attachment metadata as JSON
return JSONResponse(content=attachment.model_dump(mode="json"))
except Exception as e:
logger.error(f"Error uploading file for attachment {attachment_id}: {e}", exc_info=True)
return JSONResponse(status_code=500, content={"error": f"Failed to upload file: {str(e)}"})
@app.get("/preview/{attachment_id}")
async def preview_image(attachment_id: str):
"""Serve image preview/thumbnail.
For simplicity, this serves the full image. In production, you should
generate and cache thumbnails.
"""
logger.debug(f"Serving preview for attachment: {attachment_id}")
try:
file_path = attachment_store.get_file_path(attachment_id)
if not file_path.exists():
return JSONResponse(status_code=404, content={"error": "File not found"})
# Determine media type from file extension or attachment metadata
# For simplicity, we'll try to load from the store
try:
attachment = await data_store.load_attachment(attachment_id, {"user_id": DEFAULT_USER_ID})
media_type = attachment.mime_type
except Exception:
# Default to binary if we can't determine
media_type = "application/octet-stream"
return FileResponse(file_path, media_type=media_type)
except Exception as e:
logger.error(f"Error serving preview for attachment {attachment_id}: {e}", exc_info=True)
return JSONResponse(status_code=500, content={"error": str(e)})
if __name__ == "__main__":
# Run the server
logger.info(f"Starting ChatKit Weather Agent server on {SERVER_HOST}:{SERVER_PORT}")
uvicorn.run(app, host=SERVER_HOST, port=SERVER_PORT, log_level="info")
@@ -0,0 +1,121 @@
# Copyright (c) Microsoft. All rights reserved.
"""File-based AttachmentStore implementation for ChatKit.
This module provides a simple AttachmentStore implementation that stores
uploaded files on the local filesystem. In production, you should use
cloud storage like S3, Azure Blob Storage, or Google Cloud Storage.
"""
from pathlib import Path
from typing import Any, TYPE_CHECKING
from chatkit.store import AttachmentStore
from chatkit.types import Attachment, AttachmentCreateParams, FileAttachment, ImageAttachment
from pydantic import AnyUrl
if TYPE_CHECKING:
from store import SQLiteStore
class FileBasedAttachmentStore(AttachmentStore[dict[str, Any]]):
"""File-based AttachmentStore that stores files on local disk.
This implementation stores uploaded files in a local directory and provides
upload URLs that point to the FastAPI upload endpoint. It supports both
image and file attachments.
Features:
- Stores files in a local uploads directory
- Generates upload URLs for two-phase upload
- Generates preview URLs for images
- Proper cleanup on deletion
Note: This is for demonstration purposes. In production, use cloud storage
with signed URLs for better security and scalability.
"""
def __init__(
self,
uploads_dir: str = "./uploads",
base_url: str = "http://localhost:8001",
data_store: "SQLiteStore | None" = None,
):
"""Initialize the file-based attachment store.
Args:
uploads_dir: Directory where uploaded files will be stored
base_url: Base URL for generating upload and preview URLs
data_store: Optional data store to persist attachment metadata
"""
self.uploads_dir = Path(uploads_dir)
self.base_url = base_url.rstrip("/")
self.data_store = data_store
# Create uploads directory if it doesn't exist
self.uploads_dir.mkdir(parents=True, exist_ok=True)
def get_file_path(self, attachment_id: str) -> Path:
"""Get the filesystem path for an attachment."""
return self.uploads_dir / attachment_id
async def delete_attachment(self, attachment_id: str, context: dict[str, Any]) -> None:
"""Delete an attachment and its file from disk."""
file_path = self.get_file_path(attachment_id)
if file_path.exists():
file_path.unlink()
async def create_attachment(
self, input: AttachmentCreateParams, context: dict[str, Any]
) -> Attachment:
"""Create an attachment with upload URL for two-phase upload.
This creates the attachment metadata and returns upload URLs that
the client will use to POST the actual file bytes.
"""
# Generate unique ID for this attachment
attachment_id = self.generate_attachment_id(input.mime_type, context)
# Generate upload URL that points to our FastAPI upload endpoint
upload_url = f"{self.base_url}/upload/{attachment_id}"
# Create appropriate attachment type based on MIME type
if input.mime_type.startswith("image/"):
# For images, also provide a preview URL
preview_url = f"{self.base_url}/preview/{attachment_id}"
attachment = ImageAttachment(
id=attachment_id,
type="image",
mime_type=input.mime_type,
name=input.name,
upload_url=AnyUrl(upload_url),
preview_url=AnyUrl(preview_url),
)
else:
# For files, just provide upload URL
attachment = FileAttachment(
id=attachment_id,
type="file",
mime_type=input.mime_type,
name=input.name,
upload_url=AnyUrl(upload_url),
)
# Save attachment metadata to data store so it's available during upload
if self.data_store is not None:
await self.data_store.save_attachment(attachment, context)
return attachment
async def read_attachment_bytes(self, attachment_id: str) -> bytes:
"""Read the raw bytes of an uploaded attachment.
This is used by the ThreadItemConverter to create base64-encoded
content for sending to the Agent Framework.
"""
file_path = self.get_file_path(attachment_id)
if not file_path.exists():
raise FileNotFoundError(f"Attachment {attachment_id} not found on disk")
return file_path.read_bytes()
@@ -0,0 +1,52 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>ChatKit + Agent Framework Demo</title>
<script src="https://cdn.platform.openai.com/deployments/chatkit/chatkit.js"></script>
<style>
* {
margin: 0;
padding: 0;
box-sizing: border-box;
}
body {
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, 'Helvetica Neue', Arial, sans-serif;
height: 100vh;
display: flex;
flex-direction: column;
}
header {
padding: 1rem;
background: #f5f5f5;
border-bottom: 1px solid #ddd;
}
h1 {
font-size: 1.5rem;
margin-bottom: 0.5rem;
}
p {
color: #666;
font-size: 0.9rem;
}
#root {
flex: 1;
overflow: hidden;
}
</style>
</head>
<body>
<header>
<h1>ChatKit + Agent Framework Demo</h1>
<p>Simple weather assistant powered by Agent Framework and ChatKit</p>
</header>
<div id="root"></div>
<script type="module" src="/src/main.tsx"></script>
</body>
</html>
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,27 @@
{
"name": "chatkit-agent-framework-demo",
"version": "0.1.0",
"private": true,
"type": "module",
"scripts": {
"dev": "vite",
"build": "vite build",
"preview": "vite preview"
},
"engines": {
"node": ">=18.18",
"npm": ">=9"
},
"dependencies": {
"@openai/chatkit-react": "^0",
"react": "^19.2.0",
"react-dom": "^19.2.0"
},
"devDependencies": {
"@types/react": "^19.2.0",
"@types/react-dom": "^19.2.0",
"@vitejs/plugin-react-swc": "^3.5.0",
"typescript": "^5.4.0",
"vite": "^7.1.9"
}
}
@@ -0,0 +1,33 @@
import { ChatKit, useChatKit } from "@openai/chatkit-react";
const CHATKIT_API_URL = "/chatkit";
const CHATKIT_API_DOMAIN_KEY =
import.meta.env.VITE_CHATKIT_API_DOMAIN_KEY ?? "domain_pk_localhost_dev";
export default function App() {
const chatkit = useChatKit({
api: {
url: CHATKIT_API_URL,
domainKey: CHATKIT_API_DOMAIN_KEY,
uploadStrategy: { type: "two_phase" },
},
startScreen: {
greeting: "Hello! I'm your weather and image analysis assistant. Ask me about the weather in any location or upload images for me to analyze.",
prompts: [
{ label: "Weather in New York", prompt: "What's the weather in New York?" },
{ label: "Select City to Get Weather", prompt: "Show me the city selector for weather" },
{ label: "Current Time", prompt: "What time is it?" },
{ label: "Analyze an Image", prompt: "I'll upload an image for you to analyze" },
],
},
composer: {
placeholder: "Ask about weather or upload an image...",
attachments: {
enabled: true,
accept: { "image/*": [".png", ".jpg", ".jpeg", ".gif", ".webp"] },
},
},
});
return <ChatKit control={chatkit.control} style={{ height: "100%" }} />;
}
@@ -0,0 +1,15 @@
import { StrictMode } from "react";
import { createRoot } from "react-dom/client";
import App from "./App";
const container = document.getElementById("root");
if (!container) {
throw new Error("Root element with id 'root' not found");
}
createRoot(container).render(
<StrictMode>
<App />
</StrictMode>,
);
@@ -0,0 +1 @@
/// <reference types="vite/client" />
@@ -0,0 +1,21 @@
{
"compilerOptions": {
"target": "ES2020",
"useDefineForClassFields": true,
"lib": ["ES2020", "DOM", "DOM.Iterable"],
"module": "ESNext",
"skipLibCheck": true,
"moduleResolution": "bundler",
"allowImportingTsExtensions": true,
"resolveJsonModule": true,
"isolatedModules": true,
"noEmit": true,
"jsx": "react-jsx",
"strict": true,
"noUnusedLocals": true,
"noUnusedParameters": true,
"noFallthroughCasesInSwitch": true
},
"include": ["src"],
"references": [{ "path": "./tsconfig.node.json" }]
}
@@ -0,0 +1,10 @@
{
"compilerOptions": {
"composite": true,
"skipLibCheck": true,
"module": "ESNext",
"moduleResolution": "bundler",
"allowSyntheticDefaultImports": true
},
"include": ["vite.config.ts"]
}
@@ -0,0 +1,24 @@
import { defineConfig } from "vite";
import react from "@vitejs/plugin-react-swc";
const backendTarget = process.env.BACKEND_URL ?? "http://127.0.0.1:8001";
export default defineConfig({
plugins: [react()],
server: {
host: "0.0.0.0",
port: 5171,
proxy: {
"/chatkit": {
target: backendTarget,
changeOrigin: true,
},
},
// For production deployments, you need to add your public domains to this list
allowedHosts: [
// You can remove these examples added just to demonstrate how to configure the allowlist
".ngrok.io",
".trycloudflare.com",
],
},
});
@@ -0,0 +1,361 @@
# Copyright (c) Microsoft. All rights reserved.
"""SQLite-based store implementation for ChatKit data persistence.
This module provides a complete Store implementation using SQLite for data persistence.
It includes proper thread safety, user isolation, and follows the ChatKit Store protocol.
"""
import sqlite3
import uuid
from typing import Any
from chatkit.store import Store, NotFoundError
from chatkit.types import (
Attachment,
Page,
ThreadItem,
ThreadMetadata,
)
from pydantic import BaseModel
class ThreadData(BaseModel):
"""Model for serializing thread data to SQLite."""
thread: ThreadMetadata
class ItemData(BaseModel):
"""Model for serializing thread item data to SQLite."""
item: ThreadItem
class AttachmentData(BaseModel):
"""Model for serializing attachment data to SQLite."""
attachment: Attachment
class SQLiteStore(Store[dict[str, Any]]):
"""SQLite-based store implementation for ChatKit data.
This implementation follows the pattern from the ChatKit Python tests
and provides persistent storage for threads, messages, and attachments.
Features:
- Thread-safe SQLite connections with WAL mode
- User isolation for multi-tenant support
- Proper error handling and transaction management
- Complete Store protocol implementation
Note: This is for demonstration purposes. In production, you should
implement proper error handling, connection pooling, and migration strategies.
"""
def __init__(self, db_path: str | None = None):
self.db_path = db_path or "chatkit_demo.db" # Use file-based DB for demo
self._create_tables()
def _create_connection(self):
# Enable thread safety and WAL mode for better concurrent access
conn = sqlite3.connect(self.db_path, check_same_thread=False)
conn.execute("PRAGMA journal_mode=WAL")
return conn
def _create_tables(self):
with self._create_connection() as conn:
# Create threads table
conn.execute(
"""CREATE TABLE IF NOT EXISTS threads (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL,
created_at TEXT NOT NULL,
data TEXT NOT NULL
)"""
)
# Create items table
conn.execute(
"""CREATE TABLE IF NOT EXISTS items (
id TEXT PRIMARY KEY,
thread_id TEXT NOT NULL,
user_id TEXT NOT NULL,
created_at TEXT NOT NULL,
data TEXT NOT NULL
)"""
)
# Create attachments table
conn.execute(
"""CREATE TABLE IF NOT EXISTS attachments (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL,
data TEXT NOT NULL
)"""
)
conn.commit()
def generate_thread_id(self, context: dict[str, Any]) -> str:
return f"thr_{uuid.uuid4().hex[:8]}"
def generate_item_id(
self,
item_type: str,
thread: ThreadMetadata,
context: dict[str, Any],
) -> str:
prefix_map = {
"message": "msg",
"tool_call": "tc",
"task": "tsk",
"workflow": "wf",
"attachment": "atc",
}
prefix = prefix_map.get(item_type, "itm")
return f"{prefix}_{uuid.uuid4().hex[:8]}"
async def load_thread(self, thread_id: str, context: dict[str, Any]) -> ThreadMetadata:
user_id = context.get("user_id", "demo_user")
with self._create_connection() as conn:
cursor = conn.execute(
"SELECT data FROM threads WHERE id = ? AND user_id = ?",
(thread_id, user_id),
).fetchone()
if cursor is None:
raise NotFoundError(f"Thread {thread_id} not found")
thread_data = ThreadData.model_validate_json(cursor[0])
return thread_data.thread
async def save_thread(self, thread: ThreadMetadata, context: dict[str, Any]) -> None:
user_id = context.get("user_id", "demo_user")
with self._create_connection() as conn:
thread_data = ThreadData(thread=thread)
# Replace existing thread data
conn.execute(
"DELETE FROM threads WHERE id = ? AND user_id = ?",
(thread.id, user_id),
)
conn.execute(
"INSERT INTO threads (id, user_id, created_at, data) VALUES (?, ?, ?, ?)",
(
thread.id,
user_id,
thread.created_at.isoformat(),
thread_data.model_dump_json(),
),
)
conn.commit()
async def load_thread_items(
self,
thread_id: str,
after: str | None,
limit: int,
order: str,
context: dict[str, Any],
) -> Page[ThreadItem]:
user_id = context.get("user_id", "demo_user")
with self._create_connection() as conn:
created_after: str | None = None
if after:
after_cursor = conn.execute(
"SELECT created_at FROM items WHERE id = ? AND user_id = ?",
(after, user_id),
).fetchone()
if after_cursor is None:
raise NotFoundError(f"Item {after} not found")
created_after = after_cursor[0]
query = """
SELECT data FROM items
WHERE thread_id = ? AND user_id = ?
"""
params: list[Any] = [thread_id, user_id]
if created_after:
query += " AND created_at > ?" if order == "asc" else " AND created_at < ?"
params.append(created_after)
query += f" ORDER BY created_at {order} LIMIT ?"
params.append(limit + 1)
items_cursor = conn.execute(query, params).fetchall()
items = [
ItemData.model_validate_json(row[0]).item for row in items_cursor
]
has_more = len(items) > limit
if has_more:
items = items[:limit]
return Page[ThreadItem](
data=items,
has_more=has_more,
after=items[-1].id if items else None
)
async def save_attachment(self, attachment: Attachment, context: dict[str, Any]) -> None:
user_id = context.get("user_id", "demo_user")
with self._create_connection() as conn:
attachment_data = AttachmentData(attachment=attachment)
conn.execute(
"INSERT OR REPLACE INTO attachments (id, user_id, data) VALUES (?, ?, ?)",
(
attachment.id,
user_id,
attachment_data.model_dump_json(),
),
)
conn.commit()
async def load_attachment(self, attachment_id: str, context: dict[str, Any]) -> Attachment:
user_id = context.get("user_id", "demo_user")
with self._create_connection() as conn:
cursor = conn.execute(
"SELECT data FROM attachments WHERE id = ? AND user_id = ?",
(attachment_id, user_id),
).fetchone()
if cursor is None:
raise NotFoundError(f"Attachment {attachment_id} not found")
attachment_data = AttachmentData.model_validate_json(cursor[0])
return attachment_data.attachment
async def delete_attachment(self, attachment_id: str, context: dict[str, Any]) -> None:
user_id = context.get("user_id", "demo_user")
with self._create_connection() as conn:
conn.execute(
"DELETE FROM attachments WHERE id = ? AND user_id = ?",
(attachment_id, user_id),
)
conn.commit()
async def load_threads(
self,
limit: int,
after: str | None,
order: str,
context: dict[str, Any],
) -> Page[ThreadMetadata]:
user_id = context.get("user_id", "demo_user")
with self._create_connection() as conn:
created_after: str | None = None
if after:
after_cursor = conn.execute(
"SELECT created_at FROM threads WHERE id = ? AND user_id = ?",
(after, user_id),
).fetchone()
if after_cursor is None:
raise NotFoundError(f"Thread {after} not found")
created_after = after_cursor[0]
query = "SELECT data FROM threads WHERE user_id = ?"
params: list[Any] = [user_id]
if created_after:
query += " AND created_at > ?" if order == "asc" else " AND created_at < ?"
params.append(created_after)
query += f" ORDER BY created_at {order} LIMIT ?"
params.append(limit + 1)
threads_cursor = conn.execute(query, params).fetchall()
threads = [
ThreadData.model_validate_json(row[0]).thread for row in threads_cursor
]
has_more = len(threads) > limit
if has_more:
threads = threads[:limit]
return Page[ThreadMetadata](
data=threads,
has_more=has_more,
after=threads[-1].id if threads else None
)
async def add_thread_item(
self, thread_id: str, item: ThreadItem, context: dict[str, Any]
) -> None:
user_id = context.get("user_id", "demo_user")
with self._create_connection() as conn:
item_data = ItemData(item=item)
conn.execute(
"INSERT INTO items (id, thread_id, user_id, created_at, data) VALUES (?, ?, ?, ?, ?)",
(
item.id,
thread_id,
user_id,
item.created_at.isoformat(),
item_data.model_dump_json(),
),
)
conn.commit()
async def save_item(self, thread_id: str, item: ThreadItem, context: dict[str, Any]) -> None:
user_id = context.get("user_id", "demo_user")
with self._create_connection() as conn:
item_data = ItemData(item=item)
conn.execute(
"UPDATE items SET data = ? WHERE id = ? AND thread_id = ? AND user_id = ?",
(
item_data.model_dump_json(),
item.id,
thread_id,
user_id,
),
)
conn.commit()
async def load_item(self, thread_id: str, item_id: str, context: dict[str, Any]) -> ThreadItem:
user_id = context.get("user_id", "demo_user")
with self._create_connection() as conn:
cursor = conn.execute(
"SELECT data FROM items WHERE id = ? AND thread_id = ? AND user_id = ?",
(item_id, thread_id, user_id),
).fetchone()
if cursor is None:
raise NotFoundError(f"Item {item_id} not found in thread {thread_id}")
item_data = ItemData.model_validate_json(cursor[0])
return item_data.item
async def delete_thread(self, thread_id: str, context: dict[str, Any]) -> None:
user_id = context.get("user_id", "demo_user")
with self._create_connection() as conn:
conn.execute(
"DELETE FROM threads WHERE id = ? AND user_id = ?",
(thread_id, user_id),
)
conn.execute(
"DELETE FROM items WHERE thread_id = ? AND user_id = ?",
(thread_id, user_id),
)
conn.commit()
async def delete_thread_item(
self, thread_id: str, item_id: str, context: dict[str, Any]
) -> None:
user_id = context.get("user_id", "demo_user")
with self._create_connection() as conn:
conn.execute(
"DELETE FROM items WHERE id = ? AND thread_id = ? AND user_id = ?",
(item_id, thread_id, user_id),
)
conn.commit()
@@ -0,0 +1,437 @@
# Copyright (c) Microsoft. All rights reserved.
"""Weather widget rendering for ChatKit integration sample."""
import base64
from dataclasses import dataclass
from chatkit.actions import ActionConfig
from chatkit.widgets import Box, Button, Card, Col, Image, Row, Text, Title, WidgetRoot
WEATHER_ICON_COLOR = "#1D4ED8"
WEATHER_ICON_ACCENT = "#DBEAFE"
# Popular cities for the selector
POPULAR_CITIES = [
{"value": "seattle", "label": "Seattle, WA", "description": "Pacific Northwest"},
{"value": "new_york", "label": "New York, NY", "description": "East Coast"},
{"value": "san_francisco", "label": "San Francisco, CA", "description": "Bay Area"},
{"value": "chicago", "label": "Chicago, IL", "description": "Midwest"},
{"value": "miami", "label": "Miami, FL", "description": "Southeast"},
{"value": "austin", "label": "Austin, TX", "description": "Southwest"},
{"value": "boston", "label": "Boston, MA", "description": "New England"},
{"value": "denver", "label": "Denver, CO", "description": "Mountain West"},
{"value": "portland", "label": "Portland, OR", "description": "Pacific Northwest"},
{"value": "atlanta", "label": "Atlanta, GA", "description": "Southeast"},
]
# Mapping from city values to display names for weather queries
CITY_VALUE_TO_NAME = {city["value"]: city["label"] for city in POPULAR_CITIES}
def _sun_svg() -> str:
"""Generate SVG for sunny weather icon."""
color = WEATHER_ICON_COLOR
accent = WEATHER_ICON_ACCENT
return (
'<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64" fill="none">'
f'<circle cx="32" cy="32" r="13" fill="{accent}" stroke="{color}" stroke-width="3"/>'
f'<g stroke="{color}" stroke-width="3" stroke-linecap="round">'
'<line x1="32" y1="8" x2="32" y2="16"/>'
'<line x1="32" y1="48" x2="32" y2="56"/>'
'<line x1="8" y1="32" x2="16" y2="32"/>'
'<line x1="48" y1="32" x2="56" y2="32"/>'
'<line x1="14.93" y1="14.93" x2="20.55" y2="20.55"/>'
'<line x1="43.45" y1="43.45" x2="49.07" y2="49.07"/>'
'<line x1="14.93" y1="49.07" x2="20.55" y2="43.45"/>'
'<line x1="43.45" y1="20.55" x2="49.07" y2="14.93"/>'
"</g>"
"</svg>"
)
def _cloud_svg() -> str:
"""Generate SVG for cloudy weather icon."""
color = WEATHER_ICON_COLOR
accent = WEATHER_ICON_ACCENT
return (
'<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64" fill="none">'
f'<path d="M22 46H44C50.075 46 55 41.075 55 35S50.075 24 44 24H42.7C41.2 16.2 34.7 10 26.5 10 18 10 11.6 16.1 11 24.3 6.5 25.6 3 29.8 3 35s4.925 11 11 11h8Z" '
f'fill="{accent}" stroke="{color}" stroke-width="3" stroke-linejoin="round"/>'
"</svg>"
)
def _rain_svg() -> str:
"""Generate SVG for rainy weather icon."""
color = WEATHER_ICON_COLOR
accent = WEATHER_ICON_ACCENT
return (
'<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64" fill="none">'
f'<path d="M22 40H44C50.075 40 55 35.075 55 29S50.075 18 44 18H42.7C41.2 10.2 34.7 4 26.5 4 18 4 11.6 10.1 11 18.3 6.5 19.6 3 23.8 3 29s4.925 11 11 11h8Z" '
f'fill="{accent}" stroke="{color}" stroke-width="3" stroke-linejoin="round"/>'
f'<g stroke="{color}" stroke-width="3" stroke-linecap="round">'
'<line x1="20" y1="48" x2="24" y2="56"/>'
'<line x1="30" y1="50" x2="34" y2="58"/>'
'<line x1="40" y1="48" x2="44" y2="56"/>'
"</g>"
"</svg>"
)
def _storm_svg() -> str:
"""Generate SVG for stormy weather icon."""
color = WEATHER_ICON_COLOR
accent = WEATHER_ICON_ACCENT
return (
'<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64" fill="none">'
f'<path d="M22 40H44C50.075 40 55 35.075 55 29S50.075 18 44 18H42.7C41.2 10.2 34.7 4 26.5 4 18 4 11.6 10.1 11 18.3 6.5 19.6 3 23.8 3 29s4.925 11 11 11h8Z" '
f'fill="{accent}" stroke="{color}" stroke-width="3" stroke-linejoin="round"/>'
f'<path d="M34 46L28 56H34L30 64L42 50H36L40 46Z" '
f'fill="{color}" stroke="{color}" stroke-width="2" stroke-linejoin="round"/>'
"</svg>"
)
def _snow_svg() -> str:
"""Generate SVG for snowy weather icon."""
color = WEATHER_ICON_COLOR
accent = WEATHER_ICON_ACCENT
return (
'<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64" fill="none">'
f'<path d="M22 40H44C50.075 40 55 35.075 55 29S50.075 18 44 18H42.7C41.2 10.2 34.7 4 26.5 4 18 4 11.6 10.1 11 18.3 6.5 19.6 3 23.8 3 29s4.925 11 11 11h8Z" '
f'fill="{accent}" stroke="{color}" stroke-width="3" stroke-linejoin="round"/>'
f'<g stroke="{color}" stroke-width="2" stroke-linecap="round">'
'<line x1="20" y1="48" x2="20" y2="56"/>'
'<line x1="17" y1="51" x2="23" y2="53"/>'
'<line x1="17" y1="53" x2="23" y2="51"/>'
'<line x1="36" y1="48" x2="36" y2="56"/>'
'<line x1="33" y1="51" x2="39" y2="53"/>'
'<line x1="33" y1="53" x2="39" y2="51"/>'
"</g>"
"</svg>"
)
def _fog_svg() -> str:
"""Generate SVG for foggy weather icon."""
color = WEATHER_ICON_COLOR
accent = WEATHER_ICON_ACCENT
return (
'<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64" fill="none">'
f'<path d="M22 40H44C50.075 40 55 35.075 55 29S50.075 18 44 18H42.7C41.2 10.2 34.7 4 26.5 4 18 4 11.6 10.1 11 18.3 6.5 19.6 3 23.8 3 29s4.925 11 11 11h8Z" '
f'fill="{accent}" stroke="{color}" stroke-width="3" stroke-linejoin="round"/>'
f'<g stroke="{color}" stroke-width="3" stroke-linecap="round">'
'<line x1="18" y1="50" x2="42" y2="50"/>'
'<line x1="24" y1="56" x2="48" y2="56"/>'
"</g>"
"</svg>"
)
def _encode_svg(svg: str) -> str:
"""Encode SVG as base64 data URI."""
encoded = base64.b64encode(svg.encode("utf-8")).decode("ascii")
return f"data:image/svg+xml;base64,{encoded}"
# Weather condition to icon mapping
WEATHER_ICONS = {
"sunny": _encode_svg(_sun_svg()),
"cloudy": _encode_svg(_cloud_svg()),
"rainy": _encode_svg(_rain_svg()),
"stormy": _encode_svg(_storm_svg()),
"snowy": _encode_svg(_snow_svg()),
"foggy": _encode_svg(_fog_svg()),
}
DEFAULT_WEATHER_ICON = _encode_svg(_cloud_svg())
@dataclass
class WeatherData:
"""Weather data container."""
location: str
condition: str
temperature: int
humidity: int
wind_speed: int
def render_weather_widget(data: WeatherData) -> WidgetRoot:
"""Render a weather widget from weather data.
Args:
data: WeatherData containing weather information
Returns:
A ChatKit WidgetRoot (Card) displaying the weather information
"""
# Get weather icon
weather_icon_src = WEATHER_ICONS.get(data.condition.lower(), DEFAULT_WEATHER_ICON)
# Build the widget
header = Box(
padding=5,
background="surface-tertiary",
children=[
Row(
justify="between",
align="center",
children=[
Col(
align="start",
gap=1,
children=[
Text(
value=data.location,
size="lg",
weight="semibold",
),
Text(
value="Current conditions",
color="tertiary",
size="xs",
),
],
),
Box(
padding=3,
radius="full",
background="blue-100",
children=[
Image(
src=weather_icon_src,
alt=data.condition,
size=28,
fit="contain",
)
],
),
],
),
Row(
align="start",
gap=4,
children=[
Title(
value=f"{data.temperature}°C",
size="lg",
weight="semibold",
),
Col(
align="start",
gap=1,
children=[
Text(
value=data.condition.title(),
color="secondary",
size="sm",
weight="medium",
),
],
),
],
),
],
)
# Details section
details = Box(
padding=5,
gap=4,
children=[
Text(value="Weather details", weight="semibold", size="sm"),
Row(
gap=3,
wrap="wrap",
children=[
_detail_chip("Humidity", f"{data.humidity}%"),
_detail_chip("Wind", f"{data.wind_speed} km/h"),
],
),
],
)
return Card(
key="weather",
padding=0,
children=[header, details],
)
def _detail_chip(label: str, value: str) -> Box:
"""Create a detail chip widget component."""
return Box(
padding=3,
radius="xl",
background="surface-tertiary",
width=150,
minWidth=150,
maxWidth=150,
minHeight=80,
maxHeight=80,
flex="0 0 auto",
children=[
Col(
align="stretch",
gap=2,
children=[
Text(value=label, size="xs", weight="medium", color="tertiary"),
Row(
justify="center",
margin={"top": 2},
children=[Text(value=value, weight="semibold", size="lg")],
),
],
)
],
)
def weather_widget_copy_text(data: WeatherData) -> str:
"""Generate plain text representation of weather data.
Args:
data: WeatherData containing weather information
Returns:
Plain text description for copy/paste functionality
"""
return (
f"Weather in {data.location}:\n"
f"• Condition: {data.condition.title()}\n"
f"• Temperature: {data.temperature}°C\n"
f"• Humidity: {data.humidity}%\n"
f"• Wind: {data.wind_speed} km/h"
)
def render_city_selector_widget() -> WidgetRoot:
"""Render an interactive city selector widget.
This widget displays popular cities as a visual selection interface.
Users can click or ask about any city to get weather information.
Returns:
A ChatKit WidgetRoot (Card) with city selection display
"""
# Create location icon SVG
location_icon = _encode_svg(
'<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64" fill="none">'
f'<path d="M32 8c-8.837 0-16 7.163-16 16 0 12 16 32 16 32s16-20 16-32c0-8.837-7.163-16-16-16z" '
f'fill="{WEATHER_ICON_ACCENT}" stroke="{WEATHER_ICON_COLOR}" stroke-width="3" stroke-linejoin="round"/>'
f'<circle cx="32" cy="24" r="6" fill="{WEATHER_ICON_COLOR}"/>'
"</svg>"
)
# Header section
header = Box(
padding=5,
background="surface-tertiary",
children=[
Row(
gap=3,
align="center",
children=[
Box(
padding=3,
radius="full",
background="blue-100",
children=[
Image(
src=location_icon,
alt="Location",
size=28,
fit="contain",
)
],
),
Col(
align="start",
gap=1,
children=[
Title(
value="Popular Cities",
size="md",
weight="semibold",
),
Text(
value="Select a city or ask about any location",
color="tertiary",
size="xs",
),
],
),
],
),
],
)
# Create city chips in a grid layout
city_chips: list[Button] = []
for city in POPULAR_CITIES:
# Create a button that sends an action to query weather for the selected city
chip = Button(
label=city["label"],
variant="outline",
size="md",
onClickAction=ActionConfig(
type="city_selected",
payload={"city_value": city["value"], "city_label": city["label"]},
handler="server", # Handle on server-side
),
)
city_chips.append(chip)
# Arrange in rows of 3
city_rows: list[Row] = []
for i in range(0, len(city_chips), 3):
row_chips: list[Button] = city_chips[i : i + 3]
city_rows.append(
Row(
gap=3,
wrap="wrap",
justify="start",
children=list(row_chips), # Convert to generic list
)
)
# Cities display section
cities_section = Box(
padding=5,
gap=3,
children=[
*city_rows,
Box(
padding=3,
radius="md",
background="blue-50",
children=[
Text(
value="💡 Click any city to get its weather, or ask about any other location!",
size="xs",
color="secondary",
),
],
),
],
)
return Card(
key="city_selector",
padding=0,
children=[header, cities_section],
)
def city_selector_copy_text() -> str:
"""Generate plain text representation of city selector.
Returns:
Plain text description for copy/paste functionality
"""
cities_list = "\n".join([f"{city['label']}" for city in POPULAR_CITIES])
return f"Popular cities (click to get weather):\n{cities_list}\n\nYou can also ask about weather in any other location!"
@@ -0,0 +1,119 @@
# Tools Examples
This folder contains examples demonstrating how to use AI functions (tools) with the Agent Framework. AI functions allow agents to interact with external systems, perform computations, and execute custom logic.
## Examples
| File | Description |
|------|-------------|
| [`ai_function_declaration_only.py`](ai_function_declaration_only.py) | Demonstrates how to create function declarations without implementations. Useful for testing agent reasoning about tool usage or when tools are defined elsewhere. Shows how agents request tool calls even when the tool won't be executed. |
| [`ai_function_from_dict_with_dependency_injection.py`](ai_function_from_dict_with_dependency_injection.py) | Shows how to create AI functions from dictionary definitions using dependency injection. The function implementation is injected at runtime during deserialization, enabling dynamic tool creation and configuration. Note: This serialization/deserialization feature is in active development. |
| [`ai_function_recover_from_failures.py`](ai_function_recover_from_failures.py) | Demonstrates graceful error handling when tools raise exceptions. Shows how agents receive error information and can recover from failures, deciding whether to retry or respond differently based on the exception. |
| [`ai_function_with_approval.py`](ai_function_with_approval.py) | Shows how to implement user approval workflows for function calls without using threads. Demonstrates both streaming and non-streaming approval patterns where users can approve or reject function executions before they run. |
| [`ai_function_with_approval_and_threads.py`](ai_function_with_approval_and_threads.py) | Demonstrates tool approval workflows using threads for automatic conversation history management. Shows how threads simplify approval workflows by automatically storing and retrieving conversation context. Includes both approval and rejection examples. |
| [`ai_function_with_max_exceptions.py`](ai_function_with_max_exceptions.py) | Shows how to limit the number of times a tool can fail with exceptions using `max_invocation_exceptions`. Useful for preventing expensive tools from being called repeatedly when they keep failing. |
| [`ai_function_with_max_invocations.py`](ai_function_with_max_invocations.py) | Demonstrates limiting the total number of times a tool can be invoked using `max_invocations`. Useful for rate-limiting expensive operations or ensuring tools are only called a specific number of times per conversation. |
| [`ai_functions_in_class.py`](ai_functions_in_class.py) | Shows how to use `ai_function` decorator with class methods to create stateful tools. Demonstrates how class state can control tool behavior dynamically, allowing you to adjust tool functionality at runtime by modifying class properties. |
## Key Concepts
### AI Function Features
- **Function Declarations**: Define tool schemas without implementations for testing or external tools
- **Dependency Injection**: Create tools from configurations with runtime-injected implementations
- **Error Handling**: Gracefully handle and recover from tool execution failures
- **Approval Workflows**: Require user approval before executing sensitive or important operations
- **Invocation Limits**: Control how many times tools can be called or fail
- **Stateful Tools**: Use class methods as tools to maintain state and dynamically control behavior
### Common Patterns
#### Basic Tool Definition
```python
from agent_framework import ai_function
from typing import Annotated
@ai_function
def my_tool(param: Annotated[str, "Description"]) -> str:
"""Tool description for the AI."""
return f"Result: {param}"
```
#### Tool with Approval
```python
@ai_function(approval_mode="always_require")
def sensitive_operation(data: Annotated[str, "Data to process"]) -> str:
"""This requires user approval before execution."""
return f"Processed: {data}"
```
#### Tool with Invocation Limits
```python
@ai_function(max_invocations=3)
def limited_tool() -> str:
"""Can only be called 3 times total."""
return "Result"
@ai_function(max_invocation_exceptions=2)
def fragile_tool() -> str:
"""Can only fail 2 times before being disabled."""
return "Result"
```
#### Stateful Tools with Classes
```python
class MyTools:
def __init__(self, mode: str = "normal"):
self.mode = mode
def process(self, data: Annotated[str, "Data to process"]) -> str:
"""Process data based on current mode."""
if self.mode == "safe":
return f"Safely processed: {data}"
return f"Processed: {data}"
# Create instance and use methods as tools
tools = MyTools(mode="safe")
agent = client.create_agent(tools=tools.process)
# Change behavior dynamically
tools.mode = "normal"
```
### Error Handling
When tools raise exceptions:
1. The exception is captured and sent to the agent as a function result
2. The agent receives the error message and can reason about what went wrong
3. The agent can retry with different parameters, use alternative tools, or explain the issue to the user
4. With invocation limits, tools can be disabled after repeated failures
### Approval Workflows
Two approaches for handling approvals:
1. **Without Threads**: Manually manage conversation context, including the query, approval request, and response in each iteration
2. **With Threads**: Thread automatically manages conversation history, simplifying the approval workflow
## Usage Tips
- Use **declaration-only** functions when you want to test agent reasoning without execution
- Use **dependency injection** for dynamic tool configuration and plugin architectures
- Implement **approval workflows** for operations that modify data, spend money, or require human oversight
- Set **invocation limits** to prevent runaway costs or infinite loops with expensive tools
- Handle **exceptions gracefully** to create robust agents that can recover from failures
- Use **class-based tools** when you need to maintain state or dynamically adjust tool behavior at runtime
## Running the Examples
Each example is a standalone Python script that can be run directly:
```bash
uv run python ai_function_with_approval.py
```
Make sure you have the necessary environment variables configured (like `OPENAI_API_KEY` or Azure credentials) before running the examples.
@@ -0,0 +1,75 @@
# Copyright (c) Microsoft. All rights reserved.
from agent_framework import AIFunction
from agent_framework.openai import OpenAIResponsesClient
"""
Example of how to create a function that only consists of a declaration without an implementation.
This is useful when you want the agent to use tools that are defined elsewhere or when you want
to test the agent's ability to reason about tool usage without executing them.
The only difference is that you provide an AIFunction without a function.
If you need a input_model, you can still provide that as well.
"""
async def main():
function_declaration = AIFunction[None, None](
name="get_current_time",
description="Get the current time in ISO 8601 format.",
)
agent = OpenAIResponsesClient().create_agent(
name="DeclarationOnlyToolAgent",
instructions="You are a helpful agent that uses tools.",
tools=function_declaration,
)
query = "What is the current time?"
print(f"User: {query}")
result = await agent.run(query)
print(f"Result: {result.to_json(indent=2)}\n")
"""
Expected result:
User: What is the current time?
Result: {
"type": "agent_run_response",
"messages": [
{
"type": "chat_message",
"role": {
"type": "role",
"value": "assistant"
},
"contents": [
{
"type": "function_call",
"call_id": "call_0flN9rfGLK8LhORy4uMDiRSC",
"name": "get_current_time",
"arguments": "{}",
"fc_id": "fc_0fd5f269955c589f016904c46584348195b84a8736e61248de"
}
],
"author_name": "DeclarationOnlyToolAgent",
"additional_properties": {}
}
],
"response_id": "resp_0fd5f269955c589f016904c462d5cc819599d28384ba067edc",
"created_at": "2025-10-31T15:14:58.000000Z",
"usage_details": {
"type": "usage_details",
"input_token_count": 63,
"output_token_count": 145,
"total_token_count": 208,
"openai.reasoning_tokens": 128
},
"additional_properties": {}
}
"""
if __name__ == "__main__":
import asyncio
asyncio.run(main())
@@ -0,0 +1,188 @@
# Copyright (c) Microsoft. All rights reserved.
import asyncio
from typing import Annotated
from agent_framework import FunctionCallContent, FunctionResultContent, ai_function
from agent_framework.openai import OpenAIResponsesClient
"""
Some tools are very expensive to run, so you may want to limit the number of times
it tries to call them and fails. This sample shows a tool that can only raise exceptions a
limited number of times.
"""
# we trick the AI into calling this function with 0 as denominator to trigger the exception
@ai_function(max_invocation_exceptions=1)
def safe_divide(
a: Annotated[int, "Numerator"],
b: Annotated[int, "Denominator"],
) -> str:
"""Divide two numbers can be used with 0 as denominator."""
try:
result = a / b # Will raise ZeroDivisionError
except ZeroDivisionError as exc:
print(f" Tool failed with error: {exc}")
raise
return f"{a} / {b} = {result}"
async def main():
# tools = Tools()
agent = OpenAIResponsesClient().create_agent(
name="ToolAgent",
instructions="Use the provided tools.",
tools=[safe_divide],
)
thread = agent.get_new_thread()
print("=" * 60)
print("Step 1: Call divide(10, 0) - tool raises exception")
response = await agent.run("Divide 10 by 0", thread=thread)
print(f"Response: {response.text}")
print("=" * 60)
print("Step 2: Call divide(100, 0) - will refuse to execute due to max_invocation_exceptions")
response = await agent.run("Divide 100 by 0", thread=thread)
print(f"Response: {response.text}")
print("=" * 60)
print(f"Number of tool calls attempted: {safe_divide.invocation_count}")
print(f"Number of tool calls failed: {safe_divide.invocation_exception_count}")
print("Replay the conversation:")
assert thread.message_store
assert thread.message_store.list_messages
for idx, msg in enumerate(await thread.message_store.list_messages()):
if msg.text:
print(f"{idx + 1} {msg.author_name or msg.role}: {msg.text} ")
for content in msg.contents:
if isinstance(content, FunctionCallContent):
print(
f"{idx + 1} {msg.author_name}: calling function: {content.name} with arguments: {content.arguments}"
)
if isinstance(content, FunctionResultContent):
print(f"{idx + 1} {msg.role}: {content.result if content.result else content.exception}")
"""
Expected Output:
============================================================
Step 1: Call divide(10, 0) - tool raises exception
Tool failed with error: division by zero
[2025-10-31 15:39:53 - /Users/edvan/Work/agent-framework/python/packages/core/agent_framework/_tools.py:718 - ERROR]
Function failed. Error: division by zero
Response: Division by zero is undefined in standard arithmetic. There is no finite value for 10 ÷ 0.
If you want alternatives:
- A valid example: 10 ÷ 2 = 5.
- To handle safely in code, you can check the denominator first (e.g., in Python: if b == 0:
handle error else: compute a/b).
- If youre curious about limits: as x → 0+, 10/x → +∞; as x → 0, 10/x → −∞; there is no finite limit.
Would you like me to show a safe division snippet in a specific language, or compute something else?
============================================================
Step 2: Call divide(100, 0) - will refuse to execute due to max_invocations
[2025-10-31 15:40:09 - /Users/edvan/Work/agent-framework/python/packages/core/agent_framework/_tools.py:718 - ERROR]
Function failed. Error: Function 'safe_divide' has reached its maximum exception limit, you tried to use this
tool too many times and it kept failing.
Response: Division by zero is undefined in standard arithmetic, so 100 ÷ 0 has no finite value.
If youre coding and want safe handling, here are quick patterns in a few languages:
- Python
def safe_divide(a, b):
if b == 0:
return None # or raise an exception
return a / b
safe_divide(100, 0) # -> None
- JavaScript
function safeDivide(a, b) {
if (b === 0) return undefined; // or throw
return a / b;
}
safeDivide(100, 0) // -> undefined
- Java
public static Double safeDivide(double a, double b) {
if (b == 0.0) throw new ArithmeticException("Divide by zero");
return a / b;
}
safeDivide(100, 0) // -> exception
- C/C++
double safeDivide(double a, double b) {
if (b == 0.0) return std::numeric_limits<double>::infinity(); // or handle error
return a / b;
}
Note: In many languages, dividing by zero with floating-point numbers yields Infinity (or -Infinity) or NaN,
but integer division typically raises an error.
Would you like a snippet in a specific language or to see a math explanation (limits) for what happens as the
divisor approaches zero?
============================================================
Number of tool calls attempted: 1
Number of tool calls failed: 1
Replay the conversation:
1 user: Divide 10 by 0
2 ToolAgent: calling function: safe_divide with arguments: {"a":10,"b":0}
3 tool: division by zero
4 ToolAgent: Division by zero is undefined in standard arithmetic. There is no finite value for 10 ÷ 0.
If you want alternatives:
- A valid example: 10 ÷ 2 = 5.
- To handle safely in code, you can check the denominator first (e.g., in Python: if b == 0:
handle error else: compute a/b).
- If youre curious about limits: as x → 0+, 10/x → +∞; as x → 0, 10/x → −∞; there is no finite limit.
Would you like me to show a safe division snippet in a specific language, or compute something else?
5 user: Divide 100 by 0
6 ToolAgent: calling function: safe_divide with arguments: {"a":100,"b":0}
7 tool: Function 'safe_divide' has reached its maximum exception limit, you tried to use this tool too many times
and it kept failing.
8 ToolAgent: Division by zero is undefined in standard arithmetic, so 100 ÷ 0 has no finite value.
If youre coding and want safe handling, here are quick patterns in a few languages:
- Python
def safe_divide(a, b):
if b == 0:
return None # or raise an exception
return a / b
safe_divide(100, 0) # -> None
- JavaScript
function safeDivide(a, b) {
if (b === 0) return undefined; // or throw
return a / b;
}
safeDivide(100, 0) // -> undefined
- Java
public static Double safeDivide(double a, double b) {
if (b == 0.0) throw new ArithmeticException("Divide by zero");
return a / b;
}
safeDivide(100, 0) // -> exception
- C/C++
double safeDivide(double a, double b) {
if (b == 0.0) return std::numeric_limits<double>::infinity(); // or handle error
return a / b;
}
Note: In many languages, dividing by zero with floating-point numbers yields Infinity (or -Infinity) or NaN,
but integer division typically raises an error.
Would you like a snippet in a specific language or to see a math explanation (limits) for what happens as the
divisor approaches zero?
"""
if __name__ == "__main__":
asyncio.run(main())
@@ -0,0 +1,89 @@
# Copyright (c) Microsoft. All rights reserved.
import asyncio
from typing import Annotated
from agent_framework import FunctionCallContent, FunctionResultContent, ai_function
from agent_framework.openai import OpenAIResponsesClient
"""
For tools you can specify if there is a maximum number of invocations allowed.
This sample shows a tool that can only be invoked once.
"""
@ai_function(max_invocations=1)
def unicorn_function(times: Annotated[int, "The number of unicorns to return."]) -> str:
"""This function returns precious unicorns!"""
return f"{'🦄' * times}"
async def main():
# tools = Tools()
agent = OpenAIResponsesClient().create_agent(
name="ToolAgent",
instructions="Use the provided tools.",
tools=[unicorn_function],
)
thread = agent.get_new_thread()
print("=" * 60)
print("Step 1: Call unicorn_function")
response = await agent.run("Call 5 unicorns!", thread=thread)
print(f"Response: {response.text}")
print("=" * 60)
print("Step 2: Call unicorn_function again - will refuse to execute due to max_invocations")
response = await agent.run("Call 10 unicorns and use the function to do it.", thread=thread)
print(f"Response: {response.text}")
print("=" * 60)
print(f"Number of tool calls attempted: {unicorn_function.invocation_count}")
print(f"Number of tool calls failed: {unicorn_function.invocation_exception_count}")
print("Replay the conversation:")
assert thread.message_store
assert thread.message_store.list_messages
for idx, msg in enumerate(await thread.message_store.list_messages()):
if msg.text:
print(f"{idx + 1} {msg.author_name or msg.role}: {msg.text} ")
for content in msg.contents:
if isinstance(content, FunctionCallContent):
print(
f"{idx + 1} {msg.author_name}: calling function: {content.name} with arguments: {content.arguments}"
)
if isinstance(content, FunctionResultContent):
print(f"{idx + 1} {msg.role}: {content.result if content.result else content.exception}")
"""
Expected Output:
============================================================
Step 1: Call unicorn_function
Response: Five unicorns summoned: 🦄🦄🦄🦄🦄✨
============================================================
Step 2: Call unicorn_function again - will refuse to execute due to max_invocations
[2025-10-31 15:54:40 - /Users/edvan/Work/agent-framework/python/packages/core/agent_framework/_tools.py:718 - ERROR]
Function failed. Error: Function 'unicorn_function' has reached its maximum invocation limit,
you can no longer use this tool.
Response: The unicorn function has reached its maximum invocation limit. I cant call it again right now.
Here are 10 unicorns manually: 🦄 🦄 🦄 🦄 🦄 🦄 🦄 🦄 🦄 🦄
Would you like me to try again later, or generate something else?
============================================================
Number of tool calls attempted: 1
Number of tool calls failed: 0
Replay the conversation:
1 user: Call 5 unicorns!
2 ToolAgent: calling function: unicorn_function with arguments: {"times":5}
3 tool: 🦄🦄🦄🦄🦄✨
4 ToolAgent: Five unicorns summoned: 🦄🦄🦄🦄🦄✨
5 user: Call 10 unicorns and use the function to do it.
6 ToolAgent: calling function: unicorn_function with arguments: {"times":10}
7 tool: Function 'unicorn_function' has reached its maximum invocation limit, you can no longer use this tool.
8 ToolAgent: The unicorn function has reached its maximum invocation limit. I cant call it again right now.
Here are 10 unicorns manually: 🦄 🦄 🦄 🦄 🦄 🦄 🦄 🦄 🦄 🦄
Would you like me to try again later, or generate something else?
"""
if __name__ == "__main__":
asyncio.run(main())
@@ -0,0 +1,100 @@
# Copyright (c) Microsoft. All rights reserved.
import asyncio
from typing import Annotated
from agent_framework import ai_function
from agent_framework.openai import OpenAIResponsesClient
"""
This sample demonstrates using ai_function within a class,
showing how to manage state within the class that affects tool behavior.
And how to use ai_function-decorated methods as tools in an agent in order to adjust the behavior of a tool.
"""
class MyFunctionClass:
def __init__(self, safe: bool = False) -> None:
"""Simple class with two ai_functions: divide and add.
The safe parameter controls whether divide raises on division by zero or returns `infinity` for divide by zero.
"""
self.safe = safe
def divide(
self,
a: Annotated[int, "Numerator"],
b: Annotated[int, "Denominator"],
) -> str:
"""Divide two numbers, safe to use also with 0 as denominator."""
result = "" if b == 0 and self.safe else a / b
return f"{a} / {b} = {result}"
def add(
self,
x: Annotated[int, "First number"],
y: Annotated[int, "Second number"],
) -> str:
return f"{x} + {y} = {x + y}"
async def main():
# Creating my function class with safe division enabled
tools = MyFunctionClass(safe=True)
# Applying the ai_function decorator to one of the methods of the class
add_function = ai_function(description="Add two numbers.")(tools.add)
agent = OpenAIResponsesClient().create_agent(
name="ToolAgent",
instructions="Use the provided tools.",
)
print("=" * 60)
print("Step 1: Call divide(10, 0) - tool returns infinity")
query = "Divide 10 by 0"
response = await agent.run(
query,
tools=[add_function, tools.divide],
)
print(f"Response: {response.text}")
print("=" * 60)
print("Step 2: Call set safe to False and call again")
# Disabling safe mode to allow exceptions
tools.safe = False
response = await agent.run(query, tools=[add_function, tools.divide])
print(f"Response: {response.text}")
print("=" * 60)
"""
Expected Output:
============================================================
Step 1: Call divide(10, 0) - tool returns infinity
Response: Division by zero is undefined in standard arithmetic. There is no real number that equals 10 divided by 0.
- If you look at limits: as x → 0+ (denominator approaches 0 from the positive side), 10/x → +∞; as x → 0, 10/x → −∞.
- Some calculators may display "infinity" or give an error, but that's not a real number.
If you want a numeric surrogate, you can use a small nonzero denominator, e.g., 10/0.001 = 10000. Would you like to
see more on limits or handle it with a tiny epsilon?
============================================================
Step 2: Call set safe to False and call again
[2025-10-31 16:17:44 - /Users/edvan/Work/agent-framework/python/packages/core/agent_framework/_tools.py:718 - ERROR]
Function failed. Error: division by zero
Response: Division by zero is undefined in standard arithmetic. There is no number y such that 0 × y = 10.
If youre looking at limits:
- as x → 0+, 10/x → +∞
- as x → 0, 10/x → −∞
So the limit does not exist.
In programming, dividing by zero usually raises an error or results in special values (e.g., NaN or ∞) depending
on the language.
If you want, tell me what youd like to do instead (e.g., compute 10 divided by 2, or handle division by zero safely
in code), and I can help with examples.
============================================================
"""
if __name__ == "__main__":
asyncio.run(main())
@@ -0,0 +1,58 @@
# Copyright (c) Microsoft. All rights reserved.
import asyncio
from typing import Annotated
from agent_framework.openai import OpenAIResponsesClient
"""
This sample demonstrates how to configure function invocation settings
for an client and use a simple ai_function as a tool in an agent.
This behavior is the same for all chat client types.
"""
def add(
x: Annotated[int, "First number"],
y: Annotated[int, "Second number"],
) -> str:
return f"{x} + {y} = {x + y}"
async def main():
client = OpenAIResponsesClient()
if client.function_invocation_configuration is not None:
client.function_invocation_configuration.include_detailed_errors = True
client.function_invocation_configuration.max_iterations = 40
print(f"Function invocation configured as: \n{client.function_invocation_configuration.to_json(indent=2)}")
agent = client.create_agent(name="ToolAgent", instructions="Use the provided tools.", tools=add)
print("=" * 60)
print("Call add(239847293, 29834)")
query = "Add 239847293 and 29834"
response = await agent.run(query)
print(f"Response: {response.text}")
"""
Expected Output:
============================================================
Function invocation configured as:
{
"type": "function_invocation_configuration",
"enabled": true,
"max_iterations": 40,
"max_consecutive_errors_per_request": 3,
"terminate_on_unknown_calls": false,
"additional_tools": [],
"include_detailed_errors": true
}
============================================================
Call add(239847293, 29834)
Response: 239,877,127
"""
if __name__ == "__main__":
asyncio.run(main())