mirror of
https://github.com/microsoft/agent-framework.git
synced 2026-06-16 21:04:09 +08:00
Python: [BREAKING] Moved to a single get_response and run API (#3379)
* WIP * big update to new ResponseStream model * fixed tests and typing * fixed tests and typing * fixed tools typevar import * fix * mypy fix * mypy fixes and some cleanup * fix missing quoted names * and client * fix imports agui * fix anthropic override * fix agui * fix ag ui * fix import * fix anthropic types * fix mypy * refactoring * updated typing * fix 3.11 * fixes * redid layering of chat clients and agents * redid layering of chat clients and agents * Fix lint, type, and test issues after rebase - Add @overload decorators to AgentProtocol.run() for type compatibility - Add missing docstring params (middleware, function_invocation_configuration) - Fix TODO format (TD002) by adding author tags - Fix broken observability tests from upstream: - Replace non-existent use_instrumentation with direct instantiation - Replace non-existent use_agent_instrumentation with AgentTelemetryLayer mixin - Fix get_streaming_response to use get_response(stream=True) - Add AgentInitializationError import - Update streaming exception tests to match actual behavior * Fix AgentExecutionException import error in test_agents.py - Replace non-existent AgentExecutionException with AgentRunException * Fix test import and asyncio deprecation issues - Add 'tests' to pythonpath in ag-ui pyproject.toml for utils_test_ag_ui import - Replace deprecated asyncio.get_event_loop().run_until_complete with asyncio.run * Fix azure-ai test failures - Update _prepare_options patching to use correct class path - Fix test_to_azure_ai_agent_tools_web_search_missing_connection to clear env vars * Convert ag-ui utils_test_ag_ui.py to conftest.py - Move test utilities to conftest.py for proper pytest discovery - Update all test imports to use conftest instead of utils_test_ag_ui - Remove old utils_test_ag_ui.py file - Revert pythonpath change in pyproject.toml * fix: use relative imports for ag-ui test utilities * fix agui * Rename Bare*Client to Raw*Client and BaseChatClient - Renamed BareChatClient to BaseChatClient (abstract base class) - Renamed BareOpenAIChatClient to RawOpenAIChatClient - Renamed BareOpenAIResponsesClient to RawOpenAIResponsesClient - Renamed BareAzureAIClient to RawAzureAIClient - Added warning docstrings to Raw* classes about layer ordering - Updated README in samples/getting_started/agents/custom with layer docs - Added test for span ordering with function calling * Fix layer ordering: FunctionInvocationLayer before ChatTelemetryLayer This ensures each inner LLM call gets its own telemetry span, resulting in the correct span sequence: chat -> execute_tool -> chat Updated all production clients and test mocks to use correct ordering: - ChatMiddlewareLayer (first) - FunctionInvocationLayer (second) - ChatTelemetryLayer (third) - BaseChatClient/Raw...Client (fourth) * Remove run_stream usage * Fix conversation_id propagation * Python: Add BaseAgent implementation for Claude Agent SDK (#3509) * Added ClaudeAgent implementation * Updated streaming logic * Small updates * Small update * Fixes * Small fix * Naming improvements * Updated imports * Addressed comments * Updated package versions * Update Claude agent connector layering * fix test and plugin * Store function middleware in invocation layer * Fix telemetry streaming and ag-ui tests * Remove legacy ag-ui tests folder * updates * Remove terminate flag from FunctionInvocationContext, use MiddlewareTermination instead - Remove terminate attribute from FunctionInvocationContext - Add result attribute to MiddlewareTermination to carry function results - FunctionMiddlewarePipeline.execute() now lets MiddlewareTermination propagate - _auto_invoke_function captures context.result in exception before re-raising - _try_execute_function_calls catches MiddlewareTermination and sets should_terminate - Fix handoff middleware to append to chat_client.function_middleware directly - Update tests to use raise MiddlewareTermination instead of context.terminate - Add middleware flow documentation in samples/concepts/tools/README.md - Fix ag-ui to use FunctionMiddlewarePipeline instead of removed create_function_middleware_pipeline * fix: remove references to removed terminate flag in purview tests, add type ignore * fix: move _test_utils.py from package to test folder * fix: call get_final_response() to trigger context provider notification in streaming test * fix: correct broken links in tools README * docs: clarify default middleware behavior in summary table * fix: ensure inner stream result hooks are called when using map()/from_awaitable() * Fix mypy type errors * Address PR review comments on observability.py - Remove TODO comment about unconsumed streams, add explanatory note instead - Remove redundant _close_span cleanup hook (already called in _finalize_stream) - Clarify behavior: cleanup hooks run after stream iteration, if stream is not consumed the span remains open until garbage collected * Remove gen_ai.client.operation.duration from span attributes Duration is a metrics-only attribute per OpenTelemetry semantic conventions. It should be recorded to the histogram but not set as a span attribute. * Remove duration from _get_response_attributes, pass directly to _capture_response Duration is a metrics-only attribute. It's now passed directly to _capture_response instead of being included in the attributes dict that gets set on the span. * Remove redundant _close_span cleanup hook in AgentTelemetryLayer _finalize_stream already calls _close_span() in its finally block, so adding it as a separate cleanup hook is redundant. * Use weakref.finalize to close span when stream is garbage collected If a user creates a streaming response but never consumes it, the cleanup hooks won't run. Now we register a weak reference finalizer that will close the span when the stream object is garbage collected, ensuring spans don't leak in this scenario. * Fix _get_finalizers_from_stream to use _result_hooks attribute Renamed function to _get_result_hooks_from_stream and fixed it to look for the _result_hooks attribute which is the correct name in ResponseStream class. * Add missing asyncio import in test_request_info_mixin.py * Fix leftover merge conflict marker in image_generation sample * Update integration tests * Fix integration tests: increase max_iterations from 1 to 2 Tests with tool_choice options require at least 2 iterations: 1. First iteration to get function call and execute the tool 2. Second iteration to get the final text response With max_iterations=1, streaming tests would return early with only the function call/result but no final text content. * Fix duplicate function call error in conversation-based APIs When using conversation_id (for Responses/Assistants APIs), the server already has the function call message from the previous response. We should only send the new function result message, not all messages including the function call which would cause a duplicate ID error. Fix: When conversation_id is set, only send the last message (the tool result) instead of all response.messages. * Add regression test for conversation_id propagation between tool iterations Port test from PR #3664 with updates for new streaming API pattern. Tests that conversation_id is properly updated in options dict during function invocation loop iterations. * Fix tool_choice=required to return after tool execution When tool_choice is 'required', the user's intent is to force exactly one tool call. After the tool executes, return immediately with the function call and result - don't continue to call the model again. This fixes integration tests that were failing with empty text responses because with tool_choice=required, the model would keep returning function calls instead of text. Also adds regression tests for: - conversation_id propagation between tool iterations (from PR #3664) - tool_choice=required returns after tool execution * Document tool_choice behavior in tools README - Add table explaining tool_choice values (auto, none, required) - Explain why tool_choice=required returns immediately after tool execution - Add code example showing the difference between required and auto - Update flow diagram to show the early return path for tool_choice=required * Fix tool_choice=None behavior - don't default to 'auto' Remove the hardcoded default of 'auto' for tool_choice in ChatAgent init. When tool_choice is not specified (None), it will now not be sent to the API, allowing the API's default behavior to be used. Users who want tool_choice='auto' can still explicitly set it either in default_options or at runtime. Fixes #3585 * Fix tool_choice=none should not remove tools In OpenAI Assistants client, tools were not being sent when tool_choice='none'. This was incorrect - tool_choice='none' means the model won't call tools, but tools should still be available in the request (they may be used later in the conversation). Fixes #3585 * Add test for tool_choice=none preserving tools Adds a regression test to ensure that when tool_choice='none' is set but tools are provided, the tools are still sent to the API. This verifies the fix for #3585. * Fix tool_choice=none should not remove tools in all clients Apply the same fix to OpenAI Responses client and Azure AI client: - OpenAI Responses: Remove else block that popped tool_choice/parallel_tool_calls - Azure AI: Remove tool_choice != 'none' check when adding tools When tool_choice='none', the model won't call tools, but tools should still be sent to the API so they're available for future turns. Also update README to clarify tool_choice=required supports multiple tools. Fixes #3585 * Keep tool_choice even when tools is None Move tool_choice processing outside of the 'if tools' block in OpenAI Responses client so tool_choice is sent to the API even when no tools are provided. * Update test to match new parallel_tool_calls behavior Changed test_prepare_options_removes_parallel_tool_calls_when_no_tools to test_prepare_options_preserves_parallel_tool_calls_when_no_tools to reflect that parallel_tool_calls is now preserved even when no tools are present, consistent with the tool_choice behavior. * Fix ChatMessage API and Role enum usage after rebase - Update ChatMessage instantiation to use keyword args (role=, text=, contents=) - Fix Role enum comparisons to use .value for string comparison - Add created_at to AgentResponse in error handling - Fix AgentResponse.from_updates -> from_agent_run_response_updates - Fix DurableAgentStateMessage.from_chat_message to convert Role enum to string - Add Role import where needed * Fix additional ChatMessage API and method name changes - Fix ChatMessage usage in workflow files (use text= instead of contents= for strings) - Fix AgentResponse.from_updates -> from_agent_run_response_updates in workflow files - Fix test files for ChatMessage and Role enum usage * Fix remaining ChatMessage API usage in test files * Fix more ChatMessage and Role API changes in source and test files - Fix ChatMessage in _magentic.py replan method - Fix Role enum comparison in test assertions - Fix remaining test files with old ChatMessage syntax * Fix ChatMessage and Role API changes across packages - Add Role import where missing - Fix ChatMessage signature: positional args to keyword args (role=, text=, contents=) - Fix Role enum comparisons: .role.value instead of .role string - Fix FinishReason enum usage in ag-ui event converters - Rename AgentResponse.from_updates to from_agent_run_response_updates in ag-ui Fixes API compatibility after Types API Review improvements merge * Fix ChatMessage and Role API changes in github_copilot tests * Fix ChatMessage and Role API changes in redis and github_copilot packages - Fix redis provider: Role enum comparison using .value - Fix redis tests: ChatMessage signature and Role comparisons - Fix github_copilot tests: ChatMessage signature and Role comparisons - Update docstring examples in redis chat message store * Fix ChatMessage and Role API changes in devui package - Fix executor: ChatMessage signature change - Fix conversations: Role enum to string conversion in two places - Fix tests: ChatMessage signatures and Role comparisons * Fix ChatMessage and Role API changes in a2a and lab packages - Fix a2a tests: Role comparisons and ChatMessage signatures - Fix lab tau2 source: Role enum comparison in flip_messages, log_messages, sliding_window - Fix lab tau2 tests: ChatMessage signatures and Role comparisons * Remove duplicate test files from ag-ui/tests (tests are in ag_ui_tests) * Fix ChatMessage and Role API changes across packages After rebasing on upstream/main which merged PR #3647 (Types API Review improvements), fix all packages to use the new API: - ChatMessage: Use keyword args (role=, text=, contents=) instead of positional args - Role: Compare using .value attribute since it's now an enum Packages fixed: - ag-ui: Fixed Role value extraction bugs in _message_adapters.py - anthropic: Fixed ChatMessage and Role comparisons in tests - azure-ai: Fixed Role comparison in _client.py - azure-ai-search: Fixed ChatMessage and Role in source/tests - bedrock: Fixed ChatMessage signatures in tests - chatkit: Fixed ChatMessage and Role in source/tests - copilotstudio: Fixed ChatMessage and Role in tests - declarative: Fixed ChatMessage in _executors_agents.py - mem0: Fixed ChatMessage and Role in source/tests - purview: Fixed ChatMessage in source/tests * Fix mypy errors for ChatMessage and Role API changes - durabletask: Use str() fallback in role value extraction - core: Fix ChatMessage in _orchestrator_helpers.py to use keyword args - core: Add type ignore for _conversation_state.py contents deserialization - ag-ui: Fix type ignore comments (call-overload instead of arg-type) - azure-ai-search: Fix get_role_value type hint to accept Any - lab: Move get_role_value to module level with Any type hint * Improve CI test timeout configuration - Increase job timeout from 10 to 15 minutes - Reduce per-test timeout to 60s (was 900s/300s) - Add --timeout_method thread for better timeout handling - Add --timeout-verbose to see which tests are slow - Reduce retries from 3 to 2 and delay from 10s to 5s This ensures individual test timeouts are shorter than the job timeout, providing better visibility when tests hang. With 60s timeout and 2 retries, worst case per test is ~180s. * Fix ChatMessage API usage in docstrings and source - Fix ChatMessage positional args in docstrings: _serialization.py, _threads.py, _middleware.py - Fix ChatMessage in tau2 runner.py - Fix role comparison in _orchestrator_helpers.py to use .value - Fix role comparison in _group_chat.py docstring example - Fix role assertions in test_durable_entities.py to use .value * Revert tool_choice/parallel_tool_calls changes - must be removed when no tools OpenAI API requires tool_choice and parallel_tool_calls to only be present when tools are specified. Restored the logic that removes these options when there are no tools. - Restored check in _chat_client.py to remove tool_choice and parallel_tool_calls when no tools present - Restored same logic in _responses_client.py - Reverted test to expect the correct behavior * fixed issue in tests * fix: resolve merge conflict markers in ag-ui tests * fix: restructure ag-ui tests and fix Role/FinishReason to use string types * fix: streaming function invocation and middleware termination - Refactor streaming function invocation to use get_final_response() on inner streams - Fix MiddlewareTermination to accept result parameter for passing results - Fix _AutoHandoffMiddleware to use MiddlewareTermination instead of context.terminate - Fix AgentMiddlewareLayer.run() to properly forward function/chat middleware - Remove duplicate middleware registration in AgentMiddlewareLayer.__init__ - Fix exception handling in _auto_invoke_function to properly capture termination - Fix mypy errors in core package - Update tests to use stream=True parameter for unified run API * fix all tests command * Refactor integration tests to use pytest fixtures - Merge testutils.py into conftest.py for azurefunctions integration tests - Merge dt_testutils.py into conftest.py for durabletask integration tests - Convert all integration tests to use fixtures instead of direct imports (fixes ModuleNotFoundError with --import-mode=importlib) - Add sample_helper fixture for azurefunctions tests - Add agent_client_factory and orchestration_helper fixtures for durabletask - Integration tests now skip with descriptive messages when services unavailable - Restructure devui tests into tests/devui/ with proper conftest.py - Add test organization guidelines to CODING_STANDARD.md - Remove __init__.py from test directories per pytest best practices * Fix pytest_collection_modifyitems to only skip integration tests The hook was skipping all tests in the test session, not just integration tests. Now it only skips items in the integration_tests directory. * Fix mem0 tests failing on Python 3.13 Use patch.object on the imported module instead of @patch with string path to ensure the mock takes effect regardless of import timing. * fix mem0 * another attempt for mem0 * fix for mem0 * fix mem0 * Increase worker initialization wait time in durabletask tests Increase from 2 to 8 seconds to allow time for: - Python startup and module imports - Azure OpenAI client creation - Agent registration with DTS worker - Worker connection to DTS This helps prevent test failures in CI where the first tests may run before the worker is fully ready to process requests. * Fix streaming test to use ResponseStream with finalizer The _consume_stream method now expects a ResponseStream that can provide a final AgentResponse via get_final_response(). Update the test to use ResponseStream with AgentResponse.from_updates as the finalizer. * Fix MockToolCallingAgent to use new ResponseStream API and update samples * small updates to run_stream to run * fix sub workflow * temp fix for az func test --------- Co-authored-by: Dmytro Struk <13853051+dmytrostruk@users.noreply.github.com>
This commit is contained in:
Eduard van Valkenburg
committed by
GitHub
Unverified
GitHub
parent
d1205896a1
commit
3dc59c83b5
@@ -95,7 +95,7 @@ This directory contains samples demonstrating the capabilities of Microsoft Agen
|
||||
| File | Description |
|
||||
|------|-------------|
|
||||
| [`getting_started/agents/custom/custom_agent.py`](./getting_started/agents/custom/custom_agent.py) | Custom Agent Implementation Example |
|
||||
| [`getting_started/agents/custom/custom_chat_client.py`](./getting_started/agents/custom/custom_chat_client.py) | Custom Chat Client Implementation Example |
|
||||
| [`getting_started/chat_client/custom_chat_client.py`](./getting_started/chat_client/custom_chat_client.py) | Custom Chat Client Implementation Example |
|
||||
|
||||
### Ollama
|
||||
|
||||
|
||||
@@ -52,7 +52,7 @@ python samples/autogen-migration/orchestrations/04_magentic_one.py
|
||||
## Tips for Migration
|
||||
|
||||
- **Default behavior differences**: AutoGen's `AssistantAgent` is single-turn by default (`max_tool_iterations=1`), while AF's `ChatAgent` is multi-turn and continues tool execution automatically.
|
||||
- **Thread management**: AF agents are stateless by default. Use `agent.get_new_thread()` and pass it to `run()`/`run_stream()` to maintain conversation state, similar to AutoGen's conversation context.
|
||||
- **Thread management**: AF agents are stateless by default. Use `agent.get_new_thread()` and pass it to `run()` to maintain conversation state, similar to AutoGen's conversation context.
|
||||
- **Tools**: AutoGen uses `FunctionTool` wrappers; AF uses `@tool` decorators with automatic schema inference.
|
||||
- **Orchestration patterns**:
|
||||
- `RoundRobinGroupChat` → `SequentialBuilder` or `WorkflowBuilder`
|
||||
|
||||
@@ -82,7 +82,7 @@ async def run_agent_framework() -> None:
|
||||
# Run the workflow
|
||||
print("[Agent Framework] Sequential conversation:")
|
||||
current_executor = None
|
||||
async for event in workflow.run_stream("Create a brief summary about electric vehicles"):
|
||||
async for event in workflow.run("Create a brief summary about electric vehicles", stream=True):
|
||||
if isinstance(event, WorkflowOutputEvent):
|
||||
# Print executor name header when switching to a new agent
|
||||
if current_executor != event.executor_id:
|
||||
@@ -153,7 +153,7 @@ async def run_agent_framework_with_cycle() -> None:
|
||||
# Run the workflow
|
||||
print("[Agent Framework with Cycle] Cyclic conversation:")
|
||||
current_executor = None
|
||||
async for event in workflow.run_stream("Create a brief summary about electric vehicles"):
|
||||
async for event in workflow.run("Create a brief summary about electric vehicles", stream=True):
|
||||
if isinstance(event, WorkflowOutputEvent) and isinstance(event.data, AgentResponseUpdate):
|
||||
# Print executor name header when switching to a new agent
|
||||
if current_executor != event.executor_id:
|
||||
|
||||
@@ -101,7 +101,7 @@ async def run_agent_framework() -> None:
|
||||
# Run with a question that requires expert selection
|
||||
print("[Agent Framework] Group chat conversation:")
|
||||
current_executor = None
|
||||
async for event in workflow.run_stream("How do I connect to a PostgreSQL database using Python?"):
|
||||
async for event in workflow.run("How do I connect to a PostgreSQL database using Python?", stream=True):
|
||||
if isinstance(event, WorkflowOutputEvent) and isinstance(event.data, AgentResponseUpdate):
|
||||
# Print executor name header when switching to a new agent
|
||||
if current_executor != event.executor_id:
|
||||
|
||||
@@ -161,7 +161,7 @@ async def run_agent_framework() -> None:
|
||||
stream_line_open = False
|
||||
pending_requests: list[RequestInfoEvent] = []
|
||||
|
||||
async for event in workflow.run_stream(scripted_responses[0]):
|
||||
async for event in workflow.run(scripted_responses[0], stream=True):
|
||||
if isinstance(event, WorkflowOutputEvent) and isinstance(event.data, AgentResponseUpdate):
|
||||
# Print executor name header when switching to a new agent
|
||||
if current_executor != event.executor_id:
|
||||
|
||||
@@ -112,7 +112,7 @@ async def run_agent_framework() -> None:
|
||||
last_message_id: str | None = None
|
||||
output_event: WorkflowOutputEvent | None = None
|
||||
print("[Agent Framework] Magentic conversation:")
|
||||
async for event in workflow.run_stream("Research Python async patterns and write a simple example"):
|
||||
async for event in workflow.run("Research Python async patterns and write a simple example", stream=True):
|
||||
if isinstance(event, WorkflowOutputEvent) and isinstance(event.data, AgentResponseUpdate):
|
||||
message_id = event.data.message_id
|
||||
if message_id != last_message_id:
|
||||
|
||||
+2
-2
@@ -32,7 +32,7 @@ async def run_autogen() -> None:
|
||||
|
||||
print("\n[AutoGen] Streaming response:")
|
||||
# Stream response with Console for token streaming
|
||||
await Console(agent.run_stream(task="Count from 1 to 5"))
|
||||
await Console(agent.run(task="Count from 1 to 5", stream=True))
|
||||
|
||||
|
||||
async def run_agent_framework() -> None:
|
||||
@@ -60,7 +60,7 @@ async def run_agent_framework() -> None:
|
||||
print("\n[Agent Framework] Streaming response:")
|
||||
# Stream response
|
||||
print(" ", end="")
|
||||
async for chunk in agent.run_stream("Count from 1 to 5"):
|
||||
async for chunk in agent.run("Count from 1 to 5", thread=thread, stream=True):
|
||||
if chunk.text:
|
||||
print(chunk.text, end="", flush=True)
|
||||
print()
|
||||
|
||||
@@ -43,7 +43,7 @@ async def run_autogen() -> None:
|
||||
|
||||
# Run coordinator with streaming - it will delegate to writer
|
||||
print("[AutoGen]")
|
||||
await Console(coordinator.run_stream(task="Create a tagline for a coffee shop"))
|
||||
await Console(coordinator.run(task="Create a tagline for a coffee shop", stream=True))
|
||||
|
||||
|
||||
async def run_agent_framework() -> None:
|
||||
@@ -80,7 +80,7 @@ async def run_agent_framework() -> None:
|
||||
# Track accumulated function calls (they stream in incrementally)
|
||||
accumulated_calls: dict[str, FunctionCallContent] = {}
|
||||
|
||||
async for chunk in coordinator.run_stream("Create a tagline for a coffee shop"):
|
||||
async for chunk in coordinator.run("Create a tagline for a coffee shop", stream=True):
|
||||
# Stream text tokens
|
||||
if chunk.text:
|
||||
print(chunk.text, end="", flush=True)
|
||||
|
||||
@@ -0,0 +1,10 @@
|
||||
# Concept Samples
|
||||
|
||||
This folder contains samples that dive deep into specific Agent Framework concepts.
|
||||
|
||||
## Samples
|
||||
|
||||
| Sample | Description |
|
||||
|--------|-------------|
|
||||
| [response_stream.py](response_stream.py) | Deep dive into `ResponseStream` - the streaming abstraction for AI responses. Covers the four hook types (transform hooks, cleanup hooks, finalizer, result hooks), two consumption patterns (iteration vs direct finalization), and the `wrap()` API for layering streams without double-consumption. |
|
||||
| [typed_options.py](typed_options.py) | Demonstrates TypedDict-based chat options for type-safe configuration with IDE autocomplete support. |
|
||||
@@ -0,0 +1,360 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
import asyncio
|
||||
from collections.abc import AsyncIterable, Sequence
|
||||
|
||||
from agent_framework import ChatResponse, ChatResponseUpdate, Content, ResponseStream, Role
|
||||
|
||||
"""ResponseStream: A Deep Dive
|
||||
|
||||
This sample explores the ResponseStream class - a powerful abstraction for working with
|
||||
streaming responses in the Agent Framework.
|
||||
|
||||
=== Why ResponseStream Exists ===
|
||||
|
||||
When working with AI models, responses can be delivered in two ways:
|
||||
1. **Non-streaming**: Wait for the complete response, then return it all at once
|
||||
2. **Streaming**: Receive incremental updates as they're generated
|
||||
|
||||
Streaming provides a better user experience (faster time-to-first-token, progressive rendering)
|
||||
but introduces complexity:
|
||||
- How do you process updates as they arrive?
|
||||
- How do you also get a final, complete response?
|
||||
- How do you ensure the underlying stream is only consumed once?
|
||||
- How do you add custom logic (hooks) at different stages?
|
||||
|
||||
ResponseStream solves all these problems by wrapping an async iterable and providing:
|
||||
- Multiple consumption patterns (iteration OR direct finalization)
|
||||
- Hook points for transformation, cleanup, finalization, and result processing
|
||||
- The `wrap()` API to layer behavior without double-consuming the stream
|
||||
|
||||
=== The Four Hook Types ===
|
||||
|
||||
ResponseStream provides four ways to inject custom logic. All can be passed via constructor
|
||||
or added later via fluent methods:
|
||||
|
||||
1. **Transform Hooks** (`transform_hooks=[]` or `.with_transform_hook()`)
|
||||
- Called for EACH update as it's yielded during iteration
|
||||
- Can transform updates before they're returned to the consumer
|
||||
- Multiple hooks are called in order, each receiving the previous hook's output
|
||||
- Only triggered during iteration (not when calling get_final_response directly)
|
||||
|
||||
2. **Cleanup Hooks** (`cleanup_hooks=[]` or `.with_cleanup_hook()`)
|
||||
- Called ONCE when iteration completes (stream fully consumed), BEFORE finalizer
|
||||
- Used for cleanup: closing connections, releasing resources, logging
|
||||
- Cannot modify the stream or response
|
||||
- Triggered regardless of how the stream ends (normal completion or exception)
|
||||
|
||||
3. **Finalizer** (`finalizer=` constructor parameter)
|
||||
- Called ONCE when `get_final_response()` is invoked
|
||||
- Receives the list of collected updates and converts to the final type
|
||||
- There is only ONE finalizer per stream (set at construction)
|
||||
|
||||
4. **Result Hooks** (`result_hooks=[]` or `.with_result_hook()`)
|
||||
- Called ONCE after the finalizer produces its result
|
||||
- Transform the final response before returning
|
||||
- Multiple result hooks are called in order, each receiving the previous result
|
||||
- Can return None to keep the previous value unchanged
|
||||
|
||||
=== Two Consumption Patterns ===
|
||||
|
||||
**Pattern 1: Async Iteration**
|
||||
```python
|
||||
async for update in response_stream:
|
||||
print(update.text) # Process each update
|
||||
# Stream is now consumed; updates are stored internally
|
||||
```
|
||||
- Transform hooks are called for each yielded item
|
||||
- Cleanup hooks are called after the last item
|
||||
- The stream collects all updates internally for later finalization
|
||||
- Does not run the finalizer automatically
|
||||
|
||||
**Pattern 2: Direct Finalization**
|
||||
```python
|
||||
final = await response_stream.get_final_response()
|
||||
```
|
||||
- If the stream hasn't been iterated, it auto-iterates (consuming all updates)
|
||||
- The finalizer converts collected updates to a final response
|
||||
- Result hooks transform the response
|
||||
- You get the complete response without ever seeing individual updates
|
||||
|
||||
** Pattern 3: Combined Usage **
|
||||
|
||||
When you first iterate the stream and then call `get_final_response()`, the following occurs:
|
||||
- Iteration yields updates with transform hooks applied
|
||||
- Cleanup hooks run after iteration completes
|
||||
- Calling `get_final_response()` uses the already collected updates to produce the final response
|
||||
- Note that it does not re-iterate the stream since it's already been consumed
|
||||
|
||||
```python
|
||||
async for update in response_stream:
|
||||
print(update.text) # See each update
|
||||
final = await response_stream.get_final_response() # Get the aggregated result
|
||||
```
|
||||
|
||||
=== Chaining with .map() and .with_finalizer() ===
|
||||
|
||||
When building a ChatAgent on top of a ChatClient, we face a challenge:
|
||||
- The ChatClient returns a ResponseStream[ChatResponseUpdate, ChatResponse]
|
||||
- The ChatAgent needs to return a ResponseStream[AgentResponseUpdate, AgentResponse]
|
||||
- We can't iterate the ChatClient's stream twice!
|
||||
|
||||
The `.map()` and `.with_finalizer()` methods solve this by creating new ResponseStreams that:
|
||||
- Delegate iteration to the inner stream (only consuming it once)
|
||||
- Maintain their OWN separate transform hooks, result hooks, and cleanup hooks
|
||||
- Allow type-safe transformation of updates and final responses
|
||||
|
||||
**`.map(transform)`**: Creates a new stream that transforms each update.
|
||||
- Returns a new ResponseStream with the transformed update type
|
||||
- Falls back to the inner stream's finalizer if no new finalizer is set
|
||||
|
||||
**`.with_finalizer(finalizer)`**: Creates a new stream with a different finalizer.
|
||||
- Returns a new ResponseStream with the new final type
|
||||
- The inner stream's finalizer and result_hooks ARE still called (see below)
|
||||
|
||||
**IMPORTANT**: When chaining these methods via `get_final_response()`:
|
||||
1. The inner stream's finalizer runs first (on the original updates)
|
||||
2. The inner stream's result_hooks run (on the inner final result)
|
||||
3. The outer stream's finalizer runs (on the transformed updates)
|
||||
4. The outer stream's result_hooks run (on the outer final result)
|
||||
|
||||
This ensures that post-processing hooks registered on the inner stream (e.g., context
|
||||
provider notifications, telemetry, thread updates) are still executed even when the
|
||||
stream is wrapped/mapped.
|
||||
|
||||
```python
|
||||
# ChatAgent does something like this internally:
|
||||
chat_stream = chat_client.get_response(messages, stream=True)
|
||||
agent_stream = (
|
||||
chat_stream
|
||||
.map(_to_agent_update, _to_agent_response)
|
||||
.with_result_hook(_notify_thread) # Outer hook runs AFTER inner hooks
|
||||
)
|
||||
```
|
||||
|
||||
This ensures:
|
||||
- The underlying ChatClient stream is only consumed once
|
||||
- The agent can add its own transform hooks, result hooks, and cleanup logic
|
||||
- Each layer (ChatClient, ChatAgent, middleware) can add independent behavior
|
||||
- Inner stream post-processing (like context provider notification) still runs
|
||||
- Types flow naturally through the chain
|
||||
"""
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
"""Demonstrate the various ResponseStream patterns and capabilities."""
|
||||
|
||||
# =========================================================================
|
||||
# Example 1: Basic ResponseStream with iteration
|
||||
# =========================================================================
|
||||
print("=== Example 1: Basic Iteration ===\n")
|
||||
|
||||
async def generate_updates() -> AsyncIterable[ChatResponseUpdate]:
|
||||
"""Simulate a streaming response from an AI model."""
|
||||
words = ["Hello", " ", "from", " ", "the", " ", "streaming", " ", "response", "!"]
|
||||
for word in words:
|
||||
await asyncio.sleep(0.05) # Simulate network delay
|
||||
yield ChatResponseUpdate(contents=[Content.from_text(word)], role=Role.ASSISTANT)
|
||||
|
||||
def combine_updates(updates: Sequence[ChatResponseUpdate]) -> ChatResponse:
|
||||
"""Finalizer that combines all updates into a single response."""
|
||||
return ChatResponse.from_chat_response_updates(updates)
|
||||
|
||||
stream = ResponseStream(generate_updates(), finalizer=combine_updates)
|
||||
|
||||
print("Iterating through updates:")
|
||||
async for update in stream:
|
||||
print(f" Update: '{update.text}'")
|
||||
|
||||
# After iteration, we can still get the final response
|
||||
final = await stream.get_final_response()
|
||||
print(f"\nFinal response: '{final.text}'")
|
||||
|
||||
# =========================================================================
|
||||
# Example 2: Using get_final_response() without iteration
|
||||
# =========================================================================
|
||||
print("\n=== Example 2: Direct Finalization (No Iteration) ===\n")
|
||||
|
||||
# Create a fresh stream (streams can only be consumed once)
|
||||
stream2 = ResponseStream(generate_updates(), finalizer=combine_updates)
|
||||
|
||||
# Skip iteration entirely - get_final_response() auto-consumes the stream
|
||||
final2 = await stream2.get_final_response()
|
||||
print(f"Got final response directly: '{final2.text}'")
|
||||
print(f"Number of updates collected internally: {len(stream2.updates)}")
|
||||
|
||||
# =========================================================================
|
||||
# Example 3: Transform hooks - transform updates during iteration
|
||||
# =========================================================================
|
||||
print("\n=== Example 3: Transform Hooks ===\n")
|
||||
|
||||
update_count = {"value": 0}
|
||||
|
||||
def counting_hook(update: ChatResponseUpdate) -> ChatResponseUpdate:
|
||||
"""Hook that counts and annotates each update."""
|
||||
update_count["value"] += 1
|
||||
# Return the update (or a modified version)
|
||||
return update
|
||||
|
||||
def uppercase_hook(update: ChatResponseUpdate) -> ChatResponseUpdate:
|
||||
"""Hook that converts text to uppercase."""
|
||||
if update.text:
|
||||
return ChatResponseUpdate(
|
||||
contents=[Content.from_text(update.text.upper())], role=update.role, response_id=update.response_id
|
||||
)
|
||||
return update
|
||||
|
||||
# Pass transform_hooks directly to constructor
|
||||
stream3 = ResponseStream(
|
||||
generate_updates(),
|
||||
finalizer=combine_updates,
|
||||
transform_hooks=[counting_hook, uppercase_hook], # First counts, then uppercases
|
||||
)
|
||||
|
||||
print("Iterating with hooks applied:")
|
||||
async for update in stream3:
|
||||
print(f" Received: '{update.text}'") # Will be uppercase
|
||||
|
||||
print(f"\nTotal updates processed: {update_count['value']}")
|
||||
|
||||
# =========================================================================
|
||||
# Example 4: Cleanup hooks - cleanup after stream consumption
|
||||
# =========================================================================
|
||||
print("\n=== Example 4: Cleanup Hooks ===\n")
|
||||
|
||||
cleanup_performed = {"value": False}
|
||||
|
||||
async def cleanup_hook() -> None:
|
||||
"""Cleanup hook for releasing resources after stream consumption."""
|
||||
print(" [Cleanup] Cleaning up resources...")
|
||||
cleanup_performed["value"] = True
|
||||
|
||||
# Pass cleanup_hooks directly to constructor
|
||||
stream4 = ResponseStream(
|
||||
generate_updates(),
|
||||
finalizer=combine_updates,
|
||||
cleanup_hooks=[cleanup_hook],
|
||||
)
|
||||
|
||||
print("Starting iteration (cleanup happens after):")
|
||||
async for update in stream4:
|
||||
pass # Just consume the stream
|
||||
print(f"Cleanup was performed: {cleanup_performed['value']}")
|
||||
|
||||
# =========================================================================
|
||||
# Example 5: Result hooks - transform the final response
|
||||
# =========================================================================
|
||||
print("\n=== Example 5: Result Hooks ===\n")
|
||||
|
||||
def add_metadata_hook(response: ChatResponse) -> ChatResponse:
|
||||
"""Result hook that adds metadata to the response."""
|
||||
response.additional_properties["processed"] = True
|
||||
response.additional_properties["word_count"] = len((response.text or "").split())
|
||||
return response
|
||||
|
||||
def wrap_in_quotes_hook(response: ChatResponse) -> ChatResponse:
|
||||
"""Result hook that wraps the response text in quotes."""
|
||||
if response.text:
|
||||
return ChatResponse(
|
||||
messages=f'"{response.text}"',
|
||||
role=Role.ASSISTANT,
|
||||
additional_properties=response.additional_properties,
|
||||
)
|
||||
return response
|
||||
|
||||
# Finalizer converts updates to response, then result hooks transform it
|
||||
stream5 = ResponseStream(
|
||||
generate_updates(),
|
||||
finalizer=combine_updates,
|
||||
result_hooks=[add_metadata_hook, wrap_in_quotes_hook], # First adds metadata, then wraps in quotes
|
||||
)
|
||||
|
||||
final5 = await stream5.get_final_response()
|
||||
print(f"Final text: {final5.text}")
|
||||
print(f"Metadata: {final5.additional_properties}")
|
||||
|
||||
# =========================================================================
|
||||
# Example 6: The wrap() API - layering without double-consumption
|
||||
# =========================================================================
|
||||
print("\n=== Example 6: wrap() API for Layering ===\n")
|
||||
|
||||
# Simulate what ChatClient returns
|
||||
inner_stream = ResponseStream(generate_updates(), finalizer=combine_updates)
|
||||
|
||||
# Simulate what ChatAgent does: wrap the inner stream
|
||||
def to_agent_format(update: ChatResponseUpdate) -> ChatResponseUpdate:
|
||||
"""Map ChatResponseUpdate to agent format (simulated transformation)."""
|
||||
# In real code, this would convert to AgentResponseUpdate
|
||||
return ChatResponseUpdate(
|
||||
contents=[Content.from_text(f"[AGENT] {update.text}")], role=update.role, response_id=update.response_id
|
||||
)
|
||||
|
||||
def to_agent_response(updates: Sequence[ChatResponseUpdate]) -> ChatResponse:
|
||||
"""Finalizer that converts updates to agent response (simulated)."""
|
||||
# In real code, this would create an AgentResponse
|
||||
text = "".join(u.text or "" for u in updates)
|
||||
return ChatResponse(
|
||||
text=f"[AGENT FINAL] {text}",
|
||||
role=Role.ASSISTANT,
|
||||
additional_properties={"layer": "agent"},
|
||||
)
|
||||
|
||||
# .map() creates a new stream that:
|
||||
# 1. Delegates iteration to inner_stream (only consuming it once)
|
||||
# 2. Transforms each update via the transform function
|
||||
# 3. Uses the provided finalizer (required since update type may change)
|
||||
outer_stream = inner_stream.map(to_agent_format, to_agent_response)
|
||||
|
||||
print("Iterating the mapped stream:")
|
||||
async for update in outer_stream:
|
||||
print(f" {update.text}")
|
||||
|
||||
final_outer = await outer_stream.get_final_response()
|
||||
print(f"\nMapped final: {final_outer.text}")
|
||||
print(f"Mapped metadata: {final_outer.additional_properties}")
|
||||
|
||||
# Important: the inner stream was only consumed once!
|
||||
print(f"Inner stream consumed: {inner_stream._consumed}")
|
||||
|
||||
# =========================================================================
|
||||
# Example 7: Combining all patterns
|
||||
# =========================================================================
|
||||
print("\n=== Example 7: Full Integration ===\n")
|
||||
|
||||
stats = {"updates": 0, "characters": 0}
|
||||
|
||||
def track_stats(update: ChatResponseUpdate) -> ChatResponseUpdate:
|
||||
"""Track statistics as updates flow through."""
|
||||
stats["updates"] += 1
|
||||
stats["characters"] += len(update.text or "")
|
||||
return update
|
||||
|
||||
def log_cleanup() -> None:
|
||||
"""Log when stream consumption completes."""
|
||||
print(f" [Cleanup] Stream complete: {stats['updates']} updates, {stats['characters']} chars")
|
||||
|
||||
def add_stats_to_response(response: ChatResponse) -> ChatResponse:
|
||||
"""Result hook to include the statistics in the final response."""
|
||||
response.additional_properties["stats"] = stats.copy()
|
||||
return response
|
||||
|
||||
# All hooks can be passed via constructor
|
||||
full_stream = ResponseStream(
|
||||
generate_updates(),
|
||||
finalizer=combine_updates,
|
||||
transform_hooks=[track_stats],
|
||||
result_hooks=[add_stats_to_response],
|
||||
cleanup_hooks=[log_cleanup],
|
||||
)
|
||||
|
||||
print("Processing with all hooks active:")
|
||||
async for update in full_stream:
|
||||
print(f" -> '{update.text}'")
|
||||
|
||||
final_full = await full_stream.get_final_response()
|
||||
print(f"\nFinal: '{final_full.text}'")
|
||||
print(f"Stats: {final_full.additional_properties['stats']}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,499 @@
|
||||
# Tools and Middleware: Request Flow Architecture
|
||||
|
||||
This document describes the complete request flow when using an Agent with middleware and tools, from the initial `Agent.run()` call through middleware layers, function invocation, and back to the caller.
|
||||
|
||||
## Overview
|
||||
|
||||
The Agent Framework uses a layered architecture with three distinct middleware/processing layers:
|
||||
|
||||
1. **Agent Middleware Layer** - Wraps the entire agent execution
|
||||
2. **Chat Middleware Layer** - Wraps calls to the chat client
|
||||
3. **Function Middleware Layer** - Wraps individual tool/function invocations
|
||||
|
||||
Each layer provides interception points where you can modify inputs, inspect outputs, or alter behavior.
|
||||
|
||||
## Flow Diagram
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant User
|
||||
participant Agent as Agent.run()
|
||||
participant AML as AgentMiddlewareLayer
|
||||
participant AMP as AgentMiddlewarePipeline
|
||||
participant RawAgent as RawChatAgent.run()
|
||||
participant CML as ChatMiddlewareLayer
|
||||
participant CMP as ChatMiddlewarePipeline
|
||||
participant FIL as FunctionInvocationLayer
|
||||
participant Client as BaseChatClient._inner_get_response()
|
||||
participant LLM as LLM Service
|
||||
participant FMP as FunctionMiddlewarePipeline
|
||||
participant Tool as FunctionTool.invoke()
|
||||
|
||||
User->>Agent: run(messages, thread, options, middleware)
|
||||
|
||||
Note over Agent,AML: Agent Middleware Layer
|
||||
Agent->>AML: run() with middleware param
|
||||
AML->>AML: categorize_middleware() → split by type
|
||||
AML->>AMP: execute(AgentRunContext)
|
||||
|
||||
loop Agent Middleware Chain
|
||||
AMP->>AMP: middleware[i].process(context, next)
|
||||
Note right of AMP: Can modify: messages, options, thread
|
||||
end
|
||||
|
||||
AMP->>RawAgent: run() via final_handler
|
||||
|
||||
alt Non-Streaming (stream=False)
|
||||
RawAgent->>RawAgent: _prepare_run_context() [async]
|
||||
Note right of RawAgent: Builds: thread_messages, chat_options, tools
|
||||
RawAgent->>CML: chat_client.get_response(stream=False)
|
||||
else Streaming (stream=True)
|
||||
RawAgent->>RawAgent: ResponseStream.from_awaitable()
|
||||
Note right of RawAgent: Defers async prep to stream consumption
|
||||
RawAgent-->>User: Returns ResponseStream immediately
|
||||
Note over RawAgent,CML: Async work happens on iteration
|
||||
RawAgent->>RawAgent: _prepare_run_context() [deferred]
|
||||
RawAgent->>CML: chat_client.get_response(stream=True)
|
||||
end
|
||||
|
||||
Note over CML,CMP: Chat Middleware Layer
|
||||
CML->>CMP: execute(ChatContext)
|
||||
|
||||
loop Chat Middleware Chain
|
||||
CMP->>CMP: middleware[i].process(context, next)
|
||||
Note right of CMP: Can modify: messages, options
|
||||
end
|
||||
|
||||
CMP->>FIL: get_response() via final_handler
|
||||
|
||||
Note over FIL,Tool: Function Invocation Loop
|
||||
loop Max Iterations (default: 40)
|
||||
FIL->>Client: _inner_get_response(messages, options)
|
||||
Client->>LLM: API Call
|
||||
LLM-->>Client: Response (may include tool_calls)
|
||||
Client-->>FIL: ChatResponse
|
||||
|
||||
alt Response has function_calls
|
||||
FIL->>FIL: _extract_function_calls()
|
||||
FIL->>FIL: _try_execute_function_calls()
|
||||
|
||||
Note over FIL,Tool: Function Middleware Layer
|
||||
loop For each function_call
|
||||
FIL->>FMP: execute(FunctionInvocationContext)
|
||||
loop Function Middleware Chain
|
||||
FMP->>FMP: middleware[i].process(context, next)
|
||||
Note right of FMP: Can modify: arguments
|
||||
end
|
||||
FMP->>Tool: invoke(arguments)
|
||||
Tool-->>FMP: result
|
||||
FMP-->>FIL: Content.from_function_result()
|
||||
end
|
||||
|
||||
FIL->>FIL: Append tool results to messages
|
||||
|
||||
alt tool_choice == "required"
|
||||
Note right of FIL: Return immediately with function call + result
|
||||
FIL-->>CMP: ChatResponse
|
||||
else tool_choice == "auto" or other
|
||||
Note right of FIL: Continue loop for text response
|
||||
end
|
||||
else No function_calls
|
||||
FIL-->>CMP: ChatResponse
|
||||
end
|
||||
end
|
||||
|
||||
CMP-->>CML: ChatResponse
|
||||
Note right of CMP: Can observe/modify result
|
||||
|
||||
CML-->>RawAgent: ChatResponse / ResponseStream
|
||||
|
||||
alt Non-Streaming
|
||||
RawAgent->>RawAgent: _finalize_response_and_update_thread()
|
||||
else Streaming
|
||||
Note right of RawAgent: .map() transforms updates
|
||||
Note right of RawAgent: .with_result_hook() runs post-processing
|
||||
end
|
||||
|
||||
RawAgent-->>AMP: AgentResponse / ResponseStream
|
||||
Note right of AMP: Can observe/modify result
|
||||
AMP-->>AML: AgentResponse
|
||||
AML-->>Agent: AgentResponse
|
||||
Agent-->>User: AgentResponse / ResponseStream
|
||||
```
|
||||
|
||||
## Layer Details
|
||||
|
||||
### 1. Agent Middleware Layer (`AgentMiddlewareLayer`)
|
||||
|
||||
**Entry Point:** `Agent.run(messages, thread, options, middleware)`
|
||||
|
||||
**Context Object:** `AgentRunContext`
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `agent` | `AgentProtocol` | The agent being invoked |
|
||||
| `messages` | `list[ChatMessage]` | Input messages (mutable) |
|
||||
| `thread` | `AgentThread \| None` | Conversation thread |
|
||||
| `options` | `Mapping[str, Any]` | Chat options dict |
|
||||
| `stream` | `bool` | Whether streaming is enabled |
|
||||
| `metadata` | `dict` | Shared data between middleware |
|
||||
| `result` | `AgentResponse \| None` | Set after `next()` is called |
|
||||
| `kwargs` | `Mapping[str, Any]` | Additional run arguments |
|
||||
|
||||
**Key Operations:**
|
||||
1. `categorize_middleware()` separates middleware by type (agent, chat, function)
|
||||
2. Chat and function middleware are forwarded to `chat_client`
|
||||
3. `AgentMiddlewarePipeline.execute()` runs the agent middleware chain
|
||||
4. Final handler calls `RawChatAgent.run()`
|
||||
|
||||
**What Can Be Modified:**
|
||||
- `context.messages` - Add, remove, or modify input messages
|
||||
- `context.options` - Change model parameters, temperature, etc.
|
||||
- `context.thread` - Replace or modify the thread
|
||||
- `context.result` - Override the final response (after `next()`)
|
||||
|
||||
### 2. Chat Middleware Layer (`ChatMiddlewareLayer`)
|
||||
|
||||
**Entry Point:** `chat_client.get_response(messages, options)`
|
||||
|
||||
**Context Object:** `ChatContext`
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `chat_client` | `ChatClientProtocol` | The chat client |
|
||||
| `messages` | `Sequence[ChatMessage]` | Messages to send |
|
||||
| `options` | `Mapping[str, Any]` | Chat options |
|
||||
| `stream` | `bool` | Whether streaming |
|
||||
| `metadata` | `dict` | Shared data between middleware |
|
||||
| `result` | `ChatResponse \| None` | Set after `next()` is called |
|
||||
| `kwargs` | `Mapping[str, Any]` | Additional arguments |
|
||||
|
||||
**Key Operations:**
|
||||
1. `ChatMiddlewarePipeline.execute()` runs the chat middleware chain
|
||||
2. Final handler calls `FunctionInvocationLayer.get_response()`
|
||||
3. Stream hooks can be registered for streaming responses
|
||||
|
||||
**What Can Be Modified:**
|
||||
- `context.messages` - Inject system prompts, filter content
|
||||
- `context.options` - Change model, temperature, tool_choice
|
||||
- `context.result` - Override the response (after `next()`)
|
||||
|
||||
### 3. Function Invocation Layer (`FunctionInvocationLayer`)
|
||||
|
||||
**Entry Point:** `FunctionInvocationLayer.get_response()`
|
||||
|
||||
This layer manages the tool execution loop:
|
||||
|
||||
1. **Calls** `BaseChatClient._inner_get_response()` to get LLM response
|
||||
2. **Extracts** function calls from the response
|
||||
3. **Executes** functions through the Function Middleware Pipeline
|
||||
4. **Appends** results to messages and loops back to step 1
|
||||
|
||||
**Configuration:** `FunctionInvocationConfiguration`
|
||||
|
||||
| Setting | Default | Description |
|
||||
|---------|---------|-------------|
|
||||
| `enabled` | `True` | Enable auto-invocation |
|
||||
| `max_iterations` | `40` | Maximum tool execution loops |
|
||||
| `max_consecutive_errors_per_request` | `3` | Error threshold before stopping |
|
||||
| `terminate_on_unknown_calls` | `False` | Raise error for unknown tools |
|
||||
| `additional_tools` | `[]` | Extra tools to register |
|
||||
| `include_detailed_errors` | `False` | Include exceptions in results |
|
||||
|
||||
**`tool_choice` Behavior:**
|
||||
|
||||
The `tool_choice` option controls how the model uses available tools:
|
||||
|
||||
| Value | Behavior |
|
||||
|-------|----------|
|
||||
| `"auto"` | Model decides whether to call a tool or respond with text. After tool execution, the loop continues to get a text response. |
|
||||
| `"none"` | Model is prevented from calling tools, will only respond with text. |
|
||||
| `"required"` | Model **must** call a tool. After tool execution, returns immediately with the function call and result—**no additional model call** is made. |
|
||||
| `{"mode": "required", "required_function_name": "fn"}` | Model must call the specified function. Same return behavior as `"required"`. |
|
||||
|
||||
**Why `tool_choice="required"` returns immediately:**
|
||||
|
||||
When you set `tool_choice="required"`, your intent is to force one or more tool calls (not all models supports multiple, either by name or when using `required` without a name). The framework respects this by:
|
||||
1. Getting the model's function call(s)
|
||||
2. Executing the tool(s)
|
||||
3. Returning the response(s) with both the function call message(s) and the function result(s)
|
||||
|
||||
This avoids an infinite loop (model forced to call tools → executes → model forced to call tools again) and gives you direct access to the tool result.
|
||||
|
||||
```python
|
||||
# With tool_choice="required", response contains function call + result only
|
||||
response = await client.get_response(
|
||||
"What's the weather?",
|
||||
options={"tool_choice": "required", "tools": [get_weather]}
|
||||
)
|
||||
|
||||
# response.messages contains:
|
||||
# [0] Assistant message with function_call content
|
||||
# [1] Tool message with function_result content
|
||||
# (No text response from model)
|
||||
|
||||
# To get a text response after tool execution, use tool_choice="auto"
|
||||
response = await client.get_response(
|
||||
"What's the weather?",
|
||||
options={"tool_choice": "auto", "tools": [get_weather]}
|
||||
)
|
||||
# response.text contains the model's interpretation of the weather data
|
||||
```
|
||||
|
||||
### 4. Function Middleware Layer (`FunctionMiddlewarePipeline`)
|
||||
|
||||
**Entry Point:** Called per function invocation within `_auto_invoke_function()`
|
||||
|
||||
**Context Object:** `FunctionInvocationContext`
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `function` | `FunctionTool` | The function being invoked |
|
||||
| `arguments` | `BaseModel` | Validated Pydantic arguments |
|
||||
| `metadata` | `dict` | Shared data between middleware |
|
||||
| `result` | `Any` | Set after `next()` is called |
|
||||
| `kwargs` | `Mapping[str, Any]` | Runtime kwargs |
|
||||
|
||||
**What Can Be Modified:**
|
||||
- `context.arguments` - Modify validated arguments before execution
|
||||
- `context.result` - Override the function result (after `next()`)
|
||||
- Raise `MiddlewareTermination` to skip execution and terminate the function invocation loop
|
||||
|
||||
**Special Behavior:** When `MiddlewareTermination` is raised in function middleware, it signals that the function invocation loop should exit **without making another LLM call**. This is useful when middleware determines that no further processing is needed (e.g., a termination condition is met).
|
||||
|
||||
```python
|
||||
class TerminatingMiddleware(FunctionMiddleware):
|
||||
async def process(self, context: FunctionInvocationContext, next):
|
||||
if self.should_terminate(context):
|
||||
context.result = "terminated by middleware"
|
||||
raise MiddlewareTermination # Exit function invocation loop
|
||||
await next(context)
|
||||
```
|
||||
|
||||
## Arguments Added/Altered at Each Layer
|
||||
|
||||
### Agent Layer → Chat Layer
|
||||
|
||||
```python
|
||||
# RawChatAgent._prepare_run_context() builds:
|
||||
{
|
||||
"thread": AgentThread, # Validated/created thread
|
||||
"input_messages": [...], # Normalized input messages
|
||||
"thread_messages": [...], # Messages from thread + context + input
|
||||
"agent_name": "...", # Agent name for attribution
|
||||
"chat_options": {
|
||||
"model_id": "...",
|
||||
"conversation_id": "...", # From thread.service_thread_id
|
||||
"tools": [...], # Normalized tools + MCP tools
|
||||
"temperature": ...,
|
||||
"max_tokens": ...,
|
||||
# ... other options
|
||||
},
|
||||
"filtered_kwargs": {...}, # kwargs minus 'chat_options'
|
||||
"finalize_kwargs": {...}, # kwargs with 'thread' added
|
||||
}
|
||||
```
|
||||
|
||||
### Chat Layer → Function Layer
|
||||
|
||||
```python
|
||||
# Passed through to FunctionInvocationLayer:
|
||||
{
|
||||
"messages": [...], # Prepared messages
|
||||
"options": {...}, # Mutable copy of chat_options
|
||||
"function_middleware": [...], # Function middleware from kwargs
|
||||
}
|
||||
```
|
||||
|
||||
### Function Layer → Tool Invocation
|
||||
|
||||
```python
|
||||
# FunctionInvocationContext receives:
|
||||
{
|
||||
"function": FunctionTool, # The tool to invoke
|
||||
"arguments": BaseModel, # Validated from function_call.arguments
|
||||
"kwargs": {
|
||||
# Runtime kwargs (filtered, no conversation_id)
|
||||
},
|
||||
}
|
||||
```
|
||||
|
||||
### Tool Result → Back Up
|
||||
|
||||
```python
|
||||
# Content.from_function_result() creates:
|
||||
{
|
||||
"type": "function_result",
|
||||
"call_id": "...", # From function_call.call_id
|
||||
"result": ..., # Serialized tool output
|
||||
"exception": "..." | None, # Error message if failed
|
||||
}
|
||||
```
|
||||
|
||||
## Middleware Control Flow
|
||||
|
||||
There are three ways to exit a middleware's `process()` method:
|
||||
|
||||
### 1. Return Normally (with or without calling `next`)
|
||||
|
||||
Returns control to the upstream middleware, allowing its post-processing code to run.
|
||||
|
||||
```python
|
||||
class CachingMiddleware(FunctionMiddleware):
|
||||
async def process(self, context: FunctionInvocationContext, next):
|
||||
# Option A: Return early WITHOUT calling next (skip downstream)
|
||||
if cached := self.cache.get(context.function.name):
|
||||
context.result = cached
|
||||
return # Upstream post-processing still runs
|
||||
|
||||
# Option B: Call next, then return normally
|
||||
await next(context)
|
||||
self.cache[context.function.name] = context.result
|
||||
return # Normal completion
|
||||
```
|
||||
|
||||
### 2. Raise `MiddlewareTermination`
|
||||
|
||||
Immediately exits the entire middleware chain. Upstream middleware's post-processing code is **skipped**.
|
||||
|
||||
```python
|
||||
class BlockedFunctionMiddleware(FunctionMiddleware):
|
||||
async def process(self, context: FunctionInvocationContext, next):
|
||||
if context.function.name in self.blocked_functions:
|
||||
context.result = "Function blocked by policy"
|
||||
raise MiddlewareTermination("Blocked") # Skips ALL post-processing
|
||||
await next(context)
|
||||
```
|
||||
|
||||
### 3. Raise Any Other Exception
|
||||
|
||||
Bubbles up to the caller. The middleware chain is aborted and the exception propagates.
|
||||
|
||||
```python
|
||||
class ValidationMiddleware(FunctionMiddleware):
|
||||
async def process(self, context: FunctionInvocationContext, next):
|
||||
if not self.is_valid(context.arguments):
|
||||
raise ValueError("Invalid arguments") # Bubbles up to user
|
||||
await next(context)
|
||||
```
|
||||
|
||||
## `return` vs `raise MiddlewareTermination`
|
||||
|
||||
The key difference is what happens to **upstream middleware's post-processing**:
|
||||
|
||||
```python
|
||||
class MiddlewareA(AgentMiddleware):
|
||||
async def process(self, context, next):
|
||||
print("A: before")
|
||||
await next(context)
|
||||
print("A: after") # Does this run?
|
||||
|
||||
class MiddlewareB(AgentMiddleware):
|
||||
async def process(self, context, next):
|
||||
print("B: before")
|
||||
context.result = "early result"
|
||||
# Choose one:
|
||||
return # Option 1
|
||||
# raise MiddlewareTermination() # Option 2
|
||||
```
|
||||
|
||||
With middleware registered as `[MiddlewareA, MiddlewareB]`:
|
||||
|
||||
| Exit Method | Output |
|
||||
|-------------|--------|
|
||||
| `return` | `A: before` → `B: before` → `A: after` |
|
||||
| `raise MiddlewareTermination` | `A: before` → `B: before` (no `A: after`) |
|
||||
|
||||
**Use `return`** when you want upstream middleware to still process the result (e.g., logging, metrics).
|
||||
|
||||
**Use `raise MiddlewareTermination`** when you want to completely bypass all remaining processing (e.g., blocking a request, returning cached response without any modification).
|
||||
|
||||
## Calling `next()` or Not
|
||||
|
||||
The decision to call `next(context)` determines whether downstream middleware and the actual operation execute:
|
||||
|
||||
### Without calling `next()` - Skip downstream
|
||||
|
||||
```python
|
||||
async def process(self, context, next):
|
||||
context.result = "replacement result"
|
||||
return # Downstream middleware and actual execution are SKIPPED
|
||||
```
|
||||
|
||||
- Downstream middleware: ❌ NOT executed
|
||||
- Actual operation (LLM call, function invocation): ❌ NOT executed
|
||||
- Upstream middleware post-processing: ✅ Still runs (unless `MiddlewareTermination` raised)
|
||||
- Result: Whatever you set in `context.result`
|
||||
|
||||
### With calling `next()` - Full execution
|
||||
|
||||
```python
|
||||
async def process(self, context, next):
|
||||
# Pre-processing
|
||||
await next(context) # Execute downstream + actual operation
|
||||
# Post-processing (context.result now contains real result)
|
||||
return
|
||||
```
|
||||
|
||||
- Downstream middleware: ✅ Executed
|
||||
- Actual operation: ✅ Executed
|
||||
- Upstream middleware post-processing: ✅ Runs
|
||||
- Result: The actual result (possibly modified in post-processing)
|
||||
|
||||
### Summary Table
|
||||
|
||||
| Exit Method | Call `next()`? | Downstream Executes? | Actual Op Executes? | Upstream Post-Processing? |
|
||||
|-------------|----------------|---------------------|---------------------|--------------------------|
|
||||
| `return` (or implicit) | Yes | ✅ | ✅ | ✅ Yes |
|
||||
| `return` | No | ❌ | ❌ | ✅ Yes |
|
||||
| `raise MiddlewareTermination` | No | ❌ | ❌ | ❌ No |
|
||||
| `raise MiddlewareTermination` | Yes | ✅ | ✅ | ❌ No |
|
||||
| `raise OtherException` | Either | Depends | Depends | ❌ No (exception propagates) |
|
||||
|
||||
> **Note:** The first row (`return` after calling `next()`) is the default behavior. Python functions implicitly return `None` at the end, so simply calling `await next(context)` without an explicit `return` statement achieves this pattern.
|
||||
|
||||
## Streaming vs Non-Streaming
|
||||
|
||||
The `run()` method handles streaming and non-streaming differently:
|
||||
|
||||
### Non-Streaming (`stream=False`)
|
||||
|
||||
Returns `Awaitable[AgentResponse]`:
|
||||
|
||||
```python
|
||||
async def _run_non_streaming():
|
||||
ctx = await self._prepare_run_context(...) # Async preparation
|
||||
response = await self.chat_client.get_response(stream=False, ...)
|
||||
await self._finalize_response_and_update_thread(...)
|
||||
return AgentResponse(...)
|
||||
```
|
||||
|
||||
### Streaming (`stream=True`)
|
||||
|
||||
Returns `ResponseStream[AgentResponseUpdate, AgentResponse]` **synchronously**:
|
||||
|
||||
```python
|
||||
# Async preparation is deferred using ResponseStream.from_awaitable()
|
||||
async def _get_stream():
|
||||
ctx = await self._prepare_run_context(...) # Deferred until iteration
|
||||
return self.chat_client.get_response(stream=True, ...)
|
||||
|
||||
return (
|
||||
ResponseStream.from_awaitable(_get_stream())
|
||||
.map(
|
||||
transform=map_chat_to_agent_update, # Transform each update
|
||||
finalizer=self._finalize_response_updates, # Build final response
|
||||
)
|
||||
.with_result_hook(_post_hook) # Post-processing after finalization
|
||||
)
|
||||
```
|
||||
|
||||
Key points:
|
||||
- `ResponseStream.from_awaitable()` wraps an async function, deferring execution until the stream is consumed
|
||||
- `.map()` transforms `ChatResponseUpdate` → `AgentResponseUpdate` and provides the finalizer
|
||||
- `.with_result_hook()` runs after finalization (e.g., notify thread of new messages)
|
||||
|
||||
## See Also
|
||||
|
||||
- [Middleware Samples](../../getting_started/middleware/) - Examples of custom middleware
|
||||
- [Function Tool Samples](../../getting_started/tools/) - Creating and using tools
|
||||
@@ -118,7 +118,7 @@ agent_messages = await converter.to_agent_input(user_message_item)
|
||||
|
||||
# Running agent and streaming back to ChatKit
|
||||
async for event in stream_agent_response(
|
||||
self.weather_agent.run_stream(agent_messages),
|
||||
self.weather_agent.run(agent_messages, stream=True),
|
||||
thread_id=thread.id,
|
||||
):
|
||||
yield event
|
||||
|
||||
@@ -18,7 +18,7 @@ from typing import Annotated, Any
|
||||
import uvicorn
|
||||
|
||||
# Agent Framework imports
|
||||
from agent_framework import AgentResponseUpdate, ChatAgent, ChatMessage, FunctionResultContent, tool
|
||||
from agent_framework import AgentResponseUpdate, ChatAgent, ChatMessage, FunctionResultContent, Role, tool
|
||||
from agent_framework.azure import AzureOpenAIChatClient
|
||||
|
||||
# Agent Framework ChatKit integration
|
||||
@@ -281,7 +281,7 @@ class WeatherChatKitServer(ChatKitServer[dict[str, Any]]):
|
||||
|
||||
title_prompt = [
|
||||
ChatMessage(
|
||||
role="user",
|
||||
role=Role.USER,
|
||||
text=(
|
||||
f"Generate a very short, concise title (max 40 characters) for a conversation "
|
||||
f"that starts with:\n\n{conversation_context}\n\n"
|
||||
@@ -366,7 +366,7 @@ class WeatherChatKitServer(ChatKitServer[dict[str, Any]]):
|
||||
logger.info(f"Running agent with {len(agent_messages)} message(s)")
|
||||
|
||||
# Run the Agent Framework agent with streaming
|
||||
agent_stream = self.weather_agent.run_stream(agent_messages)
|
||||
agent_stream = self.weather_agent.run(agent_messages, stream=True)
|
||||
|
||||
# Create an intercepting stream that extracts function results while passing through updates
|
||||
async def intercept_stream() -> AsyncIterator[AgentResponseUpdate]:
|
||||
@@ -458,12 +458,12 @@ class WeatherChatKitServer(ChatKitServer[dict[str, Any]]):
|
||||
weather_data: WeatherData | None = None
|
||||
|
||||
# Create an agent message asking about the weather
|
||||
agent_messages = [ChatMessage("user", [f"What's the weather in {city_label}?"])]
|
||||
agent_messages = [ChatMessage(role=Role.USER, text=f"What's the weather in {city_label}?")]
|
||||
|
||||
logger.debug(f"Processing weather query: {agent_messages[0].text}")
|
||||
|
||||
# Run the Agent Framework agent with streaming
|
||||
agent_stream = self.weather_agent.run_stream(agent_messages)
|
||||
agent_stream = self.weather_agent.run(agent_messages, stream=True)
|
||||
|
||||
# Create an intercepting stream that extracts function results while passing through updates
|
||||
async def intercept_stream() -> AsyncIterator[AgentResponseUpdate]:
|
||||
|
||||
@@ -189,7 +189,7 @@ async def _run_workflow_with_client(query: str, chat_client: AzureAIClient) -> d
|
||||
workflow, agent_map = await _create_workflow(chat_client.project_client, chat_client.credential)
|
||||
|
||||
# Process workflow events
|
||||
events = workflow.run_stream(query)
|
||||
events = workflow.run(query, stream=True)
|
||||
workflow_output = await _process_workflow_events(events, conversation_ids, response_ids)
|
||||
|
||||
return {
|
||||
|
||||
@@ -38,7 +38,7 @@ async def main() -> None:
|
||||
query = "Can you compare Python decorators with C# attributes?"
|
||||
print(f"User: {query}")
|
||||
print("Agent: ", end="", flush=True)
|
||||
async for chunk in agent.run_stream(query):
|
||||
async for chunk in agent.run(query, stream=True):
|
||||
for content in chunk.contents:
|
||||
if isinstance(content, TextReasoningContent):
|
||||
print(f"\033[32m{content.text}\033[0m", end="", flush=True)
|
||||
|
||||
@@ -55,7 +55,7 @@ async def streaming_example() -> None:
|
||||
query = "What's the weather like in Portland and in Paris?"
|
||||
print(f"User: {query}")
|
||||
print("Agent: ", end="", flush=True)
|
||||
async for chunk in agent.run_stream(query):
|
||||
async for chunk in agent.run(query, stream=True):
|
||||
if chunk.text:
|
||||
print(chunk.text, end="", flush=True)
|
||||
print("\n")
|
||||
|
||||
@@ -59,7 +59,7 @@ async def streaming_example() -> None:
|
||||
query = "What's the weather in Paris?"
|
||||
print(f"User: {query}")
|
||||
print("Agent: ", end="", flush=True)
|
||||
async for chunk in agent.run_stream(query):
|
||||
async for chunk in agent.run(query, stream=True):
|
||||
if chunk.text:
|
||||
print(chunk.text, end="", flush=True)
|
||||
print("\n")
|
||||
|
||||
@@ -49,7 +49,7 @@ async def main() -> None:
|
||||
query = "Can you compare Python decorators with C# attributes?"
|
||||
print(f"User: {query}")
|
||||
print("Agent: ", end="", flush=True)
|
||||
async for chunk in agent.run_stream(query):
|
||||
async for chunk in agent.run(query, stream=True):
|
||||
for content in chunk.contents:
|
||||
if isinstance(content, TextReasoningContent):
|
||||
print(f"\033[32m{content.text}\033[0m", end="", flush=True)
|
||||
|
||||
@@ -53,7 +53,7 @@ async def main() -> None:
|
||||
print(f"User: {query}")
|
||||
print("Agent: ", end="", flush=True)
|
||||
files: list[HostedFileContent] = []
|
||||
async for chunk in agent.run_stream(query):
|
||||
async for chunk in agent.run(query, stream=True):
|
||||
for content in chunk.contents:
|
||||
match content.type:
|
||||
case "text":
|
||||
|
||||
@@ -68,7 +68,7 @@ async def streaming_example() -> None:
|
||||
query = "What's the weather like in Tokyo?"
|
||||
print(f"User: {query}")
|
||||
print("Agent: ", end="", flush=True)
|
||||
async for chunk in agent.run_stream(query):
|
||||
async for chunk in agent.run(query, stream=True):
|
||||
if chunk.text:
|
||||
print(chunk.text, end="", flush=True)
|
||||
print("\n")
|
||||
|
||||
@@ -22,7 +22,7 @@ async def logging_middleware(
|
||||
context: FunctionInvocationContext,
|
||||
next: Callable[[FunctionInvocationContext], Awaitable[None]],
|
||||
) -> None:
|
||||
"""Middleware that logs tool invocations to show the delegation flow."""
|
||||
"""MiddlewareTypes that logs tool invocations to show the delegation flow."""
|
||||
print(f"[Calling tool: {context.function.name}]")
|
||||
print(f"[Request: {context.arguments}]")
|
||||
|
||||
|
||||
+2
-2
@@ -11,7 +11,7 @@ from agent_framework import (
|
||||
Content,
|
||||
HostedCodeInterpreterTool,
|
||||
HostedFileContent,
|
||||
tool,
|
||||
TextContent,
|
||||
)
|
||||
from agent_framework.azure import AzureAIProjectAgentProvider
|
||||
from azure.identity.aio import AzureCliCredential
|
||||
@@ -178,7 +178,7 @@ async def streaming_example() -> None:
|
||||
file_contents_found: list[HostedFileContent] = []
|
||||
text_chunks: list[str] = []
|
||||
|
||||
async for update in agent.run_stream(QUERY):
|
||||
async for update in agent.run(QUERY, stream=True):
|
||||
if isinstance(update, AgentResponseUpdate):
|
||||
for content in update.contents:
|
||||
if content.type == "text":
|
||||
|
||||
+1
-1
@@ -78,7 +78,7 @@ async def streaming_example() -> None:
|
||||
text_chunks: list[str] = []
|
||||
file_ids_found: list[str] = []
|
||||
|
||||
async for update in agent.run_stream(QUERY):
|
||||
async for update in agent.run(QUERY, stream=True):
|
||||
if isinstance(update, AgentResponseUpdate):
|
||||
for content in update.contents:
|
||||
if content.type == "text":
|
||||
|
||||
@@ -68,7 +68,7 @@ async def streaming_example() -> None:
|
||||
|
||||
shown_reasoning_label = False
|
||||
shown_text_label = False
|
||||
async for chunk in agent.run_stream(query):
|
||||
async for chunk in agent.run(query, stream=True):
|
||||
for content in chunk.contents:
|
||||
if content.type == "text_reasoning":
|
||||
if not shown_reasoning_label:
|
||||
|
||||
@@ -66,7 +66,7 @@ async def streaming_example() -> None:
|
||||
query = "What's the weather like in Portland?"
|
||||
print(f"User: {query}")
|
||||
print("Agent: ", end="", flush=True)
|
||||
async for chunk in agent.run_stream(query):
|
||||
async for chunk in agent.run(query, stream=True):
|
||||
if chunk.text:
|
||||
print(chunk.text, end="", flush=True)
|
||||
print("\n")
|
||||
|
||||
+1
-1
@@ -87,7 +87,7 @@ async def main() -> None:
|
||||
print("Agent: ", end="", flush=True)
|
||||
# Stream the response and collect citations
|
||||
citations: list[Annotation] = []
|
||||
async for chunk in agent.run_stream(user_input):
|
||||
async for chunk in agent.run(user_input, stream=True):
|
||||
if chunk.text:
|
||||
print(chunk.text, end="", flush=True)
|
||||
# Collect citations from Azure AI Search responses
|
||||
|
||||
+1
-1
@@ -58,7 +58,7 @@ async def main() -> None:
|
||||
|
||||
# Stream the response and collect citations
|
||||
citations: list[Annotation] = []
|
||||
async for chunk in agent.run_stream(user_input):
|
||||
async for chunk in agent.run(user_input, stream=True):
|
||||
if chunk.text:
|
||||
print(chunk.text, end="", flush=True)
|
||||
|
||||
|
||||
+1
-5
@@ -4,7 +4,6 @@ import asyncio
|
||||
import os
|
||||
|
||||
from agent_framework import (
|
||||
AgentResponseUpdate,
|
||||
HostedCodeInterpreterTool,
|
||||
HostedFileContent,
|
||||
)
|
||||
@@ -60,10 +59,7 @@ async def main() -> None:
|
||||
# Collect file_ids from the response
|
||||
file_ids: list[str] = []
|
||||
|
||||
async for chunk in agent.run_stream(query):
|
||||
if not isinstance(chunk, AgentResponseUpdate):
|
||||
continue
|
||||
|
||||
async for chunk in agent.run(query, stream=True):
|
||||
for content in chunk.contents:
|
||||
if content.type == "text":
|
||||
print(content.text, end="", flush=True)
|
||||
|
||||
@@ -58,7 +58,7 @@ async def streaming_example() -> None:
|
||||
query = "What's the weather like in Portland?"
|
||||
print(f"User: {query}")
|
||||
print("Agent: ", end="", flush=True)
|
||||
async for chunk in agent.run_stream(query):
|
||||
async for chunk in agent.run(query, stream=True):
|
||||
if chunk.text:
|
||||
print(chunk.text, end="", flush=True)
|
||||
print("\n")
|
||||
|
||||
+1
-1
@@ -55,7 +55,7 @@ async def main() -> None:
|
||||
print(f"User: {query}")
|
||||
print("Agent: ", end="", flush=True)
|
||||
generated_code = ""
|
||||
async for chunk in agent.run_stream(query):
|
||||
async for chunk in agent.run(query, stream=True):
|
||||
if chunk.text:
|
||||
print(chunk.text, end="", flush=True)
|
||||
code_interpreter_chunk = get_code_interpreter_chunk(chunk)
|
||||
|
||||
@@ -60,7 +60,7 @@ async def streaming_example() -> None:
|
||||
query = "What's the weather like in Portland?"
|
||||
print(f"User: {query}")
|
||||
print("Agent: ", end="", flush=True)
|
||||
async for chunk in agent.run_stream(query):
|
||||
async for chunk in agent.run(query, stream=True):
|
||||
if chunk.text:
|
||||
print(chunk.text, end="", flush=True)
|
||||
print("\n")
|
||||
|
||||
@@ -58,7 +58,7 @@ async def streaming_example() -> None:
|
||||
query = "What's the weather like in Portland?"
|
||||
print(f"User: {query}")
|
||||
print("Agent: ", end="", flush=True)
|
||||
async for chunk in agent.run_stream(query):
|
||||
async for chunk in agent.run(query, stream=True):
|
||||
if chunk.text:
|
||||
print(chunk.text, end="", flush=True)
|
||||
print("\n")
|
||||
|
||||
+4
-4
@@ -30,10 +30,10 @@ async def handle_approvals_without_thread(query: str, agent: "AgentProtocol"):
|
||||
f"User Input Request for function from {agent.name}: {user_input_needed.function_call.name}"
|
||||
f" with arguments: {user_input_needed.function_call.arguments}"
|
||||
)
|
||||
new_inputs.append(ChatMessage("assistant", [user_input_needed]))
|
||||
new_inputs.append(ChatMessage(role="assistant", contents=[user_input_needed]))
|
||||
user_approval = input("Approve function call? (y/n): ")
|
||||
new_inputs.append(
|
||||
ChatMessage("user", [user_input_needed.to_function_approval_response(user_approval.lower() == "y")])
|
||||
ChatMessage(role="user", contents=[user_input_needed.to_function_approval_response(user_approval.lower() == "y")])
|
||||
)
|
||||
|
||||
result = await agent.run(new_inputs)
|
||||
@@ -71,8 +71,8 @@ async def handle_approvals_with_thread_streaming(query: str, agent: "AgentProtoc
|
||||
new_input_added = True
|
||||
while new_input_added:
|
||||
new_input_added = False
|
||||
new_input.append(ChatMessage("user", [query]))
|
||||
async for update in agent.run_stream(new_input, thread=thread, store=True):
|
||||
new_input.append(ChatMessage(role="user", text=query))
|
||||
async for update in agent.run(new_input, thread=thread, options={"store": True}, stream=True):
|
||||
if update.user_input_requests:
|
||||
for user_input_needed in update.user_input_requests:
|
||||
print(
|
||||
|
||||
@@ -39,7 +39,7 @@ async def streaming_example() -> None:
|
||||
query = "What is the capital of Spain?"
|
||||
print(f"User: {query}")
|
||||
print("Agent: ", end="", flush=True)
|
||||
async for chunk in agent.run_stream(query):
|
||||
async for chunk in agent.run(query, stream=True):
|
||||
if chunk.text:
|
||||
print(chunk.text, end="", flush=True)
|
||||
print("\n")
|
||||
|
||||
@@ -7,20 +7,63 @@ This folder contains examples demonstrating how to implement custom agents and c
|
||||
| File | Description |
|
||||
|------|-------------|
|
||||
| [`custom_agent.py`](custom_agent.py) | Shows how to create custom agents by extending the `BaseAgent` class. Demonstrates the `EchoAgent` implementation with both streaming and non-streaming responses, proper thread management, and message history handling. |
|
||||
| [`custom_chat_client.py`](custom_chat_client.py) | Demonstrates how to create custom chat clients by extending the `BaseChatClient` class. Shows the `EchoingChatClient` implementation and how to integrate it with `ChatAgent` using the `create_agent()` method. |
|
||||
| [`custom_chat_client.py`](../../chat_client/custom_chat_client.py) | Demonstrates how to create custom chat clients by extending the `BaseChatClient` class. Shows a `EchoingChatClient` implementation and how to integrate it with `ChatAgent` using the `as_agent()` method. |
|
||||
|
||||
## Key Takeaways
|
||||
|
||||
### Custom Agents
|
||||
- Custom agents give you complete control over the agent's behavior
|
||||
- You must implement both `run()` (for complete responses) and `run_stream()` (for streaming responses)
|
||||
- You must implement both `run()` for both the `stream=True` and `stream=False` cases
|
||||
- Use `self._normalize_messages()` to handle different input message formats
|
||||
- Use `self._notify_thread_of_new_messages()` to properly manage conversation history
|
||||
|
||||
### Custom Chat Clients
|
||||
- Custom chat clients allow you to integrate any backend service or create new LLM providers
|
||||
- You must implement both `_inner_get_response()` and `_inner_get_streaming_response()`
|
||||
- You must implement `_inner_get_response()` with a stream parameter to handle both streaming and non-streaming responses
|
||||
- Custom chat clients can be used with `ChatAgent` to leverage all agent framework features
|
||||
- Use the `create_agent()` method to easily create agents from your custom chat clients
|
||||
- Use the `as_agent()` method to easily create agents from your custom chat clients
|
||||
|
||||
Both approaches allow you to extend the framework for your specific use cases while maintaining compatibility with the broader Agent Framework ecosystem.
|
||||
Both approaches allow you to extend the framework for your specific use cases while maintaining compatibility with the broader Agent Framework ecosystem.
|
||||
|
||||
## Understanding Raw Client Classes
|
||||
|
||||
The framework provides `Raw...Client` classes (e.g., `RawOpenAIChatClient`, `RawOpenAIResponsesClient`, `RawAzureAIClient`) that are intermediate implementations without middleware, telemetry, or function invocation support.
|
||||
|
||||
### Warning: Raw Clients Should Not Normally Be Used Directly
|
||||
|
||||
**The `Raw...Client` classes should not normally be used directly.** They do not include the middleware, telemetry, or function invocation support that you most likely need. If you do use them, you should carefully consider which additional layers to apply.
|
||||
|
||||
### Layer Ordering
|
||||
|
||||
There is a defined ordering for applying layers that you should follow:
|
||||
|
||||
1. **ChatMiddlewareLayer** - Should be applied **first** because it also prepares function middleware
|
||||
2. **FunctionInvocationLayer** - Handles tool/function calling loop
|
||||
3. **ChatTelemetryLayer** - Must be **inside** the function calling loop for correct per-call telemetry
|
||||
4. **Raw...Client** - The base implementation (e.g., `RawOpenAIChatClient`)
|
||||
|
||||
Example of correct layer composition:
|
||||
|
||||
```python
|
||||
class MyCustomClient(
|
||||
ChatMiddlewareLayer[TOptions],
|
||||
FunctionInvocationLayer[TOptions],
|
||||
ChatTelemetryLayer[TOptions],
|
||||
RawOpenAIChatClient[TOptions], # or BaseChatClient for custom implementations
|
||||
Generic[TOptions],
|
||||
):
|
||||
"""Custom client with all layers correctly applied."""
|
||||
pass
|
||||
```
|
||||
|
||||
### Use Fully-Featured Clients Instead
|
||||
|
||||
For most use cases, use the fully-featured public client classes which already have all layers correctly composed:
|
||||
|
||||
- `OpenAIChatClient` - OpenAI Chat completions with all layers
|
||||
- `OpenAIResponsesClient` - OpenAI Responses API with all layers
|
||||
- `AzureOpenAIChatClient` - Azure OpenAI Chat with all layers
|
||||
- `AzureOpenAIResponsesClient` - Azure OpenAI Responses with all layers
|
||||
- `AzureAIClient` - Azure AI Project with all layers
|
||||
|
||||
These clients handle the layer composition correctly and provide the full feature set out of the box.
|
||||
|
||||
@@ -11,6 +11,8 @@ from agent_framework import (
|
||||
BaseAgent,
|
||||
ChatMessage,
|
||||
Content,
|
||||
Role,
|
||||
TextContent,
|
||||
)
|
||||
|
||||
"""
|
||||
@@ -25,7 +27,7 @@ class EchoAgent(BaseAgent):
|
||||
"""A simple custom agent that echoes user messages with a prefix.
|
||||
|
||||
This demonstrates how to create a fully custom agent by extending BaseAgent
|
||||
and implementing the required run() and run_stream() methods.
|
||||
and implementing the required run() method with stream support.
|
||||
"""
|
||||
|
||||
echo_prefix: str = "Echo: "
|
||||
@@ -53,30 +55,45 @@ class EchoAgent(BaseAgent):
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
async def run(
|
||||
def run(
|
||||
self,
|
||||
messages: str | ChatMessage | list[str] | list[ChatMessage] | None = None,
|
||||
*,
|
||||
stream: bool = False,
|
||||
thread: AgentThread | None = None,
|
||||
**kwargs: Any,
|
||||
) -> "AsyncIterable[AgentResponseUpdate] | asyncio.Future[AgentResponse]":
|
||||
"""Execute the agent and return a response.
|
||||
|
||||
Args:
|
||||
messages: The message(s) to process.
|
||||
stream: If True, return an async iterable of updates. If False, return an awaitable response.
|
||||
thread: The conversation thread (optional).
|
||||
**kwargs: Additional keyword arguments.
|
||||
|
||||
Returns:
|
||||
When stream=False: An awaitable AgentResponse containing the agent's reply.
|
||||
When stream=True: An async iterable of AgentResponseUpdate objects.
|
||||
"""
|
||||
if stream:
|
||||
return self._run_stream(messages=messages, thread=thread, **kwargs)
|
||||
return self._run(messages=messages, thread=thread, **kwargs)
|
||||
|
||||
async def _run(
|
||||
self,
|
||||
messages: str | ChatMessage | list[str] | list[ChatMessage] | None = None,
|
||||
*,
|
||||
thread: AgentThread | None = None,
|
||||
**kwargs: Any,
|
||||
) -> AgentResponse:
|
||||
"""Execute the agent and return a complete response.
|
||||
|
||||
Args:
|
||||
messages: The message(s) to process.
|
||||
thread: The conversation thread (optional).
|
||||
**kwargs: Additional keyword arguments.
|
||||
|
||||
Returns:
|
||||
An AgentResponse containing the agent's reply.
|
||||
"""
|
||||
"""Non-streaming implementation."""
|
||||
# Normalize input messages to a list
|
||||
normalized_messages = self._normalize_messages(messages)
|
||||
|
||||
if not normalized_messages:
|
||||
response_message = ChatMessage(
|
||||
"assistant",
|
||||
[Content.from_text(text="Hello! I'm a custom echo agent. Send me a message and I'll echo it back.")],
|
||||
role=Role.ASSISTANT,
|
||||
contents=[Content.from_text(text="Hello! I'm a custom echo agent. Send me a message and I'll echo it back.")],
|
||||
)
|
||||
else:
|
||||
# For simplicity, echo the last user message
|
||||
@@ -86,7 +103,7 @@ class EchoAgent(BaseAgent):
|
||||
else:
|
||||
echo_text = f"{self.echo_prefix}[Non-text message received]"
|
||||
|
||||
response_message = ChatMessage("assistant", [Content.from_text(text=echo_text)])
|
||||
response_message = ChatMessage(role=Role.ASSISTANT, contents=[Content.from_text(text=echo_text)])
|
||||
|
||||
# Notify the thread of new messages if provided
|
||||
if thread is not None:
|
||||
@@ -94,23 +111,14 @@ class EchoAgent(BaseAgent):
|
||||
|
||||
return AgentResponse(messages=[response_message])
|
||||
|
||||
async def run_stream(
|
||||
async def _run_stream(
|
||||
self,
|
||||
messages: str | ChatMessage | list[str] | list[ChatMessage] | None = None,
|
||||
*,
|
||||
thread: AgentThread | None = None,
|
||||
**kwargs: Any,
|
||||
) -> AsyncIterable[AgentResponseUpdate]:
|
||||
"""Execute the agent and yield streaming response updates.
|
||||
|
||||
Args:
|
||||
messages: The message(s) to process.
|
||||
thread: The conversation thread (optional).
|
||||
**kwargs: Additional keyword arguments.
|
||||
|
||||
Yields:
|
||||
AgentResponseUpdate objects containing chunks of the response.
|
||||
"""
|
||||
"""Streaming implementation."""
|
||||
# Normalize input messages to a list
|
||||
normalized_messages = self._normalize_messages(messages)
|
||||
|
||||
@@ -132,7 +140,7 @@ class EchoAgent(BaseAgent):
|
||||
|
||||
yield AgentResponseUpdate(
|
||||
contents=[Content.from_text(text=chunk_text)],
|
||||
role="assistant",
|
||||
role=Role.ASSISTANT,
|
||||
)
|
||||
|
||||
# Small delay to simulate streaming
|
||||
@@ -140,7 +148,7 @@ class EchoAgent(BaseAgent):
|
||||
|
||||
# Notify the thread of the complete response if provided
|
||||
if thread is not None:
|
||||
complete_response = ChatMessage("assistant", [Content.from_text(text=response_text)])
|
||||
complete_response = ChatMessage(role=Role.ASSISTANT, contents=[Content.from_text(text=response_text)])
|
||||
await self._notify_thread_of_new_messages(thread, normalized_messages, complete_response)
|
||||
|
||||
|
||||
@@ -167,7 +175,7 @@ async def main() -> None:
|
||||
query2 = "This is a streaming test"
|
||||
print(f"\nUser: {query2}")
|
||||
print("Agent: ", end="", flush=True)
|
||||
async for chunk in echo_agent.run_stream(query2):
|
||||
async for chunk in echo_agent.run(query2, stream=True):
|
||||
if chunk.text:
|
||||
print(chunk.text, end="", flush=True)
|
||||
print()
|
||||
|
||||
@@ -61,7 +61,7 @@ async def streaming_example() -> None:
|
||||
query = "What's the weather like in Tokyo?"
|
||||
print(f"User: {query}")
|
||||
print("Agent: ", end="", flush=True)
|
||||
async for chunk in agent.run_stream(query):
|
||||
async for chunk in agent.run(query, stream=True):
|
||||
if chunk.text:
|
||||
print(chunk.text, end="", flush=True)
|
||||
print("\n")
|
||||
|
||||
@@ -54,7 +54,7 @@ async def streaming_example() -> None:
|
||||
query = "What time is it in San Francisco? Use a tool call"
|
||||
print(f"User: {query}")
|
||||
print("Agent: ", end="", flush=True)
|
||||
async for chunk in agent.run_stream(query):
|
||||
async for chunk in agent.run(query, stream=True):
|
||||
if chunk.text:
|
||||
print(chunk.text, end="", flush=True)
|
||||
print("\n")
|
||||
|
||||
@@ -2,7 +2,6 @@
|
||||
|
||||
import asyncio
|
||||
|
||||
from agent_framework import TextReasoningContent
|
||||
from agent_framework.ollama import OllamaChatClient
|
||||
|
||||
"""
|
||||
@@ -18,7 +17,7 @@ https://ollama.com/
|
||||
"""
|
||||
|
||||
|
||||
async def reasoning_example() -> None:
|
||||
async def main() -> None:
|
||||
print("=== Response Reasoning Example ===")
|
||||
|
||||
agent = OllamaChatClient().as_agent(
|
||||
@@ -30,16 +29,10 @@ async def reasoning_example() -> None:
|
||||
print(f"User: {query}")
|
||||
# Enable Reasoning on per request level
|
||||
result = await agent.run(query)
|
||||
reasoning = "".join((c.text or "") for c in result.messages[-1].contents if isinstance(c, TextReasoningContent))
|
||||
reasoning = "".join((c.text or "") for c in result.messages[-1].contents if c.type == "text_reasoning")
|
||||
print(f"Reasoning: {reasoning}")
|
||||
print(f"Answer: {result}\n")
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
print("=== Basic Ollama Chat Client Agent Reasoning ===")
|
||||
|
||||
await reasoning_example()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
|
||||
@@ -33,7 +33,7 @@ async def main() -> None:
|
||||
print(f"User: {message}")
|
||||
if stream:
|
||||
print("Assistant: ", end="")
|
||||
async for chunk in client.get_streaming_response(message, tools=get_time):
|
||||
async for chunk in client.get_response(message, tools=get_time, stream=True):
|
||||
if str(chunk):
|
||||
print(str(chunk), end="")
|
||||
print("")
|
||||
|
||||
@@ -68,7 +68,7 @@ async def streaming_example() -> None:
|
||||
query = "What's the weather like in Portland?"
|
||||
print(f"User: {query}")
|
||||
print("Agent: ", end="", flush=True)
|
||||
async for chunk in agent.run_stream(query):
|
||||
async for chunk in agent.run(query, stream=True):
|
||||
if chunk.text:
|
||||
print(chunk.text, end="", flush=True)
|
||||
print("\n")
|
||||
|
||||
@@ -72,7 +72,7 @@ async def streaming_example() -> None:
|
||||
query = "What's the weather like in Portland?"
|
||||
print(f"User: {query}")
|
||||
print("Agent: ", end="", flush=True)
|
||||
async for chunk in agent.run_stream(query):
|
||||
async for chunk in agent.run(query, stream=True):
|
||||
if chunk.text:
|
||||
print(chunk.text, end="", flush=True)
|
||||
print("\n")
|
||||
|
||||
+1
-1
@@ -60,7 +60,7 @@ async def main() -> None:
|
||||
print(f"User: {query}")
|
||||
print("Agent: ", end="", flush=True)
|
||||
generated_code = ""
|
||||
async for chunk in agent.run_stream(query):
|
||||
async for chunk in agent.run(query, stream=True):
|
||||
if chunk.text:
|
||||
print(chunk.text, end="", flush=True)
|
||||
code_interpreter_chunk = get_code_interpreter_chunk(chunk)
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
import asyncio
|
||||
import os
|
||||
|
||||
from agent_framework import HostedFileSearchTool, HostedVectorStoreContent
|
||||
from agent_framework import Content, HostedFileSearchTool
|
||||
from agent_framework.openai import OpenAIAssistantProvider
|
||||
from openai import AsyncOpenAI
|
||||
|
||||
@@ -15,7 +15,7 @@ for document-based question answering and information retrieval.
|
||||
"""
|
||||
|
||||
|
||||
async def create_vector_store(client: AsyncOpenAI) -> tuple[str, HostedVectorStoreContent]:
|
||||
async def create_vector_store(client: AsyncOpenAI) -> tuple[str, Content]:
|
||||
"""Create a vector store with sample documents."""
|
||||
file = await client.files.create(
|
||||
file=("todays_weather.txt", b"The weather today is sunny with a high of 75F."), purpose="user_data"
|
||||
@@ -28,7 +28,7 @@ async def create_vector_store(client: AsyncOpenAI) -> tuple[str, HostedVectorSto
|
||||
if result.last_error is not None:
|
||||
raise Exception(f"Vector store file processing failed with status: {result.last_error.message}")
|
||||
|
||||
return file.id, HostedVectorStoreContent(vector_store_id=vector_store.id)
|
||||
return file.id, Content.from_hosted_vector_store(vector_store_id=vector_store.id)
|
||||
|
||||
|
||||
async def delete_vector_store(client: AsyncOpenAI, file_id: str, vector_store_id: str) -> None:
|
||||
@@ -56,8 +56,10 @@ async def main() -> None:
|
||||
|
||||
print(f"User: {query}")
|
||||
print("Agent: ", end="", flush=True)
|
||||
async for chunk in agent.run_stream(
|
||||
query, tool_resources={"file_search": {"vector_store_ids": [vector_store.vector_store_id]}}
|
||||
async for chunk in agent.run(
|
||||
query,
|
||||
stream=True,
|
||||
options={"tool_resources": {"file_search": {"vector_store_ids": [vector_store.vector_store_id]}}},
|
||||
):
|
||||
if chunk.text:
|
||||
print(chunk.text, end="", flush=True)
|
||||
|
||||
@@ -54,7 +54,7 @@ async def streaming_example() -> None:
|
||||
query = "What's the weather like in Portland?"
|
||||
print(f"User: {query}")
|
||||
print("Agent: ", end="", flush=True)
|
||||
async for chunk in agent.run_stream(query):
|
||||
async for chunk in agent.run(query, stream=True):
|
||||
if chunk.text:
|
||||
print(chunk.text, end="", flush=True)
|
||||
print("\n")
|
||||
|
||||
+2
-1
@@ -74,8 +74,9 @@ async def streaming_example() -> None:
|
||||
print(f"User: {query}")
|
||||
|
||||
chunks: list[str] = []
|
||||
async for chunk in agent.run_stream(
|
||||
async for chunk in agent.run(
|
||||
query,
|
||||
stream=True,
|
||||
options={
|
||||
"response_format": {
|
||||
"type": "json_schema",
|
||||
|
||||
@@ -34,7 +34,7 @@ async def main() -> None:
|
||||
|
||||
if stream:
|
||||
print("Assistant: ", end="")
|
||||
async for chunk in agent.run_stream(message):
|
||||
async for chunk in agent.run(message, stream=True):
|
||||
if chunk.text:
|
||||
print(chunk.text, end="")
|
||||
print("")
|
||||
|
||||
@@ -1,10 +1,11 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
import asyncio
|
||||
from collections.abc import Awaitable, Callable
|
||||
from random import randint
|
||||
from typing import Annotated
|
||||
|
||||
from agent_framework import ChatAgent, tool
|
||||
from agent_framework import ChatAgent, ChatContext, ChatMessage, ChatResponse, Role, chat_middleware, tool
|
||||
from agent_framework.openai import OpenAIResponsesClient
|
||||
from pydantic import Field
|
||||
|
||||
@@ -16,6 +17,47 @@ response generation, showing both streaming and non-streaming responses.
|
||||
"""
|
||||
|
||||
|
||||
@chat_middleware
|
||||
async def security_and_override_middleware(
|
||||
context: ChatContext,
|
||||
next: Callable[[ChatContext], Awaitable[None]],
|
||||
) -> None:
|
||||
"""Function-based middleware that implements security filtering and response override."""
|
||||
print("[SecurityMiddleware] Processing input...")
|
||||
|
||||
# Security check - block sensitive information
|
||||
blocked_terms = ["password", "secret", "api_key", "token"]
|
||||
|
||||
for message in context.messages:
|
||||
if message.text:
|
||||
message_lower = message.text.lower()
|
||||
for term in blocked_terms:
|
||||
if term in message_lower:
|
||||
print(f"[SecurityMiddleware] BLOCKED: Found '{term}' in message")
|
||||
|
||||
# Override the response instead of calling AI
|
||||
context.result = ChatResponse(
|
||||
messages=[
|
||||
ChatMessage(
|
||||
role=Role.ASSISTANT,
|
||||
text="I cannot process requests containing sensitive information. "
|
||||
"Please rephrase your question without including passwords, secrets, or other "
|
||||
"sensitive data.",
|
||||
)
|
||||
]
|
||||
)
|
||||
|
||||
# Set terminate flag to stop execution
|
||||
context.terminate = True
|
||||
return
|
||||
|
||||
# Continue to next middleware or AI execution
|
||||
await next(context)
|
||||
|
||||
print("[SecurityMiddleware] Response generated.")
|
||||
print(type(context.result))
|
||||
|
||||
|
||||
# NOTE: approval_mode="never_require" is for sample brevity. Use "always_require" in production; see samples/getting_started/tools/function_tool_with_approval.py and samples/getting_started/tools/function_tool_with_approval_and_threads.py.
|
||||
@tool(approval_mode="never_require")
|
||||
def get_weather(
|
||||
@@ -47,25 +89,29 @@ async def streaming_example() -> None:
|
||||
print("=== Streaming Response Example ===")
|
||||
|
||||
agent = ChatAgent(
|
||||
chat_client=OpenAIResponsesClient(),
|
||||
chat_client=OpenAIResponsesClient(
|
||||
middleware=[security_and_override_middleware],
|
||||
),
|
||||
instructions="You are a helpful weather agent.",
|
||||
tools=get_weather,
|
||||
# tools=get_weather,
|
||||
)
|
||||
|
||||
query = "What's the weather like in Portland?"
|
||||
print(f"User: {query}")
|
||||
print("Agent: ", end="", flush=True)
|
||||
async for chunk in agent.run_stream(query):
|
||||
response = agent.run(query, stream=True)
|
||||
async for chunk in response:
|
||||
if chunk.text:
|
||||
print(chunk.text, end="", flush=True)
|
||||
print("\n")
|
||||
print(f"Final Result: {await response.get_final_response()}")
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
print("=== Basic OpenAI Responses Client Agent Example ===")
|
||||
|
||||
await non_streaming_example()
|
||||
await streaming_example()
|
||||
await non_streaming_example()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
+2
-2
@@ -3,7 +3,7 @@
|
||||
import asyncio
|
||||
import base64
|
||||
|
||||
from agent_framework import Content, HostedImageGenerationTool, ImageGenerationToolResultContent
|
||||
from agent_framework import HostedImageGenerationTool
|
||||
from agent_framework.openai import OpenAIResponsesClient
|
||||
|
||||
"""
|
||||
@@ -70,7 +70,7 @@ async def main() -> None:
|
||||
# Show information about the generated image
|
||||
for message in result.messages:
|
||||
for content in message.contents:
|
||||
if isinstance(content, ImageGenerationToolResultContent) and content.outputs:
|
||||
if content.type == "image_generation" and content.outputs:
|
||||
for output in content.outputs:
|
||||
if output.type in ("data", "uri") and output.uri:
|
||||
show_image_info(output.uri)
|
||||
|
||||
@@ -55,7 +55,7 @@ async def streaming_reasoning_example() -> None:
|
||||
print(f"User: {query}")
|
||||
print(f"{agent.name}: ", end="", flush=True)
|
||||
usage = None
|
||||
async for chunk in agent.run_stream(query):
|
||||
async for chunk in agent.run(query, stream=True):
|
||||
if chunk.contents:
|
||||
for content in chunk.contents:
|
||||
if content.type == "text_reasoning":
|
||||
|
||||
+1
-1
@@ -67,7 +67,7 @@ async def main():
|
||||
await output_dir.mkdir(exist_ok=True)
|
||||
|
||||
print(" Streaming response:")
|
||||
async for update in agent.run_stream(query):
|
||||
async for update in agent.run(query, stream=True):
|
||||
for content in update.contents:
|
||||
# Handle partial images
|
||||
# The final partial image IS the complete, full-quality image. Each partial
|
||||
|
||||
+1
-1
@@ -21,7 +21,7 @@ async def logging_middleware(
|
||||
context: FunctionInvocationContext,
|
||||
next: Callable[[FunctionInvocationContext], Awaitable[None]],
|
||||
) -> None:
|
||||
"""Middleware that logs tool invocations to show the delegation flow."""
|
||||
"""MiddlewareTypes that logs tool invocations to show the delegation flow."""
|
||||
print(f"[Calling tool: {context.function.name}]")
|
||||
print(f"[Request: {context.arguments}]")
|
||||
|
||||
|
||||
+2
-5
@@ -4,9 +4,6 @@ import asyncio
|
||||
|
||||
from agent_framework import (
|
||||
ChatAgent,
|
||||
CodeInterpreterToolCallContent,
|
||||
CodeInterpreterToolResultContent,
|
||||
Content,
|
||||
HostedCodeInterpreterTool,
|
||||
)
|
||||
from agent_framework.openai import OpenAIResponsesClient
|
||||
@@ -35,8 +32,8 @@ async def main() -> None:
|
||||
print(f"Result: {result}\n")
|
||||
|
||||
for message in result.messages:
|
||||
code_blocks = [c for c in message.contents if isinstance(c, CodeInterpreterToolCallContent)]
|
||||
outputs = [c for c in message.contents if isinstance(c, CodeInterpreterToolResultContent)]
|
||||
code_blocks = [c for c in message.contents if c.type == "code_interpreter_tool_input"]
|
||||
outputs = [c for c in message.contents if c.type == "code_interpreter_tool_result"]
|
||||
if code_blocks:
|
||||
code_inputs = code_blocks[0].inputs or []
|
||||
for content in code_inputs:
|
||||
|
||||
+4
-4
@@ -2,7 +2,7 @@
|
||||
|
||||
import asyncio
|
||||
|
||||
from agent_framework import ChatAgent, HostedFileSearchTool, HostedVectorStoreContent
|
||||
from agent_framework import ChatAgent, Content, HostedFileSearchTool
|
||||
from agent_framework.openai import OpenAIResponsesClient
|
||||
|
||||
"""
|
||||
@@ -15,7 +15,7 @@ for direct document-based question answering and information retrieval.
|
||||
# Helper functions
|
||||
|
||||
|
||||
async def create_vector_store(client: OpenAIResponsesClient) -> tuple[str, HostedVectorStoreContent]:
|
||||
async def create_vector_store(client: OpenAIResponsesClient) -> tuple[str, Content]:
|
||||
"""Create a vector store with sample documents."""
|
||||
file = await client.client.files.create(
|
||||
file=("todays_weather.txt", b"The weather today is sunny with a high of 75F."), purpose="user_data"
|
||||
@@ -28,7 +28,7 @@ async def create_vector_store(client: OpenAIResponsesClient) -> tuple[str, Hoste
|
||||
if result.last_error is not None:
|
||||
raise Exception(f"Vector store file processing failed with status: {result.last_error.message}")
|
||||
|
||||
return file.id, HostedVectorStoreContent(vector_store_id=vector_store.id)
|
||||
return file.id, Content.from_hosted_vector_store(vector_store_id=vector_store.id)
|
||||
|
||||
|
||||
async def delete_vector_store(client: OpenAIResponsesClient, file_id: str, vector_store_id: str) -> None:
|
||||
@@ -55,7 +55,7 @@ async def main() -> None:
|
||||
|
||||
if stream:
|
||||
print("Assistant: ", end="")
|
||||
async for chunk in agent.run_stream(message):
|
||||
async for chunk in agent.run(message, stream=True):
|
||||
if chunk.text:
|
||||
print(chunk.text, end="")
|
||||
print("")
|
||||
|
||||
+4
-4
@@ -29,10 +29,10 @@ async def handle_approvals_without_thread(query: str, agent: "AgentProtocol"):
|
||||
f"User Input Request for function from {agent.name}: {user_input_needed.function_call.name}"
|
||||
f" with arguments: {user_input_needed.function_call.arguments}"
|
||||
)
|
||||
new_inputs.append(ChatMessage("assistant", [user_input_needed]))
|
||||
new_inputs.append(ChatMessage(role="assistant", contents=[user_input_needed]))
|
||||
user_approval = input("Approve function call? (y/n): ")
|
||||
new_inputs.append(
|
||||
ChatMessage("user", [user_input_needed.to_function_approval_response(user_approval.lower() == "y")])
|
||||
ChatMessage(role="user", contents=[user_input_needed.to_function_approval_response(user_approval.lower() == "y")])
|
||||
)
|
||||
|
||||
result = await agent.run(new_inputs)
|
||||
@@ -70,8 +70,8 @@ async def handle_approvals_with_thread_streaming(query: str, agent: "AgentProtoc
|
||||
new_input_added = True
|
||||
while new_input_added:
|
||||
new_input_added = False
|
||||
new_input.append(ChatMessage("user", [query]))
|
||||
async for update in agent.run_stream(new_input, thread=thread, store=True):
|
||||
new_input.append(ChatMessage(role="user", text=query))
|
||||
async for update in agent.run(new_input, thread=thread, stream=True, options={"store": True}):
|
||||
if update.user_input_requests:
|
||||
for user_input_needed in update.user_input_requests:
|
||||
print(
|
||||
|
||||
+2
-2
@@ -35,7 +35,7 @@ async def streaming_with_mcp(show_raw_stream: bool = False) -> None:
|
||||
query1 = "How to create an Azure storage account using az cli?"
|
||||
print(f"User: {query1}")
|
||||
print(f"{agent.name}: ", end="")
|
||||
async for chunk in agent.run_stream(query1):
|
||||
async for chunk in agent.run(query1, stream=True):
|
||||
if show_raw_stream:
|
||||
print("Streamed event: ", chunk.raw_representation.raw_representation) # type:ignore
|
||||
elif chunk.text:
|
||||
@@ -46,7 +46,7 @@ async def streaming_with_mcp(show_raw_stream: bool = False) -> None:
|
||||
query2 = "What is Microsoft Agent Framework?"
|
||||
print(f"User: {query2}")
|
||||
print(f"{agent.name}: ", end="")
|
||||
async for chunk in agent.run_stream(query2):
|
||||
async for chunk in agent.run(query2, stream=True):
|
||||
if show_raw_stream:
|
||||
print("Streamed event: ", chunk.raw_representation.raw_representation) # type:ignore
|
||||
elif chunk.text:
|
||||
|
||||
+2
-1
@@ -74,8 +74,9 @@ async def streaming_example() -> None:
|
||||
print(f"User: {query}")
|
||||
|
||||
chunks: list[str] = []
|
||||
async for chunk in agent.run_stream(
|
||||
async for chunk in agent.run(
|
||||
query,
|
||||
stream=True,
|
||||
options={
|
||||
"response_format": {
|
||||
"type": "json_schema",
|
||||
|
||||
+4
-4
@@ -59,16 +59,16 @@ async def streaming_example() -> None:
|
||||
query = "Tell me about Tokyo, Japan"
|
||||
print(f"User: {query}")
|
||||
|
||||
# Get structured response from streaming agent using AgentResponse.from_agent_response_generator
|
||||
# Get structured response from streaming agent using AgentResponse.from_update_generator
|
||||
# This method collects all streaming updates and combines them into a single AgentResponse
|
||||
result = await AgentResponse.from_agent_response_generator(
|
||||
agent.run_stream(query, options={"response_format": OutputStruct}),
|
||||
result = await AgentResponse.from_update_generator(
|
||||
agent.run(query, stream=True, options={"response_format": OutputStruct}),
|
||||
output_format_type=OutputStruct,
|
||||
)
|
||||
|
||||
# Access the structured output using the parsed value
|
||||
if structured_data := result.value:
|
||||
print("Structured Output (from streaming with AgentResponse.from_agent_response_generator):")
|
||||
print("Structured Output (from streaming with AgentResponse.from_update_generator):")
|
||||
print(f"City: {structured_data.city}")
|
||||
print(f"Description: {structured_data.description}")
|
||||
else:
|
||||
|
||||
+1
-1
@@ -34,7 +34,7 @@ async def main() -> None:
|
||||
|
||||
if stream:
|
||||
print("Assistant: ", end="")
|
||||
async for chunk in agent.run_stream(message):
|
||||
async for chunk in agent.run(message, stream=True):
|
||||
if chunk.text:
|
||||
print(chunk.text, end="")
|
||||
print("")
|
||||
|
||||
@@ -14,6 +14,7 @@ This folder contains simple examples demonstrating direct usage of various chat
|
||||
| [`openai_assistants_client.py`](openai_assistants_client.py) | Direct usage of OpenAI Assistants Client for basic chat interactions with OpenAI assistants. |
|
||||
| [`openai_chat_client.py`](openai_chat_client.py) | Direct usage of OpenAI Chat Client for chat interactions with OpenAI models. |
|
||||
| [`openai_responses_client.py`](openai_responses_client.py) | Direct usage of OpenAI Responses Client for structured response generation with OpenAI models. |
|
||||
| [`custom_chat_client.py`](custom_chat_client.py) | Demonstrates how to create custom chat clients by extending the `BaseChatClient` class. Shows a `EchoingChatClient` implementation and how to integrate it with `ChatAgent` using the `as_agent()` method. |
|
||||
|
||||
## Environment Variables
|
||||
|
||||
@@ -37,4 +38,4 @@ Depending on which client you're using, set the appropriate environment variable
|
||||
- `OLLAMA_HOST`: Your Ollama server URL (defaults to `http://localhost:11434` if not set)
|
||||
- `OLLAMA_MODEL_ID`: The Ollama model to use for chat (e.g., `llama3.2`, `llama2`, `codellama`)
|
||||
|
||||
> **Note**: For Ollama, ensure you have Ollama installed and running locally with at least one model downloaded. Visit [https://ollama.com/](https://ollama.com/) for installation instructions.
|
||||
> **Note**: For Ollama, ensure you have Ollama installed and running locally with at least one model downloaded. Visit [https://ollama.com/](https://ollama.com/) for installation instructions.
|
||||
|
||||
@@ -36,7 +36,7 @@ async def main() -> None:
|
||||
print(f"User: {message}")
|
||||
if stream:
|
||||
print("Assistant: ", end="")
|
||||
async for chunk in client.get_streaming_response(message, tools=get_weather):
|
||||
async for chunk in client.get_response(message, tools=get_weather, stream=True):
|
||||
if str(chunk):
|
||||
print(str(chunk), end="")
|
||||
print("")
|
||||
|
||||
@@ -36,7 +36,7 @@ async def main() -> None:
|
||||
print(f"User: {message}")
|
||||
if stream:
|
||||
print("Assistant: ", end="")
|
||||
async for chunk in client.get_streaming_response(message, tools=get_weather):
|
||||
async for chunk in client.get_response(message, tools=get_weather, stream=True):
|
||||
if str(chunk):
|
||||
print(str(chunk), end="")
|
||||
print("")
|
||||
|
||||
@@ -36,7 +36,7 @@ async def main() -> None:
|
||||
print(f"User: {message}")
|
||||
if stream:
|
||||
print("Assistant: ", end="")
|
||||
async for chunk in client.get_streaming_response(message, tools=get_weather):
|
||||
async for chunk in client.get_response(message, tools=get_weather, stream=True):
|
||||
if str(chunk):
|
||||
print(str(chunk), end="")
|
||||
print("")
|
||||
|
||||
@@ -42,21 +42,19 @@ async def main() -> None:
|
||||
stream = True
|
||||
print(f"User: {message}")
|
||||
if stream:
|
||||
response = await ChatResponse.from_update_generator(
|
||||
client.get_streaming_response(message, tools=get_weather, options={"response_format": OutputStruct}),
|
||||
response = await ChatResponse.from_chat_response_generator(
|
||||
client.get_response(message, tools=get_weather, options={"response_format": OutputStruct}, stream=True),
|
||||
output_format_type=OutputStruct,
|
||||
)
|
||||
try:
|
||||
result = response.value
|
||||
if result := response.try_parse_value(OutputStruct):
|
||||
print(f"Assistant: {result}")
|
||||
except Exception:
|
||||
else:
|
||||
print(f"Assistant: {response.text}")
|
||||
else:
|
||||
response = await client.get_response(message, tools=get_weather, options={"response_format": OutputStruct})
|
||||
try:
|
||||
result = response.value
|
||||
if result := response.try_parse_value(OutputStruct):
|
||||
print(f"Assistant: {result}")
|
||||
except Exception:
|
||||
else:
|
||||
print(f"Assistant: {response.text}")
|
||||
|
||||
|
||||
|
||||
+57
-35
@@ -3,40 +3,54 @@
|
||||
import asyncio
|
||||
import random
|
||||
import sys
|
||||
from collections.abc import AsyncIterable, MutableSequence
|
||||
from typing import Any, ClassVar, Generic
|
||||
from collections.abc import AsyncIterable, Awaitable, Mapping, Sequence
|
||||
from typing import Any, ClassVar, Generic, TypedDict
|
||||
|
||||
from agent_framework import (
|
||||
BaseChatClient,
|
||||
ChatMessage,
|
||||
ChatMiddlewareLayer,
|
||||
ChatOptions,
|
||||
ChatResponse,
|
||||
ChatResponseUpdate,
|
||||
Content,
|
||||
use_chat_middleware,
|
||||
use_function_invocation,
|
||||
FunctionInvocationLayer,
|
||||
ResponseStream,
|
||||
Role,
|
||||
)
|
||||
from agent_framework._clients import TOptions_co
|
||||
from agent_framework.observability import ChatTelemetryLayer
|
||||
|
||||
if sys.version_info >= (3, 13):
|
||||
from typing import TypeVar
|
||||
else:
|
||||
from typing_extensions import TypeVar
|
||||
if sys.version_info >= (3, 12):
|
||||
from typing import override # type: ignore # pragma: no cover
|
||||
else:
|
||||
from typing_extensions import override # type: ignore[import] # pragma: no cover
|
||||
|
||||
|
||||
"""
|
||||
Custom Chat Client Implementation Example
|
||||
|
||||
This sample demonstrates implementing a custom chat client by extending BaseChatClient class,
|
||||
showing integration with ChatAgent and both streaming and non-streaming responses.
|
||||
This sample demonstrates implementing a custom chat client and optionally composing
|
||||
middleware, telemetry, and function invocation layers explicitly.
|
||||
"""
|
||||
|
||||
TOptions_co = TypeVar(
|
||||
"TOptions_co",
|
||||
bound=TypedDict, # type: ignore[valid-type]
|
||||
default="ChatOptions",
|
||||
covariant=True,
|
||||
)
|
||||
|
||||
|
||||
@use_function_invocation
|
||||
@use_chat_middleware
|
||||
class EchoingChatClient(BaseChatClient[TOptions_co], Generic[TOptions_co]):
|
||||
"""A custom chat client that echoes messages back with modifications.
|
||||
|
||||
This demonstrates how to implement a custom chat client by extending BaseChatClient
|
||||
and implementing the required _inner_get_response() and _inner_get_streaming_response() methods.
|
||||
and implementing the required _inner_get_response() method.
|
||||
"""
|
||||
|
||||
OTEL_PROVIDER_NAME: ClassVar[str] = "EchoingChatClient"
|
||||
@@ -52,13 +66,14 @@ class EchoingChatClient(BaseChatClient[TOptions_co], Generic[TOptions_co]):
|
||||
self.prefix = prefix
|
||||
|
||||
@override
|
||||
async def _inner_get_response(
|
||||
def _inner_get_response(
|
||||
self,
|
||||
*,
|
||||
messages: MutableSequence[ChatMessage],
|
||||
options: dict[str, Any],
|
||||
messages: Sequence[ChatMessage],
|
||||
stream: bool = False,
|
||||
options: Mapping[str, Any],
|
||||
**kwargs: Any,
|
||||
) -> ChatResponse:
|
||||
) -> Awaitable[ChatResponse] | ResponseStream[ChatResponseUpdate, ChatResponse]:
|
||||
"""Echo back the user's message with a prefix."""
|
||||
if not messages:
|
||||
response_text = "No messages to echo!"
|
||||
@@ -66,7 +81,7 @@ class EchoingChatClient(BaseChatClient[TOptions_co], Generic[TOptions_co]):
|
||||
# Echo the last user message
|
||||
last_user_message = None
|
||||
for message in reversed(messages):
|
||||
if message.role == "user":
|
||||
if message.role == Role.USER:
|
||||
last_user_message = message
|
||||
break
|
||||
|
||||
@@ -75,39 +90,46 @@ class EchoingChatClient(BaseChatClient[TOptions_co], Generic[TOptions_co]):
|
||||
else:
|
||||
response_text = f"{self.prefix} [No text message found]"
|
||||
|
||||
response_message = ChatMessage("assistant", [Content.from_text(text=response_text)])
|
||||
response_message = ChatMessage(role=Role.ASSISTANT, contents=[Content.from_text(response_text)])
|
||||
|
||||
return ChatResponse(
|
||||
response = ChatResponse(
|
||||
messages=[response_message],
|
||||
model_id="echo-model-v1",
|
||||
response_id=f"echo-resp-{random.randint(1000, 9999)}",
|
||||
)
|
||||
|
||||
@override
|
||||
async def _inner_get_streaming_response(
|
||||
self,
|
||||
*,
|
||||
messages: MutableSequence[ChatMessage],
|
||||
options: dict[str, Any],
|
||||
**kwargs: Any,
|
||||
) -> AsyncIterable[ChatResponseUpdate]:
|
||||
"""Stream back the echoed message character by character."""
|
||||
# Get the complete response first
|
||||
response = await self._inner_get_response(messages=messages, options=options, **kwargs)
|
||||
if not stream:
|
||||
|
||||
if response.messages:
|
||||
response_text = response.messages[0].text or ""
|
||||
async def _get_response() -> ChatResponse:
|
||||
return response
|
||||
|
||||
# Stream character by character
|
||||
for char in response_text:
|
||||
return _get_response()
|
||||
|
||||
async def _stream() -> AsyncIterable[ChatResponseUpdate]:
|
||||
response_text_local = response_message.text or ""
|
||||
for char in response_text_local:
|
||||
yield ChatResponseUpdate(
|
||||
contents=[Content.from_text(text=char)],
|
||||
role="assistant",
|
||||
contents=[Content.from_text(char)],
|
||||
role=Role.ASSISTANT,
|
||||
response_id=f"echo-stream-resp-{random.randint(1000, 9999)}",
|
||||
model_id="echo-model-v1",
|
||||
)
|
||||
await asyncio.sleep(0.05)
|
||||
|
||||
return ResponseStream(_stream(), finalizer=lambda updates: response)
|
||||
|
||||
|
||||
class EchoingChatClientWithLayers( # type: ignore[misc,type-var]
|
||||
ChatMiddlewareLayer[TOptions_co],
|
||||
ChatTelemetryLayer[TOptions_co],
|
||||
FunctionInvocationLayer[TOptions_co],
|
||||
EchoingChatClient[TOptions_co],
|
||||
Generic[TOptions_co],
|
||||
):
|
||||
"""Echoing chat client that explicitly composes middleware, telemetry, and function layers."""
|
||||
|
||||
OTEL_PROVIDER_NAME: ClassVar[str] = "EchoingChatClientWithLayers"
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
"""Demonstrates how to implement and use a custom chat client with ChatAgent."""
|
||||
@@ -116,7 +138,7 @@ async def main() -> None:
|
||||
# Create the custom chat client
|
||||
print("--- EchoingChatClient Example ---")
|
||||
|
||||
echo_client = EchoingChatClient(prefix="🔊 Echo:")
|
||||
echo_client = EchoingChatClientWithLayers(prefix="🔊 Echo:")
|
||||
|
||||
# Use the chat client directly
|
||||
print("Using chat client directly:")
|
||||
@@ -141,7 +163,7 @@ async def main() -> None:
|
||||
query2 = "Stream this message back to me"
|
||||
print(f"\nUser: {query2}")
|
||||
print("Agent: ", end="", flush=True)
|
||||
async for chunk in echo_agent.run_stream(query2):
|
||||
async for chunk in echo_agent.run(query2, stream=True):
|
||||
if chunk.text:
|
||||
print(chunk.text, end="", flush=True)
|
||||
print()
|
||||
@@ -34,7 +34,7 @@ async def main() -> None:
|
||||
print(f"User: {message}")
|
||||
if stream:
|
||||
print("Assistant: ", end="")
|
||||
async for chunk in client.get_streaming_response(message, tools=get_weather):
|
||||
async for chunk in client.get_response(message, tools=get_weather, stream=True):
|
||||
if str(chunk):
|
||||
print(str(chunk), end="")
|
||||
print("")
|
||||
|
||||
@@ -34,7 +34,7 @@ async def main() -> None:
|
||||
print(f"User: {message}")
|
||||
if stream:
|
||||
print("Assistant: ", end="")
|
||||
async for chunk in client.get_streaming_response(message, tools=get_weather):
|
||||
async for chunk in client.get_response(message, tools=get_weather, stream=True):
|
||||
if chunk.text:
|
||||
print(chunk.text, end="")
|
||||
print("")
|
||||
|
||||
@@ -30,14 +30,14 @@ def get_weather(
|
||||
async def main() -> None:
|
||||
client = OpenAIResponsesClient()
|
||||
message = "What's the weather in Amsterdam and in Paris?"
|
||||
stream = False
|
||||
stream = True
|
||||
print(f"User: {message}")
|
||||
if stream:
|
||||
print("Assistant: ", end="")
|
||||
async for chunk in client.get_streaming_response(message, tools=get_weather):
|
||||
if chunk.text:
|
||||
print(chunk.text, end="")
|
||||
print("")
|
||||
response = client.get_response(message, stream=True, tools=get_weather)
|
||||
# TODO: review names of the methods, could be related to things like HTTP clients?
|
||||
response.with_update_hook(lambda chunk: print(chunk.text, end=""))
|
||||
await response.get_final_response()
|
||||
else:
|
||||
response = await client.get_response(message, tools=get_weather)
|
||||
print(f"Assistant: {response}")
|
||||
|
||||
+1
-1
@@ -130,7 +130,7 @@ async def main() -> None:
|
||||
print("Agent: ", end="", flush=True)
|
||||
|
||||
# Stream response
|
||||
async for chunk in agent.run_stream(user_input):
|
||||
async for chunk in agent.run(user_input, stream=True):
|
||||
if chunk.text:
|
||||
print(chunk.text, end="", flush=True)
|
||||
|
||||
|
||||
+1
-1
@@ -86,7 +86,7 @@ async def main() -> None:
|
||||
print("Agent: ", end="", flush=True)
|
||||
|
||||
# Stream response
|
||||
async for chunk in agent.run_stream(user_input):
|
||||
async for chunk in agent.run(user_input, stream=True):
|
||||
if chunk.text:
|
||||
print(chunk.text, end="", flush=True)
|
||||
|
||||
|
||||
@@ -14,6 +14,8 @@ from agent_framework import (
|
||||
ChatResponseUpdate,
|
||||
Content,
|
||||
FunctionInvocationContext,
|
||||
Role,
|
||||
TextContent,
|
||||
chat_middleware,
|
||||
function_middleware,
|
||||
tool,
|
||||
@@ -42,7 +44,7 @@ async def security_filter_middleware(
|
||||
|
||||
# Check only the last message (most recent user input)
|
||||
last_message = context.messages[-1] if context.messages else None
|
||||
if last_message and last_message.role == "user" and last_message.text:
|
||||
if last_message and last_message.role == Role.USER and last_message.text:
|
||||
message_lower = last_message.text.lower()
|
||||
for term in blocked_terms:
|
||||
if term in message_lower:
|
||||
@@ -52,12 +54,12 @@ async def security_filter_middleware(
|
||||
"or other sensitive data."
|
||||
)
|
||||
|
||||
if context.is_streaming:
|
||||
if context.stream:
|
||||
# Streaming mode: return async generator
|
||||
async def blocked_stream() -> AsyncIterable[ChatResponseUpdate]:
|
||||
yield ChatResponseUpdate(
|
||||
contents=[Content.from_text(text=error_message)],
|
||||
role="assistant",
|
||||
role=Role.ASSISTANT,
|
||||
)
|
||||
|
||||
context.result = blocked_stream()
|
||||
@@ -66,7 +68,7 @@ async def security_filter_middleware(
|
||||
context.result = ChatResponse(
|
||||
messages=[
|
||||
ChatMessage(
|
||||
role="assistant",
|
||||
role=Role.ASSISTANT,
|
||||
text=error_message,
|
||||
)
|
||||
]
|
||||
|
||||
@@ -3,8 +3,8 @@
|
||||
This worker registers agents as durable entities and continuously listens for requests.
|
||||
The worker should run as a background service, processing incoming agent requests.
|
||||
|
||||
Prerequisites:
|
||||
- Set AZURE_OPENAI_ENDPOINT and AZURE_OPENAI_CHAT_DEPLOYMENT_NAME
|
||||
Prerequisites:
|
||||
- Set AZURE_OPENAI_ENDPOINT and AZURE_OPENAI_CHAT_DEPLOYMENT_NAME
|
||||
(plus AZURE_OPENAI_API_KEY or Azure CLI authentication)
|
||||
- Start a Durable Task Scheduler (e.g., using Docker)
|
||||
"""
|
||||
@@ -25,7 +25,7 @@ logger = logging.getLogger(__name__)
|
||||
|
||||
def create_joker_agent() -> ChatAgent:
|
||||
"""Create the Joker agent using Azure OpenAI.
|
||||
|
||||
|
||||
Returns:
|
||||
ChatAgent: The configured Joker agent
|
||||
"""
|
||||
@@ -41,12 +41,12 @@ def get_worker(
|
||||
log_handler: logging.Handler | None = None
|
||||
) -> DurableTaskSchedulerWorker:
|
||||
"""Create a configured DurableTaskSchedulerWorker.
|
||||
|
||||
|
||||
Args:
|
||||
taskhub: Task hub name (defaults to TASKHUB env var or "default")
|
||||
endpoint: Scheduler endpoint (defaults to ENDPOINT env var or "http://localhost:8080")
|
||||
log_handler: Optional logging handler for worker logging
|
||||
|
||||
|
||||
Returns:
|
||||
Configured DurableTaskSchedulerWorker instance
|
||||
"""
|
||||
@@ -69,10 +69,10 @@ def get_worker(
|
||||
|
||||
def setup_worker(worker: DurableTaskSchedulerWorker) -> DurableAIAgentWorker:
|
||||
"""Set up the worker with agents registered.
|
||||
|
||||
|
||||
Args:
|
||||
worker: The DurableTaskSchedulerWorker instance
|
||||
|
||||
|
||||
Returns:
|
||||
DurableAIAgentWorker with agents registered
|
||||
"""
|
||||
|
||||
@@ -4,8 +4,8 @@ This worker registers two agents - a weather assistant and a math assistant - ea
|
||||
with their own specialized tools. This demonstrates how to host multiple agents
|
||||
with different capabilities in a single worker process.
|
||||
|
||||
Prerequisites:
|
||||
- Set AZURE_OPENAI_ENDPOINT and AZURE_OPENAI_CHAT_DEPLOYMENT_NAME
|
||||
Prerequisites:
|
||||
- Set AZURE_OPENAI_ENDPOINT and AZURE_OPENAI_CHAT_DEPLOYMENT_NAME
|
||||
(plus AZURE_OPENAI_API_KEY or Azure CLI authentication)
|
||||
- Start a Durable Task Scheduler (e.g., using Docker)
|
||||
"""
|
||||
@@ -15,6 +15,7 @@ import logging
|
||||
import os
|
||||
from typing import Any
|
||||
|
||||
from agent_framework import tool
|
||||
from agent_framework.azure import AzureOpenAIChatClient, DurableAIAgentWorker
|
||||
from azure.identity import AzureCliCredential, DefaultAzureCredential
|
||||
from durabletask.azuremanaged.worker import DurableTaskSchedulerWorker
|
||||
@@ -28,6 +29,7 @@ WEATHER_AGENT_NAME = "WeatherAgent"
|
||||
MATH_AGENT_NAME = "MathAgent"
|
||||
|
||||
|
||||
@tool
|
||||
def get_weather(location: str) -> dict[str, Any]:
|
||||
"""Get current weather for a location."""
|
||||
logger.info(f"🔧 [TOOL CALLED] get_weather(location={location})")
|
||||
@@ -41,11 +43,10 @@ def get_weather(location: str) -> dict[str, Any]:
|
||||
return result
|
||||
|
||||
|
||||
@tool
|
||||
def calculate_tip(bill_amount: float, tip_percentage: float = 15.0) -> dict[str, Any]:
|
||||
"""Calculate tip amount and total bill."""
|
||||
logger.info(
|
||||
f"🔧 [TOOL CALLED] calculate_tip(bill_amount={bill_amount}, tip_percentage={tip_percentage})"
|
||||
)
|
||||
logger.info(f"🔧 [TOOL CALLED] calculate_tip(bill_amount={bill_amount}, tip_percentage={tip_percentage})")
|
||||
tip = bill_amount * (tip_percentage / 100)
|
||||
total = bill_amount + tip
|
||||
result = {
|
||||
@@ -60,7 +61,7 @@ def calculate_tip(bill_amount: float, tip_percentage: float = 15.0) -> dict[str,
|
||||
|
||||
def create_weather_agent():
|
||||
"""Create the Weather agent using Azure OpenAI.
|
||||
|
||||
|
||||
Returns:
|
||||
ChatAgent: The configured Weather agent with weather tool
|
||||
"""
|
||||
@@ -73,7 +74,7 @@ def create_weather_agent():
|
||||
|
||||
def create_math_agent():
|
||||
"""Create the Math agent using Azure OpenAI.
|
||||
|
||||
|
||||
Returns:
|
||||
ChatAgent: The configured Math agent with calculation tools
|
||||
"""
|
||||
@@ -85,17 +86,15 @@ def create_math_agent():
|
||||
|
||||
|
||||
def get_worker(
|
||||
taskhub: str | None = None,
|
||||
endpoint: str | None = None,
|
||||
log_handler: logging.Handler | None = None
|
||||
taskhub: str | None = None, endpoint: str | None = None, log_handler: logging.Handler | None = None
|
||||
) -> DurableTaskSchedulerWorker:
|
||||
"""Create a configured DurableTaskSchedulerWorker.
|
||||
|
||||
|
||||
Args:
|
||||
taskhub: Task hub name (defaults to TASKHUB env var or "default")
|
||||
endpoint: Scheduler endpoint (defaults to ENDPOINT env var or "http://localhost:8080")
|
||||
log_handler: Optional logging handler for worker logging
|
||||
|
||||
|
||||
Returns:
|
||||
Configured DurableTaskSchedulerWorker instance
|
||||
"""
|
||||
@@ -112,16 +111,16 @@ def get_worker(
|
||||
secure_channel=endpoint_url != "http://localhost:8080",
|
||||
taskhub=taskhub_name,
|
||||
token_credential=credential,
|
||||
log_handler=log_handler
|
||||
log_handler=log_handler,
|
||||
)
|
||||
|
||||
|
||||
def setup_worker(worker: DurableTaskSchedulerWorker) -> DurableAIAgentWorker:
|
||||
"""Set up the worker with multiple agents registered.
|
||||
|
||||
|
||||
Args:
|
||||
worker: The DurableTaskSchedulerWorker instance
|
||||
|
||||
|
||||
Returns:
|
||||
DurableAIAgentWorker with agents registered
|
||||
"""
|
||||
|
||||
@@ -4,10 +4,12 @@
|
||||
|
||||
In a real application, these would call actual weather and events APIs.
|
||||
"""
|
||||
|
||||
from typing import Annotated
|
||||
|
||||
from agent_framework import tool
|
||||
|
||||
|
||||
@tool
|
||||
def get_weather_forecast(
|
||||
destination: Annotated[str, "The destination city or location"],
|
||||
date: Annotated[str, 'The date for the forecast (e.g., "2025-01-15" or "next Monday")'],
|
||||
@@ -64,6 +66,7 @@ Low: {low_f}°F ({low_c}°C)
|
||||
Recommendation: {recommendation}"""
|
||||
|
||||
|
||||
@tool
|
||||
def get_local_events(
|
||||
destination: Annotated[str, "The destination city or location"],
|
||||
date: Annotated[str, 'The date to search for events (e.g., "2025-01-15" or "next week")'],
|
||||
|
||||
@@ -18,7 +18,7 @@ from azure.identity.aio import AzureCliCredential
|
||||
from pydantic import Field
|
||||
|
||||
"""
|
||||
Agent-Level and Run-Level Middleware Example
|
||||
Agent-Level and Run-Level MiddlewareTypes Example
|
||||
|
||||
This sample demonstrates the difference between agent-level and run-level middleware:
|
||||
|
||||
@@ -107,7 +107,7 @@ async def debugging_middleware(
|
||||
"""Run-level debugging middleware for troubleshooting specific runs."""
|
||||
print("[Debug] Debug mode enabled for this run")
|
||||
print(f"[Debug] Messages count: {len(context.messages)}")
|
||||
print(f"[Debug] Is streaming: {context.is_streaming}")
|
||||
print(f"[Debug] Is streaming: {context.stream}")
|
||||
|
||||
# Log existing metadata from agent middleware
|
||||
if context.metadata:
|
||||
@@ -163,7 +163,7 @@ async def function_logging_middleware(
|
||||
|
||||
async def main() -> None:
|
||||
"""Example demonstrating agent-level and run-level middleware."""
|
||||
print("=== Agent-Level and Run-Level Middleware Example ===\n")
|
||||
print("=== Agent-Level and Run-Level MiddlewareTypes Example ===\n")
|
||||
|
||||
# For authentication, run `az login` command in terminal or replace AzureCliCredential with preferred
|
||||
# authentication option.
|
||||
|
||||
@@ -18,7 +18,7 @@ from azure.identity.aio import AzureCliCredential
|
||||
from pydantic import Field
|
||||
|
||||
"""
|
||||
Chat Middleware Example
|
||||
Chat MiddlewareTypes Example
|
||||
|
||||
This sample demonstrates how to use chat middleware to observe and override
|
||||
inputs sent to AI models. Chat middleware intercepts chat requests before they reach
|
||||
@@ -31,8 +31,8 @@ the underlying AI service, allowing you to:
|
||||
The example covers:
|
||||
- Class-based chat middleware inheriting from ChatMiddleware
|
||||
- Function-based chat middleware with @chat_middleware decorator
|
||||
- Middleware registration at agent level (applies to all runs)
|
||||
- Middleware registration at run level (applies to specific run only)
|
||||
- MiddlewareTypes registration at agent level (applies to all runs)
|
||||
- MiddlewareTypes registration at run level (applies to specific run only)
|
||||
"""
|
||||
|
||||
|
||||
@@ -137,7 +137,7 @@ async def security_and_override_middleware(
|
||||
async def class_based_chat_middleware() -> None:
|
||||
"""Demonstrate class-based middleware at agent level."""
|
||||
print("\n" + "=" * 60)
|
||||
print("Class-based Chat Middleware (Agent Level)")
|
||||
print("Class-based Chat MiddlewareTypes (Agent Level)")
|
||||
print("=" * 60)
|
||||
|
||||
# For authentication, run `az login` command in terminal or replace AzureCliCredential with preferred
|
||||
@@ -161,7 +161,7 @@ async def class_based_chat_middleware() -> None:
|
||||
async def function_based_chat_middleware() -> None:
|
||||
"""Demonstrate function-based middleware at agent level."""
|
||||
print("\n" + "=" * 60)
|
||||
print("Function-based Chat Middleware (Agent Level)")
|
||||
print("Function-based Chat MiddlewareTypes (Agent Level)")
|
||||
print("=" * 60)
|
||||
|
||||
async with (
|
||||
@@ -191,7 +191,7 @@ async def function_based_chat_middleware() -> None:
|
||||
async def run_level_middleware() -> None:
|
||||
"""Demonstrate middleware registration at run level."""
|
||||
print("\n" + "=" * 60)
|
||||
print("Run-level Chat Middleware")
|
||||
print("Run-level Chat MiddlewareTypes")
|
||||
print("=" * 60)
|
||||
|
||||
async with (
|
||||
@@ -204,14 +204,14 @@ async def run_level_middleware() -> None:
|
||||
) as agent,
|
||||
):
|
||||
# Scenario 1: Run without any middleware
|
||||
print("\n--- Scenario 1: No Middleware ---")
|
||||
print("\n--- Scenario 1: No MiddlewareTypes ---")
|
||||
query = "What's the weather in Tokyo?"
|
||||
print(f"User: {query}")
|
||||
result = await agent.run(query)
|
||||
print(f"Response: {result.text if result.text else 'No response'}")
|
||||
|
||||
# Scenario 2: Run with specific middleware for this call only (both enhancement and security)
|
||||
print("\n--- Scenario 2: With Run-level Middleware ---")
|
||||
print("\n--- Scenario 2: With Run-level MiddlewareTypes ---")
|
||||
print(f"User: {query}")
|
||||
result = await agent.run(
|
||||
query,
|
||||
@@ -223,7 +223,7 @@ async def run_level_middleware() -> None:
|
||||
print(f"Response: {result.text if result.text else 'No response'}")
|
||||
|
||||
# Scenario 3: Security test with run-level middleware
|
||||
print("\n--- Scenario 3: Security Test with Run-level Middleware ---")
|
||||
print("\n--- Scenario 3: Security Test with Run-level MiddlewareTypes ---")
|
||||
query = "Can you help me with my secret API key?"
|
||||
print(f"User: {query}")
|
||||
result = await agent.run(
|
||||
@@ -235,7 +235,7 @@ async def run_level_middleware() -> None:
|
||||
|
||||
async def main() -> None:
|
||||
"""Run all chat middleware examples."""
|
||||
print("Chat Middleware Examples")
|
||||
print("Chat MiddlewareTypes Examples")
|
||||
print("========================")
|
||||
|
||||
await class_based_chat_middleware()
|
||||
|
||||
@@ -20,7 +20,7 @@ from azure.identity.aio import AzureCliCredential
|
||||
from pydantic import Field
|
||||
|
||||
"""
|
||||
Class-based Middleware Example
|
||||
Class-based MiddlewareTypes Example
|
||||
|
||||
This sample demonstrates how to implement middleware using class-based approach by inheriting
|
||||
from AgentMiddleware and FunctionMiddleware base classes. The example includes:
|
||||
@@ -95,7 +95,7 @@ class LoggingFunctionMiddleware(FunctionMiddleware):
|
||||
|
||||
async def main() -> None:
|
||||
"""Example demonstrating class-based middleware."""
|
||||
print("=== Class-based Middleware Example ===")
|
||||
print("=== Class-based MiddlewareTypes Example ===")
|
||||
|
||||
# For authentication, run `az login` command in terminal or replace AzureCliCredential with preferred
|
||||
# authentication option.
|
||||
|
||||
@@ -12,7 +12,7 @@ from agent_framework.azure import AzureAIAgentClient
|
||||
from azure.identity.aio import AzureCliCredential
|
||||
|
||||
"""
|
||||
Decorator Middleware Example
|
||||
Decorator MiddlewareTypes Example
|
||||
|
||||
This sample demonstrates how to use @agent_middleware and @function_middleware decorators
|
||||
to explicitly mark middleware functions without requiring type annotations.
|
||||
@@ -52,22 +52,22 @@ def get_current_time() -> str:
|
||||
@agent_middleware # Decorator marks this as agent middleware - no type annotations needed
|
||||
async def simple_agent_middleware(context, next): # type: ignore - parameters intentionally untyped to demonstrate decorator functionality
|
||||
"""Agent middleware that runs before and after agent execution."""
|
||||
print("[Agent Middleware] Before agent execution")
|
||||
print("[Agent MiddlewareTypes] Before agent execution")
|
||||
await next(context)
|
||||
print("[Agent Middleware] After agent execution")
|
||||
print("[Agent MiddlewareTypes] After agent execution")
|
||||
|
||||
|
||||
@function_middleware # Decorator marks this as function middleware - no type annotations needed
|
||||
async def simple_function_middleware(context, next): # type: ignore - parameters intentionally untyped to demonstrate decorator functionality
|
||||
"""Function middleware that runs before and after function calls."""
|
||||
print(f"[Function Middleware] Before calling: {context.function.name}") # type: ignore
|
||||
print(f"[Function MiddlewareTypes] Before calling: {context.function.name}") # type: ignore
|
||||
await next(context)
|
||||
print(f"[Function Middleware] After calling: {context.function.name}") # type: ignore
|
||||
print(f"[Function MiddlewareTypes] After calling: {context.function.name}") # type: ignore
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
"""Example demonstrating decorator-based middleware."""
|
||||
print("=== Decorator Middleware Example ===")
|
||||
print("=== Decorator MiddlewareTypes Example ===")
|
||||
|
||||
# For authentication, run `az login` command in terminal or replace AzureCliCredential with preferred
|
||||
# authentication option.
|
||||
|
||||
@@ -10,7 +10,7 @@ from azure.identity.aio import AzureCliCredential
|
||||
from pydantic import Field
|
||||
|
||||
"""
|
||||
Exception Handling with Middleware
|
||||
Exception Handling with MiddlewareTypes
|
||||
|
||||
This sample demonstrates how to use middleware for centralized exception handling in function calls.
|
||||
The example shows:
|
||||
@@ -54,7 +54,7 @@ async def exception_handling_middleware(
|
||||
|
||||
async def main() -> None:
|
||||
"""Example demonstrating exception handling with middleware."""
|
||||
print("=== Exception Handling Middleware Example ===")
|
||||
print("=== Exception Handling MiddlewareTypes Example ===")
|
||||
|
||||
# For authentication, run `az login` command in terminal or replace AzureCliCredential with preferred
|
||||
# authentication option.
|
||||
|
||||
@@ -16,7 +16,7 @@ from azure.identity.aio import AzureCliCredential
|
||||
from pydantic import Field
|
||||
|
||||
"""
|
||||
Function-based Middleware Example
|
||||
Function-based MiddlewareTypes Example
|
||||
|
||||
This sample demonstrates how to implement middleware using simple async functions instead of classes.
|
||||
The example includes:
|
||||
@@ -80,7 +80,7 @@ async def logging_function_middleware(
|
||||
|
||||
async def main() -> None:
|
||||
"""Example demonstrating function-based middleware."""
|
||||
print("=== Function-based Middleware Example ===")
|
||||
print("=== Function-based MiddlewareTypes Example ===")
|
||||
|
||||
# For authentication, run `az login` command in terminal or replace AzureCliCredential with preferred
|
||||
# authentication option.
|
||||
|
||||
@@ -17,7 +17,7 @@ from azure.identity.aio import AzureCliCredential
|
||||
from pydantic import Field
|
||||
|
||||
"""
|
||||
Middleware Termination Example
|
||||
MiddlewareTypes Termination Example
|
||||
|
||||
This sample demonstrates how middleware can terminate execution using the `context.terminate` flag.
|
||||
The example includes:
|
||||
@@ -40,7 +40,7 @@ def get_weather(
|
||||
|
||||
|
||||
class PreTerminationMiddleware(AgentMiddleware):
|
||||
"""Middleware that terminates execution before calling the agent."""
|
||||
"""MiddlewareTypes that terminates execution before calling the agent."""
|
||||
|
||||
def __init__(self, blocked_words: list[str]):
|
||||
self.blocked_words = [word.lower() for word in blocked_words]
|
||||
@@ -79,7 +79,7 @@ class PreTerminationMiddleware(AgentMiddleware):
|
||||
|
||||
|
||||
class PostTerminationMiddleware(AgentMiddleware):
|
||||
"""Middleware that allows processing but terminates after reaching max responses across multiple runs."""
|
||||
"""MiddlewareTypes that allows processing but terminates after reaching max responses across multiple runs."""
|
||||
|
||||
def __init__(self, max_responses: int = 1):
|
||||
self.max_responses = max_responses
|
||||
@@ -109,7 +109,7 @@ class PostTerminationMiddleware(AgentMiddleware):
|
||||
|
||||
async def pre_termination_middleware() -> None:
|
||||
"""Demonstrate pre-termination middleware that blocks requests with certain words."""
|
||||
print("\n--- Example 1: Pre-termination Middleware ---")
|
||||
print("\n--- Example 1: Pre-termination MiddlewareTypes ---")
|
||||
async with (
|
||||
AzureCliCredential() as credential,
|
||||
AzureAIAgentClient(credential=credential).as_agent(
|
||||
@@ -136,7 +136,7 @@ async def pre_termination_middleware() -> None:
|
||||
|
||||
async def post_termination_middleware() -> None:
|
||||
"""Demonstrate post-termination middleware that limits responses across multiple runs."""
|
||||
print("\n--- Example 2: Post-termination Middleware ---")
|
||||
print("\n--- Example 2: Post-termination MiddlewareTypes ---")
|
||||
async with (
|
||||
AzureCliCredential() as credential,
|
||||
AzureAIAgentClient(credential=credential).as_agent(
|
||||
@@ -170,7 +170,7 @@ async def post_termination_middleware() -> None:
|
||||
|
||||
async def main() -> None:
|
||||
"""Example demonstrating middleware termination functionality."""
|
||||
print("=== Middleware Termination Example ===")
|
||||
print("=== MiddlewareTypes Termination Example ===")
|
||||
await pre_termination_middleware()
|
||||
await post_termination_middleware()
|
||||
|
||||
|
||||
@@ -1,7 +1,8 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
import asyncio
|
||||
from collections.abc import AsyncIterable, Awaitable, Callable
|
||||
import re
|
||||
from collections.abc import Awaitable, Callable
|
||||
from random import randint
|
||||
from typing import Annotated
|
||||
|
||||
@@ -9,16 +10,19 @@ from agent_framework import (
|
||||
AgentResponse,
|
||||
AgentResponseUpdate,
|
||||
AgentRunContext,
|
||||
ChatContext,
|
||||
ChatMessage,
|
||||
Content,
|
||||
ChatResponse,
|
||||
ChatResponseUpdate,
|
||||
ResponseStream,
|
||||
Role,
|
||||
tool,
|
||||
)
|
||||
from agent_framework.azure import AzureAIAgentClient
|
||||
from azure.identity.aio import AzureCliCredential
|
||||
from agent_framework.openai import OpenAIResponsesClient
|
||||
from pydantic import Field
|
||||
|
||||
"""
|
||||
Result Override with Middleware (Regular and Streaming)
|
||||
Result Override with MiddlewareTypes (Regular and Streaming)
|
||||
|
||||
This sample demonstrates how to use middleware to intercept and modify function results
|
||||
after execution, supporting both regular and streaming agent responses. The example shows:
|
||||
@@ -26,7 +30,7 @@ after execution, supporting both regular and streaming agent responses. The exam
|
||||
- How to execute the original function first and then modify its result
|
||||
- Replacing function outputs with custom messages or transformed data
|
||||
- Using middleware for result filtering, formatting, or enhancement
|
||||
- Detecting streaming vs non-streaming execution using context.is_streaming
|
||||
- Detecting streaming vs non-streaming execution using context.stream
|
||||
- Overriding streaming results with custom async generators
|
||||
|
||||
The weather override middleware lets the original weather function execute normally,
|
||||
@@ -45,10 +49,8 @@ def get_weather(
|
||||
return f"The weather in {location} is {conditions[randint(0, 3)]} with a high of {randint(10, 30)}°C."
|
||||
|
||||
|
||||
async def weather_override_middleware(
|
||||
context: AgentRunContext, next: Callable[[AgentRunContext], Awaitable[None]]
|
||||
) -> None:
|
||||
"""Middleware that overrides weather results for both streaming and non-streaming cases."""
|
||||
async def weather_override_middleware(context: ChatContext, next: Callable[[ChatContext], Awaitable[None]]) -> None:
|
||||
"""Chat middleware that overrides weather results for both streaming and non-streaming cases."""
|
||||
|
||||
# Let the original agent execution complete first
|
||||
await next(context)
|
||||
@@ -57,56 +59,159 @@ async def weather_override_middleware(
|
||||
if context.result is not None:
|
||||
# Create custom weather message
|
||||
chunks = [
|
||||
"Weather Advisory - ",
|
||||
"due to special atmospheric conditions, ",
|
||||
"all locations are experiencing perfect weather today! ",
|
||||
"Temperature is a comfortable 22°C with gentle breezes. ",
|
||||
"Perfect day for outdoor activities!",
|
||||
]
|
||||
|
||||
if context.is_streaming:
|
||||
# For streaming: create an async generator that yields chunks
|
||||
async def override_stream() -> AsyncIterable[AgentResponseUpdate]:
|
||||
for chunk in chunks:
|
||||
yield AgentResponseUpdate(contents=[Content.from_text(text=chunk)])
|
||||
if context.stream and isinstance(context.result, ResponseStream):
|
||||
index = {"value": 0}
|
||||
|
||||
context.result = override_stream()
|
||||
def _update_hook(update: ChatResponseUpdate) -> ChatResponseUpdate:
|
||||
for content in update.contents or []:
|
||||
if not content.text:
|
||||
continue
|
||||
content.text = f"Weather Advisory: [{index['value']}] {content.text}"
|
||||
index["value"] += 1
|
||||
return update
|
||||
|
||||
context.result.with_update_hook(_update_hook)
|
||||
else:
|
||||
# For non-streaming: just replace with the string message
|
||||
custom_message = "".join(chunks)
|
||||
context.result = AgentResponse(messages=[ChatMessage("assistant", [custom_message])])
|
||||
# For non-streaming: just replace with a new message
|
||||
current_text = context.result.text or ""
|
||||
custom_message = f"Weather Advisory: [0] {''.join(chunks)} Original message was: {current_text}"
|
||||
context.result = ChatResponse(messages=[ChatMessage(role=Role.ASSISTANT, text=custom_message)])
|
||||
|
||||
|
||||
async def validate_weather_middleware(context: ChatContext, next: Callable[[ChatContext], Awaitable[None]]) -> None:
|
||||
"""Chat middleware that simulates result validation for both streaming and non-streaming cases."""
|
||||
await next(context)
|
||||
|
||||
validation_note = "Validation: weather data verified."
|
||||
|
||||
if context.result is None:
|
||||
return
|
||||
|
||||
if context.stream and isinstance(context.result, ResponseStream):
|
||||
|
||||
def _append_validation_note(response: ChatResponse) -> ChatResponse:
|
||||
response.messages.append(ChatMessage(role=Role.ASSISTANT, text=validation_note))
|
||||
return response
|
||||
|
||||
context.result.with_finalizer(_append_validation_note)
|
||||
elif isinstance(context.result, ChatResponse):
|
||||
context.result.messages.append(ChatMessage(role=Role.ASSISTANT, text=validation_note))
|
||||
|
||||
|
||||
async def agent_cleanup_middleware(
|
||||
context: AgentRunContext, next: Callable[[AgentRunContext], Awaitable[None]]
|
||||
) -> None:
|
||||
"""Agent middleware that validates chat middleware effects and cleans the result."""
|
||||
await next(context)
|
||||
|
||||
if context.result is None:
|
||||
return
|
||||
|
||||
validation_note = "Validation: weather data verified."
|
||||
|
||||
state = {"found_prefix": False}
|
||||
|
||||
def _sanitize(response: AgentResponse) -> AgentResponse:
|
||||
found_prefix = state["found_prefix"]
|
||||
found_validation = False
|
||||
cleaned_messages: list[ChatMessage] = []
|
||||
|
||||
for message in response.messages:
|
||||
text = message.text
|
||||
if text is None:
|
||||
cleaned_messages.append(message)
|
||||
continue
|
||||
|
||||
if validation_note in text:
|
||||
found_validation = True
|
||||
text = text.replace(validation_note, "").strip()
|
||||
if not text:
|
||||
continue
|
||||
|
||||
if "Weather Advisory:" in text:
|
||||
found_prefix = True
|
||||
text = text.replace("Weather Advisory:", "")
|
||||
|
||||
text = re.sub(r"\[\d+\]\s*", "", text)
|
||||
|
||||
cleaned_messages.append(
|
||||
ChatMessage(
|
||||
role=message.role,
|
||||
text=text.strip(),
|
||||
author_name=message.author_name,
|
||||
message_id=message.message_id,
|
||||
additional_properties=message.additional_properties,
|
||||
raw_representation=message.raw_representation,
|
||||
)
|
||||
)
|
||||
|
||||
if not found_prefix:
|
||||
raise RuntimeError("Expected chat middleware prefix not found in agent response.")
|
||||
if not found_validation:
|
||||
raise RuntimeError("Expected validation note not found in agent response.")
|
||||
|
||||
cleaned_messages.append(ChatMessage(role=Role.ASSISTANT, text=" Agent: OK"))
|
||||
response.messages = cleaned_messages
|
||||
return response
|
||||
|
||||
if context.stream and isinstance(context.result, ResponseStream):
|
||||
|
||||
def _clean_update(update: AgentResponseUpdate) -> AgentResponseUpdate:
|
||||
for content in update.contents or []:
|
||||
if not content.text:
|
||||
continue
|
||||
text = content.text
|
||||
if "Weather Advisory:" in text:
|
||||
state["found_prefix"] = True
|
||||
text = text.replace("Weather Advisory:", "")
|
||||
text = re.sub(r"\[\d+\]\s*", "", text)
|
||||
content.text = text
|
||||
return update
|
||||
|
||||
context.result.with_update_hook(_clean_update)
|
||||
context.result.with_finalizer(_sanitize)
|
||||
elif isinstance(context.result, AgentResponse):
|
||||
context.result = _sanitize(context.result)
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
"""Example demonstrating result override with middleware for both streaming and non-streaming."""
|
||||
print("=== Result Override Middleware Example ===")
|
||||
print("=== Result Override MiddlewareTypes Example ===")
|
||||
|
||||
# For authentication, run `az login` command in terminal or replace AzureCliCredential with preferred
|
||||
# authentication option.
|
||||
async with (
|
||||
AzureCliCredential() as credential,
|
||||
AzureAIAgentClient(credential=credential).as_agent(
|
||||
name="WeatherAgent",
|
||||
instructions="You are a helpful weather assistant. Use the weather tool to get current conditions.",
|
||||
tools=get_weather,
|
||||
middleware=[weather_override_middleware],
|
||||
) as agent,
|
||||
):
|
||||
# Non-streaming example
|
||||
print("\n--- Non-streaming Example ---")
|
||||
query = "What's the weather like in Seattle?"
|
||||
print(f"User: {query}")
|
||||
result = await agent.run(query)
|
||||
print(f"Agent: {result}")
|
||||
agent = OpenAIResponsesClient(
|
||||
middleware=[validate_weather_middleware, weather_override_middleware],
|
||||
).as_agent(
|
||||
name="WeatherAgent",
|
||||
instructions="You are a helpful weather assistant. Use the weather tool to get current conditions.",
|
||||
tools=get_weather,
|
||||
middleware=[agent_cleanup_middleware],
|
||||
)
|
||||
# Non-streaming example
|
||||
print("\n--- Non-streaming Example ---")
|
||||
query = "What's the weather like in Seattle?"
|
||||
print(f"User: {query}")
|
||||
result = await agent.run(query)
|
||||
print(f"Agent: {result}")
|
||||
|
||||
# Streaming example
|
||||
print("\n--- Streaming Example ---")
|
||||
query = "What's the weather like in Portland?"
|
||||
print(f"User: {query}")
|
||||
print("Agent: ", end="", flush=True)
|
||||
async for chunk in agent.run_stream(query):
|
||||
if chunk.text:
|
||||
print(chunk.text, end="", flush=True)
|
||||
# Streaming example
|
||||
print("\n--- Streaming Example ---")
|
||||
query = "What's the weather like in Portland?"
|
||||
print(f"User: {query}")
|
||||
print("Agent: ", end="", flush=True)
|
||||
response = agent.run(query, stream=True)
|
||||
async for chunk in response:
|
||||
if chunk.text:
|
||||
print(chunk.text, end="", flush=True)
|
||||
print("\n")
|
||||
print(f"Final Result: {(await response.get_final_response()).text}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
@@ -16,9 +16,9 @@ session data, etc.) to tools and sub-agents.
|
||||
|
||||
Patterns Demonstrated:
|
||||
|
||||
1. **Pattern 1: Single Agent with Middleware & Closure** (Lines 130-180)
|
||||
1. **Pattern 1: Single Agent with MiddlewareTypes & Closure** (Lines 130-180)
|
||||
- Best for: Single agent with multiple tools
|
||||
- How: Middleware stores kwargs in container, tools access via closure
|
||||
- How: MiddlewareTypes stores kwargs in container, tools access via closure
|
||||
- Pros: Simple, explicit state management
|
||||
- Cons: Requires container instance per agent
|
||||
|
||||
@@ -28,7 +28,7 @@ Patterns Demonstrated:
|
||||
- Pros: Automatic, works with nested delegation, clean separation
|
||||
- Cons: None - this is the recommended pattern for hierarchical agents
|
||||
|
||||
3. **Pattern 3: Mixed - Hierarchical with Middleware** (Lines 250-300)
|
||||
3. **Pattern 3: Mixed - Hierarchical with MiddlewareTypes** (Lines 250-300)
|
||||
- Best for: Complex scenarios needing both delegation and state management
|
||||
- How: Combines automatic kwargs propagation with middleware processing
|
||||
- Pros: Maximum flexibility, can transform/validate context at each level
|
||||
@@ -36,7 +36,7 @@ Patterns Demonstrated:
|
||||
|
||||
Key Concepts:
|
||||
- Runtime Context: Session-specific data like API tokens, user IDs, tenant info
|
||||
- Middleware: Intercepts function calls to access/modify kwargs
|
||||
- MiddlewareTypes: Intercepts function calls to access/modify kwargs
|
||||
- Closure: Functions capturing variables from outer scope
|
||||
- kwargs Propagation: Automatic forwarding of runtime context through delegation chains
|
||||
"""
|
||||
@@ -56,7 +56,7 @@ class SessionContextContainer:
|
||||
context: FunctionInvocationContext,
|
||||
next: Callable[[FunctionInvocationContext], Awaitable[None]],
|
||||
) -> None:
|
||||
"""Middleware that extracts runtime context from kwargs and stores in container.
|
||||
"""MiddlewareTypes that extracts runtime context from kwargs and stores in container.
|
||||
|
||||
This middleware runs before tool execution and makes runtime context
|
||||
available to tools via the container instance.
|
||||
@@ -68,7 +68,7 @@ class SessionContextContainer:
|
||||
|
||||
# Log what we captured (for demonstration)
|
||||
if self.api_token or self.user_id:
|
||||
print("[Middleware] Captured runtime context:")
|
||||
print("[MiddlewareTypes] Captured runtime context:")
|
||||
print(f" - API Token: {'[PRESENT]' if self.api_token else '[NOT PROVIDED]'}")
|
||||
print(f" - User ID: {'[PRESENT]' if self.user_id else '[NOT PROVIDED]'}")
|
||||
print(f" - Session Metadata Keys: {list(self.session_metadata.keys())}")
|
||||
@@ -140,7 +140,7 @@ async def send_notification(
|
||||
async def pattern_1_single_agent_with_closure() -> None:
|
||||
"""Pattern 1: Single agent with middleware and closure for runtime context."""
|
||||
print("\n" + "=" * 70)
|
||||
print("PATTERN 1: Single Agent with Middleware & Closure")
|
||||
print("PATTERN 1: Single Agent with MiddlewareTypes & Closure")
|
||||
print("=" * 70)
|
||||
print("Use case: Single agent with multiple tools sharing runtime context")
|
||||
print()
|
||||
@@ -234,7 +234,7 @@ async def pattern_1_single_agent_with_closure() -> None:
|
||||
|
||||
print(f"\nAgent: {result4.text}")
|
||||
|
||||
print("\n✓ Pattern 1 complete - Middleware & closure pattern works for single agents")
|
||||
print("\n✓ Pattern 1 complete - MiddlewareTypes & closure pattern works for single agents")
|
||||
|
||||
|
||||
# Pattern 2: Hierarchical agents with automatic kwargs propagation
|
||||
@@ -353,7 +353,7 @@ async def pattern_2_hierarchical_with_kwargs_propagation() -> None:
|
||||
|
||||
|
||||
class AuthContextMiddleware:
|
||||
"""Middleware that validates and transforms runtime context."""
|
||||
"""MiddlewareTypes that validates and transforms runtime context."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.validated_tokens: list[str] = []
|
||||
@@ -387,7 +387,7 @@ async def protected_operation(operation: Annotated[str, Field(description="Opera
|
||||
async def pattern_3_hierarchical_with_middleware() -> None:
|
||||
"""Pattern 3: Hierarchical agents with middleware processing at each level."""
|
||||
print("\n" + "=" * 70)
|
||||
print("PATTERN 3: Hierarchical with Middleware Processing")
|
||||
print("PATTERN 3: Hierarchical with MiddlewareTypes Processing")
|
||||
print("=" * 70)
|
||||
print("Use case: Multi-level validation/transformation of runtime context")
|
||||
print()
|
||||
@@ -433,7 +433,7 @@ async def pattern_3_hierarchical_with_middleware() -> None:
|
||||
)
|
||||
|
||||
print(f"\n[Validation Summary] Validated tokens: {len(auth_middleware.validated_tokens)}")
|
||||
print("✓ Pattern 3 complete - Middleware can validate/transform context at each level")
|
||||
print("✓ Pattern 3 complete - MiddlewareTypes can validate/transform context at each level")
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
|
||||
@@ -14,7 +14,7 @@ from azure.identity.aio import AzureCliCredential
|
||||
from pydantic import Field
|
||||
|
||||
"""
|
||||
Shared State Function-based Middleware Example
|
||||
Shared State Function-based MiddlewareTypes Example
|
||||
|
||||
This sample demonstrates how to implement function-based middleware within a class to share state.
|
||||
The example includes:
|
||||
@@ -88,7 +88,7 @@ class MiddlewareContainer:
|
||||
|
||||
async def main() -> None:
|
||||
"""Example demonstrating shared state function-based middleware."""
|
||||
print("=== Shared State Function-based Middleware Example ===")
|
||||
print("=== Shared State Function-based MiddlewareTypes Example ===")
|
||||
|
||||
# Create middleware container with shared state
|
||||
middleware_container = MiddlewareContainer()
|
||||
|
||||
@@ -14,7 +14,7 @@ from azure.identity import AzureCliCredential
|
||||
from pydantic import Field
|
||||
|
||||
"""
|
||||
Thread Behavior Middleware Example
|
||||
Thread Behavior MiddlewareTypes Example
|
||||
|
||||
This sample demonstrates how middleware can access and track thread state across multiple agent runs.
|
||||
The example shows:
|
||||
@@ -48,13 +48,13 @@ async def thread_tracking_middleware(
|
||||
context: AgentRunContext,
|
||||
next: Callable[[AgentRunContext], Awaitable[None]],
|
||||
) -> None:
|
||||
"""Middleware that tracks and logs thread behavior across runs."""
|
||||
"""MiddlewareTypes that tracks and logs thread behavior across runs."""
|
||||
thread_messages = []
|
||||
if context.thread and context.thread.message_store:
|
||||
thread_messages = await context.thread.message_store.list_messages()
|
||||
|
||||
print(f"[Middleware pre-execution] Current input messages: {len(context.messages)}")
|
||||
print(f"[Middleware pre-execution] Thread history messages: {len(thread_messages)}")
|
||||
print(f"[MiddlewareTypes pre-execution] Current input messages: {len(context.messages)}")
|
||||
print(f"[MiddlewareTypes pre-execution] Thread history messages: {len(thread_messages)}")
|
||||
|
||||
# Call next to execute the agent
|
||||
await next(context)
|
||||
@@ -64,12 +64,12 @@ async def thread_tracking_middleware(
|
||||
if context.thread and context.thread.message_store:
|
||||
updated_thread_messages = await context.thread.message_store.list_messages()
|
||||
|
||||
print(f"[Middleware post-execution] Updated thread messages: {len(updated_thread_messages)}")
|
||||
print(f"[MiddlewareTypes post-execution] Updated thread messages: {len(updated_thread_messages)}")
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
"""Example demonstrating thread behavior in middleware across multiple runs."""
|
||||
print("=== Thread Behavior Middleware Example ===")
|
||||
print("=== Thread Behavior MiddlewareTypes Example ===")
|
||||
|
||||
# For authentication, run `az login` command in terminal or replace AzureCliCredential with preferred
|
||||
# authentication option.
|
||||
|
||||
+1
-1
@@ -107,7 +107,7 @@ async def run_chat_client() -> None:
|
||||
message = "What's the weather in Amsterdam and in Paris?"
|
||||
print(f"User: {message}")
|
||||
print("Assistant: ", end="")
|
||||
async for chunk in client.get_streaming_response(message, tools=get_weather):
|
||||
async for chunk in client.get_response(message, tools=get_weather, stream=True):
|
||||
if str(chunk):
|
||||
print(str(chunk), end="")
|
||||
print("")
|
||||
|
||||
@@ -81,7 +81,7 @@ async def run_chat_client(client: "ChatClientProtocol", stream: bool = False) ->
|
||||
print(f"User: {message}")
|
||||
if stream:
|
||||
print("Assistant: ", end="")
|
||||
async for chunk in client.get_streaming_response(message, tools=get_weather):
|
||||
async for chunk in client.get_response(message, tools=get_weather, stream=True):
|
||||
if str(chunk):
|
||||
print(str(chunk), end="")
|
||||
print("")
|
||||
|
||||
@@ -50,9 +50,10 @@ async def main():
|
||||
for question in questions:
|
||||
print(f"\nUser: {question}")
|
||||
print(f"{agent.name}: ", end="")
|
||||
async for update in agent.run_stream(
|
||||
async for update in agent.run(
|
||||
question,
|
||||
thread=thread,
|
||||
stream=True,
|
||||
):
|
||||
if update.text:
|
||||
print(update.text, end="")
|
||||
|
||||
@@ -87,10 +87,7 @@ async def main():
|
||||
for question in questions:
|
||||
print(f"\nUser: {question}")
|
||||
print(f"{agent.name}: ", end="")
|
||||
async for update in agent.run_stream(
|
||||
question,
|
||||
thread=thread,
|
||||
):
|
||||
async for update in agent.run(question, thread=thread, stream=True):
|
||||
if update.text:
|
||||
print(update.text, end="")
|
||||
|
||||
|
||||
@@ -67,10 +67,7 @@ async def main():
|
||||
for question in questions:
|
||||
print(f"\nUser: {question}")
|
||||
print(f"{agent.name}: ", end="")
|
||||
async for update in agent.run_stream(
|
||||
question,
|
||||
thread=thread,
|
||||
):
|
||||
async for update in agent.run(question, thread=thread, stream=True):
|
||||
if update.text:
|
||||
print(update.text, end="")
|
||||
|
||||
|
||||
+1
-1
@@ -71,7 +71,7 @@ async def run_chat_client(client: "ChatClientProtocol", stream: bool = False) ->
|
||||
print(f"User: {message}")
|
||||
if stream:
|
||||
print("Assistant: ", end="")
|
||||
async for chunk in client.get_streaming_response(message, tools=get_weather):
|
||||
async for chunk in client.get_response(message, tools=get_weather, stream=True):
|
||||
if str(chunk):
|
||||
print(str(chunk), end="")
|
||||
print("")
|
||||
|
||||
+1
-1
@@ -71,7 +71,7 @@ async def run_chat_client(client: "ChatClientProtocol", stream: bool = False) ->
|
||||
print(f"User: {message}")
|
||||
if stream:
|
||||
print("Assistant: ", end="")
|
||||
async for chunk in client.get_streaming_response(message, tools=get_weather):
|
||||
async for chunk in client.get_response(message, stream=True, tools=get_weather):
|
||||
if str(chunk):
|
||||
print(str(chunk), end="")
|
||||
print("")
|
||||
|
||||
@@ -92,7 +92,7 @@ async def run_sequential_workflow() -> None:
|
||||
print(f"Starting workflow with input: '{input_text}'")
|
||||
|
||||
output_event = None
|
||||
async for event in workflow.run_stream("Hello world"):
|
||||
async for event in workflow.run("Hello world", stream=True):
|
||||
if isinstance(event, WorkflowOutputEvent):
|
||||
# The WorkflowOutputEvent contains the final result.
|
||||
output_event = event
|
||||
|
||||
@@ -87,7 +87,7 @@ async def main() -> None:
|
||||
|
||||
# Keep track of the last response to format output nicely in streaming mode
|
||||
last_response_id: str | None = None
|
||||
async for event in workflow.run_stream(task):
|
||||
async for event in workflow.run(task, stream=True):
|
||||
if isinstance(event, WorkflowOutputEvent):
|
||||
data = event.data
|
||||
if isinstance(data, AgentResponseUpdate):
|
||||
|
||||
@@ -240,7 +240,7 @@ Share your perspective authentically. Feel free to:
|
||||
|
||||
# Keep track of the last response to format output nicely in streaming mode
|
||||
last_response_id: str | None = None
|
||||
async for event in workflow.run_stream(f"Please begin the discussion on: {topic}"):
|
||||
async for event in workflow.run(f"Please begin the discussion on: {topic}", stream=True):
|
||||
if isinstance(event, WorkflowOutputEvent):
|
||||
data = event.data
|
||||
if isinstance(data, AgentResponseUpdate):
|
||||
|
||||
@@ -105,7 +105,7 @@ async def main() -> None:
|
||||
|
||||
# Keep track of the last response to format output nicely in streaming mode
|
||||
last_response_id: str | None = None
|
||||
async for event in workflow.run_stream(task):
|
||||
async for event in workflow.run(task, stream=True):
|
||||
if isinstance(event, WorkflowOutputEvent):
|
||||
data = event.data
|
||||
if isinstance(data, AgentResponseUpdate):
|
||||
|
||||
@@ -111,7 +111,7 @@ async def main() -> None:
|
||||
print("Request:", request)
|
||||
|
||||
last_response_id: str | None = None
|
||||
async for event in workflow.run_stream(request):
|
||||
async for event in workflow.run(request, stream=True):
|
||||
if isinstance(event, HandoffSentEvent):
|
||||
print(f"\nHandoff Event: from {event.source} to {event.target}\n")
|
||||
elif isinstance(event, WorkflowOutputEvent):
|
||||
|
||||
@@ -233,12 +233,12 @@ async def main() -> None:
|
||||
]
|
||||
|
||||
# Start the workflow with the initial user message
|
||||
# run_stream() returns an async iterator of WorkflowEvent
|
||||
# run(..., stream=True) returns an async iterator of WorkflowEvent
|
||||
print("[Starting workflow with initial user message...]\n")
|
||||
initial_message = "Hello, I need assistance with my recent purchase."
|
||||
print(f"- User: {initial_message}")
|
||||
workflow_result = await workflow.run(initial_message)
|
||||
pending_requests = _handle_events(workflow_result)
|
||||
workflow_result = workflow.run(initial_message, stream=True)
|
||||
pending_requests = _handle_events([event async for event in workflow_result])
|
||||
|
||||
# Process the request/response cycle
|
||||
# The workflow will continue requesting input until:
|
||||
|
||||
@@ -187,7 +187,7 @@ async def main() -> None:
|
||||
all_file_ids: list[str] = []
|
||||
|
||||
print(f"User: {user_inputs[0]}")
|
||||
events = await _drain(workflow.run_stream(user_inputs[0]))
|
||||
events = await _drain(workflow.run(user_inputs[0], stream=True))
|
||||
requests, file_ids = _handle_events(events)
|
||||
all_file_ids.extend(file_ids)
|
||||
input_index += 1
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user