Python: [BREAKING] Renamed next middleware parameter to call_next (#3735)

* Renamed next middleware parameter to call_next

* Resolved comments
This commit is contained in:
Dmytro Struk
2026-02-09 13:21:27 -08:00
committed by GitHub
Unverified
parent 977c3adfb2
commit e4ca3e60f8
26 changed files with 529 additions and 430 deletions
+32 -32
View File
@@ -37,7 +37,7 @@ sequenceDiagram
AML->>AMP: execute(AgentContext)
loop Agent Middleware Chain
AMP->>AMP: middleware[i].process(context, next)
AMP->>AMP: middleware[i].process(context, call_next)
Note right of AMP: Can modify: messages, options, thread
end
@@ -60,7 +60,7 @@ sequenceDiagram
CML->>CMP: execute(ChatContext)
loop Chat Middleware Chain
CMP->>CMP: middleware[i].process(context, next)
CMP->>CMP: middleware[i].process(context, call_next)
Note right of CMP: Can modify: messages, options
end
@@ -81,7 +81,7 @@ sequenceDiagram
loop For each function_call
FIL->>FMP: execute(FunctionInvocationContext)
loop Function Middleware Chain
FMP->>FMP: middleware[i].process(context, next)
FMP->>FMP: middleware[i].process(context, call_next)
Note right of FMP: Can modify: arguments
end
FMP->>Tool: invoke(arguments)
@@ -137,7 +137,7 @@ sequenceDiagram
| `options` | `Mapping[str, Any]` | Chat options dict |
| `stream` | `bool` | Whether streaming is enabled |
| `metadata` | `dict` | Shared data between middleware |
| `result` | `AgentResponse \| None` | Set after `next()` is called |
| `result` | `AgentResponse \| None` | Set after `call_next()` is called |
| `kwargs` | `Mapping[str, Any]` | Additional run arguments |
**Key Operations:**
@@ -150,7 +150,7 @@ sequenceDiagram
- `context.messages` - Add, remove, or modify input messages
- `context.options` - Change model parameters, temperature, etc.
- `context.thread` - Replace or modify the thread
- `context.result` - Override the final response (after `next()`)
- `context.result` - Override the final response (after `call_next()`)
### 2. Chat Middleware Layer (`ChatMiddlewareLayer`)
@@ -165,7 +165,7 @@ sequenceDiagram
| `options` | `Mapping[str, Any]` | Chat options |
| `stream` | `bool` | Whether streaming |
| `metadata` | `dict` | Shared data between middleware |
| `result` | `ChatResponse \| None` | Set after `next()` is called |
| `result` | `ChatResponse \| None` | Set after `call_next()` is called |
| `kwargs` | `Mapping[str, Any]` | Additional arguments |
**Key Operations:**
@@ -176,7 +176,7 @@ sequenceDiagram
**What Can Be Modified:**
- `context.messages` - Inject system prompts, filter content
- `context.options` - Change model, temperature, tool_choice
- `context.result` - Override the response (after `next()`)
- `context.result` - Override the response (after `call_next()`)
### 3. Function Invocation Layer (`FunctionInvocationLayer`)
@@ -251,23 +251,23 @@ response = await client.get_response(
| `function` | `FunctionTool` | The function being invoked |
| `arguments` | `BaseModel` | Validated Pydantic arguments |
| `metadata` | `dict` | Shared data between middleware |
| `result` | `Any` | Set after `next()` is called |
| `result` | `Any` | Set after `call_next()` is called |
| `kwargs` | `Mapping[str, Any]` | Runtime kwargs |
**What Can Be Modified:**
- `context.arguments` - Modify validated arguments before execution
- `context.result` - Override the function result (after `next()`)
- `context.result` - Override the function result (after `call_next()`)
- Raise `MiddlewareTermination` to skip execution and terminate the function invocation loop
**Special Behavior:** When `MiddlewareTermination` is raised in function middleware, it signals that the function invocation loop should exit **without making another LLM call**. This is useful when middleware determines that no further processing is needed (e.g., a termination condition is met).
```python
class TerminatingMiddleware(FunctionMiddleware):
async def process(self, context: FunctionInvocationContext, next):
async def process(self, context: FunctionInvocationContext, call_next):
if self.should_terminate(context):
context.result = "terminated by middleware"
raise MiddlewareTermination # Exit function invocation loop
await next(context)
await call_next(context)
```
## Arguments Added/Altered at Each Layer
@@ -334,20 +334,20 @@ class TerminatingMiddleware(FunctionMiddleware):
There are three ways to exit a middleware's `process()` method:
### 1. Return Normally (with or without calling `next`)
### 1. Return Normally (with or without calling `call_next`)
Returns control to the upstream middleware, allowing its post-processing code to run.
```python
class CachingMiddleware(FunctionMiddleware):
async def process(self, context: FunctionInvocationContext, next):
# Option A: Return early WITHOUT calling next (skip downstream)
async def process(self, context: FunctionInvocationContext, call_next):
# Option A: Return early WITHOUT calling call_next (skip downstream)
if cached := self.cache.get(context.function.name):
context.result = cached
return # Upstream post-processing still runs
# Option B: Call next, then return normally
await next(context)
# Option B: Call call_next, then return normally
await call_next(context)
self.cache[context.function.name] = context.result
return # Normal completion
```
@@ -358,11 +358,11 @@ Immediately exits the entire middleware chain. Upstream middleware's post-proces
```python
class BlockedFunctionMiddleware(FunctionMiddleware):
async def process(self, context: FunctionInvocationContext, next):
async def process(self, context: FunctionInvocationContext, call_next):
if context.function.name in self.blocked_functions:
context.result = "Function blocked by policy"
raise MiddlewareTermination("Blocked") # Skips ALL post-processing
await next(context)
await call_next(context)
```
### 3. Raise Any Other Exception
@@ -371,10 +371,10 @@ Bubbles up to the caller. The middleware chain is aborted and the exception prop
```python
class ValidationMiddleware(FunctionMiddleware):
async def process(self, context: FunctionInvocationContext, next):
async def process(self, context: FunctionInvocationContext, call_next):
if not self.is_valid(context.arguments):
raise ValueError("Invalid arguments") # Bubbles up to user
await next(context)
await call_next(context)
```
## `return` vs `raise MiddlewareTermination`
@@ -383,13 +383,13 @@ The key difference is what happens to **upstream middleware's post-processing**:
```python
class MiddlewareA(AgentMiddleware):
async def process(self, context, next):
async def process(self, context, call_next):
print("A: before")
await next(context)
await call_next(context)
print("A: after") # Does this run?
class MiddlewareB(AgentMiddleware):
async def process(self, context, next):
async def process(self, context, call_next):
print("B: before")
context.result = "early result"
# Choose one:
@@ -408,14 +408,14 @@ With middleware registered as `[MiddlewareA, MiddlewareB]`:
**Use `raise MiddlewareTermination`** when you want to completely bypass all remaining processing (e.g., blocking a request, returning cached response without any modification).
## Calling `next()` or Not
## Calling `call_next()` or Not
The decision to call `next(context)` determines whether downstream middleware and the actual operation execute:
The decision to call `call_next(context)` determines whether downstream middleware and the actual operation execute:
### Without calling `next()` - Skip downstream
### Without calling `call_next()` - Skip downstream
```python
async def process(self, context, next):
async def process(self, context, call_next):
context.result = "replacement result"
return # Downstream middleware and actual execution are SKIPPED
```
@@ -425,12 +425,12 @@ async def process(self, context, next):
- Upstream middleware post-processing: ✅ Still runs (unless `MiddlewareTermination` raised)
- Result: Whatever you set in `context.result`
### With calling `next()` - Full execution
### With calling `call_next()` - Full execution
```python
async def process(self, context, next):
async def process(self, context, call_next):
# Pre-processing
await next(context) # Execute downstream + actual operation
await call_next(context) # Execute downstream + actual operation
# Post-processing (context.result now contains real result)
return
```
@@ -442,7 +442,7 @@ async def process(self, context, next):
### Summary Table
| Exit Method | Call `next()`? | Downstream Executes? | Actual Op Executes? | Upstream Post-Processing? |
| Exit Method | Call `call_next()`? | Downstream Executes? | Actual Op Executes? | Upstream Post-Processing? |
|-------------|----------------|---------------------|---------------------|--------------------------|
| `return` (or implicit) | Yes | ✅ | ✅ | ✅ Yes |
| `return` | No | ❌ | ❌ | ✅ Yes |
@@ -450,7 +450,7 @@ async def process(self, context, next):
| `raise MiddlewareTermination` | Yes | ✅ | ✅ | ❌ No |
| `raise OtherException` | Either | Depends | Depends | ❌ No (exception propagates) |
> **Note:** The first row (`return` after calling `next()`) is the default behavior. Python functions implicitly return `None` at the end, so simply calling `await next(context)` without an explicit `return` statement achieves this pattern.
> **Note:** The first row (`return` after calling `call_next()`) is the default behavior. Python functions implicitly return `None` at the end, so simply calling `await call_next(context)` without an explicit `return` statement achieves this pattern.
## Streaming vs Non-Streaming
@@ -20,13 +20,13 @@ multiple specialized agents, each focusing on specific tasks.
async def logging_middleware(
context: FunctionInvocationContext,
next: Callable[[FunctionInvocationContext], Awaitable[None]],
call_next: Callable[[FunctionInvocationContext], Awaitable[None]],
) -> None:
"""MiddlewareTypes that logs tool invocations to show the delegation flow."""
print(f"[Calling tool: {context.function.name}]")
print(f"[Request: {context.arguments}]")
await next(context)
await call_next(context)
print(f"[Response: {context.result}]")
@@ -28,7 +28,7 @@ response generation, showing both streaming and non-streaming responses.
@chat_middleware
async def security_and_override_middleware(
context: ChatContext,
next: Callable[[ChatContext], Awaitable[None]],
call_next: Callable[[ChatContext], Awaitable[None]],
) -> None:
"""Function-based middleware that implements security filtering and response override."""
print("[SecurityMiddleware] Processing input...")
@@ -59,7 +59,7 @@ async def security_and_override_middleware(
raise MiddlewareTermination
# Continue to next middleware or AI execution
await next(context)
await call_next(context)
print("[SecurityMiddleware] Response generated.")
print(type(context.result))
@@ -19,13 +19,13 @@ multiple specialized agents, each focusing on specific tasks.
async def logging_middleware(
context: FunctionInvocationContext,
next: Callable[[FunctionInvocationContext], Awaitable[None]],
call_next: Callable[[FunctionInvocationContext], Awaitable[None]],
) -> None:
"""MiddlewareTypes that logs tool invocations to show the delegation flow."""
print(f"[Calling tool: {context.function.name}]")
print(f"[Request: {context.arguments}]")
await next(context)
await call_next(context)
print(f"[Response: {context.result}]")
@@ -37,7 +37,7 @@ def cleanup_resources():
@chat_middleware
async def security_filter_middleware(
context: ChatContext,
next: Callable[[ChatContext], Awaitable[None]],
call_next: Callable[[ChatContext], Awaitable[None]],
) -> None:
"""Chat middleware that blocks requests containing sensitive information."""
blocked_terms = ["password", "secret", "api_key", "token"]
@@ -76,13 +76,13 @@ async def security_filter_middleware(
raise MiddlewareTermination
await next(context)
await call_next(context)
@function_middleware
async def atlantis_location_filter_middleware(
context: FunctionInvocationContext,
next: Callable[[FunctionInvocationContext], Awaitable[None]],
call_next: Callable[[FunctionInvocationContext], Awaitable[None]],
) -> None:
"""Function middleware that blocks weather requests for Atlantis."""
# Check if location parameter is "atlantis"
@@ -94,7 +94,7 @@ async def atlantis_location_filter_middleware(
)
raise MiddlewareTermination
await next(context)
await call_next(context)
# NOTE: approval_mode="never_require" is for sample brevity. Use "always_require" in production; see samples/getting_started/tools/function_tool_with_approval.py and samples/getting_started/tools/function_tool_with_approval_and_threads.py.
@@ -49,7 +49,7 @@ def get_weather(
class SecurityAgentMiddleware(AgentMiddleware):
"""Agent-level security middleware that validates all requests."""
async def process(self, context: AgentContext, next: Callable[[AgentContext], Awaitable[None]]) -> None:
async def process(self, context: AgentContext, call_next: Callable[[AgentContext], Awaitable[None]]) -> None:
print("[SecurityMiddleware] Checking security for all requests...")
# Check for security violations in the last user message
@@ -58,22 +58,22 @@ class SecurityAgentMiddleware(AgentMiddleware):
query = last_message.text.lower()
if any(word in query for word in ["password", "secret", "credentials"]):
print("[SecurityMiddleware] Security violation detected! Blocking request.")
return # Don't call next() to prevent execution
return # Don't call call_next() to prevent execution
print("[SecurityMiddleware] Security check passed.")
context.metadata["security_validated"] = True
await next(context)
await call_next(context)
async def performance_monitor_middleware(
context: AgentContext,
next: Callable[[AgentContext], Awaitable[None]],
call_next: Callable[[AgentContext], Awaitable[None]],
) -> None:
"""Agent-level performance monitoring for all runs."""
print("[PerformanceMonitor] Starting performance monitoring...")
start_time = time.time()
await next(context)
await call_next(context)
end_time = time.time()
duration = end_time - start_time
@@ -85,7 +85,7 @@ async def performance_monitor_middleware(
class HighPriorityMiddleware(AgentMiddleware):
"""Run-level middleware for high priority requests."""
async def process(self, context: AgentContext, next: Callable[[AgentContext], Awaitable[None]]) -> None:
async def process(self, context: AgentContext, call_next: Callable[[AgentContext], Awaitable[None]]) -> None:
print("[HighPriority] Processing high priority request with expedited handling...")
# Read metadata set by agent-level middleware
@@ -96,13 +96,13 @@ class HighPriorityMiddleware(AgentMiddleware):
context.metadata["priority"] = "high"
context.metadata["expedited"] = True
await next(context)
await call_next(context)
print("[HighPriority] High priority processing completed")
async def debugging_middleware(
context: AgentContext,
next: Callable[[AgentContext], Awaitable[None]],
call_next: Callable[[AgentContext], Awaitable[None]],
) -> None:
"""Run-level debugging middleware for troubleshooting specific runs."""
print("[Debug] Debug mode enabled for this run")
@@ -115,7 +115,7 @@ async def debugging_middleware(
context.metadata["debug_enabled"] = True
await next(context)
await call_next(context)
print("[Debug] Debug information collected")
@@ -126,7 +126,7 @@ class CachingMiddleware(AgentMiddleware):
def __init__(self) -> None:
self.cache: dict[str, AgentResponse] = {}
async def process(self, context: AgentContext, next: Callable[[AgentContext], Awaitable[None]]) -> None:
async def process(self, context: AgentContext, call_next: Callable[[AgentContext], Awaitable[None]]) -> None:
# Create a simple cache key from the last message
last_message = context.messages[-1] if context.messages else None
cache_key: str = last_message.text if last_message and last_message.text else "no_message"
@@ -134,12 +134,12 @@ class CachingMiddleware(AgentMiddleware):
if cache_key in self.cache:
print(f"[Cache] Cache HIT for: '{cache_key[:30]}...'")
context.result = self.cache[cache_key] # type: ignore
return # Don't call next(), return cached result
return # Don't call call_next(), return cached result
print(f"[Cache] Cache MISS for: '{cache_key[:30]}...'")
context.metadata["cache_key"] = cache_key
await next(context)
await call_next(context)
# Cache the result if we have one
if context.result:
@@ -149,14 +149,14 @@ class CachingMiddleware(AgentMiddleware):
async def function_logging_middleware(
context: FunctionInvocationContext,
next: Callable[[FunctionInvocationContext], Awaitable[None]],
call_next: Callable[[FunctionInvocationContext], Awaitable[None]],
) -> None:
"""Function middleware that logs all function calls."""
function_name = context.function.name
args = context.arguments
print(f"[FunctionLog] Calling function: {function_name} with args: {args}")
await next(context)
await call_next(context)
print(f"[FunctionLog] Function {function_name} completed")
@@ -57,7 +57,7 @@ class InputObserverMiddleware(ChatMiddleware):
async def process(
self,
context: ChatContext,
next: Callable[[ChatContext], Awaitable[None]],
call_next: Callable[[ChatContext], Awaitable[None]],
) -> None:
"""Observe and modify input messages before they are sent to AI."""
print("[InputObserverMiddleware] Observing input messages:")
@@ -91,7 +91,7 @@ class InputObserverMiddleware(ChatMiddleware):
context.messages[:] = modified_messages
# Continue to next middleware or AI execution
await next(context)
await call_next(context)
# Observe that processing is complete
print("[InputObserverMiddleware] Processing completed")
@@ -100,7 +100,7 @@ class InputObserverMiddleware(ChatMiddleware):
@chat_middleware
async def security_and_override_middleware(
context: ChatContext,
next: Callable[[ChatContext], Awaitable[None]],
call_next: Callable[[ChatContext], Awaitable[None]],
) -> None:
"""Function-based middleware that implements security filtering and response override."""
print("[SecurityMiddleware] Processing input...")
@@ -131,7 +131,7 @@ async def security_and_override_middleware(
raise MiddlewareTermination
# Continue to next middleware or AI execution
await next(context)
await call_next(context)
async def class_based_chat_middleware() -> None:
@@ -50,7 +50,7 @@ class SecurityAgentMiddleware(AgentMiddleware):
async def process(
self,
context: AgentContext,
next: Callable[[AgentContext], Awaitable[None]],
call_next: Callable[[AgentContext], Awaitable[None]],
) -> None:
# Check for potential security violations in the query
# Look at the last user message
@@ -63,11 +63,11 @@ class SecurityAgentMiddleware(AgentMiddleware):
context.result = AgentResponse(
messages=[ChatMessage("assistant", ["Detected sensitive information, the request is blocked."])]
)
# Simply don't call next() to prevent execution
# Simply don't call call_next() to prevent execution
return
print("[SecurityAgentMiddleware] Security check passed.")
await next(context)
await call_next(context)
class LoggingFunctionMiddleware(FunctionMiddleware):
@@ -76,14 +76,14 @@ class LoggingFunctionMiddleware(FunctionMiddleware):
async def process(
self,
context: FunctionInvocationContext,
next: Callable[[FunctionInvocationContext], Awaitable[None]],
call_next: Callable[[FunctionInvocationContext], Awaitable[None]],
) -> None:
function_name = context.function.name
print(f"[LoggingFunctionMiddleware] About to call function: {function_name}.")
start_time = time.time()
await next(context)
await call_next(context)
end_time = time.time()
duration = end_time - start_time
@@ -50,18 +50,18 @@ def get_current_time() -> str:
@agent_middleware # Decorator marks this as agent middleware - no type annotations needed
async def simple_agent_middleware(context, next): # type: ignore - parameters intentionally untyped to demonstrate decorator functionality
async def simple_agent_middleware(context, call_next): # type: ignore - parameters intentionally untyped to demonstrate decorator functionality
"""Agent middleware that runs before and after agent execution."""
print("[Agent MiddlewareTypes] Before agent execution")
await next(context)
await call_next(context)
print("[Agent MiddlewareTypes] After agent execution")
@function_middleware # Decorator marks this as function middleware - no type annotations needed
async def simple_function_middleware(context, next): # type: ignore - parameters intentionally untyped to demonstrate decorator functionality
async def simple_function_middleware(context, call_next): # type: ignore - parameters intentionally untyped to demonstrate decorator functionality
"""Function middleware that runs before and after function calls."""
print(f"[Function MiddlewareTypes] Before calling: {context.function.name}") # type: ignore
await next(context)
await call_next(context)
print(f"[Function MiddlewareTypes] After calling: {context.function.name}") # type: ignore
@@ -35,13 +35,13 @@ def unstable_data_service(
async def exception_handling_middleware(
context: FunctionInvocationContext, next: Callable[[FunctionInvocationContext], Awaitable[None]]
context: FunctionInvocationContext, call_next: Callable[[FunctionInvocationContext], Awaitable[None]]
) -> None:
function_name = context.function.name
try:
print(f"[ExceptionHandlingMiddleware] Executing function: {function_name}")
await next(context)
await call_next(context)
print(f"[ExceptionHandlingMiddleware] Function {function_name} completed successfully.")
except TimeoutError as e:
print(f"[ExceptionHandlingMiddleware] Caught TimeoutError: {e}")
@@ -27,7 +27,7 @@ The example includes:
Function-based middleware is ideal for simple, stateless operations and provides a more
lightweight approach compared to class-based middleware. Both agent and function middleware
can be implemented as async functions that accept context and next parameters.
can be implemented as async functions that accept context and call_next parameters.
"""
@@ -43,7 +43,7 @@ def get_weather(
async def security_agent_middleware(
context: AgentContext,
next: Callable[[AgentContext], Awaitable[None]],
call_next: Callable[[AgentContext], Awaitable[None]],
) -> None:
"""Agent middleware that checks for security violations."""
# Check for potential security violations in the query
@@ -53,16 +53,16 @@ async def security_agent_middleware(
query = last_message.text
if "password" in query.lower() or "secret" in query.lower():
print("[SecurityAgentMiddleware] Security Warning: Detected sensitive information, blocking request.")
# Simply don't call next() to prevent execution
# Simply don't call call_next() to prevent execution
return
print("[SecurityAgentMiddleware] Security check passed.")
await next(context)
await call_next(context)
async def logging_function_middleware(
context: FunctionInvocationContext,
next: Callable[[FunctionInvocationContext], Awaitable[None]],
call_next: Callable[[FunctionInvocationContext], Awaitable[None]],
) -> None:
"""Function middleware that logs function calls."""
function_name = context.function.name
@@ -70,7 +70,7 @@ async def logging_function_middleware(
start_time = time.time()
await next(context)
await call_next(context)
end_time = time.time()
duration = end_time - start_time
@@ -23,7 +23,7 @@ MiddlewareTypes Termination Example
This sample demonstrates how middleware can terminate execution using the `context.terminate` flag.
The example includes:
- PreTerminationMiddleware: Terminates execution before calling next() to prevent agent processing
- PreTerminationMiddleware: Terminates execution before calling call_next() to prevent agent processing
- PostTerminationMiddleware: Allows processing to complete but terminates further execution
This is useful for implementing security checks, rate limiting, or early exit conditions.
@@ -49,7 +49,7 @@ class PreTerminationMiddleware(AgentMiddleware):
async def process(
self,
context: AgentContext,
next: Callable[[AgentContext], Awaitable[None]],
call_next: Callable[[AgentContext], Awaitable[None]],
) -> None:
# Check if the user message contains any blocked words
last_message = context.messages[-1] if context.messages else None
@@ -75,7 +75,7 @@ class PreTerminationMiddleware(AgentMiddleware):
# Set terminate flag to prevent further processing
raise MiddlewareTermination
await next(context)
await call_next(context)
class PostTerminationMiddleware(AgentMiddleware):
@@ -88,7 +88,7 @@ class PostTerminationMiddleware(AgentMiddleware):
async def process(
self,
context: AgentContext,
next: Callable[[AgentContext], Awaitable[None]],
call_next: Callable[[AgentContext], Awaitable[None]],
) -> None:
print(f"[PostTerminationMiddleware] Processing request (response count: {self.response_count})")
@@ -101,7 +101,7 @@ class PostTerminationMiddleware(AgentMiddleware):
raise MiddlewareTermination
# Allow the agent to process normally
await next(context)
await call_next(context)
# Increment response count after processing
self.response_count += 1
@@ -48,11 +48,11 @@ def get_weather(
return f"The weather in {location} is {conditions[randint(0, 3)]} with a high of {randint(10, 30)}°C."
async def weather_override_middleware(context: ChatContext, next: Callable[[ChatContext], Awaitable[None]]) -> None:
async def weather_override_middleware(context: ChatContext, call_next: Callable[[ChatContext], Awaitable[None]]) -> None:
"""Chat middleware that overrides weather results for both streaming and non-streaming cases."""
# Let the original agent execution complete first
await next(context)
await call_next(context)
# Check if there's a result to override (agent called weather function)
if context.result is not None:
@@ -83,9 +83,9 @@ async def weather_override_middleware(context: ChatContext, next: Callable[[Chat
context.result = ChatResponse(messages=[ChatMessage(role="assistant", text=custom_message)])
async def validate_weather_middleware(context: ChatContext, next: Callable[[ChatContext], Awaitable[None]]) -> None:
async def validate_weather_middleware(context: ChatContext, call_next: Callable[[ChatContext], Awaitable[None]]) -> None:
"""Chat middleware that simulates result validation for both streaming and non-streaming cases."""
await next(context)
await call_next(context)
validation_note = "Validation: weather data verified."
@@ -103,9 +103,9 @@ async def validate_weather_middleware(context: ChatContext, next: Callable[[Chat
context.result.messages.append(ChatMessage(role="assistant", text=validation_note))
async def agent_cleanup_middleware(context: AgentContext, next: Callable[[AgentContext], Awaitable[None]]) -> None:
async def agent_cleanup_middleware(context: AgentContext, call_next: Callable[[AgentContext], Awaitable[None]]) -> None:
"""Agent middleware that validates chat middleware effects and cleans the result."""
await next(context)
await call_next(context)
if context.result is None:
return
@@ -54,7 +54,7 @@ class SessionContextContainer:
async def inject_context_middleware(
self,
context: FunctionInvocationContext,
next: Callable[[FunctionInvocationContext], Awaitable[None]],
call_next: Callable[[FunctionInvocationContext], Awaitable[None]],
) -> None:
"""MiddlewareTypes that extracts runtime context from kwargs and stores in container.
@@ -74,7 +74,7 @@ class SessionContextContainer:
print(f" - Session Metadata Keys: {list(self.session_metadata.keys())}")
# Continue to tool execution
await next(context)
await call_next(context)
# Create a container instance that will be shared via closure
@@ -278,19 +278,19 @@ async def pattern_2_hierarchical_with_kwargs_propagation() -> None:
@function_middleware
async def email_kwargs_tracker(
context: FunctionInvocationContext, next: Callable[[FunctionInvocationContext], Awaitable[None]]
context: FunctionInvocationContext, call_next: Callable[[FunctionInvocationContext], Awaitable[None]]
) -> None:
email_agent_kwargs.update(context.kwargs)
print(f"[EmailAgent] Received runtime context: {list(context.kwargs.keys())}")
await next(context)
await call_next(context)
@function_middleware
async def sms_kwargs_tracker(
context: FunctionInvocationContext, next: Callable[[FunctionInvocationContext], Awaitable[None]]
context: FunctionInvocationContext, call_next: Callable[[FunctionInvocationContext], Awaitable[None]]
) -> None:
sms_agent_kwargs.update(context.kwargs)
print(f"[SMSAgent] Received runtime context: {list(context.kwargs.keys())}")
await next(context)
await call_next(context)
client = OpenAIChatClient(model_id="gpt-4o-mini")
@@ -359,7 +359,7 @@ class AuthContextMiddleware:
self.validated_tokens: list[str] = []
async def validate_and_track(
self, context: FunctionInvocationContext, next: Callable[[FunctionInvocationContext], Awaitable[None]]
self, context: FunctionInvocationContext, call_next: Callable[[FunctionInvocationContext], Awaitable[None]]
) -> None:
"""Validate API token and track usage."""
api_token = context.kwargs.get("api_token")
@@ -375,7 +375,7 @@ class AuthContextMiddleware:
else:
print("[AuthMiddleware] No API token provided")
await next(context)
await call_next(context)
@tool(approval_mode="never_require")
@@ -57,7 +57,7 @@ class MiddlewareContainer:
async def call_counter_middleware(
self,
context: FunctionInvocationContext,
next: Callable[[FunctionInvocationContext], Awaitable[None]],
call_next: Callable[[FunctionInvocationContext], Awaitable[None]],
) -> None:
"""First middleware: increments call count in shared state."""
# Increment the shared call count
@@ -66,18 +66,18 @@ class MiddlewareContainer:
print(f"[CallCounter] This is function call #{self.call_count}")
# Call the next middleware/function
await next(context)
await call_next(context)
async def result_enhancer_middleware(
self,
context: FunctionInvocationContext,
next: Callable[[FunctionInvocationContext], Awaitable[None]],
call_next: Callable[[FunctionInvocationContext], Awaitable[None]],
) -> None:
"""Second middleware: uses shared call count to enhance function results."""
print(f"[ResultEnhancer] Current total calls so far: {self.call_count}")
# Call the next middleware/function
await next(context)
await call_next(context)
# After function execution, enhance the result using shared state
if context.result:
@@ -21,14 +21,14 @@ The example shows:
- How AgentContext.thread property behaves across multiple runs
- How middleware can access conversation history through the thread
- The timing of when thread messages are populated (before vs after next() call)
- The timing of when thread messages are populated (before vs after call_next() call)
- How to track thread state changes across runs
Key behaviors demonstrated:
1. First run: context.messages is populated, context.thread is initially empty (before next())
2. After next(): thread contains input message + response from agent
1. First run: context.messages is populated, context.thread is initially empty (before call_next())
2. After call_next(): thread contains input message + response from agent
3. Second run: context.messages contains only current input, thread contains previous history
4. After next(): thread contains full conversation history (all previous + current messages)
4. After call_next(): thread contains full conversation history (all previous + current messages)
"""
@@ -46,7 +46,7 @@ def get_weather(
async def thread_tracking_middleware(
context: AgentContext,
next: Callable[[AgentContext], Awaitable[None]],
call_next: Callable[[AgentContext], Awaitable[None]],
) -> None:
"""MiddlewareTypes that tracks and logs thread behavior across runs."""
thread_messages = []
@@ -56,8 +56,8 @@ async def thread_tracking_middleware(
print(f"[MiddlewareTypes pre-execution] Current input messages: {len(context.messages)}")
print(f"[MiddlewareTypes pre-execution] Thread history messages: {len(thread_messages)}")
# Call next to execute the agent
await next(context)
# Call call_next to execute the agent
await call_next(context)
# Check thread state after agent execution
updated_thread_messages = []