mirror of
https://github.com/microsoft/agent-framework.git
synced 2026-06-16 21:04:09 +08:00
Python: Fix hyperlight WasmSandbox cross-thread Drop and harden hosted-agent sample (#5603)
* update hyperlight to beta and move samples, add hosted agent sample * Python: Fix hyperlight WasmSandbox cross-thread Drop and harden sample Root cause: when a worker-side closure raised, the exception's __traceback__ retained frame locals that included the partially constructed PyO3 sandbox. Future.result() re-raised that exception on the caller thread, and when the caller's exception was eventually GC'd the frame locals were released off-thread, dec_ref'ing the unsendable sandbox from the wrong thread and tripping the PyO3 panic '_native_wasm::WasmSandbox is unsendable, but is being dropped on another thread'. Fix: * Add _SandboxWorker._run_on_worker which catches every exception on the worker, drops __traceback__ there, deletes the original exception, and re-raises a fresh instance on the caller thread. initialize and execute route through it; dispose keeps its bare-submit semantics. * Add an opt-in diagnostic module _drop_diagnostic (no-op unless HYPERLIGHT_TRACE_DROPS=1) that installs a sys.unraisablehook and dumps owner-thread + per-thread stacks on any future cross-thread unsendable Drop. Useful for triaging similar PyO3 regressions. * Tests: cross-thread invocation, traceback-leak isolation, _SandboxEntry attribute-shape check, and a stale-reference stress test driven through asyncio.to_thread. Sample (samples/04-hosting/foundry-hosted-agents/responses/06_hyperlight_codeact): * Dockerfile installs agent-framework-* from in-tree source with python/ as build context so unreleased fixes can be validated end-to-end. * call_server.py pins the Responses API version. * main.py enables include_detailed_errors=True so future tool failures surface the actual exception text instead of a bare 'Error: Function failed.' string. * README.md documents the in-tree-package build and the Hyperlight hypervisor requirement (/dev/kvm on Linux, MSHV on Windows). Hosted environments without hypervisor passthrough surface 'No Hypervisor was found for Sandbox'; this is a hosting constraint, not a hyperlight bug. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Python: remove _drop_diagnostic from hyperlight package The diagnostic module was useful while bisecting the cross-thread Drop bug, but it is no longer needed now that _SandboxWorker._run_on_worker prevents the panic at the source. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Python: address PR review feedback on hyperlight - Use lazy agent_framework.hyperlight import in sample main.py. - Env-driven endpoint (FOUNDRY_AGENT_ENDPOINT) in call_server.py; remove personal URLs. - Align agent.yaml model deployment with manifest (gpt-4.1-mini). - Tighten Dockerfile requirements guard; drop dangling deploy.ps1 reference. - Preserve exception args when sanitizing tracebacks in _run_on_worker. - Add public _SandboxWorker.is_alive(); update test to avoid private attr. - Add namespace coverage tests for agent_framework.hyperlight lazy loader. - Add prominent note: Foundry hosted-agent runtime does not yet support Hyperlight (no hypervisor exposed); container works locally with /dev/kvm. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Python: bump hyperlight-sandbox dependencies to 0.4.x Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Python: renumber hyperlight codeact sample to 08 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Coerce worker exception args to strings for cross-thread safety Stringify exc.args on the worker thread before propagating, so any PyO3 unsendable object captured in args (e.g. via a caller-supplied callback or underlying SDK) cannot be Dropped on the calling thread. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * moved sample --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
Eduard van Valkenburg
committed by
GitHub
Unverified
GitHub
parent
36b9b41e3b
commit
57c901a245
@@ -0,0 +1,31 @@
|
||||
# Hyperlight CodeAct context provider
|
||||
|
||||
Demonstrates the provider-owned [Hyperlight](https://github.com/hyperlight-dev/hyperlight)
|
||||
CodeAct flow. `HyperlightCodeActProvider` injects an `execute_code` tool into the
|
||||
agent and keeps the registered sandbox tools (`compute`, `fetch_data`) hidden
|
||||
from the model — the model must call them from inside the sandbox using
|
||||
`call_tool(...)`.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install agent-framework agent-framework-hyperlight --pre
|
||||
```
|
||||
|
||||
> The Hyperlight Wasm backend is currently published only for `linux/x86_64` and
|
||||
> `win32/AMD64` with Python `<3.14`. On other platforms `execute_code` will fail
|
||||
> at runtime when it tries to create the sandbox.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- An Azure AI Foundry project endpoint (`FOUNDRY_PROJECT_ENDPOINT`)
|
||||
- A deployed model (`FOUNDRY_MODEL`)
|
||||
- Azure CLI authenticated (`az login`)
|
||||
|
||||
## Run
|
||||
|
||||
```bash
|
||||
python code_act.py
|
||||
```
|
||||
|
||||
See [`code_act.py`](code_act.py) for the full annotated example.
|
||||
@@ -0,0 +1,187 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import os
|
||||
from collections.abc import Awaitable, Callable
|
||||
from typing import Annotated, Any, Literal
|
||||
|
||||
from agent_framework import Agent, FunctionInvocationContext, function_middleware, tool
|
||||
from agent_framework.foundry import FoundryChatClient
|
||||
from agent_framework.hyperlight import HyperlightCodeActProvider
|
||||
from azure.identity import AzureCliCredential
|
||||
from dotenv import load_dotenv
|
||||
|
||||
"""This sample demonstrates the provider-owned Hyperlight CodeAct flow.
|
||||
|
||||
The sample keeps `compute` and `fetch_data` off the direct agent tool surface and
|
||||
registers them only with `HyperlightCodeActProvider`. The model therefore sees a
|
||||
single `execute_code` tool and must call the provider-owned tools from inside
|
||||
the sandbox with `call_tool(...)`.
|
||||
"""
|
||||
|
||||
load_dotenv()
|
||||
|
||||
_CYAN = "\033[36m"
|
||||
_YELLOW = "\033[33m"
|
||||
_GREEN = "\033[32m"
|
||||
_DIM = "\033[2m"
|
||||
_RESET = "\033[0m"
|
||||
|
||||
|
||||
class _ColoredFormatter(logging.Formatter):
|
||||
"""Dim logger output so it does not compete with sample prints."""
|
||||
|
||||
def format(self, record: logging.LogRecord) -> str:
|
||||
return f"{_DIM}{super().format(record)}{_RESET}"
|
||||
|
||||
|
||||
logging.basicConfig(level=logging.WARNING)
|
||||
logging.getLogger().handlers[0].setFormatter(
|
||||
_ColoredFormatter("[%(asctime)s] %(levelname)s: %(message)s"),
|
||||
)
|
||||
|
||||
|
||||
@function_middleware
|
||||
async def log_function_calls(
|
||||
context: FunctionInvocationContext,
|
||||
call_next: Callable[[], Awaitable[None]],
|
||||
) -> None:
|
||||
"""Log tool calls, including readable execute_code blocks."""
|
||||
import time
|
||||
|
||||
function_name = context.function.name
|
||||
arguments = context.arguments if isinstance(context.arguments, dict) else {}
|
||||
|
||||
if function_name == "execute_code" and "code" in arguments:
|
||||
print(f"\n{_YELLOW}{'─' * 60}")
|
||||
print("▶ execute_code")
|
||||
print(f"{'─' * 60}{_RESET}")
|
||||
print(arguments["code"])
|
||||
print(f"{_YELLOW}{'─' * 60}{_RESET}")
|
||||
else:
|
||||
pairs = ", ".join(f"{name}={value!r}" for name, value in arguments.items())
|
||||
print(f"\n{_YELLOW}▶ {function_name}({pairs}){_RESET}")
|
||||
|
||||
start = time.perf_counter()
|
||||
await call_next()
|
||||
elapsed = time.perf_counter() - start
|
||||
|
||||
result = context.result
|
||||
if function_name == "execute_code" and isinstance(result, list):
|
||||
for output in result:
|
||||
if output.type == "text" and output.text:
|
||||
print(f"{_GREEN}stdout:\n{output.text}{_RESET}")
|
||||
elif output.type == "error" and output.error_details:
|
||||
print(f"{_YELLOW}stderr:\n{output.error_details}{_RESET}")
|
||||
else:
|
||||
print(f"{_YELLOW}◀ {function_name} → {result!r}{_RESET}")
|
||||
|
||||
print(f"{_DIM} ({elapsed:.4f}s){_RESET}")
|
||||
|
||||
|
||||
@tool(approval_mode="never_require")
|
||||
def compute(
|
||||
operation: Annotated[
|
||||
Literal["add", "subtract", "multiply", "divide"],
|
||||
"Math operation: add, subtract, multiply, or divide.",
|
||||
],
|
||||
a: Annotated[float, "First numeric operand."],
|
||||
b: Annotated[float, "Second numeric operand."],
|
||||
) -> float:
|
||||
"""Perform a math operation for sandboxed code."""
|
||||
operations = {
|
||||
"add": a + b,
|
||||
"subtract": a - b,
|
||||
"multiply": a * b,
|
||||
"divide": a / b if b else float("inf"),
|
||||
}
|
||||
return operations[operation]
|
||||
|
||||
|
||||
@tool(approval_mode="never_require")
|
||||
async def fetch_data(
|
||||
table: Annotated[str, "Name of the simulated table to query."],
|
||||
) -> list[dict[str, Any]]:
|
||||
"""Fetch records from a named table."""
|
||||
await asyncio.sleep(0.5)
|
||||
data: dict[str, list[dict[str, Any]]] = {
|
||||
"users": [
|
||||
{"id": 1, "name": "Alice", "role": "admin"},
|
||||
{"id": 2, "name": "Bob", "role": "user"},
|
||||
{"id": 3, "name": "Charlie", "role": "admin"},
|
||||
],
|
||||
"products": [
|
||||
{"id": 101, "name": "Widget", "price": 9.99},
|
||||
{"id": 102, "name": "Gadget", "price": 19.99},
|
||||
],
|
||||
}
|
||||
return data.get(table, [])
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
"""Run the provider-owned Hyperlight CodeAct sample."""
|
||||
# 1. Create the Hyperlight-backed provider and register sandbox tools on it.
|
||||
codeact = HyperlightCodeActProvider(
|
||||
tools=[compute, fetch_data],
|
||||
approval_mode="never_require",
|
||||
)
|
||||
|
||||
# 2. Create the client and the agent.
|
||||
agent = Agent(
|
||||
client=FoundryChatClient(
|
||||
project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"],
|
||||
model=os.environ["FOUNDRY_MODEL"],
|
||||
credential=AzureCliCredential(),
|
||||
),
|
||||
name="HyperlightCodeActProviderAgent",
|
||||
instructions="You are a helpful assistant.",
|
||||
context_providers=[codeact],
|
||||
middleware=[log_function_calls],
|
||||
)
|
||||
|
||||
# 3. Run a request that should use execute_code plus provider-owned tools.
|
||||
query = (
|
||||
"Fetch all users, find admins, multiply 7*(3*2), and print the users, "
|
||||
"admins, and multiplication result. Use execute_code and call_tool(...) "
|
||||
"inside the sandbox."
|
||||
)
|
||||
print(f"{_CYAN}{'=' * 60}")
|
||||
print("Hyperlight CodeAct provider sample")
|
||||
print(f"{'=' * 60}{_RESET}")
|
||||
print(f"{_CYAN}User: {query}{_RESET}")
|
||||
result = await agent.run(query)
|
||||
print(f"{_CYAN}Agent: {result.text}{_RESET}")
|
||||
|
||||
|
||||
"""
|
||||
Sample output (shape only):
|
||||
|
||||
============================================================
|
||||
Hyperlight CodeAct provider sample
|
||||
============================================================
|
||||
User: Fetch all users, find admins, multiply 7*(3*2), ...
|
||||
|
||||
────────────────────────────────────────────────────────────
|
||||
▶ execute_code
|
||||
────────────────────────────────────────────────────────────
|
||||
users = call_tool("fetch_data", table="users")
|
||||
admins = [user for user in users if user["role"] == "admin"]
|
||||
result = call_tool("compute", operation="multiply", a=7, b=6)
|
||||
print("Users:", users)
|
||||
print("Admins:", admins)
|
||||
print("7 * 6 =", result)
|
||||
────────────────────────────────────────────────────────────
|
||||
stdout:
|
||||
Users: [...]
|
||||
Admins: [...]
|
||||
7 * 6 = 42.0
|
||||
(0.0xxx s)
|
||||
Agent: ...
|
||||
"""
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,37 @@
|
||||
# Hyperlight local code interpreter
|
||||
|
||||
Demonstrates the standalone [Hyperlight](https://github.com/hyperlight-dev/hyperlight)
|
||||
`HyperlightExecuteCodeTool` — a sandboxed local code interpreter that the agent
|
||||
can invoke directly. Two patterns are shown:
|
||||
|
||||
| File | Pattern |
|
||||
|------|---------|
|
||||
| [`local_code_interpreter.py`](local_code_interpreter.py) | **Standalone tool** — `HyperlightExecuteCodeTool` is added to the agent tool list and self-describes its sandbox tools, so no extra agent instructions are needed. Best for quick prototyping. |
|
||||
| [`local_code_interpreter_manual_wiring.py`](local_code_interpreter_manual_wiring.py) | **Manual static wiring** — sandbox tools and CodeAct instructions are built once and passed to the `Agent` constructor alongside a direct-only tool (`send_email`). Best when the tool set is fixed for the agent's lifetime. |
|
||||
|
||||
For the recommended provider-driven pattern (with dynamic tool / capability
|
||||
management), see
|
||||
[`../../context_providers/code_act/`](../../context_providers/code_act/).
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
pip install agent-framework agent-framework-hyperlight --pre
|
||||
```
|
||||
|
||||
> The Hyperlight Wasm backend is currently published only for `linux/x86_64` and
|
||||
> `win32/AMD64` with Python `<3.14`. On other platforms `execute_code` will fail
|
||||
> at runtime when it tries to create the sandbox.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- An Azure AI Foundry project endpoint (`FOUNDRY_PROJECT_ENDPOINT`)
|
||||
- A deployed model (`FOUNDRY_MODEL`)
|
||||
- Azure CLI authenticated (`az login`)
|
||||
|
||||
## Run
|
||||
|
||||
```bash
|
||||
python local_code_interpreter.py
|
||||
python local_code_interpreter_manual_wiring.py
|
||||
```
|
||||
@@ -0,0 +1,109 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
from typing import Annotated, Any, Literal
|
||||
|
||||
from agent_framework import Agent, tool
|
||||
from agent_framework.foundry import FoundryChatClient
|
||||
from agent_framework.hyperlight import HyperlightExecuteCodeTool
|
||||
from azure.identity import AzureCliCredential
|
||||
from dotenv import load_dotenv
|
||||
|
||||
"""This sample demonstrates the standalone Hyperlight execute_code tool.
|
||||
|
||||
The sample adds `HyperlightExecuteCodeTool` directly to the agent. The tool's
|
||||
own description advertises `call_tool(...)`, the registered sandbox tools, and
|
||||
the current capability configuration, so no extra CodeAct-specific agent
|
||||
instructions are required.
|
||||
"""
|
||||
|
||||
load_dotenv()
|
||||
|
||||
|
||||
@tool(approval_mode="never_require")
|
||||
def compute(
|
||||
operation: Annotated[
|
||||
Literal["add", "subtract", "multiply", "divide"],
|
||||
"Math operation: add, subtract, multiply, or divide.",
|
||||
],
|
||||
a: Annotated[float, "First numeric operand."],
|
||||
b: Annotated[float, "Second numeric operand."],
|
||||
) -> float:
|
||||
"""Perform a math operation used by sandboxed code."""
|
||||
operations = {
|
||||
"add": a + b,
|
||||
"subtract": a - b,
|
||||
"multiply": a * b,
|
||||
"divide": a / b if b else float("inf"),
|
||||
}
|
||||
return operations[operation]
|
||||
|
||||
|
||||
@tool(approval_mode="never_require")
|
||||
def fetch_data(
|
||||
table: Annotated[str, "Name of the simulated table to query."],
|
||||
) -> list[dict[str, Any]]:
|
||||
"""Fetch simulated records from a named table."""
|
||||
data: dict[str, list[dict[str, Any]]] = {
|
||||
"users": [
|
||||
{"id": 1, "name": "Alice", "role": "admin"},
|
||||
{"id": 2, "name": "Bob", "role": "user"},
|
||||
{"id": 3, "name": "Charlie", "role": "admin"},
|
||||
],
|
||||
"products": [
|
||||
{"id": 101, "name": "Widget", "price": 9.99},
|
||||
{"id": 102, "name": "Gadget", "price": 19.99},
|
||||
],
|
||||
}
|
||||
return data.get(table, [])
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
"""Run the standalone execute_code sample."""
|
||||
# 1. Create the packaged execute_code tool and register sandbox tools on it.
|
||||
execute_code = HyperlightExecuteCodeTool(
|
||||
tools=[compute, fetch_data],
|
||||
approval_mode="never_require",
|
||||
)
|
||||
|
||||
# 2. Create the client and the agent.
|
||||
agent = Agent(
|
||||
client=FoundryChatClient(
|
||||
project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"],
|
||||
model=os.environ["FOUNDRY_MODEL"],
|
||||
credential=AzureCliCredential(),
|
||||
),
|
||||
name="HyperlightExecuteCodeToolAgent",
|
||||
instructions="You are a helpful assistant.",
|
||||
tools=execute_code,
|
||||
)
|
||||
|
||||
# 3. Run one request through the direct-tool surface.
|
||||
print("=" * 60)
|
||||
print("Hyperlight execute_code tool sample")
|
||||
print("=" * 60)
|
||||
query = (
|
||||
"Fetch all users, find admins, multiply 6*7, and print the users, admins, "
|
||||
"and multiplication result. Use one execute_code call."
|
||||
)
|
||||
print(f"User: {query}")
|
||||
result = await agent.run(query)
|
||||
print(f"Agent: {result.text}")
|
||||
|
||||
|
||||
"""
|
||||
Sample output (shape only):
|
||||
|
||||
============================================================
|
||||
Hyperlight execute_code tool sample
|
||||
============================================================
|
||||
User: Fetch all users, find admins, multiply 6*7, ...
|
||||
Agent: ...
|
||||
"""
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
+132
@@ -0,0 +1,132 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
from typing import Annotated, Any, Literal
|
||||
|
||||
from agent_framework import Agent, tool
|
||||
from agent_framework.foundry import FoundryChatClient
|
||||
from agent_framework.hyperlight import HyperlightExecuteCodeTool
|
||||
from azure.identity import AzureCliCredential
|
||||
from dotenv import load_dotenv
|
||||
|
||||
"""This sample demonstrates manual static wiring of CodeAct without a provider.
|
||||
|
||||
Instead of using `HyperlightCodeActProvider` with `context_providers=`, this
|
||||
sample creates a `HyperlightExecuteCodeTool` directly, extracts its CodeAct
|
||||
instructions once, and passes both to the `Agent` constructor at build time.
|
||||
|
||||
This avoids the per-run provider lifecycle (`before_run` / `after_run`) and is
|
||||
well-suited when the tool registry, file mounts, and network allow-list are
|
||||
fixed for the agent's lifetime. The tradeoff is that dynamic tool or capability
|
||||
changes between runs are not supported — any mutations to the tool would not
|
||||
update the agent's instructions automatically.
|
||||
"""
|
||||
|
||||
load_dotenv()
|
||||
|
||||
|
||||
@tool(approval_mode="never_require")
|
||||
def compute(
|
||||
operation: Annotated[
|
||||
Literal["add", "subtract", "multiply", "divide"],
|
||||
"Math operation: add, subtract, multiply, or divide.",
|
||||
],
|
||||
a: Annotated[float, "First numeric operand."],
|
||||
b: Annotated[float, "Second numeric operand."],
|
||||
) -> float:
|
||||
"""Perform a math operation used by sandboxed code."""
|
||||
operations = {
|
||||
"add": a + b,
|
||||
"subtract": a - b,
|
||||
"multiply": a * b,
|
||||
"divide": a / b if b else float("inf"),
|
||||
}
|
||||
return operations[operation]
|
||||
|
||||
|
||||
@tool(approval_mode="never_require")
|
||||
def fetch_data(
|
||||
table: Annotated[str, "Name of the simulated table to query."],
|
||||
) -> list[dict[str, Any]]:
|
||||
"""Fetch simulated records from a named table."""
|
||||
data: dict[str, list[dict[str, Any]]] = {
|
||||
"users": [
|
||||
{"id": 1, "name": "Alice", "role": "admin"},
|
||||
{"id": 2, "name": "Bob", "role": "user"},
|
||||
{"id": 3, "name": "Charlie", "role": "admin"},
|
||||
],
|
||||
"products": [
|
||||
{"id": 101, "name": "Widget", "price": 9.99},
|
||||
{"id": 102, "name": "Gadget", "price": 19.99},
|
||||
],
|
||||
}
|
||||
return data.get(table, [])
|
||||
|
||||
|
||||
@tool(approval_mode="never_require")
|
||||
def send_email(
|
||||
to: Annotated[str, "Recipient email address."],
|
||||
subject: Annotated[str, "Email subject line."],
|
||||
body: Annotated[str, "Email body text."],
|
||||
) -> str:
|
||||
"""Simulate sending an email (direct-only tool, not available inside the sandbox)."""
|
||||
return f"Email sent to {to}: {subject}"
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
"""Run the manual static-wiring sample."""
|
||||
# 1. Create the execute_code tool and register sandbox tools on it.
|
||||
execute_code = HyperlightExecuteCodeTool(
|
||||
tools=[compute, fetch_data],
|
||||
approval_mode="never_require",
|
||||
)
|
||||
|
||||
# 2. Build CodeAct instructions once. Setting tools_visible_to_model=False
|
||||
# tells the instructions builder that sandbox tools are not in the agent's
|
||||
# direct tool list, so the model must use call_tool(...) inside execute_code.
|
||||
codeact_instructions = execute_code.build_instructions(tools_visible_to_model=False)
|
||||
|
||||
# 3. Create the client and the agent with everything wired at construction time.
|
||||
# - send_email is a direct-only tool (not available inside the sandbox).
|
||||
# - execute_code carries sandbox tools (compute, fetch_data) via call_tool.
|
||||
agent = Agent(
|
||||
client=FoundryChatClient(
|
||||
project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"],
|
||||
model=os.environ["FOUNDRY_MODEL"],
|
||||
credential=AzureCliCredential(),
|
||||
),
|
||||
name="ManualWiringAgent",
|
||||
instructions=f"You are a helpful assistant.\n\n{codeact_instructions}",
|
||||
tools=[send_email, execute_code],
|
||||
)
|
||||
|
||||
# 4. Run a request that exercises both the sandbox and the direct tool.
|
||||
print("=" * 60)
|
||||
print("Manual static-wiring CodeAct sample")
|
||||
print("=" * 60)
|
||||
query = (
|
||||
"Fetch all users, find admins, multiply 6*7, and print the users, admins, "
|
||||
"and multiplication result. Use one execute_code call. "
|
||||
"Then send an email to admin@example.com summarising the results."
|
||||
)
|
||||
print(f"User: {query}")
|
||||
result = await agent.run(query)
|
||||
print(f"Agent: {result.text}")
|
||||
|
||||
|
||||
"""
|
||||
Sample output (shape only):
|
||||
|
||||
============================================================
|
||||
Manual static-wiring CodeAct sample
|
||||
============================================================
|
||||
User: Fetch all users, find admins, multiply 6*7, ...
|
||||
Agent: ...
|
||||
"""
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,6 @@
|
||||
.venv
|
||||
__pycache__
|
||||
*.pyc
|
||||
*.pyo
|
||||
*.pyd
|
||||
.Python
|
||||
@@ -0,0 +1,2 @@
|
||||
FOUNDRY_PROJECT_ENDPOINT="..."
|
||||
AZURE_AI_MODEL_DEPLOYMENT_NAME="..."
|
||||
@@ -0,0 +1,36 @@
|
||||
# Build this image with the repository's `python/` directory as the build context so
|
||||
# the in-tree agent-framework packages can be installed from source. From the repo root:
|
||||
#
|
||||
# docker build \
|
||||
# -f python/samples/04-hosting/foundry-hosted-agents/responses/08_hyperlight_codeact/Dockerfile \
|
||||
# -t <acr>.azurecr.io/<image>:<tag> \
|
||||
# python/
|
||||
FROM python:3.12-slim
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
# Copy the in-tree agent-framework packages we need. Order matters for editable
|
||||
# installs because of inter-package dependencies; we install in dependency order
|
||||
# below. Hyperlight backends are platform gated, so we install them via pip
|
||||
# resolution rather than copying the wheels.
|
||||
COPY packages/core /opt/af/core
|
||||
COPY packages/openai /opt/af/openai
|
||||
COPY packages/foundry /opt/af/foundry
|
||||
COPY packages/foundry_hosting /opt/af/foundry_hosting
|
||||
COPY packages/hyperlight /opt/af/hyperlight
|
||||
|
||||
# Copy just the sample we care about into the user agent location.
|
||||
COPY samples/04-hosting/foundry-hosted-agents/responses/08_hyperlight_codeact/ /app/user_agent/
|
||||
WORKDIR /app/user_agent
|
||||
|
||||
RUN pip install --no-cache-dir --upgrade pip \
|
||||
&& pip install --no-cache-dir /opt/af/core \
|
||||
&& pip install --no-cache-dir /opt/af/openai \
|
||||
&& pip install --no-cache-dir /opt/af/foundry \
|
||||
&& pip install --no-cache-dir /opt/af/foundry_hosting \
|
||||
&& pip install --no-cache-dir /opt/af/hyperlight \
|
||||
&& if grep -Eq '^[[:space:]]*[^#[:space:]]' requirements.txt; then pip install --no-cache-dir -r requirements.txt; fi
|
||||
|
||||
EXPOSE 8088
|
||||
|
||||
CMD ["python", "main.py"]
|
||||
@@ -0,0 +1,85 @@
|
||||
# What this sample demonstrates
|
||||
|
||||
An [Agent Framework](https://github.com/microsoft/agent-framework) agent that
|
||||
runs Python in a [Hyperlight](https://github.com/hyperlight-dev/hyperlight)
|
||||
WebAssembly sandbox via the **CodeAct** pattern, hosted using the **Responses
|
||||
protocol**. The model is only given a single `execute_code` tool. Local Python
|
||||
tools (`compute`, `fetch_data`) are registered on `HyperlightCodeActProvider`
|
||||
and are reachable from inside the sandbox via `call_tool(...)`, never as
|
||||
direct LLM tools. All of this can be run as a container, however not under all circumstances.
|
||||
|
||||
> **⚠️ Foundry hosted-agent runtime support is in progress.**
|
||||
> Hyperlight requires a hypervisor (`/dev/kvm` on Linux, MSHV on Windows). The
|
||||
> default Foundry hosted-agent runtime does not currently expose a hypervisor
|
||||
> to the workload container, so deploying this sample as a Foundry hosted
|
||||
> agent will fail at runtime with
|
||||
> `Failed to create sandbox: ... No Hypervisor was found for Sandbox`.
|
||||
> The sample container itself works end-to-end when run locally with
|
||||
> `docker run --device=/dev/kvm ...` (see [Hypervisor requirement](#hypervisor-requirement)
|
||||
> below). We are working with the platform team to enable a hypervisor-capable
|
||||
> hosting target.
|
||||
|
||||
## How It Works
|
||||
|
||||
### Model integration
|
||||
|
||||
The agent uses `FoundryChatClient` to talk to a Foundry-hosted model deployment.
|
||||
A `HyperlightCodeActProvider` is attached as a context provider, which on every
|
||||
run injects the `execute_code` tool plus the CodeAct instructions that teach the
|
||||
model how to author Python that calls `call_tool(...)` for sandbox-only tools.
|
||||
|
||||
See [`main.py`](main.py) for the full implementation.
|
||||
|
||||
### Agent hosting
|
||||
|
||||
The agent is hosted with `ResponsesHostServer` from
|
||||
`agent-framework-foundry-hosting`, which exposes a REST endpoint compatible with
|
||||
the OpenAI Responses protocol.
|
||||
|
||||
> The Hyperlight Wasm backend is currently published only for `linux/x86_64` and
|
||||
> `win32/AMD64` with Python `<3.14`. The hosted container runs `python:3.12-slim`
|
||||
> on linux/x86_64, which is supported.
|
||||
|
||||
### Hypervisor requirement
|
||||
|
||||
Hyperlight executes guest WebAssembly inside a micro-VM and **requires a
|
||||
hypervisor on the host**:
|
||||
|
||||
- **Linux:** `/dev/kvm` must be present *and* the container must have access to
|
||||
it (`docker run --device=/dev/kvm ...`).
|
||||
- **Windows:** the Microsoft Hypervisor Platform (MSHV) must be enabled.
|
||||
|
||||
Without a hypervisor, sandbox creation fails with:
|
||||
|
||||
```
|
||||
Failed to create sandbox: failed to build ProtoWasmSandbox: No Hypervisor was found for Sandbox
|
||||
```
|
||||
|
||||
This affects hosted environments that don't expose `/dev/kvm` to the workload
|
||||
container (most managed PaaS, including the default Foundry hosted-agent
|
||||
runtime). To run this sample as a hosted agent you need a hosting target with
|
||||
nested virtualization and `/dev/kvm` device passthrough — for example an Azure
|
||||
VM, AKS nodes with KVM enabled, or Azure Container Instances configured for
|
||||
nested virt.
|
||||
|
||||
## Running the Agent Host
|
||||
|
||||
Follow the instructions in the
|
||||
[Running the Agent Host Locally](../../foundry-hosted-agents//README.md#running-the-agent-host-locally)
|
||||
section of the README in the Foundry Hosted Agent directory.
|
||||
|
||||
## Interacting with the agent
|
||||
|
||||
Send a POST request to the server with a JSON body containing an `"input"`
|
||||
field. The model should respond by calling `execute_code` with Python that uses
|
||||
`call_tool(...)` to reach the sandbox-only tools:
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:8088/responses \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"input": "Fetch all users, find the admins, multiply 7 by 6, and print the users, admins and multiplication result. Use execute_code with call_tool(...)."}'
|
||||
```
|
||||
|
||||
## Deploying the Agent to Foundry
|
||||
|
||||
Deploying this container to Foundry will not work yet, as soon as it does, we will update this sample.
|
||||
@@ -0,0 +1,24 @@
|
||||
name: agent-framework-agent-with-hyperlight-codeact-responses
|
||||
description: >
|
||||
An Agent Framework agent with a Hyperlight CodeAct sandbox hosted by Foundry.
|
||||
metadata:
|
||||
tags:
|
||||
- Agent Framework
|
||||
- AI Agent Hosting
|
||||
- Azure AI AgentServer
|
||||
- Responses Protocol
|
||||
- Streaming
|
||||
- Hyperlight CodeAct
|
||||
template:
|
||||
name: agent-framework-agent-with-hyperlight-codeact-responses
|
||||
kind: hosted
|
||||
protocols:
|
||||
- protocol: responses
|
||||
version: 1.0.0
|
||||
environment_variables:
|
||||
- name: AZURE_AI_MODEL_DEPLOYMENT_NAME
|
||||
value: "{{AZURE_AI_MODEL_DEPLOYMENT_NAME}}"
|
||||
resources:
|
||||
- kind: model
|
||||
id: gpt-4.1-mini
|
||||
name: AZURE_AI_MODEL_DEPLOYMENT_NAME
|
||||
@@ -0,0 +1,23 @@
|
||||
# yaml-language-server: $schema=https://raw.githubusercontent.com/microsoft/AgentSchema/refs/heads/main/schemas/v1.0/ContainerAgent.yaml
|
||||
|
||||
kind: hosted
|
||||
name: agent-framework-agent-with-hyperlight-codeact-responses
|
||||
description: |
|
||||
An Agent Framework agent with a Hyperlight CodeAct sandbox hosted by Foundry.
|
||||
metadata:
|
||||
tags:
|
||||
- Agent Framework
|
||||
- AI Agent Hosting
|
||||
- Azure AI AgentServer
|
||||
- Responses Protocol
|
||||
- Streaming
|
||||
- Hyperlight CodeAct
|
||||
protocols:
|
||||
- protocol: responses
|
||||
version: 1.0.0
|
||||
resources:
|
||||
cpu: "1"
|
||||
memory: 2Gi
|
||||
environment_variables:
|
||||
- name: AZURE_AI_MODEL_DEPLOYMENT_NAME
|
||||
value: gpt-4.1-mini
|
||||
@@ -0,0 +1,41 @@
|
||||
# /// script
|
||||
# requires-python = ">=3.10"
|
||||
# dependencies = [
|
||||
# "openai>=1.50,<3",
|
||||
# "azure-identity>=1.19,<2",
|
||||
# ]
|
||||
# ///
|
||||
# Run with: uv run call_server.py
|
||||
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
"""Call the deployed Hyperlight CodeAct Foundry hosted agent via the OpenAI client."""
|
||||
|
||||
import os
|
||||
|
||||
from azure.identity import AzureCliCredential
|
||||
from openai import OpenAI
|
||||
|
||||
# Set FOUNDRY_AGENT_ENDPOINT to your deployed agent endpoint, e.g.
|
||||
# https://<your-foundry-resource>.services.ai.azure.com/api/projects/<project>/agents/<agent-name>
|
||||
ENDPOINT = os.environ.get(
|
||||
"FOUNDRY_AGENT_ENDPOINT",
|
||||
"https://<your-foundry-resource>.services.ai.azure.com"
|
||||
"/api/projects/<project>/agents/<agent-name>",
|
||||
)
|
||||
SCOPE = "https://ai.azure.com/.default"
|
||||
PROMPT = (
|
||||
"Fetch all users, find the admins, multiply 7 by 6, and print the users, "
|
||||
"admins and multiplication result. Use execute_code with call_tool(...)."
|
||||
)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
token = AzureCliCredential().get_token(SCOPE).token
|
||||
client = OpenAI(base_url=ENDPOINT, api_key=token, default_query={"api-version": "v1"})
|
||||
response = client.responses.create(model="hosted-agent", input=PROMPT)
|
||||
print(response.output_text)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,89 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
from typing import Annotated, Any, Literal
|
||||
|
||||
from agent_framework import Agent, tool
|
||||
from agent_framework.foundry import FoundryChatClient
|
||||
from agent_framework.hyperlight import HyperlightCodeActProvider
|
||||
from agent_framework_foundry_hosting import ResponsesHostServer
|
||||
from azure.identity import DefaultAzureCredential
|
||||
from dotenv import load_dotenv
|
||||
|
||||
# Load environment variables from .env file
|
||||
load_dotenv()
|
||||
|
||||
|
||||
@tool(approval_mode="never_require")
|
||||
def compute(
|
||||
operation: Annotated[
|
||||
Literal["add", "subtract", "multiply", "divide"],
|
||||
"Math operation: add, subtract, multiply, or divide.",
|
||||
],
|
||||
a: Annotated[float, "First numeric operand."],
|
||||
b: Annotated[float, "Second numeric operand."],
|
||||
) -> float:
|
||||
"""Perform a math operation for sandboxed code."""
|
||||
operations = {
|
||||
"add": a + b,
|
||||
"subtract": a - b,
|
||||
"multiply": a * b,
|
||||
"divide": a / b if b else float("inf"),
|
||||
}
|
||||
return operations[operation]
|
||||
|
||||
|
||||
@tool(approval_mode="never_require")
|
||||
async def fetch_data(
|
||||
table: Annotated[str, "Name of the simulated table to query."],
|
||||
) -> list[dict[str, Any]]:
|
||||
"""Fetch records from a named table."""
|
||||
await asyncio.sleep(0.5)
|
||||
data: dict[str, list[dict[str, Any]]] = {
|
||||
"users": [
|
||||
{"id": 1, "name": "Alice", "role": "admin"},
|
||||
{"id": 2, "name": "Bob", "role": "user"},
|
||||
{"id": 3, "name": "Charlie", "role": "admin"},
|
||||
],
|
||||
"products": [
|
||||
{"id": 101, "name": "Widget", "price": 9.99},
|
||||
{"id": 102, "name": "Gadget", "price": 19.99},
|
||||
],
|
||||
}
|
||||
return data.get(table, [])
|
||||
|
||||
|
||||
def main():
|
||||
# 1. Create the Foundry chat client.
|
||||
client = FoundryChatClient(
|
||||
project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"],
|
||||
model=os.environ["AZURE_AI_MODEL_DEPLOYMENT_NAME"],
|
||||
credential=DefaultAzureCredential(),
|
||||
function_invocation_configuration={"include_detailed_errors": True},
|
||||
)
|
||||
|
||||
# 2. Register sandbox tools on a Hyperlight CodeAct provider. The model only
|
||||
# sees `execute_code`; `compute` and `fetch_data` are reachable from
|
||||
# inside the sandbox via `call_tool(...)`.
|
||||
codeact = HyperlightCodeActProvider(
|
||||
tools=[compute, fetch_data],
|
||||
approval_mode="never_require",
|
||||
)
|
||||
|
||||
# 3. Build the agent. History is managed by the hosting infrastructure, so
|
||||
# request the model not to persist server-side conversation state.
|
||||
agent = Agent(
|
||||
client=client,
|
||||
instructions="You are a helpful assistant. Keep your answers brief.",
|
||||
context_providers=[codeact],
|
||||
default_options={"store": False},
|
||||
)
|
||||
|
||||
# 4. Serve the agent over the Foundry Responses protocol.
|
||||
server = ResponsesHostServer(agent)
|
||||
server.run()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,3 @@
|
||||
# agent-framework, agent-framework-foundry-hosting, and agent-framework-hyperlight
|
||||
# are installed from local source by the Dockerfile (build context = python/).
|
||||
# Add any sample-only third-party deps here.
|
||||
@@ -222,4 +222,4 @@ This will package your agent and deploy it to the Foundry environment, making it
|
||||
|
||||
For the full deployment guide, see the [official deployment guide](https://learn.microsoft.com/en-us/azure/foundry/agents/how-to/deploy-hosted-agent).
|
||||
|
||||
Once deployed, learn more about how to manage deployed agents in the [official management guide](https://learn.microsoft.com/en-us/azure/foundry/agents/how-to/manage-hosted-agent).
|
||||
Once deployed, learn more about how to manage deployed agents in the [official management guide](https://learn.microsoft.com/en-us/azure/foundry/agents/how-to/manage-hosted-agent).
|
||||
|
||||
Reference in New Issue
Block a user