diff --git a/.gitignore b/.gitignore index 2267fb20c4..c846efea7b 100644 --- a/.gitignore +++ b/.gitignore @@ -203,6 +203,8 @@ temp*/ # AI .claude/ +.omc/ +.omx/ WARP.md **/memory-bank/ **/projectBrief.md diff --git a/docs/decisions/0025-foundry-toolbox-support.md b/docs/decisions/0025-foundry-toolbox-support.md new file mode 100644 index 0000000000..a68b98b3bf --- /dev/null +++ b/docs/decisions/0025-foundry-toolbox-support.md @@ -0,0 +1,454 @@ +--- +status: proposed +contact: evmattso +date: 2026-04-10 +deciders: evmattso +--- + +# Foundry Toolbox Support in FoundryChatClient + +## What is the goal of this feature? + +Enable Agent Framework users to consume Foundry **toolboxes** — named, versioned bundles of tool definitions stored server-side in an Azure AI Foundry project — directly from `FoundryChatClient`, without dropping to the raw `azure-ai-projects` SDK. + +A user who has configured a toolbox in the Foundry portal (or via the raw SDK) should be able to load it into an agent with a single call: + +```python +toolbox = await client.get_toolbox("research_tools") +agent = Agent(client=client, instructions="...", tools=toolbox) +``` + +**Success metric:** an agent can consume a toolbox with no manual handling of version-resolution logic on the user's side. + +## What is the problem being solved? + +`azure-ai-projects==2.1.0a20260409002` ships a new `BetaToolboxesOperations` surface, reachable as `AIProjectClient.beta.toolboxes` on the raw SDK client (and therefore as `FoundryChatClient.project_client.beta.toolboxes` through our wrapper), that lets teams: +- Group related hosted tools (code interpreter, file search, MCP, web search, etc.) under a named toolbox +- Version toolboxes immutably, so agents can pin to a specific configuration for production stability +- Share toolboxes across multiple agents in a project + +However, consuming a toolbox from the framework today requires: +1. Knowing the raw SDK accessor path (`client.project_client.beta.toolboxes`) +2. Making two calls for the common case — `.get(name)` to find the default version, then `.get_version(name, version)` to actually retrieve tools +3. Manually unpacking `toolbox.tools` before passing them to `Agent(tools=...)` + +None of this is hard, but it's the kind of boilerplate that should live in the client. Every other hosted tool in `FoundryChatClient` (code interpreter, file search, web search, image generation, MCP) already has a factory method (`get_code_interpreter_tool()`, etc.). Toolbox support should fit the same shape on the chat-client composition surface. + +## API Changes + +### One new method on the FoundryChatClient surface + +The public toolbox-consumption surface lands on: + +- `RawFoundryChatClient` (inherited by `FoundryChatClient`) in `_chat_client.py` + +The implementation delegates to shared helper functions in `_tools.py` so there is a single source of truth for the SDK calls. + +**Scope note:** `FoundryAgent` is intentionally not part of this design. `FoundryAgent` is the runtime surface for invoking an already-configured server-side Foundry agent; if that agent should use a toolbox, the toolbox/tools should already be configured on the Foundry side (UI or `azure-ai-projects` authoring flow) before MAF connects to it. + +**Scope note:** Authoring a server-side agent whose definition references a toolbox (via `PromptAgentDefinition(tools=toolbox.tools, ...)` + `client.agents.create_version(...)`) is deliberately outside MAF scope. That is an `azure-ai-projects` / service-resource authoring concern, not a future MAF feature. Users who need it should use the raw Azure SDK directly. + +```python +async def get_toolbox( + self, + name: str, + *, + version: str | None = None, +) -> ToolboxVersionObject: + """Fetch a Foundry toolbox by name. + + If ``version`` is ``None``, resolves the toolbox's current default version + (two requests). If ``version`` is specified, fetches that version directly + (single request). + + :param name: The name of the toolbox. + :param version: Optional immutable version identifier to pin to. + :return: A ``ToolboxVersionObject``. Pass its ``tools`` attribute to + ``Agent(tools=toolbox.tools)``. + :raises azure.core.exceptions.ResourceNotFoundError: If the toolbox or + version does not exist. + """ + +``` + +### Return types: raw SDK models, no custom wrappers + +Methods return the `azure.ai.projects.models` types directly: + +- `get_toolbox()` → `ToolboxVersionObject` (has `.name`, `.version`, `.tools`, `.id`, `.created_at`, `.description`, `.metadata`, `.policies`) + +No custom wrapper classes are defined. Returning the SDK types directly: +- Eliminates maintenance overhead of keeping a custom wrapper aligned with SDK changes +- Matches the existing convention — `get_code_interpreter_tool()` returns the raw `CodeInterpreterTool` SDK type +- Means any new fields the SDK adds to these types flow through automatically + +`Agent(..., tools=...)` will accept the fetched toolbox object directly by flattening to `toolbox.tools` internally. + +### Design decisions + +**Instance methods, not `@staticmethod` factories.** Existing `get_code_interpreter_tool()` / `get_mcp_tool()` / etc. are `@staticmethod` because they're pure factories with no network I/O. Toolbox fetching requires the project client, so these new methods must be instance methods. This is a deliberate departure from the existing-factory pattern, justified by the async-with-I/O nature of the operation. + +**Raw SDK type passthrough (no custom wrappers).** There is only one toolbox type in the Foundry SDK and maintaining a shadow wrapper would create alignment risk as the SDK evolves. The raw `ToolboxVersionObject` and `ToolboxObject` carry all the fields users need. Individual tools inside `toolbox.tools` are the same `azure.ai.projects.models.Tool` subclasses returned by other factory methods. + +**Two-request default-version path.** When `version=None`, implementation calls `.get(name)` to find `default_version`, then `.get_version(name, default_version)` for the tools. Caching the default-version mapping was considered and rejected — default versions can change server-side via `update(default_version=...)`, and a stale cache would silently give callers the wrong tools. Two requests at agent setup is acceptable. + +**No discovery/listing surface in MAF.** Discovery is intentionally left to the raw `azure-ai-projects` client. MAF does not currently expose project-resource listing surfaces for many other Foundry resources (deployments, vector stores, agents, etc.), so the toolbox design stays narrowly focused on explicit retrieval by name/version. + +**Shared helpers in `_tools.py`.** The SDK-call helper function (`fetch_toolbox`) lives in a shared module so the chat-client surface stays thin and the request logic remains centralized. + +**`tools=toolbox` convenience, not a new wrapper type.** Although `get_toolbox()` returns the raw `ToolboxVersionObject`, Agent Framework can still support `tools=toolbox` / `tools=[toolbox]` by flattening the toolbox's `.tools` internally. That matches existing SDK ergonomics where some higher-level objects can be placed directly in `tools=` and unpacked underneath, without introducing a public `FoundryToolbox` wrapper. + +**Errors pass through unchanged.** `ResourceNotFoundError`, `HttpResponseError`, etc. from the SDK propagate as-is. No framework-specific exception hierarchy. + +## E2E Code Samples + +### Primary sample + +New file: `samples/02-agents/providers/foundry/foundry_chat_client_with_toolbox.py` + +```python +import asyncio + +from agent_framework import Agent +from agent_framework.foundry import FoundryChatClient +from azure.identity import AzureCliCredential + + +async def main() -> None: + client = FoundryChatClient(credential=AzureCliCredential()) + + toolbox = await client.get_toolbox("research_tools") + print(f"Loaded toolbox {toolbox.name}@{toolbox.version} ({len(toolbox.tools)} tools)") + + agent = Agent( + client=client, + instructions="You are a research assistant.", + tools=toolbox, + ) + + result = await agent.run("What are the latest developments in quantum error correction?") + print(f"Result: {result}") + + +if __name__ == "__main__": + asyncio.run(main()) +``` + +### Version pinning + +```python +toolbox = await client.get_toolbox("research_tools", version="v3") +``` + +### Combining multiple toolboxes + +```python +toolbox_a = await client.get_toolbox("research_tools") +toolbox_b = await client.get_toolbox("some_other_tools", version="v3") + +agent = Agent( + client=client, + instructions="...", + tools=[toolbox_a, toolbox_b], +) +``` + +### Combining toolbox tools with locally defined tools + +```python +toolbox = await client.get_toolbox("research_tools") + +def get_internal_metrics(metric_name: str) -> dict: + """Custom tool that reads from an internal dashboard.""" + ... + +agent = Agent( + client=client, + instructions="...", + tools=[get_internal_metrics, toolbox], +) +``` + +### Selecting only some tools from a toolbox + +Developers will not always want to pass the entire toolbox through unchanged. A +small helper in the Foundry package provides local post-fetch selection without +changing the raw return type of `get_toolbox()`. + +```python +from agent_framework.foundry import select_toolbox_tools + +toolbox = await client.get_toolbox("research_tools") + +selected_tools = select_toolbox_tools( + toolbox, + include_names=["githubmcp", "code_interpreter"], +) + +agent = Agent( + client=client, + instructions="Use only the selected toolbox tools.", + tools=selected_tools, +) +``` + +Supported filters: + +```python +from agent_framework.foundry import FoundryHostedToolType, select_toolbox_tools + +selected_tools = select_toolbox_tools( + toolbox, + include_types=["mcp", "code_interpreter"], # type: Collection[FoundryHostedToolType] + exclude_names=["internal_admin_tool"], +) +``` + +Helper signature: + +```python +type FoundryHostedToolType = Literal[ + "code_interpreter", + "file_search", + "image_generation", + "mcp", + "web_search", +] | str + +def select_toolbox_tools( + tools: ToolboxVersionObject | Sequence[Tool | dict[str, Any]], + *, + include_names: Collection[str] | None = None, + exclude_names: Collection[str] | None = None, + include_types: Collection[FoundryHostedToolType] | None = None, + exclude_types: Collection[FoundryHostedToolType] | None = None, + predicate: Callable[[Tool | dict[str, Any]], bool] | None = None, +) -> list[Tool | dict[str, Any]]: + ... +``` + +Normalized name precedence for `include_names` / `exclude_names`: + +1. MCP `server_label` +2. generic tool `name` +3. fallback tool `type` + +This keeps `get_toolbox()` as a thin fetch API and makes selection an explicit, +local post-processing step, while still allowing the ergonomic +`select_toolbox_tools(toolbox, ...)` call shape. + +## Native vs MCP consumption of a Foundry toolbox + +A Foundry toolbox can be consumed two ways. This design adds new implementation work only for the first: + +1. **Native consumption (in scope).** Tools execute inside Foundry's agent runtime. `get_toolbox()` returns the `ToolboxVersionObject` whose `.tools` attribute carries typed tool configs that the runtime interprets server-side. This design is specifically for `FoundryChatClient`-backed local agent composition. + +2. **MCP consumption (already supported through existing MCP abstractions).** A Foundry toolbox can also be exposed as an MCP server. In that case, use the existing `MCPStreamableHTTPTool(name=..., url=...)` — it already handles this path with any chat client (Foundry, OpenAI, Anthropic, etc.). No new Foundry-specific API is needed for MCP-exposed toolboxes in this design. + +### MCPStreamableHTTPTool example for a Foundry toolbox endpoint + +If Foundry gives you an MCP endpoint for the toolbox (for example from the +toolbox details UI / endpoint surface), the existing MCP client path is: + +```python +from agent_framework import Agent, MCPStreamableHTTPTool +from agent_framework.openai import OpenAIChatClient + +toolbox_mcp = MCPStreamableHTTPTool( + name="research_tools", + url="https://", +) + +agent = Agent( + client=OpenAIChatClient(), + instructions="You are a research assistant.", + tools=[toolbox_mcp], +) +``` + +This is a different integration shape than `get_toolbox(...).tools`: + +- `get_toolbox(...).tools` = **native Foundry hosted-tool configs** interpreted by the + Foundry runtime +- `MCPStreamableHTTPTool(name=..., url=...)` = **live MCP server connection** to a + toolbox endpoint + +The design in this spec adds first-class support only for the native hosted-tool +path. The MCP path is already served by the framework's existing MCP abstractions. + +These paths are not unified because they have fundamentally different execution models. Native toolbox tools are declarative configs the Foundry runtime executes; MCP consumption is a live wire protocol to a running server. + +**MCP authentication inside a toolbox** is handled server-side via `project_connection_id` on individual `MCPTool` entries (OAuth connection objects configured in the Foundry project). The client never holds bearer tokens. Consent flow handling (`CONSENT_REQUIRED` → user-visible consent URL) happens during `agent.run()`, not during toolbox fetching — see Non-goals. + +## Testing Strategy + +Unit tests in `packages/foundry/tests/test_toolbox.py` with mocked `project_client.beta.toolboxes`. A single opt-in live round-trip, `test_integration_get_toolbox_round_trip_against_real_project`, is marked `@pytest.mark.integration`; it is skipped by default and only runs when the required Foundry credentials are available. + +Coverage: + +- `get_toolbox(name, version="v3")` — explicit version, single request. Assert `.get` not called, `.get_version` awaited once, returns `ToolboxVersionObject`. +- `get_toolbox(name)` — default-version resolution. Assert `.get` then `.get_version` called in order with correct args. +- Error propagation — `ResourceNotFoundError` from `.get` propagates unchanged. +- Tool passthrough — heterogeneous tool list (`CodeInterpreterTool`, `MCPTool(project_connection_id=...)`) passes through unchanged. Asserts `project_connection_id` survives. +- Agent integration smoke — `tools=toolbox` / `tools=[toolbox]` flatten to the underlying toolbox tools. +- Multiple toolbox composition smoke — `tools=[toolbox_a, toolbox_b]` flattens into a single agent tool list. +- `get_toolbox_tool_name()` — selection-name precedence is MCP `server_label`, then `name`, then `type`. +- `select_toolbox_tools(toolbox, include_names=...)` — selects by normalized tool names directly from a fetched toolbox object. +- `select_toolbox_tools(toolbox, include_types=...)` — selects by tool types with `Literal`-guided IDE completion. +- `select_toolbox_tools(..., exclude_names=..., predicate=...)` — supports exclusion + custom predicates. + +Deliberately **not** covered: +- Runtime consent-flow handling for OAuth MCP tools (see Non-goals). +- Toolbox discovery/listing (`list_toolboxes`, `list_toolbox_versions`) — deliberately left to the raw Azure SDK. +- Full CRUD (`create_version`, `update`, `delete`) and server-side agent authoring — see Non-goals. + +Live Foundry API integration is exercised only through the opt-in `@pytest.mark.integration` round-trip noted above; it is not part of the default test run. + +## Framework dependency: `normalize_tools` flattening + +The core `normalize_tools` function in `packages/core/agent_framework/_tools.py` already supports flattening composite tool inputs. Toolbox support extends that behavior so a fetched `ToolboxVersionObject` is treated as a composite tool source and flattened to its `.tools`. + +That enables: + +- `tools=toolbox` +- `tools=[toolbox]` +- `tools=[local_tool, toolbox]` +- `tools=[toolbox_a, toolbox_b]` + +while still keeping `select_toolbox_tools(toolbox.tools, ...)` available for partial selection before the final agent construction step. + +## Telemetry + +Telemetry for toolbox support has two separate goals: + +1. **Observe toolbox API access** — `get_toolbox()` +2. **Observe toolbox usage during agent runs** — when users pass toolbox-derived tools into `Agent(..., tools=...)` + +### Request telemetry for toolbox API access + +When Agent Framework constructs the `AIProjectClient` internally for `FoundryChatClient`, it already sets: + +```python +user_agent=AGENT_FRAMEWORK_USER_AGENT +``` + +That means toolbox API requests made through: + +- `project_client.beta.toolboxes.get(...)` +- `project_client.beta.toolboxes.get_version(...)` + +carry the standard MAF user-agent marker and can be queried in backend request logs the same way as other Foundry SDK calls made through framework-owned clients. + +Important constraint: if the caller passes an already-constructed `project_client`, Agent Framework does **not** mutate it to inject the MAF user-agent. In that case, toolbox API request telemetry reflects whatever user-agent behavior that external client was configured with. + +### Runtime telemetry for toolbox usage on agent runs + +Tool-level telemetry already captures which hosted Foundry tools are available / invoked during agent execution. The remaining gap is **toolbox provenance**: once the user writes `tools=toolbox` (or otherwise flattens the toolbox into tool configs), the framework sees only raw tool configs and no longer knows which toolbox name/version supplied them. + +The design for closing the **client-side** observability gap is **internal provenance tracking**, not user-supplied metadata and not a new public wrapper type. + +#### Provenance model + +Note: this section is still under investigation. + +When `get_toolbox()` or `list_toolbox_versions()` returns a `ToolboxVersionObject`, Agent Framework will attach private provenance metadata to: + +- the returned toolbox object +- each tool inside `toolbox.tools` + +Recommended shape (private, internal-only): + +```python +tool._maf_toolbox_sources = [ + { + "id": toolbox.id, + "name": toolbox.name, + "version": toolbox.version, + } +] +``` + +Key properties of this approach: + +- **No new public API surface** — users still work with raw `ToolboxVersionObject` / `ToolboxObject` +- **No user burden** — callers do not need to stamp metadata manually +- **Provenance follows the tool objects** — works with: + - `tools=toolbox.tools` + - `tools=[toolbox_a.tools, toolbox_b.tools]` + - `tools=[*toolbox_a.tools, *toolbox_b.tools]` +- **Private attributes are not serialized** into the actual request payload sent to the model/service, so this metadata does not leak into the tool definition body + +This is intentionally preferred over introducing a new public `FoundryToolbox` wrapper purely for telemetry, and preferred over a separate global provenance registry. The provenance lives on the existing tool objects so list-copying and chat-option merging naturally preserve it. + +#### Span enrichment + +When Agent / chat telemetry computes span attributes for a run, it should inspect the final tool list and aggregate the private toolbox provenance from any tool objects that carry it. The aggregated values are then emitted as attributes on the existing run/chat spans. + +Suggested custom attributes: + +- `agent_framework.foundry.toolbox.ids` +- `agent_framework.foundry.toolbox.names` +- `agent_framework.foundry.toolbox.versions` +- or a single compact attribute such as `agent_framework.foundry.toolbox.sources=["research_tools@1","some_other_tools@3"]` + +The single compact `toolbox.sources` form is preferred for initial implementation because it is easy to query and easy to render from combined tool lists. + +#### Scope of telemetry changes + +This design does **not** require new spans. It enriches existing telemetry: + +- toolbox API access continues to rely on request logs + Azure SDK distributed tracing + MAF user-agent +- agent/chat execution spans gain toolbox provenance attributes when toolbox-derived tools are present + +Implementation-wise, this design most likely touches: + +- `packages/foundry/agent_framework_foundry/_tools.py` — to stamp provenance on fetched toolbox objects / tools +- `packages/core/agent_framework/observability.py` — to aggregate provenance into span attributes + +#### Important limitation: no server-side toolbox telemetry solution yet + +Private provenance attached to tool objects is only useful on the client side. It +does **not** go over the wire to the Foundry service because those private fields +are intentionally not serialized into the request payload. + +That means this design can support: + +- local OpenTelemetry / exporter spans emitted by Agent Framework +- local attribution of a run to one or more fetched toolboxes + +but it does **not** solve: + +- server-side request-log attribution of a model/tool run back to a toolbox +- backend/database queries that need the service itself to know "this tool came from toolbox X" + +At the moment, we do not have a satisfactory design for server-side toolbox +telemetry. The service would require additional structured information on the +request, and there is no accepted mechanism in this design yet for projecting +toolbox provenance into a server-visible field/header/metadata shape. + +So the telemetry story in this spec is explicitly limited to **client-side +toolbox telemetry**. Server-side toolbox attribution remains an open question and +requires either: + +- new service/API support, or +- a later framework design for emitting additional server-visible request metadata. + +#### Deliberate non-goals for telemetry + +- No requirement for users to pass explicit toolbox metadata in `default_options["metadata"]` or `run(..., options=...)` +- No new public `FoundryToolbox` wrapper type just to preserve attribution +- No attempted server-side attribution mechanism in this design (for example a custom request header or request metadata field) until there is a validated end-to-end contract for it + +## Non-goals / Future Work + +Explicitly out of scope for this design. Each is a separate design and PR when needed. + +1. **Create/update/delete toolboxes from code.** CRUD is rare in agent consumption flows. Users who need it drop to `client.project_client.beta.toolboxes.create_version(...)`, `.update(...)`, `.delete(...)` directly. + +2. **Server-side agent authoring from toolbox.** Creating a `PromptAgentDefinition(tools=toolbox.tools)` + `client.agents.create_version(...)` is a future feature covering agent authoring from code. The toolbox read API provides the building blocks; the authoring helpers are a separate design. + +3. **OAuth consent-flow runtime handling.** When a toolbox contains MCP tools with `project_connection_id` pointing to an OAuth connection, the runtime may return `CONSENT_REQUIRED` mid-run. This is a runtime concern separate from toolbox fetching. + +4. **Live integration tests.** This PR ships unit tests only. + +5. **Toolbox caching or refresh APIs.** Each `get_toolbox()` call hits the network. Users who want caching wrap the call themselves. diff --git a/python/packages/core/agent_framework/_feature_stage.py b/python/packages/core/agent_framework/_feature_stage.py index 1bda62b5d3..761b7860a4 100644 --- a/python/packages/core/agent_framework/_feature_stage.py +++ b/python/packages/core/agent_framework/_feature_stage.py @@ -49,6 +49,7 @@ class ExperimentalFeature(str, Enum): EVALS = "EVALS" FILE_HISTORY = "FILE_HISTORY" SKILLS = "SKILLS" + TOOLBOXES = "TOOLBOXES" class ReleaseCandidateFeature(str, Enum): diff --git a/python/packages/core/agent_framework/_tools.py b/python/packages/core/agent_framework/_tools.py index 75b21d9932..5f5e91b656 100644 --- a/python/packages/core/agent_framework/_tools.py +++ b/python/packages/core/agent_framework/_tools.py @@ -12,6 +12,7 @@ from collections.abc import ( AsyncIterable, Awaitable, Callable, + Iterable, Mapping, Sequence, ) @@ -859,6 +860,15 @@ def normalize_tools( Returns: A normalized list where callable inputs are converted to ``FunctionTool`` using :func:`tool`, and existing tool objects are passed through unchanged. + + Tool-collection wrappers are flattened in two forms: + + - non-tool, non-callable iterables + - mapping-like objects that expose a ``.tools`` collection (for example + ``ToolboxVersionObject`` from azure-ai-projects) + + This lets callers write ``tools=[toolbox, my_func]`` and have the + toolbox's contents spread in alongside individual tools. """ if not tools: return [] @@ -883,6 +893,24 @@ def normalize_tools( if callable(tool_item): # type: ignore[reportUnknownArgumentType] normalized.append(tool(tool_item)) continue + # Mapping-like tool collections (for example ToolboxVersionObject) are + # not flattened by the generic Iterable branch below because they are + # also Mapping instances. If they expose a ``tools`` collection, spread + # that collection into the normalized list. + collection_tools = getattr(tool_item, "tools", None) # type: ignore[reportUnknownArgumentType] + if isinstance(collection_tools, Iterable) and not isinstance( + collection_tools, (str, bytes, bytearray, Mapping) + ): + normalized.extend(normalize_tools(list(collection_tools))) # type: ignore[reportUnknownArgumentType] + continue + # Tool-collection wrapper (e.g. FoundryToolbox): a non-tool, non-callable + # iterable. Flatten its contents so ``tools=[toolbox, my_func]`` works. + # Strings, mappings, and Pydantic BaseModel are excluded — BaseModel + # instances iterate over (field, value) tuples, not tools, so they + # should pass through as leaf tool specs (handled below). + if isinstance(tool_item, Iterable) and not isinstance(tool_item, (str, bytes, bytearray, Mapping, BaseModel)): + normalized.extend(normalize_tools(list(tool_item))) # type: ignore[reportUnknownArgumentType] + continue normalized.append(tool_item) # type: ignore[reportUnknownArgumentType] return normalized diff --git a/python/packages/core/agent_framework/foundry/__init__.py b/python/packages/core/agent_framework/foundry/__init__.py index b1d2b88450..c1e47cd6b8 100644 --- a/python/packages/core/agent_framework/foundry/__init__.py +++ b/python/packages/core/agent_framework/foundry/__init__.py @@ -20,6 +20,7 @@ _IMPORTS: dict[str, tuple[str, str]] = { "FoundryEmbeddingOptions": ("agent_framework_foundry", "agent-framework-foundry"), "FoundryEmbeddingSettings": ("agent_framework_foundry", "agent-framework-foundry"), "FoundryEvals": ("agent_framework_foundry", "agent-framework-foundry"), + "FoundryHostedToolType": ("agent_framework_foundry", "agent-framework-foundry"), "FoundryMemoryProvider": ("agent_framework_foundry", "agent-framework-foundry"), "FoundryLocalChatOptions": ("agent_framework_foundry_local", "agent-framework-foundry-local"), "FoundryLocalClient": ("agent_framework_foundry_local", "agent-framework-foundry-local"), @@ -31,6 +32,9 @@ _IMPORTS: dict[str, tuple[str, str]] = { "RawFoundryEmbeddingClient": ("agent_framework_foundry", "agent-framework-foundry"), "evaluate_foundry_target": ("agent_framework_foundry", "agent-framework-foundry"), "evaluate_traces": ("agent_framework_foundry", "agent-framework-foundry"), + "get_toolbox_tool_name": ("agent_framework_foundry", "agent-framework-foundry"), + "get_toolbox_tool_type": ("agent_framework_foundry", "agent-framework-foundry"), + "select_toolbox_tools": ("agent_framework_foundry", "agent-framework-foundry"), } diff --git a/python/packages/core/agent_framework/foundry/__init__.pyi b/python/packages/core/agent_framework/foundry/__init__.pyi index 47eb92b3af..87cc7a3bda 100644 --- a/python/packages/core/agent_framework/foundry/__init__.pyi +++ b/python/packages/core/agent_framework/foundry/__init__.pyi @@ -12,6 +12,7 @@ from agent_framework_foundry import ( FoundryEmbeddingOptions, FoundryEmbeddingSettings, FoundryEvals, + FoundryHostedToolType, FoundryMemoryProvider, RawFoundryAgent, RawFoundryAgentChatClient, @@ -19,6 +20,9 @@ from agent_framework_foundry import ( RawFoundryEmbeddingClient, evaluate_foundry_target, evaluate_traces, + get_toolbox_tool_name, + get_toolbox_tool_type, + select_toolbox_tools, ) from agent_framework_foundry_local import ( FoundryLocalChatOptions, @@ -35,6 +39,7 @@ __all__ = [ "FoundryEmbeddingOptions", "FoundryEmbeddingSettings", "FoundryEvals", + "FoundryHostedToolType", "FoundryLocalChatOptions", "FoundryLocalClient", "FoundryLocalSettings", @@ -46,4 +51,7 @@ __all__ = [ "RawFoundryEmbeddingClient", "evaluate_foundry_target", "evaluate_traces", + "get_toolbox_tool_name", + "get_toolbox_tool_type", + "select_toolbox_tools", ] diff --git a/python/packages/core/tests/core/test_tools.py b/python/packages/core/tests/core/test_tools.py index 91ba663d84..6fa7172295 100644 --- a/python/packages/core/tests/core/test_tools.py +++ b/python/packages/core/tests/core/test_tools.py @@ -1144,3 +1144,160 @@ def test_parse_annotation_with_annotated_and_literal(): # endregion + + +# region normalize_tools flattening of tool-collection wrappers + + +def _make_flatten_function_tool(name: str) -> FunctionTool: + """Build a FunctionTool for flattening tests.""" + + @tool(name=name, description=f"{name} tool") + def _impl(x: int) -> int: + return x + + return _impl # type: ignore[return-value] + + +def test_normalize_tools_flattens_tool_collection_wrapper() -> None: + """A non-tool, non-callable iterable inside the tools list is flattened.""" + from agent_framework._tools import normalize_tools + + inner_a = _make_flatten_function_tool("inner_a") + inner_b = _make_flatten_function_tool("inner_b") + + class ToolBundle: + """Minimal stand-in for a tool-collection wrapper like FoundryToolbox.""" + + def __init__(self, tools: list[FunctionTool]) -> None: + self._tools = tools + + def __iter__(self): + return iter(self._tools) + + bundle = ToolBundle([inner_a, inner_b]) + + normalized = normalize_tools([bundle]) + + assert len(normalized) == 2 + assert normalized[0] is inner_a + assert normalized[1] is inner_b + + +def test_normalize_tools_combines_bundle_with_individual_tools() -> None: + """The canonical ``tools=[bundle, my_func]`` call site spreads bundle + individual.""" + from agent_framework._tools import normalize_tools + + bundled = _make_flatten_function_tool("bundled") + standalone = _make_flatten_function_tool("standalone") + + class ToolBundle: + def __init__(self, tools: list[FunctionTool]) -> None: + self._tools = tools + + def __iter__(self): + return iter(self._tools) + + normalized = normalize_tools([ToolBundle([bundled]), standalone]) + + assert len(normalized) == 2 + assert normalized[0] is bundled + assert normalized[1] is standalone + + +def test_normalize_tools_flattens_nested_bundles() -> None: + """Bundles inside bundles are flattened recursively via the recursive call.""" + from agent_framework._tools import normalize_tools + + inner = _make_flatten_function_tool("deep") + + class ToolBundle: + def __init__(self, tools: list[Any]) -> None: + self._tools = tools + + def __iter__(self): + return iter(self._tools) + + nested = ToolBundle([ToolBundle([inner])]) + + normalized = normalize_tools([nested]) + + assert len(normalized) == 1 + assert normalized[0] is inner + + +def test_normalize_tools_bundle_only_form() -> None: + """Passing a bundle directly (no outer list) also flattens its contents. + + ``tools=bundle`` — the outer wrap-in-list happens in the non-Sequence + branch, then the flattening logic kicks in on the inner pass. + """ + from agent_framework._tools import normalize_tools + + a = _make_flatten_function_tool("a") + b = _make_flatten_function_tool("b") + + class ToolBundle: + def __init__(self, tools: list[FunctionTool]) -> None: + self._tools = tools + + def __iter__(self): + return iter(self._tools) + + normalized = normalize_tools(ToolBundle([a, b])) # type: ignore[arg-type] + + assert len(normalized) == 2 + assert normalized[0] is a + assert normalized[1] is b + + +def test_normalize_tools_does_not_flatten_known_tool_types() -> None: + """FunctionTool / dict / callable are detected before the flatten branch.""" + from agent_framework._tools import normalize_tools + + func_tool = _make_flatten_function_tool("ft") + dict_tool: dict[str, Any] = {"type": "code_interpreter", "container": {"type": "auto"}} + + def plain_callable(x: int) -> int: + return x + + normalized = normalize_tools([func_tool, dict_tool, plain_callable]) + + assert len(normalized) == 3 + assert normalized[0] is func_tool + assert normalized[1] is dict_tool + # plain_callable was wrapped in a FunctionTool via the @tool helper + assert isinstance(normalized[2], FunctionTool) + + +def test_normalize_tools_flattens_mapping_like_toolbox_with_tools_attr() -> None: + """Mapping-like toolbox objects with ``.tools`` should still flatten.""" + from collections.abc import Mapping as MappingABC + + from agent_framework._tools import normalize_tools + + bundled = _make_flatten_function_tool("bundled") + standalone = _make_flatten_function_tool("standalone") + + class ToolBundleMapping(MappingABC[str, Any]): + def __init__(self, tools: list[FunctionTool]) -> None: + self.tools = tools + self._data = {"name": "research_tools", "version": "v1", "tools": tools} + + def __getitem__(self, key: str) -> Any: + return self._data[key] + + def __iter__(self): + return iter(self._data) + + def __len__(self) -> int: + return len(self._data) + + normalized = normalize_tools([ToolBundleMapping([bundled]), standalone]) + + assert len(normalized) == 2 + assert normalized[0] is bundled + assert normalized[1] is standalone + + +# endregion diff --git a/python/packages/foundry/README.md b/python/packages/foundry/README.md index e22fb523a5..26f9a6e309 100644 --- a/python/packages/foundry/README.md +++ b/python/packages/foundry/README.md @@ -1,3 +1,66 @@ # Agent Framework Foundry This package contains the Microsoft Foundry integrations for Microsoft Agent Framework, including Foundry chat clients, preconfigured Foundry agents, Foundry embedding clients, and Foundry memory providers. + +## Toolboxes + +A *toolbox* is a named, versioned bundle of hosted tool configurations — code interpreter, file search, image generation, MCP, web search, and so on — stored inside a Microsoft Foundry project. Toolboxes let you manage tool configuration once and reuse it across agents. + +### Authoring a toolbox + +Toolboxes can be authored two ways: + +- **Foundry portal** — create and version toolboxes through the UI without touching code. +- **Programmatically** — use the [`azure-ai-projects`](https://pypi.org/project/azure-ai-projects/) SDK to create, update, and version toolboxes from Python. + +> Toolbox authoring APIs (`ToolboxVersionObject`, `ToolboxObject`, `project_client.beta.toolboxes.*`) require `azure-ai-projects>=2.1.0`. Earlier versions can only consume toolboxes that already exist. + +### Using toolboxes with `FoundryAgent` + +For hosted `FoundryAgent`, the toolbox must already be attached to the agent in the Microsoft Foundry project. Once attached, the agent invokes its toolbox tools transparently — no client-side wiring required — and you interact with the agent the same way you would with any other tool-equipped Foundry agent. + +### Using toolboxes with `FoundryChatClient` + +There are two patterns for wiring a toolbox into a `FoundryChatClient`-backed agent. + +**1. Fetch, optionally filter, and pass the tools directly** + +Load the toolbox from the Microsoft Foundry project, optionally select a subset of its tools, and hand them to an `Agent` alongside any other tools you own: + +```python +from agent_framework import Agent +from agent_framework.foundry import FoundryChatClient, select_toolbox_tools + +client = FoundryChatClient(...) +toolbox = await client.get_toolbox("my-toolbox", version="3") + +# Pass the whole toolbox: +agent = Agent(client=client, tools=toolbox) + +# Or filter to a subset first: +selected = select_toolbox_tools(toolbox, include_types=["code_interpreter", "mcp"]) +agent = Agent(client=client, tools=selected) +``` + +See [`foundry_chat_client_with_toolbox.py`](../../samples/02-agents/providers/foundry/foundry_chat_client_with_toolbox.py) for a full example, including combining multiple toolboxes. + +**2. Connect to the toolbox's MCP endpoint with `MCPStreamableHTTPTool`** + +Each toolbox is reachable as an MCP server. Instead of fetching and fanning out its individual tool definitions, you can point a MAF `MCPStreamableHTTPTool` at the toolbox's MCP endpoint — the agent then discovers and calls its tools over MCP at runtime: + +```python +from agent_framework import Agent, MCPStreamableHTTPTool +from agent_framework.foundry import FoundryChatClient + +async with Agent( + client=FoundryChatClient(...), + instructions="You are a helpful assistant. Use the toolbox tools when useful.", + tools=MCPStreamableHTTPTool( + name="my_toolbox", + description="Tools served by my Foundry toolbox", + url="https://", + ), +) as agent: + result = await agent.run("What tools are available?") + print(result.text) +``` diff --git a/python/packages/foundry/agent_framework_foundry/__init__.py b/python/packages/foundry/agent_framework_foundry/__init__.py index fbd1376735..b70d1720f2 100644 --- a/python/packages/foundry/agent_framework_foundry/__init__.py +++ b/python/packages/foundry/agent_framework_foundry/__init__.py @@ -16,6 +16,7 @@ from ._foundry_evals import ( evaluate_traces, ) from ._memory_provider import FoundryMemoryProvider +from ._tools import FoundryHostedToolType, get_toolbox_tool_name, get_toolbox_tool_type, select_toolbox_tools try: __version__ = importlib.metadata.version(__name__) @@ -30,6 +31,7 @@ __all__ = [ "FoundryEmbeddingOptions", "FoundryEmbeddingSettings", "FoundryEvals", + "FoundryHostedToolType", "FoundryMemoryProvider", "RawFoundryAgent", "RawFoundryAgentChatClient", @@ -38,4 +40,7 @@ __all__ = [ "__version__", "evaluate_foundry_target", "evaluate_traces", + "get_toolbox_tool_name", + "get_toolbox_tool_type", + "select_toolbox_tools", ] diff --git a/python/packages/foundry/agent_framework_foundry/_agent.py b/python/packages/foundry/agent_framework_foundry/_agent.py index bf5d936d9d..0c7f93ba1f 100644 --- a/python/packages/foundry/agent_framework_foundry/_agent.py +++ b/python/packages/foundry/agent_framework_foundry/_agent.py @@ -34,6 +34,8 @@ from azure.ai.projects.aio import AIProjectClient from azure.core.credentials import TokenCredential from azure.core.credentials_async import AsyncTokenCredential +from ._tools import sanitize_foundry_response_tool + if sys.version_info >= (3, 13): from typing import TypeVar # type: ignore # pragma: no cover else: @@ -307,6 +309,20 @@ class RawFoundryAgentChatClient( # type: ignore[misc] """Skip model check — model is configured on the Foundry agent.""" pass + @override + def _prepare_tools_for_openai( + self, + tools: ToolTypes | Callable[..., Any] | Sequence[ToolTypes | Callable[..., Any]] | None, + ) -> list[Any]: + """Prepare tools for Foundry agent Responses API calls. + + Mirrors ``RawFoundryChatClient`` sanitization so toolbox-fetched MCP + tools with extra read-model fields continue to work through the agent + surface. + """ + response_tools = super()._prepare_tools_for_openai(tools) + return [sanitize_foundry_response_tool(tool_item) for tool_item in response_tools] + def _prepare_messages_for_azure_ai(self, messages: Sequence[Message]) -> tuple[list[Message], str | None]: """Extract system/developer messages as instructions for Azure AI. diff --git a/python/packages/foundry/agent_framework_foundry/_chat_client.py b/python/packages/foundry/agent_framework_foundry/_chat_client.py index d9e2483fbe..7c9eb3a68c 100644 --- a/python/packages/foundry/agent_framework_foundry/_chat_client.py +++ b/python/packages/foundry/agent_framework_foundry/_chat_client.py @@ -16,6 +16,7 @@ from agent_framework import ( load_settings, ) from agent_framework._compaction import CompactionStrategy, TokenizerProtocol +from agent_framework._feature_stage import ExperimentalFeature, experimental from agent_framework.observability import ChatTelemetryLayer from agent_framework_openai._chat_client import OpenAIChatOptions, RawOpenAIChatClient from azure.ai.projects.aio import AIProjectClient @@ -32,6 +33,8 @@ from azure.ai.projects.models import MCPTool as FoundryMCPTool from azure.core.credentials import TokenCredential from azure.core.credentials_async import AsyncTokenCredential +from ._tools import fetch_toolbox, sanitize_foundry_response_tool + if sys.version_info >= (3, 13): from typing import TypeVar # type: ignore # pragma: no cover else: @@ -46,7 +49,8 @@ else: from typing_extensions import TypedDict # type: ignore # pragma: no cover if TYPE_CHECKING: - from agent_framework import ChatAndFunctionMiddlewareTypes + from agent_framework import ChatAndFunctionMiddlewareTypes, ToolTypes + from azure.ai.projects.models import ToolboxVersionObject logger: logging.Logger = logging.getLogger("agent_framework.foundry") @@ -218,6 +222,21 @@ class RawFoundryChatClient( # type: ignore[misc] raise ValueError("model must be a non-empty string") options["model"] = self.model + @override + def _prepare_tools_for_openai( + self, + tools: ToolTypes | Callable[..., Any] | Sequence[ToolTypes | Callable[..., Any]] | None, + ) -> list[Any]: + """Prepare tools for Foundry Responses API calls. + + Foundry toolbox reads can surface MCP tool objects with extra fields + (for example ``name``) that are accepted by the toolbox API but rejected + by the Responses API. Sanitize those hosted-tool payloads before sending + them downstream. + """ + response_tools = super()._prepare_tools_for_openai(tools) + return [sanitize_foundry_response_tool(tool_item) for tool_item in response_tools] + async def configure_azure_monitor( self, enable_sensitive_data: bool = False, @@ -460,6 +479,37 @@ class RawFoundryChatClient( # type: ignore[misc] # endregion + # region Toolbox methods (instance methods — these hit the network) + + @experimental(feature_id=ExperimentalFeature.TOOLBOXES) + async def get_toolbox( + self, + name: str, + *, + version: str | None = None, + ) -> ToolboxVersionObject: + """Fetch a Foundry toolbox by name. + + If ``version`` is omitted, resolves the toolbox's current default version + (two requests). If ``version`` is specified, fetches that version directly + (single request). + + Args: + name: The name of the toolbox. + + Keyword Args: + version: Optional immutable version identifier to pin to. + + Returns: + A ``ToolboxVersionObject``. Pass its ``tools`` attribute to + ``Agent(tools=toolbox.tools)``. + + Raises: + azure.core.exceptions.ResourceNotFoundError: If the toolbox or + the requested version does not exist. + """ + return await fetch_toolbox(self.project_client, name, version) + class FoundryChatClient( # type: ignore[misc] FunctionInvocationLayer[FoundryChatOptionsT], diff --git a/python/packages/foundry/agent_framework_foundry/_tools.py b/python/packages/foundry/agent_framework_foundry/_tools.py new file mode 100644 index 0000000000..3c22872e18 --- /dev/null +++ b/python/packages/foundry/agent_framework_foundry/_tools.py @@ -0,0 +1,166 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Shared tool helpers for Foundry chat clients. + +Includes: + +* *Toolbox* helpers — a *toolbox* is a named, versioned bundle of tool + definitions stored in an Azure AI Foundry project. +* Responses-API payload sanitization for Foundry hosted tools. +""" + +from __future__ import annotations + +from collections.abc import Callable, Collection, Mapping, Sequence +from typing import TYPE_CHECKING, Any, Literal, TypeAlias, cast + +from agent_framework._feature_stage import ExperimentalFeature, experimental +from azure.ai.projects.models import MCPTool as FoundryMCPTool + +if TYPE_CHECKING: + from azure.ai.projects.aio import AIProjectClient + from azure.ai.projects.models import Tool, ToolboxVersionObject + +FoundryHostedToolType: TypeAlias = ( + Literal[ + "code_interpreter", + "file_search", + "image_generation", + "mcp", + "web_search", + ] + | str +) +ToolboxToolSelectionInput: TypeAlias = "ToolboxVersionObject | Sequence[Tool | dict[str, Any]]" + + +@experimental(feature_id=ExperimentalFeature.TOOLBOXES) +async def fetch_toolbox( + project_client: AIProjectClient, + name: str, + version: str | None = None, +) -> ToolboxVersionObject: + """Fetch a toolbox version via an ``AIProjectClient``. + + If ``version`` is omitted, resolves the toolbox's current default + version (two requests: one to ``.get(name)`` for the default version + pointer, one to ``.get_version(name, version)`` for the tools). If + ``version`` is specified, fetches that version directly (single request). + """ + if version is None: + handle = await project_client.beta.toolboxes.get(name) + version = handle.default_version + return await project_client.beta.toolboxes.get_version(name, version) + + +@experimental(feature_id=ExperimentalFeature.TOOLBOXES) +def get_toolbox_tool_name(tool: Tool | dict[str, Any]) -> str | None: + """Return the best-effort display/selection name for a toolbox tool. + + Selection precedence: + 1. MCP ``server_label`` + 2. Generic tool ``name`` + 3. Tool ``type`` + """ + if isinstance(tool, dict): + if server_label := tool.get("server_label"): + return str(server_label) + if name := tool.get("name"): + return str(name) + if tool_type := tool.get("type"): + return str(tool_type) + return None + + if server_label := getattr(tool, "server_label", None): + return str(server_label) + if name := getattr(tool, "name", None): + return str(name) + if tool_type := getattr(tool, "type", None): + return str(tool_type) + return None + + +@experimental(feature_id=ExperimentalFeature.TOOLBOXES) +def get_toolbox_tool_type(tool: Tool | dict[str, Any]) -> str | None: + """Return the raw tool ``type`` if present.""" + tool_type = tool.get("type") if isinstance(tool, dict) else getattr(tool, "type", None) + return str(tool_type) if tool_type is not None else None + + +@experimental(feature_id=ExperimentalFeature.TOOLBOXES) +def select_toolbox_tools( + tools: ToolboxToolSelectionInput, + *, + include_names: Collection[str] | None = None, + exclude_names: Collection[str] | None = None, + include_types: Collection[FoundryHostedToolType] | None = None, + exclude_types: Collection[FoundryHostedToolType] | None = None, + predicate: Callable[[Tool | dict[str, Any]], bool] | None = None, +) -> list[Tool | dict[str, Any]]: + """Filter toolbox tools by normalized name, raw type, and/or predicate. + + Normalized name precedence: + 1. ``server_label`` for MCP tools + 2. ``name`` + 3. ``type`` + """ + tool_items: Sequence[Tool | dict[str, Any]] = ( + tools if isinstance(tools, Sequence) else cast("Sequence[Tool | dict[str, Any]]", tools.tools) + ) + include_name_set = {str(item) for item in include_names} if include_names is not None else None + exclude_name_set = {str(item) for item in exclude_names} if exclude_names is not None else None + include_type_set = {str(item) for item in include_types} if include_types is not None else None + exclude_type_set = {str(item) for item in exclude_types} if exclude_types is not None else None + + selected: list[Tool | dict[str, Any]] = [] + for tool in tool_items: + tool_name = get_toolbox_tool_name(tool) + tool_type = get_toolbox_tool_type(tool) + + if include_name_set is not None and tool_name not in include_name_set: + continue + if exclude_name_set is not None and tool_name in exclude_name_set: + continue + if include_type_set is not None and tool_type not in include_type_set: + continue + if exclude_type_set is not None and tool_type in exclude_type_set: + continue + if predicate is not None and not predicate(tool): + continue + + selected.append(tool) + + return selected + + +@experimental(feature_id=ExperimentalFeature.TOOLBOXES) +def sanitize_foundry_response_tool(tool_item: Any) -> Any: + """Return a Responses-API-safe tool payload for Foundry hosted tools. + + Azure AI Projects toolbox reads can currently return hosted tool objects with + extra read-model decoration fields such as top-level ``name`` and + ``description``. Azure AI Foundry rejects at least ``name`` on Responses API + requests with: + + ``Unknown parameter: 'tools[0].name'``. + + We defensively strip these decoration fields for non-function hosted tools so + the round-trip + ``toolbox.tools -> Agent(..., tools=...) -> run()`` works, while the Azure + SDK/service behavior is corrected upstream. + """ + if isinstance(tool_item, FoundryMCPTool): + sanitized: dict[str, Any] = dict(cast("Mapping[str, Any]", tool_item)) + sanitized.pop("name", None) + sanitized.pop("description", None) + return sanitized + + if isinstance(tool_item, Mapping): + mapping = cast("Mapping[str, Any]", tool_item) + if "type" in mapping and mapping.get("type") not in {"function", "custom"}: + sanitized = dict(mapping) + sanitized.pop("name", None) + sanitized.pop("description", None) + return sanitized + + return cast(Any, tool_item) diff --git a/python/packages/foundry/pyproject.toml b/python/packages/foundry/pyproject.toml index 69d58ee3e5..67feb98c98 100644 --- a/python/packages/foundry/pyproject.toml +++ b/python/packages/foundry/pyproject.toml @@ -26,7 +26,7 @@ dependencies = [ "agent-framework-core>=1.0.1,<2", "agent-framework-openai>=1.0.1,<2", "azure-ai-inference>=1.0.0b9,<1.0.0b10", - "azure-ai-projects>=2.0.0,<3.0", + "azure-ai-projects>=2.1.0,<3.0", ] [tool.uv] diff --git a/python/packages/foundry/tests/foundry/test_foundry_chat_client.py b/python/packages/foundry/tests/foundry/test_foundry_chat_client.py index 40fc06d3ef..a7c5beb822 100644 --- a/python/packages/foundry/tests/foundry/test_foundry_chat_client.py +++ b/python/packages/foundry/tests/foundry/test_foundry_chat_client.py @@ -15,6 +15,7 @@ from agent_framework import ChatResponse, Content, Message, SupportsChatGetRespo from agent_framework._telemetry import AGENT_FRAMEWORK_USER_AGENT from agent_framework.exceptions import ChatClientException, ChatClientInvalidRequestException from agent_framework_openai import OpenAIContentFilterException +from azure.ai.projects.models import MCPTool as FoundryMCPTool from azure.core.exceptions import ResourceNotFoundError from azure.identity import AzureCliCredential from openai import BadRequestError @@ -608,6 +609,82 @@ def test_get_mcp_tool_with_project_connection_id() -> None: assert tool_config["server_label"] == "Docs_MCP" +def test_prepare_tools_for_openai_strips_extraneous_name_from_foundry_mcp_tool() -> None: + """Toolbox-returned MCP tools may carry ``name``; Foundry Responses rejects it.""" + project_client = MagicMock() + project_client.get_openai_client.return_value = _make_mock_openai_client() + client = FoundryChatClient(project_client=project_client, model="test-model") + + tool = FoundryMCPTool( + server_label="githubmcp", + server_url="https://api.githubcopilot.com/mcp", + ) + tool["project_connection_id"] = "githubmcp" + tool["name"] = "githubmcp" + + response_tools = client._prepare_tools_for_openai([tool]) + + assert len(response_tools) == 1 + prepared = response_tools[0] + assert prepared["type"] == "mcp" + assert prepared["server_label"] == "githubmcp" + assert prepared["project_connection_id"] == "githubmcp" + assert "name" not in prepared + + +def test_prepare_tools_for_openai_strips_read_model_fields_from_toolbox_code_interpreter() -> None: + """Toolbox-returned code interpreter tools may carry read-model-only name/description.""" + project_client = MagicMock() + project_client.get_openai_client.return_value = _make_mock_openai_client() + client = FoundryChatClient(project_client=project_client, model="test-model") + + tool = { + "type": "code_interpreter", + "name": "code_interpreter_t6bbtm", + "description": "Toolbox read model description", + "container": {"file_ids": [], "type": "auto"}, + } + + response_tools = client._prepare_tools_for_openai([tool]) + + assert len(response_tools) == 1 + prepared = response_tools[0] + assert prepared["type"] == "code_interpreter" + assert prepared["container"] == {"file_ids": [], "type": "auto"} + assert "name" not in prepared + assert "description" not in prepared + + +def test_prepare_tools_for_openai_strips_name_from_non_function_hosted_tool_dicts() -> None: + """All non-function hosted tool payloads should drop top-level read-model names.""" + project_client = MagicMock() + project_client.get_openai_client.return_value = _make_mock_openai_client() + client = FoundryChatClient(project_client=project_client, model="test-model") + + response_tools = client._prepare_tools_for_openai([ + { + "type": "file_search", + "name": "file_search_tool_123", + "description": "toolbox decoration", + "vector_store_ids": ["vs_123"], + }, + { + "type": "web_search", + "name": "web_search_tool_456", + "description": "toolbox decoration", + }, + ]) + + assert len(response_tools) == 2 + assert response_tools[0]["type"] == "file_search" + assert response_tools[0]["vector_store_ids"] == ["vs_123"] + assert "name" not in response_tools[0] + assert "description" not in response_tools[0] + assert response_tools[1]["type"] == "web_search" + assert "name" not in response_tools[1] + assert "description" not in response_tools[1] + + @pytest.mark.flaky @pytest.mark.integration @skip_if_foundry_integration_tests_disabled diff --git a/python/packages/foundry/tests/test_toolbox.py b/python/packages/foundry/tests/test_toolbox.py new file mode 100644 index 0000000000..1933084e10 --- /dev/null +++ b/python/packages/foundry/tests/test_toolbox.py @@ -0,0 +1,435 @@ +# Copyright (c) Microsoft. All rights reserved. + +"""Unit tests for toolbox helpers on FoundryChatClient. + +Return types are the raw azure-ai-projects SDK models (ToolboxVersionObject, +ToolboxObject) — no custom wrapper. Tests verify the chat-client get path and +tool-selection ergonomics. +""" + +from __future__ import annotations + +import datetime as dt +import os +from typing import Any +from unittest.mock import AsyncMock, MagicMock + +import pytest + +try: + from azure.ai.projects.models import ( + AutoCodeInterpreterToolParam, + CodeInterpreterTool, + Tool, + ToolboxObject, + ToolboxVersionObject, + ) +except ImportError: + pytest.skip( + "Toolbox types require azure-ai-projects>=2.1.0 (unreleased).", + allow_module_level=True, + ) + +from azure.core.exceptions import ResourceNotFoundError +from azure.identity import AzureCliCredential + +# --------------------------------------------------------------------------- # +# Helpers # +# --------------------------------------------------------------------------- # + + +class _AsyncIter: + """Minimal async-iterable for mocking ``AsyncItemPaged`` in tests.""" + + def __init__(self, items: list[Any]) -> None: + self._items = items + + def __aiter__(self) -> _AsyncIter: + self._iter = iter(self._items) + return self + + async def __anext__(self) -> Any: + try: + return next(self._iter) + except StopIteration: + raise StopAsyncIteration from None + + +def _make_code_interpreter() -> CodeInterpreterTool: + return CodeInterpreterTool(container=AutoCodeInterpreterToolParam()) + + +def _make_version_object( + *, + name: str = "research_tools", + version: str = "v1", + tools: list[Tool] | None = None, + description: str | None = None, +) -> ToolboxVersionObject: + return ToolboxVersionObject( + id=f"tbv_{name}_{version}", + name=name, + version=version, + metadata={}, + created_at=dt.datetime(2026, 4, 10, tzinfo=dt.timezone.utc), + tools=tools if tools is not None else [_make_code_interpreter()], + description=description, + ) + + +def _make_mock_foundry_client(*, project_client: MagicMock) -> Any: + """Build a FoundryChatClient wired to a mock project_client.""" + from agent_framework_foundry import FoundryChatClient + + project_client.get_openai_client = MagicMock(return_value=MagicMock()) + return FoundryChatClient(project_client=project_client, model="test-model") + + +# --------------------------------------------------------------------------- # +# get_toolbox — explicit version path # +# --------------------------------------------------------------------------- # + + +async def test_get_toolbox_with_explicit_version_makes_single_request() -> None: + project_client = MagicMock() + version_obj = _make_version_object(name="research_tools", version="v3") + project_client.beta.toolboxes.get_version = AsyncMock(return_value=version_obj) + project_client.beta.toolboxes.get = AsyncMock( + side_effect=AssertionError("get() must not be called when version is explicit") + ) + + client = _make_mock_foundry_client(project_client=project_client) + + toolbox = await client.get_toolbox("research_tools", version="v3") + + assert isinstance(toolbox, ToolboxVersionObject) + assert toolbox.name == "research_tools" + assert toolbox.version == "v3" + project_client.beta.toolboxes.get_version.assert_awaited_once_with("research_tools", "v3") + project_client.beta.toolboxes.get.assert_not_called() + + +# --------------------------------------------------------------------------- # +# get_toolbox — default-version path + error + passthrough + smoke # +# --------------------------------------------------------------------------- # + + +async def test_get_toolbox_default_version_resolves_then_fetches() -> None: + project_client = MagicMock() + handle = ToolboxObject(id="tb_1", name="research_tools", default_version="v5") + version_obj = _make_version_object(name="research_tools", version="v5") + + project_client.beta.toolboxes.get = AsyncMock(return_value=handle) + project_client.beta.toolboxes.get_version = AsyncMock(return_value=version_obj) + + client = _make_mock_foundry_client(project_client=project_client) + + toolbox = await client.get_toolbox("research_tools") + + assert toolbox.version == "v5" + project_client.beta.toolboxes.get.assert_awaited_once_with("research_tools") + project_client.beta.toolboxes.get_version.assert_awaited_once_with("research_tools", "v5") + + +async def test_get_toolbox_propagates_resource_not_found() -> None: + project_client = MagicMock() + project_client.beta.toolboxes.get = AsyncMock(side_effect=ResourceNotFoundError("no such toolbox")) + + client = _make_mock_foundry_client(project_client=project_client) + + with pytest.raises(ResourceNotFoundError): + await client.get_toolbox("missing_toolbox") + + +async def test_get_toolbox_tool_passthrough_preserves_heterogeneous_types() -> None: + """Ensure all Tool subclasses pass through unchanged — critical for MCP tools + with project_connection_id, which must reach the runtime untouched.""" + from azure.ai.projects.models import MCPTool as FoundryMCPTool + + mcp_tool = FoundryMCPTool( + server_label="github_oauth", + server_url="https://api.githubcopilot.com/mcp", + ) + mcp_tool["project_connection_id"] = "conn_abc" + + project_client = MagicMock() + version_obj = _make_version_object( + name="mixed", + version="v1", + tools=[_make_code_interpreter(), mcp_tool], + ) + project_client.beta.toolboxes.get_version = AsyncMock(return_value=version_obj) + + client = _make_mock_foundry_client(project_client=project_client) + + toolbox = await client.get_toolbox("mixed", version="v1") + + assert len(toolbox.tools) == 2 + assert isinstance(toolbox.tools[0], CodeInterpreterTool) + assert isinstance(toolbox.tools[1], FoundryMCPTool) + assert toolbox.tools[1]["project_connection_id"] == "conn_abc" + + +async def test_toolbox_tools_can_be_passed_to_agent() -> None: + """Integration smoke: toolbox.tools can be passed directly to Agent(tools=...) .""" + from agent_framework import Agent + + project_client = MagicMock() + version_obj = _make_version_object(name="research_tools", version="v1", tools=[_make_code_interpreter()]) + project_client.beta.toolboxes.get_version = AsyncMock(return_value=version_obj) + + client = _make_mock_foundry_client(project_client=project_client) + + toolbox = await client.get_toolbox("research_tools", version="v1") + + agent = Agent( + client=client, + instructions="You are a test agent.", + tools=toolbox.tools, + ) + + agent_tools = agent.default_options["tools"] + assert len(agent_tools) == 1 + assert agent_tools[0]["type"] == "code_interpreter" + + +async def test_multiple_toolbox_tool_lists_can_be_combined_in_agent() -> None: + """Nested toolbox ``.tools`` lists flatten into one tool list on Agent construction.""" + from agent_framework import Agent + + project_client = MagicMock() + project_client.get_openai_client = MagicMock(return_value=MagicMock()) + client = _make_mock_foundry_client(project_client=project_client) + + toolbox_a = _make_version_object(name="research_tools", version="v1", tools=[_make_code_interpreter()]) + toolbox_b = _make_version_object(name="some_other_tools", version="v3", tools=[_make_code_interpreter()]) + + agent = Agent( + client=client, + instructions="You are a test agent.", + tools=[toolbox_a.tools, toolbox_b.tools], + ) + + agent_tools = agent.default_options["tools"] + assert len(agent_tools) == 2 + assert agent_tools[0]["type"] == "code_interpreter" + assert agent_tools[1]["type"] == "code_interpreter" + + +# --------------------------------------------------------------------------- # +# toolbox tool selection helpers # +# --------------------------------------------------------------------------- # + + +def test_get_toolbox_tool_name_prefers_server_label_then_name_then_type() -> None: + from azure.ai.projects.models import MCPTool as FoundryMCPTool + + from agent_framework_foundry import get_toolbox_tool_name + + mcp_tool = FoundryMCPTool( + server_label="githubmcp", + server_url="https://api.githubcopilot.com/mcp", + ) + assert get_toolbox_tool_name(mcp_tool) == "githubmcp" + + named_tool = {"type": "code_interpreter", "name": "ci_tool"} + assert get_toolbox_tool_name(named_tool) == "ci_tool" + + unnamed_tool = {"type": "web_search"} + assert get_toolbox_tool_name(unnamed_tool) == "web_search" + + +def test_select_toolbox_tools_filters_by_names() -> None: + from azure.ai.projects.models import MCPTool as FoundryMCPTool + + from agent_framework_foundry import select_toolbox_tools + + tools: list[Tool | dict[str, Any]] = [ + FoundryMCPTool(server_label="githubmcp", server_url="https://api.githubcopilot.com/mcp"), + {"type": "code_interpreter", "name": "python_runner"}, + {"type": "web_search"}, + ] + + selected = select_toolbox_tools(tools, include_names=["githubmcp", "python_runner"]) + + assert len(selected) == 2 + assert selected[0] is tools[0] + assert selected[1] is tools[1] + + +def test_select_toolbox_tools_filters_by_typed_tool_types() -> None: + from agent_framework_foundry import select_toolbox_tools + + tools: list[Tool | dict[str, Any]] = [ + {"type": "mcp", "server_label": "githubmcp"}, + {"type": "code_interpreter", "name": "python_runner"}, + {"type": "web_search"}, + ] + + selected = select_toolbox_tools(tools, include_types=["mcp", "code_interpreter"]) + + assert len(selected) == 2 + assert selected[0]["type"] == "mcp" + assert selected[1]["type"] == "code_interpreter" + + +def test_select_toolbox_tools_accepts_toolbox_object_directly() -> None: + from agent_framework_foundry import select_toolbox_tools + + toolbox = _make_version_object( + name="research_tools", + version="v1", + tools=[ + {"type": "mcp", "server_label": "githubmcp"}, # type: ignore[list-item] + {"type": "code_interpreter", "name": "python_runner"}, # type: ignore[list-item] + {"type": "web_search"}, # type: ignore[list-item] + ], + ) + + selected = select_toolbox_tools(toolbox, include_types=["mcp", "code_interpreter"]) + + assert len(selected) == 2 + assert selected[0]["type"] == "mcp" + assert selected[1]["type"] == "code_interpreter" + + +async def test_fetched_toolbox_can_be_combined_with_function_tool() -> None: + from agent_framework import Agent, FunctionTool, tool + + project_client = MagicMock() + version_obj = _make_version_object(name="research_tools", version="v1", tools=[_make_code_interpreter()]) + project_client.beta.toolboxes.get_version = AsyncMock(return_value=version_obj) + + client = _make_mock_foundry_client(project_client=project_client) + toolbox = await client.get_toolbox("research_tools", version="v1") + + @tool(name="local_lookup", description="A local helper tool") + def local_lookup(query: str) -> str: + return query + + agent = Agent( + client=client, + instructions="You are a test agent.", + tools=[toolbox, local_lookup], + ) + + agent_tools = agent.default_options["tools"] + assert len(agent_tools) == 2 + assert agent_tools[0]["type"] == "code_interpreter" + assert isinstance(agent_tools[1], FunctionTool) + assert agent_tools[1].name == "local_lookup" + + +def test_select_toolbox_tools_supports_excludes_and_predicate() -> None: + from agent_framework_foundry import select_toolbox_tools + + tools: list[Tool | dict[str, Any]] = [ + {"type": "mcp", "server_label": "githubmcp"}, + {"type": "mcp", "server_label": "learnmcp"}, + {"type": "web_search"}, + ] + + selected = select_toolbox_tools( + tools, + exclude_names=["learnmcp"], + predicate=lambda tool: tool.get("type") == "mcp", # type: ignore[union-attr] + ) + + assert len(selected) == 1 + assert selected[0]["server_label"] == "githubmcp" + + +async def test_selected_toolbox_subset_can_be_combined_with_function_tool() -> None: + from agent_framework import Agent, FunctionTool, tool + + from agent_framework_foundry import select_toolbox_tools + + project_client = MagicMock() + version_obj = _make_version_object( + name="research_tools", + version="v1", + tools=[ + {"type": "mcp", "server_label": "githubmcp"}, # type: ignore[list-item] + {"type": "code_interpreter", "name": "python_runner"}, # type: ignore[list-item] + {"type": "web_search"}, # type: ignore[list-item] + ], + ) + project_client.beta.toolboxes.get_version = AsyncMock(return_value=version_obj) + + client = _make_mock_foundry_client(project_client=project_client) + toolbox = await client.get_toolbox("research_tools", version="v1") + selected_tools = select_toolbox_tools(toolbox, include_types=["mcp", "code_interpreter"]) + + @tool(name="local_lookup", description="A local helper tool") + def local_lookup(query: str) -> str: + return query + + agent = Agent( + client=client, + instructions="You are a test agent.", + tools=[selected_tools, local_lookup], + ) + + agent_tools = agent.default_options["tools"] + assert len(agent_tools) == 3 + assert agent_tools[0]["type"] == "mcp" + assert agent_tools[1]["type"] == "code_interpreter" + assert isinstance(agent_tools[2], FunctionTool) + assert agent_tools[2].name == "local_lookup" + + +# --------------------------------------------------------------------------- # +# Integration # +# --------------------------------------------------------------------------- # + + +skip_if_foundry_integration_tests_disabled = pytest.mark.skipif( + os.getenv("FOUNDRY_PROJECT_ENDPOINT", "") in ("", "https://test-project.services.ai.azure.com/") + or os.getenv("FOUNDRY_MODEL", "") == "", + reason="No real FOUNDRY_PROJECT_ENDPOINT or FOUNDRY_MODEL provided; skipping integration tests.", +) + + +@pytest.mark.flaky +@pytest.mark.integration +@skip_if_foundry_integration_tests_disabled +async def test_integration_get_toolbox_round_trip_against_real_project() -> None: + """Create a toolbox via the raw SDK, fetch via FoundryChatClient, then delete. + + Self-contained to avoid depending on toolboxes that may be cleaned up + externally. Exercises both the default-version resolution path + (``get`` + ``get_version``) and the explicit-version path. + """ + from uuid import uuid4 + + from agent_framework import Agent + + from agent_framework_foundry import FoundryChatClient + + client = FoundryChatClient(credential=AzureCliCredential()) + project_client = client.project_client + + toolbox_name = f"af-int-toolbox-{uuid4().hex[:12]}" + created = await project_client.beta.toolboxes.create_version( + name=toolbox_name, + tools=[CodeInterpreterTool()], + description=f"{toolbox_name} integration test", + ) + assert isinstance(created, ToolboxVersionObject) + try: + toolbox_default = await client.get_toolbox(toolbox_name) + assert toolbox_default.name == toolbox_name + assert toolbox_default.tools, "Default-version fetch returned no tools" + + toolbox_pinned = await client.get_toolbox(toolbox_name, version=created.version) + assert toolbox_pinned.version == created.version + assert toolbox_pinned.tools + + agent = Agent( + client=client, + instructions="You are a test agent.", + tools=toolbox_pinned.tools, + ) + assert len(agent.default_options["tools"]) == len(toolbox_pinned.tools) + finally: + await project_client.beta.toolboxes.delete(toolbox_name) diff --git a/python/samples/02-agents/context_providers/README.md b/python/samples/02-agents/context_providers/README.md index 04f3a1395f..7c34e10518 100644 --- a/python/samples/02-agents/context_providers/README.md +++ b/python/samples/02-agents/context_providers/README.md @@ -7,6 +7,7 @@ These samples demonstrate how to use context providers to enrich agent conversat | File / Folder | Description | |---------------|-------------| | [`simple_context_provider.py`](simple_context_provider.py) | Implement a custom context provider by extending `ContextProvider` to extract and inject structured user information across turns. | +| [`foundry_toolbox_context_provider.py`](foundry_toolbox_context_provider.py) | Compose a Microsoft Foundry toolbox with a `ContextProvider` that caches the toolbox once and picks a subset of its tools per-turn via `select_toolbox_tools`, driven by keywords in the latest user message. | | [`azure_ai_foundry_memory.py`](azure_ai_foundry_memory.py) | Use `FoundryMemoryProvider` to add semantic memory — automatically retrieves, searches, and stores memories via Azure AI Foundry. | | [`azure_ai_search/`](azure_ai_search/) | Retrieval Augmented Generation (RAG) with Azure AI Search in semantic and agentic modes. See its own [README](azure_ai_search/README.md). | | [`mem0/`](mem0/) | Memory-powered context using the Mem0 integration (open-source and managed). See its own [README](mem0/README.md). | @@ -19,6 +20,12 @@ These samples demonstrate how to use context providers to enrich agent conversat - `FOUNDRY_MODEL`: Model deployment name - Azure CLI authentication (`az login`) +**For `foundry_toolbox_context_provider.py`:** +- `FOUNDRY_PROJECT_ENDPOINT`: Your Microsoft Foundry project endpoint +- `FOUNDRY_MODEL`: Model deployment name +- A toolbox already configured in that project; set `TOOLBOX_NAME` / `TOOLBOX_VERSION` at the top of the sample +- Azure CLI authentication (`az login`) + **For `azure_ai_foundry_memory.py`:** - `FOUNDRY_PROJECT_ENDPOINT`: Your Azure AI Foundry project endpoint - `FOUNDRY_MODEL`: Chat/responses model deployment name diff --git a/python/samples/02-agents/context_providers/foundry_toolbox_context_provider.py b/python/samples/02-agents/context_providers/foundry_toolbox_context_provider.py new file mode 100644 index 0000000000..d889c7c1ac --- /dev/null +++ b/python/samples/02-agents/context_providers/foundry_toolbox_context_provider.py @@ -0,0 +1,207 @@ +# Copyright (c) Microsoft. All rights reserved. + +import asyncio +import os +from typing import Any + +from agent_framework import Agent, AgentSession, ContextProvider, Message, SessionContext +from agent_framework.foundry import ( + FoundryChatClient, + get_toolbox_tool_name, + get_toolbox_tool_type, + select_toolbox_tools, +) +from azure.identity import AzureCliCredential +from dotenv import load_dotenv +from pydantic import BaseModel + +# Load environment variables from .env file +load_dotenv() + +""" +Foundry Toolbox + Context Provider Example + +This sample composes a Foundry toolbox with a ContextProvider so the agent's +tool list is chosen dynamically per-turn. It uses the chat client itself as a lightweight "tool router": the +latest user message plus a short menu of toolbox tools is sent to the model +with a Pydantic ``response_format``, and the returned tool names drive +``select_toolbox_tools``. The toolbox is fetched once and cached on the +provider's state dict; subsequent turns reuse the cache. + +Prerequisites: +- A Microsoft Foundry project +- A toolbox already configured in that project (set TOOLBOX_NAME below) +- FOUNDRY_PROJECT_ENDPOINT and FOUNDRY_MODEL environment variables set +- Azure CLI authentication (`az login`) +""" + +# Replace with your own Foundry toolbox name and version. +TOOLBOX_NAME = "research_toolbox" +# Set to None to resolve the toolbox's current default version at fetch time. +TOOLBOX_VERSION: str | None = None + +# Generic queries that exercise the router without assuming any specific tool +# types are configured. The first is introspective, the second forces a +# non-empty pick for whichever tools the toolbox actually contains, and the +# third should route to nothing. +QUERIES: list[str] = [ + "Introduce yourself and briefly describe the tools you can use to help me.", + "Pick the tool you think is most useful and demonstrate it with a short example.", + "Say hi in one short sentence - no tools needed.", +] + + +def create_sample_toolbox(name: str) -> str: + """Create (or replace) a toolbox version in the Foundry project. + + Toolboxes are normally configured in the Foundry portal or a deployment + script, not the application itself. This helper exists so the sample can + be run end-to-end without first setting a toolbox up by hand — delete any + existing toolbox under ``name``, then create a fresh version containing a + single MCP tool. Returns the created version identifier. + """ + from azure.ai.projects import AIProjectClient + from azure.ai.projects.models import MCPTool, Tool + from azure.core.exceptions import ResourceNotFoundError + + with ( + AzureCliCredential() as credential, + AIProjectClient(credential=credential, endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"]) as project_client, + ): + try: + project_client.beta.toolboxes.delete(name) + print(f"Toolbox `{name}` deleted") + except ResourceNotFoundError: + pass + + tools: list[Tool] = [ + MCPTool( + server_label="api_specs", + server_url="https://gitmcp.io/Azure/azure-rest-api-specs", + require_approval="never", + ) + ] + + created = project_client.beta.toolboxes.create_version( + name=name, + description="Toolbox version with MCP require_approval set to 'never'.", + tools=tools, + ) + print(f"Created toolbox {created.name}@{created.version} ({len(created.tools)} tool(s))") + return created.version + + +class ToolSelection(BaseModel): + """Structured output for the per-turn tool router.""" + + tool_names: list[str] + + +ROUTER_INSTRUCTIONS = ( + "You are a tool router. Given the user's latest message and a menu of " + "available tools (one per line, formatted as 'NAME - TYPE'), return the " + "NAMES of the tools that would plausibly help answer the message. Return " + "an empty list if no tool is needed." +) + + +class DynamicToolboxProvider(ContextProvider): + """Fetches a Foundry toolbox once and lets the model pick tools per-turn.""" + + DEFAULT_SOURCE_ID = "foundry_toolbox" + + def __init__( + self, + source_id: str = DEFAULT_SOURCE_ID, + *, + client: FoundryChatClient, + toolbox_name: str, + toolbox_version: str | None = None, + ) -> None: + super().__init__(source_id) + self._client = client + self._toolbox_name = toolbox_name + self._toolbox_version = toolbox_version + + async def before_run( + self, + *, + agent: Any, + session: AgentSession | None, + context: SessionContext, + state: dict[str, Any], + ) -> None: + """Cache the toolbox on first call, then let the model pick tools per-turn.""" + toolbox = state.get("toolbox") + if toolbox is None: + toolbox = await self._client.get_toolbox(self._toolbox_name, version=self._toolbox_version) + state["toolbox"] = toolbox + print(f"[{self.source_id}] Loaded toolbox {toolbox.name}@{toolbox.version} ({len(toolbox.tools)} tool(s))") + + user_messages = [m for m in context.get_messages(include_input=True) if getattr(m, "role", None) == "user"] + if not user_messages: + context.extend_tools(self.source_id, list(toolbox.tools)) + return + + picks = await self._route_tools(user_messages[-1].text, toolbox.tools) + if picks: + tools = select_toolbox_tools(toolbox, include_names=picks) + print(f"[{self.source_id}] Router picked {sorted(picks)} - surfacing {len(tools)} tool(s)") + else: + tools = list(toolbox.tools) + print(f"[{self.source_id}] Router picked nothing - surfacing all {len(tools)} tool(s)") + context.extend_tools(self.source_id, tools) + + async def _route_tools(self, user_text: str, tools: Any) -> list[str]: + """Ask the model which toolbox tools to surface for this turn.""" + menu = "\n".join(f"- {get_toolbox_tool_name(t)} - {get_toolbox_tool_type(t)}" for t in tools) + prompt = ( + f"User message:\n{user_text}\n\n" + f"Available tools:\n{menu}\n\n" + "Return the names of tools that should be surfaced for this turn." + ) + response = await self._client.get_response( + messages=[Message("user", [prompt])], + options={ + "instructions": ROUTER_INSTRUCTIONS, + "response_format": ToolSelection, + }, + ) + selection: ToolSelection = response.value # type: ignore + return selection.tool_names + + +async def main() -> None: + client = FoundryChatClient( + project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"], + model=os.environ["FOUNDRY_MODEL"], + credential=AzureCliCredential(), + ) + + # Comment out if the toolbox already exists in your Foundry project. + create_sample_toolbox(TOOLBOX_NAME) + + toolbox_provider = DynamicToolboxProvider( + client=client, + toolbox_name=TOOLBOX_NAME, + toolbox_version=TOOLBOX_VERSION, + ) + + async with Agent( + client=client, + instructions=( + "You are a helpful assistant. Use the tools available to you on each " + "turn to answer the user. If no tools are relevant, reply directly." + ), + context_providers=[toolbox_provider], + ) as agent: + session = agent.create_session() + + for query in QUERIES: + print(f"\nUser: {query}") + result = await agent.run(query, session=session) + print(f"Assistant: {result}") + + +if __name__ == "__main__": + asyncio.run(main()) diff --git a/python/samples/02-agents/providers/foundry/README.md b/python/samples/02-agents/providers/foundry/README.md index 2025b0f4fe..120c4d9a1c 100644 --- a/python/samples/02-agents/providers/foundry/README.md +++ b/python/samples/02-agents/providers/foundry/README.md @@ -26,6 +26,8 @@ This folder contains Azure AI Foundry and Foundry Local samples for Agent Framew | [`foundry_chat_client_with_hosted_mcp.py`](foundry_chat_client_with_hosted_mcp.py) | Foundry Chat Client with hosted MCP | | [`foundry_chat_client_with_local_mcp.py`](foundry_chat_client_with_local_mcp.py) | Foundry Chat Client with local MCP | | [`foundry_chat_client_with_session.py`](foundry_chat_client_with_session.py) | Foundry Chat Client with session management | +| [`foundry_chat_client_with_toolbox.py`](foundry_chat_client_with_toolbox.py) | Foundry Chat Client with Foundry toolbox loading and multi-toolbox composition | +| [`foundry_chat_client_with_toolbox_mcp.py`](foundry_chat_client_with_toolbox_mcp.py) | Foundry Chat Client connected to a toolbox via its MCP endpoint using `MCPStreamableHTTPTool` | ## FoundryLocalClient Samples diff --git a/python/samples/02-agents/providers/foundry/foundry_chat_client_with_toolbox.py b/python/samples/02-agents/providers/foundry/foundry_chat_client_with_toolbox.py new file mode 100644 index 0000000000..8a532331ae --- /dev/null +++ b/python/samples/02-agents/providers/foundry/foundry_chat_client_with_toolbox.py @@ -0,0 +1,174 @@ +# Copyright (c) Microsoft. All rights reserved. + +import asyncio +import os + +from agent_framework import Agent +from agent_framework.foundry import FoundryChatClient, select_toolbox_tools +from azure.identity import AzureCliCredential +from dotenv import load_dotenv + +# Load environment variables from .env file +load_dotenv() + +""" +Foundry Chat Client with Toolbox Example + +This sample demonstrates loading a named, versioned Foundry toolbox into an +Agent via ``FoundryChatClient.get_toolbox()``. A toolbox is a server-side +bundle of tool configurations (code interpreter, file search, MCP, web search, +etc.) configured in the Foundry portal or via the raw SDK. + +Prerequisites: +- A Microsoft Foundry project +- A toolbox already configured in that project (set TOOLBOX_NAME below) +- FOUNDRY_PROJECT_ENDPOINT and FOUNDRY_MODEL environment variables set +""" + +# Replace with your own Foundry toolbox name and version. +TOOLBOX_NAME = "research_toolbox" +TOOLBOX_VERSION = "1" +# Used only by combine_toolboxes() — swap in a second toolbox you own. +SECOND_TOOLBOX_NAME = "analysis_toolbox" +SECOND_TOOLBOX_VERSION = "1" + +# Replace with any question that exercises the tools configured in your toolbox. +QUERY = "Introduce yourself and briefly describe the tools you can use to help me." + + +def create_sample_toolbox(name: str) -> str: + """Create (or replace) a toolbox version in the Foundry project. + + Toolboxes are normally configured in the Foundry portal or a deployment + script, not the application itself. This helper exists so the samples can + be run end-to-end without first setting a toolbox up by hand — delete any + existing toolbox under ``name``, then create a fresh version containing a + single MCP tool. Returns the created version identifier. + """ + from azure.ai.projects import AIProjectClient + from azure.ai.projects.models import MCPTool, Tool + from azure.core.exceptions import ResourceNotFoundError + + with ( + AzureCliCredential() as credential, + AIProjectClient(credential=credential, endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"]) as project_client, + ): + try: + project_client.beta.toolboxes.delete(name) + print(f"Toolbox `{name}` deleted") + except ResourceNotFoundError: + pass + + tools: list[Tool] = [ + MCPTool( + server_label="api_specs", + server_url="https://gitmcp.io/Azure/azure-rest-api-specs", + require_approval="never", + ) + ] + + created = project_client.beta.toolboxes.create_version( + name=name, + description="Toolbox version with MCP require_approval set to 'never'.", + tools=tools, + ) + print(f"Created toolbox {created.name}@{created.version} ({len(created.tools)} tool(s))") + return created.version + + +async def main() -> None: + """Example showing how to use a single Foundry toolbox with FoundryChatClient.""" + print("=== Foundry Chat Client with Toolbox Example ===") + + # For authentication, run `az login` in your terminal or replace + # AzureCliCredential with your preferred authentication option. + client = FoundryChatClient( + credential=AzureCliCredential(), + project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"], + model=os.environ["FOUNDRY_MODEL"], + ) + + # Comment out if the toolbox already exists in your Foundry project. + create_sample_toolbox(TOOLBOX_NAME) + + # Omit ``version`` to resolve the toolbox's current default version at runtime. + toolbox = await client.get_toolbox(TOOLBOX_NAME) + print(f"Loaded toolbox {toolbox.name}@{toolbox.version} ({len(toolbox.tools)} tool(s))") + + agent = Agent( + client=client, + instructions="You are a research assistant. Use the available tools to answer questions.", + tools=toolbox, + ) + + print(f"User: {QUERY}") + result = await agent.run(QUERY) + print(f"Result: {result}\n") + + +async def combine_toolboxes() -> None: + """Alternative flow: combine the tools from multiple Foundry toolboxes.""" + client = FoundryChatClient( + credential=AzureCliCredential(), + project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"], + model=os.environ["FOUNDRY_MODEL"], + ) + + # Comment out if the toolboxes already exist in your Foundry project. + create_sample_toolbox(TOOLBOX_NAME) + create_sample_toolbox(SECOND_TOOLBOX_NAME) + + toolbox_a = await client.get_toolbox(TOOLBOX_NAME, version=TOOLBOX_VERSION) + toolbox_b = await client.get_toolbox(SECOND_TOOLBOX_NAME, version=SECOND_TOOLBOX_VERSION) + print( + "Loaded toolboxes: " + f"{toolbox_a.name}@{toolbox_a.version} ({len(toolbox_a.tools)} tool(s)), " + f"{toolbox_b.name}@{toolbox_b.version} ({len(toolbox_b.tools)} tool(s))" + ) + + agent = Agent( + client=client, + instructions="You are a research assistant. Use all available tools to answer questions.", + tools=[toolbox_a, toolbox_b], + ) + + print(f"User: {QUERY}") + result = await agent.run(QUERY) + print(f"Combined-toolbox result: {result}\n") + + +async def select_tools_from_toolbox() -> None: + """Alternative flow: keep only a subset of toolbox tools before agent creation.""" + client = FoundryChatClient( + credential=AzureCliCredential(), + project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"], + model=os.environ["FOUNDRY_MODEL"], + ) + + # Comment out if the toolbox already exists in your Foundry project. + create_sample_toolbox(TOOLBOX_NAME) + + toolbox = await client.get_toolbox(TOOLBOX_NAME, version=TOOLBOX_VERSION) + print(f"Loaded toolbox {toolbox.name}@{toolbox.version} ({len(toolbox.tools)} tool(s))") + + selected_tools = select_toolbox_tools( + toolbox, + include_types=["code_interpreter", "mcp"], + ) + print(f"Selected {len(selected_tools)} toolbox tools for the agent") + + agent = Agent( + client=client, + instructions="You are a research assistant. Use only the selected toolbox tools.", + tools=selected_tools, + ) + + print(f"User: {QUERY}") + result = await agent.run(QUERY) + print(f"Selected-toolbox result: {result}\n") + + +if __name__ == "__main__": + asyncio.run(main()) + # asyncio.run(combine_toolboxes()) + # asyncio.run(select_tools_from_toolbox()) diff --git a/python/samples/02-agents/providers/foundry/foundry_chat_client_with_toolbox_mcp.py b/python/samples/02-agents/providers/foundry/foundry_chat_client_with_toolbox_mcp.py new file mode 100644 index 0000000000..1fbfe20a9a --- /dev/null +++ b/python/samples/02-agents/providers/foundry/foundry_chat_client_with_toolbox_mcp.py @@ -0,0 +1,118 @@ +# Copyright (c) Microsoft. All rights reserved. + +import asyncio +import os +from collections.abc import Callable +from typing import Any + +from agent_framework import Agent, MCPStreamableHTTPTool +from agent_framework.foundry import FoundryChatClient +from azure.core.credentials import TokenCredential +from azure.identity import AzureCliCredential, DefaultAzureCredential, get_bearer_token_provider +from dotenv import load_dotenv + +# Load environment variables from .env file +load_dotenv() + +""" +Foundry Toolbox via MAF ``MCPStreamableHTTPTool`` + +Instead of fetching the toolbox and fanning out individual tool specs, point +MAF's ``MCPStreamableHTTPTool`` at the toolbox's MCP endpoint. The agent +discovers and calls the toolbox's tools over MCP at runtime. + +Prerequisites: +- A Microsoft Foundry project with a toolbox configured +- FOUNDRY_PROJECT_ENDPOINT and FOUNDRY_MODEL environment variables set +- FOUNDRY_TOOLBOX_ENDPOINT: the toolbox's MCP endpoint URL, e.g. + ``https://.services.ai.azure.com/api/projects//toolsets//mcp?api-version=v1`` +- Azure CLI authentication (``az login``) +""" + +# Must match the ```` segment of FOUNDRY_TOOLBOX_ENDPOINT. +TOOLBOX_NAME = "research_toolbox" + + +def create_sample_toolbox(name: str) -> str: + """Create (or replace) a toolbox version in the Foundry project. + + Toolboxes are normally configured in the Foundry portal or a deployment + script, not the application itself. This helper exists so the sample can + be run end-to-end without first setting a toolbox up by hand — delete any + existing toolbox under ``name``, then create a fresh version containing a + single MCP tool. Returns the created version identifier. + """ + from azure.ai.projects import AIProjectClient + from azure.ai.projects.models import MCPTool, Tool + from azure.core.exceptions import ResourceNotFoundError + + with ( + AzureCliCredential() as credential, + AIProjectClient(credential=credential, endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"]) as project_client, + ): + try: + project_client.beta.toolboxes.delete(name) + print(f"Toolbox `{name}` deleted") + except ResourceNotFoundError: + pass + + tools: list[Tool] = [ + MCPTool( + server_label="api_specs", + server_url="https://gitmcp.io/Azure/azure-rest-api-specs", + require_approval="never", + ) + ] + + created = project_client.beta.toolboxes.create_version( + name=name, + description="Toolbox version with MCP require_approval set to 'never'.", + tools=tools, + ) + print(f"Created toolbox {created.name}@{created.version} ({len(created.tools)} tool(s))") + return created.version + + +def make_toolbox_header_provider(credential: TokenCredential) -> Callable[[dict[str, Any]], dict[str, str]]: + """Build a header_provider that injects a fresh Azure AI bearer token on every MCP request.""" + get_token = get_bearer_token_provider(credential, "https://ai.azure.com/.default") + + def provide(_kwargs: dict[str, Any]) -> dict[str, str]: + return { + "Authorization": f"Bearer {get_token()}", + } + + return provide + + +async def main() -> None: + credential = DefaultAzureCredential() + + # Comment out if the toolbox already exists in your Foundry project. + create_sample_toolbox(TOOLBOX_NAME) + + toolbox_tool = MCPStreamableHTTPTool( + name="foundry_toolbox", + description="Tools exposed by the configured Foundry toolbox", + url=os.environ["FOUNDRY_TOOLBOX_ENDPOINT"], + header_provider=make_toolbox_header_provider(credential), + load_prompts=False, + ) + + async with Agent( + client=FoundryChatClient( + project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"], + model=os.environ["FOUNDRY_MODEL"], + credential=credential, + ), + instructions="You are a helpful assistant. Use the available toolbox tools to answer the user.", + tools=toolbox_tool, + ) as agent: + query = "What tools do you have access to?" + print(f"User: {query}") + result = await agent.run(query) + print(f"Assistant: {result}") + + +if __name__ == "__main__": + asyncio.run(main()) diff --git a/python/uv.lock b/python/uv.lock index 370fd7e46d..92fc51361d 100644 --- a/python/uv.lock +++ b/python/uv.lock @@ -496,7 +496,7 @@ requires-dist = [ { name = "agent-framework-core", editable = "packages/core" }, { name = "agent-framework-openai", editable = "packages/openai" }, { name = "azure-ai-inference", specifier = ">=1.0.0b9,<1.0.0b10" }, - { name = "azure-ai-projects", specifier = ">=2.0.0,<3.0" }, + { name = "azure-ai-projects", specifier = ">=2.1.0,<3.0" }, ] [[package]] @@ -1048,7 +1048,7 @@ wheels = [ [[package]] name = "azure-ai-projects" -version = "2.0.1" +version = "2.1.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "azure-core", marker = "sys_platform == 'darwin' or sys_platform == 'linux' or sys_platform == 'win32'" }, @@ -1058,9 +1058,9 @@ dependencies = [ { name = "openai", marker = "sys_platform == 'darwin' or sys_platform == 'linux' or sys_platform == 'win32'" }, { name = "typing-extensions", marker = "sys_platform == 'darwin' or sys_platform == 'linux' or sys_platform == 'win32'" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/86/f9/a15c8a16e35e6d620faebabc6cc4f9e2f4b7f1d962cc6f58931c46947e24/azure_ai_projects-2.0.1.tar.gz", hash = "sha256:c8c64870aa6b89903af69a4ff28b4eff3df9744f14615ea572cae87394946a0c", size = 491774, upload-time = "2026-03-12T19:59:02.712Z" } +sdist = { url = "https://files.pythonhosted.org/packages/72/76/3fdede8eddfe5927a571898a15f0288ba30fee78e5ba099f88df3ded70af/azure_ai_projects-2.1.0.tar.gz", hash = "sha256:f0749fa9a174255aa1a5550fb6078208521518472907a4c6dd552767d9b39caa", size = 543343, upload-time = "2026-04-20T17:06:48.751Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/8d/f7/290ca39501c06c6e23b46ba9f7f3dfb05ecc928cde105fed85d6845060dd/azure_ai_projects-2.0.1-py3-none-any.whl", hash = "sha256:dfda540d256e67a52bf81c75418b6bf92b811b96693fe45787e154a888ad2396", size = 236560, upload-time = "2026-03-12T19:59:04.249Z" }, + { url = "https://files.pythonhosted.org/packages/f7/f6/4984e7772a97c7a9e6505a3de8e55a5070fa2b02cd7e980da91e0d9b9b97/azure_ai_projects-2.1.0-py3-none-any.whl", hash = "sha256:6f259d8eb9167d2dfd372006d0221a8118faeaeb05829fa898b595bc6f19c699", size = 274309, upload-time = "2026-04-20T17:06:50.542Z" }, ] [[package]]