Python: Remove bespoke Foundry toolbox helpers; standardize on MCP for toolbox consumption (#5671)

* Remove Foundry toolbox helpers; standardize on MCP for toolbox consumption

- Remove RawFoundryChatClient.get_toolbox() and its fetch_toolbox import
- Remove fetch_toolbox, select_toolbox_tools, get_toolbox_tool_name,
  get_toolbox_tool_type, FoundryHostedToolType, ToolboxToolSelectionInput
  from agent_framework_foundry._tools
- Remove ExperimentalFeature.TOOLBOXES from _feature_stage.py (no consumers)
- Drop toolbox re-exports from agent_framework_foundry/__init__.py and
  agent_framework.foundry namespace
- Update _sanitize_foundry_response_tool docstring to remove toolbox framing;
  sanitization logic itself is unchanged
- Update _agent.py docstring: 'toolbox-fetched MCP' → 'hosted MCP'
- Delete tests/test_toolbox.py (all tests covered removed helpers)
- Update test_foundry_chat_client.py: rename/redoc tests that mentioned
  toolbox but test sanitization that remains
- Delete foundry_chat_client_with_toolbox.py (bespoke toolbox API sample)
- Delete foundry_toolbox_context_provider.py (relied on select_toolbox_tools)
- Rename foundry_chat_client_with_toolbox_mcp.py →
  foundry_chat_client_with_toolbox.py (canonical MCP pattern)
- Rewrite 04_foundry_toolbox/main.py to use MCPStreamableHTTPTool
- Update provider/README, context_providers/README, 04_foundry_toolbox/README

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(samples): update 06_files sample to consume toolbox via MCP (#5670)

Replace removed get_toolbox/select_toolbox_tools APIs with
MCPStreamableHTTPTool, using allowed_tools=["code_interpreter"] to
select only the code interpreter from the toolbox endpoint.

Update .env.example and README to use FOUNDRY_TOOLBOX_ENDPOINT
instead of TOOLBOX_NAME.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(foundry): remove non-existent toolbox helper APIs from README (#5670)

Remove the 'fetch, optionally filter, and pass tools directly' pattern
from the FoundryChatClient toolbox documentation, as select_toolbox_tools
and get_toolbox were removed. Only the MCP endpoint pattern is documented.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(foundry): remove residual toolbox docstring references and reproduction report

Remove REPRODUCTION_REPORT.md (workflow artifact that should not be committed),
and update two remaining docstring references that still said 'toolbox reads'
/'toolbox definition' after the toolbox helpers were removed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Remove bespoke Foundry toolbox helpers; standardize on MCP for toolbox consumption

Fixes #5670

* fix(#5670): resolve toolbox endpoint from TOOLBOX_NAME fallback; add namespace regression tests

- Add _resolve_toolbox_endpoint() helper in 04_foundry_toolbox/main.py and
  06_files/main.py that prefers FOUNDRY_TOOLBOX_ENDPOINT but falls back to
  deriving the MCP URL from FOUNDRY_PROJECT_ENDPOINT + TOOLBOX_NAME — fixing
  the startup KeyError when agents are deployed via azd provision (which injects
  TOOLBOX_NAME, not FOUNDRY_TOOLBOX_ENDPOINT).
- Update 04_foundry_toolbox/.env.example to use FOUNDRY_TOOLBOX_ENDPOINT
  (consistent with 06_files).
- Add TOOLBOX_NAME env var to 06_files/agent.yaml so deployed agents have it
  available for the fallback derivation.
- Update both READMEs to document the two ways to supply the toolbox endpoint.
- Add test_foundry_namespace_no_longer_exposes_toolbox_helpers() with negative
  assertions for FoundryHostedToolType, get_toolbox_tool_name,
  get_toolbox_tool_type, and select_toolbox_tools — guarding against accidental
  re-introduction of removed symbols.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(samples): fail fast on empty FOUNDRY_TOOLBOX_ENDPOINT; add unit tests

Addresses review feedback for #5670:

- In _resolve_toolbox_endpoint() (04_foundry_toolbox/main.py and
  06_files/main.py) change the walrus-operator check from a truthy
  test to an explicit 'is not None' guard.  An explicitly set empty
  string now raises ValueError immediately with a clear message
  instead of silently falling through to the fallback URL
  construction.

- Add tests/samples/hosting/test_toolbox_endpoint.py covering both
  sample modules:
    (a) FOUNDRY_TOOLBOX_ENDPOINT set → returned as-is
    (b) FOUNDRY_TOOLBOX_ENDPOINT set to empty string → ValueError
    (c) fallback constructs URL from FOUNDRY_PROJECT_ENDPOINT + TOOLBOX_NAME,
        stripping trailing slashes
    (d) neither variable group set → KeyError

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address review feedback: remove extraneous test and docstring content

- Remove test_foundry_namespace_no_longer_exposes_toolbox_helpers (no longer warranted)
- Remove docstring from _agent.py _prepare_tools_for_openai (extraneous)
- Trim _chat_client.py _prepare_tools_for_openai docstring to one-liner (toolbox references no longer relevant)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: remove remaining extraneous docstring from RawFoundryChatClient._prepare_tools_for_openai

Address review comment on PR #5671: reviewer noted the description
isn't warranted now that toolbox helpers have been removed. Matches
the pattern in RawFoundryAgentChatClient which has no docstring.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
Evan Mattson
2026-05-07 08:56:16 +09:00
committed by GitHub
Unverified
parent 51ad460d5f
commit e56e6dad4d
26 changed files with 314 additions and 1161 deletions
@@ -641,8 +641,8 @@ def test_prepare_tools_for_openai_strips_extraneous_name_from_foundry_mcp_tool()
assert "name" not in prepared
def test_prepare_tools_for_openai_strips_read_model_fields_from_toolbox_code_interpreter() -> None:
"""Toolbox-returned code interpreter tools may carry read-model-only name/description."""
def test_prepare_tools_for_openai_strips_read_model_fields_from_hosted_code_interpreter() -> None:
"""Hosted code interpreter tools may carry read-model-only name/description."""
project_client = MagicMock()
project_client.get_openai_client.return_value = _make_mock_openai_client()
client = FoundryChatClient(project_client=project_client, model="test-model")
@@ -650,7 +650,7 @@ def test_prepare_tools_for_openai_strips_read_model_fields_from_toolbox_code_int
tool = {
"type": "code_interpreter",
"name": "code_interpreter_t6bbtm",
"description": "Toolbox read model description",
"description": "Hosted tool read model description",
"container": {"file_ids": [], "type": "auto"},
}
@@ -665,7 +665,7 @@ def test_prepare_tools_for_openai_strips_read_model_fields_from_toolbox_code_int
def test_prepare_tools_for_openai_injects_default_container_for_code_interpreter_dict() -> None:
"""Toolbox-returned code_interpreter without a container must get a default injected.
"""Hosted code_interpreter without a container must get a default injected.
The Azure SDK treats ``container`` as optional, but the Responses API rejects
``code_interpreter`` entries without one. The sanitizer backfills ``{"type": "auto"}``.
@@ -691,7 +691,7 @@ def test_prepare_tools_for_openai_injects_default_container_for_code_interpreter
def test_prepare_tools_for_openai_injects_default_container_for_code_interpreter_sdk_instance() -> None:
"""SDK ``CodeInterpreterTool`` instances without a container must also be backfilled.
Reproduces the toolbox creation path that calls
Reproduces the hosted tool creation path that calls
``CodeInterpreterTool(name="code_interpreter")`` without a container.
"""
from azure.ai.projects.models import CodeInterpreterTool
@@ -771,13 +771,13 @@ def test_prepare_tools_for_openai_strips_name_from_non_function_hosted_tool_dict
{
"type": "file_search",
"name": "file_search_tool_123",
"description": "toolbox decoration",
"description": "hosted tool decoration",
"vector_store_ids": ["vs_123"],
},
{
"type": "web_search",
"name": "web_search_tool_456",
"description": "toolbox decoration",
"description": "hosted tool decoration",
},
])
@@ -1,435 +0,0 @@
# Copyright (c) Microsoft. All rights reserved.
"""Unit tests for toolbox helpers on FoundryChatClient.
Return types are the raw azure-ai-projects SDK models (ToolboxVersionObject,
ToolboxObject) — no custom wrapper. Tests verify the chat-client get path and
tool-selection ergonomics.
"""
from __future__ import annotations
import datetime as dt
import os
from typing import Any
from unittest.mock import AsyncMock, MagicMock
import pytest
try:
from azure.ai.projects.models import (
AutoCodeInterpreterToolParam,
CodeInterpreterTool,
Tool,
ToolboxObject,
ToolboxVersionObject,
)
except ImportError:
pytest.skip(
"Toolbox types require azure-ai-projects>=2.1.0 (unreleased).",
allow_module_level=True,
)
from azure.core.exceptions import ResourceNotFoundError
from azure.identity import AzureCliCredential
# --------------------------------------------------------------------------- #
# Helpers #
# --------------------------------------------------------------------------- #
class _AsyncIter:
"""Minimal async-iterable for mocking ``AsyncItemPaged`` in tests."""
def __init__(self, items: list[Any]) -> None:
self._items = items
def __aiter__(self) -> _AsyncIter:
self._iter = iter(self._items)
return self
async def __anext__(self) -> Any:
try:
return next(self._iter)
except StopIteration:
raise StopAsyncIteration from None
def _make_code_interpreter() -> CodeInterpreterTool:
return CodeInterpreterTool(container=AutoCodeInterpreterToolParam())
def _make_version_object(
*,
name: str = "research_tools",
version: str = "v1",
tools: list[Tool] | None = None,
description: str | None = None,
) -> ToolboxVersionObject:
return ToolboxVersionObject(
id=f"tbv_{name}_{version}",
name=name,
version=version,
metadata={},
created_at=dt.datetime(2026, 4, 10, tzinfo=dt.timezone.utc),
tools=tools if tools is not None else [_make_code_interpreter()],
description=description,
)
def _make_mock_foundry_client(*, project_client: MagicMock) -> Any:
"""Build a FoundryChatClient wired to a mock project_client."""
from agent_framework_foundry import FoundryChatClient
project_client.get_openai_client = MagicMock(return_value=MagicMock())
return FoundryChatClient(project_client=project_client, model="test-model")
# --------------------------------------------------------------------------- #
# get_toolbox — explicit version path #
# --------------------------------------------------------------------------- #
async def test_get_toolbox_with_explicit_version_makes_single_request() -> None:
project_client = MagicMock()
version_obj = _make_version_object(name="research_tools", version="v3")
project_client.beta.toolboxes.get_version = AsyncMock(return_value=version_obj)
project_client.beta.toolboxes.get = AsyncMock(
side_effect=AssertionError("get() must not be called when version is explicit")
)
client = _make_mock_foundry_client(project_client=project_client)
toolbox = await client.get_toolbox("research_tools", version="v3")
assert isinstance(toolbox, ToolboxVersionObject)
assert toolbox.name == "research_tools"
assert toolbox.version == "v3"
project_client.beta.toolboxes.get_version.assert_awaited_once_with("research_tools", "v3")
project_client.beta.toolboxes.get.assert_not_called()
# --------------------------------------------------------------------------- #
# get_toolbox — default-version path + error + passthrough + smoke #
# --------------------------------------------------------------------------- #
async def test_get_toolbox_default_version_resolves_then_fetches() -> None:
project_client = MagicMock()
handle = ToolboxObject(id="tb_1", name="research_tools", default_version="v5")
version_obj = _make_version_object(name="research_tools", version="v5")
project_client.beta.toolboxes.get = AsyncMock(return_value=handle)
project_client.beta.toolboxes.get_version = AsyncMock(return_value=version_obj)
client = _make_mock_foundry_client(project_client=project_client)
toolbox = await client.get_toolbox("research_tools")
assert toolbox.version == "v5"
project_client.beta.toolboxes.get.assert_awaited_once_with("research_tools")
project_client.beta.toolboxes.get_version.assert_awaited_once_with("research_tools", "v5")
async def test_get_toolbox_propagates_resource_not_found() -> None:
project_client = MagicMock()
project_client.beta.toolboxes.get = AsyncMock(side_effect=ResourceNotFoundError("no such toolbox"))
client = _make_mock_foundry_client(project_client=project_client)
with pytest.raises(ResourceNotFoundError):
await client.get_toolbox("missing_toolbox")
async def test_get_toolbox_tool_passthrough_preserves_heterogeneous_types() -> None:
"""Ensure all Tool subclasses pass through unchanged — critical for MCP tools
with project_connection_id, which must reach the runtime untouched."""
from azure.ai.projects.models import MCPTool as FoundryMCPTool
mcp_tool = FoundryMCPTool(
server_label="github_oauth",
server_url="https://api.githubcopilot.com/mcp",
)
mcp_tool["project_connection_id"] = "conn_abc"
project_client = MagicMock()
version_obj = _make_version_object(
name="mixed",
version="v1",
tools=[_make_code_interpreter(), mcp_tool],
)
project_client.beta.toolboxes.get_version = AsyncMock(return_value=version_obj)
client = _make_mock_foundry_client(project_client=project_client)
toolbox = await client.get_toolbox("mixed", version="v1")
assert len(toolbox.tools) == 2
assert isinstance(toolbox.tools[0], CodeInterpreterTool)
assert isinstance(toolbox.tools[1], FoundryMCPTool)
assert toolbox.tools[1]["project_connection_id"] == "conn_abc"
async def test_toolbox_tools_can_be_passed_to_agent() -> None:
"""Integration smoke: toolbox.tools can be passed directly to Agent(tools=...) ."""
from agent_framework import Agent
project_client = MagicMock()
version_obj = _make_version_object(name="research_tools", version="v1", tools=[_make_code_interpreter()])
project_client.beta.toolboxes.get_version = AsyncMock(return_value=version_obj)
client = _make_mock_foundry_client(project_client=project_client)
toolbox = await client.get_toolbox("research_tools", version="v1")
agent = Agent(
client=client,
instructions="You are a test agent.",
tools=toolbox.tools,
)
agent_tools = agent.default_options["tools"]
assert len(agent_tools) == 1
assert agent_tools[0]["type"] == "code_interpreter"
async def test_multiple_toolbox_tool_lists_can_be_combined_in_agent() -> None:
"""Nested toolbox ``.tools`` lists flatten into one tool list on Agent construction."""
from agent_framework import Agent
project_client = MagicMock()
project_client.get_openai_client = MagicMock(return_value=MagicMock())
client = _make_mock_foundry_client(project_client=project_client)
toolbox_a = _make_version_object(name="research_tools", version="v1", tools=[_make_code_interpreter()])
toolbox_b = _make_version_object(name="some_other_tools", version="v3", tools=[_make_code_interpreter()])
agent = Agent(
client=client,
instructions="You are a test agent.",
tools=[toolbox_a.tools, toolbox_b.tools],
)
agent_tools = agent.default_options["tools"]
assert len(agent_tools) == 2
assert agent_tools[0]["type"] == "code_interpreter"
assert agent_tools[1]["type"] == "code_interpreter"
# --------------------------------------------------------------------------- #
# toolbox tool selection helpers #
# --------------------------------------------------------------------------- #
def test_get_toolbox_tool_name_prefers_server_label_then_name_then_type() -> None:
from azure.ai.projects.models import MCPTool as FoundryMCPTool
from agent_framework_foundry import get_toolbox_tool_name
mcp_tool = FoundryMCPTool(
server_label="githubmcp",
server_url="https://api.githubcopilot.com/mcp",
)
assert get_toolbox_tool_name(mcp_tool) == "githubmcp"
named_tool = {"type": "code_interpreter", "name": "ci_tool"}
assert get_toolbox_tool_name(named_tool) == "ci_tool"
unnamed_tool = {"type": "web_search"}
assert get_toolbox_tool_name(unnamed_tool) == "web_search"
def test_select_toolbox_tools_filters_by_names() -> None:
from azure.ai.projects.models import MCPTool as FoundryMCPTool
from agent_framework_foundry import select_toolbox_tools
tools: list[Tool | dict[str, Any]] = [
FoundryMCPTool(server_label="githubmcp", server_url="https://api.githubcopilot.com/mcp"),
{"type": "code_interpreter", "name": "python_runner"},
{"type": "web_search"},
]
selected = select_toolbox_tools(tools, include_names=["githubmcp", "python_runner"])
assert len(selected) == 2
assert selected[0] is tools[0]
assert selected[1] is tools[1]
def test_select_toolbox_tools_filters_by_typed_tool_types() -> None:
from agent_framework_foundry import select_toolbox_tools
tools: list[Tool | dict[str, Any]] = [
{"type": "mcp", "server_label": "githubmcp"},
{"type": "code_interpreter", "name": "python_runner"},
{"type": "web_search"},
]
selected = select_toolbox_tools(tools, include_types=["mcp", "code_interpreter"])
assert len(selected) == 2
assert selected[0]["type"] == "mcp"
assert selected[1]["type"] == "code_interpreter"
def test_select_toolbox_tools_accepts_toolbox_object_directly() -> None:
from agent_framework_foundry import select_toolbox_tools
toolbox = _make_version_object(
name="research_tools",
version="v1",
tools=[
{"type": "mcp", "server_label": "githubmcp"}, # type: ignore[list-item]
{"type": "code_interpreter", "name": "python_runner"}, # type: ignore[list-item]
{"type": "web_search"}, # type: ignore[list-item]
],
)
selected = select_toolbox_tools(toolbox, include_types=["mcp", "code_interpreter"])
assert len(selected) == 2
assert selected[0]["type"] == "mcp"
assert selected[1]["type"] == "code_interpreter"
async def test_fetched_toolbox_can_be_combined_with_function_tool() -> None:
from agent_framework import Agent, FunctionTool, tool
project_client = MagicMock()
version_obj = _make_version_object(name="research_tools", version="v1", tools=[_make_code_interpreter()])
project_client.beta.toolboxes.get_version = AsyncMock(return_value=version_obj)
client = _make_mock_foundry_client(project_client=project_client)
toolbox = await client.get_toolbox("research_tools", version="v1")
@tool(name="local_lookup", description="A local helper tool")
def local_lookup(query: str) -> str:
return query
agent = Agent(
client=client,
instructions="You are a test agent.",
tools=[toolbox, local_lookup],
)
agent_tools = agent.default_options["tools"]
assert len(agent_tools) == 2
assert agent_tools[0]["type"] == "code_interpreter"
assert isinstance(agent_tools[1], FunctionTool)
assert agent_tools[1].name == "local_lookup"
def test_select_toolbox_tools_supports_excludes_and_predicate() -> None:
from agent_framework_foundry import select_toolbox_tools
tools: list[Tool | dict[str, Any]] = [
{"type": "mcp", "server_label": "githubmcp"},
{"type": "mcp", "server_label": "learnmcp"},
{"type": "web_search"},
]
selected = select_toolbox_tools(
tools,
exclude_names=["learnmcp"],
predicate=lambda tool: tool.get("type") == "mcp", # type: ignore[union-attr]
)
assert len(selected) == 1
assert selected[0]["server_label"] == "githubmcp"
async def test_selected_toolbox_subset_can_be_combined_with_function_tool() -> None:
from agent_framework import Agent, FunctionTool, tool
from agent_framework_foundry import select_toolbox_tools
project_client = MagicMock()
version_obj = _make_version_object(
name="research_tools",
version="v1",
tools=[
{"type": "mcp", "server_label": "githubmcp"}, # type: ignore[list-item]
{"type": "code_interpreter", "name": "python_runner"}, # type: ignore[list-item]
{"type": "web_search"}, # type: ignore[list-item]
],
)
project_client.beta.toolboxes.get_version = AsyncMock(return_value=version_obj)
client = _make_mock_foundry_client(project_client=project_client)
toolbox = await client.get_toolbox("research_tools", version="v1")
selected_tools = select_toolbox_tools(toolbox, include_types=["mcp", "code_interpreter"])
@tool(name="local_lookup", description="A local helper tool")
def local_lookup(query: str) -> str:
return query
agent = Agent(
client=client,
instructions="You are a test agent.",
tools=[selected_tools, local_lookup],
)
agent_tools = agent.default_options["tools"]
assert len(agent_tools) == 3
assert agent_tools[0]["type"] == "mcp"
assert agent_tools[1]["type"] == "code_interpreter"
assert isinstance(agent_tools[2], FunctionTool)
assert agent_tools[2].name == "local_lookup"
# --------------------------------------------------------------------------- #
# Integration #
# --------------------------------------------------------------------------- #
skip_if_foundry_integration_tests_disabled = pytest.mark.skipif(
os.getenv("FOUNDRY_PROJECT_ENDPOINT", "") in ("", "https://test-project.services.ai.azure.com/")
or os.getenv("FOUNDRY_MODEL", "") == "",
reason="No real FOUNDRY_PROJECT_ENDPOINT or FOUNDRY_MODEL provided; skipping integration tests.",
)
@pytest.mark.flaky
@pytest.mark.integration
@skip_if_foundry_integration_tests_disabled
async def test_integration_get_toolbox_round_trip_against_real_project() -> None:
"""Create a toolbox via the raw SDK, fetch via FoundryChatClient, then delete.
Self-contained to avoid depending on toolboxes that may be cleaned up
externally. Exercises both the default-version resolution path
(``get`` + ``get_version``) and the explicit-version path.
"""
from uuid import uuid4
from agent_framework import Agent
from agent_framework_foundry import FoundryChatClient
client = FoundryChatClient(credential=AzureCliCredential())
project_client = client.project_client
toolbox_name = f"af-int-toolbox-{uuid4().hex[:12]}"
created = await project_client.beta.toolboxes.create_version(
name=toolbox_name,
tools=[CodeInterpreterTool()],
description=f"{toolbox_name} integration test",
)
assert isinstance(created, ToolboxVersionObject)
try:
toolbox_default = await client.get_toolbox(toolbox_name)
assert toolbox_default.name == toolbox_name
assert toolbox_default.tools, "Default-version fetch returned no tools"
toolbox_pinned = await client.get_toolbox(toolbox_name, version=created.version)
assert toolbox_pinned.version == created.version
assert toolbox_pinned.tools
agent = Agent(
client=client,
instructions="You are a test agent.",
tools=toolbox_pinned.tools,
)
assert len(agent.default_options["tools"]) == len(toolbox_pinned.tools)
finally:
await project_client.beta.toolboxes.delete(toolbox_name)