agent-framework

Python: Remove bespoke Foundry toolbox helpers; standardize on MCP for toolbox consumption (#5671 )

* Remove Foundry toolbox helpers; standardize on MCP for toolbox consumption

- Remove RawFoundryChatClient.get_toolbox() and its fetch_toolbox import
- Remove fetch_toolbox, select_toolbox_tools, get_toolbox_tool_name,
  get_toolbox_tool_type, FoundryHostedToolType, ToolboxToolSelectionInput
  from agent_framework_foundry._tools
- Remove ExperimentalFeature.TOOLBOXES from _feature_stage.py (no consumers)
- Drop toolbox re-exports from agent_framework_foundry/__init__.py and
  agent_framework.foundry namespace
- Update _sanitize_foundry_response_tool docstring to remove toolbox framing;
  sanitization logic itself is unchanged
- Update _agent.py docstring: 'toolbox-fetched MCP' → 'hosted MCP'
- Delete tests/test_toolbox.py (all tests covered removed helpers)
- Update test_foundry_chat_client.py: rename/redoc tests that mentioned
  toolbox but test sanitization that remains
- Delete foundry_chat_client_with_toolbox.py (bespoke toolbox API sample)
- Delete foundry_toolbox_context_provider.py (relied on select_toolbox_tools)
- Rename foundry_chat_client_with_toolbox_mcp.py →
  foundry_chat_client_with_toolbox.py (canonical MCP pattern)
- Rewrite 04_foundry_toolbox/main.py to use MCPStreamableHTTPTool
- Update provider/README, context_providers/README, 04_foundry_toolbox/README

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(samples): update 06_files sample to consume toolbox via MCP (#5670)

Replace removed get_toolbox/select_toolbox_tools APIs with
MCPStreamableHTTPTool, using allowed_tools=["code_interpreter"] to
select only the code interpreter from the toolbox endpoint.

Update .env.example and README to use FOUNDRY_TOOLBOX_ENDPOINT
instead of TOOLBOX_NAME.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(foundry): remove non-existent toolbox helper APIs from README (#5670)

Remove the 'fetch, optionally filter, and pass tools directly' pattern
from the FoundryChatClient toolbox documentation, as select_toolbox_tools
and get_toolbox were removed. Only the MCP endpoint pattern is documented.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(foundry): remove residual toolbox docstring references and reproduction report

Remove REPRODUCTION_REPORT.md (workflow artifact that should not be committed),
and update two remaining docstring references that still said 'toolbox reads'
/'toolbox definition' after the toolbox helpers were removed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Remove bespoke Foundry toolbox helpers; standardize on MCP for toolbox consumption

Fixes #5670

* fix(#5670): resolve toolbox endpoint from TOOLBOX_NAME fallback; add namespace regression tests

- Add _resolve_toolbox_endpoint() helper in 04_foundry_toolbox/main.py and
  06_files/main.py that prefers FOUNDRY_TOOLBOX_ENDPOINT but falls back to
  deriving the MCP URL from FOUNDRY_PROJECT_ENDPOINT + TOOLBOX_NAME — fixing
  the startup KeyError when agents are deployed via azd provision (which injects
  TOOLBOX_NAME, not FOUNDRY_TOOLBOX_ENDPOINT).
- Update 04_foundry_toolbox/.env.example to use FOUNDRY_TOOLBOX_ENDPOINT
  (consistent with 06_files).
- Add TOOLBOX_NAME env var to 06_files/agent.yaml so deployed agents have it
  available for the fallback derivation.
- Update both READMEs to document the two ways to supply the toolbox endpoint.
- Add test_foundry_namespace_no_longer_exposes_toolbox_helpers() with negative
  assertions for FoundryHostedToolType, get_toolbox_tool_name,
  get_toolbox_tool_type, and select_toolbox_tools — guarding against accidental
  re-introduction of removed symbols.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(samples): fail fast on empty FOUNDRY_TOOLBOX_ENDPOINT; add unit tests

Addresses review feedback for #5670:

- In _resolve_toolbox_endpoint() (04_foundry_toolbox/main.py and
  06_files/main.py) change the walrus-operator check from a truthy
  test to an explicit 'is not None' guard.  An explicitly set empty
  string now raises ValueError immediately with a clear message
  instead of silently falling through to the fallback URL
  construction.

- Add tests/samples/hosting/test_toolbox_endpoint.py covering both
  sample modules:
    (a) FOUNDRY_TOOLBOX_ENDPOINT set → returned as-is
    (b) FOUNDRY_TOOLBOX_ENDPOINT set to empty string → ValueError
    (c) fallback constructs URL from FOUNDRY_PROJECT_ENDPOINT + TOOLBOX_NAME,
        stripping trailing slashes
    (d) neither variable group set → KeyError

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address review feedback: remove extraneous test and docstring content

- Remove test_foundry_namespace_no_longer_exposes_toolbox_helpers (no longer warranted)
- Remove docstring from _agent.py _prepare_tools_for_openai (extraneous)
- Trim _chat_client.py _prepare_tools_for_openai docstring to one-liner (toolbox references no longer relevant)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: remove remaining extraneous docstring from RawFoundryChatClient._prepare_tools_for_openai

Address review comment on PR #5671: reviewer noted the description
isn't warranted now that toolbox helpers have been removed. Matches
the pattern in RawFoundryAgentChatClient which has no docstring.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Evan Mattson · 2026-05-06 23:56:16 +00:00

e56e6dad4d

Python: Support GPT-5 verbosity option and restore Foundry agent_reference (#5619 )

* Python: Support GPT-5 verbosity option and restore Foundry agent_reference

Adds verbosity as a typed Literal["low","medium","high"] field on
OpenAIChatOptions (Responses API) and OpenAIChatCompletionOptions (Chat
Completions API), set in the same way as the existing reasoning options.
For the Responses API, top-level verbosity is translated to the nested
text.verbosity shape the OpenAI service expects. The same field flows
through to FoundryChatClient via the existing FoundryChatOptions alias.

Also fixes #5582: PR #5447 removed the agent_reference injection from
RawFoundryAgentChatClient._prepare_options, so first-turn calls against
a Foundry Prompt Agent went out without model and without agent_reference
and were rejected by the Responses API with "Missing required parameter:
'model'". Restores the injection on the non-preview path
(allow_preview=False) and adds a guard test that asserts the preview
path does not inject agent_reference, since the preview SDK injects it
via project_client.get_openai_client(agent_name=...).

Closes #5516
Closes #5582

* Python: Address Copilot review on PR #5619

- Foundry verbosity sample docstring: replace the misleading "set deployment
  name on model=" instruction with the actual env-var pattern the sample relies
  on (FOUNDRY_PROJECT_ENDPOINT and FOUNDRY_MODEL).
- _build_agent_reference docstring: clarify the helper is used for both
  Prompt Agents and HostedAgents on the non-preview path.
- Add a Responses API test that locks in the documented precedence rule:
  when both top-level verbosity and text["verbosity"] are supplied, the
  top-level value wins.

* Python: Drop redundant Foundry verbosity sample and list OpenAI sample in README

- Remove samples/02-agents/providers/foundry/foundry_chat_client_verbosity.py
  per review feedback. The verbosity functionality is identical across the
  OpenAI and Foundry clients (FoundryChatOptions is an alias of
  OpenAIChatOptions), so a single sample on the OpenAI side is sufficient.
- Add the new client_verbosity.py entry to the OpenAI samples README.

Evan Mattson · 2026-05-04 21:21:40 +00:00

f3db60fa65

Python: Reduce flaky integration tests and improve CI signal quality (#5454 )

* Enable Ollama integration tests in CI and rename report to Integration Test Report

- Install Ollama, cache models (qwen2.5:0.5b + nomic-embed-text), and start
  server in the Misc integration job for both workflow files
- Set OLLAMA_MODEL and OLLAMA_EMBEDDING_MODEL env vars so the 5 Ollama tests
  are no longer skipped
- Rename Flaky Test Report to Integration Test Report throughout (job names,
  artifact names, cache keys, file names, script titles/docstrings)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Bump Ollama model to qwen2.5:1.5b for better instruction following

The 0.5b model was too small to reliably follow simple prompts like
'Say Hello World', causing test assertion failures. The 1.5b model
follows instructions more reliably while still being small enough
for fast CI pulls (~1GB).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Re-enable reliable streaming integration tests

Remove the hard skip on test_03_reliable_streaming tests that was
temporarily disabled for instability investigation. CI infrastructure
(Azurite, DTS emulator, Redis, func CLI) is already in place.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Re-enable skipped Functions/DurableTask tests and bump timeout to 480s

- Remove hard skips from 4 tests in test_11_workflow_parallel.py
- Remove hard skip from test_conditional_branching in test_06_dt_multi_agent_orchestration_conditionals.py
- Increase pytest --timeout from 360 to 480 for Functions+DurableTask CI job
- Updated in both python-merge-tests.yml and python-integration-tests.yml

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Re-skip failing Functions/DurableTask tests with specific root causes

- test_11_workflow_parallel (4 tests): xdist worker crashes during execution
- test_conditional_branching: orchestration fails with RuntimeError, not a timeout
- Keep 480s timeout bump for remaining Functions tests

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix auth routing in samples 06/11: api_key -> credential for Azure OpenAI

Both samples passed a bearer token provider via api_key= which caused the
client to route to api.openai.com instead of Azure OpenAI, resulting in
401 Unauthorized. Changed to credential= which correctly triggers Azure
routing and picks up AZURE_OPENAI_ENDPOINT from the environment.

- samples/azure_functions/11_workflow_parallel/function_app.py: 1 fix
- samples/durabletask/06_multi_agent_orchestration_conditionals/worker.py: 2 fixes
- Re-enable 4 parallel workflow tests and 1 conditional branching test

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Re-skip parallel workflow tests: xdist worker distribution issue

The 4 parallel workflow tests crash because xdist worksteal distributes
them across separate workers, each spawning its own func process against
shared emulators. Auth fix (api_key->credential) was valid and stays.
test_conditional_branching now passes with the auth fix.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix E501 line-too-long in azurefunctions parallel test skip reasons

Wrap skip reason strings to stay within 120 char line limit.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add retry logic and port-conflict fix for Ollama CI setup

- Kill any auto-started Ollama before launching serve (fixes port
  conflict: 'address already in use')
- Retry ollama pull up to 3 times with 15s backoff (fixes 429 rate
  limit failures)
- Applied to both python-merge-tests.yml and python-integration-tests.yml

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix flaky integration tests and re-enable skipped tests

- Foundry agent: add allow_preview=True to custom client test
- Foundry hosting: raise max_output_tokens 50->200, add temperature,
  relax assertion in test_temperature_and_max_tokens
- Foundry embedding: update skip reason with root cause (endpoint mismatch)
- OpenAI file search: fix vector store indexing race condition by polling
  file_counts before querying; fix get_streaming_response -> get_response(stream=True)
- Azure OpenAI file search: remove skip (transient 500 resolved)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Remove temperature from foundry hosting test (unsupported by CI model)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Stabilize Ollama tool call integration tests with no-arg function

Use a no-argument greet() function instead of hello_world(arg1) for
integration tests. The 1.5B model in CI is unreliable at generating
correct tool call arguments, causing 'Argument parsing failed' errors.
A no-arg function eliminates this flakiness entirely.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Increase reliable streaming test timeouts from 30s to 60s

The LLM call through Azure OpenAI + Redis streaming pipeline can exceed
30s in CI due to cold starts or throttling. Raise to 60s to reduce
flaky timeouts while still bounded by pytest's 120s per-test limit.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Re-enable workflow parallel tests with xdist_group marker

The tests were skipped because xdist distributes module tests across
workers, each spawning their own func process (port conflicts). Adding
xdist_group forces all tests in this module onto a single worker so
the module-scoped function_app_for_test fixture works correctly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Revert "Re-enable workflow parallel tests with xdist_group marker"

This reverts commit 455c28da62.

* Rename flaky_report to integration_test_report and add try/finally cleanup

- Rename scripts/flaky_report/ to scripts/integration_test_report/ to
  reflect expanded scope beyond flaky-test detection
- Update workflow references in both CI files
- Wrap file search integration tests in try/finally to ensure vector
  store cleanup runs even on test failure or timeout

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix Ollama pull failure propagation and Azure OpenAI vector store readiness

- Ollama CI: fail the step immediately if model pull fails after 3
  retries instead of silently proceeding to tests
- Azure OpenAI file search: add the same vector-store readiness polling
  that was applied to the non-Azure OpenAI tests, preventing eventual
  consistency race conditions

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* remove load_dotenv from test file

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Giles Odigwe · 2026-05-01 00:41:39 +00:00

540193ccef

Python: [BREAKING] Standardize orchestration terminal outputs as AgentResponse (#5301 )

* Fix orchestration outputs so as_agent() returns the final answer only. Align other orchestration outputs

* Fix orchestration output issues from review comments

1. Sample cleanup: Remove commented-out FoundryChatClient block and update
prerequisites to reference OPENAI_CHAT_MODEL_ID instead of FOUNDRY_* vars.

2. Sequential approval output: Change _EndWithConversation.end_with_agent_executor_response
from a no-op sink to yield response.agent_response. When the last participant is
AgentApprovalExecutor (via with_request_info), _EndWithConversation is the output
executor so the yield produces the terminal answer. When the last participant is a
regular AgentExecutor, _EndWithConversation is not in output_executors so the yield
is silently filtered out.

3. Forward data events through WorkflowExecutor: _process_workflow_result now also
forwards 'data' events from sub-workflows so that emit_intermediate_data=True on
AgentExecutor works correctly when wrapped in AgentApprovalExecutor.

4. Concurrent docstring: Update _AggregateAgentConversations docstring to say
'deterministic participant order' instead of 'completion order'.

5. Add test_concurrent_intermediate_outputs_emits_data_events verifying that
ConcurrentBuilder(intermediate_outputs=True) emits per-participant data events
alongside the single aggregated output event.