Python: Add timeout parameter to FoundryAgent to fix ConnectTimeout on multi-turn conversations (#6263)

* Python: fix ConnectTimeout on multi-turn FoundryAgent conversations (#6241) Expose a `timeout` parameter on `RawFoundryAgentChatClient`, `_FoundryAgentChatClient`, `RawFoundryAgent`, `FoundryAgent`, and `RawOpenAIChatClient` so callers can override the HTTP timeout used by the underlying AsyncOpenAI client. Root cause: `RawFoundryAgentChatClient.__init__` called `project_client.get_openai_client()` without configuring any timeout, inheriting the OpenAI SDK default of `httpx.Timeout(connect=5.0)`. When connections are recycled between turns under load, the 5 s connect timeout fires and surfaces as `openai.APITimeoutError`. Fix: - `load_openai_service_settings` (`_shared.py`): accept `timeout` and include it in `client_args` for all three `AsyncOpenAI`/ `AsyncAzureOpenAI` construction paths. - `RawOpenAIChatClient.__init__` (`_chat_client.py`): accept `timeout` and forward to `load_openai_service_settings`. - `RawFoundryAgentChatClient.__init__` (`_agent.py`): accept `timeout` and set `openai_client.timeout = timeout` on the client returned by `get_openai_client()` before passing it to the base class. - `_FoundryAgentChatClient`, `RawFoundryAgent`, `FoundryAgent`: accept and propagate `timeout` through the construction chain. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add timeout parameter to FoundryAgent and RawOpenAIChatClient Expose a timeout parameter on RawFoundryAgentChatClient, _FoundryAgentChatClient, RawFoundryAgent, FoundryAgent, and RawOpenAIChatClient. When provided, the value is applied to the underlying AsyncOpenAI client so that connect timeouts under load or after connection recycling can be tuned by callers. Previously, get_openai_client() was called without any timeout override, so the SDK default of httpx.Timeout(connect=5.0) was inherited and could fire on multi-turn conversations where the underlying connection is recycled between turns. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Python: Add `timeout` parameter to `FoundryAgent` to fix `ConnectTimeout` on multi-turn conversations Fixes #6241 * fix(foundry): use with_options to avoid mutating shared OpenAI client timeout (#6241) Replace direct assignment with in RawFoundryAgentChatClient.__init__. The Azure AI Projects SDK caches and returns a shared AsyncOpenAI client per AIProjectClient. Mutating its .timeout attribute leaked the override to all other code paths sharing that client (other agents, user code). with_options() returns a new client instance with the override applied, leaving the original shared client untouched. Update tests to assert with_options is called with the correct timeout and that the original shared client's timeout attribute is not mutated. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * test(foundry): assert with_options return value flows to instance.client (#6241) The four timeout propagation tests verified that with_options was called but did not confirm that the returned (timeout-configured) client was actually stored on the instance. A silent discard of the return value would have left the tests green while the timeout had no effect. Each test now captures the constructed instance and asserts: assert <instance>.client is openai_client_mock.with_options.return_value Affected tests: - test_raw_foundry_agent_chat_client_init_applies_timeout_to_openai_client - test_raw_foundry_agent_chat_client_init_applies_timeout_with_preview_enabled - test_foundry_agent_chat_client_init_propagates_timeout - test_foundry_agent_init_propagates_timeout_to_openai_client Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <copilot@github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-16 21:04:09 +08:00 · 2026-06-05 03:25:18 +09:00
parent bc0e65d716
commit 6b94315161
5 changed files with 173 additions and 3 deletions
@@ -191,6 +191,7 @@ class RawFoundryAgentChatClient(  # type: ignore[misc]
        compaction_strategy: CompactionStrategy | None = None,
        tokenizer: TokenizerProtocol | None = None,
        additional_properties: dict[str, Any] | None = None,
+        timeout: float | None = None,
    ) -> None:
        """Initialize a raw Foundry Agent client.

@@ -211,6 +212,8 @@ class RawFoundryAgentChatClient(  # type: ignore[misc]
            compaction_strategy: Optional per-client compaction override.
            tokenizer: Optional tokenizer for compaction strategies.
            additional_properties: Additional properties stored on the client instance.
+            timeout: HTTP timeout in seconds for requests. When not provided, the
+                OpenAI SDK default is used (connect: 5s, total: 600s).
        """
        settings = load_settings(
            FoundryAgentSettings,
@@ -260,8 +263,11 @@ class RawFoundryAgentChatClient(  # type: ignore[misc]
            openai_client_kwargs["default_headers"] = dict(default_headers)
        if allow_preview:
            openai_client_kwargs["agent_name"] = self.agent_name
+        openai_client = self.project_client.get_openai_client(**openai_client_kwargs)
+        if timeout is not None:
+            openai_client = openai_client.with_options(timeout=timeout)
        super().__init__(
-            async_client=self.project_client.get_openai_client(**openai_client_kwargs),
+            async_client=openai_client,
            default_headers=default_headers,
            instruction_role=instruction_role,
            compaction_strategy=compaction_strategy,
@@ -537,6 +543,7 @@ class _FoundryAgentChatClient(  # type: ignore[misc]
        additional_properties: dict[str, Any] | None = None,
        middleware: (Sequence[ChatAndFunctionMiddlewareTypes] | None) = None,
        function_invocation_configuration: FunctionInvocationConfiguration | None = None,
+        timeout: float | None = None,
    ) -> None:
        """Initialize a Foundry Agent client with full middleware support.

@@ -556,6 +563,8 @@ class _FoundryAgentChatClient(  # type: ignore[misc]
            additional_properties: Additional properties stored on the client instance.
            middleware: Optional sequence of middleware.
            function_invocation_configuration: Optional function invocation configuration.
+            timeout: HTTP timeout in seconds for requests. When not provided, the
+                OpenAI SDK default is used (connect: 5s, total: 600s).
        """
        super().__init__(
            project_endpoint=project_endpoint,
@@ -573,6 +582,7 @@ class _FoundryAgentChatClient(  # type: ignore[misc]
            additional_properties=additional_properties,
            middleware=middleware,
            function_invocation_configuration=function_invocation_configuration,
+            timeout=timeout,
        )


@@ -625,6 +635,7 @@ class RawFoundryAgent(  # type: ignore[misc]
        compaction_strategy: CompactionStrategy | None = None,
        tokenizer: TokenizerProtocol | None = None,
        additional_properties: Mapping[str, Any] | None = None,
+        timeout: float | None = None,
    ) -> None:
        """Initialize a Foundry Agent.

@@ -657,6 +668,8 @@ class RawFoundryAgent(  # type: ignore[misc]
            compaction_strategy: Optional agent-level in-run compaction override.
            tokenizer: Optional agent-level tokenizer override.
            additional_properties: Additional properties stored on the local agent wrapper.
+            timeout: HTTP timeout in seconds for requests. When not provided, the
+                OpenAI SDK default is used (connect: 5s, total: 600s).
        """
        # Create the client
        actual_client_type = client_type or _FoundryAgentChatClient
@@ -675,6 +688,7 @@ class RawFoundryAgent(  # type: ignore[misc]
            "default_headers": default_headers,
            "env_file_path": env_file_path,
            "env_file_encoding": env_file_encoding,
+            "timeout": timeout,
        }
        if function_invocation_configuration is not None:
            if not issubclass(actual_client_type, FunctionInvocationLayer):
@@ -912,6 +926,7 @@ class FoundryAgent(  # type: ignore[misc]
        compaction_strategy: CompactionStrategy | None = None,
        tokenizer: TokenizerProtocol | None = None,
        additional_properties: Mapping[str, Any] | None = None,
+        timeout: float | None = None,
    ) -> None:
        """Initialize a Foundry Agent with full middleware and telemetry.

@@ -958,6 +973,8 @@ class FoundryAgent(  # type: ignore[misc]
            compaction_strategy: Optional agent-level in-run compaction override.
            tokenizer: Optional agent-level tokenizer override.
            additional_properties: Additional properties stored on the local agent wrapper.
+            timeout: HTTP timeout in seconds for requests. When not provided, the
+                OpenAI SDK default is used (connect: 5s, total: 600s).
        """
        super().__init__(
            project_endpoint=project_endpoint,
@@ -983,4 +1000,5 @@ class FoundryAgent(  # type: ignore[misc]
            compaction_strategy=compaction_strategy,
            tokenizer=tokenizer,
            additional_properties=additional_properties,
+            timeout=timeout,
        )
@@ -109,9 +109,67 @@ def test_raw_foundry_agent_chat_client_init_uses_explicit_parameters() -> None:
    assert "compaction_strategy" in signature.parameters
    assert "tokenizer" in signature.parameters
    assert "additional_properties" in signature.parameters
+    assert "timeout" in signature.parameters
    assert all(parameter.kind != inspect.Parameter.VAR_KEYWORD for parameter in signature.parameters.values())


+def test_raw_foundry_agent_chat_client_init_applies_timeout_to_openai_client() -> None:
+    """Test that timeout is applied via with_options without mutating the shared OpenAI client."""
+
+    mock_project = MagicMock()
+    openai_client_mock = MagicMock()
+    openai_client_mock.timeout = 5.0
+    mock_project.get_openai_client.return_value = openai_client_mock
+
+    client = RawFoundryAgentChatClient(
+        project_client=mock_project,
+        agent_name="test-agent",
+        timeout=60.0,
+    )
+
+    openai_client_mock.with_options.assert_called_once_with(timeout=60.0)
+    assert openai_client_mock.timeout == 5.0, "Original shared client must not be mutated"
+    assert client.client is openai_client_mock.with_options.return_value
+
+
+def test_raw_foundry_agent_chat_client_init_timeout_none_leaves_client_unchanged() -> None:
+    """Test that timeout=None does not call with_options and leaves the shared client intact."""
+
+    mock_project = MagicMock()
+    openai_client_mock = MagicMock()
+    openai_client_mock.timeout = 5.0
+    mock_project.get_openai_client.return_value = openai_client_mock
+
+    RawFoundryAgentChatClient(
+        project_client=mock_project,
+        agent_name="test-agent",
+        timeout=None,
+    )
+
+    openai_client_mock.with_options.assert_not_called()
+    assert openai_client_mock.timeout == 5.0
+
+
+def test_raw_foundry_agent_chat_client_init_applies_timeout_with_preview_enabled() -> None:
+    """Test that timeout uses with_options even when allow_preview=True (hosted agent path)."""
+
+    mock_project = MagicMock()
+    openai_client_mock = MagicMock()
+    openai_client_mock.timeout = 5.0
+    mock_project.get_openai_client.return_value = openai_client_mock
+
+    client = RawFoundryAgentChatClient(
+        project_client=mock_project,
+        agent_name="hosted-agent",
+        allow_preview=True,
+        timeout=120.0,
+    )
+
+    openai_client_mock.with_options.assert_called_once_with(timeout=120.0)
+    assert openai_client_mock.timeout == 5.0, "Original shared client must not be mutated"
+    assert client.client is openai_client_mock.with_options.return_value
+
+
 def test_raw_foundry_agent_chat_client_as_agent_preserves_client_type() -> None:
    """Test that as_agent() wraps the client in FoundryAgent using the same client class."""

@@ -552,9 +610,29 @@ def test_foundry_agent_chat_client_init_uses_explicit_parameters() -> None:
    assert "compaction_strategy" in signature.parameters
    assert "tokenizer" in signature.parameters
    assert "additional_properties" in signature.parameters
+    assert "timeout" in signature.parameters
    assert all(parameter.kind != inspect.Parameter.VAR_KEYWORD for parameter in signature.parameters.values())


+def test_foundry_agent_chat_client_init_propagates_timeout() -> None:
+    """Test that _FoundryAgentChatClient calls with_options instead of mutating the shared client."""
+
+    mock_project = MagicMock()
+    openai_client_mock = MagicMock()
+    openai_client_mock.timeout = 5.0
+    mock_project.get_openai_client.return_value = openai_client_mock
+
+    client = _FoundryAgentChatClient(
+        project_client=mock_project,
+        agent_name="test-agent",
+        timeout=45.0,
+    )
+
+    openai_client_mock.with_options.assert_called_once_with(timeout=45.0)
+    assert openai_client_mock.timeout == 5.0, "Original shared client must not be mutated"
+    assert client.client is openai_client_mock.with_options.return_value
+
+
 def test_raw_foundry_agent_init_creates_client() -> None:
    """Test that RawFoundryAgent creates a client internally."""

@@ -629,6 +707,7 @@ def test_raw_foundry_agent_init_uses_explicit_parameters() -> None:
    assert "compaction_strategy" in signature.parameters
    assert "tokenizer" in signature.parameters
    assert "additional_properties" in signature.parameters
+    assert "timeout" in signature.parameters
    assert all(parameter.kind != inspect.Parameter.VAR_KEYWORD for parameter in signature.parameters.values())


@@ -641,9 +720,47 @@ def test_foundry_agent_init_uses_explicit_parameters() -> None:
    assert "compaction_strategy" in signature.parameters
    assert "tokenizer" in signature.parameters
    assert "additional_properties" in signature.parameters
+    assert "timeout" in signature.parameters
    assert all(parameter.kind != inspect.Parameter.VAR_KEYWORD for parameter in signature.parameters.values())


+def test_foundry_agent_init_propagates_timeout_to_openai_client() -> None:
+    """Test that FoundryAgent uses with_options instead of mutating the shared OpenAI client."""
+
+    mock_project = MagicMock()
+    openai_client_mock = MagicMock()
+    openai_client_mock.timeout = 5.0
+    mock_project.get_openai_client.return_value = openai_client_mock
+
+    agent = FoundryAgent(
+        project_client=mock_project,
+        agent_name="test-agent",
+        timeout=90.0,
+    )
+
+    openai_client_mock.with_options.assert_called_once_with(timeout=90.0)
+    assert openai_client_mock.timeout == 5.0, "Original shared client must not be mutated"
+    assert agent.client.client is openai_client_mock.with_options.return_value
+
+
+def test_foundry_agent_init_timeout_none_leaves_client_default() -> None:
+    """Test that FoundryAgent with timeout=None does not call with_options or mutate the client."""
+
+    mock_project = MagicMock()
+    openai_client_mock = MagicMock()
+    openai_client_mock.timeout = 5.0
+    mock_project.get_openai_client.return_value = openai_client_mock
+
+    FoundryAgent(
+        project_client=mock_project,
+        agent_name="test-agent",
+        timeout=None,
+    )
+
+    openai_client_mock.with_options.assert_not_called()
+    assert openai_client_mock.timeout == 5.0
+
+
 def test_raw_foundry_agent_init_rejects_invalid_client_type() -> None:
    """Test that invalid client_type raises TypeError."""

@@ -385,6 +385,7 @@ class RawOpenAIChatClient(  # type: ignore[misc]
        additional_properties: dict[str, Any] | None = None,
        env_file_path: str | None = None,
        env_file_encoding: str | None = None,
+        timeout: float | None = None,
    ) -> None:
        """Initialize a raw OpenAI Chat client.

@@ -406,6 +407,7 @@ class RawOpenAIChatClient(  # type: ignore[misc]
            env_file_path: Optional ``.env`` file that is checked before the process environment
                for ``OPENAI_*`` values.
            env_file_encoding: Encoding for the ``.env`` file.
+            timeout: Optional timeout in seconds for requests.
        """
        ...

@@ -427,6 +429,7 @@ class RawOpenAIChatClient(  # type: ignore[misc]
        additional_properties: dict[str, Any] | None = None,
        env_file_path: str | None = None,
        env_file_encoding: str | None = None,
+        timeout: float | None = None,
    ) -> None:
        """Initialize a raw OpenAI Chat client.

@@ -455,6 +458,7 @@ class RawOpenAIChatClient(  # type: ignore[misc]
            env_file_path: Optional ``.env`` file that is checked before process environment
                variables for ``AZURE_OPENAI_*`` values.
            env_file_encoding: Encoding for the ``.env`` file.
+            timeout: Optional timeout in seconds for requests.
        """
        ...

@@ -476,6 +480,7 @@ class RawOpenAIChatClient(  # type: ignore[misc]
        additional_properties: dict[str, Any] | None = None,
        env_file_path: str | None = None,
        env_file_encoding: str | None = None,
+        timeout: float | None = None,
    ) -> None:
        """Initialize a raw OpenAI Chat client.

@@ -511,6 +516,8 @@ class RawOpenAIChatClient(  # type: ignore[misc]
                variables. The same file is used for both ``OPENAI_*`` and ``AZURE_OPENAI_*``
                lookups.
            env_file_encoding: Encoding for the ``.env`` file.
+            timeout: HTTP timeout in seconds for requests. When not provided, the
+                OpenAI SDK default is used (connect: 5s, total: 600s).

        Notes:
            Environment resolution and routing precedence are:
@@ -541,6 +548,7 @@ class RawOpenAIChatClient(  # type: ignore[misc]
            openai_model_fields=("chat_model", "model"),
            azure_model_fields=("chat_model", "model"),
            responses_mode=True,
+            timeout=timeout,
        )

        self.client = client
@@ -162,6 +162,7 @@ def load_openai_service_settings(
    openai_model_fields: Sequence[OpenAIModelSettingName] = ("model",),
    azure_model_fields: Sequence[OpenAIModelSettingName] = ("model",),
    responses_mode: bool = False,
+    timeout: float | None = None,
 ) -> tuple[dict[str, Any], AsyncOpenAI, bool]:
    """Load OpenAI settings, including Azure OpenAI model aliases.

@@ -218,6 +219,8 @@ def load_openai_service_settings(
            }
            if base_url := openai_settings.get("base_url"):
                client_args["base_url"] = base_url
+            if timeout is not None:
+                client_args["timeout"] = timeout
            return openai_settings, AsyncOpenAI(**client_args), False  # type: ignore[return-value]
        checked_openai = True
    azure_settings = load_settings(
@@ -299,8 +302,12 @@ def load_openai_service_settings(
            openai_args["api_key"] = _ensure_async_token_provider(client_args["azure_ad_token_provider"])
        elif "api_key" in client_args:
            openai_args["api_key"] = client_args["api_key"]
+        if timeout is not None:
+            openai_args["timeout"] = timeout
        return azure_settings, AsyncOpenAI(**openai_args), True  # type: ignore[return-value]

+    if timeout is not None:
+        client_args["timeout"] = timeout
    return azure_settings, AsyncAzureOpenAI(**client_args), True  # type: ignore[return-value]


@@ -36,7 +36,7 @@ from agent_framework.exceptions import (
    ChatClientInvalidRequestException,
    SettingNotFoundError,
 )
-from openai import BadRequestError
+from openai import AsyncOpenAI, BadRequestError
 from openai.types.responses.response_reasoning_item import Summary
 from openai.types.responses.response_reasoning_summary_text_delta_event import (
    ResponseReasoningSummaryTextDeltaEvent,
@@ -55,7 +55,7 @@ from pydantic import BaseModel
 from pytest import param

 from agent_framework_openai import OpenAIChatClient
-from agent_framework_openai._chat_client import OPENAI_LOCAL_SHELL_CALL_ITEM_ID_KEY
+from agent_framework_openai._chat_client import OPENAI_LOCAL_SHELL_CALL_ITEM_ID_KEY, RawOpenAIChatClient
 from agent_framework_openai._exceptions import OpenAIContentFilterException

 skip_if_openai_integration_tests_disabled = pytest.mark.skipif(
@@ -194,6 +194,26 @@ def test_init_uses_explicit_parameters() -> None:
    assert all(parameter.kind != inspect.Parameter.VAR_KEYWORD for parameter in signature.parameters.values())


+def test_raw_openai_chat_client_init_uses_explicit_parameters() -> None:
+    signature = inspect.signature(RawOpenAIChatClient.__init__)
+
+    assert "additional_properties" in signature.parameters
+    assert "compaction_strategy" in signature.parameters
+    assert "tokenizer" in signature.parameters
+    assert "timeout" in signature.parameters
+    assert all(parameter.kind != inspect.Parameter.VAR_KEYWORD for parameter in signature.parameters.values())
+
+
+def test_raw_openai_chat_client_accepts_preconfigured_client_with_timeout() -> None:
+    """Test that timeout is accepted without error when async_client is pre-provided."""
+
+    mock_client = MagicMock(spec=AsyncOpenAI)
+    mock_client.timeout = 5.0
+
+    client = RawOpenAIChatClient(async_client=mock_client, timeout=30.0)
+    assert client is not None
+
+
 def test_openai_chat_client_supports_all_tool_protocols() -> None:
    assert isinstance(OpenAIChatClient, SupportsCodeInterpreterTool)
    assert isinstance(OpenAIChatClient, SupportsWebSearchTool)