agent/agent-framework
2
0
Fork 0
mirror of https://github.com/microsoft/agent-framework.git synced 2026-06-16 21:04:09 +08:00
Code Issues Packages Projects Releases Wiki Activity

Python: [BREAKING] Moved to a single get_response and run API (#3379)

Browse Source 
* WIP

* big update to new ResponseStream model

* fixed tests and typing

* fixed tests and typing

* fixed tools typevar import

* fix

* mypy fix

* mypy fixes and some cleanup

* fix missing quoted names

* and client

* fix  imports agui

* fix anthropic override

* fix agui

* fix ag ui

* fix import

* fix anthropic types

* fix mypy

* refactoring

* updated typing

* fix 3.11

* fixes

* redid layering of chat clients and agents

* redid layering of chat clients and agents

* Fix lint, type, and test issues after rebase

- Add @overload decorators to AgentProtocol.run() for type compatibility
- Add missing docstring params (middleware, function_invocation_configuration)
- Fix TODO format (TD002) by adding author tags
- Fix broken observability tests from upstream:
  - Replace non-existent use_instrumentation with direct instantiation
  - Replace non-existent use_agent_instrumentation with AgentTelemetryLayer mixin
  - Fix get_streaming_response to use get_response(stream=True)
  - Add AgentInitializationError import
  - Update streaming exception tests to match actual behavior

* Fix AgentExecutionException import error in test_agents.py

- Replace non-existent AgentExecutionException with AgentRunException

* Fix test import and asyncio deprecation issues

- Add 'tests' to pythonpath in ag-ui pyproject.toml for utils_test_ag_ui import
- Replace deprecated asyncio.get_event_loop().run_until_complete with asyncio.run

* Fix azure-ai test failures

- Update _prepare_options patching to use correct class path
- Fix test_to_azure_ai_agent_tools_web_search_missing_connection to clear env vars

* Convert ag-ui utils_test_ag_ui.py to conftest.py

- Move test utilities to conftest.py for proper pytest discovery
- Update all test imports to use conftest instead of utils_test_ag_ui
- Remove old utils_test_ag_ui.py file
- Revert pythonpath change in pyproject.toml

* fix: use relative imports for ag-ui test utilities

* fix agui

* Rename Bare*Client to Raw*Client and BaseChatClient

- Renamed BareChatClient to BaseChatClient (abstract base class)
- Renamed BareOpenAIChatClient to RawOpenAIChatClient
- Renamed BareOpenAIResponsesClient to RawOpenAIResponsesClient
- Renamed BareAzureAIClient to RawAzureAIClient
- Added warning docstrings to Raw* classes about layer ordering
- Updated README in samples/getting_started/agents/custom with layer docs
- Added test for span ordering with function calling

* Fix layer ordering: FunctionInvocationLayer before ChatTelemetryLayer

This ensures each inner LLM call gets its own telemetry span, resulting in
the correct span sequence: chat -> execute_tool -> chat

Updated all production clients and test mocks to use correct ordering:
- ChatMiddlewareLayer (first)
- FunctionInvocationLayer (second)
- ChatTelemetryLayer (third)
- BaseChatClient/Raw...Client (fourth)

* Remove run_stream usage

* Fix conversation_id propagation

* Python: Add BaseAgent implementation for Claude Agent SDK (#3509)

* Added ClaudeAgent implementation

* Updated streaming logic

* Small updates

* Small update

* Fixes

* Small fix

* Naming improvements

* Updated imports

* Addressed comments

* Updated package versions

* Update Claude agent connector layering

* fix test and plugin

* Store function middleware in invocation layer

* Fix telemetry streaming and ag-ui tests

* Remove legacy ag-ui tests folder

* updates

* Remove terminate flag from FunctionInvocationContext, use MiddlewareTermination instead

- Remove terminate attribute from FunctionInvocationContext
- Add result attribute to MiddlewareTermination to carry function results
- FunctionMiddlewarePipeline.execute() now lets MiddlewareTermination propagate
- _auto_invoke_function captures context.result in exception before re-raising
- _try_execute_function_calls catches MiddlewareTermination and sets should_terminate
- Fix handoff middleware to append to chat_client.function_middleware directly
- Update tests to use raise MiddlewareTermination instead of context.terminate
- Add middleware flow documentation in samples/concepts/tools/README.md
- Fix ag-ui to use FunctionMiddlewarePipeline instead of removed create_function_middleware_pipeline

* fix: remove references to removed terminate flag in purview tests, add type ignore

* fix: move _test_utils.py from package to test folder

* fix: call get_final_response() to trigger context provider notification in streaming test

* fix: correct broken links in tools README

* docs: clarify default middleware behavior in summary table

* fix: ensure inner stream result hooks are called when using map()/from_awaitable()

* Fix mypy type errors

* Address PR review comments on observability.py

- Remove TODO comment about unconsumed streams, add explanatory note instead
- Remove redundant _close_span cleanup hook (already called in _finalize_stream)
- Clarify behavior: cleanup hooks run after stream iteration, if stream is not
  consumed the span remains open until garbage collected

* Remove gen_ai.client.operation.duration from span attributes

Duration is a metrics-only attribute per OpenTelemetry semantic conventions.
It should be recorded to the histogram but not set as a span attribute.

* Remove duration from _get_response_attributes, pass directly to _capture_response

Duration is a metrics-only attribute. It's now passed directly to _capture_response
instead of being included in the attributes dict that gets set on the span.

* Remove redundant _close_span cleanup hook in AgentTelemetryLayer

_finalize_stream already calls _close_span() in its finally block,
so adding it as a separate cleanup hook is redundant.

* Use weakref.finalize to close span when stream is garbage collected

If a user creates a streaming response but never consumes it, the cleanup
hooks won't run. Now we register a weak reference finalizer that will close
the span when the stream object is garbage collected, ensuring spans don't
leak in this scenario.

* Fix _get_finalizers_from_stream to use _result_hooks attribute

Renamed function to _get_result_hooks_from_stream and fixed it to
look for the _result_hooks attribute which is the correct name in
ResponseStream class.

* Add missing asyncio import in test_request_info_mixin.py

* Fix leftover merge conflict marker in image_generation sample

* Update integration tests

* Fix integration tests: increase max_iterations from 1 to 2

Tests with tool_choice options require at least 2 iterations:
1. First iteration to get function call and execute the tool
2. Second iteration to get the final text response

With max_iterations=1, streaming tests would return early with only
the function call/result but no final text content.

* Fix duplicate function call error in conversation-based APIs

When using conversation_id (for Responses/Assistants APIs), the server
already has the function call message from the previous response. We
should only send the new function result message, not all messages
including the function call which would cause a duplicate ID error.

Fix: When conversation_id is set, only send the last message (the tool
result) instead of all response.messages.

* Add regression test for conversation_id propagation between tool iterations

Port test from PR #3664 with updates for new streaming API pattern.
Tests that conversation_id is properly updated in options dict during
function invocation loop iterations.

* Fix tool_choice=required to return after tool execution

When tool_choice is 'required', the user's intent is to force exactly one
tool call. After the tool executes, return immediately with the function
call and result - don't continue to call the model again.

This fixes integration tests that were failing with empty text responses
because with tool_choice=required, the model would keep returning function
calls instead of text.

Also adds regression tests for:
- conversation_id propagation between tool iterations (from PR #3664)
- tool_choice=required returns after tool execution

* Document tool_choice behavior in tools README

- Add table explaining tool_choice values (auto, none, required)
- Explain why tool_choice=required returns immediately after tool execution
- Add code example showing the difference between required and auto
- Update flow diagram to show the early return path for tool_choice=required

* Fix tool_choice=None behavior - don't default to 'auto'

Remove the hardcoded default of 'auto' for tool_choice in ChatAgent init.
When tool_choice is not specified (None), it will now not be sent to the
API, allowing the API's default behavior to be used.

Users who want tool_choice='auto' can still explicitly set it either in
default_options or at runtime.

Fixes #3585

* Fix tool_choice=none should not remove tools

In OpenAI Assistants client, tools were not being sent when
tool_choice='none'. This was incorrect - tool_choice='none' means
the model won't call tools, but tools should still be available
in the request (they may be used later in the conversation).

Fixes #3585

* Add test for tool_choice=none preserving tools

Adds a regression test to ensure that when tool_choice='none' is set but
tools are provided, the tools are still sent to the API. This verifies
the fix for #3585.

* Fix tool_choice=none should not remove tools in all clients

Apply the same fix to OpenAI Responses client and Azure AI client:
- OpenAI Responses: Remove else block that popped tool_choice/parallel_tool_calls
- Azure AI: Remove tool_choice != 'none' check when adding tools

When tool_choice='none', the model won't call tools, but tools should
still be sent to the API so they're available for future turns.

Also update README to clarify tool_choice=required supports multiple tools.

Fixes #3585

* Keep tool_choice even when tools is None

Move tool_choice processing outside of the 'if tools' block in OpenAI
Responses client so tool_choice is sent to the API even when no tools
are provided.

* Update test to match new parallel_tool_calls behavior

Changed test_prepare_options_removes_parallel_tool_calls_when_no_tools to
test_prepare_options_preserves_parallel_tool_calls_when_no_tools to reflect
that parallel_tool_calls is now preserved even when no tools are present,
consistent with the tool_choice behavior.

* Fix ChatMessage API and Role enum usage after rebase

- Update ChatMessage instantiation to use keyword args (role=, text=, contents=)
- Fix Role enum comparisons to use .value for string comparison
- Add created_at to AgentResponse in error handling
- Fix AgentResponse.from_updates -> from_agent_run_response_updates
- Fix DurableAgentStateMessage.from_chat_message to convert Role enum to string
- Add Role import where needed

* Fix additional ChatMessage API and method name changes

- Fix ChatMessage usage in workflow files (use text= instead of contents= for strings)
- Fix AgentResponse.from_updates -> from_agent_run_response_updates in workflow files
- Fix test files for ChatMessage and Role enum usage

* Fix remaining ChatMessage API usage in test files

* Fix more ChatMessage and Role API changes in source and test files

- Fix ChatMessage in _magentic.py replan method
- Fix Role enum comparison in test assertions
- Fix remaining test files with old ChatMessage syntax

* Fix ChatMessage and Role API changes across packages

- Add Role import where missing
- Fix ChatMessage signature: positional args to keyword args (role=, text=, contents=)
- Fix Role enum comparisons: .role.value instead of .role string
- Fix FinishReason enum usage in ag-ui event converters
- Rename AgentResponse.from_updates to from_agent_run_response_updates in ag-ui

Fixes API compatibility after Types API Review improvements merge

* Fix ChatMessage and Role API changes in github_copilot tests

* Fix ChatMessage and Role API changes in redis and github_copilot packages

- Fix redis provider: Role enum comparison using .value
- Fix redis tests: ChatMessage signature and Role comparisons
- Fix github_copilot tests: ChatMessage signature and Role comparisons
- Update docstring examples in redis chat message store

* Fix ChatMessage and Role API changes in devui package

- Fix executor: ChatMessage signature change
- Fix conversations: Role enum to string conversion in two places
- Fix tests: ChatMessage signatures and Role comparisons

* Fix ChatMessage and Role API changes in a2a and lab packages

- Fix a2a tests: Role comparisons and ChatMessage signatures
- Fix lab tau2 source: Role enum comparison in flip_messages, log_messages, sliding_window
- Fix lab tau2 tests: ChatMessage signatures and Role comparisons

* Remove duplicate test files from ag-ui/tests (tests are in ag_ui_tests)

* Fix ChatMessage and Role API changes across packages

After rebasing on upstream/main which merged PR #3647 (Types API Review
improvements), fix all packages to use the new API:

- ChatMessage: Use keyword args (role=, text=, contents=) instead of
  positional args
- Role: Compare using .value attribute since it's now an enum

Packages fixed:
- ag-ui: Fixed Role value extraction bugs in _message_adapters.py
- anthropic: Fixed ChatMessage and Role comparisons in tests
- azure-ai: Fixed Role comparison in _client.py
- azure-ai-search: Fixed ChatMessage and Role in source/tests
- bedrock: Fixed ChatMessage signatures in tests
- chatkit: Fixed ChatMessage and Role in source/tests
- copilotstudio: Fixed ChatMessage and Role in tests
- declarative: Fixed ChatMessage in _executors_agents.py
- mem0: Fixed ChatMessage and Role in source/tests
- purview: Fixed ChatMessage in source/tests

* Fix mypy errors for ChatMessage and Role API changes

- durabletask: Use str() fallback in role value extraction
- core: Fix ChatMessage in _orchestrator_helpers.py to use keyword args
- core: Add type ignore for _conversation_state.py contents deserialization
- ag-ui: Fix type ignore comments (call-overload instead of arg-type)
- azure-ai-search: Fix get_role_value type hint to accept Any
- lab: Move get_role_value to module level with Any type hint

* Improve CI test timeout configuration

- Increase job timeout from 10 to 15 minutes
- Reduce per-test timeout to 60s (was 900s/300s)
- Add --timeout_method thread for better timeout handling
- Add --timeout-verbose to see which tests are slow
- Reduce retries from 3 to 2 and delay from 10s to 5s

This ensures individual test timeouts are shorter than the job
timeout, providing better visibility when tests hang.

With 60s timeout and 2 retries, worst case per test is ~180s.

* Fix ChatMessage API usage in docstrings and source

- Fix ChatMessage positional args in docstrings: _serialization.py, _threads.py, _middleware.py
- Fix ChatMessage in tau2 runner.py
- Fix role comparison in _orchestrator_helpers.py to use .value
- Fix role comparison in _group_chat.py docstring example
- Fix role assertions in test_durable_entities.py to use .value

* Revert tool_choice/parallel_tool_calls changes - must be removed when no tools

OpenAI API requires tool_choice and parallel_tool_calls to only be
present when tools are specified. Restored the logic that removes
these options when there are no tools.

- Restored check in _chat_client.py to remove tool_choice and
  parallel_tool_calls when no tools present
- Restored same logic in _responses_client.py
- Reverted test to expect the correct behavior

* fixed issue in tests

* fix: resolve merge conflict markers in ag-ui tests

* fix: restructure ag-ui tests and fix Role/FinishReason to use string types

* fix: streaming function invocation and middleware termination

- Refactor streaming function invocation to use get_final_response() on inner streams
- Fix MiddlewareTermination to accept result parameter for passing results
- Fix _AutoHandoffMiddleware to use MiddlewareTermination instead of context.terminate
- Fix AgentMiddlewareLayer.run() to properly forward function/chat middleware
- Remove duplicate middleware registration in AgentMiddlewareLayer.__init__
- Fix exception handling in _auto_invoke_function to properly capture termination
- Fix mypy errors in core package
- Update tests to use stream=True parameter for unified run API

* fix all tests command

* Refactor integration tests to use pytest fixtures

- Merge testutils.py into conftest.py for azurefunctions integration tests
- Merge dt_testutils.py into conftest.py for durabletask integration tests
- Convert all integration tests to use fixtures instead of direct imports
  (fixes ModuleNotFoundError with --import-mode=importlib)
- Add sample_helper fixture for azurefunctions tests
- Add agent_client_factory and orchestration_helper fixtures for durabletask
- Integration tests now skip with descriptive messages when services unavailable
- Restructure devui tests into tests/devui/ with proper conftest.py
- Add test organization guidelines to CODING_STANDARD.md
- Remove __init__.py from test directories per pytest best practices

* Fix pytest_collection_modifyitems to only skip integration tests

The hook was skipping all tests in the test session, not just
integration tests. Now it only skips items in the integration_tests
directory.

* Fix mem0 tests failing on Python 3.13

Use patch.object on the imported module instead of @patch with string
path to ensure the mock takes effect regardless of import timing.

* fix mem0

* another attempt for mem0

* fix for mem0

* fix mem0

* Increase worker initialization wait time in durabletask tests

Increase from 2 to 8 seconds to allow time for:
- Python startup and module imports
- Azure OpenAI client creation
- Agent registration with DTS worker
- Worker connection to DTS

This helps prevent test failures in CI where the first tests may run
before the worker is fully ready to process requests.

* Fix streaming test to use ResponseStream with finalizer

The _consume_stream method now expects a ResponseStream that can provide
a final AgentResponse via get_final_response(). Update the test to use
ResponseStream with AgentResponse.from_updates as the finalizer.

* Fix MockToolCallingAgent to use new ResponseStream API and update samples

* small updates to run_stream to run

* fix sub workflow

* temp fix for az func test

---------

Co-authored-by: Dmytro Struk <13853051+dmytrostruk@users.noreply.github.com>
This commit is contained in:
Eduard van Valkenburg
2026-02-05 20:09:58 +00:00
committed by GitHub
co-authored by Dmytro Struk
parent d1205896a1
commit 3dc59c83b5
372 changed files with 11583 additions and 9465 deletions
Download Patch File Download Diff File Expand all files Collapse all files
python/packages/core/tests/openai/test_openai_assistants_client.py
+56 -25
View File
@@ -695,7 +695,7 @@ def test_prepare_options_basic(mock_async_openai: MagicMock) -> None:
"top_p": 0.9,
}
messages = [ChatMessage("user", ["Hello"])]
messages = [ChatMessage(role="user", text="Hello")]
# Call the method
run_options, tool_results = chat_client._prepare_options(messages, options) # type: ignore
@@ -724,7 +724,7 @@ def test_prepare_options_with_tool_tool(mock_async_openai: MagicMock) -> None:
"tool_choice": "auto",
}
messages = [ChatMessage("user", ["Hello"])]
messages = [ChatMessage(role="user", text="Hello")]
# Call the method
run_options, tool_results = chat_client._prepare_options(messages, options) # type: ignore
@@ -749,7 +749,7 @@ def test_prepare_options_with_code_interpreter(mock_async_openai: MagicMock) ->
"tool_choice": "auto",
}
messages = [ChatMessage("user", ["Calculate something"])]
messages = [ChatMessage(role="user", text="Calculate something")]
# Call the method
run_options, tool_results = chat_client._prepare_options(messages, options) # type: ignore
@@ -762,23 +762,52 @@ def test_prepare_options_with_code_interpreter(mock_async_openai: MagicMock) ->
def test_prepare_options_tool_choice_none(mock_async_openai: MagicMock) -> None:
"""Test _prepare_options with tool_choice set to 'none'."""
"""Test _prepare_options with tool_choice set to 'none' and no tools."""
chat_client = create_test_openai_assistants_client(mock_async_openai)
options = {
"tool_choice": "none",
}
messages = [ChatMessage("user", ["Hello"])]
messages = [ChatMessage(role="user", text="Hello")]
# Call the method
run_options, tool_results = chat_client._prepare_options(messages, options) # type: ignore
# Should set tool_choice to none and not include tools
# Should set tool_choice to none - no tools because none were provided
assert run_options["tool_choice"] == "none"
assert "tools" not in run_options
def test_prepare_options_tool_choice_none_with_tools(mock_async_openai: MagicMock) -> None:
"""Test _prepare_options with tool_choice='none' but tools provided.
When tool_choice='none', the model won't call tools, but tools should still
be sent to the API so they're available for future turns in the conversation.
"""
chat_client = create_test_openai_assistants_client(mock_async_openai)
# Create a function tool
@tool(approval_mode="never_require")
def test_func(arg: str) -> str:
return arg
options = {
"tool_choice": "none",
"tools": [test_func],
}
messages = [ChatMessage(role="user", text="Hello")]
# Call the method
run_options, tool_results = chat_client._prepare_options(messages, options) # type: ignore
# Should set tool_choice to none BUT still include tools
assert run_options["tool_choice"] == "none"
assert "tools" in run_options
assert len(run_options["tools"]) == 1
def test_prepare_options_required_function(mock_async_openai: MagicMock) -> None:
"""Test _prepare_options with required function tool choice."""
chat_client = create_test_openai_assistants_client(mock_async_openai)
@@ -790,7 +819,7 @@ def test_prepare_options_required_function(mock_async_openai: MagicMock) -> None
"tool_choice": tool_choice,
}
messages = [ChatMessage("user", ["Hello"])]
messages = [ChatMessage(role="user", text="Hello")]
# Call the method
run_options, tool_results = chat_client._prepare_options(messages, options) # type: ignore
@@ -816,7 +845,7 @@ def test_prepare_options_with_file_search_tool(mock_async_openai: MagicMock) ->
"tool_choice": "auto",
}
messages = [ChatMessage("user", ["Search for information"])]
messages = [ChatMessage(role="user", text="Search for information")]
# Call the method
run_options, tool_results = chat_client._prepare_options(messages, options) # type: ignore
@@ -841,7 +870,7 @@ def test_prepare_options_with_mapping_tool(mock_async_openai: MagicMock) -> None
"tool_choice": "auto",
}
messages = [ChatMessage("user", ["Use custom tool"])]
messages = [ChatMessage(role="user", text="Use custom tool")]
# Call the method
run_options, tool_results = chat_client._prepare_options(messages, options) # type: ignore
@@ -863,7 +892,7 @@ def test_prepare_options_with_pydantic_response_format(mock_async_openai: MagicM
model_config = ConfigDict(extra="forbid")
chat_client = create_test_openai_assistants_client(mock_async_openai)
messages = [ChatMessage("user", ["Test"])]
messages = [ChatMessage(role="user", text="Test")]
options = {"response_format": TestResponse}
run_options, _ = chat_client._prepare_options(messages, options) # type: ignore
@@ -879,8 +908,8 @@ def test_prepare_options_with_system_message(mock_async_openai: MagicMock) -> No
chat_client = create_test_openai_assistants_client(mock_async_openai)
messages = [
ChatMessage("system", ["You are a helpful assistant."]),
ChatMessage("user", ["Hello"]),
ChatMessage(role="system", text="You are a helpful assistant."),
ChatMessage(role="user", text="Hello"),
]
# Call the method
@@ -900,7 +929,7 @@ def test_prepare_options_with_image_content(mock_async_openai: MagicMock) -> Non
# Create message with image content
image_content = Content.from_uri(uri="https://example.com/image.jpg", media_type="image/jpeg")
messages = [ChatMessage("user", [image_content])]
messages = [ChatMessage(role="user", contents=[image_content])]
# Call the method
run_options, tool_results = chat_client._prepare_options(messages, {}) # type: ignore
@@ -1020,7 +1049,7 @@ async def test_get_response() -> None:
"It's a beautiful day for outdoor activities.",
)
)
messages.append(ChatMessage("user", ["What's the weather like today?"]))
messages.append(ChatMessage(role="user", text="What's the weather like today?"))
# Test that the client can be used to get a response
response = await openai_assistants_client.get_response(messages=messages)
@@ -1038,7 +1067,7 @@ async def test_get_response_tools() -> None:
assert isinstance(openai_assistants_client, ChatClientProtocol)
messages: list[ChatMessage] = []
messages.append(ChatMessage("user", ["What's the weather like in Seattle?"]))
messages.append(ChatMessage(role="user", text="What's the weather like in Seattle?"))
# Test that the client can be used to get a response
response = await openai_assistants_client.get_response(
@@ -1066,10 +1095,10 @@ async def test_streaming() -> None:
"It's a beautiful day for outdoor activities.",
)
)
messages.append(ChatMessage("user", ["What's the weather like today?"]))
messages.append(ChatMessage(role="user", text="What's the weather like today?"))
# Test that the client can be used to get a response
response = openai_assistants_client.get_streaming_response(messages=messages)
response = openai_assistants_client.get_response(stream=True, messages=messages)
full_message: str = ""
async for chunk in response:
@@ -1090,10 +1119,11 @@ async def test_streaming_tools() -> None:
assert isinstance(openai_assistants_client, ChatClientProtocol)
messages: list[ChatMessage] = []
messages.append(ChatMessage("user", ["What's the weather like in Seattle?"]))
messages.append(ChatMessage(role="user", text="What's the weather like in Seattle?"))
# Test that the client can be used to get a response
response = openai_assistants_client.get_streaming_response(
response = openai_assistants_client.get_response(
stream=True,
messages=messages,
options={
"tools": [get_weather],
@@ -1118,7 +1148,7 @@ async def test_with_existing_assistant() -> None:
# First create an assistant to use in the test
async with OpenAIAssistantsClient(model_id=INTEGRATION_TEST_MODEL) as temp_client:
# Get the assistant ID by triggering assistant creation
messages = [ChatMessage("user", ["Hello"])]
messages = [ChatMessage(role="user", text="Hello")]
await temp_client.get_response(messages=messages)
assistant_id = temp_client.assistant_id
@@ -1129,7 +1159,7 @@ async def test_with_existing_assistant() -> None:
assert isinstance(openai_assistants_client, ChatClientProtocol)
assert openai_assistants_client.assistant_id == assistant_id
messages = [ChatMessage("user", ["What can you do?"])]
messages = [ChatMessage(role="user", text="What can you do?")]
# Test that the client can be used to get a response
response = await openai_assistants_client.get_response(messages=messages)
@@ -1148,7 +1178,7 @@ async def test_file_search() -> None:
assert isinstance(openai_assistants_client, ChatClientProtocol)
messages: list[ChatMessage] = []
messages.append(ChatMessage("user", ["What's the weather like today?"]))
messages.append(ChatMessage(role="user", text="What's the weather like today?"))
file_id, vector_store = await create_vector_store(openai_assistants_client)
response = await openai_assistants_client.get_response(
@@ -1174,10 +1204,11 @@ async def test_file_search_streaming() -> None:
assert isinstance(openai_assistants_client, ChatClientProtocol)
messages: list[ChatMessage] = []
messages.append(ChatMessage("user", ["What's the weather like today?"]))
messages.append(ChatMessage(role="user", text="What's the weather like today?"))
file_id, vector_store = await create_vector_store(openai_assistants_client)
response = openai_assistants_client.get_streaming_response(
response = openai_assistants_client.get_response(
stream=True,
messages=messages,
options={
"tools": [HostedFileSearchTool()],
@@ -1224,7 +1255,7 @@ async def test_openai_assistants_agent_basic_run_streaming():
) as agent:
# Run streaming query
full_message: str = ""
async for chunk in agent.run_stream("Please respond with exactly: 'This is a streaming response test.'"):
async for chunk in agent.run("Please respond with exactly: 'This is a streaming response test.'", stream=True):
assert chunk is not None
assert isinstance(chunk, AgentResponseUpdate)
if chunk.text:
python/packages/core/tests/openai/test_openai_chat_client.py
+32 -31
View File
@@ -154,7 +154,7 @@ def test_serialize_with_org_id(openai_unit_test_env: dict[str, str]) -> None:
async def test_content_filter_exception_handling(openai_unit_test_env: dict[str, str]) -> None:
"""Test that content filter errors are properly handled."""
client = OpenAIChatClient()
messages = [ChatMessage("user", ["test message"])]
messages = [ChatMessage(role="user", text="test message")]
# Create a mock BadRequestError with content_filter code
mock_response = MagicMock()
@@ -209,7 +209,7 @@ def get_weather(location: str) -> str:
async def test_exception_message_includes_original_error_details() -> None:
"""Test that exception messages include original error details in the new format."""
client = OpenAIChatClient(model_id="test-model", api_key="test-key")
messages = [ChatMessage("user", ["test message"])]
messages = [ChatMessage(role="user", text="test message")]
mock_response = MagicMock()
original_error_message = "Invalid API request format"
@@ -652,12 +652,12 @@ def test_function_approval_content_is_skipped_in_preparation(openai_unit_test_en
)
# Test that approval request is skipped
message_with_request = ChatMessage("assistant", [approval_request])
message_with_request = ChatMessage(role="assistant", contents=[approval_request])
prepared_request = client._prepare_message_for_openai(message_with_request)
assert len(prepared_request) == 0 # Should be empty - approval content is skipped
# Test that approval response is skipped
message_with_response = ChatMessage("user", [approval_response])
message_with_response = ChatMessage(role="user", contents=[approval_response])
prepared_response = client._prepare_message_for_openai(message_with_response)
assert len(prepared_response) == 0 # Should be empty - approval content is skipped
@@ -752,7 +752,7 @@ def test_prepare_options_without_model_id(openai_unit_test_env: dict[str, str])
client = OpenAIChatClient()
client.model_id = None # Remove model_id
messages = [ChatMessage("user", ["test"])]
messages = [ChatMessage(role="user", text="test")]
with pytest.raises(ValueError, match="model_id must be a non-empty string"):
client._prepare_options(messages, {})
@@ -786,7 +786,7 @@ def test_prepare_options_with_instructions(openai_unit_test_env: dict[str, str])
"""Test that instructions are prepended as system message."""
client = OpenAIChatClient()
messages = [ChatMessage("user", ["Hello"])]
messages = [ChatMessage(role="user", text="Hello")]
options = {"instructions": "You are a helpful assistant."}
prepared_options = client._prepare_options(messages, options)
@@ -836,7 +836,7 @@ def test_tool_choice_required_with_function_name(openai_unit_test_env: dict[str,
"""Test that tool_choice with required mode and function name is correctly prepared."""
client = OpenAIChatClient()
messages = [ChatMessage("user", ["test"])]
messages = [ChatMessage(role="user", text="test")]
options = {
"tools": [get_weather],
"tool_choice": {"mode": "required", "required_function_name": "get_weather"},
@@ -854,7 +854,7 @@ def test_response_format_dict_passthrough(openai_unit_test_env: dict[str, str])
"""Test that response_format as dict is passed through directly."""
client = OpenAIChatClient()
messages = [ChatMessage("user", ["test"])]
messages = [ChatMessage(role="user", text="test")]
custom_format = {
"type": "json_schema",
"json_schema": {"name": "Test", "schema": {"type": "object"}},
@@ -894,7 +894,7 @@ def test_prepare_options_removes_parallel_tool_calls_when_no_tools(openai_unit_t
"""Test that parallel_tool_calls is removed when no tools are present."""
client = OpenAIChatClient()
messages = [ChatMessage("user", ["test"])]
messages = [ChatMessage(role="user", text="test")]
options = {"allow_multiple_tool_calls": True}
prepared_options = client._prepare_options(messages, options)
@@ -906,7 +906,7 @@ def test_prepare_options_removes_parallel_tool_calls_when_no_tools(openai_unit_t
async def test_streaming_exception_handling(openai_unit_test_env: dict[str, str]) -> None:
"""Test that streaming errors are properly handled."""
client = OpenAIChatClient()
messages = [ChatMessage("user", ["test"])]
messages = [ChatMessage(role="user", text="test")]
# Create a mock error during streaming
mock_error = Exception("Streaming error")
@@ -915,12 +915,8 @@ async def test_streaming_exception_handling(openai_unit_test_env: dict[str, str]
patch.object(client.client.chat.completions, "create", side_effect=mock_error),
pytest.raises(ServiceResponseException),
):
async def consume_stream():
async for _ in client._inner_get_streaming_response(messages=messages, options={}): # type: ignore
pass
await consume_stream()
async for _ in client._inner_get_response(messages=messages, stream=True, options={}): # type: ignore
pass
# region Integration Tests
@@ -955,11 +951,11 @@ class OutputStruct(BaseModel):
param("tools", [get_weather], True, id="tools_function"),
param("tool_choice", "auto", True, id="tool_choice_auto"),
param("tool_choice", "none", True, id="tool_choice_none"),
param("tool_choice", "required", True, id="tool_choice_required_any"),
param("tool_choice", "required", False, id="tool_choice_required_any"),
param(
"tool_choice",
{"mode": "required", "required_function_name": "get_weather"},
True,
False,
id="tool_choice_required",
),
param("response_format", OutputStruct, True, id="response_format_pydantic"),
@@ -1001,21 +997,21 @@ async def test_integration_options(
check that the feature actually works correctly.
"""
client = OpenAIChatClient()
# to ensure toolmode required does not endlessly loop
client.function_invocation_configuration.max_iterations = 1
# Need at least 2 iterations for tool_choice tests: one to get function call, one to get final response
client.function_invocation_configuration["max_iterations"] = 2
for streaming in [False, True]:
# Prepare test message
if option_name.startswith("tools") or option_name.startswith("tool_choice"):
# Use weather-related prompt for tool tests
messages = [ChatMessage("user", ["What is the weather in Seattle?"])]
messages = [ChatMessage(role="user", text="What is the weather in Seattle?")]
elif option_name.startswith("response_format"):
# Use prompt that works well with structured output
messages = [ChatMessage("user", ["The weather in Seattle is sunny"])]
messages.append(ChatMessage("user", ["What is the weather in Seattle?"]))
messages = [ChatMessage(role="user", text="The weather in Seattle is sunny")]
messages.append(ChatMessage(role="user", text="What is the weather in Seattle?"))
else:
# Generic prompt for simple options
messages = [ChatMessage("user", ["Say 'Hello World' briefly."])]
messages = [ChatMessage(role="user", text="Say 'Hello World' briefly.")]
# Build options dict
options: dict[str, Any] = {option_name: option_value}
@@ -1026,13 +1022,13 @@ async def test_integration_options(
if streaming:
# Test streaming mode
response_gen = client.get_streaming_response(
response_stream = client.get_response(
messages=messages,
stream=True,
options=options,
)
output_format = option_value if option_name.startswith("response_format") else None
response = await ChatResponse.from_update_generator(response_gen, output_format_type=output_format)
response = await response_stream.get_final_response()
else:
# Test non-streaming mode
response = await client.get_response(
@@ -1042,8 +1038,13 @@ async def test_integration_options(
assert response is not None
assert isinstance(response, ChatResponse)
assert response.text is not None, f"No text in response for option '{option_name}'"
assert len(response.text) > 0, f"Empty response for option '{option_name}'"
assert response.messages is not None
if not option_name.startswith("tool_choice") and (
(isinstance(option_value, str) and option_value != "required")
or (isinstance(option_value, dict) and option_value.get("mode") != "required")
):
assert response.text is not None, f"No text in response for option '{option_name}'"
assert len(response.text) > 0, f"Empty response for option '{option_name}'"
# Validate based on option type
if needs_validation:
@@ -1080,7 +1081,7 @@ async def test_integration_web_search() -> None:
},
}
if streaming:
response = await ChatResponse.from_update_generator(client.get_streaming_response(**content))
response = await client.get_response(stream=True, **content).get_final_response()
else:
response = await client.get_response(**content)
@@ -1105,7 +1106,7 @@ async def test_integration_web_search() -> None:
},
}
if streaming:
response = await ChatResponse.from_update_generator(client.get_streaming_response(**content))
response = await client.get_response(stream=True, **content).get_final_response()
else:
response = await client.get_response(**content)
assert response.text is not None
python/packages/core/tests/openai/test_openai_chat_client_base.py
+21 -36
View File
@@ -69,7 +69,7 @@ async def test_cmc(
openai_unit_test_env: dict[str, str],
):
mock_create.return_value = mock_chat_completion_response
chat_history.append(ChatMessage("user", ["hello world"]))
chat_history.append(ChatMessage(role="user", text="hello world"))
openai_chat_completion = OpenAIChatClient()
await openai_chat_completion.get_response(messages=chat_history)
@@ -88,7 +88,7 @@ async def test_cmc_chat_options(
openai_unit_test_env: dict[str, str],
):
mock_create.return_value = mock_chat_completion_response
chat_history.append(ChatMessage("user", ["hello world"]))
chat_history.append(ChatMessage(role="user", text="hello world"))
openai_chat_completion = OpenAIChatClient()
await openai_chat_completion.get_response(
@@ -109,7 +109,7 @@ async def test_cmc_no_fcc_in_response(
openai_unit_test_env: dict[str, str],
):
mock_create.return_value = mock_chat_completion_response
chat_history.append(ChatMessage("user", ["hello world"]))
chat_history.append(ChatMessage(role="user", text="hello world"))
orig_chat_history = deepcopy(chat_history)
openai_chat_completion = OpenAIChatClient()
@@ -131,7 +131,7 @@ async def test_cmc_structured_output_no_fcc(
openai_unit_test_env: dict[str, str],
):
mock_create.return_value = mock_chat_completion_response
chat_history.append(ChatMessage("user", ["hello world"]))
chat_history.append(ChatMessage(role="user", text="hello world"))
# Define a mock response format
class Test(BaseModel):
@@ -153,10 +153,11 @@ async def test_scmc_chat_options(
openai_unit_test_env: dict[str, str],
):
mock_create.return_value = mock_streaming_chat_completion_response
chat_history.append(ChatMessage("user", ["hello world"]))
chat_history.append(ChatMessage(role="user", text="hello world"))
openai_chat_completion = OpenAIChatClient()
async for msg in openai_chat_completion.get_streaming_response(
async for msg in openai_chat_completion.get_response(
stream=True,
messages=chat_history,
):
assert isinstance(msg, ChatResponseUpdate)
@@ -178,7 +179,7 @@ async def test_cmc_general_exception(
openai_unit_test_env: dict[str, str],
):
mock_create.return_value = mock_chat_completion_response
chat_history.append(ChatMessage("user", ["hello world"]))
chat_history.append(ChatMessage(role="user", text="hello world"))
openai_chat_completion = OpenAIChatClient()
with pytest.raises(ServiceResponseException):
@@ -195,7 +196,7 @@ async def test_cmc_additional_properties(
openai_unit_test_env: dict[str, str],
):
mock_create.return_value = mock_chat_completion_response
chat_history.append(ChatMessage("user", ["hello world"]))
chat_history.append(ChatMessage(role="user", text="hello world"))
openai_chat_completion = OpenAIChatClient()
await openai_chat_completion.get_response(messages=chat_history, options={"reasoning_effort": "low"})
@@ -233,11 +234,12 @@ async def test_get_streaming(
stream = MagicMock(spec=AsyncStream)
stream.__aiter__.return_value = [content1, content2]
mock_create.return_value = stream
chat_history.append(ChatMessage("user", ["hello world"]))
chat_history.append(ChatMessage(role="user", text="hello world"))
orig_chat_history = deepcopy(chat_history)
openai_chat_completion = OpenAIChatClient()
async for msg in openai_chat_completion.get_streaming_response(
async for msg in openai_chat_completion.get_response(
stream=True,
messages=chat_history,
):
assert isinstance(msg, ChatResponseUpdate)
@@ -272,11 +274,12 @@ async def test_get_streaming_singular(
stream = MagicMock(spec=AsyncStream)
stream.__aiter__.return_value = [content1, content2]
mock_create.return_value = stream
chat_history.append(ChatMessage("user", ["hello world"]))
chat_history.append(ChatMessage(role="user", text="hello world"))
orig_chat_history = deepcopy(chat_history)
openai_chat_completion = OpenAIChatClient()
async for msg in openai_chat_completion.get_streaming_response(
async for msg in openai_chat_completion.get_response(
stream=True,
messages=chat_history,
):
assert isinstance(msg, ChatResponseUpdate)
@@ -311,14 +314,15 @@ async def test_get_streaming_structured_output_no_fcc(
stream = MagicMock(spec=AsyncStream)
stream.__aiter__.return_value = [content1, content2]
mock_create.return_value = stream
chat_history.append(ChatMessage("user", ["hello world"]))
chat_history.append(ChatMessage(role="user", text="hello world"))
# Define a mock response format
class Test(BaseModel):
name: str
openai_chat_completion = OpenAIChatClient()
async for msg in openai_chat_completion.get_streaming_response(
async for msg in openai_chat_completion.get_response(
stream=True,
messages=chat_history,
response_format=Test,
):
@@ -334,13 +338,14 @@ async def test_get_streaming_no_fcc_in_response(
openai_unit_test_env: dict[str, str],
):
mock_create.return_value = mock_streaming_chat_completion_response
chat_history.append(ChatMessage("user", ["hello world"]))
chat_history.append(ChatMessage(role="user", text="hello world"))
orig_chat_history = deepcopy(chat_history)
openai_chat_completion = OpenAIChatClient()
[
msg
async for msg in openai_chat_completion.get_streaming_response(
async for msg in openai_chat_completion.get_response(
stream=True,
messages=chat_history,
)
]
@@ -352,26 +357,6 @@ async def test_get_streaming_no_fcc_in_response(
)
@patch.object(AsyncChatCompletions, "create", new_callable=AsyncMock)
async def test_get_streaming_no_stream(
mock_create: AsyncMock,
chat_history: list[ChatMessage],
openai_unit_test_env: dict[str, str],
mock_chat_completion_response: ChatCompletion, # AsyncStream[ChatCompletionChunk]?
):
mock_create.return_value = mock_chat_completion_response
chat_history.append(ChatMessage("user", ["hello world"]))
openai_chat_completion = OpenAIChatClient()
with pytest.raises(ServiceResponseException):
[
msg
async for msg in openai_chat_completion.get_streaming_response(
messages=chat_history,
)
]
# region UTC Timestamp Tests
python/packages/core/tests/openai/test_openai_responses_client.py
+88 -109
View File
@@ -1,6 +1,5 @@
# Copyright (c) Microsoft. All rights reserved.
import asyncio
import base64
import json
import os
@@ -196,51 +195,48 @@ def test_serialize_with_org_id(openai_unit_test_env: dict[str, str]) -> None:
assert "User-Agent" not in dumped_settings.get("default_headers", {})
def test_get_response_with_invalid_input() -> None:
async def test_get_response_with_invalid_input() -> None:
"""Test get_response with invalid inputs to trigger exception handling."""
client = OpenAIResponsesClient(model_id="invalid-model", api_key="test-key")
# Test with empty messages which should trigger ServiceInvalidRequestError
with pytest.raises(ServiceInvalidRequestError, match="Messages are required"):
asyncio.run(client.get_response(messages=[]))
await client.get_response(messages=[])
def test_get_response_with_all_parameters() -> None:
async def test_get_response_with_all_parameters() -> None:
"""Test get_response with all possible parameters to cover parameter handling logic."""
client = OpenAIResponsesClient(model_id="test-model", api_key="test-key")
# Test with comprehensive parameter set - should fail due to invalid API key
with pytest.raises(ServiceResponseException):
asyncio.run(
client.get_response(
messages=[ChatMessage("user", ["Test message"])],
options={
"include": ["message.output_text.logprobs"],
"instructions": "You are a helpful assistant",
"max_tokens": 100,
"parallel_tool_calls": True,
"model_id": "gpt-4",
"previous_response_id": "prev-123",
"reasoning": {"chain_of_thought": "enabled"},
"service_tier": "auto",
"response_format": OutputStruct,
"seed": 42,
"store": True,
"temperature": 0.7,
"tool_choice": "auto",
"tools": [get_weather],
"top_p": 0.9,
"user": "test-user",
"truncation": "auto",
"timeout": 30.0,
"additional_properties": {"custom": "value"},
},
)
await client.get_response(
messages=[ChatMessage(role="user", text="Test message")],
options={
"include": ["message.output_text.logprobs"],
"instructions": "You are a helpful assistant",
"max_tokens": 100,
"parallel_tool_calls": True,
"model_id": "gpt-4",
"previous_response_id": "prev-123",
"reasoning": {"chain_of_thought": "enabled"},
"service_tier": "auto",
"response_format": OutputStruct,
"seed": 42,
"store": True,
"temperature": 0.7,
"tool_choice": "auto",
"tools": [get_weather],
"top_p": 0.9,
"user": "test-user",
"truncation": "auto",
"timeout": 30.0,
"additional_properties": {"custom": "value"},
},
)
def test_web_search_tool_with_location() -> None:
async def test_web_search_tool_with_location() -> None:
"""Test HostedWebSearchTool with location parameters."""
client = OpenAIResponsesClient(model_id="test-model", api_key="test-key")
@@ -258,15 +254,13 @@ def test_web_search_tool_with_location() -> None:
# Should raise an authentication error due to invalid API key
with pytest.raises(ServiceResponseException):
asyncio.run(
client.get_response(
messages=[ChatMessage("user", ["What's the weather?"])],
options={"tools": [web_search_tool], "tool_choice": "auto"},
)
await client.get_response(
messages=[ChatMessage(role="user", text="What's the weather?")],
options={"tools": [web_search_tool], "tool_choice": "auto"},
)
def test_file_search_tool_with_invalid_inputs() -> None:
async def test_file_search_tool_with_invalid_inputs() -> None:
"""Test HostedFileSearchTool with invalid vector store inputs."""
client = OpenAIResponsesClient(model_id="test-model", api_key="test-key")
@@ -275,15 +269,13 @@ def test_file_search_tool_with_invalid_inputs() -> None:
# Should raise an error due to invalid inputs
with pytest.raises(ValueError, match="HostedFileSearchTool requires inputs to be of type"):
asyncio.run(
client.get_response(
messages=[ChatMessage("user", ["Search files"])],
options={"tools": [file_search_tool]},
)
await client.get_response(
messages=[ChatMessage(role="user", text="Search files")],
options={"tools": [file_search_tool]},
)
def test_code_interpreter_tool_variations() -> None:
async def test_code_interpreter_tool_variations() -> None:
"""Test HostedCodeInterpreterTool with and without file inputs."""
client = OpenAIResponsesClient(model_id="test-model", api_key="test-key")
@@ -291,11 +283,9 @@ def test_code_interpreter_tool_variations() -> None:
code_tool_empty = HostedCodeInterpreterTool()
with pytest.raises(ServiceResponseException):
asyncio.run(
client.get_response(
messages=[ChatMessage("user", ["Run some code"])],
options={"tools": [code_tool_empty]},
)
await client.get_response(
messages=[ChatMessage(role="user", text="Run some code")],
options={"tools": [code_tool_empty]},
)
# Test code interpreter with files
@@ -304,15 +294,13 @@ def test_code_interpreter_tool_variations() -> None:
)
with pytest.raises(ServiceResponseException):
asyncio.run(
client.get_response(
messages=[ChatMessage("user", ["Process these files"])],
options={"tools": [code_tool_with_files]},
)
await client.get_response(
messages=[ChatMessage(role="user", text="Process these files")],
options={"tools": [code_tool_with_files]},
)
def test_content_filter_exception() -> None:
async def test_content_filter_exception() -> None:
"""Test that content filter errors in get_response are properly handled."""
client = OpenAIResponsesClient(model_id="test-model", api_key="test-key")
@@ -326,12 +314,12 @@ def test_content_filter_exception() -> None:
with patch.object(client.client.responses, "create", side_effect=mock_error):
with pytest.raises(OpenAIContentFilterException) as exc_info:
asyncio.run(client.get_response(messages=[ChatMessage("user", ["Test message"])]))
await client.get_response(messages=[ChatMessage(role="user", text="Test message")])
assert "content error" in str(exc_info.value)
def test_hosted_file_search_tool_validation() -> None:
async def test_hosted_file_search_tool_validation() -> None:
"""Test get_response HostedFileSearchTool validation."""
client = OpenAIResponsesClient(model_id="test-model", api_key="test-key")
@@ -340,15 +328,13 @@ def test_hosted_file_search_tool_validation() -> None:
empty_file_search_tool = HostedFileSearchTool()
with pytest.raises((ValueError, ServiceInvalidRequestError)):
asyncio.run(
client.get_response(
messages=[ChatMessage("user", ["Test"])],
options={"tools": [empty_file_search_tool]},
)
await client.get_response(
messages=[ChatMessage(role="user", text="Test")],
options={"tools": [empty_file_search_tool]},
)
def test_chat_message_parsing_with_function_calls() -> None:
async def test_chat_message_parsing_with_function_calls() -> None:
"""Test get_response message preparation with function call and result content types in conversation flow."""
client = OpenAIResponsesClient(model_id="test-model", api_key="test-key")
@@ -363,14 +349,14 @@ def test_chat_message_parsing_with_function_calls() -> None:
function_result = Content.from_function_result(call_id="test-call-id", result="Function executed successfully")
messages = [
ChatMessage("user", ["Call a function"]),
ChatMessage("assistant", [function_call]),
ChatMessage("tool", [function_result]),
ChatMessage(role="user", text="Call a function"),
ChatMessage(role="assistant", contents=[function_call]),
ChatMessage(role="tool", contents=[function_result]),
]
# This should exercise the message parsing logic - will fail due to invalid API key
with pytest.raises(ServiceResponseException):
asyncio.run(client.get_response(messages=messages))
await client.get_response(messages=messages)
async def test_response_format_parse_path() -> None:
@@ -391,7 +377,7 @@ async def test_response_format_parse_path() -> None:
with patch.object(client.client.responses, "parse", return_value=mock_parsed_response):
response = await client.get_response(
messages=[ChatMessage("user", ["Test message"])],
messages=[ChatMessage(role="user", text="Test message")],
options={"response_format": OutputStruct, "store": True},
)
assert response.response_id == "parsed_response_123"
@@ -418,7 +404,7 @@ async def test_response_format_parse_path_with_conversation_id() -> None:
with patch.object(client.client.responses, "parse", return_value=mock_parsed_response):
response = await client.get_response(
messages=[ChatMessage("user", ["Test message"])],
messages=[ChatMessage(role="user", text="Test message")],
options={"response_format": OutputStruct, "store": True},
)
assert response.response_id == "parsed_response_123"
@@ -441,7 +427,7 @@ async def test_bad_request_error_non_content_filter() -> None:
with patch.object(client.client.responses, "parse", side_effect=mock_error):
with pytest.raises(ServiceResponseException) as exc_info:
await client.get_response(
messages=[ChatMessage("user", ["Test message"])],
messages=[ChatMessage(role="user", text="Test message")],
options={"response_format": OutputStruct},
)
@@ -449,7 +435,7 @@ async def test_bad_request_error_non_content_filter() -> None:
async def test_streaming_content_filter_exception_handling() -> None:
"""Test that content filter errors in get_streaming_response are properly handled."""
"""Test that content filter errors in get_response(..., stream=True) are properly handled."""
client = OpenAIResponsesClient(model_id="test-model", api_key="test-key")
# Mock the OpenAI client to raise a BadRequestError with content_filter code
@@ -462,7 +448,7 @@ async def test_streaming_content_filter_exception_handling() -> None:
mock_create.side_effect.code = "content_filter"
with pytest.raises(OpenAIContentFilterException, match="service encountered a content error"):
response_stream = client.get_streaming_response(messages=[ChatMessage("user", ["Test"])])
response_stream = client.get_response(stream=True, messages=[ChatMessage(role="user", text="Test")])
async for _ in response_stream:
break
@@ -806,7 +792,7 @@ def test_prepare_message_for_openai_with_function_approval_response() -> None:
function_call=function_call,
)
message = ChatMessage("user", [approval_response])
message = ChatMessage(role="user", contents=[approval_response])
call_id_to_id: dict[str, str] = {}
result = client._prepare_message_for_openai(message, call_id_to_id)
@@ -828,7 +814,7 @@ def test_chat_message_with_error_content() -> None:
error_code="TEST_ERR",
)
message = ChatMessage("assistant", [error_content])
message = ChatMessage(role="assistant", contents=[error_content])
call_id_to_id: dict[str, str] = {}
result = client._prepare_message_for_openai(message, call_id_to_id)
@@ -853,7 +839,7 @@ def test_chat_message_with_usage_content() -> None:
}
)
message = ChatMessage("assistant", [usage_content])
message = ChatMessage(role="assistant", contents=[usage_content])
call_id_to_id: dict[str, str] = {}
result = client._prepare_message_for_openai(message, call_id_to_id)
@@ -1357,28 +1343,18 @@ async def test_end_to_end_mcp_approval_flow(span_exporter) -> None:
# Patch the create call to return the two mocked responses in sequence
with patch.object(client.client.responses, "create", side_effect=[mock_response1, mock_response2]) as mock_create:
# First call: get the approval request
response = await client.get_response(messages=[ChatMessage("user", ["Trigger approval"])])
response = await client.get_response(messages=[ChatMessage(role="user", text="Trigger approval")])
assert response.messages[0].contents[0].type == "function_approval_request"
req = response.messages[0].contents[0]
assert req.id == "approval-1"
# Build a user approval and send it (include required function_call)
approval = Content.from_function_approval_response(approved=True, id=req.id, function_call=req.function_call)
approval_message = ChatMessage("user", [approval])
approval_message = ChatMessage(role="user", contents=[approval])
_ = await client.get_response(messages=[approval_message])
# Ensure two calls were made and the second includes the mcp_approval_response
# After approval is processed, the model is called again to get the final response
assert mock_create.call_count == 2
_, kwargs = mock_create.call_args_list[1]
sent_input = kwargs.get("input")
assert isinstance(sent_input, list)
found = False
for item in sent_input:
if isinstance(item, dict) and item.get("type") == "mcp_approval_response":
assert item["approval_request_id"] == "approval-1"
assert item["approve"] is True
found = True
assert found
def test_usage_details_basic() -> None:
@@ -1616,10 +1592,10 @@ def test_streaming_annotation_added_with_unknown_type() -> None:
assert len(response.contents) == 0
def test_service_response_exception_includes_original_error_details() -> None:
async def test_service_response_exception_includes_original_error_details() -> None:
"""Test that ServiceResponseException messages include original error details in the new format."""
client = OpenAIResponsesClient(model_id="test-model", api_key="test-key")
messages = [ChatMessage("user", ["test message"])]
messages = [ChatMessage(role="user", text="test message")]
mock_response = MagicMock()
original_error_message = "Request rate limit exceeded"
@@ -1634,26 +1610,28 @@ def test_service_response_exception_includes_original_error_details() -> None:
patch.object(client.client.responses, "parse", side_effect=mock_error),
pytest.raises(ServiceResponseException) as exc_info,
):
asyncio.run(client.get_response(messages=messages, options={"response_format": OutputStruct}))
await client.get_response(messages=messages, options={"response_format": OutputStruct})
exception_message = str(exc_info.value)
assert "service failed to complete the prompt:" in exception_message
assert original_error_message in exception_message
def test_get_streaming_response_with_response_format() -> None:
"""Test get_streaming_response with response_format."""
async def test_get_response_streaming_with_response_format() -> None:
"""Test get_response streaming with response_format."""
client = OpenAIResponsesClient(model_id="test-model", api_key="test-key")
messages = [ChatMessage("user", ["Test streaming with format"])]
messages = [ChatMessage(role="user", text="Test streaming with format")]
# It will fail due to invalid API key, but exercises the code path
with pytest.raises(ServiceResponseException):
async def run_streaming():
async for _ in client.get_streaming_response(messages=messages, options={"response_format": OutputStruct}):
async for _ in client.get_response(
stream=True, messages=messages, options={"response_format": OutputStruct}
):
pass
asyncio.run(run_streaming())
await run_streaming()
def test_prepare_content_for_openai_image_content() -> None:
@@ -2090,7 +2068,7 @@ def test_parse_response_from_openai_image_generation_fallback():
async def test_prepare_options_store_parameter_handling() -> None:
client = OpenAIResponsesClient(model_id="test-model", api_key="test-key")
messages = [ChatMessage("user", ["Test message"])]
messages = [ChatMessage(role="user", text="Test message")]
test_conversation_id = "test-conversation-123"
chat_options = ChatOptions(store=True, conversation_id=test_conversation_id)
@@ -2116,7 +2094,7 @@ async def test_prepare_options_store_parameter_handling() -> None:
async def test_conversation_id_precedence_kwargs_over_options() -> None:
"""When both kwargs and options contain conversation_id, kwargs wins."""
client = OpenAIResponsesClient(model_id="test-model", api_key="test-key")
messages = [ChatMessage("user", ["Hello"])]
messages = [ChatMessage(role="user", text="Hello")]
# options has a stale response id, kwargs carries the freshest one
opts = {"conversation_id": "resp_old_123"}
@@ -2216,21 +2194,21 @@ async def test_integration_options(
check that the feature actually works correctly.
"""
openai_responses_client = OpenAIResponsesClient()
# to ensure toolmode required does not endlessly loop
openai_responses_client.function_invocation_configuration.max_iterations = 1
# Need at least 2 iterations for tool_choice tests: one to get function call, one to get final response
openai_responses_client.function_invocation_configuration["max_iterations"] = 2
for streaming in [False, True]:
# Prepare test message
if option_name.startswith("tools") or option_name.startswith("tool_choice"):
# Use weather-related prompt for tool tests
messages = [ChatMessage("user", ["What is the weather in Seattle?"])]
messages = [ChatMessage(role="user", text="What is the weather in Seattle?")]
elif option_name.startswith("response_format"):
# Use prompt that works well with structured output
messages = [ChatMessage("user", ["The weather in Seattle is sunny"])]
messages.append(ChatMessage("user", ["What is the weather in Seattle?"]))
messages = [ChatMessage(role="user", text="The weather in Seattle is sunny")]
messages.append(ChatMessage(role="user", text="What is the weather in Seattle?"))
else:
# Generic prompt for simple options
messages = [ChatMessage("user", ["Say 'Hello World' briefly."])]
messages = [ChatMessage(role="user", text="Say 'Hello World' briefly.")]
# Build options dict
options: dict[str, Any] = {option_name: option_value}
@@ -2241,13 +2219,13 @@ async def test_integration_options(
if streaming:
# Test streaming mode
response_gen = openai_responses_client.get_streaming_response(
response_stream = openai_responses_client.get_response(
stream=True,
messages=messages,
options=options,
)
output_format = option_value if option_name.startswith("response_format") else None
response = await ChatResponse.from_update_generator(response_gen, output_format_type=output_format)
response = await response_stream.get_final_response()
else:
# Test non-streaming mode
response = await openai_responses_client.get_response(
@@ -2295,7 +2273,7 @@ async def test_integration_web_search() -> None:
},
}
if streaming:
response = await ChatResponse.from_update_generator(client.get_streaming_response(**content))
response = await client.get_response(stream=True, **content).get_final_response()
else:
response = await client.get_response(**content)
@@ -2320,7 +2298,7 @@ async def test_integration_web_search() -> None:
},
}
if streaming:
response = await ChatResponse.from_update_generator(client.get_streaming_response(**content))
response = await client.get_response(stream=True, **content).get_final_response()
else:
response = await client.get_response(**content)
assert response.text is not None
@@ -2370,7 +2348,8 @@ async def test_integration_streaming_file_search() -> None:
file_id, vector_store = await create_vector_store(openai_responses_client)
# Test that the client will use the web search tool
response = openai_responses_client.get_streaming_response(
response = openai_responses_client.get_response(
stream=True,
messages=[
ChatMessage(
role="user",