* Fix agent_with_hosted_mcp sample to use AzureOpenAIResponsesClient (#4861)
The agent_with_hosted_mcp sample used AzureOpenAIChatClient with an MCP tool
dict, but the Chat Completions API only supports 'function' and 'custom' tool
types, not 'mcp'. This caused a 400 error at runtime.
Switch the sample to AzureOpenAIResponsesClient which natively supports MCP
tools via the Responses API. Use get_mcp_tool() to construct the tool config.
Changes:
- main.py: Replace AzureOpenAIChatClient with AzureOpenAIResponsesClient
- requirements.txt: Update azure-ai-agentserver-agentframework to 1.0.0b16
and use agent-framework-azure-ai package
- agent.yaml: Use AZURE_OPENAI_RESPONSES_DEPLOYMENT_NAME env var
- Add regression test documenting chat client MCP tool passthrough behavior
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: Fix agent_with_hosted_mcp sample to use Responses API client for MCP tools
Fixes#4861
* Remove REPRODUCTION_REPORT.md investigation artifact (#4861)
Remove the reproduction report markdown file from the test directory.
Investigation notes belong in the GitHub issue or PR description,
not as committed files in the source tree. The regression test in
test_openai_chat_client.py already provides automated verification.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add MCP tool API rejection regression test (#4861)
Add test_mcp_tool_dict_causes_api_rejection to verify that MCP tool
dicts passed through to the Chat Completions API result in a clear
ChatClientException rather than being silently dropped. This completes
the regression test coverage requested in code review.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* small fix
* Revert deletion of dotnet local.settings.json files
Restore the two local.settings.json files that were accidentally deleted in this PR.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: Remove unsupported memory scoping params from samples and docs
Fixes#4353
The `Mem0ContextProvider` and `RedisContextProvider` no longer support
`thread_id` or `scope_to_per_operation_thread_id` parameters. This commit
updates the affected samples and READMEs to use only the currently
supported API (`user_id`, `agent_id`, `application_id`).
Changes:
- mem0_sessions.py: Remove `thread_id` and
`scope_to_per_operation_thread_id` from examples 1 and 2, rewrite to
demonstrate user-scoped and agent-scoped memory patterns
- redis_sessions.py: Update module docstring to remove references to
removed thread scoping params
- mem0/README.md: Update Memory Scoping docs to reflect current API
- redis/README.md: Remove `thread_id` and
`scope_to_per_operation_thread_id` references from docs
* Address Copilot review: rename thread_scope functions, fix docstring
- Rename `example_global_thread_scope` -> `example_global_memory_scope`
- Rename `example_per_operation_thread_scope` -> `example_agent_scoped_memory`
- Update example 2 docstring to mention `application_id` alongside
`user_id` and `agent_id` since it's set in the provider config
- Update module docstring scenario 2 to include `application_id`
* fix: rebase onto main, address giles17 review feedback
- Resolve merge conflicts by rebasing all 4 original files onto current main
- Address giles17's agent review suggestions:
- mem0_basic.py: update comment to remove thread_id from scoping list
- mem0_oss.py: update comment to remove thread_id from scoping list
- redis_sessions.py: rename Example 2 from "Agent-Scoped Memory" to
"Hybrid Vector Search" to accurately describe what it demonstrates
- redis/README.md: update Example 2 description to match renamed example
---------
Co-authored-by: Tao Chen <taochen@microsoft.com>
Co-authored-by: Giles Odigwe <79032838+giles17@users.noreply.github.com>
* Add Python A2A agent-as-function-tools sample
Port of the .NET A2AAgent_AsFunctionTools sample to Python.
Resolves a remote A2A agent card, converts each skill to a
FunctionTool via as_tool(), and registers them with a host agent
using AzureOpenAIResponsesClient.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Sanitize A2A skill names before passing to as_tool()
as_tool() only auto-sanitizes when name is omitted. Since we pass
skill.name explicitly, we need to strip special characters ourselves.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: Add header_provider to MCPStreamableHTTPTool (#4808)
Add a header_provider callback parameter to MCPStreamableHTTPTool that
enables injecting dynamic per-request HTTP headers from runtime kwargs
(originating from FunctionInvocationContext.kwargs set in agent middleware).
The implementation uses contextvars and httpx event hooks to ensure headers
are task-local and safe for concurrent tool calls:
- header_provider receives the runtime kwargs dict and returns headers
- call_tool sets a ContextVar before delegating to MCPTool.call_tool
- An httpx request event hook reads from the ContextVar and injects headers
Example usage:
mcp_tool = MCPStreamableHTTPTool(
name="web-api",
url="https://api.example.com/mcp",
header_provider=lambda kwargs: {
"X-Auth-Token": kwargs.get("auth_token", ""),
},
)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address review feedback for #4808: Python: [Bug]: Unable to pass AgentContext to MCPStreamableHTTPTool
* Add test for header_provider via FunctionTool.invoke with FunctionInvocationContext
Addresses PR review comment: exercises the full pipeline from
FunctionInvocationContext.kwargs through FunctionTool.invoke to
MCPStreamableHTTPTool.call_tool and header_provider, rather than
testing call_tool in isolation.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address review feedback for #4808: review comment fixes
* Fix streamable MCP transport defaults
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix Azure AI test client mocks
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix MCP runtime kwarg regressions
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Stabilize MCP tool runtime kwargs
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Use context kwargs in MCP wrappers
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* updated mcp samples
* fix link
---------
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Eduard van Valkenburg
·
2026-03-31 17:23:49 +00:00
* Fix sample bugs: incorrect API params, wrong client types, and invalid options
- typed_options.py: Fix AnthropicClient model->model_id, wrap raw strings in Message objects for get_response(), fix reasoning_effort->reasoning dict, fix budget_tokens minimum (1024), use OpenAIChatClient not FoundryChatClient, remove unused import
- client_reasoning.py: Fix deprecated model_id to model param
- client_with_hosted_mcp.py: Remove invalid store=True kwarg from Agent.run()
- code_defined_skill.py: Fix precision kwarg to use function_invocation_kwargs
- Various other samples: Fix deprecated API usage and incorrect params
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address PR review comments
- client_with_hosted_mcp.py: Fix remaining store=True kwarg on line 68 to use options dict
- client_with_session.py: Change store=True to store=False to match in-memory persistence demo intent
- typed_options.py: Remove non-existent import and model key from docstring example
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* new sample fixes
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Foundry Evals integration for Python
Merged and refactored eval module per Eduard's PR review:
- Merge _eval.py + _local_eval.py into single _evaluation.py
- Convert EvalItem from dataclass to regular class
- Rename to_dict() to to_eval_data()
- Convert _AgentEvalData to TypedDict
- Simplify check system: unified async pattern with isawaitable
- Parallelize checks and evaluators with asyncio.gather
- Add all/any mode to tool_called_check
- Fix bool(passed) truthy bug in _coerce_result
- Remove deprecated function_evaluator/async_function_evaluator aliases
- Remove _MinimalAgent, tighten evaluate_agent signature
- Set self.name in __init__ (LocalEvaluator, FoundryEvals)
- Limit FoundryEvals to AsyncOpenAI only
- Type project_client as AIProjectClient
- Remove NotImplementedError continuous eval code
- Add evaluation samples in 02-agents/ and 03-workflows/
- Update all imports and tests (167 passing)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: resolve mypy redundant-cast errors while keeping pyright happy
Use cast(list[Any], x) with type: ignore[redundant-cast] comments to
satisfy both mypy (which considers casting Any redundant) and pyright
strict mode (which needs explicit casts to narrow Unknown types).
Also fix evaluator decorator check_name type annotation to be
explicitly str, resolving mypy str|Any|None mismatch.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: CI failures — pyupgrade, evaluator overloads, sample API, reset attr
- Apply pyupgrade: Sequence from collections.abc, remove forward-ref quotes
- Add @overload signatures to evaluator() for proper @evaluator usage
- Fix evaluate_workflow sample to use WorkflowBuilder(start_executor=) API
- Fix _workflow.py executor.reset() to use getattr pattern for pyright
- Remove unused EvalResults forward-ref string in default_factory lambda
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: skip gRPC-dependent observability test
The test_configure_otel_providers_with_env_file_and_vs_code_port test
triggers gRPC OTLP exporter creation, but the grpc dependency is
optional and not installed by default. Add skipif decorator matching
the pattern used by all other gRPC exporter tests in the same file.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: add nosec B101 for bandit assert check
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* style: align eval samples with repo conventions
- Move module docstrings before imports (after copyright header)
- Add -> None return type to all main() and helper functions
- Fix line-too-long in multiturn sample conversation data
- Add Workflow import for typed return in all_patterns_sample
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address PR review feedback: async fixes, sample bugs, deprecation warnings
- Simplify _ensure_async_result to direct await (async-only clients)
- Replace get_event_loop() with get_running_loop()
- Narrow _fetch_output_items exception handling to specific types
- Add warning log when _filter_tool_evaluators falls back to defaults
- Add DeprecationWarning to options alias in Agent.__init__
- Add DeprecationWarning to evaluate_response()
- Rename raw key to _raw_arguments in convert_message fallback
- Fix evaluate_agent_sample.py: replace evals.select() with FoundryEvals()
- Fix evaluate_multiturn_sample.py: use Message/Content/FunctionTool types
- Fix evaluate_workflow_sample.py: replace evals.select() with FoundryEvals()
- Update test mocks to use AsyncMock for awaited API calls
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add test coverage for review feedback items
- Add num_repetitions=2 positive test verifying 2×items and 4 agent calls
- Add _poll_eval_run tests: timeout, failed, and canceled paths
- Add evaluate_traces tests: validation error, response_ids path, trace_ids path
- Add evaluate_foundry_target happy-path test with target/query verification
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix ruff ISC004 lint error and apply formatter
- Wrap implicit string concatenation in parens in evaluate_multiturn_sample.py
- Apply ruff formatter to 6 other files with minor formatting drift
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Remove core type changes (extracted to fix/workflow-stale-session branch)
Reverts changes to _agents.py, _agent_executor.py, and _workflow.py
back to upstream/main. These fixes are now in a separate PR.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address PR review round 2: bugs, tests, and architecture
Code fixes:
- Fix _normalize_queries inverted condition (single query now replicates
to match expected_count)
- Fix substring match bug: 'end' in 'backend' matched; use exact set
lookup for executor ID filtering
- Fix used_available_tools sample: tool_definitions→tools param, use
FunctionTool attribute access instead of dict .get()
- Add None-check in _resolve_openai_client for misconfigured project
- Add Returns section to evaluate_workflow docstring
- Cache inspect.signature in @evaluator wrapper (avoid per-item reflection)
Architecture:
- Extract _evaluate_via_responses as module-level helper; evaluate_traces
now calls it directly instead of creating a FoundryEvals instance
- Move Foundry-specific typed-content conversion out of core to_eval_data;
core now returns plain role/content dicts, FoundryEvals applies
AgentEvalConverter in _evaluate_via_dataset
Tests:
- evaluate_response() deprecation warning emission and delegation
- num_repetitions > 1 with expected_output and expected_tool_calls
- Mock output_items.list in test_evaluate_calls_evals_api
- Update to_eval_data assertions for plain-dict format
- Unknown param error now raised at @evaluator decoration time
Skipped (separate PR): executor reset loop, xfail removal, options alias
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix CI: revert test_full_conversation, fix pyright errors
- Revert test_full_conversation.py to upstream/main (the session
preservation test was incorrectly changed to assert clearing)
- Fix pyright reportUnnecessaryComparison on get_openai_client() None
check by adding ignore comment
- Fix pyright reportPrivateUsage: add public EvalItem.split_messages()
method and use it in FoundryEvals._evaluate_via_dataset instead of
accessing private _split_conversation
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address PR review round 3: reliability, test gaps, cleanup
- Add try/except guard for non-numeric score in _coerce_result
- Add poll_interval minimum bound (0.1s) to prevent tight loops
- Add runtime async client check in _resolve_openai_client
- Remove _ensure_async_result wrapper (10 call sites → direct await)
- Better error message when queries provided without agent
- Import-time asserts for evaluator set consistency
- Remove 28 redundant @pytest.mark.asyncio decorators
- Add doc note about _raw_arguments sensitive data
- Tests: tool_called_check mode=any, _normalize_queries branches,
_extract_result_counts paths, _extract_per_evaluator, bare check
via evaluate_agent, output_items assertion, modulo wrapping,
async client check, queries-without-agent error
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix CI: ruff S101 assert, pyright and mypy arg-type errors
- Replace module-level assert with if/raise for evaluator set
consistency checks (ruff S101 disallows bare assert)
- Add type: ignore[arg-type] and pyright: ignore[reportArgumentType]
on OpenAI SDK evals API calls that pass dicts where typed params
are expected (SDK accepts dicts at runtime)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address PR review round 4: bugs, reliability, test fixes
- Fix all_passed ignoring parent result_counts when sub_results present
- Fix _extract_tool_calls: parse string arguments via json.loads before
falling back to None (real LLM responses use string arguments)
- Sanitize _raw_arguments to '[unparseable]' to avoid leaking sensitive
tool-call data to external evaluation services
- Add NOTE comment on to_eval_data message serialization dropping
non-text content (tool calls, results)
- Eliminate double conversation split in _evaluate_via_dataset: build
JSONL dicts directly from split_messages + AgentEvalConverter
- Raise poll_interval floor from 0.1s to 1.0s to prevent rate-limit
exhaustion
- Fix MagicMock(name=...) bug in test: sets display name not .name attr
- Fix mock_output_item.sample: use MagicMock object instead of dict so
_fetch_output_items exercises error/usage/input/output extraction
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address PR review round 5: reliability, docs, test coverage
Code fixes:
- Move import-time RuntimeError checks to unit tests (avoids breaking
imports for all users on developer set-drift mistake)
- _filter_tool_evaluators now raises ValueError when all evaluators
require tools but no items have tools (was silently substituting)
- Add poll_interval upper bound (60s) to prevent single-iteration sleep
- Log exc_info=True in _fetch_output_items for debugging API changes
- Fix evaluate() docstring: remove claim about Responses API optimization
- Validate target dict has 'type' key in evaluate_foundry_target
- Document to_eval_data() limitation: non-text content is omitted
Tests:
- TestEvaluatorSetConsistency: verify _AGENT/_TOOL subsets of _BUILTIN
- TestEvaluateTracesAgentId: agent_id-only path with lookback_hours
- TestFilterToolEvaluatorsRaises: ValueError on all-tool no-items
- TestEvaluateFoundryTargetValidation: target without 'type' key
- Assert items==[] on failed/canceled poll results
- Mock output_items.list in response_ids test for full flow
- TestAllPassedSubResults: result_counts=None + sub_results delegation
and parent failures override sub_results
- TestBuildOverallItemEmpty: empty workflow outputs returns None
Skipped r5-07 (_raw_arguments length hint): marginal debugging value,
could leak content size information.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix error message: evaluate_responses() → evaluate_traces(response_ids=...)
The referenced function doesn't exist; the correct API is
evaluate_traces(response_ids=...) from the azure-ai package.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Remove dead to_eval_data() method, fix docstring claims
- Remove to_eval_data() from EvalItem (dead code after r4-05 JSONL refactor)
- Migrate 15 tests from to_eval_data() to split_messages()
- Update sample to use split_messages() + Message properties
- Remove unimplemented Responses API optimization docstring claim
- Update split_messages() docstring to not reference removed method
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Reduce default eval timeout from 600s to 180s (3 minutes)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Remove dead _evaluate_via_responses method from FoundryEvals
The method was never called — evaluate() uses _evaluate_via_dataset,
and evaluate_traces() calls _evaluate_via_responses_impl directly.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Revert unrelated formatting changes to get-started samples
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix pyright: remove phantom FoundryMemoryProvider import, apply ruff format
- Remove import of non-existent _foundry_memory_provider module
(incorrectly kept during rebase conflict resolution)
- Apply ruff formatter to test_local_eval.py and get-started samples
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix eval samples: use FoundryChatClient for Agent()
The upstream provider-leading client refactor (#4818) made client=
a required parameter on Agent(). Update the three getting-started
eval samples to use FoundryChatClient with FOUNDRY_PROJECT_ENDPOINT,
matching the standard pattern from 01-get-started samples.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Simplify self-reflection sample using FoundryEvals
Replace ~80 lines of manual OpenAI evals API code (create_eval,
run_eval, manual polling, raw JSONL params) with FoundryEvals:
- evaluate_groundedness() uses FoundryEvals.evaluate() with EvalItem
- Remove create_openai_client(), create_eval(), run_eval() functions
- Remove openai SDK type imports (DataSourceConfigCustom, etc.)
- run_self_reflection_batch creates FoundryEvals instance once,
reuses it for all iterations across all prompts
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Update eval samples to FoundryChatClient and FOUNDRY_PROJECT_ENDPOINT
- Migrate all foundry_evals samples from AzureOpenAIResponsesClient to FoundryChatClient
- Update env var from AZURE_AI_PROJECT_ENDPOINT to FOUNDRY_PROJECT_ENDPOINT
- Use AzureCliCredential consistently across all samples
- Fix README.md: correct function names (evaluate_dataset -> FoundryEvals.evaluate, evaluate_responses -> evaluate_traces)
- Update self_reflection .env.example and README.md
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix lint errors in eval samples (E501, ASYNC240, formatting)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Remove evaluate_all_patterns_sample.py (redundant with focused samples)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix async credential mismatch: use azure.identity.aio for async AIProjectClient
AIProjectClient from azure.ai.projects.aio requires an async credential.
Switch all foundry_evals samples from azure.identity.AzureCliCredential
to azure.identity.aio.AzureCliCredential. Also pass project_client to
FoundryChatClient instead of duplicating endpoint+credential.
Close credential in self_reflection sample to avoid resource leak.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Revert test_observability.py to upstream/main (not our test)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address moonbox3 review: sphinx docstrings, pagination, isinstance check
- Convert all Example:: / Typical usage:: code blocks to .. code-block:: python
format matching codebase convention (both _evaluation.py and _foundry_evals.py)
- Add async pagination in _fetch_output_items via async for (handles large result sets)
- Replace hasattr(__aenter__) with isinstance(client, AsyncOpenAI) in _resolve_openai_client
- Move AsyncOpenAI import from TYPE_CHECKING to runtime (needed for isinstance)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix test failures and address remaining moonbox3 review comments
- Fix tests: use MagicMock(spec=AsyncOpenAI) for project_client mocks
(isinstance check now requires proper type, not duck-typing)
- Fix tests: replace mock_page.__iter__ with _AsyncPage helper for async for
- Fix evaluate_response: auto-extract queries from response messages when
query is not provided (previously always raised ValueError)
- Add debug logging when skipping internal _-prefixed executor IDs
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address Tao's PR review comments on Foundry Evals
- T1: Add comment explaining builtin.* pass-through in _resolve_evaluator
- T2: Add comment referencing OpenAI evals API for testing_criteria dict
- T3: Document Mustache-style {{item.*}} template placeholders
- T4: Document poll loop 60s sleep upper bound rationale
- T5: Narrow run type to RunRetrieveResponse, use typed field access
instead of vars()/getattr dance in _extract_result_counts and
_extract_per_evaluator; use run.error and run.report_url directly
- T6: Clarify openai_client docstring re: Azure Foundry endpoint
- T8: Remove misleading empty expected_tool_calls from sample
- Update tests to match real SDK PerTestingCriteriaResult shape
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Remove unnecessary Any union from run type annotations
RunRetrieveResponse is the correct type — no backward compat needed
for a brand new feature.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Accept FoundryChatClient instead of raw AsyncOpenAI
FoundryEvals now takes client: FoundryChatClient as its primary
parameter instead of openai_client: AsyncOpenAI. The builtin.*
evaluators require a Foundry endpoint, so the type should reflect that.
- FoundryEvals.__init__: client: FoundryChatClient replaces openai_client
- evaluate_traces / evaluate_foundry_target: same change
- _resolve_openai_client: extracts .client from FoundryChatClient
- project_client fallback retained for standalone functions
- All samples updated to construct FoundryChatClient and pass as client=
- Tests updated (openai_client= → client=)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Remove implicit 60s upper bound on poll interval
If a developer sets a higher poll_interval, respect it. Only clamp
to remaining time and enforce a 1s minimum for rate-limit protection.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Remove 1s floor on poll interval — let the developer control it
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Update python/samples/05-end-to-end/evaluation/foundry_evals/.env.example
Co-authored-by: Eduard van Valkenburg <eavanvalkenburg@users.noreply.github.com>
* Update python/samples/02-agents/evaluation/evaluate_agent.py
Co-authored-by: Eduard van Valkenburg <eavanvalkenburg@users.noreply.github.com>
* Address eavanvalkenburg review (round 2) on Python eval PR
- Rename model_deployment -> model across FoundryEvals and all samples
- Make model param optional, resolves from client.model
- Convert EvalResults from dataclass to regular class
- Remove deprecated evaluate_response() function
- Refactor splitters: BUILT_IN_SPLITTERS dict + standalone functions
- Change per_turn_items from classmethod to staticmethod
- Simplify EvalCheck type alias to use Awaitable[CheckResult]
- Remove errored property from EvalResults
- Remove default value from Evaluator protocol eval_name
- Rename assert_passed -> raise_for_status, add EvalNotPassedError
- Type agent param as SupportsAgentRun | None
- Fix Arguments docstring
- Update __init__.py exports
- Update all tests and samples
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Move FoundryEvals to foundry package, split tool eval sample
- Move _foundry_evals.py from azure-ai to foundry package
- Move test_foundry_evals.py to foundry/tests/
- Update lazy re-exports in agent_framework.foundry namespace
- Update .pyi type stubs
- All samples now import from agent_framework.foundry
- Split tool-call evaluation into evaluate_tool_calls_sample.py
- Fix all_passed to check errored count from result_counts
- Fix raise_for_status to include errored item details
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Auto-create FoundryChatClient from env vars when no client provided
FoundryEvals() now works zero-config when FOUNDRY_PROJECT_ENDPOINT and
FOUNDRY_MODEL environment variables are set. Auto-creates a FoundryChatClient
under the hood, matching the established env var pattern.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix pyright errors: remove dead _normalize_queries, suppress EvalAPIError check
- Remove unused _normalize_queries function and its tests
- Add pyright ignore for EvalAPIError None check (defensive guard)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Support multimodal image content in eval pipeline
Add image (data/uri) content handling to AgentEvalConverter.convert_message()
so that Content.from_data() and Content.from_uri() image payloads are
preserved as input_image parts in the Foundry evaluator format.
- Handle Content type='data' and type='uri' → emit input_image parts
- Add 6 unit tests for image content through convert_message/convert_messages
- Add integration test verifying images flow through EvalItem → JSONL path
- Add evaluate_multimodal.py sample demonstrating local image eval
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address remaining review comments
- Fix project_client docstring to say async-only (not sync/async)
- Add builtin evaluator name validation warning in _resolve_evaluator
- Replace getattr with typed attribute access in _poll_eval_run,
_extract_result_counts, _extract_per_evaluator, _fetch_output_items
- Remove cast import from _foundry_evals (no longer needed)
- Tighten _coerce_result: honour explicit 'passed' when both 'score'
and 'passed' are present; remove performative cast
- Fix self_reflection sample: add env file existence check
- Fix traces sample: correct Pattern 2 section label
- Update all Foundry eval samples to FoundryChatClient + FOUNDRY_MODEL
(remove AIProjectClient + AZURE_AI_MODEL_DEPLOYMENT_NAME pattern)
- Add eval_name and OpenAI client docs to FoundryEvals docstring
- Update test mocks to match typed SDK objects (_MockResultCounts)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix ruff lint errors (E501, SIM108, SIM102)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix pyright errors: type-narrow dict to dict[str, Any], add ignore comments
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Replace ConversationSplitter type alias with Protocol
ConversationSplitter is now a runtime-checkable Protocol with a named
'conversation' parameter, making the expected signature self-documenting.
ConversationSplit enum members gain a __call__ method so they satisfy
the protocol directly -- ConversationSplit.LAST_TURN(conversation) works.
This simplifies _split_conversation from an isinstance dispatch to a
single split(conversation) call.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Standardize on AZURE_AI_MODEL_DEPLOYMENT_NAME and fix Unicode in samples
- Replace FOUNDRY_MODEL with AZURE_AI_MODEL_DEPLOYMENT_NAME in all
eval samples to match repo convention
- Replace Unicode symbols with ASCII equivalents in all eval sample
print statements to avoid cp1252 encoding errors on Windows
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Update python/samples/03-workflows/evaluation/evaluate_workflow.py
Co-authored-by: Eduard van Valkenburg <eavanvalkenburg@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Eduard van Valkenburg <eavanvalkenburg@users.noreply.github.com>
* Rename ADR 0020 to 0023 (foundry evals integration)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: alliscode <bentho@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Eduard van Valkenburg <eavanvalkenburg@users.noreply.github.com>
* Python: Fix broken samples for GitHub Copilot, declarative, and Responses API
- Add missing on_permission_request handler to github_copilot_basic and
github_copilot_with_session samples (required by copilot SDK)
- Increase timeout for remote MCP query in github_copilot_with_mcp sample
- Soften session isolation claim in github_copilot_with_session sample
- Fix inline_yaml sample: pass project_endpoint via client_kwargs instead
of relying on YAML connection block (AzureAIClient expects
project_endpoint, not endpoint)
- Handle raw JSON schemas in Responses client _convert_response_format
so declarative outputSchema works with the Responses API
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Improve raw JSON schema detection heuristic and add tests
- Broaden raw schema detection to handle anyOf, oneOf, allOf, $ref, $defs
keywords and JSON Schema primitive types, not just 'properties'
- Apply same raw schema handling to azure-ai _shared.py for consistency
- Add unit tests for both openai and azure-ai response_format conversion
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Move ag_ui_workflow_handoff demo to 05-end-to-end (#4895)
Move the AG-UI workflow handoff demo from python/samples/demos/ to
python/samples/05-end-to-end/ to follow the current folder structure
convention. Update README paths accordingly.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix review feedback: remove build artifacts, fix README paths (#4895)
- Add .gitignore to frontend/ to exclude *.tsbuildinfo, vite.config.js,
and vite.config.d.ts build artifacts from version control
- Remove the 4 tracked build artifact files from the tree
- Fix step 2 cd path in README to be relative after 'cd python'
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Clarify working directory context in README Step 2 (#4895)
Step 2 uses a python/-relative path (samples/...) which assumes the
user is still in the python/ directory from Step 1. Add a brief note
making this explicit.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* [BREAKING] Refactor middleware layering and raw clients
Reorder chat client layers so function invocation wraps chat middleware, and chat middleware stays outside telemetry while still running for each inner model call. Add middleware pipeline caching, refresh docs and samples, and split Anthropic into raw and public clients to match the standard layering model.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Tighten typing ignores in ancillary modules
Add targeted typing ignores in workflow visualization and lab modules so pyright stays clean alongside the middleware refactor work.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix categorize_middleware to unpack tuple/Sequence and use relative MRO assertions
- Broaden isinstance check in categorize_middleware from list to Sequence
so tuples and other Sequence types are properly unpacked instead of
being appended as a single item.
- Replace fragile hardcoded MRO index assertions in anthropic test with
relative ordering via mro.index().
- Add regression tests for categorize_middleware with tuple, list, and
None inputs.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix middleware string decomposition, add middleware param to FunctionInvocationLayer, and add tests (#4710)
- Guard categorize_middleware Sequence check against str/bytes to prevent
character-by-character decomposition of accidentally passed strings
- Add explicit middleware parameter to FunctionInvocationLayer.get_response
and merge it into client_kwargs before categorization, fixing the
inconsistency where only OpenAIChatClient supported this parameter
- Add assertions that RawAnthropicClient does not inherit convenience layers
- Add chat middleware cache test with non-empty base middleware
- Add tests for single unwrapped middleware item and string input
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Apply pre-commit auto-fixes
* Apply pre-commit auto-fixes
* Address review feedback for #4710: review comment fixes
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <copilot@github.com>
Eduard van Valkenburg
·
2026-03-20 00:43:37 +00:00
* updated automation tasks and commands, with alias for the time being
* Restore aggregate test exclusions
Preserve the legacy all-tests scope for test --all by excluding lab and devui from the default aggregate sweep, while still allowing explicit package selection. Also ignore hidden/generated test directories such as .mypy_cache during aggregate discovery.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* updated versions in pre-commit
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Eduard van Valkenburg
·
2026-03-18 18:39:11 +00:00
* Add two hosted agent samples using the foundry agent
* Refactor formatting and improve readability in main.py
* Add agent-framework dependency to requirements and update copyright notice in main.py files
* Refactor agent imports and update credential handling in hosted agent samples
* Update agent framework dependency in requirements for hosted agents
* chore: update Python version to 3.14 and improve Dockerfile for hosted agents
* feat: add hosted agent samples for Azure AI with local tools and multi-agent workflows
* fix: update Azure AI client import and refactor agent initialization in hotel agent sample
* feat: add hosted agent samples for Seattle hotel search and writer-reviewer workflow
* fix: correct agent name in YAML configuration for local tools agent
* Python: clean up kwargs across agents, chat clients, tools, and sessions (#3642)
Audit and refactor public **kwargs usage across core agents, chat clients,
tools, sessions, and provider packages per the migration strategy codified
in CODING_STANDARD.md.
Key changes:
- Add explicit runtime buckets: function_invocation_kwargs and client_kwargs
on RawAgent.run() and chat client get_response() layers.
- Refactor FunctionTool to prefer explicit ctx: FunctionInvocationContext
injection; legacy **kwargs tools still work via _forward_runtime_kwargs.
- Refactor Agent.as_tool() to use direct JSON schema, always-streaming
wrapper, approval_mode parameter, and UserInputRequiredException
propagation (integrates PR #4568 behavior).
- Remove implicit session bleeding into FunctionInvocationContext; tools
that need a session must receive it via function_invocation_kwargs.
- Lower chat-client layers after FunctionInvocationLayer accept only
compatibility **kwargs (client_kwargs flattened, function_invocation_kwargs
ignored).
- Add layered docstring composition from Raw... implementations via
_docstrings.py helper.
- Clean up provider constructors to use explicit additional_properties.
- Deprecation warnings on legacy direct kwargs paths.
- Update samples, tests, and typing across all 23 packages.
Resolves#3642
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* clarified docstring
* feedback fixes
* Add unit tests for _docstrings.py build/apply helpers
Tests cover: no docstring source, no extra kwargs, appending to existing
Keyword Args section, inserting after Args, inserting in plain docstrings,
multiline descriptions, ordering, and apply_layered_docstring.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add test for propagate_session TypeError on non-AgentSession values
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add tests for multi-content and empty UserInputRequiredException propagation
Cover the branching logic in _try_execute_function_calls for:
- Multiple user_input_request items in a single exception (extra_user_input_contents path)
- Empty contents list (fallback function_result path)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add tests for DurableAIAgent.get_session forwarding service_session_id
Verifies get_session correctly forwards service_session_id and session_id
to the executor's get_new_session, replacing the removed kwargs test.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Simplify ag-ui test stub to read session from client_kwargs only
Remove dual-mode detection (client_kwargs vs raw kwargs fallback) from
the test mock. Session is now read exclusively from client_kwargs,
matching the settled public calling convention.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* updated create and get sessions in durable
* fixed docstrings
* fix test
* updated session handling
* updated from main
* updated tests
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Eduard van Valkenburg
·
2026-03-13 08:58:32 +00:00
* Implement annotation-based context compaction
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Handle missing compaction attributes in BaseChatClient
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix CI typing and bandit issues
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Optimize incremental compaction annotation pass
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* refinement
* Python: add ToolResultCompactionStrategy and CompactionProvider
Add ToolResultCompactionStrategy that collapses older tool-call groups
into short summary messages (e.g. [Tool calls: get_weather]) while
keeping the most recent groups verbatim. This mirrors the .NET
ToolResultCompactionStrategy from PR #4533.
Add CompactionProvider as a context-provider that auto-applies compaction
before each agent turn and stores compacted history in session state
after each turn.
Includes tests and samples for both features.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* refinement and alignment with dotnet PR
* updated tool result compaction
* updated tool result compaction
* Python: add ToolResultCompactionStrategy, CompactionProvider, and skip_excluded
- ToolResultCompactionStrategy collapses older tool-call groups into
[Tool results: func_name: result] summaries with bidirectional tracing
(same pattern as SummarizationStrategy).
- CompactionProvider as BaseContextProvider with separate before_strategy
and after_strategy parameters. before_strategy compacts loaded context;
after_strategy compacts stored history via history_source_id.
- InMemoryHistoryProvider gains skip_excluded flag to filter out messages
marked as excluded by compaction strategies.
- Tests, samples, and exports updated.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fixed checks
* fix mypy
* Fix: ensure summary messages from both strategies get full compaction annotations
SummarizationStrategy was not calling annotate_message_groups after
inserting its summary message, so the summary lacked core group
annotations (id, kind, index, has_reasoning, _excluded). Added the
missing call. ToolResultCompactionStrategy already had it.
Added tests verifying both strategies produce fully annotated summaries.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* updated propagation
* fix mypy
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Eduard van Valkenburg
·
2026-03-11 19:23:00 +00:00
* support skill scripts execution
* fix mixed line endings
* address comments and fix syntax issues
* use few try/except instead of one
* change samples
* validate either script path or script resource is set not both
* fix: separate LLM args from runtime kwargs in skill script execution
* address pr review comments
* address PR review comments
* Update python/packages/core/agent_framework/_skills.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update python/packages/core/agent_framework/_skills.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update python/packages/core/agent_framework/_skills.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* 1. Fixing the caching bug where parameters_schema would re-inspect on every call when the result was None
2. Updating the arguments tool description to be more generic (not CLI-specific)
* fix failing tests
* address pr review comments
* address pr review comments
* allow resource function returning any instead of sting
* address PR review comments
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Python: Add A2A server sample and fix client streaming bug
Add a pure Python A2A server sample so testing the A2A client no longer
requires running the .NET server. The server uses the a2a-sdk's
A2AStarletteApplication with uvicorn and supports three agent types
(invoice, policy, logistics) backed by AzureOpenAIResponsesClient.
New files:
- a2a_server.py: Main server entry point with CLI args
- agent_executor.py: Bridges a2a-sdk AgentExecutor to Agent Framework
- agent_definitions.py: Agent and AgentCard factory definitions
- invoice_data.py: Mock invoice data and query tool functions
- a2a_server.http: REST Client requests for testing
Also fixes a streaming bug in agent_with_a2a.py where async with was
used on ResponseStream which does not support the async context manager
protocol. Changed to async for to match all other samples.
Closes#4045
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address PR review: handle CancelledError and fix end_date filtering
- Re-raise asyncio.CancelledError before the broad exception handler
so cooperative cancellation is not swallowed.
- Make end_date filter inclusive of the full day by comparing with
< end + timedelta(days=1) instead of <= midnight.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The sample was passing raw strings in a list to get_response(), which
expects Message objects. This caused an AttributeError since strings
don't have a 'role' attribute.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add multi-turn streaming sample and rename multi-turn samples
- Rename 03_multi_turn.py to 03a_multi_turn.py
- Add 03b_multi_turn_streaming.py showing streaming with session history
- The new sample demonstrates calling get_final_response() after
iterating the stream to persist conversation history
- Update READMEs to reflect the new file names
Closes#4447
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Auto-finalize ResponseStream on iteration completion
When a ResponseStream is fully consumed via async iteration,
automatically trigger finalization (finalizer + result hooks).
This ensures session history is persisted in streaming multi-turn
conversations without requiring an explicit get_final_response() call.
- Add auto-finalize call in __anext__ on StopAsyncIteration
- Guard inner stream finalization to prevent double-execution
- Re-check _finalized after iteration in get_final_response()
- Add tests for auto-finalization and streaming session history
- Revert sample file renames from previous commit
Closes#4447
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* README fix
* Fix SIM102 lint: combine nested if statements
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Prepare azure-ai-projects 2.0 GA compatibility
Add allow_preview support for internal AIProjectClient creation, keep backward compatibility for renamed SDK model classes, and align Azure AI/core paths and tests for GA validation workflows.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* upgrade to ai-project==2.0.0
* Python: remove azure-ai-projects keyword-guard paths
Assume azure-ai-projects 2.0+ in Azure AI client/provider/responses code paths by removing _supports_keyword_argument gating and related fallback branching.
Also fix pyright typing in FoundryMemoryProvider memory store calls by using ResponseInputItemParam-typed items.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* check fixes
* Python: remove unsupported foundry_features option
Drop foundry_features from Azure AI client and provider surfaces because azure-ai-projects 2.0.0 does not expose that create_version parameter.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: add allow_preview to Foundry memory provider
Propagate allow_preview when FoundryMemoryProvider constructs an AIProjectClient and update tests accordingly.
Also finish wiring allow_preview through AzureAIClient-facing surfaces and related docs.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* aligning docstrings
* udpated lock
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Eduard van Valkenburg
·
2026-03-09 10:12:47 +00:00
* Update github_copilot package for github-copilot-sdk>=0.1.32 (#4549)
- Update requires-python from >=3.10 to >=3.11
- Remove Python 3.10 classifier
- Update mypy python_version to 3.11
- Update dependency to github-copilot-sdk>=0.1.32
- Fix ToolResult API: use snake_case kwargs (text_result_for_llm,
result_type) instead of camelCase (textResultForLlm, resultType)
- Update test assertions to use attribute access on ToolResult
- Add ToolResult type assertions to tool handler tests
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix tests to use ToolInvocation dataclass instead of plain dict (#4549)
Update test_github_copilot_agent.py to pass ToolInvocation objects to tool
handlers instead of plain dicts, matching the github-copilot-sdk>=0.1.32 API
where ToolInvocation is a dataclass with an .arguments attribute.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add regression tests for ToolInvocation contract (#4549)
Add tests to lock in the new ToolInvocation-based calling convention:
- test_tool_handler_rejects_raw_dict_invocation: verifies passing a raw
dict (old calling convention) raises TypeError/AttributeError
- test_tool_handler_with_empty_arguments: verifies ToolInvocation with
empty arguments works correctly for no-arg tools
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Revert requires-python to >=3.10 to avoid breaking CI (#4549)
The repo CI runs with Python 3.10 (uv sync --all-packages) and all other
packages require >=3.10. Raising this package to >=3.11 would break the
shared install flow. The SDK dependency version constraint (>=0.1.32) will
enforce any Python version requirement from the SDK itself.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix min Python version for github_copilot package to >=3.11
github-copilot-sdk>=0.1.32 requires Python>=3.11, which conflicts
with the package's declared >=3.10 minimum, breaking uv sync.
* Bump py version for GH workflows to 3.11, exclude GHCP sdk from 3.10 items
* Fix uv command
* Fixes
* Update samples
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: Add propagate_session parameter to as_tool() for session sharing
Add opt-in session propagation in agent-as-tool scenarios. When
propagate_session=True, the parent agent's AgentSession is forwarded
to the sub-agent's run() call, allowing both agents to share session
state (history, metadata, session_id).
- Add propagate_session parameter to BaseAgent.as_tool() (default False)
- Include session in additional_function_arguments so it flows to tools
- Add 3 tests for propagation on/off and shared state verification
- Add sample showing session propagation with observability middleware
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Clarify propagate_session docstring per review feedback
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Added shell tool
* Fixed CI error
* Add ShellTool support for OpenAI and Anthropic providers
- Add shell_tool_call, shell_tool_result, and shell_command_output content types
- Add ShellTool class and shell_tool decorator to core
- Add get_hosted_shell_tool() to OpenAI Responses client
- Handle shell_call and shell_call_output parsing in OpenAI (sync and streaming)
- Map ShellTool to Anthropic bash tool API format
- Parse bash_code_execution_tool_result as shell_tool_result in Anthropic
- Add unit tests for all new functionality
- Add sample scripts for hosted and local shell execution
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Addressed comments
* Reverted ruff change
* Fixed tests
* Addressed comments
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(python): use AgentResponse.value instead of model_validate_json in HITL sample
Since the agent is configured with response_format=GuessOutput, the
AgentResponse already provides .value with the parsed Pydantic model.
Using .value is more idiomatic and avoids redundant JSON parsing.
Fixes#4396
* fix: add safety guard for AgentResponse.value being None
Address Copilot review feedback: .value is optional and may be None
if response_format isn't propagated through the streaming path.
Add an explicit None check with a clear error message.
* Phase 2: Embedding clients for Ollama, Bedrock, and Azure AI Inference
Add embedding client implementations to existing provider packages:
- OllamaEmbeddingClient: Text embeddings via Ollama's embed API
- BedrockEmbeddingClient: Text embeddings via Amazon Titan on Bedrock
- AzureAIInferenceEmbeddingClient: Text and image embeddings via Azure AI
Inference, supporting Content | str input with separate model IDs for
text (AZURE_AI_INFERENCE_EMBEDDING_MODEL_ID) and image
(AZURE_AI_INFERENCE_IMAGE_EMBEDDING_MODEL_ID) endpoints
Additional changes:
- Rename EmbeddingCoT -> EmbeddingT, EmbeddingOptionsCoT -> EmbeddingOptionsT
- Add otel_provider_name passthrough to all embedding clients
- Register integration pytest marker in all packages
- Add lazy-loading namespace exports for Ollama and Bedrock embeddings
- Add image embedding sample using Cohere-embed-v3-english
- Add azure-ai-inference dependency to azure-ai package
Part of #1188
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix mypy duplicate name and ruff lint issues
- Rename second 'vector' variable to 'img_vector' in image embedding loop
- Combine nested with statements in tests
- Remove unused result assignments in tests
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* updates from feedback
* Fix CI failures in embedding usage handling
- Fix Azure AI embedding mypy issues by normalizing vectors to list[float],
safely accumulating optional usage token fields, and filtering None entries
before constructing GeneratedEmbeddings
- Avoid Bandit false positive by initializing usage details as an empty dict
- Update OpenAI embedding tests to assert canonical usage keys
(input_token_count/total_token_count)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Eduard van Valkenburg
·
2026-02-25 17:45:08 +00:00
Christian Glessner ·
Eduard van Valkenburg ·
Tao Chen ·
Giles Odigwe ·
L. Elaine Dazzio ·
Ben Thomas ·
Copilot ·
Evan Mattson ·
Hui Miao ·
Laveesh Rohra ·
SergeyMenshykh ·
Dmytro Struk ·
dependabot[bot] ·
Chris ·