* Add ADR-0020: Foundry Evals integration design
Captures the design for integrating Azure AI Foundry Evaluations with
agent-framework. Key decisions:
- EvalItem with conversation (list[Message]) as single source of truth
- query/response derived from configurable conversation split strategies
- Tools as list[FunctionTool] (including auto-extracted MCP tools)
- FoundryEvals provider with auto-detection of evaluator capabilities
- LocalEvaluator with @function_evaluator decorator for local checks
- Consistent Python/C# APIs: evaluate_agent, evaluate_workflow
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Mark ADR 0020 Foundry Evals as accepted
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: alliscode <bentho@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: clean up kwargs across agents, chat clients, tools, and sessions (#3642)
Audit and refactor public **kwargs usage across core agents, chat clients,
tools, sessions, and provider packages per the migration strategy codified
in CODING_STANDARD.md.
Key changes:
- Add explicit runtime buckets: function_invocation_kwargs and client_kwargs
on RawAgent.run() and chat client get_response() layers.
- Refactor FunctionTool to prefer explicit ctx: FunctionInvocationContext
injection; legacy **kwargs tools still work via _forward_runtime_kwargs.
- Refactor Agent.as_tool() to use direct JSON schema, always-streaming
wrapper, approval_mode parameter, and UserInputRequiredException
propagation (integrates PR #4568 behavior).
- Remove implicit session bleeding into FunctionInvocationContext; tools
that need a session must receive it via function_invocation_kwargs.
- Lower chat-client layers after FunctionInvocationLayer accept only
compatibility **kwargs (client_kwargs flattened, function_invocation_kwargs
ignored).
- Add layered docstring composition from Raw... implementations via
_docstrings.py helper.
- Clean up provider constructors to use explicit additional_properties.
- Deprecation warnings on legacy direct kwargs paths.
- Update samples, tests, and typing across all 23 packages.
Resolves#3642
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* clarified docstring
* feedback fixes
* Add unit tests for _docstrings.py build/apply helpers
Tests cover: no docstring source, no extra kwargs, appending to existing
Keyword Args section, inserting after Args, inserting in plain docstrings,
multiline descriptions, ordering, and apply_layered_docstring.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add test for propagate_session TypeError on non-AgentSession values
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add tests for multi-content and empty UserInputRequiredException propagation
Cover the branching logic in _try_execute_function_calls for:
- Multiple user_input_request items in a single exception (extra_user_input_contents path)
- Empty contents list (fallback function_result path)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add tests for DurableAIAgent.get_session forwarding service_session_id
Verifies get_session correctly forwards service_session_id and session_id
to the executor's get_new_session, replacing the removed kwargs test.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Simplify ag-ui test stub to read session from client_kwargs only
Remove dual-mode detection (client_kwargs vs raw kwargs fallback) from
the test mock. Session is now read exclusively from client_kwargs,
matching the settled public calling convention.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* updated create and get sessions in durable
* fixed docstrings
* fix test
* updated session handling
* updated from main
* updated tests
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Implement annotation-based context compaction
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Handle missing compaction attributes in BaseChatClient
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix CI typing and bandit issues
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Optimize incremental compaction annotation pass
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* refinement
* Python: add ToolResultCompactionStrategy and CompactionProvider
Add ToolResultCompactionStrategy that collapses older tool-call groups
into short summary messages (e.g. [Tool calls: get_weather]) while
keeping the most recent groups verbatim. This mirrors the .NET
ToolResultCompactionStrategy from PR #4533.
Add CompactionProvider as a context-provider that auto-applies compaction
before each agent turn and stores compacted history in session state
after each turn.
Includes tests and samples for both features.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* refinement and alignment with dotnet PR
* updated tool result compaction
* updated tool result compaction
* Python: add ToolResultCompactionStrategy, CompactionProvider, and skip_excluded
- ToolResultCompactionStrategy collapses older tool-call groups into
[Tool results: func_name: result] summaries with bidirectional tracing
(same pattern as SummarizationStrategy).
- CompactionProvider as BaseContextProvider with separate before_strategy
and after_strategy parameters. before_strategy compacts loaded context;
after_strategy compacts stored history via history_source_id.
- InMemoryHistoryProvider gains skip_excluded flag to filter out messages
marked as excluded by compaction strategies.
- Tests, samples, and exports updated.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fixed checks
* fix mypy
* Fix: ensure summary messages from both strategies get full compaction annotations
SummarizationStrategy was not calling annotate_message_groups after
inserting its summary message, so the summary lacked core group
annotations (id, kind, index, has_reasoning, _excluded). Added the
missing call. ToolResultCompactionStrategy already had it.
Added tests verifying both strategies produce fully annotated summaries.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* updated propagation
* fix mypy
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Initial plan
* Fix broken Strands Agents documentation links in ADR 0001
Replace 5 broken strandsagents.com URLs (returning 404) with stable
GitHub source code links in docs/decisions/0001-agent-run-response.md.
The Strands Agents docs site restructured from /api-reference/python/
to /api/python/, breaking the old links.
Co-authored-by: SergeyMenshykh <68852919+SergeyMenshykh@users.noreply.github.com>
* Update Strands Agents links to use official documentation site
Replace GitHub source links with official strandsagents.com/docs/api/python/
documentation URLs in docs/decisions/0001-agent-run-response.md.
Co-authored-by: SergeyMenshykh <68852919+SergeyMenshykh@users.noreply.github.com>
* Update Strands Agents links to use specific documentation URLs
- Streaming: strandsagents.com/docs/user-guide/concepts/streaming/
- Structured output: strandsagents.com/docs/user-guide/concepts/agents/structured-output/
- AgentResult/stop_reason: strandsagents.com/docs/api/python/strands.agent.agent_result/#agentresult
Co-authored-by: SergeyMenshykh <68852919+SergeyMenshykh@users.noreply.github.com>
* Deduplicate Strands AgentResult link in stop-reason row
Replaced the duplicate hyperlink on `stop_reason` with inline code,
keeping a single AgentResult link to the same URL.
Co-authored-by: SergeyMenshykh <68852919+SergeyMenshykh@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: SergeyMenshykh <68852919+SergeyMenshykh@users.noreply.github.com>
* Phase 2: Embedding clients for Ollama, Bedrock, and Azure AI Inference
Add embedding client implementations to existing provider packages:
- OllamaEmbeddingClient: Text embeddings via Ollama's embed API
- BedrockEmbeddingClient: Text embeddings via Amazon Titan on Bedrock
- AzureAIInferenceEmbeddingClient: Text and image embeddings via Azure AI
Inference, supporting Content | str input with separate model IDs for
text (AZURE_AI_INFERENCE_EMBEDDING_MODEL_ID) and image
(AZURE_AI_INFERENCE_IMAGE_EMBEDDING_MODEL_ID) endpoints
Additional changes:
- Rename EmbeddingCoT -> EmbeddingT, EmbeddingOptionsCoT -> EmbeddingOptionsT
- Add otel_provider_name passthrough to all embedding clients
- Register integration pytest marker in all packages
- Add lazy-loading namespace exports for Ollama and Bedrock embeddings
- Add image embedding sample using Cohere-embed-v3-english
- Add azure-ai-inference dependency to azure-ai package
Part of #1188
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix mypy duplicate name and ruff lint issues
- Rename second 'vector' variable to 'img_vector' in image embedding loop
- Combine nested with statements in tests
- Remove unused result assignments in tests
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* updates from feedback
* Fix CI failures in embedding usage handling
- Fix Azure AI embedding mypy issues by normalizing vectors to list[float],
safely accumulating optional usage token fields, and filtering None entries
before constructing GeneratedEmbeddings
- Avoid Bandit false positive by initializing usage details as an empty dict
- Update OpenAI embedding tests to assert canonical usage keys
(input_token_count/total_token_count)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* updated integration tests and guidance
* fixed merge test
* updated integration tests
* fix: remove duplicate --dist loadfile flag from pytest-xdist config
Only one --dist mode can be active at a time; the second value silently
overrides the first. Keep --dist worksteal (dynamic load balancing) and
remove the redundant --dist loadfile from all workflow files and
pyproject.toml configs.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* docs: add keep-in-sync notes for merge and integration test workflows
Both python-merge-tests.yml and python-integration-tests.yml share the
same parallel job structure. Added sync reminders in workflow file
comments, the python-testing SKILL.md, and CODING_STANDARD.md.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* refactor: remove RUN_INTEGRATION_TESTS flag
Integration test gating now uses two mechanisms:
- `@pytest.mark.integration` for test selection via `-m` filtering
- `skip_if_*_disabled` for credential/service availability checks
The RUN_INTEGRATION_TESTS env var was redundant since the marker handles
selection and the skip decorators already check for actual credentials.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: sync missing env vars from merge-tests to integration-tests
Add OPENAI_EMBEDDINGS_MODEL_ID and AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME
to python-integration-tests.yml to match python-merge-tests.yml.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: remove remaining RUN_INTEGRATION_TESTS from embedding tests and docs
Missed test_openai_embedding_client.py and vector-stores README in the
earlier cleanup.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* set functions tests to 3.10
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat(python): Add embedding abstractions and OpenAI implementation (Phase 1)
This PR contains two parts:
1. **Overall migration plan** for porting vector stores and embeddings from
Semantic Kernel to Agent Framework (docs/features/vector-stores-and-embeddings/README.md)
covering all 10 phases from core abstractions through connectors and TextSearch.
2. **Phase 1 implementation** — core embedding abstractions and OpenAI/Azure OpenAI
embedding clients:
Core types (_types.py):
- EmbeddingGenerationOptions TypedDict (total=False)
- Embedding[EmbeddingT] generic class with model_id, dimensions, created_at
- GeneratedEmbeddings[EmbeddingT, EmbeddingOptionsT] list container with options, usage
- EmbeddingInputT (default str) and EmbeddingT (default list[float]) TypeVars
Protocol + base class (_clients.py):
- SupportsGetEmbeddings protocol — Generic[EmbeddingInputT, EmbeddingT, OptionsContraT]
- BaseEmbeddingClient ABC — Generic[EmbeddingInputT, EmbeddingT, OptionsCoT]
Telemetry (observability.py):
- EmbeddingTelemetryLayer with gen_ai.operation.name = "embeddings"
OpenAI implementation (openai/_embedding_client.py):
- RawOpenAIEmbeddingClient, OpenAIEmbeddingClient, OpenAIEmbeddingOptions
- Uses _ensure_client() factory pattern
Azure OpenAI implementation (azure/_embedding_client.py):
- AzureOpenAIEmbeddingClient following AzureOpenAIChatClient pattern
- Supports API key, Entra ID credentials, env var configuration
Tests:
- 47 unit tests for types, protocol, base class, OpenAI, and Azure clients
- 6 integration tests (gated behind RUN_INTEGRATION_TESTS + credentials)
Samples:
- samples/02-agents/embeddings/openai_embeddings.py
- samples/02-agents/embeddings/azure_openai_embeddings.py
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: Add AzureOpenAIEmbeddingClient to azure __init__.pyi stub
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* ci: Add embedding env vars to Python integration tests
Map OPENAI_EMBEDDING_MODEL_ID and AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME
from GitHub vars to the integration test environment.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: Handle base64 encoding_format in OpenAI embedding client
When encoding_format='base64' is used, the OpenAI API returns base64-encoded
floats instead of a JSON array. Decode these automatically to list[float]
so the return type stays consistent regardless of encoding format.
Also adds a unit test for base64 decoding and fixes minor docstring/import issues.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: Only record INPUT_TOKENS for embedding telemetry
Embeddings have no output/completion tokens. Remove OUTPUT_TOKENS recording
which was double-counting prompt_tokens via the total_tokens fallback.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: Resolve mypy variance error and lint warning
Use contravariant/covariant TypeVars for SupportsGetEmbeddings Protocol.
Combine nested if into single statement in telemetry layer.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: Make EmbeddingCoT invariant for mypy compatibility
GeneratedEmbeddings is invariant in its type param, so the Protocol
TypeVar cannot be covariant.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: Address PR review - empty values guard, service_url for telemetry
- Add early return for empty values in get_embeddings to avoid unnecessary API calls
- Add service_url() method to RawOpenAIEmbeddingClient for proper telemetry endpoint reporting
- Add test for empty values behavior
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: Fix OpenAI chat client compatibility with third-party endpoints and OTel 0.4.14 (#4161)
* Fix system message content sent as list instead of string
Some OpenAI-compatible endpoints (e.g. NVIDIA NIM) reject system messages
when content is a list of content parts. This change flattens system and
developer message content to a plain string in the Chat Completions client.
Fixes https://github.com/microsoft/agent-framework/issues/1407
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix compatibility with opentelemetry-semantic-conventions-ai 0.4.14
Version 0.4.14 removed several LLM_* attributes from SpanAttributes
(LLM_SYSTEM, LLM_REQUEST_MODEL, LLM_RESPONSE_MODEL, LLM_REQUEST_MAX_TOKENS,
LLM_REQUEST_TEMPERATURE, LLM_REQUEST_TOP_P, LLM_TOKEN_TYPE).
Move these to the OtelAttr enum with their well-known gen_ai.* string values
and update all references in observability.py and tests.
Fixes https://github.com/microsoft/agent-framework/issues/4160
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Flatten text-only message content to string for all roles
Extend the system/developer fix to all message roles. Text-only content
lists are now post-processed into plain strings, while multimodal content
(text + images/audio) remains as a list. This fixes compatibility with
OpenAI-like endpoints that cannot deserialize list content (e.g. Foundry
Local's Neutron backend).
Partially fixes https://github.com/microsoft/agent-framework/issues/4084
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix streaming text lost when usage data in same chunk
Some providers (e.g. Gemini) include both usage data and text content
in the same streaming chunk. The early return on chunk.usage caused
text and tool call parsing to be skipped entirely. Remove the early
return and process usage alongside text/tool calls.
Fixes https://github.com/microsoft/agent-framework/issues/3434
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix mypy errors in _chat_client.py
Rename shadowed variable 'args' in system/developer branch to 'sys_args'
and rename loop variable 'content' to 'msg_content' to avoid type conflict.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* reorder imports
* fix: Use OtelAttr.REQUEST_MODEL instead of removed SpanAttributes.LLM_REQUEST_MODEL
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* docs: Add score_threshold to vector store plan
Reference SK .NET PR #13501 for score threshold filtering semantics.
Include score_threshold in SearchOptions from Phase 3.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* docs: Add reference to roji's SK .NET MEVD work for SQL connectors
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: Clear env vars in construction tests to avoid CI leakage
Tests for missing API key / model ID now use monkeypatch.delenv to ensure
env vars from the integration test environment don't prevent the expected
ValueError from being raised.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* restructure: Python samples into progressive 01-05 layout
- 01-get-started/: 6 numbered steps (hello agent → hosting)
- 02-agents/: all agent concept samples (tools, middleware, providers, etc.)
- 03-workflows/: ALL existing workflow samples preserved as-is
- 04-hosting/: azure-functions, durabletask, a2a
- 05-end-to-end/: demos, evaluation, hosted agents
- Old files moved to _to_delete/ for review
- Added AGENTS.md with structure documentation
- autogen-migration/ and semantic-kernel-migration/ preserved at root
* fix: switch to AzureOpenAI Foundry, fix CI failures
- Switch all 01-get-started samples to AzureOpenAIResponsesClient with
Azure AI Foundry project endpoint (AZURE_AI_PROJECT_ENDPOINT +
AZURE_OPENAI_RESPONSES_DEPLOYMENT_NAME + AzureCliCredential)
- Add _to_delete/ and 05-end-to-end/ to pyrightconfig.samples.json excludes
- Fix test paths in packages/ that referenced old getting_started/ dirs:
durabletask conftest + streaming test, azurefunctions conftest,
devui conftest + capture_messages + openai_sdk_integration
- Fix workflow_as_agent_human_in_the_loop.py import (sibling import)
- Update hosting READMEs and tool comment paths
- Replace root README.md with new structure overview
- Update AGENTS.md to document Azure OpenAI Foundry as default provider
* cleanup: remove _to_delete folder, copy resource files to active dirs
All files in _to_delete/ were either:
- Exact duplicates of files in the new structure (240 files)
- Same file with only comment path updates (100 files)
- One import-fix diff (workflow_as_agent_human_in_the_loop.py)
- One superseded minimal_sample.py
Resource files (sample.pdf, countries.json, employees.pdf, weather.json)
copied to 02-agents/sample_assets/ and 02-agents/resources/ since active
samples reference them.
* fix: address PR review comments, centralize resources, remove root duplicates
- Fix type annotation in 04_memory.py (string union -> proper types)
- Fix old sample paths in observability files
- Fix grammar/spelling in observability samples
- Move sample_assets/ and resources/ to shared/ folder
- Remove 8 duplicate observability files from 02-agents root
- Update resource path references in multimodal_input and provider samples
* fix: update broken links from old getting_started paths to new structure
- Update relative paths in READMEs: getting_started/ → 01-get-started/,
02-agents/, 03-workflows/, 04-hosting/, 05-end-to-end/
- Fix absolute GitHub URLs in package READMEs
- Fix broken link in ollama package README
* fix: convert absolute GitHub URLs to relative paths for link checker
Absolute URLs to python/samples/ on main branch 404 until PR merges.
Converted to relative paths that linkspector can verify locally.
* fix: update link for handoff sample moved to orchestrations/
* fix: update chatkit-integration README path from demos/ to 05-end-to-end/
* fix: update broken links in orchestrations README to match flat directory structure
* Add ADR for Python ContextMiddleware unification
* Add session serialization/deserialization design to ADR
* Add Related Issues section mapping to ADR
* Update session management: create_session, get_session_by_id, agent.serialize_session
* ADR: Add hooks alternative, context compaction discussion, and PR feedback
- Add Option 3: ContextHooks with before_run/after_run pattern
- Add detailed pros/cons for both wrapper and hooks approaches
- Add Open Discussion section on context compaction strategies
- Clarify response_messages is read-only (use AgentMiddleware for modifications)
- Add SimpleRAG examples showing input-only filtering
- Clarify default storage only added when NO middleware configured
- Add RAGWithBuffer examples for self-managed history
- Rename hook methods to before_run/after_run
* ADR: Restructure and add .NET comparison
- Add class hierarchy clarification for both options
- Merge detailed design sections (side-by-side comparison)
- Move detailed design before decision outcome
- Move compaction discussion after decision
- Add .NET implementation comparison (feature equivalence)
- Update .NET method names to match actual implementation
- Rename hook methods to before_run/after_run
- Fix storage context table for injected context
* tweaks
* fix smart load
* ADR: Add naming discussion note for ContextHooks
- Note that class and method names are open for discussion
- Add alternative method naming options table
- Include invoking/invoked as option matching current Python and .NET
* Update context middleware design: remove smart mode, add attribution filtering
- Remove smart mode for load_messages (now explicit bool, default True)
- Add attribution marker in additional_properties for message filtering
- Update validation to warn on multiple or zero storage loaders
- Add note about ChatReducer naming from .NET
- Note that attribution should not be propagated to storage
* Add Decision 2: Instance Ownership (instances in session vs agent)
- Option A: Instances in Session (current proposal)
- Option B: Instances in Agent, State in Session
- B1: Simple dict state with optional return
- B2: SessionState object with mutable wrapper
- Updated examples to use Hooks pattern (before_run/after_run)
- Added open discussion on hook factories in Option B model
* Update ADR: Choose ContextPlugin with before_run/after_run and Option B1
Decision outcomes:
- Option 3 (Hooks pattern) with ContextPlugin class name
- Methods: before_run/after_run
- Option B1: Instances in Agent, State in Session (simple dict)
- Whole state dict passed to plugins (mutable, no return needed)
- Added trust note: plugins reason over messages, so they're trusted by default
Status changed from proposed to accepted.
* Add agent and session params to before_run/after_run methods
Signature now: before_run(agent, session, context, state)
* Remove ContextPluginRunner, store plugins directly on agent
Simpler design: agent stores Sequence[ContextPlugin] and calls
before_run/after_run directly in the run method.
* Update workplan to 2 PRs for simpler review
* updated doc
* Refine ADR: serialization, ownership, decorators, session methods, exports
- Add to_dict()/from_dict() on AgentSession with 'type' discriminator
- Present serialization as Option A (direct) vs Option B (through agent)
- Rewrite ownership section as 2x2 matrix (orthogonal decision)
- Move Instance Ownership Options before Decision Outcome
- Fix get_session to use service_session_id, split from create_session
- Add decorator-based provider convenience API (@before_run/@after_run)
- Add _ prefix naming strategy for all PR1 types (core + external)
- Constructor compatibility table for existing providers
- Add load_messages=False skip logic to all agent run loops
- Clarify abstract vs non-abstract in execution pattern samples
- Update auto-provision: trigger on conversation_id or store=True
- Document root package exports (ContextProvider, HistoryProvider, etc.)
- Rename section heading to 'Key Design Considerations'
* Rename ADR to 0016-python-context-middleware.md
* Fix broken link: #3-unified-storage-middleware → #3-unified-storage
* WIP
* big update to new ResponseStream model
* fixed tests and typing
* fixed tests and typing
* fixed tools typevar import
* fix
* mypy fix
* mypy fixes and some cleanup
* fix missing quoted names
* and client
* fix imports agui
* fix anthropic override
* fix agui
* fix ag ui
* fix import
* fix anthropic types
* fix mypy
* refactoring
* updated typing
* fix 3.11
* fixes
* redid layering of chat clients and agents
* redid layering of chat clients and agents
* Fix lint, type, and test issues after rebase
- Add @overload decorators to AgentProtocol.run() for type compatibility
- Add missing docstring params (middleware, function_invocation_configuration)
- Fix TODO format (TD002) by adding author tags
- Fix broken observability tests from upstream:
- Replace non-existent use_instrumentation with direct instantiation
- Replace non-existent use_agent_instrumentation with AgentTelemetryLayer mixin
- Fix get_streaming_response to use get_response(stream=True)
- Add AgentInitializationError import
- Update streaming exception tests to match actual behavior
* Fix AgentExecutionException import error in test_agents.py
- Replace non-existent AgentExecutionException with AgentRunException
* Fix test import and asyncio deprecation issues
- Add 'tests' to pythonpath in ag-ui pyproject.toml for utils_test_ag_ui import
- Replace deprecated asyncio.get_event_loop().run_until_complete with asyncio.run
* Fix azure-ai test failures
- Update _prepare_options patching to use correct class path
- Fix test_to_azure_ai_agent_tools_web_search_missing_connection to clear env vars
* Convert ag-ui utils_test_ag_ui.py to conftest.py
- Move test utilities to conftest.py for proper pytest discovery
- Update all test imports to use conftest instead of utils_test_ag_ui
- Remove old utils_test_ag_ui.py file
- Revert pythonpath change in pyproject.toml
* fix: use relative imports for ag-ui test utilities
* fix agui
* Rename Bare*Client to Raw*Client and BaseChatClient
- Renamed BareChatClient to BaseChatClient (abstract base class)
- Renamed BareOpenAIChatClient to RawOpenAIChatClient
- Renamed BareOpenAIResponsesClient to RawOpenAIResponsesClient
- Renamed BareAzureAIClient to RawAzureAIClient
- Added warning docstrings to Raw* classes about layer ordering
- Updated README in samples/getting_started/agents/custom with layer docs
- Added test for span ordering with function calling
* Fix layer ordering: FunctionInvocationLayer before ChatTelemetryLayer
This ensures each inner LLM call gets its own telemetry span, resulting in
the correct span sequence: chat -> execute_tool -> chat
Updated all production clients and test mocks to use correct ordering:
- ChatMiddlewareLayer (first)
- FunctionInvocationLayer (second)
- ChatTelemetryLayer (third)
- BaseChatClient/Raw...Client (fourth)
* Remove run_stream usage
* Fix conversation_id propagation
* Python: Add BaseAgent implementation for Claude Agent SDK (#3509)
* Added ClaudeAgent implementation
* Updated streaming logic
* Small updates
* Small update
* Fixes
* Small fix
* Naming improvements
* Updated imports
* Addressed comments
* Updated package versions
* Update Claude agent connector layering
* fix test and plugin
* Store function middleware in invocation layer
* Fix telemetry streaming and ag-ui tests
* Remove legacy ag-ui tests folder
* updates
* Remove terminate flag from FunctionInvocationContext, use MiddlewareTermination instead
- Remove terminate attribute from FunctionInvocationContext
- Add result attribute to MiddlewareTermination to carry function results
- FunctionMiddlewarePipeline.execute() now lets MiddlewareTermination propagate
- _auto_invoke_function captures context.result in exception before re-raising
- _try_execute_function_calls catches MiddlewareTermination and sets should_terminate
- Fix handoff middleware to append to chat_client.function_middleware directly
- Update tests to use raise MiddlewareTermination instead of context.terminate
- Add middleware flow documentation in samples/concepts/tools/README.md
- Fix ag-ui to use FunctionMiddlewarePipeline instead of removed create_function_middleware_pipeline
* fix: remove references to removed terminate flag in purview tests, add type ignore
* fix: move _test_utils.py from package to test folder
* fix: call get_final_response() to trigger context provider notification in streaming test
* fix: correct broken links in tools README
* docs: clarify default middleware behavior in summary table
* fix: ensure inner stream result hooks are called when using map()/from_awaitable()
* Fix mypy type errors
* Address PR review comments on observability.py
- Remove TODO comment about unconsumed streams, add explanatory note instead
- Remove redundant _close_span cleanup hook (already called in _finalize_stream)
- Clarify behavior: cleanup hooks run after stream iteration, if stream is not
consumed the span remains open until garbage collected
* Remove gen_ai.client.operation.duration from span attributes
Duration is a metrics-only attribute per OpenTelemetry semantic conventions.
It should be recorded to the histogram but not set as a span attribute.
* Remove duration from _get_response_attributes, pass directly to _capture_response
Duration is a metrics-only attribute. It's now passed directly to _capture_response
instead of being included in the attributes dict that gets set on the span.
* Remove redundant _close_span cleanup hook in AgentTelemetryLayer
_finalize_stream already calls _close_span() in its finally block,
so adding it as a separate cleanup hook is redundant.
* Use weakref.finalize to close span when stream is garbage collected
If a user creates a streaming response but never consumes it, the cleanup
hooks won't run. Now we register a weak reference finalizer that will close
the span when the stream object is garbage collected, ensuring spans don't
leak in this scenario.
* Fix _get_finalizers_from_stream to use _result_hooks attribute
Renamed function to _get_result_hooks_from_stream and fixed it to
look for the _result_hooks attribute which is the correct name in
ResponseStream class.
* Add missing asyncio import in test_request_info_mixin.py
* Fix leftover merge conflict marker in image_generation sample
* Update integration tests
* Fix integration tests: increase max_iterations from 1 to 2
Tests with tool_choice options require at least 2 iterations:
1. First iteration to get function call and execute the tool
2. Second iteration to get the final text response
With max_iterations=1, streaming tests would return early with only
the function call/result but no final text content.
* Fix duplicate function call error in conversation-based APIs
When using conversation_id (for Responses/Assistants APIs), the server
already has the function call message from the previous response. We
should only send the new function result message, not all messages
including the function call which would cause a duplicate ID error.
Fix: When conversation_id is set, only send the last message (the tool
result) instead of all response.messages.
* Add regression test for conversation_id propagation between tool iterations
Port test from PR #3664 with updates for new streaming API pattern.
Tests that conversation_id is properly updated in options dict during
function invocation loop iterations.
* Fix tool_choice=required to return after tool execution
When tool_choice is 'required', the user's intent is to force exactly one
tool call. After the tool executes, return immediately with the function
call and result - don't continue to call the model again.
This fixes integration tests that were failing with empty text responses
because with tool_choice=required, the model would keep returning function
calls instead of text.
Also adds regression tests for:
- conversation_id propagation between tool iterations (from PR #3664)
- tool_choice=required returns after tool execution
* Document tool_choice behavior in tools README
- Add table explaining tool_choice values (auto, none, required)
- Explain why tool_choice=required returns immediately after tool execution
- Add code example showing the difference between required and auto
- Update flow diagram to show the early return path for tool_choice=required
* Fix tool_choice=None behavior - don't default to 'auto'
Remove the hardcoded default of 'auto' for tool_choice in ChatAgent init.
When tool_choice is not specified (None), it will now not be sent to the
API, allowing the API's default behavior to be used.
Users who want tool_choice='auto' can still explicitly set it either in
default_options or at runtime.
Fixes#3585
* Fix tool_choice=none should not remove tools
In OpenAI Assistants client, tools were not being sent when
tool_choice='none'. This was incorrect - tool_choice='none' means
the model won't call tools, but tools should still be available
in the request (they may be used later in the conversation).
Fixes#3585
* Add test for tool_choice=none preserving tools
Adds a regression test to ensure that when tool_choice='none' is set but
tools are provided, the tools are still sent to the API. This verifies
the fix for #3585.
* Fix tool_choice=none should not remove tools in all clients
Apply the same fix to OpenAI Responses client and Azure AI client:
- OpenAI Responses: Remove else block that popped tool_choice/parallel_tool_calls
- Azure AI: Remove tool_choice != 'none' check when adding tools
When tool_choice='none', the model won't call tools, but tools should
still be sent to the API so they're available for future turns.
Also update README to clarify tool_choice=required supports multiple tools.
Fixes#3585
* Keep tool_choice even when tools is None
Move tool_choice processing outside of the 'if tools' block in OpenAI
Responses client so tool_choice is sent to the API even when no tools
are provided.
* Update test to match new parallel_tool_calls behavior
Changed test_prepare_options_removes_parallel_tool_calls_when_no_tools to
test_prepare_options_preserves_parallel_tool_calls_when_no_tools to reflect
that parallel_tool_calls is now preserved even when no tools are present,
consistent with the tool_choice behavior.
* Fix ChatMessage API and Role enum usage after rebase
- Update ChatMessage instantiation to use keyword args (role=, text=, contents=)
- Fix Role enum comparisons to use .value for string comparison
- Add created_at to AgentResponse in error handling
- Fix AgentResponse.from_updates -> from_agent_run_response_updates
- Fix DurableAgentStateMessage.from_chat_message to convert Role enum to string
- Add Role import where needed
* Fix additional ChatMessage API and method name changes
- Fix ChatMessage usage in workflow files (use text= instead of contents= for strings)
- Fix AgentResponse.from_updates -> from_agent_run_response_updates in workflow files
- Fix test files for ChatMessage and Role enum usage
* Fix remaining ChatMessage API usage in test files
* Fix more ChatMessage and Role API changes in source and test files
- Fix ChatMessage in _magentic.py replan method
- Fix Role enum comparison in test assertions
- Fix remaining test files with old ChatMessage syntax
* Fix ChatMessage and Role API changes across packages
- Add Role import where missing
- Fix ChatMessage signature: positional args to keyword args (role=, text=, contents=)
- Fix Role enum comparisons: .role.value instead of .role string
- Fix FinishReason enum usage in ag-ui event converters
- Rename AgentResponse.from_updates to from_agent_run_response_updates in ag-ui
Fixes API compatibility after Types API Review improvements merge
* Fix ChatMessage and Role API changes in github_copilot tests
* Fix ChatMessage and Role API changes in redis and github_copilot packages
- Fix redis provider: Role enum comparison using .value
- Fix redis tests: ChatMessage signature and Role comparisons
- Fix github_copilot tests: ChatMessage signature and Role comparisons
- Update docstring examples in redis chat message store
* Fix ChatMessage and Role API changes in devui package
- Fix executor: ChatMessage signature change
- Fix conversations: Role enum to string conversion in two places
- Fix tests: ChatMessage signatures and Role comparisons
* Fix ChatMessage and Role API changes in a2a and lab packages
- Fix a2a tests: Role comparisons and ChatMessage signatures
- Fix lab tau2 source: Role enum comparison in flip_messages, log_messages, sliding_window
- Fix lab tau2 tests: ChatMessage signatures and Role comparisons
* Remove duplicate test files from ag-ui/tests (tests are in ag_ui_tests)
* Fix ChatMessage and Role API changes across packages
After rebasing on upstream/main which merged PR #3647 (Types API Review
improvements), fix all packages to use the new API:
- ChatMessage: Use keyword args (role=, text=, contents=) instead of
positional args
- Role: Compare using .value attribute since it's now an enum
Packages fixed:
- ag-ui: Fixed Role value extraction bugs in _message_adapters.py
- anthropic: Fixed ChatMessage and Role comparisons in tests
- azure-ai: Fixed Role comparison in _client.py
- azure-ai-search: Fixed ChatMessage and Role in source/tests
- bedrock: Fixed ChatMessage signatures in tests
- chatkit: Fixed ChatMessage and Role in source/tests
- copilotstudio: Fixed ChatMessage and Role in tests
- declarative: Fixed ChatMessage in _executors_agents.py
- mem0: Fixed ChatMessage and Role in source/tests
- purview: Fixed ChatMessage in source/tests
* Fix mypy errors for ChatMessage and Role API changes
- durabletask: Use str() fallback in role value extraction
- core: Fix ChatMessage in _orchestrator_helpers.py to use keyword args
- core: Add type ignore for _conversation_state.py contents deserialization
- ag-ui: Fix type ignore comments (call-overload instead of arg-type)
- azure-ai-search: Fix get_role_value type hint to accept Any
- lab: Move get_role_value to module level with Any type hint
* Improve CI test timeout configuration
- Increase job timeout from 10 to 15 minutes
- Reduce per-test timeout to 60s (was 900s/300s)
- Add --timeout_method thread for better timeout handling
- Add --timeout-verbose to see which tests are slow
- Reduce retries from 3 to 2 and delay from 10s to 5s
This ensures individual test timeouts are shorter than the job
timeout, providing better visibility when tests hang.
With 60s timeout and 2 retries, worst case per test is ~180s.
* Fix ChatMessage API usage in docstrings and source
- Fix ChatMessage positional args in docstrings: _serialization.py, _threads.py, _middleware.py
- Fix ChatMessage in tau2 runner.py
- Fix role comparison in _orchestrator_helpers.py to use .value
- Fix role comparison in _group_chat.py docstring example
- Fix role assertions in test_durable_entities.py to use .value
* Revert tool_choice/parallel_tool_calls changes - must be removed when no tools
OpenAI API requires tool_choice and parallel_tool_calls to only be
present when tools are specified. Restored the logic that removes
these options when there are no tools.
- Restored check in _chat_client.py to remove tool_choice and
parallel_tool_calls when no tools present
- Restored same logic in _responses_client.py
- Reverted test to expect the correct behavior
* fixed issue in tests
* fix: resolve merge conflict markers in ag-ui tests
* fix: restructure ag-ui tests and fix Role/FinishReason to use string types
* fix: streaming function invocation and middleware termination
- Refactor streaming function invocation to use get_final_response() on inner streams
- Fix MiddlewareTermination to accept result parameter for passing results
- Fix _AutoHandoffMiddleware to use MiddlewareTermination instead of context.terminate
- Fix AgentMiddlewareLayer.run() to properly forward function/chat middleware
- Remove duplicate middleware registration in AgentMiddlewareLayer.__init__
- Fix exception handling in _auto_invoke_function to properly capture termination
- Fix mypy errors in core package
- Update tests to use stream=True parameter for unified run API
* fix all tests command
* Refactor integration tests to use pytest fixtures
- Merge testutils.py into conftest.py for azurefunctions integration tests
- Merge dt_testutils.py into conftest.py for durabletask integration tests
- Convert all integration tests to use fixtures instead of direct imports
(fixes ModuleNotFoundError with --import-mode=importlib)
- Add sample_helper fixture for azurefunctions tests
- Add agent_client_factory and orchestration_helper fixtures for durabletask
- Integration tests now skip with descriptive messages when services unavailable
- Restructure devui tests into tests/devui/ with proper conftest.py
- Add test organization guidelines to CODING_STANDARD.md
- Remove __init__.py from test directories per pytest best practices
* Fix pytest_collection_modifyitems to only skip integration tests
The hook was skipping all tests in the test session, not just
integration tests. Now it only skips items in the integration_tests
directory.
* Fix mem0 tests failing on Python 3.13
Use patch.object on the imported module instead of @patch with string
path to ensure the mock takes effect regardless of import timing.
* fix mem0
* another attempt for mem0
* fix for mem0
* fix mem0
* Increase worker initialization wait time in durabletask tests
Increase from 2 to 8 seconds to allow time for:
- Python startup and module imports
- Azure OpenAI client creation
- Agent registration with DTS worker
- Worker connection to DTS
This helps prevent test failures in CI where the first tests may run
before the worker is fully ready to process requests.
* Fix streaming test to use ResponseStream with finalizer
The _consume_stream method now expects a ResponseStream that can provide
a final AgentResponse via get_final_response(). Update the test to use
ResponseStream with AgentResponse.from_updates as the finalizer.
* Fix MockToolCallingAgent to use new ResponseStream API and update samples
* small updates to run_stream to run
* fix sub workflow
* temp fix for az func test
---------
Co-authored-by: Dmytro Struk <13853051+dmytrostruk@users.noreply.github.com>
* Add an AsyncLocal AgentRunContext
* Update AgentRunContext session naming
* Make AgentRunContext readonly and add ADR
* Make session nullable and add unit tests
* Add unit tests for setting the context in AIAgent
* Apply suggestions from code review
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Fix sample in ADR
* Fix broken unit test
* Add unit test for checking if middleware can access AgentRunContext
* Fix build error after merge.
* Fix AgentRunContextTests after merge from main
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* ADR for simplified get response
* updated some language, added agent option and code comparison
* small update in sample
* added workflows and expanded some points
* changed decision and number
* updated with stream=False default
* Initial plan
* Infrastructure setup
* Plan for minimal client
* Plan update
* Basic agentic chat
* cleanup
* Cleanups
* More cleanups
* Cleanups
* More cleanups
* Test plan
* Sample
* Fix streaming and error handling
* Fix notifications
* Cleanups
* cleanup sample
* Additional tests
* Additional tests
* Run dotnet format
* Remove unnecessary files
* Mark packages as non packable
* Fix build
* Address feedback
* Fix build
* Fix remaining warnings
* Feedback
* Feedback and cleanup
* Cleanup
* Cleanups
* Cleanups
* Cleanups
* Retrieve existing messages from the store to send them along the way and update the sample client
* Run dotnet format
* Add ADR for AG-UI
* Switch to use the SG and use a convention for run ids
* Cleanup MapAGUI API
* Fix formatting
* Fix solution
* Fix solution
* Rename AI Agent packages to use Microsoft.Agents.AI
* Fix for build
* Fix formatting
* Fix formatting
* Ignore in VSTHRD200 in migration samples
* Ignore in VSTHRD200 in migration samples
* Add some missing projects and run format
* Fix build errors
* Address code review feedback
* Fix merge issues
---------
Co-authored-by: Mark Wallace <markwallace@microsoft.com>
* update to packaging section
* added options
* clarification and typo
* updated python packaging guidance, moved discussion to appendix
* moved discussion into ADR
* updated folder strcutre
* Add a script to produce docs by language, and add multi-turn-conversation docs.
* Fix typo.
* Remove java pivots, move source templates and update comments.
* Add links to docs from root readme
westeyandGitHub
Evan MattsonandGitHub
Chris GillumandGitHub
Shawn HenryandGitHub
Jeff HandleyandGitHub
Javier Calvarro NelsonandGitHub
Eric ZhuandGitHub
Tao ChenandGitHub
