mirror of
https://github.com/microsoft/agent-framework.git
synced 2026-06-16 21:04:09 +08:00
af-foundry-evals-python
4 Commits
-
Python: fix reasoning model workflow handoff and history serialization (#4083)
* fix: strip function_call and text_reasoning from cross-agent workflow handoff When a reasoning model (e.g. gpt-5-mini) runs as Agent 1 in a workflow, its response includes text_reasoning items (with server-scoped IDs like rs_XXXX) and function_call items. Forwarding these to Agent 2 in a fresh conversation caused API errors because the reasoning/call IDs are scoped to the original stored response context. Changes: - Strip 'function_call', 'text_reasoning', 'function_approval_request', and 'function_approval_response' from handoff messages in _agent_executor.py - Keep 'function_result' so the actual tool output content is preserved for the next agent's context - Update unit tests to reflect that function_result messages survive handoff (messages grow from 2→3: user, tool(result), assistant(summary)) - Fix incorrect test assertions in test_function_invocation_stop_clears_* that assumed the client layer updates session.service_session_id - Also fixed _extract_function_calls to search all messages with call_id deduplication, and the error-limit stop path to submit function_call_output items before halting (via tool_choice=none cleanup call) Relates to: https://github.com/microsoft/agent-framework/issues/4047 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: reasoning model workflow handoff and history serialization Fixes multiple related issues when using reasoning models (gpt-5-mini, gpt-5.2) in multi-agent workflows that chain agents via from_response or replay full conversation history via AgentExecutorRequest. ## Reasoning items always emitted on output_item.added When a reasoning model produces encrypted or hidden reasoning (no visible text), the Responses API still fires a reasoning output item without any reasoning_text.delta events. Previously no text_reasoning Content was emitted in that case, making it invisible to downstream logic. Both the non-streaming (_parse_response_from_openai) and streaming (output_item.added) paths now always emit at least one text_reasoning Content — with empty text if no content is available — so co-occurrence detection and serialization guards work reliably. ## Reasoning items only serialized when paired with a function_call The Responses API only accepts reasoning items in input when they directly preceded a function_call in the original response. Sending a reasoning item that preceded a text response (no tool call) causes: "reasoning was provided without its required following item" _prepare_message_for_openai now checks has_function_call per message and skips text_reasoning serialization when there is no accompanying function_call. ## summary field is an array, not an object The reasoning item summary field sent to the Responses API must be an array of objects ([{"type": "summary_text", "text": ...}]), not a single object. Fixed _prepare_content_for_openai accordingly. ## service_session_id cleared when explicit history is provided When a workflow coordinator replays a full conversation (including function calls from a previous agent run) back to an executor via AgentExecutorRequest or from_response, the executor's session still held a service_session_id (previous_response_id) from the prior run. The API then received the same function-call items twice — once from previous_response_id (server-stored) and once from the explicit input — causing: "Duplicate item found with id fc_...". AgentExecutor.run (when should_respond=True) and from_response now reset self._session.service_session_id = None before running so that explicit input is the sole source of conversation context. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * small improvements in text reasoning * refactor: add reset_service_session to AgentExecutorRequest for explicit history replay Replace the implicit 'always clear service_session_id when should_respond=True' with an explicit opt-in field on AgentExecutorRequest. The old approach used should_respond=True as a proxy for 'full history replay', but that conflates two distinct intents: - Orchestrations group chat sends should_respond=True with an empty/single-message list (not a full replay) — unnecessarily clearing service_session_id. - HITL / feedback coordinators send the full prior conversation and truly need a fresh service session ID to avoid duplicate-item API errors. Changes: - Add AgentExecutorRequest.reset_service_session: bool = False - AgentExecutor.run only clears service_session_id when this flag is True - AgentExecutor.from_response unchanged (always clears; always full conversation) - Set reset_service_session=True in all full-history-replay call sites: agents_with_HITL.py, azure_chat_agents_tool_calls_with_feedback.py, autogen-migration round-robin coordinator, tau2 runner - Update _FullHistoryReplayCoordinator test helper to pass the flag Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * comment update * fixes from feedback * fix test * reverted changes to agent executor * fix: remove reset_service_session from tau2 runner Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * two other reverts * fix sample --------- Co-authored-by: Giles Odigwe <79032838+giles17@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Eduard van Valkenburg ·
2026-02-19 21:02:20 +00:00 -
Python: Add load_dotenv() to samples for .env file support (#4043)
* Initial plan * Add load_dotenv() to 303 Python samples for environment variable loading Co-authored-by: TaoChenOSU <12570346+TaoChenOSU@users.noreply.github.com> * Update SAMPLE_GUIDELINES.md to document load_dotenv() requirement Co-authored-by: TaoChenOSU <12570346+TaoChenOSU@users.noreply.github.com> * Update samples README.md to document .env file usage Co-authored-by: TaoChenOSU <12570346+TaoChenOSU@users.noreply.github.com> * Run ruff format on all changed sample files Co-authored-by: TaoChenOSU <12570346+TaoChenOSU@users.noreply.github.com> * Clarify load_dotenv() usage in README - local dev vs production Co-authored-by: TaoChenOSU <12570346+TaoChenOSU@users.noreply.github.com> * Remove deprecated getting_started folder as requested Co-authored-by: TaoChenOSU <12570346+TaoChenOSU@users.noreply.github.com> * Document env_file_path parameter for per-client configuration Co-authored-by: TaoChenOSU <12570346+TaoChenOSU@users.noreply.github.com> * Merge main branch to resolve conflicts Co-authored-by: TaoChenOSU <12570346+TaoChenOSU@users.noreply.github.com> * Fix run_evaluation.py file that was empty in merge commit Co-authored-by: TaoChenOSU <12570346+TaoChenOSU@users.noreply.github.com> * Remove dotnet changes from merge - out of scope for this PR Co-authored-by: TaoChenOSU <12570346+TaoChenOSU@users.noreply.github.com> * Remove package and test changes from merge - only sample changes needed Co-authored-by: TaoChenOSU <12570346+TaoChenOSU@users.noreply.github.com> * Remove test_func_utils.py - only sample changes needed Co-authored-by: TaoChenOSU <12570346+TaoChenOSU@users.noreply.github.com> * Revert sample files not in original changeset - keep only load_dotenv additions Co-authored-by: TaoChenOSU <12570346+TaoChenOSU@users.noreply.github.com> * Move load_dotenv() outside snippet tag in 06_host_your_agent.py Co-authored-by: TaoChenOSU <12570346+TaoChenOSU@users.noreply.github.com> * Fix comment placement - move load_dotenv before code comments Co-authored-by: TaoChenOSU <12570346+TaoChenOSU@users.noreply.github.com> * Fix load_dotenv() placement across all samples - after docstring, before code comments Co-authored-by: TaoChenOSU <12570346+TaoChenOSU@users.noreply.github.com> * Merge latest main branch with load_dotenv changes Co-authored-by: TaoChenOSU <12570346+TaoChenOSU@users.noreply.github.com> * Remove non-sample changes from merge - keep only load_dotenv additions Co-authored-by: TaoChenOSU <12570346+TaoChenOSU@users.noreply.github.com> * Revert non-load_dotenv sample changes from merge Co-authored-by: TaoChenOSU <12570346+TaoChenOSU@users.noreply.github.com> * Fix run_evaluation.py - use main's improved version (file already had load_dotenv) Co-authored-by: TaoChenOSU <12570346+TaoChenOSU@users.noreply.github.com> * Manual update * Manual update 2 * Fix Role usage and load_dotenv placement per PR review feedback Co-authored-by: eavanvalkenburg <13749212+eavanvalkenburg@users.noreply.github.com> * Fix Role usage - use string literals not enum attributes Co-authored-by: eavanvalkenburg <13749212+eavanvalkenburg@users.noreply.github.com> * Fix SAMPLE_GUIDELINES.md example - load_dotenv before docstring per guidance Co-authored-by: eavanvalkenburg <13749212+eavanvalkenburg@users.noreply.github.com> * Move load_dotenv() before docstrings in all samples per SAMPLE_GUIDELINES ordering Co-authored-by: eavanvalkenburg <13749212+eavanvalkenburg@users.noreply.github.com> * Address PR review: rename files, fix placement, add session usage, remove note Co-authored-by: eavanvalkenburg <13749212+eavanvalkenburg@users.noreply.github.com> * Update Redis README to reference renamed file redis_history_provider.py Co-authored-by: eavanvalkenburg <13749212+eavanvalkenburg@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: TaoChenOSU <12570346+TaoChenOSU@users.noreply.github.com> Co-authored-by: Tao Chen <taochen@microsoft.com> Co-authored-by: eavanvalkenburg <13749212+eavanvalkenburg@users.noreply.github.com> Co-authored-by: Eduard van Valkenburg <eavanvalkenburg@users.noreply.github.com>
Copilot ·
2026-02-19 10:55:13 +00:00 -
Python: Fix workflow samples for bugbash: part 1 (#4055)
* Fix workflow samples for bugbash: part 1 * Fix mypy * Fix tests
Tao Chen ·
2026-02-18 23:08:53 +00:00 -
Python: restructure: Python samples into progressive 01-05 layout (#3862)
* restructure: Python samples into progressive 01-05 layout - 01-get-started/: 6 numbered steps (hello agent → hosting) - 02-agents/: all agent concept samples (tools, middleware, providers, etc.) - 03-workflows/: ALL existing workflow samples preserved as-is - 04-hosting/: azure-functions, durabletask, a2a - 05-end-to-end/: demos, evaluation, hosted agents - Old files moved to _to_delete/ for review - Added AGENTS.md with structure documentation - autogen-migration/ and semantic-kernel-migration/ preserved at root * fix: switch to AzureOpenAI Foundry, fix CI failures - Switch all 01-get-started samples to AzureOpenAIResponsesClient with Azure AI Foundry project endpoint (AZURE_AI_PROJECT_ENDPOINT + AZURE_OPENAI_RESPONSES_DEPLOYMENT_NAME + AzureCliCredential) - Add _to_delete/ and 05-end-to-end/ to pyrightconfig.samples.json excludes - Fix test paths in packages/ that referenced old getting_started/ dirs: durabletask conftest + streaming test, azurefunctions conftest, devui conftest + capture_messages + openai_sdk_integration - Fix workflow_as_agent_human_in_the_loop.py import (sibling import) - Update hosting READMEs and tool comment paths - Replace root README.md with new structure overview - Update AGENTS.md to document Azure OpenAI Foundry as default provider * cleanup: remove _to_delete folder, copy resource files to active dirs All files in _to_delete/ were either: - Exact duplicates of files in the new structure (240 files) - Same file with only comment path updates (100 files) - One import-fix diff (workflow_as_agent_human_in_the_loop.py) - One superseded minimal_sample.py Resource files (sample.pdf, countries.json, employees.pdf, weather.json) copied to 02-agents/sample_assets/ and 02-agents/resources/ since active samples reference them. * fix: address PR review comments, centralize resources, remove root duplicates - Fix type annotation in 04_memory.py (string union -> proper types) - Fix old sample paths in observability files - Fix grammar/spelling in observability samples - Move sample_assets/ and resources/ to shared/ folder - Remove 8 duplicate observability files from 02-agents root - Update resource path references in multimodal_input and provider samples * fix: update broken links from old getting_started paths to new structure - Update relative paths in READMEs: getting_started/ → 01-get-started/, 02-agents/, 03-workflows/, 04-hosting/, 05-end-to-end/ - Fix absolute GitHub URLs in package READMEs - Fix broken link in ollama package README * fix: convert absolute GitHub URLs to relative paths for link checker Absolute URLs to python/samples/ on main branch 404 until PR merges. Converted to relative paths that linkspector can verify locally. * fix: update link for handoff sample moved to orchestrations/ * fix: update chatkit-integration README path from demos/ to 05-end-to-end/ * fix: update broken links in orchestrations README to match flat directory structure
2026-02-12 17:36:36 +00:00