* Improve PR template and breaking-change label automation
- Add a structured "Related Issue" section using GitHub closing keywords
- Add a Review Guide prompt (major changes, impact, reviewer focus) with a
note that the focus item is for human reviewers only
- Add checklist items for issue linkage / no duplicate PRs and invert the
breaking-change item (checked = not breaking)
- Extend label-title-prefix to prepend [BREAKING] when the "breaking change"
label is added
- Add label-breaking-change workflow to apply the "breaking change" label
when a PR title contains [BREAKING]
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add pull-requests agent skill with dotnet/python links
- Add root .github/skills/pull-requests/SKILL.md covering PR description
authoring (following the PR template) and the review-comment workflow
(review -> plan -> user review -> implement -> reply to all -> resolve)
- Symlink the skill from python/.github/skills and dotnet/.github/skills
- Reference the skill from python/AGENTS.md and dotnet/AGENTS.md
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fold breaking-change labeling into label-pr workflow
Move the title -> 'breaking change' label logic into the existing label-pr
workflow (which already applies the python/.NET labels) and drop the separate
label-breaking-change workflow.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address PR title prefix review feedback
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Pin patched MessagePack for .NET restore
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Revert MessagePack central pin
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Move title prefix tests out of tracked GitHub tests
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Exclude skill docs from CI path filters
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Match skill symlinks in CI path exclusions
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Exclude AGENTS docs from CI path filters
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Scope title-prefix normalization to a real prefix
The normalization branch in addTitlePrefix matched ^Python (no colon), so
titles like "Python samples improvements" or "Pythonic refactor" were treated
as already-prefixed and only re-cased, never receiving the "Python: " prefix.
Scope the match to ^<prefix>:\s* so only an actual existing prefix is
normalized; otherwise the prefix is prepended. Same fix applies to the .NET
prefix (e.g. ".NETStandard bump").
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Eduard van Valkenburg
·
2026-06-15 10:55:23 +00:00
* Fix ollama_chat_client.py sample: pass tools via options dict
The sample was passing tools as a direct keyword argument to
get_response(), which caused a TypeError. The tools parameter
must be passed inside the options dict per the SupportsChatGetResponse
protocol.
Fixes#6411
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Wrap tools in a list as expected by OllamaChatClient
_prepare_tools_for_ollama iterates the tools value, so it must be a
list rather than a bare FunctionTool instance.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: Add AgentLoopMiddleware for re-running agents in a loop
Add `AgentLoopMiddleware`, an `AgentMiddleware` that re-runs the wrapped
agent in a loop. A single configurable class covers three common patterns,
each with a convenience classmethod factory:
- Ralph loop (`.ralph(...)`): no exit criteria, with feedback tracking
(`record_feedback`/`progress`), progress injection (`inject_progress`),
optional fresh context per iteration (`fresh_context`), and an early-stop
completion signal (`is_complete`).
- Predicate (`.with_predicate(...)`): loop while a `should_continue` callable
returns True (e.g. paired with `todos_remaining`/`background_tasks_running`).
- Judge (`.with_judge(...)`): a second chat client decides whether the original
request was answered, using a `JudgeVerdict` structured-output response.
The loop also auto-resolves pending function-approval / user-input requests via
an `on_approval_request` callable (bounded by `max_approval_rounds`), and the
next iteration's input is controlled by `next_message`. Supports both streaming
and non-streaming runs.
Exports `AgentLoopMiddleware`, `JudgeVerdict`, `todos_remaining`, and
`background_tasks_running`. Adds tests, a sample, and docs.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: Refine AgentLoopMiddleware API and sample
- with_judge: add criteria list with {{criteria}} templating into judge
instructions plus an agent-side instruction; add fresh_context, additional
judge feedback relay; default judge max_iterations.
- should_continue is now required and positional; supports (bool, str|None)
feedback tuples surfaced to next_message/record_feedback via feedback kwarg.
- Judge forwards full multi-modal request and response messages.
- Default max_iterations=10 (explicit None = unbounded); removed is_complete and
Ralph terminology; ShouldContinueResult is a real TypeAlias.
- Sample: stream all loops, print iteration counts via injected user-block
boundaries (robust to function calling), <role>: content formatting, per-method
expected output, and a looping todo sample.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: Fix CI checks for AgentLoopMiddleware
- Resolve pyright errors in _loop.py: drop the always-true final_result None
check (the while loop always assigns it) and cast finish_reason to the
AgentResponse constructor's expected type.
- Apply pyupgrade --py310-plus: import TypeAlias from typing.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: Resolve mypy/pyright disagreement on finish_reason
pyright infers AgentResponse.finish_reason as including str and rejects the
direct assignment, while mypy considers a cast redundant. Drop the cast and
suppress only pyright with a targeted reportArgumentType ignore, satisfying
both type checkers.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: Add todo+judge AgentLoopMiddleware sample
Add a second AgentLoopMiddleware sample that composes two criteria in one
should_continue predicate: a TodoProvider check (evaluated first) and a
report-style judge chat client (evaluated once todos are complete) that grades
the assembled report against shared requirements. Register it in the middleware
samples README.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: Compose todo+judge loops as two middleware
Rework the todo+judge sample to compose two AgentLoopMiddleware on the agent
itself (middleware=[judge_loop, todo_loop]) instead of a single hand-written
predicate. The inner todos_remaining loop drafts the report todo-by-todo and the
outer with_judge loop re-runs it until an editor chat client judges the report
publication-ready, reusing the built-in helpers.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Reset session for fresh_context loops via snapshot/restore
AgentLoopMiddleware.fresh_context previously only reset context.messages,
so with an attached session each iteration still reloaded the local
transcript or re-threaded the service-side conversation id and the model
saw the accumulated history. Snapshot the session once before the loop
(via to_dict) and restore it (from_dict + field copy) between iterations,
so every pass starts from the pre-loop baseline. The final iteration's
pass is persisted (no restore after the terminating iteration), so a
subsequent agent.run continues from there.
Removed the obsolete warning, updated docstrings and core AGENTS.md, and
added tests: a snapshot/restore round-trip, a session-reset
streaming x fresh_context x inject_progress x store matrix across multiple
runs and loop iterations, and response_format parsing across the loop.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Updated samples and docstrings
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Eduard van Valkenburg
·
2026-06-12 14:35:54 +00:00
* feat(ag-ui): add thread snapshot store primitives
Key decisions:\n- Introduce an AGUIThreadSnapshot model limited to replayable messages, optional Shared State, and optional interrupt state.\n- Define AGUIThreadSnapshotStore as an async protocol keyed by explicit Snapshot Scope and AG-UI Thread id.\n- Add InMemoryAGUIThreadSnapshotStore as memory-only, latest-only, bounded local/demo/test storage; no file-backed store is introduced.\n- Require snapshot_scope_resolver whenever an endpoint is configured with a snapshot store, including pre-wrapped runners, so thread ids are not authorization boundaries.\n\nFiles changed:\n- packages/ag-ui/agent_framework_ag_ui/_snapshots.py\n- packages/ag-ui/agent_framework_ag_ui/__init__.py\n- packages/ag-ui/agent_framework_ag_ui/_agent.py\n- packages/ag-ui/agent_framework_ag_ui/_workflow.py\n- packages/ag-ui/agent_framework_ag_ui/_endpoint.py\n- packages/core/agent_framework/ag_ui/__init__.py\n- packages/core/agent_framework/ag_ui/__init__.pyi\n- packages/ag-ui/tests/ag_ui/test_snapshots.py\n- packages/ag-ui/tests/ag_ui/test_endpoint.py\n- packages/ag-ui/tests/ag_ui/test_public_exports.py\n- packages/ag-ui/AGENTS.md\n\nVerification:\n- uv run pytest packages/ag-ui/tests/ag_ui/test_snapshots.py packages/ag-ui/tests/ag_ui/test_public_exports.py packages/ag-ui/tests/ag_ui/test_endpoint.py::test_endpoint_requires_snapshot_scope_resolver_when_store_configured packages/ag-ui/tests/ag_ui/test_endpoint.py::test_endpoint_accepts_snapshot_store_with_scope_resolver -q\n- uv run pytest packages/ag-ui/tests/ag_ui/test_endpoint.py::test_endpoint_requires_snapshot_scope_resolver_when_store_configured packages/ag-ui/tests/ag_ui/test_endpoint.py::test_endpoint_requires_snapshot_scope_resolver_when_wrapped_runner_has_store packages/ag-ui/tests/ag_ui/test_endpoint.py::test_endpoint_accepts_snapshot_store_with_scope_resolver -q\n- uv run poe syntax -P ag-ui -C\n- uv run poe pyright -P ag-ui\n- uv run poe syntax -P core -C\n- uv run poe pyright -P core\n- uv run poe typing -P ag-ui\n- uv run poe typing -P core\n- uv run poe test -P ag-ui\n- uv run poe check -P ag-ui\n- git diff --check\n- git diff --cached --check\n\nBlockers / next iteration:\n- No blockers. Next slice can use the store contract to capture and hydrate agent snapshots.\n- uv repeatedly refreshed azure-ai-projects in uv.lock during local runs; reverted the generated lockfile churn because this change does not alter dependencies.\n- The poe-check commit hook was skipped after manual verification because it reformatted unrelated core MCP files outside this task.
* feat(ag-ui): hydrate agent threads from snapshots
Key decisions:
- Resolve Snapshot Scope per endpoint request and pass it to the AG-UI runner only when snapshot storage is active.
- Treat empty messages with no resume payload as an agent Hydrate Request when a scoped snapshot store is configured, replaying stored Shared State and message snapshots without invoking the wrapped agent.
- Save the latest replayable agent message snapshot and Shared State at normal completion under Snapshot Scope plus AG-UI Thread id; no durable or file-backed store is introduced.
Files changed:
- packages/ag-ui/agent_framework_ag_ui/_agent_run.py
- packages/ag-ui/agent_framework_ag_ui/_endpoint.py
- packages/ag-ui/agent_framework_ag_ui/_snapshots.py
- packages/ag-ui/tests/ag_ui/test_endpoint.py
Verification:
- uv run pytest packages/ag-ui/tests/ag_ui/test_endpoint.py::test_agent_endpoint_hydrates_stored_thread_snapshot_without_invoking_agent -q
- uv run pytest packages/ag-ui/tests/ag_ui/test_endpoint.py::test_agent_endpoint_hydrates_stored_thread_snapshot_without_invoking_agent packages/ag-ui/tests/ag_ui/test_endpoint.py::test_agent_endpoint_hydrates_snapshots_by_scope_and_thread -q
- uv run pytest packages/ag-ui/tests/ag_ui/test_endpoint.py::test_endpoint_empty_messages packages/ag-ui/tests/ag_ui/test_endpoint.py::test_agent_endpoint_hydrates_stored_thread_snapshot_without_invoking_agent packages/ag-ui/tests/ag_ui/test_endpoint.py::test_agent_endpoint_hydrates_snapshots_by_scope_and_thread -q
- uv run poe syntax -P ag-ui -C
- uv run poe pyright -P ag-ui
- uv run poe typing -P ag-ui
- uv run poe test -P ag-ui
- uv run poe check -P ag-ui
- git diff --check
- git diff --cached --check
Blockers / next iteration:
- No blockers. Next slice can reconstruct normal new-user agent turns from stored snapshots.
- uv repeatedly refreshed azure-ai-projects in uv.lock during local runs; reverted the generated lockfile churn because this change does not alter dependencies.
- The poe-check commit hook was skipped after manual verification because it refreshed unrelated uv.lock dependency resolution.
* feat(ag-ui): reconstruct agent turns from snapshots
Key decisions:
- Load scoped thread snapshots for non-hydrate agent requests only when snapshot storage is active and no resume payload is present.
- Rebuild prior AG-UI history from stored snapshot messages, preserving the incoming new user suffix and treating stored snapshot content as authoritative over conflicting prior client history.
- Merge stored Shared State with request state overrides before schema defaults and existing state-context injection.
Files changed:
- packages/ag-ui/agent_framework_ag_ui/_agent_run.py
- packages/ag-ui/tests/ag_ui/test_endpoint.py
Verification:
- uv run pytest packages/ag-ui/tests/ag_ui/test_endpoint.py::test_agent_endpoint_prepends_stored_snapshot_for_new_user_turn -q
- uv run pytest packages/ag-ui/tests/ag_ui/test_endpoint.py::test_agent_endpoint_deduplicates_full_history_and_merges_fresh_state -q
- uv run pytest packages/ag-ui/tests/ag_ui/test_endpoint.py::test_endpoint_empty_messages packages/ag-ui/tests/ag_ui/test_endpoint.py::test_agent_endpoint_hydrates_stored_thread_snapshot_without_invoking_agent packages/ag-ui/tests/ag_ui/test_endpoint.py::test_agent_endpoint_hydrates_snapshots_by_scope_and_thread packages/ag-ui/tests/ag_ui/test_endpoint.py::test_agent_endpoint_prepends_stored_snapshot_for_new_user_turn packages/ag-ui/tests/ag_ui/test_endpoint.py::test_agent_endpoint_deduplicates_full_history_and_merges_fresh_state -q
- uv run pytest packages/ag-ui/tests/ag_ui/test_endpoint.py -q
- uv run poe syntax -P ag-ui -C
- uv run poe pyright -P ag-ui
- uv run poe test -P ag-ui
- uv run poe check -P ag-ui
- uv run poe typing -P ag-ui
- git diff --check
- git diff --cached --check
Blockers / next iteration:
- No blockers. Next slice can enable workflow AG-UI Thread Snapshot persistence and hydration.
- uv repeatedly refreshed azure-ai-projects in uv.lock during local runs; reverted the generated lockfile churn because this change does not alter dependencies.
- The poe-check commit hook was skipped after manual verification because it refreshes unrelated uv.lock dependency resolution.
* feat(ag-ui): hydrate workflow threads from snapshots
Key decisions:
- Handle workflow Hydrate Requests before resolving or invoking the wrapped workflow when snapshot storage and Snapshot Scope are active.
- Capture only replayable workflow protocol data: workflow-emitted state snapshots, workflow-emitted message snapshots, and synthesized messages from text/tool output.
- Keep workflow snapshot capture inactive without configured persistence, and skip saving snapshots when the workflow stream emits RUN_ERROR.
Files changed:
- packages/ag-ui/agent_framework_ag_ui/_workflow.py
- packages/ag-ui/tests/ag_ui/test_endpoint.py
Verification:
- uv run pytest packages/ag-ui/tests/ag_ui/test_endpoint.py::test_workflow_endpoint_hydrates_emitted_snapshots_without_invoking_workflow packages/ag-ui/tests/ag_ui/test_endpoint.py::test_workflow_endpoint_hydrates_synthesized_text_and_tool_snapshot -q
- uv run pytest packages/ag-ui/tests/ag_ui/test_endpoint.py -q
- uv run pytest packages/ag-ui/tests/ag_ui/golden/test_scenario_workflow.py -q
- uv run poe syntax -P ag-ui -C
- uv run poe pyright -P ag-ui
- uv run poe test -P ag-ui
- uv run poe typing -P ag-ui
- uv run poe check -P ag-ui
- git diff --check
- git diff --cached --check
Blockers / next iteration:
- No blockers. Next slice can preserve interruption state and protect snapshots on errors across agent and workflow endpoints.
- uv repeatedly refreshed azure-ai-projects in uv.lock during local runs; reverted the generated lockfile churn because this change does not alter dependencies.
- The poe-check commit hook was skipped after manual verification because it refreshes unrelated uv.lock dependency resolution.
* feat(ag-ui): preserve interrupted thread snapshots
Key decisions:
- Capture workflow RUN_FINISHED interrupt metadata in replayable AG-UI Thread Snapshots so Hydrate Requests can restore pending workflow actions without invoking or resuming the workflow.
- Keep failed agent and workflow runs from replacing the last good snapshot; RUN_ERROR streams leave the previous snapshot available for hydration.
- Verify interruption hydration through endpoint-level AG-UI streams for both agent and workflow wrappers, including Shared State replay and no wrapped runner invocation.
Files changed:
- packages/ag-ui/agent_framework_ag_ui/_workflow.py
- packages/ag-ui/tests/ag_ui/test_endpoint.py
Verification:
- uv run pytest packages/ag-ui/tests/ag_ui/test_endpoint.py::test_workflow_endpoint_hydrates_interrupted_thread_without_invoking_workflow -q
- uv run pytest packages/ag-ui/tests/ag_ui/test_endpoint.py::test_agent_endpoint_hydrates_interrupted_thread_without_invoking_agent packages/ag-ui/tests/ag_ui/test_endpoint.py::test_agent_endpoint_run_error_does_not_overwrite_previous_snapshot packages/ag-ui/tests/ag_ui/test_endpoint.py::test_workflow_endpoint_hydrates_interrupted_thread_without_invoking_workflow packages/ag-ui/tests/ag_ui/test_endpoint.py::test_workflow_endpoint_run_error_does_not_overwrite_previous_snapshot -q
- uv run pytest packages/ag-ui/tests/ag_ui/test_endpoint.py -q
- uv run pytest packages/ag-ui/tests/ag_ui/golden/test_scenario_workflow.py -q
- uv run poe syntax -P ag-ui -C
- uv run poe pyright -P ag-ui
- uv run poe test -P ag-ui
- uv run poe typing -P ag-ui
- uv run poe check -P ag-ui
- git diff --check
- git diff --cached --check
Blockers / next iteration:
- No blockers. Next slice can document AG-UI Thread Snapshot security and usage.
- uv repeatedly refreshed azure-ai-projects in uv.lock during local runs; reverted the generated lockfile churn because this change does not alter dependencies.
- The poe-check commit hook was skipped after manual verification because it refreshes unrelated uv.lock dependency resolution.
* docs(ag-ui): document thread snapshot security
Key decisions:
- Document AG-UI Thread Snapshot persistence as opt-in and disabled unless a snapshot_store is configured.
- Place Snapshot Scope guidance next to endpoint authentication guidance, making clear that AG-UI Thread ids identify threads but do not authorize snapshot access.
- Describe built-in storage as in-memory only, process-local, latest-only, and not durable production storage; durable stores remain app-owned implementations of AGUIThreadSnapshotStore.
- Call out snapshot confidentiality impact and that no file-backed AG-UI snapshot store is provided.
Files changed:
- packages/ag-ui/README.md
Verification:
- uv run python scripts/check_md_code_blocks.py packages/ag-ui/README.md --no-glob
- git diff --check
- git diff --cached --check
- commit hook without SKIP ran changed-package lint/format and AG-UI README markdown-code-lint successfully before stopping because uv.lock was modified
- uv run poe markdown-code-lint (failed due existing unrelated packages/mistral/README.md missing agent_framework_mistral import resolution; changed AG-UI README blocks passed)
Blockers / next iteration:
- No blockers. Local issue/PRD planning artifacts remain uncommitted.
- uv refreshed azure-ai-projects in uv.lock during markdown lint and the commit hook; reverted the generated lockfile churn because this documentation change does not alter dependencies.
- The poe-check commit hook was skipped after manual verification because it refreshes unrelated uv.lock dependency resolution.
* fix(ag-ui): harden thread snapshot persistence edge cases
- Persist the completed confirm_changes turn with interrupt=None so hydration
no longer replays a stale pending interrupt after the user responds; resume
requests prepend stored history so the persisted thread is not truncated.
- Defer endpoint default_state application to the runners when snapshot
persistence is active, filling only keys missing from both the stored
snapshot state and the request state so defaults never reset persisted
Shared State.
- Always fold the turn's output into the persisted messages snapshot even when
the outbound MESSAGES_SNAPSHOT event is suppressed for predictive tools
without confirmation.
- Load the stored snapshot on workflow follow-up turns, reconstruct full
thread history into the run input, and seed the snapshot builder with merged
state so saving a new turn no longer replaces prior history.
- Move snapshot message reconstruction helpers to _run_common for reuse by the
workflow runner; load stored agent snapshots on resume turns for state merge.
- Add endpoint regression tests for all four scenarios.
* fix(ag-ui): protect snapshot history on resume and harden suffix trust
- Prepend stored thread history when persisting snapshots for resume runs on
both the agent and workflow paths, so a resumed interrupt no longer
overwrites the stored thread with just the resume turn's output.
- Filter the incoming message suffix during thread reconstruction: only user
turns and tool results answering backend-issued tool calls (stored tool
calls or pending interrupts) may extend authoritative history. Client-forged
assistant and tool messages are dropped and logged instead of being
persisted and replayed.
- Close the workflow snapshot builder's tool-call group when a tool result or
text message lands, so synthesized transcripts keep tool results adjacent to
their tool_calls message and stay valid as provider replay history.
- Export DEFAULT_MAX_THREAD_SNAPSHOTS from agent_framework_ag_ui and expose
SnapshotScopeResolver through the core ag_ui facade and stub.
- Add regression tests for agent and workflow resume history preservation,
forged suffix rejection, builder tool-call grouping, and the export surface.
* fix(ag-ui): tolerate snapshot save failures and scope workflow cache
- Wrap snapshot_store.save() on both the agent and workflow paths so a
transient store failure (timeout, connection refused) is logged instead of
propagating. Previously a failing save converted an already-streamed
successful run into RUN_ERROR, and on the workflow path emitted RUN_ERROR
after RUN_FINISHED, violating the single-terminal-event invariant. The
previous snapshot stays available for hydration.
- Key the workflow_factory instance cache by (snapshot_scope, thread_id). The
Snapshot Scope is the authorization boundary, so the same thread id under
different scopes no longer shares an in-memory workflow instance.
clear_thread_workflow accepts an optional snapshot_scope and clears all
scopes for the thread when omitted.
- Add tests: save-failure tolerance for agent and workflow endpoints,
scope-isolated workflow cache, async snapshot_scope_resolver support, and
in-memory store key validation errors.
* fix(ci): ignore all dotnet.microsoft.com links in linkspector
The existing ignore pattern only matched https://dotnet.microsoft.com/download,
but Microsoft sites insert a locale segment between host and path
(e.g. /en-us/download/dotnet/10.0), so localized links slip past the pattern
and get checked. dotnet.microsoft.com bot-blocks CI link checkers with
intermittent 403s across the whole site, which fails markdown-link-check on
unrelated pull requests since linkspector scans the entire repository.
Ignore the domain wholesale, matching how platform.openai.com is already
handled for the same reason. A 403 from bot blocking is indistinguishable
from a removed page, so the checker cannot produce a meaningful signal for
this domain either way.
* ag-ui: simplify raw_messages assignment and drop OrderedDict
- Replace list(cast(...)) with a typed annotation for raw_messages
(_agent_run.py:866) per review suggestion
- Replace OrderedDict with a plain dict in InMemoryAGUIThreadSnapshotStore
(_snapshots.py:136); regular dicts are insertion-order-safe since
Python 3.7, so OrderedDict is unnecessary. Update _evict_oldest to use
next(iter(...)) for FIFO removal instead of popitem(last=False).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address review feedback for #2458: review comment fixes
---------
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Integrate shell tool into AgentHarness
* Validate shell_executor exposes as_function() with a clear TypeError
Addresses PR review feedback: a public factory should fail fast with an
actionable error rather than a cryptic AttributeError when an incompatible
shell_executor is supplied. Validation happens upfront, regardless of whether
the client supports shell tools.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Type shell harness params via TYPE_CHECKING import
Addresses PR review feedback: type shell_executor and
shell_environment_provider_options instead of Any, using a TYPE_CHECKING
import from agent_framework_tools.shell. The import never executes at
runtime, so there is no circular dependency, and the lazy runtime import of
ShellEnvironmentProvider is retained. Since ShellExecutor is a protocol
without as_function(), the validated getattr result is invoked directly.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix MCP allowed_tools empty list handling
When allowed_tools is set to an empty list [], the falsy check
'if not self.allowed_tools' incorrectly treats it as unconfigured
(same as None), causing all tools to be exposed. Change to an
explicit 'is None' check so that an empty list correctly results
in no tools being allowed.
Co-authored-by: Azure SRE Agent <noreply@microsoft.com>
* Clarify allowed_tools docstring: None vs [] semantics
Per Eduard's review on PR #6296: explicitly document that None exposes all tools and [] exposes none, across all four MCPTool / MCPStdioTool / MCPStreamableHTTPTool / MCPWebsocketTool docstrings.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* allowed_tools docstring: recommend load_tools=False for full disable
Per Eduard's follow-up on PR #6296: `load_tools=False` is the cleaner idiom when you don't want to expose any tools. Reframe `allowed_tools=[]` in the docstring as a runtime guard / inspection-only path and cross-reference `load_tools`.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Azure SRE Agent <noreply@microsoft.com>
Co-authored-by: Giles Odigwe <79032838+giles17@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Parse structuredContent from MCP CallToolResult (#3313)
The _parse_tool_result_from_mcp method only iterated over the content
field from CallToolResult, ignoring the structuredContent field entirely.
MCP servers that return JSON data via structuredContent (e.g., Power BI
MCP) appeared to return None.
Add handling for structuredContent: when present, serialize it as JSON
text and append it to the result list. This preserves the data for the
LLM while maintaining backward compatibility with existing behavior.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: Parse MCP CallToolResult.structuredContent field to prevent tool results returning None
Fixes#3313
* Address review feedback: add default=str to json.dumps and remove .checkpoints/
- Add default=str to json.dumps for structuredContent serialization so
non-JSON-serializable values (e.g. bytes) degrade gracefully instead
of raising TypeError
- Remove all .checkpoints/ runtime artifacts from the repository
- Add **/.checkpoints/ to .gitignore to prevent future accidental commits
- Add test for non-serializable structuredContent values
Fixes#3313
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address review feedback for #3313: Python: MCP CallToolResult.structuredContent field is not parsed, causing tool results to return None
---------
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add sampling guardrails to MCP tools
Add approval, token, and request-count controls to the MCP sampling
callback used when an MCPTool is configured with a chat client.
- Add `sampling_approval_callback`, `sampling_max_tokens`, and
`sampling_max_requests` parameters to `MCPTool` and its
`MCPStdioTool`, `MCPStreamableHTTPTool`, and `MCPWebsocketTool`
subclasses, positioned directly after `client`.
- Gate each server-initiated `sampling/createMessage` request behind the
approval callback, which denies by default when no callback is provided.
- Clamp the requested `maxTokens` to `sampling_max_tokens` and enforce a
per-session request count via `sampling_max_requests`.
- Log incoming sampling requests at WARNING level (counts only).
- Export `SamplingApprovalCallback` from the public API.
- Add tests, a sample, and documentation updates.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Make sampling denial message context-aware
Distinguish the deny-by-default case (no approval callback configured)
from an explicit denial by a configured `sampling_approval_callback`, so
the returned ErrorData message is accurate for callback-driven denials
and exceptions.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Eduard van Valkenburg
·
2026-06-10 10:17:36 +00:00
* Parallelize Purview PSPC cold cache path
* Cache Purview payment-required state for scope refresh
* Cache Purview payment-required state for scope refresh
* Align Purview policy action dedupe and 402 caching
Deduplicate combined policy actions by action and restriction action so restriction-only actions are preserved
without duplicating identical entries. Cache tenant-level payment-required state from background scope refresh so
subsequent calls short-circuit consistently.
* .NET: Implement best-effort caching for background job scope retrieval and add unit tests for cache write failures
* Purview - feat: Enhance ScopedContentProcessor to queue ContentActivityJob when no applicable scopes are found and update related tests
* docs: Update purview package README and AGENTS documentation to reflect caching optimizations and policy enforcement scenarios
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* docs: clarify checkpoint storage security model and deserialization trust boundaries
Add Security Model documentation sections to the checkpoint encoding and
Azure Functions serialization modules explaining:
- Checkpoint storage is a trusted data source requiring access controls
- The RestrictedUnpickler allowlist is defense-in-depth, not a security boundary
- Developer responsibilities for securing storage backends
- Guidance on using allowed_types and strip_pickle_markers
Co-authored-by: Azure SRE Agent <noreply@microsoft.com>
* Apply suggestions from code review
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Azure SRE Agent <noreply@microsoft.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* fix: use getattr for non-OpenAI provider response compatibility
Fixes#6234Fixes#6235
Use getattr with None fallback for system_fingerprint and output
attributes to prevent AttributeError when non-OpenAI providers
return response objects without these fields.
* fix: use typed variable for response output to satisfy pyright
Fixes#6235
Use getattr with None fallback for the output attribute, and assign
to a typed list variable before the match statement to help pyright
narrow the response item types correctly.
* fix: rename response_outputs to avoid name collision with case-block variable
Fixes#6235
Rename outputs to response_outputs on line 1974 to avoid mypy error
about conflicting variable names in the match statement's case blocks.
Also use list[Any] for explicit generic type annotation.
* fix: use cast(list[Any]) for response output to satisfy pyright
Fixes#6235
The getattr() call returns Unknown type which pyright cannot narrow
in the match statement. Use an explicit cast to list[Any].
* fix: use hasattr guard instead of getattr for response.output
Fixes#6235
Using hasattr(response, 'output') and then accessing response.output
directly gives pyright enough type information to verify the match
statement exhaustiveness. This avoids the cast(list[Any]) approach
which pyright still flagged as partially unknown.
* fix: use ternary operator for response_outputs assignment
Replace if-else block with ternary expression to satisfy ruff SIM108 lint rule.
This fixes the Package Checks (3.11) CI failure.
* fix: use ternary with cast for ruff SIM108 and pyright type safety
Replace if-else block with ternary expression using cast(list[Any], ...)
to satisfy:
- ruff SIM108 (use ternary instead of if-else)
- ruff E501 (line length < 120)
- pyright type narrowing (cast preserves type info lost in ternary)
All local checks pass: ruff check, ruff format, pyright, 298 tests.
* fix: replace hasattr+cast with try/except to preserve pyright types
---------
Co-authored-by: Tao Chen <taochen@microsoft.com>
* Filter MCP tool kwargs to declared params via allowlist
Previously MCPTool combined framework runtime kwargs (from
FunctionInvocationContext.kwargs) with the LLM-supplied arguments and
stripped only a hardcoded denylist of known framework keys before
forwarding to the MCP server. Any new framework-injected kwarg leaked to
the server unless the denylist was updated.
Switch to an allowlist built from each tool's declared parameters
(inputSchema.properties). Only declared params are forwarded; everything
else is stripped. Add an `additional_tool_argument_names` constructor
argument so users can opt extra names back in, globally (Sequence[str])
and/or per remote tool name (Mapping with reserved "*" global key). The
existing denylist is kept as a safety net for framework-named params a
server declares in its schema; explicitly opted-in extras always win. The
reserved _meta handling is unchanged.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address MCP allowlist review comments and fix reload arg loss
- Fix pyright reportUnknownArgumentType in _load_tools (cast schema properties).
- Register declared param names before the existing-tool skip guard so that
tool-list reloads preserve the allowlist for already-loaded tools (previously
unchanged tools silently dropped all declared args after a background reload).
- Handle bare-string values in an additional_tool_argument_names mapping instead
of iterating their characters.
- Clarify the framework denylist comment: explicit extras override the denylist.
- Make the extras-override-denylist test unambiguous (opt in a denylisted name).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Eduard van Valkenburg
·
2026-06-09 07:37:11 +00:00
* feat(claude): bump claude-agent-sdk to 0.2.87
Upgrade claude-agent-sdk dependency from >=0.1.36,<0.1.49 to >=0.2.87,<0.3.
Changes:
- Bump version pin in pyproject.toml
- Add 'xhigh' effort level to ClaudeAgentOptions (Opus 4.7 specific)
- Expose new upstream SDK options: skills, session_id, task_budget,
include_hook_events, strict_mcp_config, continue_conversation,
fork_session
- Add TaskBudget type import
- Update uv.lock
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* chore: lower claude-agent-sdk floor to >=0.1.36
Keep the lower bound at 0.1.36 since the 0.1→0.2 transition was additive
and our code works on older versions as long as new options aren't used.
This avoids forcing unnecessary upgrades on existing users.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: replace TaskBudget import with inline type for SDK compat
TaskBudget was added in claude-agent-sdk 0.2.93 but does not exist in
0.2.87. Use dict[str, int] inline type instead so type checking passes
against 0.2.87. Lock file pinned to 0.2.87.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix per-service-call history persistence with server-storing clients
When an Agent set require_per_service_call_history_persistence=True together
with a HistoryProvider, and the chat client stored history server-side by
default (e.g. OpenAIChatClient, STORES_BY_DEFAULT=True), the external history
provider was silently never persisted.
Unify persistence on the per-service-call middleware: when the flag is set and
a HistoryProvider exists, the middleware is always installed and owns
persistence. service_stores_history now only selects middleware behavior:
- service does not store: load providers and drive the function loop with a
local sentinel conversation id, or
- service stores: skip loading (the service owns history) and persist each
service call while the real conversation id flows through.
Also rationalize chat-options handling in _prepare_run_context:
- _merge_options now skips None overrides and strips remaining None values, so
an unset `store` is never forwarded and the service decides its own default.
- Resolve `store` and `conversation_id` once from a single combined view
(effective_options) instead of probing both default and runtime dicts; the
auto-injection and per-service-call resolution now agree on conversation_id.
Fixes#5798
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Correct as_agent() docstring: persistence is per service call, not once per run
Address PR review: when the client stores history server-side, the
per-service-call middleware still persists after each model call; only
provider loading is skipped. The previous "persist once per run()" wording
contradicted the implementation.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address PR review: docs, missing-conversation-id warning, and tests
- Clarify that require_per_service_call_history_persistence is a no-op when no
HistoryProvider is present (docstrings in _agents.py and _clients.py).
- Warn on every service call when the client stores history server-side but
returns no conversation_id, so the (uncommon) loss of cross-turn resumability
cannot fail silently.
- Add tests: storing client + existing conversation_id does not raise and the id
propagates; two runs on the same session keep persisting with a stable
service_session_id and no provider loading; storing-without-conversation-id
warns per call.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Eduard van Valkenburg
·
2026-06-09 05:47:57 +00:00
* fix(mem0): parallel memory retrieval logic and strict type compliance
* fix(mem0): align parallel retrieval types for pyright and mypy
* fix(mem0): handle asyncio.CancelledError in search response and update test description
* fix(mem0): improve error handling for asyncio.CancelledError and update test names for clarity
* fix(mem0): improve retrieval response handling
* fix(gemini): preserve schema response_format
* fix(gemini): satisfy pyright strict in response schema extraction
Cast Any-narrowed mappings to Mapping[str, Any] in the structured-output
schema helpers so pyright strict no longer reports partially-unknown
member, argument, and variable types. Pass response_format["format"]
straight into the recursive extractor, which already guards non-mapping
inputs. No behavior change.
* fix(gemini): use Sequence[object] cast to satisfy both mypy and pyright
The Sequence[Any] cast pyright strict needs to know the loop element type
is reported as a redundant-cast by mypy, which already narrows the
isinstance branch to Sequence[Any]. Cast to Sequence[object] instead:
pyright gets a fully known element type and mypy no longer sees an
identical-type cast. No behavior change.
---------
Co-authored-by: Evan Mattson <evan.mattson@microsoft.com>
* MCP long-running task support in Python
* Fix pyupgrade and AGENTS.md reconnect description
- pyupgrade: drop forward-reference string annotations in _mcp.py (Python 3.10+ resolves them natively now that MCPTaskOptions is defined before use).
- AGENTS.md: align reconnect description with current behavior. Phase 1 (initial tools/call) does NOT retry on connection loss; raises 'connection lost; task state unknown' instead, so a server that accepted the request but lost the response cannot start the operation twice. Phase 2 (tasks/get / tasks/result) still reconnects once against the same task_id.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix bandit nosec marker for CI pipeline
* Address PR feedbacks
* Clarifiied comments and addressed more PR feedbacks.
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: fix ConnectTimeout on multi-turn FoundryAgent conversations (#6241)
Expose a `timeout` parameter on `RawFoundryAgentChatClient`,
`_FoundryAgentChatClient`, `RawFoundryAgent`, `FoundryAgent`, and
`RawOpenAIChatClient` so callers can override the HTTP timeout used by
the underlying AsyncOpenAI client.
Root cause: `RawFoundryAgentChatClient.__init__` called
`project_client.get_openai_client()` without configuring any timeout,
inheriting the OpenAI SDK default of `httpx.Timeout(connect=5.0)`.
When connections are recycled between turns under load, the 5 s connect
timeout fires and surfaces as `openai.APITimeoutError`.
Fix:
- `load_openai_service_settings` (`_shared.py`): accept `timeout` and
include it in `client_args` for all three `AsyncOpenAI`/
`AsyncAzureOpenAI` construction paths.
- `RawOpenAIChatClient.__init__` (`_chat_client.py`): accept `timeout`
and forward to `load_openai_service_settings`.
- `RawFoundryAgentChatClient.__init__` (`_agent.py`): accept `timeout`
and set `openai_client.timeout = timeout` on the client returned by
`get_openai_client()` before passing it to the base class.
- `_FoundryAgentChatClient`, `RawFoundryAgent`, `FoundryAgent`: accept
and propagate `timeout` through the construction chain.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add timeout parameter to FoundryAgent and RawOpenAIChatClient
Expose a timeout parameter on RawFoundryAgentChatClient,
_FoundryAgentChatClient, RawFoundryAgent, FoundryAgent, and
RawOpenAIChatClient. When provided, the value is applied to the
underlying AsyncOpenAI client so that connect timeouts under load
or after connection recycling can be tuned by callers.
Previously, get_openai_client() was called without any timeout
override, so the SDK default of httpx.Timeout(connect=5.0) was
inherited and could fire on multi-turn conversations where the
underlying connection is recycled between turns.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: Add `timeout` parameter to `FoundryAgent` to fix `ConnectTimeout` on multi-turn conversations
Fixes#6241
* fix(foundry): use with_options to avoid mutating shared OpenAI client timeout (#6241)
Replace direct assignment with
in
RawFoundryAgentChatClient.__init__.
The Azure AI Projects SDK caches and returns a shared AsyncOpenAI client
per AIProjectClient. Mutating its .timeout attribute leaked the override
to all other code paths sharing that client (other agents, user code).
with_options() returns a new client instance with the override applied,
leaving the original shared client untouched.
Update tests to assert with_options is called with the correct timeout
and that the original shared client's timeout attribute is not mutated.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* test(foundry): assert with_options return value flows to instance.client (#6241)
The four timeout propagation tests verified that with_options was called
but did not confirm that the returned (timeout-configured) client was
actually stored on the instance. A silent discard of the return value
would have left the tests green while the timeout had no effect.
Each test now captures the constructed instance and asserts:
assert <instance>.client is openai_client_mock.with_options.return_value
Affected tests:
- test_raw_foundry_agent_chat_client_init_applies_timeout_to_openai_client
- test_raw_foundry_agent_chat_client_init_applies_timeout_with_preview_enabled
- test_foundry_agent_chat_client_init_propagates_timeout
- test_foundry_agent_init_propagates_timeout_to_openai_client
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix magentic manager warning
* Use typing_extensions.Sentinel for _MISSING sentinel value
Replace the bare object() sentinel with typing_extensions.Sentinel per
PEP 661 (now final). Sentinel provides a proper name and repr
('<_MISSING>') and is the idiomatic approach going forward.
Refs #4306
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: correct Sentinel type annotation for max_stall_count param (#6261)
Use int | Sentinel for max_stall_count parameter type annotation instead
of int with cast(Any, _MISSING) to properly express that the parameter
can hold either an int or the _MISSING sentinel value. This fixes the
pyright reportUnnecessaryComparison errors caused by the types int and
Sentinel having no overlap.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Rename _MISSING sentinel to UNSET in orchestrations
The sentinel is user-visible as a default in public init signatures, so
use UNSET (no leading underscore) instead of the private _MISSING name.
Drop the now-unnecessary reportPrivateUsage ignores on the UNSET imports.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix compaction message-id collisions and tool-loop summary persistence
Fixes two bugs in the compaction strategies:
- #5237: incremental group annotation assigned message ids by position
within the re-annotated slice, so moving the re-annotation start back to
a previous group start restarted ids at 0 and produced collisions
(e.g. a user message reusing an assistant message's id), merging groups
and causing tool-result compaction to wrongly exclude messages.
group_messages/_ensure_message_ids now take an id_offset and guard
against existing-id collisions; annotate_message_groups threads the
slice start index through as the offset.
- #4991: the function-invocation loop copied the message list each
iteration, so summaries inserted by compaction landed in a throwaway
copy and were lost across tool-loop iterations (only the persistent
excluded flags survived). _prepare_messages_for_model_call now compacts
the list in place when messages is a list, so inserted summaries persist.
Adds regression tests (incremental id uniqueness, existing-id collision
avoidance, idempotency, and tool-loop summary persistence including
streaming and conversation-id modes).
Also adds a summarization.py sample demonstrating SummarizationStrategy
directly with a real client, and reworks advanced.py with tool-call
groups and a real summarizer.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Guard incremental message-id assignment against prefix-id collisions
Addresses PR review on #5237: _ensure_message_ids only guarded against
collisions within the re-annotated slice. A preexisting (e.g. user-supplied)
id in the preserved prefix could still be reassigned in the suffix when the
id was numerically out of position, merging groups across the re-annotation
boundary again.
group_messages/_ensure_message_ids now accept reserved_ids, and
annotate_message_groups passes the preserved prefix's ids so auto-assigned
suffix ids never collide across the full list. Adds a regression test
reproducing the out-of-position prefix-id collision.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Eduard van Valkenburg
·
2026-06-04 08:37:59 +00:00
* Add MCP-based skills discovery (McpSkill, McpSkillsSource, McpSkillResource)
Implement Agent Skills discovery over MCP following the SEP-2640 convention:
- McpSkillsSource: reads skill://index.json to discover skills served by an MCP server
- McpSkill: lazily fetches SKILL.md content via resources/read on demand
- McpSkillResource: wraps MCP resource results (text and binary)
- Path traversal protection in get_resource for defense in depth
- Samples for Foundry Toolbox and standalone MCP skills server
- Comprehensive unit tests (514 lines)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address PR review comments: rename to MCP* convention, fix error handling and samples
- Rename McpSkill/McpSkillResource/McpSkillsSource to MCPSkill/MCPSkillResource/MCPSkillsSource
- Add data-URI prefix stripping for blob resource decoding
- Let non-McpError exceptions propagate from get_resource()
- Fix contradictory test comment
- Use interactive input() in mcp_based_skill sample
- Remove misleading sample output block
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Restore debug logging for McpError in get_resource()
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Use AzureCliCredential in Foundry toolbox skills sample for consistency
Replace DefaultAzureCredential with AzureCliCredential to match the
credential convention used in all other samples.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Use MCPStreamableHTTPTool in MCP skills sample
Replace raw mcp library imports (ClientSession, streamable_http_client)
with the framework's MCPStreamableHTTPTool to keep MCP server connections
consistent regardless of whether skills are enabled.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Branch on McpError.error.code so only not-found errors return empty
Previously _try_read_index() and get_resource() swallowed every McpError
as 'no skills available', making auth failures, server crashes, and
connection drops indistinguishable from a server that simply has no
skills.
Now only two codes are treated as not-found:
- -32002 (MCP-spec Resource not found)
- -32601 (METHOD_NOT_FOUND — server lacks resources/read)
All other McpError codes and non-McpError exceptions propagate with a
warning log, surfacing real failures visibly.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add tests for non-McpError and non-not-found error propagation in MCP skills
Cover the re-raise branch in MCPSkill.get_resource for plain
ConnectionError/TimeoutError, the generic McpError (code 0) propagation
on get_resource, and TimeoutError propagation in _try_read_index.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Revert "Use MCPStreamableHTTPTool in MCP skills sample"
This reverts commit f31ed0ded9.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Introduce MCP_SKILLS experimental feature for MCP skill classes
Add a separate MCP_SKILLS feature ID to ExperimentalFeature enum and
use it for MCPSkillResource, MCPSkill, and MCPSkillsSource, since their
promotion timeline is partly outside of our control.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: progressive tool exposure via FunctionInvocationContext
Add first-class progressive tool exposure to the Python core function-calling
loop. Tools can now add or remove real FunctionTool schemas at runtime via the
injected FunctionInvocationContext, taking effect on the next iteration of the
loop.
- FunctionInvocationContext gains a live `tools` list plus experimental
`add_tools()` / `remove_tools()` helpers (feature: PROGRESSIVE_TOOLS).
- The function-calling loop establishes a run-local, normalized tools list and
threads it into the context at both invocation paths so mutations propagate.
- Add a sample (dynamic_tool_exposure.py) and a tools samples README, including
a note that CodeAct providers (Monty/Hyperlight) use their own provider-level
tool management instead.
Supersedes #3877.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Validate non-negative input in dynamic_tool_exposure sample tools
Address review feedback: factorial and fibonacci now return an error
message for negative n instead of producing incorrect results.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Make add_tools atomic and surface swallowed function errors
Address review feedback on progressive tool exposure:
- add_tools now validates the full batch against a throwaway copy before
committing, so a duplicate-name clash partway through a sequence leaves
the live tool list unchanged (all-or-nothing).
- _auto_invoke_function now logs a warning (with traceback) when a tool
raises, so contract errors such as a duplicate-name ValueError from
add_tools are debuggable without enabling include_detailed_errors.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Avoid retaining tracebacks when logging swallowed function errors
Logging with exc_info=exc fed the exception traceback to the logging
machinery, whose frame references created reference cycles collected
lazily by the cyclic GC. On Windows that could drop a hyperlight
WasmSandbox on a non-owning thread ("unsendable, dropped on another
thread"), crashing the xdist worker. Log a pre-formatted message with
the exception repr instead, so no traceback object is retained.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* added missing decorator
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Eduard van Valkenburg
·
2026-06-03 09:01:07 +00:00
* Fix FoundryAgent stripping model from PromptAgent requests
Move run_options.pop('model', None) inside the _uses_foundry_agent_session()
conditional so that model is only stripped for hosted agent sessions (where
the server manages the model) and preserved for PromptAgent requests that
require it in the Responses API call.
Fixes#5525
* test: add coverage for resp_* continuation preserving model
Adds test_raw_foundry_agent_chat_client_prepare_options_preserves_model_for_resp_continuation
to explicitly verify that HostedAgent v1 / v2-no-session paths (where conversation_id
starts with resp_) preserve model and previous_response_id without triggering the
hosted-session gate.
---------
Co-authored-by: Benke Qu <bequ@microsoft.com>
Co-authored-by: Evan Mattson <35585003+moonbox3@users.noreply.github.com>
* Python: Fix OTLP HTTP base-endpoint losing /v1/{signal} auto-append
Per the OTel spec, OTEL_EXPORTER_OTLP_ENDPOINT is a *base* URL for HTTP —
the SDK auto-appends /v1/traces, /v1/metrics, /v1/logs when it reads the
env var directly. Signal-specific endpoint env vars are *full* URLs used
verbatim.
_get_exporters_from_env read the base endpoint and forwarded it as the
constructor ``endpoint=`` argument, which the SDK always treats as a full
signal URL. As a result, with OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
and HTTP protocol, the exporter sent to http://localhost:4318 instead of
http://localhost:4318/v1/traces (and likewise for metrics/logs).
Replicate the spec's auto-append here when falling back to the base
endpoint under HTTP. gRPC behavior is unchanged.
* Python: Fix mypy type errors in OTLP endpoint assignment
Pre-declare traces_endpoint, metrics_endpoint, logs_endpoint as
str | None before the if/else block. Mypy inferred str from the
if-branch f-string assignments and then rejected the str | None
expressions in the else-branch as incompatible.
* feat(bedrock): add structured output support via Converse API (Fixes#5966)
* fix(bedrock): improve unsupported model exception handling and schema parsing
* refactor(bedrock): use generic traversal for strict schema enforcement
* address Copilot review comments on structured output
* refine bedrock structured output: guard additionalProperties, TypeError check, docs + test
* fix(bedrock): widen response_format to Mapping and add missing test coverage
* Python: feat(evals): RubricScore type + EvalScoreResult.dimensions
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: feat(foundry-evals): RubricDimension + GeneratedEvaluatorRef + accept in evaluators=
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: feat(evals): parse rubric_scores from output items + assertion helpers
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: feat(evals): BaseAgent.as_eval_source / Workflow.as_eval_source
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: feat(foundry-evals): EvalGenerationSource + generate_rubric helper
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: feat(foundry-evals): YAML config loader + sample
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: fix(evals): address PR review feedback
Addresses 4 Copilot review comments on PR #6101:
1. assert_dimension_score_at_least: drop the (not evaluator or found_any) guard so require_applicable=True correctly raises when the named evaluator produces no entries for the dimension. Adds TestRubricAssertions covering the regression.
2. GeneratedEvaluatorRef docstring: reword to describe actual behaviour (pinning recommended, not required) so it matches the dataclass default and FoundryEvals warning path.
3. _poll_generation_job: switch from asyncio.get_event_loop() to get_running_loop() and bound the per-iteration sleep by remaining time, matching _poll_eval_run.
4. generate_rubric: type category as Literal['quality','safety'] and validate at the entry point with a ValueError; drop the silent 'invalid -> quality' rewrite in _generation_job_to_ref. Adds a regression test.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: feat(foundry-evals): hosted-agent-aware rubric generation
* Auto-detect hosted Foundry agents in agent_as_eval_source: when the
agent's chat_client exposes a string agent_name (the convention used
by RawFoundryAgentChatClient for PromptAgents/HostedAgents), emit a
type='agent' EvalGenerationSource so the service fetches instructions
and tools from the agent registry instead of relying on the local
wrapper (which holds neither for hosted agents).
* Add hosted_agent_version kwarg and a new agent_version field on
EvalGenerationSource so PromptAgent runs can pin to a specific hosted
version for reproducible rubric generation.
* Add force_prompt_source escape hatch to bypass auto-detection and
always emit a rendered prompt dossier - useful when the local wrapper
carries overrides the service-side agent doesnt see.
* Fix _to_sdk_source for dataset sources: SDK ctor takes name=/version=,
not dataset_name=/dataset_version=. The mismatch would raise TypeError
against the real azure-ai-projects 2.3.0a* SDK; only unmocked
integration paths were affected.
Tests cover: auto-detection happy path, versionless hosted agent,
explicit hosted_agent_version forwarding, force_prompt_source override,
non-string chat_client attrs (MagicMock test doubles) not mis-detected,
agent_version forwarded through _to_sdk_source, and the corrected
dataset SDK kwarg names.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(foundry-evals): accept canonical dimension_scores key per docs
The published Foundry rubric-evaluator output (Microsoft Learn 'Rubric evaluators' reference) places per-dimension breakdowns under properties.dimension_scores, not properties.rubric_scores. The parser now tries dimension_scores first and falls back to rubric_scores for preview-build compatibility, and tolerates non-list payloads (e.g. MagicMock auto-attrs) by trying the next candidate when parsing yields zero entries.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat(foundry-evals): add manual create_rubric_evaluator
Adds FoundryEvals.create_rubric_evaluator as the agent-framework surface over project_client.beta.evaluators.create_version. This is the manual counterpart to generate_rubric: callers supply RubricDimension instances (authored locally, ported from another framework, or hand-tuned) and we POST a RubricBasedEvaluatorDefinition. The service auto-attaches the non-editable residual dimension (general_quality for quality, general_policy_compliance for safety).
Per the Microsoft Learn 'Rubric evaluators' reference, the auto-generation path (create_generation_job) is primarily a portal/UI feature; external SDK clients with rich local agent context are better served by manual create_version. This keeps generate_rubric for users who want to round-trip through a Foundry-registered agent.
Validation up front: weight must be in [1,10], ids unique, descriptions non-empty, pass_threshold in [0,1]. The returned GeneratedEvaluatorRef is identical in shape to one obtained from generate_rubric, so downstream evaluators= lists work unchanged.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* samples(foundry-evals): manual rubric sample + namespace re-exports
Adds evaluate_with_manual_rubric_sample.py demonstrating the end-to-end dev scenario for FoundryEvals.create_rubric_evaluator: hand-author a list of RubricDimension, register via create_rubric_evaluator, then use the pinned GeneratedEvaluatorRef alongside built-in evaluators in an agent regression run.
Also re-exports RubricDimension, GeneratedEvaluatorRef, build_sources, and load_evals_config from agent_framework.foundry (both the lazy runtime shim and the type stub) so the rubric samples can import everything from a single namespace; the auto-generate sample was previously broken because the shim was missing build_sources / load_evals_config.
Updates the foundry-evals README with a chooser entry for the two rubric paths.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat(foundry-evals): remove rubric creation flows; keep consumption only
Reframes agent-framework as a pure consumer of Foundry rubric evaluators: scoring against rubrics that already exist (authored in the Foundry portal or via the dedicated SDK / REST surface) instead of creating them from the SDK.
Removed creation surface area:
- FoundryEvals.generate_rubric (auto-generate path) and create_rubric_evaluator (manual path), plus all _GenerationSdkTypes / _ManualRubricSdkTypes / _to_sdk_dimensions / _coalesce_generation_sources / _to_sdk_source / _poll_generation_job / _generation_job_to_ref / _evaluator_version_to_ref / _get_beta_evaluators / _import_*_sdk_types helpers.
- EvalGenerationSource (the input source discriminator), RubricDimension (the input dimension type), agent_as_eval_source / workflow_as_eval_source / _detect_hosted_foundry_agent helpers, and the YAML-config loader (_evals_config.py with RubricGenerationSpec / RubricSourceSpec / parse_evals_config / load_evals_config / build_sources).
- BaseAgent.as_eval_source / Workflow.as_eval_source plus the _render_agent_dossier / _render_workflow_dossier helpers in core. These existed only to feed the now-removed generation pipeline.
- Samples evaluate_with_generated_rubric_sample.py, evaluate_with_manual_rubric_sample.py, and evaluators.yaml. Replaced with a short README section showing how to reference an existing rubric evaluator via GeneratedEvaluatorRef.
Kept (consumption surface):
- GeneratedEvaluatorRef, slimmed to (name, version, display_name). Still accepted alongside built-in evaluator strings in FoundryEvals(evaluators=[...]). Versionless refs still warn.
- RubricScore on EvalScoreResult.dimensions plus EvalResults.assert_dimension_score_at_least for per-dimension CI gates.
- _parse_dimension_entries / _extract_rubric_scores output parsing (both canonical dimension_scores and the legacy rubric_scores key).
Tests: 160/160 foundry unit tests and 71/71 core local-eval tests pass; pyright is clean across changed files. The pre-existing tests/core/test_telemetry.py::test_detect_hosted_fallback_import_error failure is unrelated and reproduces on the prior commit.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* samples(foundry-evals): add evaluate_with_rubric_sample
Adds a runnable end-to-end sample showing how to consume a pre-existing rubric evaluator created in Foundry: reference it with GeneratedEvaluatorRef(name, version), mix it with built-in evaluators in FoundryEvals, and gate CI with assert_dimension_score_at_least on a specific dimension.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(foundry-evals): satisfy mypy on _fetch_output_items
mypy infers OutputItemListResponse.sample as dict[str, object] | None while pyright correctly infers the typed Sample model. Cast to Any so both type checkers accept the attribute access pattern, rename the local to avoid shadowing the inner-loop sample binding, and drop the now-stale pyright suppressions.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* docs(foundry-evals): drop unpublished rubric-evaluators learn.microsoft.com link
The Adaptive Evals authoring docs are not yet published on Microsoft Learn, so the link 404s. Keep the descriptive text without the broken hyperlink; we can re-add it once the docs ship.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* test(foundry-evals): hoist repeated local imports to module top
Per code review feedback (eavanvalkenburg): the test file repeated 'from agent_framework_foundry._foundry_evals import ...' inside 22 test bodies and 'from agent_framework_foundry import GeneratedEvaluatorRef' inside 8 more. Move all of them to the existing top-level imports; the symbols are the same across tests and the local imports were redundant.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Ben Thomas <25218250+alliscode@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: safely serialize function-call arguments in core observability
Apply make_json_safe() to content.arguments in _to_otel_part() before
building the otel message dict, so that dataclass/framework payloads
(e.g. workflow request_info events) do not cause a TypeError when
_capture_messages() calls json.dumps().
Lift make_json_safe() into agent_framework._serialization (no new
external deps — dataclasses/datetime only) so the core observability
path can use it without a dependency on the ag-ui adapter.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(core): safely serialize workflow request_info payloads in observability (#5733)
- Add make_json_safe() helper to recursively convert non-serializable objects
- Use make_json_safe() in _to_otel_part() for function_call arguments
- Fix CustomPayload test class to use @dataclass (resolves B903 lint error)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(serialization): guard callability and normalize dict keys in make_json_safe (#5733)
- Use callable(getattr(obj, method, None)) instead of hasattr() so that
non-callable attributes named model_dump/to_dict/dict do not raise
TypeError at runtime.
- Wrap each call in try/except TypeError to handle callables with
mandatory arguments gracefully.
- Convert dict keys to str() so that non-string keys (e.g. datetime,
int) cannot cause json.dumps to raise TypeError.
- Add regression tests for both scenarios.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address observability serialization review feedback
---------
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Eduard van Valkenburg ·
westey ·
Giles Odigwe ·
Peter Ibekwe ·
Evan Mattson ·
chetantoshniwal ·
Copilot ·
Taisir Hassan ·
Willow Lopez ·
SergeyMenshykh ·
Vedant Sonani ·
Tao Chen ·
cooleryu ·
Yufeng He ·
semenshi-m ·
Benke Qu ·
Dineshsuriya D ·
Hameed Kunkanoor ·
Thota Sai Karthik ·
Ben Thomas ·