* Foundry Evals integration for .NET
- Core evaluation framework: EvalItem, LocalEvaluator, FunctionEvaluator, EvalChecks
- IAgentEvaluator interface with MeaiEvaluatorAdapter bridge
- AgentEvaluationExtensions for agent.EvaluateAsync() overloads
- FoundryEvals wrapping MEAI quality/safety evaluators
- ConversationSplitters (LastTurn, Full) and IConversationSplitter
- EvalItem.PerTurnItems() for multi-turn decomposition
- HasImageContent for multimodal content detection
- WorkflowEvaluationExtensions for per-agent workflow evaluation
- 7 eval samples mirroring Python parity:
02-agents/Evaluation: SimpleEval, ExpectedOutputs, Multimodal
03-workflows/Evaluation: WorkflowEval
05-end-to-end/Evaluation: FoundryQuality, MixedProviders, ConversationSplits
- Comprehensive unit tests (1958 passing)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Rewrite FoundryEvals to use real Foundry Evals API
Replace MEAI evaluator shim with actual OpenAI EvaluationClient protocol
methods. FoundryEvals now creates eval definitions, submits runs, polls
for completion, and fetches per-item results server-side.
- New constructor: FoundryEvals(AIProjectClient, model, evaluators)
- Add FoundryEvalConverter for MEAI ChatMessage -> Foundry JSON format
- Add EvalId, RunId, ReportUrl to AgentEvaluationResults
- All 20 built-in evaluator constants now work (agent, tool, quality, safety)
- Remove Microsoft.Extensions.AI.Evaluation.Quality/Safety dependencies
- Update all samples for new constructor (no more ChatConfiguration)
- Replace BuildEvaluators tests with ResolveEvaluator tests
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add response output to CustomEvals and ExpectedOutputs samples
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address review: pagination, validation, error handling, tests
FoundryEvals fixes:
- Add pagination for output items (has_more/after cursor)
- Add guard clauses for pollIntervalSeconds/timeoutSeconds <= 0
- Fix double TryGetProperty for passed field parsing
- Throw on all-tool-evaluators with no tool definitions
- Fix XML doc (default 300s, not 180s)
New tests (30 added, 1989 total):
- EvalChecks: NonEmpty, ContainsExpected (pass/fail/skip/case),
HasImageContent, ToolCallsPresent
- FoundryEvalConverter: ConvertMessage (text, image, function call,
function results fan-out, empty fallback, mixed content),
ConvertEvalItem, BuildTestingCriteria (quality/agent/tool/groundedness
data mappings), BuildItemSchema
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix review: null-refs, Data.ToString() bug, ContainsExpected, add tests
- Fix NullReferenceException in sample Response display (pattern matching)
- Fix WorkflowEvaluationExtensions Data?.ToString() producing type names
instead of message text (pattern-match ChatMessage/AgentResponse/list)
- Change EvalChecks.ContainsExpected to return Passed=false when no
ExpectedOutput (was silently passing, masking misconfiguration)
- Add EvalItem constructor tests with LastTurn/Full/null splitters
- Add FoundryEvalConverter.ConvertMessage DataContent (base64 image) test
- Add ExtractAgentData tests with ChatMessage, list, and AgentResponse data
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix review: conversation fidelity, eval caching, fallback tests
- WorkflowEvaluationExtensions: preserve full response messages (tool calls,
intermediate) instead of synthetic 2-message conversation. Cast completed
Data to AgentResponse and use Messages when available, fallback to text.
- FoundryEvals: cache evalId per schema shape (hasContext, hasTools) so
subsequent EvaluateAsync calls create runs under the same eval definition.
- MeaiEvaluatorAdapter: code already correctly passes queryMessages (not full
conversation) to IEvaluator — no change needed, verified by inspection.
- Add tests: AgentResponse full messages preservation, unknown object
ToString() fallback for ExtractAgentData.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Rename AzureAI→Foundry: move eval files, update references
- Move FoundryEvals.cs and FoundryEvalConverter.cs from
Microsoft.Agents.AI.AzureAI to Microsoft.Agents.AI.Foundry
- Update namespace from AzureAI to Foundry in both files
- Add explicit usings required by Foundry project (no implicit usings)
- Move FoundryEvalConverter tests to Foundry.UnitTests project
(avoids ReplacingRedactor type conflict from dual project refs)
- Update all sample csproj references and using statements
- Remove Foundry project reference from AI UnitTests
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* PR review round 4: wire up tool extraction, remove eval cache, fix null safety
- BuildEvalItem: extract tools from agent via GetService<ChatOptions>() into EvalItem.Tools (Python parity)
- FoundryEvals: remove eval ID cache - each call creates fresh definition (matches Python behavior)
- FoundryEvals: replace null-forgiving operators with descriptive InvalidOperationException
- MixedProviders sample: remove unnecessary explicit PackageReferences (transitively provided)
- FoundryEvalConverter: document that tool results take precedence over text content
- Add LocalEvaluator zero-checks test documenting 0 metrics = failed behavior
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python-dotnet parity: 9 feature gaps filled
New checks:
- ToolCallArgsMatch() — verify tool call names + argument subset match
- ToolCalledCheck(ToolCalledMode.Any, ...) — match any of the specified tools
- ToolCalledMode enum (All/Any)
FoundryEvals enhancements:
- Default evaluators now [Relevance, Coherence, TaskAdherence] (was Relevance, Coherence)
- Auto-add ToolCallAccuracy when items have tool definitions
- EvaluateTracesAsync — evaluate by response_ids, trace_ids, or agent_id
- EvaluateFoundryTargetAsync — evaluate deployed Foundry targets
Result type enrichment:
- AgentEvaluationResults: added Status, Error, PerEvaluator, DetailedItems
- New EvalItemResult/EvalScoreResult/PerEvaluatorResult types
- FoundryEvals populates all new fields from API responses
Workflow fix:
- Skip internal executors (_*, input-conversation, end-conversation, end)
Tests: 8 new tests covering ToolCallArgsMatch, ToolCalledMode.Any, internal executor filtering
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add MeaiEvaluatorAdapter and PerTurnItems edge case tests
- 3 tests for MeaiEvaluatorAdapter: query message forwarding, synthetic
response fallback, multiple items aggregation
- 3 tests for EvalItem.PerTurnItems: empty conversation, no user messages,
system+assistant only
- StubEvaluator and StubChatClient test helpers
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Blocking link check for outdated package in DevUI.
* Replace Dictionary<string, object> payloads with typed wire models
Introduce internal FoundryEvalWireModels.cs with compile-time-safe types
for the OpenAI Evals API wire format. The OpenAI .NET SDK (2.9.1) only
provides protocol-level methods with BinaryContent/ClientResult — no
typed request models. These internal models replace scattered dictionary
literals with [JsonPropertyName]-annotated classes, giving:
- Compile-time safety (typos become build errors)
- Single point of change when the API evolves
- IntelliSense discoverability
- Cleaner serialization via JsonPolymorphic for content items
Models: WireContentItem hierarchy (text, image, tool_call, tool_result),
WireMessage, WireEvalItemPayload, WireTestingCriterion, WireItemSchema,
WireCreateEvalRequest, WireCreateRunRequest, and data source variants.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Skip metric when Foundry returns neither score nor passed
When an evaluator returns no score and no passed value, the previous
code created BooleanMetric(name, false), which falsely failed items
via ItemPassed. Now we skip the MEAI metric entirely for indeterminate
results — the raw data remains available in DetailedItems for diagnostics.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address PR #4914 review comments: fix tool evaluator bug and add tests
- Fix duplicate ToolCallAccuracy: resolve evaluator names before checking
against ToolEvaluators set (Comment 2)
- Make FilterToolEvaluators internal for testability; add tests for the
ArgumentException edge case when all evaluators are tool-type (Comment 3)
- Add CancellationToken test for LocalEvaluator (Comment 4)
- Add EvaluateAsync integration test on Run with sequential workflow and
per-agent SubResults verification (Comment 5)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address Peter's review comments on PR #4914
- Add trailing newline to Evaluation_FoundryQuality.csproj (Comment 6)
- Make evaluator name lookups case-insensitive: switch BuiltinEvaluators,
ToolEvaluators, AgentEvaluators, and ResolveEvaluator's StartsWith check
from Ordinal to OrdinalIgnoreCase (Comment 7)
- Add Trace.TraceWarning when Foundry returns fewer results than submitted
items, indicating expected vs actual count before padding (Comment 8)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add Microsoft.Extensions.AI.Evaluation packages to Directory.Packages.props
These were removed in #5269 as unused, but are needed by the Foundry
and core evaluation integration added in this PR.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: alliscode <bentho@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat: add finish_reason support to AgentResponse and AgentResponseUpdate
Add finish_reason field to AgentResponse and AgentResponseUpdate classes,
propagate it through _process_update() and map_chat_to_agent_update(),
and add comprehensive unit tests.
Fixes#4622
* feat: add finish_reason to AgentResponse and AgentResponseUpdate
* style: add copyright header to test_finish_reason.py
* docs: add finish_reason to AgentResponse and AgentResponseUpdate docstrings
* refactor: move finish_reason tests into test_types.py per review feedback
Move all finish_reason test cases from the separate test_finish_reason.py
file into test_types.py as requested by eavanvalkenburg. Tests are placed
in a new '# region finish_reason' section at the end of the file.
* fix: use model instead of model_id in _process_update
Address PR review feedback from @eavanvalkenburg — ChatResponse and
ChatResponseUpdate both use 'model', not 'model_id'.
* fix: resolve SIM102 lint error in _process_update
Combine nested if statements for AgentResponse finish_reason check
to satisfy ruff SIM102 rule, with line wrapping to stay under 120 chars.
* fix: resolve pyright reportArgumentType in map_chat_to_agent_update
Add type: ignore[arg-type] for FinishReason NewType widening when
passing ChatResponseUpdate.finish_reason to AgentResponseUpdate.
Matches existing patterns in the codebase (40+ similar ignores).
* Fix url_citation annotations dropped in streaming (#5029)
Add url_citation branch to the streaming annotation handler in
_parse_chunk_from_openai, mirroring the existing non-streaming path.
The handler creates an Annotation with type='citation', title, url,
and annotated_regions (TextSpanRegion), wrapped in Content.from_text.
Update test_streaming_annotation_added_with_unknown_type to use a
truly unknown type, and add new tests for url_citation (with and
without url).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address review feedback for #5029: Python: [Bug]: url_citation annotations silently dropped in Foundry streaming (SharePoint grounding citations lost)
---------
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Evan Mattson <35585003+moonbox3@users.noreply.github.com>
- Update Anthropic from 12.11.0 to 12.13.0
- Update Anthropic.Foundry from 0.4.2 to 0.5.0
- Change Anthropic project from release candidate to preview
- Add new IBetaService members (Agents, Environments, Sessions, Vaults) to test mock
Fixes#5246
When a custom @executor transforms agent output and sends a plain str,
the downstream AgentExecutor.from_str handler loses the full conversation
context. This adds a with_text() helper that creates a new
AgentExecutorResponse with replaced text while preserving the prior
conversation chain, so AgentExecutor.from_response is invoked instead.
- Add with_text(text) method to AgentExecutorResponse dataclass
- Add 3 regression tests in test_full_conversation.py
Co-authored-by: Evan Mattson <35585003+moonbox3@users.noreply.github.com>
* Improve workflow unit tests
* Update test name prefix for clarity.
* Update tests to surface any errors.
* fix check-point restore-time race in off-thread workflow event stream
* Fixes an intermittent checkpoint-restore race in in-process workflow runs.
The local MCP server can't be used for hosted tools tests because
Anthropic's backend needs to reach the MCP URL from their infrastructure
(not localhost on the CI runner). Revert to learn.microsoft.com/api/mcp
but catch BadRequestError, InternalServerError, APIConnectionError, and
APITimeoutError and pytest.skip so upstream outages don't block the
merge queue.
* Python: use local MCP server for hosted tools test and broaden image assertion
The hosted tools integration test was hitting rate limits on the external
learn.microsoft.com MCP server, causing persistent failures that retries
couldn't recover from. Switch to the local MCP server already spun up in
CI via LOCAL_MCP_URL, skipping when the env var isn't set.
Also broaden the image description assertion to accept common synonyms
(cottage, mansion, villa, etc.) instead of just "house", since the model
legitimately uses varied vocabulary for the same image.
* Address review feedback: validate LOCAL_MCP_URL scheme and use word boundaries
- Skip hosted tools test when LOCAL_MCP_URL lacks http/https scheme,
matching the pattern used in test_mcp.py.
- Use regex word boundaries for image assertion to avoid false matches
like "villain" matching "villa".
The misc-integration job (Anthropic, Ollama, MCP) frequently fails on merge to main when the upstream MCP server (e.g. learn.microsoft.com/api/mcp) returns a transient rate-limit error. The previous 5s retry delay is too short to ride out the upstream backoff window, so all retries fail and the merge queue is blocked. Bumping to 30s gives the upstream a chance to recover before pytest-retry re-runs the test.
* Add agent-framework-gemini package
* Add AGENTS.md documentation
* Add LICENSE file
* Add README.md for agent-framework-gemini package
* Add Google Gemini API keys to .env.example
* Add Google Gemini chat client implementation
* Add tests for GeminiChatClient
* Add Google Gemini agent examples
* Fix client inheritence order
* Update Gemini agent examples
* Update documentation
* Update AGENTS.md
* Add tests for JSON string handling in GeminiChatClient
* Add final response assembly test in GeminiChatClient
* Add tests for handling empty candidates in GeminiChatClient
* Improve Pydantic response handling in GeminiChatClient
* Add tests for function result resolution and callable tool normalization
* Add test for function result resolution when call_id is generated
* Refactor GeminiChatClient to correct inheritance order
Also updates constructor parameter order for environment file handling
* Enhance documentation and clarify Gemini-specific fields
* Update ThinkingConfig with new attributes and type
* Add tests for GoogleSearch and GoogleMaps configs
* Suppress valid-type mypy error on GeminiChatOptionsT
* Move service_url method near overrides
* Order _prepare_config kwargs by base then Gemini-specific
* Use FunctionCallingConfigMode for clarity and type safety
* Fix code_execution doc
* Add agent-framework-gemini to project dependencies
* Remove package from core dependencies
Initial release will be done without agent-framework-gemini in
core[all].
* Move integration tests into one file
* Remove __init__.py file from gemini tests directory
* Introduce RawGeminiChatClient as lightweight chat client
Updated GeminiChatClient to inherit from RawGeminiChatClient, maintaining full functionality with added features.
* Updated variable names from `model_id` to `model`
Across the codebase, including environment variables and client initialization. Adjusted related tests and sample scripts to reflect this change, ensuring consistency in the usage of the Gemini model identifier.
* Update AGENTS.md
* Update Gemini package to alpha status
* Fix docstrings in Gemini tests
* Change 'model_id' to 'model' in response handling
* Fix model property change in response handling
* Add built-in tool factory methods to Gemini client
Replaces boolean tool options (code_execution, google_search_grounding,
google_maps_grounding) with static factory methods that return types.Tool
objects: get_code_interpreter_tool, get_web_search_tool, get_mcp_tool,
get_file_search_tool, and get_maps_grounding_tool.
Simplifies _prepare_tools to a single translation boundary between
FunctionTool (framework) and FunctionDeclaration (Gemini API), with
types.Tool objects passed through unchanged.
* Surface code execution parts
_parse_parts now maps executable_code and code_execution_result
parts to text Content objects so callers can see the code run
and its output. Unknown part types log at debug level rather than
being silently dropped.
* Update Gemini client documentation
* Unify Gemini model name
Co-authored-by: Eduard van Valkenburg <eavanvalkenburg@users.noreply.github.com>
* Update Agent Framework core version
Co-authored-by: Eduard van Valkenburg <eavanvalkenburg@users.noreply.github.com>
* Add Python 3.14 in classifiers
* Replace kwargs with parameters in tool factories
* Refactor chat options handling in Gemini client
* Add tests for handling unknown and consumed keys
* Update Gemini documentation
Now reflects new options and built-in tool factory methods
* Change build system to flit
Co-authored-by: Eduard van Valkenburg <eavanvalkenburg@users.noreply.github.com>
* Fix build system in pyproject.toml
* Fix type checking for generate_content_stream
---------
Co-authored-by: Eduard van Valkenburg <eavanvalkenburg@users.noreply.github.com>
* Python: Skip get_final_response in OTel _finalize_stream when stream errored
When a streaming error occurs, _finalize_stream (a cleanup hook registered by
AgentTelemetryLayer) was unconditionally calling get_final_response(), which
triggers all registered result hooks including after_run context providers.
This caused providers to fire incorrectly on error paths.
Guard against this by checking result_stream._consumed: True only after
StopAsyncIteration (normal completion), False when an exception was raised.
The fix applies to both the chat client and agent telemetry layers.
Closes#5231
* Python: Expose consumed/stream_error on ResponseStream and capture error in OTel span
Address Copilot review feedback on #5232:
- Add `_stream_error: Exception | None` to ResponseStream, set in __anext__'s
except branch so cleanup hooks can inspect the failure.
- Expose public `consumed` and `stream_error` properties to avoid coupling
observability.py to private stream internals.
- Update both _finalize_stream closures (chat and agent layers) to use the
public properties and call capture_exception() with the stream error before
returning early, ensuring the OTel span records the failure rather than
closing silently.
* Python: Address Copilot review feedback on stream error handling
- Use stream_error is not None as the guard in _finalize_stream instead of
not consumed, so the early-return path is keyed precisely to actual errors
rather than any non-normal completion state.
- Clear _stream_error after _run_cleanup_hooks() completes to avoid retaining
the exception traceback (and any large object graphs it references) on the
stream instance beyond the cleanup phase.
* Python: Remove consumed/stream_error properties, use private attrs directly
Per review feedback: since observability.py and _types.py are in the same
package, accessing _stream_error directly is fine and the public properties
are unnecessary.
* Python: Fix Pyright reportPrivateUsage via inline ignore comments
Keep _stream_error private (consistent with rest of ResponseStream), and
suppress reportPrivateUsage at the call sites in observability.py with
inline pyright: ignore comments — access is intentional within the package.
* AG-UI deterministic state updates from tool results
* fix(ag-ui): address PR #5201 review comments
1. Add missing AGUIEventConverter, AGUIHttpService, __version__ to
_IMPORTS in core ag_ui lazy-export list to match the .pyi stub.
2. Coalesce predictive and deterministic state snapshots into a single
StateSnapshotEvent when both mechanisms are active on the same tool
result, reducing redundant snapshot traffic.
3. Update state_update() docstring to clarify that a predictive snapshot
may be emitted before the deterministic one when predict_state_config
is active.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix HandoffBuilder dropping function-level middleware when cloning agents (#5173)
_clone_chat_agent() was using agent.agent_middleware (agent-level only)
instead of agent.middleware (all types), which silently dropped any
function middleware registered on the original agent.
Changed to use agent.middleware to preserve all middleware types
(agent, function, and chat) during cloning.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: Fix HandoffBuilder dropping function-level middleware when cloning agents
Fixes#5173
* Fix false-positive middleware regression test (#5173)
The test used isinstance(m, FunctionMiddleware) which matched
_AutoHandoffMiddleware (always appended during build) instead of the
user's @function_middleware decorator. Assert directly that
tracking_middleware is present in the cloned agent's middleware list.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address review feedback for #5173: Python: [Bug]: HandoffBuilder drops function-level middleware when cloning agents
---------
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: Add allowed_checkpoint_types support to CosmosCheckpointStorage (#5200)
Add allowed_checkpoint_types parameter to CosmosCheckpointStorage for
parity with FileCheckpointStorage. This ensures both providers use the
same restricted pickle deserialization by default.
Changes:
- Accept allowed_checkpoint_types kwarg in __init__, stored as frozenset
- Convert _document_to_checkpoint from @staticmethod to instance method
- Forward allowed_types to decode_checkpoint_value on all load paths
- Update class docstring to describe the new parameter
- Add tests covering built-in safe types, app type opt-in/blocking,
and all load paths (load, list_checkpoints, get_latest)
- Add changelog entry noting the breaking behavior change
BREAKING CHANGE: CosmosCheckpointStorage now uses restricted pickle
deserialization by default. Checkpoints containing application-defined
types will require passing those types via allowed_checkpoint_types.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: Add `allowed_checkpoint_types` support to `CosmosCheckpointStorage` for parity with `FileCheckpointStorage`
Fixes#5200
* Address PR review: add pickle security warning and fix docstring examples
- Reintroduce explicit security warning about pickle deserialization risks
- Convert Example:: block to .. code-block:: python with imports for
consistency with other docstring examples
- Note: PR title should be updated to include [BREAKING] prefix per
changelog convention (comment #3, requires GitHub UI change)
Fixes#5200
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix python-feature-lifecycle skill YAML frontmatter
Remove copyright comment that preceded the YAML frontmatter delimiter,
which prevented the skill from loading. The --- block must be the very
first line of SKILL.md.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: update broken eslint-react plugin links in devui README
The upstream eslint-react repo moved plugins from packages/plugins/
to the top-level plugins/ directory, causing 404 errors detected by
linkspector CI.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat: Refactor Handoff Orchestration and add HITL support
* Change HandoffAgentExecutor to use factory-based instantiation
* Extract shared request collection logic in AIAgentUnservicedRequestsCollector
* Refactor HandoffAgentExecutor to use the "ContinueTurn" pattern as in AIAgentHostExecutor
* fix: Remove '$' from exception strings
Rename authored identifiers, XML docs, log messages, and comments
from 'folder' to 'directory' across the file skills codebase for
consistency with the agentskills.io specification and .NET conventions.
Public API changes (experimental):
- ScriptFolders → ScriptDirectories
- ResourceFolders → ResourceDirectories
.NET BCL API calls (Directory.Exists, Path.GetDirectoryName, etc.)
were already using 'directory' and are unchanged.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* support reflection for discovery of resources and scripts in class-based skills
* fix format issues
* refactor samples to use reflection
* Validate resource member signatures during discovery
Add discovery-time validation in AgentClassSkill.DiscoverResources() to
fail fast when [AgentSkillResource] is applied to members with incompatible
signatures:
- Reject indexer properties (getter has parameters)
- Reject methods with parameters other than IServiceProvider or
CancellationToken
Throws InvalidOperationException with actionable error messages instead of
allowing silent runtime failures when ReadAsync invokes the AIFunction with
no named arguments.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* prevent duplicates
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Bump Python version to 1.1.0 for a release
* Fix changelog
* 1.0.1 instead of 1.1.0
* Update CHANGELOG.md
* update version and changelog
* Bump lower bounds
* Python: Migrate GitHub Copilot package to SDK 0.2.x
Replace all imports from the non-existent copilot.types module with
correct SDK 0.2.x module paths (copilot.session, copilot.client,
copilot.tools, copilot.generated.session_events). Fix PermissionRequest
attribute access from dict-style .get() to dataclass attribute access.
Add OTel telemetry support to Copilot samples via configure_otel_providers
and document new telemetry environment variables in samples README.
* Python: Fix remaining copilot.types import in sample validation script
* Python: Include model in default_options for telemetry span attributes
* Python: Address review feedback on log_level and session kwargs typing
* Python: Scope PR to SDK 0.2.x migration only, remove net-new OTel features
- Remove RawGitHubCopilotAgent split and AgentTelemetryLayer inheritance
- Remove TelemetryConfig plumbing and OTLP/file telemetry settings
- Remove configure_otel_providers() calls from samples
- Remove telemetry env var rows from samples README
- Retain only: import path fixes, PermissionRequest attribute access fix,
log_level default fix, session kwargs typed fix, dependency pin
* Python: Update tests for SDK 0.2.x API changes
- SubprocessConfig replaces CopilotClientOptions dict
- create_session and resume_session now use keyword args
- send and send_and_wait take plain string prompt instead of MessageOptions
- on_permission_request is always required; deny-all fallback replaces omission
* Python: Pin github-copilot-sdk to >=0.2.0,<=0.2.0
Tighten the upper bound from <0.3.0 to <=0.2.0 to avoid pulling in 0.2.1+
which has breaking API changes relative to 0.2.0. The lower bound stays at
>=0.2.0 since this migration requires the 0.2.x import paths; 0.1.x would
fail at import time.
* Python: Pin github-copilot-sdk to >=0.2.1,<=0.2.1
---------
Co-authored-by: Evan Mattson <35585003+moonbox3@users.noreply.github.com>
* Harden Python checkpoint persistence defaults
Add RestrictedUnpickler to _checkpoint_encoding.py that limits which
types may be instantiated during pickle deserialization. By default
FileCheckpointStorage now uses the restricted unpickler, allowing only:
- Built-in Python value types (primitives, datetime, uuid, decimal,
collections, etc.)
- All agent_framework.* internal types
- Additional types specified via the new allowed_checkpoint_types
parameter on FileCheckpointStorage
This narrows the default type surface area for persisted checkpoints
while keeping framework-owned scenarios working without extra
configuration. Developers can extend the allowed set by passing
"module:qualname" strings to allowed_checkpoint_types.
The decode_checkpoint_value function retains backward-compatible
unrestricted behavior when called without the new allowed_types kwarg.
Fixes#4894
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: resolve mypy no-any-return error in checkpoint encoding
Add explicit type annotation for super().find_class() return value
to satisfy mypy's no-any-return check.
Fixes#4894
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Simplify find_class return in _RestrictedUnpickler (#4894)
Remove unnecessary intermediate variable and apply # noqa: S301 # nosec
directly on the super().find_class() call, matching the established
pattern used on the pickle.loads() call in the same file.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address review feedback for #4894: Python: Harden Python checkpoint persistence defaults
* Restore # noqa: S301 on line 102 of _checkpoint_encoding.py (#4894)
The review feedback correctly identified that removing the # noqa: S301
suppression from the find_class return statement would cause a ruff S301
lint failure, since the project enables bandit ("S") rules. This
restores consistency with lines 82 and 246 in the same file.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address review feedback for #4894: Python: Harden Python checkpoint persistence defaults
* Address PR review comments on checkpoint encoding (#4894)
- Move module docstring to proper position after __future__ import
- Fix find_class return type annotation to type[Any]
- Add missing # noqa: S301 pragma on find_class return
- Improve error message to reference both allowed_types param and
FileCheckpointStorage.allowed_checkpoint_types
- Add -> None return annotation to FileCheckpointStorage.__init__
- Replace tempfile.mktemp with TemporaryDirectory in test
- Replace contextlib.suppress with pytest.raises for precise assertion
- Remove unused contextlib import
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address PR #4941 review comments: fix docstring position and return type
- Move module docstring before 'from __future__' import so it populates
__doc__ (comment #4)
- Change find_class return annotation from type[Any] to type to avoid
misleading callers about non-type returns like copyreg._reconstructor
(comment #2)
Comments #1, #3, #5, #6, #7, #8 were already addressed in the current code.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address review feedback for #4894: review comment fixes
* fix: use pickle.UnpicklingError in RestrictedUnpickler and improve docstring (#4894)
- Change _RestrictedUnpickler.find_class to raise pickle.UnpicklingError
instead of WorkflowCheckpointException, since it is pickle-level concern
that gets wrapped by the caller in _base64_to_unpickle.
- Remove now-unnecessary WorkflowCheckpointException re-raise in
_base64_to_unpickle (pickle.UnpicklingError is caught by the generic
except Exception handler and wrapped).
- Expand decode_checkpoint_value docstring to show a concrete example of
the module:qualname format with a user-defined class.
- Add regression test verifying find_class raises pickle.UnpicklingError.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: address PR #4941 review comments for checkpoint encoding
- Comment 1 (line 103): Already resolved in prior commit — _RestrictedUnpickler
now raises pickle.UnpicklingError instead of WorkflowCheckpointException.
- Comment 2 (line 140): Add concrete usage examples to decode_checkpoint_value
docstring showing both direct allowed_types usage and FileCheckpointStorage
allowed_checkpoint_types usage. Rename 'SafeState' to 'MyState' across all
docstrings for consistency, making it clear this is a user-defined class name.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: replace deprecated 'builtin' repo with pre-commit-hooks in pre-commit config
pre-commit 4.x no longer supports 'repo: builtin'. Merge those hooks into
the existing pre-commit-hooks repo entry.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* style: apply pyupgrade formatting to docstring example
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: resolve pre-commit hook paths for monorepo git root
The poe-check and bandit hooks referenced paths relative to python/
but pre-commit runs hooks from the git root (monorepo root). Fix
poe-check entry to cd into python/ first, and update bandit config
path to python/pyproject.toml.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix pre-commit config paths for prek --cd python execution
Revert bandit config path from 'python/pyproject.toml' to 'pyproject.toml'
and poe-check entry from explicit 'cd python' wrapper to direct invocation,
since prek --cd python already sets the working directory to python/.
Also apply ruff formatting fixes to cosmos checkpoint storage files.
Fixes#4894
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: add builtins:getattr to checkpoint deserialization allowlist
Pickle uses builtins:getattr to reconstruct enum members (e.g.,
WorkflowMessage.type which is a MessageType enum). Without it in the
allowlist, checkpoint roundtrip tests fail with
WorkflowCheckpointException.
Fixes#4894
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address review feedback for #4894: review comment fixes
---------
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix reasoning text done events duplicating streamed delta content (#5157)
The OpenAI Responses API sends both reasoning_text.delta (incremental
chunks) and reasoning_text.done (full accumulated text) events. The
chat client was emitting Content for both, causing ag-ui to append the
full done text onto already-accumulated delta text, producing
duplicated reasoning output.
Stop emitting Content for reasoning_text.done and
reasoning_summary_text.done events, matching how output_text.done is
already handled (not emitted). The deltas contain all the content;
the done event is redundant.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(openai): emit reasoning done content as fallback when no deltas observed (#5157)
Address PR review feedback:
- Track item_ids that received reasoning deltas via seen_reasoning_delta_item_ids set
- Emit content from done events only when no deltas were received for the
item_id, preventing silent content loss on stream resumption
- Add comment documenting code_interpreter done event asymmetry
- Replace redundant ag-ui test with deduplication-focused test
- Add integration test for delta+done sequence in OpenAI chat client tests
- Add fallback path tests for done events without preceding deltas
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address review feedback for #5157: Python: [Bug]: "type": "response.reasoning_text.delta" and "response.reasoning_text.done" both get exposed as "text_reasoning"
* Fix AG-UI reasoning streaming to use proper Start/End pattern (#5157)
_emit_text_reasoning now follows the same streaming pattern as _emit_text:
- Emits ReasoningStartEvent/ReasoningMessageStartEvent only on the first
delta for a given message_id
- Emits only ReasoningMessageContentEvent for subsequent deltas
- Defers ReasoningMessageEndEvent/ReasoningEndEvent until
_close_reasoning_block is called (on content type switch or end-of-run)
This produces the correct protocol pattern:
ReasoningStartEvent
ReasoningMessageStartEvent
ReasoningMessageContentEvent(delta1)
ReasoningMessageContentEvent(delta2)
ReasoningMessageEndEvent
ReasoningEndEvent
Instead of wrapping every delta in a full Start→End sequence.
Backward compatibility is preserved: calling _emit_text_reasoning without
a flow argument still produces the full sequence per call.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix import ordering lint error in AG-UI test file (#5157)
Move inline import of TextMessageContentEvent to the top-level import
block and ensure alphabetical ordering to satisfy ruff I001 rule.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix mypy error: rename loop variable to avoid type conflict with WorkflowEvent
The 'event' variable was already typed as WorkflowEvent[Any] from the
async for loop at line 590. Reusing it in the _close_reasoning_block
loop (which returns list[BaseEvent]) caused an incompatible assignment
error. Renamed to 'reasoning_evt' to avoid the conflict.
Fixes#5162
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address review feedback for #5157: review comment fixes
* narrow test result reporting to explicit pytest JUnit XML
* Fix test args
* Fix pytest-results-action in merge workflow and remove committed test artifacts
Apply the same JUnit XML fix from python-tests.yml to python-merge-tests.yml:
add --junitxml=pytest.xml to all test commands and narrow the results action
path from ./python/**.xml to ./python/pytest.xml. Also remove accidentally
committed pytest.xml and python-coverage.xml and add them to .gitignore.
---------
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* .NET: Add JsonSerializerOptions support to programmatic skill APIs
Allow callers to pass custom JsonSerializerOptions when creating inline
resources and scripts via AgentInlineSkill, AgentClassSkill,
AgentInlineSkillResource, and AgentInlineSkillScript. A skill-level
default can be set on AgentInlineSkill and overridden per-resource/
script call.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Update dotnet/tests/Microsoft.Agents.AI.UnitTests/AgentSkills/TestSkillTypes.cs
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
_prepare_options() now removes tools, tool_choice, and parallel_tool_calls
from run_options after injecting agent_reference. The Foundry API rejects
requests containing both fields. FunctionTools are still invoked client-side
by the function invocation layer.
Fixes#5087
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Evan Mattson <35585003+moonbox3@users.noreply.github.com>
* Guard against empty text in _parse_structured_response_value (#5145)
When using response_format with background=True (Responses API), polling
an in-progress response produces empty text. _parse_structured_response_value
unconditionally passed this to model_validate_json/json.loads, causing
ValidationError or JSONDecodeError.
Add an early return of None when text is empty, matching the existing
guard for response_format=None. This allows .value to safely return None
for in-progress background responses.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: Fix `response_format` crash on background polling with empty text
Fixes#5145
---------
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Raise clear handler registration error for unresolved TypeVar (#4943)
Detect unresolved TypeVar in message parameter annotations during handler
registration in both _validate_handler_signature (Executor) and
_validate_function_signature (FunctionExecutor). Raises a ValueError with
an actionable message recommending @handler(input=..., output=...) or
@executor(input=..., output=...) instead of letting TypeVar leak through
to a confusing TypeCompatibilityError during workflow edge validation.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address review feedback for #4943: reorder checks and harden function executor
- Move TypeVar check before validate_workflow_context_annotation in
_executor.py so users see the more actionable error first
- Wrap get_type_hints in try/except in _function_executor.py matching
the defensive pattern in _executor.py
- Repurpose duplicate test to cover bounded TypeVar rejection
- Add test_function_executor_allows_concrete_types for test symmetry
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Narrow get_type_hints except clause and add missing tests (#4943)
- Narrow `except Exception` to `except (NameError, AttributeError, RecursionError)`
in both _executor.py and _function_executor.py so unexpected failures in
get_type_hints are not silently swallowed.
- Add test_handler_unresolvable_annotation_raises to test_function_executor_future.py
exercising the except branch of get_type_hints in the function executor path.
- Add test_function_executor_rejects_bounded_typevar_in_message_annotation to
test_function_executor.py for parity with the Executor bounded TypeVar test.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add error ordering test for TypeVar vs WorkflowContext priority (#4943)
Add test_handler_typevar_error_takes_priority_over_context_error to verify
that when a handler has both a TypeVar message and an unannotated ctx, the
TypeVar error is raised first (the more actionable issue).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: Fix image content serialization sending null file_id to Foundry API
Omit file_id from input_image dict when not present instead of including
it as null, which Azure AI Foundry's stricter schema validation rejects.
* Python: Fix Foundry API rejecting rich content in function_call_output
Azure AI Foundry does not support list-format output in function_call_output
items. Add SUPPORTS_RICH_FUNCTION_OUTPUT flag (default True) to
RawOpenAIChatClient, set to False in RawFoundryChatClient so Foundry
falls back to string output for tool results with images/files.
Also omit file_id from input_image dicts when not set, since Foundry
rejects explicit nulls.
* Python: Surface rich tool content as user message when Foundry lacks support
When SUPPORTS_RICH_FUNCTION_OUTPUT is False, image/file items from tool
results are injected as a follow-up user message so the model can still
process the visual content via Foundry's supported user message format.
* Xfail Foundry image integration test for the meantime
---------
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: Concurrent Workflow Sample
* Switch to using Azure AI Projects APIs
* Remove agent streaming outputs by changing emitEvents to false on TurnToken
* Disable forwarding input from agent host executors
* Make output format more legible
* refactor: Update Concurrent sample to use message delivery event callback
Adds a public CreateSessionAsync(string conversationId, CancellationToken)
method to FoundryAgent that delegates to the inner ChatClientAgent,
allowing users to create sessions with existing server-side conversation IDs.
Fixes#5138
* add class-based skills
* address formating issues
* Remove generated filtered-unit.slnx and add to .gitignore
The filtered solution file is generated dynamically by
eng/scripts/New-FilteredSolution.ps1 during CI. Checking it in
risks it becoming stale and out-of-sync with the real solution.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Remove generated filtered-unit.slnx and add to .gitignore
The filtered solution file is generated dynamically by
eng/scripts/New-FilteredSolution.ps1 during CI. Checking it in
risks it becoming stale and out-of-sync with the real solution.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* discover scripts and resource from folders defined in spec
* Remove Step05 and Step06 DI skill samples
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* address review comments
* fix build error
* Fix mixed path separators in skill folder discovery on .NET Framework
Path.Combine with forward-slash folder names (e.g. "scripts/f1") produces
mixed separators on Windows, causing the StartsWith containment check to
fail against Path.GetFullPath-resolved file paths. Wrap in Path.GetFullPath
to canonicalize separators before the containment comparison.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* address comment
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-07 12:15:46 +00:00
572 changed files with 26522 additions and 3782 deletions
@@ -9,9 +9,16 @@ The `verify-samples` project (`dotnet/eng/verify-samples/`) is an automated tool
## Running verify-samples
**Important:** By default, samples must be pre-built before running verify-samples. Build the solution first, or pass `--build` to build samples during the run:
varendpoint=Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT")??thrownewInvalidOperationException("AZURE_OPENAI_ENDPOINT environment variable is not set.");
$env:AZURE_OPENAI_DEPLOYMENT_NAME="gpt-4o-mini"# Optional, defaults to gpt-4o-mini
$env:AZURE_OPENAI_DEPLOYMENT_NAME="gpt-5.4-mini"# Optional, defaults to gpt-5.4-mini
```
**Note**: This demo uses Azure CLI credentials for authentication. Make sure you're logged in with `az login` and have access to the Azure OpenAI resource.
varendpoint=Environment.GetEnvironmentVariable("AZURE_AI_PROJECT_ENDPOINT")??thrownewInvalidOperationException("AZURE_AI_PROJECT_ENDPOINT is not set.");
@@ -22,5 +22,5 @@ Set the following environment variables:
```powershell
$env:AZURE_AI_PROJECT_ENDPOINT="https://your-foundry-service.services.ai.azure.com/api/projects/your-foundry-project"# Replace with your Microsoft Foundry resource endpoint
$env:AZURE_AI_MODEL_DEPLOYMENT_NAME="gpt-4o-mini"# Optional, defaults to gpt-4o-mini
$env:AZURE_AI_MODEL_DEPLOYMENT_NAME="gpt-5.4-mini"# Optional, defaults to gpt-5.4-mini
varendpoint=Environment.GetEnvironmentVariable("AZURE_AI_PROJECT_ENDPOINT")??thrownewInvalidOperationException("AZURE_AI_PROJECT_ENDPOINT is not set.");
@@ -22,5 +22,5 @@ Set the following environment variables:
```powershell
$env:AZURE_AI_PROJECT_ENDPOINT="https://your-foundry-service.services.ai.azure.com/api/projects/your-foundry-project"# Replace with your Microsoft Foundry resource endpoint
$env:AZURE_AI_MODEL_DEPLOYMENT_NAME="gpt-4o-mini"# Optional, defaults to gpt-4o-mini
$env:AZURE_AI_MODEL_DEPLOYMENT_NAME="gpt-5.4-mini"# Optional, defaults to gpt-5.4-mini
@@ -15,7 +15,7 @@ using Microsoft.Agents.AI.Foundry;
stringfoundryEndpoint=Environment.GetEnvironmentVariable("AZURE_AI_PROJECT_ENDPOINT")??thrownewInvalidOperationException("AZURE_AI_PROJECT_ENDPOINT is not set.");
varendpoint=Environment.GetEnvironmentVariable("AZURE_AI_PROJECT_ENDPOINT")??thrownewInvalidOperationException("AZURE_AI_PROJECT_ENDPOINT is not set.");
@@ -11,7 +11,7 @@ using Microsoft.Extensions.Hosting;
usingModelContextProtocol.Server;
varendpoint=Environment.GetEnvironmentVariable("AZURE_AI_PROJECT_ENDPOINT")??thrownewInvalidOperationException("AZURE_AI_PROJECT_ENDPOINT is not set.");
@@ -22,7 +22,7 @@ To use the [MCP Inspector](https://modelcontextprotocol.io/docs/tools/inspector)
1. Open a web browser and navigate to the URL displayed in the terminal. If not opened automatically, this will open the MCP Inspector interface.
1. In the MCP Inspector interface, add the following environment variables to allow your MCP server to access Microsoft Foundry Project to create and run the agent:
- AZURE_AI_PROJECT_ENDPOINT = https://your-resource.openai.azure.com/ # Replace with your Microsoft Foundry Project endpoint
- AZURE_AI_MODEL_DEPLOYMENT_NAME = gpt-4o-mini # Replace with your model deployment name
- AZURE_AI_MODEL_DEPLOYMENT_NAME = gpt-5.4-mini # Replace with your model deployment name
1. Find and click the `Connect` button in the MCP Inspector interface to connect to the MCP server.
1. As soon as the connection is established, open the `Tools` tab in the MCP Inspector interface and select the `Joker` tool from the list.
1. Specify your prompt as a value for the `query` argument, for example: `Tell me a joke about a pirate` and click the `Run Tool` button to run the tool.
varendpoint=Environment.GetEnvironmentVariable("AZURE_AI_PROJECT_ENDPOINT")??thrownewInvalidOperationException("AZURE_AI_PROJECT_ENDPOINT is not set.");
varbingConnectionId=Environment.GetEnvironmentVariable("AZURE_AI_BING_CONNECTION_ID")??thrownewInvalidOperationException("AZURE_AI_BING_CONNECTION_ID is not set.");
// Configure extended network timeout for long-running Deep Research tasks.
@@ -59,7 +59,7 @@ Set the following environment variables:
```powershell
$env:AZURE_OPENAI_ENDPOINT="https://your-resource.openai.azure.com/"# Replace with your Azure OpenAI resource endpoint
$env:AZURE_OPENAI_DEPLOYMENT_NAME="gpt-4o-mini"# Optional, defaults to gpt-4o-mini
$env:AZURE_OPENAI_DEPLOYMENT_NAME="gpt-5.4-mini"# Optional, defaults to gpt-5.4-mini
```
If the variables are not set, you will be prompted for the values when running the samples.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Tao Chen
L. Elaine DazzioandGitHub
Kartik MadanandGitHub
Roger BarretoandGitHub
Peter IbekweandGitHub
Dineshsuriya DandGitHub
