* Show more authentication methods in Foundry Toolbox MCP
* Remove hardcoded toolbox version num
* Add Foundry MCP OAuth consent handling
* Use message instead of the dedicated item type
* Go back to using OAuthConsentRequestOutputItem
* WIP: sample testing
* Update error code
* Address review on Foundry Toolbox MCP samples
Reviewed feedback addressed:
- Drop the branch-pinned `git+https://...@feature/...` entries from
`04_foundry_toolbox/requirements.txt`; restore the simple comment + `mcp`
runtime dep. The git pins were only useful while iterating on the PR and
shouldn't ship. (eavanvalkenburg)
- Fix the `/toolsets/` typo in both `04_foundry_toolbox/README.md` and
`06_files/README.md`. Verified empirically against the
research_toolbox in the test workspace: the toolbox MCP gateway lives at
`/toolboxes/{name}/mcp?api-version=v1` and requires the
`Foundry-Features: Toolboxes=V1Preview` header. `/toolsets/{name}/mcp`
returns 403 with `preview_feature_required: Toolsets=V1Preview` (a
different opt-in feature).
- Wrap `httpx.AsyncClient(...)` in `async with ... as http_client:` in both
samples so the connection pool is cleaned up. (Copilot reviewer)
- Make the `TOOLBOX_NAME` env var consistent in both samples. Previously the
tool name silently fell back to `"toolbox"` when `TOOLBOX_NAME` was unset,
but `resolve_toolbox_endpoint()` still required `TOOLBOX_NAME` and would
raise `KeyError`. The samples now resolve the endpoint once and derive the
tool name from the resolved URL when `TOOLBOX_NAME` isn't set, so the
local tool name always matches the upstream toolbox identity regardless
of which env var the user set. (Copilot reviewer)
- Rename `_responses.is_consent_error` to `consent_url_from_error`: the
helper returns `str | None` (the consent URL), not a bool, so the new
name matches behavior. Update the test class accordingly. (eavanvalkenburg)
- Tighten `_handle_inner_agent`'s lazy-entry catch from `Exception` to
`AgentFrameworkException`, the type the MCP layer actually wraps consent
errors in via `MCPStreamableHTTPTool.__aenter__` →
`ToolExecutionException(inner_exception=mcp_error)`. Network failures,
cancellations, and other non-framework exceptions now propagate normally
instead of being briefly caught and re-raised. The test helper
`_make_consent_error` is updated to use `ToolExecutionException` so it
matches the real-world wrapping. (eavanvalkenburg)
- Clarify the `github_pat` description in `agent.manifest.yaml` to note
it's only needed when the PAT-based connection (`github-mcp-pat-conn`)
is chosen; users selecting the OAuth2 connection (`github-mcp-oauth-conn`)
can leave it empty. (Copilot reviewer)
Validation: ran both samples end-to-end against a real Foundry toolbox
(`research_toolbox`) -- the samples connect successfully and the agent
lists the toolbox's MCP tools (`api_specs___fetch_azure_rest_api_docs`,
etc.). `uv run poe test -P foundry_hosting` passes (119 tests), pyright +
mypy clean.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* docs: fix broken Foundry samples link in 04_foundry_toolbox README
The previous URL pointed to an old location of the toolbox supported-scenarios
doc; the doc moved to /samples/python/hosted-agents/SUPPORTED_TOOLBOX_SCENARIOS.md
and the old /samples/python/toolbox/azd path now 404s.
Caught by the markdown-link-check CI step.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Eduard van Valkenburg <eavanvalkenburg@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Enable instrumentation by default
* Update samples
* Optimization when span is not recording
* Address Copilot comments
* Revert uv.lock
* Add warning
* Formatting
* Fix mypy
* Add disable_instrumentation() with sticky user-intent semantics
Add a public disable_instrumentation() entry point so users can explicitly opt
out of Agent Framework telemetry, with a sticky-disable flag that makes the
user's intent "leading" — no framework code path (foundry's
configure_azure_monitor, configure_otel_providers, enable_instrumentation,
enable_sensitive_telemetry, or direct OBSERVABILITY_SETTINGS.enable_*
writes) can re-enable instrumentation until the user explicitly clears the
disable with enable_instrumentation(force=True) /
enable_sensitive_telemetry(force=True).
Also addresses the two remaining unresolved review threads on the PR:
1. test_observability_settings_defaults_instrumentation_true pins the new
"ENABLE_INSTRUMENTATION defaults to True when env unset" behavior.
2. test_enable_instrumentation_reads_env_sensitive_data restores coverage
for the post-import load_dotenv() fallback path.
Implementation:
- ObservabilitySettings.enable_instrumentation / enable_sensitive_data become
properties backed by _enable_*. While _user_disabled is True, the getters
return False and the setters drop True writes (defense in depth so third-
party writes can't subvert the disable).
- Public is_user_disabled read-only property lets integrations (e.g. foundry's
configure_azure_monitor) cheaply check the disable state without poking at
privates.
- enable_instrumentation() and enable_sensitive_telemetry() short-circuit with
an info log when disabled; gain a force=True kwarg that clears the disable.
- configure_otel_providers() still creates providers / exporters / views so a
later force-enable can use them, but logs an info message when called while
disabled.
- Foundry's FoundryChatClient.configure_azure_monitor and
FoundryAgent.configure_azure_monitor early-return when the user has
disabled, so Azure Monitor's global providers aren't installed unnecessarily.
Tests: 11 new tests covering default-on, env re-read at call time, sticky
behavior against each re-enable surface (enable_instrumentation,
enable_sensitive_telemetry, configure_otel_providers, direct attribute
writes), force=True override, re-arming the disable, and the __all__ export.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* docs: document disable_instrumentation() and force=True paths
Add a "Disabling instrumentation" section to the observability sample README
that walks through:
- The distinction between the ENABLE_INSTRUMENTATION env var (initial,
non-sticky) and disable_instrumentation() (process-wide, sticky).
- Why the sticky semantics matter: framework integrations like
FoundryChatClient.configure_azure_monitor() can call
enable_instrumentation() as part of their setup, and the user's opt-out
needs to win.
- All five surfaces guarded by the sticky disable (property reads, public
enable functions, configure_otel_providers, direct attribute writes,
is_user_disabled-aware integrations).
- The force=True escape hatch on both enable_instrumentation() and
enable_sensitive_telemetry().
- How third-party integrations should consult OBSERVABILITY_SETTINGS.is_user_disabled.
- The limits of the disable (does not tear down existing providers /
in-flight spans / third-party instrumentation, does not persist across
processes).
Cross-links the new section from the ENABLE_INSTRUMENTATION row in the env
vars table.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* docs: soften disable_instrumentation() overclaim about telemetry guarantees
Replace 'no telemetry will be emitted no matter what' (which is too strong,
since callers can still pass force=True or mutate private attributes) with
language framing the disable as a user-intent contract that library and
framework code is expected to honor: the framework actively short-circuits
the public enable paths, force=True and private-attribute writes are
acknowledged as out-of-contract escape hatches that integrations should
not use on the user's behalf.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* docs: correct observability Dependencies section
- opentelemetry-sdk is no longer a hard dependency; it is lazily imported by
create_resource(), create_metric_views(), and configure_otel_providers()
with a clear ImportError when missing. Day-to-day instrumentation works
with opentelemetry-api alone provided some other component configures the
global OpenTelemetry providers (Azure Monitor, an APM agent, application
bootstrap, etc.).
- opentelemetry-semantic-conventions-ai is no longer used anywhere in the
source; remove it from the listed dependencies.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* docs: replace stale observability migration guide with current PR's only relevant migration
The old guide documented the move away from setup_observability(otlp_endpoint=...)
which was an earlier-release API change unrelated to this PR and stale enough that
it's more confusing than helpful at this point. Replace it with a short note on the
single migration this PR introduces: callers of
enable_instrumentation(enable_sensitive_data=True) should switch to
enable_sensitive_telemetry(). Cross-link to the Disabling instrumentation section
for the rare 'force on without enabling sensitive data' use case where
enable_instrumentation() still applies.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Eduard van Valkenburg <eavanvalkenburg@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Record actual served model as response model for Azure OpenAI
* Formatting
* Fix tests
* Fix pipeline error
* Comments
* Address review: surface served model via ChatResponse.model
Apply blocking review feedback from PR #5910:
- Use ChatResponse.model / ChatResponseUpdate.model as the source of truth
for the Azure x-ms-served-model header value, instead of stashing it in
additional_properties and overriding it again in observability.
Observability already reads response.model; the chat client now overwrites
it post-parse when the served-model header is present. Empirically the
Azure Responses API returns the deployment alias in body.model and the
actual snapshot (e.g. gpt-5-nano-2025-08-07) in this header.
- Move the AZURE_OPENAI_SERVED_MODEL_HEADER constant out of observability.py
and into RawOpenAIChatClient (as the SERVED_MODEL_HEADER ClassVar). The
header is Azure-OpenAI-Responses-API-specific so observability does not
need to know about it.
- Revert the streaming text_format path to client.responses.stream(...) and
drop the _pydantic_model_to_text_format_param helper. That helper imported
from openai.lib._parsing._responses (a private SDK path) and the swap to
responses.create(stream=True) dropped client-side output_parsed for
structured-output streaming. The streaming-with-text_format path is the
only one that does not surface the served-model header - documented inline.
- Wrap the raw streaming responses in async with so the underlying socket
closes deterministically (continuation_token retrieve + create paths).
- Fix the empty-string / whitespace-only header at the source by stripping
in _extract_served_model and returning None when nothing remains.
- Revert unrelated formatting-only churn in _skills.py and test_mcp.py.
- Update unit tests to assert against chat_response.model / update.model
and add an aggregated streaming assertion plus a pin that the
streaming-with-text_format path does not get the header.
Verified end-to-end against Azure OpenAI Responses API: deployment alias
gpt-5-nano now reports gpt-5-nano-2025-08-07 as ChatResponse.model in both
the non-streaming and streaming paths.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: preserve streaming structured output finalization
Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/f62076ef-558d-49e8-8fe2-f38d527c9639
Co-authored-by: eavanvalkenburg <13749212+eavanvalkenburg@users.noreply.github.com>
* refactor: name streaming response finalizer
Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/f62076ef-558d-49e8-8fe2-f38d527c9639
Co-authored-by: eavanvalkenburg <13749212+eavanvalkenburg@users.noreply.github.com>
* fix: capture streaming response format after prepare
Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/f62076ef-558d-49e8-8fe2-f38d527c9639
Co-authored-by: eavanvalkenburg <13749212+eavanvalkenburg@users.noreply.github.com>
* refactor: clarify streaming response format capture
Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/f62076ef-558d-49e8-8fe2-f38d527c9639
Co-authored-by: eavanvalkenburg <13749212+eavanvalkenburg@users.noreply.github.com>
* test: use public API for streaming structured output
Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/f62076ef-558d-49e8-8fe2-f38d527c9639
Co-authored-by: eavanvalkenburg <13749212+eavanvalkenburg@users.noreply.github.com>
* Inline the served-model header override at its two call sites
The `_apply_served_model_header` helper was a 1-line wrapper around
`_extract_served_model`. Inlining the `if served_model is not None: ...`
matches the pattern already used in the streaming paths and folds the
explanatory docstring onto `_extract_served_model` (which is now the
single place that knows about the header).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Eduard van Valkenburg <eavanvalkenburg@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: eavanvalkenburg <13749212+eavanvalkenburg@users.noreply.github.com>
* Python: Wrapper + Samples 1st (#5177)
* Experiment
* Update dependency and add non streaming
* Add more samples
* Rename samples
* Add invocations
* Comments 1
* Comments 2
* Comments 3
* Improve README
* Add local shell sample
* WIP: Add eval and memory samples
* Update user agent prefix
* Update user agent prefix doc
* Update dependency (#5215)
* Add tests and more content types (#5235)
* Add tests
* fix tests and sample
* Fix formatting
* Remove function approval contents
* Python: Refine samples and upgrade packages (#5261)
* Refine samples and upgrade pacakges
* Upgrade to a new package that fixes a bug
* Update model env var
* Move samples (#5281)
* Python: Upgrade agentserver packages (#5284)
* Upgrade agentserver packages
* Fix new types
* Python: Add special handling for workflows (#5298)
* Add special handling for workflows
* Address comments
* Improve samples (#5372)
* Python: Add more types (#5378)
* Add more type supports
* Upgrade packages
* Remove TODOs in README
* Fix README
* Comments and mypy
* User agent scoped
* Fix README
* Fix pre commit
* Fix pre commit 2
* Fix pre commit 3
* Fix pre commit 4
* Fix pre commit 5
* Fix pre commit 6
* Add azure-monitor-opentelemetry to dev deps
Fixes Samples & Markdown CI failure. The PR's new transitive dep on
azure-monitor-opentelemetry-exporter (via azure-ai-agentserver-core) makes
pyright resolve the azure.monitor.opentelemetry namespace, flipping the
check_md_code_blocks diagnostic for `configure_azure_monitor` from
reportMissingImports (filtered) to reportAttributeAccessIssue (not filtered).
Installing the umbrella azure-monitor-opentelemetry package in dev makes
pyright resolve the symbol correctly, matching the install guidance the
observability README already gives users.
---------
Co-authored-by: Evan Mattson <evan.mattson@microsoft.com>
* feat(workflows): Make telemetry opt-in via WithOpenTelemetry()
- Add WorkflowTelemetryOptions class with EnableSensitiveData property
- Add WorkflowTelemetryContext to manage ActivitySource lifecycle
- Add WithOpenTelemetry() extension method on WorkflowBuilder
- Update all workflow components to use telemetry context:
- WorkflowBuilder, Workflow, Executor
- InProcessRunnerContext, InProcessRunner
- LockstepRunEventStream, StreamingRunEventStream
- All edge runners (Direct, FanIn, FanOut, Response)
- Telemetry is now disabled by default
- Users must call WithOpenTelemetry() to enable spans/activities
BREAKING CHANGE: Workflow telemetry is now opt-in. Users who relied on
automatic telemetry must add .WithOpenTelemetry() to their workflow builder.
* refactor: Pass telemetry context as parameter instead of via interface
- Remove IWorkflowContextWithTelemetry interface
- Add internal ExecuteAsync overload that accepts WorkflowTelemetryContext
- Public ExecuteAsync delegates with WorkflowTelemetryContext.Disabled
- InProcessRunner passes TelemetryContext when calling ExecuteAsync
- BoundContext now implements IWorkflowContext (not the removed interface)
* Add optional ActivitySource parameter to WithOpenTelemetry
Allow users to provide their own ActivitySource when enabling telemetry,
giving them better control over the ActivitySource lifecycle. When not
provided, the framework creates one internally (existing behavior).
Changes:
- Add optional activitySource parameter to WithOpenTelemetry() extension
- Update WorkflowTelemetryContext to accept external ActivitySource
- Add unit test for user-provided ActivitySource scenario
* Add component-level telemetry control with disable flags
Allow users to selectively disable specific activity types via
WorkflowTelemetryOptions. All activities are enabled by default.
New disable flags:
- DisableWorkflowBuild: Disables workflow.build activities
- DisableWorkflowRun: Disables workflow_invoke activities
- DisableExecutorProcess: Disables executor.process activities
- DisableEdgeGroupProcess: Disables edge_group.process activities
- DisableMessageSend: Disables message.send activities
Added helper methods to WorkflowTelemetryContext for each activity type
and updated all activity creation sites to use them.
* Implement EnableSensitiveData to log executor input/output
When EnableSensitiveData is true in WorkflowTelemetryOptions, executor
input and output are logged as JSON-serialized attributes in the
executor.process activity.
New activity tags:
- executor.input: JSON serialized input message
- executor.output: JSON serialized output result (non-void only)
Added suppression attributes for AOT/trimming warnings since this is
an opt-in feature for debugging/diagnostics.
* Refactor activity start methods to centralize tagging logic
Move tagging logic into WorkflowTelemetryContext methods:
- StartExecutorProcessActivity now accepts executorId, executorType,
messageType, and message; sets all tags including executor.input
when EnableSensitiveData is true
- Added SetExecutorOutput method to set executor.output after execution
- StartMessageSendActivity now accepts sourceId, targetId, and message;
sets all tags including message.content when EnableSensitiveData is true
Simplified Executor.cs and InProcessRunnerContext.cs by removing
inline tagging code. Added message.content tag constant.
* Revert Python changes
* Update samples and code cleanup
* Fix file formatting
* Add comment
* Add telemetry configuration to declarative workflow
* Remove delays in tests
* Address comments
* WIP: with_output_from
* Add with_output_from to other modules; next: workflow as agent
* WIP: remove agent run events
* orchestrations
* WIP: update samples; next start at guessing_game_With_human_input.py
* Update all samples
* WIP: consolidate workflow as agent streaming vs non-streaming
* Consolidate workflow as agent streaming vs non-streaming
* Move request info event processing to a share method
* Final pass on the samples
* Fix mypy
* Fix mypy
* Comments
---------
Co-authored-by: Evan Mattson <35585003+moonbox3@users.noreply.github.com>
* Python: Add coverage threshold gate for PR checks (#3392)
- Add python-check-coverage.py script to enforce coverage threshold on specific modules
- Modify python-test-coverage.yml to run coverage check after tests
- Initial enforced module: agent_framework_azure_ai at 85% threshold
- Other modules are reported for visibility but don't block merges
* Fail if module not found
* Force unit test job to run
* Comment 1
* Fix coverage check to use full package paths for submodule support
* Update report format
* Group chat refactoring Part 1; Next: HIL and handoff
* Add agent approval flow; next samples
* WIP: samples
* WIP: HIL samples
* Group chat HIL working; next: handoff
* Fix group chat tool approval sample
* WIP: refactor handoff; next handoff handling
* Handoff done; next handoff samples and concurrent and sequential
* Handoff samples, concurrent, and sequential done; next Magentic
* WIP: magentic; next test with samples + HIL
* Magentic Working; next fix all samples and tests
* Fix handoff samples; next tests
* WIP: fixing tests; some orchestration as agent samples are failing
* Group chat unit tests done
* Handoff unit tests done
* Remove old orchestration_request_info and fix related tests
* Magentic unit tests done
* Fix samples
* Fix test
* Fix test 2
* mypy
* Address comments
* Update readme
* Address comments
* Address comments 2
* Replace display name
* Add factory pattern to sequential orchestration builder
* Use temp list to avoid override
* Add sample and some other fixes
* Fix comments
* Small fix
* Update readme
* Add workflow as an agent with observability sample
* Address comment
* Fix formatting
* enable sensitive data
* enable sensitive data for sub agents
* adjust aggregator handlers
* Prototype: Add request_info API and @response_handler
* Add original_request as a parameter to the response handler
* Prototype: request interception in sub workflows
* Prototype: request interception in sub workflows 2
* WIP: Make checkpointing work
* checkpointing with sub workflow
* Fix function executor
* Allow sub-workflow to output directly
* Remove ReqeustInfoExecutor and related classes; Debugging checkpoint_with_human_in_the_loop
* Fix Handoff and sample
* fix pending requests in checkpoint
* Fix unit tests
* Fix formatting
* Resolve comments
* Address comment
* Add checkpoint tests
* Add tests
* misc
* fix mypy
* fix mypy
* Use request type as part of the key
* Log warning if there is not response handler for a request
* Update Internal edge group comments
* REcord message type in executor processing span
* Update sample
* Improve tests
* Further observability cleanup and update telemetry samples
* Add VS Code Extension config
* Fix unit tests
* Fix unit tests
* Add more comments
* Remove live metric
* Improve type utils
* Add sample
* Add Union
* Add more test cases
* Add more test cases
* Fix RequestResponse typing to only coerce mapping original_request
---------
Co-authored-by: Evan Mattson <evan.mattson@microsoft.com>
* Introducing edge groups
* Add conditional and partitioning edge groups; next add samples and tests
* Add unit tests
* Add samples
* Address comments 1
* Address comments 2
* Update conditional edge group to take in cases and default
* Minor updates to sample
* Collapsing Paritioning Edge group and Conditional Edge group to source edge group
* Improve sample clarity
* Name consolidation
---------
Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
Tao Chen
