Compare commits

...
Author SHA1 Message Date
Giles Odigwe e33d3e5bc6 remove load_dotenv from test file 2026-04-30 14:02:42 -07:00
Giles OdigweandCopilot 097095c1ea Fix Ollama pull failure propagation and Azure OpenAI vector store readiness
- Ollama CI: fail the step immediately if model pull fails after 3
  retries instead of silently proceeding to tests
- Azure OpenAI file search: add the same vector-store readiness polling
  that was applied to the non-Azure OpenAI tests, preventing eventual
  consistency race conditions

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-30 11:42:18 -07:00
Giles Odigwe 0edd5f1b32 Merge branch 'main' into flaky-test-report 2026-04-30 10:28:28 -07:00
Giles OdigweandCopilot 52589ab474 Rename flaky_report to integration_test_report and add try/finally cleanup
- Rename scripts/flaky_report/ to scripts/integration_test_report/ to
  reflect expanded scope beyond flaky-test detection
- Update workflow references in both CI files
- Wrap file search integration tests in try/finally to ensure vector
  store cleanup runs even on test failure or timeout

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-30 10:22:00 -07:00
Giles Odigwe d2de5ba1b5 Revert "Re-enable workflow parallel tests with xdist_group marker"
This reverts commit 455c28da62.
2026-04-30 09:43:06 -07:00
6cd81286a9 .NET: dotnet: Add hosted-agent User-Agent supplement to outgoing requests (#5453)
* dotnet: Add hosted-agent User-Agent supplement to outgoing requests

When an agent runs inside a Foundry Hosted Agent, the outgoing
User-Agent header now includes 'agent-framework-hosted/{version}'
alongside the existing 'MEAI/{version}' segment.

- Add HostedAgentContext with AsyncLocal<string?> property
- MeaiUserAgentPolicy reads the supplement per-call
- AgentFrameworkResponseHandler sets/restores the context

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: update hosted UA format to foundry-hosting/agent-framework-dotnet/{version}

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Trying to get UA flowing, no luck yet.

* .NET: Polyfill MEAI OpenAIResponsesChatClient to add hosted-agent User-Agent supplement

When AgentFrameworkResponseHandler resolves an agent (i.e. we are running in a
hosted context), TryApplyUserAgent walks the agent's IChatClient decorator chain
to find MEAI's internal OpenAIResponsesChatClient and reflectively swaps its
inner _responseClient field with a DelegatingResponsesClient wrapper. The
wrapper overrides the public-virtual protocol methods to add a per-call
HostedAgentUserAgentPolicy to the RequestOptions and delegate to the inner
ResponsesClient. The OpenAI SDK's internal streaming overloads bottom out in
calls to the public-virtual non-streaming overloads via virtual dispatch on
this, so streaming is covered without overriding any non-virtual member.

The wrapper accepts any ResponsesClient-derived inner — both the Foundry
ProjectResponsesClient and the native OpenAI ResponsesClient — and preserves
the inner client's full pipeline (Transport, RetryPolicy, NetworkTimeout,
OrganizationId / ProjectId / UserAgentApplicationId, custom policies).

- Add DelegatingResponsesClient + HostedAgentUserAgentPolicy in Microsoft.Agents.AI.Foundry.Hosting.
- Add TryApplyUserAgent next to ApplyOpenTelemetry in FoundryHostingExtensions; wire it into AgentFrameworkResponseHandler.GetAgent for both keyed and default-agent paths.
- Drop earlier-iteration dead code: AddHostedAgentTelemetry extension, HostedUserAgentPolicy class, HostedAgentContext.cs, and the never-called ToRequestOptions helper.
- Revert RequestOptionsExtensions.MeaiUserAgentPolicy to MEAI-only (the supplement is now injected by the polyfill).
- Revert unrelated whitespace change in Agent_Step25_ToolboxServerSideTools sample.
- Tests cover streaming AND non-streaming, retry policy preservation, OrganizationId/ProjectId/UserAgentApplicationId pass-through, idempotency, native OpenAI ResponsesClient, and reflection guards for MEAI/OpenAI shape drift.

* .NET: Address review feedback on hosted-agent User-Agent polyfill

- TryApplyUserAgent: replace silent null-return with ArgumentNullException to match the codebase's convention.
- Add idempotency test (TryApplyUserAgent_CalledTwiceOnSameAgent_DoesNotDoubleWrap) — runs the polyfill twice on the same agent and asserts the wire UA contains exactly one foundry-hosting segment, proving the 'current is DelegatingResponsesClient' guard prevents nested wrapping.
- Add retry-double-append test (Polyfill_RetryWithinCall_DoesNotDuplicateSupplementInUserAgent) — exercises the HostedAgentUserAgentPolicy Contains-guard via a custom retry policy that re-runs the inner pipeline on the same message.
- Replace TryApplyUserAgent_NullAgent_ReturnsNullWithoutThrowing with TryApplyUserAgent_NullAgent_ThrowsArgumentNullException to match the new contract.

* .NET: Drop null check from TryApplyUserAgent and its now-redundant test

The two call sites in AgentFrameworkResponseHandler.GetAgent already null-check the agent before invoking TryApplyUserAgent, so the defensive ArgumentNullException is unreachable. Remove it and the corresponding test.

* .NET: Remove unused Microsoft.Shared.Diagnostics import in ServiceCollectionExtensions

The Throw.IfNull helper from this namespace was used by the now-removed null check in TryApplyUserAgent. Drop the unused import to satisfy IDE0005 in CI's full-project dotnet format run.

---------

Co-authored-by: alliscode <bentho@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Roger Barreto <19890735+rogerbarreto@users.noreply.github.com>
2026-04-30 16:37:54 +00:00
Giles OdigweandCopilot 455c28da62 Re-enable workflow parallel tests with xdist_group marker
The tests were skipped because xdist distributes module tests across
workers, each spawning their own func process (port conflicts). Adding
xdist_group forces all tests in this module onto a single worker so
the module-scoped function_app_for_test fixture works correctly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-30 09:04:46 -07:00
Giles OdigweandCopilot 7ce27ddda3 Increase reliable streaming test timeouts from 30s to 60s
The LLM call through Azure OpenAI + Redis streaming pipeline can exceed
30s in CI due to cold starts or throttling. Raise to 60s to reduce
flaky timeouts while still bounded by pytest's 120s per-test limit.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-30 08:42:08 -07:00
Giles OdigweandCopilot acf24ea2e4 Stabilize Ollama tool call integration tests with no-arg function
Use a no-argument greet() function instead of hello_world(arg1) for
integration tests. The 1.5B model in CI is unreliable at generating
correct tool call arguments, causing 'Argument parsing failed' errors.
A no-arg function eliminates this flakiness entirely.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-30 00:36:09 -07:00
Giles OdigweandCopilot 3ab3370a8e Remove temperature from foundry hosting test (unsupported by CI model)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-29 15:31:57 -07:00
Giles OdigweandCopilot 072123a8f1 Fix flaky integration tests and re-enable skipped tests
- Foundry agent: add allow_preview=True to custom client test
- Foundry hosting: raise max_output_tokens 50->200, add temperature,
  relax assertion in test_temperature_and_max_tokens
- Foundry embedding: update skip reason with root cause (endpoint mismatch)
- OpenAI file search: fix vector store indexing race condition by polling
  file_counts before querying; fix get_streaming_response -> get_response(stream=True)
- Azure OpenAI file search: remove skip (transient 500 resolved)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-29 15:06:17 -07:00
Peter IbekweandGitHub 6853f64de8 .NET: Add declarative HttpRequestAction sample (#5572)
* Add declarative HttpRequestAction support to workflows

* Clean up response body for diagnostics  and fix tests.

* Fix merge with main.

* Remove redundant fallback for request content headers.

* Add declarative InvokeHttpRequest sample

* Fix solution file and update sample yaml comments

* Add final newline to sample class to fix formatting failure
2026-04-29 19:19:31 +00:00
570a4d54c2 Python: Support OpenAI and Gemini allowed_tools tool choice (#5322)
* Support OpenAI allowed_tools in ToolMode (#5309)

Add allowed_tools field to ToolMode TypedDict, enabling users to restrict
which tools the model may call via the OpenAI allowed_tools tool_choice
type. This preserves prompt caching by keeping all tools in the tools list
while limiting which ones the model can invoke.

- Add allowed_tools: list[str] to ToolMode TypedDict
- Add validation in validate_tool_mode() (only valid when mode == "auto")
- Convert to OpenAI API format in _prepare_options()
- Add tests for validation and API payload generation

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Support OpenAI `allowed_tools` tool choice in Python SDK

Fixes #5309

* Fix #5309: Validate allowed_tools shape and add Chat Completions client support

- validate_tool_mode now checks allowed_tools is a non-string sequence of
  strings and normalizes to list[str], raising ContentError for invalid types
- Add missing allowed_tools branch in _chat_completion_client._prepare_options
  so allowed_tools is emitted as the OpenAI allowed_tools wire format instead
  of being silently dropped
- Add tests for invalid allowed_tools types (string, int, mixed), empty list,
  tuple normalization, and Chat Completions client payload generation

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: support allowed_tools with mode 'required' in addition to 'auto'

OpenAI's allowed_tools tool_choice type supports both mode 'auto' and
'required'. Update validation, client conversion, and tests to allow
both modes instead of restricting to 'auto' only.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: use Gemini VALIDATED mode for allowed_tools, warn in unsupported providers

- Use FunctionCallingConfigMode.VALIDATED instead of ANY when allowed_tools
  is set with auto mode in Gemini, preserving optional tool-call semantics.
- Handle allowed_tools in required mode with required_function_name precedence.
- Fix allowed_names guard to use identity check (is not None) so empty lists
  are preserved.
- Bump google-genai minimum to >=1.32.0 (VALIDATED added in that version).
- Add warnings in Anthropic and Bedrock when allowed_tools is set but not
  supported.
- Add Gemini unit tests for allowed_tools with auto, required, empty list,
  and required_function_name precedence scenarios.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: Chat Completions API does not support allowed_tools, add integration tests

- Chat Completions API (_chat_completion_client.py) now warns and falls
  back to plain mode when allowed_tools is set, since the /chat/completions
  endpoint does not support the allowed_tools type.
- Add allowed_tools integration test param to both OpenAIChatClient
  (Responses API) and OpenAIChatCompletionClient parametrized option tests.
- Update Chat Completions unit tests to reflect the warn-and-fallback
  behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: remove unused walrus operator variable in chat completion client

Remove assigned-but-never-used variable 'allowed' flagged by ruff F841.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-29 17:43:47 +00:00
Giles Odigwe 2e6b999bd2 Merge branch 'main' into flaky-test-report 2026-04-29 08:55:29 -07:00
Evan MattsonandGitHub f5419b9f38 Python: bump package versions for 1.2.2 release (#5561)
* Python: bump package versions for 1.2.2 release

PATCH bump (1.2.1 -> 1.2.2) for the released cohort. Five PRs land in this
window:

- agent-framework-openai: fix file_search citations breaking the assistant-
  message history roundtrip (#5557) — drives the released-tier PATCH
- agent-framework-orchestrations: [BREAKING] standardize orchestration
  terminal outputs as AgentResponse (#5301)
- agent-framework-core, agent-framework-declarative: preserve Workflow.run()
  shared state across calls, accept list[Message] in declarative start
  executor, and coerce Enum values when serializing PowerFx symbols (#5531)
- agent-framework-foundry-hosting: add hosted Durable Workflow support
  (#5531)
- agent-framework-azure-contentunderstanding: new alpha package — Azure AI
  Content Understanding context provider (#4829)
- dependencies: workspace package dependency refresh (#5555)

Per lockstep convention, all 21 beta packages stamp 1.0.0b260429 and all 4
alpha packages (now including the new contentunderstanding) stamp
1.0.0a260429. Date stamp reflects 2026-04-29 Pacific. Every non-core package
floor on agent-framework-core is raised to >=1.2.2; the new
contentunderstanding package's stale >=1.0.0 floor is brought into line.

Two follow-on fixes bundled to keep validate-dependency-bounds-test green
at lowest-direct resolution:
- Bump agent-framework-azure-contentunderstanding's azure-ai-content
  understanding lower bound from >=1.0.0 to >=1.0.1 (1.0.0 ships without
  proper typing — pyright reports 65 unknown-type errors)
- Add pyright ignore comments to core/foundry/__init__.pyi for the new
  alpha package's type-stub imports, since alpha packages are not in
  core's [all] extra and therefore aren't installed at lowest-direct

* Python: add #5552 to 1.2.2 CHANGELOG

Add the streaming-span observability fix to the Fixed section. PR is on
upstream/main but not yet pulled into origin/main; the code itself will
land via the PR merge.

* Python: address PR #5561 review feedback on dependency bounds

Two packaging fixes flagged in review:

1. agent-framework-azure-contentunderstanding: add agent-framework-foundry
   as a runtime dependency. The package's README directs users to
   `pip install agent-framework-azure-contentunderstanding --pre` and the
   basic example imports `FoundryChatClient` from `agent_framework.foundry`,
   so the documented install path was failing with ImportError. Pulling
   agent-framework-foundry into deps makes the advertised entry path
   self-contained.

2. agent-framework-foundry: bump agent-framework-openai lower bound from
   >=1.1.0 to >=1.2.2,<2. Foundry imports private modules from
   agent_framework_openai (`_chat_client.py:22`, `_agent.py:34`), so
   resolvers were free to pair foundry==1.2.2 with older OpenAI versions
   that lack this release's coordinated Responses/history fix. Lockstep the
   floor with the released cohort to prevent mismatched installs.

Both changes pass `validate-dependency-bounds-test` lower + upper at
their respective packages.
2026-04-29 17:51:48 +09:00
Tao ChenandGitHub 03e47b5232 Python: Fix spans not correctly nested when using streaming (#5552)
* Fix spans not correctly nested when using streaming

* fix pre commit

* Address comments
2026-04-29 08:21:28 +00:00
Evan MattsonandGitHub 46ab47b9e1 Python: Fix file_search citations breaking assistant history roundtrip (#5557)
* Python: Fix file_search citations breaking assistant history roundtrip

The Responses API rejects 'input_file' inside an assistant message, but the
SDK was emitting it whenever an assistant Message contained a hosted_file
content (which is what file_search citations become). Three coordinated fixes:

1. _prepare_content_for_openai now skips hosted_file for the assistant role
   instead of mapping to input_file (which the API rejects there).

2. The streaming response.output_text.annotation.added handler attaches
   file_citation, container_file_citation, and file_path as annotations on
   text content, matching the non-streaming path. Previously streaming
   produced standalone HostedFileContent items that always tripped (1).

3. output_text serialization preserves Annotation objects on roundtrip via a
   new _annotations_to_output_text helper instead of hardcoding 'annotations'
   to []. file_search citations now survive multi-agent forwarding.

Closes #5556.

* Address PR review

- _annotations_to_output_text: fan out one entry per annotated_region for
  url_citation/container_file_citation (Annotation.annotated_regions is a
  Sequence; the API form carries one start/end per entry).
- Validate region span bounds are ints before emitting; skip otherwise.
- Add test for the file_path branch (annotation with file_id only).
- Add test verifying streamed citation events coalesce onto surrounding
  text via _finalize_response so span indices reference the merged text,
  not the empty-text streaming carrier.
2026-04-29 07:38:19 +00:00
Evan MattsonandGitHub 094f9903b3 Python: Update package dependencies (#5555)
* Update dependencies

* Preserve mcp[ws] and uvicorn[standard] extras in override-dependencies

Bare-package overrides on mcp and uvicorn dropped the [ws] and [standard]
extras (and their transitive deps like httptools, watchfiles) from the
generated lock. Re-add the extras to the overrides so the lock matches
what workspace packages actually request.
2026-04-29 06:18:03 +00:00
8b71f9459a Python: Feature/hosted dwf (#5531)
* Fix declarative Workflow.as_agent() by accepting list[Message] in start executor

The declarative start executor (JoinExecutor) only advertised dict and str
in its input_types, so WorkflowAgent.__init__ rejected it with
'Workflow's start executor cannot handle list[Message]'.

Add list[Message] to the JoinExecutor handler annotation and add a
matching branch in DeclarativeActionExecutor._ensure_state_initialized
that extracts the last user-message text and falls through to the
string-input initialization path, so =System.LastMessageText works
end-to-end via as_agent().

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Populate Conversation.messages from list[Message] trigger

When Workflow.as_agent() is invoked with a list[Message], the start executor now populates Conversation.messages / Conversation.history / System.conversations.{id}.messages with prior turns only (excluding the latest user message), and surfaces the latest user message via Inputs.input and System.LastMessage*. This matches InvokeAzureAgent's contract that the messages binding holds prior turns and the executor itself appends the new user input before invoking, avoiding double-append of the trailing user turn while preserving full history (incl. assistant/system/tool roles and multi-modal content) for downstream actions.

* Coerce Enum values when serializing PowerFx symbols

MessageRole and other str-subclass Enums passed isinstance(v, str) and were forwarded to pythonnet unchanged. pythonnet then raised 'MessageRole value cannot be converted to System.String' for every PowerFx primitive when ConditionGroup/Expr eval walked the symbol table containing Conversation.messages. Reduce Enum members to their underlying value before the primitive check so eval sees plain strings/ints.

* Foundry hosting: pass full conversation history to workflow agents

_handle_inner_workflow only forwarded the latest user turn to WorkflowAgent.run, even though _handle_inner_agent already prepends history fetched from Foundry storage to the messages it sends a regular agent. Declarative workflows reset Conversation.messages on every run (state.initialize), so checkpoint replay alone does not give them prior turns - the host has to pass them in, the same way it does for non-workflow agents. Mirror that contract: fetch context.get_history() and pass [*history, *input_messages] to the workflow agent.

* feat(workflows): support combined message + checkpoint_id for multi-turn continuation

Allow Workflow.run(message=..., checkpoint_id=...) so callers can restore
prior workflow state from a checkpoint AND deliver a new message to the
start executor in a single call. The existing reset_context logic
already preserves shared state when checkpoint_id is set, so this gives
us 'fresh start executor invocation with prior state intact' - exactly
what hosted multi-turn declarative workflows need.

- _workflow.py: drop the message+checkpoint_id mutual exclusion and
  update _execute_with_message_or_checkpoint to do both (restore then
  execute) when both are provided.
- _agent.py: in _run_core's checkpoint branch, also forward
  input_messages so WorkflowAgent.run(messages, checkpoint_id=...) works
  end-to-end. Falls back to the legacy 'restore only' behavior when
  messages are absent.
- _declarative_base.py: detect continuation in _ensure_state_initialized
  by checking whether DECLARATIVE_STATE_KEY already exists in shared
  state; if so, refresh inputs/LastMessage* and append non-user trigger
  messages instead of calling state.initialize() (which would wipe
  Conversation/Local/System).
- foundry_hosting/_responses.py: collapse the host's two-call pattern
  (restore-only, then fresh run) into a single combined call now that
  the underlying APIs support it.
- tests: drop the assertion that combined message+checkpoint_id raises.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Pivot: preserve workflow state across run() calls

Replace the prior 'combined message + checkpoint_id in one run()' approach
with a cleaner default: Workflow.run no longer wipes shared state or runner-
context messages between calls. Iteration counting and per-run kwargs still
reset on a fresh-message run; checkpoint and responses runs are continuations
that preserve everything.

This lets a WorkflowAgent be invoked repeatedly on the same instance and
maintain multi-turn context (e.g. accumulated Conversation.messages) without
asking developers to opt in. Hosted-agent multi-turn pattern becomes two
explicit calls: restore-from-checkpoint (drive to idle), then run-with-message.

Key changes:
- _workflow.py: drop _state.clear() and reset_for_new_run() from run().
  Reset iteration count and run kwargs on fresh-message runs only.
  Restore 'Cannot provide both message and checkpoint_id' validation.
  Add async guard: fresh-message run with un-drained pending executor
  messages from a prior run is invalid.
- _runner.py: clear _state before import_state in restore_from_checkpoint
  so restore is authoritative (import_state merges, not replaces).
- _agent.py: revert checkpoint branch to restore-only (no message forward).
- _responses.py (foundry_hosting): two-call host pattern - restore checkpoint
  silently, then run with new user input.
- tests: state-preservation is the new default; rebuild Workflow for clean slate.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix CI lint and mypy issues from prior pivot commit

- _workflow.py: collapse nested if (SIM102), drop redundant assignment (RET504)
- _declarative_base.py: remove unused last_user_msg = tail assignment
  whose Message | None type clashed with the prior Message-typed branch

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address PR review: fix Inputs.input update and checkpoint storage path

- _declarative_base.py: continuation branch was writing 'Inputs.input' via
  state.set, which routes to the Custom namespace and never updates the
  PowerFx-visible Workflow.Inputs.input. Update state_data['Inputs'] in
  place via get_state_data / set_state_data so =Workflow.Inputs.input and
  =inputs.input see the new turn's user text on continuation.
- _declarative_base.py: refresh docstring to clarify that on a list[Message]
  trigger, Conversation.messages excludes the current user message at the
  start of the turn (agent executors append it before invoking the inner
  agent).
- _responses.py: when previous_response_id is supplied (no conversation_id),
  the prior checkpoint lives under <storage>/<previous_response_id> but new
  checkpoints must land under <storage>/<current_response_id> for the next
  turn to find them. Hold onto restore_storage from the get_latest lookup
  and pass it to the restore-only run; pass write_storage (current id) to
  the message-delivery run and to checkpoint cleanup.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix pyright errors in _declarative_base.py for CI

- Replace state._state.get(...) protected access with new public
  is_initialized() method on DeclarativeWorkflowState (also clearer intent
  for the continuation detection use case).
- Add narrow pyright ignores for the Any-typed trigger paths that pyright
  cannot fully narrow (the list[Message] isinstance loop and the
  fallback-DefaultTransform branch).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address Copilot review batch: tests + Workflow.reset escape hatch

* Add Workflow.reset() public method as recovery escape hatch when an
  in-flight run aborted (e.g. WorkflowConvergenceException) and the
  workflow is not checkpointed. Update the in-flight messages guard's
  error message to point callers at it.

* Add test_workflow_run_inflight_messages_guard exercising both the
  guard (sync + streaming) and the reset() recovery path.
* Add test_workflow_reset_rejects_concurrent_runs to lock down the
  in-progress guard on reset.

* Add test_as_agent_continuation_preserves_prior_state covering the
  is_continuation branch in _ensure_state_initialized: stamps a marker
  between calls and asserts it survives, while Inputs.input and
  System.LastMessageText refresh to the new turn.

* Add test_powerfx_safe.py regression tests for the Enum branch in
  _make_powerfx_safe (str-subclass, int-subclass, plain Enum, and
  Enums nested in dict/list).

* Drop redundant @pytest.mark.asyncio on
  test_as_agent_round_trip_with_last_message_text (asyncio_mode='auto').

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Skip restore-only pre-pass when checkpoint has pending request_info

Address Copilot review on _responses.py: the restore-only checkpoint
replay populates self._agent.pending_requests for any request_info
events captured in the checkpoint. The follow-up run(input_messages)
call would then route through WorkflowAgent._process_pending_requests,
which expects function-response content and rejects plain text input
as 'unexpected content while awaiting request info responses'.

Workflows resumed from a checkpoint that was idle-with-pending-requests
would therefore fail every subsequent plain-text user turn. Inspect the
loaded checkpoint and skip the pre-pass when its
pending_request_info_events dict is non-empty. Workflows that don't use
request_info (the current sample set) are unaffected; workflows that do
will fall through to a fresh-message run rather than silently corrupting
the routing state.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Loosen azure-ai-agentserver-* pins to major version

The exact-version pins on azure-ai-agentserver-{core,responses,invocations}
forced foundry-hosting consumers to upgrade in lockstep with every beta
bump from upstream. Switch to '>=current,<next-major' so we pick up patch
and feature updates within the same major series without a coordinated
release.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Drop Workflow.reset(); checkpointing is the recovery path

The in-flight-messages guard prevented silent misbehavior, but the
companion Workflow.reset() escape hatch only cleared _messages while
leaving iteration count, executor-local state, and shared State
mutations in an indeterminate condition after a mid-run failure. That
gave a false sense of recovery.

Recovery from a mid-run failure is supported only via checkpoint
restoration. Keep the guard and reframe its error message accordingly;
remove reset() and its tests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address Tao's review on PR 5531

- Rename Workflow._run_workflow_with_tracing parameter
  is_fresh_message_run -> is_continuation (default False, inverted).
  Fresh-message turns reset per-run accounting; continuations
  (checkpoint restores, responses replays) preserve it.
- Simplify the in-flight-messages guard: _validate_run_params already
  enforces that 'message' is mutually exclusive with 'checkpoint_id'
  and 'responses', so the additional checks were dead code.
- foundry_hosting _responses: move the restore-only pre-pass above
  emit_created/emit_in_progress; restore is preparation, not run
  progress. Drop the skip-restore gate (state preservation requires
  unconditional restore) and instead clear agent.pending_requests
  after the restore-only call. Collapse over-conditioned check.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Don't clear pending_requests after restore-only pre-pass

Pending requests in the restored checkpoint represent genuinely
outstanding HITL requests. The next user input may carry function
responses (Responses API `function_call_output` items become
FunctionResultContent / FunctionApprovalResponseContent), which
`WorkflowAgent._process_pending_requests` correctly extracts and
matches against the populated `pending_requests`. Clearing them
after restore would silently drop that state and force the next turn
to be treated as a fresh input even when the caller is responding to
the outstanding requests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: alliscode <bentho@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Evan Mattson <35585003+moonbox3@users.noreply.github.com>
2026-04-29 00:51:49 +00:00
866a325b48 Python: [BREAKING] Standardize orchestration terminal outputs as AgentResponse (#5301)
* Fix orchestration outputs so as_agent() returns the final answer only. Align other orchestration outputs

* Fix orchestration output issues from review comments

1. Sample cleanup: Remove commented-out FoundryChatClient block and update
   prerequisites to reference OPENAI_CHAT_MODEL_ID instead of FOUNDRY_* vars.

2. Sequential approval output: Change _EndWithConversation.end_with_agent_executor_response
   from a no-op sink to yield response.agent_response. When the last participant is
   AgentApprovalExecutor (via with_request_info), _EndWithConversation is the output
   executor so the yield produces the terminal answer. When the last participant is a
   regular AgentExecutor, _EndWithConversation is not in output_executors so the yield
   is silently filtered out.

3. Forward data events through WorkflowExecutor: _process_workflow_result now also
   forwards 'data' events from sub-workflows so that emit_intermediate_data=True on
   AgentExecutor works correctly when wrapped in AgentApprovalExecutor.

4. Concurrent docstring: Update _AggregateAgentConversations docstring to say
   'deterministic participant order' instead of 'completion order'.

5. Add test_concurrent_intermediate_outputs_emits_data_events verifying that
   ConcurrentBuilder(intermediate_outputs=True) emits per-participant data events
   alongside the single aggregated output event.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add tests for sequential workflow with_request_info and intermediate_outputs (#5301)

Address PR review comments 2, 3, and 5:

- Add test_sequential_request_info_last_participant_emits_output:
  Verifies that when the last participant is wrapped via with_request_info()
  (AgentApprovalExecutor), the workflow still emits a terminal output after
  approval, exercising the _EndWithConversation.end_with_agent_executor_response
  fallback path.

- Add test_sequential_request_info_with_intermediate_outputs_emits_data_events:
  Verifies that emit_intermediate_data=True works correctly through
  AgentApprovalExecutor wrapping—WorkflowExecutor._process_result already
  forwards data events from sub-workflows, so intermediate agent responses
  surface as data events in the parent workflow.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix pyright type errors from AgentResponse output refactor (#5301)

Update cast() calls in _group_chat.py and _magentic.py to use
WorkflowContext[Never, AgentResponse] instead of the old
WorkflowContext[Never, list[Message]], matching the updated method
signatures in _base_group_chat_orchestrator.py.

Fix _sequential.py _EndWithConversation.end_with_agent_executor_response
to declare WorkflowContext[Any, AgentResponse] so yield_output accepts
AgentResponse[None].

Fix _workflow_executor.py data event forwarding to handle nullable
executor_id.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix pyright reportUnknownVariableType in _agent.py (#5301)

Extract event.data into a typed local variable before the isinstance
check to avoid pyright narrowing it to AgentResponse[Unknown].

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix pyright reportMissingImports for orjson in file history samples (#5301)

Add pyright: ignore[reportMissingImports] to orjson imports that are
already guarded by try/except ImportError, matching the existing pattern
used elsewhere in the samples.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address review feedback for #5301: review comment fixes

* Address review feedback for #5301: review comment fixes

* Revert sequential_workflow_as_agent sample to FoundryChatClient

Reverts the mistaken switch from FoundryChatClient to OpenAIChatClient
in the sequential workflow as agent sample.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address ultrareview feedback: emit_data_events rename + WorkflowAgent reasoning conversion

Layered on top of the prior review-feedback work in this branch.

Renames:
- AgentExecutor.emit_intermediate_data -> emit_data_events (mechanical
  rename; orchestration semantics live at the orchestration layer, not
  the general-purpose executor). Forwarded through MagenticAgentExecutor,
  AgentApprovalExecutor, and all orchestration call sites.
- HandoffAgentExecutor._check_terminate_and_yield -> _should_terminate
  (pure predicate; no longer yields anything). HandoffBuilder docstring
  rewritten to describe the new per-agent AgentResponse output contract.

WorkflowAgent reasoning-content conversion:
- Add _rewrite_text_to_reasoning(contents) and _msg_as_reasoning(msg)
  helpers; the as_agent() path now reframes text content from data events
  as text_reasoning Content blocks before merging into the AgentResponse.
- Consumers iterate msg.contents and branch on content.type — same path
  they already use for Claude thinking and OpenAI reasoning. No new
  field on Message/AgentResponse/WorkflowEvent.
- Streaming branch constructs fresh AgentResponseUpdate instances instead
  of mutating shared payloads (regression test added).
- Helper _msg_maybe_reasoning consolidates the conditional rewrite at
  three call sites in the non-streaming conversion.

Tests:
- TestWorkflowAgentReasoningHelpers + TestWorkflowAgentDataEventReasoningConversion
  add 9 new tests covering helpers, non-streaming, streaming, mixed content,
  already-reasoning passthrough, and mutation-safety regression.
- Updated test_sequential_as_agent_with_intermediate_outputs_includes_chain
  to assert text_reasoning content for intermediate agents.

* Fix pyright: widen event.data to Any to avoid partial-unknown narrowing

The streaming conversion path narrowed event.data via isinstance against
generic AgentResponse, producing AgentResponse[Unknown] and tripping
reportUnknownVariableType/reportUnknownMemberType. Binding data: Any
before the check keeps runtime behavior identical while restoring a fully
known type for downstream access.

* Clean up design

* Scope to agent output semantics only

* yield AgentResponseUpdate streaming, AgentResponse non-streaming

* Fix mypy/pyright: widen cast types at GroupChat callsites

Eight callsites in _group_chat.py still cast to WorkflowContext[Never,
AgentResponse] but the base orchestrator methods now accept the wider
WorkflowContext[Never, AgentResponse | AgentResponseUpdate] (mode-aware
yields). W_OutT is invariant, so the narrower cast is not assignable.
Magentic was widened in the same commit; this catches the GroupChat
callsites that were missed.

* Python: skip flaky Foundry / Foundry Hosting integration tests (#5553)

These two integration tests have been failing in the merge queue across
multiple unrelated PRs (5301, 5531). Both are marked `@pytest.mark.flaky`
with 3 retries, but all attempts fail back-to-back. Skipping both with a
reason pointing to #5553 so they can be fixed properly without continuing
to block unrelated merges.

- packages/foundry_hosting/tests/test_responses_int.py::TestOptions::test_temperature_and_max_tokens
- packages/foundry/tests/foundry/test_foundry_embedding_client.py::TestFoundryEmbeddingIntegration::test_text_embedding_live

Also includes a one-line uv.lock specifier-ordering normalization
auto-applied by the poe-check pre-commit hook.

---------

Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-29 00:35:36 +00:00
Giles OdigweandCopilot e2eba0bacc Add retry logic and port-conflict fix for Ollama CI setup
- Kill any auto-started Ollama before launching serve (fixes port
  conflict: 'address already in use')
- Retry ollama pull up to 3 times with 15s backoff (fixes 429 rate
  limit failures)
- Applied to both python-merge-tests.yml and python-integration-tests.yml

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-28 16:51:07 -07:00
Giles OdigweandCopilot 386e08ed64 Fix E501 line-too-long in azurefunctions parallel test skip reasons
Wrap skip reason strings to stay within 120 char line limit.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-28 16:48:35 -07:00
Peter IbekweandGitHub 40e90c96c3 .NET: Add HttpRequestAction support to declarative workflows (#5474)
* Add declarative HttpRequestAction support to workflows

* Clean up response body for diagnostics  and fix tests.

* Fix merge with main.

* Remove redundant fallback for request content headers.
2026-04-28 21:53:19 +00:00
1e1eda65ce [Python] Add agent-framework-azure-ai-contentunderstanding package (#4829)
* feat: add agent-framework-azure-contentunderstanding package

Add Azure Content Understanding integration as a context provider for the
Agent Framework. The package automatically analyzes file attachments
(documents, images, audio, video) using Azure CU and injects structured
results (markdown, fields) into the LLM context.

Key features:
- Multi-document session state with status tracking (pending/ready/failed)
- Configurable timeout with async background fallback for large files
- Output filtering via AnalysisSection enum
- Auto-registered list_documents() and get_analyzed_document() tools
- Supports all CU modalities: documents, images, audio, video
- Content limits enforcement (pages, file size, duration)
- Binary stripping of supported files from input messages

Public API:
- ContentUnderstandingContextProvider (main class)
- AnalysisSection (output section selector enum)
- ContentLimits (configurable limits dataclass)

Tests: 46 unit tests, 91% coverage, all linting and type checks pass.

* fix: update CU fixtures with real API data, fix test assertions

- Replace synthetic fixtures with real CU API responses (sanitized)
- Update test assertions to match real data (Contoso vs CONTOSO,
  TotalAmount vs InvoiceTotal, field values from real analysis)
- Add --pre install note in README (preview package)
- Document unenforced ContentLimits fields (max_pages, duration)

* chore: add connector .gitignore, update uv.lock

* refactor: rename to azure-ai-contentunderstanding, fix CI issues

Align naming with Azure SDK convention and AF pattern:
- Directory: azure-contentunderstanding -> azure-ai-contentunderstanding
- PyPI: agent-framework-azure-contentunderstanding -> agent-framework-azure-ai-contentunderstanding
- Module: agent_framework_azure_contentunderstanding -> agent_framework_azure_ai_contentunderstanding

CI fixes:
- Inline conftest helpers to avoid cross-package import collision in xdist
- Remove PyPI badge and dead API reference link from README (package not published yet)

* feat: add samples (document_qa, invoice_processing, multimodal_chat)

- document_qa.py: Single PDF upload, CU context provider, follow-up Q&A
- invoice_processing.py: Structured field extraction with prebuilt-invoice
- multimodal_chat.py: Multi-file session with status tracking
- Add ruff per-file-ignores for samples/ directory
- Update README with samples section, env vars, and run instructions

* feat: add remaining samples (devui_multimodal_agent, large_doc_file_search)

- S3: devui_multimodal_agent/ — DevUI web UI with CU-powered file analysis
- S4: large_doc_file_search.py — CU extraction + OpenAI vector store RAG
- Update README and samples/README.md with all 5 samples

* feat: add file_search integration for large document RAG

Add FileSearchConfig — when provided, CU-extracted markdown is automatically
uploaded to an OpenAI vector store and a file_search tool is registered on
the context. This enables token-efficient RAG retrieval for large documents
without users needing to manage vector stores manually.

- FileSearchConfig dataclass (openai_client, vector_store_name)
- Auto-create vector store, upload markdown, register file_search tool
- Auto-cleanup on close()
- When file_search is enabled, skip full content injection (use RAG instead)
- Update large_doc_file_search sample to use the integration
- 4 new tests (50 total, 90% coverage)

* fix: add key-based auth support to all samples

Follow established AF pattern: check for API key env var first,
fall back to AzureCliCredential. Supports AZURE_OPENAI_API_KEY and
AZURE_CONTENTUNDERSTANDING_API_KEY environment variables.

* FEATURE(python): add analyzer auto-detection, file_search RAG, and lazy init

_context_provider.py:
- Make analyzer_id optional (default None) with auto-detection by media
  type prefix: audio->audioSearch, video->videoSearch, else documentSearch
- Add _ensure_initialized() for lazy client creation in before_run()
- Add FileSearchConfig-based vector store upload
- Fix: background-completed docs in file_search mode now upload to vector
  store instead of injecting full markdown into context messages
- Add _pending_uploads queue for deferred vector store uploads

devui_file_search_agent/ (new sample):
- DevUI agent combining CU extraction + OpenAI file_search RAG

azure_responses_agent (existing sample fix):
- Add AzureCliCredential support and AZURE_AI_PROJECT_ENDPOINT fallback

Tests (19 new), Docs updated (AGENTS.md, README.md)

* feat(cu): MIME sniffing, media-aware formatting, unified timeout, vector store expiration

- Add three-layer MIME detection (fast path → filetype binary sniff → filename
  fallback) to handle unreliable upstream MIME types (e.g. mp4 sent as
  application/octet-stream). Adds filetype>=1.2,<2 dependency.
- Media-aware output formatting: video shows duration/resolution + all fields
  as JSON; audio promotes Summary as prose; document unchanged.
- Unified timeout for all media types (removed file_search special-case that
  waited indefinitely for video/audio). All files use max_wait with background
  polling fallback.
- Vector store created with expires_after=1 day as crash safety net.
- Add 8 MIME sniffing tests (TestMimeSniffing class).

* fix: merge all CU content segments for video/audio analysis

CU's prebuilt-videoSearch and prebuilt-audioSearch analyzers split long
media files into multiple `contents[]` segments. Previously,
`_extract_sections()` only read `contents[0]`, causing truncated
duration, missing transcript, and incomplete fields for any video/audio
longer than a single scene.

Now iterates all segments and merges:
- duration: global min(startTimeMs) → max(endTimeMs)
- markdown: concatenated with `---` separators
- fields: same-named fields collected into per-segment list
- metadata (kind, resolution): taken from first segment

Single-segment results (documents, short audio) are unaffected.

Update test fixture to realistic 3-segment video structure and expand
assertions to verify multi-segment merging. Add documentation for
multi-segment processing and speaker diarization limitation.

* refactor: improve CU context provider docs and remove ContentLimits

- Improve class docstring: clarify endpoint (Azure AI Foundry URL with
  example), credential (AzureKeyCredential vs Entra ID), and analyzer_id
  (prebuilt/custom with auto-selection behavior and reference links)
- Add SUPPORTED_MEDIA_TYPES comments explaining MIME-based matching
  behavior and add missing file types per CU service docs
- Use namespaced logger to align with other packages
- Remove ContentLimits and related code/tests
- Rename DEFAULT_MAX_WAIT to DEFAULT_MAX_WAIT_SECONDS for clarity

* feat: support user-provided vector store in FileSearchConfig

- Add vector_store_id field to FileSearchConfig (None = auto-create)
- Track _owns_vector_store to only delete auto-created stores on close()
- Remove vector_store_name; use internal _DEFAULT_VECTOR_STORE_NAME
- Add inline comments for private state fields
- Document output_sections default in docstring
- Update AGENTS.md, samples, and tests

* fix: remove ContentLimits from README code block

* refactor: create CU client in __init__ instead of __aenter__

Follow Azure AI Search provider pattern: create the client eagerly in
__init__, make __aenter__ a no-op. This ensures __aexit__/close() is
always safe to call and eliminates the _ensure_initialized() workaround.

* docs: add file_search param to class docstring

* feat: introduce FileSearchBackend abstraction for cross-client support

Replace direct OpenAI client usage with FileSearchBackend ABC:
- OpenAIFileSearchBackend: for OpenAIChatClient (Responses API)
- FoundryFileSearchBackend: for FoundryChatClient (Azure Foundry)
- Shared base _OpenAICompatBackend for common vector store CRUD

FileSearchConfig now takes a backend instead of openai_client.
Factory methods from_openai() and from_foundry() for convenience.

BREAKING: FileSearchConfig(openai_client=...) -> FileSearchConfig.from_openai(...)

* refactor: FileSearchBackend abstraction + caller-owned vector store

* fix: file_search reliability and sample improvements

- Poll vector store indexing (create_and_poll) to ensure file_search
  returns results immediately after upload
- Set status to failed when vector store upload fails
- Skip get_analyzed_document tool in file_search mode to prevent
  LLM from bypassing RAG
- Simplify sample auth: single credential, direct parameters
- Use from_foundry backend for Foundry project endpoints

* perf: set max_num_results=10 for file_search to reduce token usage

* fix: move import to top of file (E402 lint)

* chore: remove unused imports

* fix: align azure-ai-contentunderstanding with MAF coding conventions

- Add module-level docstrings to __init__.py and _context_provider.py
- Use Self return type for __aenter__ (with typing_extensions fallback)
- Use explicit typed params for __aexit__ signature
- Add sync TokenCredential to AzureCredentialTypes union
- Pass AGENT_FRAMEWORK_USER_AGENT to ContentUnderstandingClient
- Remove unused ContentLimits from public API and tests
- Fix FileSearchConfig tests to match refactored backend API
- Fix lifecycle tests to match eager client initialization

* refactor: improve CU context provider API surface and fix CI

- Refactor _analyze_file to return DocumentEntry instead of mutating dict
- Remove TokenCredential from AzureCredentialTypes (fixes mypy/pyright CI)
- Remove OpenAIFileSearchBackend/FoundryFileSearchBackend from public API
  (internal to FileSearchConfig factory methods)
- Remove DocumentStatus from public exports (implementation detail)
- Update file_search comments to reflect backend-agnostic design
- Add DocumentStatus enum, analysis/upload duration tracking
- Add combined timeout for CU analysis + vector store upload

* fix: improve file_search samples and move tool guidelines to context provider

- Delete redundant devui_file_search_agent sample (duplicate of azure_openai variant)
- Move tool usage guidelines from sample agent instructions into context provider
  (extend_instructions in step 6, applied automatically for all file_search users)
- Fix file_search purpose: use from_foundry() for Azure OpenAI (purpose="assistants")
- Add filename hint in upload instructions for targeted file_search queries
- Reduce max_num_results from 10 to 3 in both devui samples
- Simplify agent instructions in both samples (remove tool-specific guidance)

* feat: improve source_id, integration tests, and content assertions

- Rename DEFAULT_SOURCE_ID to "azure_ai_contentunderstanding" (matches
  azure_ai_search convention)
- Improve source_id docstring to describe default value
- Clarify _detect_and_strip_files docstring (CU-supported files)
- Add invoice.pdf test fixture from Azure CU samples repo
- Refactor integration tests to use invoice.pdf directly (assert instead
  of skip when fixture missing)
- Add URI content test (Content.from_uri with external URL)
- Add "CONTOSO LTD." content assertion to all integration tests
- Use max_wait=None in integration tests (wait until complete)

* feat: reject duplicate filenames, add integration tests and sample comments

- Reject duplicate document keys in before_run (skip + warn LLM to rename)
- Update _derive_doc_key docstring to document uniqueness constraint
- Add unit tests for duplicate filename rejection (cross-turn and same-turn)
- Add integration test for data URI content (from_uri with base64)
- Add integration test for background analysis (max_wait timeout + resolve)
- Add filename recommendation comments to all samples' Content.from_data()

* chore: improve doc key derivation, comments, and README

- Replace hash-based doc key with uuid4 for anonymous uploads (O(1), no payload traversal)
- Remove hashlib import (no longer needed)
- Add File Naming section to README (filename importance, duplicate rejection)
- Improve inline comments (_derive_doc_key, _extract_binary, URL parsing)

* test: strengthen _format_result assertions with exact expected strings

- Replace loose 'in' checks with exact 'assert formatted == expected'
  for both multi-segment and single-segment format tests
- Add object-type fields (ShippingAddress, Speakers) to test data
  to cover nested dict/list serialization
- Add position-based ordering assertions to verify structural
  correctness (header -> markdown -> fields across segments)

* refactor: move invoice.pdf to shared sample_assets directory

- Move invoice.pdf from tests/cu/test_data/ to
  python/samples/shared/sample_assets/ as single source of truth
- Add INVOICE_PDF_PATH constant in test_integration.py pointing
  to the shared location
- Update document_qa.py, invoice_processing.py, large_doc_file_search.py
  to use invoice.pdf instead of sample.pdf

* refactor: reorganize samples into numbered dirs and simplify auth

- Move script samples into 01-get-started/ with numbered prefixes
  (01_document_qa, 02_multimodal_chat, 03_invoice_processing,
   04_large_doc_file_search)
- Move devui samples into 02-devui/ with 01-multimodal_agent and
  02-file_search_agent/{azure_openai_backend,foundry_backend}
- Move invoice.pdf to CU package-local samples/shared/sample_assets/
- Replace kwargs dicts with direct constructor calls; support both
  API key (AZURE_OPENAI_API_KEY) and AzureCliCredential
- Update README sample table with new paths

* fix: resolve CI lint errors (D205, RUF001, E501)

- Fix D205: single-line docstring summary for _detect_and_strip_files
- Fix RUF001: replace EN DASH with HYPHEN-MINUS in segment headers
- Fix E501: wrap long assertion lines in tests
- Also includes samples reorg and auth simplification

* refactor: overhaul samples — FoundryChatClient, sessions, remove get_analyzed_document

Samples:
- Switch all samples from deprecated AzureOpenAIResponsesClient to FoundryChatClient
- Add 02_multi_turn_session.py showing AgentSession persistence across turns
- Rewrite 03_multimodal_chat.py with real PDF + audio + video (parallel
  analysis), per-modality follow-ups, cross-document question, elapsed
  time, user prompts, and input token counts
- Renumber: 02->03 multimodal, 03->04 invoice, 04->05 file_search

Context provider:
- Remove get_analyzed_document tool -- full content is in conversation
  history via InMemoryHistoryProvider, no retrieval tool needed
- Remove follow-up turn instructions about tools
- Only list_documents tool remains (for status queries)
- Update README to reflect tool removal

* feat: add 05_background_analysis sample and fix 04 session/max_wait

- Add 05_background_analysis.py demonstrating non-blocking CU analysis
  with max_wait=1s, status tracking via list_documents(), and automatic
  background task resolution on subsequent turns
- Fix 04_invoice_processing.py: add max_wait=None and AgentSession
- Rename 05→06 large_doc_file_search
- Update README sample table

* docs: update README and fix sample 06

README:
- Switch Quick Start from AzureOpenAIResponsesClient to FoundryChatClient
- Add AgentSession to Quick Start example
- Fix status values: pending -> analyzing/uploading/ready/failed
- Fix env var: AZURE_OPENAI_RESPONSES_DEPLOYMENT_NAME -> AZURE_OPENAI_DEPLOYMENT_NAME
- Update samples section with new paths, link to samples/README.md
- Update multi-segment description to reflect per-segment fields

Sample 06:
- Fix from_openai -> from_foundry for Azure endpoints
- Add AgentSession and max_wait=None

* docs: rewrite README — concise format, prerequisites, CU link

* fix: resolve pyright errors in _format_result segment cast

* docs: add numbered section comments and fresh sample output to all samples

- Add numbered section comments (# 1. ..., # 2. ...) per SAMPLE_GUIDELINES
- Re-run all 6 samples and update expected output with real results
- Fix duplicate sample output blocks in 04 and 05
- Update README code example to use public invoice URL

* feat: add load_settings support for env var configuration

- Make endpoint optional in constructor — auto-loads from
  AZURE_CONTENTUNDERSTANDING_ENDPOINT env var via load_settings()
- Add ContentUnderstandingSettings TypedDict
- Add env_file_path/env_file_encoding params for .env file support
- Add 4 unit tests: env var loading, explicit override, missing
  endpoint error, missing credential error
- Update README with env var auto-resolution docs
- Follows framework convention used by all other packages

* docs: polish README — fix duplicate env var, add Next steps, service limits link

* chore: trim invoice fixture from 199K to 33 lines

Keep only VendorName, InvoiceTotal, DueDate, InvoiceDate, InvoiceId
fields and first 500 chars of markdown. Strip spans/source/coordinates.
Reduces fixture from 6.6MB to 1.2KB.

* feat: per-file analyzer_id override via additional_properties

- Read analyzer_id from Content.additional_properties for per-file override
- Resolution order: per-file > provider-level > auto-detect by media type
- Update class docstring documenting filename and analyzer_id properties
- Update sample 04 to demonstrate per-file override (prebuilt-invoice)
- Add unit test for per-file analyzer override

* Trim PDF test fixture and clarify unique filename requirement

- Trim analyze_pdf_result.json from 4427 to 23 lines by removing
  pages, words, lines, paragraphs, sections, spans, and source
  fields that are not used by any unit test.
- Add docstring note that filename must be unique within a session;
  duplicate filenames are rejected and the file will not be analyzed.

* Update python/packages/azure-ai-contentunderstanding/agent_framework_azure_ai_contentunderstanding/_context_provider.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update python/packages/azure-ai-contentunderstanding/agent_framework_azure_ai_contentunderstanding/_context_provider.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update python/packages/azure-ai-contentunderstanding/samples/02-devui/02-file_search_agent/azure_openai_backend/agent.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update python/packages/azure-ai-contentunderstanding/samples/02-devui/01-multimodal_agent/agent.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update python/packages/azure-ai-contentunderstanding/samples/01-get-started/06_large_doc_file_search.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Fix AGENTS.md to match implementation; remove unused variable in test helper

AGENTS.md:
- Remove _ensure_initialized() reference (client is created in __init__)
- Fix multi-segment docs: segments kept as list, not merged into fields
- Remove get_analyzed_document() reference (only list_documents registered)
- Update sample names to match current directory structure

test_context_provider.py:
- Simplify _make_data_uri() — remove unused 'encoded' variable

* Fix premature file_search instruction for background-completed docs

- Change _resolve_pending_tasks() instruction from 'Use file_search'
  to 'being indexed' since the upload hasn't completed yet at that point.
- Add LLM instruction on upload failure in step 1b so the agent can
  inform the user the document isn't searchable.

* fix: wrap long line in devui agent instructions (E501)

* Fix Copilot review: unused logger, stray code in README, await cancelled tasks

- _file_search.py: Remove unused logger and logging import
- 01-multimodal_agent/README.md: Remove accidentally pasted Python script
- _context_provider.py close(): Await cancelled tasks before closing
  client to prevent 'Task destroyed but pending' warnings

* Sanitize doc keys and fix duplicate filename re-injection

- Add _sanitize_doc_key() to strip control characters, collapse
  whitespace, and cap length at 255 chars — prevents prompt injection
  via crafted filenames in extend_instructions() calls.
- Track accepted doc_keys in step 3 so step 5 only injects content
  for files actually analyzed this turn, not pre-existing duplicates.
- Soften duplicate upload instruction wording (remove IMPORTANT/caps).

* fix: add type annotation to tasks_to_cancel for pyright

* Move per-session mutable state to state dict for session isolation

Previously _pending_tasks, _pending_uploads, and _uploaded_file_ids
were stored on self, shared across all sessions. This caused
cross-session leakage: Session A's background task results could be
injected into Session B's context.

Now these are stored in the per-session state dict. Global copies
(_all_pending_tasks, _all_uploaded_file_ids) are kept on self only
for best-effort cleanup in close().

Add 2 new TestSessionIsolation tests verifying that background tasks
and resolved content stay within their originating session.

* Remove unused AnalysisSection enum values

Only MARKDOWN and FIELDS are handled by _extract_sections().
Remove FIELD_GROUNDING, TABLES, PARAGRAPHS, SECTIONS to avoid
exposing dead options to users.

* Recursively flatten object/array field values for cleaner LLM output

- Use SDK .value property with recursive extraction for object/array fields
- Object: AmountDue -> {Amount: 610, CurrencyCode: USD} (was raw SDK dict)
- Array: LineItems -> list of flattened items (was raw SDK list)
- Update invoice fixture with object/array fields from prebuilt-invoice
- Add 3 unit tests for object, array, and nested object field extraction

* Preserve sub-field confidence; compare full expected JSON in tests

* Remove incorrect MIME aliases (audio/mp4, video/x-matroska)

* feat: add AnalysisInput, content_range, warnings, and category support

- Use SDK AnalysisInput model instead of raw body dict for begin_analyze
- Forward content_range from additional_properties to CU (page/time ranges)
- Extract CU warnings with code/message/target (ODataV4Format) into output
- Include content-level category from classifier analyzers
- Add 5 new tests: warnings, category, content_range forwarding
- Fix pyright with explicit casts; fix en-dash lint (RUF002)

* fix: falsy-0 bug in duration calc; improve test coverage

- Fix start_time_ms=0 treated as falsy by 'or' short-circuit, use
  'is None' checks instead for duration and segment time extraction
- Update warnings test to use RAI ContentFiltered codes
- Enrich warnings extraction to include code/message/target (ODataV4Format)
- Add multi-segment video category test with per-segment assertions

* refactor: split _context_provider.py into focused modules

- Extract _constants.py: SUPPORTED_MEDIA_TYPES, MIME_ALIASES, analyzer maps
- Extract _detection.py: file detection, MIME sniffing, doc key derivation
- Extract _extraction.py: result extraction, field flattening, LLM formatting
- _context_provider.py delegates via thin wrappers (793 lines, was 1255)
- Update test imports to use _constants.py for SUPPORTED_MEDIA_TYPES

* docs: update AGENTS.md with DocumentStatus, FileSearchBackend, and _file_search.py

* refactor: replace AnalysisSection enum with Literal type for simpler DX

- Remove AnalysisSection(str, Enum) class, replace with Literal["markdown", "fields"] type alias
- Users can now pass plain strings: output_sections=["markdown"] — no extra import needed
- AnalysisSection type alias still exported for type annotation use
- Update all samples, tests, and internal code to use string literals
- Address PR review feedback (eavanvalkenburg)

* refactor: replace asyncio.Task with continuation tokens for serializable state

- Replace state["_pending_tasks"] (asyncio.Task — not serializable) with
  state["_pending_tokens"] (dict of continuation token strings) so the
  framework can persist session state to disk/storage
- Resume pending analyses via Azure SDK continuation_token mechanism
- Fix: resumed pollers have stale cached status (done() always False),
  use asyncio.wait_for(poller.result()) with 10s min timeout instead
- Remove _background_poll(), _all_pending_tasks, and task cancellation
- Address PR review feedback (eavanvalkenburg): state must be serializable

* fix: resolve CI lint (RUF052) and mypy (call-overload) errors

* feat: add structured output (Pydantic model) to invoice processing sample

- Use response_format=InvoiceResult for schema-constrained LLM output
- Use output_sections=["fields"] only (no markdown needed for structured output)
- Add LowConfidenceField model with confidence values
- Add comments about prebuilt-invoice extensive schema vs simplified model
- Address PR review feedback (eavanvalkenburg): use structured response

* fix: use FOUNDRY_PROJECT_ENDPOINT and FOUNDRY_MODEL env vars in all samples

Replace AZURE_AI_PROJECT_ENDPOINT → FOUNDRY_PROJECT_ENDPOINT and
AZURE_OPENAI_DEPLOYMENT_NAME → FOUNDRY_MODEL across all sample .py and
README.md files. Address PR review feedback (eavanvalkenburg).

* refactor: remove background_analysis sample, use FoundryChatClient in DevUI

- Remove 05_background_analysis.py (per reviewer feedback — discuss max_wait
  design separately from samples)
- Renumber 06_large_doc_file_search.py → 05_large_doc_file_search.py
- Replace AzureOpenAIResponsesClient with FoundryChatClient in all DevUI samples
- Replace client.as_agent() with Agent(client=client, ...) everywhere
- Add max_wait comments explaining interactive vs batch usage
- Update README.md and AGENTS.md
- Address PR review feedback (eavanvalkenburg)

* fix: vector_stores API moved from beta namespace in OpenAI SDK

* docs: add comments about multi-file support and CU service limits in file_search sample

* fix: broken markdown links after sample removal and renumbering

* fix: migrate BaseContextProvider to ContextProvider (non-deprecated)

* fix: Message(text=) -> Message(contents=[]) for API compatibility

* Inline _constants.py into consuming modules

Remove _constants.py and move constants to where they are used:
- SUPPORTED_MEDIA_TYPES, MIME_ALIASES → _detection.py
- MEDIA_TYPE_ANALYZER_MAP, DEFAULT_ANALYZER → _context_provider.py

Addresses review feedback to reduce file count.

* Mark package as alpha per package management skill

- Version: 1.0.0b260401 → 1.0.0a260401
- Classifier: Development Status 4 - Beta → 3 - Alpha
- Add to PACKAGE_STATUS.md as alpha

Follows the alpha package checklist from python-package-management skill.

* Replace extend_instructions with extend_messages for status notifications

Status/error/result notifications now use extend_messages (conversation
context) instead of extend_instructions (system prompt). This avoids
system prompt bloat and keeps behavioral directives separate from
event notifications.

- 11 extend_instructions calls → extend_messages (role='user')
- 1 extend_instructions retained: tool usage guidelines (behavioral)
- 6 test assertions updated to check context_messages

All 84 unit tests + 5 live integration tests pass.

* Fix lint: E402 import order, ISC004 implicit string concatenation

- Move constants after all imports to fix E402
- Wrap multi-line strings in parentheses inside contents=[] to fix ISC004

* Fix lint: remove unused json import in invoice sample

* Fix CI: apply ruff format + fix E501 line length after reformatting

ruff format expands Message() calls to multi-line, pushing string
indentation deeper. Break long strings to fit within 120 char limit
after formatting. Also removes unused json import in sample.

* Address review feedback: keyword-only args, accept pre-built client, remove wrappers

- All __init__ args now keyword-only (matches FoundryChatClient pattern)
- New 'client' param accepts pre-built ContentUnderstandingClient
- core dep bound: >=1.0.0rc5 → >=1.0.0,<2
- Self import moved after local imports
- Removed 9 static method wrappers; callsites use module functions directly
- Tests updated to import derive_doc_key and format_result directly

* fix: remove duplicate ContentUnderstandingClient instantiation

The client was being created twice — once inside the if/else block and
again unconditionally after it. The second instantiation overwrote the
pre-built client path and failed type checking when credential was None.

* rename: azure-ai-contentunderstanding → azure-contentunderstanding

Package: agent-framework-azure-ai-contentunderstanding → agent-framework-azure-contentunderstanding
Module: agent_framework_azure_ai_contentunderstanding → agent_framework_azure_contentunderstanding
Directory: packages/azure-ai-contentunderstanding → packages/azure-contentunderstanding

Per agreement with PM and MAF team to drop 'AI' from the package name.

* feat: add ContentUnderstanding re-export to agent_framework.foundry namespace

Enables: from agent_framework.foundry import ContentUnderstandingContextProvider

Exports: ContentUnderstandingContextProvider, FileSearchConfig,
FileSearchBackend, AnalysisSection, DocumentStatus

Updates all samples and README to use the foundry namespace import.

* fix: add missing copyright headers to standalone sample scripts

* chore: remove .vscode/settings.json and add to .gitignore

* refactor: reuse FoundryChatClient.client for vector store ops in file_search sample

Address review feedback from TaoChenOSU:
- 05_large_doc_file_search.py: use client.client instead of manually
  constructing AsyncAzureOpenAI; remove openai dependency
- azure_openai_backend/agent.py: import reorder only (AIProjectClient
  kept — required for sync vector store creation in DevUI)

* fix: skip closing client when caller passes pre-built client

When a ContentUnderstandingClient is passed via client=, the caller
owns its lifecycle. Added _owns_client flag so close() only closes
the client when we created it internally.

---------

Co-authored-by: yungshinlin <yungshin@msn.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-28 20:55:59 +00:00
Evan MattsonandGitHub 3a463b8bf6 Python: bump package versions for 1.2.1 release (#5536)
* Python: bump package versions for 1.2.1 release

PATCH bump (1.2.0 -> 1.2.1) for the released cohort. The release window
covers two PRs, no new public APIs:

- agent-framework-core: prevent inner_exception from being lost in
  AgentFrameworkException (#5167)
- samples: add requirements.txt and .env.example to the a2a/ hosting
  sample for pip-based setup (#5510)

Per lockstep convention, all 21 beta packages stamp 1.0.0b260428 and all
3 alpha packages stamp 1.0.0a260428, regardless of per-package code
churn. Every non-core package floor on agent-framework-core is raised to
>=1.2.1 to keep cohort signaling consistent. Date stamp reflects the
local (Asia) cut date 2026-04-28.

* Python: silence pyright unknown-type warnings in hosted-env detection

`azure.ai.agentserver.core` is probed at runtime via `importlib.util.find_spec`
and is not a declared dependency. The existing `# pyright: ignore[reportMissingImports]`
suppresses the missing-import warning, but at `lowest-direct` resolution pyright
still reports the imported symbol (`AgentConfig`) and its members (`from_env`,
`is_hosted`) as unknown, breaking `validate-dependency-bounds-test` for
`packages/core`.

Extend the existing ignore to cover `reportUnknownVariableType` on the import
and `reportUnknownMemberType` on the call site so the bounds check returns to
green. Behavior is unchanged.

Latent since #5455 (shipped in 1.2.0).

* Python: raise agent-framework-gemini lower bound to google-genai>=1.65.0

The Gemini chat client references several `google.genai.types` symbols
(`FileSearch`, `ThinkingLevel`, `SearchTypes`, `McpServer`,
`StreamableHttpTransport`, plus call-site keyword args `mcp_servers` and
`search_types`) that are not present at the lower bound of `google-genai>=1.0.0`.
At `lowest-direct` resolution this caused `validate-dependency-bounds-test` to
fail for `packages/gemini` with eleven `reportAttributeAccessIssue` /
`reportUnknownVariableType` errors.

Walking the upstream `google.genai.types` API:
- `GoogleMaps`, `AuthConfig`: present from 1.40.0
- `FileSearch`: introduced in 1.49.0
- `ThinkingLevel`: introduced in 1.55.0
- `SearchTypes`, `McpServer`, `StreamableHttpTransport`: introduced in 1.65.0

Bump the lower bound to 1.65.0 — the minimum version that exposes every symbol
the package actually uses. Keep the `<2.0.0` upper cap unchanged. With this
bump `validate-dependency-bounds-test` passes for both lower and upper
resolution scenarios across all 27 workspace packages.

Latent since #4847 (Gemini package introduction in 1.1.0); aggravated by
subsequent feature additions that pulled in newer `types.*` symbols.

* Python: add dependabot bumps to 1.2.1 CHANGELOG

Catalog the 15 dependabot dependency updates that merged on `upstream/main`
between python-1.2.0 and the 1.2.1 cut window under a new Changed section:

- Workspace dev/runtime deps: `rich`, `prek`, `python-multipart`, `pyasn1`,
  `pytest` (ag-ui, devui, lab), `uv` (lab)
- Frontend deps: `vite` (devui, chatkit), `postcss` (devui, chatkit, handoff),
  `picomatch` (devui, handoff)

CHANGELOG-only — no source or pyproject.toml changes. PRs themselves merged
upstream independently of this release branch and will be brought in via the
PR merge.
2026-04-28 18:23:26 +09:00
dependabot[bot]GitHubmoonbox3dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
74a5ea8dca Python: Bump prek from 0.3.8 to 0.3.9 in /python (#5228)
* Bump prek from 0.3.8 to 0.3.9 in /python

Bumps [prek](https://github.com/j178/prek) from 0.3.8 to 0.3.9.
- [Release notes](https://github.com/j178/prek/releases)
- [Changelog](https://github.com/j178/prek/blob/master/CHANGELOG.md)
- [Commits](https://github.com/j178/prek/compare/v0.3.8...v0.3.9)

---
updated-dependencies:
- dependency-name: prek
  dependency-version: 0.3.9
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Fix CI: bump prek to 0.3.9 in lab package and update uv.lock

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/f17751e5-c5a8-4d42-9555-6bf708a2ef47

Co-authored-by: moonbox3 <35585003+moonbox3@users.noreply.github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: moonbox3 <35585003+moonbox3@users.noreply.github.com>
2026-04-28 09:08:58 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
df6041bcc1 Bump vite in /python/samples/05-end-to-end/chatkit-integration/frontend (#5126)
Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) from 7.1.12 to 7.3.2.
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/v7.3.2/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v7.3.2/packages/vite)

---
updated-dependencies:
- dependency-name: vite
  dependency-version: 7.3.2
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-28 08:08:36 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
e6c29f8fa4 Bump vite from 7.1.12 to 7.3.2 in /python/packages/devui/frontend (#5127)
Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) from 7.1.12 to 7.3.2.
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/v7.3.2/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v7.3.2/packages/vite)

---
updated-dependencies:
- dependency-name: vite
  dependency-version: 7.3.2
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-28 08:08:21 +00:00
dependabot[bot]GitHubmoonbox3dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2c35be877d Bump pytest from 9.0.2 to 9.0.3 in /python/packages/ag-ui (#5461)
* Bump pytest from 9.0.2 to 9.0.3 in /python/packages/ag-ui

Bumps [pytest](https://github.com/pytest-dev/pytest) from 9.0.2 to 9.0.3.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest/compare/9.0.2...9.0.3)

---
updated-dependencies:
- dependency-name: pytest
  dependency-version: 9.0.3
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>

* Fix CI: bump pytest to 9.0.3 across all workspace packages and update uv.lock

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/a9996d8b-3fb7-436b-a22f-f4a8c436b213

Co-authored-by: moonbox3 <35585003+moonbox3@users.noreply.github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: moonbox3 <35585003+moonbox3@users.noreply.github.com>
2026-04-28 07:59:51 +00:00
dependabot[bot]GitHubmoonbox3dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
0a27c74245 Python: Bump pytest from 9.0.2 to 9.0.3 in /python/packages/devui (#5492)
* Bump pytest from 9.0.2 to 9.0.3 in /python/packages/devui

Bumps [pytest](https://github.com/pytest-dev/pytest) from 9.0.2 to 9.0.3.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest/compare/9.0.2...9.0.3)

---
updated-dependencies:
- dependency-name: pytest
  dependency-version: 9.0.3
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>

* Fix CI: bump pytest to 9.0.3 in all workspace pyproject.toml files and update uv.lock

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/60822a0e-8a54-412f-9999-051cf1ce2c7c

Co-authored-by: moonbox3 <35585003+moonbox3@users.noreply.github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: moonbox3 <35585003+moonbox3@users.noreply.github.com>
2026-04-28 07:59:45 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
7c4837744b Bump picomatch (#4936)
Bumps [picomatch](https://github.com/micromatch/picomatch) from 4.0.3 to 4.0.4.
- [Release notes](https://github.com/micromatch/picomatch/releases)
- [Changelog](https://github.com/micromatch/picomatch/blob/master/CHANGELOG.md)
- [Commits](https://github.com/micromatch/picomatch/compare/4.0.3...4.0.4)

---
updated-dependencies:
- dependency-name: picomatch
  dependency-version: 4.0.4
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-28 07:27:15 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
870f10829e Update rich requirement in /python (#5227)
Updates the requirements on [rich](https://github.com/Textualize/rich) to permit the latest version.
- [Release notes](https://github.com/Textualize/rich/releases)
- [Changelog](https://github.com/Textualize/rich/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Textualize/rich/compare/v13.7.1...v15.0.0)

---
updated-dependencies:
- dependency-name: rich
  dependency-version: 15.0.0
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-28 07:26:03 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
5ba7f8aa6f Bump python-multipart from 0.0.22 to 0.0.26 in /python (#5286)
Bumps [python-multipart](https://github.com/Kludex/python-multipart) from 0.0.22 to 0.0.26.
- [Release notes](https://github.com/Kludex/python-multipart/releases)
- [Changelog](https://github.com/Kludex/python-multipart/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Kludex/python-multipart/compare/0.0.22...0.0.26)

---
updated-dependencies:
- dependency-name: python-multipart
  dependency-version: 0.0.26
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-28 07:25:28 +00:00
dependabot[bot]GitHubmoonbox3dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
35a0b51523 Python: Bump uv from 0.11.3 to 0.11.6 in /python/packages/lab (#5469)
* Bump uv from 0.11.3 to 0.11.6 in /python/packages/lab

Bumps [uv](https://github.com/astral-sh/uv) from 0.11.3 to 0.11.6.
- [Release notes](https://github.com/astral-sh/uv/releases)
- [Changelog](https://github.com/astral-sh/uv/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/uv/compare/0.11.3...0.11.6)

---
updated-dependencies:
- dependency-name: uv
  dependency-version: 0.11.6
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>

* Fix CI: update uv from 0.11.3 to 0.11.6 in python/pyproject.toml and regenerate uv.lock

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/a1a7c648-b26f-44e7-bace-d56ed8489053

Co-authored-by: moonbox3 <35585003+moonbox3@users.noreply.github.com>

* Fix code quality CI: update uv-pre-commit rev from 0.11.3 to 0.11.6 in .pre-commit-config.yaml

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/cdfdd211-9f1e-4570-bc7c-86fd15240e91

Co-authored-by: moonbox3 <35585003+moonbox3@users.noreply.github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: moonbox3 <35585003+moonbox3@users.noreply.github.com>
2026-04-28 07:24:43 +00:00
dependabot[bot]GitHubmoonbox3dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
d28c841c50 Python: Bump pytest from 9.0.2 to 9.0.3 in /python/packages/lab (#5470)
* Bump pytest from 9.0.2 to 9.0.3 in /python/packages/lab

Bumps [pytest](https://github.com/pytest-dev/pytest) from 9.0.2 to 9.0.3.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest/compare/9.0.2...9.0.3)

---
updated-dependencies:
- dependency-name: pytest
  dependency-version: 9.0.3
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update pytest from 9.0.2 to 9.0.3 across all workspace packages

Fix dependency conflict: agent-framework workspace packages were pinning
pytest==9.0.2 while agent-framework-lab required pytest==9.0.3, causing
uv dependency resolution to fail. Updated all pyproject.toml files and
regenerated uv.lock to use pytest==9.0.3 consistently.

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/d274f7c5-b5ed-4b18-8eab-4db3cfd9d1bf

Co-authored-by: moonbox3 <35585003+moonbox3@users.noreply.github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: moonbox3 <35585003+moonbox3@users.noreply.github.com>
2026-04-28 07:24:23 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
7d305d461c Bump postcss from 8.5.6 to 8.5.10 in /python/packages/devui/frontend (#5484)
Bumps [postcss](https://github.com/postcss/postcss) from 8.5.6 to 8.5.10.
- [Release notes](https://github.com/postcss/postcss/releases)
- [Changelog](https://github.com/postcss/postcss/blob/main/CHANGELOG.md)
- [Commits](https://github.com/postcss/postcss/compare/8.5.6...8.5.10)

---
updated-dependencies:
- dependency-name: postcss
  dependency-version: 8.5.10
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-28 07:24:05 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
8f4efe5fb9 Bump postcss (#5491)
Bumps [postcss](https://github.com/postcss/postcss) from 8.5.6 to 8.5.10.
- [Release notes](https://github.com/postcss/postcss/releases)
- [Changelog](https://github.com/postcss/postcss/blob/main/CHANGELOG.md)
- [Commits](https://github.com/postcss/postcss/compare/8.5.6...8.5.10)

---
updated-dependencies:
- dependency-name: postcss
  dependency-version: 8.5.10
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-28 07:23:50 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
362c4c5f84 Bump postcss (#5527)
Bumps [postcss](https://github.com/postcss/postcss) from 8.5.6 to 8.5.12.
- [Release notes](https://github.com/postcss/postcss/releases)
- [Changelog](https://github.com/postcss/postcss/blob/main/CHANGELOG.md)
- [Commits](https://github.com/postcss/postcss/compare/8.5.6...8.5.12)

---
updated-dependencies:
- dependency-name: postcss
  dependency-version: 8.5.12
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-28 07:23:17 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
27a6f47a3b Bump picomatch in /python/packages/devui/frontend (#4921)
Bumps  and [picomatch](https://github.com/micromatch/picomatch). These dependencies needed to be updated together.

Updates `picomatch` from 4.0.3 to 4.0.4
- [Release notes](https://github.com/micromatch/picomatch/releases)
- [Changelog](https://github.com/micromatch/picomatch/blob/master/CHANGELOG.md)
- [Commits](https://github.com/micromatch/picomatch/compare/4.0.3...4.0.4)

Updates `picomatch` from 2.3.1 to 2.3.2
- [Release notes](https://github.com/micromatch/picomatch/releases)
- [Changelog](https://github.com/micromatch/picomatch/blob/master/CHANGELOG.md)
- [Commits](https://github.com/micromatch/picomatch/compare/4.0.3...4.0.4)

---
updated-dependencies:
- dependency-name: picomatch
  dependency-version: 4.0.4
  dependency-type: indirect
- dependency-name: picomatch
  dependency-version: 2.3.2
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-28 06:35:09 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
198a3a1ab1 Bump pyasn1 from 0.6.2 to 0.6.3 in /python (#4748)
Bumps [pyasn1](https://github.com/pyasn1/pyasn1) from 0.6.2 to 0.6.3.
- [Release notes](https://github.com/pyasn1/pyasn1/releases)
- [Changelog](https://github.com/pyasn1/pyasn1/blob/main/CHANGES.rst)
- [Commits](https://github.com/pyasn1/pyasn1/compare/v0.6.2...v0.6.3)

---
updated-dependencies:
- dependency-name: pyasn1
  dependency-version: 0.6.3
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-28 06:34:40 +00:00
Tao ChenandGitHub 88347f6494 Python: Update hosting agent samples + fixes (#5485)
* Update foundry hosting samples

* Add file data type support

* Fix file content and add more tests

* Fix README

* Address comments

* Fix int tests

* remove temp
2026-04-28 04:24:05 +00:00
9b22ecd119 Python: Add requirements.txt and .env.example to the a2a/ sample for pip-based setup (#5510)
* Add requirements.txt and .env.example to a2a sample

Beginners following the a2a/ sample had no pip-based install path:
the directory lacked requirements.txt and .env.example, unlike every
other 04-hosting/ sample.

- Add requirements.txt with editable local package paths matching the
  pattern used in azure_functions/ and similar hosting samples
- Add .env.example documenting FOUNDRY_PROJECT_ENDPOINT, FOUNDRY_MODEL,
  and A2A_AGENT_HOST
- Update README Quick Start to cover both pip (.venv) and uv workflows

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Add `requirements.txt` and `.env.example` to the `a2a/` sample for pip-based setup

Fixes #5395

* fix(a2a-sample): address PR review feedback for issue #5395

- Remove 'from repo root' wording from Option B uv heading in README
  to avoid contradicting the 'run from this directory' instruction
- Fix A2A_AGENT_HOST default in .env.example from 5001 to 5000 to match
  function-tools flow; add clarifying comments about port usage
- Add note for pip users explaining they can replace 'uv run python'
  with 'python' once the virtual environment is activated

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address review feedback for #5395: Python: [Samples][Python] a2a/ sample missing requirements.txt — beginners cannot install dependencies

---------

Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-27 22:22:07 +00:00
SergeyMenshykhandGitHub 2eb0705ee0 .NET: [Breaking] Support string[] arguments for file-based skill scripts (#5475)
* support arguments of string[] shape for file-based skill scripts

* suppress breaking changes errors

* address feedback

* remove unnecessary usung directive
2026-04-27 15:37:10 +00:00
Giles OdigweandCopilot 374526515d Re-skip parallel workflow tests: xdist worker distribution issue
The 4 parallel workflow tests crash because xdist worksteal distributes
them across separate workers, each spawning its own func process against
shared emulators. Auth fix (api_key->credential) was valid and stays.
test_conditional_branching now passes with the auth fix.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-27 06:38:19 -07:00
Giles OdigweandCopilot a6e0ab5603 Fix auth routing in samples 06/11: api_key -> credential for Azure OpenAI
Both samples passed a bearer token provider via api_key= which caused the
client to route to api.openai.com instead of Azure OpenAI, resulting in
401 Unauthorized. Changed to credential= which correctly triggers Azure
routing and picks up AZURE_OPENAI_ENDPOINT from the environment.

- samples/azure_functions/11_workflow_parallel/function_app.py: 1 fix
- samples/durabletask/06_multi_agent_orchestration_conditionals/worker.py: 2 fixes
- Re-enable 4 parallel workflow tests and 1 conditional branching test

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-27 06:12:28 -07:00
Giles OdigweandCopilot f6f87477c9 Re-skip failing Functions/DurableTask tests with specific root causes
- test_11_workflow_parallel (4 tests): xdist worker crashes during execution
- test_conditional_branching: orchestration fails with RuntimeError, not a timeout
- Keep 480s timeout bump for remaining Functions tests

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-27 05:37:50 -07:00
bahtyarandGitHub dad3652f46 Python: fix: prevent inner_exception from being lost in AgentFrameworkException (#5167)
* fix: prevent inner_exception from being lost in AgentFrameworkException

The __init__ method unconditionally called super().__init__() after
the conditional call with inner_exception, effectively overwriting the
exception args and losing the inner_exception reference.

Add else branch so super().__init__() is only called once with the
correct arguments.

Fixes #5155

Signed-off-by: bahtya <bahtyar153@qq.com>

* test: add explicit tests for AgentFrameworkException inner_exception handling

- test_exception_with_inner_exception: verifies args include inner exception
- test_exception_without_inner_exception: verifies args only contain message
- test_exception_inner_exception_none_explicit: verifies explicit None

Covers both branches of the if/else in __init__.

* fix: export AgentFrameworkException from package

Bahtya

---------

Signed-off-by: bahtya <bahtyar153@qq.com>
2026-04-27 04:53:06 +00:00
Shyju KrishnankuttyandGitHub 56fb634f0e .NET: Support returning durable workflow results from HTTP trigger endpoint (#5321)
* Adding support for "wait for response" when invoking workflow http endpoint.

* update changelog.

* PR comment fixes.

* Address PR review feedback.

- Return 404 Not Found when no orchestration with the given ID exists
- Return 200 OK for failed workflows (the HTTP operation succeeded;
  the workflow outcome is conveyed via the response body)
- Rename 'status' to 'workflowStatus' in WorkflowRunResponse to avoid
  inconsistency with AgentRunSuccessResponse which uses integer status
- Add optional 'error' field (omitted from JSON when null) to
  WorkflowRunResponse for failed workflow details
2026-04-25 00:55:28 +00:00
56c3f8d825 .NET: Bump OpenTelemetry packages to 1.15.3 (#5478)
* Bump OpenTelemetry packages to 1.15.3 to fix known vulnerabilities

Update OpenTelemetry packages from 1.15.0 to 1.15.3 in Directory.Packages.props
to resolve NU1902 warnings-as-errors for CVEs GHSA-g94r-2vxg-569j,
GHSA-mr8r-92fq-pj8p, and GHSA-q834-8qmm-v933.

Add explicit PackageReference for OpenTelemetry.Exporter.OpenTelemetryProtocol
in Foundry.Hosting and OpenTelemetry.Api + OpenTelemetry.Exporter.OpenTelemetryProtocol
in Hosted-Invocations-EchoAgent to override transitive 1.15.0 resolution in
projects with CentralPackageTransitivePinningEnabled=false.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Bump OpenTelemetry Extensions and Instrumentation packages to 1.15.x

Align the full OpenTelemetry package set to the 1.15.x family:
- OpenTelemetry.Extensions.Hosting: 1.14.0 -> 1.15.3
- OpenTelemetry.Instrumentation.AspNetCore: 1.14.0 -> 1.15.2
- OpenTelemetry.Instrumentation.Http: 1.14.0 -> 1.15.1
- OpenTelemetry.Instrumentation.Runtime: 1.14.0 -> 1.15.1

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-24 21:56:30 +00:00
Giles OdigweandCopilot 9316f2c2f8 Re-enable skipped Functions/DurableTask tests and bump timeout to 480s
- Remove hard skips from 4 tests in test_11_workflow_parallel.py
- Remove hard skip from test_conditional_branching in test_06_dt_multi_agent_orchestration_conditionals.py
- Increase pytest --timeout from 360 to 480 for Functions+DurableTask CI job
- Updated in both python-merge-tests.yml and python-integration-tests.yml

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-24 11:00:29 -07:00
Giles OdigweandCopilot dc64d63a2a Re-enable reliable streaming integration tests
Remove the hard skip on test_03_reliable_streaming tests that was
temporarily disabled for instability investigation. CI infrastructure
(Azurite, DTS emulator, Redis, func CLI) is already in place.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-24 07:35:39 -07:00
Evan MattsonandGitHub 0b69d7fd15 Python: Bump Python package versions for 1.2.0 release (#5468)
* Bump Python package versions for 1.2.0 release

Released tier bumps 1.1.1 -> 1.2.0 (core, openai, foundry, root) to
reflect additive public APIs landed since 1.1.0: functional workflow API
(#4238) and FunctionTool SKIP_PARSING sentinel (#5424). All beta packages
stamped 1.0.0b260424, alpha packages 1.0.0a260424. All 26 non-core
agent-framework-core floors raised to >=1.2.0,<2. CHANGELOG consolidates
the never-tagged 1.1.1 entries with the post-merge additions into [1.2.0].

* Update CHANGELOG footer links for 1.2.0

Advance [Unreleased] comparison base from python-1.1.0 to python-1.2.0
and add a [1.2.0] reference link comparing python-1.1.0...python-1.2.0
so the heading links resolve correctly.

* Fix CHANGELOG: restore [1.1.1] section and add proper [1.2.0]

Previous commit incorrectly renamed the [1.1.1] header to [1.2.0], which
wiped the historical 1.1.1 entries and wrongly attributed them to 1.2.0.
This restores [1.1.1] to its origin/main content and adds a new [1.2.0]
section above containing only the commits in python-1.1.1..HEAD:

- #4238 functional workflow API
- #5142 GitHub Copilot OpenTelemetry
- #2403 A2A bridge support
- #5070 oauth_consent_request events in Foundry clients
- #5447 FoundryAgent hosted agent sessions
- #5459 hosting server dependency upgrade + types
- #5389 AG-UI reasoning/multimodal parsing fix
- #5440 stop [TOOLBOXES] warning spam
- #5455 user agent prefix fix

Also corrects the [1.2.0] compare base to python-1.1.1 (not 1.1.0) and
adds the missing [1.1.1] reference link.
2026-04-24 19:54:59 +09:00
7b70f80036 Python: Surface oauth_consent_request events from Responses API in Foundry clients (#5070)
* Fix Foundry clients not surfacing oauth_consent_request events (#5054)

Override _parse_chunk_from_openai in both RawFoundryChatClient and
RawFoundryAgentChatClient to intercept response.output_item.added
events with item.type == 'oauth_consent_request'. The consent link
is validated (HTTPS required) and converted to
Content.from_oauth_consent_request, which the AG-UI layer already
knows how to emit as a CUSTOM event.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address PR review feedback for #5054 OAuth consent parsing

- Extract shared helper (try_parse_oauth_consent_event) to avoid
  duplicated logic between RawFoundryChatClient and
  RawFoundryAgentChatClient
- Use urllib.parse.urlparse() for HTTPS validation instead of
  case-sensitive startswith check
- Sanitize log messages to avoid leaking consent_link tokens;
  log only item id
- Add model=self.model to ChatResponseUpdate to match parent behavior
- Add assertions on role, raw_representation, and model in happy-path
  tests
- Add test for empty-string consent_link
- Add test verifying non-oauth events delegate to super()

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Handle response.oauth_consent_requested top-level event (#5054)

Add support for the top-level response.oauth_consent_requested stream
event in addition to the response.output_item.added variant. The
service may emit either form; handle both so the consent link is
reliably surfaced.

Extract _validate_consent_link helper within _oauth_helpers.py to
reduce nesting.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address review feedback for #5054: Python: [Bug]: `FoundryAgent` (Responses API) Does Not Surface `oauth_consent_request` as a CUSTOM AG-UI Event

* Address review feedback: defensive getattr and dedicated helper tests (#5054)

- Use getattr(event, 'type', None) in try_parse_oauth_consent_event
  for defensive access against malformed events without a type attribute
- Add test_oauth_helpers.py with unit tests for _validate_consent_link
  and try_parse_oauth_consent_event covering edge cases:
  - HTTPS URL with empty netloc (https:///path)
  - Warning log messages for rejected consent links
  - Event objects missing 'type' attribute

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address review feedback for #5054: Python: [Bug]: `FoundryAgent` (Responses API) Does Not Surface `oauth_consent_request` as a CUSTOM AG-UI Event

* Fix mypy: match _parse_chunk_from_openai signature with superclass

Add seen_reasoning_delta_item_ids parameter to _parse_chunk_from_openai
overrides in both RawFoundryChatClient and RawFoundryAgentChatClient to
match the updated superclass signature on main. Update super() calls and
test assertions accordingly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Evan Mattson <evan.mattson@microsoft.com>
2026-04-24 09:59:14 +00:00
da32e8cf80 Python: (core): Add functional workflow API (#4238)
* Add functional workflow api

* cleanup

* More cleanup

* address copilot feedback

* Address PR feedbacK

* updates

* PR feedback

* Address review comments on functional workflow samples

- Swap 05/06 get-started samples: agent workflow first (motivates
  why workflows exist), simple text workflow second
- Rename text_pipeline → text_workflow, poem_pipeline → poem_workflow
- Add @step to agent workflow sample (05) to demonstrate caching
- Switch agent samples to AzureOpenAIResponsesClient with Foundry
- Remove .as_agent() from agent_integration.py to focus on the key
  difference between inline agent calls vs @step-cached calls
- Add commented-out Agent.run example in hitl_review.py
- Add clarifying comment in _functional.py that event streaming is
  buffered (not true per-token streaming)
- Add naive_group_chat.py functional sample: round-robin group chat
  as a plain Python loop
- Update READMEs to reflect new file names and group chat sample

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix pyright type errors

* Address PR review comments on functional workflow API

1. Allow request_info inside @step: Auto-inject RunContext into step
   functions that declare a RunContext parameter (by type or name 'ctx'),
   and expose get_run_context() for programmatic access.

2. Handle None responses: Log a warning when a response value is None,
   and document the behavior in request_info docstring.

3. Add executor_bypassed event type: Replace executor_invoked +
   executor_completed with a single executor_bypassed event when a step
   replays from cache, making cached vs live execution explicit.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add regression tests for PR review comments on functional workflow API

The three review comments (request_info in @step, None response handling,
executor_bypassed event type) were already addressed in 7da7db4e. This
commit adds cross-cutting regression tests that exercise the interactions
between these features:

- HITL in step with caching: preceding step bypassed on resume
- Full checkpoint lifecycle with HITL step (interrupt -> resume -> restore)
- None response inside step-level request_info logs warning
- WorkflowInterrupted from step does not emit executor_failed

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address PR #4238 review comments on functional workflow API

Comment 1 (request_info in @step): Already supported. Added comment in
StepWrapper.__call__ explaining why WorkflowInterrupted (BaseException)
safely bypasses the except Exception handler.

Comment 2 (None response): Added docstring to _get_response clarifying
the (found, value) return tuple semantics and None handling.

Comment 3 (bypass event type): executor_bypassed is already a dedicated
event type in WorkflowEventType. Updated comment at the bypass site to
make the deliberate event type choice explicit.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add experimental API warnings to functional workflow module

Mark all public classes and decorators (workflow, step, RunContext,
FunctionalWorkflow, StepWrapper, FunctionalWorkflowAgent) as
experimental and subject to change or removal.

* Address PR #4238 review comments from @eavanvalkenburg

- RunContext docstring leads with purpose (opt-in handle for HITL,
  custom events, state) so readers importing it from the public surface
  understand its role before the mechanics (#2993513452).
- Rename `06_first_functional_workflow.py` to
  `06_functional_workflow_basics.py`; the previous filename was
  confusing since it followed `05_functional_workflow_with_agents.py`
  (#2993531979).
- Simplify `05_functional_workflow_with_agents.py` to call agents
  directly without a @step wrapper; the step-vs-no-step contrast lives
  in `03-workflows/functional/agent_integration.py`, keeping the
  get-started sample minimal (#2993525532).
- Switch functional samples to `FoundryChatClient` for consistency with
  the rest of 01-get-started and 03-workflows (follow-up on #2876988570).
- Use walrus in `hitl_review.py` final-state assertion (#2993572182).
- Add expected-output block to `basic_streaming_pipeline.py` (#2993557609).
- Clarify in `parallel_pipeline.py` that `@step` composes with
  `asyncio.gather` (#2993597282).
- `naive_group_chat.py` threads `list[Message]` between turns instead
  of stringifying the transcript, preserving role/authorship (#2993583231).

Drive-by: pre-commit hook sorts an unrelated import block in
`samples/04-hosting/foundry-hosted-agents/responses/02_local_tools/main.py`.

* Fix 10 functional-workflow API bugs from /ultrareview pass

- bug_001: `ctx.request_info()` without an explicit `request_id` now derives
  a deterministic `auto::<index>` id from the call-counter, so HITL resume
  works correctly on the documented default path.  A uuid was regenerated on
  every replay, making resume impossible.

- bug_002: `StepWrapper.__call__` no longer deepcopies arguments on the
  cache-hit replay branch.  The copy is only performed on the live-execution
  path (for the event log) and falls back to the original mapping if deepcopy
  fails, so steps whose args aren't deepcopyable (locks, sockets, sessions)
  can still resume from checkpoint.

- bug_007: `_set_responses` now prunes each resolved `request_id` from
  `_pending_requests`, and the cache-hit branch in `request_info` does the
  same.  Previously, answered requests were re-serialized into every
  subsequent checkpoint and the final checkpoint falsely claimed pending
  requests even after the workflow completed.

- bug_008: `_compute_signature_hash` now mixes the function's `co_code` and
  `co_names` into the checkpoint signature, so changes to the workflow body
  invalidate older checkpoints even when steps are accessed via module /
  class attributes (which `_discover_step_names` can't see statically).
  `RunContext._record_observed_step` records observed step names for
  diagnostics.

- bug_010: `FunctionalWorkflow.run()` docstring corrected — says "at least
  one of message/responses/checkpoint_id" and explicitly notes `responses`
  may be combined with `checkpoint_id` (the validator already allowed this).

- bug_013: `FunctionalWorkflowAgent` now surfaces `request_info` events as
  `FunctionApprovalRequestContent` items (mirroring graph `WorkflowAgent`),
  threads `responses=` and `checkpoint_id=` through to the underlying
  workflow, and exposes `pending_requests`.  Previously `.as_agent()`
  returned empty `AgentResponse` for HITL workflows — effectively unusable.

- bug_014: `FunctionalWorkflow` now clears `_last_message`,
  `_last_step_cache`, and `_last_pending_request_ids` on clean completion.
  `run()` validates that `responses=` keys intersect the currently-pending
  request set (or raises with a clear error) instead of silently replaying
  against stale singleton state from a prior run.

- bug_015: `FunctionalWorkflow.as_agent` signature now matches graph
  `Workflow.as_agent`: accepts `name`, `description`, `context_providers`,
  and `**kwargs`.  `FunctionalWorkflowAgent` stores the overrides.

- bug_017: `RunContext.set_state` raises `ValueError` for underscore-
  prefixed keys (the framework's `_step_cache` / `_original_message` keys
  would silently clobber user state on checkpoint save and user
  underscore-prefixed state was dropped on restore).  Docstring documents
  the reserved prefix.

- merged_bug_003: Workflow function arity is validated at decoration time.
  Multiple non-ctx parameters raise `ValueError` immediately (previously
  every arg past the first was silently dropped at call time).  Passing a
  non-None `message` to a ctx-only workflow raises `ValueError` instead of
  silently discarding the message.

Test coverage: +18 regression tests covering every fix.  Full workflow
suite now 766 passed, 1 skipped, 2 xfailed; full core suite 2338 passed.

* Deslop functional.py fix commit

- Remove dead instrumentation added in the prior commit that was never
  consumed: `RunContext._observed_step_names`,
  `RunContext._record_observed_step`, `FunctionalWorkflow._runtime_step_names`,
  and `FunctionalWorkflowAgent._extra_kwargs`.  The signature hash relies on
  `co_code` alone, which covers the attribute-access case without the
  collection-scaffolding.
- Trim over-explanatory comments that restated what the code does or what
  it no longer does.  Keep only the comments that answer "why" for the
  non-obvious bits (deterministic id contract, defensive deepcopy, stale
  replay guard).
- Compress the `_compute_signature_hash` and FunctionalWorkflow `__init__`
  block docstrings without losing the user-facing reasoning.

Net -49 lines.  Regression lock preserved (766 passed, 1 skipped, 2 xfailed).

* Fix functional workflow review feedback

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <copilot@github.com>
2026-04-24 09:41:20 +00:00
62e02da698 Python: update FoundryAgent for hosted agent sessions (#5447)
* fixes to FoundryAgent to connect to new hosted agents

Co-authored-by: Copilot <copilot@github.com>

* fix mypy

Co-authored-by: Copilot <copilot@github.com>

* Python: remove Foundry service session helpers

Remove the public hosted-agent service session CRUD helpers from FoundryAgent and drop the related feature-stage inventory entry.

Update the hosted-agent sample to create and delete service sessions directly through the preview AIProjectClient APIs, and tighten a few test harnesses surfaced by full workspace validation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix from merge

* fix hosted env detection

Co-authored-by: Copilot <copilot@github.com>

* reverted sample update

* fix tests and code

Co-authored-by: Copilot <copilot@github.com>

* remove aenter

* skipping some tests

Co-authored-by: Copilot <copilot@github.com>

---------

Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-24 09:25:03 +00:00
Giles OdigweandCopilot 733bfb9bfe Bump Ollama model to qwen2.5:1.5b for better instruction following
The 0.5b model was too small to reliably follow simple prompts like
'Say Hello World', causing test assertion failures. The 1.5b model
follows instructions more reliably while still being small enough
for fast CI pulls (~1GB).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-23 14:51:16 -07:00
Giles OdigweandCopilot 101f50134c Enable Ollama integration tests in CI and rename report to Integration Test Report
- Install Ollama, cache models (qwen2.5:0.5b + nomic-embed-text), and start
  server in the Misc integration job for both workflow files
- Set OLLAMA_MODEL and OLLAMA_EMBEDDING_MODEL env vars so the 5 Ollama tests
  are no longer skipped
- Rename Flaky Test Report to Integration Test Report throughout (job names,
  artifact names, cache keys, file names, script titles/docstrings)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-23 14:25:38 -07:00
285 changed files with 22575 additions and 3120 deletions
+106 -18
View File
@@ -157,6 +157,8 @@ jobs:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
ANTHROPIC_CHAT_MODEL: ${{ vars.ANTHROPIC_CHAT_MODEL_ID }}
LOCAL_MCP_URL: ${{ vars.LOCAL_MCP__URL }}
OLLAMA_MODEL: qwen2.5:1.5b
OLLAMA_EMBEDDING_MODEL: nomic-embed-text
defaults:
run:
working-directory: python
@@ -171,6 +173,43 @@ jobs:
with:
python-version: ${{ env.UV_PYTHON }}
os: ${{ runner.os }}
- name: Install Ollama
run: curl -fsSL https://ollama.com/install.sh | sh
working-directory: .
- name: Cache Ollama models
uses: actions/cache@v4
with:
path: ~/.ollama/models
key: ollama-models-qwen2.5-1.5b-nomic-embed-text-v1
- name: Start Ollama and pull models
run: |
# Stop any Ollama instance auto-started by the install script
pkill ollama || true
sleep 2
ollama serve &
for i in $(seq 1 30); do
if curl -sf http://localhost:11434/api/tags > /dev/null 2>&1; then
break
fi
sleep 1
done
# Pull models with retry for transient 429 rate limits
for model in qwen2.5:1.5b nomic-embed-text; do
pulled=false
for attempt in 1 2 3; do
if ollama pull "$model"; then
pulled=true
break
fi
echo "Retry $attempt for $model (waiting 15s)..."
sleep 15
done
if [ "$pulled" != "true" ]; then
echo "ERROR: Failed to pull $model after 3 attempts"
exit 1
fi
done
working-directory: .
- name: Start local MCP server
id: local-mcp
uses: ./.github/actions/setup-local-mcp-server
@@ -271,7 +310,7 @@ jobs:
-m integration
-n logical --dist worksteal
-x
--timeout=360 --session-timeout=900 --timeout_method thread
--timeout=480 --session-timeout=900 --timeout_method thread
--retries 2 --retry-delay 5
--junitxml=pytest.xml
- name: Upload test results
@@ -336,6 +375,53 @@ jobs:
path: ./python/pytest.xml
if-no-files-found: ignore
# Foundry Hosting integration tests
python-tests-foundry-hosting:
name: Python Integration Tests - Foundry Hosting
runs-on: ubuntu-latest
environment: integration
timeout-minutes: 60
env:
FOUNDRY_PROJECT_ENDPOINT: ${{ vars.FOUNDRY_PROJECT_ENDPOINT }}
FOUNDRY_MODEL: ${{ vars.FOUNDRY_MODEL }}
defaults:
run:
working-directory: python
steps:
- uses: actions/checkout@v6
with:
ref: ${{ inputs.checkout-ref }}
persist-credentials: false
- name: Set up python and install the project
id: python-setup
uses: ./.github/actions/python-setup
with:
python-version: ${{ env.UV_PYTHON }}
os: ${{ runner.os }}
- name: Azure CLI Login
uses: azure/login@v2
with:
client-id: ${{ secrets.AZURE_CLIENT_ID }}
tenant-id: ${{ secrets.AZURE_TENANT_ID }}
subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
- name: Test with pytest (Foundry Hosting integration)
timeout-minutes: 15
run: >
uv run pytest --import-mode=importlib
packages/foundry_hosting/tests
-m integration
-n logical --dist worksteal
--timeout=120 --session-timeout=900 --timeout_method thread
--retries 2 --retry-delay 5
--junitxml=pytest.xml
- name: Upload test results
if: always()
uses: actions/upload-artifact@v7
with:
name: test-results-foundry-hosting
path: ./python/pytest.xml
if-no-files-found: ignore
# Azure Cosmos integration tests
python-tests-cosmos:
name: Python Integration Tests - Cosmos
@@ -388,9 +474,9 @@ jobs:
path: ./python/pytest.xml
if-no-files-found: ignore
# Flaky test trend report (aggregates per-job JUnit XML results)
python-flaky-test-report:
name: Flaky Test Report
# Integration test trend report (aggregates per-job JUnit XML results)
python-integration-test-report:
name: Integration Test Report
if: >
always() &&
(contains(join(needs.*.result, ','), 'success') ||
@@ -402,6 +488,7 @@ jobs:
python-tests-misc-integration,
python-tests-functions,
python-tests-foundry,
python-tests-foundry-hosting,
python-tests-cosmos,
]
runs-on: ubuntu-latest
@@ -423,36 +510,36 @@ jobs:
with:
pattern: test-results-*
path: test-results/
- name: Restore flaky report history cache
- name: Restore report history cache
uses: actions/cache/restore@v4
with:
path: python/flaky-report-history.json
key: flaky-report-history-integration-${{ github.run_id }}
path: python/integration-report-history.json
key: integration-report-history-integration-${{ github.run_id }}
restore-keys: |
flaky-report-history-integration-
integration-report-history-integration-
- name: Generate trend report
run: >
uv run python scripts/flaky_report/aggregate.py
uv run python scripts/integration_test_report/aggregate.py
../test-results/
flaky-report-history.json
flaky-test-report.md
integration-report-history.json
integration-test-report.md
- name: Post to Job Summary
if: always()
run: cat flaky-test-report.md >> $GITHUB_STEP_SUMMARY
- name: Save flaky report history cache
run: cat integration-test-report.md >> $GITHUB_STEP_SUMMARY
- name: Save report history cache
if: always()
uses: actions/cache/save@v4
with:
path: python/flaky-report-history.json
key: flaky-report-history-integration-${{ github.run_id }}
path: python/integration-report-history.json
key: integration-report-history-integration-${{ github.run_id }}
- name: Upload unified trend report
if: always()
uses: actions/upload-artifact@v7
with:
name: flaky-test-report
name: integration-test-report
path: |
python/flaky-test-report.md
python/flaky-report-history.json
python/integration-test-report.md
python/integration-report-history.json
python-integration-tests-check:
if: always()
@@ -465,6 +552,7 @@ jobs:
python-tests-misc-integration,
python-tests-functions,
python-tests-foundry,
python-tests-foundry-hosting,
python-tests-cosmos
]
steps:
+123 -18
View File
@@ -38,6 +38,7 @@ jobs:
miscChanged: ${{ steps.filter.outputs.misc }}
functionsChanged: ${{ steps.filter.outputs.functions }}
foundryChanged: ${{ steps.filter.outputs.foundry }}
foundryHostingChanged: ${{ steps.filter.outputs.foundry_hosting }}
cosmosChanged: ${{ steps.filter.outputs.cosmos }}
steps:
- uses: actions/checkout@v6
@@ -80,6 +81,8 @@ jobs:
- 'python/packages/foundry/**'
- 'python/samples/**/providers/foundry/**'
- 'python/samples/02-agents/embeddings/foundry_embeddings.py'
foundry_hosting:
- 'python/packages/foundry_hosting/**'
cosmos:
- 'python/packages/azure-cosmos/**'
# run only if 'python' files were changed
@@ -275,6 +278,8 @@ jobs:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
ANTHROPIC_CHAT_MODEL: ${{ vars.ANTHROPIC_CHAT_MODEL_ID }}
LOCAL_MCP_URL: ${{ vars.LOCAL_MCP__URL }}
OLLAMA_MODEL: qwen2.5:1.5b
OLLAMA_EMBEDDING_MODEL: nomic-embed-text
defaults:
run:
working-directory: python
@@ -286,6 +291,43 @@ jobs:
with:
python-version: ${{ env.UV_PYTHON }}
os: ${{ runner.os }}
- name: Install Ollama
run: curl -fsSL https://ollama.com/install.sh | sh
working-directory: .
- name: Cache Ollama models
uses: actions/cache@v4
with:
path: ~/.ollama/models
key: ollama-models-qwen2.5-1.5b-nomic-embed-text-v1
- name: Start Ollama and pull models
run: |
# Stop any Ollama instance auto-started by the install script
pkill ollama || true
sleep 2
ollama serve &
for i in $(seq 1 30); do
if curl -sf http://localhost:11434/api/tags > /dev/null 2>&1; then
break
fi
sleep 1
done
# Pull models with retry for transient 429 rate limits
for model in qwen2.5:1.5b nomic-embed-text; do
pulled=false
for attempt in 1 2 3; do
if ollama pull "$model"; then
pulled=true
break
fi
echo "Retry $attempt for $model (waiting 15s)..."
sleep 15
done
if [ "$pulled" != "true" ]; then
echo "ERROR: Failed to pull $model after 3 attempts"
exit 1
fi
done
working-directory: .
- name: Start local MCP server
id: local-mcp
uses: ./.github/actions/setup-local-mcp-server
@@ -400,7 +442,7 @@ jobs:
-m integration
-n logical --dist worksteal
-x
--timeout=360 --session-timeout=900 --timeout_method thread
--timeout=480 --session-timeout=900 --timeout_method thread
--retries 2 --retry-delay 5
--junitxml=pytest.xml
working-directory: ./python
@@ -488,6 +530,67 @@ jobs:
path: ./python/pytest.xml
if-no-files-found: ignore
# Foundry Hosting integration tests
python-tests-foundry-hosting:
name: Python Tests - Foundry Hosting Integration
needs: paths-filter
if: >
github.event_name != 'pull_request' &&
needs.paths-filter.outputs.pythonChanges == 'true' &&
(github.event_name != 'merge_group' ||
needs.paths-filter.outputs.foundryHostingChanged == 'true' ||
needs.paths-filter.outputs.coreChanged == 'true')
runs-on: ubuntu-latest
environment: integration
env:
FOUNDRY_PROJECT_ENDPOINT: ${{ vars.FOUNDRY_PROJECT_ENDPOINT }}
FOUNDRY_MODEL: ${{ vars.FOUNDRY_MODEL }}
defaults:
run:
working-directory: python
steps:
- uses: actions/checkout@v6
- name: Set up python and install the project
id: python-setup
uses: ./.github/actions/python-setup
with:
python-version: ${{ env.UV_PYTHON }}
os: ${{ runner.os }}
- name: Azure CLI Login
if: github.event_name != 'pull_request'
uses: azure/login@v2
with:
client-id: ${{ secrets.AZURE_CLIENT_ID }}
tenant-id: ${{ secrets.AZURE_TENANT_ID }}
subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
- name: Test with pytest (Foundry Hosting integration)
timeout-minutes: 15
run: >
uv run pytest --import-mode=importlib
packages/foundry_hosting/tests
-m integration
-n logical --dist worksteal
--timeout=120 --session-timeout=900 --timeout_method thread
--retries 2 --retry-delay 5
--junitxml=pytest.xml
working-directory: ./python
- name: Surface failing tests
if: always()
uses: pmeier/pytest-results-action@v0.7.2
with:
path: ./python/pytest.xml
summary: true
display-options: fEX
fail-on-empty: false
title: Foundry Hosting integration test results
- name: Upload test results
if: always()
uses: actions/upload-artifact@v7
with:
name: test-results-foundry-hosting
path: ./python/pytest.xml
if-no-files-found: ignore
# TODO: Add python-tests-lab
# Azure Cosmos integration tests
@@ -555,9 +658,9 @@ jobs:
path: ./python/pytest.xml
if-no-files-found: ignore
# Flaky test trend report (aggregates per-job JUnit XML results)
python-flaky-test-report:
name: Flaky Test Report
# Integration test trend report (aggregates per-job JUnit XML results)
python-integration-test-report:
name: Integration Test Report
if: >
always() &&
(contains(join(needs.*.result, ','), 'success') ||
@@ -569,6 +672,7 @@ jobs:
python-tests-misc-integration,
python-tests-functions,
python-tests-foundry,
python-tests-foundry-hosting,
python-tests-cosmos,
]
runs-on: ubuntu-latest
@@ -587,36 +691,36 @@ jobs:
with:
pattern: test-results-*
path: test-results/
- name: Restore flaky report history cache
- name: Restore report history cache
uses: actions/cache/restore@v4
with:
path: python/flaky-report-history.json
key: flaky-report-history-merge-${{ github.run_id }}
path: python/integration-report-history.json
key: integration-report-history-merge-${{ github.run_id }}
restore-keys: |
flaky-report-history-merge-
integration-report-history-merge-
- name: Generate trend report
run: >
uv run python scripts/flaky_report/aggregate.py
uv run python scripts/integration_test_report/aggregate.py
../test-results/
flaky-report-history.json
flaky-test-report.md
integration-report-history.json
integration-test-report.md
- name: Post to Job Summary
if: always()
run: cat flaky-test-report.md >> $GITHUB_STEP_SUMMARY
- name: Save flaky report history cache
run: cat integration-test-report.md >> $GITHUB_STEP_SUMMARY
- name: Save report history cache
if: always()
uses: actions/cache/save@v4
with:
path: python/flaky-report-history.json
key: flaky-report-history-merge-${{ github.run_id }}
path: python/integration-report-history.json
key: integration-report-history-merge-${{ github.run_id }}
- name: Upload unified trend report
if: always()
uses: actions/upload-artifact@v7
with:
name: flaky-test-report
name: integration-test-report
path: |
python/flaky-test-report.md
python/flaky-report-history.json
python/integration-test-report.md
python/integration-report-history.json
python-integration-tests-check:
if: always()
@@ -629,6 +733,7 @@ jobs:
python-tests-misc-integration,
python-tests-functions,
python-tests-foundry,
python-tests-foundry-hosting,
python-tests-cosmos,
]
steps:
+4
View File
@@ -242,3 +242,7 @@ python/dotnet-ref
# Generated filtered solution files (created by eng/scripts/New-FilteredSolution.ps1)
dotnet/filtered-*.slnx
**/*.lscache
# Local tool state
.omc/
.omx/
+9 -9
View File
@@ -56,15 +56,15 @@
<PackageVersion Include="System.Threading.Tasks.Extensions" Version="4.6.3" />
<PackageVersion Include="System.Net.Security" Version="4.3.2" />
<!-- OpenTelemetry -->
<PackageVersion Include="OpenTelemetry" Version="1.15.0" />
<PackageVersion Include="OpenTelemetry.Api" Version="1.15.0" />
<PackageVersion Include="OpenTelemetry.Exporter.Console" Version="1.15.0" />
<PackageVersion Include="OpenTelemetry.Exporter.InMemory" Version="1.15.0" />
<PackageVersion Include="OpenTelemetry.Exporter.OpenTelemetryProtocol" Version="1.15.0" />
<PackageVersion Include="OpenTelemetry.Extensions.Hosting" Version="1.14.0" />
<PackageVersion Include="OpenTelemetry.Instrumentation.AspNetCore" Version="1.14.0" />
<PackageVersion Include="OpenTelemetry.Instrumentation.Http" Version="1.14.0" />
<PackageVersion Include="OpenTelemetry.Instrumentation.Runtime" Version="1.14.0" />
<PackageVersion Include="OpenTelemetry" Version="1.15.3" />
<PackageVersion Include="OpenTelemetry.Api" Version="1.15.3" />
<PackageVersion Include="OpenTelemetry.Exporter.Console" Version="1.15.3" />
<PackageVersion Include="OpenTelemetry.Exporter.InMemory" Version="1.15.3" />
<PackageVersion Include="OpenTelemetry.Exporter.OpenTelemetryProtocol" Version="1.15.3" />
<PackageVersion Include="OpenTelemetry.Extensions.Hosting" Version="1.15.3" />
<PackageVersion Include="OpenTelemetry.Instrumentation.AspNetCore" Version="1.15.2" />
<PackageVersion Include="OpenTelemetry.Instrumentation.Http" Version="1.15.1" />
<PackageVersion Include="OpenTelemetry.Instrumentation.Runtime" Version="1.15.1" />
<!-- Microsoft.AspNetCore.* -->
<PackageVersion Include="Microsoft.AspNetCore.Authentication.JwtBearer" Version="10.0.0" />
<PackageVersion Include="Microsoft.AspNetCore.Authentication.OpenIdConnect" Version="10.0.0" />
+5 -4
View File
@@ -163,10 +163,10 @@
<Project Path="samples/02-agents/AgentsWithFoundry/Agent_Step25_ToolboxServerSideTools/Agent_Step25_ToolboxServerSideTools.csproj" />
</Folder>
<Folder Name="/Samples/02-agents/Evaluation/">
<Project Path="samples/02-agents/Evaluation/Evaluation_SimpleEval/Evaluation_SimpleEval.csproj" />
<Project Path="samples/02-agents/Evaluation/Evaluation_CustomEvals/Evaluation_CustomEvals.csproj" />
<Project Path="samples/02-agents/Evaluation/Evaluation_ExpectedOutputs/Evaluation_ExpectedOutputs.csproj" />
<Project Path="samples/02-agents/Evaluation/Evaluation_Multimodal/Evaluation_Multimodal.csproj" />
<Project Path="samples/02-agents/Evaluation/Evaluation_SimpleEval/Evaluation_SimpleEval.csproj" />
</Folder>
<Folder Name="/Samples/02-agents/AgentWithMemory/">
<File Path="samples/02-agents/AgentWithMemory/README.md" />
@@ -226,6 +226,7 @@
<Project Path="samples/03-workflows/Declarative/HostedWorkflow/HostedWorkflow.csproj" />
<Project Path="samples/03-workflows/Declarative/InputArguments/InputArguments.csproj" />
<Project Path="samples/03-workflows/Declarative/InvokeFunctionTool/InvokeFunctionTool.csproj" />
<Project Path="samples/03-workflows/Declarative/InvokeHttpRequest/InvokeHttpRequest.csproj" />
<Project Path="samples/03-workflows/Declarative/InvokeMcpTool/InvokeMcpTool.csproj" />
<Project Path="samples/03-workflows/Declarative/Marketing/Marketing.csproj" />
<Project Path="samples/03-workflows/Declarative/StudentTeacher/StudentTeacher.csproj" />
@@ -347,17 +348,17 @@
<File Path="samples/02-agents/A2A/README.md" />
<Project Path="samples/02-agents/A2A/A2AAgent_AsFunctionTools/A2AAgent_AsFunctionTools.csproj" />
<Project Path="samples/02-agents/A2A/A2AAgent_PollingForTaskCompletion/A2AAgent_PollingForTaskCompletion.csproj" />
<Project Path="samples/02-agents/A2A/A2AAgent_StreamReconnection/A2AAgent_StreamReconnection.csproj" />
<Project Path="samples/02-agents/A2A/A2AAgent_ProtocolSelection/A2AAgent_ProtocolSelection.csproj" />
<Project Path="samples/02-agents/A2A/A2AAgent_StreamReconnection/A2AAgent_StreamReconnection.csproj" />
</Folder>
<Folder Name="/Samples/05-end-to-end/">
<Project Path="samples/05-end-to-end/AgentWithPurview/AgentWithPurview.csproj" />
<Project Path="samples/05-end-to-end/M365Agent/M365Agent.csproj" />
</Folder>
<Folder Name="/Samples/05-end-to-end/Evaluation/">
<Project Path="samples/05-end-to-end/Evaluation/Evaluation_ConversationSplits/Evaluation_ConversationSplits.csproj" />
<Project Path="samples/05-end-to-end/Evaluation/Evaluation_FoundryQuality/Evaluation_FoundryQuality.csproj" />
<Project Path="samples/05-end-to-end/Evaluation/Evaluation_MixedProviders/Evaluation_MixedProviders.csproj" />
<Project Path="samples/05-end-to-end/Evaluation/Evaluation_ConversationSplits/Evaluation_ConversationSplits.csproj" />
</Folder>
<Folder Name="/Samples/05-end-to-end/A2AClientServer/">
<File Path="samples/05-end-to-end/A2AClientServer/README.md" />
@@ -543,8 +544,8 @@
<Project Path="src/Microsoft.Agents.AI.Declarative/Microsoft.Agents.AI.Declarative.csproj" />
<Project Path="src/Microsoft.Agents.AI.DevUI/Microsoft.Agents.AI.DevUI.csproj" />
<Project Path="src/Microsoft.Agents.AI.DurableTask/Microsoft.Agents.AI.DurableTask.csproj" />
<Project Path="src/Microsoft.Agents.AI.Foundry/Microsoft.Agents.AI.Foundry.csproj" />
<Project Path="src/Microsoft.Agents.AI.Foundry.Hosting/Microsoft.Agents.AI.Foundry.Hosting.csproj" />
<Project Path="src/Microsoft.Agents.AI.Foundry/Microsoft.Agents.AI.Foundry.csproj" />
<Project Path="src/Microsoft.Agents.AI.GitHub.Copilot/Microsoft.Agents.AI.GitHub.Copilot.csproj" />
<Project Path="src/Microsoft.Agents.AI.Hosting.A2A.AspNetCore/Microsoft.Agents.AI.Hosting.A2A.AspNetCore.csproj" />
<Project Path="src/Microsoft.Agents.AI.Hosting.A2A/Microsoft.Agents.AI.Hosting.A2A.csproj" />
@@ -5,16 +5,16 @@
// This is provided for demonstration purposes only.
using System.Diagnostics;
using System.Text.Json;
using Microsoft.Agents.AI;
using Microsoft.Extensions.AI;
/// <summary>
/// Executes file-based skill scripts as local subprocesses.
/// </summary>
/// <remarks>
/// This runner uses the script's absolute path, converts the arguments
/// to CLI flags, and returns captured output. It is intended for
/// demonstration purposes only.
/// This runner uses the script's absolute path and converts the arguments
/// to CLI arguments. When the LLM sends a JSON array, each element is used
/// as a positional argument. It is intended for demonstration purposes only.
/// </remarks>
internal static class SubprocessScriptRunner
{
@@ -24,7 +24,8 @@ internal static class SubprocessScriptRunner
public static async Task<object?> RunAsync(
AgentFileSkill skill,
AgentFileSkillScript script,
AIFunctionArguments arguments,
JsonElement? arguments,
IServiceProvider? serviceProvider,
CancellationToken cancellationToken)
{
if (!File.Exists(script.FullPath))
@@ -61,24 +62,27 @@ internal static class SubprocessScriptRunner
startInfo.FileName = script.FullPath;
}
if (arguments is not null)
if (arguments is { ValueKind: JsonValueKind.Array } json)
{
foreach (var (key, value) in arguments)
// Positional CLI arguments
foreach (var element in json.EnumerateArray())
{
if (value is bool boolValue)
if (element.ValueKind != JsonValueKind.String)
{
if (boolValue)
{
startInfo.ArgumentList.Add(NormalizeKey(key));
}
}
else if (value is not null)
{
startInfo.ArgumentList.Add(NormalizeKey(key));
startInfo.ArgumentList.Add(value.ToString()!);
throw new InvalidOperationException(
$"File-based skill scripts only accept string CLI arguments but received a JSON element of kind '{element.ValueKind}'. " +
"All array elements must be JSON strings.");
}
startInfo.ArgumentList.Add(element.GetString()!);
}
}
else if (arguments is not null && arguments.Value.ValueKind != JsonValueKind.Null && arguments.Value.ValueKind != JsonValueKind.Undefined)
{
throw new InvalidOperationException(
$"Expected a JSON array of CLI arguments but received {arguments.Value.ValueKind}. " +
"File-based skill scripts expect positional arguments as a JSON array of strings.");
}
Process? process = null;
try
@@ -128,10 +132,4 @@ internal static class SubprocessScriptRunner
process?.Dispose();
}
}
/// <summary>
/// Normalizes a parameter key to a consistent --flag format.
/// Models may return keys with or without leading dashes (e.g., "value" vs "--value").
/// </summary>
private static string NormalizeKey(string key) => "--" + key.TrimStart('-');
}
@@ -0,0 +1,38 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFrameworks>net10.0</TargetFrameworks>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
</PropertyGroup>
<PropertyGroup>
<InjectIsExternalInitOnLegacy>true</InjectIsExternalInitOnLegacy>
<InjectSharedFoundryAgents>true</InjectSharedFoundryAgents>
<InjectSharedWorkflowsExecution>true</InjectSharedWorkflowsExecution>
<InjectSharedWorkflowsSettings>true</InjectSharedWorkflowsSettings>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Extensions.Configuration" />
<PackageReference Include="Microsoft.Extensions.Configuration.Binder" />
<PackageReference Include="Microsoft.Extensions.Configuration.EnvironmentVariables" />
<PackageReference Include="Microsoft.Extensions.Configuration.Json" />
<PackageReference Include="Microsoft.Extensions.Configuration.UserSecrets" />
<PackageReference Include="Microsoft.Extensions.DependencyInjection" />
<PackageReference Include="Microsoft.Extensions.Logging" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\..\..\..\src\Microsoft.Agents.AI.Workflows.Declarative\Microsoft.Agents.AI.Workflows.Declarative.csproj" />
<ProjectReference Include="..\..\..\..\src\Microsoft.Agents.AI.Workflows.Declarative.Foundry\Microsoft.Agents.AI.Workflows.Declarative.Foundry.csproj" />
</ItemGroup>
<ItemGroup>
<None Include="InvokeHttpRequest.yaml">
<CopyToOutputDirectory>Always</CopyToOutputDirectory>
</None>
</ItemGroup>
</Project>
@@ -0,0 +1,76 @@
#
# This workflow demonstrates using HttpRequestAction to call a REST API directly
# from the workflow without going through an AI agent first.
#
# HttpRequestAction allows workflows to:
# - Fetch data from external HTTP endpoints
# - Store the parsed response in workflow variables for later use
# - Add the response body to the conversation so a downstream agent can
# answer questions based on it
#
# This sample fetches public metadata for the dotnet/runtime repository from
# the GitHub REST API (no authentication required) and uses an agent to
# answer follow-up questions about it.
#
# Example input:
# How many subscribers does the repository have?
#
kind: Workflow
trigger:
kind: OnConversationStart
id: workflow_invoke_http_request_demo
actions:
# Capture the original user message for input to the follow-up agent.
- kind: SetVariable
id: set_user_message
variable: Local.InputMessage
value: =System.LastMessage
# Set the repository org/name used to form the request URL.
- kind: SetVariable
id: set_repo_name
variable: Local.RepoName
value: microsoft/agent-framework
# Invoke the GitHub repo API. The response body is parsed into Local.RepoInfo
# and also added to the conversation (via conversationId) so the agent below
# can answer questions based on it.
- kind: HttpRequestAction
id: fetch_repo_info
conversationId: =System.ConversationId
method: GET
url: =Concatenate("https://api.github.com/repos/", Local.RepoName)
headers:
Accept: application/vnd.github+json
User-Agent: agent-framework-sample
response: Local.RepoInfo
# Display a confirmation message showing key fields from the parsed response.
- kind: SendMessage
id: show_repo_summary
message: "Fetched repo: visibility={Local.RepoInfo.visibility}, description={Local.RepoInfo.description}"
# Use the agent to summarize the repo using the conversation context.
- kind: InvokeAzureAgent
id: summarize_repo
conversationId: =System.ConversationId
agent:
name: GitHubRepoInfoAgent
input:
messages: =UserMessage("Please provide a brief summary of this GitHub repository based on the data already in the conversation.")
output:
autoSend: true
messages: Local.AgentResponse
# Allow the user to ask follow-up questions about the repo in a loop.
- kind: InvokeAzureAgent
id: invoke_followup
conversationId: =System.ConversationId
agent:
name: GitHubRepoInfoAgent
input:
messages: =Local.InputMessage
externalLoop:
when: =Upper(System.LastMessage.Text) <> "EXIT"
@@ -0,0 +1,95 @@
// Copyright (c) Microsoft. All rights reserved.
using Azure.AI.Projects;
using Azure.AI.Projects.Agents;
using Azure.Identity;
using Microsoft.Agents.AI.Workflows.Declarative;
using Microsoft.Extensions.Configuration;
using Shared.Foundry;
using Shared.Workflows;
namespace Demo.Workflows.Declarative.InvokeHttpRequest;
/// <summary>
/// Demonstrates a workflow that uses HttpRequestAction to call a REST API
/// directly from the workflow.
/// </summary>
/// <remarks>
/// <para>
/// The HttpRequestAction allows workflows to issue HTTP requests and:
/// </para>
/// <list type="bullet">
/// <item>Fetch data from external REST endpoints</item>
/// <item>Store the parsed response in workflow variables</item>
/// <item>Add the response body to the conversation so an agent can answer
/// questions based on it</item>
/// </list>
/// <para>
/// This sample fetches public metadata for the dotnet/runtime repository from
/// the GitHub REST API (no authentication required) and uses a Foundry agent
/// to answer follow-up questions about it. Type "EXIT" to end the conversation.
/// </para>
/// <para>
/// See the README.md file in the parent folder (../README.md) for detailed
/// information about the configuration required to run this sample.
/// </para>
/// </remarks>
internal sealed class Program
{
public static async Task Main(string[] args)
{
// Initialize configuration
IConfiguration configuration = Application.InitializeConfig();
Uri foundryEndpoint = new(configuration.GetValue(Application.Settings.FoundryEndpoint));
// Ensure sample agent exists in Foundry. The agent has no tools - it answers
// questions about the GitHub repository using only the JSON data that the
// HttpRequestAction adds to the conversation.
await CreateAgentAsync(foundryEndpoint, configuration);
// Get input from command line or console
string workflowInput = Application.GetInput(args);
// The default HttpRequestHandler is sufficient for this sample because the
// GitHub REST endpoint used here does not require authentication. For
// authenticated endpoints, supply a custom Func<HttpRequestInfo, ..., HttpClient?>
// to DefaultHttpRequestHandler so each request can be routed through a
// pre-configured (cached) HttpClient with the appropriate credentials.
await using DefaultHttpRequestHandler httpRequestHandler = new();
// Create the workflow factory with the HTTP request handler
WorkflowFactory workflowFactory = new("InvokeHttpRequest.yaml", foundryEndpoint)
{
HttpRequestHandler = httpRequestHandler
};
// Execute the workflow
WorkflowRunner runner = new() { UseJsonCheckpoints = true };
await runner.ExecuteAsync(workflowFactory.CreateWorkflow, workflowInput);
}
private static async Task CreateAgentAsync(Uri foundryEndpoint, IConfiguration configuration)
{
// WARNING: DefaultAzureCredential is convenient for development but requires careful consideration in production.
AIProjectClient aiProjectClient = new(foundryEndpoint, new DefaultAzureCredential());
await aiProjectClient.CreateAgentAsync(
agentName: "GitHubRepoInfoAgent",
agentDefinition: DefineAgent(configuration),
agentDescription: "Answers questions about a GitHub repository using HTTP response data in the conversation");
}
private static DeclarativeAgentDefinition DefineAgent(IConfiguration configuration)
{
return new DeclarativeAgentDefinition(configuration.GetValue(Application.Settings.FoundryModel))
{
Instructions =
"""
Answer the user's questions about the GitHub repository using only the
JSON data already present in the conversation history.
If the answer is not contained in the conversation, say so plainly
rather than guessing. Be concise and helpful.
"""
};
}
}
@@ -65,6 +65,53 @@ Workflow orchestration started for CancelOrder. Orchestration runId: abc123def45
>
> If not provided, a unique run ID is auto-generated.
### Wait for the Workflow Result
By default, the HTTP endpoint returns `202 Accepted` immediately with the run ID. If you want to wait for the workflow to complete and get the result in the response, add the `x-ms-wait-for-response: true` header:
Bash (Linux/macOS/WSL):
```bash
curl -X POST http://localhost:7071/api/workflows/CancelOrder/run \
-H "Content-Type: text/plain" \
-H "x-ms-wait-for-response: true" \
-d "12345"
```
PowerShell:
```powershell
Invoke-RestMethod -Method Post `
-Uri http://localhost:7071/api/workflows/CancelOrder/run `
-ContentType text/plain `
-Headers @{ "x-ms-wait-for-response" = "true" } `
-Body "12345"
```
The response will contain the workflow result as plain text (200 OK):
```text
Cancellation email sent for order 12345 to jerry@example.com.
```
To get the result as JSON, also include the `Accept: application/json` header:
```bash
curl -X POST http://localhost:7071/api/workflows/CancelOrder/run \
-H "Content-Type: text/plain" \
-H "x-ms-wait-for-response: true" \
-H "Accept: application/json" \
-d "12345"
```
```json
{
"runId": "abc123def456",
"workflowStatus": "Completed",
"result": "Cancellation email sent for order 12345 to jerry@example.com."
}
```
In the function app logs, you will see the sequential execution of each executor:
```text
@@ -7,6 +7,21 @@ Content-Type: text/plain
12345
### Cancel an order and wait for the result
POST {{authority}}/api/workflows/CancelOrder/run
Content-Type: text/plain
x-ms-wait-for-response: true
12345
### Cancel an order and wait for the result (JSON response)
POST {{authority}}/api/workflows/CancelOrder/run
Content-Type: text/plain
Accept: application/json
x-ms-wait-for-response: true
12345
### Cancel an order with a custom run ID
POST {{authority}}/api/workflows/CancelOrder/run?runId=my-custom-id-123
Content-Type: text/plain
@@ -19,6 +34,13 @@ Content-Type: text/plain
12345
### Get order status and wait for the result
POST {{authority}}/api/workflows/OrderStatus/run
Content-Type: text/plain
x-ms-wait-for-response: true
12345
### Batch cancel orders with a complex JSON input
POST {{authority}}/api/workflows/BatchCancelOrders/run
Content-Type: application/json
@@ -13,6 +13,8 @@
<ItemGroup>
<PackageReference Include="Azure.AI.AgentServer.Invocations" />
<PackageReference Include="DotNetEnv" />
<PackageReference Include="OpenTelemetry.Api" />
<PackageReference Include="OpenTelemetry.Exporter.OpenTelemetryProtocol" />
</ItemGroup>
<!-- For contributors: uses ProjectReference to build against local source -->
@@ -297,6 +297,7 @@ public class AgentFrameworkResponseHandler : ResponseHandler
var agent = this._serviceProvider.GetKeyedService<AIAgent>(agentName);
if (agent is not null)
{
FoundryHostingExtensions.TryApplyUserAgent(agent);
return FoundryHostingExtensions.ApplyOpenTelemetry(agent);
}
@@ -310,12 +311,13 @@ public class AgentFrameworkResponseHandler : ResponseHandler
var defaultAgent = this._serviceProvider.GetService<AIAgent>();
if (defaultAgent is not null)
{
FoundryHostingExtensions.TryApplyUserAgent(defaultAgent);
return FoundryHostingExtensions.ApplyOpenTelemetry(defaultAgent);
}
var errorMessage = string.IsNullOrEmpty(agentName)
? "No agent name specified in the request (via agent.name or metadata[\"entity_id\"]) and no default AIAgent is registered."
: $"Agent '{agentName}' not found. Ensure it is registered via AddAIAgent(\"{agentName}\", ...) or as a default AIAgent.";
: $"Agent '{agentName}' not found. Ensure it is registered via AddFoundryResponses(services, agent) or services.AddKeyedSingleton<AIAgent>(\"{agentName}\", ...).";
throw new InvalidOperationException(errorMessage);
}
@@ -352,7 +354,7 @@ public class AgentFrameworkResponseHandler : ResponseHandler
var errorMessage = string.IsNullOrEmpty(agentName)
? "No agent name specified in the request (via agent.name or metadata[\"entity_id\"]) and no default AgentSessionStore is registered."
: $"Agent '{agentName}' not found. Ensure it is registered via AddAIAgent(\"{agentName}\", ...) or as a default AgentSessionStore.";
: $"AgentSessionStore for agent '{agentName}' not found. Ensure it is registered via AddFoundryResponses(services, agent, agentSessionStore) or services.AddKeyedSingleton<AgentSessionStore>(\"{agentName}\", ...).";
throw new InvalidOperationException(errorMessage);
}
@@ -0,0 +1,113 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.ClientModel;
using System.ClientModel.Primitives;
using System.Collections.Generic;
using System.Threading.Tasks;
using OpenAI;
using OpenAI.Responses;
#pragma warning disable OPENAI001, SCME0001
namespace Microsoft.Agents.AI.Foundry.Hosting;
/// <summary>
/// A <see cref="ResponsesClient"/> subclass that delegates every protocol-level request to a
/// wrapped <see cref="ResponsesClient"/>. Before each call, a
/// <see cref="HostedAgentUserAgentPolicy"/> is added to the per-call
/// <see cref="RequestOptions"/> so the wrapped client's pipeline appends the hosted-agent
/// <c>User-Agent</c> segment on the wire.
/// </summary>
/// <remarks>
/// <para>
/// The streaming overloads MEAI binds via reflection (<c>internal CreateResponseStreamingAsync(CreateResponseOptions, RequestOptions)</c>
/// and <c>internal GetResponseStreamingAsync(GetResponseOptions, RequestOptions)</c>) bottom out
/// in calls to the public-virtual non-streaming protocol overloads on <see langword="this"/>. Overriding those
/// non-streaming overloads is therefore sufficient to intercept both streaming and non-streaming traffic.
/// </para>
/// <para>
/// The base pipeline supplied to <see cref="ResponsesClient(ClientPipeline, OpenAIClientOptions)"/>
/// is a dummy pipeline whose terminal transport throws if invoked. Every override on this class
/// delegates to the inner client BEFORE any code path reaches <see cref="ResponsesClient.Pipeline"/>, so the dummy is
/// never expected to run; the throwing transport surfaces any unexpected escape route loudly.
/// </para>
/// </remarks>
internal sealed class DelegatingResponsesClient : ResponsesClient
{
private readonly ResponsesClient _inner;
public DelegatingResponsesClient(ResponsesClient inner)
: base(BuildDummyPipeline(), new OpenAIClientOptions { Endpoint = inner?.Endpoint })
{
this._inner = inner ?? throw new ArgumentNullException(nameof(inner));
}
public override async Task<ClientResult> CreateResponseAsync(BinaryContent content, RequestOptions? options = null)
=> await this._inner.CreateResponseAsync(content, AddUserAgentPolicy(options)).ConfigureAwait(false);
public override ClientResult CreateResponse(BinaryContent content, RequestOptions? options = null)
=> this._inner.CreateResponse(content, AddUserAgentPolicy(options));
public override async Task<ClientResult> GetResponseAsync(string responseId, IEnumerable<IncludedResponseProperty>? include, bool? stream, int? startingAfter, bool? includeObfuscation, RequestOptions options)
=> await this._inner.GetResponseAsync(responseId, include, stream, startingAfter, includeObfuscation, AddUserAgentPolicy(options)).ConfigureAwait(false);
public override ClientResult GetResponse(string responseId, IEnumerable<IncludedResponseProperty>? include, bool? stream, int? startingAfter, bool? includeObfuscation, RequestOptions options)
=> this._inner.GetResponse(responseId, include, stream, startingAfter, includeObfuscation, AddUserAgentPolicy(options));
public override async Task<ClientResult> DeleteResponseAsync(string responseId, RequestOptions options)
=> await this._inner.DeleteResponseAsync(responseId, AddUserAgentPolicy(options)).ConfigureAwait(false);
public override ClientResult DeleteResponse(string responseId, RequestOptions options)
=> this._inner.DeleteResponse(responseId, AddUserAgentPolicy(options));
public override async Task<ClientResult> CancelResponseAsync(string responseId, RequestOptions options)
=> await this._inner.CancelResponseAsync(responseId, AddUserAgentPolicy(options)).ConfigureAwait(false);
public override ClientResult CancelResponse(string responseId, RequestOptions options)
=> this._inner.CancelResponse(responseId, AddUserAgentPolicy(options));
public override async Task<ClientResult> GetInputTokenCountAsync(string contentType, BinaryContent content, RequestOptions? options = null)
=> await this._inner.GetInputTokenCountAsync(contentType, content, AddUserAgentPolicy(options)).ConfigureAwait(false);
public override ClientResult GetInputTokenCount(string contentType, BinaryContent content, RequestOptions? options = null)
=> this._inner.GetInputTokenCount(contentType, content, AddUserAgentPolicy(options));
public override async Task<ClientResult> CompactResponseAsync(string contentType, BinaryContent content, RequestOptions? options = null)
=> await this._inner.CompactResponseAsync(contentType, content, AddUserAgentPolicy(options)).ConfigureAwait(false);
public override ClientResult CompactResponse(string contentType, BinaryContent content, RequestOptions? options = null)
=> this._inner.CompactResponse(contentType, content, AddUserAgentPolicy(options));
public override async Task<ClientResult> GetResponseInputItemCollectionPageAsync(string responseId, int? limit, string order, string after, string before, RequestOptions options)
=> await this._inner.GetResponseInputItemCollectionPageAsync(responseId, limit, order, after, before, AddUserAgentPolicy(options)).ConfigureAwait(false);
public override ClientResult GetResponseInputItemCollectionPage(string responseId, int? limit, string order, string after, string before, RequestOptions options)
=> this._inner.GetResponseInputItemCollectionPage(responseId, limit, order, after, before, AddUserAgentPolicy(options));
private static RequestOptions AddUserAgentPolicy(RequestOptions? options)
{
options ??= new RequestOptions();
options.AddPolicy(HostedAgentUserAgentPolicy.Instance, PipelinePosition.PerCall);
return options;
}
private static ClientPipeline BuildDummyPipeline()
{
var options = new ClientPipelineOptions
{
Transport = new ThrowingTransport(),
};
return ClientPipeline.Create(options, default, default, default);
}
private sealed class ThrowingTransport : PipelineTransport
{
private const string Message =
"DelegatingResponsesClient transport invoked bypassed the override-and-delegate design. This exception should be unreachable and should never be thrown following the correct usage of DelegatingResponsesClient.";
protected override PipelineMessage CreateMessageCore() => throw new InvalidOperationException(Message);
protected override void ProcessCore(PipelineMessage message) => throw new InvalidOperationException(Message);
protected override ValueTask ProcessCoreAsync(PipelineMessage message) => throw new InvalidOperationException(Message);
}
}
@@ -0,0 +1,84 @@
// Copyright (c) Microsoft. All rights reserved.
using System.ClientModel.Primitives;
using System.Collections.Generic;
using System.Reflection;
using System.Threading.Tasks;
namespace Microsoft.Agents.AI.Foundry.Hosting;
/// <summary>
/// Pipeline policy that appends the hosted-agent <c>User-Agent</c> segment
/// (e.g. <c>"foundry-hosting/agent-framework-dotnet/{version}"</c>) to outgoing requests.
/// </summary>
/// <remarks>
/// <para>
/// The supplement value is computed once from the Microsoft.Agents.AI.Foundry.Hosting
/// assembly's informational version. The policy is idempotent on retries: if the segment
/// is already present in the <c>User-Agent</c> header, the policy does not append it again.
/// </para>
/// <para>
/// This policy is added at request time (per-call <see cref="PipelinePosition"/>)
/// by <see cref="DelegatingResponsesClient"/> when invoking the wrapped
/// <see cref="OpenAI.Responses.ResponsesClient"/>. It is only registered when an agent is
/// resolved by the Foundry hosting layer.
/// </para>
/// </remarks>
internal sealed class HostedAgentUserAgentPolicy : PipelinePolicy
{
public static HostedAgentUserAgentPolicy Instance { get; } = new HostedAgentUserAgentPolicy();
private static readonly string s_supplementValue = CreateSupplementValue();
public override void Process(PipelineMessage message, IReadOnlyList<PipelinePolicy> pipeline, int currentIndex)
{
AppendHeader(message);
ProcessNext(message, pipeline, currentIndex);
}
public override async ValueTask ProcessAsync(PipelineMessage message, IReadOnlyList<PipelinePolicy> pipeline, int currentIndex)
{
AppendHeader(message);
await ProcessNextAsync(message, pipeline, currentIndex).ConfigureAwait(false);
}
private static void AppendHeader(PipelineMessage message)
{
if (message.Request.Headers.TryGetValue("User-Agent", out var existing) && !string.IsNullOrEmpty(existing))
{
// Guard against double-append on retries or when the policy
// is registered on multiple pipeline positions.
if (existing.Contains(s_supplementValue))
{
return;
}
message.Request.Headers.Set("User-Agent", $"{existing} {s_supplementValue}");
}
else
{
message.Request.Headers.Set("User-Agent", s_supplementValue);
}
}
private static string CreateSupplementValue()
{
const string Name = "foundry-hosting/agent-framework-dotnet";
if (typeof(HostedAgentUserAgentPolicy).Assembly.GetCustomAttribute<AssemblyInformationalVersionAttribute>()?.InformationalVersion is string version)
{
int pos = version.IndexOf('+');
if (pos >= 0)
{
version = version.Substring(0, pos);
}
if (version.Length > 0)
{
return $"{Name}/{version}";
}
}
return Name;
}
}
@@ -34,6 +34,7 @@
<PackageReference Include="Azure.AI.Projects" VersionOverride="2.1.0-beta.1" />
<PackageReference Include="Azure.Identity" />
<PackageReference Include="ModelContextProtocol" />
<PackageReference Include="OpenTelemetry.Exporter.OpenTelemetryProtocol" />
</ItemGroup>
<ItemGroup>
@@ -3,16 +3,15 @@
using System;
using System.Diagnostics.CodeAnalysis;
using System.Reflection;
using System.Threading.Tasks;
using Azure.AI.AgentServer.Responses;
using Azure.Core;
using Azure.Identity;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Routing;
using Microsoft.Extensions.AI;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.DependencyInjection.Extensions;
using Microsoft.Shared.DiagnosticIds;
using OpenAI.Responses;
namespace Microsoft.Agents.AI.Foundry.Hosting;
@@ -36,7 +35,7 @@ public static class FoundryHostingExtensions
/// <para>
/// Example:
/// <code>
/// builder.AddAIAgent("my-agent", ...);
/// builder.Services.AddKeyedSingleton&lt;AIAgent&gt;("my-agent", myAgent);
/// builder.Services.AddFoundryResponses();
///
/// var app = builder.Build();
@@ -181,13 +180,6 @@ public static class FoundryHostingExtensions
{
ArgumentNullException.ThrowIfNull(endpoints);
endpoints.MapResponsesServer(prefix);
if (endpoints is IApplicationBuilder app)
{
// Ensure the middleware is added to the pipeline
app.UseMiddleware<AgentFrameworkUserAgentMiddleware>();
}
return endpoints;
}
@@ -216,46 +208,85 @@ public static class FoundryHostingExtensions
.Build();
}
private sealed class AgentFrameworkUserAgentMiddleware(RequestDelegate next)
/// <summary>
/// Attempts to wrap the agent's underlying <see cref="ResponsesClient"/>
/// with a <see cref="DelegatingResponsesClient"/> so every outgoing Responses-API request
/// carries the hosted-agent <c>User-Agent</c> segment.
/// </summary>
/// <remarks>
/// <para>
/// Best-effort and idempotent. The method is a no-op when:
/// <list type="bullet">
/// <item><description><paramref name="agent"/> exposes no <see cref="IChatClient"/>;</description></item>
/// <item><description>the chat client is not backed by MEAI's internal <c>OpenAIResponsesChatClient</c> (e.g., a non-OpenAI provider or a custom impl);</description></item>
/// <item><description>the inner <see cref="ResponsesClient"/> is already a <see cref="DelegatingResponsesClient"/>.</description></item>
/// </list>
/// </para>
/// <para>
/// Works for any <see cref="ResponsesClient"/>-derived inner client — both the Foundry-specific
/// <see cref="Azure.AI.Extensions.OpenAI.ProjectResponsesClient"/> and the native OpenAI
/// <see cref="ResponsesClient"/> obtained from <see cref="OpenAI.OpenAIClient"/>. The wrapper preserves
/// the inner client's pipeline (Transport, RetryPolicy, NetworkTimeout, OrganizationId / ProjectId /
/// UserAgentApplicationId, custom policies) because every override delegates to the inner instance.
/// </para>
/// <para>
/// Returns the same <paramref name="agent"/> instance unchanged. Mutation happens via
/// reflection on MEAI's private <c>_responseClient</c> field; the agent itself is not wrapped.
/// </para>
/// </remarks>
internal static AIAgent TryApplyUserAgent(AIAgent agent)
{
private static readonly string s_userAgentValue = CreateUserAgentValue();
public async Task InvokeAsync(HttpContext context)
var chatClient = agent.GetService<IChatClient>();
if (chatClient is null)
{
var headers = context.Request.Headers;
var userAgent = headers.UserAgent.ToString();
if (string.IsNullOrEmpty(userAgent))
{
headers.UserAgent = s_userAgentValue;
}
else if (!userAgent.Contains(s_userAgentValue, StringComparison.OrdinalIgnoreCase))
{
headers.UserAgent = $"{userAgent} {s_userAgentValue}";
}
await next(context).ConfigureAwait(false);
return agent;
}
private static string CreateUserAgentValue()
var meaiType = s_meaiResponsesChatClientType;
if (meaiType is null)
{
const string Name = "agent-framework-dotnet";
if (typeof(AgentFrameworkUserAgentMiddleware).Assembly.GetCustomAttribute<AssemblyInformationalVersionAttribute>()?.InformationalVersion is string version)
{
int pos = version.IndexOf('+');
if (pos >= 0)
{
version = version.Substring(0, pos);
}
if (version.Length > 0)
{
return $"{Name}/{version}";
}
}
return Name;
return agent;
}
var meaiInstance = chatClient.GetService(meaiType);
if (meaiInstance is null)
{
return agent;
}
var field = s_meaiResponseClientField;
if (field is null)
{
return agent;
}
var current = field.GetValue(meaiInstance) as ResponsesClient;
if (current is null or DelegatingResponsesClient)
{
return agent;
}
field.SetValue(meaiInstance, new DelegatingResponsesClient(current));
return agent;
}
/// <summary>
/// MEAI's internal <c>OpenAIResponsesChatClient</c> type, resolved once via reflection.
/// <see langword="null"/> if the type cannot be found (e.g., MEAI version drift).
/// </summary>
[UnconditionalSuppressMessage("Trimming", "IL2026:RequiresUnreferencedCode",
Justification = "MEAI's OpenAIResponsesChatClient is referenced through MicrosoftExtensionsAIResponsesExtensions and survives trimming.")]
[UnconditionalSuppressMessage("Trimming", "IL2073:RequiresUnreferencedCode",
Justification = "MEAI's OpenAIResponsesChatClient is referenced through MicrosoftExtensionsAIResponsesExtensions and survives trimming.")]
private static readonly Type? s_meaiResponsesChatClientType =
typeof(MicrosoftExtensionsAIResponsesExtensions).Assembly.GetType("Microsoft.Extensions.AI.OpenAIResponsesChatClient");
/// <summary>
/// MEAI's internal <c>_responseClient</c> field on <c>OpenAIResponsesChatClient</c>,
/// resolved once via reflection. <see langword="null"/> if the field cannot be found.
/// </summary>
[UnconditionalSuppressMessage("Trimming", "IL2080:RequiresDynamicallyAccessedMembers",
Justification = "OpenAIResponsesChatClient and its private fields are preserved by the polyfill design; MEAI does the same reflection internally.")]
private static readonly FieldInfo? s_meaiResponseClientField =
s_meaiResponsesChatClientType?.GetField("_responseClient", BindingFlags.NonPublic | BindingFlags.Instance);
}
@@ -3,7 +3,6 @@
using System.ClientModel.Primitives;
using System.Collections.Generic;
using System.Reflection;
using System.Threading;
using System.Threading.Tasks;
namespace Microsoft.Agents.AI;
@@ -13,20 +12,6 @@ internal static class RequestOptionsExtensions
/// <summary>Gets the singleton <see cref="PipelinePolicy"/> that adds a MEAI user-agent header.</summary>
internal static PipelinePolicy UserAgentPolicy => MeaiUserAgentPolicy.Instance;
/// <summary>Creates a <see cref="RequestOptions"/> configured for use with Foundry Agents.</summary>
public static RequestOptions ToRequestOptions(this CancellationToken cancellationToken, bool streaming)
{
RequestOptions requestOptions = new()
{
CancellationToken = cancellationToken,
BufferResponse = !streaming
};
requestOptions.AddPolicy(MeaiUserAgentPolicy.Instance, PipelinePosition.PerCall);
return requestOptions;
}
/// <summary>Provides a pipeline policy that adds a "MEAI/x.y.z" user-agent header.</summary>
private sealed class MeaiUserAgentPolicy : PipelinePolicy
{
@@ -21,6 +21,8 @@ internal static class BuiltInFunctions
internal const string HttpPrefix = "http-";
internal const string McpToolPrefix = "mcptool-";
private const string WaitForResponseHeaderName = "x-ms-wait-for-response";
internal static readonly string RunAgentHttpFunctionEntryPoint = $"{typeof(BuiltInFunctions).FullName!}.{nameof(RunAgentHttpAsync)}";
internal static readonly string RunAgentEntityFunctionEntryPoint = $"{typeof(BuiltInFunctions).FullName!}.{nameof(InvokeAgentAsync)}";
internal static readonly string RunAgentMcpToolFunctionEntryPoint = $"{typeof(BuiltInFunctions).FullName!}.{nameof(RunMcpToolAsync)}";
@@ -62,6 +64,11 @@ internal static class BuiltInFunctions
StartOrchestrationOptions? options = instanceId is not null ? new StartOrchestrationOptions(instanceId) : null;
string resolvedInstanceId = await client.ScheduleNewOrchestrationInstanceAsync(orchestrationFunctionName, orchestrationInput, options);
if (ShouldWaitForResponse(req, defaultValue: false))
{
return await WaitForWorkflowCompletionAsync(req, client, context, resolvedInstanceId);
}
HttpResponseData response = req.CreateResponse(HttpStatusCode.Accepted);
await response.WriteStringAsync($"Workflow orchestration started for {workflowName}. Orchestration runId: {resolvedInstanceId}");
return response;
@@ -304,15 +311,7 @@ internal static class BuiltInFunctions
}
// Check if we should wait for response (default is true)
bool waitForResponse = true;
if (req.Headers.TryGetValues("x-ms-wait-for-response", out IEnumerable<string>? waitForResponseValues))
{
string? waitForResponseValue = waitForResponseValues.FirstOrDefault();
if (!string.IsNullOrEmpty(waitForResponseValue) && bool.TryParse(waitForResponseValue, out bool parsedValue))
{
waitForResponse = parsedValue;
}
}
bool waitForResponse = ShouldWaitForResponse(req, defaultValue: true);
AIAgent agentProxy = client.AsDurableAgentProxy(context, agentName);
@@ -428,6 +427,95 @@ internal static class BuiltInFunctions
return metadata.ReadOutputAs<DurableWorkflowResult>()?.Result;
}
/// <summary>
/// Waits for a workflow orchestration to complete and returns an appropriate HTTP response.
/// </summary>
private static async Task<HttpResponseData> WaitForWorkflowCompletionAsync(
HttpRequestData req,
DurableTaskClient client,
FunctionContext context,
string instanceId)
{
bool acceptsJson = AcceptsJson(req);
OrchestrationMetadata? metadata = await client.WaitForInstanceCompletionAsync(
instanceId,
getInputsAndOutputs: true,
cancellation: context.CancellationToken);
if (metadata is null)
{
return await CreateErrorResponseAsync(req, context, HttpStatusCode.NotFound,
$"No workflow orchestration with ID '{instanceId}' was found.", acceptsJson);
}
if (metadata.RuntimeStatus is OrchestrationRuntimeStatus.Failed)
{
string errorMessage = metadata.FailureDetails?.ErrorMessage ?? "Unknown error";
HttpResponseData failedResponse = req.CreateResponse(HttpStatusCode.OK);
if (acceptsJson)
{
await failedResponse.WriteAsJsonAsync(
new WorkflowRunResponse(instanceId, metadata.RuntimeStatus.ToString(), Result: null, Error: errorMessage),
context.CancellationToken);
}
else
{
failedResponse.Headers.Add("Content-Type", "text/plain");
await failedResponse.WriteStringAsync(errorMessage, context.CancellationToken);
}
return failedResponse;
}
if (metadata.RuntimeStatus is not OrchestrationRuntimeStatus.Completed)
{
return await CreateErrorResponseAsync(req, context, HttpStatusCode.InternalServerError,
$"Workflow orchestration '{instanceId}' ended with unexpected status '{metadata.RuntimeStatus}'.", acceptsJson);
}
string? result = metadata.ReadOutputAs<DurableWorkflowResult>()?.Result;
HttpResponseData response = req.CreateResponse(HttpStatusCode.OK);
if (acceptsJson)
{
JsonElement? resultElement = null;
if (!string.IsNullOrEmpty(result))
{
try
{
using JsonDocument doc = JsonDocument.Parse(result);
resultElement = doc.RootElement.Clone();
}
catch (JsonException)
{
// Result is a plain string (not valid JSON) — serialize it as a JSON string element.
var buffer = new System.Buffers.ArrayBufferWriter<byte>();
using (var writer = new Utf8JsonWriter(buffer))
{
writer.WriteStringValue(result);
}
using JsonDocument fallbackDoc = JsonDocument.Parse(buffer.WrittenMemory);
resultElement = fallbackDoc.RootElement.Clone();
}
}
await response.WriteAsJsonAsync(
new WorkflowRunResponse(instanceId, metadata.RuntimeStatus.ToString(), resultElement),
context.CancellationToken);
}
else
{
response.Headers.Add("Content-Type", "text/plain");
await response.WriteStringAsync(result ?? string.Empty, context.CancellationToken);
}
return response;
}
/// <summary>
/// Creates an error response with the specified status code and error message.
/// </summary>
@@ -435,18 +523,18 @@ internal static class BuiltInFunctions
/// <param name="context">The function context.</param>
/// <param name="statusCode">The HTTP status code.</param>
/// <param name="errorMessage">The error message.</param>
/// <param name="acceptsJson">Optional pre-computed value indicating whether the client accepts JSON. When <see langword="null"/>, the value is determined from the request's <c>Accept</c> header.</param>
/// <returns>The HTTP response data containing the error.</returns>
private static async Task<HttpResponseData> CreateErrorResponseAsync(
HttpRequestData req,
FunctionContext context,
HttpStatusCode statusCode,
string errorMessage)
string errorMessage,
bool? acceptsJson = null)
{
HttpResponseData response = req.CreateResponse(statusCode);
bool acceptsJson = req.Headers.TryGetValues("Accept", out IEnumerable<string>? acceptValues) &&
acceptValues.Contains("application/json", StringComparer.OrdinalIgnoreCase);
if (acceptsJson)
if (acceptsJson ?? AcceptsJson(req))
{
ErrorResponse errorResponse = new((int)statusCode, errorMessage);
await response.WriteAsJsonAsync(errorResponse, context.CancellationToken);
@@ -479,10 +567,7 @@ internal static class BuiltInFunctions
HttpResponseData response = req.CreateResponse(statusCode);
response.Headers.Add("x-ms-thread-id", sessionId);
bool acceptsJson = req.Headers.TryGetValues("Accept", out IEnumerable<string>? acceptValues) &&
acceptValues.Contains("application/json", StringComparer.OrdinalIgnoreCase);
if (acceptsJson)
if (AcceptsJson(req))
{
AgentRunSuccessResponse successResponse = new((int)statusCode, sessionId, agentResponse);
await response.WriteAsJsonAsync(successResponse, context.CancellationToken);
@@ -511,10 +596,7 @@ internal static class BuiltInFunctions
HttpResponseData response = req.CreateResponse(HttpStatusCode.Accepted);
response.Headers.Add("x-ms-thread-id", sessionId);
bool acceptsJson = req.Headers.TryGetValues("Accept", out IEnumerable<string>? acceptValues) &&
acceptValues.Contains("application/json", StringComparer.OrdinalIgnoreCase);
if (acceptsJson)
if (AcceptsJson(req))
{
AgentRunAcceptedResponse acceptedResponse = new((int)HttpStatusCode.Accepted, sessionId);
await response.WriteAsJsonAsync(acceptedResponse, context.CancellationToken);
@@ -528,6 +610,34 @@ internal static class BuiltInFunctions
return response;
}
/// <summary>
/// Returns <see langword="true"/> when the caller has requested waiting for the workflow/agent to complete,
/// as indicated by the <c>x-ms-wait-for-response</c> header. Falls back to <paramref name="defaultValue"/>
/// when the header is absent or not a valid boolean.
/// </summary>
private static bool ShouldWaitForResponse(HttpRequestData req, bool defaultValue)
{
if (req.Headers.TryGetValues(WaitForResponseHeaderName, out IEnumerable<string>? values) &&
bool.TryParse(values.FirstOrDefault(), out bool parsed))
{
return parsed;
}
return defaultValue;
}
/// <summary>
/// Returns <see langword="true"/> when the request accepts the <c>application/json</c> media type.
/// </summary>
private static bool AcceptsJson(HttpRequestData req)
{
return req.Headers.TryGetValues("Accept", out IEnumerable<string>? acceptValues) &&
acceptValues
.SelectMany(v => v.Split(',', StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries))
.Select(v => v.Split(';', 2)[0].Trim())
.Contains("application/json", StringComparer.OrdinalIgnoreCase);
}
private static string GetAgentName(FunctionContext context)
{
// Check if the function name starts with the HttpPrefix
@@ -591,6 +701,19 @@ internal static class BuiltInFunctions
[property: JsonPropertyName("eventName")] string? EventName,
[property: JsonPropertyName("response")] JsonElement Response);
/// <summary>
/// Represents a workflow run response when waiting for completion.
/// </summary>
/// <param name="RunId">The orchestration run ID.</param>
/// <param name="WorkflowStatus">The orchestration runtime status (e.g., "Completed", "Failed").</param>
/// <param name="Result">The workflow result as a JSON element so POCOs serialize as nested objects rather than escaped strings.</param>
/// <param name="Error">An optional error message when the workflow has failed.</param>
private sealed record WorkflowRunResponse(
[property: JsonPropertyName("runId")] string RunId,
[property: JsonPropertyName("workflowStatus")] string WorkflowStatus,
[property: JsonPropertyName("result")] JsonElement? Result,
[property: JsonPropertyName("error"), JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)] string? Error = null);
/// <summary>
/// A service provider that combines the original service provider with an additional DurableTaskClient instance.
/// </summary>
@@ -2,6 +2,7 @@
## [Unreleased]
- Support returning workflow results from HTTP trigger endpoint ([#5321](https://github.com/microsoft/agent-framework/pull/5321))
- Added MCP tool trigger support for durable workflows ([#4768](https://github.com/microsoft/agent-framework/pull/4768))
- Added Azure Functions hosting support for durable workflows ([#4436](https://github.com/microsoft/agent-framework/pull/4436))
@@ -26,6 +26,12 @@ public sealed class DeclarativeWorkflowOptions(ResponseAgentProvider agentProvid
/// </summary>
public IMcpToolHandler? McpToolHandler { get; init; }
/// <summary>
/// Gets or sets the HTTP request handler for executing <c>HttpRequestAction</c> actions within workflows.
/// If not set, HTTP request actions will fail with an appropriate error message.
/// </summary>
public IHttpRequestHandler? HttpRequestHandler { get; init; }
/// <summary>
/// Defines the configuration settings for the workflow.
/// </summary>
@@ -0,0 +1,289 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Net.Http;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
namespace Microsoft.Agents.AI.Workflows.Declarative;
/// <summary>
/// Default implementation of <see cref="IHttpRequestHandler"/> built on <see cref="HttpClient"/>.
/// </summary>
/// <remarks>
/// <para>
/// This handler supports per-request authentication via an optional <c>httpClientProvider</c> callback that
/// returns a pre-configured <see cref="HttpClient"/> for a given request (e.g. authenticated, custom handler).
/// When the provider returns <see langword="null"/>, or no provider is supplied, a shared internal <see cref="HttpClient"/>
/// is used.
/// </para>
/// <para>
/// The handler applies the per-request <see cref="HttpRequestInfo.Timeout"/> using a linked <see cref="CancellationTokenSource"/>
/// so it does not mutate <see cref="HttpClient.Timeout"/> on shared instances.
/// </para>
/// </remarks>
public sealed class DefaultHttpRequestHandler : IHttpRequestHandler, IAsyncDisposable
{
private readonly Func<HttpRequestInfo, CancellationToken, Task<HttpClient?>>? _httpClientProvider;
private readonly Lazy<HttpClient> _ownedHttpClient;
/// <summary>
/// Initializes a new instance of the <see cref="DefaultHttpRequestHandler"/> class that uses an
/// internally owned <see cref="HttpClient"/> for all requests. The internal client is disposed
/// when <see cref="DisposeAsync"/> is called.
/// </summary>
public DefaultHttpRequestHandler()
: this(httpClientProvider: null)
{
}
/// <summary>
/// Initializes a new instance of the <see cref="DefaultHttpRequestHandler"/> class that uses the
/// supplied <see cref="HttpClient"/> for all requests.
/// </summary>
/// <param name="httpClient">
/// The <see cref="HttpClient"/> to use for all requests. The caller retains ownership of this
/// instance; it is not disposed by <see cref="DisposeAsync"/>.
/// </param>
/// <exception cref="ArgumentNullException"><paramref name="httpClient"/> is <see langword="null"/>.</exception>
public DefaultHttpRequestHandler(HttpClient httpClient)
: this(CreateSingleClientProvider(httpClient))
{
}
/// <summary>
/// Initializes a new instance of the <see cref="DefaultHttpRequestHandler"/> class that selects
/// an <see cref="HttpClient"/> per request via a caller-supplied callback — for example, to route
/// different URLs through differently authenticated clients.
/// </summary>
/// <param name="httpClientProvider">
/// An optional callback invoked for each request. The callback receives the <see cref="HttpRequestInfo"/>
/// and should return a pre-configured <see cref="HttpClient"/> (e.g. with authentication or a custom
/// transport). Return <see langword="null"/> to fall back to the handler's shared internal
/// <see cref="HttpClient"/>.
/// </param>
/// <remarks>
/// <para>
/// <b>Ownership</b>: the caller is solely responsible for the lifetime of clients returned by this
/// callback. <see cref="DefaultHttpRequestHandler"/> will <b>not</b> dispose provider-returned
/// clients; only the handler's internally owned fallback client is disposed by <see cref="DisposeAsync"/>.
/// </para>
/// <para>
/// <b>Reuse</b>: callers are expected to cache and reuse clients (for example, keyed by base URL or
/// auth scope) across requests. Returning a newly allocated <see cref="HttpClient"/> on every
/// invocation will leak sockets and handler resources.
/// </para>
/// </remarks>
public DefaultHttpRequestHandler(Func<HttpRequestInfo, CancellationToken, Task<HttpClient?>>? httpClientProvider)
{
this._httpClientProvider = httpClientProvider;
this._ownedHttpClient = new Lazy<HttpClient>(() => new HttpClient(), LazyThreadSafetyMode.ExecutionAndPublication);
}
private static Func<HttpRequestInfo, CancellationToken, Task<HttpClient?>> CreateSingleClientProvider(HttpClient httpClient)
{
if (httpClient is null)
{
throw new ArgumentNullException(nameof(httpClient));
}
return (_, _) => Task.FromResult<HttpClient?>(httpClient);
}
/// <inheritdoc/>
public async Task<HttpRequestResult> SendAsync(HttpRequestInfo request, CancellationToken cancellationToken = default)
{
if (request is null)
{
throw new ArgumentNullException(nameof(request));
}
if (string.IsNullOrWhiteSpace(request.Url))
{
throw new ArgumentException("Request URL must be provided.", nameof(request));
}
if (string.IsNullOrWhiteSpace(request.Method))
{
throw new ArgumentException("Request method must be provided.", nameof(request));
}
HttpClient? providedClient = null;
if (this._httpClientProvider is not null)
{
providedClient = await this._httpClientProvider(request, cancellationToken).ConfigureAwait(false);
}
HttpClient client = providedClient ?? this._ownedHttpClient.Value;
using HttpRequestMessage httpRequest = BuildHttpRequestMessage(request);
using CancellationTokenSource? timeoutCts = request.Timeout is { } timeout && timeout > TimeSpan.Zero
? CancellationTokenSource.CreateLinkedTokenSource(cancellationToken)
: null;
timeoutCts?.CancelAfter(request.Timeout!.Value);
CancellationToken effectiveToken = timeoutCts?.Token ?? cancellationToken;
using HttpResponseMessage httpResponse = await client
.SendAsync(httpRequest, HttpCompletionOption.ResponseContentRead, effectiveToken)
.ConfigureAwait(false);
string? body = httpResponse.Content is null
? null
#if NET
: await httpResponse.Content.ReadAsStringAsync(effectiveToken).ConfigureAwait(false);
#else
: await httpResponse.Content.ReadAsStringAsync().ConfigureAwait(false);
#endif
Dictionary<string, IReadOnlyList<string>> headers = new(StringComparer.OrdinalIgnoreCase);
AppendHeaders(headers, httpResponse.Headers);
if (httpResponse.Content is not null)
{
AppendHeaders(headers, httpResponse.Content.Headers);
}
return new HttpRequestResult
{
StatusCode = (int)httpResponse.StatusCode,
IsSuccessStatusCode = httpResponse.IsSuccessStatusCode,
Body = body,
Headers = headers,
};
}
/// <inheritdoc/>
public ValueTask DisposeAsync()
{
if (this._ownedHttpClient.IsValueCreated)
{
this._ownedHttpClient.Value.Dispose();
}
return default;
}
private static HttpRequestMessage BuildHttpRequestMessage(HttpRequestInfo request)
{
HttpMethod method = ResolveMethod(request.Method);
string requestUri = ResolveRequestUri(request);
HttpRequestMessage httpRequest = new(method, requestUri);
if (request.Body is not null)
{
string contentType = string.IsNullOrWhiteSpace(request.BodyContentType)
? "text/plain"
: request.BodyContentType!;
httpRequest.Content = new StringContent(request.Body, Encoding.UTF8);
// Replace the default content-type header (including charset) with the declared type.
httpRequest.Content.Headers.Remove("Content-Type");
httpRequest.Content.Headers.TryAddWithoutValidation("Content-Type", contentType);
}
if (request.Headers is not null)
{
foreach (KeyValuePair<string, string> header in request.Headers)
{
if (string.IsNullOrEmpty(header.Key))
{
continue;
}
// Content-* headers belong on HttpContent; all others belong on the request.
if (header.Key.StartsWith("Content-", StringComparison.OrdinalIgnoreCase) && httpRequest.Content is not null)
{
httpRequest.Content.Headers.Remove(header.Key);
httpRequest.Content.Headers.TryAddWithoutValidation(header.Key, header.Value);
continue;
}
if (!httpRequest.Headers.TryAddWithoutValidation(header.Key, header.Value))
{
httpRequest.Content?.Headers.TryAddWithoutValidation(header.Key, header.Value);
}
}
}
return httpRequest;
}
private static HttpMethod ResolveMethod(string method)
{
string normalized = method.Trim().ToUpperInvariant();
return normalized switch
{
"GET" => HttpMethod.Get,
"POST" => HttpMethod.Post,
"PUT" => HttpMethod.Put,
"DELETE" => HttpMethod.Delete,
#if NET
"PATCH" => HttpMethod.Patch,
#else
"PATCH" => new HttpMethod("PATCH"),
#endif
_ => new HttpMethod(normalized),
};
}
private static string ResolveRequestUri(HttpRequestInfo request)
{
string baseUrl = request.Url;
if (request.QueryParameters is null || request.QueryParameters.Count == 0)
{
return baseUrl;
}
StringBuilder queryBuilder = new();
foreach (KeyValuePair<string, string> parameter in request.QueryParameters)
{
if (string.IsNullOrEmpty(parameter.Key))
{
continue;
}
if (queryBuilder.Length > 0)
{
queryBuilder.Append('&');
}
queryBuilder.Append(Uri.EscapeDataString(parameter.Key))
.Append('=')
.Append(Uri.EscapeDataString(parameter.Value ?? string.Empty));
}
if (queryBuilder.Length == 0)
{
return baseUrl;
}
char separator = baseUrl.Contains('?') ? '&' : '?';
return string.Concat(baseUrl, separator.ToString(), queryBuilder.ToString());
}
private static void AppendHeaders(
Dictionary<string, IReadOnlyList<string>> target,
System.Net.Http.Headers.HttpHeaders source)
{
foreach (KeyValuePair<string, IEnumerable<string>> header in source)
{
string[] values = header.Value.ToArray();
if (target.TryGetValue(header.Key, out IReadOnlyList<string>? existing))
{
List<string> combined = new(existing);
combined.AddRange(values);
target[header.Key] = combined;
}
else
{
target[header.Key] = values;
}
}
}
}
@@ -16,6 +16,60 @@ internal static class ChatMessageExtensions
public static RecordValue ToRecord(this ChatMessage message) =>
FormulaValue.NewRecordFromFields(message.GetMessageFields());
/// <summary>
/// Merges the user-authored <paramref name="input"/> with the round-tripped
/// <paramref name="inputMessage"/> returned by <c>AgentProvider.CreateMessageAsync</c>
/// to produce the value stored in <c>System.LastMessage</c>.
/// </summary>
/// <remarks>
/// The agent service often strips or alters <see cref="TextContent"/> on round-trip,
/// while replacing inline media (<see cref="DataContent"/>, <see cref="UriContent"/>)
/// with server-side references (typically <see cref="HostedFileContent"/>).
/// We want both: the original text (so <c>=System.LastMessage.Text</c> works) and
/// the server's media references (so subsequent actions don't re-upload large blobs).
/// <para>
/// Strategy: keep <paramref name="inputMessage"/> as the base — it has the server-generated
/// <see cref="ChatMessage.MessageId"/> and any provider-augmented metadata, and is forward-
/// compatible with new properties added on <see cref="ChatMessage"/> in the abstractions
/// layer. Only the <see cref="ChatMessage.Contents"/> list is mutated to substitute
/// original <see cref="TextContent"/> items in place (and append any extras the round-trip
/// dropped). Non-text content items returned by the service are left untouched so
/// server-side references survive.
/// </para>
/// </remarks>
public static ChatMessage MergeForLastMessage(this ChatMessage input, ChatMessage? inputMessage)
{
if (inputMessage is null)
{
return input;
}
// Build a queue of the original text items, in order. Fall back to ChatMessage.Text
// if the input has no explicit TextContent entries.
Queue<TextContent> originalTexts = new(input.Contents.OfType<TextContent>());
if (originalTexts.Count == 0 && !string.IsNullOrEmpty(input.Text))
{
originalTexts.Enqueue(new TextContent(input.Text));
}
// Replace TextContent items in inputMessage.Contents with the originals, in order.
for (int i = 0; i < inputMessage.Contents.Count && originalTexts.Count > 0; i++)
{
if (inputMessage.Contents[i] is TextContent)
{
inputMessage.Contents[i] = originalTexts.Dequeue();
}
}
// Append any remaining original text items that the round-trip dropped entirely.
while (originalTexts.Count > 0)
{
inputMessage.Contents.Add(originalTexts.Dequeue());
}
return inputMessage;
}
public static TableValue ToTable(this IEnumerable<ChatMessage> messages) =>
FormulaValue.NewTable(TypeSchema.Message.RecordType, messages.Select(message => message.ToRecord()));
@@ -0,0 +1,103 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
using System.Threading;
using System.Threading.Tasks;
namespace Microsoft.Agents.AI.Workflows.Declarative;
/// <summary>
/// Defines the contract for executing HTTP requests emitted by <c>HttpRequestAction</c> within declarative workflows.
/// </summary>
/// <remarks>
/// This interface allows the HTTP request dispatch to be abstracted, enabling different implementations
/// for local development, hosted workflows, authenticated scenarios, and testing.
/// </remarks>
public interface IHttpRequestHandler
{
/// <summary>
/// Sends an HTTP request and returns the response.
/// </summary>
/// <param name="request">The HTTP request to send.</param>
/// <param name="cancellationToken">A token to observe cancellation.</param>
/// <returns>The <see cref="HttpRequestResult"/> describing the HTTP response.</returns>
Task<HttpRequestResult> SendAsync(
HttpRequestInfo request,
CancellationToken cancellationToken = default);
}
/// <summary>
/// Describes an HTTP request to be sent by an <see cref="IHttpRequestHandler"/>.
/// </summary>
[SuppressMessage("Design", "CA1056:URI-like properties should not be strings", Justification = "URL is carried as a string to preserve the declarative expression result and to avoid forcing handler implementations to construct a Uri eagerly.")]
public sealed class HttpRequestInfo
{
/// <summary>
/// Gets the HTTP method to use (GET, POST, PUT, PATCH, DELETE).
/// </summary>
public string Method { get; init; } = "GET";
/// <summary>
/// Gets the absolute URL to send the request to.
/// </summary>
public string Url { get; init; } = string.Empty;
/// <summary>
/// Gets the headers to include on the request, excluding the <c>Content-Type</c> header (which is supplied via <see cref="BodyContentType"/>).
/// </summary>
public IReadOnlyDictionary<string, string>? Headers { get; init; }
/// <summary>
/// Gets the <c>Content-Type</c> of the request body, or <see langword="null"/> if no body is sent.
/// </summary>
public string? BodyContentType { get; init; }
/// <summary>
/// Gets the serialized request body, or <see langword="null"/> if no body is sent.
/// </summary>
public string? Body { get; init; }
/// <summary>
/// Gets the maximum amount of time to wait for the request to complete, or <see langword="null"/> to use the handler default.
/// </summary>
public TimeSpan? Timeout { get; init; }
/// <summary>
/// Gets the query parameters to append to the request URL, with values already formatted as strings.
/// </summary>
public IReadOnlyDictionary<string, string>? QueryParameters { get; init; }
/// <summary>
/// Gets the name of the declared remote connection, or <see langword="null"/> if no connection is declared.
/// This maps to the Foundry project connection Id and is only used when running in foundry service.
/// </summary>
public string? ConnectionName { get; init; }
}
/// <summary>
/// Represents the result of an HTTP request executed by an <see cref="IHttpRequestHandler"/>.
/// </summary>
public sealed class HttpRequestResult
{
/// <summary>
/// Gets the HTTP status code returned by the server.
/// </summary>
public int StatusCode { get; init; }
/// <summary>
/// Gets a value indicating whether the status code is in the range 200-299.
/// </summary>
public bool IsSuccessStatusCode { get; init; }
/// <summary>
/// Gets the response body, or <see langword="null"/> if no body was returned.
/// </summary>
public string? Body { get; init; }
/// <summary>
/// Gets the response headers keyed by header name. Each header may have multiple values.
/// </summary>
public IReadOnlyDictionary<string, IReadOnlyList<string>>? Headers { get; init; }
}
@@ -43,7 +43,11 @@ internal sealed class DeclarativeWorkflowExecutor<TInput>(
await declarativeContext.QueueConversationUpdateAsync(conversationId, isExternal: true, cancellationToken).ConfigureAwait(false);
ChatMessage inputMessage = await options.AgentProvider.CreateMessageAsync(conversationId, input, cancellationToken).ConfigureAwait(false);
await declarativeContext.SetLastMessageAsync(inputMessage).ConfigureAwait(false);
// Use the original input for System.LastMessage to ensure Text is preserved (the
// service may strip text on round-trip), but substitute server-side media references
// (e.g., HostedFileContent) so subsequent actions don't re-upload large blobs.
await declarativeContext.SetLastMessageAsync(input.MergeForLastMessage(inputMessage)).ConfigureAwait(false);
await context.SendResultMessageAsync(this.Id, cancellationToken).ConfigureAwait(false);
}
@@ -529,6 +529,18 @@ internal sealed class WorkflowActionVisitor : DialogActionVisitor
this._workflowModel.AddNode(new DelegateActionExecutor(postId, this._workflowState, action.CompleteAsync), action.ParentId);
}
protected override void Visit(HttpRequestAction item)
{
this.Trace(item);
if (this._workflowOptions.HttpRequestHandler is null)
{
throw new DeclarativeModelException("HTTP request handler not configured. Set HttpRequestHandler in DeclarativeWorkflowOptions to use HttpRequestAction actions.");
}
this.ContinueWith(new HttpRequestExecutor(item, this._workflowOptions.HttpRequestHandler, this._workflowOptions.AgentProvider, this._workflowState));
}
#region Not supported
protected override void Visit(AnswerQuestionWithAI item) => this.NotSupported(item);
@@ -573,8 +585,6 @@ internal sealed class WorkflowActionVisitor : DialogActionVisitor
protected override void Visit(GetConversationMembers item) => this.NotSupported(item);
protected override void Visit(HttpRequestAction item) => this.NotSupported(item);
protected override void Visit(RecognizeIntent item) => this.NotSupported(item);
protected override void Visit(TransferConversation item) => this.NotSupported(item);
@@ -58,7 +58,6 @@ public abstract class RootExecutor<TInput> : Executor<TInput>, IResettableExecut
public override async ValueTask HandleAsync(TInput message, IWorkflowContext context, CancellationToken cancellationToken)
{
DeclarativeWorkflowContext declarativeContext = new(context, this._state);
await this.ExecuteAsync(message, declarativeContext, cancellationToken).ConfigureAwait(false);
ChatMessage input = (this._inputTransform ?? DefaultInputTransform).Invoke(message);
@@ -69,7 +68,13 @@ public abstract class RootExecutor<TInput> : Executor<TInput>, IResettableExecut
await declarativeContext.QueueConversationUpdateAsync(this._conversationId, isExternal: true, cancellationToken).ConfigureAwait(false);
ChatMessage inputMessage = await this._agentProvider.CreateMessageAsync(this._conversationId, input, cancellationToken).ConfigureAwait(false);
await declarativeContext.SetLastMessageAsync(inputMessage).ConfigureAwait(false);
// Use the original input for System.LastMessage to ensure Text is preserved (the
// service may strip text on round-trip), but substitute server-side media references
// (e.g., HostedFileContent) so subsequent actions don't re-upload large blobs.
await declarativeContext.SetLastMessageAsync(input.MergeForLastMessage(inputMessage)).ConfigureAwait(false);
await this.ExecuteAsync(message, declarativeContext, cancellationToken).ConfigureAwait(false);
await declarativeContext.SendResultMessageAsync(this.Id, cancellationToken).ConfigureAwait(false);
}
@@ -0,0 +1,346 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Agents.AI.Workflows.Declarative.Extensions;
using Microsoft.Agents.AI.Workflows.Declarative.Interpreter;
using Microsoft.Agents.AI.Workflows.Declarative.Kit;
using Microsoft.Agents.AI.Workflows.Declarative.PowerFx;
using Microsoft.Agents.ObjectModel;
using Microsoft.Extensions.AI;
using Microsoft.PowerFx.Types;
using Microsoft.Shared.Diagnostics;
namespace Microsoft.Agents.AI.Workflows.Declarative.ObjectModel;
/// <summary>
/// Executor for the <see cref="HttpRequestAction"/> action.
/// Dispatches the request through the configured <see cref="IHttpRequestHandler"/> and assigns
/// the response body and headers to the declared property paths.
/// </summary>
internal sealed class HttpRequestExecutor(
HttpRequestAction model,
IHttpRequestHandler httpRequestHandler,
ResponseAgentProvider agentProvider,
WorkflowFormulaState state) :
DeclarativeActionExecutor<HttpRequestAction>(model, state)
{
/// <inheritdoc/>
protected override async ValueTask<object?> ExecuteAsync(IWorkflowContext context, CancellationToken cancellationToken = default)
{
string method = this.GetMethod();
string url = this.GetUrl();
Dictionary<string, string>? headers = this.GetHeaders();
Dictionary<string, string>? queryParameters = this.GetQueryParameters();
(string? body, string? contentType) = this.GetBody();
TimeSpan? timeout = this.GetTimeout();
string? conversationId = this.GetConversationId();
string? connectionName = this.GetConnectionName();
HttpRequestInfo requestInfo = new()
{
Method = method,
Url = url,
Headers = headers,
QueryParameters = queryParameters,
Body = body,
BodyContentType = contentType,
Timeout = timeout,
ConnectionName = connectionName,
};
HttpRequestResult result;
try
{
result = await httpRequestHandler.SendAsync(requestInfo, cancellationToken).ConfigureAwait(false);
}
catch (OperationCanceledException) when (!cancellationToken.IsCancellationRequested)
{
throw this.Exception($"HTTP request to '{url}' timed out.");
}
catch (Exception exception) when (exception is not DeclarativeActionException)
{
throw this.Exception($"HTTP request to '{url}' failed: {exception.Message}", exception);
}
if (result.IsSuccessStatusCode)
{
await this.AssignResponseAsync(context, result.Body).ConfigureAwait(false);
await this.AssignResponseHeadersAsync(context, result.Headers).ConfigureAwait(false);
await this.AddResponseToConversationAsync(conversationId, result.Body, cancellationToken).ConfigureAwait(false);
return default;
}
// Non-success status code - throw.
// Also publish response headers for diagnostic purposes.
await this.AssignResponseHeadersAsync(context, result.Headers).ConfigureAwait(false);
string bodyPreview = FormatBodyForDiagnostics(result.Body);
string message = bodyPreview.Length == 0
? $"HTTP request to '{url}' failed with status code {result.StatusCode}."
: $"HTTP request to '{url}' failed with status code {result.StatusCode}. Body: '{bodyPreview}'";
throw this.Exception(message);
}
// Response bodies can echo secrets (tokens, PII) and may be very large (multi-MB HTML error pages).
// Exception messages are often logged and persisted, so we clip the body to bound both exposure
// and message size. Full bodies are still available via the success path (assigned to Response).
private const int MaxBodyDiagnosticLength = 256;
private const string BodyTruncationSuffix = " \u2026 [truncated]";
private static string FormatBodyForDiagnostics(string? body)
{
if (string.IsNullOrEmpty(body))
{
return string.Empty;
}
int sourceLen = body!.Length;
bool truncated = sourceLen > MaxBodyDiagnosticLength;
int copyLen = truncated ? MaxBodyDiagnosticLength : sourceLen;
int finalLen = copyLen + (truncated ? BodyTruncationSuffix.Length : 0);
// Size the buffer for the final string so we only allocate once for the chars
// and once for the string itself. For a 10 KB error body we touch 256 chars instead of 10,000.
char[] buffer = new char[finalLen];
for (int i = 0; i < copyLen; i++)
{
char c = body[i];
buffer[i] = c is '\r' or '\n' or '\t' ? ' ' : c;
}
if (truncated)
{
BodyTruncationSuffix.CopyTo(0, buffer, copyLen, BodyTruncationSuffix.Length);
}
return new string(buffer);
}
private async ValueTask AddResponseToConversationAsync(string? conversationId, string? responseBody, CancellationToken cancellationToken)
{
if (conversationId is null || string.IsNullOrEmpty(responseBody))
{
return;
}
ChatMessage message = new(ChatRole.Assistant, responseBody);
await agentProvider.CreateMessageAsync(conversationId, message, cancellationToken).ConfigureAwait(false);
}
private async ValueTask AssignResponseAsync(IWorkflowContext context, string? responseBody)
{
if (this.Model.Response is not { Path: { } responsePath })
{
return;
}
await this.AssignAsync(responsePath, ParseResponseBody(responseBody), context).ConfigureAwait(false);
}
private async ValueTask AssignResponseHeadersAsync(IWorkflowContext context, IReadOnlyDictionary<string, IReadOnlyList<string>>? responseHeaders)
{
if (this.Model.ResponseHeaders is not { Path: { } headersPath })
{
return;
}
if (responseHeaders is null || responseHeaders.Count == 0)
{
await this.AssignAsync(headersPath, FormulaValue.NewBlank(), context).ConfigureAwait(false);
return;
}
// Flatten multi-value headers by joining with commas (standard HTTP header folding).
Dictionary<string, object?> flattened = new(StringComparer.OrdinalIgnoreCase);
foreach (KeyValuePair<string, IReadOnlyList<string>> header in responseHeaders)
{
flattened[header.Key] = string.Join(",", header.Value);
}
await this.AssignAsync(headersPath, flattened.ToFormula(), context).ConfigureAwait(false);
}
private static FormulaValue ParseResponseBody(string? responseBody)
{
if (string.IsNullOrEmpty(responseBody))
{
return FormulaValue.NewBlank();
}
// Attempt to parse as JSON so records/tables are exposed naturally to the workflow.
try
{
using JsonDocument jsonDocument = JsonDocument.Parse(responseBody);
object? parsedValue = jsonDocument.RootElement.ValueKind switch
{
JsonValueKind.Object => jsonDocument.ParseRecord(VariableType.RecordType),
JsonValueKind.Array => jsonDocument.ParseList(jsonDocument.RootElement.GetListTypeFromJson()),
JsonValueKind.String => jsonDocument.RootElement.GetString(),
JsonValueKind.Number => jsonDocument.RootElement.TryGetInt64(out long l)
? l
: jsonDocument.RootElement.GetDouble(),
JsonValueKind.True => true,
JsonValueKind.False => false,
JsonValueKind.Null => null,
_ => responseBody,
};
return parsedValue.ToFormula();
}
catch (JsonException)
{
// Not valid JSON — return the raw string.
return FormulaValue.New(responseBody);
}
}
private string GetMethod()
{
EnumExpression<HttpMethodTypeWrapper>? methodExpression = this.Model.Method;
if (methodExpression is null)
{
return "GET";
}
HttpMethodTypeWrapper wrapper = this.Evaluator.GetValue(methodExpression).Value;
return !string.IsNullOrEmpty(wrapper.UnknownValue) ? wrapper.UnknownValue! : wrapper.Value.ToString().ToUpperInvariant();
}
private string GetUrl() =>
this.Evaluator.GetValue(
Throw.IfNull(
this.Model.Url,
$"{nameof(this.Model)}.{nameof(this.Model.Url)}")).Value;
private Dictionary<string, string>? GetHeaders()
{
if (this.Model.Headers is null || this.Model.Headers.Count == 0)
{
return null;
}
Dictionary<string, string> result = new(StringComparer.OrdinalIgnoreCase);
foreach (KeyValuePair<string, StringExpression> header in this.Model.Headers)
{
string value = this.Evaluator.GetValue(header.Value).Value;
if (!string.IsNullOrEmpty(value))
{
result[header.Key] = value;
}
}
return result.Count == 0 ? null : result;
}
private (string? Body, string? ContentType) GetBody()
{
switch (this.Model.Body)
{
case null:
case NoRequestContent:
return (null, null);
case JsonRequestContent jsonContent when jsonContent.Content is not null:
{
FormulaValue formula = this.Evaluator.GetValue(jsonContent.Content).Value.ToFormula();
string json = formula.ToJson().ToJsonString();
return (json, "application/json");
}
case RawRequestContent rawContent:
{
string? content = rawContent.Content is null
? null
: this.Evaluator.GetValue(rawContent.Content).Value;
string? contentType = rawContent.ContentType is null
? null
: this.Evaluator.GetValue(rawContent.ContentType).Value;
return (content, string.IsNullOrEmpty(contentType) ? null : contentType);
}
default:
return (null, null);
}
}
private TimeSpan? GetTimeout()
{
if (this.Model.RequestTimeoutInMilliseconds is null || this.Model.RequestTimeoutInMillisecondsIsDefaultValue)
{
return null;
}
long value = this.Evaluator.GetValue(this.Model.RequestTimeoutInMilliseconds).Value;
return value > 0 ? TimeSpan.FromMilliseconds(value) : null;
}
private Dictionary<string, string>? GetQueryParameters()
{
if (this.Model.QueryParameters is null || this.Model.QueryParameters.Count == 0)
{
return null;
}
Dictionary<string, string> result = new(StringComparer.Ordinal);
foreach (KeyValuePair<string, ValueExpression> parameter in this.Model.QueryParameters)
{
if (string.IsNullOrEmpty(parameter.Key) || parameter.Value is null)
{
continue;
}
object? rawValue = this.Evaluator.GetValue(parameter.Value).Value.ToObject();
string? formatted = FormatQueryValue(rawValue);
if (formatted is not null)
{
result[parameter.Key] = formatted;
}
}
return result.Count == 0 ? null : result;
}
private static string? FormatQueryValue(object? value) =>
value switch
{
null => null,
string s => s,
bool b => b ? "true" : "false",
IFormattable formattable => formattable.ToString(null, System.Globalization.CultureInfo.InvariantCulture),
_ => value.ToString(),
};
private string? GetConversationId()
{
if (this.Model.ConversationId is null)
{
return null;
}
string value = this.Evaluator.GetValue(this.Model.ConversationId).Value;
return value.Length == 0 ? null : value;
}
private string? GetConnectionName()
{
RemoteConnection? connection = this.Model.Connection;
if (connection is null)
{
return null;
}
string? name = connection.Name is null
? null
: this.Evaluator.GetValue(connection.Name).Value;
return string.IsNullOrEmpty(name) ? null : name;
}
}
@@ -1,6 +1,20 @@
<?xml version="1.0" encoding="utf-8"?>
<!-- https://learn.microsoft.com/dotnet/fundamentals/package-validation/diagnostic-ids -->
<Suppressions xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentFileSkillScriptRunner.BeginInvoke(Microsoft.Agents.AI.AgentFileSkill,Microsoft.Agents.AI.AgentFileSkillScript,Microsoft.Extensions.AI.AIFunctionArguments,System.Threading.CancellationToken,System.AsyncCallback,System.Object)</Target>
<Left>lib/net10.0/Microsoft.Agents.AI.dll</Left>
<Right>lib/net10.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentFileSkillScriptRunner.Invoke(Microsoft.Agents.AI.AgentFileSkill,Microsoft.Agents.AI.AgentFileSkillScript,Microsoft.Extensions.AI.AIFunctionArguments,System.Threading.CancellationToken)</Target>
<Left>lib/net10.0/Microsoft.Agents.AI.dll</Left>
<Right>lib/net10.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentInlineSkill.#ctor(Microsoft.Agents.AI.AgentSkillFrontmatter,System.String)</Target>
@@ -29,6 +43,13 @@
<Right>lib/net10.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkillScript.RunAsync(Microsoft.Agents.AI.AgentSkill,Microsoft.Extensions.AI.AIFunctionArguments,System.Threading.CancellationToken)</Target>
<Left>lib/net10.0/Microsoft.Agents.AI.dll</Left>
<Right>lib/net10.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkillsProvider.#ctor(Microsoft.Agents.AI.AgentInlineSkill[])</Target>
@@ -43,6 +64,20 @@
<Right>lib/net10.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentFileSkillScriptRunner.BeginInvoke(Microsoft.Agents.AI.AgentFileSkill,Microsoft.Agents.AI.AgentFileSkillScript,Microsoft.Extensions.AI.AIFunctionArguments,System.Threading.CancellationToken,System.AsyncCallback,System.Object)</Target>
<Left>lib/net472/Microsoft.Agents.AI.dll</Left>
<Right>lib/net472/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentFileSkillScriptRunner.Invoke(Microsoft.Agents.AI.AgentFileSkill,Microsoft.Agents.AI.AgentFileSkillScript,Microsoft.Extensions.AI.AIFunctionArguments,System.Threading.CancellationToken)</Target>
<Left>lib/net472/Microsoft.Agents.AI.dll</Left>
<Right>lib/net472/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentInlineSkill.#ctor(Microsoft.Agents.AI.AgentSkillFrontmatter,System.String)</Target>
@@ -71,6 +106,13 @@
<Right>lib/net472/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkillScript.RunAsync(Microsoft.Agents.AI.AgentSkill,Microsoft.Extensions.AI.AIFunctionArguments,System.Threading.CancellationToken)</Target>
<Left>lib/net472/Microsoft.Agents.AI.dll</Left>
<Right>lib/net472/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkillsProvider.#ctor(Microsoft.Agents.AI.AgentInlineSkill[])</Target>
@@ -85,6 +127,20 @@
<Right>lib/net472/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentFileSkillScriptRunner.BeginInvoke(Microsoft.Agents.AI.AgentFileSkill,Microsoft.Agents.AI.AgentFileSkillScript,Microsoft.Extensions.AI.AIFunctionArguments,System.Threading.CancellationToken,System.AsyncCallback,System.Object)</Target>
<Left>lib/net8.0/Microsoft.Agents.AI.dll</Left>
<Right>lib/net8.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentFileSkillScriptRunner.Invoke(Microsoft.Agents.AI.AgentFileSkill,Microsoft.Agents.AI.AgentFileSkillScript,Microsoft.Extensions.AI.AIFunctionArguments,System.Threading.CancellationToken)</Target>
<Left>lib/net8.0/Microsoft.Agents.AI.dll</Left>
<Right>lib/net8.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentInlineSkill.#ctor(Microsoft.Agents.AI.AgentSkillFrontmatter,System.String)</Target>
@@ -113,6 +169,13 @@
<Right>lib/net8.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkillScript.RunAsync(Microsoft.Agents.AI.AgentSkill,Microsoft.Extensions.AI.AIFunctionArguments,System.Threading.CancellationToken)</Target>
<Left>lib/net8.0/Microsoft.Agents.AI.dll</Left>
<Right>lib/net8.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkillsProvider.#ctor(Microsoft.Agents.AI.AgentInlineSkill[])</Target>
@@ -127,6 +190,20 @@
<Right>lib/net8.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentFileSkillScriptRunner.BeginInvoke(Microsoft.Agents.AI.AgentFileSkill,Microsoft.Agents.AI.AgentFileSkillScript,Microsoft.Extensions.AI.AIFunctionArguments,System.Threading.CancellationToken,System.AsyncCallback,System.Object)</Target>
<Left>lib/net9.0/Microsoft.Agents.AI.dll</Left>
<Right>lib/net9.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentFileSkillScriptRunner.Invoke(Microsoft.Agents.AI.AgentFileSkill,Microsoft.Agents.AI.AgentFileSkillScript,Microsoft.Extensions.AI.AIFunctionArguments,System.Threading.CancellationToken)</Target>
<Left>lib/net9.0/Microsoft.Agents.AI.dll</Left>
<Right>lib/net9.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentInlineSkill.#ctor(Microsoft.Agents.AI.AgentSkillFrontmatter,System.String)</Target>
@@ -155,6 +232,13 @@
<Right>lib/net9.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkillScript.RunAsync(Microsoft.Agents.AI.AgentSkill,Microsoft.Extensions.AI.AIFunctionArguments,System.Threading.CancellationToken)</Target>
<Left>lib/net9.0/Microsoft.Agents.AI.dll</Left>
<Right>lib/net9.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkillsProvider.#ctor(Microsoft.Agents.AI.AgentInlineSkill[])</Target>
@@ -169,6 +253,20 @@
<Right>lib/net9.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentFileSkillScriptRunner.BeginInvoke(Microsoft.Agents.AI.AgentFileSkill,Microsoft.Agents.AI.AgentFileSkillScript,Microsoft.Extensions.AI.AIFunctionArguments,System.Threading.CancellationToken,System.AsyncCallback,System.Object)</Target>
<Left>lib/netstandard2.0/Microsoft.Agents.AI.dll</Left>
<Right>lib/netstandard2.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentFileSkillScriptRunner.Invoke(Microsoft.Agents.AI.AgentFileSkill,Microsoft.Agents.AI.AgentFileSkillScript,Microsoft.Extensions.AI.AIFunctionArguments,System.Threading.CancellationToken)</Target>
<Left>lib/netstandard2.0/Microsoft.Agents.AI.dll</Left>
<Right>lib/netstandard2.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentInlineSkill.#ctor(Microsoft.Agents.AI.AgentSkillFrontmatter,System.String)</Target>
@@ -197,6 +295,13 @@
<Right>lib/netstandard2.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkillScript.RunAsync(Microsoft.Agents.AI.AgentSkill,Microsoft.Extensions.AI.AIFunctionArguments,System.Threading.CancellationToken)</Target>
<Left>lib/netstandard2.0/Microsoft.Agents.AI.dll</Left>
<Right>lib/netstandard2.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkillsProvider.#ctor(Microsoft.Agents.AI.AgentInlineSkill[])</Target>
@@ -211,4 +316,39 @@
<Right>lib/netstandard2.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0005</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkillScript.RunAsync(Microsoft.Agents.AI.AgentSkill,System.Nullable{System.Text.Json.JsonElement},System.IServiceProvider,System.Threading.CancellationToken)</Target>
<Left>lib/net10.0/Microsoft.Agents.AI.dll</Left>
<Right>lib/net10.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0005</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkillScript.RunAsync(Microsoft.Agents.AI.AgentSkill,System.Nullable{System.Text.Json.JsonElement},System.IServiceProvider,System.Threading.CancellationToken)</Target>
<Left>lib/net472/Microsoft.Agents.AI.dll</Left>
<Right>lib/net472/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0005</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkillScript.RunAsync(Microsoft.Agents.AI.AgentSkill,System.Nullable{System.Text.Json.JsonElement},System.IServiceProvider,System.Threading.CancellationToken)</Target>
<Left>lib/net8.0/Microsoft.Agents.AI.dll</Left>
<Right>lib/net8.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0005</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkillScript.RunAsync(Microsoft.Agents.AI.AgentSkill,System.Nullable{System.Text.Json.JsonElement},System.IServiceProvider,System.Threading.CancellationToken)</Target>
<Left>lib/net9.0/Microsoft.Agents.AI.dll</Left>
<Right>lib/net9.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0005</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkillScript.RunAsync(Microsoft.Agents.AI.AgentSkill,System.Nullable{System.Text.Json.JsonElement},System.IServiceProvider,System.Threading.CancellationToken)</Target>
<Left>lib/netstandard2.0/Microsoft.Agents.AI.dll</Left>
<Right>lib/netstandard2.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
</Suppressions>
@@ -35,7 +35,8 @@ public abstract class AgentSkill
/// Gets the full skill content.
/// </summary>
/// <remarks>
/// For file-based skills this is the raw SKILL.md file content.
/// For file-based skills this is the raw SKILL.md file content, optionally
/// augmented with a synthesized scripts block when scripts are present.
/// For code-defined skills this is a synthesized XML document
/// containing name, description, and body (instructions, resources, scripts).
/// </remarks>
@@ -1,10 +1,10 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Diagnostics.CodeAnalysis;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.AI;
using Microsoft.Shared.DiagnosticIds;
using Microsoft.Shared.Diagnostics;
@@ -46,8 +46,9 @@ public abstract class AgentSkillScript
/// Runs the script with the given arguments.
/// </summary>
/// <param name="skill">The skill that owns this script.</param>
/// <param name="arguments">Arguments for script execution.</param>
/// <param name="arguments">Raw JSON arguments for script execution, preserving the original format (object or array) sent by the caller.</param>
/// <param name="serviceProvider">Optional service provider for dependency injection.</param>
/// <param name="cancellationToken">Cancellation token.</param>
/// <returns>The script execution result.</returns>
public abstract Task<object?> RunAsync(AgentSkill skill, AIFunctionArguments arguments, CancellationToken cancellationToken = default);
public abstract Task<object?> RunAsync(AgentSkill skill, JsonElement? arguments, IServiceProvider? serviceProvider, CancellationToken cancellationToken = default);
}
@@ -6,6 +6,7 @@ using System.Diagnostics.CodeAnalysis;
using System.Linq;
using System.Security;
using System.Text;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.AI;
@@ -243,7 +244,7 @@ public sealed partial class AgentSkillsProvider : AIContextProvider
}
AIFunction scriptFunction = AIFunctionFactory.Create(
(string skillName, string scriptName, IDictionary<string, object?>? arguments = null, IServiceProvider? serviceProvider = null, CancellationToken cancellationToken = default) =>
(string skillName, string scriptName, JsonElement? arguments = null, IServiceProvider? serviceProvider = null, CancellationToken cancellationToken = default) =>
this.RunSkillScriptAsync(skills, skillName, scriptName, arguments, serviceProvider, cancellationToken),
name: "run_skill_script",
description: "Runs a script associated with a skill.");
@@ -340,7 +341,7 @@ public sealed partial class AgentSkillsProvider : AIContextProvider
}
}
private async Task<object?> RunSkillScriptAsync(IList<AgentSkill> skills, string skillName, string scriptName, IDictionary<string, object?>? arguments = null, IServiceProvider? serviceProvider = null, CancellationToken cancellationToken = default)
private async Task<object?> RunSkillScriptAsync(IList<AgentSkill> skills, string skillName, string scriptName, JsonElement? arguments = null, IServiceProvider? serviceProvider = null, CancellationToken cancellationToken = default)
{
if (string.IsNullOrWhiteSpace(skillName))
{
@@ -366,7 +367,7 @@ public sealed partial class AgentSkillsProvider : AIContextProvider
try
{
return await script.RunAsync(skill, new AIFunctionArguments(arguments) { Services = serviceProvider }, cancellationToken).ConfigureAwait(false);
return await script.RunAsync(skill, arguments, serviceProvider, cancellationToken).ConfigureAwait(false);
}
catch (Exception ex)
{
@@ -15,6 +15,8 @@ public sealed class AgentFileSkill : AgentSkill
{
private readonly IReadOnlyList<AgentSkillResource> _resources;
private readonly IReadOnlyList<AgentSkillScript> _scripts;
private readonly string _originalContent;
private string? _content;
/// <summary>
/// Initializes a new instance of the <see cref="AgentFileSkill"/> class.
@@ -32,7 +34,7 @@ public sealed class AgentFileSkill : AgentSkill
IReadOnlyList<AgentSkillScript>? scripts = null)
{
this.Frontmatter = Throw.IfNull(frontmatter);
this.Content = Throw.IfNull(content);
this._originalContent = Throw.IfNull(content);
this.Path = Throw.IfNullOrWhitespace(path);
this._resources = resources ?? [];
this._scripts = scripts ?? [];
@@ -42,7 +44,18 @@ public sealed class AgentFileSkill : AgentSkill
public override AgentSkillFrontmatter Frontmatter { get; }
/// <inheritdoc/>
public override string Content { get; }
/// <remarks>
/// Returns the raw SKILL.md content. When the skill has scripts, a
/// <c>&lt;scripts&gt;&lt;script name="..."&gt;&lt;parameters_schema&gt;...&lt;/parameters_schema&gt;&lt;/script&gt;&lt;/scripts&gt;</c>
/// block is appended with a per-script entry describing the expected argument format.
/// The result is cached after the first access.
/// </remarks>
public override string Content
{
get => this._content ??= this._scripts is { Count: > 0 }
? this._originalContent + AgentInlineSkillContentBuilder.BuildScriptsBlock(this._scripts)
: this._originalContent;
}
/// <summary>
/// Gets the directory path where the skill was discovered.
@@ -2,9 +2,9 @@
using System;
using System.Diagnostics.CodeAnalysis;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.AI;
using Microsoft.Shared.DiagnosticIds;
using Microsoft.Shared.Diagnostics;
@@ -16,6 +16,11 @@ namespace Microsoft.Agents.AI;
[Experimental(DiagnosticIds.Experiments.AgentsAIExperiments)]
public sealed class AgentFileSkillScript : AgentSkillScript
{
/// <summary>
/// Cached JSON schema element describing the expected argument format: a string array of CLI arguments.
/// </summary>
private static readonly JsonElement s_defaultSchema = CreateDefaultSchema();
private readonly AgentFileSkillScriptRunner? _runner;
/// <summary>
@@ -37,7 +42,14 @@ public sealed class AgentFileSkillScript : AgentSkillScript
public string FullPath { get; }
/// <inheritdoc/>
public override async Task<object?> RunAsync(AgentSkill skill, AIFunctionArguments arguments, CancellationToken cancellationToken = default)
/// <remarks>
/// Returns a fixed schema describing a string array of CLI arguments:
/// <c>{"type":"array","items":{"type":"string"}}</c>.
/// </remarks>
public override JsonElement? ParametersSchema => s_defaultSchema;
/// <inheritdoc/>
public override async Task<object?> RunAsync(AgentSkill skill, JsonElement? arguments, IServiceProvider? serviceProvider, CancellationToken cancellationToken = default)
{
if (skill is not AgentFileSkill fileSkill)
{
@@ -51,6 +63,12 @@ public sealed class AgentFileSkillScript : AgentSkillScript
$"Supply a script runner when constructing {nameof(AgentFileSkillsSource)} to enable script execution.");
}
return await this._runner(fileSkill, this, arguments, cancellationToken).ConfigureAwait(false);
return await this._runner(fileSkill, this, arguments, serviceProvider, cancellationToken).ConfigureAwait(false);
}
private static JsonElement CreateDefaultSchema()
{
using JsonDocument document = JsonDocument.Parse("""{"type":"array","items":{"type":"string"}}""");
return document.RootElement.Clone();
}
}
@@ -1,9 +1,10 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Diagnostics.CodeAnalysis;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.AI;
using Microsoft.Shared.DiagnosticIds;
namespace Microsoft.Agents.AI;
@@ -13,15 +14,19 @@ namespace Microsoft.Agents.AI;
/// </summary>
/// <remarks>
/// Implementations determine the execution strategy (e.g., local subprocess, hosted code execution environment).
/// The <paramref name="arguments"/> parameter preserves the raw JSON sent by the caller, in the shape
/// described by <see cref="AgentFileSkillScript.ParametersSchema"/>.
/// </remarks>
/// <param name="skill">The skill that owns the script.</param>
/// <param name="script">The file-based script to run.</param>
/// <param name="arguments">Optional arguments for the script, provided by the agent/LLM.</param>
/// <param name="arguments">Raw JSON arguments for the script, in the shape described by <see cref="AgentFileSkillScript.ParametersSchema"/>.</param>
/// <param name="serviceProvider">Optional service provider for dependency injection.</param>
/// <param name="cancellationToken">Cancellation token.</param>
/// <returns>The script execution result.</returns>
[Experimental(DiagnosticIds.Experiments.AgentsAIExperiments)]
public delegate Task<object?> AgentFileSkillScriptRunner(
AgentFileSkill skill,
AgentFileSkillScript script,
AIFunctionArguments arguments,
JsonElement? arguments,
IServiceProvider? serviceProvider,
CancellationToken cancellationToken);
@@ -59,36 +59,60 @@ internal static class AgentInlineSkillContentBuilder
if (scripts is { Count: > 0 })
{
sb.Append("\n\n<scripts>\n");
foreach (var script in scripts)
{
var parametersSchema = script.ParametersSchema;
if (script.Description is null && parametersSchema is null)
{
sb.Append($" <script name=\"{EscapeXmlString(script.Name)}\"/>\n");
}
else
{
sb.Append(script.Description is not null
? $" <script name=\"{EscapeXmlString(script.Name)}\" description=\"{EscapeXmlString(script.Description)}\">\n"
: $" <script name=\"{EscapeXmlString(script.Name)}\">\n");
if (parametersSchema is not null)
{
sb.Append($" <parameters_schema>{EscapeXmlString(parametersSchema.Value.GetRawText(), preserveQuotes: true)}</parameters_schema>\n");
}
sb.Append(" </script>\n");
}
}
sb.Append("</scripts>");
sb.Append('\n');
sb.Append(BuildScriptsBlock(scripts));
}
return sb.ToString();
}
/// <summary>
/// Builds a <c>&lt;scripts&gt;...&lt;/scripts&gt;</c> XML block for the given scripts.
/// Each script is emitted as a <c>&lt;script name="..."&gt;</c> element with optional
/// <c>description</c> attribute and <c>&lt;parameters_schema&gt;</c> child element.
/// </summary>
/// <param name="scripts">The scripts to include in the block.</param>
/// <returns>An XML string starting with <c>\n&lt;scripts&gt;</c>, or an empty string if the list is empty.</returns>
public static string BuildScriptsBlock(IReadOnlyList<AgentSkillScript> scripts)
{
_ = Throw.IfNull(scripts);
if (scripts.Count == 0)
{
return string.Empty;
}
var sb = new StringBuilder();
sb.Append("\n<scripts>\n");
foreach (var script in scripts)
{
var parametersSchema = script.ParametersSchema;
if (script.Description is null && parametersSchema is null)
{
sb.Append($" <script name=\"{EscapeXmlString(script.Name)}\"/>\n");
}
else
{
sb.Append(script.Description is not null
? $" <script name=\"{EscapeXmlString(script.Name)}\" description=\"{EscapeXmlString(script.Description)}\">\n"
: $" <script name=\"{EscapeXmlString(script.Name)}\">\n");
if (parametersSchema is not null)
{
sb.Append($" <parameters_schema>{EscapeXmlString(parametersSchema.Value.GetRawText(), preserveQuotes: true)}</parameters_schema>\n");
}
sb.Append(" </script>\n");
}
}
sb.Append("</scripts>");
return sb.ToString();
}
/// <summary>
/// Escapes XML special characters: always escapes <c>&amp;</c>, <c>&lt;</c>, <c>&gt;</c>,
/// <c>&quot;</c>, and <c>&apos;</c>. When <paramref name="preserveQuotes"/> is <see langword="true"/>,
@@ -1,6 +1,7 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
using System.Reflection;
using System.Text.Json;
@@ -67,8 +68,42 @@ internal sealed class AgentInlineSkillScript : AgentSkillScript
public override JsonElement? ParametersSchema => this._function.JsonSchema;
/// <inheritdoc/>
public override async Task<object?> RunAsync(AgentSkill skill, AIFunctionArguments arguments, CancellationToken cancellationToken = default)
public override async Task<object?> RunAsync(AgentSkill skill, JsonElement? arguments, IServiceProvider? serviceProvider, CancellationToken cancellationToken = default)
{
return await this._function.InvokeAsync(arguments, cancellationToken).ConfigureAwait(false);
var funcArgs = ConvertToFunctionArguments(arguments);
funcArgs.Services = serviceProvider;
return await this._function.InvokeAsync(funcArgs, cancellationToken).ConfigureAwait(false);
}
/// <summary>
/// Converts a raw <see cref="JsonElement"/> to <see cref="AIFunctionArguments"/> for delegate invocation.
/// </summary>
/// <exception cref="InvalidOperationException">
/// Thrown when <paramref name="arguments"/> is provided but is not a JSON object.
/// Inline skill scripts expect arguments as a JSON object whose properties map to the delegate's parameters.
/// </exception>
private static AIFunctionArguments ConvertToFunctionArguments(JsonElement? arguments)
{
if (arguments is null ||
arguments.Value.ValueKind == JsonValueKind.Null ||
arguments.Value.ValueKind == JsonValueKind.Undefined)
{
return [];
}
if (arguments.Value.ValueKind != JsonValueKind.Object)
{
throw new InvalidOperationException(
$"Inline skill scripts expect arguments as a JSON object but received a JSON element of kind '{arguments.Value.ValueKind}'.");
}
var dict = new Dictionary<string, object?>();
foreach (var property in arguments.Value.EnumerateObject())
{
dict[property.Name] = property.Value;
}
return new AIFunctionArguments(dict);
}
}
@@ -25,6 +25,9 @@ internal sealed class WorkflowFactory(string workflowFile, Uri foundryEndpoint)
// Assign to provide MCP tool capabilities
public IMcpToolHandler? McpToolHandler { get; init; }
// Assign to enable HttpRequestAction support
public IHttpRequestHandler? HttpRequestHandler { get; init; }
/// <summary>
/// Create the workflow from the declarative YAML. Includes definition of the
/// <see cref="DeclarativeWorkflowOptions" /> and the associated <see cref="ResponseAgentProvider"/>.
@@ -46,6 +49,7 @@ internal sealed class WorkflowFactory(string workflowFile, Uri foundryEndpoint)
ConversationId = this.ConversationId,
LoggerFactory = this.LoggerFactory,
McpToolHandler = this.McpToolHandler,
HttpRequestHandler = this.HttpRequestHandler,
};
string workflowPath = Path.Combine(AppContext.BaseDirectory, workflowFile);
@@ -162,7 +162,10 @@ internal sealed class WorkflowRunner
case RequestInfoEvent requestInfo:
Debug.WriteLine($"REQUEST #{requestInfo.Request.RequestId}");
externalResponse = requestInfo.Request;
if (response is null || !string.Equals(requestInfo.Request.RequestId, response.RequestId, StringComparison.Ordinal))
{
externalResponse = requestInfo.Request;
}
break;
case ConversationUpdateEvent invokeEvent:
@@ -0,0 +1,453 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.ClientModel;
using System.ClientModel.Primitives;
using System.Collections.Generic;
using System.Net;
using System.Net.Http;
using System.Reflection;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using Azure.AI.Extensions.OpenAI;
using Microsoft.Agents.AI.Foundry.Hosting;
using Microsoft.Extensions.AI;
using OpenAI;
using OpenAI.Responses;
#pragma warning disable OPENAI001, SCME0001, SCME0002, MEAI001
namespace Microsoft.Agents.AI.Foundry.UnitTests.Hosting;
/// <summary>
/// Verifies that <see cref="DelegatingResponsesClient"/> preserves user-supplied client options
/// (Transport, RetryPolicy, UserAgentApplicationId, OrganizationId, ProjectId) and adds the
/// hosted-agent User-Agent supplement on every outgoing request, including streaming.
/// Covers both the Azure-flavored <see cref="ProjectResponsesClient"/> and the native OpenAI
/// <see cref="ResponsesClient"/>.
/// </summary>
public sealed partial class DelegatingResponsesClientTests
{
private const string TestEndpoint = "https://fake-foundry.example.com/api/projects/fake-prj";
private const string OpenAIEndpoint = "https://fake-openai.example.com/v1";
private const string Deployment = "fake-deployment";
[System.Text.RegularExpressions.GeneratedRegex("foundry-hosting/agent-framework-dotnet")]
private static partial System.Text.RegularExpressions.Regex SupplementRegex();
[Fact]
public async Task Polyfill_NonStreaming_PreservesAppId_ThroughCustomTransport_AddsSupplementAsync()
{
// Arrange
using var handler = new RecordingHandler(MinimalResponseJson());
#pragma warning disable CA5399
using var httpClient = new HttpClient(handler);
#pragma warning restore CA5399
var inner = BuildInner(httpClient, userAgentApplicationId: "MY_APP_ID");
var chat = MakeWithDelegating(inner);
// Act
_ = await chat.GetResponseAsync("hello");
// Assert
var req = Assert.Single(handler.Requests);
Assert.Contains("MY_APP_ID", req.UserAgent);
Assert.Contains("MEAI/", req.UserAgent);
Assert.Contains("foundry-hosting/agent-framework-dotnet", req.UserAgent);
Assert.StartsWith(TestEndpoint, req.Uri);
}
[Fact]
public async Task Polyfill_Streaming_PreservesAppId_ThroughCustomTransport_AddsSupplementAsync()
{
// Arrange
using var handler = new RecordingHandler(MinimalSseResponse());
#pragma warning disable CA5399
using var httpClient = new HttpClient(handler);
#pragma warning restore CA5399
var inner = BuildInner(httpClient, userAgentApplicationId: "MY_APP_ID");
var chat = MakeWithDelegating(inner);
// Act
await foreach (var _ in chat.GetStreamingResponseAsync("hello"))
{
}
// Assert
var req = Assert.Single(handler.Requests);
Assert.Contains("MY_APP_ID", req.UserAgent);
Assert.Contains("MEAI/", req.UserAgent);
Assert.Contains("foundry-hosting/agent-framework-dotnet", req.UserAgent);
Assert.StartsWith(TestEndpoint, req.Uri);
}
[Fact]
public async Task Polyfill_PreservesOrganizationAndProjectHeadersAsync()
{
// Arrange
using var handler = new RecordingHandler(MinimalResponseJson());
#pragma warning disable CA5399
using var httpClient = new HttpClient(handler);
#pragma warning restore CA5399
var inner = BuildInner(httpClient,
userAgentApplicationId: "MY_APP_ID",
organizationId: "org_xyz",
projectId: "proj_abc");
var chat = MakeWithDelegating(inner);
// Act
_ = await chat.GetResponseAsync("hello");
// Assert
var req = Assert.Single(handler.Requests);
Assert.Contains("MY_APP_ID", req.UserAgent);
Assert.Contains("foundry-hosting/agent-framework-dotnet", req.UserAgent);
}
[Fact]
public async Task Polyfill_HonorsUserSuppliedRetryPolicy_ByCountingRetriesAsync()
{
// Arrange
var retryPolicy = new CountingRetryPolicy(extraAttempts: 2);
using var handler = new RecordingHandler(MinimalResponseJson());
#pragma warning disable CA5399
using var httpClient = new HttpClient(handler);
#pragma warning restore CA5399
var inner = BuildInner(httpClient, userAgentApplicationId: "MY_APP_ID", retryPolicy: retryPolicy);
var chat = MakeWithDelegating(inner);
// Act
_ = await chat.GetResponseAsync("hello");
// Assert: retry policy ran (1 + 2 extras = 3 attempts).
Assert.Equal(3, handler.Requests.Count);
Assert.Equal(3, retryPolicy.InvocationCount);
foreach (var req in handler.Requests)
{
Assert.Contains("MY_APP_ID", req.UserAgent);
Assert.Contains("MEAI/", req.UserAgent);
Assert.Contains("foundry-hosting/agent-framework-dotnet", req.UserAgent);
}
}
[Fact]
public async Task Baseline_NonStreaming_DoesNotInjectSupplementAsync()
{
// Arrange
using var handler = new RecordingHandler(MinimalResponseJson());
#pragma warning disable CA5399
using var httpClient = new HttpClient(handler);
#pragma warning restore CA5399
var inner = BuildInner(httpClient, userAgentApplicationId: "MY_APP_ID");
var chat = inner.AsIChatClient(Deployment);
// Act
_ = await chat.GetResponseAsync("hello");
// Assert
var req = Assert.Single(handler.Requests);
Assert.Contains("MY_APP_ID", req.UserAgent);
Assert.Contains("MEAI/", req.UserAgent);
Assert.DoesNotContain("foundry-hosting/agent-framework-dotnet", req.UserAgent);
}
[Fact]
public async Task Polyfill_NativeOpenAIResponsesClient_NonStreaming_AddsSupplementAsync()
{
// Arrange: use the NATIVE OpenAI SDK ResponsesClient (no Foundry / Azure project involved).
using var handler = new RecordingHandler(MinimalResponseJson());
#pragma warning disable CA5399
using var httpClient = new HttpClient(handler);
#pragma warning restore CA5399
var inner = BuildOpenAIInner(httpClient, userAgentApplicationId: "MY_APP_ID");
var chat = MakeWithDelegating(inner);
// Act
_ = await chat.GetResponseAsync("hello");
// Assert
var req = Assert.Single(handler.Requests);
Assert.Contains("MY_APP_ID", req.UserAgent);
Assert.Contains("MEAI/", req.UserAgent);
Assert.Contains("foundry-hosting/agent-framework-dotnet", req.UserAgent);
Assert.StartsWith(OpenAIEndpoint, req.Uri);
}
[Fact]
public async Task Polyfill_NativeOpenAIResponsesClient_Streaming_AddsSupplementAsync()
{
// Arrange
using var handler = new RecordingHandler(MinimalSseResponse());
#pragma warning disable CA5399
using var httpClient = new HttpClient(handler);
#pragma warning restore CA5399
var inner = BuildOpenAIInner(httpClient, userAgentApplicationId: "MY_APP_ID");
var chat = MakeWithDelegating(inner);
// Act
await foreach (var _ in chat.GetStreamingResponseAsync("hello"))
{
}
// Assert
var req = Assert.Single(handler.Requests);
Assert.Contains("MY_APP_ID", req.UserAgent);
Assert.Contains("MEAI/", req.UserAgent);
Assert.Contains("foundry-hosting/agent-framework-dotnet", req.UserAgent);
Assert.StartsWith(OpenAIEndpoint, req.Uri);
}
[Theory]
[InlineData("DeleteResponseAsync")]
[InlineData("CancelResponseAsync")]
[InlineData("GetInputTokenCountAsync")]
[InlineData("CompactResponseAsync")]
[InlineData("GetResponseInputItemCollectionPageAsync")]
public async Task Polyfill_AncillaryProtocolMethod_AddsSupplementAsync(string method)
{
// Arrange: hit the wrapper DIRECTLY (no MEAI in the chain) to simulate user code that
// grabs the underlying ResponsesClient via chat.GetService<ResponsesClient>() and invokes
// a non-Create/Get protocol method. This is the regression path: without overriding these,
// the wrapper's dummy throwing pipeline would fire.
using var handler = new RecordingHandler(MinimalResponseJson());
#pragma warning disable CA5399
using var httpClient = new HttpClient(handler);
#pragma warning restore CA5399
var inner = BuildOpenAIInner(httpClient, userAgentApplicationId: "MY_APP_ID");
var wrapper = new DelegatingResponsesClient(inner);
// Act
switch (method)
{
case "DeleteResponseAsync":
_ = await wrapper.DeleteResponseAsync("resp_1", options: null!);
break;
case "CancelResponseAsync":
_ = await wrapper.CancelResponseAsync("resp_1", options: null!);
break;
case "GetInputTokenCountAsync":
_ = await wrapper.GetInputTokenCountAsync("application/json", BinaryContent.Create(BinaryData.FromString("{}")));
break;
case "CompactResponseAsync":
_ = await wrapper.CompactResponseAsync("application/json", BinaryContent.Create(BinaryData.FromString("{}")));
break;
case "GetResponseInputItemCollectionPageAsync":
_ = await wrapper.GetResponseInputItemCollectionPageAsync("resp_1", limit: null, order: "asc", after: "a", before: "b", options: null!);
break;
default:
Assert.Fail($"Unhandled method: {method}");
break;
}
// Assert
var req = Assert.Single(handler.Requests);
Assert.Contains("MY_APP_ID", req.UserAgent);
Assert.Contains("foundry-hosting/agent-framework-dotnet", req.UserAgent);
}
[Fact]
public async Task Polyfill_RetryWithinCall_DoesNotDuplicateSupplementInUserAgentAsync()
{
// Arrange: a custom retry policy that re-runs the inner pipeline on the SAME message,
// so the per-call HostedAgentUserAgentPolicy fires multiple times against the same headers.
// The policy's Contains-guard must prevent the supplement from appearing twice.
var retryPolicy = new CountingRetryPolicy(extraAttempts: 2);
using var handler = new RecordingHandler(MinimalResponseJson());
#pragma warning disable CA5399
using var httpClient = new HttpClient(handler);
#pragma warning restore CA5399
var inner = BuildInner(httpClient, userAgentApplicationId: "MY_APP_ID", retryPolicy: retryPolicy);
var chat = MakeWithDelegating(inner);
// Act
_ = await chat.GetResponseAsync("hello");
// Assert: each retry attempt must have exactly ONE foundry-hosting segment, never two.
Assert.Equal(3, handler.Requests.Count);
foreach (var req in handler.Requests)
{
int matches = SupplementRegex().Matches(req.UserAgent).Count;
Assert.True(matches == 1, $"Expected exactly one foundry-hosting segment per retry attempt, got {matches}. UA: {req.UserAgent}");
}
}
[Fact]
public async Task TryApplyUserAgent_CalledTwiceOnSameAgent_DoesNotDoubleWrapAsync()
{
// Arrange: build a real ChatClientAgent whose IChatClient resolves to MEAI's
// OpenAIResponsesChatClient → ProjectResponsesClient (with a fake transport).
using var handler = new RecordingHandler(MinimalResponseJson());
#pragma warning disable CA5399
using var httpClient = new HttpClient(handler);
#pragma warning restore CA5399
var inner = BuildInner(httpClient, userAgentApplicationId: "MY_APP_ID");
IChatClient chatClient = inner.AsIChatClient(Deployment);
AIAgent agent = new ChatClientAgent(chatClient);
// Act: apply twice.
FoundryHostingExtensions.TryApplyUserAgent(agent);
FoundryHostingExtensions.TryApplyUserAgent(agent);
// Assert: invoking the agent produces exactly ONE outbound request whose UA contains
// the supplement EXACTLY ONCE (would be twice if the wrapper were nested).
_ = await chatClient.GetResponseAsync("hello");
var req = Assert.Single(handler.Requests);
int matches = SupplementRegex().Matches(req.UserAgent).Count;
Assert.True(matches == 1, $"Expected exactly one foundry-hosting segment, got {matches}. UA: {req.UserAgent}");
}
[Fact]
public void OpenAIResponsesChatClient_ResponseClientField_ReflectionGuard()
{
// Guards the polyfill's reflection target. Failure here means MEAI internals
// changed and the polyfill needs updating.
var meaiType = typeof(MicrosoftExtensionsAIResponsesExtensions).Assembly
.GetType("Microsoft.Extensions.AI.OpenAIResponsesChatClient");
Assert.NotNull(meaiType);
var field = meaiType!.GetField("_responseClient", BindingFlags.NonPublic | BindingFlags.Instance);
Assert.NotNull(field);
Assert.True(typeof(ResponsesClient).IsAssignableFrom(field!.FieldType),
$"Expected _responseClient to be assignable to ResponsesClient but was {field.FieldType}.");
}
[Fact]
public void ResponsesClient_PipelineProperty_ReflectionGuard()
{
// The polyfill design assumes ResponsesClient.Pipeline remains accessible.
var pipelineProp = typeof(ResponsesClient).GetProperty("Pipeline", BindingFlags.Public | BindingFlags.Instance);
Assert.NotNull(pipelineProp);
Assert.Equal(typeof(ClientPipeline), pipelineProp!.PropertyType);
}
private static IChatClient MakeWithDelegating(ResponsesClient inner)
{
IChatClient meai = inner.AsIChatClient(Deployment);
var meaiType = meai.GetType();
var field = meaiType.GetField("_responseClient", BindingFlags.NonPublic | BindingFlags.Instance)!;
field.SetValue(meai, new DelegatingResponsesClient(inner));
return meai;
}
private static ProjectResponsesClient BuildInner(
HttpClient httpClient,
string? userAgentApplicationId = null,
string? organizationId = null,
string? projectId = null,
PipelinePolicy? retryPolicy = null)
{
var options = new ProjectResponsesClientOptions
{
Transport = new HttpClientPipelineTransport(httpClient),
};
if (userAgentApplicationId is not null)
{
options.UserAgentApplicationId = userAgentApplicationId;
}
if (organizationId is not null)
{
options.OrganizationId = organizationId;
}
if (projectId is not null)
{
options.ProjectId = projectId;
}
if (retryPolicy is not null)
{
options.RetryPolicy = retryPolicy;
}
return new ProjectResponsesClient(new Uri(TestEndpoint), new FakeAuthenticationTokenProvider(), options);
}
private static ResponsesClient BuildOpenAIInner(
HttpClient httpClient,
string? userAgentApplicationId = null)
{
var options = new OpenAIClientOptions
{
Transport = new HttpClientPipelineTransport(httpClient),
Endpoint = new Uri(OpenAIEndpoint),
};
if (userAgentApplicationId is not null)
{
options.UserAgentApplicationId = userAgentApplicationId;
}
return new ResponsesClient(new ApiKeyCredential("test-key"), options);
}
private static string MinimalResponseJson() => """
{
"id":"resp_1","object":"response","created_at":1700000000,"status":"completed",
"model":"fake","output":[],"usage":{"input_tokens":1,"output_tokens":1,"total_tokens":2}
}
""";
private static string MinimalSseResponse()
{
var sb = new StringBuilder();
sb.Append("event: response.completed\n");
sb.Append("data: ").Append("""{"type":"response.completed","response":{"id":"resp_1","object":"response","created_at":1700000000,"status":"completed","model":"fake","output":[],"usage":{"input_tokens":1,"output_tokens":1,"total_tokens":2}}}""").Append("\n\n");
sb.Append("data: [DONE]\n\n");
return sb.ToString();
}
private sealed class RecordingHandler : HttpClientHandler
{
private readonly string _body;
public List<RecordedRequest> Requests { get; } = [];
public RecordingHandler(string body)
{
this._body = body;
}
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
string ua = request.Headers.TryGetValues("User-Agent", out var values)
? string.Join(",", values)
: "(none)";
this.Requests.Add(new RecordedRequest(request.Method.Method, request.RequestUri?.ToString() ?? "?", ua));
var resp = new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new StringContent(this._body, Encoding.UTF8, "application/json"),
RequestMessage = request,
};
return Task.FromResult(resp);
}
}
private readonly record struct RecordedRequest(string Method, string Uri, string UserAgent);
private sealed class CountingRetryPolicy : PipelinePolicy
{
private readonly int _extraAttempts;
public int InvocationCount { get; private set; }
public CountingRetryPolicy(int extraAttempts)
{
this._extraAttempts = extraAttempts;
}
public override void Process(PipelineMessage message, IReadOnlyList<PipelinePolicy> pipeline, int currentIndex)
{
for (int i = 0; i <= this._extraAttempts; i++)
{
this.InvocationCount++;
ProcessNext(message, pipeline, currentIndex);
}
}
public override async ValueTask ProcessAsync(PipelineMessage message, IReadOnlyList<PipelinePolicy> pipeline, int currentIndex)
{
for (int i = 0; i <= this._extraAttempts; i++)
{
this.InvocationCount++;
await ProcessNextAsync(message, pipeline, currentIndex).ConfigureAwait(false);
}
}
}
}
@@ -0,0 +1,165 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.ClientModel.Primitives;
using System.Collections.Generic;
using System.Net;
using System.Net.Http;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using Azure.AI.Extensions.OpenAI;
using Microsoft.Agents.AI.Foundry.Hosting;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting.Server;
using Microsoft.AspNetCore.TestHost;
using Microsoft.Extensions.AI;
using Microsoft.Extensions.DependencyInjection;
#pragma warning disable OPENAI001, SCME0001, SCME0002, MEAI001
namespace Microsoft.Agents.AI.Foundry.UnitTests.Hosting;
/// <summary>
/// End-to-end tests that exercise the FULL hosted ASP.NET Core pipeline:
/// inbound HTTP → MapFoundryResponses → AgentFrameworkResponseHandler → TryApplyUserAgent →
/// agent invocation → outbound HTTP from inside the hosted environment.
/// Verifies that the hosted-agent <c>User-Agent</c> supplement reaches the outbound wire,
/// not just the inbound request.
/// </summary>
public sealed class HostedOutboundUserAgentTests : IAsyncDisposable
{
private const string TestEndpoint = "https://fake-foundry.example.com/api/projects/fake-prj";
private const string Deployment = "fake-deployment";
private WebApplication? _app;
private HttpClient? _inboundClient;
private RecordingHandler? _outboundHandler;
public async ValueTask DisposeAsync()
{
this._inboundClient?.Dispose();
this._outboundHandler?.Dispose();
if (this._app is not null)
{
await this._app.DisposeAsync();
}
}
[Fact]
public async Task Hosted_InboundResponsesRequest_TriggersOutboundCall_WithFoundryHostingSupplementAsync()
{
// Arrange: spin up a real ASP.NET Core TestServer that hosts an AIAgent backed by MEAI's
// OpenAIResponsesChatClient → ProjectResponsesClient → fake HTTP transport. This is the
// exact production stack minus the network: the only thing not real is the wire transport.
await this.StartHostedServerAsync();
// Act: send an inbound /openai/v1/responses request as the Foundry runtime would.
using var inboundRequest = new HttpRequestMessage(HttpMethod.Post, "/responses")
{
Content = new StringContent(InboundResponsesRequestJson(), Encoding.UTF8, "application/json"),
};
using var inboundResponse = await this._inboundClient!.SendAsync(inboundRequest);
var inboundBody = await inboundResponse.Content.ReadAsStringAsync();
// Assert: at least one OUTBOUND request reached the fake transport, AND it carries the
// foundry-hosting/agent-framework-dotnet/{version} supplement on its User-Agent.
// (We don't care about the inbound response shape — only that the agent's call to MEAI
// triggered an outbound request whose UA reaches the sandbox boundary correctly.)
Assert.True(this._outboundHandler!.Requests.Count > 0,
$"Expected at least one outbound request. Inbound status: {(int)inboundResponse.StatusCode}, body: {inboundBody}");
var outbound = this._outboundHandler.Requests[0];
Assert.StartsWith(TestEndpoint, outbound.Uri);
Assert.Contains("MEAI/", outbound.UserAgent);
Assert.Contains("foundry-hosting/agent-framework-dotnet", outbound.UserAgent);
}
private async Task StartHostedServerAsync()
{
var builder = WebApplication.CreateBuilder();
builder.WebHost.UseTestServer();
// Build a real ChatClientAgent whose IChatClient is MEAI's OpenAIResponsesChatClient
// wrapping a ProjectResponsesClient backed by a fake HTTP handler. After AgentFrameworkResponseHandler
// resolves this agent, TryApplyUserAgent will swap the inner _responseClient with our wrapper.
this._outboundHandler = new RecordingHandler(MinimalResponseJson());
#pragma warning disable CA5399
var outboundHttpClient = new HttpClient(this._outboundHandler);
#pragma warning restore CA5399
var projectOptions = new ProjectResponsesClientOptions
{
Transport = new HttpClientPipelineTransport(outboundHttpClient),
};
var projectResponsesClient = new ProjectResponsesClient(
new Uri(TestEndpoint),
new FakeAuthenticationTokenProvider(),
projectOptions);
IChatClient chatClient = projectResponsesClient.AsIChatClient(Deployment);
AIAgent agent = new ChatClientAgent(chatClient);
builder.Services.AddFoundryResponses(agent);
builder.Services.AddLogging();
this._app = builder.Build();
this._app.MapFoundryResponses();
await this._app.StartAsync();
var testServer = this._app.Services.GetRequiredService<IServer>() as TestServer
?? throw new InvalidOperationException("TestServer not found");
this._inboundClient = testServer.CreateClient();
}
private static string InboundResponsesRequestJson() => """
{
"model": "fake-deployment",
"input": [
{
"type": "message",
"id": "msg_1",
"status": "completed",
"role": "user",
"content": [{ "type": "input_text", "text": "Hello" }]
}
]
}
""";
private static string MinimalResponseJson() => """
{
"id":"resp_1","object":"response","created_at":1700000000,"status":"completed",
"model":"fake","output":[],"usage":{"input_tokens":1,"output_tokens":1,"total_tokens":2}
}
""";
private sealed class RecordingHandler : HttpClientHandler
{
private readonly string _body;
public List<RecordedRequest> Requests { get; } = [];
public RecordingHandler(string body)
{
this._body = body;
}
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
string ua = request.Headers.TryGetValues("User-Agent", out var values)
? string.Join(",", values)
: "(none)";
this.Requests.Add(new RecordedRequest(request.RequestUri?.ToString() ?? "?", ua));
var resp = new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new StringContent(this._body, Encoding.UTF8, "application/json"),
RequestMessage = request,
};
return Task.FromResult(resp);
}
}
private readonly record struct RecordedRequest(string Uri, string UserAgent);
}
@@ -4,8 +4,10 @@ using System;
using System.Linq;
using Azure.AI.AgentServer.Responses;
using Microsoft.Agents.AI.Foundry.Hosting;
using Microsoft.Extensions.AI;
using Microsoft.Extensions.DependencyInjection;
using Moq;
using OpenAI.Responses;
namespace Microsoft.Agents.AI.Foundry.UnitTests.Hosting;
@@ -93,4 +95,45 @@ public class ServiceCollectionExtensionsTests
Assert.Same(instrumented, result);
}
[Fact]
public void TryApplyUserAgent_AgentWithoutChatClient_NoOp()
{
// Arrange: agent.GetService<IChatClient>() returns null.
var mockAgent = new Mock<AIAgent>();
// Act
var result = FoundryHostingExtensions.TryApplyUserAgent(mockAgent.Object);
// Assert
Assert.Same(mockAgent.Object, result);
}
[Fact]
public void TryApplyUserAgent_AgentWithNonMeaiChatClient_NoOp()
{
// Arrange: chat client that does not return MEAI's OpenAIResponsesChatClient via GetService.
var mockChatClient = new Mock<IChatClient>();
mockChatClient.Setup(c => c.GetService(It.IsAny<Type>(), It.IsAny<object?>())).Returns(null!);
var mockAgent = new Mock<AIAgent>();
mockAgent.Setup(a => a.GetService(typeof(IChatClient), It.IsAny<object?>())).Returns(mockChatClient.Object);
// Act
var result = FoundryHostingExtensions.TryApplyUserAgent(mockAgent.Object);
// Assert
Assert.Same(mockAgent.Object, result);
}
[Fact]
public void MeaiOpenAIResponsesChatClient_TypeFullName_ReflectionGuard()
{
// Guards the polyfill's reflection target type-name.
var meaiType = typeof(MicrosoftExtensionsAIResponsesExtensions).Assembly
.GetType("Microsoft.Extensions.AI.OpenAIResponsesChatClient");
Assert.NotNull(meaiType);
Assert.True(typeof(IChatClient).IsAssignableFrom(meaiType!),
$"Expected MEAI {meaiType!.FullName} to implement IChatClient.");
}
}
@@ -1,134 +0,0 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Net.Http;
using System.Text.RegularExpressions;
using System.Threading.Tasks;
using Microsoft.Agents.AI.Foundry.Hosting;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting.Server;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.TestHost;
using Microsoft.Extensions.DependencyInjection;
using Moq;
namespace Microsoft.Agents.AI.Foundry.UnitTests.Hosting;
/// <summary>
/// Tests for the <c>AgentFrameworkUserAgentMiddleware</c> registered by
/// <see cref="FoundryHostingExtensions.MapFoundryResponses"/>.
/// </summary>
public sealed partial class UserAgentMiddlewareTests : IAsyncDisposable
{
private const string VersionedUserAgentPattern = @"agent-framework-dotnet/\d+\.\d+\.\d+(-[\w.]+)?";
private WebApplication? _app;
private HttpClient? _httpClient;
public async ValueTask DisposeAsync()
{
this._httpClient?.Dispose();
if (this._app != null)
{
await this._app.DisposeAsync();
}
}
[Fact]
public async Task MapFoundryResponses_NoUserAgentHeader_SetsAgentFrameworkUserAgentAsync()
{
// Arrange
await this.CreateTestServerAsync();
using var request = new HttpRequestMessage(HttpMethod.Get, "/test-ua");
// Act
var response = await this._httpClient!.SendAsync(request);
var userAgent = await response.Content.ReadAsStringAsync();
// Assert
Assert.Matches(VersionedUserAgentPattern, userAgent);
}
[Fact]
public async Task MapFoundryResponses_WithExistingUserAgent_AppendsAgentFrameworkUserAgentAsync()
{
// Arrange
await this.CreateTestServerAsync();
using var request = new HttpRequestMessage(HttpMethod.Get, "/test-ua");
request.Headers.TryAddWithoutValidation("User-Agent", "MyApp/1.0");
// Act
var response = await this._httpClient!.SendAsync(request);
var userAgent = await response.Content.ReadAsStringAsync();
// Assert
Assert.StartsWith("MyApp/1.0", userAgent);
Assert.Matches(VersionedUserAgentPattern, userAgent);
}
[Fact]
public async Task MapFoundryResponses_AlreadyContainsUserAgent_DoesNotDuplicateAsync()
{
// Arrange
await this.CreateTestServerAsync();
// First request to capture the actual middleware-generated value
using var firstRequest = new HttpRequestMessage(HttpMethod.Get, "/test-ua");
var firstResponse = await this._httpClient!.SendAsync(firstRequest);
var middlewareValue = await firstResponse.Content.ReadAsStringAsync();
// Act: send a second request that already contains the middleware value
using var secondRequest = new HttpRequestMessage(HttpMethod.Get, "/test-ua");
secondRequest.Headers.TryAddWithoutValidation("User-Agent", $"MyApp/2.0 {middlewareValue}");
var secondResponse = await this._httpClient!.SendAsync(secondRequest);
var userAgent = await secondResponse.Content.ReadAsStringAsync();
// Assert: should remain unchanged (no duplication)
Assert.Equal($"MyApp/2.0 {middlewareValue}", userAgent);
Assert.Single(VersionedUserAgentRegex().Matches(userAgent));
}
[Fact]
public async Task MapFoundryResponses_UserAgentValue_ContainsVersionAsync()
{
// Arrange
await this.CreateTestServerAsync();
using var request = new HttpRequestMessage(HttpMethod.Get, "/test-ua");
// Act
var response = await this._httpClient!.SendAsync(request);
var userAgent = await response.Content.ReadAsStringAsync();
// Assert: should match "agent-framework-dotnet/x.y.z" pattern
Assert.Matches(VersionedUserAgentPattern, userAgent);
}
private async Task CreateTestServerAsync()
{
var builder = WebApplication.CreateBuilder();
builder.WebHost.UseTestServer();
var mockAgent = new Mock<AIAgent>();
builder.Services.AddFoundryResponses(mockAgent.Object);
this._app = builder.Build();
this._app.MapFoundryResponses();
// Test endpoint that echoes the User-Agent header after middleware processing
this._app.MapGet("/test-ua", (HttpContext ctx) =>
Results.Text(ctx.Request.Headers.UserAgent.ToString()));
await this._app.StartAsync();
var testServer = this._app.Services.GetRequiredService<IServer>() as TestServer
?? throw new InvalidOperationException("TestServer not found");
this._httpClient = testServer.CreateClient();
}
[GeneratedRegex(VersionedUserAgentPattern)]
private static partial Regex VersionedUserAgentRegex();
}
@@ -0,0 +1,115 @@
// Copyright (c) Microsoft. All rights reserved.
using System.ClientModel.Primitives;
using System.Net;
using System.Net.Http;
using System.Reflection;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
namespace Microsoft.Agents.AI.Foundry.UnitTests;
/// <summary>
/// Verifies the per-call <c>MeaiUserAgentPolicy</c> exposed via
/// <see cref="RequestOptionsExtensions.UserAgentPolicy"/>. The policy is reachable through the
/// public <see cref="FoundryAgent"/> constructors (which add it to the internally-built
/// <see cref="Azure.AI.Projects.AIProjectClient"/>'s pipeline), so its behavior is part of the
/// public API surface.
/// </summary>
public sealed class RequestOptionsExtensionsTests
{
[Fact]
public async Task MeaiUserAgentPolicy_AddsMeaiSegment_ToOutgoingRequestAsync()
{
// Arrange
using var handler = new RecordingHandler();
#pragma warning disable CA5399
using var httpClient = new HttpClient(handler);
#pragma warning restore CA5399
var pipeline = ClientPipeline.Create(
new ClientPipelineOptions { Transport = new HttpClientPipelineTransport(httpClient) },
perCallPolicies: [RequestOptionsExtensions.UserAgentPolicy],
perTryPolicies: default,
beforeTransportPolicies: default);
// Act
var message = pipeline.CreateMessage();
message.Request.Method = "POST";
message.Request.Uri = new System.Uri("https://example.test/anything");
await pipeline.SendAsync(message);
// Assert
Assert.Equal(1, handler.Count);
Assert.NotNull(handler.LastUserAgent);
Assert.Contains("MEAI/", handler.LastUserAgent);
}
[Fact]
public async Task MeaiUserAgentPolicy_DoesNotAddFoundryHostingSegmentAsync()
{
// Arrange
using var handler = new RecordingHandler();
#pragma warning disable CA5399
using var httpClient = new HttpClient(handler);
#pragma warning restore CA5399
var pipeline = ClientPipeline.Create(
new ClientPipelineOptions { Transport = new HttpClientPipelineTransport(httpClient) },
perCallPolicies: [RequestOptionsExtensions.UserAgentPolicy],
perTryPolicies: default,
beforeTransportPolicies: default);
// Act
var message = pipeline.CreateMessage();
message.Request.Method = "POST";
message.Request.Uri = new System.Uri("https://example.test/anything");
await pipeline.SendAsync(message);
// Assert: the policy is MEAI-only; the foundry-hosting supplement is added elsewhere
// (by the polyfill DelegatingResponsesClient → HostedAgentUserAgentPolicy).
Assert.NotNull(handler.LastUserAgent);
Assert.DoesNotContain("foundry-hosting/agent-framework-dotnet", handler.LastUserAgent);
}
[Fact]
public void UserAgentPolicy_ExposesSingletonInstance()
{
// Two reads of the static property must return the same instance — the policy is stateless and shared.
var first = RequestOptionsExtensions.UserAgentPolicy;
var second = RequestOptionsExtensions.UserAgentPolicy;
Assert.Same(first, second);
}
[Fact]
public void MeaiUserAgentPolicy_ValueIncludesAFFoundryAssemblyVersion_ReflectionGuard()
{
// The policy emits "MEAI/{Microsoft.Agents.AI.Foundry assembly InformationalVersion}".
// If the assembly metadata stops being readable, the policy falls back to "MEAI" without a version,
// which is a measurable telemetry regression.
var attr = typeof(RequestOptionsExtensions).Assembly
.GetCustomAttribute<AssemblyInformationalVersionAttribute>();
Assert.NotNull(attr);
Assert.False(string.IsNullOrEmpty(attr!.InformationalVersion));
}
private sealed class RecordingHandler : HttpClientHandler
{
public int Count { get; private set; }
public string? LastUserAgent { get; private set; }
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
this.Count++;
this.LastUserAgent = request.Headers.TryGetValues("User-Agent", out var values)
? string.Join(",", values)
: null;
var resp = new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new StringContent("{}", Encoding.UTF8, "application/json"),
RequestMessage = request,
};
return Task.FromResult(resp);
}
}
}
@@ -3,6 +3,7 @@
using System.Diagnostics;
using System.Reflection;
using System.Text;
using System.Text.Json;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Logging;
using ModelContextProtocol.Client;
@@ -125,6 +126,45 @@ public sealed class WorkflowSamplesValidation(ITestOutputHelper outputHelper) :
},
message: "OrderStatus workflow completed",
timeout: s_orchestrationTimeout);
// Test the CancelOrder workflow with x-ms-wait-for-response header
this._outputHelper.WriteLine("Starting CancelOrder workflow with x-ms-wait-for-response: true...");
using HttpRequestMessage waitRequest = new(HttpMethod.Post, cancelOrderUri);
waitRequest.Content = new StringContent("55555", Encoding.UTF8, "text/plain");
waitRequest.Headers.Add("x-ms-wait-for-response", "true");
using HttpResponseMessage waitResponse = await s_sharedHttpClient.SendAsync(waitRequest);
Assert.True(waitResponse.IsSuccessStatusCode, $"CancelOrder wait-for-response request failed with status: {waitResponse.StatusCode}");
string waitResponseText = await waitResponse.Content.ReadAsStringAsync();
this._outputHelper.WriteLine($"CancelOrder wait-for-response result: {waitResponseText}");
// The response should contain the workflow result (not just "started for CancelOrder")
Assert.DoesNotContain("Workflow orchestration started", waitResponseText);
Assert.Contains("55555", waitResponseText);
// Test the wait-for-response with Accept: application/json header
this._outputHelper.WriteLine("Starting CancelOrder workflow with x-ms-wait-for-response and Accept: application/json...");
using HttpRequestMessage jsonWaitRequest = new(HttpMethod.Post, cancelOrderUri);
jsonWaitRequest.Content = new StringContent("77777", Encoding.UTF8, "text/plain");
jsonWaitRequest.Headers.Add("x-ms-wait-for-response", "true");
jsonWaitRequest.Headers.Add("Accept", "application/json");
using CancellationTokenSource jsonWaitCts = new(s_orchestrationTimeout);
using HttpResponseMessage jsonWaitResponse = await s_sharedHttpClient.SendAsync(jsonWaitRequest, jsonWaitCts.Token);
Assert.True(jsonWaitResponse.IsSuccessStatusCode, $"CancelOrder JSON wait-for-response request failed with status: {jsonWaitResponse.StatusCode}");
string jsonWaitResponseText = await jsonWaitResponse.Content.ReadAsStringAsync();
this._outputHelper.WriteLine($"CancelOrder JSON wait-for-response result: {jsonWaitResponseText}");
using JsonDocument jsonDoc = JsonDocument.Parse(jsonWaitResponseText);
JsonElement root = jsonDoc.RootElement;
Assert.True(root.TryGetProperty("runId", out _), "JSON response missing 'runId' property");
Assert.True(root.TryGetProperty("workflowStatus", out JsonElement statusEl), "JSON response missing 'workflowStatus' property");
Assert.Equal("Completed", statusEl.GetString());
Assert.True(root.TryGetProperty("result", out JsonElement resultEl), "JSON response missing 'result' property");
Assert.Contains("77777", resultEl.GetString());
});
}
@@ -8,7 +8,6 @@ using System.Reflection;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.AI;
using Microsoft.Extensions.DependencyInjection;
namespace Microsoft.Agents.AI.UnitTests.AgentSkills;
@@ -128,8 +127,9 @@ public sealed class AgentClassSkillTests
// Act — script with custom type deserialization
var script = skill.Scripts![0];
var inputJson = JsonSerializer.SerializeToElement(new LookupRequest { Query = "test", MaxResults = 5 }, jso);
var args = new AIFunctionArguments { ["request"] = inputJson };
var scriptResult = await script.RunAsync(skill, args, CancellationToken.None);
using var argsDoc = JsonDocument.Parse($$"""{ "request": {{inputJson.GetRawText()}} }""");
var args = argsDoc.RootElement;
var scriptResult = await script.RunAsync(skill, args, null, CancellationToken.None);
// Assert
Assert.NotNull(scriptResult);
@@ -173,12 +173,14 @@ public sealed class AgentClassSkillTests
// Act & Assert — static method
var doWorkScript = skill.Scripts!.First(s => s.Name == "do-work");
var doWorkResult = await doWorkScript.RunAsync(skill, new AIFunctionArguments { ["input"] = "hello" }, CancellationToken.None);
using var doWorkDoc = JsonDocument.Parse("""{"input":"hello"}""");
var doWorkResult = await doWorkScript.RunAsync(skill, doWorkDoc.RootElement, null, CancellationToken.None);
Assert.Equal("HELLO", doWorkResult?.ToString());
// Act & Assert — instance method
var appendScript = skill.Scripts!.First(s => s.Name == "append");
var appendResult = await appendScript.RunAsync(skill, new AIFunctionArguments { ["input"] = "test" }, CancellationToken.None);
using var appendDoc = JsonDocument.Parse("""{"input":"test"}""");
var appendResult = await appendScript.RunAsync(skill, appendDoc.RootElement, null, CancellationToken.None);
Assert.Equal("test-suffix", appendResult?.ToString());
}
@@ -367,7 +369,7 @@ public sealed class AgentClassSkillTests
// Act & Assert — all scripts produce values
foreach (var script in skill.Scripts!)
{
var result = await script.RunAsync(skill, new AIFunctionArguments(), CancellationToken.None);
var result = await script.RunAsync(skill, null, null, CancellationToken.None);
Assert.NotNull(result);
}
}
@@ -382,8 +384,9 @@ public sealed class AgentClassSkillTests
// Act & Assert — script with custom JSO
var script = skill.Scripts![0];
var inputJson = JsonSerializer.SerializeToElement(new LookupRequest { Query = "test", MaxResults = 3 }, jso);
var args = new AIFunctionArguments { ["request"] = inputJson };
var scriptResult = await script.RunAsync(skill, args, CancellationToken.None);
using var argsDoc = JsonDocument.Parse($$"""{ "request": {{inputJson.GetRawText()}} }""");
var args = argsDoc.RootElement;
var scriptResult = await script.RunAsync(skill, args, null, CancellationToken.None);
Assert.NotNull(scriptResult);
Assert.Contains("test", scriptResult!.ToString()!);
Assert.Contains("3", scriptResult!.ToString()!);
@@ -497,8 +500,9 @@ public sealed class AgentClassSkillTests
var script = skill.Scripts!.First(s => s.Name == "Lookup");
var jso = SkillTestJsonContext.Default.Options;
var inputJson = JsonSerializer.SerializeToElement(new LookupRequest { Query = "fallback", MaxResults = 7 }, jso);
var args = new AIFunctionArguments { ["request"] = inputJson };
var result = await script.RunAsync(skill, args, CancellationToken.None);
using var argsDoc = JsonDocument.Parse($$"""{ "request": {{inputJson.GetRawText()}} }""");
var args = argsDoc.RootElement;
var result = await script.RunAsync(skill, args, null, CancellationToken.None);
// Assert
Assert.NotNull(result);
@@ -531,8 +535,9 @@ public sealed class AgentClassSkillTests
var script = skill.Scripts!.First(s => s.Name == "Lookup");
var jso = SkillTestJsonContext.Default.Options;
var inputJson = JsonSerializer.SerializeToElement(new LookupRequest { Query = "explicit", MaxResults = 2 }, jso);
var args = new AIFunctionArguments { ["request"] = inputJson };
var result = await script.RunAsync(skill, args, CancellationToken.None);
using var argsDoc = JsonDocument.Parse($$"""{ "request": {{inputJson.GetRawText()}} }""");
var args = argsDoc.RootElement;
var result = await script.RunAsync(skill, args, null, CancellationToken.None);
// Assert
Assert.NotNull(result);
@@ -1,9 +1,9 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.AI;
namespace Microsoft.Agents.AI.UnitTests.AgentSkills;
@@ -16,13 +16,13 @@ public sealed class AgentFileSkillScriptTests
public async Task RunAsync_SkillIsNotAgentFileSkill_ThrowsInvalidOperationExceptionAsync()
{
// Arrange
static Task<object?> RunnerAsync(AgentFileSkill s, AgentFileSkillScript sc, AIFunctionArguments a, CancellationToken ct) => Task.FromResult<object?>("result");
static Task<object?> RunnerAsync(AgentFileSkill s, AgentFileSkillScript sc, JsonElement? a, IServiceProvider? sp, CancellationToken ct) => Task.FromResult<object?>("result");
var script = CreateScript("test-script", "/path/to/script.py", RunnerAsync);
var nonFileSkill = new TestAgentSkill("my-skill", "A skill", "Instructions.");
// Act & Assert
await Assert.ThrowsAsync<InvalidOperationException>(
() => script.RunAsync(nonFileSkill, new AIFunctionArguments(), CancellationToken.None));
() => script.RunAsync(nonFileSkill, null, null, CancellationToken.None));
}
[Fact]
@@ -30,7 +30,7 @@ public sealed class AgentFileSkillScriptTests
{
// Arrange
var runnerCalled = false;
Task<object?> runnerAsync(AgentFileSkill skill, AgentFileSkillScript scriptArg, AIFunctionArguments args, CancellationToken ct)
Task<object?> runnerAsync(AgentFileSkill skill, AgentFileSkillScript scriptArg, JsonElement? args, IServiceProvider? sp, CancellationToken ct)
{
runnerCalled = true;
return Task.FromResult<object?>("executed");
@@ -42,7 +42,7 @@ public sealed class AgentFileSkillScriptTests
"/skills/my-skill");
// Act
var result = await script.RunAsync(fileSkill, new AIFunctionArguments(), CancellationToken.None);
var result = await script.RunAsync(fileSkill, null, null, CancellationToken.None);
// Assert
Assert.True(runnerCalled);
@@ -55,7 +55,7 @@ public sealed class AgentFileSkillScriptTests
// Arrange
AgentFileSkill? capturedSkill = null;
AgentFileSkillScript? capturedScript = null;
Task<object?> runnerAsync(AgentFileSkill skill, AgentFileSkillScript scriptArg, AIFunctionArguments args, CancellationToken ct)
Task<object?> runnerAsync(AgentFileSkill skill, AgentFileSkillScript scriptArg, JsonElement? args, IServiceProvider? sp, CancellationToken ct)
{
capturedSkill = skill;
capturedScript = scriptArg;
@@ -68,7 +68,7 @@ public sealed class AgentFileSkillScriptTests
"/skills/owner-skill");
// Act
await script.RunAsync(fileSkill, new AIFunctionArguments(), CancellationToken.None);
await script.RunAsync(fileSkill, null, null, CancellationToken.None);
// Assert
Assert.Same(fileSkill, capturedSkill);
@@ -79,7 +79,7 @@ public sealed class AgentFileSkillScriptTests
public void Script_HasCorrectNameAndPath()
{
// Arrange & Act
static Task<object?> RunnerAsync(AgentFileSkill s, AgentFileSkillScript sc, AIFunctionArguments a, CancellationToken ct) => Task.FromResult<object?>(null);
static Task<object?> RunnerAsync(AgentFileSkill s, AgentFileSkillScript sc, JsonElement? a, IServiceProvider? sp, CancellationToken ct) => Task.FromResult<object?>(null);
var script = CreateScript("my-script", "/path/to/my-script.py", RunnerAsync);
// Assert
@@ -87,10 +87,173 @@ public sealed class AgentFileSkillScriptTests
Assert.Equal("/path/to/my-script.py", script.FullPath);
}
[Fact]
public void ParametersSchema_ReturnsExpectedArraySchema()
{
// Arrange
static Task<object?> RunnerAsync(AgentFileSkill s, AgentFileSkillScript sc, JsonElement? a, IServiceProvider? sp, CancellationToken ct) => Task.FromResult<object?>(null);
var script = CreateScript("my-script", "/path/to/script.py", RunnerAsync);
// Act
var schema = script.ParametersSchema;
// Assert
Assert.NotNull(schema);
var raw = schema!.Value.GetRawText();
Assert.Contains("\"type\":\"array\"", raw);
Assert.Contains("\"items\":{\"type\":\"string\"}", raw);
}
[Fact]
public void Content_WithScripts_AppendsPerScriptEntries()
{
// Arrange
static Task<object?> RunnerAsync(AgentFileSkill s, AgentFileSkillScript sc, JsonElement? a, IServiceProvider? sp, CancellationToken ct) => Task.FromResult<object?>(null);
var script1 = CreateScript("build", "/scripts/build.sh", RunnerAsync);
var script2 = CreateScript("deploy", "/scripts/deploy.sh", RunnerAsync);
var fileSkill = new AgentFileSkill(
new AgentSkillFrontmatter("my-skill", "A skill"),
"Original content",
"/skills/my-skill",
scripts: [script1, script2]);
// Act
var content = fileSkill.Content;
// Assert — content starts with original and appends per-script entries
Assert.StartsWith("Original content", content);
Assert.Contains("<scripts>", content);
Assert.Contains("<script name=\"build\">", content);
Assert.Contains("<script name=\"deploy\">", content);
Assert.Contains("<parameters_schema>", content);
Assert.Contains("</scripts>", content);
}
[Fact]
public void Content_WithoutScripts_ReturnsOriginalContent()
{
// Arrange
var fileSkill = new AgentFileSkill(
new AgentSkillFrontmatter("my-skill", "A skill"),
"Original content only",
"/skills/my-skill");
// Act
var content = fileSkill.Content;
// Assert
Assert.Equal("Original content only", content);
}
[Fact]
public void Content_WithScripts_IsCached()
{
// Arrange
static Task<object?> RunnerAsync(AgentFileSkill s, AgentFileSkillScript sc, JsonElement? a, IServiceProvider? sp, CancellationToken ct) => Task.FromResult<object?>(null);
var script = CreateScript("test", "/scripts/test.sh", RunnerAsync);
var fileSkill = new AgentFileSkill(
new AgentSkillFrontmatter("my-skill", "A skill"),
"Content",
"/skills/my-skill",
scripts: [script]);
// Act
var content1 = fileSkill.Content;
var content2 = fileSkill.Content;
// Assert
Assert.Same(content1, content2);
}
[Fact]
public async Task RunAsync_ForwardsJsonArrayArgumentsToRunnerAsync()
{
// Arrange
JsonElement? capturedArgs = null;
Task<object?> runnerAsync(AgentFileSkill skill, AgentFileSkillScript scriptArg, JsonElement? args, IServiceProvider? sp, CancellationToken ct)
{
capturedArgs = args;
return Task.FromResult<object?>("done");
}
var script = CreateScript("array-test", "/scripts/test.sh", runnerAsync);
var fileSkill = new AgentFileSkill(
new AgentSkillFrontmatter("my-skill", "A skill"),
"Content",
"/skills/my-skill");
using var arrayArgsDoc = JsonDocument.Parse("""["arg1","arg2","arg3"]""");
var arrayArgs = arrayArgsDoc.RootElement;
// Act
await script.RunAsync(fileSkill, arrayArgs, null, CancellationToken.None);
// Assert — the raw JSON array is forwarded unchanged
Assert.NotNull(capturedArgs);
Assert.Equal(JsonValueKind.Array, capturedArgs!.Value.ValueKind);
Assert.Equal("""["arg1","arg2","arg3"]""", capturedArgs.Value.GetRawText());
}
[Fact]
public async Task RunAsync_ForwardsServiceProviderToRunnerAsync()
{
// Arrange
IServiceProvider? capturedProvider = null;
Task<object?> runnerAsync(AgentFileSkill skill, AgentFileSkillScript scriptArg, JsonElement? args, IServiceProvider? sp, CancellationToken ct)
{
capturedProvider = sp;
return Task.FromResult<object?>("done");
}
var script = CreateScript("sp-test", "/scripts/test.sh", runnerAsync);
var fileSkill = new AgentFileSkill(
new AgentSkillFrontmatter("my-skill", "A skill"),
"Content",
"/skills/my-skill");
var mockProvider = new TestServiceProvider();
// Act
await script.RunAsync(fileSkill, null, mockProvider, CancellationToken.None);
// Assert
Assert.Same(mockProvider, capturedProvider);
}
[Fact]
public async Task RunAsync_NoRunner_ThrowsInvalidOperationExceptionAsync()
{
// Arrange — create script without a runner
var script = CreateScript("no-runner", "/scripts/test.sh", runner: null);
var fileSkill = new AgentFileSkill(
new AgentSkillFrontmatter("my-skill", "A skill"),
"Content",
"/skills/my-skill");
// Act & Assert
await Assert.ThrowsAsync<InvalidOperationException>(
() => script.RunAsync(fileSkill, null, null, CancellationToken.None));
}
[Fact]
public void Content_WithScripts_ContainsDefaultParametersSchema()
{
// Arrange
static Task<object?> RunnerAsync(AgentFileSkill s, AgentFileSkillScript sc, JsonElement? a, IServiceProvider? sp, CancellationToken ct) => Task.FromResult<object?>(null);
var script = CreateScript("test", "/scripts/test.sh", RunnerAsync);
var fileSkill = new AgentFileSkill(
new AgentSkillFrontmatter("my-skill", "A skill"),
"Original content",
"/skills/my-skill",
scripts: [script]);
// Act
var content = fileSkill.Content;
// Assert — the appended block contains the actual default schema from AgentFileSkillScript
Assert.Contains("""{"type":"array","items":{"type":"string"}}""", content);
}
/// <summary>
/// Helper to create an <see cref="AgentFileSkillScript"/> via reflection since the constructor is internal.
/// </summary>
private static AgentFileSkillScript CreateScript(string name, string fullPath, AgentFileSkillScriptRunner executor)
private static AgentFileSkillScript CreateScript(string name, string fullPath, AgentFileSkillScriptRunner? runner)
{
var ctor = typeof(AgentFileSkillScript).GetConstructor(
System.Reflection.BindingFlags.NonPublic | System.Reflection.BindingFlags.Instance,
@@ -98,6 +261,14 @@ public sealed class AgentFileSkillScriptTests
[typeof(string), typeof(string), typeof(AgentFileSkillScriptRunner)],
null) ?? throw new InvalidOperationException("Could not find internal constructor.");
return (AgentFileSkillScript)ctor.Invoke([name, fullPath, executor]);
return (AgentFileSkillScript)ctor.Invoke([name, fullPath, runner]);
}
/// <summary>
/// Minimal <see cref="IServiceProvider"/> for testing service forwarding.
/// </summary>
private sealed class TestServiceProvider : IServiceProvider
{
public object? GetService(Type serviceType) => null;
}
}
@@ -3,9 +3,9 @@
using System;
using System.IO;
using System.Linq;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.AI;
namespace Microsoft.Agents.AI.UnitTests.AgentSkills;
@@ -15,7 +15,7 @@ namespace Microsoft.Agents.AI.UnitTests.AgentSkills;
public sealed class AgentFileSkillsSourceScriptTests : IDisposable
{
private static readonly string[] s_rubyExtension = new[] { ".rb" };
private static readonly AgentFileSkillScriptRunner s_noOpExecutor = (skill, script, args, ct) => Task.FromResult<object?>(null);
private static readonly AgentFileSkillScriptRunner s_noOpExecutor = (skill, script, args, sp, ct) => Task.FromResult<object?>(null);
private readonly string _testRoot;
@@ -139,7 +139,7 @@ public sealed class AgentFileSkillsSourceScriptTests : IDisposable
var executorCalled = false;
var source = new AgentFileSkillsSource(
this._testRoot,
(skill, script, args, ct) =>
(skill, script, args, sp, ct) =>
{
executorCalled = true;
Assert.Equal("exec-skill", skill.Frontmatter.Name);
@@ -150,7 +150,7 @@ public sealed class AgentFileSkillsSourceScriptTests : IDisposable
// Act
var skills = await source.GetSkillsAsync(CancellationToken.None);
var scriptResult = await skills[0].Scripts![0].RunAsync(skills[0], new AIFunctionArguments(), CancellationToken.None);
var scriptResult = await skills[0].Scripts![0].RunAsync(skills[0], null, null, CancellationToken.None);
// Assert
Assert.True(executorCalled);
@@ -178,7 +178,7 @@ public sealed class AgentFileSkillsSourceScriptTests : IDisposable
var script = skills[0].Scripts![0];
// Assert — running the script throws because no runner was provided
await Assert.ThrowsAsync<InvalidOperationException>(() => script.RunAsync(skills[0], new AIFunctionArguments(), CancellationToken.None));
await Assert.ThrowsAsync<InvalidOperationException>(() => script.RunAsync(skills[0], null, null, CancellationToken.None));
}
[Fact]
@@ -204,10 +204,10 @@ public sealed class AgentFileSkillsSourceScriptTests : IDisposable
{
// Arrange
CreateSkillWithScript(this._testRoot, "args-skill", "Args test", "Body.", "scripts/test.py", "print('ok')");
AIFunctionArguments? capturedArgs = null;
JsonElement? capturedArgs = null;
var source = new AgentFileSkillsSource(
this._testRoot,
(skill, script, args, ct) =>
(skill, script, args, sp, ct) =>
{
capturedArgs = args;
return Task.FromResult<object?>("done");
@@ -215,17 +215,15 @@ public sealed class AgentFileSkillsSourceScriptTests : IDisposable
// Act
var skills = await source.GetSkillsAsync(CancellationToken.None);
var arguments = new AIFunctionArguments
{
["value"] = 26.2,
["factor"] = 1.60934
};
await skills[0].Scripts![0].RunAsync(skills[0], arguments, CancellationToken.None);
using var argumentsDoc = JsonDocument.Parse("""{"value":26.2,"factor":1.60934}""");
var arguments = argumentsDoc.RootElement;
await skills[0].Scripts![0].RunAsync(skills[0], arguments, null, CancellationToken.None);
// Assert
Assert.NotNull(capturedArgs);
Assert.Equal(26.2, capturedArgs["value"]);
Assert.Equal(1.60934, capturedArgs["factor"]);
Assert.Equal(JsonValueKind.Object, capturedArgs!.Value.ValueKind);
Assert.Equal(26.2, capturedArgs.Value.GetProperty("value").GetDouble());
Assert.Equal(1.60934, capturedArgs.Value.GetProperty("factor").GetDouble());
}
[Fact]
@@ -5,7 +5,6 @@ using System.Reflection;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.AI;
namespace Microsoft.Agents.AI.UnitTests.AgentSkills;
@@ -22,7 +21,7 @@ public sealed class AgentInlineSkillScriptTests
var skill = new AgentInlineSkill("test-skill", "Test.", "Instructions.");
// Act
var result = await script.RunAsync(skill, new AIFunctionArguments(), CancellationToken.None);
var result = await script.RunAsync(skill, null, null, CancellationToken.None);
// Assert
Assert.Equal("hello", result?.ToString());
@@ -34,10 +33,11 @@ public sealed class AgentInlineSkillScriptTests
// Arrange
var script = new AgentInlineSkillScript("add", (int a, int b) => a + b);
var skill = new AgentInlineSkill("calc-skill", "Calc.", "Instructions.");
var args = new AIFunctionArguments { ["a"] = 3, ["b"] = 7 };
using var argsDoc = JsonDocument.Parse("""{"a":3,"b":7}""");
var args = argsDoc.RootElement;
// Act
var result = await script.RunAsync(skill, args, CancellationToken.None);
var result = await script.RunAsync(skill, args, null, CancellationToken.None);
// Assert
Assert.Equal(10, int.Parse(result?.ToString()!));
@@ -129,10 +129,11 @@ public sealed class AgentInlineSkillScriptTests
}, serializerOptions: jso);
var skill = new AgentInlineSkill("test-skill", "Test.", "Instructions.");
var inputJson = JsonSerializer.SerializeToElement(new LookupRequest { Query = "test", MaxResults = 5 }, jso);
var args = new AIFunctionArguments { ["request"] = inputJson };
using var argsDoc = JsonDocument.Parse($$"""{ "request": {{inputJson.GetRawText()}} }""");
var args = argsDoc.RootElement;
// Act
var result = await script.RunAsync(skill, args, CancellationToken.None);
var result = await script.RunAsync(skill, args, null, CancellationToken.None);
// Assert — the custom input type was deserialized and the response was produced
Assert.NotNull(result);
@@ -145,10 +146,11 @@ public sealed class AgentInlineSkillScriptTests
// Arrange
var script = new AgentInlineSkillScript("echo", (string message) => message);
var skill = new AgentInlineSkill("test-skill", "Test.", "Instructions.");
var args = new AIFunctionArguments { ["message"] = "hello world" };
using var argsDoc = JsonDocument.Parse("""{"message":"hello world"}""");
var args = argsDoc.RootElement;
// Act
var result = await script.RunAsync(skill, args, CancellationToken.None);
var result = await script.RunAsync(skill, args, null, CancellationToken.None);
// Assert
Assert.Equal("hello world", result?.ToString());
@@ -175,10 +177,11 @@ public sealed class AgentInlineSkillScriptTests
var method = typeof(AgentInlineSkillScriptTests).GetMethod(nameof(StaticScriptHelper), BindingFlags.NonPublic | BindingFlags.Static)!;
var script = new AgentInlineSkillScript("static-method-script", method, target: null);
var skill = new AgentInlineSkill("test-skill", "Test.", "Instructions.");
var args = new AIFunctionArguments { ["input"] = "hello" };
using var argsDoc = JsonDocument.Parse("""{"input":"hello"}""");
var args = argsDoc.RootElement;
// Act
var result = await script.RunAsync(skill, args, CancellationToken.None);
var result = await script.RunAsync(skill, args, null, CancellationToken.None);
// Assert
Assert.Equal("HELLO", result?.ToString());
@@ -191,10 +194,11 @@ public sealed class AgentInlineSkillScriptTests
var method = typeof(AgentInlineSkillScriptTests).GetMethod(nameof(InstanceScriptHelper), BindingFlags.NonPublic | BindingFlags.Instance)!;
var script = new AgentInlineSkillScript("instance-method-script", method, target: this);
var skill = new AgentInlineSkill("test-skill", "Test.", "Instructions.");
var args = new AIFunctionArguments { ["input"] = "test" };
using var argsDoc2 = JsonDocument.Parse("""{"input":"test"}""");
var args2 = argsDoc2.RootElement;
// Act
var result = await script.RunAsync(skill, args, CancellationToken.None);
var result = await script.RunAsync(skill, args2, null, CancellationToken.None);
// Assert
Assert.Equal("test-suffix", result?.ToString());
@@ -223,7 +227,63 @@ public sealed class AgentInlineSkillScriptTests
Assert.Contains("input", schema!.Value.GetRawText());
}
[Fact]
public async Task RunAsync_WithNonObjectArguments_ThrowsInvalidOperationExceptionAsync()
{
// Arrange — inline scripts require a JSON object for arguments
var script = new AgentInlineSkillScript("noop", () => "ok");
var skill = new AgentInlineSkill("test-skill", "Test.", "Instructions.");
using var arrayArgsDoc = JsonDocument.Parse("""["a","b"]""");
var arrayArgs = arrayArgsDoc.RootElement;
// Act & Assert — non-object JSON should fail fast rather than silently dropping arguments
await Assert.ThrowsAsync<InvalidOperationException>(
() => script.RunAsync(skill, arrayArgs, null, CancellationToken.None));
}
[Fact]
public async Task RunAsync_WithNullArguments_TreatsAsNoArgumentsAsync()
{
// Arrange — a parameterless delegate should succeed when given null arguments
var script = new AgentInlineSkillScript("noop", () => "ok");
var skill = new AgentInlineSkill("test-skill", "Test.", "Instructions.");
// Act
var result = await script.RunAsync(skill, null, null, CancellationToken.None);
// Assert
Assert.Equal("ok", result?.ToString());
}
[Fact]
public async Task RunAsync_ServiceProviderIsForwardedAsync()
{
// Arrange — delegate that resolves a service from the IServiceProvider
IServiceProvider? capturedProvider = null;
var script = new AgentInlineSkillScript("svc-test", (IServiceProvider sp) =>
{
capturedProvider = sp;
return "done";
});
var skill = new AgentInlineSkill("test-skill", "Test.", "Instructions.");
var mockProvider = new TestServiceProvider();
// Act
await script.RunAsync(skill, null, mockProvider, CancellationToken.None);
// Assert
Assert.Same(mockProvider, capturedProvider);
}
private static string StaticScriptHelper(string input) => input.ToUpperInvariant();
private string InstanceScriptHelper(string input) => input + "-suffix";
/// <summary>
/// Minimal <see cref="IServiceProvider"/> for testing service forwarding.
/// </summary>
private sealed class TestServiceProvider : IServiceProvider
{
public object? GetService(Type serviceType) => null;
}
}
@@ -433,10 +433,11 @@ public sealed class AgentInlineSkillTests
TotalCount = request.MaxResults,
});
var inputJson = JsonSerializer.SerializeToElement(new LookupRequest { Query = "test", MaxResults = 3 }, jso);
var args = new AIFunctionArguments { ["request"] = inputJson };
using var argsDoc = JsonDocument.Parse($$"""{ "request": {{inputJson.GetRawText()}} }""");
var args = argsDoc.RootElement;
// Act
var result = await skill.Scripts![0].RunAsync(skill, args, CancellationToken.None);
var result = await skill.Scripts![0].RunAsync(skill, args, null, CancellationToken.None);
// Assert — the custom input was deserialized via skill-level JSO and response was produced
Assert.NotNull(result);
@@ -456,10 +457,11 @@ public sealed class AgentInlineSkillTests
TotalCount = request.MaxResults,
}, serializerOptions: scriptJso);
var inputJson = JsonSerializer.SerializeToElement(new LookupRequest { Query = "override", MaxResults = 7 }, scriptJso);
var args = new AIFunctionArguments { ["request"] = inputJson };
using var argsDoc = JsonDocument.Parse($$"""{ "request": {{inputJson.GetRawText()}} }""");
var args = argsDoc.RootElement;
// Act
var result = await skill.Scripts![0].RunAsync(skill, args, CancellationToken.None);
var result = await skill.Scripts![0].RunAsync(skill, args, null, CancellationToken.None);
// Assert — per-script JSO takes effect and custom types are properly marshaled
Assert.NotNull(result);
@@ -4,6 +4,7 @@ using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.AI;
@@ -15,7 +16,7 @@ namespace Microsoft.Agents.AI.UnitTests.AgentSkills;
/// </summary>
public sealed class AgentSkillsProviderTests : IDisposable
{
private static readonly AgentFileSkillScriptRunner s_noOpExecutor = (skill, script, args, ct) => Task.FromResult<object?>(null);
private static readonly AgentFileSkillScriptRunner s_noOpExecutor = (skill, script, args, sp, ct) => Task.FromResult<object?>(null);
private readonly string _testRoot;
private readonly TestAIAgent _agent = new();
@@ -462,7 +463,7 @@ public sealed class AgentSkillsProviderTests : IDisposable
// Act — call UseFileScriptRunner AFTER UseFileSkill (the bug scenario)
var provider = new AgentSkillsProviderBuilder()
.UseFileSkill(this._testRoot)
.UseFileScriptRunner((skill, script, args, ct) =>
.UseFileScriptRunner((skill, script, args, sp, ct) =>
{
executorCalled = true;
return Task.FromResult<object?>("executed");
@@ -487,6 +488,62 @@ public sealed class AgentSkillsProviderTests : IDisposable
Assert.True(executorCalled);
}
[Fact]
public async Task RunSkillScript_ForwardsJsonArgumentsAndServiceProviderToRunnerAsync()
{
// Arrange — create a skill with a script file
string skillDir = Path.Combine(this._testRoot, "fwd-skill");
Directory.CreateDirectory(Path.Combine(skillDir, "scripts"));
File.WriteAllText(
Path.Combine(skillDir, "SKILL.md"),
"---\nname: fwd-skill\ndescription: Forwarding test\n---\nBody.");
File.WriteAllText(
Path.Combine(skillDir, "scripts", "run.py"),
"print('ok')");
JsonElement? capturedArgs = null;
IServiceProvider? capturedServiceProvider = null;
var provider = new AgentSkillsProviderBuilder()
.UseFileSkill(this._testRoot)
.UseFileScriptRunner((skill, script, args, sp, ct) =>
{
capturedArgs = args;
capturedServiceProvider = sp;
return Task.FromResult<object?>("executed");
})
.Build();
var mockServiceProvider = new TestServiceProvider();
var invokingContext = new AIContextProvider.InvokingContext(this._agent, session: null, new AIContext());
var result = await provider.InvokingAsync(invokingContext, CancellationToken.None);
var runScriptTool = result.Tools!.First(t => t.Name == "run_skill_script") as AIFunction;
// Act — invoke with JsonElement arguments and a service provider
using var argsJsonDoc = JsonDocument.Parse("""["arg1","arg2"]""");
var argsJson = argsJsonDoc.RootElement;
await runScriptTool!.InvokeAsync(new AIFunctionArguments(new Dictionary<string, object?>
{
["skillName"] = "fwd-skill",
["scriptName"] = "scripts/run.py",
["arguments"] = argsJson,
})
{
Services = mockServiceProvider,
});
// Assert — JsonElement arguments and service provider are forwarded to the runner
Assert.NotNull(capturedArgs);
Assert.Equal(JsonValueKind.Array, capturedArgs!.Value.ValueKind);
Assert.Equal("""["arg1","arg2"]""", capturedArgs.Value.GetRawText());
Assert.Same(mockServiceProvider, capturedServiceProvider);
}
private sealed class TestServiceProvider : IServiceProvider
{
public object? GetService(Type serviceType) => null;
}
private static void CreateSkillIn(string root, string name, string description, string body)
{
string skillDir = Path.Combine(root, name);
@@ -15,7 +15,7 @@ public sealed class FileAgentSkillLoaderTests : IDisposable
private static readonly string[] s_customExtensions = [".custom"];
private static readonly string[] s_validExtensions = [".md", ".json", ".custom"];
private static readonly string[] s_mixedValidInvalidExtensions = [".md", "json"];
private static readonly AgentFileSkillScriptRunner s_noOpExecutor = (skill, script, args, ct) => Task.FromResult<object?>(null);
private static readonly AgentFileSkillScriptRunner s_noOpExecutor = (skill, script, args, sp, ct) => Task.FromResult<object?>(null);
private readonly string _testRoot;
@@ -60,10 +60,20 @@ public abstract class IntegrationTest : IDisposable
protected async ValueTask<DeclarativeWorkflowOptions> CreateOptionsAsync(bool externalConversation = false, params IEnumerable<AIFunction> functionTools)
{
return await this.CreateOptionsAsync(externalConversation, mcpToolProvider: null, functionTools).ConfigureAwait(false);
return await this.CreateOptionsAsync(externalConversation, mcpToolProvider: null, httpRequestHandler: null, functionTools).ConfigureAwait(false);
}
protected async ValueTask<DeclarativeWorkflowOptions> CreateOptionsAsync(bool externalConversation, IMcpToolHandler? mcpToolProvider, params IEnumerable<AIFunction> functionTools)
{
return await this.CreateOptionsAsync(externalConversation, mcpToolProvider, httpRequestHandler: null, functionTools).ConfigureAwait(false);
}
protected async ValueTask<DeclarativeWorkflowOptions> CreateOptionsAsync(bool externalConversation, IHttpRequestHandler? httpRequestHandler, params IEnumerable<AIFunction> functionTools)
{
return await this.CreateOptionsAsync(externalConversation, mcpToolProvider: null, httpRequestHandler, functionTools).ConfigureAwait(false);
}
protected async ValueTask<DeclarativeWorkflowOptions> CreateOptionsAsync(bool externalConversation, IMcpToolHandler? mcpToolProvider, IHttpRequestHandler? httpRequestHandler, params IEnumerable<AIFunction> functionTools)
{
AzureAgentProvider agentProvider =
new(this.TestEndpoint, TestAzureCliCredentials.CreateAzureCliCredential())
@@ -82,7 +92,8 @@ public abstract class IntegrationTest : IDisposable
{
ConversationId = conversationId,
LoggerFactory = this.Output,
McpToolHandler = mcpToolProvider
McpToolHandler = mcpToolProvider,
HttpRequestHandler = httpRequestHandler,
};
}
@@ -45,6 +45,15 @@ public sealed class InvokeToolWorkflowTest(ITestOutputHelper output) : Integrati
#endregion
#region InvokeHttpRequest Tests
[RetryTheory(3, 5000)]
[InlineData("HttpRequest.yaml", "visibility: public")]
public Task ValidateHttpRequestAsync(string workflowFileName, string? expectedResultContains) =>
this.RunHttpRequestTestAsync(workflowFileName, expectedResultContains);
#endregion
#region InvokeFunctionTool Test Helpers
/// <summary>
@@ -250,6 +259,40 @@ public sealed class InvokeToolWorkflowTest(ITestOutputHelper output) : Integrati
#endregion
#region InvokeHttpRequest Test Helpers
/// <summary>
/// Runs an HttpRequestAction workflow test with the specified configuration.
/// </summary>
private async Task RunHttpRequestTestAsync(
string workflowFileName,
string? expectedResultContains = null)
{
// Arrange
string workflowPath = GetWorkflowPath(workflowFileName);
await using DefaultHttpRequestHandler httpRequestHandler = new();
DeclarativeWorkflowOptions workflowOptions = await this.CreateOptionsAsync(
externalConversation: false,
httpRequestHandler: httpRequestHandler);
Workflow workflow = DeclarativeWorkflowBuilder.Build<string>(workflowPath, workflowOptions);
WorkflowHarness harness = new(workflow, runId: Path.GetFileNameWithoutExtension(workflowPath));
// Act
WorkflowEvents workflowEvents = await harness.RunWorkflowAsync("start").ConfigureAwait(false);
// Assert - Verify executor and action events
AssertWorkflowEventsEmitted(workflowEvents);
// Assert - Verify expected result if specified
if (expectedResultContains is not null)
{
AssertResultContains(workflowEvents, expectedResultContains);
}
}
#endregion
#region Shared Helpers
private static void AssertWorkflowEventsEmitted(WorkflowEvents workflowEvents)
@@ -0,0 +1,32 @@
#
# This workflow tests invoking HttpRequestAction end-to-end.
# Uses the public GitHub API (unauthenticated) to fetch repo metadata.
#
kind: Workflow
trigger:
kind: OnConversationStart
id: workflow_http_request_test
actions:
# Set the repo owner used to form the request URL.
- kind: SetVariable
id: set_repo_owner
variable: Local.RepoOwner
value: dotnet
# Invoke the GitHub repo API.
- kind: HttpRequestAction
id: fetch_repo_info
conversationId: =System.ConversationId
method: GET
url: =Concatenate("https://api.github.com/repos/", Local.RepoOwner, "/runtime")
headers:
Accept: application/vnd.github+json
User-Agent: agent-framework-integration-test
response: Local.RepoInfo
# Surface the Repo visibility field from the parsed JSON response.
- kind: SendMessage
id: show_visibility
message: "visibility: {Local.RepoInfo.visibility}"
@@ -181,6 +181,7 @@ public sealed class DeclarativeWorkflowTest(ITestOutputHelper output) : Workflow
[InlineData("ResetVariable.yaml", 2, "clear_var")]
[InlineData("MixedScopes.yaml", 2, "activity_input")]
[InlineData("CaseInsensitive.yaml", 6, "end_when_match")]
[InlineData("HttpRequest.yaml", 1, "http_request")]
public async Task ExecuteActionAsync(string workflowFile, int expectedCount, string expectedId)
{
await this.RunWorkflowAsync(workflowFile);
@@ -200,7 +201,6 @@ public sealed class DeclarativeWorkflowTest(ITestOutputHelper output) : Workflow
[InlineData(typeof(EmitEvent.Builder))]
[InlineData(typeof(GetActivityMembers.Builder))]
[InlineData(typeof(GetConversationMembers.Builder))]
[InlineData(typeof(HttpRequestAction.Builder))]
[InlineData(typeof(InvokeAIBuilderModelAction.Builder))]
[InlineData(typeof(InvokeConnectorAction.Builder))]
[InlineData(typeof(InvokeCustomModelAction.Builder))]
@@ -266,6 +266,7 @@ public sealed class DeclarativeWorkflowTest(ITestOutputHelper output) : Workflow
[InlineData("SendActivity.yaml", "activity_input")]
[InlineData("SetVariable.yaml", "set_var")]
[InlineData("SetTextVariable.yaml", "set_text")]
[InlineData("HttpRequest.yaml", "http_request")]
public async Task CancelRunAsync(string workflowPath, string expectedExecutedId)
{
// Arrange
@@ -374,7 +375,12 @@ public sealed class DeclarativeWorkflowTest(ITestOutputHelper output) : Workflow
{
using StreamReader yamlReader = File.OpenText(Path.Combine("Workflows", workflowPath));
Mock<ResponseAgentProvider> mockAgentProvider = CreateMockProvider($"{workflowInput}");
DeclarativeWorkflowOptions workflowContext = new(mockAgentProvider.Object) { LoggerFactory = this.Output };
DeclarativeWorkflowOptions workflowContext =
new(mockAgentProvider.Object)
{
LoggerFactory = this.Output,
HttpRequestHandler = CreateMockHttpRequestHandler().Object,
};
return DeclarativeWorkflowBuilder.Build<TInput>(yamlReader, workflowContext);
}
@@ -385,4 +391,18 @@ public sealed class DeclarativeWorkflowTest(ITestOutputHelper output) : Workflow
mockAgentProvider.Setup(provider => provider.CreateMessageAsync(It.IsAny<string>(), It.IsAny<ChatMessage>(), It.IsAny<CancellationToken>())).Returns(Task.FromResult(new ChatMessage(ChatRole.Assistant, input)));
return mockAgentProvider;
}
private static Mock<IHttpRequestHandler> CreateMockHttpRequestHandler()
{
Mock<IHttpRequestHandler> mockHandler = new(MockBehavior.Loose);
mockHandler
.Setup(handler => handler.SendAsync(It.IsAny<HttpRequestInfo>(), It.IsAny<CancellationToken>()))
.Returns(() => Task.FromResult(new HttpRequestResult
{
StatusCode = 200,
IsSuccessStatusCode = true,
Body = "{\"ok\":true}",
}));
return mockHandler;
}
}
@@ -0,0 +1,510 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Net;
using System.Net.Http;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using FluentAssertions;
namespace Microsoft.Agents.AI.Workflows.Declarative.UnitTests;
/// <summary>
/// Unit tests for <see cref="DefaultHttpRequestHandler"/>.
/// </summary>
public sealed class DefaultHttpRequestHandlerTests
{
private static readonly string[] s_setCookieValues = ["a=1", "b=2"];
private const string TestUrl = "https://api.example.test/resource";
#region Constructor Tests
[Fact]
public async Task ConstructorWithNoParametersCreatesInstanceAsync()
{
// Act
await using DefaultHttpRequestHandler handler = new();
// Assert
handler.Should().NotBeNull();
}
[Fact]
public async Task ConstructorWithNullProviderCreatesInstanceAsync()
{
// Act
await using DefaultHttpRequestHandler handler = new(httpClientProvider: null);
// Assert
handler.Should().NotBeNull();
}
[Fact]
public void ConstructorWithNullHttpClientThrows()
{
// Act
Action act = () => _ = new DefaultHttpRequestHandler((HttpClient)null!);
// Assert
act.Should().Throw<ArgumentNullException>();
}
[Fact]
public async Task ConstructorWithHttpClientUsesSuppliedClientForAllRequestsAsync()
{
// Arrange
TestHttpMessageHandler messageHandler = new((req, _) =>
Task.FromResult(new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new StringContent("ok", Encoding.UTF8, "text/plain"),
}));
using HttpClient suppliedClient = new(messageHandler);
await using DefaultHttpRequestHandler handler = new(suppliedClient);
HttpRequestInfo request = new() { Method = "GET", Url = TestUrl };
// Act
HttpRequestResult result = await handler.SendAsync(request);
// Assert - the supplied HttpClient's underlying handler saw the request
messageHandler.LastRequest.Should().NotBeNull();
messageHandler.LastRequest!.RequestUri!.ToString().Should().Be(TestUrl);
result.Body.Should().Be("ok");
}
[Fact]
public async Task DisposeAsyncDoesNotDisposeCallerSuppliedHttpClientAsync()
{
// Arrange
TestHttpMessageHandler messageHandler = new((req, _) =>
Task.FromResult(new HttpResponseMessage(HttpStatusCode.OK)));
using HttpClient suppliedClient = new(messageHandler);
// Act
DefaultHttpRequestHandler handler = new(suppliedClient);
await handler.DisposeAsync();
// Assert - supplied client remains usable (not disposed)
Func<Task> act = async () => await suppliedClient.GetAsync(new Uri(TestUrl));
await act.Should().NotThrowAsync<ObjectDisposedException>();
}
#endregion
#region Argument Validation Tests
[Fact]
public async Task SendAsyncWithNullRequestThrowsAsync()
{
// Arrange
await using DefaultHttpRequestHandler handler = new();
// Act
Func<Task> act = async () => await handler.SendAsync(null!);
// Assert
await act.Should().ThrowAsync<ArgumentNullException>();
}
[Fact]
public async Task SendAsyncWithEmptyUrlThrowsAsync()
{
// Arrange
await using DefaultHttpRequestHandler handler = new();
HttpRequestInfo request = new() { Method = "GET", Url = "" };
// Act
Func<Task> act = async () => await handler.SendAsync(request);
// Assert
await act.Should().ThrowAsync<ArgumentException>();
}
[Fact]
public async Task SendAsyncWithEmptyMethodThrowsAsync()
{
// Arrange
await using DefaultHttpRequestHandler handler = new();
HttpRequestInfo request = new() { Method = "", Url = TestUrl };
// Act
Func<Task> act = async () => await handler.SendAsync(request);
// Assert
await act.Should().ThrowAsync<ArgumentException>();
}
#endregion
#region Send Behavior Tests
[Fact]
public async Task SendAsyncUsesProvidedHttpClientAsync()
{
// Arrange
TestHttpMessageHandler messageHandler = new((req, _) =>
Task.FromResult(new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new StringContent("hello", Encoding.UTF8, "text/plain"),
}));
await using DefaultHttpRequestHandler handler = new((_, _) => Task.FromResult<HttpClient?>(new HttpClient(messageHandler)));
HttpRequestInfo request = new() { Method = "GET", Url = TestUrl };
// Act
HttpRequestResult result = await handler.SendAsync(request);
// Assert
messageHandler.LastRequest.Should().NotBeNull();
messageHandler.LastRequest!.Method.Should().Be(HttpMethod.Get);
messageHandler.LastRequest.RequestUri!.ToString().Should().Be(TestUrl);
result.StatusCode.Should().Be(200);
result.IsSuccessStatusCode.Should().BeTrue();
result.Body.Should().Be("hello");
}
[Fact]
public async Task SendAsyncMapsAllKnownMethodsAsync()
{
// Arrange
TestHttpMessageHandler messageHandler = new((req, _) =>
Task.FromResult(new HttpResponseMessage(HttpStatusCode.OK)));
await using DefaultHttpRequestHandler handler = new((_, _) => Task.FromResult<HttpClient?>(new HttpClient(messageHandler)));
foreach (string method in new[] { "GET", "POST", "PUT", "PATCH", "DELETE", "CUSTOM" })
{
HttpRequestInfo request = new() { Method = method, Url = TestUrl };
// Act
await handler.SendAsync(request);
// Assert
messageHandler.LastRequest!.Method.Method.Should().Be(method);
}
}
[Fact]
public async Task SendAsyncNormalizesWhitespaceAroundCustomMethodAsync()
{
// Arrange
TestHttpMessageHandler messageHandler = new((req, _) =>
Task.FromResult(new HttpResponseMessage(HttpStatusCode.OK)));
await using DefaultHttpRequestHandler handler = new((_, _) => Task.FromResult<HttpClient?>(new HttpClient(messageHandler)));
HttpRequestInfo request = new() { Method = " custom ", Url = TestUrl };
// Act
await handler.SendAsync(request);
// Assert - fallback path should apply the same Trim/ToUpperInvariant normalization.
messageHandler.LastRequest!.Method.Method.Should().Be("CUSTOM");
}
[Fact]
public async Task SendAsyncAppliesBodyAndContentTypeAsync()
{
// Arrange
TestHttpMessageHandler messageHandler = new((req, _) =>
Task.FromResult(new HttpResponseMessage(HttpStatusCode.OK)));
await using DefaultHttpRequestHandler handler = new((_, _) => Task.FromResult<HttpClient?>(new HttpClient(messageHandler)));
HttpRequestInfo request = new()
{
Method = "POST",
Url = TestUrl,
Body = "{\"hello\":\"world\"}",
BodyContentType = "application/json",
};
// Act
await handler.SendAsync(request);
// Assert
messageHandler.LastRequestBody.Should().Be("{\"hello\":\"world\"}");
messageHandler.LastRequestContentType.Should().Be("application/json");
}
[Fact]
public async Task SendAsyncAppliesRequestHeadersAsync()
{
// Arrange
TestHttpMessageHandler messageHandler = new((req, _) =>
Task.FromResult(new HttpResponseMessage(HttpStatusCode.OK)));
await using DefaultHttpRequestHandler handler = new((_, _) => Task.FromResult<HttpClient?>(new HttpClient(messageHandler)));
HttpRequestInfo request = new()
{
Method = "GET",
Url = TestUrl,
Headers = new Dictionary<string, string>
{
["Authorization"] = "Bearer secret",
["Accept"] = "application/json",
},
};
// Act
await handler.SendAsync(request);
// Assert
messageHandler.LastRequest!.Headers.Authorization!.ToString().Should().Be("Bearer secret");
messageHandler.LastRequest.Headers.Accept.Should().Contain(mediaType => mediaType.MediaType == "application/json");
}
[Fact]
public async Task SendAsyncRoutesContentHeadersToBodyAsync()
{
// Arrange
TestHttpMessageHandler messageHandler = new((req, _) =>
Task.FromResult(new HttpResponseMessage(HttpStatusCode.OK)));
await using DefaultHttpRequestHandler handler = new((_, _) => Task.FromResult<HttpClient?>(new HttpClient(messageHandler)));
HttpRequestInfo request = new()
{
Method = "POST",
Url = TestUrl,
Body = "raw",
BodyContentType = "text/plain",
Headers = new Dictionary<string, string>
{
["Content-Language"] = "en-US",
},
};
// Act
await handler.SendAsync(request);
// Assert
messageHandler.LastRequest!.Content!.Headers.ContentLanguage.Should().Contain("en-US");
}
[Fact]
public async Task SendAsyncCapturesResponseHeadersAsync()
{
// Arrange
TestHttpMessageHandler messageHandler = new((req, _) =>
{
#pragma warning disable CA2025
HttpResponseMessage response = new(HttpStatusCode.OK)
{
Content = new StringContent("ok", Encoding.UTF8, "text/plain"),
};
response.Headers.Add("X-Request-Id", "request-1");
response.Headers.Add("Set-Cookie", s_setCookieValues);
return Task.FromResult(response);
#pragma warning restore CA2025
});
await using DefaultHttpRequestHandler handler = new((_, _) => Task.FromResult<HttpClient?>(new HttpClient(messageHandler)));
HttpRequestInfo request = new() { Method = "GET", Url = TestUrl };
// Act
HttpRequestResult result = await handler.SendAsync(request);
// Assert
result.Headers.Should().NotBeNull();
result.Headers!.Should().ContainKey("X-Request-Id");
result.Headers!["Set-Cookie"].Should().BeEquivalentTo(s_setCookieValues);
// Content headers also flattened in.
result.Headers!.Should().ContainKey("Content-Type");
}
[Fact]
public async Task SendAsyncReturnsFailureStatusWithoutThrowingAsync()
{
// Arrange
TestHttpMessageHandler messageHandler = new((req, _) =>
Task.FromResult(new HttpResponseMessage(HttpStatusCode.BadRequest)
{
Content = new StringContent("bad request", Encoding.UTF8, "text/plain"),
}));
await using DefaultHttpRequestHandler handler = new((_, _) => Task.FromResult<HttpClient?>(new HttpClient(messageHandler)));
HttpRequestInfo request = new() { Method = "GET", Url = TestUrl };
// Act
HttpRequestResult result = await handler.SendAsync(request);
// Assert
result.IsSuccessStatusCode.Should().BeFalse();
result.StatusCode.Should().Be(400);
result.Body.Should().Be("bad request");
}
[Fact]
public async Task SendAsyncTimeoutCancelsRequestAsync()
{
// Arrange
TestHttpMessageHandler messageHandler = new(async (req, ct) =>
{
await Task.Delay(TimeSpan.FromSeconds(5), ct).ConfigureAwait(false);
return new HttpResponseMessage(HttpStatusCode.OK);
});
await using DefaultHttpRequestHandler handler = new((_, _) => Task.FromResult<HttpClient?>(new HttpClient(messageHandler)));
HttpRequestInfo request = new()
{
Method = "GET",
Url = TestUrl,
Timeout = TimeSpan.FromMilliseconds(50),
};
// Act
Func<Task> act = async () => await handler.SendAsync(request);
// Assert
await act.Should().ThrowAsync<OperationCanceledException>();
}
[Fact]
public async Task SendAsyncFallsBackToOwnedClientWhenProviderReturnsNullAsync()
{
// Arrange
int providerCallCount = 0;
await using DefaultHttpRequestHandler handler = new((_, _) =>
{
providerCallCount++;
return Task.FromResult<HttpClient?>(null);
});
HttpRequestInfo request = new() { Method = "GET", Url = "http://127.0.0.1:1/" };
// Act - owned client will attempt real network and fail, but provider path should have been consulted first.
Func<Task> act = async () => await handler.SendAsync(request);
// Assert
await act.Should().ThrowAsync<Exception>();
providerCallCount.Should().Be(1);
}
#endregion
#region DisposeAsync
[Fact]
public async Task DisposeAsyncCompletesAsync()
{
// Arrange
DefaultHttpRequestHandler handler = new();
// Act
Func<Task> act = async () => await handler.DisposeAsync();
// Assert
await act.Should().NotThrowAsync();
}
[Fact]
public async Task DisposeAsyncCalledMultipleTimesSucceedsAsync()
{
// Arrange
DefaultHttpRequestHandler handler = new();
// Act
await handler.DisposeAsync();
Func<Task> second = async () => await handler.DisposeAsync();
// Assert
await second.Should().NotThrowAsync();
}
#endregion
#region Query Parameters and Connection Tests
[Fact]
public async Task QueryParametersAreAppendedToUrlAsync()
{
// Arrange
TestHttpMessageHandler fake = new(static (req, _) =>
Task.FromResult(new HttpResponseMessage(HttpStatusCode.OK) { Content = new StringContent(string.Empty) }));
await using DefaultHttpRequestHandler handler = new((_, _) => Task.FromResult<HttpClient?>(new HttpClient(fake)));
HttpRequestInfo info = new()
{
Method = "GET",
Url = TestUrl,
QueryParameters = new Dictionary<string, string>
{
["filter"] = "active items",
["ids"] = "1,2,3",
},
};
// Act
await handler.SendAsync(info);
// Assert
fake.LastRequest.Should().NotBeNull();
string? query = fake.LastRequest!.RequestUri!.Query;
query.Should().Contain("filter=active%20items");
query.Should().Contain("ids=1%2C2%2C3");
}
[Fact]
public async Task QueryParametersPreserveExistingQueryStringAsync()
{
// Arrange
TestHttpMessageHandler fake = new(static (req, _) =>
Task.FromResult(new HttpResponseMessage(HttpStatusCode.OK) { Content = new StringContent(string.Empty) }));
await using DefaultHttpRequestHandler handler = new((_, _) => Task.FromResult<HttpClient?>(new HttpClient(fake)));
HttpRequestInfo info = new()
{
Method = "GET",
Url = TestUrl + "?existing=yes",
QueryParameters = new Dictionary<string, string>
{
["added"] = "true",
},
};
// Act
await handler.SendAsync(info);
// Assert
fake.LastRequest!.RequestUri!.Query.Should().Be("?existing=yes&added=true");
}
#endregion
private sealed class TestHttpMessageHandler : HttpMessageHandler
{
private readonly Func<HttpRequestMessage, CancellationToken, Task<HttpResponseMessage>> _responseFactory;
public TestHttpMessageHandler(Func<HttpRequestMessage, CancellationToken, Task<HttpResponseMessage>> responseFactory)
{
this._responseFactory = responseFactory;
}
public HttpRequestMessage? LastRequest { get; private set; }
public string? LastRequestBody { get; private set; }
public string? LastRequestContentType { get; private set; }
protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
this.LastRequest = request;
if (request.Content is not null)
{
#if NET
this.LastRequestBody = await request.Content.ReadAsStringAsync(cancellationToken).ConfigureAwait(false);
#else
this.LastRequestBody = await request.Content.ReadAsStringAsync().ConfigureAwait(false);
#endif
this.LastRequestContentType = request.Content.Headers.ContentType?.MediaType;
}
return await this._responseFactory(request, cancellationToken).ConfigureAwait(false);
}
}
}
@@ -769,4 +769,165 @@ public sealed class ChatMessageExtensionsTests
break;
}
}
[Fact]
public void MergeForLastMessageReturnsInputWhenInputMessageIsNull()
{
// Arrange
ChatMessage input = new(ChatRole.User, "hello") { MessageId = "local" };
// Act
ChatMessage result = input.MergeForLastMessage(null);
// Assert
Assert.Same(input, result);
}
[Fact]
public void MergeForLastMessageReturnsSameInstanceAsRoundTripped()
{
// Arrange: returning the round-tripped instance keeps the merge forward-compatible
// with future ChatMessage properties (e.g., new metadata fields) without explicit copies.
ChatMessage input = new(ChatRole.User, "original");
ChatMessage roundTripped = new(ChatRole.User, "stripped") { MessageId = "server" };
// Act
ChatMessage result = input.MergeForLastMessage(roundTripped);
// Assert
Assert.Same(roundTripped, result);
}
[Fact]
public void MergeForLastMessagePrefersOriginalTextOverRoundTrippedText()
{
// Arrange
ChatMessage input = new(ChatRole.User, "original text");
ChatMessage roundTripped = new(ChatRole.User, "stripped") { MessageId = "server-id" };
// Act
ChatMessage result = input.MergeForLastMessage(roundTripped);
// Assert
Assert.Equal("server-id", result.MessageId);
Assert.Equal("original text", result.Text);
TextContent text = Assert.IsType<TextContent>(Assert.Single(result.Contents));
Assert.Equal("original text", text.Text);
}
[Fact]
public void MergeForLastMessageReplacesTextInPlaceAndKeepsServerMedia()
{
// Arrange
HostedFileContent serverRef = new("file-abc");
ChatMessage input = new(ChatRole.User, [new TextContent("look at this:"), new DataContent("data:image/jpeg;base64,QUJD", "image/jpeg")]);
ChatMessage roundTripped = new(ChatRole.User, [new TextContent("stripped"), serverRef]) { MessageId = "server-id" };
// Act
ChatMessage result = input.MergeForLastMessage(roundTripped);
// Assert: server's text slot is replaced with original text; server's media reference is preserved.
Assert.Equal("server-id", result.MessageId);
Assert.Collection(result.Contents,
c => Assert.Equal("look at this:", Assert.IsType<TextContent>(c).Text),
c => Assert.Same(serverRef, c));
}
[Fact]
public void MergeForLastMessageAppendsOriginalTextWhenRoundTripHasNoTextSlot()
{
// Arrange: round-tripped message has only media (no text slot to replace).
HostedFileContent serverRef = new("file-1");
ChatMessage input = new(ChatRole.User, [new TextContent("middle"), new DataContent("data:image/jpeg;base64,QUE=", "image/jpeg")]);
ChatMessage roundTripped = new(ChatRole.User, [serverRef]) { MessageId = "id" };
// Act
ChatMessage result = input.MergeForLastMessage(roundTripped);
// Assert: media kept; original text appended at end.
Assert.Collection(result.Contents,
c => Assert.Same(serverRef, c),
c => Assert.Equal("middle", Assert.IsType<TextContent>(c).Text));
}
[Fact]
public void MergeForLastMessageReplacesMultipleTextSlotsInOrder()
{
// Arrange: input has two text items; round-tripped has two text slots interleaved with media.
HostedFileContent firstRef = new("file-1");
HostedFileContent secondRef = new("file-2");
ChatMessage input = new(ChatRole.User, [new TextContent("first"), new TextContent("second")]);
ChatMessage roundTripped = new(ChatRole.User, [firstRef, new TextContent("a"), secondRef, new TextContent("b")]) { MessageId = "id" };
// Act
ChatMessage result = input.MergeForLastMessage(roundTripped);
// Assert
Assert.Collection(result.Contents,
c => Assert.Same(firstRef, c),
c => Assert.Equal("first", Assert.IsType<TextContent>(c).Text),
c => Assert.Same(secondRef, c),
c => Assert.Equal("second", Assert.IsType<TextContent>(c).Text));
}
[Fact]
public void MergeForLastMessageFallsBackToInputTextWhenInputHasNoTextContent()
{
// Arrange: ChatMessage(role, "string") populates Text but no explicit TextContent
// when Contents is initially empty in some construction paths. Verify we still
// recover the original Text via input.Text.
ChatMessage input = new(ChatRole.User, "fallback text");
ChatMessage roundTripped = new(ChatRole.User, [new TextContent("stripped")]) { MessageId = "id" };
// Act
ChatMessage result = input.MergeForLastMessage(roundTripped);
// Assert
Assert.Equal("fallback text", Assert.IsType<TextContent>(Assert.Single(result.Contents)).Text);
}
[Fact]
public void MergeForLastMessagePreservesServerAuthoredProperties()
{
// Arrange: server (round-trip) is authoritative for metadata. Returning the
// round-tripped instance means any future ChatMessage property is automatically
// preserved without code changes here.
ChatMessage input = new(ChatRole.User, "hi")
{
AuthorName = "client-side",
AdditionalProperties = new AdditionalPropertiesDictionary { ["client"] = "value" },
};
ChatMessage roundTripped = new(ChatRole.User, [new TextContent("stripped")])
{
MessageId = "server",
AuthorName = "server-side",
AdditionalProperties = new AdditionalPropertiesDictionary { ["server"] = "value" },
};
// Act
ChatMessage result = input.MergeForLastMessage(roundTripped);
// Assert
Assert.Equal("server", result.MessageId);
Assert.Equal("server-side", result.AuthorName);
Assert.NotNull(result.AdditionalProperties);
Assert.True(result.AdditionalProperties.ContainsKey("server"));
Assert.False(result.AdditionalProperties.ContainsKey("client"));
}
[Fact]
public void MergeForLastMessageHandlesEmptyInputContents()
{
// Arrange
ChatMessage input = new(ChatRole.User, new List<AIContent>());
HostedFileContent serverRef = new("file-only");
ChatMessage roundTripped = new(ChatRole.User, [serverRef]) { MessageId = "id" };
// Act
ChatMessage result = input.MergeForLastMessage(roundTripped);
// Assert: nothing to splice; round-tripped returned unchanged.
Assert.Same(roundTripped, result);
Assert.Equal("file-only", Assert.IsType<HostedFileContent>(Assert.Single(result.Contents)).FileId);
}
}
@@ -0,0 +1,759 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Agents.AI.Workflows.Declarative.ObjectModel;
using Microsoft.Agents.AI.Workflows.Declarative.PowerFx;
using Microsoft.Agents.ObjectModel;
using Microsoft.Extensions.AI;
using Microsoft.PowerFx.Types;
using Moq;
namespace Microsoft.Agents.AI.Workflows.Declarative.UnitTests.ObjectModel;
/// <summary>
/// Tests for <see cref="HttpRequestExecutor"/>.
/// </summary>
public sealed class HttpRequestExecutorTest(ITestOutputHelper output) : WorkflowActionExecutorTest(output)
{
private const string TestUrl = "https://api.example.com/data";
private readonly Mock<ResponseAgentProvider> _agentProvider = new(MockBehavior.Loose);
[Fact]
public void InvalidModel()
{
// Arrange
Mock<IHttpRequestHandler> mockHandler = new();
// Act & Assert
Assert.Throws<DeclarativeModelException>(() => new HttpRequestExecutor(
new HttpRequestAction(),
mockHandler.Object,
this._agentProvider.Object,
this.State));
}
[Fact]
public void HttpRequestIsDiscreteAction()
{
// Arrange
Mock<IHttpRequestHandler> mockHandler = new();
HttpRequestAction model = this.CreateModel(
displayName: nameof(HttpRequestIsDiscreteAction),
url: TestUrl,
method: HttpMethodType.Get);
HttpRequestExecutor action = new(model, mockHandler.Object, this._agentProvider.Object, this.State);
// Act & Assert — IsDiscreteAction should be true for HttpRequest (single-step action).
VerifyIsDiscrete(action, isDiscrete: true);
}
[Fact]
public async Task HttpGetReturnsJsonObjectAsync()
{
// Arrange
this.State.InitializeSystem();
const string ResponseVar = "Result";
HttpRequestAction model = this.CreateModel(
displayName: nameof(HttpGetReturnsJsonObjectAsync),
url: TestUrl,
method: HttpMethodType.Get,
responseVariable: ResponseVar);
MockHttpRequestHandler handler = new(HttpRequestResult("{\"key\":\"value\",\"number\":42}"));
HttpRequestExecutor action = new(model, handler.Object, this._agentProvider.Object, this.State);
// Act
await this.ExecuteAsync(action);
// Assert
VerifyModel(model, action);
Assert.IsType<RecordValue>(this.State.Get(ResponseVar), exactMatch: false);
handler.VerifySent(info => info.Method == "GET" && info.Url == TestUrl);
}
[Fact]
public async Task HttpGetReturnsPlainStringAsync()
{
// Arrange
this.State.InitializeSystem();
const string ResponseVar = "Result";
HttpRequestAction model = this.CreateModel(
displayName: nameof(HttpGetReturnsPlainStringAsync),
url: TestUrl,
method: HttpMethodType.Get,
responseVariable: ResponseVar);
MockHttpRequestHandler handler = new(HttpRequestResult("not-json content"));
HttpRequestExecutor action = new(model, handler.Object, this._agentProvider.Object, this.State);
// Act
await this.ExecuteAsync(action);
// Assert
VerifyModel(model, action);
this.VerifyState(ResponseVar, FormulaValue.New("not-json content"));
}
[Fact]
public async Task HttpGetWithEmptyBodyYieldsBlankAsync()
{
// Arrange
this.State.InitializeSystem();
const string ResponseVar = "Result";
HttpRequestAction model = this.CreateModel(
displayName: nameof(HttpGetWithEmptyBodyYieldsBlankAsync),
url: TestUrl,
method: HttpMethodType.Get,
responseVariable: ResponseVar);
MockHttpRequestHandler handler = new(HttpRequestResult(null));
HttpRequestExecutor action = new(model, handler.Object, this._agentProvider.Object, this.State);
// Act
await this.ExecuteAsync(action);
// Assert
VerifyModel(model, action);
this.VerifyUndefined(ResponseVar);
}
[Fact]
public async Task HttpGetForwardsHeadersAsync()
{
// Arrange
this.State.InitializeSystem();
HttpRequestAction model = this.CreateModel(
displayName: nameof(HttpGetForwardsHeadersAsync),
url: TestUrl,
method: HttpMethodType.Get,
headers: new Dictionary<string, string>
{
["Authorization"] = "Bearer token",
["Accept"] = "application/json",
});
MockHttpRequestHandler handler = new(HttpRequestResult("{}"));
HttpRequestExecutor action = new(model, handler.Object, this._agentProvider.Object, this.State);
// Act
await this.ExecuteAsync(action);
// Assert
VerifyModel(model, action);
handler.VerifySent(info =>
info.Headers?["Authorization"] == "Bearer token" &&
info.Headers?["Accept"] == "application/json");
}
[Fact]
public async Task HttpPostWithJsonBodyAsync()
{
// Arrange
this.State.InitializeSystem();
HttpRequestAction model = this.CreateModel(
displayName: nameof(HttpPostWithJsonBodyAsync),
url: TestUrl,
method: HttpMethodType.Post,
jsonBody: new StringDataValue("hello"));
MockHttpRequestHandler handler = new(HttpRequestResult("{}"));
HttpRequestExecutor action = new(model, handler.Object, this._agentProvider.Object, this.State);
// Act
await this.ExecuteAsync(action);
// Assert
VerifyModel(model, action);
handler.VerifySent(info =>
info.Method == "POST" &&
info.BodyContentType == "application/json" &&
info.Body == "\"hello\"");
}
[Fact]
public async Task HttpPostWithRawBodyAsync()
{
// Arrange
this.State.InitializeSystem();
HttpRequestAction model = this.CreateModel(
displayName: nameof(HttpPostWithRawBodyAsync),
url: TestUrl,
method: HttpMethodType.Post,
rawBody: "raw body content",
rawContentType: "text/plain");
MockHttpRequestHandler handler = new(HttpRequestResult(""));
HttpRequestExecutor action = new(model, handler.Object, this._agentProvider.Object, this.State);
// Act
await this.ExecuteAsync(action);
// Assert
VerifyModel(model, action);
handler.VerifySent(info =>
info.BodyContentType == "text/plain" &&
info.Body == "raw body content");
}
[Fact]
public async Task HttpRequestRaisesOnErrorByDefaultAsync()
{
// Arrange
this.State.InitializeSystem();
HttpRequestAction model = this.CreateModel(
displayName: nameof(HttpRequestRaisesOnErrorByDefaultAsync),
url: TestUrl,
method: HttpMethodType.Get);
MockHttpRequestHandler handler = new(HttpRequestResult("server error", statusCode: 500, isSuccess: false));
HttpRequestExecutor action = new(model, handler.Object, this._agentProvider.Object, this.State);
// Act & Assert
await Assert.ThrowsAsync<DeclarativeActionException>(() => this.ExecuteAsync(action));
}
[Fact]
public async Task HttpRequestFailureExceptionTruncatesLongBodyAsync()
{
// Arrange
this.State.InitializeSystem();
HttpRequestAction model = this.CreateModel(
displayName: nameof(HttpRequestFailureExceptionTruncatesLongBodyAsync),
url: TestUrl,
method: HttpMethodType.Get);
string longBody = new('x', 10_000);
MockHttpRequestHandler handler = new(HttpRequestResult(longBody, statusCode: 500, isSuccess: false));
HttpRequestExecutor action = new(model, handler.Object, this._agentProvider.Object, this.State);
// Act
DeclarativeActionException exception =
await Assert.ThrowsAsync<DeclarativeActionException>(() => this.ExecuteAsync(action));
// Assert - message contains status and truncation marker, bounded in length, never the full body.
Assert.Contains("500", exception.Message);
Assert.Contains("[truncated]", exception.Message);
Assert.DoesNotContain(longBody, exception.Message);
Assert.True(exception.Message.Length < 512, $"Exception message too long: {exception.Message.Length} chars.");
}
[Fact]
public async Task HttpRequestFailureExceptionOmitsEmptyBodyAsync()
{
// Arrange
this.State.InitializeSystem();
HttpRequestAction model = this.CreateModel(
displayName: nameof(HttpRequestFailureExceptionOmitsEmptyBodyAsync),
url: TestUrl,
method: HttpMethodType.Get);
MockHttpRequestHandler handler = new(HttpRequestResult(body: null, statusCode: 404, isSuccess: false));
HttpRequestExecutor action = new(model, handler.Object, this._agentProvider.Object, this.State);
// Act
DeclarativeActionException exception =
await Assert.ThrowsAsync<DeclarativeActionException>(() => this.ExecuteAsync(action));
// Assert - status present, no stray "Body: ''" noise.
Assert.Contains("404", exception.Message);
Assert.DoesNotContain("Body:", exception.Message);
}
[Fact]
public async Task HttpRequestFailureExceptionSanitizesControlCharsAsync()
{
// Arrange
this.State.InitializeSystem();
HttpRequestAction model = this.CreateModel(
displayName: nameof(HttpRequestFailureExceptionSanitizesControlCharsAsync),
url: TestUrl,
method: HttpMethodType.Get);
MockHttpRequestHandler handler = new(HttpRequestResult("line1\r\nline2\tend", statusCode: 400, isSuccess: false));
HttpRequestExecutor action = new(model, handler.Object, this._agentProvider.Object, this.State);
// Act
DeclarativeActionException exception =
await Assert.ThrowsAsync<DeclarativeActionException>(() => this.ExecuteAsync(action));
// Assert - CR/LF/TAB collapsed to spaces so the message stays on one line.
Assert.DoesNotContain("\r", exception.Message);
Assert.DoesNotContain("\n", exception.Message);
Assert.DoesNotContain("\t", exception.Message);
Assert.Contains("line1", exception.Message);
Assert.Contains("line2", exception.Message);
}
[Fact]
public async Task HttpRequestPassesTimeoutToHandlerAsync()
{
// Arrange
this.State.InitializeSystem();
HttpRequestAction model = this.CreateModel(
displayName: nameof(HttpRequestPassesTimeoutToHandlerAsync),
url: TestUrl,
method: HttpMethodType.Get,
timeoutMilliseconds: 1500);
MockHttpRequestHandler handler = new(HttpRequestResult("{}"));
HttpRequestExecutor action = new(model, handler.Object, this._agentProvider.Object, this.State);
// Act
await this.ExecuteAsync(action);
// Assert
VerifyModel(model, action);
handler.VerifySent(info =>
info.Timeout is not null &&
info.Timeout.Value == TimeSpan.FromMilliseconds(1500));
}
[Fact]
public async Task HttpRequestTimeoutRaisesDeclarativeExceptionAsync()
{
// Arrange
this.State.InitializeSystem();
HttpRequestAction model = this.CreateModel(
displayName: nameof(HttpRequestTimeoutRaisesDeclarativeExceptionAsync),
url: TestUrl,
method: HttpMethodType.Get);
MockHttpRequestHandler handler = new(
HttpRequestResult("{}"),
throwOnSend: new OperationCanceledException());
HttpRequestExecutor action = new(model, handler.Object, this._agentProvider.Object, this.State);
// Act & Assert
await Assert.ThrowsAsync<DeclarativeActionException>(() => this.ExecuteAsync(action));
}
[Fact]
public async Task HttpRequestTransportFailureRaisesDeclarativeExceptionAsync()
{
// Arrange
this.State.InitializeSystem();
HttpRequestAction model = this.CreateModel(
displayName: nameof(HttpRequestTransportFailureRaisesDeclarativeExceptionAsync),
url: TestUrl,
method: HttpMethodType.Get);
MockHttpRequestHandler handler = new(
HttpRequestResult("{}"),
throwOnSend: new InvalidOperationException("transport failure"));
HttpRequestExecutor action = new(model, handler.Object, this._agentProvider.Object, this.State);
// Act & Assert
await Assert.ThrowsAsync<DeclarativeActionException>(() => this.ExecuteAsync(action));
}
[Fact]
public async Task HttpRequestStoresResponseHeadersAsync()
{
// Arrange
this.State.InitializeSystem();
const string HeaderVar = "Headers";
HttpRequestAction model = this.CreateModel(
displayName: nameof(HttpRequestStoresResponseHeadersAsync),
url: TestUrl,
method: HttpMethodType.Get,
responseHeadersVariable: HeaderVar);
Dictionary<string, IReadOnlyList<string>> responseHeaders = new(StringComparer.OrdinalIgnoreCase)
{
["X-Request-Id"] = ["abc-123"],
["Set-Cookie"] = ["a=1", "b=2"],
};
MockHttpRequestHandler handler = new(HttpRequestResult("{}", headers: responseHeaders));
HttpRequestExecutor action = new(model, handler.Object, this._agentProvider.Object, this.State);
// Act
await this.ExecuteAsync(action);
// Assert
VerifyModel(model, action);
FormulaValue storedHeaders = this.State.Get(HeaderVar);
Assert.IsType<RecordValue>(storedHeaders, exactMatch: false);
}
[Fact]
public async Task HttpRequestForwardsQueryParametersAsync()
{
// Arrange
this.State.InitializeSystem();
HttpRequestAction model = this.CreateModel(
displayName: nameof(HttpRequestForwardsQueryParametersAsync),
url: TestUrl,
method: HttpMethodType.Get,
queryParameters: new Dictionary<string, DataValue>
{
["filter"] = StringDataValue.Create("active"),
["limit"] = NumberDataValue.Create(10),
["includeDeleted"] = BooleanDataValue.Create(false),
});
MockHttpRequestHandler handler = new(HttpRequestResult("{}"));
HttpRequestExecutor action = new(model, handler.Object, this._agentProvider.Object, this.State);
// Act
await this.ExecuteAsync(action);
// Assert
VerifyModel(model, action);
handler.VerifySent(info =>
info.QueryParameters?.Count == 3 &&
info.QueryParameters["filter"] == "active" &&
info.QueryParameters["limit"] == "10" &&
info.QueryParameters["includeDeleted"] == "false");
}
[Fact]
public async Task HttpRequestAddsResponseToConversationAsync()
{
// Arrange
this.State.InitializeSystem();
const string ConversationId = "conv-12345";
const string ResponseBody = "response-text";
this._agentProvider
.Setup(p => p.CreateMessageAsync(It.IsAny<string>(), It.IsAny<ChatMessage>(), It.IsAny<CancellationToken>()))
.Returns<string, ChatMessage, CancellationToken>((_, message, _) => Task.FromResult(message));
HttpRequestAction model = this.CreateModel(
displayName: nameof(HttpRequestAddsResponseToConversationAsync),
url: TestUrl,
method: HttpMethodType.Get,
conversationId: ConversationId);
MockHttpRequestHandler handler = new(HttpRequestResult(ResponseBody));
HttpRequestExecutor action = new(model, handler.Object, this._agentProvider.Object, this.State);
// Act
await this.ExecuteAsync(action);
// Assert
VerifyModel(model, action);
this._agentProvider.Verify(
p => p.CreateMessageAsync(
ConversationId,
It.Is<ChatMessage>(m => m.Role == ChatRole.Assistant && m.Text == ResponseBody),
It.IsAny<CancellationToken>()),
Times.Once);
}
[Fact]
public async Task HttpRequestWithoutConversationIdSkipsConversationAsync()
{
// Arrange
this.State.InitializeSystem();
HttpRequestAction model = this.CreateModel(
displayName: nameof(HttpRequestWithoutConversationIdSkipsConversationAsync),
url: TestUrl,
method: HttpMethodType.Get);
MockHttpRequestHandler handler = new(HttpRequestResult("response"));
HttpRequestExecutor action = new(model, handler.Object, this._agentProvider.Object, this.State);
// Act
await this.ExecuteAsync(action);
// Assert
VerifyModel(model, action);
this._agentProvider.Verify(
p => p.CreateMessageAsync(It.IsAny<string>(), It.IsAny<ChatMessage>(), It.IsAny<CancellationToken>()),
Times.Never);
}
[Fact]
public async Task HttpRequestForwardsConnectionNameAsync()
{
// Arrange
this.State.InitializeSystem();
const string ConnectionName = "my-connection";
HttpRequestAction model = this.CreateModel(
displayName: nameof(HttpRequestForwardsConnectionNameAsync),
url: TestUrl,
method: HttpMethodType.Get,
connectionName: ConnectionName);
MockHttpRequestHandler handler = new(HttpRequestResult("{}"));
HttpRequestExecutor action = new(model, handler.Object, this._agentProvider.Object, this.State);
// Act
await this.ExecuteAsync(action);
// Assert
VerifyModel(model, action);
handler.VerifySent(info => info.ConnectionName == ConnectionName);
}
[Fact]
public async Task HttpRequestEmptyConversationIdSkipsConversationAsync()
{
// Arrange - empty-string conversationId should be treated as unset.
this.State.InitializeSystem();
HttpRequestAction model = this.CreateModel(
displayName: nameof(HttpRequestEmptyConversationIdSkipsConversationAsync),
url: TestUrl,
method: HttpMethodType.Get,
conversationId: "");
MockHttpRequestHandler handler = new(HttpRequestResult("response"));
HttpRequestExecutor action = new(model, handler.Object, this._agentProvider.Object, this.State);
// Act
await this.ExecuteAsync(action);
// Assert
VerifyModel(model, action);
this._agentProvider.Verify(
p => p.CreateMessageAsync(It.IsAny<string>(), It.IsAny<ChatMessage>(), It.IsAny<CancellationToken>()),
Times.Never);
}
[Fact]
public async Task HttpRequestEmptyResponseBodySkipsConversationAsync()
{
// Arrange - conversationId set, but empty body should not produce a conversation message.
this.State.InitializeSystem();
HttpRequestAction model = this.CreateModel(
displayName: nameof(HttpRequestEmptyResponseBodySkipsConversationAsync),
url: TestUrl,
method: HttpMethodType.Get,
conversationId: "conv-1");
MockHttpRequestHandler handler = new(HttpRequestResult(""));
HttpRequestExecutor action = new(model, handler.Object, this._agentProvider.Object, this.State);
// Act
await this.ExecuteAsync(action);
// Assert
VerifyModel(model, action);
this._agentProvider.Verify(
p => p.CreateMessageAsync(It.IsAny<string>(), It.IsAny<ChatMessage>(), It.IsAny<CancellationToken>()),
Times.Never);
}
[Fact]
public async Task HttpGetReturnsJsonArrayAsync()
{
// Arrange - exercises JsonValueKind.Array branch of ParseResponseBody.
this.State.InitializeSystem();
const string ResponseVar = "Result";
HttpRequestAction model = this.CreateModel(
displayName: nameof(HttpGetReturnsJsonArrayAsync),
url: TestUrl,
method: HttpMethodType.Get,
responseVariable: ResponseVar);
MockHttpRequestHandler handler = new(HttpRequestResult("[1, 2, 3]"));
HttpRequestExecutor action = new(model, handler.Object, this._agentProvider.Object, this.State);
// Act
await this.ExecuteAsync(action);
// Assert
VerifyModel(model, action);
FormulaValue stored = this.State.Get(ResponseVar);
Assert.IsType<TableValue>(stored, exactMatch: false);
}
[Fact]
public async Task HttpGetWithEmptyHeaderValueDropsHeaderAsync()
{
// Arrange - empty header values should be filtered out (matches GetHeaders guard).
this.State.InitializeSystem();
HttpRequestAction model = this.CreateModel(
displayName: nameof(HttpGetWithEmptyHeaderValueDropsHeaderAsync),
url: TestUrl,
method: HttpMethodType.Get,
headers: new Dictionary<string, string>
{
["X-Trace"] = "trace-1",
["X-Empty"] = "",
});
MockHttpRequestHandler handler = new(HttpRequestResult("{}"));
HttpRequestExecutor action = new(model, handler.Object, this._agentProvider.Object, this.State);
// Act
await this.ExecuteAsync(action);
// Assert
VerifyModel(model, action);
handler.VerifySent(info =>
info.Headers?.ContainsKey("X-Trace") == true &&
info.Headers?.ContainsKey("X-Empty") == false);
}
[Fact]
public async Task HttpRequestZeroTimeoutNotForwardedAsync()
{
// Arrange - non-positive timeouts should not be forwarded (handler default applies).
this.State.InitializeSystem();
HttpRequestAction model = this.CreateModel(
displayName: nameof(HttpRequestZeroTimeoutNotForwardedAsync),
url: TestUrl,
method: HttpMethodType.Get,
timeoutMilliseconds: 0);
MockHttpRequestHandler handler = new(HttpRequestResult("{}"));
HttpRequestExecutor action = new(model, handler.Object, this._agentProvider.Object, this.State);
// Act
await this.ExecuteAsync(action);
// Assert
VerifyModel(model, action);
handler.VerifySent(info => info.Timeout is null);
}
private static HttpRequestResult HttpRequestResult(
string? body,
int statusCode = 200,
bool isSuccess = true,
IReadOnlyDictionary<string, IReadOnlyList<string>>? headers = null) =>
new()
{
StatusCode = statusCode,
IsSuccessStatusCode = isSuccess,
Body = body,
Headers = headers,
};
private HttpRequestAction CreateModel(
string displayName,
string url,
HttpMethodType method,
string? responseVariable = null,
string? responseHeadersVariable = null,
IReadOnlyDictionary<string, string>? headers = null,
IReadOnlyDictionary<string, DataValue>? queryParameters = null,
string? conversationId = null,
string? connectionName = null,
DataValue? jsonBody = null,
string? rawBody = null,
string? rawContentType = null,
long? timeoutMilliseconds = null,
string? continueOnErrorStatusVariable = null,
string? continueOnErrorBodyVariable = null)
{
HttpRequestAction.Builder builder = new()
{
Id = this.CreateActionId(),
DisplayName = this.FormatDisplayName(displayName),
Url = new StringExpression.Builder(StringExpression.Literal(url)),
Method = new EnumExpression<HttpMethodTypeWrapper>.Builder(
EnumExpression<HttpMethodTypeWrapper>.Literal(HttpMethodTypeWrapper.Get(method))),
};
if (responseVariable is not null)
{
builder.Response = PropertyPath.Create(FormatVariablePath(responseVariable));
}
if (responseHeadersVariable is not null)
{
builder.ResponseHeaders = PropertyPath.Create(FormatVariablePath(responseHeadersVariable));
}
if (headers is not null)
{
foreach (KeyValuePair<string, string> header in headers)
{
builder.Headers.Add(header.Key, new StringExpression.Builder(StringExpression.Literal(header.Value)));
}
}
if (queryParameters is not null)
{
foreach (KeyValuePair<string, DataValue> parameter in queryParameters)
{
builder.QueryParameters.Add(parameter.Key, new ValueExpression.Builder(ValueExpression.Literal(parameter.Value)));
}
}
if (conversationId is not null)
{
builder.ConversationId = new StringExpression.Builder(StringExpression.Literal(conversationId));
}
if (connectionName is not null)
{
builder.Connection = new RemoteConnection.Builder
{
Name = new StringExpression.Builder(StringExpression.Literal(connectionName)),
};
}
if (jsonBody is not null)
{
builder.Body = new JsonRequestContent.Builder()
{
Content = new ValueExpression.Builder(ValueExpression.Literal(jsonBody)),
};
}
else if (rawBody is not null)
{
RawRequestContent.Builder rawBuilder = new()
{
Content = new StringExpression.Builder(StringExpression.Literal(rawBody)),
};
if (rawContentType is not null)
{
rawBuilder.ContentType = new StringExpression.Builder(StringExpression.Literal(rawContentType));
}
builder.Body = rawBuilder;
}
if (timeoutMilliseconds is not null)
{
builder.RequestTimeoutInMilliseconds = new IntExpression.Builder(IntExpression.Literal(timeoutMilliseconds.Value));
}
if (continueOnErrorStatusVariable is not null || continueOnErrorBodyVariable is not null)
{
ContinueOnErrorBehavior.Builder continueBuilder = new();
if (continueOnErrorStatusVariable is not null)
{
continueBuilder.StatusCode = PropertyPath.Create(FormatVariablePath(continueOnErrorStatusVariable));
}
if (continueOnErrorBodyVariable is not null)
{
continueBuilder.ErrorResponseBody = PropertyPath.Create(FormatVariablePath(continueOnErrorBodyVariable));
}
builder.ErrorHandling = continueBuilder;
}
return AssignParent<HttpRequestAction>(builder);
}
private sealed class MockHttpRequestHandler : Mock<IHttpRequestHandler>
{
private HttpRequestInfo? _lastRequest;
public MockHttpRequestHandler(HttpRequestResult result, Exception? throwOnSend = null)
{
this.Setup(handler => handler.SendAsync(It.IsAny<HttpRequestInfo>(), It.IsAny<CancellationToken>()))
.Returns<HttpRequestInfo, CancellationToken>((info, _) =>
{
this._lastRequest = info;
if (throwOnSend is not null)
{
throw throwOnSend;
}
return Task.FromResult(result);
});
}
public void VerifySent(Func<HttpRequestInfo, bool> predicate)
{
Assert.NotNull(this._lastRequest);
Assert.True(predicate(this._lastRequest!), "Sent HTTP request did not match expected predicate.");
}
}
}
@@ -0,0 +1,15 @@
kind: Workflow
trigger:
kind: OnConversationStart
id: my_workflow
actions:
- kind: HttpRequestAction
id: http_request
method: GET
url: =Concatenate("https://api.example.test/items/", System.LastMessageText)
headers:
Accept: application/json
response: Local.HttpResult
responseHeaders: Local.HttpHeaders
+1 -1
View File
@@ -61,7 +61,7 @@ repos:
additional_dependencies: ["bandit[toml]"]
- repo: https://github.com/astral-sh/uv-pre-commit
# uv version.
rev: 0.10.10
rev: 0.11.6
hooks:
# Update the uv lockfile
- id: uv-lock
+1
View File
@@ -69,6 +69,7 @@ python/
### Azure Integrations
- [foundry](packages/foundry/README.md) - Microsoft Foundry chat, agent, memory, and embedding integrations
- [azure-contentunderstanding](packages/azure-contentunderstanding/AGENTS.md) - Azure Content Understanding context provider
- [azure-ai-search](packages/azure-ai-search/AGENTS.md) - Azure AI Search RAG
- [azure-cosmos](packages/azure-cosmos/AGENTS.md) - Azure Cosmos DB-backed history provider
- [azurefunctions](packages/azurefunctions/AGENTS.md) - Azure Functions hosting
+64 -3
View File
@@ -7,6 +7,64 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
## [1.2.2] - 2026-04-29
### Added
- **agent-framework-azure-contentunderstanding**: New alpha package — Azure AI Content Understanding context provider that auto-analyzes file attachments (documents, images, audio, video) and injects structured results into the LLM context, with multi-document session state, configurable timeout, output filtering via `AnalysisSection`, and auto-registered `list_documents` / `get_analyzed_document` tools ([#4829](https://github.com/microsoft/agent-framework/pull/4829))
- **agent-framework-foundry-hosting**: Add hosted Durable Workflow support — propagate full conversation history to workflow agents and wire `Workflow.as_agent()` end-to-end via the foundry hosting layer ([#5531](https://github.com/microsoft/agent-framework/pull/5531))
### Changed
- **agent-framework-orchestrations**: [BREAKING] Standardize orchestration terminal outputs as `AgentResponse` so `Workflow.as_agent()` returns the final answer only; aligns sequential-approval (`with_request_info`) and concurrent (`intermediate_outputs=True`) flows on the same output contract ([#5301](https://github.com/microsoft/agent-framework/pull/5301))
- **agent-framework-core**, **agent-framework-declarative**: Preserve `Workflow.run()` shared state across calls so multi-turn `WorkflowAgent` invocations retain context, accept `list[Message]` input in the declarative start executor, and coerce `Enum` values when serializing PowerFx symbols ([#5531](https://github.com/microsoft/agent-framework/pull/5531))
- **dependencies**: Update workspace package dependencies and preserve `mcp[ws]` / `uvicorn[standard]` extras through override-dependencies in `/python` ([#5555](https://github.com/microsoft/agent-framework/pull/5555))
### Fixed
- **agent-framework-core**: Fix observability spans not being correctly nested when using streaming ([#5552](https://github.com/microsoft/agent-framework/pull/5552))
- **agent-framework-openai**: Fix `file_search` citations breaking the assistant-message history roundtrip — skip `hosted_file` content in the assistant role so the Responses API no longer rejects `input_file` ([#5557](https://github.com/microsoft/agent-framework/pull/5557))
## [1.2.1] - 2026-04-28
### Added
- **agent-framework-foundry-hosting**: Add file data type support to hosted-agent Responses, refresh `foundry-hosted-agents` samples, and add response test coverage ([#5485](https://github.com/microsoft/agent-framework/pull/5485))
- **samples**: Add `requirements.txt` and `.env.example` to the `a2a/` hosting sample for pip-based setup ([#5510](https://github.com/microsoft/agent-framework/pull/5510))
### Changed
- **dependencies**: Update `rich` requirement from `<15.0.0,>=13.7.1` to `>=13.7.1,<16.0.0` in `/python` ([#5227](https://github.com/microsoft/agent-framework/pull/5227))
- **dependencies**: Bump `prek` from `0.3.8` to `0.3.9` in `/python` ([#5228](https://github.com/microsoft/agent-framework/pull/5228))
- **dependencies**: Bump `python-multipart` from `0.0.22` to `0.0.26` in `/python` ([#5286](https://github.com/microsoft/agent-framework/pull/5286))
- **dependencies**: Bump `pyasn1` from `0.6.2` to `0.6.3` in `/python` ([#4748](https://github.com/microsoft/agent-framework/pull/4748))
- **dependencies**: Bump `pytest` from `9.0.2` to `9.0.3` in `/python/packages/ag-ui` ([#5461](https://github.com/microsoft/agent-framework/pull/5461))
- **dependencies**: Bump `pytest` from `9.0.2` to `9.0.3` in `/python/packages/devui` ([#5492](https://github.com/microsoft/agent-framework/pull/5492))
- **dependencies**: Bump `pytest` from `9.0.2` to `9.0.3` in `/python/packages/lab` ([#5470](https://github.com/microsoft/agent-framework/pull/5470))
- **dependencies**: Bump `uv` from `0.11.3` to `0.11.6` in `/python/packages/lab` ([#5469](https://github.com/microsoft/agent-framework/pull/5469))
- **dependencies**: Bump `vite` from `7.1.12` to `7.3.2` in `/python/packages/devui/frontend` ([#5127](https://github.com/microsoft/agent-framework/pull/5127))
- **dependencies**: Bump `vite` from `7.1.12` to `7.3.2` in `/python/samples/05-end-to-end/chatkit-integration/frontend` ([#5126](https://github.com/microsoft/agent-framework/pull/5126))
- **dependencies**: Bump `postcss` from `8.5.6` to `8.5.10` in `/python/packages/devui/frontend` ([#5484](https://github.com/microsoft/agent-framework/pull/5484))
- **dependencies**: Bump `postcss` from `8.5.6` to `8.5.10` in `/python/samples/05-end-to-end/chatkit-integration/frontend` ([#5491](https://github.com/microsoft/agent-framework/pull/5491))
- **dependencies**: Bump `postcss` from `8.5.6` to `8.5.12` in `/python/samples/05-end-to-end/ag_ui_workflow_handoff/frontend` ([#5527](https://github.com/microsoft/agent-framework/pull/5527))
- **dependencies**: Bump `picomatch` from `4.0.3` to `4.0.4` in `/python/packages/devui/frontend` ([#4921](https://github.com/microsoft/agent-framework/pull/4921))
- **dependencies**: Bump `picomatch` from `4.0.3` to `4.0.4` in `/python/samples/05-end-to-end/ag_ui_workflow_handoff/frontend` ([#4936](https://github.com/microsoft/agent-framework/pull/4936))
### Fixed
- **agent-framework-core**: Prevent `inner_exception` from being lost in `AgentFrameworkException` ([#5167](https://github.com/microsoft/agent-framework/pull/5167))
## [1.2.0] - 2026-04-24
### Added
- **agent-framework-core**: Add functional workflow API ([#4238](https://github.com/microsoft/agent-framework/pull/4238))
- **agent-framework-core**, **agent-framework-github-copilot**: Add OpenTelemetry integration for `GitHubCopilotAgent` ([#5142](https://github.com/microsoft/agent-framework/pull/5142))
- **agent-framework-a2a**: Add Agent Framework to A2A bridge support ([#2403](https://github.com/microsoft/agent-framework/pull/2403))
- **agent-framework-foundry**: Surface `oauth_consent_request` events from Responses API in Foundry clients ([#5070](https://github.com/microsoft/agent-framework/pull/5070))
### Changed
- **agent-framework-core**, **agent-framework-foundry**: Update `FoundryAgent` for hosted agent sessions ([#5447](https://github.com/microsoft/agent-framework/pull/5447))
- **agent-framework-foundry-hosting**: Upgrade hosting server dependency and add more type support ([#5459](https://github.com/microsoft/agent-framework/pull/5459))
### Fixed
- **agent-framework-ag-ui**: Fix reasoning role and multimodal media parsing to follow specification ([#5389](https://github.com/microsoft/agent-framework/pull/5389))
- **agent-framework-foundry**: Stop emitting `[TOOLBOXES]` warning for every `FoundryChatClient` call ([#5440](https://github.com/microsoft/agent-framework/pull/5440))
- **agent-framework-anthropic**, **agent-framework-azure-ai-search**, **agent-framework-azure-cosmos**: Fix user agent prefix ([#5455](https://github.com/microsoft/agent-framework/pull/5455))
## [1.1.1] - 2026-04-23
### Added
@@ -26,8 +84,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- **agent-framework-openai**: Exclude null `file_id` from `input_image` payload to prevent schema 400 errors ([#5125](https://github.com/microsoft/agent-framework/pull/5125))
- **agent-framework-foundry**: Reconcile Toolbox hosted-tool payloads with the Responses API ([#5414](https://github.com/microsoft/agent-framework/pull/5414))
- **agent-framework-ag-ui**: Pass client `thread_id` as `session_id` when constructing `AgentSession` ([#5384](https://github.com/microsoft/agent-framework/pull/5384))
- **agent-framework-hyperlight**: Thread-confine `WasmSandbox` interactions via per-entry `ThreadPoolExecutor` to eliminate the PyO3 `unsendable` panic when touched from asyncio worker threads
([#5424](https://github.com/microsoft/agent-framework/pull/5424))
- **agent-framework-hyperlight**: Thread-confine `WasmSandbox` interactions via per-entry `ThreadPoolExecutor` to eliminate the PyO3 `unsendable` panic when touched from asyncio worker threads ([#5424](https://github.com/microsoft/agent-framework/pull/5424))
## [1.1.0] - 2026-04-21
@@ -961,7 +1018,11 @@ Release candidate for **agent-framework-core** and **agent-framework-azure-ai**
For more information, see the [announcement blog post](https://devblogs.microsoft.com/foundry/introducing-microsoft-agent-framework-the-open-source-engine-for-agentic-ai-apps/).
[Unreleased]: https://github.com/microsoft/agent-framework/compare/python-1.1.0...HEAD
[Unreleased]: https://github.com/microsoft/agent-framework/compare/python-1.2.2...HEAD
[1.2.2]: https://github.com/microsoft/agent-framework/compare/python-1.2.1...python-1.2.2
[1.2.1]: https://github.com/microsoft/agent-framework/compare/python-1.2.0...python-1.2.1
[1.2.0]: https://github.com/microsoft/agent-framework/compare/python-1.1.1...python-1.2.0
[1.1.1]: https://github.com/microsoft/agent-framework/compare/python-1.1.0...python-1.1.1
[1.1.0]: https://github.com/microsoft/agent-framework/compare/python-1.0.1...python-1.1.0
[1.0.1]: https://github.com/microsoft/agent-framework/compare/python-1.0.0...python-1.0.1
[1.0.0]: https://github.com/microsoft/agent-framework/compare/python-1.0.0rc6...python-1.0.0
+1
View File
@@ -18,6 +18,7 @@ Status is grouped into these buckets:
| `agent-framework-a2a` | `python/packages/a2a` | `beta` |
| `agent-framework-ag-ui` | `python/packages/ag-ui` | `beta` |
| `agent-framework-anthropic` | `python/packages/anthropic` | `beta` |
| `agent-framework-azure-contentunderstanding` | `python/packages/azure-contentunderstanding` | `alpha` |
| `agent-framework-azure-ai-search` | `python/packages/azure-ai-search` | `beta` |
| `agent-framework-azure-cosmos` | `python/packages/azure-cosmos` | `beta` |
| `agent-framework-azurefunctions` | `python/packages/azurefunctions` | `beta` |
+2 -2
View File
@@ -4,7 +4,7 @@ description = "A2A integration for Microsoft Agent Framework."
authors = [{ name = "Microsoft", email = "af-support@microsoft.com"}]
readme = "README.md"
requires-python = ">=3.10"
version = "1.0.0b260423"
version = "1.0.0b260429"
license-files = ["LICENSE"]
urls.homepage = "https://aka.ms/agent-framework"
urls.source = "https://github.com/microsoft/agent-framework/tree/main/python"
@@ -23,7 +23,7 @@ classifiers = [
"Typing :: Typed",
]
dependencies = [
"agent-framework-core>=1.1.1,<2",
"agent-framework-core>=1.2.2,<2",
"a2a-sdk>=0.3.5,<0.3.24",
]
+3 -3
View File
@@ -1,6 +1,6 @@
[project]
name = "agent-framework-ag-ui"
version = "1.0.0b260423"
version = "1.0.0b260429"
description = "AG-UI protocol integration for Agent Framework"
readme = "README.md"
license-files = ["LICENSE"]
@@ -22,7 +22,7 @@ classifiers = [
"Typing :: Typed",
]
dependencies = [
"agent-framework-core>=1.1.1,<2",
"agent-framework-core>=1.2.2,<2",
"ag-ui-protocol>=0.1.16,<0.2",
"fastapi>=0.115.0,<0.133.1",
"uvicorn[standard]>=0.30.0,<0.42.0"
@@ -30,7 +30,7 @@ dependencies = [
[project.optional-dependencies]
dev = [
"pytest==9.0.2",
"pytest==9.0.3",
"httpx==0.28.1",
]
@@ -872,6 +872,8 @@ class RawAnthropicClient(
tool_mode = validate_tool_mode(options.get("tool_choice"))
if tool_mode is None:
return result or None
if "allowed_tools" in tool_mode:
logger.warning("allowed_tools is not supported by Anthropic; the setting will be ignored")
allow_multiple = options.get("allow_multiple_tool_calls")
match tool_mode.get("mode"):
case "auto":
+2 -2
View File
@@ -4,7 +4,7 @@ description = "Anthropic integration for Microsoft Agent Framework."
authors = [{ name = "Microsoft", email = "af-support@microsoft.com"}]
readme = "README.md"
requires-python = ">=3.10"
version = "1.0.0b260423"
version = "1.0.0b260429"
license-files = ["LICENSE"]
urls.homepage = "https://aka.ms/agent-framework"
urls.source = "https://github.com/microsoft/agent-framework/tree/main/python"
@@ -23,7 +23,7 @@ classifiers = [
"Typing :: Typed",
]
dependencies = [
"agent-framework-core>=1.1.1,<2",
"agent-framework-core>=1.2.2,<2",
"anthropic>=0.80.0,<0.80.1",
]
@@ -4,7 +4,7 @@ description = "Azure AI Search integration for Microsoft Agent Framework."
authors = [{ name = "Microsoft", email = "af-support@microsoft.com"}]
readme = "README.md"
requires-python = ">=3.10"
version = "1.0.0b260423"
version = "1.0.0b260429"
license-files = ["LICENSE"]
urls.homepage = "https://aka.ms/agent-framework"
urls.source = "https://github.com/microsoft/agent-framework/tree/main/python"
@@ -23,7 +23,7 @@ classifiers = [
"Typing :: Typed",
]
dependencies = [
"agent-framework-core>=1.1.1,<2",
"agent-framework-core>=1.2.2,<2",
"azure-search-documents>=11.7.0b2,<11.7.0b3",
]
@@ -0,0 +1,3 @@
# Local-only files (not committed)
_local_only/
*_local_only*
@@ -0,0 +1,71 @@
# AGENTS.md — azure-contentunderstanding
## Package Overview
`agent-framework-azure-contentunderstanding` integrates Azure Content Understanding (CU)
into the Agent Framework as a context provider. It automatically analyzes file attachments
(documents, images, audio, video) and injects structured results into the LLM context.
## Public API
| Symbol | Type | Description |
|--------|------|-------------|
| `ContentUnderstandingContextProvider` | class | Main context provider — extends `ContextProvider` |
| `AnalysisSection` | enum | Output section selector (MARKDOWN, FIELDS, etc.) |
| `DocumentStatus` | enum | Document lifecycle state (ANALYZING, UPLOADING, READY, FAILED) |
| `FileSearchBackend` | ABC | Abstract vector store file operations interface |
| `FileSearchConfig` | dataclass | Configuration for CU + vector store RAG mode |
## Architecture
- **`_context_provider.py`** — Main provider implementation. Overrides `before_run()` to detect
file attachments, call the CU API, manage session state with multi-document tracking,
and auto-register retrieval tools for follow-up turns.
- **Analyzer auto-detection** — When `analyzer_id=None` (default), `_resolve_analyzer_id()`
selects the CU analyzer based on media type prefix: `audio/` → `prebuilt-audioSearch`,
`video/` → `prebuilt-videoSearch`, everything else → `prebuilt-documentSearch`.
- **Multi-segment output** — CU splits long video/audio into multiple scene segments
(each a separate `contents[]` entry with its own `startTimeMs`, `endTimeMs`, `markdown`,
and `fields`). `_extract_sections()` produces:
- `segments`: list of per-segment dicts, each with `markdown`, `fields`, `start_time_s`, `end_time_s`
- `markdown`: concatenated at top level with `---` separators (for file_search uploads)
- `duration_seconds`: computed from global `min(startTimeMs)` → `max(endTimeMs)`
- Metadata (`kind`, `resolution`): taken from the first segment
- **Speaker diarization (not identification)** — CU transcripts label speakers as
`<Speaker 1>`, `<Speaker 2>`, etc. CU does **not** identify speakers by name.
- **file_search RAG** — When `FileSearchConfig` is provided, CU-extracted markdown is
uploaded to an OpenAI vector store and a `file_search` tool is registered on the context
instead of injecting the full document content. This enables token-efficient retrieval
for large documents.
- **`_models.py`** — `AnalysisSection` enum, `DocumentStatus` enum, `DocumentEntry` TypedDict,
`FileSearchConfig` dataclass.
- **`_file_search.py`** — `FileSearchBackend` ABC, `OpenAIFileSearchBackend`,
`FoundryFileSearchBackend`.
## Key Patterns
- Follows the Azure AI Search context provider pattern (same lifecycle, config style).
- Uses provider-scoped `state` dict for multi-document tracking across turns.
- Auto-registers `list_documents()` tool via `context.extend_tools()`.
- Configurable timeout (`max_wait`) with `asyncio.create_task()` background fallback.
- Strips supported binary attachments from `input_messages` to prevent LLM API errors.
- Explicit `analyzer_id` always overrides auto-detection (user preference wins).
- Vector store resources are cleaned up in `close()` / `__aexit__`.
## Samples
| Sample | Description |
|--------|-------------|
| `01_document_qa.py` | Upload a PDF via URL, ask questions about it |
| `02_multi_turn_session.py` | AgentSession persistence across turns |
| `03_multimodal_chat.py` | PDF + audio + video parallel analysis |
| `04_invoice_processing.py` | Structured field extraction with `prebuilt-invoice` analyzer |
| `05_large_doc_file_search.py` | CU extraction + OpenAI vector store RAG |
| `02-devui/01-multimodal_agent/` | DevUI web UI for CU-powered chat |
| `02-devui/02-file_search_agent/` | DevUI web UI combining CU + file_search RAG |
## Running Tests
```bash
uv run poe test -P azure-contentunderstanding
```
@@ -0,0 +1,21 @@
MIT License
Copyright (c) Microsoft Corporation.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE
@@ -0,0 +1,127 @@
# Get Started with Azure Content Understanding in Microsoft Agent Framework
Please install this package via pip:
```bash
pip install agent-framework-azure-contentunderstanding --pre
```
## Azure Content Understanding Integration
### Prerequisites
Before using this package, you need an Azure Content Understanding resource:
1. An active **Azure subscription** ([create one for free](https://azure.microsoft.com/pricing/purchase-options/azure-account))
2. A **Microsoft Foundry resource** created in a [supported region](https://learn.microsoft.com/azure/ai-services/content-understanding/language-region-support)
3. **Default model deployments** configured for your resource (GPT-4.1, GPT-4.1-mini, text-embedding-3-large)
Follow the [prerequisites section](https://learn.microsoft.com/azure/ai-services/content-understanding/quickstart/use-rest-api?tabs=portal%2Cdocument&pivots=programming-language-rest#prerequisites) in the Azure Content Understanding quickstart for setup instructions.
### Introduction
The Azure Content Understanding integration provides a context provider that automatically analyzes file attachments (documents, images, audio, video) using [Azure Content Understanding](https://learn.microsoft.com/azure/ai-services/content-understanding/) and injects structured results into the LLM context.
- **Document & image analysis**: State-of-the-art OCR with markdown extraction, table preservation, and structured field extraction — handles scanned PDFs, handwritten content, and complex layouts
- **Audio & video analysis**: Transcription, speaker diarization, and per-segment summaries
- **Background processing**: Configurable timeout with async background fallback for large files
- **file_search integration**: Optional vector store upload for token-efficient RAG on large documents
> Learn more about Azure Content Understanding capabilities at [https://learn.microsoft.com/azure/ai-services/content-understanding/](https://learn.microsoft.com/azure/ai-services/content-understanding/)
### Basic Usage Example
See the [samples directory](samples/) which demonstrates:
- Single PDF upload and Q&A ([01_document_qa](samples/01-get-started/01_document_qa.py))
- Multi-turn sessions with cached results ([02_multi_turn_session](samples/01-get-started/02_multi_turn_session.py))
- PDF + audio + video parallel analysis ([03_multimodal_chat](samples/01-get-started/03_multimodal_chat.py))
- Structured field extraction with prebuilt-invoice ([04_invoice_processing](samples/01-get-started/04_invoice_processing.py))
- CU extraction + OpenAI vector store RAG ([05_large_doc_file_search](samples/01-get-started/05_large_doc_file_search.py))
- Interactive web UI with DevUI ([02-devui](samples/02-devui/))
```python
import asyncio
from agent_framework import Agent, AgentSession, Message, Content
from agent_framework.foundry import FoundryChatClient
from agent_framework.foundry import ContentUnderstandingContextProvider
from azure.identity import AzureCliCredential
credential = AzureCliCredential()
cu = ContentUnderstandingContextProvider(
endpoint="https://my-resource.cognitiveservices.azure.com/",
credential=credential,
max_wait=None, # block until CU extraction completes before sending to LLM
)
client = FoundryChatClient(
project_endpoint="https://your-project.services.ai.azure.com",
model="gpt-4.1",
credential=credential,
)
async def main():
async with cu:
agent = Agent(
client=client,
name="DocumentQA",
instructions="You are a helpful document analyst.",
context_providers=[cu],
)
session = AgentSession()
response = await agent.run(
Message(role="user", contents=[
Content.from_text("What's on this invoice?"),
Content.from_uri(
"https://raw.githubusercontent.com/Azure-Samples/"
"azure-ai-content-understanding-assets/main/document/invoice.pdf",
media_type="application/pdf",
additional_properties={"filename": "invoice.pdf"},
),
]),
session=session,
)
print(response.text)
asyncio.run(main())
```
### Supported File Types
| Category | Types |
|----------|-------|
| Documents | PDF, DOCX, XLSX, PPTX, HTML, TXT, Markdown |
| Images | JPEG, PNG, TIFF, BMP |
| Audio | WAV, MP3, M4A, FLAC, OGG |
| Video | MP4, MOV, AVI, WebM |
For the complete list of supported file types and size limits, see [Azure Content Understanding service limits](https://learn.microsoft.com/azure/ai-services/content-understanding/service-limits#input-file-limits).
### Environment Variables
The provider supports automatic endpoint resolution from environment variables.
When ``endpoint`` is not passed to the constructor, it is loaded from
``AZURE_CONTENTUNDERSTANDING_ENDPOINT``:
```python
# Endpoint auto-loaded from AZURE_CONTENTUNDERSTANDING_ENDPOINT env var
cu = ContentUnderstandingContextProvider(credential=credential)
```
Set these in your shell or in a `.env` file:
```bash
AZURE_CONTENTUNDERSTANDING_ENDPOINT=https://your-cu-resource.cognitiveservices.azure.com/
AZURE_AI_PROJECT_ENDPOINT=https://your-project.services.ai.azure.com
AZURE_OPENAI_DEPLOYMENT_NAME=gpt-4.1
```
You also need to be logged in with `az login` (for `AzureCliCredential`).
### Next steps
- Explore the [samples directory](samples/) for complete code examples
- Read the [Azure Content Understanding documentation](https://learn.microsoft.com/azure/ai-services/content-understanding/) for detailed service information
- Learn more about the [Microsoft Agent Framework](https://aka.ms/agent-framework)
@@ -0,0 +1,28 @@
# Copyright (c) Microsoft. All rights reserved.
"""Azure Content Understanding integration for Microsoft Agent Framework.
Provides a context provider that analyzes file attachments (documents, images,
audio, video) using Azure Content Understanding and injects structured results
into the LLM context.
"""
import importlib.metadata
from ._context_provider import ContentUnderstandingContextProvider
from ._file_search import FileSearchBackend
from ._models import AnalysisSection, DocumentStatus, FileSearchConfig
try:
__version__ = importlib.metadata.version(__name__)
except importlib.metadata.PackageNotFoundError:
__version__ = "0.0.0"
__all__ = [
"AnalysisSection",
"ContentUnderstandingContextProvider",
"DocumentStatus",
"FileSearchBackend",
"FileSearchConfig",
"__version__",
]
@@ -0,0 +1,858 @@
# Copyright (c) Microsoft. All rights reserved.
"""Azure Content Understanding context provider using ContextProvider.
This module provides ``ContentUnderstandingContextProvider``, built on the
:class:`ContextProvider` hooks pattern. It automatically detects file
attachments, analyzes them via the Azure Content Understanding API, and
injects structured results into the LLM context.
"""
from __future__ import annotations
import asyncio
import json
import logging
import sys
import time
from datetime import datetime, timezone
from typing import TYPE_CHECKING, Any, ClassVar, TypedDict
from agent_framework import (
AGENT_FRAMEWORK_USER_AGENT,
Content,
ContextProvider,
FunctionTool,
Message,
SessionContext,
)
from agent_framework._sessions import AgentSession
from agent_framework._settings import load_settings
from azure.ai.contentunderstanding.aio import ContentUnderstandingClient
from azure.ai.contentunderstanding.models import AnalysisInput, AnalysisResult
from azure.core.credentials import AzureKeyCredential
from azure.core.credentials_async import AsyncTokenCredential
if TYPE_CHECKING:
from agent_framework._agents import SupportsAgentRun
from ._detection import (
detect_and_strip_files,
)
from ._extraction import extract_sections, format_result
from ._models import AnalysisSection, DocumentEntry, DocumentStatus, FileSearchConfig
if sys.version_info >= (3, 11):
from typing import Self # pragma: no cover
else:
from typing_extensions import Self # pragma: no cover
logger = logging.getLogger("agent_framework.azure_contentunderstanding")
AzureCredentialTypes = AzureKeyCredential | AsyncTokenCredential
# Mapping from media type prefix to the appropriate prebuilt CU analyzer.
# Used when analyzer_id is None (auto-detect mode).
MEDIA_TYPE_ANALYZER_MAP: dict[str, str] = {
"audio/": "prebuilt-audioSearch",
"video/": "prebuilt-videoSearch",
}
DEFAULT_ANALYZER: str = "prebuilt-documentSearch"
class ContentUnderstandingSettings(TypedDict, total=False):
"""Settings for ContentUnderstandingContextProvider with auto-loading from environment.
Settings are resolved in this order: explicit keyword arguments, values from an
explicitly provided .env file, then environment variables with the prefix
``AZURE_CONTENTUNDERSTANDING_``.
Keys:
endpoint: Azure AI Foundry endpoint URL.
Can be set via environment variable ``AZURE_CONTENTUNDERSTANDING_ENDPOINT``.
"""
endpoint: str | None
class ContentUnderstandingContextProvider(ContextProvider):
"""Context provider that analyzes file attachments using Azure Content Understanding.
Automatically detects supported file attachments in the agent's input,
analyzes them via CU, and injects the structured results (markdown, fields)
into the LLM context. Supports multiple documents per session with background
processing for long-running analyses. Optionally integrates with a vector
store backend for ``file_search``-based RAG retrieval on LLM clients that
support it.
Args:
endpoint: Azure AI Foundry endpoint URL
(e.g., ``"https://<your-foundry-resource>.services.ai.azure.com/"``).
Can also be set via environment variable
``AZURE_CONTENTUNDERSTANDING_ENDPOINT``.
credential: An ``AzureKeyCredential`` for API key auth or an
``AsyncTokenCredential`` (e.g., ``DefaultAzureCredential``) for
Microsoft Entra ID auth.
analyzer_id: A prebuilt or custom CU analyzer ID. When ``None``
(default), a prebuilt analyzer is chosen automatically based on
the file's media type: ``prebuilt-documentSearch`` for documents
and images, ``prebuilt-audioSearch`` for audio, and
``prebuilt-videoSearch`` for video.
Analyzer reference: https://learn.microsoft.com/azure/ai-services/content-understanding/concepts/analyzer-reference
Prebuilt analyzers: https://learn.microsoft.com/azure/ai-services/content-understanding/concepts/prebuilt-analyzers
max_wait: Max seconds to wait for analysis before deferring to background.
``None`` waits until complete.
output_sections: Which CU output sections to pass to LLM.
Defaults to ``["markdown", "fields"]``.
file_search: Optional configuration for uploading CU-extracted markdown to
a vector store for token-efficient RAG retrieval. When provided, full
content injection is replaced by ``file_search`` tool registration.
The ``FileSearchConfig`` abstraction is backend-agnostic — use
``FileSearchConfig.from_openai()`` or ``FileSearchConfig.from_foundry()``
for supported providers, or supply a custom ``FileSearchBackend``
implementation for other vector store services.
source_id: Unique identifier for this provider instance, used for message
attribution and tool registration. Defaults to ``"azure_contentunderstanding"``.
env_file_path: Path to a ``.env`` file for loading settings.
env_file_encoding: Encoding of the ``.env`` file.
Per-file ``additional_properties`` on ``Content`` objects:
The provider reads the following keys from
``Content.additional_properties`` (passed via ``Content.from_data()``
or ``Content.from_uri()``):
``filename`` (str):
The document key used for tracking, status, and LLM references.
Without a filename, a UUID-based key is generated.
Must be unique within a session — uploading a file with a
duplicate filename will be rejected and the file will not be
analyzed.
``analyzer_id`` (str):
Per-file analyzer override. Takes priority over the provider-level
``analyzer_id``. Useful for mixing analyzers in the same turn
(e.g., ``prebuilt-invoice`` for invoices alongside
``prebuilt-documentSearch`` for general documents).
``content_range`` (str):
Subset of the input to analyze. For documents, use 1-based page
numbers (e.g., ``"1-3"`` for pages 1-3, ``"1,3,5-"`` for pages
1, 3, and 5 onward). For audio/video, use milliseconds
(e.g., ``"0-60000"`` for the first 60 seconds).
Example::
Content.from_data(
pdf_bytes,
"application/pdf",
additional_properties={
"filename": "invoice.pdf",
"analyzer_id": "prebuilt-invoice",
"content_range": "1-3",
},
)
"""
DEFAULT_SOURCE_ID: ClassVar[str] = "azure_contentunderstanding"
DEFAULT_MAX_WAIT_SECONDS: ClassVar[float] = 5.0
def __init__(
self,
*,
endpoint: str | None = None,
credential: AzureCredentialTypes | None = None,
client: ContentUnderstandingClient | None = None,
analyzer_id: str | None = None,
max_wait: float | None = DEFAULT_MAX_WAIT_SECONDS,
output_sections: list[AnalysisSection] | None = None,
file_search: FileSearchConfig | None = None,
source_id: str = DEFAULT_SOURCE_ID,
env_file_path: str | None = None,
env_file_encoding: str | None = None,
) -> None:
super().__init__(source_id)
if client is not None:
# Use the pre-built client directly — endpoint/credential are ignored.
self._client = client
self._owns_client = False
self._endpoint = ""
self._credential = None
else:
# Build a new client from endpoint + credential.
settings = load_settings(
ContentUnderstandingSettings,
env_prefix="AZURE_CONTENTUNDERSTANDING_",
required_fields=["endpoint"],
endpoint=endpoint,
env_file_path=env_file_path,
env_file_encoding=env_file_encoding,
)
resolved_endpoint: str = settings["endpoint"] # type: ignore[assignment] # validated by load_settings
if credential is None:
raise ValueError(
"Azure credential is required. Provide a 'credential' keyword argument "
"(e.g., AzureKeyCredential or AzureCliCredential), or pass a pre-built "
"'client' (ContentUnderstandingClient) instead."
)
self._endpoint = resolved_endpoint
self._credential = credential
self._client = ContentUnderstandingClient(
self._endpoint, self._credential, user_agent=AGENT_FRAMEWORK_USER_AGENT
)
self._owns_client = True
self.analyzer_id = analyzer_id
self.max_wait = max_wait
self.output_sections: list[AnalysisSection] = output_sections or ["markdown", "fields"]
self.file_search = file_search
# Global list of uploaded file IDs — used only by close() for
# best-effort cleanup. The authoritative per-session copy lives in
# state["_uploaded_file_ids"] (populated in before_run). This global
# list may contain entries from multiple sessions; that is intentional
# for cleanup.
self._all_uploaded_file_ids: list[str] = []
async def __aenter__(self) -> Self:
"""Async context manager entry."""
return self
async def __aexit__(
self,
exc_type: type[BaseException] | None,
exc_val: BaseException | None,
exc_tb: Any,
) -> None:
"""Async context manager exit — cleanup clients."""
await self.close()
async def close(self) -> None:
"""Close the underlying CU client and clean up resources.
Uses global tracking lists for best-effort cleanup across all
sessions that used this provider instance.
"""
# Clean up uploaded files; the vector store itself is caller-managed.
if self.file_search and self._all_uploaded_file_ids:
await self._cleanup_uploaded_files()
# Only close the client if we created it internally.
# When a pre-built client was passed in, the caller owns its lifecycle.
if self._owns_client:
await self._client.close()
async def before_run(
self,
*,
agent: SupportsAgentRun,
session: AgentSession,
context: SessionContext,
state: dict[str, Any],
) -> None:
"""Analyze file attachments and inject results into the LLM context.
This method is called automatically by the framework before each LLM invocation.
"""
documents: dict[str, DocumentEntry] = state.setdefault("documents", {})
# Per-session mutable state — isolated per session to prevent cross-session leakage.
# _pending_tokens stores serializable continuation tokens (not asyncio.Task objects)
# so that state can be persisted to disk/storage by the framework.
# Structure: {doc_key: {"continuation_token": <opaque Azure SDK string>,
# "analyzer_id": <CU analyzer used for this file>}}
pending_tokens: dict[str, dict[str, str]] = state.setdefault("_pending_tokens", {})
pending_uploads: list[tuple[str, DocumentEntry]] = state.setdefault("_pending_uploads", [])
# 1. Resolve pending background analyses via continuation tokens
await self._resolve_pending_tokens(pending_tokens, pending_uploads, documents, context)
# 1b. Upload any documents that completed in the background (file_search mode)
if pending_uploads:
# Use a bounded timeout so before_run() stays responsive and does not block
# indefinitely on slow vector store indexing.
upload_timeout = getattr(self, "max_wait", None)
remaining_uploads: list[tuple[str, DocumentEntry]] = []
for upload_key, upload_entry in pending_uploads:
try:
if upload_timeout is not None:
await asyncio.wait_for(
self._upload_to_vector_store(upload_key, upload_entry, state=state),
timeout=upload_timeout,
)
else:
await self._upload_to_vector_store(upload_key, upload_entry, state=state)
except asyncio.TimeoutError:
# Leave timed-out uploads pending so they can be retried on a later turn.
logger.warning(
"Timed out while uploading document '%s' to vector store; will retry later.",
upload_key,
)
remaining_uploads.append((upload_key, upload_entry))
except Exception:
# Log unexpected failures and drop the upload entry; this matches prior
# behavior where all pending uploads were cleared regardless of outcome.
logger.exception(
"Error while uploading document '%s' to vector store; dropping from pending list.",
upload_key,
)
context.extend_messages(
self.source_id,
[
Message(
role="user",
contents=[
(
f"Document '{upload_key}' was analyzed but failed to upload "
"to the vector store. The document content is not available for search."
)
],
)
],
)
state["_pending_uploads"] = remaining_uploads
pending_uploads = remaining_uploads
# 2. Detect CU-supported file attachments, strip them from input, and return for analysis
new_files = detect_and_strip_files(context)
# 3. Analyze new files using CU (track elapsed time for combined timeout)
file_start_times: dict[str, float] = {}
accepted_keys: set[str] = set() # doc_keys successfully accepted for analysis this turn
for doc_key, content_item, binary_data in new_files:
# Reject duplicate filenames — re-analyzing would orphan vector store entries
if doc_key in documents:
logger.warning("Duplicate document key '%s' — skipping (already exists in session).", doc_key)
context.extend_messages(
self.source_id,
[
Message(
role="user",
contents=[
(
f"The user tried to upload '{doc_key}', but a file with that name "
"was already uploaded earlier in this session. The new upload was rejected "
"and was not analyzed. Tell the user that a file with the same name "
"already exists and they need to rename the file before uploading again."
)
],
)
],
)
continue
file_start_times[doc_key] = time.monotonic()
doc_entry = await self._analyze_file(doc_key, content_item, binary_data, context, pending_tokens)
if doc_entry:
documents[doc_key] = doc_entry
accepted_keys.add(doc_key)
# 4. Inject content for ready documents and register tools
if documents:
self._register_tools(documents, context)
# 5. On upload turns, inject content for docs accepted this turn
for doc_key in accepted_keys:
entry = documents.get(doc_key)
if entry and entry["status"] == DocumentStatus.READY and entry["result"]:
# Upload to vector store if file_search is configured
if self.file_search:
# Combined timeout: subtract CU analysis time from max_wait
remaining: float | None = None
if self.max_wait is not None:
elapsed = time.monotonic() - file_start_times.get(doc_key, time.monotonic())
remaining = max(0.0, self.max_wait - elapsed)
uploaded = await self._upload_to_vector_store(doc_key, entry, timeout=remaining, state=state)
if uploaded:
context.extend_messages(
self.source_id,
[
Message(
role="user",
contents=[
(
f"The user just uploaded '{entry['filename']}'. It has been analyzed "
"using Azure Content Understanding and indexed in a vector store. "
f"When using file_search, include '{entry['filename']}' in your query "
"to retrieve content from this specific document."
)
],
)
],
)
elif entry.get("error"):
# Upload failed (not timeout — actual error)
context.extend_messages(
self.source_id,
[
Message(
role="user",
contents=[
(
f"Document '{entry['filename']}' was analyzed but failed to upload "
"to the vector store. The document content is not available for search."
)
],
)
],
)
else:
# Upload deferred to background (timeout)
context.extend_messages(
self.source_id,
[
Message(
role="user",
contents=[
(
f"Document '{entry['filename']}' has been analyzed and is being indexed. "
"Ask about it again in a moment."
)
],
)
],
)
else:
# Without file_search, inject full content into context
context.extend_messages(
self,
[
Message(role="user", contents=[format_result(entry["filename"], entry["result"])]),
],
)
context.extend_messages(
self.source_id,
[
Message(
role="user",
contents=[
(
f"The user just uploaded '{entry['filename']}'."
" It has been analyzed using Azure Content Understanding."
" The document content (markdown) and extracted fields"
" (JSON) are provided above."
" If the user's question is ambiguous,"
" prioritize this most recently uploaded document."
" Use specific field values and cite page numbers"
" when answering."
)
],
)
],
)
# 6. Register file_search tool (for LLM clients that support it)
if self.file_search:
context.extend_tools(
self.source_id,
[self.file_search.file_search_tool],
)
context.extend_instructions(
self.source_id,
"Tool usage guidelines:\n"
"- Use file_search ONLY when answering questions about document content.\n"
"- Use list_documents() for status queries (e.g. 'list docs', 'what's uploaded?').\n"
"- Do NOT call file_search for status queries — it wastes tokens.",
)
# ------------------------------------------------------------------
# Analyzer Resolution
# ------------------------------------------------------------------
def _resolve_analyzer_id(self, media_type: str) -> str:
"""Return the analyzer ID to use for the given media type.
When ``self.analyzer_id`` is set, it is always returned (explicit
override). Otherwise the media type prefix is matched against the
known mapping, falling back to ``prebuilt-documentSearch``.
"""
if self.analyzer_id is not None:
return self.analyzer_id
for prefix, analyzer in MEDIA_TYPE_ANALYZER_MAP.items():
if media_type.startswith(prefix):
return analyzer
return DEFAULT_ANALYZER
# ------------------------------------------------------------------
# Analysis
# ------------------------------------------------------------------
async def _analyze_file(
self,
doc_key: str,
content: Content,
binary_data: bytes | None,
context: SessionContext,
pending_tokens: dict[str, dict[str, str]] | None = None,
) -> DocumentEntry | None:
"""Analyze a single file via CU with timeout handling.
The analyzer is resolved in priority order:
1. Per-file override via ``content.additional_properties["analyzer_id"]``
2. Provider-level default via ``self.analyzer_id``
3. Auto-detect by media type (document/audio/video)
Returns:
A ``DocumentEntry`` (ready, analyzing, or failed), or ``None`` if
file data could not be extracted.
"""
media_type = content.media_type or "application/octet-stream"
filename = doc_key
# Per-file analyzer override from additional_properties
props = content.additional_properties or {}
per_file_analyzer = props.get("analyzer_id")
content_range = props.get("content_range")
resolved_analyzer = per_file_analyzer or self._resolve_analyzer_id(media_type)
t0 = time.monotonic()
try:
# Start CU analysis
if content.type == "uri" and content.uri and not content.uri.startswith("data:"):
poller = await self._client.begin_analyze(
resolved_analyzer,
inputs=[AnalysisInput(url=content.uri, content_range=content_range)],
)
elif binary_data:
poller = await self._client.begin_analyze_binary(
resolved_analyzer,
binary_input=binary_data,
content_type=media_type,
)
else:
context.extend_messages(
self.source_id,
[Message(role="user", contents=[f"Could not extract file data from '{filename}'."])],
)
return None
# Wait with timeout; defer to background polling on timeout.
try:
result = await asyncio.wait_for(poller.result(), timeout=self.max_wait)
except asyncio.TimeoutError:
# Save continuation token for resuming on next before_run().
# Continuation tokens are serializable strings, so state can
# be persisted to disk/storage without issues.
token = poller.continuation_token()
logger.info("Analysis of '%s' timed out; deferring to background via continuation token.", filename)
if pending_tokens is not None:
pending_tokens[doc_key] = {
"continuation_token": token,
"analyzer_id": resolved_analyzer,
}
context.extend_messages(
self.source_id,
[
Message(
role="user",
contents=[f"Document '{filename}' is being analyzed. Ask about it again in a moment."],
)
],
)
return DocumentEntry(
status=DocumentStatus.ANALYZING,
filename=filename,
media_type=media_type,
analyzer_id=resolved_analyzer,
analyzed_at=None,
analysis_duration_s=None,
upload_duration_s=None,
result=None,
error=None,
)
# Analysis completed within timeout
analysis_duration = round(time.monotonic() - t0, 2)
extracted = self._extract_sections(result)
logger.info("Analyzed '%s' with analyzer '%s' in %.1fs.", filename, resolved_analyzer, analysis_duration)
return DocumentEntry(
status=DocumentStatus.READY,
filename=filename,
media_type=media_type,
analyzer_id=resolved_analyzer,
analyzed_at=datetime.now(tz=timezone.utc).isoformat(),
analysis_duration_s=analysis_duration,
upload_duration_s=None,
result=extracted,
error=None,
)
except asyncio.TimeoutError:
raise
except Exception as e:
logger.warning("CU analysis error for '%s': %s", filename, e)
context.extend_messages(
self.source_id,
[Message(role="user", contents=[f"Could not analyze '{filename}': {e}"])],
)
return DocumentEntry(
status=DocumentStatus.FAILED,
filename=filename,
media_type=media_type,
analyzer_id=resolved_analyzer,
analyzed_at=datetime.now(tz=timezone.utc).isoformat(),
analysis_duration_s=round(time.monotonic() - t0, 2),
upload_duration_s=None,
result=None,
error=str(e),
)
# ------------------------------------------------------------------
# Pending Token Resolution
# ------------------------------------------------------------------
async def _resolve_pending_tokens(
self,
pending_tokens: dict[str, dict[str, str]],
pending_uploads: list[tuple[str, DocumentEntry]],
documents: dict[str, DocumentEntry],
context: SessionContext,
) -> None:
"""Resume pending CU analyses using serializable continuation tokens.
When a file's CU analysis exceeds ``max_wait``, a continuation token
(an opaque string from the Azure SDK) is saved in ``state`` instead of
an ``asyncio.Task``. This keeps state fully serializable — it can be
persisted to disk/storage by the framework.
On the next ``before_run()`` call, this method resumes each pending
operation by passing the token back to ``begin_analyze()``. If the
server-side operation has completed, the result is available
immediately; otherwise the token is kept for the next turn.
"""
if not pending_tokens:
return
logger.info("Resolving %d pending analysis token(s).", len(pending_tokens))
completed_keys: list[str] = []
for doc_key, token_info in pending_tokens.items():
entry = documents.get(doc_key)
if not entry:
completed_keys.append(doc_key)
continue
try:
poller = await self._client.begin_analyze( # type: ignore[call-overload, reportUnknownVariableType]
token_info["analyzer_id"],
continuation_token=token_info["continuation_token"], # pyright: ignore[reportCallIssue]
)
# Use wait_for to avoid blocking before_run indefinitely.
# poller.done() always returns False for resumed pollers (stale
# cached status), so we call poller.result() which polls the server.
#
# Timeout: at least 10s regardless of max_wait. The upload-turn
# max_wait can be very short (e.g. 5s) for responsiveness, but
# on resolution turns the resumed poller needs a network round-trip
# to fetch the result. If the analysis is still running after 10s,
# the token is kept and retried on the next turn.
MIN_RESOLUTION_TIMEOUT = 10.0
resolution_timeout = max(self.max_wait or MIN_RESOLUTION_TIMEOUT, MIN_RESOLUTION_TIMEOUT)
try:
result: AnalysisResult = await asyncio.wait_for(
poller.result(), # pyright: ignore[reportUnknownMemberType, reportUnknownArgumentType]
timeout=resolution_timeout,
) # pyright: ignore[reportUnknownVariableType]
except asyncio.TimeoutError:
# Still running — update token and keep for next turn
new_token: str = poller.continuation_token() # pyright: ignore[reportUnknownMemberType, reportUnknownVariableType]
token_info["continuation_token"] = new_token
logger.info("Analysis for '%s' still running; keeping token for next turn.", doc_key)
continue
completed_keys.append(doc_key)
extracted = self._extract_sections(result) # pyright: ignore[reportUnknownArgumentType]
entry["status"] = DocumentStatus.READY
entry["analyzed_at"] = datetime.now(tz=timezone.utc).isoformat()
entry["result"] = extracted
entry["error"] = None
logger.info("Background analysis of '%s' completed.", entry["filename"])
# Inject newly ready content
if self.file_search:
pending_uploads.append((doc_key, entry))
else:
context.extend_messages(
self,
[
Message(role="user", contents=[format_result(entry["filename"], extracted)]),
],
)
context.extend_messages(
self.source_id,
[
Message(
role="user",
contents=[
f"Document '{entry['filename']}' analysis is now complete."
+ (
" The document is being indexed in the vector store and will become"
" searchable via file_search shortly."
if self.file_search
else " The content is provided above."
)
],
)
],
)
except Exception as e:
completed_keys.append(doc_key)
logger.warning("Background analysis of '%s' failed: %s", entry.get("filename", doc_key), e)
entry["status"] = DocumentStatus.FAILED
entry["analyzed_at"] = datetime.now(tz=timezone.utc).isoformat()
entry["error"] = str(e)
context.extend_messages(
self.source_id,
[Message(role="user", contents=[f"Document '{entry['filename']}' analysis failed: {e}"])],
)
for key in completed_keys:
del pending_tokens[key]
# ------------------------------------------------------------------
# Output Extraction & Formatting (delegates to _extraction module)
# ------------------------------------------------------------------
def _extract_sections(self, result: AnalysisResult) -> dict[str, object]:
return extract_sections(result, self.output_sections)
# ------------------------------------------------------------------
# Tool Registration
# ------------------------------------------------------------------
def _register_tools(
self,
documents: dict[str, DocumentEntry],
context: SessionContext,
) -> None:
"""Register document tools on the context.
Only ``list_documents`` is registered — the full document content is
already injected into conversation history on the upload turn, so a
separate retrieval tool is not needed.
"""
context.extend_tools(
self.source_id,
[self._make_list_documents_tool(documents)],
)
@staticmethod
def _make_list_documents_tool(documents: dict[str, DocumentEntry]) -> FunctionTool:
"""Create a tool that lists all tracked documents with their status."""
docs_ref = documents
def list_documents() -> str:
"""List all documents that have been uploaded and their analysis status."""
entries: list[dict[str, object]] = []
for name, entry in docs_ref.items():
entries.append({
"name": name,
"status": entry["status"],
"media_type": entry["media_type"],
"analyzed_at": entry["analyzed_at"],
"analysis_duration_s": entry["analysis_duration_s"],
"upload_duration_s": entry["upload_duration_s"],
})
return json.dumps(entries, indent=2, default=str)
return FunctionTool(
name="list_documents",
description=(
"List all documents that have been uploaded in this session "
"with their analysis status (analyzing, uploading, ready, or failed)."
),
func=list_documents,
)
# ------------------------------------------------------------------
# file_search Vector Store Integration
# ------------------------------------------------------------------
async def _upload_to_vector_store(
self,
doc_key: str,
entry: DocumentEntry,
*,
timeout: float | None = None,
state: dict[str, Any] | None = None,
) -> bool:
"""Upload CU-extracted markdown to the caller's vector store.
Delegates to the configured ``FileSearchBackend`` (OpenAI, Foundry,
or a custom implementation). The upload includes file upload **and**
vector store indexing (embedding + ingestion) — ``create_and_poll``
waits for the index to be fully ready before returning.
Args:
doc_key: Document identifier.
entry: The document entry with extracted results.
timeout: Max seconds to wait for upload + indexing. ``None`` waits
indefinitely. On timeout the upload is deferred to the
per-session ``_pending_uploads`` queue for the next
``before_run()`` call.
state: Per-session state dict for tracking uploaded file IDs and
pending uploads.
Returns:
True if the upload succeeded, False otherwise.
"""
if not self.file_search:
return False
result = entry.get("result")
if not result:
return False
# Upload the full formatted content (markdown + fields + segments),
# not just raw markdown — consistent with what non-file_search mode injects.
formatted = format_result(entry["filename"], result)
if not formatted:
return False
entry["status"] = DocumentStatus.UPLOADING
t0 = time.monotonic()
try:
upload_coro = self.file_search.backend.upload_file(
self.file_search.vector_store_id, f"{doc_key}.md", formatted.encode("utf-8")
)
file_id = await asyncio.wait_for(upload_coro, timeout=timeout)
upload_duration = round(time.monotonic() - t0, 2)
# Track in per-session state and global list (for close() cleanup)
if state is not None:
state.setdefault("_uploaded_file_ids", []).append(file_id)
self._all_uploaded_file_ids.append(file_id)
entry["status"] = DocumentStatus.READY
entry["upload_duration_s"] = upload_duration
logger.info("Uploaded '%s' to vector store in %.1fs (%s bytes).", doc_key, upload_duration, len(formatted))
return True
except asyncio.TimeoutError:
logger.info("Vector store upload for '%s' timed out; deferring to background.", doc_key)
entry["status"] = DocumentStatus.UPLOADING
if state is not None:
state.setdefault("_pending_uploads", []).append((doc_key, entry))
return False
except Exception as e:
logger.warning("Failed to upload '%s' to vector store: %s", doc_key, e)
entry["status"] = DocumentStatus.FAILED
entry["upload_duration_s"] = round(time.monotonic() - t0, 2)
entry["error"] = f"Vector store upload failed: {e}"
return False
async def _cleanup_uploaded_files(self) -> None:
"""Delete files uploaded by this provider via the configured backend.
The vector store itself is caller-managed and is not deleted here.
"""
if not self.file_search:
return
backend = self.file_search.backend
try:
for file_id in self._all_uploaded_file_ids:
await backend.delete_file(file_id)
self._all_uploaded_file_ids.clear()
except Exception as e:
logger.warning("Failed to clean up uploaded files: %s", e)
@@ -0,0 +1,234 @@
# Copyright (c) Microsoft. All rights reserved.
"""File detection utilities for Azure Content Understanding context provider.
Functions for scanning input messages, sniffing MIME types, deriving
document keys, and extracting binary data from content items.
"""
from __future__ import annotations
import base64
import logging
import mimetypes
import re
import uuid
import filetype
from agent_framework import Content, SessionContext
logger = logging.getLogger("agent_framework.azure_contentunderstanding")
# MIME types used to match against the resolved media type for routing files to CU analysis.
# The media type may be provided via Content.media_type or inferred (e.g., via sniffing or filename)
# when missing or generic (such as application/octet-stream). Only files whose resolved media type is
# in this set will be processed; others are skipped.
#
# Supported input file types:
# https://learn.microsoft.com/azure/ai-services/content-understanding/service-limits#input-file-limits
SUPPORTED_MEDIA_TYPES: frozenset[str] = frozenset({
# Documents and images
"application/pdf",
"image/jpeg",
"image/png",
"image/tiff",
"image/bmp",
"image/heif",
"image/heic",
"application/vnd.openxmlformats-officedocument.wordprocessingml.document",
"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
"application/vnd.openxmlformats-officedocument.presentationml.presentation",
# Text
"text/plain",
"text/html",
"text/markdown",
"text/rtf",
"text/xml",
"application/xml",
"message/rfc822",
"application/vnd.ms-outlook",
# Audio
"audio/wav",
"audio/mpeg",
"audio/mp3",
"audio/mp4",
"audio/m4a",
"audio/flac",
"audio/ogg",
"audio/opus",
"audio/webm",
"audio/x-ms-wma",
"audio/aac",
"audio/amr",
"audio/3gpp",
# Video
"video/mp4",
"video/quicktime",
"video/x-msvideo",
"video/webm",
"video/x-flv",
"video/x-ms-wmv",
"video/x-ms-asf",
"video/x-matroska",
})
# Mapping from filetype's MIME output to our canonical SUPPORTED_MEDIA_TYPES values.
# filetype uses some x-prefixed variants that differ from our set.
MIME_ALIASES: dict[str, str] = {
"audio/x-wav": "audio/wav",
"audio/x-flac": "audio/flac",
"video/x-m4v": "video/mp4",
}
def detect_and_strip_files(
context: SessionContext,
) -> list[tuple[str, Content, bytes | None]]:
"""Scan input messages for supported file content and prepare for CU analysis.
Scans for type ``data`` or ``uri`` content supported by Azure Content
Understanding, strips them from messages to prevent raw binary being sent
to the LLM, and returns metadata for CU analysis.
Detected files are tracked via ``doc_key`` (derived from filename, URL,
or UUID) and their analysis status is managed in session state.
When the upstream MIME type is unreliable (``application/octet-stream``
or missing), binary content sniffing via ``filetype`` is used to
determine the real media type, with ``mimetypes.guess_type`` as a
filename-based fallback.
Returns:
List of (doc_key, content_item, binary_data) tuples for files to analyze.
"""
results: list[tuple[str, Content, bytes | None]] = []
strip_ids: set[int] = set()
for msg in context.input_messages:
for c in msg.contents:
if c.type not in ("data", "uri"):
continue
media_type = c.media_type
# Fast path: already a known supported type
if media_type and media_type in SUPPORTED_MEDIA_TYPES:
binary_data = extract_binary(c)
results.append((derive_doc_key(c), c, binary_data))
strip_ids.add(id(c))
continue
# Slow path: unreliable MIME — sniff binary content
if (not media_type) or (media_type == "application/octet-stream"):
binary_data = extract_binary(c)
resolved = sniff_media_type(binary_data, c)
if resolved and (resolved in SUPPORTED_MEDIA_TYPES):
c.media_type = resolved
results.append((derive_doc_key(c), c, binary_data))
strip_ids.add(id(c))
# Strip detected files from input so raw binary isn't sent to LLM
msg.contents = [c for c in msg.contents if id(c) not in strip_ids]
return results
def sniff_media_type(binary_data: bytes | None, content: Content) -> str | None:
"""Sniff the actual MIME type from binary data, with filename fallback.
Uses ``filetype`` (magic-bytes) first, then ``mimetypes.guess_type``
on the filename. Normalizes filetype's variant MIME values (e.g.
``audio/x-wav`` -> ``audio/wav``) via ``MIME_ALIASES``.
"""
# 1. Binary sniffing via filetype (needs only first 261 bytes)
if binary_data:
kind = filetype.guess(binary_data[:262]) # type: ignore[reportUnknownMemberType]
if kind:
mime: str = kind.mime # type: ignore[reportUnknownMemberType]
return MIME_ALIASES.get(mime, mime)
# 2. Filename extension fallback — try additional_properties first,
# then extract basename from external URL path
filename: str | None = None
if content.additional_properties:
filename = content.additional_properties.get("filename")
if not filename and content.uri and not content.uri.startswith("data:"):
# Extract basename from URL path (e.g. "https://example.com/report.pdf?v=1" -> "report.pdf")
filename = content.uri.split("?")[0].split("#")[0].rsplit("/", 1)[-1]
if filename:
guessed, _ = mimetypes.guess_type(filename) # uses file extension to guess MIME type
if guessed:
return MIME_ALIASES.get(guessed, guessed)
return None
def is_supported_content(content: Content) -> bool:
"""Check if a content item is a supported file type for CU analysis."""
if content.type not in ("data", "uri"):
return False
media_type = content.media_type
if not media_type:
return False
return media_type in SUPPORTED_MEDIA_TYPES
def sanitize_doc_key(raw: str) -> str:
"""Sanitize a document key to prevent prompt injection.
Removes control characters (newlines, tabs, etc.), collapses
whitespace, strips surrounding whitespace, and caps length at
255 characters.
"""
# Remove control characters (C0/C1 controls, including \n, \r, \t)
cleaned = re.sub(r"[\x00-\x1f\x7f-\x9f]", "", raw)
# Collapse whitespace
cleaned = " ".join(cleaned.split())
# Cap length
return cleaned[:255] if cleaned else f"doc_{uuid.uuid4().hex[:8]}"
def derive_doc_key(content: Content) -> str:
"""Derive a unique document key from content metadata.
The key is used to track documents in session state. Duplicate keys
within a session are rejected (not re-analyzed) to prevent orphaned
vector store entries.
The returned key is sanitized to prevent prompt injection via
crafted filenames (control characters removed, length capped).
Priority: filename > URL basename > generated UUID.
"""
# 1. Filename from additional_properties
if content.additional_properties:
filename = content.additional_properties.get("filename")
if filename and isinstance(filename, str):
return sanitize_doc_key(filename)
# 2. URL path basename for external URIs (e.g. "https://example.com/report.pdf" -> "report.pdf")
if content.type == "uri" and content.uri and not content.uri.startswith("data:"):
path = content.uri.split("?")[0].split("#")[0] # strip query params and fragments
# rstrip("/") handles trailing slashes (e.g. ".../files/" -> ".../files")
# rsplit("/", 1)[-1] splits from the right once to get the last path segment
basename = path.rstrip("/").rsplit("/", 1)[-1]
if basename:
return sanitize_doc_key(basename)
# 3. Fallback: generate a unique ID for anonymous uploads (no filename, no URL)
return f"doc_{uuid.uuid4().hex[:8]}"
def extract_binary(content: Content) -> bytes | None:
"""Extract binary data from a data URI content item.
Only handles ``data:`` URIs (base64-encoded). Returns ``None`` for
external URLs -- those are passed directly to CU via ``begin_analyze``.
"""
if content.uri and content.uri.startswith("data:"):
try:
_, data_part = content.uri.split(",", 1)
return base64.b64decode(data_part)
except Exception:
logger.warning("Failed to decode base64 data URI")
return None
return None
@@ -0,0 +1,297 @@
# Copyright (c) Microsoft. All rights reserved.
"""Output extraction and formatting for Azure Content Understanding results.
Converts CU ``AnalysisResult`` objects into plain Python dicts suitable
for LLM consumption, and formats them as human-readable text.
"""
from __future__ import annotations
import json
from typing import Any, cast
from azure.ai.contentunderstanding.models import AnalysisResult
from ._models import AnalysisSection
def extract_sections(
result: AnalysisResult,
output_sections: list[AnalysisSection],
) -> dict[str, object]:
"""Extract configured sections from a CU analysis result.
For single-segment results (documents, images, short audio), returns a flat
dict with ``markdown`` and ``fields`` at the top level.
For multi-segment results (e.g. video split into scenes), fields are kept
with their respective segments in a ``segments`` list so the LLM can see
which fields belong to which part of the content:
- ``segments``: list of per-segment dicts with ``markdown``, ``fields``,
``start_time_s``, and ``end_time_s``
- ``markdown``: still concatenated at top level for file_search uploads
- ``duration_seconds``: computed from the global time span
- ``kind`` / ``resolution``: taken from the first segment
"""
extracted: dict[str, object] = {}
contents = result.contents
if not contents:
return extracted
# --- Warnings from the CU service (ODataV4Format with code/message/target) ---
if result.warnings:
warnings_out: list[dict[str, str]] = []
for w in result.warnings:
entry: dict[str, str] = {}
code = getattr(w, "code", None)
if code:
entry["code"] = code
msg = getattr(w, "message", None)
entry["message"] = msg if msg else str(w)
target = getattr(w, "target", None)
if target:
entry["target"] = target
warnings_out.append(entry)
extracted["warnings"] = warnings_out
# --- Media metadata (from first segment) ---
first = contents[0]
kind = getattr(first, "kind", None)
if kind:
extracted["kind"] = kind
width = getattr(first, "width", None)
height = getattr(first, "height", None)
if width and height:
extracted["resolution"] = f"{width}x{height}"
# Compute total duration from the global time span of all segments.
global_start: int | None = None
global_end: int | None = None
for content in contents:
s = getattr(content, "start_time_ms", None)
if s is None:
s = getattr(content, "startTimeMs", None)
e = getattr(content, "end_time_ms", None)
if e is None:
e = getattr(content, "endTimeMs", None)
if s is not None:
global_start = s if global_start is None else min(global_start, s)
if e is not None:
global_end = e if global_end is None else max(global_end, e)
if global_start is not None and global_end is not None:
extracted["duration_seconds"] = round((global_end - global_start) / 1000, 1)
is_multi_segment = len(contents) > 1
# --- Single-segment: flat output (documents, images, short audio) ---
if not is_multi_segment:
if "markdown" in output_sections and contents[0].markdown:
extracted["markdown"] = contents[0].markdown
if "fields" in output_sections and contents[0].fields:
fields: dict[str, object] = {}
for name, field in contents[0].fields.items():
entry_dict: dict[str, object] = {
"type": getattr(field, "type", None),
"value": extract_field_value(field),
}
confidence = getattr(field, "confidence", None)
if confidence is not None:
entry_dict["confidence"] = confidence
fields[name] = entry_dict
if fields:
extracted["fields"] = fields
# Content-level category (e.g. from classifier analyzers)
category = getattr(contents[0], "category", None)
if category:
extracted["category"] = category
return extracted
# --- Multi-segment: per-segment output (video scenes, long audio) ---
# Each segment keeps its own markdown + fields together so the LLM can
# see which fields (e.g. Summary) belong to which part of the content.
segments_out: list[dict[str, object]] = []
md_parts: list[str] = [] # also collect for top-level concatenated markdown
for content in contents:
seg: dict[str, object] = {}
# Time range for this segment
s = getattr(content, "start_time_ms", None)
if s is None:
s = getattr(content, "startTimeMs", None)
e = getattr(content, "end_time_ms", None)
if e is None:
e = getattr(content, "endTimeMs", None)
if s is not None:
seg["start_time_s"] = round(s / 1000, 1)
if e is not None:
seg["end_time_s"] = round(e / 1000, 1)
# Per-segment markdown
if "markdown" in output_sections and content.markdown:
seg["markdown"] = content.markdown
md_parts.append(content.markdown)
# Per-segment fields
if "fields" in output_sections and content.fields:
seg_fields: dict[str, object] = {}
for name, field in content.fields.items():
seg_entry: dict[str, object] = {
"type": getattr(field, "type", None),
"value": extract_field_value(field),
}
confidence = getattr(field, "confidence", None)
if confidence is not None:
seg_entry["confidence"] = confidence
seg_fields[name] = seg_entry
if seg_fields:
seg["fields"] = seg_fields
# Per-segment category (e.g. from classifier analyzers)
category = getattr(content, "category", None)
if category:
seg["category"] = category
segments_out.append(seg)
extracted["segments"] = segments_out
# Top-level concatenated markdown (used by file_search for vector store upload)
if md_parts:
extracted["markdown"] = "\n\n---\n\n".join(md_parts)
return extracted
def extract_field_value(field: Any) -> object:
"""Extract the plain Python value from a CU ``ContentField``.
Uses the SDK's ``.value`` convenience property, which dynamically
reads the correct ``value_*`` attribute for each field type.
Object and array types are recursively flattened so that the
output contains only plain Python primitives (str, int, float,
date, dict, list) -- no SDK model objects or raw wire format
(``valueNumber``, ``spans``, ``source``, etc.).
"""
field_type = getattr(field, "type", None)
raw = getattr(field, "value", None)
# Object fields -> recursively resolve nested sub-fields
if field_type == "object" and raw is not None and isinstance(raw, dict):
return {str(k): flatten_field(v) for k, v in cast(dict[str, Any], raw).items()}
# Array fields -> list of flattened items (each with value + optional confidence)
if field_type == "array" and raw is not None and isinstance(raw, list):
return [flatten_field(item) for item in cast(list[Any], raw)]
# Scalar fields (string, number, date, etc.) -- .value returns native Python type
return raw
def flatten_field(field: Any) -> object:
"""Flatten a CU ``ContentField`` into a ``{type, value, confidence}`` dict.
Used for sub-fields inside object and array types to preserve
per-field confidence scores. Confidence is omitted when ``None``
to reduce token usage.
"""
field_type = getattr(field, "type", None)
value = extract_field_value(field)
confidence = getattr(field, "confidence", None)
result: dict[str, object] = {"type": field_type, "value": value}
if confidence is not None:
result["confidence"] = confidence
return result
def format_result(filename: str, result: dict[str, object]) -> str:
"""Format extracted CU result for LLM consumption.
For multi-segment results (video/audio with ``segments``), each segment's
markdown and fields are grouped together so the LLM can see which fields
belong to which part of the content.
"""
kind = result.get("kind")
is_video = kind == "audioVisual"
is_audio = kind == "audio"
# Header -- media-aware label
if is_video:
label = "Video analysis"
elif is_audio:
label = "Audio analysis"
else:
label = "Document analysis"
parts: list[str] = [f'{label} of "{filename}":']
# Media metadata line (duration, resolution)
meta_items: list[str] = []
duration = result.get("duration_seconds")
if duration is not None:
mins, secs = divmod(int(duration), 60) # type: ignore[call-overload]
meta_items.append(f"Duration: {mins}:{secs:02d}")
resolution = result.get("resolution")
if resolution:
meta_items.append(f"Resolution: {resolution}")
if meta_items:
parts.append(" | ".join(meta_items))
# --- Multi-segment: format each segment with its own content + fields ---
raw_segments = result.get("segments")
segments: list[dict[str, object]] = (
cast(list[dict[str, object]], raw_segments) if isinstance(raw_segments, list) else []
)
if segments:
for i, seg in enumerate(segments):
# Segment header with time range
start = seg.get("start_time_s")
end = seg.get("end_time_s")
if start is not None and end is not None:
s_min, s_sec = divmod(int(start), 60) # type: ignore[call-overload]
e_min, e_sec = divmod(int(end), 60) # type: ignore[call-overload]
parts.append(f"\n### Segment {i + 1} ({s_min}:{s_sec:02d} - {e_min}:{e_sec:02d})")
else:
parts.append(f"\n### Segment {i + 1}")
# Segment markdown
seg_md = seg.get("markdown")
if seg_md:
parts.append(f"\n```markdown\n{seg_md}\n```")
# Segment fields
seg_fields = seg.get("fields")
if isinstance(seg_fields, dict) and seg_fields:
fields_json = json.dumps(seg_fields, indent=2, default=str)
parts.append(f"\n**Fields:**\n```json\n{fields_json}\n```")
return "\n".join(parts)
# --- Single-segment: flat format ---
fields_raw = result.get("fields")
fields: dict[str, object] = cast(dict[str, object], fields_raw) if isinstance(fields_raw, dict) else {}
# For audio: promote Summary field as prose before markdown
if is_audio and fields:
summary_field = fields.get("Summary")
if isinstance(summary_field, dict):
sf = cast(dict[str, object], summary_field)
if sf.get("value"):
parts.append(f"\n## Summary\n\n{sf['value']}")
# Markdown content
markdown = result.get("markdown")
if markdown:
parts.append(f"\n## Content\n\n```markdown\n{markdown}\n```")
# Fields section
if fields:
remaining = dict(fields)
if is_audio:
remaining = {k: v for k, v in remaining.items() if k != "Summary"}
if remaining:
fields_json = json.dumps(remaining, indent=2, default=str)
parts.append(f"\n## Extracted Fields\n\n```json\n{fields_json}\n```")
return "\n".join(parts)
@@ -0,0 +1,101 @@
# Copyright (c) Microsoft. All rights reserved.
"""File search backend abstraction for vector store file operations.
Provides a unified interface for uploading CU-extracted content to
vector stores across different LLM clients. Two implementations:
- ``OpenAIFileSearchBackend`` — for ``OpenAIChatClient`` (Responses API)
- ``FoundryFileSearchBackend`` — for ``FoundryChatClient`` (Responses API via Azure)
Both share the same OpenAI-compatible vector store file API but differ
in the file upload ``purpose`` value.
Vector store creation, tool construction, and lifecycle management are
the caller's responsibility — the backend only handles file upload/delete.
"""
from __future__ import annotations
import io
from abc import ABC, abstractmethod
from typing import Any
class FileSearchBackend(ABC):
"""Abstract interface for vector store file operations.
Implementations handle the differences between OpenAI and Foundry
file upload APIs (e.g., different ``purpose`` values).
Vector store creation, deletion, and ``file_search`` tool construction
are **not** part of this interface — those are managed by the caller.
"""
@abstractmethod
async def upload_file(self, vector_store_id: str, filename: str, content: bytes) -> str:
"""Upload a file to a vector store and return the file ID."""
@abstractmethod
async def delete_file(self, file_id: str) -> None:
"""Delete a previously uploaded file by ID."""
class _OpenAICompatBackend(FileSearchBackend):
"""Shared base for OpenAI-compatible file upload backends.
Both OpenAI and Foundry use the same ``client.files.*`` and
``client.vector_stores.files.*`` API surface. Subclasses only
override the file upload ``purpose``.
"""
_FILE_PURPOSE: str # Subclasses must set this
def __init__(self, client: Any) -> None:
self._client = client
async def upload_file(self, vector_store_id: str, filename: str, content: bytes) -> str:
uploaded = await self._client.files.create(
file=(filename, io.BytesIO(content)),
purpose=self._FILE_PURPOSE,
)
# Use create_and_poll to wait for indexing to complete before returning.
# Without this, file_search queries may return no results immediately
# after upload because the vector store index isn't ready yet.
await self._client.vector_stores.files.create_and_poll(
vector_store_id=vector_store_id,
file_id=uploaded.id,
)
return uploaded.id # type: ignore[no-any-return]
async def delete_file(self, file_id: str) -> None:
await self._client.files.delete(file_id)
class OpenAIFileSearchBackend(_OpenAICompatBackend):
"""File search backend for OpenAI Responses API.
Use with ``OpenAIChatClient`` or ``AzureOpenAIResponsesClient``.
Requires an ``AsyncOpenAI`` or ``AsyncAzureOpenAI`` client.
Args:
client: An async OpenAI client (``AsyncOpenAI`` or ``AsyncAzureOpenAI``)
that supports ``client.files.*`` and ``client.vector_stores.*`` APIs.
"""
_FILE_PURPOSE = "user_data"
class FoundryFileSearchBackend(_OpenAICompatBackend):
"""File search backend for Azure AI Foundry.
Use with ``FoundryChatClient``. Requires the OpenAI-compatible client
obtained from ``FoundryChatClient.client`` (i.e.,
``project_client.get_openai_client()``).
Args:
client: The OpenAI-compatible async client from a ``FoundryChatClient``
(access via ``foundry_client.client``).
"""
_FILE_PURPOSE = "assistants"
@@ -0,0 +1,115 @@
# Copyright (c) Microsoft. All rights reserved.
from __future__ import annotations
from dataclasses import dataclass
from enum import Enum
from typing import Any, Literal, TypedDict
from ._file_search import FileSearchBackend, FoundryFileSearchBackend, OpenAIFileSearchBackend
class DocumentStatus(str, Enum):
"""Analysis lifecycle state of a tracked document."""
ANALYZING = "analyzing"
"""CU analysis is in progress (deferred to background)."""
UPLOADING = "uploading"
"""Analysis complete; vector store upload + indexing is in progress."""
READY = "ready"
"""Analysis (and upload, if applicable) completed successfully."""
FAILED = "failed"
"""Analysis or upload failed."""
AnalysisSection = Literal["markdown", "fields"]
"""Which sections of the CU output to pass to the LLM.
- ``"markdown"``: Full document text with tables as HTML, reading order preserved.
- ``"fields"``: Extracted typed fields with confidence scores (when available).
"""
class DocumentEntry(TypedDict):
"""Tracks the analysis state of a single document in session state."""
status: DocumentStatus
filename: str
media_type: str
analyzer_id: str
analyzed_at: str | None
analysis_duration_s: float | None
upload_duration_s: float | None
result: dict[str, object] | None
error: str | None
@dataclass
class FileSearchConfig:
"""Configuration for uploading CU-extracted content to an existing vector store.
When provided to ``ContentUnderstandingContextProvider``, analyzed document
markdown is automatically uploaded to the specified vector store and the
given ``file_search`` tool is registered on the context. This enables
token-efficient RAG retrieval on follow-up turns for large documents.
The caller is responsible for creating and managing the vector store and
the ``file_search`` tool. Use :meth:`from_openai` or :meth:`from_foundry`
factory methods for convenience.
Args:
backend: A ``FileSearchBackend`` that handles file upload/delete
operations for the target vector store.
vector_store_id: The ID of a pre-existing vector store to upload to.
file_search_tool: A ``file_search`` tool object created via the LLM
client's ``get_file_search_tool()`` factory method. This is
registered on the context via ``extend_tools`` so the LLM can
retrieve uploaded content.
"""
backend: FileSearchBackend
vector_store_id: str
file_search_tool: Any
@staticmethod
def from_openai(
client: Any,
*,
vector_store_id: str,
file_search_tool: Any,
) -> FileSearchConfig:
"""Create a config for OpenAI Responses API (``OpenAIChatClient``).
Args:
client: An ``AsyncOpenAI`` or ``AsyncAzureOpenAI`` client.
vector_store_id: The ID of the vector store to upload to.
file_search_tool: Tool from ``OpenAIChatClient.get_file_search_tool()``.
"""
return FileSearchConfig(
backend=OpenAIFileSearchBackend(client),
vector_store_id=vector_store_id,
file_search_tool=file_search_tool,
)
@staticmethod
def from_foundry(
client: Any,
*,
vector_store_id: str,
file_search_tool: Any,
) -> FileSearchConfig:
"""Create a config for Azure AI Foundry (``FoundryChatClient``).
Args:
client: The OpenAI-compatible client from ``FoundryChatClient.client``.
vector_store_id: The ID of the vector store to upload to.
file_search_tool: Tool from ``FoundryChatClient.get_file_search_tool()``.
"""
return FileSearchConfig(
backend=FoundryFileSearchBackend(client),
vector_store_id=vector_store_id,
file_search_tool=file_search_tool,
)
@@ -0,0 +1,101 @@
[project]
name = "agent-framework-azure-contentunderstanding"
description = "Azure Content Understanding integration for Microsoft Agent Framework."
authors = [{ name = "Microsoft", email = "af-support@microsoft.com" }]
readme = "README.md"
requires-python = ">=3.10"
version = "1.0.0a260429"
license-files = ["LICENSE"]
urls.homepage = "https://aka.ms/agent-framework"
urls.source = "https://github.com/microsoft/agent-framework/tree/main/python"
urls.release_notes = "https://github.com/microsoft/agent-framework/releases?q=tag%3Apython-1&expanded=true"
urls.issues = "https://github.com/microsoft/agent-framework/issues"
classifiers = [
"License :: OSI Approved :: MIT License",
"Development Status :: 3 - Alpha",
"Intended Audience :: Developers",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Programming Language :: Python :: 3.14",
"Typing :: Typed",
]
dependencies = [
"agent-framework-core>=1.2.2,<2",
"agent-framework-foundry>=1.2.2,<2",
"azure-ai-contentunderstanding>=1.0.1,<1.1",
"aiohttp>=3.9,<4",
"filetype>=1.2,<2",
]
[tool.uv]
prerelease = "if-necessary-or-explicit"
environments = [
"sys_platform == 'darwin'",
"sys_platform == 'linux'",
"sys_platform == 'win32'"
]
[tool.uv-dynamic-versioning]
fallback-version = "0.0.0"
[tool.pytest.ini_options]
testpaths = 'tests'
addopts = "-ra -q -r fEX"
asyncio_mode = "auto"
asyncio_default_fixture_loop_scope = "function"
timeout = 120
markers = [
"integration: marks tests as integration tests that require external services",
]
[tool.ruff]
extend = "../../pyproject.toml"
[tool.ruff.lint.per-file-ignores]
"**/tests/**" = ["D", "INP", "TD", "ERA001", "RUF", "S"]
"samples/**" = ["D", "INP", "ERA001", "RUF", "S", "T201", "CPY"]
[tool.coverage.run]
omit = ["**/__init__.py"]
[tool.pyright]
extends = "../../pyproject.toml"
include = ["agent_framework_azure_contentunderstanding"]
exclude = ['tests']
[tool.mypy]
plugins = ['pydantic.mypy']
strict = true
python_version = "3.10"
ignore_missing_imports = true
disallow_untyped_defs = true
no_implicit_optional = true
check_untyped_defs = true
warn_return_any = true
show_error_codes = true
warn_unused_ignores = false
disallow_incomplete_defs = true
disallow_untyped_decorators = true
[tool.bandit]
targets = ["agent_framework_azure_contentunderstanding"]
exclude_dirs = ["tests"]
[tool.poe]
executor.type = "uv"
include = "../../shared_tasks.toml"
[tool.poe.tasks.mypy]
help = "Run MyPy for this package."
cmd = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_azure_contentunderstanding"
[tool.poe.tasks.test]
help = "Run the default unit test suite for this package."
cmd = 'pytest -m "not integration" --cov=agent_framework_azure_contentunderstanding --cov-report=term-missing:skip-covered tests'
[build-system]
requires = ["flit-core >= 3.11,<4.0"]
build-backend = "flit_core.buildapi"
@@ -0,0 +1,117 @@
# Copyright (c) Microsoft. All rights reserved.
# /// script
# requires-python = ">=3.10"
# dependencies = [
# "agent-framework-azure-contentunderstanding",
# "agent-framework-foundry",
# "azure-identity",
# ]
# ///
# Run with: uv run packages/azure-contentunderstanding/samples/01-get-started/01_document_qa.py
import asyncio
import os
from pathlib import Path
from agent_framework import Agent, Content, Message
from agent_framework.foundry import ContentUnderstandingContextProvider, FoundryChatClient
from azure.identity import AzureCliCredential
from dotenv import load_dotenv
load_dotenv()
"""
Document Q&A — PDF upload with CU-powered extraction
This sample demonstrates the simplest CU integration: upload a PDF and
ask questions about it. Azure Content Understanding extracts structured
markdown with table preservation — superior to LLM-only vision for
scanned PDFs, handwritten content, and complex layouts.
Environment variables:
FOUNDRY_PROJECT_ENDPOINT — Azure AI Foundry project endpoint
FOUNDRY_MODEL — Model deployment name (e.g. gpt-4.1)
AZURE_CONTENTUNDERSTANDING_ENDPOINT — CU endpoint URL
"""
# Path to a sample PDF — uses the shared sample asset if available,
# otherwise falls back to a public URL
SAMPLE_PDF_PATH = Path(__file__).resolve().parents[1] / "shared" / "sample_assets" / "invoice.pdf"
async def main() -> None:
credential = AzureCliCredential()
# Set up Azure Content Understanding context provider
cu = ContentUnderstandingContextProvider(
endpoint=os.environ["AZURE_CONTENTUNDERSTANDING_ENDPOINT"],
credential=credential,
analyzer_id="prebuilt-documentSearch", # RAG-optimized document analyzer
max_wait=None, # wait until CU analysis finishes (no background deferral)
)
# Set up the LLM client
client = FoundryChatClient(
project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"],
model=os.environ["FOUNDRY_MODEL"],
credential=credential,
)
# Create agent with CU context provider.
# The provider extracts document content via CU and injects it into the
# LLM context so the agent can answer questions about the document.
async with cu:
agent = Agent(
client=client,
name="DocumentQA",
instructions=(
"You are a helpful document analyst. Use the analyzed document "
"content and extracted fields to answer questions precisely."
),
context_providers=[cu],
)
# --- Turn 1: Upload PDF and ask a question ---
# 4. Upload PDF and ask questions
# The CU provider extracts markdown + fields from the PDF and injects
# the full content into context so the agent can answer precisely.
print("--- Upload PDF and ask questions ---")
pdf_bytes = SAMPLE_PDF_PATH.read_bytes()
response = await agent.run(
Message(
role="user",
contents=[
Content.from_text(
"What is this document about? Who is the vendor, and what is the total amount due?"
),
Content.from_data(
pdf_bytes,
"application/pdf",
# Always provide filename — used as the document key
additional_properties={"filename": SAMPLE_PDF_PATH.name},
),
],
)
)
usage = response.usage_details or {}
print(f"Agent: {response}")
print(f" [Input tokens: {usage.get('input_token_count', 'N/A')}]\n")
if __name__ == "__main__":
asyncio.run(main())
"""
Sample output:
--- Upload PDF and ask questions ---
Agent: This document is an **invoice** for services and fees billed to
**MICROSOFT CORPORATION** (Invoice **INV-100**), including line items
(e.g., Consulting Services, Document Fee, Printing Fee) and a billing summary.
- **Vendor:** **CONTOSO LTD.**
- **Total amount due:** **$610.00**
[Input tokens: 988]
"""
@@ -0,0 +1,143 @@
# Copyright (c) Microsoft. All rights reserved.
# /// script
# requires-python = ">=3.10"
# dependencies = [
# "agent-framework-azure-contentunderstanding",
# "agent-framework-foundry",
# "azure-identity",
# ]
# ///
# Run with: uv run packages/azure-contentunderstanding/samples/01-get-started/02_multi_turn_session.py
import asyncio
import os
from pathlib import Path
from agent_framework import Agent, AgentSession, Content, Message
from agent_framework.foundry import ContentUnderstandingContextProvider, FoundryChatClient
from azure.identity import AzureCliCredential
from dotenv import load_dotenv
load_dotenv()
"""
Multi-Turn Session — Cached results across turns
This sample demonstrates multi-turn document Q&A using an AgentSession.
The session persists CU analysis results and conversation history across
turns so the agent can answer follow-up questions about previously
uploaded documents without re-analyzing them.
Key concepts:
- AgentSession keeps CU state and conversation history across agent.run() calls
- Turn 1: CU analyzes the PDF and injects full content into context
- Turn 2: Unrelated question — agent answers from general knowledge
- Turn 3: Detailed question — agent uses document content from conversation
history (injected in Turn 1) to answer precisely
Environment variables:
FOUNDRY_PROJECT_ENDPOINT — Azure AI Foundry project endpoint
FOUNDRY_MODEL — Model deployment name (e.g. gpt-4.1)
AZURE_CONTENTUNDERSTANDING_ENDPOINT — CU endpoint URL
"""
SAMPLE_PDF_PATH = Path(__file__).resolve().parents[1] / "shared" / "sample_assets" / "invoice.pdf"
async def main() -> None:
# 1. Set up credentials and CU context provider
credential = AzureCliCredential()
cu = ContentUnderstandingContextProvider(
endpoint=os.environ["AZURE_CONTENTUNDERSTANDING_ENDPOINT"],
credential=credential,
analyzer_id="prebuilt-documentSearch",
max_wait=None, # wait until CU analysis finishes (no background deferral)
)
# 2. Set up the LLM client
client = FoundryChatClient(
project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"],
model=os.environ["FOUNDRY_MODEL"],
credential=credential,
)
# 3. Create agent and persistent session
async with cu:
agent = Agent(
client=client,
name="DocumentQA",
instructions=(
"You are a helpful document analyst. Use the analyzed document "
"content and extracted fields to answer questions precisely."
),
context_providers=[cu],
)
# Create a persistent session — this keeps CU state across turns
session = AgentSession()
# 4. Turn 1: Upload PDF
# CU analyzes the PDF and injects full content into context.
print("--- Turn 1: Upload PDF ---")
pdf_bytes = SAMPLE_PDF_PATH.read_bytes()
response = await agent.run(
Message(
role="user",
contents=[
Content.from_text("What is this document about?"),
Content.from_data(
pdf_bytes,
"application/pdf",
additional_properties={"filename": SAMPLE_PDF_PATH.name},
),
],
),
session=session, # <-- persist state across turns
)
usage = response.usage_details or {}
print(f"Agent: {response}")
print(f" [Input tokens: {usage.get('input_token_count', 'N/A')}]\n")
# 5. Turn 2: Unrelated question
# No document needed — agent answers from general knowledge.
print("--- Turn 2: Unrelated question ---")
response = await agent.run("What is the capital of France?", session=session)
usage = response.usage_details or {}
print(f"Agent: {response}")
print(f" [Input tokens: {usage.get('input_token_count', 'N/A')}]\n")
# 6. Turn 3: Detailed follow-up
# The agent answers from the full document content that was injected
# into conversation history in Turn 1. No re-analysis or tool call needed.
print("--- Turn 3: Detailed follow-up ---")
response = await agent.run(
"What is the shipping address on the invoice?",
session=session,
)
usage = response.usage_details or {}
print(f"Agent: {response}")
print(f" [Input tokens: {usage.get('input_token_count', 'N/A')}]\n")
if __name__ == "__main__":
asyncio.run(main())
"""
Sample output:
--- Turn 1: Upload PDF ---
Agent: This document is an **invoice** from **CONTOSO LTD.** to **MICROSOFT
CORPORATION**. Amount Due: $610.00. Invoice INV-100, dated 11/15/2019.
[Input tokens: 975]
--- Turn 2: Unrelated question ---
Agent: Paris.
[Input tokens: 1134]
--- Turn 3: Detailed follow-up ---
Agent: Shipping address (SHIP TO): Microsoft Delivery, 123 Ship St,
Redmond WA, 98052.
[Input tokens: 1155]
"""
@@ -0,0 +1,186 @@
# Copyright (c) Microsoft. All rights reserved.
# /// script
# requires-python = ">=3.10"
# dependencies = [
# "agent-framework-azure-contentunderstanding",
# "agent-framework-foundry",
# "azure-identity",
# ]
# ///
# Run with: uv run packages/azure-contentunderstanding/samples/01-get-started/03_multimodal_chat.py
import asyncio
import os
import time
from pathlib import Path
from agent_framework import Agent, AgentSession, Content, Message
from agent_framework.foundry import ContentUnderstandingContextProvider, FoundryChatClient
from azure.identity import AzureCliCredential
from dotenv import load_dotenv
load_dotenv()
"""
Multi-Modal Chat — PDF, audio, and video in a single turn
This sample demonstrates CU's multi-modal capability: upload a PDF invoice,
an audio call recording, and a video file all at once. The provider analyzes
all three in parallel using the right CU analyzer for each media type.
The provider auto-detects the media type and selects the right CU analyzer:
- PDF/images → prebuilt-documentSearch
- Audio → prebuilt-audioSearch
- Video → prebuilt-videoSearch
Environment variables:
FOUNDRY_PROJECT_ENDPOINT — Azure AI Foundry project endpoint
FOUNDRY_MODEL — Model deployment name (e.g. gpt-4.1)
AZURE_CONTENTUNDERSTANDING_ENDPOINT — CU endpoint URL
"""
# Local PDF from package assets
SAMPLE_PDF = Path(__file__).resolve().parents[1] / "shared" / "sample_assets" / "invoice.pdf"
# Public audio/video from Azure CU samples repo (raw GitHub URLs)
_CU_ASSETS = "https://raw.githubusercontent.com/Azure-Samples/azure-ai-content-understanding-assets/main"
AUDIO_URL = f"{_CU_ASSETS}/audio/callCenterRecording.mp3"
VIDEO_URL = f"{_CU_ASSETS}/videos/sdk_samples/FlightSimulator.mp4"
async def main() -> None:
# 1. Set up credentials and CU context provider
credential = AzureCliCredential()
# No analyzer_id specified — the provider auto-detects from media type:
# PDF/images → prebuilt-documentSearch
# Audio → prebuilt-audioSearch
# Video → prebuilt-videoSearch
cu = ContentUnderstandingContextProvider(
endpoint=os.environ["AZURE_CONTENTUNDERSTANDING_ENDPOINT"],
credential=credential,
max_wait=None, # wait until each analysis finishes
)
# 2. Set up the LLM client
client = FoundryChatClient(
project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"],
model=os.environ["FOUNDRY_MODEL"],
credential=credential,
)
# 3. Create agent and session
async with cu:
agent = Agent(
client=client,
name="MultiModalAgent",
instructions=(
"You are a helpful assistant that can analyze documents, audio, "
"and video files. Answer questions using the extracted content."
),
context_providers=[cu],
)
session = AgentSession()
# --- Turn 1: Upload all 3 modalities at once ---
# The provider analyzes all files in parallel using the appropriate
# CU analyzer for each media type. All results are injected into
# the same context so the agent can answer about all of them.
turn1_prompt = (
"I'm uploading three files: an invoice PDF, a call center "
"audio recording, and a flight simulator video. "
"Give a brief summary of each file."
)
print("--- Turn 1: Upload PDF + audio + video (parallel analysis) ---")
print(" (CU analysis may take a few minutes for these audio/video files...)")
print(f"User: {turn1_prompt}")
t0 = time.perf_counter()
response = await agent.run(
Message(
role="user",
contents=[
Content.from_text(turn1_prompt),
Content.from_data(
SAMPLE_PDF.read_bytes(),
"application/pdf",
additional_properties={"filename": "invoice.pdf"},
),
Content.from_uri(
AUDIO_URL,
media_type="audio/mp3",
additional_properties={"filename": "callCenterRecording.mp3"},
),
Content.from_uri(
VIDEO_URL,
media_type="video/mp4",
additional_properties={"filename": "FlightSimulator.mp4"},
),
],
),
session=session,
)
elapsed = time.perf_counter() - t0
usage = response.usage_details or {}
print(f" [Analyzed in {elapsed:.1f}s | Input tokens: {usage.get('input_token_count', 'N/A')}]")
print(f"Agent: {response}\n")
# --- Turn 2: Detail question about the PDF ---
turn2_prompt = "What are the line items and their amounts on the invoice?"
print("--- Turn 2: PDF detail ---")
print(f"User: {turn2_prompt}")
response = await agent.run(turn2_prompt, session=session)
usage = response.usage_details or {}
print(f" [Input tokens: {usage.get('input_token_count', 'N/A')}]")
print(f"Agent: {response}\n")
# --- Turn 3: Detail question about the audio ---
turn3_prompt = "What was the customer's issue in the call recording?"
print("--- Turn 3: Audio detail ---")
print(f"User: {turn3_prompt}")
response = await agent.run(turn3_prompt, session=session)
usage = response.usage_details or {}
print(f" [Input tokens: {usage.get('input_token_count', 'N/A')}]")
print(f"Agent: {response}\n")
# --- Turn 4: Detail question about the video ---
turn4_prompt = "What key scenes or actions are shown in the flight simulator video?"
print("--- Turn 4: Video detail ---")
print(f"User: {turn4_prompt}")
response = await agent.run(turn4_prompt, session=session)
usage = response.usage_details or {}
print(f" [Input tokens: {usage.get('input_token_count', 'N/A')}]")
print(f"Agent: {response}\n")
# --- Turn 5: Cross-document question ---
turn5_prompt = (
"Across all three files, which one contains financial data, "
"which one involves a customer interaction, and which one is "
"a visual demonstration?"
)
print("--- Turn 5: Cross-document question ---")
print(f"User: {turn5_prompt}")
response = await agent.run(turn5_prompt, session=session)
usage = response.usage_details or {}
print(f" [Input tokens: {usage.get('input_token_count', 'N/A')}]")
print(f"Agent: {response}\n")
if __name__ == "__main__":
asyncio.run(main())
"""
Sample output:
--- Turn 1: Upload PDF + audio + video (parallel analysis) ---
User: I'm uploading three files...
(CU analysis may take 1-2 minutes for audio/video files...)
[Analyzed in ~94s | Input tokens: ~2939]
Agent: ### invoice.pdf: An invoice from CONTOSO LTD. to MICROSOFT CORPORATION...
### callCenterRecording.mp3: A customer service call about point balance...
### FlightSimulator.mp4: A clip discussing neural text-to-speech...
--- Turn 2-5: Detail and cross-document questions ---
(Agent answers from conversation history without re-analysis)
"""
@@ -0,0 +1,193 @@
# Copyright (c) Microsoft. All rights reserved.
# /// script
# requires-python = ">=3.10"
# dependencies = [
# "agent-framework-azure-contentunderstanding",
# "agent-framework-foundry",
# "azure-identity",
# "pydantic",
# ]
# ///
# Run with: uv run packages/azure-contentunderstanding/samples/01-get-started/04_invoice_processing.py
import asyncio
import os
from pathlib import Path
from agent_framework import Agent, AgentSession, Content, Message
from agent_framework.foundry import ContentUnderstandingContextProvider, FoundryChatClient
from azure.identity import AzureCliCredential
from dotenv import load_dotenv
from pydantic import BaseModel, Field
load_dotenv()
"""
Invoice Processing — Structured output with prebuilt-invoice analyzer
This sample demonstrates CU's structured field extraction combined with
LLM structured output (Pydantic model). The prebuilt-invoice analyzer extracts
typed fields (VendorName, InvoiceTotal, DueDate, LineItems, etc.) with
confidence scores. We use output_sections=["fields"] only (no markdown needed)
since we want the LLM to produce a structured JSON response from the extracted
fields, not summarize document text.
Environment variables:
FOUNDRY_PROJECT_ENDPOINT — Azure AI Foundry project endpoint
FOUNDRY_MODEL — Model deployment name (e.g. gpt-4.1)
AZURE_CONTENTUNDERSTANDING_ENDPOINT — CU endpoint URL
"""
SAMPLE_PDF_PATH = Path(__file__).resolve().parents[1] / "shared" / "sample_assets" / "invoice.pdf"
# Structured output model — the LLM will return JSON matching this schema
# Structured output models — the LLM returns JSON matching this schema.
#
# Note: the prebuilt-invoice analyzer extracts an extensive set of fields
# (VendorName, BillingAddress, ShippingAddress, TaxDetails, PONumber, etc.).
# This sample defines a simplified schema to extract only the fields of
# interest to the caller. The LLM maps the full CU field output to this
# subset automatically.
# Learn more about prebuilt analyzers: https://learn.microsoft.com/azure/ai-services/content-understanding/concepts/prebuilt-analyzers
class LineItem(BaseModel):
description: str
quantity: float | None = None
unit_price: float | None = None
amount: float | None = None
class LowConfidenceField(BaseModel):
field_name: str
confidence: float
class InvoiceResult(BaseModel):
vendor_name: str
total_amount: float | None = None
currency: str = "USD"
due_date: str | None = None
line_items: list[LineItem] = Field(default_factory=list)
low_confidence_fields: list[LowConfidenceField] = Field(
default_factory=list,
description="Fields with confidence < 0.8, including their confidence score",
)
async def main() -> None:
# 1. Set up credentials and CU context provider
credential = AzureCliCredential()
# Default analyzer is prebuilt-documentSearch (RAG-optimized).
# Per-file override via additional_properties["analyzer_id"] lets us
# use prebuilt-invoice for structured field extraction on specific files.
#
# Only request "fields" (not "markdown") — we want the extracted typed
# fields for structured output, not the raw document text.
cu = ContentUnderstandingContextProvider(
endpoint=os.environ["AZURE_CONTENTUNDERSTANDING_ENDPOINT"],
credential=credential,
analyzer_id="prebuilt-documentSearch", # default for all files
max_wait=None, # wait until CU analysis finishes
output_sections=["fields"], # fields only — structured output doesn't need markdown
)
# 2. Set up the LLM client
client = FoundryChatClient(
project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"],
model=os.environ["FOUNDRY_MODEL"],
credential=credential,
)
# 3. Create agent and session
async with cu:
agent = Agent(
client=client,
name="InvoiceProcessor",
instructions=(
"You are an invoice processing assistant. Extract invoice data from "
"the provided CU fields (JSON with confidence scores). Return structured "
"output matching the requested schema. Flag fields with confidence < 0.8 "
"in the low_confidence_fields list."
),
context_providers=[cu],
)
session = AgentSession()
# 4. Upload an invoice PDF — uses structured output (Pydantic model)
print("--- Upload Invoice (Structured Output) ---")
pdf_bytes = SAMPLE_PDF_PATH.read_bytes()
response = await agent.run(
Message(
role="user",
contents=[
Content.from_text(
"Process this invoice. Extract the vendor name, total amount, due date, and all line items."
),
Content.from_data(
pdf_bytes,
"application/pdf",
# Per-file analyzer override: use prebuilt-invoice for
# structured field extraction (VendorName, InvoiceTotal, etc.)
# instead of the provider default (prebuilt-documentSearch).
additional_properties={
"filename": SAMPLE_PDF_PATH.name,
"analyzer_id": "prebuilt-invoice",
},
),
],
),
session=session,
options={"response_format": InvoiceResult},
)
# Parse the structured output from JSON text
try:
invoice = InvoiceResult.model_validate_json(response.text)
print(f"Vendor: {invoice.vendor_name}")
print(f"Total: {invoice.currency} {invoice.total_amount}")
print(f"Due date: {invoice.due_date}")
print(f"Line items ({len(invoice.line_items)}):")
for item in invoice.line_items:
print(f" - {item.description}: {item.amount}")
if invoice.low_confidence_fields:
print("âš  Low confidence fields:")
for f in invoice.low_confidence_fields:
print(f" - {f.field_name}: {f.confidence:.3f}")
except Exception:
print(f"Agent (raw): {response.text}\n")
# 5. Follow-up: free-text question about the invoice
print("\n--- Follow-up (Free Text) ---")
response = await agent.run(
"What is the payment term? Are there any fields with low confidence?",
session=session,
)
print(f"Agent: {response}\n")
if __name__ == "__main__":
asyncio.run(main())
"""
Sample output:
--- Upload Invoice (Structured Output) ---
Vendor: CONTOSO LTD.
Total: USD 110.0
Due date: 2019-12-15
Line items (3):
- Consulting Services: 60.0
- Document Fee: 30.0
- Printing Fee: 10.0
âš  Low confidence: VendorName, CustomerName
--- Follow-up (Free Text) ---
Agent: The payment terms are not explicitly stated on the invoice...
"""
@@ -0,0 +1,166 @@
# Copyright (c) Microsoft. All rights reserved.
# /// script
# requires-python = ">=3.10"
# dependencies = [
# "agent-framework-azure-contentunderstanding",
# "agent-framework-foundry",
# "azure-identity",
# ]
# ///
# Run with: uv run packages/azure-contentunderstanding/samples/01-get-started/05_large_doc_file_search.py
import asyncio
import os
from pathlib import Path
from agent_framework import Agent, AgentSession, Content, Message
from agent_framework.foundry import (
ContentUnderstandingContextProvider,
FileSearchConfig,
FoundryChatClient,
)
from azure.identity import AzureCliCredential
from dotenv import load_dotenv
load_dotenv()
"""
Large Document + file_search RAG — CU extraction + OpenAI vector store
For large documents (100+ pages) or long audio/video, injecting the full
CU-extracted content into the LLM context is impractical. This sample shows
how to use the built-in file_search integration: CU extracts markdown and
automatically uploads it to an OpenAI vector store for token-efficient RAG.
When ``FileSearchConfig`` is provided, the provider:
1. Extracts markdown via CU (handles scanned PDFs, audio, video)
2. Uploads the extracted markdown to a vector store
3. Registers a ``file_search`` tool on the agent context
4. Cleans up the vector store on close
Architecture:
Large PDF -> CU extracts markdown -> auto-upload to vector store -> file_search
Follow-up -> file_search retrieves top-k chunks -> LLM answers
NOTE: Requires an async OpenAI client for vector store operations.
This sample uses a single small invoice PDF for simplicity. In practice,
you can upload multiple files in the same session (each is indexed
separately in the vector store), and this pattern is most valuable for
large documents (up to 300 pages), long audio recordings, or video files
where full-context injection would exceed the LLM's context window.
CU supports PDFs up to 300 pages / 200 MB, and audio files up to 300 MB
— see the full service limits:
https://learn.microsoft.com/azure/ai-services/content-understanding/service-limits#input-file-limits
Environment variables:
FOUNDRY_PROJECT_ENDPOINT — Azure AI Foundry project endpoint
FOUNDRY_MODEL — Model deployment name (e.g. gpt-4.1)
AZURE_CONTENTUNDERSTANDING_ENDPOINT — CU endpoint URL
"""
SAMPLE_PDF_PATH = Path(__file__).resolve().parents[1] / "shared" / "sample_assets" / "invoice.pdf"
async def main() -> None:
# 1. Set up credentials and LLM client
credential = AzureCliCredential()
client = FoundryChatClient(
project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"],
model=os.environ["FOUNDRY_MODEL"],
credential=credential,
)
# 2. Get the async OpenAI client from FoundryChatClient for vector store operations
openai_client = client.client
# 3. Create vector store and file_search tool
vector_store = await openai_client.vector_stores.create(
name="cu_large_doc_demo",
expires_after={"anchor": "last_active_at", "days": 1},
)
file_search_tool = client.get_file_search_tool(vector_store_ids=[vector_store.id])
# 4. Configure CU provider with file_search integration
# When file_search is set, CU-extracted markdown is automatically uploaded
# to the vector store and the file_search tool is registered on the context.
cu = ContentUnderstandingContextProvider(
endpoint=os.environ["AZURE_CONTENTUNDERSTANDING_ENDPOINT"],
credential=credential,
analyzer_id="prebuilt-documentSearch",
max_wait=None, # wait until CU analysis + vector store upload finishes
file_search=FileSearchConfig.from_foundry(
openai_client,
vector_store_id=vector_store.id,
file_search_tool=file_search_tool,
),
)
pdf_bytes = SAMPLE_PDF_PATH.read_bytes()
# The provider handles everything: CU extraction + vector store upload + file_search tool
async with cu:
agent = Agent(
client=client,
name="LargeDocAgent",
instructions=(
"You are a document analyst. Use the file_search tool to find "
"relevant sections from the document and answer precisely. "
"Cite specific sections when answering."
),
context_providers=[cu],
)
session = AgentSession()
# Turn 1: Upload — CU extracts and uploads to vector store automatically
print("--- Turn 1: Upload document ---")
response = await agent.run(
Message(
role="user",
contents=[
Content.from_text("What are the key points in this document?"),
Content.from_data(
pdf_bytes,
"application/pdf",
additional_properties={"filename": SAMPLE_PDF_PATH.name},
),
],
),
session=session,
)
print(f"Agent: {response}\n")
# Turn 2: Follow-up — file_search retrieves relevant chunks (token efficient)
print("--- Turn 2: Follow-up (RAG) ---")
response = await agent.run(
"What numbers or financial metrics are mentioned?",
session=session,
)
print(f"Agent: {response}\n")
# Explicitly delete the vector store created for this sample
await openai_client.vector_stores.delete(vector_store.id)
print("Done. Vector store deleted.")
if __name__ == "__main__":
asyncio.run(main())
"""
Sample output:
--- Turn 1: Upload document ---
Agent: An invoice from Contoso Ltd. to Microsoft Corporation (INV-100).
Line items: Consulting Services $60, Document Fee $30, Printing Fee $10.
Subtotal $100, Sales tax $10, Total $110, Previous balance $500, Amount due $610.
--- Turn 2: Follow-up (RAG) ---
Agent: Subtotal $100.00, Sales tax $10.00, Total $110.00,
Previous unpaid balance $500.00, Amount due $610.00.
Line items: 2 hours @ $30 = $60, 3 @ $10 = $30, 10 pages @ $1 = $10.
Done. Vector store cleaned up automatically.
"""
@@ -0,0 +1,33 @@
# DevUI Multi-Modal Agent
Interactive web UI for uploading and chatting with documents, images, audio, and video using Azure Content Understanding.
## Setup
1. Set environment variables (or create a `.env` file in `python/`):
```bash
FOUNDRY_PROJECT_ENDPOINT=https://your-project.api.azureml.ms
AZURE_OPENAI_RESPONSES_DEPLOYMENT_NAME=gpt-4.1
AZURE_CONTENTUNDERSTANDING_ENDPOINT=https://your-cu-resource.cognitiveservices.azure.com/
```
2. Log in with Azure CLI:
```bash
az login
```
3. Run with DevUI:
```bash
uv run poe devui --agent packages/azure-contentunderstanding/samples/devui_multimodal_agent
```
4. Open the DevUI URL in your browser and start uploading files.
## What You Can Do
- **Upload PDFs** — including scanned/image-based PDFs that LLM vision struggles with
- **Upload images** — handwritten notes, infographics, charts
- **Upload audio** — meeting recordings, call center calls (transcription with speaker ID)
- **Upload video** — product demos, training videos (frame extraction + transcription)
- **Ask questions** across all uploaded documents
- **Check status** — "which documents are ready?" uses the auto-registered `list_documents()` tool
@@ -0,0 +1,6 @@
# Copyright (c) Microsoft. All rights reserved.
"""DevUI Multi-Modal Agent with Azure Content Understanding."""
from .agent import agent
__all__ = ["agent"]
@@ -0,0 +1,66 @@
# Copyright (c) Microsoft. All rights reserved.
"""DevUI Multi-Modal Agent — file upload + CU-powered analysis.
This agent uses Azure Content Understanding to analyze uploaded files
(PDFs, scanned documents, handwritten images, audio recordings, video)
and answer questions about them through the DevUI web interface.
Unlike the standard azure_responses_agent which sends files directly to the LLM,
this agent uses CU for structured extraction — superior for scanned PDFs,
handwritten content, audio transcription, and video analysis.
Required environment variables:
FOUNDRY_PROJECT_ENDPOINT — Azure AI Foundry project endpoint
FOUNDRY_MODEL — Model deployment name (e.g. gpt-4.1)
AZURE_CONTENTUNDERSTANDING_ENDPOINT — CU endpoint URL
Run with DevUI:
uv run poe devui --agent packages/azure-contentunderstanding/samples/devui_multimodal_agent
"""
import os
from agent_framework import Agent
from agent_framework.foundry import ContentUnderstandingContextProvider, FoundryChatClient
from azure.core.credentials import AzureKeyCredential
from azure.identity import AzureCliCredential
from dotenv import load_dotenv
load_dotenv()
# --- Auth ---
_credential = AzureCliCredential()
_cu_api_key = os.environ.get("AZURE_CONTENTUNDERSTANDING_API_KEY")
_cu_credential = AzureKeyCredential(_cu_api_key) if _cu_api_key else _credential
cu = ContentUnderstandingContextProvider(
endpoint=os.environ["AZURE_CONTENTUNDERSTANDING_ENDPOINT"],
credential=_cu_credential,
# max_wait controls how long before_run() waits for CU analysis before
# deferring to background. For interactive DevUI use, a short timeout
# (e.g. 5s) keeps the chat responsive — the agent tells the user the
# file is still being analyzed and resolves it on the next turn.
# Use max_wait=None to always wait for analysis to complete.
max_wait=5.0,
)
client = FoundryChatClient(
project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"],
model=os.environ["FOUNDRY_MODEL"],
credential=_credential,
)
agent = Agent(
client=client,
name="MultiModalDocAgent",
instructions=(
"You are a helpful document analysis assistant. "
"When a user uploads files, they are automatically analyzed using Azure Content Understanding. "
"Use list_documents() to check which documents are ready, pending, or failed "
"and to see which files are available for answering questions. "
"Tell the user if any documents are still being analyzed. "
"You can process PDFs, scanned documents, handwritten images, audio recordings, and video files. "
"When answering, cite specific content from the documents."
),
context_providers=[cu],
)
@@ -0,0 +1,51 @@
# DevUI File Search Agent
Interactive web UI for uploading and chatting with documents, images, audio, and video using Azure Content Understanding + OpenAI file_search RAG.
## How It Works
1. **Upload** any supported file (PDF, image, audio, video) via the DevUI chat
2. **CU analyzes** the file — auto-selects the right analyzer per media type
3. **Markdown extracted** by CU is uploaded to an OpenAI vector store
4. **file_search** tool is registered — LLM retrieves top-k relevant chunks
5. **Ask questions** across all uploaded documents with token-efficient RAG
## Setup
1. Set environment variables (or create a `.env` file in `python/`):
```bash
FOUNDRY_PROJECT_ENDPOINT=https://your-project.services.ai.azure.com/
AZURE_OPENAI_RESPONSES_DEPLOYMENT_NAME=gpt-4.1
AZURE_CONTENTUNDERSTANDING_ENDPOINT=https://your-cu-resource.services.ai.azure.com/
```
2. Log in with Azure CLI:
```bash
az login
```
3. Run with DevUI:
```bash
devui packages/azure-contentunderstanding/samples/devui_azure_openai_file_search_agent
```
4. Open the DevUI URL in your browser and start uploading files.
## Supported File Types
| Type | Formats | CU Analyzer (auto-detected) |
|------|---------|----------------------------|
| Documents | PDF, DOCX, XLSX, PPTX, HTML, TXT, Markdown | `prebuilt-documentSearch` |
| Images | JPEG, PNG, TIFF, BMP | `prebuilt-documentSearch` |
| Audio | WAV, MP3, FLAC, OGG, M4A | `prebuilt-audioSearch` |
| Video | MP4, MOV, AVI, WebM | `prebuilt-videoSearch` |
## vs. devui_multimodal_agent
| Feature | multimodal_agent | file_search_agent |
|---------|-----------------|-------------------|
| CU extraction | âś… Full content injected | âś… Content indexed in vector store |
| RAG | ❌ | ✅ file_search retrieves top-k chunks |
| Large docs (100+ pages) | ⚠️ May exceed context window | ✅ Token-efficient |
| Multiple large files | ⚠️ Context overflow risk | ✅ All indexed, searchable |
| Best for | Small docs, quick inspection | Large docs, multi-file Q&A |
@@ -0,0 +1,6 @@
# Copyright (c) Microsoft. All rights reserved.
"""DevUI Multi-Modal Agent with CU + file_search RAG."""
from .agent import agent
__all__ = ["agent"]
@@ -0,0 +1,105 @@
# Copyright (c) Microsoft. All rights reserved.
"""DevUI Multi-Modal Agent — CU extraction + file_search RAG.
This agent combines Azure Content Understanding with OpenAI file_search
for token-efficient RAG over large or multi-modal documents.
Upload flow:
1. CU extracts high-quality markdown (handles scanned PDFs, audio, video)
2. Extracted markdown is auto-uploaded to an OpenAI vector store
3. file_search tool is registered so the LLM retrieves top-k chunks
4. Vector store is configured to auto-expire after inactivity
This is ideal for large documents (100+ pages), long audio recordings,
or multiple files in the same conversation where full-context injection
would exceed the LLM's context window.
Analyzer auto-detection:
When no analyzer_id is specified, the provider auto-selects the
appropriate CU analyzer based on media type:
- Documents/images → prebuilt-documentSearch
- Audio → prebuilt-audioSearch
- Video → prebuilt-videoSearch
Required environment variables:
FOUNDRY_PROJECT_ENDPOINT — Azure AI Foundry project endpoint
FOUNDRY_MODEL — Model deployment name (e.g. gpt-4.1)
AZURE_CONTENTUNDERSTANDING_ENDPOINT — CU endpoint URL
Run with DevUI:
devui packages/azure-contentunderstanding/samples/devui_azure_openai_file_search_agent
"""
import os
from agent_framework import Agent
from agent_framework.foundry import (
ContentUnderstandingContextProvider,
FileSearchConfig,
FoundryChatClient,
)
from azure.ai.projects import AIProjectClient
from azure.core.credentials import AzureKeyCredential
from azure.identity import AzureCliCredential
from dotenv import load_dotenv
load_dotenv()
# --- Auth ---
_credential = AzureCliCredential()
_cu_api_key = os.environ.get("AZURE_CONTENTUNDERSTANDING_API_KEY")
_cu_credential = AzureKeyCredential(_cu_api_key) if _cu_api_key else _credential
_endpoint = os.environ["FOUNDRY_PROJECT_ENDPOINT"]
# --- LLM client + sync vector store setup ---
# DevUI loads agent modules synchronously at startup while an event loop is already
# running, so we cannot use async APIs here. A sync AIProjectClient is used for
# one-time vector store creation; runtime file uploads use client.client (async).
client = FoundryChatClient(
project_endpoint=_endpoint,
model=os.environ["FOUNDRY_MODEL"],
credential=_credential,
)
_sync_project = AIProjectClient(endpoint=_endpoint, credential=_credential) # type: ignore[arg-type]
_sync_openai = _sync_project.get_openai_client()
_vector_store = _sync_openai.vector_stores.create(
name="devui_cu_file_search",
expires_after={"anchor": "last_active_at", "days": 1},
)
_sync_openai.close()
_file_search_tool = client.get_file_search_tool(
vector_store_ids=[_vector_store.id],
max_num_results=3, # limit chunks to reduce input token usage
)
# --- CU context provider with file_search ---
# client.client is the async OpenAI client used for runtime file uploads.
# No analyzer_id → auto-selects per media type (documents, audio, video)
cu = ContentUnderstandingContextProvider(
endpoint=os.environ["AZURE_CONTENTUNDERSTANDING_ENDPOINT"],
credential=_cu_credential,
file_search=FileSearchConfig.from_foundry(
client.client, # reuse the LLM client's internal AsyncAzureOpenAI for file uploads
vector_store_id=_vector_store.id,
file_search_tool=_file_search_tool,
),
)
agent = Agent(
client=client,
name="FileSearchDocAgent",
instructions=(
"You are a helpful document analysis assistant with RAG capabilities. "
"When a user uploads files, they are automatically analyzed using Azure Content Understanding "
"and indexed in a vector store for efficient retrieval. "
"Analysis takes time (seconds for documents, longer for audio/video) — if a document "
"is still pending, let the user know and suggest they ask again shortly. "
"You can process PDFs, scanned documents, handwritten images, audio recordings, and video files. "
"Multiple files can be uploaded and queried in the same conversation. "
"When answering, cite specific content from the documents."
),
context_providers=[cu],
)
@@ -0,0 +1,34 @@
# DevUI Foundry File Search Agent
Interactive web UI for uploading and chatting with documents, images, audio, and video using Azure Content Understanding + Foundry file_search RAG.
This is the **Foundry** variant. For the Azure OpenAI Responses API variant, see `devui_azure_openai_file_search_agent`.
## How It Works
1. **Upload** any supported file (PDF, image, audio, video) via the DevUI chat
2. **CU analyzes** the file — auto-selects the right analyzer per media type
3. **Markdown extracted** by CU is uploaded to a Foundry vector store
4. **file_search** tool is registered — LLM retrieves top-k relevant chunks
5. **Ask questions** across all uploaded documents with token-efficient RAG
## Setup
1. Set environment variables (or create a `.env` file in `python/`):
```bash
FOUNDRY_PROJECT_ENDPOINT=https://your-project.services.ai.azure.com/
FOUNDRY_MODEL=gpt-4.1
AZURE_CONTENTUNDERSTANDING_ENDPOINT=https://your-cu-resource.services.ai.azure.com/
```
2. Log in with Azure CLI:
```bash
az login
```
3. Run with DevUI:
```bash
devui packages/azure-contentunderstanding/samples/devui_foundry_file_search_agent
```
4. Open the DevUI URL in your browser and start uploading files.
@@ -0,0 +1 @@
# Copyright (c) Microsoft. All rights reserved.
@@ -0,0 +1,109 @@
# Copyright (c) Microsoft. All rights reserved.
"""DevUI Multi-Modal Agent — CU extraction + file_search RAG via Azure AI Foundry.
This agent combines Azure Content Understanding with Foundry's file_search
for token-efficient RAG over large or multi-modal documents.
Upload flow:
1. CU extracts high-quality markdown (handles scanned PDFs, audio, video)
2. Extracted markdown is uploaded to a Foundry vector store
3. file_search tool is registered so the LLM retrieves top-k chunks
4. Uploaded files are cleaned up on server shutdown
This sample uses ``FoundryChatClient`` and ``FoundryFileSearchBackend``.
For the OpenAI Responses API variant, see ``devui_azure_openai_file_search_agent``.
Analyzer auto-detection:
When no analyzer_id is specified, the provider auto-selects the
appropriate CU analyzer based on media type:
- Documents/images → prebuilt-documentSearch
- Audio → prebuilt-audioSearch
- Video → prebuilt-videoSearch
Required environment variables:
FOUNDRY_PROJECT_ENDPOINT — Azure AI Foundry project endpoint
FOUNDRY_MODEL — Model deployment name (e.g. gpt-4.1)
AZURE_CONTENTUNDERSTANDING_ENDPOINT — CU endpoint URL
Run with DevUI:
devui packages/azure-contentunderstanding/samples/devui_foundry_file_search_agent
"""
import os
from agent_framework import Agent
from agent_framework.foundry import (
ContentUnderstandingContextProvider,
FileSearchConfig,
FoundryChatClient,
)
from azure.core.credentials import AzureKeyCredential
from azure.identity import AzureCliCredential
from dotenv import load_dotenv
from openai import AzureOpenAI
load_dotenv()
# --- Auth ---
# AzureCliCredential for Foundry. CU API key optional if on a different resource.
_credential = AzureCliCredential()
_cu_api_key = os.environ.get("AZURE_CONTENTUNDERSTANDING_API_KEY")
_cu_credential = AzureKeyCredential(_cu_api_key) if _cu_api_key else _credential
# --- Foundry LLM client ---
client = FoundryChatClient(
project_endpoint=os.environ.get("FOUNDRY_PROJECT_ENDPOINT", ""),
model=os.environ.get("FOUNDRY_MODEL", ""),
credential=_credential,
)
# --- Create vector store (sync client to avoid event loop conflicts in DevUI) ---
_token = _credential.get_token("https://ai.azure.com/.default").token
_sync_openai = AzureOpenAI(
azure_endpoint=os.environ.get("FOUNDRY_PROJECT_ENDPOINT", ""),
azure_ad_token=_token,
api_version="2025-04-01-preview",
)
_vector_store = _sync_openai.vector_stores.create(
name="devui_cu_foundry_file_search",
expires_after={"anchor": "last_active_at", "days": 1},
)
_sync_openai.close()
_file_search_tool = client.get_file_search_tool(
vector_store_ids=[_vector_store.id],
max_num_results=3, # limit chunks to reduce input token usage
)
# --- CU context provider with file_search ---
# No analyzer_id → auto-selects per media type (documents, audio, video)
cu = ContentUnderstandingContextProvider(
endpoint=os.environ["AZURE_CONTENTUNDERSTANDING_ENDPOINT"],
credential=_cu_credential,
# max_wait is the combined budget for CU analysis + vector store upload.
# For file_search mode, 10s gives enough time for small documents to be
# analyzed and indexed in one turn. Larger files (audio, video) will
# be deferred to background and resolved on the next turn.
max_wait=10.0,
file_search=FileSearchConfig.from_foundry(
client.client,
vector_store_id=_vector_store.id,
file_search_tool=_file_search_tool,
),
)
agent = Agent(
client=client,
name="FoundryFileSearchDocAgent",
instructions=(
"You are a helpful document analysis assistant with RAG capabilities. "
"When a user uploads files, they are automatically analyzed using Azure Content Understanding "
"and indexed in a vector store for efficient retrieval. "
"Analysis takes time (seconds for documents, longer for audio/video) — if a document "
"is still pending, let the user know and suggest they ask again shortly. "
"You can process PDFs, scanned documents, handwritten images, audio recordings, and video files. "
"Multiple files can be uploaded and queried in the same conversation. "
"When answering, cite specific content from the documents."
),
context_providers=[cu],
)
@@ -0,0 +1,39 @@
# Azure Content Understanding Samples
These samples demonstrate how to use the `agent-framework-azure-contentunderstanding` package to add document, image, audio, and video understanding to your agents.
## Prerequisites
1. Azure CLI logged in: `az login`
2. Environment variables set (or `.env` file in the `python/` directory):
```
FOUNDRY_PROJECT_ENDPOINT=https://your-project.services.ai.azure.com
FOUNDRY_MODEL=gpt-4.1
AZURE_CONTENTUNDERSTANDING_ENDPOINT=https://your-cu-resource.cognitiveservices.azure.com/
```
## Samples
### 01-get-started — Script samples (easy → advanced)
| # | Sample | Description | Run |
|---|--------|-------------|-----|
| 01 | [Document Q&A](01-get-started/01_document_qa.py) | Upload a PDF, ask questions with CU-powered extraction | `uv run samples/01-get-started/01_document_qa.py` |
| 02 | [Multi-Turn Session](01-get-started/02_multi_turn_session.py) | AgentSession persistence across turns | `uv run samples/01-get-started/02_multi_turn_session.py` |
| 03 | [Multi-Modal Chat](01-get-started/03_multimodal_chat.py) | PDF + audio + video parallel analysis | `uv run samples/01-get-started/03_multimodal_chat.py` |
| 04 | [Invoice Processing](01-get-started/04_invoice_processing.py) | Structured field extraction with prebuilt-invoice | `uv run samples/01-get-started/04_invoice_processing.py` |
| 05 | [Large Doc + file_search](01-get-started/05_large_doc_file_search.py) | CU extraction + OpenAI vector store RAG | `uv run samples/01-get-started/05_large_doc_file_search.py` |
### 02-devui — Interactive web UI samples
| # | Sample | Description | Run |
|---|--------|-------------|-----|
| 01 | [Multi-Modal Agent](02-devui/01-multimodal_agent/) | Web UI for file upload + CU-powered chat | `devui samples/02-devui/01-multimodal_agent` |
| 02a | [file_search (Azure OpenAI backend)](02-devui/02-file_search_agent/azure_openai_backend/) | DevUI with CU + Azure OpenAI vector store | `devui samples/02-devui/02-file_search_agent/azure_openai_backend` |
| 02b | [file_search (Foundry backend)](02-devui/02-file_search_agent/foundry_backend/) | DevUI with CU + Foundry vector store | `devui samples/02-devui/02-file_search_agent/foundry_backend` |
## Install (preview)
```bash
pip install --pre agent-framework-azure-contentunderstanding
```
@@ -0,0 +1,106 @@
# Copyright (c) Microsoft. All rights reserved.
from __future__ import annotations
import asyncio
import json
from pathlib import Path
from typing import Any
from unittest.mock import AsyncMock, MagicMock
import pytest
from azure.ai.contentunderstanding.models import AnalysisResult
FIXTURES_DIR = Path(__file__).parent / "fixtures"
def _load_fixture(name: str) -> dict[str, Any]:
return json.loads((FIXTURES_DIR / name).read_text()) # type: ignore[no-any-return]
@pytest.fixture
def pdf_fixture_raw() -> dict[str, Any]:
return _load_fixture("analyze_pdf_result.json")
@pytest.fixture
def pdf_analysis_result(pdf_fixture_raw: dict[str, Any]) -> AnalysisResult:
return AnalysisResult(pdf_fixture_raw)
@pytest.fixture
def audio_fixture_raw() -> dict[str, Any]:
return _load_fixture("analyze_audio_result.json")
@pytest.fixture
def audio_analysis_result(audio_fixture_raw: dict[str, Any]) -> AnalysisResult:
return AnalysisResult(audio_fixture_raw)
@pytest.fixture
def invoice_fixture_raw() -> dict[str, Any]:
return _load_fixture("analyze_invoice_result.json")
@pytest.fixture
def invoice_analysis_result(invoice_fixture_raw: dict[str, Any]) -> AnalysisResult:
return AnalysisResult(invoice_fixture_raw)
@pytest.fixture
def video_fixture_raw() -> dict[str, Any]:
return _load_fixture("analyze_video_result.json")
@pytest.fixture
def video_analysis_result(video_fixture_raw: dict[str, Any]) -> AnalysisResult:
return AnalysisResult(video_fixture_raw)
@pytest.fixture
def image_fixture_raw() -> dict[str, Any]:
return _load_fixture("analyze_image_result.json")
@pytest.fixture
def image_analysis_result(image_fixture_raw: dict[str, Any]) -> AnalysisResult:
return AnalysisResult(image_fixture_raw)
@pytest.fixture
def mock_cu_client() -> AsyncMock:
"""Create a mock ContentUnderstandingClient."""
client = AsyncMock()
client.close = AsyncMock()
return client
def make_mock_poller(result: AnalysisResult) -> AsyncMock:
"""Create a mock poller that returns the given result immediately."""
poller = AsyncMock()
poller.result = AsyncMock(return_value=result)
poller.continuation_token = MagicMock(return_value="mock_continuation_token")
poller.done = MagicMock(return_value=True)
return poller
def make_slow_poller(result: AnalysisResult, delay: float = 10.0) -> MagicMock:
"""Create a mock poller that simulates a timeout then eventually returns."""
poller = MagicMock()
async def slow_result() -> AnalysisResult:
await asyncio.sleep(delay)
return result
poller.result = slow_result
poller.continuation_token = MagicMock(return_value="mock_slow_continuation_token")
poller.done = MagicMock(return_value=False)
return poller
def make_failing_poller(error: Exception) -> AsyncMock:
"""Create a mock poller that raises an exception."""
poller = AsyncMock()
poller.result = AsyncMock(side_effect=error)
return poller
@@ -0,0 +1,13 @@
{
"id": "synthetic-audio-001",
"status": "Succeeded",
"analyzer_id": "prebuilt-audioSearch",
"api_version": "2025-05-01-preview",
"created_at": "2026-03-21T10:05:00Z",
"contents": [
{
"markdown": "## Call Center Recording\n\n**Duration:** 2 minutes 15 seconds\n**Speakers:** 2\n\n### Transcript\n\n**Speaker 1 (Agent):** Thank you for calling Contoso support. My name is Sarah. How can I help you today?\n\n**Speaker 2 (Customer):** Hi Sarah, I'm calling about my recent order number ORD-5678. It was supposed to arrive yesterday but I haven't received it.\n\n**Speaker 1 (Agent):** I'm sorry to hear that. Let me look up your order. Can you confirm your name and email address?\n\n**Speaker 2 (Customer):** Sure, it's John Smith, john.smith@example.com.\n\n**Speaker 1 (Agent):** Thank you, John. I can see your order was shipped on March 18th. It looks like there was a delay with the carrier. The updated delivery estimate is March 22nd.\n\n**Speaker 2 (Customer):** That's helpful, thank you. Is there anything I can do to track it?\n\n**Speaker 1 (Agent):** Yes, I'll send you a tracking link to your email right away. Is there anything else I can help with?\n\n**Speaker 2 (Customer):** No, that's all. Thanks for your help.\n\n**Speaker 1 (Agent):** You're welcome! Have a great day.",
"fields": {}
}
]
}
@@ -0,0 +1,857 @@
{
"analyzerId": "prebuilt-documentSearch",
"apiVersion": "2025-11-01",
"createdAt": "2026-03-21T22:44:21Z",
"stringEncoding": "codePoint",
"warnings": [],
"contents": [
{
"path": "input1",
"markdown": "# Contoso Q1 2025 Financial Summary\n\nTotal revenue for Q1 2025 was $42.7 million, an increase of 18% over Q1 2024.\nOperating expenses were $31.2 million. Net profit was $11.5 million. The largest\nrevenue segment was Cloud Services at $19.3 million, followed by Professional\nServices at $14.8 million and Product Licensing at $8.6 million. Headcount at end of\nQ1 was 1,247 employees across 8 offices worldwide.\n",
"fields": {
"Summary": {
"type": "string",
"valueString": "The document provides a financial summary for Contoso in Q1 2025, reporting total revenue of $42.7 million, an 18% increase from Q1 2024. Operating expenses were $31.2 million, resulting in a net profit of $11.5 million. The largest revenue segment was Cloud Services with $19.3 million, followed by Professional Services at $14.8 million and Product Licensing at $8.6 million. The company had 1,247 employees across 8 offices worldwide at the end of Q1.",
"spans": [
{
"offset": 37,
"length": 77
},
{
"offset": 115,
"length": 80
},
{
"offset": 196,
"length": 77
},
{
"offset": 274,
"length": 84
},
{
"offset": 359,
"length": 50
}
],
"confidence": 0.592,
"source": "D(1,212.0000,334.0000,1394.0000,334.0000,1394.0000,374.0000,212.0000,374.0000);D(1,213.0000,379.0000,1398.0000,379.0000,1398.0000,422.0000,213.0000,422.0000);D(1,212.0000,423.0000,1389.0000,423.0000,1389.0000,464.0000,212.0000,464.0000);D(1,213.0000,468.0000,1453.0000,468.0000,1453.0000,510.0000,213.0000,510.0000);D(1,213.0000,512.0000,1000.0000,512.0000,1000.0000,554.0000,213.0000,554.0000)"
}
},
"kind": "document",
"startPageNumber": 1,
"endPageNumber": 1,
"unit": "pixel",
"pages": [
{
"pageNumber": 1,
"angle": -0.0242,
"width": 1700,
"height": 2200,
"spans": [
{
"offset": 0,
"length": 410
}
],
"words": [
{
"content": "Contoso",
"span": {
"offset": 2,
"length": 7
},
"confidence": 0.99,
"source": "D(1,214,222,401,222,401,274,214,273)"
},
{
"content": "Q1",
"span": {
"offset": 10,
"length": 2
},
"confidence": 0.957,
"source": "D(1,414,222,473,222,473,275,414,274)"
},
{
"content": "2025",
"span": {
"offset": 13,
"length": 4
},
"confidence": 0.929,
"source": "D(1,494,222,607,222,607,276,494,275)"
},
{
"content": "Financial",
"span": {
"offset": 18,
"length": 9
},
"confidence": 0.975,
"source": "D(1,624,222,819,223,819,277,624,276)"
},
{
"content": "Summary",
"span": {
"offset": 28,
"length": 7
},
"confidence": 0.991,
"source": "D(1,836,223,1050,225,1050,279,836,277)"
},
{
"content": "Total",
"span": {
"offset": 37,
"length": 5
},
"confidence": 0.996,
"source": "D(1,212,335,287,334,288,374,212,373)"
},
{
"content": "revenue",
"span": {
"offset": 43,
"length": 7
},
"confidence": 0.994,
"source": "D(1,299,334,417,334,418,374,299,374)"
},
{
"content": "for",
"span": {
"offset": 51,
"length": 3
},
"confidence": 0.994,
"source": "D(1,427,334,467,334,467,374,427,374)"
},
{
"content": "Q1",
"span": {
"offset": 55,
"length": 2
},
"confidence": 0.944,
"source": "D(1,475,334,515,334,515,374,475,374)"
},
{
"content": "2025",
"span": {
"offset": 58,
"length": 4
},
"confidence": 0.876,
"source": "D(1,528,334,604,334,604,374,529,374)"
},
{
"content": "was",
"span": {
"offset": 63,
"length": 3
},
"confidence": 0.991,
"source": "D(1,613,334,672,334,672,374,613,374)"
},
{
"content": "$",
"span": {
"offset": 67,
"length": 1
},
"confidence": 0.999,
"source": "D(1,681,334,698,334,698,374,681,374)"
},
{
"content": "42.7",
"span": {
"offset": 68,
"length": 4
},
"confidence": 0.946,
"source": "D(1,700,334,765,334,765,374,700,374)"
},
{
"content": "million",
"span": {
"offset": 73,
"length": 7
},
"confidence": 0.977,
"source": "D(1,775,334,867,334,867,374,776,374)"
},
{
"content": ",",
"span": {
"offset": 80,
"length": 1
},
"confidence": 0.998,
"source": "D(1,870,334,877,334,877,374,870,374)"
},
{
"content": "an",
"span": {
"offset": 82,
"length": 2
},
"confidence": 0.998,
"source": "D(1,888,334,922,334,922,374,888,374)"
},
{
"content": "increase",
"span": {
"offset": 85,
"length": 8
},
"confidence": 0.991,
"source": "D(1,934,334,1058,335,1059,374,934,374)"
},
{
"content": "of",
"span": {
"offset": 94,
"length": 2
},
"confidence": 0.982,
"source": "D(1,1069,335,1098,335,1098,374,1069,374)"
},
{
"content": "18",
"span": {
"offset": 97,
"length": 2
},
"confidence": 0.963,
"source": "D(1,1108,335,1142,335,1142,374,1108,374)"
},
{
"content": "%",
"span": {
"offset": 99,
"length": 1
},
"confidence": 0.998,
"source": "D(1,1143,335,1171,335,1171,374,1143,374)"
},
{
"content": "over",
"span": {
"offset": 101,
"length": 4
},
"confidence": 0.946,
"source": "D(1,1181,335,1248,335,1248,374,1181,374)"
},
{
"content": "Q1",
"span": {
"offset": 106,
"length": 2
},
"confidence": 0.875,
"source": "D(1,1256,335,1295,335,1295,374,1256,374)"
},
{
"content": "2024",
"span": {
"offset": 109,
"length": 4
},
"confidence": 0.683,
"source": "D(1,1310,335,1384,335,1384,374,1310,374)"
},
{
"content": ".",
"span": {
"offset": 113,
"length": 1
},
"confidence": 0.991,
"source": "D(1,1385,335,1394,335,1394,374,1385,374)"
},
{
"content": "Operating",
"span": {
"offset": 115,
"length": 9
},
"confidence": 0.996,
"source": "D(1,213,380,358,380,358,422,213,422)"
},
{
"content": "expenses",
"span": {
"offset": 125,
"length": 8
},
"confidence": 0.997,
"source": "D(1,369,380,513,379,513,421,369,421)"
},
{
"content": "were",
"span": {
"offset": 134,
"length": 4
},
"confidence": 0.998,
"source": "D(1,521,379,595,379,595,421,521,421)"
},
{
"content": "$",
"span": {
"offset": 139,
"length": 1
},
"confidence": 0.999,
"source": "D(1,603,379,620,379,620,421,603,421)"
},
{
"content": "31.2",
"span": {
"offset": 140,
"length": 4
},
"confidence": 0.938,
"source": "D(1,623,379,686,379,686,421,623,421)"
},
{
"content": "million",
"span": {
"offset": 145,
"length": 7
},
"confidence": 0.913,
"source": "D(1,696,379,790,379,790,421,696,421)"
},
{
"content": ".",
"span": {
"offset": 152,
"length": 1
},
"confidence": 0.975,
"source": "D(1,793,379,800,379,800,421,793,421)"
},
{
"content": "Net",
"span": {
"offset": 154,
"length": 3
},
"confidence": 0.976,
"source": "D(1,811,379,862,379,862,420,811,421)"
},
{
"content": "profit",
"span": {
"offset": 158,
"length": 6
},
"confidence": 0.993,
"source": "D(1,871,379,947,379,947,420,871,420)"
},
{
"content": "was",
"span": {
"offset": 165,
"length": 3
},
"confidence": 0.997,
"source": "D(1,954,379,1012,379,1012,420,953,420)"
},
{
"content": "$",
"span": {
"offset": 169,
"length": 1
},
"confidence": 0.998,
"source": "D(1,1021,379,1039,379,1039,420,1021,420)"
},
{
"content": "11.5",
"span": {
"offset": 170,
"length": 4
},
"confidence": 0.954,
"source": "D(1,1043,379,1106,379,1106,421,1043,420)"
},
{
"content": "million",
"span": {
"offset": 175,
"length": 7
},
"confidence": 0.837,
"source": "D(1,1118,379,1208,379,1208,421,1118,421)"
},
{
"content": ".",
"span": {
"offset": 182,
"length": 1
},
"confidence": 0.978,
"source": "D(1,1210,379,1217,379,1217,421,1210,421)"
},
{
"content": "The",
"span": {
"offset": 184,
"length": 3
},
"confidence": 0.949,
"source": "D(1,1228,379,1285,379,1285,421,1228,421)"
},
{
"content": "largest",
"span": {
"offset": 188,
"length": 7
},
"confidence": 0.978,
"source": "D(1,1295,379,1398,379,1398,421,1295,421)"
},
{
"content": "revenue",
"span": {
"offset": 196,
"length": 7
},
"confidence": 0.995,
"source": "D(1,212,425,334,425,334,464,212,464)"
},
{
"content": "segment",
"span": {
"offset": 204,
"length": 7
},
"confidence": 0.996,
"source": "D(1,344,425,472,424,472,464,344,464)"
},
{
"content": "was",
"span": {
"offset": 212,
"length": 3
},
"confidence": 0.998,
"source": "D(1,480,424,541,424,541,464,480,464)"
},
{
"content": "Cloud",
"span": {
"offset": 216,
"length": 5
},
"confidence": 0.997,
"source": "D(1,550,424,636,424,637,464,551,464)"
},
{
"content": "Services",
"span": {
"offset": 222,
"length": 8
},
"confidence": 0.995,
"source": "D(1,647,424,774,424,774,464,647,464)"
},
{
"content": "at",
"span": {
"offset": 231,
"length": 2
},
"confidence": 0.996,
"source": "D(1,784,424,812,424,812,464,784,464)"
},
{
"content": "$",
"span": {
"offset": 234,
"length": 1
},
"confidence": 0.998,
"source": "D(1,820,424,837,424,837,464,820,464)"
},
{
"content": "19.3",
"span": {
"offset": 235,
"length": 4
},
"confidence": 0.879,
"source": "D(1,840,424,903,423,903,463,840,464)"
},
{
"content": "million",
"span": {
"offset": 240,
"length": 7
},
"confidence": 0.876,
"source": "D(1,915,423,1006,423,1006,463,915,463)"
},
{
"content": ",",
"span": {
"offset": 247,
"length": 1
},
"confidence": 0.999,
"source": "D(1,1008,423,1015,423,1015,463,1008,463)"
},
{
"content": "followed",
"span": {
"offset": 249,
"length": 8
},
"confidence": 0.978,
"source": "D(1,1026,423,1148,424,1148,463,1026,463)"
},
{
"content": "by",
"span": {
"offset": 258,
"length": 2
},
"confidence": 0.986,
"source": "D(1,1160,424,1194,424,1194,463,1160,463)"
},
{
"content": "Professional",
"span": {
"offset": 261,
"length": 12
},
"confidence": 0.965,
"source": "D(1,1204,424,1389,424,1389,463,1204,463)"
},
{
"content": "Services",
"span": {
"offset": 274,
"length": 8
},
"confidence": 0.991,
"source": "D(1,213,469,341,469,341,510,213,510)"
},
{
"content": "at",
"span": {
"offset": 283,
"length": 2
},
"confidence": 0.997,
"source": "D(1,352,469,380,469,380,510,352,510)"
},
{
"content": "$",
"span": {
"offset": 286,
"length": 1
},
"confidence": 0.998,
"source": "D(1,388,469,405,469,405,510,388,510)"
},
{
"content": "14.8",
"span": {
"offset": 287,
"length": 4
},
"confidence": 0.973,
"source": "D(1,410,469,472,469,472,510,410,510)"
},
{
"content": "million",
"span": {
"offset": 292,
"length": 7
},
"confidence": 0.987,
"source": "D(1,483,469,575,469,575,510,483,510)"
},
{
"content": "and",
"span": {
"offset": 300,
"length": 3
},
"confidence": 0.999,
"source": "D(1,585,469,638,469,638,510,585,510)"
},
{
"content": "Product",
"span": {
"offset": 304,
"length": 7
},
"confidence": 0.995,
"source": "D(1,652,469,765,469,765,510,652,510)"
},
{
"content": "Licensing",
"span": {
"offset": 312,
"length": 9
},
"confidence": 0.993,
"source": "D(1,777,469,914,469,914,510,777,510)"
},
{
"content": "at",
"span": {
"offset": 322,
"length": 2
},
"confidence": 0.998,
"source": "D(1,925,469,953,469,953,510,925,510)"
},
{
"content": "$",
"span": {
"offset": 325,
"length": 1
},
"confidence": 0.998,
"source": "D(1,961,469,978,469,978,510,961,510)"
},
{
"content": "8.6",
"span": {
"offset": 326,
"length": 3
},
"confidence": 0.958,
"source": "D(1,980,469,1025,469,1025,510,980,510)"
},
{
"content": "million",
"span": {
"offset": 330,
"length": 7
},
"confidence": 0.908,
"source": "D(1,1036,469,1128,468,1128,510,1036,510)"
},
{
"content": ".",
"span": {
"offset": 337,
"length": 1
},
"confidence": 0.987,
"source": "D(1,1130,468,1137,468,1137,510,1130,510)"
},
{
"content": "Headcount",
"span": {
"offset": 339,
"length": 9
},
"confidence": 0.934,
"source": "D(1,1150,468,1310,468,1310,510,1150,510)"
},
{
"content": "at",
"span": {
"offset": 349,
"length": 2
},
"confidence": 0.993,
"source": "D(1,1318,468,1348,468,1348,510,1318,510)"
},
{
"content": "end",
"span": {
"offset": 352,
"length": 3
},
"confidence": 0.947,
"source": "D(1,1355,468,1410,468,1410,510,1355,510)"
},
{
"content": "of",
"span": {
"offset": 356,
"length": 2
},
"confidence": 0.974,
"source": "D(1,1419,468,1453,468,1453,509,1419,509)"
},
{
"content": "Q1",
"span": {
"offset": 359,
"length": 2
},
"confidence": 0.931,
"source": "D(1,213,512,252,512,252,554,213,554)"
},
{
"content": "was",
"span": {
"offset": 362,
"length": 3
},
"confidence": 0.847,
"source": "D(1,267,512,326,512,326,554,267,554)"
},
{
"content": "1,247",
"span": {
"offset": 366,
"length": 5
},
"confidence": 0.523,
"source": "D(1,338,512,419,512,419,554,338,554)"
},
{
"content": "employees",
"span": {
"offset": 372,
"length": 9
},
"confidence": 0.972,
"source": "D(1,429,513,591,512,591,554,429,554)"
},
{
"content": "across",
"span": {
"offset": 382,
"length": 6
},
"confidence": 0.972,
"source": "D(1,601,512,697,512,697,554,601,554)"
},
{
"content": "8",
"span": {
"offset": 389,
"length": 1
},
"confidence": 0.946,
"source": "D(1,708,512,725,512,725,553,708,554)"
},
{
"content": "offices",
"span": {
"offset": 391,
"length": 7
},
"confidence": 0.95,
"source": "D(1,736,512,831,512,831,553,736,553)"
},
{
"content": "worldwide",
"span": {
"offset": 399,
"length": 9
},
"confidence": 0.988,
"source": "D(1,840,512,989,512,989,552,840,553)"
},
{
"content": ".",
"span": {
"offset": 408,
"length": 1
},
"confidence": 0.996,
"source": "D(1,991,512,1000,512,1000,552,991,552)"
}
],
"lines": [
{
"content": "Contoso Q1 2025 Financial Summary",
"source": "D(1,214,221,1050,225,1050,279,213,273)",
"span": {
"offset": 2,
"length": 33
}
},
{
"content": "Total revenue for Q1 2025 was $42.7 million, an increase of 18% over Q1 2024.",
"source": "D(1,212,334,1394,335,1394,374,212,374)",
"span": {
"offset": 37,
"length": 77
}
},
{
"content": "Operating expenses were $31.2 million. Net profit was $11.5 million. The largest",
"source": "D(1,213,379,1398,378,1398,421,213,422)",
"span": {
"offset": 115,
"length": 80
}
},
{
"content": "revenue segment was Cloud Services at $19.3 million, followed by Professional",
"source": "D(1,212,424,1389,423,1389,463,212,464)",
"span": {
"offset": 196,
"length": 77
}
},
{
"content": "Services at $14.8 million and Product Licensing at $8.6 million. Headcount at end of",
"source": "D(1,213,469,1453,468,1453,510,213,511)",
"span": {
"offset": 274,
"length": 84
}
},
{
"content": "Q1 was 1,247 employees across 8 offices worldwide.",
"source": "D(1,213,512,1000,512,1000,554,213,554)",
"span": {
"offset": 359,
"length": 50
}
}
]
}
],
"paragraphs": [
{
"role": "title",
"content": "Contoso Q1 2025 Financial Summary",
"source": "D(1,214,219,1050,225,1050,279,213,273)",
"span": {
"offset": 0,
"length": 35
}
},
{
"content": "Total revenue for Q1 2025 was $42.7 million, an increase of 18% over Q1 2024. Operating expenses were $31.2 million. Net profit was $11.5 million. The largest revenue segment was Cloud Services at $19.3 million, followed by Professional Services at $14.8 million and Product Licensing at $8.6 million. Headcount at end of Q1 was 1,247 employees across 8 offices worldwide.",
"source": "D(1,212,334,1453,333,1454,553,212,554)",
"span": {
"offset": 37,
"length": 372
}
}
],
"sections": [
{
"span": {
"offset": 0,
"length": 409
},
"elements": [
"/paragraphs/0",
"/paragraphs/1"
]
}
],
"analyzerId": "prebuilt-documentSearch",
"mimeType": "image/png"
}
]
}
@@ -0,0 +1,114 @@
{
"analyzerId": "prebuilt-invoice",
"apiVersion": "2025-11-01",
"createdAt": "2026-03-21T22:44:33Z",
"stringEncoding": "codePoint",
"warnings": [],
"contents": [
{
"markdown": "# Master Services Agreement\n\nClient: Alpine Industries Inc.\n\nContract Reference: MSA-2025-ALP-00847\n\nEffective Date: January 15, 2025\nPrepared for: Robert Chen, Chief Executive Officer, Alpine Industries Inc.\n\nAddress: 742 Evergreen Blvd, Denver, CO 80203\n\nThis Master Services Agreement (the 'Agreement') is entered into by and between Alpine Industries\nInc. (the 'Client') and TechServe Global Partners (the 'Provider'). This agreement governs the provision\nof managed technology services as descri",
"fields": {
"VendorName": {
"type": "string",
"valueString": "TechServe Global Partners",
"confidence": 0.71
},
"DueDate": {
"type": "date",
"valueDate": "2025-02-15",
"confidence": 0.793
},
"InvoiceDate": {
"type": "date",
"valueDate": "2025-01-15",
"confidence": 0.693
},
"InvoiceId": {
"type": "string",
"valueString": "INV-100",
"confidence": 0.489
},
"AmountDue": {
"type": "object",
"valueObject": {
"Amount": {
"type": "number",
"valueNumber": 610,
"confidence": 0.758
},
"CurrencyCode": {
"type": "string",
"valueString": "USD"
}
}
},
"SubtotalAmount": {
"type": "object",
"valueObject": {
"Amount": {
"type": "number",
"valueNumber": 100,
"confidence": 0.902
},
"CurrencyCode": {
"type": "string",
"valueString": "USD"
}
}
},
"LineItems": {
"type": "array",
"valueArray": [
{
"type": "object",
"valueObject": {
"Description": {
"type": "string",
"valueString": "Consulting Services",
"confidence": 0.664
},
"Quantity": {
"type": "number",
"valueNumber": 2,
"confidence": 0.957
},
"UnitPrice": {
"type": "object",
"valueObject": {
"Amount": {
"type": "number",
"valueNumber": 30,
"confidence": 0.956
},
"CurrencyCode": {
"type": "string",
"valueString": "USD"
}
}
}
}
},
{
"type": "object",
"valueObject": {
"Description": {
"type": "string",
"valueString": "Document Fee",
"confidence": 0.712
},
"Quantity": {
"type": "number",
"valueNumber": 3,
"confidence": 0.939
}
}
}
]
}
},
"kind": "document",
"startPageNumber": 1,
"endPageNumber": 100
}
]
}

Some files were not shown because too many files have changed in this diff Show More