Compare commits

...
Author SHA1 Message Date
copilot-swe-agent[bot]andGitHub aee043d17e Revert unnecessary bool simplification in WorkflowAgent input selection 2026-05-27 21:30:44 +00:00
copilot-swe-agent[bot]andGitHub 5c52926123 Polish WorkflowAgent restore resume logic and tests 2026-05-27 18:38:44 +00:00
copilot-swe-agent[bot]andGitHub 02e5600ede Fix WorkflowAgent pending request resume after session restore 2026-05-27 18:37:40 +00:00
copilot-swe-agent[bot]andGitHub 9361cda413 Initial plan 2026-05-27 18:29:13 +00:00
ef86fb51d5 Python: Add a HarnessAgent with available features and sample (#6041)
* Add a HarnessAgent with available features and sample

* Fix formatting

* Address PR comments and fix mypy error

* Add web search support to HarnessAgent

* Fix build warning

* Apply suggestions from code review

Co-authored-by: Eduard van Valkenburg <eavanvalkenburg@users.noreply.github.com>

* Address PR comments

* Address PR comments

* Address further PR comments.

* Fix markdown broken link

---------

Co-authored-by: Eduard van Valkenburg <eavanvalkenburg@users.noreply.github.com>
2026-05-27 14:54:00 +01:00
d5c07f2623 Python: feat(foundry): add to_prompt_agent / deploy_as_prompt_agent (experimental) (#5959)
* feat(foundry): add experimental to_prompt_agent converter

Adds `to_prompt_agent(agent)`, an experimental converter
(`ExperimentalFeature.TO_PROMPT_AGENT`) that turns an Agent Framework
`Agent` into a Foundry `PromptAgentDefinition` ready to publish via
`AIProjectClient.agents.create_version(...)`.

Behaviour:

* `agent.client` must be a `FoundryChatClient` (or subclass); otherwise
  `TypeError` is raised. The model deployment name is lifted from the
  bound client so the same Agent definition used for local runs can be
  published as a hosted prompt agent without restating the model.
* Foundry SDK tool instances (from `FoundryChatClient.get_*_tool()`) are
  passed through unchanged. AF `FunctionTool`s (and `@tool`-decorated
  callables) are emitted as Foundry `FunctionTool` declarations.
* Local AF MCP tools cannot be expressed in a `PromptAgentDefinition`;
  the converter raises `ValueError` and points at
  `FoundryChatClient.get_mcp_tool()` for hosted MCP servers.
* The converter walks both `agent.default_options["tools"]` and
  `agent.mcp_tools` because `normalize_tools()` splits local MCP off
  into its own list.

Re-exported through the `agent_framework.foundry` lazy-loading namespace
(updates both `__init__.py` and the `__init__.pyi` type stub).

Adds a portable-agent sample showing the same `Agent` driven through
both `agent.run(...)` and `to_prompt_agent(agent)`, and a README section
covering the new converter.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore(samples): remove snippet tags from portable agent sample

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore(samples): inline FoundryChatClient and enable prompt-agent publish

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore(samples): drop async credential context manager

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(foundry): trim README to_prompt_agent example to publish-only flow

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(foundry): note FoundryAgent runs @tool callables for deployed prompt agents

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(foundry): address review comments on to_prompt_agent converter

* Construct `PromptAgentDefinition` `Tool` from a dict via `**tool_item`
  unpacking rather than the positional Mapping constructor \u2014 cleaner and
  matches the typical Pydantic / Azure SDK pattern.
* Drop the redundant `isinstance(mcp_tool, MCPTool)` guard in
  `_convert_tools`; the parameter is already typed `Iterable[MCPTool]` so
  the second `raise` was unreachable. The remaining single `raise`
  fires for every entry as intended.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(foundry): match Agent.__init__ model resolution in to_prompt_agent

* Read the model from `agent.default_options.get("model")` first,
  falling back to `agent.client.model`. This mirrors the order
  `Agent.__init__` uses (`_agents.py:740`) when assembling
  default_options, so the model the agent runs with is the same model
  the converter publishes \u2014 e.g. when the caller passes
  `default_options={"model": "..."}` to override the bound client.
* Updated the missing-model error message to point at both the client
  and the default_options paths.
* Added tests:
  * tool-only agent with no `instructions` produces a definition
    where `instructions` is `None` and is omitted from the dict
    payload (`Agent.__init__` strips None values from default_options
    before storing them).
  * `default_options['model']` wins over the bound client's model.
  * Fallback to client.model when default_options has no model.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat(foundry): add deploy_as_prompt_agent helper + samples

Adds `deploy_as_prompt_agent(agent)`, a convenience wrapper around
`to_prompt_agent` that reuses the bound FoundryChatClient's project
client to call `project_client.agents.create_version(...)`. Defaults
`agent_name` / `description` from `agent.name` / `agent.description`
so the Agent stays the single source of truth.

* Exposed from `agent_framework_foundry` and the lazy-loading
  `agent_framework.foundry` namespace (including the .pyi stub).
* Marked experimental with the existing
  `ExperimentalFeature.TO_PROMPT_AGENT` tag.
* Tests cover the happy path, name/description defaulting, explicit
  override, no-name error, metadata + description forwarding, extra
  kwargs passthrough, and the experimental metadata.

Samples:
* Renamed the existing sample to `creating_prompt_agents.py`, drops
  'portable' wording, presents `deploy_as_prompt_agent` first as the
  recommended path and `to_prompt_agent` + `AIProjectClient` as the
  two-step alternative, and adds a cleanup step that deletes the
  published agent so re-runs stay idempotent.
* New `using_prompt_agents.py` shows the end-to-end loop: deploy the
  agent, connect to it with `FoundryAgent` passing the same local
  `@tool` callable, run a query against the deployed prompt agent,
  then clean up.

README updated to introduce `deploy_as_prompt_agent` as the
recommended path and link to both runnable samples.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(foundry): restore missing-model ValueError in to_prompt_agent

The check was accidentally dropped while reworking docstrings in the
previous commit. Test `test_to_prompt_agent_rejects_missing_model`
exercises this path and was failing on CI as a result.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* refactor(foundry): rename deploy_as_prompt_agent -> create_prompt_agent

Renames the helper across the foundry package, core lazy-loader stubs,
tests, README and samples. The new name better matches the action
performed (a prompt-agent definition is created in Foundry) and is
consistent with the surrounding ''create_*'' API surface.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* refactor(foundry): drop create_prompt_agent, enrich to_prompt_agent params

Remove the create_prompt_agent helper and consolidate on to_prompt_agent.
Expose every PromptAgentDefinition parameter that has either an Agent
Framework equivalent (sourced from default_options) or no equivalent
(accepted as a keyword argument).

* default_options-sourced (with kwarg overrides):
  temperature, top_p, string tool_choice
* kwarg-only Foundry knobs:
  reasoning, text, structured_inputs, rai_config, ToolChoiceParam tool_choice

Precedence is always: explicit keyword > default_options entry > unset.

Tests cover every path (defaults, default_options, kwargs, kwarg override).
Samples and README rewritten around the enriched to_prompt_agent.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* refactor(foundry): single source of truth for prompt-agent options

Stop duplicating the generation-parameter surface between FoundryChatOptions
and to_prompt_agent. Translate every field with an Agent Framework equivalent
(temperature, top_p, tool_choice, reasoning, response_format/text/verbosity)
from agent.default_options via a new RawFoundryChatClient helper
_prepare_prompt_agent_options. Only Foundry-specific fields with no AF
equivalent — structured_inputs and rai_config — remain as keyword arguments
on to_prompt_agent.

- tool_choice is dropped when there are no tools (mirrors _prepare_options
  semantics and avoids polluting tool-less prompt agents with Agent.__init__'s
  'auto' default).
- response_format Pydantic models route through
  openai.lib._parsing._responses.type_to_text_format_param; dict shapes go
  through the existing _prepare_response_and_text_format helper.
- default_options is not mutated; text dict is defensively copied.

Tests, README, and creating_prompt_agents.py sample updated to reflect the
new single-source model.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(foundry): consolidate prompt-agent sample

Drop creating_prompt_agents.py (the publish-only variant) and rename
using_prompt_agents.py to foundry_prompt_agents.py so the single sample
covers the full convert -> publish -> connect -> run loop. Update the
README link list accordingly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(foundry): run local Agent + deployed agent in same sample

Add an agent.run() call against the local Agent before publishing, then run
the deployed prompt agent on the same query. Expand the docstring with a
compare-and-contrast covering runtime/latency, configurability, and
persistence/sharing differences between the two execution paths.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* test(foundry): cover conflicting response_format + text.format in to_prompt_agent

Exercises the ValueError path when a Pydantic response_format would overwrite
an explicit text.format mapping with a different shape. Lifts _chat_client.py
coverage from 89% to 90%.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* refactor(foundry): move _prepare_prompt_agent_options into _to_prompt_agent

Lift the translation helper off RawFoundryChatClient and into the
_to_prompt_agent module as a module-private function that takes the client
as its first argument. The chat client no longer needs to carry a method
whose only consumer is the prompt-agent converter, while still serving as
the source of the request-path helper (_prepare_response_and_text_format)
that the converter reuses for dict-shaped response_format values.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(python): codify GA terminology + post-run docs review

Add two pieces of guidance to python/AGENTS.md:

* Terminology - reserve 'GA' for hosted services; use 'released' or 'stable'
  for Agent Framework code/features to match the feature-lifecycle stages.
* Maintaining Documentation - review AGENTS.md and skills at the end of every
  run and update any guidance the conversation made stale; before adding a
  new principle, ask the user to confirm it should be captured.

Also pulls in a docstring fix in foundry_prompt_agents.py that swaps the
stray 'GA' for 'released', applying the new terminology rule.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* address PR review: strict=True default, Tool._deserialize dispatch, sample cleanup safety

- FunctionTool published as strict=True so the server-side schema validation
  matches what the local FoundryAgent(tools=[same_callable]) dispatcher
  enforces. AF FunctionTool has no 'strict' attribute, so the safer default
  is used uniformly instead of silently downgrading to a permissive contract.
- _validate_mapping_tool now dispatches through ProjectsTool._deserialize so
  dict-shaped tools rehydrate to the concrete subclass (FunctionTool,
  WebSearchTool, ...) via the 'type' discriminator instead of returning a
  generic Tool. Added a test that asserts isinstance(WebSearchTool) and a
  new test for the function-typed dict path.
- foundry_prompt_agents.py sample now wraps credential + project client in
  async with and the create_version / run flow in try/finally so a failure
  on connect or run still deletes the published prompt agent rather than
  leaving an orphaned, billable resource in the user's Foundry project.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(ci): correct linkspector ignorePattern typo (./pulls -> ./pull)

GitHub PR URLs use the singular segment /pull/N (compare to /issues/N
for issues). The existing './pulls' ignore pattern never matched
anything as a result, so legitimately stale PR links (e.g. PRs deleted
from forks) surface as linkspector failures on unrelated PRs.

This is the same convention the './issues' rule above already follows.
Fixes the markdown-link-check failure on a dangling link in
dotnet/src/Microsoft.Agents.AI.DurableTask/CHANGELOG.md.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-27 13:31:21 +00:00
westeyandGitHub ae989b92e7 Python: Add a BackgroundAgentsProvider for python (#6069)
* Add a BackgroundAgentsProvider for python

* Address PR comments and fix linting warnings

* Address PR comment
2026-05-27 09:12:01 +00:00
3242d8a4c4 Python: Fix DevUI streaming memory growth regression (#6038)
* Fix DevUI streaming memory growth regression

Bounds retained streaming/debug state in DevUI and strengthens browser regression coverage for long streamed responses.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address DevUI memory review feedback

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix DevUI bundle trailing whitespace

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-27 07:48:29 +00:00
e1e6e3d35e Python: fix(openai): guard against null delta in streaming chunks from non-co… (#5734)
* fix(openai): guard against null delta in streaming chunks from non-compliant providers (#5732)

* chore: resolve nit and align with project style

---------

Co-authored-by: Sergey Borisov <sergey.borisov@dataimpact.io>
Co-authored-by: Giles Odigwe <79032838+giles17@users.noreply.github.com>
2026-05-27 07:42:46 +00:00
Peter IbekweandGitHub 08697f8037 Persist ForeachExecutor iteration state across checkpoints (#6051) 2026-05-26 18:26:12 +00:00
b0f5fa541c .NET: Updating version for dotnet release 1.7.0 (#6093)
* Updating version for dotnet release 1.6.3

* Change to minor version bump.

---------

Co-authored-by: Ben Thomas <25218250+alliscode@users.noreply.github.com>
2026-05-26 18:11:18 +00:00
e3290a2d22 Adding shell tool project to release solution (#6092)
Co-authored-by: Ben Thomas <25218250+alliscode@users.noreply.github.com>
2026-05-26 17:56:04 +00:00
200488cb08 Python: Add Python parity sample for invoking Foundry Toolbox tools from declarative workflows (#5933)
* Add Python parity sample for invoking Foundry Toolbox tools from declarative workflows

* Python: address PR review on declarative toolbox sample

Two security fixes for PR #5933:

1. Add safe_mode flag to WorkflowFactory (default True) mirroring
   AgentFactory. Gates =Env.* exposure inside DeclarativeWorkflowState
   PowerFx symbols via _safe_mode_context, so workflow YAML loaded from
   untrusted sources no longer leaks the host's full os.environ snapshot
   into PowerFx evaluation. The flag is also forwarded to the
   internally-constructed AgentFactory so inline agent definitions
   follow the same policy.

2. Pin the invoke_foundry_toolbox_mcp sample's _client_provider to the
   resolved toolbox endpoint. The bearer-authenticated httpx client is
   now only returned when MCPToolInvocation.server_url matches the
   toolbox URL case-insensitively; any other URL gets None (the default
   unauthenticated path), preventing the Foundry AAD bearer token from
   being attached to a mis-configured or injected server URL. Mirrors
   the .NET sample's httpClientProvider guard.

The sample is updated to opt in to safe_mode=False because its YAML
intentionally uses =Env.FOUNDRY_TOOLBOX_* to keep configuration in env
vars under the developer's control.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix pyright issues.

* Addressed PR comments.

* Fix CI pipelines.

* Resolve PR comments

* Revamped sample to address PR comments.

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-26 15:36:33 +00:00
westeyandGitHub bd4fc64b4d Python: Align ModeProvider tool names and instructions (#6071)
* Align ModeProvider tool names and instructions

* Address PR comments
2026-05-26 14:37:34 +00:00
Peter IbekweandGitHub b2e77067e9 Fix Foreach body exit wiring in declarative workflows (#6050) 2026-05-26 06:37:35 +00:00
08541ee5a9 .NET: [Breaking] Refactor AgentSkill API to async resource and script lookup (#6030)
* .NET: Refactor AgentSkill API to async resource and script lookup

Replace property-based AgentSkill.Content, Resources, and Scripts with
async-by-name lookup methods plus boolean availability flags:

- Content (string getter) -> GetContentAsync(CancellationToken)
- Resources (full list) -> HasResources + GetResourceAsync(name, ct)
- Scripts (full list) -> HasScripts + GetScriptAsync(name, ct)

This makes the API friendlier for sources like MCP where enumerating all
resources up front is expensive or impossible, and allows skill implementations
to fetch content lazily.

Subclass changes:
- AgentFileSkill and AgentInlineSkill implement the new async API while
  preserving content caching.
- AgentClassSkill<TSelf> keeps virtual Resources/Scripts properties for
  reflection-based discovery and seals the new HasResources/HasScripts/
  GetResourceAsync/GetScriptAsync overrides. Its previously non-thread-safe
  lazy initialization is replaced with Lazy<T> (default thread-safety) wired
  up in a new protected constructor, so concurrent first-access from multiple
  threads is safe.
- AgentSkillsProvider calls the new async API and exposes 
ead_skill_resource
  / load_skill / 
un_skill_script tools that await the per-name lookups.

Includes baseline CompatibilitySuppressions.xml entries for the removed
property getters.

Tests:
- Direct coverage for HasResources, HasScripts, GetResourceAsync, and
  GetScriptAsync on all three skill implementations (positive, missing-name,
  and no-resources/no-scripts cases).
- Thread-safety regression test for AgentClassSkill<TSelf> that exercises
  concurrent first-access to Resources, Scripts, and GetContentAsync from
  many tasks and asserts all observers see the same cached instance.
- Provider-level coverage for the 
ead_skill_resource tool (invocation +
  error paths) and for the previously untested error paths of load_skill
  and 
un_skill_script (empty names, skill/resource/script not found).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address PR review comments

- Move GetScriptAsync inside try/catch in RunSkillScriptAsync for error-handling parity
- Remove dead _reflectedResources branch from AgentSkillTestExtensions
- Fix XML docs to reference virtual Resources/Scripts properties (not sealed methods)
- Add Async suffix to async test methods per naming convention
- Make no-await tests synchronous to eliminate CS1998

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix formatting: add UTF-8 BOM and remove unused using

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix XML cref: Resources/Scripts are on AgentClassSkill<TSelf>

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Remove HasResources and HasScripts properties from AgentSkill

Drop the virtual HasResources and HasScripts properties from AgentSkill
and all concrete subclasses (AgentFileSkill, AgentInlineSkill,
AgentClassSkill). AgentSkillsProvider now always includes all three
tools (load_skill, read_skill_resource, run_skill_script) and both
instruction blocks, since the tools already handle missing
resources/scripts gracefully.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add blank line for readability in file-based skills sample

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix HostedAgentSkillsPatternTests for always-included tools

Update assertions to expect read_skill_resource and run_skill_script
tools are always present, matching the new behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-25 17:16:03 +00:00
dc4bafbc1e .NET: Add Hosted-AgentSkills sample with Foundry Skills integration (#6013)
* .NET: Add Hosted-AgentSkills sample for Foundry Skills integration

Add a new hosted agent sample that demonstrates how to load behavioral
guidelines from Foundry Skills at startup using AgentSkillsProvider and
the progressive disclosure pattern (advertise -> load on demand).

The sample:
- Downloads SKILL.md files from Foundry via ProjectAgentSkills SDK
- Extracts ZIP archives with zip-slip protection
- Wires skills into AgentSkillsProvider as an AIContextProvider
- Hosts the agent via the Responses protocol

Ships two Contoso Outdoors skills matching the Python sample (PR #5822):
- support-style: tone, formatting, signature guidelines
- escalation-policy: when and how to escalate tickets

Includes convenience provisioning gated behind PROVISION_SAMPLE_SKILLS
env var, clearly documented as NOT a production pattern.

Closes #5776

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* .NET: Add unit tests and integration test for Hosted-AgentSkills

Unit tests (14 tests, all passing):
- ZIP extraction with zip-slip guard (valid archive, traversal attack,
  sibling-prefix attack, directory entries)
- Skill name validation (rejects dots, separators, traversal patterns)
- AgentSkillsProvider with downloaded skills (advertises both skills,
  load_skill returns canary tokens, unknown skill returns error)

Container integration test:
- New 'agent-skills' scenario in the test container that creates
  Contoso Outdoors skills on disk and wires AgentSkillsProvider
- AgentSkillsHostedAgentFixture + 4 integration tests verifying:
  - Routine questions load support-style skill (STYLE-CANARY-3318)
  - Escalation triggers load escalation-policy (ESC-CANARY-7742)
  - Skills are advertised in system prompt
  - load_skill tool is invoked via FunctionCallContent

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* .NET: Add smoke test, bootstrap, and docs for agent-skills integration

- Add scripts/smoke.ps1 for local Docker smoke testing: builds the
  contributor image, runs the container, verifies both skills are loaded
  via canary tokens (STYLE-CANARY-3318, ESC-CANARY-7742)
- Add 'agent-skills' to the bootstrap script scenario list
- Add agent-skills row to the integration test README scenarios table
- Exclude HostedAgentSkillsPatternTests from net472 (uses net8.0+ APIs)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* .NET: Update commented-out package versions to latest across all hosted samples

Update the end-user PackageReference versions (in the commented-out
sections) from 1.0.0 to the current latest NuGet versions:

- Microsoft.Agents.AI: 1.6.1
- Microsoft.Agents.AI.Foundry: 1.6.1-preview.260514.1
- Microsoft.Agents.AI.Foundry.Hosting: 1.6.1-preview.260514.1
- Microsoft.Agents.AI.Hosting: 1.6.1-preview.260514.1
- Microsoft.Agents.AI.OpenAI: 1.6.1
- Microsoft.Agents.AI.Workflows: 1.6.1

Also adds explicit versions to Hosted-Workflow-Handoff which had bare
PackageReference entries without Version attributes.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* .NET: Fix broken markdown links in Hosted-AgentSkills README

Remove references to non-existent ../../README.md. Replace with
inline instructions matching other hosted samples that don't have
a parent README.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* .NET: Use OS-appropriate string comparison in zip-slip guard

Use Ordinal on Unix (case-sensitive FS) and OrdinalIgnoreCase on
Windows to prevent case-based path bypass on Linux containers.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-25 09:32:04 +00:00
de6d0267f2 .NET: fix parallel tool call rendering in AGUI translation layer (#6009)
Fix three interlocked bugs that prevent parallel tool calls from rendering
correctly in AG-UI protocol clients:

Bug #1: Scope synthetic MessageId fallback to text events only. The shared
streamingMessageId was leaking into ToolCallStartEvent.ParentMessageId,
causing all parallel tool calls to collapse into one FE card.

Bug #2: Make ToolCallResultEvent.MessageId deterministically unique using
result-{CallId} format. MEAI's FunctionInvokingChatClient batches all
results with a shared MessageId, collapsing them in FE reconciliation.

Bug #3: Coalesce consecutive assistant-tool-call messages in AsChatMessages.
Once Bug #1 is fixed, the FE produces separate AGUIAssistantMessage per
tool call. On multi-turn replay these become consecutive assistant messages
without intervening tool results, triggering HTTP 400 from Azure OpenAI.

Remove the now-dead ContainsToolResult helper introduced by PR #5800.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-25 09:31:29 +00:00
westeyandGitHub 0099a6e2fa .NET: HarnessConsole: Improve rendering perf / reduce flickering (#6014)
* HarnessConsole: Improve rendering perf / reduce flickering

* Address PR comments
2026-05-25 09:25:58 +00:00
114 changed files with 7993 additions and 1020 deletions
+6 -2
View File
@@ -1,15 +1,19 @@
{
"name": "Python 3",
"image": "mcr.microsoft.com/devcontainers/python:3.13-bullseye",
"image": "mcr.microsoft.com/devcontainers/python:3.14-bookworm",
"features": {
"ghcr.io/va-h/devcontainers-features/uv:1": {},
"ghcr.io/devcontainers/features/azure-cli:1.2.8": {}
"ghcr.io/devcontainers/features/docker-in-docker:3": {},
"ghcr.io/devcontainers/features/azure-cli:1.2.9": {},
"ghcr.io/devcontainers/features/copilot-cli:1": {}
},
"postCreateCommand": "bash ./devsetup.sh",
"workspaceFolder": "/workspaces/agent-framework/python/",
"customizations": {
"vscode": {
"extensions": [
"GitHub.copilot",
"GitHub.vscode-github-actions",
"ms-python.python",
"ms-windows-ai-studio.windows-ai-studio",
"littlefoxteam.vscode-python-test-adapter"
+1 -1
View File
@@ -8,7 +8,7 @@ ignorePatterns:
- pattern: "./blob"
- pattern: "./issues"
- pattern: "./discussions"
- pattern: "./pulls"
- pattern: "./pull"
- pattern: "https:\/\/platform.openai.com"
- pattern: "http:\/\/localhost"
- pattern: "http:\/\/127.0.0.1"
+3
View File
@@ -359,6 +359,9 @@
<Folder Name="/Samples/04-hosting/FoundryHostedAgents/responses/Hosted-Workflow-Handoff/">
<Project Path="samples/04-hosting/FoundryHostedAgents/responses/Hosted-Workflow-Handoff/HostedWorkflowHandoff.csproj" />
</Folder>
<Folder Name="/Samples/04-hosting/FoundryHostedAgents/responses/Hosted-AgentSkills/">
<Project Path="samples/04-hosting/FoundryHostedAgents/responses/Hosted-AgentSkills/HostedAgentSkills.csproj" />
</Folder>
<Folder Name="/Samples/04-hosting/DurableAgents/" />
<Folder Name="/Samples/04-hosting/DurableAgents/AzureFunctions/">
<File Path="samples/04-hosting/DurableAgents/AzureFunctions/.editorconfig" />
+1
View File
@@ -26,6 +26,7 @@
"src\\Microsoft.Agents.AI.Mem0\\Microsoft.Agents.AI.Mem0.csproj",
"src\\Microsoft.Agents.AI.OpenAI\\Microsoft.Agents.AI.OpenAI.csproj",
"src\\Microsoft.Agents.AI.Purview\\Microsoft.Agents.AI.Purview.csproj",
"src\\Microsoft.Agents.AI.Tools.Shell\\Microsoft.Agents.AI.Tools.Shell.csproj",
"src\\Microsoft.Agents.AI.Workflows.Declarative.Foundry\\Microsoft.Agents.AI.Workflows.Declarative.Foundry.csproj",
"src\\Microsoft.Agents.AI.Workflows.Declarative\\Microsoft.Agents.AI.Workflows.Declarative.csproj",
"src\\Microsoft.Agents.AI.Workflows.Generators\\Microsoft.Agents.AI.Workflows.Generators.csproj",
+3 -3
View File
@@ -1,14 +1,14 @@
<Project>
<PropertyGroup>
<!-- Central version prefix - applies to all nuget packages. -->
<VersionPrefix>1.6.2</VersionPrefix>
<VersionPrefix>1.7.0</VersionPrefix>
<RCNumber>1</RCNumber>
<DateSuffix>260521</DateSuffix>
<DateSuffix>260526</DateSuffix>
<PackageVersion Condition="'$(IsReleaseCandidate)' == 'true'">$(VersionPrefix)-rc$(RCNumber)</PackageVersion>
<PackageVersion Condition="'$(IsReleaseCandidate)' != 'true' AND '$(VersionSuffix)' != ''">$(VersionPrefix)-$(VersionSuffix).$(DateSuffix).1</PackageVersion>
<PackageVersion Condition="'$(IsReleaseCandidate)' != 'true' AND '$(VersionSuffix)' == ''">$(VersionPrefix)-preview.$(DateSuffix).1</PackageVersion>
<PackageVersion Condition="'$(IsReleased)' == 'true'">$(VersionPrefix)</PackageVersion>
<GitTag>1.6.2</GitTag>
<GitTag>1.7.0</GitTag>
<Configurations>Debug;Release;Publish</Configurations>
<IsPackable>true</IsPackable>
@@ -24,6 +24,7 @@ string deploymentName = Environment.GetEnvironmentVariable("AZURE_OPENAI_DEPLOYM
var skillsProvider = new AgentSkillsProvider(
Path.Combine(AppContext.BaseDirectory, "skills"),
SubprocessScriptRunner.RunAsync);
// --- Agent Setup ---
AIAgent agent = new AzureOpenAIClient(new Uri(endpoint), new DefaultAzureCredential())
.GetResponsesClient()
@@ -51,7 +51,7 @@ Console.WriteLine($"Agent: {response.Text}");
/// Properties annotated with <see cref="AgentSkillResourceAttribute"/> are automatically
/// discovered as skill resources, and methods annotated with <see cref="AgentSkillScriptAttribute"/>
/// are automatically discovered as skill scripts. Alternatively,
/// <see cref="AgentSkill.Resources"/> and <see cref="AgentSkill.Scripts"/> can be overridden.
/// <see cref="AgentClassSkill{TSelf}.Resources"/> and <see cref="AgentClassSkill{TSelf}.Scripts"/> can be overridden.
/// </remarks>
internal sealed class UnitConverterSkill : AgentClassSkill<UnitConverterSkill>
{
@@ -40,8 +40,8 @@ public class ListSelection : ConsoleReactiveComponent<ListSelectionProps, Consol
foreach (string line in props.Title.Split('\n'))
{
Console.Write(AnsiEscapes.MoveCursor(props.Y + row, props.X));
Console.Write(AnsiEscapes.EraseEntireLine);
Console.Write(line);
Console.Write(AnsiEscapes.EraseToEndOfLine);
row++;
}
}
@@ -52,7 +52,6 @@ public class ListSelection : ConsoleReactiveComponent<ListSelectionProps, Consol
for (int i = 0; i < totalItems; i++)
{
Console.Write(AnsiEscapes.MoveCursor(props.Y + row, props.X));
Console.Write(AnsiEscapes.EraseEntireLine);
bool isSelected = i == props.SelectedIndex;
bool isCustomTextOption = props.CustomTextPlaceholder != null && i == props.Items.Count;
@@ -72,6 +71,7 @@ public class ListSelection : ConsoleReactiveComponent<ListSelectionProps, Consol
}
Console.Write(props.Items[i]);
Console.Write(AnsiEscapes.EraseToEndOfLine);
if (isSelected)
{
@@ -101,6 +101,7 @@ public class ListSelection : ConsoleReactiveComponent<ListSelectionProps, Consol
}
Console.Write(props.CustomText);
Console.Write(AnsiEscapes.EraseToEndOfLine);
if (isSelected)
{
@@ -121,6 +122,7 @@ public class ListSelection : ConsoleReactiveComponent<ListSelectionProps, Consol
Console.Write(" ");
Console.Write(props.CustomTextPlaceholder);
Console.Write(AnsiEscapes.EraseToEndOfLine);
Console.Write(AnsiEscapes.ResetAttributes);
}
}
@@ -17,16 +17,19 @@ public record TextScrollPanelProps : ConsoleReactiveProps
/// <summary>
/// State for <see cref="TextScrollPanel"/>.
/// </summary>
/// <param name="RenderedCount">The number of items already rendered.</param>
public record TextScrollPanelState(int RenderedCount = 0) : ConsoleReactiveState;
public record TextScrollPanelState : ConsoleReactiveState;
/// <summary>
/// A component that renders pre-rendered string items within a scroll area.
/// All items are considered finalized — only new items since the last render are output.
/// Use <see cref="Reset"/> to force a full re-render.
/// The last rendered item is considered dynamic and will be re-rendered on each call.
/// All prior items are considered finalized and are not re-rendered.
/// Use <see cref="Invalidate"/> to force a full re-render.
/// </summary>
public class TextScrollPanel : ConsoleReactiveComponent<TextScrollPanelProps, TextScrollPanelState>
{
private int _renderedCount;
private int _lastItemOffsetFromBottom;
/// <summary>
/// Initializes a new instance of the <see cref="TextScrollPanel"/> class.
/// </summary>
@@ -35,12 +38,12 @@ public class TextScrollPanel : ConsoleReactiveComponent<TextScrollPanelProps, Te
this.State = new TextScrollPanelState();
}
/// <summary>
/// Resets the panel so all items will be re-rendered on the next Render call.
/// </summary>
public void Reset()
/// <inheritdoc />
public override void Invalidate()
{
this.State = new TextScrollPanelState();
this._renderedCount = 0;
this._lastItemOffsetFromBottom = 0;
base.Invalidate();
}
/// <inheritdoc />
@@ -51,16 +54,59 @@ public class TextScrollPanel : ConsoleReactiveComponent<TextScrollPanelProps, Te
return;
}
// Move cursor to the bottom of the scroll area
Console.Write(AnsiEscapes.MoveCursor(props.Y + props.Height - 1, props.X));
int bottomRow = props.Y + props.Height - 1;
// Output only new items since last rendered
for (int i = state.RenderedCount; i < props.Items.Count; i++)
// Determine the first item to render. If we previously rendered items,
// re-render the last one (it may have changed/grown) from its stored position.
int startIndex = this._renderedCount > 0 ? this._renderedCount - 1 : 0;
if (this._renderedCount > 0 && this._lastItemOffsetFromBottom > 0)
{
// Reposition cursor to where the last rendered item began
Console.Write(AnsiEscapes.MoveCursor(bottomRow - this._lastItemOffsetFromBottom, props.X));
}
else
{
// First render — position at the bottom of the scroll area
Console.Write(AnsiEscapes.MoveCursor(bottomRow, props.X));
}
// Render from startIndex onwards
for (int i = startIndex; i < props.Items.Count; i++)
{
Console.Write(props.Items[i]);
}
// Update state to track what we've rendered
this.State = new TextScrollPanelState(props.Items.Count);
// Calculate the offset from bottom for the start of the new last item
int lastItemLines = CountLines(props.Items[^1]);
this._lastItemOffsetFromBottom = lastItemLines > 0 ? lastItemLines - 1 : 0;
// Update rendered count
this._renderedCount = props.Items.Count;
}
private static int CountLines(string text)
{
if (string.IsNullOrEmpty(text))
{
return 0;
}
int count = 1;
for (int i = 0; i < text.Length; i++)
{
if (text[i] == '\n')
{
count++;
}
}
// If text ends with a newline, don't count the trailing empty line
if (text[text.Length - 1] == '\n')
{
count--;
}
return count;
}
}
@@ -0,0 +1,36 @@
// Copyright (c) Microsoft. All rights reserved.
namespace Harness.ConsoleReactiveFramework;
/// <summary>
/// Caches the result of a mapping function and only recomputes when the input changes.
/// </summary>
/// <typeparam name="TInput">The type of the input value.</typeparam>
/// <typeparam name="TOutput">The type of the mapped output value.</typeparam>
public class ConsoleReactiveMemo<TInput, TOutput>
{
private TInput? _previousInput;
private TOutput? _cachedOutput;
private bool _hasValue;
/// <summary>
/// Returns the cached output if <paramref name="input"/> equals the previously stored input;
/// otherwise invokes <paramref name="mapper"/> to compute and cache a new output.
/// </summary>
/// <param name="input">The current input value.</param>
/// <param name="mapper">A function that maps the input to an output value.</param>
/// <returns>The cached or newly computed output.</returns>
public TOutput Map(TInput input, Func<TInput, TOutput> mapper)
{
ArgumentNullException.ThrowIfNull(mapper);
if (!this._hasValue || !EqualityComparer<TInput>.Default.Equals(input, this._previousInput))
{
this._previousInput = input;
this._cachedOutput = mapper(input);
this._hasValue = true;
}
return this._cachedOutput!;
}
}
@@ -19,7 +19,6 @@ public class HarnessAppComponent : ConsoleReactiveComponent<ConsoleReactiveProps
private readonly ListSelection _listSelection = new();
private readonly TextInput _textInput = new();
private readonly TextScrollPanel _textScrollPanel = new();
private readonly TextPanel _textPanel = new();
private readonly TextPanel _queuedPanel = new();
private readonly AgentStatus _agentStatus = new();
private readonly AgentModeAndHelp _modeAndHelp = new();
@@ -341,16 +340,6 @@ public class HarnessAppComponent : ConsoleReactiveComponent<ConsoleReactiveProps
return;
}
// Determine the text panel height for the last scroll item
IReadOnlyList<string> lastItems = state.ScrollAreaContentItems.Count > 0
? [state.ScrollAreaContentItems[^1]]
: [];
int textPanelHeight = TextPanel.CalculateHeight(lastItems);
if (textPanelHeight > 0)
{
textPanelHeight++; // Extra line for spacing between text panel and rule
}
// Calculate queued items panel height
int queuedPanelHeight = TextPanel.CalculateHeight(state.QueuedItems);
@@ -444,7 +433,7 @@ public class HarnessAppComponent : ConsoleReactiveComponent<ConsoleReactiveProps
int modeAndHelpHeight = showStatusAndHelp ? AgentModeAndHelp.CalculateHeight(modeAndHelpProps) : 0;
int ruleHeight = TopBottomRule.CalculateHeight(ruleProps);
int nonScrollHeight = ruleHeight + textPanelHeight + agentStatusHeight + queuedPanelHeight + modeAndHelpHeight + 1; // +1 for bottom padding
int nonScrollHeight = ruleHeight + agentStatusHeight + queuedPanelHeight + modeAndHelpHeight + 1; // +1 for bottom padding
int scrollBottom = Math.Max(1, state.ConsoleHeight - nonScrollHeight);
// If scroll region changed or a clear is needed, reset everything
@@ -455,52 +444,36 @@ public class HarnessAppComponent : ConsoleReactiveComponent<ConsoleReactiveProps
System.Console.Write(AnsiEscapes.ResetScrollRegion);
System.Console.Write(AnsiEscapes.EraseEntireScreen);
System.Console.Write(AnsiEscapes.EraseScrollbackBuffer);
this._textScrollPanel.Reset();
this._resizedSinceLastRender = false;
// Invalidate all children so they re-render even if props haven't changed
this._rule.Invalidate();
this._textScrollPanel.Invalidate();
this._textPanel.Invalidate();
this._queuedPanel.Invalidate();
this._agentStatus.Invalidate();
this._modeAndHelp.Invalidate();
this._textInput.Invalidate();
this._listSelection.Invalidate();
this._resizedSinceLastRender = false;
}
this._scrollRegionBottom = scrollBottom;
System.Console.Write(AnsiEscapes.SetScrollRegion(scrollBottom));
// Render text scroll panel in the scroll area (all items except the last)
IReadOnlyList<string> scrollItems = state.ScrollAreaContentItems.Count > 1
? state.ScrollAreaContentItems.Take(state.ScrollAreaContentItems.Count - 1).ToList()
: [];
// Render text scroll panel in the scroll area
this._textScrollPanel.Props = new TextScrollPanelProps
{
X = 1,
Y = 1,
Width = state.ConsoleWidth,
Height = scrollBottom,
Items = scrollItems,
Items = state.ScrollAreaContentItems,
};
this._textScrollPanel.Render();
// Render the text panel for the last (dynamic) item just below the scroll region
this._textPanel.Props = new TextPanelProps
{
X = 1,
Y = scrollBottom + 1,
Width = state.ConsoleWidth,
Height = textPanelHeight,
Items = lastItems,
};
this._textPanel.Render();
// Render queued input items between text panel and agent status
int queuedPanelY = scrollBottom + textPanelHeight + 1;
// Render queued input items between scroll area and agent status
int queuedPanelY = scrollBottom + 1;
this._queuedPanel.Props = new TextPanelProps
{
X = 1,
@@ -5,17 +5,17 @@ using Microsoft.Extensions.AI;
namespace Harness.Shared.Console.ToolFormatters;
/// <summary>
/// Formats <c>AgentMode_*</c> tool calls, showing the target mode for Set operations.
/// Formats <c>mode_*</c> tool calls, showing the target mode for Set operations.
/// </summary>
public sealed class ModeToolFormatter : ToolCallFormatter
{
/// <inheritdoc/>
public override bool CanFormat(FunctionCallContent call) => call.Name.StartsWith("AgentMode_", StringComparison.Ordinal);
public override bool CanFormat(FunctionCallContent call) => call.Name.StartsWith("mode_", StringComparison.Ordinal);
/// <inheritdoc/>
public override string? FormatDetail(FunctionCallContent call) => call.Name switch
{
"AgentMode_Set" => FormatStringArg(call, "mode"),
"mode_set" => FormatStringArg(call, "mode"),
_ => null,
};
@@ -0,0 +1,14 @@
AZURE_AI_PROJECT_ENDPOINT=<your-azure-ai-project-endpoint>
ASPNETCORE_URLS=http://+:8088
ASPNETCORE_ENVIRONMENT=Development
AZURE_AI_MODEL_DEPLOYMENT_NAME=gpt-4o
AGENT_NAME=hosted-agent-skills
SKILL_NAMES=support-style,escalation-policy
# Set to true to provision sample skills to Foundry on startup (first-run convenience).
# In production, skills are provisioned externally — leave this unset or false.
PROVISION_SAMPLE_SKILLS=true
AZURE_BEARER_TOKEN=DefaultAzureCredential
# When running outside the Foundry platform the platform-injected isolation keys are absent.
# These two variables provide fallback values for local Docker debugging only.
HOSTED_USER_ISOLATION_KEY=local-dev-user
HOSTED_CHAT_ISOLATION_KEY=local-dev-chat
@@ -0,0 +1,26 @@
# Dockerfile for end-users consuming the Agent Framework via NuGet packages.
#
# This Dockerfile performs a full `dotnet restore` and `dotnet publish` inside the container,
# which only succeeds when the project references its dependencies via PackageReference (see the
# commented-out section in HostedAgentSkills.csproj). Contributors building from the
# agent-framework repository source must use Dockerfile.contributor instead because
# ProjectReference dependencies live outside this folder and cannot be restored from inside
# this build context.
#
# Use the official .NET 10.0 ASP.NET runtime as a parent image
FROM mcr.microsoft.com/dotnet/aspnet:10.0 AS base
WORKDIR /app
FROM mcr.microsoft.com/dotnet/sdk:10.0 AS build
WORKDIR /src
COPY . .
RUN dotnet restore
RUN dotnet publish -c Release -o /app/publish
# Final stage
FROM base AS final
WORKDIR /app
COPY --from=build /app/publish .
EXPOSE 8088
ENV ASPNETCORE_URLS=http://+:8088
ENTRYPOINT ["dotnet", "HostedAgentSkills.dll"]
@@ -0,0 +1,23 @@
# Dockerfile for contributors building from the agent-framework repository source.
#
# This project uses ProjectReference to the local Microsoft.Agents.AI source,
# which means a standard multi-stage Docker build cannot resolve dependencies outside
# this folder. Instead, pre-publish the app targeting the container runtime and copy
# the output into the container:
#
# dotnet publish -c Debug -f net10.0 -r linux-musl-x64 --self-contained false -o out
# docker build -f Dockerfile.contributor -t hosted-agent-skills .
# docker run --rm -p 8088:8088 \
# -e AGENT_NAME=hosted-agent-skills \
# -e HOSTED_USER_ISOLATION_KEY=alice \
# -e HOSTED_CHAT_ISOLATION_KEY=alice-chat-1 \
# --env-file .env hosted-agent-skills
#
# For end-users consuming the NuGet package (not ProjectReference), use the standard
# Dockerfile which performs a full dotnet restore + publish inside the container.
FROM mcr.microsoft.com/dotnet/aspnet:10.0-alpine AS final
WORKDIR /app
COPY out/ .
EXPOSE 8088
ENV ASPNETCORE_URLS=http://+:8088
ENTRYPOINT ["dotnet", "HostedAgentSkills.dll"]
@@ -0,0 +1,40 @@
<Project Sdk="Microsoft.NET.Sdk.Web">
<PropertyGroup>
<TargetFrameworks>net10.0</TargetFrameworks>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<CentralPackageTransitivePinningEnabled>false</CentralPackageTransitivePinningEnabled>
<RootNamespace>HostedAgentSkills</RootNamespace>
<AssemblyName>HostedAgentSkills</AssemblyName>
<NoWarn>$(NoWarn);MEAI001;OPENAI001;AAIP001</NoWarn>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Azure.AI.Projects" />
<PackageReference Include="Azure.Identity" />
<PackageReference Include="DotNetEnv" />
</ItemGroup>
<!-- For contributors: uses ProjectReference to build against local source -->
<ItemGroup>
<ProjectReference Include="..\..\..\..\..\src\Microsoft.Agents.AI\Microsoft.Agents.AI.csproj" />
<ProjectReference Include="..\..\..\..\..\src\Microsoft.Agents.AI.Foundry\Microsoft.Agents.AI.Foundry.csproj" />
<ProjectReference Include="..\..\..\..\..\src\Microsoft.Agents.AI.Foundry.Hosting\Microsoft.Agents.AI.Foundry.Hosting.csproj" />
<ProjectReference Include="..\Hosted_Shared_Contributor_Setup\Hosted_Shared_Contributor_Setup.csproj" />
</ItemGroup>
<!-- For end-users: uncomment the PackageReference below and remove the ProjectReferences above
<ItemGroup>
<PackageReference Include="Microsoft.Agents.AI" Version="1.6.1" />
<PackageReference Include="Microsoft.Agents.AI.Foundry" Version="1.6.1-preview.260514.1" />
<PackageReference Include="Microsoft.Agents.AI.Foundry.Hosting" Version="1.6.1-preview.260514.1" />
</ItemGroup>
-->
<!-- Include the skills/ directory in the publish output so the sample can provision them -->
<ItemGroup>
<None Include="skills\**\*" CopyToOutputDirectory="PreserveNewest" />
</ItemGroup>
</Project>
@@ -0,0 +1,215 @@
// Copyright (c) Microsoft. All rights reserved.
// Hosted-AgentSkills
//
// Demonstrates how to host an agent that loads its behavioral guidelines from Foundry Skills at
// startup. Skills are authored as SKILL.md files, uploaded to Foundry via the Skills REST API,
// and downloaded by the agent on boot so guideline updates ship without code changes.
//
// The agent uses AgentSkillsProvider from the Agent Framework which implements the progressive
// disclosure pattern from the Agent Skills specification (https://agentskills.io/):
// 1. Advertise — skill names and descriptions are injected into the system prompt.
// 2. Load — the model calls load_skill to retrieve the full SKILL.md body on demand.
//
// IMPORTANT: In production, skill provisioning (uploading SKILL.md files to Foundry) is an
// external concern — it is NOT the hosted agent's responsibility. The provisioning helper below
// is included for sample convenience only, so the sample is self-contained and runnable without
// a separate setup step. A real deployment pipeline would provision skills separately (e.g., via
// a CI/CD step, a CLI script, or a management portal).
#pragma warning disable AAIP001 // ProjectAgentSkills is experimental
using System.ClientModel;
using System.IO.Compression;
using Azure.AI.Projects;
using Azure.AI.Projects.Agents;
using Azure.Core;
using Azure.Identity;
using DotNetEnv;
using Hosted_Shared_Contributor_Setup;
using Microsoft.Agents.AI;
using Microsoft.Agents.AI.Foundry.Hosting;
using Microsoft.Extensions.AI;
// Load .env file if present (for local development)
Env.TraversePath().Load();
string endpoint = Environment.GetEnvironmentVariable("AZURE_AI_PROJECT_ENDPOINT")
?? throw new InvalidOperationException("AZURE_AI_PROJECT_ENDPOINT is not set.");
string deploymentName = Environment.GetEnvironmentVariable("AZURE_AI_MODEL_DEPLOYMENT_NAME") ?? "gpt-4o";
string skillNames = Environment.GetEnvironmentVariable("SKILL_NAMES")
?? throw new InvalidOperationException("SKILL_NAMES is not set. Provide a comma-separated list of skill names (e.g., support-style,escalation-policy).");
string[] requestedSkills = skillNames.Split(',', StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries);
if (requestedSkills.Length == 0)
{
throw new InvalidOperationException("SKILL_NAMES must list at least one skill name.");
}
// Validate skill names to prevent path traversal.
foreach (string name in requestedSkills)
{
if (name.Contains('.') || name.Contains('/') || name.Contains('\\') || Path.IsPathRooted(name))
{
throw new InvalidOperationException(
$"Invalid skill name '{name}': skill names must not contain path separators or dots.");
}
}
// Use a chained credential: try a temporary dev token first (for local Docker debugging),
// then fall back to DefaultAzureCredential (for local dev via dotnet run / managed identity in production).
TokenCredential credential = new ChainedTokenCredential(
new DevTemporaryTokenCredential(),
new DefaultAzureCredential());
AIProjectClient projectClient = new(new Uri(endpoint), credential);
ProjectAgentSkills skillsClient = projectClient.AgentAdministrationClient.GetAgentSkills();
// ── Provision skills (sample convenience only — NOT a production pattern) ─────
// In production, skills are provisioned externally (e.g., via CI/CD or a management script).
// This helper ensures the sample's SKILL.md files exist in Foundry so the sample is runnable
// out of the box without a separate setup step. Set PROVISION_SAMPLE_SKILLS=true to enable.
string sourceSkillsDir = Path.Combine(AppContext.BaseDirectory, "skills");
bool provisionEnabled = string.Equals(
Environment.GetEnvironmentVariable("PROVISION_SAMPLE_SKILLS"), "true", StringComparison.OrdinalIgnoreCase);
if (provisionEnabled && Directory.Exists(sourceSkillsDir))
{
await EnsureSkillsProvisionedAsync(skillsClient, sourceSkillsDir, requestedSkills);
}
// ── Download skills from Foundry ─────────────────────────────────────────────
// Pull the latest copy of each skill from Foundry into a runtime-only folder.
// This directory is recreated on every startup so the agent always picks up
// the latest version of each skill.
string downloadedSkillsDir = Path.Combine(AppContext.BaseDirectory, "downloaded_skills");
await DownloadSkillsAsync(skillsClient, requestedSkills, downloadedSkillsDir);
// ── Wire skills into the agent ───────────────────────────────────────────────
// AgentSkillsProvider implements progressive disclosure: skill names and descriptions
// are advertised in the system prompt (~100 tokens per skill), and the full SKILL.md
// body is loaded on demand when the model calls the load_skill tool.
AgentSkillsProvider skillsProvider = new(downloadedSkillsDir);
ChatClientAgent agent = projectClient.AsAIAgent(new ChatClientAgentOptions
{
Name = Environment.GetEnvironmentVariable("AGENT_NAME") ?? "hosted-agent-skills",
ChatOptions = new ChatOptions
{
ModelId = deploymentName,
Instructions = "You are a customer-support assistant for Contoso Outdoors.",
},
AIContextProviders = [skillsProvider]
});
// Host the agent as a Foundry Hosted Agent using the Responses API.
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddFoundryResponses(agent);
builder.Services.AddDevTemporaryLocalContributorSetup(); // Local Docker debugging only - must not be used in production.
var app = builder.Build();
app.MapFoundryResponses();
// Contributor-only: in Development, also map the per-agent OpenAI route shape that live Foundry uses
// so a local REPL client can target this server via AIProjectClient.AsAIAgent(Uri agentEndpoint).
// Do not use this in production. Hosted Foundry agents only support the agent-endpoint path.
app.MapDevTemporaryLocalAgentEndpoint();
app.Run();
// ── Helpers ──────────────────────────────────────────────────────────────────
// Downloads each named skill from Foundry and extracts the ZIP archive into a
// separate subdirectory under the target directory.
static async Task DownloadSkillsAsync(ProjectAgentSkills skillsClient, string[] skillNames, string targetDir)
{
if (Directory.Exists(targetDir))
{
Directory.Delete(targetDir, recursive: true);
}
Directory.CreateDirectory(targetDir);
foreach (string name in skillNames)
{
Console.WriteLine($"Downloading skill '{name}' from Foundry...");
BinaryData zipData = await skillsClient.DownloadSkillAsync(name);
string skillDir = Path.Combine(targetDir, name);
Directory.CreateDirectory(skillDir);
using var zipStream = zipData.ToStream();
using var archive = new ZipArchive(zipStream, ZipArchiveMode.Read);
SafeExtractZip(archive, skillDir);
if (!File.Exists(Path.Combine(skillDir, "SKILL.md")))
{
throw new InvalidOperationException(
$"Downloaded archive for '{name}' did not contain a SKILL.md at the root.");
}
}
}
// Extracts a ZIP archive into a destination directory, rejecting entries that would
// escape the target path (zip-slip guard).
static void SafeExtractZip(ZipArchive archive, string destinationDir)
{
string destRoot = Path.GetFullPath(destinationDir);
string destRootWithSep = Path.EndsInDirectorySeparator(destRoot)
? destRoot
: destRoot + Path.DirectorySeparatorChar;
// Use ordinal comparison on Unix (case-sensitive FS) and ordinal-ignore-case on Windows.
var comparison = OperatingSystem.IsWindows()
? StringComparison.OrdinalIgnoreCase
: StringComparison.Ordinal;
foreach (ZipArchiveEntry entry in archive.Entries)
{
string entryPath = Path.GetFullPath(Path.Combine(destRoot, entry.FullName));
if (!entryPath.StartsWith(destRootWithSep, comparison)
&& !string.Equals(entryPath, destRoot, comparison))
{
throw new InvalidOperationException(
$"Refusing to extract unsafe path '{entry.FullName}' outside of '{destRoot}'.");
}
if (string.IsNullOrEmpty(entry.Name))
{
// Directory entry — ensure it exists.
Directory.CreateDirectory(entryPath);
}
else
{
Directory.CreateDirectory(Path.GetDirectoryName(entryPath)!);
entry.ExtractToFile(entryPath, overwrite: true);
}
}
}
// Ensures each requested skill is provisioned in Foundry. For each skill name, checks whether
// the skill exists and uploads it from the local source directory if it does not.
//
// This is a sample convenience helper — in production, skill provisioning is an external concern.
static async Task EnsureSkillsProvisionedAsync(ProjectAgentSkills skillsClient, string sourceDir, string[] skillNames)
{
foreach (string name in skillNames)
{
string skillPath = Path.Combine(sourceDir, name);
if (!Directory.Exists(skillPath) || !File.Exists(Path.Combine(skillPath, "SKILL.md")))
{
continue; // No local source for this skill — skip provisioning.
}
try
{
await skillsClient.GetSkillAsync(name);
Console.WriteLine($"Skill '{name}' already exists in Foundry.");
}
catch (ClientResultException ex) when (ex.Status == 404)
{
Console.WriteLine($"Provisioning skill '{name}' from {skillPath}...");
AgentsSkill imported = await skillsClient.CreateSkillFromPackageAsync(skillPath);
Console.WriteLine($" Imported skill '{imported.Name}' (id={imported.SkillId}, has_blob={imported.HasBlob}).");
}
}
}
@@ -0,0 +1,109 @@
# What this sample demonstrates
An [Agent Framework](https://github.com/microsoft/agent-framework) agent that loads its behavioral guidelines from [**Foundry Skills**](https://learn.microsoft.com/en-us/azure/foundry/agents/how-to/tools/skills) at startup, hosted using the **Responses protocol**. Skills are authored once as `SKILL.md` files, uploaded to your Foundry project through the Skills REST API, and downloaded by the agent on boot so updates ship without code changes.
## How It Works
### Authoring skills
Each skill is a Markdown file with a YAML front matter block. This sample ships two source skills under [`skills/`](skills/):
| Skill | Purpose |
|---|---|
| [`support-style`](skills/support-style/SKILL.md) | Voice, formatting, and signature rules for Contoso Outdoors support replies. |
| [`escalation-policy`](skills/escalation-policy/SKILL.md) | When and how to escalate a customer ticket. |
Each `SKILL.md` includes a unique `*-CANARY-*` token that the model is asked to echo, so you can prove the skill was loaded from Foundry (not hallucinated) by checking the response.
> The `name` and `description` values in the YAML front matter must be **unquoted** — quoting them causes the Skills REST API to return HTTP 500 on import.
### Uploading skills
The sample includes a convenience provisioning step that checks whether each skill exists in Foundry and uploads it if not, gated behind the `PROVISION_SAMPLE_SKILLS=true` env var. **In production, skill provisioning is an external concern** — it is NOT the hosted agent's responsibility. A real deployment pipeline would provision skills separately (e.g., via a CI/CD step, a CLI script, or a management portal).
The provisioning uses `ProjectAgentSkills.CreateSkillFromPackageAsync(directoryPath)` from the `Azure.AI.Projects.Agents` SDK. The method packages the `SKILL.md` file as a ZIP and uploads it to Foundry.
### Downloading skills at agent startup
[`Program.cs`](Program.cs) reads the comma-separated `SKILL_NAMES` env var and for each skill name downloads the ZIP archive from Foundry via `ProjectAgentSkills.DownloadSkillAsync(name)`, then unpacks it into a **separate runtime directory** at `downloaded_skills/<name>/` (kept distinct from the static `skills/` source folder).
An [`AgentSkillsProvider`](../../../../../src/Microsoft.Agents.AI/Skills/AgentSkillsProvider.cs) is then built over `downloaded_skills/` and attached to the agent as a context provider. The provider follows the [Agent Skills](https://agentskills.io/) progressive-disclosure pattern:
1. **Advertise** — skill names and descriptions are injected into the system prompt at session start (~100 tokens per skill).
2. **Load** — the model calls the `load_skill` tool when it decides a skill is relevant to the user's turn, and the full `SKILL.md` body is returned.
This means the model only pays the token cost for a skill's full body when it actually needs it, and updating a skill in Foundry + restarting the agent is enough to pick up the change — no code redeploy required.
> **Note:** This sample supports instruction-only and resource-based skills. If your downloaded skills contain scripts, add a script runner when constructing the `AgentSkillsProvider`.
### Agent Hosting
The agent is hosted using the [Agent Framework](https://github.com/microsoft/agent-framework) with the Responses API hosting layer (`AddFoundryResponses` / `MapFoundryResponses`).
## Prerequisites
- An Azure AI Foundry project with a deployed model (e.g., `gpt-4o`)
- Azure CLI logged in (`az login`)
### Required RBAC
Your identity (or the Managed Identity running the container in production) needs **Azure AI User** on the Foundry project scope. This single role covers both authoring skills and downloading them.
## Running the Agent Host
Set the required environment variables and run the sample with `dotnet run`:
```bash
export AZURE_AI_PROJECT_ENDPOINT="https://<account>.services.ai.azure.com/api/projects/<project>"
export AZURE_AI_MODEL_DEPLOYMENT_NAME="gpt-4o"
export SKILL_NAMES="support-style,escalation-policy"
export PROVISION_SAMPLE_SKILLS="true" # First run only — provisions skills to Foundry
```
Or in PowerShell:
```powershell
$env:SKILL_NAMES="support-style,escalation-policy"
$env:PROVISION_SAMPLE_SKILLS="true" # First run only — provisions skills to Foundry
```
You can also place these in a `.env` file next to `Program.cs` — see [`.env.example`](.env.example).
On startup you should see:
```text
Skill 'support-style' already exists in Foundry.
Skill 'escalation-policy' already exists in Foundry.
Downloading skill 'support-style' from Foundry...
Downloading skill 'escalation-policy' from Foundry...
```
The downloaded `SKILL.md` files land under `downloaded_skills/<name>/SKILL.md` next to the published output. This directory is recreated from scratch on every run, so deleting it manually is never necessary.
## Interacting with the agent
> Send a POST request to the server with a JSON body containing an `"input"` field to interact with the agent. For example:
```bash
curl -X POST http://localhost:8088/responses -H "Content-Type: application/json" -d '{"input": "Hi, I am Alex. I just want to confirm I can return my tent within 30 days."}'
curl -X POST http://localhost:8088/responses -H "Content-Type: application/json" -d '{"input": "I want a $750 refund on Order #A-1042 right now or I am calling my lawyer."}'
```
| Prompt mentions | Skill that should drive the response |
|---|---|
| Routine return / shipping / care question | Model loads `support-style` (canary `STYLE-CANARY-3318`) — no escalation. |
| Injury, legal threat, press, or refund > $500 | Model loads `escalation-policy` (canary `ESC-CANARY-7742`) **and** `support-style`. |
Because skills are loaded on demand, the canary token in a response also proves the model actually invoked `load_skill` for the matching skill (not just saw its name in the advertised list).
## Deploying the Agent to Foundry
When deploying to Foundry, make sure `SKILL_NAMES` is set in your `azd` environment so it gets injected into the hosted container per [`agent.manifest.yaml`](agent.manifest.yaml):
```bash
azd env set SKILL_NAMES "support-style,escalation-policy"
```
The deployed agent's Managed Identity needs **Azure AI User** on the Foundry project to download skills at startup.
> The `skills/` source folder is **not** deployed to Foundry — only the downloaded skills are used at runtime. The provisioning step must have been run against the same Foundry project before the agent can download the skills.
@@ -0,0 +1,41 @@
# yaml-language-server: $schema=https://raw.githubusercontent.com/microsoft/AgentSchema/refs/heads/main/schemas/v1.0/AgentManifest.yaml
name: hosted-agent-skills
displayName: "Hosted Agent Skills"
description: >
An Agent Framework agent that downloads its behavioral guidelines from the Foundry
Skills REST API at startup, demonstrating how to decouple behavioral guidelines
(tone, escalation policy, etc.) from agent code using AgentSkillsProvider.
metadata:
tags:
- AI Agent Hosting
- Azure AI AgentServer
- Responses Protocol
- Agent Framework
- Agent Skills
- Foundry Skills
template:
name: hosted-agent-skills
kind: hosted
protocols:
- protocol: responses
version: 1.0.0
resources:
cpu: "0.25"
memory: 0.5Gi
environment_variables:
- name: AZURE_AI_MODEL_DEPLOYMENT_NAME
value: "{{AZURE_AI_MODEL_DEPLOYMENT_NAME}}"
- name: SKILL_NAMES
value: "{{SKILL_NAMES}}"
parameters:
properties:
- name: SKILL_NAMES
secret: false
description: Comma-separated list of Foundry skill names to download at startup (e.g., support-style,escalation-policy)
resources:
- kind: model
id: gpt-4.1-mini
name: AZURE_AI_MODEL_DEPLOYMENT_NAME
@@ -0,0 +1,14 @@
# yaml-language-server: $schema=https://raw.githubusercontent.com/microsoft/AgentSchema/refs/heads/main/schemas/v1.0/ContainerAgent.yaml
kind: hosted
name: hosted-agent-skills
protocols:
- protocol: responses
version: 1.0.0
resources:
cpu: "0.25"
memory: 0.5Gi
environment_variables:
- name: AZURE_AI_MODEL_DEPLOYMENT_NAME
value: ${AZURE_AI_MODEL_DEPLOYMENT_NAME}
- name: SKILL_NAMES
value: ${SKILL_NAMES}
@@ -0,0 +1,100 @@
#requires -Version 7
<#
.SYNOPSIS
Local smoke test for the Hosted-AgentSkills sample.
.DESCRIPTION
Publishes the sample, builds the contributor Docker image, runs the container, drives
two conversations via curl invocations, and asserts that the agent loaded the correct
Foundry Skill for each prompt (verified via canary tokens in the response).
Exits non-zero on failure.
Prerequisites:
- Docker
- az login (token is fetched from the host)
- .env populated with AZURE_AI_PROJECT_ENDPOINT and model deployment
- Skills provisioned to Foundry (set PROVISION_SAMPLE_SKILLS=true on first run)
.NOTES
This script is for local Docker debugging only. The Foundry platform supplies the
isolation keys for every inbound request in production and the dev fallback used here
must not be enabled in production deployments.
#>
[CmdletBinding()]
param(
[int]$Port = 8088,
[string]$ImageName = 'hosted-agent-skills-smoke',
[string]$ContainerName = 'hosted-agent-skills-smoke'
)
$ErrorActionPreference = 'Stop'
Set-Location -Path $PSScriptRoot/..
if (-not (Test-Path .env)) {
throw '.env not found. Copy .env.example to .env and fill in AZURE_AI_PROJECT_ENDPOINT.'
}
Write-Host '==> Publishing sample for linux-musl-x64 ...'
dotnet publish -c Debug -f net10.0 -r linux-musl-x64 --self-contained false -o out --tl:off | Out-Host
if ($LASTEXITCODE -ne 0) { throw 'dotnet publish failed.' }
Write-Host '==> Building docker image ...'
docker build -f Dockerfile.contributor -t $ImageName . | Out-Host
if ($LASTEXITCODE -ne 0) { throw 'docker build failed.' }
Write-Host '==> Fetching bearer token ...'
$bearer = az account get-access-token --resource https://ai.azure.com --query accessToken -o tsv
if (-not $bearer) { throw 'Failed to obtain bearer token. Run az login.' }
function Start-Container {
docker rm -f $ContainerName 2>$null | Out-Null
docker run -d --name $ContainerName -p ${Port}:8088 `
-e AGENT_NAME=hosted-agent-skills `
-e AZURE_BEARER_TOKEN=$bearer `
-e HOSTED_USER_ISOLATION_KEY=smoke-user `
-e HOSTED_CHAT_ISOLATION_KEY=smoke-chat-1 `
--env-file .env `
$ImageName | Out-Host
if ($LASTEXITCODE -ne 0) { throw "docker run failed." }
# Wait for the server to start and download skills from Foundry.
Write-Host ' Waiting for startup (skill download + server ready) ...'
Start-Sleep -Seconds 15
}
function Invoke-Agent([string]$Prompt, [string]$PreviousResponseId = $null) {
$body = @{ input = $Prompt; model = 'hosted-agent-skills' }
if ($PreviousResponseId) { $body['previous_response_id'] = $PreviousResponseId }
$json = $body | ConvertTo-Json -Compress
$resp = Invoke-RestMethod -Method Post -Uri "http://localhost:$Port/responses" -ContentType 'application/json' -Body $json
return $resp
}
function Get-ResponseText($response) {
return ($response.output | ForEach-Object { $_.content | ForEach-Object { $_.text } }) -join ' '
}
function Assert-Contains([string]$Haystack, [string]$Needle, [string]$Label) {
if ($Haystack -notmatch [regex]::Escape($Needle)) {
throw "FAILED [$Label]: expected response to contain '$Needle' but got: $Haystack"
}
Write-Host "PASS [$Label]: response contains '$Needle'."
}
try {
Start-Container
Write-Host '==> Test 1: Routine support question -> support-style skill ...'
$r1 = Invoke-Agent -Prompt 'Hi, I am Alex. I just want to confirm I can return my tent within 30 days.'
$text1 = Get-ResponseText $r1
Assert-Contains $text1 'STYLE-CANARY-3318' 'routine question: support-style canary'
Write-Host '==> Test 2: Escalation trigger -> escalation-policy skill ...'
$r2 = Invoke-Agent -Prompt 'I want a $750 refund on Order #A-1042 right now or I am calling my lawyer.'
$text2 = Get-ResponseText $r2
Assert-Contains $text2 'ESC-CANARY-7742' 'escalation trigger: escalation-policy canary'
Write-Host ''
Write-Host '==> All smoke assertions passed.'
}
finally {
docker rm -f $ContainerName 2>$null | Out-Null
}
@@ -0,0 +1,30 @@
---
name: escalation-policy
description: When and how to escalate Contoso Outdoors customer-support tickets.
---
# Contoso Outdoors Escalation Policy
You must follow this escalation policy on every conversation.
## Escalate immediately when the customer
- Reports an injury, allergic reaction, or other safety incident.
- Mentions legal action, regulators, or the press.
- Has waited more than 14 days for a refund that was already approved.
- Requests a refund larger than $500.
## How to escalate
1. Acknowledge the issue in one sentence.
2. Tell the customer you are escalating to a senior specialist.
3. Provide the escalation reference `ESC-CANARY-7742` and the SLA: a senior
specialist will reply within 1 business day.
4. Do not promise a specific outcome (refund, replacement, compensation) on
escalated tickets — only the senior specialist can commit to one.
## Do not escalate
- Routine returns within the standard 30-day window.
- Shipping status questions.
- Product care and usage questions.
@@ -0,0 +1,25 @@
---
name: support-style
description: Contoso Outdoors customer-support tone and formatting guidelines.
---
# Contoso Outdoors Support Style
You are speaking on behalf of Contoso Outdoors customer support.
## Voice
- Warm, concise, and confident — never apologetic in a hand-wringing way.
- Use the customer's name when it is known.
- Sign every response with `— Contoso Outdoors Support`.
## Formatting
- Keep replies to 13 short paragraphs unless the customer asks for detail.
- Use bullet lists only when enumerating concrete steps or options.
- Always reference order numbers as `Order #<id>` (e.g. `Order #A-1042`).
## Canary
To prove this skill was loaded, include the token `STYLE-CANARY-3318` on a
separate line at the bottom of every response, prefixed with `# `.
@@ -27,9 +27,9 @@
<!-- For end-users: uncomment the PackageReference below and remove the ProjectReferences above
<ItemGroup>
<PackageReference Include="Microsoft.Agents.AI.Foundry" Version="1.0.0" />
<PackageReference Include="Microsoft.Agents.AI.Foundry.Hosting" Version="1.0.0" />
<PackageReference Include="Microsoft.Agents.AI.OpenAI" Version="1.0.0" />
<PackageReference Include="Microsoft.Agents.AI.Foundry" Version="1.6.1-preview.260514.1" />
<PackageReference Include="Microsoft.Agents.AI.Foundry.Hosting" Version="1.6.1-preview.260514.1" />
<PackageReference Include="Microsoft.Agents.AI.OpenAI" Version="1.6.1" />
</ItemGroup>
-->
@@ -23,8 +23,8 @@
<!-- For end-users: uncomment the PackageReference below and remove the ProjectReference above
<ItemGroup>
<PackageReference Include="Microsoft.Agents.AI.Foundry" Version="1.0.0" />
<PackageReference Include="Microsoft.Agents.AI.Foundry.Hosting" Version="1.0.0" />
<PackageReference Include="Microsoft.Agents.AI.Foundry" Version="1.6.1-preview.260514.1" />
<PackageReference Include="Microsoft.Agents.AI.Foundry.Hosting" Version="1.6.1-preview.260514.1" />
<PackageReference Include="Azure.AI.Projects" />
<PackageReference Include="Azure.Identity" />
</ItemGroup>
@@ -33,8 +33,8 @@
<!-- For end-users: uncomment the PackageReference below and remove the ProjectReference above
<ItemGroup>
<PackageReference Include="Microsoft.Agents.AI.Foundry" Version="1.0.0" />
<PackageReference Include="Microsoft.Agents.AI.Foundry.Hosting" Version="1.0.0" />
<PackageReference Include="Microsoft.Agents.AI.Foundry" Version="1.6.1-preview.260514.1" />
<PackageReference Include="Microsoft.Agents.AI.Foundry.Hosting" Version="1.6.1-preview.260514.1" />
</ItemGroup>
-->
@@ -23,8 +23,8 @@
<!-- For end-users: uncomment the PackageReference below and remove the ProjectReference above
<ItemGroup>
<PackageReference Include="Microsoft.Agents.AI.Foundry" Version="1.0.0" />
<PackageReference Include="Microsoft.Agents.AI.Foundry.Hosting" Version="1.0.0" />
<PackageReference Include="Microsoft.Agents.AI.Foundry" Version="1.6.1-preview.260514.1" />
<PackageReference Include="Microsoft.Agents.AI.Foundry.Hosting" Version="1.6.1-preview.260514.1" />
<PackageReference Include="Azure.AI.Projects" />
<PackageReference Include="Azure.Identity" />
</ItemGroup>
@@ -25,8 +25,8 @@
<!-- For end-users: uncomment the PackageReference below and remove the ProjectReference above
<ItemGroup>
<PackageReference Include="Microsoft.Agents.AI.Foundry" Version="1.0.0" />
<PackageReference Include="Microsoft.Agents.AI.Foundry.Hosting" Version="1.0.0" />
<PackageReference Include="Microsoft.Agents.AI.Foundry" Version="1.6.1-preview.260514.1" />
<PackageReference Include="Microsoft.Agents.AI.Foundry.Hosting" Version="1.6.1-preview.260514.1" />
</ItemGroup>
-->
@@ -26,8 +26,8 @@
<!-- For end-users: uncomment the PackageReference below and remove the ProjectReference above
<ItemGroup>
<PackageReference Include="Microsoft.Agents.AI.Foundry" Version="1.0.0" />
<PackageReference Include="Microsoft.Agents.AI.Foundry.Hosting" Version="1.0.0" />
<PackageReference Include="Microsoft.Agents.AI.Foundry" Version="1.6.1-preview.260514.1" />
<PackageReference Include="Microsoft.Agents.AI.Foundry.Hosting" Version="1.6.1-preview.260514.1" />
</ItemGroup>
-->
@@ -25,8 +25,8 @@
<!-- For end-users: uncomment the PackageReference below and remove the ProjectReferences above
<ItemGroup>
<PackageReference Include="Microsoft.Agents.AI.Foundry" Version="1.0.0" />
<PackageReference Include="Microsoft.Agents.AI.Foundry.Hosting" Version="1.0.0" />
<PackageReference Include="Microsoft.Agents.AI.Foundry" Version="1.6.1-preview.260514.1" />
<PackageReference Include="Microsoft.Agents.AI.Foundry.Hosting" Version="1.6.1-preview.260514.1" />
</ItemGroup>
-->
@@ -25,8 +25,8 @@
<!-- For end-users: uncomment the PackageReference below and remove the ProjectReference above
<ItemGroup>
<PackageReference Include="Microsoft.Agents.AI.Foundry" Version="1.0.0" />
<PackageReference Include="Microsoft.Agents.AI.Foundry.Hosting" Version="1.0.0" />
<PackageReference Include="Microsoft.Agents.AI.Foundry" Version="1.6.1-preview.260514.1" />
<PackageReference Include="Microsoft.Agents.AI.Foundry.Hosting" Version="1.6.1-preview.260514.1" />
</ItemGroup>
-->
@@ -26,9 +26,9 @@
<!-- For end-users: uncomment the PackageReference below and remove the ProjectReferences above
<ItemGroup>
<PackageReference Include="Microsoft.Agents.AI.Foundry" Version="1.0.0" />
<PackageReference Include="Microsoft.Agents.AI.Foundry.Hosting" Version="1.0.0" />
<PackageReference Include="Microsoft.Agents.AI.OpenAI" Version="1.0.0" />
<PackageReference Include="Microsoft.Agents.AI.Foundry" Version="1.6.1-preview.260514.1" />
<PackageReference Include="Microsoft.Agents.AI.Foundry.Hosting" Version="1.6.1-preview.260514.1" />
<PackageReference Include="Microsoft.Agents.AI.OpenAI" Version="1.6.1" />
</ItemGroup>
-->
@@ -25,8 +25,8 @@
<!-- For end-users: uncomment the PackageReference below and remove the ProjectReference above
<ItemGroup>
<PackageReference Include="Microsoft.Agents.AI.Foundry" Version="1.0.0" />
<PackageReference Include="Microsoft.Agents.AI.Foundry.Hosting" Version="1.0.0" />
<PackageReference Include="Microsoft.Agents.AI.Foundry" Version="1.6.1-preview.260514.1" />
<PackageReference Include="Microsoft.Agents.AI.Foundry.Hosting" Version="1.6.1-preview.260514.1" />
</ItemGroup>
-->
@@ -32,11 +32,11 @@
<!-- For end-users: uncomment the PackageReference below and remove the ProjectReference above
<ItemGroup>
<PackageReference Include="Microsoft.Agents.AI.Foundry" />
<PackageReference Include="Microsoft.Agents.AI.Foundry.Hosting" />
<PackageReference Include="Microsoft.Agents.AI.Hosting" />
<PackageReference Include="Microsoft.Agents.AI.OpenAI" />
<PackageReference Include="Microsoft.Agents.AI.Workflows" />
<PackageReference Include="Microsoft.Agents.AI.Foundry" Version="1.6.1-preview.260514.1" />
<PackageReference Include="Microsoft.Agents.AI.Foundry.Hosting" Version="1.6.1-preview.260514.1" />
<PackageReference Include="Microsoft.Agents.AI.Hosting" Version="1.6.1-preview.260514.1" />
<PackageReference Include="Microsoft.Agents.AI.OpenAI" Version="1.6.1" />
<PackageReference Include="Microsoft.Agents.AI.Workflows" Version="1.6.1" />
</ItemGroup>
-->
@@ -27,10 +27,10 @@
<!-- For end-users: uncomment the PackageReference below and remove the ProjectReferences above
<ItemGroup>
<PackageReference Include="Microsoft.Agents.AI.Foundry" Version="1.0.0" />
<PackageReference Include="Microsoft.Agents.AI.Foundry.Hosting" Version="1.0.0" />
<PackageReference Include="Microsoft.Agents.AI.OpenAI" Version="1.0.0" />
<PackageReference Include="Microsoft.Agents.AI.Workflows" Version="1.0.0" />
<PackageReference Include="Microsoft.Agents.AI.Foundry" Version="1.6.1-preview.260514.1" />
<PackageReference Include="Microsoft.Agents.AI.Foundry.Hosting" Version="1.6.1-preview.260514.1" />
<PackageReference Include="Microsoft.Agents.AI.OpenAI" Version="1.6.1" />
<PackageReference Include="Microsoft.Agents.AI.Workflows" Version="1.6.1" />
</ItemGroup>
-->
@@ -20,8 +20,28 @@ internal static class AGUIChatMessageExtensions
this IEnumerable<AGUIMessage> aguiMessages,
JsonSerializerOptions jsonSerializerOptions)
{
// Coalesce consecutive AGUIAssistantMessages that carry tool_calls into a single
// ChatMessage. The AG-UI client (e.g. @ag-ui/client) creates a separate assistant
// message per tool call when ToolCallStartEvent.parentMessageId is empty, but
// OpenAI's chat-completion API requires every assistant message with tool_calls
// to be IMMEDIATELY followed by tool responses for each of its tool_call_ids.
// Sending two consecutive single-tool-call assistant messages before any tool
// result triggers HTTP 400 "tool_call_ids did not have response messages".
List<AIContent>? pendingContents = null;
string? pendingId = null;
foreach (var message in aguiMessages)
{
bool isAssistantWithToolCalls =
message is AGUIAssistantMessage am && am.ToolCalls is { Length: > 0 };
if (pendingContents is not null && !isAssistantWithToolCalls)
{
yield return new ChatMessage(ChatRole.Assistant, pendingContents) { MessageId = pendingId };
pendingContents = null;
pendingId = null;
}
var role = MapChatRole(message.Role);
switch (message)
@@ -84,14 +104,14 @@ internal static class AGUIChatMessageExtensions
case AGUIAssistantMessage assistantMessage when assistantMessage.ToolCalls is { Length: > 0 }:
{
var contents = new List<AIContent>();
pendingContents ??= new List<AIContent>();
pendingId ??= message.Id;
if (!string.IsNullOrEmpty(assistantMessage.Content))
{
contents.Add(new TextContent(assistantMessage.Content));
pendingContents.Add(new TextContent(assistantMessage.Content));
}
// Add tool calls
foreach (var toolCall in assistantMessage.ToolCalls)
{
Dictionary<string, object?>? arguments = null;
@@ -102,16 +122,12 @@ internal static class AGUIChatMessageExtensions
jsonSerializerOptions.GetTypeInfo(typeof(Dictionary<string, object?>)));
}
contents.Add(new FunctionCallContent(
pendingContents.Add(new FunctionCallContent(
toolCall.Id,
toolCall.Function.Name,
arguments));
}
yield return new ChatMessage(role, contents)
{
MessageId = message.Id
};
break;
}
@@ -134,6 +150,12 @@ internal static class AGUIChatMessageExtensions
}
}
}
// Flush remaining pending assistant-tool-call entry at end of stream.
if (pendingContents is not null)
{
yield return new ChatMessage(ChatRole.Assistant, pendingContents) { MessageId = pendingId };
}
}
public static IEnumerable<AGUIMessage> AsAGUIMessages(
@@ -448,24 +448,36 @@ internal static class ChatResponseUpdateAGUIExtensions
};
string? currentMessageId = null;
string? streamingMessageId = null;
string? textStreamingFallback = null;
bool textInFallback = false;
string? currentReasoningBaseId = null;
string? currentReasoningId = null;
string? currentReasoningMessageId = null;
await foreach (var chatResponse in updates.WithCancellation(cancellationToken).ConfigureAwait(false))
{
// Generate a fallback MessageId when the provider doesn't supply one.
// This ensures all AGUI events have a valid messageId regardless of agent type.
if (string.IsNullOrWhiteSpace(chatResponse.MessageId))
// The text-event surface (TextMessageStart/Content/End) requires a non-empty
// MessageId to be valid AGUI. Generate a fallback scoped to a contiguous run of
// null/empty-MessageId chunks (one logical text message). Leave the raw
// chatResponse.MessageId untouched so the tool-call surface below uses the raw
// provider value — collapsing parallel tool calls under a synthetic shared parent
// would make the FE render them as one assistant-message bubble instead of
// distinct rows.
string? textMessageId = chatResponse.MessageId;
if (string.IsNullOrWhiteSpace(textMessageId))
{
chatResponse.MessageId = ContainsToolResult(chatResponse)
? Guid.NewGuid().ToString("N")
: (streamingMessageId ??= Guid.NewGuid().ToString("N"));
textStreamingFallback ??= Guid.NewGuid().ToString("N");
textMessageId = textStreamingFallback;
textInFallback = true;
}
else if (textInFallback)
{
textStreamingFallback = null;
textInFallback = false;
}
if (chatResponse is { Contents.Count: > 0 } &&
chatResponse.Contents[0] is TextContent &&
!string.Equals(currentMessageId, chatResponse.MessageId, StringComparison.Ordinal))
!string.Equals(currentMessageId, textMessageId, StringComparison.Ordinal))
{
// Close any open reasoning block before opening a text message, so AG-UI
// events are properly bracketed. MEAI providers share one MessageId across
@@ -498,11 +510,11 @@ internal static class ChatResponseUpdateAGUIExtensions
// Start the new message
yield return new TextMessageStartEvent
{
MessageId = chatResponse.MessageId!,
MessageId = textMessageId!,
Role = chatResponse.Role!.Value.Value
};
currentMessageId = chatResponse.MessageId;
currentMessageId = textMessageId;
}
// Emit text content if present
@@ -577,9 +589,15 @@ internal static class ChatResponseUpdateAGUIExtensions
currentReasoningMessageId = null;
}
// Each tool result is a distinct tool-role message on the AGUI wire.
// MEAI's FunctionInvokingChatClient shares one synthetic MessageId
// across all FunctionResultContent items, but the FE keys messages
// by id, so emitting them with the same id collapses them in React
// reconciliation. Derive a unique, deterministic per-result id from
// the (LLM-assigned) call id.
yield return new ToolCallResultEvent
{
MessageId = chatResponse.MessageId,
MessageId = $"result-{functionResultContent.CallId}",
ToolCallId = functionResultContent.CallId,
Content = SerializeResultContent(functionResultContent, jsonSerializerOptions) ?? "",
Role = AGUIRoles.Tool
@@ -674,7 +692,7 @@ internal static class ChatResponseUpdateAGUIExtensions
// Text content event
yield return new TextMessageContentEvent
{
MessageId = chatResponse.MessageId!,
MessageId = textMessageId!,
#if !NET
Delta = Encoding.UTF8.GetString(dataContent.Data.ToArray())
#else
@@ -726,17 +744,4 @@ internal static class ChatResponseUpdateAGUIExtensions
_ => JsonSerializer.Serialize(functionResultContent.Result, options.GetTypeInfo(functionResultContent.Result.GetType())),
};
}
private static bool ContainsToolResult(ChatResponseUpdate chatResponse)
{
foreach (AIContent content in chatResponse.Contents)
{
if (content is FunctionResultContent)
{
return true;
}
}
return false;
}
}
@@ -7,7 +7,7 @@
## v1.0.0-preview.260219.1
- [BREAKING] Changed ChatHistory and AIContext Providers to have pipeline semantics ([#3806](https://github.com/microsoft/agent-framework/pull/3806))
- Marked all `RunAsync<T>` overloads as `new`, added missing ones, and added support for primitives and arrays ([#3803](https://github.com/microsoft/agent-framework/pull/3803))
- Marked all `RunAsync<T>` overloads as `new`, added missing ones, and added support for primitives and arrays #3803
- Improve session cast error message quality and consistency ([#3973](https://github.com/microsoft/agent-framework/pull/3973))
## v1.0.0-preview.260212.1
@@ -22,6 +22,11 @@ internal sealed class ForeachExecutor : DeclarativeActionExecutor<Foreach>
public static string End(string id) => $"{id}_{nameof(End)}";
}
// State keys for checkpoint persistence of iteration progress.
private const string IndexStateKey = nameof(_index);
private const string ValuesStateKey = nameof(_values);
private const string HasValueStateKey = nameof(HasValue);
private int _index;
private FormulaValue[] _values;
@@ -93,4 +98,45 @@ internal sealed class ForeachExecutor : DeclarativeActionExecutor<Foreach>
await context.QueueStateResetAsync(this.Model.Index, cancellationToken).ConfigureAwait(false);
}
}
/// <inheritdoc/>
/// <remarks>
/// Persists the iteration cursor (<see cref="_index"/>), the materialized item snapshot
/// (<see cref="_values"/> as <see cref="PortableValue"/>[]), and <see cref="HasValue"/> so a
/// foreach loop can resume mid-iteration after a checkpoint (e.g. when a <c>Question</c>
/// inside the loop body pauses the workflow and the executor is re-instantiated on resume).
/// </remarks>
protected override async ValueTask OnCheckpointingAsync(IWorkflowContext context, CancellationToken cancellationToken = default)
{
PortableValue[] portableValues = [.. this._values.Select(value => new PortableValue(value.AsPortable()))];
await context.QueueStateUpdateAsync(IndexStateKey, this._index, cancellationToken: cancellationToken).ConfigureAwait(false);
await context.QueueStateUpdateAsync(ValuesStateKey, portableValues, cancellationToken: cancellationToken).ConfigureAwait(false);
await context.QueueStateUpdateAsync(HasValueStateKey, this.HasValue, cancellationToken: cancellationToken).ConfigureAwait(false);
await base.OnCheckpointingAsync(context, cancellationToken).ConfigureAwait(false);
}
/// <inheritdoc/>
/// <remarks>
/// Restores the iteration cursor, item snapshot, and <see cref="HasValue"/> recorded by
/// <see cref="OnCheckpointingAsync"/>. The presence of the values snapshot is the source of
/// truth for "this foreach was previously checkpointed"; if it is absent the executor keeps
/// its constructor defaults (fresh-start semantics).
/// </remarks>
protected override async ValueTask OnCheckpointRestoredAsync(IWorkflowContext context, CancellationToken cancellationToken = default)
{
await base.OnCheckpointRestoredAsync(context, cancellationToken).ConfigureAwait(false);
PortableValue[]? savedValues =
await context.ReadStateAsync<PortableValue[]>(ValuesStateKey, cancellationToken: cancellationToken).ConfigureAwait(false);
if (savedValues is null)
{
return;
}
this._values = [.. savedValues.Select(value => value.ToFormula())];
this._index = await context.ReadStateAsync<int>(IndexStateKey, cancellationToken: cancellationToken).ConfigureAwait(false);
this.HasValue = await context.ReadStateAsync<bool>(HasValueStateKey, cancellationToken: cancellationToken).ConfigureAwait(false);
}
}
@@ -43,6 +43,27 @@
<Right>lib/net10.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkill.get_Content</Target>
<Left>lib/net10.0/Microsoft.Agents.AI.dll</Left>
<Right>lib/net10.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkill.get_Resources</Target>
<Left>lib/net10.0/Microsoft.Agents.AI.dll</Left>
<Right>lib/net10.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkill.get_Scripts</Target>
<Left>lib/net10.0/Microsoft.Agents.AI.dll</Left>
<Right>lib/net10.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkillScript.RunAsync(Microsoft.Agents.AI.AgentSkill,Microsoft.Extensions.AI.AIFunctionArguments,System.Threading.CancellationToken)</Target>
@@ -106,6 +127,27 @@
<Right>lib/net472/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkill.get_Content</Target>
<Left>lib/net472/Microsoft.Agents.AI.dll</Left>
<Right>lib/net472/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkill.get_Resources</Target>
<Left>lib/net472/Microsoft.Agents.AI.dll</Left>
<Right>lib/net472/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkill.get_Scripts</Target>
<Left>lib/net472/Microsoft.Agents.AI.dll</Left>
<Right>lib/net472/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkillScript.RunAsync(Microsoft.Agents.AI.AgentSkill,Microsoft.Extensions.AI.AIFunctionArguments,System.Threading.CancellationToken)</Target>
@@ -169,6 +211,27 @@
<Right>lib/net8.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkill.get_Content</Target>
<Left>lib/net8.0/Microsoft.Agents.AI.dll</Left>
<Right>lib/net8.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkill.get_Resources</Target>
<Left>lib/net8.0/Microsoft.Agents.AI.dll</Left>
<Right>lib/net8.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkill.get_Scripts</Target>
<Left>lib/net8.0/Microsoft.Agents.AI.dll</Left>
<Right>lib/net8.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkillScript.RunAsync(Microsoft.Agents.AI.AgentSkill,Microsoft.Extensions.AI.AIFunctionArguments,System.Threading.CancellationToken)</Target>
@@ -232,6 +295,27 @@
<Right>lib/net9.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkill.get_Content</Target>
<Left>lib/net9.0/Microsoft.Agents.AI.dll</Left>
<Right>lib/net9.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkill.get_Resources</Target>
<Left>lib/net9.0/Microsoft.Agents.AI.dll</Left>
<Right>lib/net9.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkill.get_Scripts</Target>
<Left>lib/net9.0/Microsoft.Agents.AI.dll</Left>
<Right>lib/net9.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkillScript.RunAsync(Microsoft.Agents.AI.AgentSkill,Microsoft.Extensions.AI.AIFunctionArguments,System.Threading.CancellationToken)</Target>
@@ -295,6 +379,27 @@
<Right>lib/netstandard2.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkill.get_Content</Target>
<Left>lib/netstandard2.0/Microsoft.Agents.AI.dll</Left>
<Right>lib/netstandard2.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkill.get_Resources</Target>
<Left>lib/netstandard2.0/Microsoft.Agents.AI.dll</Left>
<Right>lib/netstandard2.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkill.get_Scripts</Target>
<Left>lib/netstandard2.0/Microsoft.Agents.AI.dll</Left>
<Right>lib/netstandard2.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0002</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkillScript.RunAsync(Microsoft.Agents.AI.AgentSkill,Microsoft.Extensions.AI.AIFunctionArguments,System.Threading.CancellationToken)</Target>
@@ -316,6 +421,13 @@
<Right>lib/netstandard2.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0005</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkill.GetContentAsync(System.Threading.CancellationToken)</Target>
<Left>lib/net10.0/Microsoft.Agents.AI.dll</Left>
<Right>lib/net10.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0005</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkillScript.RunAsync(Microsoft.Agents.AI.AgentSkill,System.Nullable{System.Text.Json.JsonElement},System.IServiceProvider,System.Threading.CancellationToken)</Target>
@@ -323,6 +435,13 @@
<Right>lib/net10.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0005</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkill.GetContentAsync(System.Threading.CancellationToken)</Target>
<Left>lib/net472/Microsoft.Agents.AI.dll</Left>
<Right>lib/net472/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0005</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkillScript.RunAsync(Microsoft.Agents.AI.AgentSkill,System.Nullable{System.Text.Json.JsonElement},System.IServiceProvider,System.Threading.CancellationToken)</Target>
@@ -330,6 +449,13 @@
<Right>lib/net472/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0005</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkill.GetContentAsync(System.Threading.CancellationToken)</Target>
<Left>lib/net8.0/Microsoft.Agents.AI.dll</Left>
<Right>lib/net8.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0005</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkillScript.RunAsync(Microsoft.Agents.AI.AgentSkill,System.Nullable{System.Text.Json.JsonElement},System.IServiceProvider,System.Threading.CancellationToken)</Target>
@@ -337,6 +463,13 @@
<Right>lib/net8.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0005</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkill.GetContentAsync(System.Threading.CancellationToken)</Target>
<Left>lib/net9.0/Microsoft.Agents.AI.dll</Left>
<Right>lib/net9.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0005</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkillScript.RunAsync(Microsoft.Agents.AI.AgentSkill,System.Nullable{System.Text.Json.JsonElement},System.IServiceProvider,System.Threading.CancellationToken)</Target>
@@ -344,6 +477,13 @@
<Right>lib/net9.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0005</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkill.GetContentAsync(System.Threading.CancellationToken)</Target>
<Left>lib/netstandard2.0/Microsoft.Agents.AI.dll</Left>
<Right>lib/netstandard2.0/Microsoft.Agents.AI.dll</Right>
<IsBaselineSuppression>true</IsBaselineSuppression>
</Suppression>
<Suppression>
<DiagnosticId>CP0005</DiagnosticId>
<Target>M:Microsoft.Agents.AI.AgentSkillScript.RunAsync(Microsoft.Agents.AI.AgentSkill,System.Nullable{System.Text.Json.JsonElement},System.IServiceProvider,System.Threading.CancellationToken)</Target>
@@ -29,8 +29,8 @@ namespace Microsoft.Agents.AI;
/// <para>
/// This provider exposes the following tools to the agent:
/// <list type="bullet">
/// <item><description><c>AgentMode_Set</c> — Switch the agent's operating mode.</description></item>
/// <item><description><c>AgentMode_Get</c> — Retrieve the agent's current operating mode.</description></item>
/// <item><description><c>mode_set</c> — Switch the agent's operating mode.</description></item>
/// <item><description><c>mode_get</c> — Retrieve the agent's current operating mode.</description></item>
/// </list>
/// </para>
/// <para>
@@ -49,8 +49,8 @@ public sealed class AgentModeProvider : AIContextProvider
- You must check the current mode after any user input, since the user may have changed the mode themselves,
e.g. the user may have switched to 'plan' mode after a previous research task finished in 'execute' mode, meaning they want to review a plan first before execution.
Use the AgentMode_Get tool to check your current operating mode.
Use the AgentMode_Set tool to switch between modes as your work progresses. Only use AgentMode_Set if the user explicitly instructs/allows you to change modes.
Use the mode_get tool to check your current operating mode.
Use the mode_set tool to switch between modes as your work progresses. Only use mode_set if the user explicitly instructs/allows you to change modes.
You are currently operating in the {current_mode} mode.
@@ -79,7 +79,7 @@ public sealed class AgentModeProvider : AIContextProvider
4. Do short exploratory research if it helps with being able to ask sensible clarifications from the user.
5. Write the plan to a memory file, so that it is retained even if compaction happens. Make sure to update the plan file if the user requests changes.
6. Present the plan to the user and ask for approval to switch to execute mode and process the plan.
7. When approval is granted, always switch to execute mode (using the `AgentMode_Set` tool), and follow the steps for *Execute mode*.
7. When approval is granted, always switch to execute mode (using the `mode_set` tool), and follow the steps for *Execute mode*.
"""),
new(
"execute",
@@ -263,7 +263,7 @@ public sealed class AgentModeProvider : AIContextProvider
},
new AIFunctionFactoryOptions
{
Name = "AgentMode_Set",
Name = "mode_set",
Description = $"Switch the agent's operating mode. Supported modes: \"{this._modeNamesDisplay}\".",
SerializerOptions = serializerOptions,
}),
@@ -272,7 +272,7 @@ public sealed class AgentModeProvider : AIContextProvider
() => state.CurrentMode,
new AIFunctionFactoryOptions
{
Name = "AgentMode_Get",
Name = "mode_get",
Description = "Get the agent's current operating mode.",
SerializerOptions = serializerOptions,
}),
@@ -1,7 +1,8 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Shared.DiagnosticIds;
namespace Microsoft.Agents.AI;
@@ -34,29 +35,44 @@ public abstract class AgentSkill
/// <summary>
/// Gets the full skill content.
/// </summary>
/// <remarks>
/// <param name="cancellationToken">Cancellation token.</param>
/// <returns>
/// For file-based skills this is the raw SKILL.md file content, optionally
/// augmented with a synthesized scripts block when scripts are present.
/// For code-defined skills this is a synthesized XML document
/// containing name, description, and body (instructions, resources, scripts).
/// </remarks>
public abstract string Content { get; }
/// </returns>
public abstract ValueTask<string> GetContentAsync(CancellationToken cancellationToken = default);
/// <summary>
/// Gets the resources associated with this skill, or <see langword="null"/> if none.
/// Gets a resource owned by this skill by name.
/// </summary>
/// <param name="name">The resource name (e.g. an identifier or a relative path referenced inside the skill content).</param>
/// <param name="cancellationToken">Cancellation token.</param>
/// <returns>
/// The <see cref="AgentSkillResource"/>, or <see langword="null"/> when no resource with the given name exists.
/// </returns>
/// <remarks>
/// The default implementation returns <see langword="null"/>.
/// Override this property in derived classes to provide skill-specific resources.
/// The default implementation returns <see langword="null"/>. Override in derived classes that
/// expose resources.
/// </remarks>
public virtual IReadOnlyList<AgentSkillResource>? Resources => null;
public virtual ValueTask<AgentSkillResource?> GetResourceAsync(
string name,
CancellationToken cancellationToken = default) => default;
/// <summary>
/// Gets the scripts associated with this skill, or <see langword="null"/> if none.
/// Gets a script owned by this skill by name.
/// </summary>
/// <param name="name">The script name.</param>
/// <param name="cancellationToken">Cancellation token.</param>
/// <returns>
/// The <see cref="AgentSkillScript"/>, or <see langword="null"/> when no script with the given name exists.
/// </returns>
/// <remarks>
/// The default implementation returns <see langword="null"/>.
/// Override this property in derived classes to provide skill-specific scripts.
/// The default implementation returns <see langword="null"/>. Override in derived classes that
/// expose scripts.
/// </remarks>
public virtual IReadOnlyList<AgentSkillScript>? Scripts => null;
public virtual ValueTask<AgentSkillScript?> GetScriptAsync(
string name,
CancellationToken cancellationToken = default) => default;
}
@@ -186,13 +186,10 @@ public sealed partial class AgentSkillsProvider : AIContextProvider
return await base.ProvideAIContextAsync(context, cancellationToken).ConfigureAwait(false);
}
bool hasScripts = skills.Any(s => s.Scripts is { Count: > 0 });
bool hasResources = skills.Any(s => s.Resources is { Count: > 0 });
return new AIContext
{
Instructions = this.BuildSkillsInstructions(skills, includeScriptInstructions: hasScripts, hasResources),
Tools = this.BuildTools(skills, hasScripts, hasResources),
Instructions = this.BuildSkillsInstructions(skills),
Tools = this.BuildTools(skills),
};
}
@@ -219,29 +216,20 @@ public sealed partial class AgentSkillsProvider : AIContextProvider
}
}
private IList<AIFunction> BuildTools(IList<AgentSkill> skills, bool hasScripts, bool hasResources)
private IList<AIFunction> BuildTools(IList<AgentSkill> skills)
{
IList<AIFunction> tools =
[
AIFunctionFactory.Create(
(string skillName) => this.LoadSkill(skills, skillName),
(string skillName, CancellationToken cancellationToken) => this.LoadSkillAsync(skills, skillName, cancellationToken),
name: "load_skill",
description: "Loads the full content of a specific skill"),
];
if (hasResources)
{
tools.Add(AIFunctionFactory.Create(
AIFunctionFactory.Create(
(string skillName, string resourceName, IServiceProvider? serviceProvider, CancellationToken cancellationToken = default) =>
this.ReadSkillResourceAsync(skills, skillName, resourceName, serviceProvider, cancellationToken),
name: "read_skill_resource",
description: "Reads a resource associated with a skill, such as references, assets, or dynamic data."));
}
if (!hasScripts)
{
return tools;
}
description: "Reads a resource associated with a skill, such as references, assets, or dynamic data."),
];
AIFunction scriptFunction = AIFunctionFactory.Create(
(string skillName, string scriptName, JsonElement? arguments = null, IServiceProvider? serviceProvider = null, CancellationToken cancellationToken = default) =>
@@ -257,7 +245,7 @@ public sealed partial class AgentSkillsProvider : AIContextProvider
return [.. tools, scriptFunction];
}
private string? BuildSkillsInstructions(IList<AgentSkill> skills, bool includeScriptInstructions, bool includeResourceInstructions)
private string? BuildSkillsInstructions(IList<AgentSkill> skills)
{
string promptTemplate = this._options?.SkillsInstructionPrompt ?? DefaultSkillsInstructionPrompt;
@@ -270,32 +258,29 @@ public sealed partial class AgentSkillsProvider : AIContextProvider
sb.AppendLine(" </skill>");
}
string resourceInstruction = includeResourceInstructions
? """
const string ResourceInstruction =
"""
- Use `read_skill_resource` to read any referenced resources, using the name exactly as listed
(e.g. `"style-guide"` not `"style-guide.md"`, `"references/FAQ.md"` not `"FAQ.md"`).
"""
: string.Empty;
""";
string scriptInstruction = includeScriptInstructions
? "- Use `run_skill_script` to run referenced scripts, using the name exactly as listed."
: string.Empty;
const string ScriptInstruction = "- Use `run_skill_script` to run referenced scripts, using the name exactly as listed.";
return new StringBuilder(promptTemplate)
.Replace(SkillsPlaceholder, sb.ToString().TrimEnd())
.Replace(ResourceInstructionsPlaceholder, resourceInstruction)
.Replace(ScriptInstructionsPlaceholder, scriptInstruction)
.Replace(ResourceInstructionsPlaceholder, ResourceInstruction)
.Replace(ScriptInstructionsPlaceholder, ScriptInstruction)
.ToString();
}
private string LoadSkill(IList<AgentSkill> skills, string skillName)
private async Task<string> LoadSkillAsync(IList<AgentSkill> skills, string skillName, CancellationToken cancellationToken)
{
if (string.IsNullOrWhiteSpace(skillName))
{
return "Error: Skill name cannot be empty.";
}
var skill = skills?.FirstOrDefault(skill => skill.Frontmatter.Name == skillName);
var skill = skills.FirstOrDefault(skill => skill.Frontmatter.Name == skillName);
if (skill == null)
{
return $"Error: Skill '{skillName}' not found.";
@@ -303,7 +288,7 @@ public sealed partial class AgentSkillsProvider : AIContextProvider
LogSkillLoading(this._logger, skillName);
return skill.Content;
return await skill.GetContentAsync(cancellationToken).ConfigureAwait(false);
}
private async Task<object?> ReadSkillResourceAsync(IList<AgentSkill> skills, string skillName, string resourceName, IServiceProvider? serviceProvider, CancellationToken cancellationToken = default)
@@ -318,20 +303,20 @@ public sealed partial class AgentSkillsProvider : AIContextProvider
return "Error: Resource name cannot be empty.";
}
var skill = skills?.FirstOrDefault(skill => skill.Frontmatter.Name == skillName);
var skill = skills.FirstOrDefault(skill => skill.Frontmatter.Name == skillName);
if (skill == null)
{
return $"Error: Skill '{skillName}' not found.";
}
var resource = skill.Resources?.FirstOrDefault(resource => resource.Name == resourceName);
if (resource is null)
{
return $"Error: Resource '{resourceName}' not found in skill '{skillName}'.";
}
try
{
var resource = await skill.GetResourceAsync(resourceName, cancellationToken).ConfigureAwait(false);
if (resource is null)
{
return $"Error: Resource '{resourceName}' not found in skill '{skillName}'.";
}
return await resource.ReadAsync(serviceProvider, cancellationToken).ConfigureAwait(false);
}
catch (Exception ex)
@@ -353,20 +338,20 @@ public sealed partial class AgentSkillsProvider : AIContextProvider
return "Error: Script name cannot be empty.";
}
var skill = skills?.FirstOrDefault(skill => skill.Frontmatter.Name == skillName);
var skill = skills.FirstOrDefault(skill => skill.Frontmatter.Name == skillName);
if (skill == null)
{
return $"Error: Skill '{skillName}' not found.";
}
var script = skill.Scripts?.FirstOrDefault(resource => resource.Name == scriptName);
if (script is null)
{
return $"Error: Script '{scriptName}' not found in skill '{skillName}'.";
}
try
{
var script = await skill.GetScriptAsync(scriptName, cancellationToken).ConfigureAwait(false);
if (script is null)
{
return $"Error: Script '{scriptName}' not found in skill '{skillName}'.";
}
return await script.RunAsync(skill, arguments, serviceProvider, cancellationToken).ConfigureAwait(false);
}
catch (Exception ex)
@@ -2,6 +2,9 @@
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Shared.DiagnosticIds;
using Microsoft.Shared.Diagnostics;
@@ -50,11 +53,12 @@ public sealed class AgentFileSkill : AgentSkill
/// block is appended with a per-script entry describing the expected argument format.
/// The result is cached after the first access.
/// </remarks>
public override string Content
public override ValueTask<string> GetContentAsync(CancellationToken cancellationToken = default)
{
get => this._content ??= this._scripts is { Count: > 0 }
var content = this._content ??= this._scripts is { Count: > 0 }
? this._originalContent + AgentInlineSkillContentBuilder.BuildScriptsBlock(this._scripts)
: this._originalContent;
return new(content);
}
/// <summary>
@@ -63,8 +67,16 @@ public sealed class AgentFileSkill : AgentSkill
public string Path { get; }
/// <inheritdoc/>
public override IReadOnlyList<AgentSkillResource> Resources => this._resources;
public override ValueTask<AgentSkillResource?> GetResourceAsync(string name, CancellationToken cancellationToken = default)
{
var resource = this._resources.FirstOrDefault(r => r.Name == name);
return new(resource);
}
/// <inheritdoc/>
public override IReadOnlyList<AgentSkillScript> Scripts => this._scripts;
public override ValueTask<AgentSkillScript?> GetScriptAsync(string name, CancellationToken cancellationToken = default)
{
var script = this._scripts.FirstOrDefault(s => s.Name == name);
return new(script);
}
}
@@ -4,9 +4,11 @@ using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Diagnostics.CodeAnalysis;
using System.Linq;
using System.Reflection;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.AI;
using Microsoft.Shared.DiagnosticIds;
@@ -34,9 +36,9 @@ namespace Microsoft.Agents.AI;
/// discovered via reflection on <typeparamref name="TSelf"/>. This approach is compatible with Native AOT.
/// </item>
/// <item>
/// <b>Explicit override:</b> Override <see cref="AgentSkill.Resources"/> and <see cref="AgentSkill.Scripts"/>, using
/// <see cref="CreateResource(string, object, string?)"/>, <see cref="CreateResource(string, Delegate, string?, JsonSerializerOptions?)"/>,
/// and <see cref="CreateScript"/> to define inline resources and scripts. This approach is also compatible with Native AOT.
/// <b>Explicit override:</b> Override <see cref="Resources"/> and <see cref="Scripts"/>, using <see cref="CreateResource(string, object, string?)"/>,
/// <see cref="CreateResource(string, Delegate, string?, JsonSerializerOptions?)"/>, and <see cref="CreateScript"/> to define
/// inline resources and scripts. This approach is also compatible with Native AOT.
/// </item>
/// </list>
/// </para>
@@ -97,11 +99,24 @@ public abstract class AgentClassSkill<
{
private const BindingFlags DiscoveryBindingFlags = BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance | BindingFlags.Static;
private string? _content;
private bool _resourcesDiscovered;
private bool _scriptsDiscovered;
private IReadOnlyList<AgentSkillResource>? _reflectedResources;
private IReadOnlyList<AgentSkillScript>? _reflectedScripts;
private readonly Lazy<IReadOnlyList<AgentSkillResource>?> _resources;
private readonly Lazy<IReadOnlyList<AgentSkillScript>?> _scripts;
private readonly Lazy<string> _content;
/// <summary>
/// Initializes a new instance of the <see cref="AgentClassSkill{TSelf}"/> class.
/// </summary>
protected AgentClassSkill()
{
this._resources = new Lazy<IReadOnlyList<AgentSkillResource>?>(this.DiscoverResources);
this._scripts = new Lazy<IReadOnlyList<AgentSkillScript>?>(this.DiscoverScripts);
this._content = new Lazy<string>(() => AgentInlineSkillContentBuilder.Build(
this.Frontmatter.Name,
this.Frontmatter.Description,
this.Instructions,
this.Resources,
this.Scripts));
}
/// <summary>
/// Gets the raw instructions text for this skill.
@@ -126,53 +141,44 @@ public abstract class AgentClassSkill<
/// Returns a synthesized XML document containing name, description, instructions, resources, and scripts.
/// The result is cached after the first access. Override to provide custom content.
/// </remarks>
public override string Content => this._content ??= AgentInlineSkillContentBuilder.Build(
this.Frontmatter.Name,
this.Frontmatter.Description,
this.Instructions,
this.Resources,
this.Scripts);
public override ValueTask<string> GetContentAsync(CancellationToken cancellationToken = default) => new(this._content.Value);
/// <summary>
/// Gets the resources associated with this skill, or <see langword="null"/> if none.
/// </summary>
/// <remarks>
/// The default implementation returns resources discovered via reflection by scanning
/// <typeparamref name="TSelf"/> for members annotated with <see cref="AgentSkillResourceAttribute"/>.
/// This discovery is compatible with Native AOT because <typeparamref name="TSelf"/> is annotated with
/// <see cref="DynamicallyAccessedMembersAttribute"/>. The result is cached after the first access.
/// Override this property in derived classes to provide skill-specific resources.
/// </remarks>
public virtual IReadOnlyList<AgentSkillResource>? Resources => this._resources.Value;
/// <summary>
/// Gets the scripts associated with this skill, or <see langword="null"/> if none.
/// </summary>
/// <remarks>
/// The default implementation returns scripts discovered via reflection by scanning
/// <typeparamref name="TSelf"/> for methods annotated with <see cref="AgentSkillScriptAttribute"/>.
/// This discovery is compatible with Native AOT because <typeparamref name="TSelf"/> is annotated with
/// <see cref="DynamicallyAccessedMembersAttribute"/>. The result is cached after the first access.
/// Override this property in derived classes to provide skill-specific scripts.
/// </remarks>
public virtual IReadOnlyList<AgentSkillScript>? Scripts => this._scripts.Value;
/// <inheritdoc/>
/// <remarks>
/// Returns resources discovered via reflection by scanning <typeparamref name="TSelf"/> for
/// members annotated with <see cref="AgentSkillResourceAttribute"/>. This discovery is
/// compatible with Native AOT because <typeparamref name="TSelf"/> is annotated with
/// <see cref="DynamicallyAccessedMembersAttribute"/>. The result is cached after the first access.
/// </remarks>
public override IReadOnlyList<AgentSkillResource>? Resources
public sealed override ValueTask<AgentSkillResource?> GetResourceAsync(string name, CancellationToken cancellationToken = default)
{
get
{
if (!this._resourcesDiscovered)
{
this._reflectedResources = this.DiscoverResources();
this._resourcesDiscovered = true;
}
return this._reflectedResources;
}
var resource = this.Resources?.FirstOrDefault(r => r.Name == name);
return new(resource);
}
/// <inheritdoc/>
/// <remarks>
/// Returns scripts discovered via reflection by scanning <typeparamref name="TSelf"/> for
/// methods annotated with <see cref="AgentSkillScriptAttribute"/>. This discovery is
/// compatible with Native AOT because <typeparamref name="TSelf"/> is annotated with
/// <see cref="DynamicallyAccessedMembersAttribute"/>. The result is cached after the first access.
/// </remarks>
public override IReadOnlyList<AgentSkillScript>? Scripts
public sealed override ValueTask<AgentSkillScript?> GetScriptAsync(string name, CancellationToken cancellationToken = default)
{
get
{
if (!this._scriptsDiscovered)
{
this._reflectedScripts = this.DiscoverScripts();
this._scriptsDiscovered = true;
}
return this._reflectedScripts;
}
var script = this.Scripts?.FirstOrDefault(s => s.Name == name);
return new(script);
}
/// <summary>
@@ -3,7 +3,10 @@
using System;
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
using System.Linq;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.AI;
using Microsoft.Shared.DiagnosticIds;
using Microsoft.Shared.Diagnostics;
@@ -16,9 +19,9 @@ namespace Microsoft.Agents.AI;
/// <remarks>
/// All calls to <see cref="AddResource(string, object, string?)"/>,
/// <see cref="AddResource(string, Delegate, string?, JsonSerializerOptions?)"/>, and <see cref="AddScript"/>
/// must be made before the skill's <see cref="Content"/> is first accessed.
/// must be made before the skill's <see cref="GetContentAsync"/> is first called.
/// Calls made after that point will not be reflected in the generated
/// <see cref="Content"/>. In typical usage, this means configuring all
/// content. In typical usage, this means configuring all
/// resources and scripts before registering the skill with an
/// <see cref="AgentSkillsProvider"/> or <see cref="AgentSkillsProviderBuilder"/>.
/// </remarks>
@@ -90,13 +93,24 @@ public sealed class AgentInlineSkill : AgentSkill
public override AgentSkillFrontmatter Frontmatter { get; }
/// <inheritdoc/>
public override string Content => this._cachedContent ??= AgentInlineSkillContentBuilder.Build(this.Frontmatter.Name, this.Frontmatter.Description, this._instructions, this._resources, this._scripts);
public override ValueTask<string> GetContentAsync(CancellationToken cancellationToken = default)
{
return new(this._cachedContent ??= AgentInlineSkillContentBuilder.Build(this.Frontmatter.Name, this.Frontmatter.Description, this._instructions, this._resources, this._scripts));
}
/// <inheritdoc/>
public override IReadOnlyList<AgentSkillResource>? Resources => this._resources;
public override ValueTask<AgentSkillResource?> GetResourceAsync(string name, CancellationToken cancellationToken = default)
{
var resource = this._resources?.FirstOrDefault(r => r.Name == name);
return new(resource);
}
/// <inheritdoc/>
public override IReadOnlyList<AgentSkillScript>? Scripts => this._scripts;
public override ValueTask<AgentSkillScript?> GetScriptAsync(string name, CancellationToken cancellationToken = default)
{
var script = this._scripts?.FirstOrDefault(s => s.Name == name);
return new(script);
}
/// <summary>
/// Registers a static resource with this skill.
@@ -27,7 +27,7 @@ namespace Microsoft.Agents.AI;
/// </para>
/// <para>
/// This attribute is compatible with Native AOT when used with <see cref="AgentClassSkill{TSelf}"/>.
/// Alternatively, override the <see cref="AgentSkill.Resources"/> property and use
/// Alternatively, override <see cref="AgentClassSkill{TSelf}.Resources"/> and use
/// <see cref="AgentClassSkill{TSelf}.CreateResource(string, object, string?)"/> instead.
/// </para>
/// </remarks>
@@ -26,7 +26,7 @@ namespace Microsoft.Agents.AI;
/// </para>
/// <para>
/// This attribute is compatible with Native AOT when used with <see cref="AgentClassSkill{TSelf}"/>.
/// Alternatively, override the <see cref="AgentSkill.Scripts"/> property and use
/// Alternatively, override <see cref="AgentClassSkill{TSelf}.Scripts"/> and use
/// <see cref="AgentClassSkill{TSelf}.CreateScript"/> instead.
/// </para>
/// </remarks>
@@ -27,11 +27,13 @@
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\..\src\Microsoft.Agents.AI\Microsoft.Agents.AI.csproj" />
<ProjectReference Include="..\..\src\Microsoft.Agents.AI.Foundry\Microsoft.Agents.AI.Foundry.csproj" />
<ProjectReference Include="..\..\src\Microsoft.Agents.AI.Foundry.Hosting\Microsoft.Agents.AI.Foundry.Hosting.csproj" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="Azure.AI.Projects" />
<PackageReference Include="Azure.Identity" />
<PackageReference Include="Azure.Search.Documents" />
<PackageReference Include="Microsoft.Extensions.AI" />
@@ -38,6 +38,7 @@ AIAgent agent = scenario switch
"memory" => await CreateMemoryAgentAsync(projectClient, deployment).ConfigureAwait(false),
"azure-search-rag" => CreateAzureSearchRagAgent(projectClient, deployment),
"session-files" => CreateSessionFilesAgent(projectClient, deployment),
"agent-skills" => CreateAgentSkillsAgent(projectClient, deployment),
_ => throw new InvalidOperationException($"Unknown IT_SCENARIO '{scenario}'.")
};
@@ -209,6 +210,77 @@ static async Task<AIAgent> CreateMemoryAgentAsync(AIProjectClient client, string
});
}
// Agent skills scenario. Uses AgentSkillsProvider with two bundled Contoso Outdoors skills
// (support-style + escalation-policy). Skills are loaded from embedded SKILL.md files on disk,
// simulating the download-from-Foundry pattern used by the Hosted-AgentSkills sample. When the
// container starts, it writes the skills to a temp directory and wires AgentSkillsProvider over it.
#pragma warning disable MEAI001 // AgentSkillsProvider is experimental
static AIAgent CreateAgentSkillsAgent(AIProjectClient client, string deployment)
{
string skillsDir = Path.Combine(Path.GetTempPath(), "it-agent-skills-" + Guid.NewGuid().ToString("N"));
Directory.CreateDirectory(Path.Combine(skillsDir, "support-style"));
Directory.CreateDirectory(Path.Combine(skillsDir, "escalation-policy"));
File.WriteAllText(Path.Combine(skillsDir, "support-style", "SKILL.md"),
"""
---
name: support-style
description: Contoso Outdoors customer-support tone and formatting guidelines.
---
# Contoso Outdoors Support Style
You are speaking on behalf of Contoso Outdoors customer support.
## Voice
- Warm, concise, and confident.
- Use the customer's name when known.
- Sign every response with ` Contoso Outdoors Support`.
## Canary
To prove this skill was loaded, include the token `STYLE-CANARY-3318` on a
separate line at the bottom of every response, prefixed with `# `.
""");
File.WriteAllText(Path.Combine(skillsDir, "escalation-policy", "SKILL.md"),
"""
---
name: escalation-policy
description: When and how to escalate Contoso Outdoors customer-support tickets.
---
# Contoso Outdoors Escalation Policy
## Escalate immediately when the customer
- Reports an injury or safety incident.
- Mentions legal action, regulators, or the press.
- Requests a refund larger than $500.
## How to escalate
1. Acknowledge the issue.
2. Tell the customer you are escalating to a senior specialist.
3. Provide the escalation reference `ESC-CANARY-7742`.
""");
var skillsProvider = new AgentSkillsProvider(skillsDir, scriptRunner: null);
return client.AsAIAgent(new ChatClientAgentOptions
{
Name = "agent-skills-agent",
ChatOptions = new ChatOptions
{
ModelId = deployment,
Instructions = "You are a customer-support assistant for Contoso Outdoors.",
},
AIContextProviders = [skillsProvider]
});
}
#pragma warning restore MEAI001
[Description("Returns the current UTC date and time as an ISO 8601 string.")]
static string GetUtcNow() => DateTime.UtcNow.ToString("o");
@@ -0,0 +1,84 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Linq;
using System.Threading.Tasks;
using Foundry.Hosting.IntegrationTests.Fixtures;
using Microsoft.Extensions.AI;
namespace Foundry.Hosting.IntegrationTests;
/// <summary>
/// Integration tests that exercise the Agent Skills pattern in a hosted agent container.
/// The container uses <see cref="Microsoft.Agents.AI.AgentSkillsProvider"/> with two
/// Contoso Outdoors skills (support-style, escalation-policy) to verify the progressive
/// disclosure flow: skills are advertised in the system prompt and loaded on demand via
/// the <c>load_skill</c> tool when the model decides they are relevant.
/// </summary>
[Trait("Category", "FoundryHostedAgents")]
public sealed class AgentSkillsHostedAgentTests(AgentSkillsHostedAgentFixture fixture) : IClassFixture<AgentSkillsHostedAgentFixture>
{
private readonly AgentSkillsHostedAgentFixture _fixture = fixture;
[Fact(Skip = "Pending TestContainer build and end to end smoke (step 5).")]
public async Task RoutineQuestion_LoadsSupportStyleSkillAsync()
{
// Arrange
var agent = this._fixture.Agent;
// Act — ask a routine support question that should trigger the support-style skill
var response = await agent.RunAsync(
"Hi, I am Alex. I just want to confirm I can return my tent within 30 days.");
// Assert — response should contain the canary token proving the skill was loaded
Assert.False(string.IsNullOrWhiteSpace(response.Text));
Assert.Contains("STYLE-CANARY-3318", response.Text);
}
[Fact(Skip = "Pending TestContainer build and end to end smoke (step 5).")]
public async Task EscalationTrigger_LoadsEscalationPolicySkillAsync()
{
// Arrange
var agent = this._fixture.Agent;
// Act — trigger an escalation (legal threat + refund > $500)
var response = await agent.RunAsync(
"I want a $750 refund on Order #A-1042 right now or I am calling my lawyer.");
// Assert — response should contain the escalation canary token
Assert.False(string.IsNullOrWhiteSpace(response.Text));
Assert.Contains("ESC-CANARY-7742", response.Text);
}
[Fact(Skip = "Pending TestContainer build and end to end smoke (step 5).")]
public async Task SkillsAreAdvertised_LoadSkillToolIsAvailableAsync()
{
// Arrange
var agent = this._fixture.Agent;
// Act — ask the model what skills are available (triggers system prompt inspection)
var response = await agent.RunAsync(
"List the skills you have access to. Just give me their names.");
// Assert — both skills should be mentioned (they are advertised in the system prompt)
Assert.False(string.IsNullOrWhiteSpace(response.Text));
Assert.Contains("support-style", response.Text);
Assert.Contains("escalation-policy", response.Text);
}
[Fact(Skip = "Pending TestContainer build and end to end smoke (step 5).")]
public async Task LoadSkill_InvokesToolAndReturnsContentAsync()
{
// Arrange
var agent = this._fixture.Agent;
// Act — ask a question that should load a specific skill
var response = await agent.RunAsync(
"I need to know the escalation policy for customer tickets. Load the escalation-policy skill and tell me the rules.");
// Assert — the response should reference the load_skill tool invocation
Assert.False(string.IsNullOrWhiteSpace(response.Text));
Assert.True(
response.Messages.Any(m => m.Contents.OfType<FunctionCallContent>().Any(fc => fc.Name == "load_skill")),
"Expected at least one load_skill FunctionCallContent in the response messages.");
}
}
@@ -0,0 +1,14 @@
// Copyright (c) Microsoft. All rights reserved.
namespace Foundry.Hosting.IntegrationTests.Fixtures;
/// <summary>
/// Provisions a hosted agent that runs the test container in <c>IT_SCENARIO=agent-skills</c> mode.
/// The container creates two Contoso Outdoors skills (support-style, escalation-policy) on disk
/// and wires them into <see cref="Microsoft.Agents.AI.AgentSkillsProvider"/> so the model can
/// discover and load skills via the progressive disclosure pattern.
/// </summary>
public sealed class AgentSkillsHostedAgentFixture : HostedAgentFixture
{
protected override string ScenarioName => "agent-skills";
}
@@ -199,6 +199,7 @@ human-only operation; CI only adds and deletes versions under existing agents.
| `CustomStorageHostedAgentFixture` | `custom-storage` | `it-custom-storage` | Round trip with custom `IResponsesStorageProvider`; multi turn reads from the custom store (placeholder). |
| `AzureSearchRagHostedAgentFixture` | `azure-search-rag` | `it-azure-search-rag` | RAG against a real Azure AI Search index seeded with Contoso Outdoors documents; verifies the model cites the retrieved sources. |
| `SessionFilesHostedAgentFixture` | `session-files` | `it-session-files` | End-to-end: upload via `AgentSessionFiles` (alpha) into a pinned `agent_session_id`, invoke the agent, assert it reads the file via the container's `ReadFile` tool. |
| `AgentSkillsHostedAgentFixture` | `agent-skills` | `it-agent-skills` | Agent skills via `AgentSkillsProvider`: advertises two Contoso Outdoors skills (support-style, escalation-policy) in the system prompt, loads them on demand via `load_skill`, verifies canary tokens prove the skill was loaded. |
The placeholder scenarios will be wired up in the test container `Program.cs` once the
relevant `Microsoft.Agents.AI.Foundry.Hosting` API surfaces stabilize.
@@ -47,7 +47,8 @@ $Scenarios = @(
'custom-storage',
'memory',
'azure-search-rag',
'session-files'
'session-files',
'agent-skills'
)
# Resolve project ARM scope from the endpoint.
@@ -914,4 +914,147 @@ public sealed class AGUIChatMessageExtensionsTests
}
#endregion
#region Consecutive Assistant-Tool-Call Coalescing
/// <summary>
/// Bug #3 reproduction: consecutive AGUIAssistantMessages with ToolCalls should
/// be coalesced into a single ChatMessage with multiple FunctionCallContent
/// entries. Without coalescing, Azure OpenAI rejects the history with HTTP 400.
/// </summary>
[Fact]
public void AsChatMessages_ConsecutiveAssistantToolCallMessages_CoalesceIntoOneChatMessage()
{
// Arrange — 3 consecutive assistant messages with tool calls (no intervening tool msg)
List<AGUIMessage> aguiMessages =
[
new AGUIUserMessage { Id = "user-1", Content = "Run 3 queries" },
new AGUIAssistantMessage
{
Id = "asst-1",
Content = "",
ToolCalls =
[
new AGUIToolCall { Id = "call_A", Type = "function", Function = new AGUIFunctionCall { Name = "query", Arguments = "{\"q\":\"1\"}" } }
]
},
new AGUIAssistantMessage
{
Id = "asst-2",
Content = "",
ToolCalls =
[
new AGUIToolCall { Id = "call_B", Type = "function", Function = new AGUIFunctionCall { Name = "query", Arguments = "{\"q\":\"2\"}" } }
]
},
new AGUIAssistantMessage
{
Id = "asst-3",
Content = "",
ToolCalls =
[
new AGUIToolCall { Id = "call_C", Type = "function", Function = new AGUIFunctionCall { Name = "query", Arguments = "{\"q\":\"3\"}" } }
]
},
new AGUIToolMessage { Id = "tool-1", ToolCallId = "call_A", Content = "\"result1\"" },
new AGUIToolMessage { Id = "tool-2", ToolCallId = "call_B", Content = "\"result2\"" },
new AGUIToolMessage { Id = "tool-3", ToolCallId = "call_C", Content = "\"result3\"" },
new AGUIUserMessage { Id = "user-2", Content = "Run it again" },
];
// Act
List<ChatMessage> chatMessages = aguiMessages.AsChatMessages(AGUIJsonSerializerContext.Default.Options).ToList();
// Assert — the 3 consecutive assistant-tool-call messages should coalesce into 1
List<ChatMessage> assistantWithToolCalls = chatMessages
.Where(m => m.Role == ChatRole.Assistant && m.Contents.OfType<FunctionCallContent>().Any())
.ToList();
Assert.Single(assistantWithToolCalls);
// The single coalesced message should contain all 3 FunctionCallContent entries
List<FunctionCallContent> functionCalls = assistantWithToolCalls[0].Contents
.OfType<FunctionCallContent>().ToList();
Assert.Equal(3, functionCalls.Count);
Assert.Equal("call_A", functionCalls[0].CallId);
Assert.Equal("call_B", functionCalls[1].CallId);
Assert.Equal("call_C", functionCalls[2].CallId);
// MessageId should be from the first message in the coalesced group
Assert.Equal("asst-1", assistantWithToolCalls[0].MessageId);
// Total messages: user + coalesced assistant + 3 tools + user = 6
Assert.Equal(6, chatMessages.Count);
}
/// <summary>
/// A single assistant message with tool calls (not consecutive) should still
/// produce one ChatMessage — no behavior change from coalescing logic.
/// </summary>
[Fact]
public void AsChatMessages_SingleAssistantToolCallMessage_ProducesOneChatMessage()
{
// Arrange
List<AGUIMessage> aguiMessages =
[
new AGUIAssistantMessage
{
Id = "asst-1",
Content = "Here are the results",
ToolCalls =
[
new AGUIToolCall { Id = "call_A", Type = "function", Function = new AGUIFunctionCall { Name = "query", Arguments = "{}" } },
new AGUIToolCall { Id = "call_B", Type = "function", Function = new AGUIFunctionCall { Name = "query", Arguments = "{}" } },
]
},
new AGUIToolMessage { Id = "tool-1", ToolCallId = "call_A", Content = "\"r1\"" },
new AGUIToolMessage { Id = "tool-2", ToolCallId = "call_B", Content = "\"r2\"" },
];
// Act
List<ChatMessage> chatMessages = aguiMessages.AsChatMessages(AGUIJsonSerializerContext.Default.Options).ToList();
// Assert — single assistant message, not coalesced from multiple
Assert.Equal(3, chatMessages.Count);
Assert.Equal(ChatRole.Assistant, chatMessages[0].Role);
List<FunctionCallContent> calls = chatMessages[0].Contents.OfType<FunctionCallContent>().ToList();
Assert.Equal(2, calls.Count);
Assert.Equal("asst-1", chatMessages[0].MessageId);
}
/// <summary>
/// When consecutive assistant-tool-call messages are at the END of the stream
/// (no subsequent non-tool-call message to trigger flush), they should still
/// be coalesced and flushed.
/// </summary>
[Fact]
public void AsChatMessages_ConsecutiveAssistantToolCallsAtEndOfStream_FlushesCorrectly()
{
// Arrange — stream ends with consecutive assistant tool-call messages
List<AGUIMessage> aguiMessages =
[
new AGUIUserMessage { Id = "user-1", Content = "Do things" },
new AGUIAssistantMessage
{
Id = "asst-1",
ToolCalls = [new AGUIToolCall { Id = "call_X", Type = "function", Function = new AGUIFunctionCall { Name = "fn", Arguments = "{}" } }]
},
new AGUIAssistantMessage
{
Id = "asst-2",
ToolCalls = [new AGUIToolCall { Id = "call_Y", Type = "function", Function = new AGUIFunctionCall { Name = "fn", Arguments = "{}" } }]
},
];
// Act
List<ChatMessage> chatMessages = aguiMessages.AsChatMessages(AGUIJsonSerializerContext.Default.Options).ToList();
// Assert — should be user + 1 coalesced assistant = 2 messages
Assert.Equal(2, chatMessages.Count);
Assert.Equal(ChatRole.User, chatMessages[0].Role);
Assert.Equal(ChatRole.Assistant, chatMessages[1].Role);
Assert.Equal(2, chatMessages[1].Contents.OfType<FunctionCallContent>().Count());
}
#endregion
}
@@ -109,11 +109,13 @@ public sealed class AGUIStreamingMessageIdTests
}
/// <summary>
/// When ChatResponseUpdate has empty string MessageId, the AGUI layer generates
/// a fallback so ToolCallStartEvent.ParentMessageId is valid.
/// When ChatResponseUpdate has empty string MessageId, the AGUI layer passes
/// through the raw provider value for ToolCallStartEvent.ParentMessageId.
/// Tool-call chunks should NOT receive the text-event fallback GUID — that
/// would collapse parallel tool calls into one assistant message in the FE.
/// </summary>
[Fact]
public async Task ToolCalls_EmptyMessageId_GeneratesFallbackParentMessageIdAsync()
public async Task ToolCalls_EmptyMessageId_DoesNotGenerateFallbackParentMessageIdAsync()
{
// Arrange - ChatResponseUpdate with a tool call but empty MessageId
FunctionCallContent functionCall = new("call_abc123", "GetWeather")
@@ -139,14 +141,14 @@ public sealed class AGUIStreamingMessageIdTests
aguiEvents.Add(evt);
}
// Assert — ParentMessageId should have a generated fallback
// Assert — ParentMessageId should be empty (raw provider value, no synthetic fallback)
ToolCallStartEvent? toolCallStart = aguiEvents.OfType<ToolCallStartEvent>().FirstOrDefault();
Assert.NotNull(toolCallStart);
Assert.Equal("call_abc123", toolCallStart.ToolCallId);
Assert.Equal("GetWeather", toolCallStart.ToolCallName);
Assert.False(
Assert.True(
string.IsNullOrEmpty(toolCallStart.ParentMessageId),
"ParentMessageId should have a generated fallback for empty provider MessageId");
"ParentMessageId should be empty when provider omits MessageId (raw pass-through)");
}
/// <summary>
@@ -183,10 +185,13 @@ public sealed class AGUIStreamingMessageIdTests
ToolCallStartEvent toolCallStart = Assert.Single(aguiEvents.OfType<ToolCallStartEvent>());
ToolCallResultEvent toolCallResult = Assert.Single(aguiEvents.OfType<ToolCallResultEvent>());
Assert.Equal(textStart.MessageId, toolCallStart.ParentMessageId);
// Tool-call ParentMessageId should NOT leak the text fallback GUID
Assert.NotEqual(textStart.MessageId, toolCallStart.ParentMessageId);
Assert.Equal("call_abc123", toolCallResult.ToolCallId);
Assert.False(string.IsNullOrEmpty(toolCallResult.MessageId));
Assert.NotEqual(textStart.MessageId, toolCallResult.MessageId);
// Result MessageId should be deterministic based on CallId
Assert.Equal("result-call_abc123", toolCallResult.MessageId);
}
[Fact]
@@ -230,10 +235,11 @@ public sealed class AGUIStreamingMessageIdTests
ToolCallStartEvent toolCallStart = Assert.Single(aguiEvents.OfType<ToolCallStartEvent>());
ToolCallResultEvent toolCallResult = Assert.Single(aguiEvents.OfType<ToolCallResultEvent>());
Assert.Equal(textStarts[0].MessageId, toolCallStart.ParentMessageId);
// Tool-call ParentMessageId should NOT leak the text fallback GUID
Assert.NotEqual(textStarts[0].MessageId, toolCallStart.ParentMessageId);
Assert.NotEqual(textStarts[0].MessageId, toolCallResult.MessageId);
Assert.Equal(toolCallResult.MessageId, toolText.MessageId);
Assert.Equal(textStarts[^1].MessageId, toolCallResult.MessageId);
// Result MessageId should be deterministic based on CallId
Assert.Equal("result-call_abc123", toolCallResult.MessageId);
}
/// <summary>
@@ -274,6 +280,86 @@ public sealed class AGUIStreamingMessageIdTests
Assert.Equal(2, contentEvents.Count);
Assert.All(contentEvents, e => Assert.Equal("chatcmpl-abc123", e.MessageId));
}
/// <summary>
/// Bug #1 reproduction: parallel tool calls with empty MessageId should NOT all
/// share the same synthetic ParentMessageId. Each should pass through the raw
/// provider value (empty), allowing the FE to render them as distinct cards.
/// </summary>
[Fact]
public async Task ParallelToolCalls_EmptyMessageId_DoNotShareParentMessageIdAsync()
{
// Arrange — 3 parallel tool calls with empty MessageId (real OpenAI behavior)
List<ChatResponseUpdate> providerUpdates =
[
new ChatResponseUpdate(ChatRole.Assistant, "Let me run those queries.") { MessageId = "chatcmpl-real" },
new ChatResponseUpdate { Role = ChatRole.Assistant, MessageId = "", Contents = [new FunctionCallContent("call_A", "query") { Arguments = new Dictionary<string, object?> { ["q"] = "1" } }] },
new ChatResponseUpdate { Role = ChatRole.Assistant, MessageId = "", Contents = [new FunctionCallContent("call_B", "query") { Arguments = new Dictionary<string, object?> { ["q"] = "2" } }] },
new ChatResponseUpdate { Role = ChatRole.Assistant, MessageId = "", Contents = [new FunctionCallContent("call_C", "query") { Arguments = new Dictionary<string, object?> { ["q"] = "3" } }] },
];
// Act
List<BaseEvent> aguiEvents = [];
await foreach (BaseEvent evt in providerUpdates.ToAsyncEnumerableAsync()
.AsAGUIEventStreamAsync("thread-1", "run-1", AGUIJsonSerializerContext.Default.Options))
{
aguiEvents.Add(evt);
}
// Assert — all 3 tool calls should have empty ParentMessageId (raw provider value),
// NOT the text fallback GUID
List<ToolCallStartEvent> toolCallStarts = aguiEvents.OfType<ToolCallStartEvent>().ToList();
Assert.Equal(3, toolCallStarts.Count);
Assert.All(toolCallStarts, tc => Assert.True(string.IsNullOrEmpty(tc.ParentMessageId)));
// Text events should still have a valid fallback MessageId
TextMessageStartEvent textStart = Assert.Single(aguiEvents.OfType<TextMessageStartEvent>());
Assert.False(string.IsNullOrEmpty(textStart.MessageId));
}
/// <summary>
/// Bug #2 reproduction: tool results batched into one ChatResponseUpdate with a
/// shared MEAI MessageId should each get a unique deterministic MessageId.
/// </summary>
[Fact]
public async Task ToolCallResults_SharedMeaiMessageId_HaveUniqueMessageIdsPerCallAsync()
{
// Arrange — MEAI batches all FunctionResultContent into one update with shared id
List<ChatResponseUpdate> providerUpdates =
[
new ChatResponseUpdate
{
Role = ChatRole.Tool,
MessageId = "meai-shared-id",
Contents =
[
new FunctionResultContent("call_A", "result1"),
new FunctionResultContent("call_B", "result2"),
new FunctionResultContent("call_C", "result3"),
]
},
];
// Act
List<BaseEvent> aguiEvents = [];
await foreach (BaseEvent evt in providerUpdates.ToAsyncEnumerableAsync()
.AsAGUIEventStreamAsync("thread-1", "run-1", AGUIJsonSerializerContext.Default.Options))
{
aguiEvents.Add(evt);
}
// Assert — each result should have a unique MessageId
List<ToolCallResultEvent> toolResults = aguiEvents.OfType<ToolCallResultEvent>().ToList();
Assert.Equal(3, toolResults.Count);
string?[] distinctIds = toolResults.Select(r => r.MessageId).Distinct().ToArray();
Assert.Equal(3, distinctIds.Length);
// Verify deterministic format
Assert.Equal("result-call_A", toolResults[0].MessageId);
Assert.Equal("result-call_B", toolResults[1].MessageId);
Assert.Equal("result-call_C", toolResults[2].MessageId);
}
}
/// <summary>
@@ -18,7 +18,7 @@ namespace Microsoft.Agents.AI.UnitTests.AgentSkills;
public sealed class AgentClassSkillTests
{
[Fact]
public void MinimalClassSkill_HasNullOverrides_AndSynthesizesContent()
public async Task MinimalClassSkill_HasNullOverrides_AndSynthesizesContentAsync()
{
// Arrange
var skill = new MinimalClassSkill();
@@ -26,18 +26,17 @@ public sealed class AgentClassSkillTests
// Act & Assert — null overrides
Assert.Equal("minimal", skill.Frontmatter.Name);
Assert.Null(skill.Resources);
Assert.Null(skill.Scripts);
// Act & Assert — synthesized XML content
Assert.Contains("<name>minimal</name>", skill.Content);
Assert.Contains("<description>A minimal skill.</description>", skill.Content);
Assert.Contains("<instructions>", skill.Content);
Assert.Contains("Minimal skill body.", skill.Content);
Assert.Contains("</instructions>", skill.Content);
Assert.Contains("<name>minimal</name>", await skill.GetContentAsync());
Assert.Contains("<description>A minimal skill.</description>", await skill.GetContentAsync());
Assert.Contains("<instructions>", await skill.GetContentAsync());
Assert.Contains("Minimal skill body.", await skill.GetContentAsync());
Assert.Contains("</instructions>", await skill.GetContentAsync());
}
[Fact]
public void FullClassSkill_ReturnsOverriddenLists_AndCachesContent()
public async Task FullClassSkill_ReturnsOverriddenLists_AndCachesContentAsync()
{
// Arrange
var skill = new FullClassSkill();
@@ -50,11 +49,11 @@ public sealed class AgentClassSkillTests
Assert.Equal("TestScript", skill.Scripts![0].Name);
// Act & Assert — Content is cached
Assert.Same(skill.Content, skill.Content);
Assert.Same(await skill.GetContentAsync(), await skill.GetContentAsync());
// Act & Assert — Content includes parameter schema from typed script
Assert.Contains("parameters_schema", skill.Content);
Assert.Contains("value", skill.Content);
Assert.Contains("parameters_schema", await skill.GetContentAsync());
Assert.Contains("value", await skill.GetContentAsync());
}
[Fact]
@@ -117,6 +116,116 @@ public sealed class AgentClassSkillTests
Assert.Single(scriptOnly.Scripts!);
}
[Fact]
public async Task GetResourceAsync_ExistingName_ReturnsResourceAsync()
{
// Arrange
var skill = new FullClassSkill();
// Act
var resource = await skill.GetResourceAsync("test-resource");
// Assert
Assert.NotNull(resource);
Assert.Equal("test-resource", resource!.Name);
}
[Fact]
public async Task GetResourceAsync_NonExistingName_ReturnsNullAsync()
{
// Arrange
var skill = new FullClassSkill();
// Act
var resource = await skill.GetResourceAsync("missing");
// Assert
Assert.Null(resource);
}
[Fact]
public async Task GetResourceAsync_NoResources_ReturnsNullAsync()
{
// Arrange
var skill = new MinimalClassSkill();
// Act
var resource = await skill.GetResourceAsync("anything");
// Assert
Assert.Null(resource);
}
[Fact]
public async Task GetScriptAsync_ExistingName_ReturnsScriptAsync()
{
// Arrange
var skill = new FullClassSkill();
// Act
var script = await skill.GetScriptAsync("TestScript");
// Assert
Assert.NotNull(script);
Assert.Equal("TestScript", script!.Name);
}
[Fact]
public async Task GetScriptAsync_NonExistingName_ReturnsNullAsync()
{
// Arrange
var skill = new FullClassSkill();
// Act
var script = await skill.GetScriptAsync("missing");
// Assert
Assert.Null(script);
}
[Fact]
public async Task GetScriptAsync_NoScripts_ReturnsNullAsync()
{
// Arrange
var skill = new MinimalClassSkill();
// Act
var script = await skill.GetScriptAsync("anything");
// Assert
Assert.Null(script);
}
[Fact]
public async Task ConcurrentAccess_ToReflectedResourcesScriptsAndContent_InvokesDiscoveryOnceAsync()
{
// Regression test for thread-safety of Lazy<T> initialization in AgentClassSkill<TSelf>.
// AttributedFullSkill uses attribute-based discovery (no override), so it exercises
// the base class's Lazy<T> fields rather than a subclass's own caching.
var skill = new AttributedFullSkill();
const int Concurrency = 32;
var resourcesResults = new IReadOnlyList<AgentSkillResource>?[Concurrency];
var scriptsResults = new IReadOnlyList<AgentSkillScript>?[Concurrency];
var contentResults = new string[Concurrency];
// Act — invoke all three accessors concurrently from many threads.
await Task.WhenAll(Enumerable.Range(0, Concurrency).Select(i => Task.Run(async () =>
{
resourcesResults[i] = skill.Resources;
scriptsResults[i] = skill.Scripts;
contentResults[i] = await skill.GetContentAsync();
})));
// Assert — every thread observed the same cached instances (no torn state).
for (int i = 1; i < Concurrency; i++)
{
Assert.Same(resourcesResults[0], resourcesResults[i]);
Assert.Same(scriptsResults[0], scriptsResults[i]);
Assert.Same(contentResults[0], contentResults[i]);
}
}
[Fact]
public async Task CreateScriptAndResource_WithSerializerOptions_HandleCustomTypesAsync()
{
@@ -151,17 +260,14 @@ public sealed class AgentClassSkillTests
// Arrange
var skill = new AttributedScriptsSkill();
// Act
var scripts = skill.Scripts;
// Act & Assert — all scripts discovered with correct metadata
Assert.NotNull(skill.Scripts);
Assert.Equal(4, skill.Scripts!.Count);
Assert.Contains(skill.Scripts, s => s.Name == "do-work");
Assert.Contains(skill.Scripts, s => s.Name == "DefaultNamed");
Assert.Contains(skill.Scripts, s => s.Name == "append");
// Assert — all scripts discovered with correct metadata
Assert.NotNull(scripts);
Assert.Equal(4, scripts!.Count);
Assert.Contains(scripts, s => s.Name == "do-work");
Assert.Contains(scripts, s => s.Name == "DefaultNamed");
Assert.Contains(scripts, s => s.Name == "append");
var processScript = scripts.First(s => s.Name == "process");
var processScript = skill.Scripts.First(s => s.Name == "process");
Assert.Equal("Processes the input.", processScript.Description);
}
@@ -272,16 +378,16 @@ public sealed class AgentClassSkillTests
}
[Fact]
public void AttributedFullSkill_IncludesContentWithSchema_AndCachesMembers()
public async Task AttributedFullSkill_IncludesContentWithSchema_AndCachesMembersAsync()
{
// Arrange
var skill = new AttributedFullSkill();
// Act & Assert — Content includes reflected resources and scripts
Assert.Contains("<resources>", skill.Content);
Assert.Contains("conversion-table", skill.Content);
Assert.Contains("<scripts>", skill.Content);
Assert.Contains("convert", skill.Content);
Assert.Contains("<resources>", await skill.GetContentAsync());
Assert.Contains("conversion-table", await skill.GetContentAsync());
Assert.Contains("<scripts>", await skill.GetContentAsync());
Assert.Contains("convert", await skill.GetContentAsync());
// Act & Assert — discovered members are cached
Assert.Same(skill.Resources, skill.Resources);
@@ -299,33 +405,33 @@ public sealed class AgentClassSkillTests
// Arrange — skill with no attributes and no overrides; base discovery returns null (not empty list)
var skill = new NoAttributesNoOverridesSkill();
var baseType = typeof(AgentClassSkill<NoAttributesNoOverridesSkill>);
var resourcesDiscoveredField = baseType.GetField("_resourcesDiscovered", BindingFlags.Instance | BindingFlags.NonPublic);
var scriptsDiscoveredField = baseType.GetField("_scriptsDiscovered", BindingFlags.Instance | BindingFlags.NonPublic);
var reflectedResourcesField = baseType.GetField("_reflectedResources", BindingFlags.Instance | BindingFlags.NonPublic);
var reflectedScriptsField = baseType.GetField("_reflectedScripts", BindingFlags.Instance | BindingFlags.NonPublic);
var resourcesField = baseType.GetField("_resources", BindingFlags.Instance | BindingFlags.NonPublic);
var scriptsField = baseType.GetField("_scripts", BindingFlags.Instance | BindingFlags.NonPublic);
Assert.NotNull(resourcesDiscoveredField);
Assert.NotNull(scriptsDiscoveredField);
Assert.NotNull(reflectedResourcesField);
Assert.NotNull(reflectedScriptsField);
Assert.False((bool)resourcesDiscoveredField!.GetValue(skill)!);
Assert.False((bool)scriptsDiscoveredField!.GetValue(skill)!);
Assert.NotNull(resourcesField);
Assert.NotNull(scriptsField);
var resourcesLazy = (Lazy<IReadOnlyList<AgentSkillResource>?>)resourcesField!.GetValue(skill)!;
var scriptsLazy = (Lazy<IReadOnlyList<AgentSkillScript>?>)scriptsField!.GetValue(skill)!;
Assert.False(resourcesLazy.IsValueCreated);
Assert.False(scriptsLazy.IsValueCreated);
// Act & Assert
Assert.Null(skill.Resources);
Assert.Null(skill.Scripts);
Assert.True((bool)resourcesDiscoveredField.GetValue(skill)!);
Assert.True((bool)scriptsDiscoveredField.GetValue(skill)!);
Assert.Null(reflectedResourcesField!.GetValue(skill));
Assert.Null(reflectedScriptsField!.GetValue(skill));
Assert.True(resourcesLazy.IsValueCreated);
Assert.True(scriptsLazy.IsValueCreated);
Assert.Null(resourcesLazy.Value);
Assert.Null(scriptsLazy.Value);
// Repeated access should not re-trigger discovery even when discovered value is null.
Assert.Null(skill.Resources);
Assert.Null(skill.Scripts);
Assert.True((bool)resourcesDiscoveredField.GetValue(skill)!);
Assert.True((bool)scriptsDiscoveredField.GetValue(skill)!);
Assert.Null(reflectedResourcesField.GetValue(skill));
Assert.Null(reflectedScriptsField.GetValue(skill));
Assert.True(resourcesLazy.IsValueCreated);
Assert.True(scriptsLazy.IsValueCreated);
Assert.Null(resourcesLazy.Value);
Assert.Null(scriptsLazy.Value);
}
[Fact]
@@ -382,7 +488,7 @@ public sealed class AgentClassSkillTests
var jso = SkillTestJsonContext.Default.Options;
// Act & Assert — script with custom JSO
var script = skill.Scripts![0];
var script = skill.Scripts!.First(s => s.Name == "lookup");
var inputJson = JsonSerializer.SerializeToElement(new LookupRequest { Query = "test", MaxResults = 3 }, jso);
using var argsDoc = JsonDocument.Parse($$"""{ "request": {{inputJson.GetRawText()}} }""");
var args = argsDoc.RootElement;
@@ -398,13 +504,13 @@ public sealed class AgentClassSkillTests
}
[Fact]
public void Content_IncludesDescription_ForReflectedResources()
public async Task Content_IncludesDescription_ForReflectedResourcesAsync()
{
// Arrange
var skill = new AttributedResourcePropertiesSkill();
// Act
var content = skill.Content;
var content = await skill.GetContentAsync();
// Assert — descriptions from [Description] attribute appear in synthesized content
Assert.Contains("Some important data.", content);
@@ -105,7 +105,7 @@ public sealed class AgentFileSkillScriptTests
}
[Fact]
public void Content_WithScripts_AppendsPerScriptEntries()
public async Task Content_WithScripts_AppendsPerScriptEntriesAsync()
{
// Arrange
static Task<object?> RunnerAsync(AgentFileSkill s, AgentFileSkillScript sc, JsonElement? a, IServiceProvider? sp, CancellationToken ct) => Task.FromResult<object?>(null);
@@ -118,7 +118,7 @@ public sealed class AgentFileSkillScriptTests
scripts: [script1, script2]);
// Act
var content = fileSkill.Content;
var content = await fileSkill.GetContentAsync();
// Assert — content starts with original and appends per-script entries
Assert.StartsWith("Original content", content);
@@ -130,7 +130,7 @@ public sealed class AgentFileSkillScriptTests
}
[Fact]
public void Content_WithoutScripts_ReturnsOriginalContent()
public async Task Content_WithoutScripts_ReturnsOriginalContentAsync()
{
// Arrange
var fileSkill = new AgentFileSkill(
@@ -139,14 +139,14 @@ public sealed class AgentFileSkillScriptTests
"/skills/my-skill");
// Act
var content = fileSkill.Content;
var content = await fileSkill.GetContentAsync();
// Assert
Assert.Equal("Original content only", content);
}
[Fact]
public void Content_WithScripts_IsCached()
public async Task Content_WithScripts_IsCachedAsync()
{
// Arrange
static Task<object?> RunnerAsync(AgentFileSkill s, AgentFileSkillScript sc, JsonElement? a, IServiceProvider? sp, CancellationToken ct) => Task.FromResult<object?>(null);
@@ -158,8 +158,8 @@ public sealed class AgentFileSkillScriptTests
scripts: [script]);
// Act
var content1 = fileSkill.Content;
var content2 = fileSkill.Content;
var content1 = await fileSkill.GetContentAsync();
var content2 = await fileSkill.GetContentAsync();
// Assert
Assert.Same(content1, content2);
@@ -232,7 +232,7 @@ public sealed class AgentFileSkillScriptTests
}
[Fact]
public void Content_WithScripts_ContainsDefaultParametersSchema()
public async Task Content_WithScripts_ContainsDefaultParametersSchemaAsync()
{
// Arrange
static Task<object?> RunnerAsync(AgentFileSkill s, AgentFileSkillScript sc, JsonElement? a, IServiceProvider? sp, CancellationToken ct) => Task.FromResult<object?>(null);
@@ -244,7 +244,7 @@ public sealed class AgentFileSkillScriptTests
scripts: [script]);
// Act
var content = fileSkill.Content;
var content = await fileSkill.GetContentAsync();
// Assert — the appended block contains the actual default schema from AgentFileSkillScript
Assert.Contains("""{"type":"array","items":{"type":"string"}}""", content);
@@ -2,7 +2,6 @@
using System;
using System.IO;
using System.Linq;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
@@ -46,9 +45,9 @@ public sealed class AgentFileSkillsSourceScriptTests : IDisposable
// Assert
Assert.Single(skills);
var skill = skills[0];
Assert.NotNull(skill.Scripts);
Assert.Single(skill.Scripts!);
Assert.Equal("scripts/convert.py", skill.Scripts![0].Name);
var script = await skill.GetScriptAsync("scripts/convert.py");
Assert.NotNull(script);
Assert.Equal("scripts/convert.py", script!.Name);
}
[Fact]
@@ -69,14 +68,13 @@ public sealed class AgentFileSkillsSourceScriptTests : IDisposable
// Assert
Assert.Single(skills);
var scriptNames = skills[0].Scripts!.Select(s => s.Name).OrderBy(n => n, StringComparer.Ordinal).ToList();
Assert.Equal(6, scriptNames.Count);
Assert.Contains("scripts/run.cs", scriptNames);
Assert.Contains("scripts/run.csx", scriptNames);
Assert.Contains("scripts/run.js", scriptNames);
Assert.Contains("scripts/run.ps1", scriptNames);
Assert.Contains("scripts/run.py", scriptNames);
Assert.Contains("scripts/run.sh", scriptNames);
// Assert — verify all expected scripts are discoverable
foreach (var name in (string[])["scripts/run.cs", "scripts/run.csx", "scripts/run.js", "scripts/run.ps1", "scripts/run.py", "scripts/run.sh"])
{
var script = await skills[0].GetScriptAsync(name);
Assert.NotNull(script);
Assert.Equal(name, script!.Name);
}
}
[Fact]
@@ -94,7 +92,7 @@ public sealed class AgentFileSkillsSourceScriptTests : IDisposable
// Assert
Assert.Single(skills);
Assert.Empty(skills[0].Scripts!);
Assert.Null(await skills[0].GetScriptAsync("scripts/data.txt"));
}
[Fact]
@@ -109,8 +107,7 @@ public sealed class AgentFileSkillsSourceScriptTests : IDisposable
// Assert
Assert.Single(skills);
Assert.NotNull(skills[0].Scripts);
Assert.Empty(skills[0].Scripts!);
Assert.Null(await skills[0].GetScriptAsync("any-script"));
}
[Fact]
@@ -128,7 +125,7 @@ public sealed class AgentFileSkillsSourceScriptTests : IDisposable
// Assert — neither file is in the default scripts/ directory, so no scripts are discovered
Assert.Single(skills);
Assert.Empty(skills[0].Scripts!);
Assert.Null(await skills[0].GetScriptAsync("convert.py"));
}
[Fact]
@@ -150,7 +147,7 @@ public sealed class AgentFileSkillsSourceScriptTests : IDisposable
// Act
var skills = await source.GetSkillsAsync(CancellationToken.None);
var scriptResult = await skills[0].Scripts![0].RunAsync(skills[0], null, null, CancellationToken.None);
var scriptResult = await (await skills[0].GetScriptAsync("scripts/test.py"))!.RunAsync(skills[0], null, null, CancellationToken.None);
// Assert
Assert.True(executorCalled);
@@ -175,7 +172,7 @@ public sealed class AgentFileSkillsSourceScriptTests : IDisposable
// Act — discovery succeeds even without a runner
var skills = await source.GetSkillsAsync(CancellationToken.None);
var script = skills[0].Scripts![0];
var script = (await skills[0].GetScriptAsync("scripts/run.sh"))!;
// Assert — running the script throws because no runner was provided
await Assert.ThrowsAsync<InvalidOperationException>(() => script.RunAsync(skills[0], null, null, CancellationToken.None));
@@ -195,8 +192,9 @@ public sealed class AgentFileSkillsSourceScriptTests : IDisposable
// Assert
Assert.Single(skills);
Assert.Single(skills[0].Scripts!);
Assert.Equal("scripts/run.rb", skills[0].Scripts![0].Name);
var rbScript = await skills[0].GetScriptAsync("scripts/run.rb");
Assert.NotNull(rbScript);
Assert.Equal("scripts/run.rb", rbScript!.Name);
}
[Fact]
@@ -217,7 +215,7 @@ public sealed class AgentFileSkillsSourceScriptTests : IDisposable
var skills = await source.GetSkillsAsync(CancellationToken.None);
using var argumentsDoc = JsonDocument.Parse("""{"value":26.2,"factor":1.60934}""");
var arguments = argumentsDoc.RootElement;
await skills[0].Scripts![0].RunAsync(skills[0], arguments, null, CancellationToken.None);
await (await skills[0].GetScriptAsync("scripts/test.py"))!.RunAsync(skills[0], arguments, null, CancellationToken.None);
// Assert
Assert.NotNull(capturedArgs);
@@ -240,8 +238,9 @@ public sealed class AgentFileSkillsSourceScriptTests : IDisposable
// Assert — script file inside the deeply nested directory is discovered
Assert.Single(skills);
Assert.Single(skills[0].Scripts!);
Assert.Equal("f1/f2/f3/run.py", skills[0].Scripts![0].Name);
var nestedScript = await skills[0].GetScriptAsync("f1/f2/f3/run.py");
Assert.NotNull(nestedScript);
Assert.Equal("f1/f2/f3/run.py", nestedScript!.Name);
}
[Theory]
@@ -267,11 +266,12 @@ public sealed class AgentFileSkillsSourceScriptTests : IDisposable
// Assert — scripts are discovered with names identical to using directories without "./"
Assert.Single(skills);
Assert.Equal(directories.Length, skills[0].Scripts!.Count);
foreach (string directory in directories)
{
string expectedName = $"{directory.Substring(2)}/run.py";
Assert.Contains(skills[0].Scripts!, s => s.Name == expectedName);
var script = await skills[0].GetScriptAsync(expectedName);
Assert.NotNull(script);
Assert.Equal(expectedName, script!.Name);
}
}
@@ -105,13 +105,13 @@ public sealed class AgentInlineSkillTests
}
[Fact]
public void Content_ContainsNameDescriptionAndInstructions()
public async Task Content_ContainsNameDescriptionAndInstructionsAsync()
{
// Arrange
var skill = new AgentInlineSkill("my-skill", "A valid skill.", "Do the thing.");
// Act
var content = skill.Content;
var content = await skill.GetContentAsync();
// Assert
Assert.Contains("<name>my-skill</name>", content);
@@ -120,13 +120,13 @@ public sealed class AgentInlineSkillTests
}
[Fact]
public void Content_EscapesXmlCharacters()
public async Task Content_EscapesXmlCharactersAsync()
{
// Arrange
var skill = new AgentInlineSkill("my-skill", "x<y>z\"w & it's more", "1 & 2 < 3");
// Act
var content = skill.Content;
var content = await skill.GetContentAsync();
// Assert
Assert.Contains("<name>my-skill</name>", content);
@@ -135,28 +135,28 @@ public sealed class AgentInlineSkillTests
}
[Fact]
public void Content_IsCachedAcrossAccesses()
public async Task Content_IsCachedAcrossAccessesAsync()
{
// Arrange
var skill = new AgentInlineSkill("my-skill", "A valid skill.", "Instructions.");
// Act
var first = skill.Content;
var second = skill.Content;
var first = await skill.GetContentAsync();
var second = await skill.GetContentAsync();
// Assert
Assert.Same(first, second);
}
[Fact]
public void Content_IncludesResourcesAddedBeforeFirstAccess()
public async Task Content_IncludesResourcesAddedBeforeFirstAccessAsync()
{
// Arrange
var skill = new AgentInlineSkill("my-skill", "A valid skill.", "Instructions.");
skill.AddResource("config", "value1", "A config resource.");
// Act
var content = skill.Content;
var content = await skill.GetContentAsync();
// Assert
Assert.Contains("<resources>", content);
@@ -164,14 +164,14 @@ public sealed class AgentInlineSkillTests
}
[Fact]
public void Content_IncludesDelegateResourcesAddedBeforeFirstAccess()
public async Task Content_IncludesDelegateResourcesAddedBeforeFirstAccessAsync()
{
// Arrange
var skill = new AgentInlineSkill("my-skill", "A valid skill.", "Instructions.");
skill.AddResource("dynamic", () => "hello");
// Act
var content = skill.Content;
var content = await skill.GetContentAsync();
// Assert
Assert.Contains("<resources>", content);
@@ -179,14 +179,14 @@ public sealed class AgentInlineSkillTests
}
[Fact]
public void Content_IncludesScriptsAddedBeforeFirstAccess()
public async Task Content_IncludesScriptsAddedBeforeFirstAccessAsync()
{
// Arrange
var skill = new AgentInlineSkill("my-skill", "A valid skill.", "Instructions.");
skill.AddScript("run", () => "result", "Runs something.");
// Act
var content = skill.Content;
var content = await skill.GetContentAsync();
// Assert
Assert.Contains("<scripts>", content);
@@ -194,22 +194,22 @@ public sealed class AgentInlineSkillTests
}
[Fact]
public void Content_IsCachedAndNotRebuilt()
public async Task Content_IsCachedAndNotRebuiltAsync()
{
// Arrange
var skill = new AgentInlineSkill("my-skill", "A valid skill.", "Instructions.");
skill.AddResource("r1", "v1");
// Act
var first = skill.Content;
var second = skill.Content;
var first = await skill.GetContentAsync();
var second = await skill.GetContentAsync();
// Assert
Assert.Same(first, second);
}
[Fact]
public void Content_IncludesResourcesAndScriptsAddedBeforeFirstAccess()
public async Task Content_IncludesResourcesAndScriptsAddedBeforeFirstAccessAsync()
{
// Arrange
var skill = new AgentInlineSkill("my-skill", "A valid skill.", "Instructions.");
@@ -217,7 +217,7 @@ public sealed class AgentInlineSkillTests
skill.AddScript("s1", () => "ok");
// Act
var content = skill.Content;
var content = await skill.GetContentAsync();
// Assert
Assert.Contains("<resources>", content);
@@ -227,14 +227,14 @@ public sealed class AgentInlineSkillTests
}
[Fact]
public void Content_ParametersSchema_IsXmlEscaped()
public async Task Content_ParametersSchema_IsXmlEscapedAsync()
{
// Arrange
var skill = new AgentInlineSkill("my-skill", "A valid skill.", "Instructions.");
skill.AddScript("search", (string query, int limit) => $"found {limit} results for {query}");
// Act
var content = skill.Content;
var content = await skill.GetContentAsync();
// Assert — JSON schema should be present and XML content chars escaped
Assert.Contains("parameters_schema", content);
@@ -280,17 +280,103 @@ public sealed class AgentInlineSkillTests
var skill = new AgentInlineSkill("my-skill", "A valid skill.", "Instructions.");
// Act & Assert
Assert.Null(skill.Resources);
Assert.Null(skill.GetTestResources());
}
[Fact]
public void Scripts_WhenNoneAdded_ReturnsNull()
public async Task Scripts_WhenNoneAdded_ReturnsNullAsync()
{
// Arrange
var skill = new AgentInlineSkill("my-skill", "A valid skill.", "Instructions.");
// Act & Assert
Assert.Null(skill.Scripts);
Assert.Null(await skill.GetScriptAsync("nonexistent"));
}
[Fact]
public async Task GetResourceAsync_ExistingName_ReturnsResourceAsync()
{
// Arrange
var skill = new AgentInlineSkill("my-skill", "A valid skill.", "Instructions.");
skill.AddResource("r1", "v1");
skill.AddResource("r2", "v2");
// Act
var resource = await skill.GetResourceAsync("r2");
// Assert
Assert.NotNull(resource);
Assert.Equal("r2", resource!.Name);
}
[Fact]
public async Task GetResourceAsync_NonExistingName_ReturnsNullAsync()
{
// Arrange
var skill = new AgentInlineSkill("my-skill", "A valid skill.", "Instructions.");
skill.AddResource("r1", "v1");
// Act
var resource = await skill.GetResourceAsync("missing");
// Assert
Assert.Null(resource);
}
[Fact]
public async Task GetResourceAsync_NoResourcesAdded_ReturnsNullAsync()
{
// Arrange
var skill = new AgentInlineSkill("my-skill", "A valid skill.", "Instructions.");
// Act
var resource = await skill.GetResourceAsync("missing");
// Assert
Assert.Null(resource);
}
[Fact]
public async Task GetScriptAsync_ExistingName_ReturnsScriptAsync()
{
// Arrange
var skill = new AgentInlineSkill("my-skill", "A valid skill.", "Instructions.");
skill.AddScript("s1", () => "first");
skill.AddScript("s2", () => "second");
// Act
var script = await skill.GetScriptAsync("s2");
// Assert
Assert.NotNull(script);
Assert.Equal("s2", script!.Name);
}
[Fact]
public async Task GetScriptAsync_NonExistingName_ReturnsNullAsync()
{
// Arrange
var skill = new AgentInlineSkill("my-skill", "A valid skill.", "Instructions.");
skill.AddScript("s1", () => "ok");
// Act
var script = await skill.GetScriptAsync("missing");
// Assert
Assert.Null(script);
}
[Fact]
public async Task GetScriptAsync_NoScriptsAdded_ReturnsNullAsync()
{
// Arrange
var skill = new AgentInlineSkill("my-skill", "A valid skill.", "Instructions.");
// Act
var script = await skill.GetScriptAsync("missing");
// Assert
Assert.Null(script);
}
[Fact]
@@ -333,13 +419,13 @@ public sealed class AgentInlineSkillTests
}
[Fact]
public void Content_NoResourcesOrScripts_DoesNotContainResourcesOrScriptsTags()
public async Task Content_NoResourcesOrScripts_DoesNotContainResourcesOrScriptsTagsAsync()
{
// Arrange
var skill = new AgentInlineSkill("my-skill", "A valid skill.", "Instructions.");
// Act
var content = skill.Content;
var content = await skill.GetContentAsync();
// Assert
Assert.DoesNotContain("<resources>", content);
@@ -347,58 +433,58 @@ public sealed class AgentInlineSkillTests
}
[Fact]
public void Content_ResourcesAddedAfterCaching_AreNotIncluded()
public async Task Content_ResourcesAddedAfterCaching_AreNotIncludedAsync()
{
// Arrange
var skill = new AgentInlineSkill("my-skill", "A valid skill.", "Instructions.");
_ = skill.Content; // trigger caching
_ = await skill.GetContentAsync(); // trigger caching
skill.AddResource("late-resource", "late-value");
// Act
var content = skill.Content;
var content = await skill.GetContentAsync();
// Assert — the late resource should not appear because content was cached
Assert.DoesNotContain("late-resource", content);
}
[Fact]
public void Content_ScriptsAddedAfterCaching_AreNotIncluded()
public async Task Content_ScriptsAddedAfterCaching_AreNotIncludedAsync()
{
// Arrange
var skill = new AgentInlineSkill("my-skill", "A valid skill.", "Instructions.");
_ = skill.Content; // trigger caching
_ = await skill.GetContentAsync(); // trigger caching
skill.AddScript("late-script", () => "late");
// Act
var content = skill.Content;
var content = await skill.GetContentAsync();
// Assert — the late script should not appear because content was cached
Assert.DoesNotContain("late-script", content);
}
[Fact]
public void Content_ScriptWithDescription_IncludesDescriptionAttribute()
public async Task Content_ScriptWithDescription_IncludesDescriptionAttributeAsync()
{
// Arrange
var skill = new AgentInlineSkill("my-skill", "A valid skill.", "Instructions.");
skill.AddScript("my-script", () => "ok", "Runs something.");
// Act
var content = skill.Content;
var content = await skill.GetContentAsync();
// Assert
Assert.Contains("description=\"Runs something.\"", content);
}
[Fact]
public void Content_ScriptWithoutParametersOrDescription_UsesSelfClosingTag()
public async Task Content_ScriptWithoutParametersOrDescription_UsesSelfClosingTagAsync()
{
// Arrange
var skill = new AgentInlineSkill("my-skill", "A valid skill.", "Instructions.");
skill.AddScript("simple", () => "ok");
// Act
var content = skill.Content;
var content = await skill.GetContentAsync();
// Assert — parameterless Action delegates still produce a schema, so this
// verifies the script is at least included in the output
@@ -406,7 +492,7 @@ public sealed class AgentInlineSkillTests
}
[Fact]
public void Content_ResourceWithDescription_IncludesDescriptionAttribute()
public async Task Content_ResourceWithDescription_IncludesDescriptionAttributeAsync()
{
// Arrange
var skill = new AgentInlineSkill("my-skill", "A valid skill.", "Instructions.");
@@ -414,7 +500,7 @@ public sealed class AgentInlineSkillTests
skill.AddResource("no-desc", "value");
// Act
var content = skill.Content;
var content = await skill.GetContentAsync();
// Assert
Assert.Contains("description=\"A described resource.\"", content);
@@ -437,7 +523,7 @@ public sealed class AgentInlineSkillTests
var args = argsDoc.RootElement;
// Act
var result = await skill.Scripts![0].RunAsync(skill, args, null, CancellationToken.None);
var result = await (await skill.GetScriptAsync("lookup"))!.RunAsync(skill, args, null, CancellationToken.None);
// Assert — the custom input was deserialized via skill-level JSO and response was produced
Assert.NotNull(result);
@@ -461,7 +547,7 @@ public sealed class AgentInlineSkillTests
var args = argsDoc.RootElement;
// Act
var result = await skill.Scripts![0].RunAsync(skill, args, null, CancellationToken.None);
var result = await (await skill.GetScriptAsync("lookup"))!.RunAsync(skill, args, null, CancellationToken.None);
// Assert — per-script JSO takes effect and custom types are properly marshaled
Assert.NotNull(result);
@@ -477,7 +563,7 @@ public sealed class AgentInlineSkillTests
skill.AddResource("config", () => new SkillConfig { Theme = "dark", Verbose = true });
// Act
var result = await skill.Resources![0].ReadAsync();
var result = await skill.GetTestResources()![0].ReadAsync();
// Assert — the custom type was returned successfully via skill-level JSO
Assert.NotNull(result);
@@ -494,7 +580,7 @@ public sealed class AgentInlineSkillTests
skill.AddResource("config", () => new SkillConfig { Theme = "dark", Verbose = true }, serializerOptions: resourceJso);
// Act
var result = await skill.Resources![0].ReadAsync();
var result = await skill.GetTestResources()![0].ReadAsync();
// Assert — per-resource JSO takes effect and custom type is properly marshaled
Assert.NotNull(result);
@@ -0,0 +1,43 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Collections.Generic;
using System.Linq;
using System.Reflection;
namespace Microsoft.Agents.AI.UnitTests.AgentSkills;
/// <summary>
/// Test-only helpers that peek at the underlying resource list of a skill via reflection.
/// </summary>
/// <remarks>
/// The public <see cref="AgentSkill"/> API exposes resources only through
/// <see cref="AgentSkill.GetResourceAsync"/>.
/// These helpers exist purely to allow unit tests for <see cref="AgentFileSkill"/> and
/// <see cref="AgentInlineSkill"/> to inspect the concrete enumerated list a skill carries.
/// </remarks>
internal static class AgentSkillTestExtensions
{
public static IReadOnlyList<AgentSkillResource>? GetTestResources(this AgentSkill skill)
{
// AgentFileSkill / AgentInlineSkill: private "_resources" field.
for (var type = skill.GetType(); type is not null; type = type.BaseType)
{
var field = type.GetField("_resources", BindingFlags.NonPublic | BindingFlags.Instance);
if (field is not null)
{
return UnwrapList(field.GetValue(skill));
}
}
return null;
}
private static IReadOnlyList<AgentSkillResource>? UnwrapList(object? value) =>
value switch
{
null => null,
IReadOnlyList<AgentSkillResource> list => list,
IEnumerable<AgentSkillResource> seq => seq.ToList(),
_ => null,
};
}
@@ -68,11 +68,12 @@ public sealed class AgentSkillsProviderTests : IDisposable
Assert.Contains("provider-skill", result.Instructions);
Assert.Contains("Provider skill test", result.Instructions);
// Should have load_skill tool (no resources, so no read_skill_resource)
// Should have load_skill, read_skill_resource, and run_skill_script tools
Assert.NotNull(result.Tools);
var toolNames = result.Tools!.Select(t => t.Name).ToList();
Assert.Contains("load_skill", toolNames);
Assert.DoesNotContain("read_skill_resource", toolNames);
Assert.Contains("read_skill_resource", toolNames);
Assert.Contains("run_skill_script", toolNames);
}
[Fact]
@@ -316,7 +317,7 @@ public sealed class AgentSkillsProviderTests : IDisposable
}
[Fact]
public async Task InvokingCoreAsync_WithoutScripts_NoRunSkillScriptToolAsync()
public async Task InvokingCoreAsync_WithoutScripts_StillIncludesAllToolsAsync()
{
// Arrange
this.CreateSkill("no-script-skill", "No scripts", "Body.");
@@ -328,10 +329,12 @@ public sealed class AgentSkillsProviderTests : IDisposable
// Act
var result = await provider.InvokingAsync(invokingContext, CancellationToken.None);
// Assert
// Assert — all tools are always included regardless of skill content
Assert.NotNull(result.Tools);
var toolNames = result.Tools!.Select(t => t.Name).ToList();
Assert.DoesNotContain("run_skill_script", toolNames);
Assert.Contains("load_skill", toolNames);
Assert.Contains("read_skill_resource", toolNames);
Assert.Contains("run_skill_script", toolNames);
}
[Fact]
@@ -416,7 +419,7 @@ public sealed class AgentSkillsProviderTests : IDisposable
// Assert
Assert.Single(skills);
var fileSkill = Assert.IsType<AgentFileSkill>(skills[0]);
Assert.All(fileSkill.Resources, r => Assert.EndsWith(".json", r.Name));
Assert.All(fileSkill.GetTestResources()!, r => Assert.EndsWith(".json", r.Name));
}
private void CreateSkill(string name, string description, string body)
@@ -445,6 +448,279 @@ public sealed class AgentSkillsProviderTests : IDisposable
Assert.Contains("Skill body.", text);
}
[Fact]
public async Task LoadSkill_EmptySkillName_ReturnsErrorAsync()
{
// Arrange
this.CreateSkill("any-skill", "Test", "Body.");
var provider = new AgentSkillsProvider(new AgentFileSkillsSource(this._testRoot, s_noOpExecutor));
var invokingContext = new AIContextProvider.InvokingContext(this._agent, session: null, new AIContext());
var result = await provider.InvokingAsync(invokingContext, CancellationToken.None);
var loadSkillTool = result.Tools!.First(t => t.Name == "load_skill") as AIFunction;
// Act
var content = await loadSkillTool!.InvokeAsync(new AIFunctionArguments(new Dictionary<string, object?> { ["skillName"] = "" }));
// Assert
Assert.Equal("Error: Skill name cannot be empty.", content!.ToString());
}
[Fact]
public async Task LoadSkill_SkillNotFound_ReturnsErrorAsync()
{
// Arrange
this.CreateSkill("only-skill", "Test", "Body.");
var provider = new AgentSkillsProvider(new AgentFileSkillsSource(this._testRoot, s_noOpExecutor));
var invokingContext = new AIContextProvider.InvokingContext(this._agent, session: null, new AIContext());
var result = await provider.InvokingAsync(invokingContext, CancellationToken.None);
var loadSkillTool = result.Tools!.First(t => t.Name == "load_skill") as AIFunction;
// Act
var content = await loadSkillTool!.InvokeAsync(new AIFunctionArguments(new Dictionary<string, object?> { ["skillName"] = "non-existent" }));
// Assert
Assert.Equal("Error: Skill 'non-existent' not found.", content!.ToString());
}
[Fact]
public async Task InvokingCoreAsync_WithResources_IncludesReadSkillResourceToolAsync()
{
// Arrange — inline skill with a resource
var skill = new AgentInlineSkill("res-skill", "Has resources", "Body.");
skill.AddResource("config", "value1", "A config resource.");
var provider = new AgentSkillsProvider(skill);
var invokingContext = new AIContextProvider.InvokingContext(this._agent, session: null, new AIContext());
// Act
var result = await provider.InvokingAsync(invokingContext, CancellationToken.None);
// Assert
Assert.NotNull(result.Tools);
var toolNames = result.Tools!.Select(t => t.Name).ToList();
Assert.Contains("read_skill_resource", toolNames);
}
[Fact]
public async Task ReadSkillResource_ReturnsResourceContentAsync()
{
// Arrange — inline skill with a resource
var skill = new AgentInlineSkill("res-skill", "Has resources", "Body.");
skill.AddResource("config", "resource-value");
var provider = new AgentSkillsProvider(skill);
var invokingContext = new AIContextProvider.InvokingContext(this._agent, session: null, new AIContext());
var result = await provider.InvokingAsync(invokingContext, CancellationToken.None);
var readTool = result.Tools!.First(t => t.Name == "read_skill_resource") as AIFunction;
// Act
var content = await readTool!.InvokeAsync(new AIFunctionArguments(new Dictionary<string, object?>
{
["skillName"] = "res-skill",
["resourceName"] = "config",
})
{
Services = new TestServiceProvider(),
});
// Assert
Assert.Equal("resource-value", content!.ToString());
}
[Fact]
public async Task ReadSkillResource_EmptySkillName_ReturnsErrorAsync()
{
// Arrange
var skill = new AgentInlineSkill("res-skill", "Has resources", "Body.");
skill.AddResource("config", "v");
var provider = new AgentSkillsProvider(skill);
var invokingContext = new AIContextProvider.InvokingContext(this._agent, session: null, new AIContext());
var result = await provider.InvokingAsync(invokingContext, CancellationToken.None);
var readTool = result.Tools!.First(t => t.Name == "read_skill_resource") as AIFunction;
// Act
var content = await readTool!.InvokeAsync(new AIFunctionArguments(new Dictionary<string, object?>
{
["skillName"] = "",
["resourceName"] = "config",
})
{
Services = new TestServiceProvider(),
});
// Assert
Assert.Equal("Error: Skill name cannot be empty.", content!.ToString());
}
[Fact]
public async Task ReadSkillResource_EmptyResourceName_ReturnsErrorAsync()
{
// Arrange
var skill = new AgentInlineSkill("res-skill", "Has resources", "Body.");
skill.AddResource("config", "v");
var provider = new AgentSkillsProvider(skill);
var invokingContext = new AIContextProvider.InvokingContext(this._agent, session: null, new AIContext());
var result = await provider.InvokingAsync(invokingContext, CancellationToken.None);
var readTool = result.Tools!.First(t => t.Name == "read_skill_resource") as AIFunction;
// Act
var content = await readTool!.InvokeAsync(new AIFunctionArguments(new Dictionary<string, object?>
{
["skillName"] = "res-skill",
["resourceName"] = "",
})
{
Services = new TestServiceProvider(),
});
// Assert
Assert.Equal("Error: Resource name cannot be empty.", content!.ToString());
}
[Fact]
public async Task ReadSkillResource_SkillNotFound_ReturnsErrorAsync()
{
// Arrange
var skill = new AgentInlineSkill("res-skill", "Has resources", "Body.");
skill.AddResource("config", "v");
var provider = new AgentSkillsProvider(skill);
var invokingContext = new AIContextProvider.InvokingContext(this._agent, session: null, new AIContext());
var result = await provider.InvokingAsync(invokingContext, CancellationToken.None);
var readTool = result.Tools!.First(t => t.Name == "read_skill_resource") as AIFunction;
// Act
var content = await readTool!.InvokeAsync(new AIFunctionArguments(new Dictionary<string, object?>
{
["skillName"] = "non-existent",
["resourceName"] = "config",
})
{
Services = new TestServiceProvider(),
});
// Assert
Assert.Equal("Error: Skill 'non-existent' not found.", content!.ToString());
}
[Fact]
public async Task ReadSkillResource_ResourceNotFound_ReturnsErrorAsync()
{
// Arrange
var skill = new AgentInlineSkill("res-skill", "Has resources", "Body.");
skill.AddResource("config", "v");
var provider = new AgentSkillsProvider(skill);
var invokingContext = new AIContextProvider.InvokingContext(this._agent, session: null, new AIContext());
var result = await provider.InvokingAsync(invokingContext, CancellationToken.None);
var readTool = result.Tools!.First(t => t.Name == "read_skill_resource") as AIFunction;
// Act
var content = await readTool!.InvokeAsync(new AIFunctionArguments(new Dictionary<string, object?>
{
["skillName"] = "res-skill",
["resourceName"] = "missing",
})
{
Services = new TestServiceProvider(),
});
// Assert
Assert.Equal("Error: Resource 'missing' not found in skill 'res-skill'.", content!.ToString());
}
[Fact]
public async Task RunSkillScript_EmptySkillName_ReturnsErrorAsync()
{
// Arrange
string skillDir = Path.Combine(this._testRoot, "err-script-skill");
Directory.CreateDirectory(Path.Combine(skillDir, "scripts"));
File.WriteAllText(Path.Combine(skillDir, "SKILL.md"), "---\nname: err-script-skill\ndescription: Test\n---\nBody.");
File.WriteAllText(Path.Combine(skillDir, "scripts", "run.py"), "print('hi')");
var provider = new AgentSkillsProvider(new AgentFileSkillsSource(this._testRoot, s_noOpExecutor));
var invokingContext = new AIContextProvider.InvokingContext(this._agent, session: null, new AIContext());
var result = await provider.InvokingAsync(invokingContext, CancellationToken.None);
var runScriptTool = result.Tools!.First(t => t.Name == "run_skill_script") as AIFunction;
// Act
var content = await runScriptTool!.InvokeAsync(new AIFunctionArguments(new Dictionary<string, object?>
{
["skillName"] = "",
["scriptName"] = "scripts/run.py",
}));
// Assert
Assert.Equal("Error: Skill name cannot be empty.", content!.ToString());
}
[Fact]
public async Task RunSkillScript_EmptyScriptName_ReturnsErrorAsync()
{
// Arrange
string skillDir = Path.Combine(this._testRoot, "err-script2-skill");
Directory.CreateDirectory(Path.Combine(skillDir, "scripts"));
File.WriteAllText(Path.Combine(skillDir, "SKILL.md"), "---\nname: err-script2-skill\ndescription: Test\n---\nBody.");
File.WriteAllText(Path.Combine(skillDir, "scripts", "run.py"), "print('hi')");
var provider = new AgentSkillsProvider(new AgentFileSkillsSource(this._testRoot, s_noOpExecutor));
var invokingContext = new AIContextProvider.InvokingContext(this._agent, session: null, new AIContext());
var result = await provider.InvokingAsync(invokingContext, CancellationToken.None);
var runScriptTool = result.Tools!.First(t => t.Name == "run_skill_script") as AIFunction;
// Act
var content = await runScriptTool!.InvokeAsync(new AIFunctionArguments(new Dictionary<string, object?>
{
["skillName"] = "err-script2-skill",
["scriptName"] = "",
}));
// Assert
Assert.Equal("Error: Script name cannot be empty.", content!.ToString());
}
[Fact]
public async Task RunSkillScript_SkillNotFound_ReturnsErrorAsync()
{
// Arrange
string skillDir = Path.Combine(this._testRoot, "err-script3-skill");
Directory.CreateDirectory(Path.Combine(skillDir, "scripts"));
File.WriteAllText(Path.Combine(skillDir, "SKILL.md"), "---\nname: err-script3-skill\ndescription: Test\n---\nBody.");
File.WriteAllText(Path.Combine(skillDir, "scripts", "run.py"), "print('hi')");
var provider = new AgentSkillsProvider(new AgentFileSkillsSource(this._testRoot, s_noOpExecutor));
var invokingContext = new AIContextProvider.InvokingContext(this._agent, session: null, new AIContext());
var result = await provider.InvokingAsync(invokingContext, CancellationToken.None);
var runScriptTool = result.Tools!.First(t => t.Name == "run_skill_script") as AIFunction;
// Act
var content = await runScriptTool!.InvokeAsync(new AIFunctionArguments(new Dictionary<string, object?>
{
["skillName"] = "non-existent",
["scriptName"] = "scripts/run.py",
}));
// Assert
Assert.Equal("Error: Skill 'non-existent' not found.", content!.ToString());
}
[Fact]
public async Task RunSkillScript_ScriptNotFound_ReturnsErrorAsync()
{
// Arrange
string skillDir = Path.Combine(this._testRoot, "err-script4-skill");
Directory.CreateDirectory(Path.Combine(skillDir, "scripts"));
File.WriteAllText(Path.Combine(skillDir, "SKILL.md"), "---\nname: err-script4-skill\ndescription: Test\n---\nBody.");
File.WriteAllText(Path.Combine(skillDir, "scripts", "run.py"), "print('hi')");
var provider = new AgentSkillsProvider(new AgentFileSkillsSource(this._testRoot, s_noOpExecutor));
var invokingContext = new AIContextProvider.InvokingContext(this._agent, session: null, new AIContext());
var result = await provider.InvokingAsync(invokingContext, CancellationToken.None);
var runScriptTool = result.Tools!.First(t => t.Name == "run_skill_script") as AIFunction;
// Act
var content = await runScriptTool!.InvokeAsync(new AIFunctionArguments(new Dictionary<string, object?>
{
["skillName"] = "err-script4-skill",
["scriptName"] = "scripts/missing.py",
}));
// Assert
Assert.Equal("Error: Script 'scripts/missing.py' not found in skill 'err-script4-skill'.", content!.ToString());
}
[Fact]
public async Task Builder_UseFileScriptRunnerAfterUseFileSkills_RunnerIsUsedAsync()
{
@@ -998,9 +1274,5 @@ public sealed class AgentSkillsProviderTests : IDisposable
public override AgentSkillFrontmatter Frontmatter { get; }
protected override string Instructions => this._instructions;
public override IReadOnlyList<AgentSkillResource>? Resources => null;
public override IReadOnlyList<AgentSkillScript>? Scripts => null;
}
}
@@ -281,9 +281,9 @@ public sealed class FileAgentSkillLoaderTests : IDisposable
// Assert
Assert.Single(skills);
var skill = skills[0];
Assert.Equal(2, skill.Resources!.Count);
Assert.Contains(skill.Resources!, r => r.Name.Equals("references/FAQ.md", StringComparison.OrdinalIgnoreCase));
Assert.Contains(skill.Resources!, r => r.Name.Equals("assets/data.json", StringComparison.OrdinalIgnoreCase));
Assert.Equal(2, skill.GetTestResources()!.Count);
Assert.Contains(skill.GetTestResources()!, r => r.Name.Equals("references/FAQ.md", StringComparison.OrdinalIgnoreCase));
Assert.Contains(skill.GetTestResources()!, r => r.Name.Equals("assets/data.json", StringComparison.OrdinalIgnoreCase));
}
[Fact]
@@ -306,8 +306,8 @@ public sealed class FileAgentSkillLoaderTests : IDisposable
// Assert
Assert.Single(skills);
var skill = skills[0];
Assert.Single(skill.Resources!);
Assert.Equal("references/data.json", skill.Resources![0].Name);
Assert.Single(skill.GetTestResources()!);
Assert.Equal("references/data.json", skill.GetTestResources()![0].Name);
}
[Fact]
@@ -329,8 +329,8 @@ public sealed class FileAgentSkillLoaderTests : IDisposable
// Assert
Assert.Single(skills);
var skill = skills[0];
Assert.Single(skill.Resources!);
Assert.Equal("references/notes.md", skill.Resources![0].Name);
Assert.Single(skill.GetTestResources()!);
Assert.Equal("references/notes.md", skill.GetTestResources()![0].Name);
}
[Fact]
@@ -355,9 +355,9 @@ public sealed class FileAgentSkillLoaderTests : IDisposable
// Assert — only the file directly in references/ is discovered; the nested file is not
Assert.Single(skills);
var skill = skills[0];
Assert.Single(skill.Resources!);
Assert.Contains(skill.Resources!, r => r.Name.Equals("references/top.md", StringComparison.OrdinalIgnoreCase));
Assert.DoesNotContain(skill.Resources!, r => r.Name.Contains("deep.md", StringComparison.OrdinalIgnoreCase));
Assert.Single(skill.GetTestResources()!);
Assert.Contains(skill.GetTestResources()!, r => r.Name.Equals("references/top.md", StringComparison.OrdinalIgnoreCase));
Assert.DoesNotContain(skill.GetTestResources()!, r => r.Name.Contains("deep.md", StringComparison.OrdinalIgnoreCase));
}
[Fact]
@@ -380,8 +380,8 @@ public sealed class FileAgentSkillLoaderTests : IDisposable
// Assert — only .custom files should be discovered, not .json
Assert.Single(skills);
var skill = skills[0];
Assert.Single(skill.Resources!);
Assert.Equal("references/data.custom", skill.Resources![0].Name);
Assert.Single(skill.GetTestResources()!);
Assert.Equal("references/data.custom", skill.GetTestResources()![0].Name);
}
[Theory]
@@ -406,7 +406,7 @@ public sealed class FileAgentSkillLoaderTests : IDisposable
// Assert — default extensions include .md
var skills = await source.GetSkillsAsync();
Assert.Single(skills[0].Resources!);
Assert.Single(skills[0].GetTestResources()!);
}
[Fact]
@@ -442,7 +442,7 @@ public sealed class FileAgentSkillLoaderTests : IDisposable
// Assert — root-level files are NOT discovered unless "." is in ResourceDirectories
Assert.Single(skills);
Assert.Empty(skills[0].Resources!);
Assert.Empty(skills[0].GetTestResources()!);
}
[Fact]
@@ -465,9 +465,9 @@ public sealed class FileAgentSkillLoaderTests : IDisposable
// Assert — both root-level resource files (and SKILL.md excluded) should be discovered
Assert.Single(skills);
var skill = skills[0];
Assert.Equal(2, skill.Resources!.Count);
Assert.Contains(skill.Resources!, r => r.Name.Equals("guide.md", StringComparison.OrdinalIgnoreCase));
Assert.Contains(skill.Resources!, r => r.Name.Equals("config.json", StringComparison.OrdinalIgnoreCase));
Assert.Equal(2, skill.GetTestResources()!.Count);
Assert.Contains(skill.GetTestResources()!, r => r.Name.Equals("guide.md", StringComparison.OrdinalIgnoreCase));
Assert.Contains(skill.GetTestResources()!, r => r.Name.Equals("config.json", StringComparison.OrdinalIgnoreCase));
}
[Fact]
@@ -488,7 +488,7 @@ public sealed class FileAgentSkillLoaderTests : IDisposable
// Assert — non-spec directories are not scanned by default
Assert.Single(skills);
Assert.Empty(skills[0].Resources!);
Assert.Empty(skills[0].GetTestResources()!);
}
[Fact]
@@ -514,8 +514,8 @@ public sealed class FileAgentSkillLoaderTests : IDisposable
// Assert — only docs/ is scanned; references/ is NOT scanned
Assert.Single(skills);
var skill = skills[0];
Assert.Single(skill.Resources!);
Assert.Equal("docs/readme.md", skill.Resources![0].Name);
Assert.Single(skill.GetTestResources()!);
Assert.Equal("docs/readme.md", skill.GetTestResources()![0].Name);
}
[Fact]
@@ -530,7 +530,7 @@ public sealed class FileAgentSkillLoaderTests : IDisposable
// Assert
Assert.Single(skills);
Assert.Empty(skills[0].Resources!);
Assert.Empty(skills[0].GetTestResources()!);
}
[Fact]
@@ -588,7 +588,7 @@ public sealed class FileAgentSkillLoaderTests : IDisposable
File.WriteAllText(Path.Combine(refsDir, "doc.md"), "Document content here.");
var source = new AgentFileSkillsSource(this._testRoot, s_noOpExecutor);
var skills = await source.GetSkillsAsync();
var resource = skills[0].Resources!.First(r => r.Name == "references/doc.md");
var resource = skills[0].GetTestResources()!.First(r => r.Name == "references/doc.md");
// Act
var content = await resource.ReadAsync();
@@ -672,8 +672,8 @@ public sealed class FileAgentSkillLoaderTests : IDisposable
// Assert — skill should still load, the symlinked references/ is skipped, assets/legit.md is found
var skill = skills.FirstOrDefault(s => s.Frontmatter.Name == "symlink-escape-skill");
Assert.NotNull(skill);
Assert.Single(skill.Resources!);
Assert.Equal("assets/legit.md", skill.Resources![0].Name);
Assert.Single(skill.GetTestResources()!);
Assert.Equal("assets/legit.md", skill.GetTestResources()![0].Name);
}
[Fact]
@@ -714,8 +714,8 @@ public sealed class FileAgentSkillLoaderTests : IDisposable
// Assert — only assets/legit.md is found; the symlinked references/ directory is skipped entirely
var skill = skills.FirstOrDefault(s => s.Frontmatter.Name == "symlink-directory-skip");
Assert.NotNull(skill);
Assert.Single(skill.Resources!);
Assert.Equal("assets/legit.md", skill.Resources![0].Name);
Assert.Single(skill.GetTestResources()!);
Assert.Equal("assets/legit.md", skill.GetTestResources()![0].Name);
}
[Fact]
@@ -751,7 +751,7 @@ public sealed class FileAgentSkillLoaderTests : IDisposable
// Assert — skill loads but scripts from the symlinked directory are not discovered
var skill = skills.FirstOrDefault(s => s.Frontmatter.Name == "symlink-script-skip");
Assert.NotNull(skill);
Assert.Empty(skill.Scripts!);
Assert.Null(await skill.GetScriptAsync("any-script"));
}
[Fact]
@@ -791,7 +791,7 @@ public sealed class FileAgentSkillLoaderTests : IDisposable
// Assert — the symlinked intermediate segment causes the directory to be skipped
var skill = skills.FirstOrDefault(s => s.Frontmatter.Name == "symlink-intermediate");
Assert.NotNull(skill);
Assert.Empty(skill.Resources!);
Assert.Empty(skill.GetTestResources()!);
}
#endif
@@ -1020,8 +1020,8 @@ public sealed class FileAgentSkillLoaderTests : IDisposable
// Assert — only one copy of the resource despite two equivalent directory entries
Assert.Single(skills);
Assert.Single(skills[0].Resources!);
Assert.Equal("references/FAQ.md", skills[0].Resources![0].Name);
Assert.Single(skills[0].GetTestResources()!);
Assert.Equal("references/FAQ.md", skills[0].GetTestResources()![0].Name);
}
[Fact]
@@ -1043,8 +1043,8 @@ public sealed class FileAgentSkillLoaderTests : IDisposable
// Assert — trailing slash variant deduplicated
Assert.Single(skills);
Assert.Single(skills[0].Resources!);
Assert.Equal("references/data.json", skills[0].Resources![0].Name);
Assert.Single(skills[0].GetTestResources()!);
Assert.Equal("references/data.json", skills[0].GetTestResources()![0].Name);
}
[Fact]
@@ -1066,8 +1066,9 @@ public sealed class FileAgentSkillLoaderTests : IDisposable
// Assert — backslash variant deduplicated
Assert.Single(skills);
Assert.Single(skills[0].Scripts!);
Assert.Equal("scripts/run.py", skills[0].Scripts![0].Name);
var script = await skills[0].GetScriptAsync("scripts/run.py");
Assert.NotNull(script);
Assert.Equal("scripts/run.py", script!.Name);
}
[Theory]
@@ -1093,8 +1094,8 @@ public sealed class FileAgentSkillLoaderTests : IDisposable
// Assert — the resource is discovered with a name identical to using the directory without "./"
Assert.Single(skills);
Assert.Single(skills[0].Resources!);
Assert.Equal($"{directoryWithoutDotSlash}/data.json", skills[0].Resources![0].Name);
Assert.Single(skills[0].GetTestResources()!);
Assert.Equal($"{directoryWithoutDotSlash}/data.json", skills[0].GetTestResources()![0].Name);
}
[Fact]
@@ -1117,8 +1118,8 @@ public sealed class FileAgentSkillLoaderTests : IDisposable
// Assert — resource file inside the deeply nested directory is discovered
Assert.Single(skills);
var skill = skills[0];
Assert.Single(skill.Resources!);
Assert.Equal("f1/f2/f3/data.json", skill.Resources![0].Name);
Assert.Single(skill.GetTestResources()!);
Assert.Equal("f1/f2/f3/data.json", skill.GetTestResources()![0].Name);
}
private string CreateSkillDirectory(string name, string description, string body)
@@ -1188,8 +1189,9 @@ public sealed class FileAgentSkillLoaderTests : IDisposable
// Assert — script at the skill root should be discovered
var skill = skills.FirstOrDefault(s => s.Frontmatter.Name == "root-script-skill");
Assert.NotNull(skill);
Assert.Single(skill.Scripts!);
Assert.Equal("run.py", skill.Scripts![0].Name);
var script = await skill.GetScriptAsync("run.py");
Assert.NotNull(script);
Assert.Equal("run.py", script!.Name);
}
#if NET
@@ -1229,8 +1231,8 @@ public sealed class FileAgentSkillLoaderTests : IDisposable
// Assert — only legit.md should be discovered; the symlinked leak.md is skipped
var skill = skills.FirstOrDefault(s => s.Frontmatter.Name == "symlink-file-skill");
Assert.NotNull(skill);
Assert.Single(skill.Resources!);
Assert.Equal("references/legit.md", skill.Resources![0].Name);
Assert.Single(skill.GetTestResources()!);
Assert.Equal("references/legit.md", skill.GetTestResources()![0].Name);
}
#endif
}
@@ -0,0 +1,301 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.IO;
using System.IO.Compression;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.AI;
namespace Microsoft.Agents.AI.UnitTests.AgentSkills;
/// <summary>
/// Unit tests that verify the Hosted-AgentSkills sample patterns: ZIP extraction with
/// zip-slip guard, skill name validation, and AgentSkillsProvider loading from
/// downloaded skill directories (the Foundry download → extract → wire-into-provider flow).
/// </summary>
public sealed class HostedAgentSkillsPatternTests : IDisposable
{
private readonly string _testRoot;
private readonly TestAIAgent _agent = new();
public HostedAgentSkillsPatternTests()
{
this._testRoot = Path.Combine(Path.GetTempPath(), "hosted-skills-tests-" + Guid.NewGuid().ToString("N"));
Directory.CreateDirectory(this._testRoot);
}
public void Dispose()
{
if (Directory.Exists(this._testRoot))
{
Directory.Delete(this._testRoot, recursive: true);
}
}
// ── ZIP extraction tests ──────────────────────────────────────────────────
[Fact]
public void SafeExtractZip_ValidArchive_ExtractsToDestination()
{
// Arrange
string destDir = Path.Combine(this._testRoot, "valid-extract");
Directory.CreateDirectory(destDir);
byte[] zip = CreateZipWithEntry("SKILL.md", "---\nname: test\ndescription: Test\n---\nBody.");
// Act
using var archive = new ZipArchive(new MemoryStream(zip), ZipArchiveMode.Read);
SafeExtractZip(archive, destDir);
// Assert
Assert.True(File.Exists(Path.Combine(destDir, "SKILL.md")));
string content = File.ReadAllText(Path.Combine(destDir, "SKILL.md"));
Assert.Contains("name: test", content);
}
[Fact]
public void SafeExtractZip_ZipSlipAttempt_ThrowsInvalidOperationException()
{
// Arrange
string destDir = Path.Combine(this._testRoot, "zipslip-test");
Directory.CreateDirectory(destDir);
byte[] zip = CreateZipWithEntry("../../../evil.txt", "malicious content");
// Act & Assert
using var archive = new ZipArchive(new MemoryStream(zip), ZipArchiveMode.Read);
var ex = Assert.Throws<InvalidOperationException>(() => SafeExtractZip(archive, destDir));
Assert.Contains("outside of", ex.Message);
}
[Fact]
public void SafeExtractZip_SiblingPrefixAttack_ThrowsInvalidOperationException()
{
// Arrange — sibling path that starts with the dest dir name
string destDir = Path.Combine(this._testRoot, "target");
Directory.CreateDirectory(destDir);
byte[] zip = CreateZipWithEntry("../target-evil/payload.txt", "exploit");
// Act & Assert
using var archive = new ZipArchive(new MemoryStream(zip), ZipArchiveMode.Read);
var ex = Assert.Throws<InvalidOperationException>(() => SafeExtractZip(archive, destDir));
Assert.Contains("outside of", ex.Message);
}
[Fact]
public void SafeExtractZip_DirectoryEntry_CreatesDirectory()
{
// Arrange
string destDir = Path.Combine(this._testRoot, "dir-entry");
Directory.CreateDirectory(destDir);
byte[] zip = CreateZipWithDirectoryEntry("subdir/");
// Act
using var archive = new ZipArchive(new MemoryStream(zip), ZipArchiveMode.Read);
SafeExtractZip(archive, destDir);
// Assert
Assert.True(Directory.Exists(Path.Combine(destDir, "subdir")));
}
// ── Skill name validation tests ──────────────────────────────────────────
[Theory]
[InlineData("../escape")]
[InlineData("path/traversal")]
[InlineData("path\\traversal")]
[InlineData("has.dots")]
public void ValidateSkillName_InvalidNames_Rejected(string name)
{
// Act & Assert
Assert.True(IsInvalidSkillName(name), $"Expected '{name}' to be rejected.");
}
[Theory]
[InlineData("support-style")]
[InlineData("escalation-policy")]
[InlineData("my-skill-123")]
public void ValidateSkillName_ValidNames_Accepted(string name)
{
// Act & Assert
Assert.False(IsInvalidSkillName(name), $"Expected '{name}' to be accepted.");
}
// ── AgentSkillsProvider integration with downloaded skill directories ─────
[Fact]
public async Task AgentSkillsProvider_WithDownloadedSkills_AdvertisesAndLoadsAsync()
{
// Arrange — simulate the Foundry download + extract flow
string downloadDir = Path.Combine(this._testRoot, "downloaded_skills");
Directory.CreateDirectory(downloadDir);
CreateDownloadedSkill(downloadDir, "support-style",
"---\nname: support-style\ndescription: Contoso Outdoors customer-support tone and formatting guidelines.\n---\n\n# Contoso Outdoors Support Style\n\nYou are speaking on behalf of Contoso Outdoors.\n\n## Canary\n\nInclude STYLE-CANARY-3318.");
CreateDownloadedSkill(downloadDir, "escalation-policy",
"---\nname: escalation-policy\ndescription: When and how to escalate Contoso Outdoors customer-support tickets.\n---\n\n# Escalation Policy\n\nProvide ESC-CANARY-7742.");
var provider = new AgentSkillsProvider(downloadDir, scriptRunner: null);
var inputContext = new AIContext
{
Instructions = "You are a customer-support assistant for Contoso Outdoors."
};
var invokingContext = new AIContextProvider.InvokingContext(this._agent, session: null, inputContext);
// Act
var result = await provider.InvokingAsync(invokingContext, CancellationToken.None);
// Assert — skills are advertised in instructions
Assert.NotNull(result.Instructions);
Assert.Contains("support-style", result.Instructions);
Assert.Contains("escalation-policy", result.Instructions);
Assert.Contains("Contoso Outdoors customer-support tone", result.Instructions);
// Assert — load_skill tool is available
Assert.NotNull(result.Tools);
var toolNames = result.Tools!.Select(t => t.Name).ToList();
Assert.Contains("load_skill", toolNames);
// All tools are always included regardless of whether skills have resources or scripts
Assert.Contains("read_skill_resource", toolNames);
Assert.Contains("run_skill_script", toolNames);
}
[Fact]
public async Task LoadSkill_ReturnsFullContentWithCanaryAsync()
{
// Arrange
string downloadDir = Path.Combine(this._testRoot, "canary_skills");
Directory.CreateDirectory(downloadDir);
CreateDownloadedSkill(downloadDir, "support-style",
"---\nname: support-style\ndescription: Contoso tone guidelines.\n---\n\nInclude STYLE-CANARY-3318 at the bottom.");
var provider = new AgentSkillsProvider(downloadDir, scriptRunner: null);
var inputContext = new AIContext();
var invokingContext = new AIContextProvider.InvokingContext(this._agent, session: null, inputContext);
var result = await provider.InvokingAsync(invokingContext, CancellationToken.None);
var loadSkillTool = result.Tools!.First(t => t.Name == "load_skill") as AIFunction;
Assert.NotNull(loadSkillTool);
// Act
var content = await loadSkillTool!.InvokeAsync(
new AIFunctionArguments(new System.Collections.Generic.Dictionary<string, object?> { ["skillName"] = "support-style" }));
// Assert
var text = content!.ToString()!;
Assert.Contains("STYLE-CANARY-3318", text);
Assert.Contains("name: support-style", text);
}
[Fact]
public async Task LoadSkill_UnknownName_ReturnsErrorAsync()
{
// Arrange
string downloadDir = Path.Combine(this._testRoot, "error_skills");
Directory.CreateDirectory(downloadDir);
CreateDownloadedSkill(downloadDir, "support-style",
"---\nname: support-style\ndescription: Test\n---\nBody.");
var provider = new AgentSkillsProvider(downloadDir, scriptRunner: null);
var inputContext = new AIContext();
var invokingContext = new AIContextProvider.InvokingContext(this._agent, session: null, inputContext);
var result = await provider.InvokingAsync(invokingContext, CancellationToken.None);
var loadSkillTool = result.Tools!.First(t => t.Name == "load_skill") as AIFunction;
// Act
var content = await loadSkillTool!.InvokeAsync(
new AIFunctionArguments(new System.Collections.Generic.Dictionary<string, object?> { ["skillName"] = "nonexistent-skill" }));
// Assert
var text = content!.ToString()!;
Assert.Contains("Error", text);
Assert.Contains("not found", text);
}
// ── Helpers ──────────────────────────────────────────────────────────────
/// <summary>
/// Creates a downloaded skill directory with a SKILL.md file — simulating what
/// the Foundry download + ZIP extract flow produces.
/// </summary>
private static void CreateDownloadedSkill(string parentDir, string name, string content)
{
string skillDir = Path.Combine(parentDir, name);
Directory.CreateDirectory(skillDir);
File.WriteAllText(Path.Combine(skillDir, "SKILL.md"), content);
}
/// <summary>
/// Creates a ZIP archive in memory containing a single file entry.
/// </summary>
private static byte[] CreateZipWithEntry(string entryName, string content)
{
using var ms = new MemoryStream();
using (var archive = new ZipArchive(ms, ZipArchiveMode.Create, leaveOpen: true))
{
var entry = archive.CreateEntry(entryName);
using var writer = new StreamWriter(entry.Open());
writer.Write(content);
}
return ms.ToArray();
}
/// <summary>
/// Creates a ZIP archive in memory containing a single directory entry.
/// </summary>
private static byte[] CreateZipWithDirectoryEntry(string directoryName)
{
using var ms = new MemoryStream();
using (var archive = new ZipArchive(ms, ZipArchiveMode.Create, leaveOpen: true))
{
// Directory entries in ZIPs have an empty name portion and end with /
archive.CreateEntry(directoryName);
}
return ms.ToArray();
}
/// <summary>
/// Mirrors the zip-slip guard from the Hosted-AgentSkills sample Program.cs.
/// </summary>
private static void SafeExtractZip(ZipArchive archive, string destinationDir)
{
string destRoot = Path.GetFullPath(destinationDir);
string destRootWithSep = Path.EndsInDirectorySeparator(destRoot)
? destRoot
: destRoot + Path.DirectorySeparatorChar;
var comparison = OperatingSystem.IsWindows()
? StringComparison.OrdinalIgnoreCase
: StringComparison.Ordinal;
foreach (ZipArchiveEntry entry in archive.Entries)
{
string entryPath = Path.GetFullPath(Path.Combine(destRoot, entry.FullName));
if (!entryPath.StartsWith(destRootWithSep, comparison)
&& !string.Equals(entryPath, destRoot, comparison))
{
throw new InvalidOperationException(
$"Refusing to extract unsafe path '{entry.FullName}' outside of '{destRoot}'.");
}
if (string.IsNullOrEmpty(entry.Name))
{
Directory.CreateDirectory(entryPath);
}
else
{
Directory.CreateDirectory(Path.GetDirectoryName(entryPath)!);
entry.ExtractToFile(entryPath, overwrite: true);
}
}
}
/// <summary>
/// Mirrors the skill name validation from the Hosted-AgentSkills sample Program.cs.
/// </summary>
private static bool IsInvalidSkillName(string name) =>
name.Contains('.') || name.Contains('/') || name.Contains('\\') || Path.IsPathRooted(name);
}
@@ -31,13 +31,7 @@ internal sealed class TestAgentSkill : AgentSkill
public override AgentSkillFrontmatter Frontmatter => this._frontmatter;
/// <inheritdoc/>
public override string Content => this._content;
/// <inheritdoc/>
public override IReadOnlyList<AgentSkillResource>? Resources => null;
/// <inheritdoc/>
public override IReadOnlyList<AgentSkillScript>? Scripts => null;
public override ValueTask<string> GetContentAsync(CancellationToken cancellationToken = default) => new(this._content);
}
/// <summary>
@@ -73,7 +73,7 @@ public class AgentModeProviderTests
{
// Arrange
var (tools, state) = await CreateToolsWithStateAsync();
AIFunction setMode = GetTool(tools, "AgentMode_Set");
AIFunction setMode = GetTool(tools, "mode_set");
// Act
await setMode.InvokeAsync(new AIFunctionArguments() { ["mode"] = "execute" });
@@ -90,7 +90,7 @@ public class AgentModeProviderTests
{
// Arrange
var (tools, _) = await CreateToolsWithStateAsync();
AIFunction setMode = GetTool(tools, "AgentMode_Set");
AIFunction setMode = GetTool(tools, "mode_set");
// Act
object? result = await setMode.InvokeAsync(new AIFunctionArguments() { ["mode"] = "execute" });
@@ -107,8 +107,8 @@ public class AgentModeProviderTests
{
// Arrange
var (tools, provider, session) = await CreateToolsWithProviderAndSessionAsync();
AIFunction setMode = GetTool(tools, "AgentMode_Set");
AIFunction getMode = GetTool(tools, "AgentMode_Get");
AIFunction setMode = GetTool(tools, "mode_set");
AIFunction getMode = GetTool(tools, "mode_get");
// Act & Assert
await Assert.ThrowsAsync<ArgumentException>(async () =>
@@ -131,7 +131,7 @@ public class AgentModeProviderTests
{
// Arrange
var (tools, _) = await CreateToolsWithStateAsync();
AIFunction getMode = GetTool(tools, "AgentMode_Get");
AIFunction getMode = GetTool(tools, "mode_get");
// Act
object? result = await getMode.InvokeAsync(new AIFunctionArguments());
@@ -148,8 +148,8 @@ public class AgentModeProviderTests
{
// Arrange
var (tools, _) = await CreateToolsWithStateAsync();
AIFunction setMode = GetTool(tools, "AgentMode_Set");
AIFunction getMode = GetTool(tools, "AgentMode_Get");
AIFunction setMode = GetTool(tools, "mode_set");
AIFunction getMode = GetTool(tools, "mode_get");
// Act
await setMode.InvokeAsync(new AIFunctionArguments() { ["mode"] = "execute" });
@@ -236,7 +236,7 @@ public class AgentModeProviderTests
// Act
AIContext result = await provider.InvokingAsync(context);
AIFunction getMode = GetTool(result.Tools!, "AgentMode_Get");
AIFunction getMode = GetTool(result.Tools!, "mode_get");
object? modeResult = await getMode.InvokeAsync(new AIFunctionArguments());
// Assert
@@ -264,12 +264,12 @@ public class AgentModeProviderTests
// Act — first invocation changes mode
AIContext result1 = await provider.InvokingAsync(context);
AIFunction setMode = GetTool(result1.Tools!, "AgentMode_Set");
AIFunction setMode = GetTool(result1.Tools!, "mode_set");
await setMode.InvokeAsync(new AIFunctionArguments() { ["mode"] = "execute" });
// Second invocation should see the updated mode
AIContext result2 = await provider.InvokingAsync(context);
AIFunction getMode = GetTool(result2.Tools!, "AgentMode_Get");
AIFunction getMode = GetTool(result2.Tools!, "mode_get");
object? modeResult = await getMode.InvokeAsync(new AIFunctionArguments());
// Assert
@@ -579,7 +579,7 @@ public class AgentModeProviderTests
// First call to initialize
AIContext result1 = await provider.InvokingAsync(context);
AIFunction setMode = GetTool(result1.Tools!, "AgentMode_Set");
AIFunction setMode = GetTool(result1.Tools!, "mode_set");
// Change mode via the tool (agent-initiated)
await setMode.InvokeAsync(new AIFunctionArguments() { ["mode"] = "execute" });
@@ -16,6 +16,7 @@
<!-- Evaluation tests require net8.0+ (MEAI.Evaluation does not support legacy TFMs) -->
<ItemGroup Condition="!$([MSBuild]::IsTargetFrameworkCompatible('$(TargetFramework)', 'net8.0'))">
<Compile Remove="EvaluationTests.cs" />
<Compile Remove="AgentSkills\HostedAgentSkillsPatternTests.cs" />
</ItemGroup>
<ItemGroup>
@@ -1,6 +1,10 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Reflection;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Agents.AI.Workflows.Declarative.ObjectModel;
using Microsoft.Agents.ObjectModel;
@@ -198,6 +202,131 @@ public sealed class ForeachExecutorTest(ITestOutputHelper output) : WorkflowActi
indexName: "CurrentIndex");
}
/// <summary>
/// Regression test for GH-5009: a <see cref="ForeachExecutor"/> that is re-instantiated
/// during checkpoint restore (e.g. cross-process resume after a <c>Question</c> inside the
/// loop body) must continue iterating from where it left off, not exit after the first
/// iteration.
/// </summary>
[Fact]
public async Task ForeachStateRestoredAcrossCheckpointAsync()
{
// Arrange — a 3-item source table and a freshly-bound foreach executor (instance A).
const string SourceVariableName = "SourceArray";
this.SetVariableState("CurrentValue");
this.State.Set(
SourceVariableName,
FormulaValue.NewTable(
RecordType.Empty(),
FormulaValue.NewRecordFromFields(new NamedValue("value", FormulaValue.New(10))),
FormulaValue.NewRecordFromFields(new NamedValue("value", FormulaValue.New(20))),
FormulaValue.NewRecordFromFields(new NamedValue("value", FormulaValue.New(30)))));
Foreach model = this.CreateModel(
displayName: nameof(ForeachStateRestoredAcrossCheckpointAsync),
items: ValueExpression.Variable(PropertyPath.TopicVariable(SourceVariableName)),
valueName: "CurrentValue",
indexName: null);
ForeachExecutor instanceA = new(model, this.State);
// Drive instance A through ExecuteAsync (initializes _values/_index) and one TakeNextAsync
// so that _index advances to 1 and HasValue is true — the state at the point a Question
// inside the loop body would pause the workflow and trigger a checkpoint.
await this.ExecuteAsync(instanceA, ForeachExecutor.Steps.Next(instanceA.Id), instanceA.TakeNextAsync);
Assert.True(instanceA.HasValue, "Instance A should have a current item after the first TakeNextAsync.");
// Act 1 — instance A persists checkpoint state.
InMemoryWorkflowContext checkpoint = new();
await InvokeOnCheckpointingAsync(instanceA, checkpoint);
// Act 2 — a fresh instance B (simulating cross-process resume) restores from the checkpoint.
ForeachExecutor instanceB = new(model, this.State);
await InvokeOnCheckpointRestoredAsync(instanceB, checkpoint);
// Assert — HasValue carries over so the routing predicate after loopId continues to take
// the "loop body" edge instead of falling through to the loop continuation.
Assert.True(instanceB.HasValue, "Restored instance should report HasValue == true at the checkpointed cursor.");
// Drive iteration 2 and 3 through instance B; both should succeed.
await instanceB.TakeNextAsync(checkpoint, _: null, CancellationToken.None);
Assert.True(instanceB.HasValue, "Restored instance should advance to iteration 2 (value=20).");
await instanceB.TakeNextAsync(checkpoint, _: null, CancellationToken.None);
Assert.True(instanceB.HasValue, "Restored instance should advance to iteration 3 (value=30).");
// Driving past the end exits the loop normally.
await instanceB.TakeNextAsync(checkpoint, _: null, CancellationToken.None);
Assert.False(instanceB.HasValue, "Restored instance should report HasValue == false after exhausting all items.");
}
/// <summary>
/// When no checkpoint state has been written for the executor (e.g. first run), the restore
/// hook must be a no-op and leave constructor defaults in place.
/// </summary>
[Fact]
public async Task ForeachRestoreWithNoSavedStateAsync()
{
// Arrange
this.SetVariableState("CurrentValue");
Foreach model = this.CreateModel(
displayName: nameof(ForeachRestoreWithNoSavedStateAsync),
items: ValueExpression.Literal(DataValue.EmptyTable),
valueName: "CurrentValue",
indexName: null);
ForeachExecutor executor = new(model, this.State);
InMemoryWorkflowContext emptyContext = new();
// Act — restoring against an empty context must not throw and must leave the executor
// in its constructor-default state.
await InvokeOnCheckpointRestoredAsync(executor, emptyContext);
// Assert
Assert.False(executor.HasValue);
// A subsequent TakeNextAsync (without a prior ExecuteAsync) should report no value
// because _values is still the empty constructor default.
await executor.TakeNextAsync(emptyContext, _: null, CancellationToken.None);
Assert.False(executor.HasValue);
}
/// <summary>
/// Checkpoint/restore around a foreach over an empty source must roundtrip cleanly
/// (zero-length <c>PortableValue[]</c> snapshot).
/// </summary>
[Fact]
public async Task ForeachStateSurvivesEmptyValuesAsync()
{
// Arrange
this.SetVariableState("CurrentValue");
Foreach model = this.CreateModel(
displayName: nameof(ForeachStateSurvivesEmptyValuesAsync),
items: ValueExpression.Literal(DataValue.EmptyTable),
valueName: "CurrentValue",
indexName: null);
ForeachExecutor instanceA = new(model, this.State);
// Run ExecuteAsync (which sets _values = []) followed by one TakeNextAsync (which sets
// HasValue = false on an empty source).
await this.ExecuteAsync(instanceA, ForeachExecutor.Steps.Next(instanceA.Id), instanceA.TakeNextAsync);
Assert.False(instanceA.HasValue);
// Act — checkpoint and restore into a fresh instance.
InMemoryWorkflowContext checkpoint = new();
await InvokeOnCheckpointingAsync(instanceA, checkpoint);
ForeachExecutor instanceB = new(model, this.State);
await InvokeOnCheckpointRestoredAsync(instanceB, checkpoint);
// Assert — restored instance must agree that the source is empty and HasValue is false.
Assert.False(instanceB.HasValue);
await instanceB.TakeNextAsync(checkpoint, _: null, CancellationToken.None);
Assert.False(instanceB.HasValue);
}
private void SetVariableState(string valueName, string? indexName = null, FormulaValue? valueState = null)
{
this.State.Set(valueName, valueState ?? FormulaValue.New("something"));
@@ -312,4 +441,96 @@ public sealed class ForeachExecutorTest(ITestOutputHelper output) : WorkflowActi
return AssignParent<Foreach>(actionBuilder);
}
// Reflection helpers used to invoke the `protected internal` checkpoint hooks on the executor
// base class from this test project (which is in a different assembly than Microsoft.Agents.AI.Workflows
// and is not granted InternalsVisibleTo there).
private static Task InvokeOnCheckpointingAsync(Executor executor, IWorkflowContext context) =>
InvokeProtectedCheckpointHookAsync(executor, context, methodName: "OnCheckpointingAsync");
private static Task InvokeOnCheckpointRestoredAsync(Executor executor, IWorkflowContext context) =>
InvokeProtectedCheckpointHookAsync(executor, context, methodName: "OnCheckpointRestoredAsync");
private static async Task InvokeProtectedCheckpointHookAsync(Executor executor, IWorkflowContext context, string methodName)
{
MethodInfo method = typeof(Executor).GetMethod(
methodName,
BindingFlags.Instance | BindingFlags.NonPublic,
binder: null,
types: new[] { typeof(IWorkflowContext), typeof(CancellationToken) },
modifiers: null) ?? throw new InvalidOperationException($"Could not locate {methodName} on Executor.");
ValueTask invocation = (ValueTask)method.Invoke(executor, new object[] { context, CancellationToken.None })!;
await invocation;
}
/// <summary>
/// Minimal in-memory <see cref="IWorkflowContext"/> implementation used to drive the
/// checkpoint/restore overrides on <see cref="ForeachExecutor"/> directly from a unit test.
/// Records state writes in a (scope, key) dictionary and serves matching reads back. Only the
/// state-related members are exercised by the checkpoint hooks; the other members are stubbed.
/// </summary>
private sealed class InMemoryWorkflowContext : IWorkflowContext
{
private readonly Dictionary<(string? scope, string key), object?> _store = [];
public bool ConcurrentRunsEnabled => false;
public IReadOnlyDictionary<string, string>? TraceContext => null;
public ValueTask QueueStateUpdateAsync<T>(string key, T? value, string? scopeName = null, CancellationToken cancellationToken = default)
{
this._store[(scopeName, key)] = value;
return default;
}
public ValueTask<T?> ReadStateAsync<T>(string key, string? scopeName = null, CancellationToken cancellationToken = default)
{
if (this._store.TryGetValue((scopeName, key), out object? stored) && stored is T typed)
{
return new ValueTask<T?>(typed);
}
return new ValueTask<T?>(default(T));
}
public ValueTask<T> ReadOrInitStateAsync<T>(string key, Func<T> initialStateFactory, string? scopeName = null, CancellationToken cancellationToken = default)
{
if (this._store.TryGetValue((scopeName, key), out object? stored) && stored is T typed)
{
return new ValueTask<T>(typed);
}
T initial = initialStateFactory();
this._store[(scopeName, key)] = initial;
return new ValueTask<T>(initial);
}
public ValueTask<HashSet<string>> ReadStateKeysAsync(string? scopeName = null, CancellationToken cancellationToken = default)
{
HashSet<string> keys = new(
this._store.Keys
.Where(slot => string.Equals(slot.scope, scopeName, StringComparison.Ordinal))
.Select(slot => slot.key));
return new ValueTask<HashSet<string>>(keys);
}
public ValueTask QueueClearScopeAsync(string? scopeName = null, CancellationToken cancellationToken = default)
{
foreach ((string? scope, string key) slot in this._store.Keys.Where(slot => string.Equals(slot.scope, scopeName, StringComparison.Ordinal)).ToArray())
{
this._store.Remove(slot);
}
return default;
}
public ValueTask AddEventAsync(WorkflowEvent workflowEvent, CancellationToken cancellationToken = default) => default;
public ValueTask SendMessageAsync(object message, string? targetId, CancellationToken cancellationToken = default) => default;
public ValueTask YieldOutputAsync(object output, CancellationToken cancellationToken = default) => default;
public ValueTask RequestHaltAsync() => default;
}
}
+15
View File
@@ -21,6 +21,21 @@ When making changes to a package, check if the following need updates:
- The package's `AGENTS.md` file (adding/removing/renaming public APIs, architecture changes, import path changes)
- The agent skills in `.github/skills/` if conventions, commands, or workflows change
At the end of every run, re-read `AGENTS.md` and the relevant skill files and
update any guidance that the conversation revealed to be out of date,
incomplete, or misleading (renamed files, changed commands, new conventions
the user confirmed, etc.). **Before adding a new principle or rule, ask the
user whether they want it captured as a durable principle** — do not invent
team norms from a single conversation without explicit confirmation.
## Terminology
- **Avoid "GA" for Agent Framework code.** Reserve *GA* for hosted services
(e.g. "the Foundry service is GA"). For Agent Framework packages, features,
and APIs use **"released"** or **"stable"** depending on context — these
match the feature-lifecycle stages documented in the
`python-feature-lifecycle` skill.
## Pull Request Description Guidance
When preparing a PR description:
@@ -45,6 +45,7 @@ from ._compaction import (
CharacterEstimatorTokenizer,
CompactionProvider,
CompactionStrategy,
ContextWindowCompactionStrategy,
SelectiveToolCallCompactionStrategy,
SlidingWindowStrategy,
SummarizationStrategy,
@@ -79,6 +80,16 @@ from ._evaluation import (
tool_calls_present,
)
from ._feature_stage import ExperimentalFeature, ReleaseCandidateFeature
from ._harness._agent import (
DEFAULT_HARNESS_INSTRUCTIONS,
create_harness_agent,
)
from ._harness._background_agents import (
DEFAULT_BACKGROUND_AGENTS_SOURCE_ID,
BackgroundAgentsProvider,
BackgroundTaskInfo,
BackgroundTaskStatus,
)
from ._harness._memory import (
DEFAULT_MEMORY_SOURCE_ID,
MemoryContextProvider,
@@ -297,6 +308,8 @@ __all__ = [
"AGENT_FRAMEWORK_USER_AGENT",
"APP_INFO",
"COMPACTION_STATE_KEY",
"DEFAULT_BACKGROUND_AGENTS_SOURCE_ID",
"DEFAULT_HARNESS_INSTRUCTIONS",
"DEFAULT_MAX_ITERATIONS",
"DEFAULT_MEMORY_SOURCE_ID",
"DEFAULT_MODE_SOURCE_ID",
@@ -332,6 +345,9 @@ __all__ = [
"AgentSession",
"AggregatingSkillsSource",
"Annotation",
"BackgroundAgentsProvider",
"BackgroundTaskInfo",
"BackgroundTaskStatus",
"BaseAgent",
"BaseChatClient",
"BaseEmbeddingClient",
@@ -352,6 +368,7 @@ __all__ = [
"CompactionStrategy",
"Content",
"ContextProvider",
"ContextWindowCompactionStrategy",
"ContinuationToken",
"ConversationSplit",
"ConversationSplitter",
@@ -499,6 +516,7 @@ __all__ = [
"apply_compaction",
"chat_middleware",
"create_edge_runner",
"create_harness_agent",
"detect_media_type_from_base64",
"evaluate_agent",
"evaluate_workflow",
@@ -1277,6 +1277,121 @@ class CompactionProvider(ContextProvider):
# whether excluded messages are loaded on the next turn.
class ContextWindowCompactionStrategy:
"""Token-budget compaction derived from a model's context window size.
Computes an input budget from the model's context window and output token
limits, then applies a two-phase compaction pipeline:
1. **Tool result eviction** — collapses older tool-call groups into summaries
when included tokens exceed ``tool_eviction_threshold`` of the input budget.
2. **Truncation** — removes oldest non-system groups when included tokens
exceed ``truncation_threshold`` of the input budget.
The class uses two independent :class:`TokenBudgetComposedStrategy`
instances — one per phase — so each fires only when its own threshold
is exceeded.
Examples:
.. code-block:: python
from agent_framework import ContextWindowCompactionStrategy, CompactionProvider
strategy = ContextWindowCompactionStrategy(
max_context_window_tokens=128_000,
max_output_tokens=16_384,
)
provider = CompactionProvider(before_strategy=strategy)
"""
DEFAULT_TOOL_EVICTION_THRESHOLD: float = 0.5
"""Default fraction of input budget at which tool result eviction triggers."""
DEFAULT_TRUNCATION_THRESHOLD: float = 0.8
"""Default fraction of input budget at which truncation triggers."""
def __init__(
self,
*,
max_context_window_tokens: int,
max_output_tokens: int,
tokenizer: TokenizerProtocol | None = None,
tool_eviction_threshold: float = DEFAULT_TOOL_EVICTION_THRESHOLD,
truncation_threshold: float = DEFAULT_TRUNCATION_THRESHOLD,
keep_last_tool_call_groups: int = 4,
) -> None:
"""Create a context-window compaction strategy.
Keyword Args:
max_context_window_tokens: The model's maximum context window size
in tokens (e.g. 128,000).
max_output_tokens: The model's maximum output tokens per response
(e.g. 16,384).
tokenizer: Token counter for measuring message sizes. Defaults to
:class:`CharacterEstimatorTokenizer` (4 chars/token heuristic).
tool_eviction_threshold: Fraction of input budget (0.0, 1.0] at
which tool result eviction triggers. Defaults to 0.5.
truncation_threshold: Fraction of input budget (0.0, 1.0] at which
truncation triggers. Must be ≥ ``tool_eviction_threshold``.
Defaults to 0.8.
keep_last_tool_call_groups: Number of most recent tool-call groups
to retain verbatim during tool eviction. Older groups are
collapsed into summaries. Defaults to 4.
Raises:
ValueError: If thresholds are out of range or inconsistent.
"""
if max_context_window_tokens <= 0:
raise ValueError("max_context_window_tokens must be positive.")
if max_output_tokens < 0 or max_output_tokens >= max_context_window_tokens:
raise ValueError("max_output_tokens must be >= 0 and < max_context_window_tokens.")
if not (0.0 < tool_eviction_threshold <= 1.0):
raise ValueError("tool_eviction_threshold must be in (0.0, 1.0].")
if not (0.0 < truncation_threshold <= 1.0):
raise ValueError("truncation_threshold must be in (0.0, 1.0].")
if truncation_threshold < tool_eviction_threshold:
raise ValueError("truncation_threshold must be >= tool_eviction_threshold.")
resolved_tokenizer = tokenizer or CharacterEstimatorTokenizer()
input_budget = max_context_window_tokens - max_output_tokens
tool_eviction_tokens = int(input_budget * tool_eviction_threshold)
truncation_tokens = int(input_budget * truncation_threshold)
self.max_context_window_tokens = max_context_window_tokens
self.max_output_tokens = max_output_tokens
self.input_budget_tokens = input_budget
self.tool_eviction_threshold = tool_eviction_threshold
self.truncation_threshold = truncation_threshold
self._tool_eviction = TokenBudgetComposedStrategy(
token_budget=tool_eviction_tokens,
tokenizer=resolved_tokenizer,
strategies=[
ToolResultCompactionStrategy(keep_last_tool_call_groups=keep_last_tool_call_groups),
],
)
self._truncation = TokenBudgetComposedStrategy(
token_budget=truncation_tokens,
tokenizer=resolved_tokenizer,
strategies=[
TruncationStrategy(
max_n=truncation_tokens,
compact_to=tool_eviction_tokens,
tokenizer=resolved_tokenizer,
),
],
)
async def __call__(self, messages: list[Message]) -> bool:
"""Apply the two-phase compaction pipeline.
Returns:
True if compaction changed message inclusion; otherwise False.
"""
changed = await self._tool_eviction(messages)
return (await self._truncation(messages)) or changed
__all__ = [
"COMPACTION_STATE_KEY",
"EXCLUDED_KEY",
@@ -1293,6 +1408,7 @@ __all__ = [
"CharacterEstimatorTokenizer",
"CompactionProvider",
"CompactionStrategy",
"ContextWindowCompactionStrategy",
"GroupKind",
"SelectiveToolCallCompactionStrategy",
"SlidingWindowStrategy",
@@ -58,6 +58,7 @@ class ExperimentalFeature(str, Enum):
FUNCTIONAL_WORKFLOWS = "FUNCTIONAL_WORKFLOWS"
HARNESS = "HARNESS"
SKILLS = "SKILLS"
TO_PROMPT_AGENT = "TO_PROMPT_AGENT"
class ReleaseCandidateFeature(str, Enum):
@@ -0,0 +1,349 @@
# Copyright (c) Microsoft. All rights reserved.
"""Harness agent factory: a pre-configured bundled agent with batteries included.
This module provides :func:`create_harness_agent`, a factory function that assembles
the full agent pipeline from a chat client, wiring up function invocation,
per-service-call history persistence, compaction, and a rich set of default
context providers (todo, mode, memory, skills).
"""
from __future__ import annotations
import logging
from collections.abc import Callable, Sequence
from typing import TYPE_CHECKING, Any
from .._agents import Agent
from .._clients import SupportsWebSearchTool
from .._compaction import CompactionProvider, ContextWindowCompactionStrategy, ToolResultCompactionStrategy
from .._feature_stage import ExperimentalFeature, experimental
from .._sessions import ContextProvider, HistoryProvider, InMemoryHistoryProvider
from .._skills import SkillsProvider
from ._memory import MemoryContextProvider, MemoryStore
from ._mode import AgentModeProvider
from ._todo import TodoProvider
if TYPE_CHECKING:
from collections.abc import Mapping
from .._clients import SupportsChatGetResponse
from .._compaction import CompactionStrategy, TokenizerProtocol
from .._middleware import MiddlewareTypes
from .._tools import ToolTypes
logger = logging.getLogger(__name__)
DEFAULT_HARNESS_INSTRUCTIONS = """\
You are a helpful AI assistant that uses tools to complete tasks.
## General guidelines
- Think through the task before acting. Break complex work into clear steps.
- Use the tools available to you to gather information, perform actions, and verify results.
- Explain your reasoning and thought process as you work through tasks.
- Explain what you learned and what you are going to do next between tool calls, \
so the user can follow along with your thought process.
- Avoid making more than 4 tool calls in a row without explaining what you are doing.
- If a tool call fails or returns unexpected results, adapt your approach rather than \
repeating the same call.
- When you have completed the task, present a clear and concise summary of what you did \
and what you found.
"""
def _assemble_instructions(
harness_instructions: str | None,
agent_instructions: str | None,
) -> str | None:
"""Assemble final instructions from harness + agent instructions."""
harness = harness_instructions if harness_instructions is not None else DEFAULT_HARNESS_INSTRUCTIONS
return f"{harness}\n\n{agent_instructions or ''}".strip() or None
def _assemble_compaction_provider(
*,
disable_compaction: bool,
max_context_window_tokens: int,
max_output_tokens: int,
history_source_id: str,
before_compaction_strategy: CompactionStrategy | None,
after_compaction_strategy: CompactionStrategy | None,
tokenizer: TokenizerProtocol | None,
) -> CompactionProvider | None:
"""Build the compaction provider from parameters or defaults."""
if disable_compaction:
return None
before_strategy = before_compaction_strategy or ContextWindowCompactionStrategy(
max_context_window_tokens=max_context_window_tokens,
max_output_tokens=max_output_tokens,
tokenizer=tokenizer,
)
after_strategy = after_compaction_strategy or ToolResultCompactionStrategy(keep_last_tool_call_groups=2)
return CompactionProvider(
before_strategy=before_strategy,
after_strategy=after_strategy,
tokenizer=tokenizer,
history_source_id=history_source_id,
)
def _assemble_context_providers(
*,
history_provider: HistoryProvider,
compaction_provider: CompactionProvider | None,
disable_todo: bool,
todo_provider: TodoProvider | None,
disable_mode: bool,
mode_provider: AgentModeProvider | None,
disable_memory: bool,
memory_store: MemoryStore | None,
skills_provider: SkillsProvider | None,
skills_paths: Sequence[str] | None,
extra_context_providers: Sequence[ContextProvider] | None,
) -> list[ContextProvider]:
"""Assemble the ordered list of context providers."""
providers: list[ContextProvider] = []
# History first so other providers can access loaded messages.
providers.append(history_provider)
# Compaction runs after history loads messages.
if compaction_provider is not None:
providers.append(compaction_provider)
if not disable_todo:
providers.append(todo_provider or TodoProvider())
if not disable_mode:
providers.append(mode_provider or AgentModeProvider())
if not disable_memory and memory_store is not None:
providers.append(MemoryContextProvider(store=memory_store))
# Skills are opt-in: only added when skills_provider or skills_paths is provided.
if skills_provider:
providers.append(skills_provider)
if skills_paths:
providers.append(SkillsProvider.from_paths(*skills_paths))
# Append any user-supplied additional providers.
if extra_context_providers:
providers.extend(extra_context_providers)
return providers
HARNESS_AGENT_PROVIDER_NAME = "microsoft.agent_framework.harness"
@experimental(feature_id=ExperimentalFeature.HARNESS)
def create_harness_agent(
client: SupportsChatGetResponse[Any],
*,
id: str | None = None,
name: str | None = None,
description: str | None = None,
harness_instructions: str | None = None,
agent_instructions: str | None = None,
tools: ToolTypes | Callable[..., Any] | Sequence[ToolTypes | Callable[..., Any]] | None = None,
max_context_window_tokens: int,
max_output_tokens: int,
history_provider: HistoryProvider | None = None,
disable_compaction: bool = False,
before_compaction_strategy: CompactionStrategy | None = None,
after_compaction_strategy: CompactionStrategy | None = None,
tokenizer: TokenizerProtocol | None = None,
disable_todo: bool = False,
todo_provider: TodoProvider | None = None,
disable_mode: bool = False,
mode_provider: AgentModeProvider | None = None,
disable_memory: bool = False,
memory_store: MemoryStore | None = None,
skills_provider: SkillsProvider | None = None,
skills_paths: Sequence[str] | None = None,
disable_web_search: bool = False,
otel_provider_name: str | None = None,
context_providers: Sequence[ContextProvider] | None = None,
middleware: Sequence[MiddlewareTypes] | None = None,
default_options: Mapping[str, Any] | None = None,
) -> Agent[Any]:
"""Create a pre-configured agent with batteries included.
Assembles an :class:`~agent_framework.Agent` from a chat client, automatically wiring:
- **Function invocation** — automatic tool calling loop
- **Per-service-call history persistence** — persists history after every model call
- **Compaction** — context-window compaction before/after each run
- **TodoProvider** — todo list management
- **AgentModeProvider** — plan/execute mode tracking
- **MemoryContextProvider** — file-based durable memory (when ``memory_store`` provided)
- **SkillsProvider** — skill discovery and progressive loading
- **OpenTelemetry** — observability via ``AgentTelemetryLayer``
Each feature can be disabled or customized via keyword arguments.
Examples:
Basic usage:
.. code-block:: python
from agent_framework import create_harness_agent
from agent_framework.openai import OpenAIChatClient
agent = create_harness_agent(
OpenAIChatClient(model="gpt-4o"),
max_context_window_tokens=128_000,
max_output_tokens=16_384,
)
session = agent.create_session()
response = await agent.run("Plan a weekend trip to Seattle", session=session)
With customization:
.. code-block:: python
agent = create_harness_agent(
client=client,
max_context_window_tokens=200_000,
max_output_tokens=32_000,
name="research-agent",
agent_instructions="Focus on academic sources.",
disable_todo=True,
skills_paths=["./skills", "./custom-skills"],
)
Args:
client: The chat client providing access to the underlying AI model.
Keyword Args:
id: Optional agent ID (auto-generated UUID if omitted).
name: Optional agent name.
description: Optional agent description.
harness_instructions: Override the default harness-level system instructions that
govern agent behavior (how to use tools, report progress, structure responses).
These provide general "operating guidelines" independent of any specific task.
When None, ``DEFAULT_HARNESS_INSTRUCTIONS`` is used. Set to empty string ``""``
to omit harness instructions entirely.
agent_instructions: Domain or task-specific instructions appended after harness
instructions. Use this for the agent's purpose, persona, or specialization
(e.g., "You are a research assistant focused on academic sources.").
tools: Additional tools to include in the agent's toolset.
max_context_window_tokens: Maximum tokens the model's context window supports.
max_output_tokens: Maximum output tokens per response.
history_provider: Custom history provider. When None, an InMemoryHistoryProvider is used.
disable_compaction: When True, skip compaction provider setup.
before_compaction_strategy: Custom before-run compaction strategy.
Defaults to ContextWindowCompactionStrategy (token-budget aware).
after_compaction_strategy: Custom after-run compaction strategy.
Defaults to ToolResultCompactionStrategy.
tokenizer: Custom tokenizer for compaction strategies.
disable_todo: When True, skip the TodoProvider.
todo_provider: Custom TodoProvider instance. Ignored when disable_todo is True.
disable_mode: When True, skip the AgentModeProvider.
mode_provider: Custom AgentModeProvider instance. Ignored when disable_mode is True.
disable_memory: When True, skip the MemoryContextProvider.
memory_store: Memory store instance. When provided (and disable_memory is False),
a MemoryContextProvider is added.
skills_provider: Custom SkillsProvider instance for code-defined skills.
Can be combined with ``skills_paths`` to aggregate file and code-based skills.
skills_paths: Paths for file-based skill discovery (looks for SKILL.md files).
Can be combined with ``skills_provider``. When neither ``skills_provider``
nor ``skills_paths`` is provided, no SkillsProvider is added.
disable_web_search: When True, skip automatic web search tool inclusion.
When False (default), the web search tool is automatically added if the
client implements SupportsWebSearchTool. A warning is logged if the client
does not support web search.
otel_provider_name: Custom OpenTelemetry provider/source name for telemetry.
context_providers: Additional context providers to include after the built-in ones.
middleware: Additional middleware to include.
default_options: Provider-specific chat options (temperature, max_tokens, etc.).
Returns:
A fully configured :class:`~agent_framework.Agent` instance.
Raises:
ValueError: If max_context_window_tokens <= 0 or max_output_tokens < 0
or max_output_tokens >= max_context_window_tokens.
"""
if max_context_window_tokens <= 0:
raise ValueError("max_context_window_tokens must be positive.")
if max_output_tokens < 0:
raise ValueError("max_output_tokens must be non-negative.")
if max_output_tokens >= max_context_window_tokens:
raise ValueError("max_output_tokens must be less than max_context_window_tokens.")
# Build history provider.
resolved_history = history_provider or InMemoryHistoryProvider()
# Build compaction provider.
compaction_provider = _assemble_compaction_provider(
disable_compaction=disable_compaction,
max_context_window_tokens=max_context_window_tokens,
max_output_tokens=max_output_tokens,
history_source_id=resolved_history.source_id,
before_compaction_strategy=before_compaction_strategy,
after_compaction_strategy=after_compaction_strategy,
tokenizer=tokenizer,
)
# Build context providers.
assembled_providers = _assemble_context_providers(
history_provider=resolved_history,
compaction_provider=compaction_provider,
disable_todo=disable_todo,
todo_provider=todo_provider,
disable_mode=disable_mode,
mode_provider=mode_provider,
disable_memory=disable_memory,
memory_store=memory_store,
skills_provider=skills_provider,
skills_paths=skills_paths,
extra_context_providers=context_providers,
)
# Build instructions.
instructions = _assemble_instructions(harness_instructions, agent_instructions)
# Assemble tools, auto-adding web search if supported.
assembled_tools: list[ToolTypes | Callable[..., Any]] = []
if not disable_web_search:
if isinstance(client, SupportsWebSearchTool):
assembled_tools.append(client.get_web_search_tool())
else:
logger.warning(
"Web search tool not available: client %r does not implement SupportsWebSearchTool. "
"Set disable_web_search=True to suppress this warning.",
type(client).__name__,
)
if tools is not None:
if isinstance(tools, Sequence):
assembled_tools.extend(tools) # pyright: ignore[reportUnknownArgumentType]
else:
assembled_tools.append(tools)
final_tools: list[ToolTypes | Callable[..., Any]] | None = assembled_tools or None
# Build default options dict.
default_opts: dict[str, Any] = dict(default_options) if default_options else {}
default_opts.setdefault("max_tokens", max_output_tokens)
agent = Agent(
client,
instructions,
id=id,
name=name,
description=description,
tools=final_tools,
default_options=default_opts, # type: ignore[arg-type]
context_providers=assembled_providers,
middleware=list(middleware) if middleware else None,
require_per_service_call_history_persistence=True,
)
# Set the telemetry provider name after construction.
agent.otel_provider_name = otel_provider_name or HARNESS_AGENT_PROVIDER_NAME
return agent
@@ -0,0 +1,521 @@
# Copyright (c) Microsoft. All rights reserved.
"""BackgroundAgentsProvider: enables an agent to delegate work to background sub-agents asynchronously.
This module provides :class:`BackgroundAgentsProvider`, a context provider that allows
a parent agent to start background tasks on child agents, wait for their completion,
and retrieve results. Each background task runs in its own session concurrently.
"""
from __future__ import annotations
import asyncio
from collections.abc import Awaitable, MutableMapping, Sequence
from dataclasses import dataclass, field
from enum import Enum
from typing import Any, ClassVar, cast
from .._agents import SupportsAgentRun
from .._feature_stage import ExperimentalFeature, experimental
from .._serialization import SerializationMixin
from .._sessions import AgentSession, ContextProvider, SessionContext
from .._tools import tool
from .._types import AgentResponse, Message
DEFAULT_BACKGROUND_AGENTS_SOURCE_ID = "background_agents"
DEFAULT_BACKGROUND_AGENTS_INSTRUCTIONS = """\
## Background Agents
You have access to background agents that can perform work on your behalf.
- Use the `background_agents_*` tools to start tasks on background agents and check their results.
- Creating a background task does not block, and background tasks run concurrently.
- Important: Always wait for outstanding tasks to finish before you finish processing.
- Important: After retrieving results from a completed task, clear it with \
background_agents_clear_completed_task to free memory, unless you plan to continue it with \
background_agents_continue_task.
{background_agents}"""
class BackgroundTaskStatus(str, Enum):
"""Status of a background task."""
RUNNING = "running"
COMPLETED = "completed"
FAILED = "failed"
LOST = "lost"
@experimental(feature_id=ExperimentalFeature.HARNESS)
class BackgroundTaskInfo(SerializationMixin):
"""Metadata for a single background task."""
DEFAULT_EXCLUDE: ClassVar[set[str]] = set()
id: int
agent_name: str
description: str
status: BackgroundTaskStatus
result_text: str | None
error_text: str | None
__slots__ = ("agent_name", "description", "error_text", "id", "result_text", "status")
def __init__(
self,
id: int,
agent_name: str,
description: str,
status: BackgroundTaskStatus = BackgroundTaskStatus.RUNNING,
result_text: str | None = None,
error_text: str | None = None,
) -> None:
"""Initialize a background task info entry."""
self.id = id
self.agent_name = agent_name
self.description = description
self.status = status
self.result_text = result_text
self.error_text = error_text
def to_dict(self, *, exclude: set[str] | None = None, exclude_none: bool = True) -> dict[str, Any]:
"""Serialize for session state persistence."""
del exclude
data: dict[str, Any] = {
"id": self.id,
"agent_name": self.agent_name,
"description": self.description,
"status": self.status.value,
}
if not exclude_none or self.result_text is not None:
data["result_text"] = self.result_text
if not exclude_none or self.error_text is not None:
data["error_text"] = self.error_text
return data
@classmethod
def from_dict(cls, data: MutableMapping[str, Any], **kwargs: Any) -> BackgroundTaskInfo:
"""Deserialize from session state."""
return cls(
id=data["id"],
agent_name=data["agent_name"],
description=data["description"],
status=BackgroundTaskStatus(data["status"]),
result_text=data.get("result_text"),
error_text=data.get("error_text"),
)
@dataclass
class _RuntimeState:
"""Non-serializable per-session runtime state for background tasks."""
in_flight_tasks: dict[int, asyncio.Task[AgentResponse[Any]]] = field(
default_factory=lambda: {} # pyright: ignore[reportUnknownLambdaType]
)
background_sessions: dict[int, AgentSession] = field(
default_factory=lambda: {} # pyright: ignore[reportUnknownLambdaType]
)
# ---------------------------------------------------------------------------
# Module-level helper functions (following ModeProvider pattern)
# ---------------------------------------------------------------------------
async def _run_agent(awaitable: Awaitable[AgentResponse[Any]]) -> AgentResponse[Any]:
"""Wrap an Awaitable in a proper coroutine for use with asyncio.create_task."""
return await awaitable
def _validate_and_build_agent_dict(agents: Sequence[SupportsAgentRun]) -> dict[str, SupportsAgentRun]:
"""Validate agents and build a case-insensitive lookup dict.
Raises:
ValueError: If agents is empty, an agent has no name, or names are not unique.
"""
if not agents:
raise ValueError("At least one background agent must be provided.")
agent_dict: dict[str, SupportsAgentRun] = {}
for agent in agents:
name = agent.name
if not name or not name.strip():
raise ValueError("All background agents must have a non-empty name.")
key = name.lower()
if key in agent_dict:
raise ValueError(
f"Duplicate background agent name: '{name}'. Agent names must be unique (case-insensitive)."
)
agent_dict[key] = agent
return agent_dict
def _build_agent_list_text(agents: dict[str, SupportsAgentRun]) -> str:
"""Build text listing available background agents."""
lines = ["Available background agents:"]
for agent in agents.values():
line = f"- {agent.name}"
if agent.description:
line += f": {agent.description}"
lines.append(line)
return "\n".join(lines)
def _get_provider_state(session: AgentSession, *, source_id: str) -> dict[str, Any]:
"""Load or initialize serializable provider state from session."""
state = session.state.get(source_id)
if state is None:
initial: dict[str, Any] = {"next_task_id": 1, "tasks": []}
session.state[source_id] = initial
return initial
return cast(dict[str, Any], state)
def _save_provider_state(session: AgentSession, state: dict[str, Any], *, source_id: str) -> None:
"""Persist serializable state to session."""
session.state[source_id] = state
def _get_tasks(state: dict[str, Any]) -> list[BackgroundTaskInfo]:
"""Parse task list from state dict."""
return [BackgroundTaskInfo.from_dict(t) for t in state.get("tasks", [])]
def _save_tasks(state: dict[str, Any], tasks: list[BackgroundTaskInfo]) -> None:
"""Serialize task list back to state dict."""
state["tasks"] = [t.to_dict() for t in tasks]
def _finalize_task(
task_info: BackgroundTaskInfo,
completed_task: asyncio.Task[AgentResponse[Any]],
runtime: _RuntimeState,
) -> None:
"""Extract results from a completed asyncio task and update task info."""
if completed_task.cancelled():
task_info.status = BackgroundTaskStatus.FAILED
task_info.error_text = "Task was canceled."
else:
exception = completed_task.exception()
if exception is not None:
task_info.status = BackgroundTaskStatus.FAILED
task_info.error_text = str(exception)
else:
task_info.status = BackgroundTaskStatus.COMPLETED
task_info.result_text = completed_task.result().text
runtime.in_flight_tasks.pop(task_info.id, None)
def _refresh_task_state(
session: AgentSession, state: dict[str, Any], runtime: _RuntimeState, *, source_id: str
) -> list[BackgroundTaskInfo]:
"""Refresh status of in-flight tasks and return updated task list."""
tasks = _get_tasks(state)
changed = False
for task_info in tasks:
if task_info.status != BackgroundTaskStatus.RUNNING:
continue
in_flight = runtime.in_flight_tasks.get(task_info.id)
if in_flight is None:
task_info.status = BackgroundTaskStatus.LOST
changed = True
continue
if in_flight.done():
_finalize_task(task_info, in_flight, runtime)
changed = True
if changed:
_save_tasks(state, tasks)
_save_provider_state(session, state, source_id=source_id)
return tasks
# ---------------------------------------------------------------------------
# Provider class
# ---------------------------------------------------------------------------
@experimental(feature_id=ExperimentalFeature.HARNESS)
class BackgroundAgentsProvider(ContextProvider):
"""Context provider that enables an agent to delegate work to background sub-agents.
The ``BackgroundAgentsProvider`` allows a parent agent to start background tasks on child agents,
wait for their completion, and retrieve results. Each background task runs in its own session and
executes concurrently.
This provider exposes the following tools to the agent:
- ``background_agents_start_task`` — Start a background task on a named agent with text input.
- ``background_agents_wait_for_first_completion`` — Block until the first of the specified tasks completes.
- ``background_agents_get_task_results`` — Retrieve the text output of a completed background task.
- ``background_agents_get_all_tasks`` — List all background tasks with their IDs, statuses, and descriptions.
- ``background_agents_continue_task`` — Send follow-up input to a completed task's session to resume work.
- ``background_agents_clear_completed_task`` — Remove a completed task and release its session.
"""
def __init__(
self,
agents: Sequence[SupportsAgentRun],
*,
source_id: str = DEFAULT_BACKGROUND_AGENTS_SOURCE_ID,
instructions: str | None = None,
) -> None:
"""Initialize the background agents provider.
Args:
agents: Collection of background agents available for delegation.
Each agent must have a non-empty, unique name (case-insensitive).
Keyword Args:
source_id: Unique source ID for serializable task state in session.
instructions: Optional instruction override. May include ``{background_agents}``
placeholder which will be replaced with the agent listing.
Raises:
ValueError: If agents is empty, an agent has no name, or names are not unique.
"""
super().__init__(source_id)
self._agents = _validate_and_build_agent_dict(agents)
# Build instructions with agent listing.
base_instructions = instructions if instructions is not None else DEFAULT_BACKGROUND_AGENTS_INSTRUCTIONS
agent_list_text = _build_agent_list_text(self._agents)
self._instructions = base_instructions.replace("{background_agents}", agent_list_text)
# Per-session runtime state (non-serializable), keyed by session_id.
# Note: Runtime state (in-flight asyncio.Task objects, child AgentSession handles)
# is inherently non-serializable and cannot survive process restarts. If the provider
# instance is lost, _refresh_task_state() marks orphaned tasks as LOST.
self._runtime: dict[str, _RuntimeState] = {}
def _get_runtime(self, session: AgentSession) -> _RuntimeState:
"""Get or create runtime state for a session."""
session_id = session.session_id
if session_id not in self._runtime:
self._runtime[session_id] = _RuntimeState()
return self._runtime[session_id]
async def before_run(
self,
*,
agent: Any,
session: AgentSession,
context: SessionContext,
state: dict[str, Any],
) -> None:
"""Inject background agent tools and instructions before the model runs."""
del agent, state
provider_state = _get_provider_state(session, source_id=self.source_id)
runtime = self._get_runtime(session)
source_id = self.source_id
@tool(name="background_agents_start_task", approval_mode="never_require")
def background_agents_start_task(agent_name: str, input: str, description: str) -> str:
"""Start a background task on a named agent. Returns a confirmation with the task ID."""
key = agent_name.lower()
if key not in self._agents:
available = ", ".join(a.name or "" for a in self._agents.values())
return f"Error: No background agent found with name '{agent_name}'. Available agents: {available}"
bg_agent = self._agents[key]
task_id = provider_state.get("next_task_id", 1)
provider_state["next_task_id"] = task_id + 1
task_info = BackgroundTaskInfo(
id=task_id,
agent_name=agent_name,
description=description,
)
tasks = _get_tasks(provider_state)
tasks.append(task_info)
_save_tasks(provider_state, tasks)
# Create a dedicated session for this background task.
sub_session = bg_agent.create_session()
# Start the task concurrently.
async_task = asyncio.create_task(_run_agent(bg_agent.run(input, session=sub_session)))
runtime.in_flight_tasks[task_id] = async_task
runtime.background_sessions[task_id] = sub_session
_save_provider_state(session, provider_state, source_id=source_id)
return f"Background task {task_id} started on agent '{agent_name}'."
@tool(name="background_agents_wait_for_first_completion", approval_mode="never_require")
async def background_agents_wait_for_first_completion(task_ids: list[int]) -> str:
"""Block until the first of the specified background tasks completes. Returns the completed task's ID."""
if not task_ids:
return "Error: No task IDs provided."
# Collect in-flight tasks matching the requested IDs.
waitable: list[tuple[int, asyncio.Task[AgentResponse[Any]]]] = []
for tid in task_ids:
in_flight = runtime.in_flight_tasks.get(tid)
if in_flight is not None:
waitable.append((tid, in_flight))
if not waitable:
# Refresh state to catch any that completed.
tasks = _refresh_task_state(session, provider_state, runtime, source_id=source_id)
already_complete = next(
(t for t in tasks if t.id in task_ids and t.status != BackgroundTaskStatus.RUNNING), None
)
if already_complete is not None:
return (
f"Task {already_complete.id} is not running; current status: {already_complete.status.value}."
)
return "Error: None of the specified task IDs correspond to running tasks."
# Wait for the first one to complete.
done, _ = await asyncio.wait(
[t for _, t in waitable],
return_when=asyncio.FIRST_COMPLETED,
)
# Find which ID completed.
completed_id: int | None = None
for tid, task in waitable:
if task in done:
completed_id = tid
break
# Finalize the completed task.
tasks = _get_tasks(provider_state)
task_info = next((t for t in tasks if t.id == completed_id), None)
if task_info is not None and completed_id is not None:
completed_task = runtime.in_flight_tasks.get(completed_id)
if completed_task is not None:
_finalize_task(task_info, completed_task, runtime)
_save_tasks(provider_state, tasks)
_save_provider_state(session, provider_state, source_id=source_id)
status_str = task_info.status.value if task_info else "Unknown"
return f"Task {completed_id} finished with status: {status_str}."
@tool(name="background_agents_get_task_results", approval_mode="never_require")
def background_agents_get_task_results(task_id: int) -> str:
"""Get the text output of a background task by its ID."""
tasks = _refresh_task_state(session, provider_state, runtime, source_id=source_id)
task_info = next((t for t in tasks if t.id == task_id), None)
if task_info is None:
return f"Error: No task found with ID {task_id}."
if task_info.status == BackgroundTaskStatus.COMPLETED:
return task_info.result_text or "(no output)"
if task_info.status == BackgroundTaskStatus.FAILED:
return f"Task failed: {task_info.error_text or 'Unknown error'}"
if task_info.status == BackgroundTaskStatus.LOST:
return "Task state was lost (reference unavailable)."
if task_info.status == BackgroundTaskStatus.RUNNING:
return f"Task {task_id} is still running."
return f"Task {task_id} has status: {task_info.status.value}."
@tool(name="background_agents_get_all_tasks", approval_mode="never_require")
def background_agents_get_all_tasks() -> str:
"""List all background tasks with their IDs, statuses, agent names, and descriptions."""
tasks = _refresh_task_state(session, provider_state, runtime, source_id=source_id)
if not tasks:
return "No tasks."
lines = ["Tasks:"]
for t in tasks:
lines.append(f"- Task {t.id} [{t.status.value}] ({t.agent_name}): {t.description}")
return "\n".join(lines)
@tool(name="background_agents_continue_task", approval_mode="never_require")
def background_agents_continue_task(task_id: int, text: str) -> str:
"""Send follow-up input to a completed or failed task to resume its work."""
tasks = _refresh_task_state(session, provider_state, runtime, source_id=source_id)
task_info = next((t for t in tasks if t.id == task_id), None)
if task_info is None:
return f"Error: No task found with ID {task_id}."
if task_info.status == BackgroundTaskStatus.LOST:
return (
f"Error: Task {task_id} cannot be continued because its session was lost. Start a new task instead."
)
if task_info.status == BackgroundTaskStatus.RUNNING:
return f"Error: Task {task_id} is still running. Wait for it to complete before continuing."
key = task_info.agent_name.lower()
if key not in self._agents:
return f"Error: Agent '{task_info.agent_name}' is no longer available."
sub_session = runtime.background_sessions.get(task_id)
if sub_session is None:
return f"Error: Session for task {task_id} is no longer available."
bg_agent = self._agents[key]
# Reset task state and start a new run on the existing session.
task_info.status = BackgroundTaskStatus.RUNNING
task_info.result_text = None
task_info.error_text = None
_save_tasks(provider_state, tasks)
async_task = asyncio.create_task(_run_agent(bg_agent.run(text, session=sub_session)))
runtime.in_flight_tasks[task_id] = async_task
_save_provider_state(session, provider_state, source_id=source_id)
return f"Task {task_id} continued with new input."
@tool(name="background_agents_clear_completed_task", approval_mode="never_require")
def background_agents_clear_completed_task(task_id: int) -> str:
"""Remove a completed or failed task and release its session to free memory."""
tasks = _refresh_task_state(session, provider_state, runtime, source_id=source_id)
task_info = next((t for t in tasks if t.id == task_id), None)
if task_info is None:
return f"Error: No task found with ID {task_id}."
if task_info.status == BackgroundTaskStatus.RUNNING:
return f"Error: Task {task_id} is still running. Wait for it to complete before clearing."
# Remove the task from state.
tasks = [t for t in tasks if t.id != task_id]
_save_tasks(provider_state, tasks)
# Clean up runtime references.
runtime.in_flight_tasks.pop(task_id, None)
runtime.background_sessions.pop(task_id, None)
_save_provider_state(session, provider_state, source_id=source_id)
return f"Task {task_id} cleared."
# Inject instructions and current task status.
context.extend_instructions(self.source_id, [self._instructions])
context.extend_tools(
self.source_id,
[
background_agents_start_task,
background_agents_wait_for_first_completion,
background_agents_get_task_results,
background_agents_get_all_tasks,
background_agents_continue_task,
background_agents_clear_completed_task,
],
)
# Include current task status as context message if there are tasks.
# Refresh first to get accurate statuses for any tasks that completed between turns.
tasks = _refresh_task_state(session, provider_state, runtime, source_id=source_id)
if tasks:
status_lines = ["### Current background tasks"]
for t in tasks:
status_lines.append(f"- Task {t.id} [{t.status.value}] ({t.agent_name}): {t.description}")
context.extend_messages(
self.source_id,
[Message(role="user", contents=["\n".join(status_lines)])],
)
@@ -14,14 +14,19 @@ from .._types import Message
DEFAULT_MODE_SOURCE_ID = "agent_mode"
DEFAULT_MODE_INSTRUCTIONS = (
"## Agent Mode\n\n"
"You can operate in different modes. Depending on the mode you are in, "
"you will be required to follow different processes.\n\n"
"Use the get_mode tool to check your current operating mode.\n"
"Use the set_mode tool to switch between modes as your work progresses. "
"Only use set_mode if the user explicitly instructs/allows you to change modes.\n\n"
"- You can operate in different modes. Depending on the mode you are in, "
"you will be required to follow different processes.\n"
"- You must check the current mode after any user input, since the user may have changed the mode themselves, "
"e.g. the user may have switched to 'plan' mode after a previous research task finished in 'execute' mode, "
"meaning they want to review a plan first before execution.\n\n"
"Use the mode_get tool to check your current operating mode.\n"
"Use the mode_set tool to switch between modes as your work progresses. "
"Only use mode_set if the user explicitly instructs/allows you to change modes.\n\n"
"You are currently operating in the {current_mode} mode.\n\n"
"### Mandatory Mode based Workflow\n\n"
"For every new substantive user request, including short factual questions, "
"your behavior is determined by the mode you are in.\n\n"
"{available_modes}\n"
"\n"
"You are currently operating in the {current_mode} mode.\n"
)
DEFAULT_MODE_CHANGE_NOTIFICATION = (
'[Mode changed: The operating mode has been switched from "{previous_mode}" to "{current_mode}". '
@@ -31,13 +36,37 @@ DEFAULT_MODE_DESCRIPTIONS: dict[str, str] = {
"plan": (
"Use this mode when analyzing requirements, breaking down tasks, and creating plans. "
"This is the interactive mode — ask clarifying questions, discuss options, and get user approval before "
"proceeding."
"proceeding.\n\n"
"Process to follow when in plan mode:\n"
"1. Analyze the request with the purpose of building a research plan.\n"
"2. Create a list of todo items.\n"
"3. If needed, use the provided tools to do some exploratory checks to help build a plan and determine "
"what clarifying questions you may need from the user.\n"
"4. Ask for clarifications from the user where needed.\n"
" 1. Ask each clarification one by one.\n"
" 2. When asking for clarification and you have specific options in mind, present them to the user, "
"so they can choose the option instead of having to retype the entire response.\n"
" 3. Do not proceed until you have received all the needed clarifications.\n"
" 4. Do short exploratory research if it helps with being able to ask sensible clarifications from "
"the user.\n"
"5. Write the plan to a memory file, so that it is retained even if compaction happens. "
"Make sure to update the plan file if the user requests changes.\n"
"6. Present the plan to the user and ask for approval to switch to execute mode and process the plan.\n"
"7. When approval is granted, always switch to execute mode (using the `mode_set` tool), "
"and follow the steps for *Execute mode*."
),
"execute": (
"Use this mode when carrying out approved plans. Work autonomously using your best judgement — do not ask "
"the user questions or wait for feedback. Make reasonable decisions on your own so that there is a complete, "
"useful result when the user returns. If you encounter ambiguity, choose the most reasonable option and note "
"your choice."
"Use this mode when carrying out approved plans. Work autonomously using your best judgment — do not ask "
"the user questions or wait for feedback.\n\n"
"Process to follow when in execute mode:\n"
"1. If you don't have a plan or tasks yet, analyze the user request and create tasks and a plan. "
"(**Skip this step if you came from plan mode**)\n"
"2. Work autonomously — use your best judgment to make decisions and keep progressing without asking "
"the user questions. The goal is to have a complete, useful result ready when the user returns.\n"
"3. If you encounter ambiguity or an unexpected situation during execution, choose the most reasonable "
"option, note your choice, and keep going.\n"
"4. Mark tasks as completed as you finish them.\n"
"5. Continue working, thinking and calling tools until you have the research result for the user."
),
}
@@ -179,8 +208,8 @@ class AgentModeProvider(ContextProvider):
``"plan"`` (interactive planning) and ``"execute"`` (autonomous execution).
This provider exposes the following tools to the agent:
- ``set_mode``: Switch the agent's operating mode.
- ``get_mode``: Retrieve the agent's current operating mode.
- ``mode_set``: Switch the agent's operating mode.
- ``mode_get``: Retrieve the agent's current operating mode.
Public helper functions ``get_agent_mode`` and ``set_agent_mode`` allow external code to programmatically read
and change the mode.
@@ -223,7 +252,7 @@ class AgentModeProvider(ContextProvider):
def _build_instructions(self, current_mode: str) -> str:
"""Build the mode guidance injected for the current session."""
mode_lines = "".join(
f'- "{self._mode_display_names[mode]}": {description}\n'
f"#### {self._mode_display_names[mode]}\n\n{description}\n\n"
for mode, description in self.mode_descriptions.items()
)
instructions = self.instructions or DEFAULT_MODE_INSTRUCTIONS
@@ -257,8 +286,8 @@ class AgentModeProvider(ContextProvider):
provider_state = _get_mode_state(session, source_id=self.source_id)
previous_mode = provider_state.pop(_PREVIOUS_MODE_STATE_KEY, None)
@tool(name="set_mode", approval_mode="never_require")
def set_mode(mode: str) -> str:
@tool(name="mode_set", approval_mode="never_require")
def mode_set(mode: str) -> str:
"""Switch the agent's operating mode."""
# The agent invoked the tool itself, so it knows the mode just changed — bypass
# ``set_agent_mode`` to avoid triggering a notification message on the next turn.
@@ -267,8 +296,8 @@ class AgentModeProvider(ContextProvider):
tool_state["current_mode"] = normalized_mode
return json.dumps({"mode": normalized_mode, "message": f"Mode changed to '{normalized_mode}'."})
@tool(name="get_mode", approval_mode="never_require")
def get_mode() -> str:
@tool(name="mode_get", approval_mode="never_require")
def mode_get() -> str:
"""Get the agent's current operating mode."""
current_mode_value = get_agent_mode(
session,
@@ -282,11 +311,11 @@ class AgentModeProvider(ContextProvider):
self.source_id,
[self._build_instructions(current_mode)],
)
context.extend_tools(self.source_id, [set_mode, get_mode])
context.extend_tools(self.source_id, [mode_set, mode_get])
if isinstance(previous_mode, str) and previous_mode != current_mode:
# Inject a user-role message announcing the external mode change. System instructions
# always render first in the chat history, so the agent can otherwise stay anchored to
# the most recent ``set_mode`` tool call rather than the new mode.
# the most recent ``mode_set`` tool call rather than the new mode.
previous_display = self._mode_display_names.get(previous_mode, previous_mode)
current_display = self._mode_display_names.get(current_mode, current_mode)
notification = DEFAULT_MODE_CHANGE_NOTIFICATION.format(
@@ -54,6 +54,8 @@ class WorkflowAgent(BaseAgent):
# Class variable for the request info function name
REQUEST_INFO_FUNCTION_NAME: ClassVar[str] = "request_info"
_SESSION_STATE_KEY: ClassVar[str] = "workflow_agent"
_PENDING_REQUESTS_STATE_KEY: ClassVar[str] = "pending_request_info_events"
@dataclass
class RequestInfoFunctionArgs:
@@ -258,6 +260,7 @@ class WorkflowAgent(BaseAgent):
An AgentResponse representing the workflow execution results.
"""
input_messages = normalize_messages_input(messages)
self._restore_pending_requests_from_session(session)
if (
not any(
@@ -291,10 +294,11 @@ class WorkflowAgent(BaseAgent):
)
# combine the messages
session_messages: list[Message] = session_context.get_messages(include_input=True)
workflow_input_messages = input_messages if bool(self.pending_requests) else session_messages
output_events: list[WorkflowEvent[Any]] = []
async for event in self._run_core(
session_messages,
workflow_input_messages,
checkpoint_id,
checkpoint_storage,
streaming=False,
@@ -311,6 +315,7 @@ class WorkflowAgent(BaseAgent):
session_context._response = result # type: ignore[assignment]
await self._run_after_providers(session=provider_session, context=session_context)
self._persist_pending_requests_to_session(session)
return result
async def _run_stream_impl(
@@ -338,6 +343,7 @@ class WorkflowAgent(BaseAgent):
AgentResponseUpdate objects representing the workflow execution progress.
"""
input_messages = normalize_messages_input(messages)
self._restore_pending_requests_from_session(session)
if (
not any(
@@ -372,9 +378,10 @@ class WorkflowAgent(BaseAgent):
# combine the messages
session_messages: list[Message] = session_context.get_messages(include_input=True)
workflow_input_messages = input_messages if bool(self.pending_requests) else session_messages
all_updates: list[AgentResponseUpdate] = []
async for event in self._run_core(
session_messages,
workflow_input_messages,
checkpoint_id,
checkpoint_storage,
streaming=True,
@@ -392,6 +399,7 @@ class WorkflowAgent(BaseAgent):
session_context._response = AgentResponse.from_updates(all_updates) # type: ignore[assignment]
await self._run_after_providers(session=provider_session, context=session_context)
self._persist_pending_requests_to_session(session)
async def _run_core(
self,
@@ -425,6 +433,8 @@ class WorkflowAgent(BaseAgent):
async for event in self.workflow.run(
responses=function_responses,
stream=True,
checkpoint_id=checkpoint_id,
checkpoint_storage=checkpoint_storage,
function_invocation_kwargs=function_invocation_kwargs,
client_kwargs=client_kwargs,
):
@@ -432,6 +442,8 @@ class WorkflowAgent(BaseAgent):
else:
for event in await self.workflow.run(
responses=function_responses,
checkpoint_id=checkpoint_id,
checkpoint_storage=checkpoint_storage,
function_invocation_kwargs=function_invocation_kwargs,
client_kwargs=client_kwargs,
):
@@ -484,6 +496,60 @@ class WorkflowAgent(BaseAgent):
# endregion Run Methods
def _restore_pending_requests_from_session(self, session: AgentSession | None) -> None:
"""Load pending request-info events from the session state."""
if session is None:
return
agent_state = session.state.get(self._SESSION_STATE_KEY)
if not isinstance(agent_state, dict):
self.pending_requests.clear()
return
pending_requests_payload = agent_state.get(self._PENDING_REQUESTS_STATE_KEY)
if not isinstance(pending_requests_payload, dict):
self.pending_requests.clear()
return
restored_pending: dict[str, WorkflowEvent[Any]] = {}
for request_id, request_payload in pending_requests_payload.items():
if isinstance(request_payload, WorkflowEvent):
restored_pending[request_id] = request_payload
continue
if not isinstance(request_payload, dict):
logger.warning("Skipping malformed pending request payload for request_id '%s'.", request_id)
continue
try:
restored_pending[request_id] = WorkflowEvent.from_dict(request_payload)
except Exception as exc: # pragma: no cover - defensive
logger.warning(
"Failed to restore pending request payload for request_id '%s': %s",
request_id,
exc,
)
self.pending_requests.clear()
self.pending_requests.update(restored_pending)
def _persist_pending_requests_to_session(self, session: AgentSession | None) -> None:
"""Persist pending request-info events to the session state."""
if session is None:
return
agent_state = session.state.setdefault(self._SESSION_STATE_KEY, {})
if not isinstance(agent_state, dict):
logger.warning(
"Skipping pending request persistence because '%s' is not a mapping.",
self._SESSION_STATE_KEY,
)
return
agent_state[self._PENDING_REQUESTS_STATE_KEY] = {
request_id: event.to_dict() for request_id, event in self.pending_requests.items()
}
def _process_pending_requests(self, input_messages: Sequence[Message]) -> dict[str, Any]:
"""Process pending requests by extracting function responses and updating state.
@@ -41,6 +41,7 @@ _IMPORTS: dict[str, tuple[str, str]] = {
"RawFoundryEmbeddingClient": ("agent_framework_foundry", "agent-framework-foundry"),
"evaluate_foundry_target": ("agent_framework_foundry", "agent-framework-foundry"),
"evaluate_traces": ("agent_framework_foundry", "agent-framework-foundry"),
"to_prompt_agent": ("agent_framework_foundry", "agent-framework-foundry"),
}
@@ -26,6 +26,7 @@ from agent_framework_foundry import (
RawFoundryEmbeddingClient,
evaluate_foundry_target,
evaluate_traces,
to_prompt_agent,
)
from agent_framework_foundry_local import (
FoundryLocalChatOptions,
@@ -58,4 +59,5 @@ __all__ = [
"RawFoundryEmbeddingClient",
"evaluate_foundry_target",
"evaluate_traces",
"to_prompt_agent",
]
@@ -19,6 +19,7 @@ from agent_framework import (
ChatResponse,
CompactionProvider,
Content,
ContextWindowCompactionStrategy,
Message,
SelectiveToolCallCompactionStrategy,
SlidingWindowStrategy,
@@ -952,3 +953,159 @@ async def test_in_memory_history_provider_default_loads_all() -> None:
loaded = await provider.get_messages(session_id="test", state=state)
assert len(loaded) == 3
# --- ContextWindowCompactionStrategy tests ---
async def test_context_window_strategy_noop_under_threshold() -> None:
"""No compaction when total tokens are below 50% of input budget."""
# input_budget = 1000 - 200 = 800; tool eviction threshold = 50% = 400 tokens
# CharacterEstimatorTokenizer: 4 chars/token
# Each short message ~4-5 tokens, total well under 400
messages = [
Message(role="system", contents=["sys"]),
Message(role="user", contents=["hello"]),
Message(role="assistant", contents=["hi"]),
]
strategy = ContextWindowCompactionStrategy(
max_context_window_tokens=1000,
max_output_tokens=200,
)
changed = await strategy(messages)
assert changed is False
assert len(included_messages(messages)) == 3
async def test_context_window_strategy_tool_eviction_triggers_at_threshold() -> None:
"""Tool eviction fires when tokens exceed 50% but truncation does not."""
# input_budget = 20000 - 200 = 19800
# tool eviction at 50% = 9900 tokens; truncation at 80% = 15840 tokens
# CharacterEstimatorTokenizer: 4 chars/token
# Each tool result: "x" * 8000 = 8000 chars = 2000 tokens
# 5 groups * ~2000 = ~10000+ tokens (exceeds 9900, under 15840)
# Tool eviction collapses older groups; truncation threshold not reached.
messages = [
Message(role="system", contents=["system prompt"]),
Message(role="user", contents=["u1"]),
_assistant_function_call("c1"),
_tool_result("c1", "x" * 8000),
Message(role="user", contents=["u2"]),
_assistant_function_call("c2"),
_tool_result("c2", "x" * 8000),
Message(role="user", contents=["u3"]),
_assistant_function_call("c3"),
_tool_result("c3", "x" * 8000),
Message(role="user", contents=["u4"]),
_assistant_function_call("c4"),
_tool_result("c4", "x" * 8000),
Message(role="user", contents=["u5"]),
_assistant_function_call("c5"),
_tool_result("c5", "x" * 8000),
]
strategy = ContextWindowCompactionStrategy(
max_context_window_tokens=20000,
max_output_tokens=200,
keep_last_tool_call_groups=2,
)
changed = await strategy(messages)
assert changed is True
projected = included_messages(messages)
# Verify that tool results were compacted (summary messages present).
summary_msgs = [m for m in projected if m.text and "[Tool results:" in m.text]
assert len(summary_msgs) > 0
# Verify that the truncation phase did NOT fire — no messages excluded with "truncation" reason.
from agent_framework._compaction import EXCLUDE_REASON_KEY
truncation_excluded = [m for m in messages if m.additional_properties.get(EXCLUDE_REASON_KEY) == "truncation"]
assert len(truncation_excluded) == 0
async def test_context_window_strategy_truncation_triggers_above_80_pct() -> None:
"""Truncation fires when tokens exceed 80% of input budget."""
# input_budget = 1000 - 100 = 900
# tool eviction at 50% = 450 tokens; truncation at 80% = 720 tokens
# We'll create messages with no tool calls (so tool eviction does nothing)
# but exceeding 720 tokens total (>2880 chars)
messages = [
Message(role="system", contents=["sys"]),
Message(role="user", contents=["u1 " * 400]), # ~1200 chars = 300 tokens
Message(role="assistant", contents=["a1 " * 400]), # ~1200 chars = 300 tokens
Message(role="user", contents=["u2 " * 400]), # ~1200 chars = 300 tokens
Message(role="assistant", contents=["a2 " * 400]), # ~1200 chars = 300 tokens
]
strategy = ContextWindowCompactionStrategy(
max_context_window_tokens=1000,
max_output_tokens=100,
)
changed = await strategy(messages)
assert changed is True
projected = included_messages(messages)
# System message should always be preserved
assert projected[0].role == "system"
# Some messages should have been excluded
assert len(projected) < 5
async def test_context_window_strategy_keep_last_tool_call_groups_respected() -> None:
"""The keep_last_tool_call_groups parameter controls how many groups are retained."""
# Create enough tokens to trigger tool eviction (>50% of input budget)
# input_budget = 1000 - 100 = 900; threshold = 450 tokens
messages = [
Message(role="system", contents=["sys"]),
Message(role="user", contents=["u1"]),
_assistant_function_call("c1"),
_tool_result("c1", "r1 " * 200),
Message(role="user", contents=["u2"]),
_assistant_function_call("c2"),
_tool_result("c2", "r2 " * 200),
Message(role="user", contents=["u3"]),
_assistant_function_call("c3"),
_tool_result("c3", "r3 " * 200),
]
# keep_last_tool_call_groups=1: only the last group (c3) should be kept verbatim
strategy = ContextWindowCompactionStrategy(
max_context_window_tokens=1000,
max_output_tokens=100,
keep_last_tool_call_groups=1,
)
changed = await strategy(messages)
assert changed is True
projected = included_messages(messages)
# The last tool call group (c3) should be in the projected messages
has_c3 = any(
c.call_id == "c3" for m in projected for c in m.contents if c.type in ("function_call", "function_result")
)
assert has_c3
def test_context_window_strategy_validates_thresholds() -> None:
"""Invalid threshold combinations raise ValueError."""
import pytest
with pytest.raises(ValueError, match="max_context_window_tokens must be positive"):
ContextWindowCompactionStrategy(max_context_window_tokens=0, max_output_tokens=0)
with pytest.raises(ValueError, match="max_output_tokens must be >= 0"):
ContextWindowCompactionStrategy(max_context_window_tokens=1000, max_output_tokens=1000)
with pytest.raises(ValueError, match="tool_eviction_threshold must be in"):
ContextWindowCompactionStrategy(
max_context_window_tokens=1000, max_output_tokens=100, tool_eviction_threshold=0.0
)
with pytest.raises(ValueError, match="truncation_threshold must be >= tool_eviction_threshold"):
ContextWindowCompactionStrategy(
max_context_window_tokens=1000,
max_output_tokens=100,
tool_eviction_threshold=0.8,
truncation_threshold=0.5,
)
@@ -0,0 +1,396 @@
# Copyright (c) Microsoft. All rights reserved.
from __future__ import annotations
from collections.abc import AsyncIterator, Mapping
from typing import Any
import pytest
from agent_framework import (
AgentSession,
ChatResponse,
CompactionProvider,
InMemoryHistoryProvider,
Message,
SkillsProvider,
TodoProvider,
create_harness_agent,
)
from agent_framework._harness._agent import DEFAULT_HARNESS_INSTRUCTIONS, _assemble_instructions
from agent_framework._harness._mode import AgentModeProvider
from agent_framework._sessions import ContextProvider
class _FakeChatClient:
"""Minimal chat client stub for testing assembly."""
model = "test-model"
async def get_response(
self,
*,
messages: list[Message],
options: Mapping[str, Any] | None = None,
**kwargs: Any,
) -> ChatResponse:
return ChatResponse(messages=[Message(role="assistant", contents=["Hello"])])
async def get_streaming_response(
self,
*,
messages: list[Message],
options: Mapping[str, Any] | None = None,
**kwargs: Any,
) -> AsyncIterator[Any]:
yield Message(role="assistant", contents=["Hello"]) # pragma: no cover
# --- Assembly Tests ---
def test_create_harness_agent_with_defaults() -> None:
"""create_harness_agent should assemble successfully with default options."""
agent = create_harness_agent(
client=_FakeChatClient(), # type: ignore[arg-type]
max_context_window_tokens=128_000,
max_output_tokens=16_384,
)
assert agent.id is not None
def test_create_harness_agent_includes_all_default_providers() -> None:
"""Default assembly should include history, compaction, todo, mode (no skills by default)."""
agent = create_harness_agent(
client=_FakeChatClient(), # type: ignore[arg-type]
max_context_window_tokens=128_000,
max_output_tokens=16_384,
)
providers = agent.context_providers
provider_types = [type(p) for p in providers]
assert InMemoryHistoryProvider in provider_types
assert CompactionProvider in provider_types
assert TodoProvider in provider_types
assert AgentModeProvider in provider_types
assert SkillsProvider not in provider_types
def test_create_harness_agent_disable_todo() -> None:
"""disable_todo=True should exclude TodoProvider."""
agent = create_harness_agent(
client=_FakeChatClient(), # type: ignore[arg-type]
max_context_window_tokens=128_000,
max_output_tokens=16_384,
disable_todo=True,
)
provider_types = [type(p) for p in agent.context_providers]
assert TodoProvider not in provider_types
def test_create_harness_agent_disable_mode() -> None:
"""disable_mode=True should exclude AgentModeProvider."""
agent = create_harness_agent(
client=_FakeChatClient(), # type: ignore[arg-type]
max_context_window_tokens=128_000,
max_output_tokens=16_384,
disable_mode=True,
)
provider_types = [type(p) for p in agent.context_providers]
assert AgentModeProvider not in provider_types
def test_create_harness_agent_disable_memory() -> None:
"""disable_memory=True should exclude MemoryContextProvider even when memory_store is provided."""
from agent_framework import MemoryContextProvider
from agent_framework._harness._memory import MemoryStore
class _FakeMemoryStore(MemoryStore):
def list_topics(self, session, *, source_id):
return []
def get_topic(self, session, *, source_id, topic):
raise NotImplementedError
def write_topic(self, session, record, *, source_id):
pass
def delete_topic(self, session, *, source_id, topic):
pass
def get_index_text(self, session, *, source_id):
return ""
def get_transcripts_directory(self, session, *, source_id):
return ""
def read_state(self, session, *, source_id):
return {}
def rebuild_index(self, session, *, source_id):
pass
def search_transcripts(self, session, *, source_id, query):
return []
def write_state(self, session, state, *, source_id):
pass
# With memory_store provided and disable_memory=False, MemoryContextProvider should be present.
agent_with_memory = create_harness_agent(
client=_FakeChatClient(), # type: ignore[arg-type]
max_context_window_tokens=128_000,
max_output_tokens=16_384,
memory_store=_FakeMemoryStore(),
)
provider_types = [type(p) for p in agent_with_memory.context_providers]
assert MemoryContextProvider in provider_types
# With memory_store provided and disable_memory=True, MemoryContextProvider should be absent.
agent_disabled = create_harness_agent(
client=_FakeChatClient(), # type: ignore[arg-type]
max_context_window_tokens=128_000,
max_output_tokens=16_384,
memory_store=_FakeMemoryStore(),
disable_memory=True,
)
provider_types = [type(p) for p in agent_disabled.context_providers]
assert MemoryContextProvider not in provider_types
def test_create_harness_agent_skills_paths_adds_provider() -> None:
"""skills_paths should add a SkillsProvider."""
agent = create_harness_agent(
client=_FakeChatClient(), # type: ignore[arg-type]
max_context_window_tokens=128_000,
max_output_tokens=16_384,
skills_paths=["./test-skills"],
)
provider_types = [type(p) for p in agent.context_providers]
assert SkillsProvider in provider_types
def test_create_harness_agent_disable_compaction() -> None:
"""disable_compaction=True should exclude CompactionProvider."""
agent = create_harness_agent(
client=_FakeChatClient(), # type: ignore[arg-type]
max_context_window_tokens=128_000,
max_output_tokens=16_384,
disable_compaction=True,
)
provider_types = [type(p) for p in agent.context_providers]
assert CompactionProvider not in provider_types
def test_create_harness_agent_returns_full_agent() -> None:
"""Factory should return an Agent instance (with telemetry)."""
from agent_framework._agents import Agent as FullAgent
agent = create_harness_agent(
client=_FakeChatClient(), # type: ignore[arg-type]
max_context_window_tokens=128_000,
max_output_tokens=16_384,
)
assert isinstance(agent, FullAgent)
# --- Validation Tests ---
def test_create_harness_agent_rejects_invalid_context_tokens() -> None:
"""max_context_window_tokens must be positive."""
with pytest.raises(ValueError, match="max_context_window_tokens must be positive"):
create_harness_agent(
client=_FakeChatClient(), # type: ignore[arg-type]
max_context_window_tokens=0,
max_output_tokens=100,
)
def test_create_harness_agent_rejects_negative_output_tokens() -> None:
"""max_output_tokens must be non-negative."""
with pytest.raises(ValueError, match="max_output_tokens must be non-negative"):
create_harness_agent(
client=_FakeChatClient(), # type: ignore[arg-type]
max_context_window_tokens=1000,
max_output_tokens=-1,
)
def test_create_harness_agent_rejects_output_gte_context() -> None:
"""max_output_tokens must be less than max_context_window_tokens."""
with pytest.raises(ValueError, match="max_output_tokens must be less than"):
create_harness_agent(
client=_FakeChatClient(), # type: ignore[arg-type]
max_context_window_tokens=1000,
max_output_tokens=1000,
)
# --- Instructions Tests ---
def test_default_instructions() -> None:
"""None args should produce default harness instructions."""
result = _assemble_instructions(None, None)
assert result == DEFAULT_HARNESS_INSTRUCTIONS.strip()
def test_custom_agent_instructions_appended() -> None:
"""Agent instructions should be appended after harness instructions."""
result = _assemble_instructions(None, "Focus on code review.")
assert DEFAULT_HARNESS_INSTRUCTIONS in result # type: ignore[operator]
assert "Focus on code review." in result # type: ignore[operator]
def test_empty_harness_instructions_uses_agent_only() -> None:
"""Empty harness_instructions should return agent instructions only."""
result = _assemble_instructions("", "Custom only.")
assert result == "Custom only."
# --- Identity Tests ---
def test_create_harness_agent_custom_identity() -> None:
"""Custom id, name, description should propagate."""
agent = create_harness_agent(
client=_FakeChatClient(), # type: ignore[arg-type]
max_context_window_tokens=128_000,
max_output_tokens=16_384,
id="my-agent-id",
name="my-agent",
description="A test agent",
)
assert agent.id == "my-agent-id"
assert agent.name == "my-agent"
assert agent.description == "A test agent"
# --- Session Tests ---
def test_create_harness_agent_create_session() -> None:
"""create_session should return an AgentSession."""
agent = create_harness_agent(
client=_FakeChatClient(), # type: ignore[arg-type]
max_context_window_tokens=128_000,
max_output_tokens=16_384,
)
session = agent.create_session()
assert isinstance(session, AgentSession)
def test_create_harness_agent_create_session_with_id() -> None:
"""create_session should accept a custom session_id."""
agent = create_harness_agent(
client=_FakeChatClient(), # type: ignore[arg-type]
max_context_window_tokens=128_000,
max_output_tokens=16_384,
)
session = agent.create_session(session_id="custom-id")
assert session.session_id == "custom-id"
async def test_create_harness_agent_run_returns_response() -> None:
"""agent.run() should return a response."""
agent = create_harness_agent(
client=_FakeChatClient(), # type: ignore[arg-type]
max_context_window_tokens=128_000,
max_output_tokens=16_384,
)
session = agent.create_session()
response = await agent.run("hello", session=session)
assert response.messages
assert response.messages[-1].role == "assistant"
# --- Protocol Tests ---
def test_create_harness_agent_satisfies_protocol() -> None:
"""Returned agent should satisfy SupportsAgentRun protocol."""
from agent_framework import SupportsAgentRun
agent = create_harness_agent(
client=_FakeChatClient(), # type: ignore[arg-type]
max_context_window_tokens=128_000,
max_output_tokens=16_384,
)
assert isinstance(agent, SupportsAgentRun)
# --- Additional providers ---
def test_create_harness_agent_extra_context_providers() -> None:
"""Additional context_providers should be appended."""
class _CustomProvider(ContextProvider):
pass
custom = _CustomProvider("custom")
agent = create_harness_agent(
client=_FakeChatClient(), # type: ignore[arg-type]
max_context_window_tokens=128_000,
max_output_tokens=16_384,
context_providers=[custom],
)
assert custom in agent.context_providers
# --- Web Search Tool Tests ---
class _FakeWebSearchClient(_FakeChatClient):
"""Fake client that supports web search tool."""
def get_web_search_tool(self, **kwargs: Any) -> str:
return "web_search_tool_instance"
def test_create_harness_agent_auto_adds_web_search_tool() -> None:
"""Web search tool should be auto-added when client supports it."""
agent = create_harness_agent(
client=_FakeWebSearchClient(), # type: ignore[arg-type]
max_context_window_tokens=128_000,
max_output_tokens=16_384,
)
tools = agent.default_options.get("tools", [])
assert "web_search_tool_instance" in tools
def test_create_harness_agent_disable_web_search() -> None:
"""disable_web_search=True should skip auto-adding the web search tool."""
agent = create_harness_agent(
client=_FakeWebSearchClient(), # type: ignore[arg-type]
max_context_window_tokens=128_000,
max_output_tokens=16_384,
disable_web_search=True,
)
tools = agent.default_options.get("tools", [])
assert "web_search_tool_instance" not in tools
def test_create_harness_agent_no_web_search_when_unsupported() -> None:
"""Web search tool should NOT be added when client does not support it."""
agent = create_harness_agent(
client=_FakeChatClient(), # type: ignore[arg-type]
max_context_window_tokens=128_000,
max_output_tokens=16_384,
)
tools = agent.default_options.get("tools", [])
assert "web_search_tool_instance" not in tools
def test_create_harness_agent_logs_warning_when_no_web_search(caplog: pytest.LogCaptureFixture) -> None:
"""A warning should be logged when client doesn't support web search."""
import logging
with caplog.at_level(logging.WARNING, logger="agent_framework._harness._agent"):
create_harness_agent(
client=_FakeChatClient(), # type: ignore[arg-type]
max_context_window_tokens=128_000,
max_output_tokens=16_384,
)
assert any("SupportsWebSearchTool" in msg for msg in caplog.messages)
@@ -0,0 +1,538 @@
# Copyright (c) Microsoft. All rights reserved.
from __future__ import annotations
import asyncio
from typing import Any
import pytest
from agent_framework import (
AgentResponse,
AgentSession,
BackgroundAgentsProvider,
BackgroundTaskInfo,
BackgroundTaskStatus,
Message,
)
from agent_framework._sessions import SessionContext
# Suppress "coroutine was never awaited" warnings from task cancellation in tests.
# This occurs when cancelling tasks that wrap coroutines through _run_agent().
pytestmark = pytest.mark.filterwarnings("ignore::RuntimeWarning:asyncio")
# --- Test Helpers ---
class _FakeAgent:
"""Minimal agent stub for testing background agent delegation."""
def __init__(
self,
name: str,
description: str | None = None,
*,
response_text: str = "done",
delay: float = 0.0,
should_fail: bool = False,
):
self.id = f"agent-{name}"
self.name = name
self.description = description
self._response_text = response_text
self._delay = delay
self._should_fail = should_fail
def create_session(self, *, session_id: str | None = None) -> AgentSession:
return AgentSession(session_id=session_id)
def get_session(self, service_session_id: str, *, session_id: str | None = None) -> AgentSession:
return AgentSession(service_session_id=service_session_id, session_id=session_id)
async def run(
self, messages: Any = None, *, stream: bool = False, session: Any = None, **kwargs: Any
) -> AgentResponse[Any]:
if self._delay > 0:
await asyncio.sleep(self._delay)
if self._should_fail:
raise RuntimeError("Agent execution failed")
return AgentResponse(messages=[Message(role="assistant", contents=[self._response_text])])
def _make_provider(*agents: _FakeAgent) -> BackgroundAgentsProvider:
"""Create a provider with given agents."""
return BackgroundAgentsProvider(agents)
def _make_session() -> AgentSession:
"""Create a session for testing."""
return AgentSession()
async def _get_tools(provider: BackgroundAgentsProvider, session: AgentSession) -> dict[str, Any]:
"""Run before_run and return tools by name."""
context = SessionContext(input_messages=[])
await provider.before_run(agent=None, session=session, context=context, state={})
tools_by_name: dict[str, Any] = {}
for t in context.tools:
tools_by_name[t.name if hasattr(t, "name") else str(t)] = t
return tools_by_name
async def _invoke_tool(tool_obj: Any, **kwargs: Any) -> str:
"""Invoke a FunctionTool and return the raw result string."""
return await tool_obj.invoke(arguments=kwargs, skip_parsing=True)
# --- Constructor Tests ---
def test_constructor_requires_at_least_one_agent() -> None:
"""Should reject empty agent list."""
with pytest.raises(ValueError, match="At least one background agent"):
BackgroundAgentsProvider([])
def test_constructor_requires_agent_names() -> None:
"""Should reject agents with no name."""
agent = _FakeAgent("")
with pytest.raises(ValueError, match="non-empty name"):
BackgroundAgentsProvider([agent])
def test_constructor_rejects_duplicate_names() -> None:
"""Should reject duplicate agent names (case-insensitive)."""
agent1 = _FakeAgent("Research")
agent2 = _FakeAgent("research")
with pytest.raises(ValueError, match="Duplicate background agent name"):
BackgroundAgentsProvider([agent1, agent2])
def test_constructor_valid_agents() -> None:
"""Should succeed with valid unique agents."""
provider = BackgroundAgentsProvider([_FakeAgent("Alpha"), _FakeAgent("Beta")])
assert provider.source_id == "background_agents"
def test_constructor_custom_source_id() -> None:
"""Should accept custom source_id."""
provider = BackgroundAgentsProvider([_FakeAgent("Agent1")], source_id="custom_bg")
assert provider.source_id == "custom_bg"
# --- Tool Injection Tests ---
async def test_before_run_injects_six_tools() -> None:
"""before_run should inject exactly 6 tools."""
provider = _make_provider(_FakeAgent("Worker"))
tools = await _get_tools(provider, _make_session())
assert len(tools) == 6
expected_names = {
"background_agents_start_task",
"background_agents_wait_for_first_completion",
"background_agents_get_task_results",
"background_agents_get_all_tasks",
"background_agents_continue_task",
"background_agents_clear_completed_task",
}
assert set(tools.keys()) == expected_names
async def test_before_run_injects_instructions() -> None:
"""before_run should inject instructions mentioning agent names."""
provider = _make_provider(_FakeAgent("ResearchBot", "Does research"))
context = SessionContext(input_messages=[])
session = _make_session()
await provider.before_run(agent=None, session=session, context=context, state={})
all_instructions = " ".join(context.instructions)
assert "ResearchBot" in all_instructions
assert "Does research" in all_instructions
# --- Start Task Tests ---
async def test_start_task_success() -> None:
"""Should start a task and return confirmation."""
provider = _make_provider(_FakeAgent("Worker", response_text="result"))
session = _make_session()
tools = await _get_tools(provider, session)
result = await _invoke_tool(
tools["background_agents_start_task"],
agent_name="Worker",
input="do something",
description="test task",
)
assert "task 1 started" in result.lower()
assert "Worker" in result
async def test_start_task_unknown_agent() -> None:
"""Should return error for unknown agent name."""
provider = _make_provider(_FakeAgent("Worker"))
session = _make_session()
tools = await _get_tools(provider, session)
result = await _invoke_tool(
tools["background_agents_start_task"],
agent_name="NonExistent",
input="do something",
description="test",
)
assert "Error" in result
assert "NonExistent" in result
async def test_start_task_increments_ids() -> None:
"""Task IDs should increment sequentially."""
provider = _make_provider(_FakeAgent("Worker"))
session = _make_session()
tools = await _get_tools(provider, session)
r1 = await _invoke_tool(
tools["background_agents_start_task"],
agent_name="Worker",
input="task 1",
description="first",
)
r2 = await _invoke_tool(
tools["background_agents_start_task"],
agent_name="Worker",
input="task 2",
description="second",
)
assert "task 1 started" in r1.lower()
assert "task 2 started" in r2.lower()
# --- Get All Tasks Tests ---
async def test_get_all_tasks_empty() -> None:
"""Should return 'No tasks.' when no tasks exist."""
provider = _make_provider(_FakeAgent("Worker"))
session = _make_session()
tools = await _get_tools(provider, session)
result = await _invoke_tool(tools["background_agents_get_all_tasks"])
assert "No tasks" in result
async def test_get_all_tasks_shows_tasks() -> None:
"""Should list all tasks with status and description."""
provider = _make_provider(_FakeAgent("Worker"))
session = _make_session()
tools = await _get_tools(provider, session)
await _invoke_tool(
tools["background_agents_start_task"],
agent_name="Worker",
input="hello",
description="my task",
)
result = await _invoke_tool(tools["background_agents_get_all_tasks"])
assert "my task" in result
assert "Worker" in result
# --- Wait for Completion Tests ---
async def test_wait_for_first_completion() -> None:
"""Should wait and return when a task completes."""
provider = _make_provider(_FakeAgent("Fast", response_text="fast result", delay=0.01))
session = _make_session()
tools = await _get_tools(provider, session)
await _invoke_tool(
tools["background_agents_start_task"],
agent_name="Fast",
input="go",
description="fast task",
)
result = await _invoke_tool(
tools["background_agents_wait_for_first_completion"],
task_ids=[1],
)
assert "finished" in result.lower()
assert "completed" in result.lower()
async def test_wait_empty_task_ids() -> None:
"""Should return error for empty task_ids."""
provider = _make_provider(_FakeAgent("Worker"))
session = _make_session()
tools = await _get_tools(provider, session)
result = await _invoke_tool(
tools["background_agents_wait_for_first_completion"],
task_ids=[],
)
assert "Error" in result
async def test_wait_no_running_tasks() -> None:
"""Should return error when no specified tasks are running."""
provider = _make_provider(_FakeAgent("Worker"))
session = _make_session()
tools = await _get_tools(provider, session)
result = await _invoke_tool(
tools["background_agents_wait_for_first_completion"],
task_ids=[999],
)
assert "Error" in result or "not running" in result.lower()
# --- Get Task Results Tests ---
async def test_get_task_results_completed() -> None:
"""Should return result text for completed task."""
provider = _make_provider(_FakeAgent("Worker", response_text="the answer", delay=0.01))
session = _make_session()
tools = await _get_tools(provider, session)
await _invoke_tool(
tools["background_agents_start_task"],
agent_name="Worker",
input="query",
description="test",
)
# Wait for completion.
await _invoke_tool(
tools["background_agents_wait_for_first_completion"],
task_ids=[1],
)
result = await _invoke_tool(
tools["background_agents_get_task_results"],
task_id=1,
)
assert result == "the answer"
async def test_get_task_results_running() -> None:
"""Should indicate task is still running."""
provider = _make_provider(_FakeAgent("Slow", delay=10.0))
session = _make_session()
tools = await _get_tools(provider, session)
await _invoke_tool(
tools["background_agents_start_task"],
agent_name="Slow",
input="query",
description="slow task",
)
try:
result = await _invoke_tool(
tools["background_agents_get_task_results"],
task_id=1,
)
assert "still running" in result.lower()
finally:
runtime = provider._get_runtime(session)
for task in list(runtime.in_flight_tasks.values()):
task.cancel()
await asyncio.gather(*runtime.in_flight_tasks.values(), return_exceptions=True)
async def test_get_task_results_failed() -> None:
"""Should return error text for failed task."""
provider = _make_provider(_FakeAgent("Broken", should_fail=True, delay=0.01))
session = _make_session()
tools = await _get_tools(provider, session)
await _invoke_tool(
tools["background_agents_start_task"],
agent_name="Broken",
input="query",
description="will fail",
)
await _invoke_tool(
tools["background_agents_wait_for_first_completion"],
task_ids=[1],
)
result = await _invoke_tool(
tools["background_agents_get_task_results"],
task_id=1,
)
assert "failed" in result.lower()
async def test_get_task_results_not_found() -> None:
"""Should return error for non-existent task."""
provider = _make_provider(_FakeAgent("Worker"))
session = _make_session()
tools = await _get_tools(provider, session)
result = await _invoke_tool(
tools["background_agents_get_task_results"],
task_id=999,
)
assert "Error" in result
# --- Continue Task Tests ---
async def test_continue_task_after_completion() -> None:
"""Should be able to continue a completed task."""
provider = _make_provider(_FakeAgent("Worker", response_text="first result", delay=0.01))
session = _make_session()
tools = await _get_tools(provider, session)
await _invoke_tool(
tools["background_agents_start_task"],
agent_name="Worker",
input="first input",
description="continuable",
)
await _invoke_tool(
tools["background_agents_wait_for_first_completion"],
task_ids=[1],
)
result = await _invoke_tool(
tools["background_agents_continue_task"],
task_id=1,
text="follow up",
)
assert "continued" in result.lower()
async def test_continue_task_still_running() -> None:
"""Should return error if task is still running."""
provider = _make_provider(_FakeAgent("Slow", delay=10.0))
session = _make_session()
tools = await _get_tools(provider, session)
await _invoke_tool(
tools["background_agents_start_task"],
agent_name="Slow",
input="input",
description="running",
)
try:
result = await _invoke_tool(
tools["background_agents_continue_task"],
task_id=1,
text="follow up",
)
assert "still running" in result.lower()
finally:
runtime = provider._get_runtime(session)
for task in list(runtime.in_flight_tasks.values()):
task.cancel()
await asyncio.gather(*runtime.in_flight_tasks.values(), return_exceptions=True)
async def test_continue_task_not_found() -> None:
"""Should return error for non-existent task."""
provider = _make_provider(_FakeAgent("Worker"))
session = _make_session()
tools = await _get_tools(provider, session)
result = await _invoke_tool(
tools["background_agents_continue_task"],
task_id=999,
text="hello",
)
assert "Error" in result
# --- Clear Task Tests ---
async def test_clear_completed_task() -> None:
"""Should clear a completed task."""
provider = _make_provider(_FakeAgent("Worker", response_text="done", delay=0.01))
session = _make_session()
tools = await _get_tools(provider, session)
await _invoke_tool(
tools["background_agents_start_task"],
agent_name="Worker",
input="task",
description="clearable",
)
await _invoke_tool(
tools["background_agents_wait_for_first_completion"],
task_ids=[1],
)
result = await _invoke_tool(
tools["background_agents_clear_completed_task"],
task_id=1,
)
assert "cleared" in result.lower()
# Verify task is gone.
all_tasks = await _invoke_tool(tools["background_agents_get_all_tasks"])
assert "No tasks" in all_tasks
async def test_clear_running_task_error() -> None:
"""Should return error when clearing a running task."""
provider = _make_provider(_FakeAgent("Slow", delay=10.0))
session = _make_session()
tools = await _get_tools(provider, session)
await _invoke_tool(
tools["background_agents_start_task"],
agent_name="Slow",
input="task",
description="still going",
)
try:
result = await _invoke_tool(
tools["background_agents_clear_completed_task"],
task_id=1,
)
assert "still running" in result.lower()
finally:
runtime = provider._get_runtime(session)
for task in list(runtime.in_flight_tasks.values()):
task.cancel()
await asyncio.gather(*runtime.in_flight_tasks.values(), return_exceptions=True)
async def test_clear_not_found() -> None:
"""Should return error for non-existent task."""
provider = _make_provider(_FakeAgent("Worker"))
session = _make_session()
tools = await _get_tools(provider, session)
result = await _invoke_tool(
tools["background_agents_clear_completed_task"],
task_id=999,
)
assert "Error" in result
# --- BackgroundTaskInfo Tests ---
def test_task_info_serialization() -> None:
"""BackgroundTaskInfo should round-trip through to_dict/from_dict."""
info = BackgroundTaskInfo(
id=1,
agent_name="Worker",
description="test task",
status=BackgroundTaskStatus.COMPLETED,
result_text="hello",
)
data = info.to_dict()
restored = BackgroundTaskInfo.from_dict(data)
assert restored.id == 1
assert restored.agent_name == "Worker"
assert restored.status == BackgroundTaskStatus.COMPLETED
assert restored.result_text == "hello"
assert restored.error_text is None
def test_task_status_enum_values() -> None:
"""BackgroundTaskStatus should have expected values."""
assert BackgroundTaskStatus.RUNNING == "running"
assert BackgroundTaskStatus.COMPLETED == "completed"
assert BackgroundTaskStatus.FAILED == "failed"
assert BackgroundTaskStatus.LOST == "lost"
@@ -95,8 +95,10 @@ async def test_agent_mode_context_provider_normalizes_custom_modes(
)
instructions = options["instructions"]
assert isinstance(instructions, str)
assert '"Draft": Draft it.' in instructions
assert '"Final": Finalize it.' in instructions
assert "#### Draft" in instructions
assert "Draft it." in instructions
assert "#### Final" in instructions
assert "Finalize it." in instructions
assert "You are currently operating in the draft mode." in instructions
assert (
@@ -125,8 +127,8 @@ async def test_agent_mode_context_provider_serializes_tool_outputs_as_json(
)
tools = options["tools"]
assert isinstance(tools, list)
get_mode_tool = _tool_by_name(tools, "get_mode")
set_mode_tool = _tool_by_name(tools, "set_mode")
get_mode_tool = _tool_by_name(tools, "mode_get")
set_mode_tool = _tool_by_name(tools, "mode_set")
initial_mode = await get_mode_tool.invoke()
assert json.loads(initial_mode[0].text) == {"mode": mode_name}
@@ -152,13 +154,13 @@ async def test_agent_mode_context_provider_updates_agent_mode(
instructions = options["instructions"]
assert isinstance(instructions, str)
assert "## Agent Mode" in instructions
assert "Use the set_mode tool to switch between modes as your work progresses." in instructions
assert "Use the mode_set tool to switch between modes as your work progresses." in instructions
assert "ask clarifying questions, discuss options, and get user approval before proceeding" in instructions
assert "If you encounter ambiguity, choose the most reasonable option and note your choice" in instructions
assert "If you encounter ambiguity" in instructions
assert "You are currently operating in the plan mode." in instructions
get_mode_tool = _tool_by_name(tools, "get_mode")
set_mode_tool = _tool_by_name(tools, "set_mode")
get_mode_tool = _tool_by_name(tools, "mode_get")
set_mode_tool = _tool_by_name(tools, "mode_set")
initial_mode = await get_mode_tool.invoke()
assert json.loads(initial_mode[0].text) == {"mode": "plan"}
@@ -218,13 +220,13 @@ async def test_agent_mode_provider_injects_user_message_after_external_change(
provider = AgentModeProvider()
agent = Agent(client=chat_client_base, context_providers=[provider])
# First run: agent uses set_mode tool to switch to execute. The tool path must NOT queue a
# First run: agent uses mode_set tool to switch to execute. The tool path must NOT queue a
# notification because the agent already saw its own tool call in the chat history.
_, first_options = await agent._prepare_session_and_messages( # type: ignore[reportPrivateUsage]
session=session,
input_messages=[Message(role="user", contents=["Plan first."])],
)
set_mode_tool = _tool_by_name(first_options["tools"], "set_mode")
set_mode_tool = _tool_by_name(first_options["tools"], "mode_set")
await set_mode_tool.invoke(arguments={"mode": "execute"})
assert "previous_mode_for_notification" not in session.state[provider.source_id]
@@ -293,6 +293,94 @@ class TestWorkflowAgent:
# Verify cleanup - pending requests should be cleared after function response handling
assert len(agent.pending_requests) == 0
async def test_request_info_resume_after_session_restore_with_checkpoint(self):
"""Pending request metadata in AgentSession should resume the same request_id after restore."""
from agent_framework import InMemoryCheckpointStorage
simple_executor = SimpleExecutor(id="simple", response_text="SimpleResponse", streaming=False)
requesting_executor = RequestingExecutor(id="requester", streaming=False)
checkpoint_storage = InMemoryCheckpointStorage()
workflow = (
WorkflowBuilder(start_executor=simple_executor, checkpoint_storage=checkpoint_storage)
.add_edge(simple_executor, requesting_executor)
.build()
)
agent = WorkflowAgent(workflow=workflow, name="Request Restore Test Agent")
session = AgentSession()
updates: list[AgentResponseUpdate] = []
async for update in agent.run("Start request", stream=True, session=session):
updates.append(update)
approval_update = next(
(
update
for update in updates
if any(content.type == "function_approval_request" for content in update.contents)
),
None,
)
assert approval_update is not None, "Should have received a request_info approval request"
function_call = next(content for content in approval_update.contents if content.type == "function_call")
approval_request = next(
content for content in approval_update.contents if content.type == "function_approval_request"
)
request_id = approval_request.id
assert request_id is not None
assert function_call.call_id == request_id
checkpoints = await checkpoint_storage.list_checkpoints(workflow_name=workflow.name)
checkpoint_with_request = next(
(checkpoint for checkpoint in checkpoints if request_id in checkpoint.pending_request_info_events),
None,
)
assert checkpoint_with_request is not None
serialized_session = session.to_dict()
workflow_agent_state = serialized_session["state"].get("workflow_agent", {})
pending_state = workflow_agent_state.get("pending_request_info_events", {})
assert request_id in pending_state
restored_session = AgentSession.from_dict(serialized_session)
restored_simple_executor = SimpleExecutor(id="simple", response_text="SimpleResponse", streaming=False)
restored_requesting_executor = RequestingExecutor(id="requester", streaming=False)
restored_workflow = (
WorkflowBuilder(start_executor=restored_simple_executor, checkpoint_storage=checkpoint_storage)
.add_edge(restored_simple_executor, restored_requesting_executor)
.build()
)
restored_agent = WorkflowAgent(workflow=restored_workflow, name="Request Restore Test Agent")
response_args = WorkflowAgent.RequestInfoFunctionArgs(
request_id=request_id,
data="User provided answer",
).to_dict()
approval_response = Content.from_function_approval_response(
approved=True,
id=request_id,
function_call=Content.from_function_call(
call_id=request_id,
name=WorkflowAgent.REQUEST_INFO_FUNCTION_NAME,
arguments=response_args,
),
)
response_message = Message(role="user", contents=[approval_response])
continuation_result = await restored_agent.run(
response_message,
session=restored_session,
checkpoint_id=checkpoint_with_request.checkpoint_id,
checkpoint_storage=checkpoint_storage,
)
assert isinstance(continuation_result, AgentResponse)
response_texts = [message.text for message in continuation_result.messages if message.text]
assert any("Request completed with response: User provided answer" in text for text in response_texts)
assert len(restored_agent.pending_requests) == 0
def test_workflow_as_agent_method(self) -> None:
"""Test that Workflow.as_agent() creates a properly configured WorkflowAgent."""
# Create a simple workflow
@@ -27,12 +27,15 @@ from __future__ import annotations
import locale
import logging
import os
import re
import sys
import uuid
from collections.abc import Mapping
from dataclasses import dataclass
from dataclasses import dataclass, field
from decimal import Decimal as _Decimal
from enum import Enum
from types import MappingProxyType
from typing import Any, Literal, cast
from agent_framework import (
@@ -58,6 +61,100 @@ else:
logger = logging.getLogger(__name__)
_ENV_REFERENCE_RE = re.compile(r"\bEnv\.([A-Za-z_][A-Za-z0-9_]*)")
@dataclass(frozen=True)
class DeclarativeEnvConfig:
"""Configuration that populates the PowerFx ``Env`` symbol for a workflow.
Configuration values are always exposed under ``Env.<name>``;
``os.environ`` is consulted only when ``restrict_to_configuration``
is ``False`` AND the YAML literally references the name in a PowerFx
expression (the allowlist enforced via ``referenced_names``).
Attributes:
values: Caller-supplied configuration resolved by name when the
workflow YAML references ``=Env.NAME``. Always exposed in
the ``Env`` symbol regardless of ``restrict_to_configuration``.
restrict_to_configuration: When ``True`` (default), the ``Env``
symbol is populated exclusively from ``values``; ``os.environ``
is never consulted. Set to ``False`` to additionally fall back
to ``os.environ`` for names absent from ``values`` that the
workflow YAML explicitly references.
referenced_names: The set of ``Env.NAME`` symbols discovered in
PowerFx expressions inside the workflow definition. The
``os.environ`` fallback is constrained to this allowlist so
unrelated environment variables never enter the PowerFx scope.
"""
values: Mapping[str, str] = field(default_factory=lambda: MappingProxyType({}))
restrict_to_configuration: bool = True
referenced_names: frozenset[str] = field(default_factory=lambda: frozenset[str]())
def __post_init__(self) -> None:
# Defensive snapshots so the frozen guarantee extends to the
# contents of ``values`` / ``referenced_names``: caller mutations
# to the original objects after construction cannot leak into
# ``resolve()``.
object.__setattr__(self, "values", MappingProxyType(dict(self.values)))
object.__setattr__(self, "referenced_names", frozenset(self.referenced_names))
def resolve(self) -> dict[str, str]:
"""Return the resolved ``Env`` symbol mapping for the workflow.
Configuration values are always included (stringified).
``os.environ`` is consulted only when ``restrict_to_configuration``
is ``False`` and the name appears in ``referenced_names``, so
unrelated environment variables never enter the PowerFx scope.
Configuration values always win over the environment fallback.
"""
resolved = {name: str(value) for name, value in self.values.items()}
if self.restrict_to_configuration:
return resolved
for name in self.referenced_names.difference(resolved):
env_value = os.environ.get(name)
if env_value is not None:
resolved[name] = env_value
return resolved
def discover_env_references(node: Any) -> set[str]:
"""Discover ``Env.NAME`` references in PowerFx expressions inside ``node``.
Walks any nested ``Mapping``/``list``/scalar structure and inspects every
string value. To avoid false positives from doc/description fields that
happen to mention ``Env.SOMETHING`` as plain text, the scan only inspects
strings that begin with ``=`` (PowerFx expression marker, matching the
convention enforced by :meth:`DeclarativeWorkflowState.eval`).
Args:
node: A parsed workflow definition (typically the dict produced by
``yaml.safe_load``).
Returns:
The set of ``Env`` identifier names referenced in PowerFx
expressions inside ``node``.
"""
names: set[str] = set()
def visit(value: Any) -> None:
if isinstance(value, str):
if value.startswith("="):
names.update(_ENV_REFERENCE_RE.findall(value))
return
if isinstance(value, Mapping):
for inner in cast(Mapping[Any, Any], value).values(): # type: ignore[redundant-cast]
visit(inner)
return
if isinstance(value, list):
for item in cast(list[Any], value): # type: ignore[redundant-cast]
visit(item)
visit(node)
return names
class ConversationData(TypedDict):
"""Structure for conversation-related state data.
@@ -169,13 +266,18 @@ class DeclarativeWorkflowState:
- Conversation: Conversation history
"""
def __init__(self, state: State):
def __init__(self, state: State, env_config: DeclarativeEnvConfig | None = None):
"""Initialize with a State instance.
Args:
state: The workflow's state for persistence
env_config: Configuration that populates the PowerFx ``Env``
symbol when ``_to_powerfx_symbols`` is called. Defaults to
an empty configuration which results in no ``Env`` binding,
matching the safe default of the :class:`WorkflowFactory`.
"""
self._state = state
self._env_config = env_config if env_config is not None else DeclarativeEnvConfig()
def initialize(self, inputs: Mapping[str, Any] | None = None) -> None:
"""Initialize the declarative state with inputs.
@@ -714,6 +816,14 @@ class DeclarativeWorkflowState:
# Custom namespaces
**state_data.get("Custom", {}),
}
# Resolve the ``Env`` symbol from the workflow-level
# :class:`DeclarativeEnvConfig`. When both ``values`` and the
# ``os.environ`` allowlist produce no entries the symbol is
# omitted so ``=Env.X`` falls back to the literal expression
# string (preserving the legacy "unbound identifier" behaviour).
env_bound = self._env_config.resolve()
if env_bound:
symbols["Env"] = env_bound
# Debug log the Local symbols to help diagnose type issues
if local_data:
for key, value in local_data.items():
@@ -867,6 +977,11 @@ class DeclarativeActionExecutor(Executor):
action_id = id or action_def.get("id") or f"{action_def.get('kind', 'action')}_{hash(str(action_def)) % 10000}"
super().__init__(id=action_id, defer_discovery=True)
self._action_def = action_def
# The active :class:`DeclarativeEnvConfig` is stamped onto the
# executor by :class:`DeclarativeWorkflowBuilder` after construction.
# Defaults to an empty configuration so direct ``DeclarativeActionExecutor``
# construction (e.g. in unit tests) doesn't expose ``os.environ``.
self._declarative_env_config: DeclarativeEnvConfig = DeclarativeEnvConfig()
# Manually register handlers after initialization
self._handlers = {}
@@ -874,6 +989,16 @@ class DeclarativeActionExecutor(Executor):
self._discover_handlers()
self._discover_response_handlers()
def set_declarative_env_config(self, env_config: DeclarativeEnvConfig) -> None:
"""Set the workflow-level :class:`DeclarativeEnvConfig` for this executor.
Called by :class:`DeclarativeWorkflowBuilder` after each executor is
created so that ``_to_powerfx_symbols`` populates the ``Env`` symbol
according to the caller-supplied configuration on the
:class:`WorkflowFactory`.
"""
self._declarative_env_config = env_config
@property
def action_def(self) -> dict[str, Any]:
"""Get the action definition."""
@@ -886,7 +1011,7 @@ class DeclarativeActionExecutor(Executor):
def _get_state(self, state: State) -> DeclarativeWorkflowState:
"""Get the declarative workflow state wrapper."""
return DeclarativeWorkflowState(state)
return DeclarativeWorkflowState(state, env_config=self._declarative_env_config)
async def _ensure_state_initialized(
self,
@@ -24,6 +24,7 @@ from agent_framework import (
from ._declarative_base import (
ConditionResult,
DeclarativeActionExecutor,
DeclarativeEnvConfig,
LoopIterationResult,
)
from ._errors import DeclarativeWorkflowError
@@ -140,6 +141,7 @@ class DeclarativeWorkflowBuilder:
max_iterations: int | None = None,
http_request_handler: HttpRequestHandler | None = None,
mcp_tool_handler: MCPToolHandler | None = None,
env_config: DeclarativeEnvConfig | None = None,
):
"""Initialize the builder.
@@ -158,6 +160,10 @@ class DeclarativeWorkflowBuilder:
mcp_tool_handler: Handler used to dispatch InvokeMcpTool calls.
Must be supplied when the workflow contains any InvokeMcpTool;
otherwise build raises ``DeclarativeWorkflowError``.
env_config: Optional :class:`DeclarativeEnvConfig` controlling
how the ``Env`` PowerFx symbol is populated for every
executor built by this builder. Defaults to an empty
configuration (``Env`` not exposed).
"""
self._yaml_def = yaml_definition
self._workflow_id = workflow_id or yaml_definition.get("name", "declarative_workflow")
@@ -171,6 +177,7 @@ class DeclarativeWorkflowBuilder:
self._seen_explicit_ids: set[str] = set() # Track explicit IDs for duplicate detection
self._http_request_handler = http_request_handler
self._mcp_tool_handler = mcp_tool_handler
self._env_config: DeclarativeEnvConfig = env_config if env_config is not None else DeclarativeEnvConfig()
# Resolve max_iterations: explicit arg > YAML maxTurns > core default
resolved = max_iterations if max_iterations is not None else yaml_definition.get("maxTurns")
if resolved is not None and (not isinstance(resolved, int) or resolved <= 0):
@@ -221,6 +228,15 @@ class DeclarativeWorkflowBuilder:
# Resolve pending gotos (back-edges for loops, forward-edges for jumps)
self._resolve_pending_gotos(builder)
# Stamp the resolved DeclarativeEnvConfig onto every executor so they
# expose the configured Env binding through their _get_state(). This
# happens after _create_executors_for_actions and _resolve_pending_gotos
# so it covers the entry node, join nodes, evaluators, foreach
# init/next/exit nodes, and goto placeholders.
for executor in self._executors.values():
if isinstance(executor, DeclarativeActionExecutor):
executor.set_declarative_env_config(self._env_config)
return builder.build()
def _validate_workflow(self, actions: list[dict[str, Any]]) -> None:
@@ -817,10 +833,14 @@ class DeclarativeWorkflowBuilder:
condition=lambda msg: isinstance(msg, LoopIterationResult) and msg.has_next,
)
# Body exit -> Next (get all exits from body and wire to next_executor)
body_exits = self._get_source_exits(body_entry)
for body_exit in body_exits:
builder.add_edge(source=body_exit, target=next_executor)
# Wire from the LAST body action so the loop only advances after the
# whole body completes. _get_branch_exit walks the chain, skips
# terminators (Break/Continue), and returns nested If/Switch
# structures so _get_source_exits can flatten their branch exits.
body_exit = self._get_branch_exit(body_entry)
if body_exit is not None:
for source_exit in self._get_source_exits(body_exit):
builder.add_edge(source=source_exit, target=next_executor)
# Next -> body (when has_next=True, loop back)
builder.add_edge(
@@ -1008,16 +1028,12 @@ class DeclarativeWorkflowBuilder:
return entry.evaluator if is_structure else entry
def _get_branch_exit(self, branch_entry: Any) -> Any | None:
"""Get the exit executor of a branch.
"""Get the exit point of a branch for downstream wiring.
For a linear sequence of actions, returns the last executor.
For nested structures, returns None (they have their own branch_exits).
Args:
branch_entry: The first executor of the branch
Returns:
The exit executor, or None if branch is empty or ends with a structure
Returns the last executor (or its ``_exit_executor``) for a linear chain,
the nested If/Switch structure itself when the chain ends in one (so
callers can flatten ``branch_exits`` via :meth:`_get_source_exits`), or
``None`` when the branch is empty or ends in a terminator action.
"""
if branch_entry is None:
return None
@@ -26,6 +26,7 @@ from agent_framework import (
)
from .._loader import AgentFactory
from ._declarative_base import DeclarativeEnvConfig, discover_env_references
from ._declarative_builder import DeclarativeWorkflowBuilder
from ._errors import DeclarativeWorkflowError
from ._http_handler import HttpRequestHandler
@@ -93,6 +94,8 @@ class WorkflowFactory:
max_iterations: int | None = None,
http_request_handler: HttpRequestHandler | None = None,
mcp_tool_handler: MCPToolHandler | None = None,
configuration: Mapping[str, str] | None = None,
restrict_env_to_configuration: bool = True,
) -> None:
"""Initialize the workflow factory.
@@ -119,6 +122,23 @@ class WorkflowFactory:
for a default backed by :class:`agent_framework.MCPStreamableHTTPTool`,
or supply your own implementation to enforce SSRF guards, allowlisting,
or auth/connection resolution.
configuration: Optional mapping that populates the PowerFx ``Env``
symbol referenced from workflow YAML expressions (e.g.
``=Env.MY_KEY``). Keys supplied here are always exposed
under ``Env.<key>``; the process ``os.environ`` is consulted
only when ``restrict_env_to_configuration`` is ``False``.
When neither source produces a value the ``Env`` symbol is
omitted so ``=Env.X`` evaluates to the literal expression
string.
restrict_env_to_configuration: When ``True`` (default), the
``Env`` PowerFx symbol is populated exclusively from
``configuration``; ``os.environ`` is never consulted. Set to
``False`` to additionally fall back to ``os.environ`` for
names absent from ``configuration`` that the workflow YAML
explicitly references. The fallback is constrained to names
discovered in PowerFx expressions inside the workflow
definition so unrelated environment variables never enter
the PowerFx scope.
Examples:
.. code-block:: python
@@ -151,6 +171,18 @@ class WorkflowFactory:
checkpoint_storage=FileCheckpointStorage("./checkpoints"),
env_file=".env",
)
.. code-block:: python
from agent_framework.declarative import WorkflowFactory
# Inject named values for =Env.* references in the workflow YAML
factory = WorkflowFactory(
configuration={
"MY_SERVER_URL": "https://example.com",
"MY_TOOL_NAME": "search",
},
)
"""
self._agent_factory = agent_factory or AgentFactory(env_file_path=env_file)
self._agents: dict[str, SupportsAgentRun | AgentExecutor] = dict(agents) if agents else {}
@@ -160,6 +192,8 @@ class WorkflowFactory:
self._max_iterations = max_iterations
self._http_request_handler = http_request_handler
self._mcp_tool_handler = mcp_tool_handler
self._configuration: dict[str, str] = dict(configuration) if configuration else {}
self._restrict_env_to_configuration = restrict_env_to_configuration
def create_workflow_from_yaml_path(
self,
@@ -394,6 +428,16 @@ class WorkflowFactory:
if description:
normalized_def["description"] = description
# Build the DeclarativeEnvConfig from the factory's configuration and the
# set of Env references actually used in the workflow PowerFx expressions.
# The referenced-name allowlist constrains ``os.environ`` fallback (when
# enabled) so unrelated variables never enter the PowerFx scope.
env_config = DeclarativeEnvConfig(
values=dict(self._configuration),
restrict_to_configuration=self._restrict_env_to_configuration,
referenced_names=frozenset(discover_env_references(normalized_def)),
)
# Build the graph-based workflow, passing agents and tools for specialized executors
try:
graph_builder = DeclarativeWorkflowBuilder(
@@ -405,6 +449,7 @@ class WorkflowFactory:
max_iterations=self._max_iterations,
http_request_handler=self._http_request_handler,
mcp_tool_handler=self._mcp_tool_handler,
env_config=env_config,
)
workflow = graph_builder.build()
except ValueError as e:
@@ -33,7 +33,7 @@ import logging
from collections import OrderedDict
from collections.abc import Awaitable, Callable
from dataclasses import dataclass, field
from typing import TYPE_CHECKING, Any, Protocol, cast, runtime_checkable
from typing import TYPE_CHECKING, Any, ClassVar, Protocol, cast, runtime_checkable
import httpx
@@ -194,6 +194,21 @@ class DefaultMCPToolHandler:
Defaults to ``32``.
"""
LIST_TOOLS_TOOL_NAME: ClassVar[str] = "tools/list"
"""Reserved ``tool_name`` that maps an :class:`MCPToolHandler` invocation
to the MCP protocol ``tools/list`` discovery operation.
The constant matches the underlying MCP method name so a single
string travels unchanged through host code, YAML, and the protocol
wire. When this handler receives an invocation with this name it
pages through ``session.list_tools()`` and returns the catalog as a
single ``TextContent`` containing JSON of shape
``{"tools": [{name, description, inputSchema, outputSchema}, ...]}``.
Workflows can reference this name from an ``InvokeMcpTool`` declarative
action to introspect a server's tool surface without an extra round-trip
from host code.
"""
def __init__(
self,
*,
@@ -217,10 +232,27 @@ class DefaultMCPToolHandler:
self._closed = False
async def invoke_tool(self, invocation: MCPToolInvocation) -> MCPToolResult:
"""Invoke ``invocation.tool_name`` on the cached MCP client for the server."""
"""Invoke ``invocation.tool_name`` on the cached MCP client for the server.
The reserved name :attr:`LIST_TOOLS_TOOL_NAME` (``"tools/list"``) is
intercepted client-side: instead of being forwarded as a tool call,
it is translated to an MCP ``session.list_tools()`` discovery
operation (paginated automatically) and returned as a single
``TextContent`` containing a JSON tool catalog.
"""
from agent_framework import Content
from agent_framework.exceptions import ToolExecutionException
# Reserved-name args validation runs before connect: rejecting bad
# input shouldn't require establishing an MCP session.
if invocation.tool_name == self.LIST_TOOLS_TOOL_NAME and invocation.arguments:
message = f"The reserved MCP '{self.LIST_TOOLS_TOOL_NAME}' operation does not accept tool arguments."
return MCPToolResult(
outputs=[Content.from_text(f"Error: {message}")],
is_error=True,
error_message=message,
)
try:
entry = await self._get_or_create_entry(invocation)
except Exception as exc:
@@ -240,6 +272,8 @@ class DefaultMCPToolHandler:
)
try:
if invocation.tool_name == self.LIST_TOOLS_TOOL_NAME:
return await self._invoke_list_tools(entry)
raw = await entry.tool.call_tool(invocation.tool_name, **invocation.arguments)
except ToolExecutionException as exc:
logger.info(
@@ -284,6 +318,59 @@ class DefaultMCPToolHandler:
outputs = list(raw)
return MCPToolResult(outputs=outputs)
@staticmethod
async def _invoke_list_tools(entry: _CacheEntry) -> MCPToolResult:
"""Handle the reserved :attr:`LIST_TOOLS_TOOL_NAME` invocation.
Pages through ``session.list_tools()`` (mirroring the pagination loop
in :meth:`agent_framework.MCPTool.load_tools`) and serialises the
full catalog as a single ``TextContent`` containing JSON of shape
``{"tools": [{name, description, inputSchema, outputSchema}, ...]}``.
The output shape, property names, and property order are stable so
downstream PowerFx expressions can rely on the schema. ``indent=2``
produces human-readable JSON for the conversation log;
``allow_nan=False`` guards against producing non-conformant JSON
``NaN``/``Infinity`` tokens if a misbehaving server returns such
values in a schema.
"""
from agent_framework import Content
session = getattr(entry.tool, "session", None)
if session is None:
message = "MCP session is not connected; cannot list tools."
return MCPToolResult(
outputs=[Content.from_text(f"Error: {message}")],
is_error=True,
error_message=message,
)
# Lazy import keeps ``mcp`` types out of module import time.
from mcp import types as mcp_types
collected: list[Any] = []
params: mcp_types.PaginatedRequestParams | None = None
while True:
tool_list = await session.list_tools(params=params)
collected.extend(tool_list.tools)
next_cursor = getattr(tool_list, "nextCursor", None)
if not next_cursor:
break
params = mcp_types.PaginatedRequestParams(cursor=next_cursor)
payload = {
"tools": [
{
"name": tool.name,
"description": tool.description,
"inputSchema": tool.inputSchema,
"outputSchema": tool.outputSchema,
}
for tool in collected
],
}
return MCPToolResult(outputs=[Content.from_text(json.dumps(payload, indent=2, allow_nan=False))])
async def aclose(self) -> None:
"""Close all cached MCP clients and the owned httpx clients.
@@ -13,6 +13,7 @@ owned-vs-caller httpx close semantics.
from __future__ import annotations
import asyncio
import json
import sys
from typing import Any
from unittest.mock import patch
@@ -33,6 +34,55 @@ pytestmark = pytest.mark.skipif(
)
class FakeListToolsResult: # noqa: B903 - mimics ``mcp.types.ListToolsResult`` shape, not a value type
"""Stand-in for ``mcp.types.ListToolsResult`` returned by ``session.list_tools()``."""
def __init__(self, tools: list[Any], next_cursor: str | None = None) -> None:
self.tools = tools
self.nextCursor = next_cursor
class FakeMcpTool:
"""Stand-in for an MCP ``Tool`` (subset used by ``_invoke_list_tools``)."""
def __init__(
self,
name: str,
description: str | None = None,
inputSchema: dict[str, Any] | None = None,
outputSchema: dict[str, Any] | None = None,
) -> None:
self.name = name
self.description = description
self.inputSchema = inputSchema if inputSchema is not None else {"type": "object", "properties": {}}
self.outputSchema = outputSchema
class FakeMcpSession:
"""Stand-in for ``mcp.ClientSession``.
``list_tools_pages`` lets a test enqueue multiple paginated responses;
when None (default), an empty single-page result is returned. ``list_tools_error``
raises a synthetic error on the next call when set.
"""
def __init__(self) -> None:
self.list_tools_pages: list[FakeListToolsResult] | None = None
self.list_tools_calls: list[Any] = []
self.list_tools_error: BaseException | None = None
async def list_tools(self, params: Any = None) -> FakeListToolsResult:
self.list_tools_calls.append(params)
if self.list_tools_error is not None:
raise self.list_tools_error
if self.list_tools_pages is None:
return FakeListToolsResult(tools=[])
index = len(self.list_tools_calls) - 1
if index >= len(self.list_tools_pages):
return FakeListToolsResult(tools=[])
return self.list_tools_pages[index]
class FakeTool:
"""Stand-in for ``MCPStreamableHTTPTool``.
@@ -50,6 +100,7 @@ class FakeTool:
self.connect_error: BaseException | None = None
self.call_handler: Any = lambda **_a: [Content.from_text("ok")]
self._httpx_client: httpx.AsyncClient | None = None
self.session: FakeMcpSession | None = None
# Mimic MCPStreamableHTTPTool: when no caller client AND header_provider
# is set, lazily allocate an owned httpx client during connect.
FakeTool.instances.append(self)
@@ -63,6 +114,9 @@ class FakeTool:
# Mimic lazy httpx allocation when no client provided AND header_provider set.
if self.kwargs.get("http_client") is None and self.kwargs.get("header_provider") is not None:
self._httpx_client = httpx.AsyncClient()
# Mimic MCPStreamableHTTPTool: a live session becomes available after connect.
if self.session is None:
self.session = FakeMcpSession()
async def close(self) -> None:
self.close_count += 1
@@ -541,3 +595,185 @@ class TestCacheKey:
k1 = DefaultMCPToolHandler._cache_key("https://x/", None, None, {"X": "Bearer-A"})
k2 = DefaultMCPToolHandler._cache_key("https://x/", None, None, {"X": "bearer-a"})
assert k1 != k2
# ---------- tools/list reserved name --------------------------------------
class TestListTools:
"""Exercise the reserved :attr:`DefaultMCPToolHandler.LIST_TOOLS_TOOL_NAME` interception path."""
@pytest.mark.asyncio
async def test_list_tools_returns_json_catalog(self) -> None:
handler = DefaultMCPToolHandler()
with _patch_tool():
# Prime the cache so the FakeTool session exists.
await handler.invoke_tool(_invocation())
FakeTool.instances[0].session.list_tools_pages = [ # type: ignore[union-attr]
FakeListToolsResult(
tools=[
FakeMcpTool(
name="search",
description="Search docs",
inputSchema={"type": "object", "properties": {"q": {"type": "string"}}},
outputSchema={"type": "object"},
),
FakeMcpTool(name="echo", description=None, outputSchema=None),
],
),
]
result = await handler.invoke_tool(_invocation(tool_name=DefaultMCPToolHandler.LIST_TOOLS_TOOL_NAME))
assert result.is_error is False
assert len(result.outputs) == 1
payload = json.loads(result.outputs[0].text) # type: ignore[reportAttributeAccessIssue]
assert payload == {
"tools": [
{
"name": "search",
"description": "Search docs",
"inputSchema": {"type": "object", "properties": {"q": {"type": "string"}}},
"outputSchema": {"type": "object"},
},
{
"name": "echo",
"description": None,
"inputSchema": {"type": "object", "properties": {}},
"outputSchema": None,
},
],
}
@pytest.mark.asyncio
async def test_list_tools_property_order_is_stable(self) -> None:
"""JSON property order is stable: name, description, inputSchema, outputSchema."""
handler = DefaultMCPToolHandler()
with _patch_tool():
await handler.invoke_tool(_invocation())
FakeTool.instances[0].session.list_tools_pages = [ # type: ignore[union-attr]
FakeListToolsResult(tools=[FakeMcpTool(name="t1", description="d")]),
]
result = await handler.invoke_tool(_invocation(tool_name=DefaultMCPToolHandler.LIST_TOOLS_TOOL_NAME))
text = result.outputs[0].text # type: ignore[reportAttributeAccessIssue]
name_idx = text.find('"name"')
desc_idx = text.find('"description"')
input_idx = text.find('"inputSchema"')
output_idx = text.find('"outputSchema"')
assert 0 <= name_idx < desc_idx < input_idx < output_idx
@pytest.mark.asyncio
async def test_list_tools_indented_output(self) -> None:
"""Output is JSON with a 2-space indent so the conversation log is human-readable."""
handler = DefaultMCPToolHandler()
with _patch_tool():
await handler.invoke_tool(_invocation())
FakeTool.instances[0].session.list_tools_pages = [ # type: ignore[union-attr]
FakeListToolsResult(tools=[FakeMcpTool(name="t1")]),
]
result = await handler.invoke_tool(_invocation(tool_name=DefaultMCPToolHandler.LIST_TOOLS_TOOL_NAME))
text = result.outputs[0].text # type: ignore[reportAttributeAccessIssue]
# Indented output contains newlines and a 2-space indented key.
assert "\n " in text
@pytest.mark.asyncio
async def test_list_tools_rejects_arguments(self) -> None:
"""Reserved name does NOT accept tool arguments. Fails fast before connect."""
handler = DefaultMCPToolHandler()
with _patch_tool():
result = await handler.invoke_tool(
_invocation(tool_name=DefaultMCPToolHandler.LIST_TOOLS_TOOL_NAME, arguments={"q": "test"}),
)
assert result.is_error is True
assert "does not accept tool arguments" in (result.error_message or "")
# Args validation runs before connect, so no tool was instantiated.
assert FakeTool.instances == []
@pytest.mark.asyncio
async def test_list_tools_empty_args_dict_is_accepted(self) -> None:
"""An empty arguments dict is equivalent to no arguments."""
handler = DefaultMCPToolHandler()
with _patch_tool():
await handler.invoke_tool(_invocation())
result = await handler.invoke_tool(
_invocation(tool_name=DefaultMCPToolHandler.LIST_TOOLS_TOOL_NAME, arguments={}),
)
assert result.is_error is False
@pytest.mark.asyncio
async def test_list_tools_paginates(self) -> None:
"""Pagination loop calls list_tools repeatedly until nextCursor is empty."""
handler = DefaultMCPToolHandler()
with _patch_tool():
await handler.invoke_tool(_invocation())
FakeTool.instances[0].session.list_tools_pages = [ # type: ignore[union-attr]
FakeListToolsResult(tools=[FakeMcpTool(name="a")], next_cursor="cursor1"),
FakeListToolsResult(tools=[FakeMcpTool(name="b")], next_cursor="cursor2"),
FakeListToolsResult(tools=[FakeMcpTool(name="c")], next_cursor=None),
]
result = await handler.invoke_tool(_invocation(tool_name=DefaultMCPToolHandler.LIST_TOOLS_TOOL_NAME))
payload = json.loads(result.outputs[0].text) # type: ignore[reportAttributeAccessIssue]
assert [t["name"] for t in payload["tools"]] == ["a", "b", "c"]
session = FakeTool.instances[0].session
assert session is not None
assert len(session.list_tools_calls) == 3
# First call has no cursor; second/third use the cursor from the prior page.
assert session.list_tools_calls[0] is None
assert getattr(session.list_tools_calls[1], "cursor", None) == "cursor1"
assert getattr(session.list_tools_calls[2], "cursor", None) == "cursor2"
@pytest.mark.asyncio
async def test_list_tools_shares_cache_with_call_tool(self) -> None:
"""tools/list reuses the same cached MCP session as a regular call_tool."""
handler = DefaultMCPToolHandler()
with _patch_tool():
await handler.invoke_tool(_invocation(tool_name="search"))
await handler.invoke_tool(_invocation(tool_name=DefaultMCPToolHandler.LIST_TOOLS_TOOL_NAME))
assert len(FakeTool.instances) == 1
assert FakeTool.instances[0].connect_count == 1
@pytest.mark.asyncio
async def test_list_tools_propagates_session_errors_as_error_result(self) -> None:
"""Errors raised by session.list_tools become MCPToolResult(is_error=True), not crashes."""
handler = DefaultMCPToolHandler()
with _patch_tool():
await handler.invoke_tool(_invocation())
FakeTool.instances[0].session.list_tools_error = httpx.ReadTimeout("read timed out") # type: ignore[union-attr]
result = await handler.invoke_tool(_invocation(tool_name=DefaultMCPToolHandler.LIST_TOOLS_TOOL_NAME))
assert result.is_error is True
assert "ReadTimeout" in (result.error_message or "")
@pytest.mark.asyncio
async def test_list_tools_returns_error_when_session_is_none(self) -> None:
"""If somehow the cached tool has no session, return a clear error rather than crashing."""
handler = DefaultMCPToolHandler()
with _patch_tool():
await handler.invoke_tool(_invocation())
FakeTool.instances[0].session = None
result = await handler.invoke_tool(_invocation(tool_name=DefaultMCPToolHandler.LIST_TOOLS_TOOL_NAME))
assert result.is_error is True
assert "not connected" in (result.error_message or "")
@pytest.mark.asyncio
async def test_list_tools_does_not_call_call_tool(self) -> None:
"""The reserved name is intercepted; the inner call_tool path is bypassed."""
handler = DefaultMCPToolHandler()
call_tool_invoked = False
def fail(**_a: Any) -> Any:
nonlocal call_tool_invoked
call_tool_invoked = True
raise AssertionError("call_tool should not run for tools/list")
with _patch_tool():
await handler.invoke_tool(_invocation())
FakeTool.instances[0].call_handler = fail
FakeTool.instances[0].session.list_tools_pages = [ # type: ignore[union-attr]
FakeListToolsResult(tools=[]),
]
result = await handler.invoke_tool(_invocation(tool_name=DefaultMCPToolHandler.LIST_TOOLS_TOOL_NAME))
assert call_tool_invoked is False
assert result.is_error is False
def test_class_attribute_value(self) -> None:
# Constant must equal the MCP protocol method name so a single
# string travels unchanged through host code, YAML, and the wire.
assert DefaultMCPToolHandler.LIST_TOOLS_TOOL_NAME == "tools/list"
@@ -2224,6 +2224,101 @@ class TestBuilderControlFlowCreation:
class TestBuilderEdgeWiring:
"""Tests for builder edge wiring methods."""
def test_foreach_advance_edge_wired_from_last_body_action(self):
"""Advance edge must come from the last body action."""
from agent_framework_declarative._workflows import DeclarativeWorkflowBuilder
yaml_def = {
"name": "foreach_seq",
"actions": [
{"kind": "SetValue", "id": "set_items", "path": "Local.items", "value": ["A", "B"]},
{
"kind": "Foreach",
"id": "loop",
"itemsSource": "=Local.items",
"iteratorVariable": "Local.item",
"actions": [
{"kind": "SendActivity", "id": "step_1", "activity": {"text": "one"}},
{"kind": "SendActivity", "id": "step_2", "activity": {"text": "two"}},
{"kind": "SendActivity", "id": "step_3", "activity": {"text": "three"}},
],
},
],
}
workflow = DeclarativeWorkflowBuilder(yaml_def).build()
edges = {(e.source_id, e.target_id) for group in workflow.edge_groups for e in group.edges}
assert ("step_3", "loop_next") in edges
assert ("step_1", "loop_next") not in edges
assert ("step_2", "loop_next") not in edges
assert ("step_1", "step_2") in edges
assert ("step_2", "step_3") in edges
def test_foreach_advance_edge_skipped_for_terminator_body(self):
"""BreakLoop at end of body wires itself to loop_next; no duplicate edge."""
from agent_framework_declarative._workflows import DeclarativeWorkflowBuilder
yaml_def = {
"name": "foreach_terminator",
"actions": [
{"kind": "SetValue", "id": "set_items", "path": "Local.items", "value": ["A"]},
{
"kind": "Foreach",
"id": "loop",
"itemsSource": "=Local.items",
"iteratorVariable": "Local.item",
"actions": [
{"kind": "SendActivity", "id": "step_1", "activity": {"text": "one"}},
{"kind": "BreakLoop", "id": "stop"},
],
},
],
}
workflow = DeclarativeWorkflowBuilder(yaml_def).build()
all_edges = [(e.source_id, e.target_id) for group in workflow.edge_groups for e in group.edges]
assert all_edges.count(("stop", "loop_next")) == 1
assert ("step_1", "loop_next") not in all_edges
def test_foreach_advance_edge_with_if_as_last_body_action(self):
"""Trailing If in a Foreach body wires every branch exit to loop_next."""
from agent_framework_declarative._workflows import DeclarativeWorkflowBuilder
yaml_def = {
"name": "foreach_if_last",
"actions": [
{"kind": "SetValue", "id": "set_items", "path": "Local.items", "value": ["A", "B"]},
{
"kind": "Foreach",
"id": "loop",
"itemsSource": "=Local.items",
"iteratorVariable": "Local.item",
"actions": [
{"kind": "SendActivity", "id": "step_1", "activity": {"text": "one"}},
{
"kind": "If",
"id": "check",
"condition": '=Local.item = "A"',
"then": [
{"kind": "SendActivity", "id": "then_action", "activity": {"text": "then"}},
],
"else": [
{"kind": "SendActivity", "id": "else_action", "activity": {"text": "else"}},
],
},
],
},
],
}
workflow = DeclarativeWorkflowBuilder(yaml_def).build()
edges = {(e.source_id, e.target_id) for group in workflow.edge_groups for e in group.edges}
assert ("then_action", "loop_next") in edges
assert ("else_action", "loop_next") in edges
assert ("step_1", "loop_next") not in edges
def test_wire_to_target_with_if_structure(self):
"""Test wiring to an If structure routes to evaluator."""
from agent_framework import WorkflowBuilder
@@ -121,6 +121,35 @@ class TestGraphBasedWorkflowExecution:
assert "b" in outputs
assert "c" in outputs
@pytest.mark.asyncio
async def test_foreach_multi_action_body_runs_sequentially(self):
"""Body actions must complete per item before advancing."""
yaml_def = {
"name": "loop_sequential_body",
"actions": [
{"kind": "SetValue", "id": "set_items", "path": "Local.items", "value": ["A", "B"]},
{
"kind": "Foreach",
"id": "loop",
"itemsSource": "=Local.items",
"iteratorVariable": "Local.item",
"actions": [
{"kind": "SendActivity", "id": "step_1", "activity": {"text": '="1-" & Local.item'}},
{"kind": "SendActivity", "id": "step_2", "activity": {"text": '="2-" & Local.item'}},
{"kind": "SendActivity", "id": "step_3", "activity": {"text": '="3-" & Local.item'}},
],
},
],
}
builder = DeclarativeWorkflowBuilder(yaml_def)
workflow = builder.build()
events = await workflow.run(ActionTrigger())
outputs = events.get_outputs()
assert outputs == ["1-A", "2-A", "3-A", "1-B", "2-B", "3-B"]
@pytest.mark.asyncio
async def test_workflow_with_switch(self):
"""Test workflow with Switch/ConditionGroup."""
File diff suppressed because one or more lines are too long
@@ -40,11 +40,22 @@ import type {
ExtendedResponseStreamEvent,
} from "@/types";
import { useDevUIStore } from "@/stores";
import { loadStreamingState } from "@/services/streaming-state";
import { loadStreamingState, type StreamingState } from "@/services/streaming-state";
type DebugEventHandler = (event: ExtendedResponseStreamEvent | "clear") => void;
const ASSISTANT_TEXT_RENDER_INTERVAL_MS = 50;
const STREAMING_PREVIEW_PREFIX = "[Earlier streaming content omitted after refresh]\n\n";
function getRestoredStreamingText(state: StreamingState): string {
if (!state.accumulatedText) {
return "";
}
return state.accumulatedTextIsPreview
? `${STREAMING_PREVIEW_PREFIX}${state.accumulatedText}`
: state.accumulatedText;
}
interface AgentViewProps {
selectedAgent: AgentInfo;
@@ -683,13 +694,14 @@ export function AgentView({ selectedAgent, onDebugEvent }: AgentViewProps) {
const state = loadStreamingState(mostRecent.id);
if (state && !state.completed) {
accumulatedTextRef.current = state.accumulatedText || "";
const restoredText = getRestoredStreamingText(state);
accumulatedTextRef.current = restoredText;
// Add assistant message with resumed text
const assistantMsg: import("@/types/openai").ConversationMessage = {
id: state.lastMessageId || `assistant-${Date.now()}`,
type: "message",
role: "assistant",
content: state.accumulatedText ? [{ type: "text", text: state.accumulatedText }] : [],
content: restoredText ? [{ type: "text", text: restoredText }] : [],
status: "in_progress",
};
setChatItems([...allItems as import("@/types/openai").ConversationItem[], assistantMsg]);
@@ -988,13 +1000,14 @@ export function AgentView({ selectedAgent, onDebugEvent }: AgentViewProps) {
// Check for incomplete stream and restore accumulated text
const state = loadStreamingState(conversationId);
if (state?.accumulatedText) {
accumulatedTextRef.current = state.accumulatedText;
const restoredText = getRestoredStreamingText(state);
accumulatedTextRef.current = restoredText;
// Add assistant message with resumed text - streaming will continue automatically
const assistantMsg: import("@/types/openai").ConversationMessage = {
id: `assistant-${Date.now()}`,
type: "message",
role: "assistant",
content: [{ type: "output_text", text: state.accumulatedText }],
content: [{ type: "output_text", text: restoredText }],
status: "in_progress",
};
setChatItems([...items, assistantMsg]);
@@ -49,6 +49,28 @@ interface WorkflowViewProps {
onDebugEvent: DebugEventHandler;
}
function getWorkflowEventTimestamp(event: ExtendedResponseStreamEvent): number | undefined {
if ("created_at" in event && typeof event.created_at === "number" && event.created_at) {
return event.created_at;
}
const response = "response" in event ? event.response : undefined;
if (response && typeof response === "object" && "created_at" in response) {
const createdAt = response.created_at;
if (typeof createdAt === "number") {
return createdAt;
}
}
const data = "data" in event ? event.data : undefined;
if (data && typeof data === "object" && "timestamp" in data && typeof data.timestamp === "string") {
const milliseconds = new Date(data.timestamp).getTime();
return Number.isFinite(milliseconds) ? milliseconds / 1000 : undefined;
}
return undefined;
}
// TODO: CheckpointSelector is not currently used but may be needed for checkpoint resumption feature
// Smart Run Workflow Button Component moved to separate file
@@ -581,20 +603,7 @@ export function WorkflowView({
// 2. response.created_at (response.created / lifecycle events)
// 3. data.timestamp (response.workflow_event.completed ISO string)
// Fall back to a synthesized timestamp only when none is present.
const anyEvent = openAIEvent as Record<string, unknown>;
const eventTimestamp: number | undefined =
typeof anyEvent["created_at"] === "number" && anyEvent["created_at"]
? (anyEvent["created_at"] as number)
: typeof (anyEvent["response"] as Record<string, unknown> | undefined)?.["created_at"] === "number"
? ((anyEvent["response"] as Record<string, number>)["created_at"] as number)
: (() => {
const ts = (anyEvent["data"] as Record<string, unknown> | undefined)?.["timestamp"];
if (typeof ts !== "string") return undefined;
const ms = new Date(ts).getTime();
// Guard against NaN: Python isoformat() emits microseconds without Z,
// which some JS engines cannot parse. Number.isFinite rejects NaN.
return Number.isFinite(ms) ? ms / 1000 : undefined;
})();
const eventTimestamp = getWorkflowEventTimestamp(openAIEvent);
const baseTimestamp = Math.floor(Date.now() / 1000);
const lastTimestamp =
prev.length > 0
@@ -1018,20 +1027,7 @@ export function WorkflowView({
// 2. response.created_at (response.created / lifecycle events)
// 3. data.timestamp (response.workflow_event.completed ISO string)
// Fall back to a synthesized timestamp only when none is present.
const anyEvent = openAIEvent as Record<string, unknown>;
const eventTimestamp: number | undefined =
typeof anyEvent["created_at"] === "number" && anyEvent["created_at"]
? (anyEvent["created_at"] as number)
: typeof (anyEvent["response"] as Record<string, unknown> | undefined)?.["created_at"] === "number"
? ((anyEvent["response"] as Record<string, number>)["created_at"] as number)
: (() => {
const ts = (anyEvent["data"] as Record<string, unknown> | undefined)?.["timestamp"];
if (typeof ts !== "string") return undefined;
const ms = new Date(ts).getTime();
// Guard against NaN: Python isoformat() emits microseconds without Z,
// which some JS engines cannot parse. Number.isFinite rejects NaN.
return Number.isFinite(ms) ? ms / 1000 : undefined;
})();
const eventTimestamp = getWorkflowEventTimestamp(openAIEvent);
const baseTimestamp = Math.floor(Date.now() / 1000);
const lastTimestamp =
prev.length > 0
@@ -600,7 +600,10 @@ function EventItem({ event }: EventItemProps) {
event.type === "error";
return (
<div className="border-l-2 border-muted pl-3 py-2 hover:bg-muted/50 transition-colors">
<div
className="border-l-2 border-muted pl-3 py-2 hover:bg-muted/50 transition-colors"
data-devui-debug-event={eventType}
>
<div className="flex items-center gap-2 text-xs text-muted-foreground mb-1">
<Icon className={`h-3 w-3 ${colorClass}`} />
<span className="font-mono">{timestamp}</span>
@@ -1088,18 +1091,17 @@ function EventExpandedContent({
function EventsTab({
events,
processedEvents,
isStreaming,
}: {
events: ExtendedResponseStreamEvent[];
processedEvents: ExtendedResponseStreamEvent[];
isStreaming?: boolean;
}) {
const scrollRef = useRef<HTMLDivElement>(null);
// Process events to accumulate tool calls and reduce noise
const processedEvents = processEventsForDisplay(events);
// Add separators between message rounds
const eventsWithSeparators = addSeparatorsToEvents(processedEvents);
const eventsWithSeparators = useMemo(() => addSeparatorsToEvents(processedEvents), [processedEvents]);
// Reverse events so latest appears at top
const reversedEvents = [...eventsWithSeparators].reverse();
@@ -1565,10 +1567,13 @@ function TracesTab({ events }: { events: ExtendedResponseStreamEvent[] }) {
);
}
function ToolsTab({ events }: { events: ExtendedResponseStreamEvent[] }) {
// Process events first to get clean tool calls
const processedEvents = processEventsForDisplay(events);
function ToolsTab({
events,
processedEvents,
}: {
events: ExtendedResponseStreamEvent[];
processedEvents: ExtendedResponseStreamEvent[];
}) {
// Create call->result pairs in chronological order
const toolEvents: ExtendedResponseStreamEvent[] = [];
const functionCalls = processedEvents.filter(
@@ -1755,15 +1760,16 @@ export function DebugPanel({
const activeTab = useDevUIStore((state) => state.debugPanelTab);
const setActiveTab = useDevUIStore((state) => state.setDebugPanelTab);
const processedEvents = useMemo(() => processEventsForDisplay(events), [events]);
// Compute counts once for tab badges (memoized to avoid perf hits)
const counts = useMemo(() => {
const processedEvents = processEventsForDisplay(events);
const eventsCount = processedEvents.length;
const tracesCount = events.filter(e => e.type === "response.trace.completed").length;
const toolsCount = processedEvents.filter(e => e.type === "response.function_call.complete").length
+ events.filter(e => getFunctionResultFromEvent(e) !== null).length;
return { eventsCount, tracesCount, toolsCount };
}, [events]);
}, [events, processedEvents]);
return (
<div className="flex-1 border-l flex flex-col min-h-0">
@@ -1809,7 +1815,7 @@ export function DebugPanel({
</div>
<TabsContent value="events" className="flex-1 mt-0 overflow-hidden">
<EventsTab events={events} isStreaming={isStreaming} />
<EventsTab events={events} processedEvents={processedEvents} isStreaming={isStreaming} />
</TabsContent>
<TabsContent value="traces" className="flex-1 mt-0 overflow-hidden">
@@ -1817,7 +1823,7 @@ export function DebugPanel({
</TabsContent>
<TabsContent value="tools" className="flex-1 mt-0 overflow-hidden">
<ToolsTab events={events} />
<ToolsTab events={events} processedEvents={processedEvents} />
</TabsContent>
</Tabs>
</div>
@@ -519,6 +519,7 @@ class ApiClient {
lastMessageId,
lastSequenceNumber,
accumulatedText: storedState?.accumulatedText,
accumulatedTextIsPreview: storedState?.accumulatedTextIsPreview,
}),
event,
currentResponseId,
@@ -15,11 +15,13 @@ export interface StreamingState {
lastSequenceNumber: number;
timestamp: number; // When this state was last updated
completed: boolean; // Whether the stream completed successfully
accumulatedText?: string; // Accumulated text content for quick restoration
accumulatedText?: string; // Bounded tail preview for refresh restoration
accumulatedTextIsPreview?: boolean;
}
const STORAGE_KEY_PREFIX = "devui_streaming_state_";
const STATE_EXPIRY_MS = 24 * 60 * 60 * 1000; // 24 hours
const MAX_ACCUMULATED_TEXT_PREVIEW_CHARS = 16 * 1024;
interface CreateStreamingStateOptions {
conversationId: string;
@@ -27,6 +29,7 @@ interface CreateStreamingStateOptions {
lastMessageId?: string;
lastSequenceNumber?: number;
accumulatedText?: string;
accumulatedTextIsPreview?: boolean;
}
/**
@@ -36,6 +39,21 @@ function getStorageKey(conversationId: string): string {
return `${STORAGE_KEY_PREFIX}${conversationId}`;
}
function normalizeAccumulatedTextPreview(state: StreamingState): StreamingState {
if (
state.accumulatedText === undefined ||
state.accumulatedText.length <= MAX_ACCUMULATED_TEXT_PREVIEW_CHARS
) {
return state;
}
return {
...state,
accumulatedText: state.accumulatedText.slice(-MAX_ACCUMULATED_TEXT_PREVIEW_CHARS),
accumulatedTextIsPreview: true,
};
}
/**
* Read raw streaming state from storage, including completed entries.
*/
@@ -56,7 +74,7 @@ function readStreamingState(conversationId: string): StreamingState | null {
return null;
}
return state;
return normalizeAccumulatedTextPreview(state);
}
/**
@@ -68,8 +86,9 @@ export function createStreamingState({
lastMessageId,
lastSequenceNumber = -1,
accumulatedText,
accumulatedTextIsPreview = false,
}: CreateStreamingStateOptions): StreamingState {
return {
return normalizeAccumulatedTextPreview({
conversationId,
responseId,
lastMessageId,
@@ -77,7 +96,8 @@ export function createStreamingState({
timestamp: Date.now(),
completed: false,
accumulatedText,
};
accumulatedTextIsPreview,
});
}
/**
@@ -108,7 +128,15 @@ export function applyStreamingEventToState(
typeof event.delta === "string" &&
event.delta.length > 0
) {
nextState.accumulatedText = `${state.accumulatedText ?? ""}${event.delta}`;
const accumulatedText = `${state.accumulatedText ?? ""}${event.delta}`;
const isPreview =
state.accumulatedTextIsPreview ||
accumulatedText.length > MAX_ACCUMULATED_TEXT_PREVIEW_CHARS;
nextState.accumulatedText = isPreview
? accumulatedText.slice(-MAX_ACCUMULATED_TEXT_PREVIEW_CHARS)
: accumulatedText;
nextState.accumulatedTextIsPreview = isPreview;
}
return nextState;
@@ -120,7 +148,7 @@ export function applyStreamingEventToState(
export function saveStreamingState(state: StreamingState): void {
try {
const key = getStorageKey(state.conversationId);
const data = JSON.stringify(state);
const data = JSON.stringify(normalizeAccumulatedTextPreview(state));
localStorage.setItem(key, data);
} catch (error) {
console.error("Failed to save streaming state:", error);
@@ -129,7 +157,7 @@ export function saveStreamingState(state: StreamingState): void {
clearExpiredStreamingStates();
// Try again
const key = getStorageKey(state.conversationId);
const data = JSON.stringify(state);
const data = JSON.stringify(normalizeAccumulatedTextPreview(state));
localStorage.setItem(key, data);
} catch {
console.error("Failed to save streaming state even after cleanup");
@@ -18,6 +18,26 @@ import type {
import type { ConversationItem } from "@/types/openai";
import type { AttachmentItem } from "@/components/ui/attachment-gallery";
const MAX_DEBUG_EVENTS = 1000;
const MAX_DEBUG_TEXT_DELTA_CHARS = 2048;
function prepareDebugEvent(event: ExtendedResponseStreamEvent): ExtendedResponseStreamEvent {
if (
event.type !== "response.output_text.delta" ||
!("delta" in event) ||
typeof event.delta !== "string" ||
event.delta.length <= MAX_DEBUG_TEXT_DELTA_CHARS
) {
return event;
}
const omittedChars = event.delta.length - MAX_DEBUG_TEXT_DELTA_CHARS;
return {
...event,
delta: `${event.delta.slice(0, MAX_DEBUG_TEXT_DELTA_CHARS)}\n...[${omittedChars} chars omitted from debug view]`,
};
}
// ========================================
// State Interface
// ========================================
@@ -395,6 +415,7 @@ export const useDevUIStore = create<DevUIStore>()(
setStreamingEnabled: (enabled) => set({ streamingEnabled: enabled }),
addDebugEvent: (event) =>
set((state) => {
const eventForStorage = prepareDebugEvent(event);
// Generate unique timestamp for each event
// Use current time + small increment to ensure uniqueness even for rapid events
const baseTimestamp = Math.floor(Date.now() / 1000);
@@ -404,16 +425,19 @@ export const useDevUIStore = create<DevUIStore>()(
const lastTimestamp = lastEvent?._uiTimestamp ?? 0;
// Ensure new timestamp is always greater than the last one
const uniqueTimestamp = Math.max(baseTimestamp, lastTimestamp + 1);
const retainedEvents = state.debugEvents.length >= MAX_DEBUG_EVENTS
? state.debugEvents.slice(-(MAX_DEBUG_EVENTS - 1))
: state.debugEvents;
return {
debugEvents: [
...state.debugEvents,
...retainedEvents,
{
...event,
...eventForStorage,
// Add UI display timestamp when event is received (Unix seconds)
// Each event gets a unique timestamp to preserve chronological order
_uiTimestamp: ('created_at' in event && event.created_at)
? event.created_at
_uiTimestamp: ('created_at' in eventForStorage && eventForStorage.created_at)
? eventForStorage.created_at
: uniqueTimestamp,
} as ExtendedResponseStreamEvent & { _uiTimestamp: number },
],
@@ -79,6 +79,8 @@ _POST_SEND_DELAY_S = 1.0
_SAMPLE_INTERVAL_S = 0.5
_SAMPLE_WINDOW_S = 12.0
_MAX_RENDERER_GROWTH_MB = 500.0
_MAX_STREAMING_STATE_STORAGE_BYTES = 64 * 1024
_MAX_DEBUG_EVENT_DOM_ITEMS = 1000
@dataclass(frozen=True)
@@ -89,6 +91,13 @@ class _BrowserProcessRow:
command: str
@dataclass(frozen=True)
class _BrowserMemoryProbe:
streaming_state_storage_bytes: int
debug_event_dom_items: int
js_heap_bytes: int | None
class MemoryStressAgent(BaseAgent):
"""Agent that emits many small streaming chunks."""
@@ -430,6 +439,52 @@ def _sample_peak_renderer_rss_mb(root_pid: int, profile_dir: str) -> float:
return round((max(renderer_rss_kb, default=0)) / 1024, 2)
async def _sample_browser_memory_probe(client: _CDPClient, *, session_id: str) -> _BrowserMemoryProbe:
value = await client.evaluate(
"""
(() => {
const storagePrefix = "devui_streaming_state_";
const textEncoder = new TextEncoder();
let streamingStateStorageBytes = 0;
for (let index = 0; index < localStorage.length; index += 1) {
const key = localStorage.key(index);
if (!key || !key.startsWith(storagePrefix)) {
continue;
}
const item = localStorage.getItem(key) || "";
streamingStateStorageBytes += textEncoder.encode(key).length + textEncoder.encode(item).length;
}
return {
streamingStateStorageBytes,
debugEventDomItems: document.querySelectorAll("[data-devui-debug-event]").length,
jsHeapBytes: performance.memory ? performance.memory.usedJSHeapSize : null,
};
})()
""",
session_id=session_id,
)
if not isinstance(value, dict):
raise AssertionError(f"Expected browser memory probe object, got: {type(value).__name__}")
streaming_state_storage_bytes = value.get("streamingStateStorageBytes")
debug_event_dom_items = value.get("debugEventDomItems")
js_heap_bytes = value.get("jsHeapBytes")
if not isinstance(streaming_state_storage_bytes, int):
raise AssertionError("Browser memory probe did not return streamingStateStorageBytes")
if not isinstance(debug_event_dom_items, int):
raise AssertionError("Browser memory probe did not return debugEventDomItems")
if js_heap_bytes is not None and not isinstance(js_heap_bytes, int):
raise AssertionError("Browser memory probe returned invalid jsHeapBytes")
return _BrowserMemoryProbe(
streaming_state_storage_bytes=streaming_state_storage_bytes,
debug_event_dom_items=debug_event_dom_items,
js_heap_bytes=js_heap_bytes,
)
def _terminate_browser_processes(root_pid: int, profile_dir: str) -> None:
browser_rows = _collect_browser_process_rows(root_pid, profile_dir)
browser_pids = sorted({row.pid for row in browser_rows} | {root_pid}, reverse=True)
@@ -727,6 +782,7 @@ async def test_devui_streaming_renderer_memory_is_bounded(
peak_renderer_rss_mb = start_renderer_rss_mb
samples: list[tuple[float, float]] = [(0.0, start_renderer_rss_mb)]
probe_samples: list[tuple[float, _BrowserMemoryProbe]] = []
start_time = time.monotonic()
while time.monotonic() - start_time < _SAMPLE_WINDOW_S:
@@ -737,6 +793,10 @@ async def test_devui_streaming_renderer_memory_is_bounded(
elapsed_s = round(time.monotonic() - start_time, 2)
samples.append((elapsed_s, current_sample))
peak_renderer_rss_mb = max(peak_renderer_rss_mb, current_sample)
probe_samples.append((
elapsed_s,
await _sample_browser_memory_probe(client, session_id=session_id),
))
if peak_renderer_rss_mb - start_renderer_rss_mb > _MAX_RENDERER_GROWTH_MB:
break
@@ -744,13 +804,44 @@ async def test_devui_streaming_renderer_memory_is_bounded(
await asyncio.sleep(_SAMPLE_INTERVAL_S)
renderer_growth_mb = round(peak_renderer_rss_mb - start_renderer_rss_mb, 2)
max_streaming_state_storage_bytes = max(
(probe.streaming_state_storage_bytes for _, probe in probe_samples),
default=0,
)
max_debug_event_dom_items = max(
(probe.debug_event_dom_items for _, probe in probe_samples),
default=0,
)
assert renderer_growth_mb <= _MAX_RENDERER_GROWTH_MB, (
"DevUI renderer memory grew too much during a ~1.5 MB streaming response. "
f"start={start_renderer_rss_mb:.2f}MB "
f"peak={peak_renderer_rss_mb:.2f}MB "
f"growth={renderer_growth_mb:.2f}MB "
f"budget={_MAX_RENDERER_GROWTH_MB:.2f}MB "
f"samples={samples}"
f"samples={samples} "
f"probe_samples={probe_samples}"
)
assert max_streaming_state_storage_bytes <= _MAX_STREAMING_STATE_STORAGE_BYTES, (
"DevUI streaming resume state retained too much text in browser storage. "
f"peak={max_streaming_state_storage_bytes} bytes "
f"budget={_MAX_STREAMING_STATE_STORAGE_BYTES} bytes "
f"probe_samples={probe_samples}"
)
assert max_streaming_state_storage_bytes > 0, (
"DevUI streaming state storage was never written during the stress run "
"(cap assertion would be vacuous). "
f"probe_samples={probe_samples}"
)
assert max_debug_event_dom_items <= _MAX_DEBUG_EVENT_DOM_ITEMS, (
"DevUI debug panel rendered too many retained streaming events. "
f"peak={max_debug_event_dom_items} "
f"budget={_MAX_DEBUG_EVENT_DOM_ITEMS} "
f"probe_samples={probe_samples}"
)
assert max_debug_event_dom_items > 0, (
"DevUI debug panel rendered zero events during the stress run "
"(cap assertion would be vacuous). "
f"probe_samples={probe_samples}"
)
finally:
_shutdown_browser_process(browser_process, profile_dir=profile_dir)
+126
View File
@@ -106,3 +106,129 @@ Generally available factories: `get_code_interpreter_tool`,
| `get_browser_automation_tool(connection_id)` | `BrowserAutomationPreviewTool` |
| `get_bing_custom_search_tool(connection_id, instance_name, ...)` | `BingCustomSearchPreviewTool` |
| `get_a2a_tool(base_url=..., project_connection_id=..., ...)` | `A2APreviewTool` |
## Publishing an agent as a Foundry prompt agent
> **Experimental — `ExperimentalFeature.TO_PROMPT_AGENT`.** `to_prompt_agent`
> is a preview API and may change before reaching GA. The warning fires the
> first time the `TO_PROMPT_AGENT` feature is exercised in a process and is
> then deduplicated.
`to_prompt_agent(agent)` converts an `Agent` whose chat client is a
`FoundryChatClient` into a Foundry `PromptAgentDefinition` that can be
published with `AIProjectClient.agents.create_version(...)`. The model is read
from `default_options["model"]` first and falls back to the bound
`FoundryChatClient.model` (matching `Agent.__init__`'s resolution order), so
the same agent definition you run locally can be published as a hosted prompt
agent without restating the model deployment name.
Every generation parameter that has an Agent Framework equivalent is sourced
from `agent.default_options` and translated into the matching Foundry shape by
`_prepare_prompt_agent_options` (a module-private helper in
`agent_framework_foundry._to_prompt_agent` that reuses the chat client's own
request-path helpers):
| `default_options` key | `PromptAgentDefinition` field |
|---|---|
| `temperature` | `temperature` |
| `top_p` | `top_p` |
| `tool_choice` (dropped when no tools) | `tool_choice` (`str` / `ToolChoiceFunction` / `ToolChoiceAllowed`) |
| `reasoning` (dict or `Reasoning`) | `reasoning` |
| `response_format` (dict or `BaseModel`) | `text.format` |
| `verbosity` | `text.verbosity` |
| `text` | merged into `text` |
This keeps the `Agent` as the single source of truth for everything it can
already express. Only Foundry-specific fields with no Agent Framework
equivalent are accepted as keyword arguments on `to_prompt_agent`:
- `structured_inputs``dict[str, StructuredInputDefinition]`
- `rai_config``RaiConfig`
```python
import asyncio
import os
from agent_framework import Agent
from agent_framework.foundry import FoundryChatClient, to_prompt_agent
from azure.ai.projects.aio import AIProjectClient
from azure.identity.aio import AzureCliCredential
async def main() -> None:
credential = AzureCliCredential()
project_endpoint = os.environ["FOUNDRY_PROJECT_ENDPOINT"]
agent = Agent(
client=FoundryChatClient(
project_endpoint=project_endpoint,
model="gpt-4o",
credential=credential,
),
name="travel-agent",
description="Helps Contoso employees book travel.",
instructions="You are a helpful travel assistant.",
tools=[
FoundryChatClient.get_web_search_tool(),
FoundryChatClient.get_code_interpreter_tool(),
],
# Generation parameters set on the Agent flow through automatically.
default_options={
"temperature": 0.3,
"top_p": 0.95,
"reasoning": {"effort": "medium"},
},
)
definition = to_prompt_agent(agent)
project_client = AIProjectClient(endpoint=project_endpoint, credential=credential)
created = await project_client.agents.create_version(
agent_name=agent.name,
definition=definition,
description=agent.description,
)
print(f"Published {created.name} v{created.version}")
asyncio.run(main())
```
Behaviour:
- `agent.client` must be a `FoundryChatClient` (or subclass) — otherwise the
converter raises `TypeError`.
- The bound client must have a `model` set — otherwise the converter raises
`ValueError`.
- Foundry SDK tool instances returned by `FoundryChatClient.get_*_tool()` are
passed through unchanged.
- AF `FunctionTool` instances (and `@tool`-decorated callables) are emitted as
Foundry `FunctionTool` **declarations** — the prompt agent receives the
schema only, not the Python implementation. To execute the function when
invoking the deployed prompt agent, connect with `FoundryAgent` and pass the
same callable via `tools=`:
```python
from agent_framework.foundry import FoundryAgent
deployed = FoundryAgent(
project_endpoint=project_endpoint,
agent_name="travel-agent",
credential=credential,
tools=[book_hotel], # same @tool-decorated callable used at publish time
)
result = await deployed.run("Book me a hotel in Seattle for 3 nights.")
```
`FoundryAgent` runs the function locally when the prompt agent calls it, so
the declaration on the server and the implementation on the client stay in
sync via the shared `@tool` definition.
- Local Agent Framework MCP tools cannot be published as prompt-agent tools —
the converter raises `ValueError` and points at
`FoundryChatClient.get_mcp_tool(...)` for hosted MCP servers.
See the runnable example under `samples/02-agents/providers/foundry/`:
- [`foundry_prompt_agents.py`](../../samples/02-agents/providers/foundry/foundry_prompt_agents.py)
— publish with `to_prompt_agent`, then connect back with `FoundryAgent` and
execute the same local `@tool` callable that the deployed prompt agent
invokes by name.
@@ -16,6 +16,7 @@ from ._foundry_evals import (
evaluate_traces,
)
from ._memory_provider import FoundryMemoryProvider
from ._to_prompt_agent import to_prompt_agent
try:
__version__ = importlib.metadata.version(__name__)
@@ -39,4 +40,5 @@ __all__ = [
"__version__",
"evaluate_foundry_target",
"evaluate_traces",
"to_prompt_agent",
]

Some files were not shown because too many files have changed in this diff Show More