Compare commits

..
Author SHA1 Message Date
SergeyMenshykhandCopilot 547316b523 Add DI wiring verification tests for AddA2AServer
Add three tests to A2AServerServiceCollectionExtensionsTests that verify
custom keyed services are actually wired through to the A2AServer, not
just that the server resolves non-null:

- Custom IAgentHandler: verifies the keyed handler is invoked when
  processing a SendMessageRequest instead of the default A2AAgentHandler.
- Custom AgentSessionStore (no handler): verifies the keyed session
  store's GetSessionAsync is called during request processing when no
  custom handler is registered.
- Default stores end-to-end: verifies the InMemoryAgentSessionStore and
  InMemoryTaskStore defaults successfully process a request. Uses a new
  CreateAgentMockForRequests helper that includes SerializeSessionCoreAsync
  setup needed by InMemoryAgentSessionStore.

All tests call A2AServer.SendMessageAsync directly (no HTTP layer needed)
and use CancellationToken timeouts to guard against hangs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-22 16:25:16 +01:00
SergeyMenshykh 424d66ad74 fix formatting issues 2026-04-22 13:44:22 +01:00
SergeyMenshykh cd1f4c2a93 address automated code review and formatting issues 2026-04-22 13:31:14 +01:00
SergeyMenshykh 8d208f3bb3 address copilot initial feedback 2026-04-22 12:51:51 +01:00
SergeyMenshykh 373482427c restore AsyncEnumerable package version 2026-04-22 11:44:54 +01:00
SergeyMenshykhandGitHub c84203a4a8 Merge branch 'main' into a2a-agent-migration 2026-04-22 11:27:12 +01:00
SergeyMenshykhandCopilot 8f878dcd58 Remove unnecessary using directive in AgentWebChat.AgentHost
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-22 11:20:15 +01:00
c54483f81e .NET: Split A2A endpoint mapping into protocol-specific methods (#5413)
* .NET: Refactor A2A hosting registration into A2AServerServiceCollectionExtensions

- Rename A2AHostingOptions to A2AServerRegistrationOptions
- Move server registration logic from A2AEndpointRouteBuilderExtensions
  and AIAgentExtensions into new A2AServerServiceCollectionExtensions
- Remove A2AProtocolBinding and AIAgentExtensions (consolidated)
- Update samples and tests to use the new registration API

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* address copilot comments

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-22 10:38:48 +01:00
275363d15e .NET: Migrate A2A hosting to A2A SDK v1 (#5363)
* .NET: Migrate A2A hosting to A2A SDK v1

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* remove unused agent card

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-21 15:31:56 +01:00
7417eeb7e6 .NET: Use IA2AClientFactory to create A2AClient (#5277)
* Refactor A2A extensions to use IA2AClientFactory and add ProtocolSelection sample

- Update A2AAgentCardExtensions to accept IA2AClientFactory instead of A2AClientOptions
- Update A2ACardResolverExtensions to accept IA2AClientFactory
- Update A2AClientExtensions to accept IA2AClientFactory
- Update A2AAgent to use IA2AClientFactory for client creation
- Add A2AAgent_ProtocolSelection sample demonstrating protocol selection
- Add comprehensive unit tests for all changes
- Update README files with new sample reference

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Reorder params: options before loggerFactory in A2A extensions

Move A2AClientOptions parameter before ILoggerFactory in AsAIAgent
and GetAIAgentAsync extension methods to follow the repo convention
of keeping LoggerFactory and CancellationToken as the last parameters.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-15 17:29:36 +01:00
6ad9279f0f .NET: Fix stream reconnection for A2AAgent (#5275)
* Add SSE stream reconnection support to A2AAgent

Implement automatic reconnection for SSE streams that disconnect mid-task,
using the Last-Event-ID header to resume from where the stream left off.

Changes:
- Add InvokeStreamingWithReconnectAsync method to A2AAgent with configurable
  max retries and delay between attempts
- Add new log messages for reconnection events
- Add A2AAgent_StreamReconnection sample demonstrating the feature
- Update existing polling sample to use simplified SendMessageAsync API
- Add unit tests for stream reconnection logic

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* address comments

* Address PR review feedback

- Dispose SSE enumerator before GetTaskAsync fallback to release HTTP connection
- Wrap StreamWriter in using blocks with leaveOpen:true and explicit UTF-8 encoding
- Print update.Text instead of update object in stream reconnection sample

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-15 14:45:11 +01:00
c1bbaeb31d Move A2A samples from 04-hosting to 02-agents (#5267)
Move the A2A sample projects (A2AAgent_AsFunctionTools and
A2AAgent_PollingForTaskCompletion) from samples/04-hosting/A2A/ to
samples/02-agents/A2A/ to better align with the sample directory
structure. Update solution file and samples README accordingly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-15 12:08:21 +01:00
SergeyMenshykhandGitHub 6173e63f0b update a2a agent to the latest a2a sdk (#5257) 2026-04-15 11:08:05 +01:00
52 changed files with 197 additions and 1697 deletions
-4
View File
@@ -108,10 +108,6 @@ jobs:
needs: team_check
if: ${{ needs.team_check.outputs.is_team_member == 'true' }}
timeout-minutes: 60
# Advisory check: failures here should not block the PR. The reviewer
# posts comments as a best-effort signal; if the pipeline breaks, the
# PR author should still be able to merge without a red required check.
continue-on-error: true
steps:
# Safe checkout: base repo only, not the untrusted PR head.
+1 -114
View File
@@ -87,14 +87,6 @@ jobs:
-n logical --dist worksteal
--timeout=120 --session-timeout=900 --timeout_method thread
--retries 2 --retry-delay 5
--junitxml=pytest.xml
- name: Upload test results
if: always()
uses: actions/upload-artifact@v7
with:
name: test-results-openai
path: ./python/pytest.xml
if-no-files-found: ignore
# Azure OpenAI integration tests
python-tests-azure-openai:
@@ -138,14 +130,6 @@ jobs:
-n logical --dist worksteal
--timeout=120 --session-timeout=900 --timeout_method thread
--retries 2 --retry-delay 5
--junitxml=pytest.xml
- name: Upload test results
if: always()
uses: actions/upload-artifact@v7
with:
name: test-results-azure-openai
path: ./python/pytest.xml
if-no-files-found: ignore
# Misc integration tests (Anthropic, Hyperlight, Ollama, MCP)
python-tests-misc-integration:
@@ -189,14 +173,6 @@ jobs:
-n logical --dist worksteal
--timeout=120 --session-timeout=900 --timeout_method thread
--retries 2 --retry-delay 30
--junitxml=pytest.xml
- name: Upload test results
if: always()
uses: actions/upload-artifact@v7
with:
name: test-results-misc
path: ./python/pytest.xml
if-no-files-found: ignore
- name: Stop local MCP server
if: always()
shell: bash
@@ -273,14 +249,6 @@ jobs:
-x
--timeout=360 --session-timeout=900 --timeout_method thread
--retries 2 --retry-delay 5
--junitxml=pytest.xml
- name: Upload test results
if: always()
uses: actions/upload-artifact@v7
with:
name: test-results-functions
path: ./python/pytest.xml
if-no-files-found: ignore
# Foundry integration tests
python-tests-foundry:
@@ -327,14 +295,6 @@ jobs:
-n logical --dist worksteal
--timeout=120 --session-timeout=900 --timeout_method thread
--retries 2 --retry-delay 5
--junitxml=pytest.xml
- name: Upload test results
if: always()
uses: actions/upload-artifact@v7
with:
name: test-results-foundry
path: ./python/pytest.xml
if-no-files-found: ignore
# Azure Cosmos integration tests
python-tests-cosmos:
@@ -379,80 +339,7 @@ jobs:
echo "Cosmos DB emulator did not become ready in time." >&2
exit 1
- name: Test with pytest (Cosmos integration)
run: uv run --directory packages/azure-cosmos poe integration-tests -n logical --dist worksteal --timeout=120 --session-timeout=900 --timeout_method thread --retries 2 --retry-delay 5 --junitxml=${{ github.workspace }}/python/pytest.xml
- name: Upload test results
if: always()
uses: actions/upload-artifact@v7
with:
name: test-results-cosmos
path: ./python/pytest.xml
if-no-files-found: ignore
# Flaky test trend report (aggregates per-job JUnit XML results)
python-flaky-test-report:
name: Flaky Test Report
if: >
always() &&
(contains(join(needs.*.result, ','), 'success') ||
contains(join(needs.*.result, ','), 'failure'))
needs:
[
python-tests-openai,
python-tests-azure-openai,
python-tests-misc-integration,
python-tests-functions,
python-tests-foundry,
python-tests-cosmos,
]
runs-on: ubuntu-latest
defaults:
run:
working-directory: python
steps:
- uses: actions/checkout@v6
with:
ref: ${{ inputs.checkout-ref }}
persist-credentials: false
- name: Set up python and install the project
uses: ./.github/actions/python-setup
with:
python-version: ${{ env.UV_PYTHON }}
os: ${{ runner.os }}
- name: Download all test results from current run
uses: actions/download-artifact@v4
with:
pattern: test-results-*
path: test-results/
- name: Restore flaky report history cache
uses: actions/cache/restore@v4
with:
path: python/flaky-report-history.json
key: flaky-report-history-integration-${{ github.run_id }}
restore-keys: |
flaky-report-history-integration-
- name: Generate trend report
run: >
uv run python scripts/flaky_report/aggregate.py
../test-results/
flaky-report-history.json
flaky-test-report.md
- name: Post to Job Summary
if: always()
run: cat flaky-test-report.md >> $GITHUB_STEP_SUMMARY
- name: Save flaky report history cache
if: always()
uses: actions/cache/save@v4
with:
path: python/flaky-report-history.json
key: flaky-report-history-integration-${{ github.run_id }}
- name: Upload unified trend report
if: always()
uses: actions/upload-artifact@v7
with:
name: flaky-test-report
path: |
python/flaky-test-report.md
python/flaky-report-history.json
run: uv run --directory packages/azure-cosmos poe integration-tests -n logical --dist worksteal --timeout=120 --session-timeout=900 --timeout_method thread --retries 2 --retry-delay 5
python-integration-tests-check:
if: always()
+1 -110
View File
@@ -181,13 +181,6 @@ jobs:
display-options: fEX
fail-on-empty: false
title: OpenAI integration test results
- name: Upload test results
if: always()
uses: actions/upload-artifact@v7
with:
name: test-results-openai
path: ./python/pytest.xml
if-no-files-found: ignore
# Azure OpenAI integration tests
python-tests-azure-openai:
@@ -251,13 +244,6 @@ jobs:
display-options: fEX
fail-on-empty: false
title: Azure OpenAI integration test results
- name: Upload test results
if: always()
uses: actions/upload-artifact@v7
with:
name: test-results-azure-openai
path: ./python/pytest.xml
if-no-files-found: ignore
# Misc integration tests (Anthropic, Ollama, MCP)
python-tests-misc-integration:
@@ -335,13 +321,6 @@ jobs:
display-options: fEX
fail-on-empty: false
title: Misc integration test results
- name: Upload test results
if: always()
uses: actions/upload-artifact@v7
with:
name: test-results-misc
path: ./python/pytest.xml
if-no-files-found: ignore
# Azure Functions + Durable Task integration tests
python-tests-functions:
@@ -413,13 +392,6 @@ jobs:
display-options: fEX
fail-on-empty: false
title: Functions integration test results
- name: Upload test results
if: always()
uses: actions/upload-artifact@v7
with:
name: test-results-functions
path: ./python/pytest.xml
if-no-files-found: ignore
python-tests-foundry:
name: Python Integration Tests - Foundry
@@ -437,10 +409,6 @@ jobs:
FOUNDRY_MODEL: ${{ vars.FOUNDRY_MODEL }}
FOUNDRY_AGENT_NAME: ${{ vars.FOUNDRY_AGENT_NAME }}
FOUNDRY_AGENT_VERSION: ${{ vars.FOUNDRY_AGENT_VERSION }}
FOUNDRY_MODELS_ENDPOINT: ${{ vars.FOUNDRY_MODELS_ENDPOINT || '' }}
FOUNDRY_MODELS_API_KEY: ${{ secrets.FOUNDRY_MODELS_API_KEY || '' }}
FOUNDRY_EMBEDDING_MODEL: ${{ vars.FOUNDRY_EMBEDDING_MODEL || '' }}
FOUNDRY_IMAGE_EMBEDDING_MODEL: ${{ vars.FOUNDRY_IMAGE_EMBEDDING_MODEL || '' }}
LOCAL_MCP_URL: ${{ vars.LOCAL_MCP__URL }}
defaults:
run:
@@ -480,13 +448,6 @@ jobs:
display-options: fEX
fail-on-empty: false
title: Test results
- name: Upload test results
if: always()
uses: actions/upload-artifact@v7
with:
name: test-results-foundry
path: ./python/pytest.xml
if-no-files-found: ignore
# TODO: Add python-tests-lab
@@ -536,7 +497,7 @@ jobs:
echo "Cosmos DB emulator did not become ready in time." >&2
exit 1
- name: Test with pytest (Cosmos integration)
run: uv run --directory packages/azure-cosmos poe integration-tests -n logical --dist worksteal --timeout=120 --session-timeout=900 --timeout_method thread --retries 2 --retry-delay 5 --junitxml=${{ github.workspace }}/python/pytest.xml
run: uv run --directory packages/azure-cosmos poe integration-tests -n logical --dist worksteal --timeout=120 --session-timeout=900 --timeout_method thread --retries 2 --retry-delay 5 --junitxml=pytest.xml
working-directory: ./python
- name: Surface failing tests
if: always()
@@ -547,76 +508,6 @@ jobs:
display-options: fEX
fail-on-empty: false
title: Cosmos integration test results
- name: Upload test results
if: always()
uses: actions/upload-artifact@v7
with:
name: test-results-cosmos
path: ./python/pytest.xml
if-no-files-found: ignore
# Flaky test trend report (aggregates per-job JUnit XML results)
python-flaky-test-report:
name: Flaky Test Report
if: >
always() &&
(contains(join(needs.*.result, ','), 'success') ||
contains(join(needs.*.result, ','), 'failure'))
needs:
[
python-tests-openai,
python-tests-azure-openai,
python-tests-misc-integration,
python-tests-functions,
python-tests-foundry,
python-tests-cosmos,
]
runs-on: ubuntu-latest
defaults:
run:
working-directory: python
steps:
- uses: actions/checkout@v6
- name: Set up python and install the project
uses: ./.github/actions/python-setup
with:
python-version: ${{ env.UV_PYTHON }}
os: ${{ runner.os }}
- name: Download all test results from current run
uses: actions/download-artifact@v4
with:
pattern: test-results-*
path: test-results/
- name: Restore flaky report history cache
uses: actions/cache/restore@v4
with:
path: python/flaky-report-history.json
key: flaky-report-history-merge-${{ github.run_id }}
restore-keys: |
flaky-report-history-merge-
- name: Generate trend report
run: >
uv run python scripts/flaky_report/aggregate.py
../test-results/
flaky-report-history.json
flaky-test-report.md
- name: Post to Job Summary
if: always()
run: cat flaky-test-report.md >> $GITHUB_STEP_SUMMARY
- name: Save flaky report history cache
if: always()
uses: actions/cache/save@v4
with:
path: python/flaky-report-history.json
key: flaky-report-history-merge-${{ github.run_id }}
- name: Upload unified trend report
if: always()
uses: actions/upload-artifact@v7
with:
name: flaky-test-report
path: |
python/flaky-test-report.md
python/flaky-report-history.json
python-integration-tests-check:
if: always()
-18
View File
@@ -7,24 +7,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
## [1.1.1] - 2026-04-23
### Added
- **agent-framework-core**: Add `expected_output` ground-truth support to `evaluate_workflow` for similarity evaluators ([#5234](https://github.com/microsoft/agent-framework/pull/5234))
- **agent-framework-ag-ui**, **agent-framework-a2a**: Propagate `thread_id` and `forwarded_props` through AG-UI to A2A `context_id` ([#5383](https://github.com/microsoft/agent-framework/pull/5383))
- **samples**: Add second approval-required tool (`set_stop_loss`) to `concurrent_builder_tool_approval` sample ([#4875](https://github.com/microsoft/agent-framework/pull/4875))
### Changed
- **agent-framework-foundry-hosting**: Correct Development Status classifier from Beta (4) to Alpha (3) to match the package's lifecycle stage ([#5387](https://github.com/microsoft/agent-framework/pull/5387))
- **tests**: Add Python flaky test report workflow ([#5342](https://github.com/microsoft/agent-framework/pull/5342))
### Fixed
- **agent-framework-openai**: Fix OpenAI Responses streaming to propagate `created_at` from the final `response.completed` event ([#5382](https://github.com/microsoft/agent-framework/pull/5382))
- **agent-framework-openai**: Fix `OpenAIEmbeddingClient` to use `AsyncOpenAI` for `/openai/v1` endpoints ([#5137](https://github.com/microsoft/agent-framework/pull/5137))
- **agent-framework-openai**: Exclude null `file_id` from `input_image` payload to prevent schema 400 errors ([#5125](https://github.com/microsoft/agent-framework/pull/5125))
- **agent-framework-foundry**: Reconcile Toolbox hosted-tool payloads with the Responses API ([#5414](https://github.com/microsoft/agent-framework/pull/5414))
- **agent-framework-ag-ui**: Pass client `thread_id` as `session_id` when constructing `AgentSession` ([#5384](https://github.com/microsoft/agent-framework/pull/5384))
## [1.1.0] - 2026-04-21
### Added
@@ -295,10 +295,7 @@ class A2AAgent(AgentTelemetryLayer, BaseAgent):
else:
if not normalized_messages:
raise ValueError("At least one message is required when starting a new task (no continuation_token).")
a2a_message = self._prepare_message_for_a2a(
normalized_messages[-1],
context_id=session.service_session_id if session else None,
)
a2a_message = self._prepare_message_for_a2a(normalized_messages[-1])
a2a_stream = self.client.send_message(a2a_message)
provider_session = session
@@ -587,7 +584,7 @@ class A2AAgent(AgentTelemetryLayer, BaseAgent):
return AgentResponse.from_updates(updates)
return AgentResponse(messages=[], response_id=task.id, raw_representation=task)
def _prepare_message_for_a2a(self, message: Message, *, context_id: str | None = None) -> A2AMessage:
def _prepare_message_for_a2a(self, message: Message) -> A2AMessage:
"""Prepare a Message for the A2A protocol.
Transforms Agent Framework Message objects into A2A protocol Messages by:
@@ -596,13 +593,6 @@ class A2AAgent(AgentTelemetryLayer, BaseAgent):
- Converting file references (URI/data/hosted_file) to FilePart objects
- Preserving metadata and additional properties from the original message
- Setting the role to 'user' as framework messages are treated as user input
Args:
message: The framework Message to convert.
context_id: Optional fallback context identifier (e.g. derived from
``AgentSession.service_session_id``). When the *message* already
carries a ``context_id`` in its ``additional_properties`` that
value takes precedence; otherwise this fallback is used.
"""
parts: list[A2APart] = []
if not message.contents:
@@ -682,7 +672,7 @@ class A2AAgent(AgentTelemetryLayer, BaseAgent):
role=A2ARole("user"),
parts=parts,
message_id=message.message_id or uuid.uuid4().hex,
context_id=message.additional_properties.get("context_id") or context_id,
context_id=message.additional_properties.get("context_id"),
metadata=metadata,
)
+2 -2
View File
@@ -4,7 +4,7 @@ description = "A2A integration for Microsoft Agent Framework."
authors = [{ name = "Microsoft", email = "af-support@microsoft.com"}]
readme = "README.md"
requires-python = ">=3.10"
version = "1.0.0b260423"
version = "1.0.0b260421"
license-files = ["LICENSE"]
urls.homepage = "https://aka.ms/agent-framework"
urls.source = "https://github.com/microsoft/agent-framework/tree/main/python"
@@ -23,7 +23,7 @@ classifiers = [
"Typing :: Typed",
]
dependencies = [
"agent-framework-core>=1.1.1,<2",
"agent-framework-core>=1.1.0,<2",
"a2a-sdk>=0.3.5,<0.3.24",
]
@@ -46,7 +46,6 @@ class MockA2AClient:
self.responses: list[Any] = []
self.resubscribe_responses: list[Any] = []
self.get_task_response: Task | None = None
self.last_message: Any = None
def add_message_response(self, message_id: str, text: str, role: str = "agent") -> None:
"""Add a mock Message response."""
@@ -112,7 +111,6 @@ class MockA2AClient:
async def send_message(self, message: Any) -> AsyncIterator[Any]:
"""Mock send_message method that yields responses."""
self.last_message = message
self.call_count += 1
# All queued responses are delivered as a single streaming batch per call.
@@ -541,37 +539,6 @@ def test_prepare_message_for_a2a_forwards_context_id() -> None:
assert result.metadata == {"trace_id": "trace-456"}
def test_prepare_message_for_a2a_uses_fallback_context_id() -> None:
"""Test that context_id kwarg is used when message has no context_id property."""
agent = A2AAgent(client=MagicMock(), http_client=None)
message = Message(
role="user",
contents=[Content.from_text(text="Hello")],
)
result = agent._prepare_message_for_a2a(message, context_id="session-ctx-1")
assert result.context_id == "session-ctx-1"
def test_prepare_message_for_a2a_message_context_id_takes_precedence() -> None:
"""Test that message.additional_properties context_id wins over the fallback."""
agent = A2AAgent(client=MagicMock(), http_client=None)
message = Message(
role="user",
contents=[Content.from_text(text="Hello")],
additional_properties={"context_id": "explicit-ctx"},
)
result = agent._prepare_message_for_a2a(message, context_id="session-ctx-1")
assert result.context_id == "explicit-ctx"
def test_parse_contents_from_a2a_with_data_part() -> None:
"""Test conversion of A2A DataPart."""
@@ -901,43 +868,6 @@ async def test_poll_task_completed(a2a_agent: A2AAgent, mock_a2a_client: MockA2A
# endregion
# region Session context_id Integration Tests
@mark.asyncio
async def test_run_passes_session_service_session_id_as_context_id(mock_a2a_client: MockA2AClient) -> None:
"""Test that run() wires session.service_session_id to the A2A message context_id."""
agent = A2AAgent(name="Test Agent", id="test-agent", client=mock_a2a_client, http_client=None)
mock_a2a_client.add_message_response("msg-ctx", "reply")
session = AgentSession(service_session_id="svc-session-42")
await agent.run("Hello", session=session)
assert mock_a2a_client.last_message is not None
assert mock_a2a_client.last_message.context_id == "svc-session-42"
@mark.asyncio
async def test_run_message_context_id_takes_precedence_over_session(mock_a2a_client: MockA2AClient) -> None:
"""Test that an explicit context_id on the message wins over session.service_session_id."""
agent = A2AAgent(name="Test Agent", id="test-agent", client=mock_a2a_client, http_client=None)
mock_a2a_client.add_message_response("msg-ctx2", "reply")
session = AgentSession(service_session_id="svc-session-42")
message = Message(
role="user",
contents=[Content.from_text(text="Hello")],
additional_properties={"context_id": "explicit-ctx"},
)
await agent.run(messages=[message], session=session)
assert mock_a2a_client.last_message is not None
assert mock_a2a_client.last_message.context_id == "explicit-ctx"
# endregion
# region Context Provider Tests
@@ -790,9 +790,9 @@ async def run_agent_stream(
# Create session (with service session support)
if config.use_service_session:
supplied_thread_id = input_data.get("thread_id") or input_data.get("threadId")
session = AgentSession(session_id=thread_id, service_session_id=supplied_thread_id)
session = AgentSession(service_session_id=supplied_thread_id)
else:
session = AgentSession(session_id=thread_id)
session = AgentSession()
# Inject metadata for AG-UI orchestration (Feature #2: Azure-safe truncation)
base_metadata: dict[str, Any] = {
+2 -2
View File
@@ -1,6 +1,6 @@
[project]
name = "agent-framework-ag-ui"
version = "1.0.0b260423"
version = "1.0.0b260421"
description = "AG-UI protocol integration for Agent Framework"
readme = "README.md"
license-files = ["LICENSE"]
@@ -22,7 +22,7 @@ classifiers = [
"Typing :: Typed",
]
dependencies = [
"agent-framework-core>=1.1.1,<2",
"agent-framework-core>=1.1.0,<2",
"ag-ui-protocol==0.1.13",
"fastapi>=0.115.0,<0.133.1",
"uvicorn[standard]>=0.30.0,<0.42.0"
@@ -183,7 +183,6 @@ class StubAgent(SupportsAgentRun):
self.client = client or SimpleNamespace(function_invocation_configuration=None)
self.messages_received: list[Any] = []
self.tools_received: list[Any] | None = None
self.last_session: AgentSession | None = None
@overload
def run(
@@ -217,7 +216,6 @@ class StubAgent(SupportsAgentRun):
async def _stream() -> AsyncIterator[AgentResponseUpdate]:
self.messages_received = [] if messages is None else list(messages) # type: ignore[arg-type]
self.last_session = session
self.tools_received = kwargs.get("tools")
for update in self.updates:
yield update
@@ -1640,115 +1640,3 @@ class TestReasoningInSnapshot:
# close: MsgEnd(block2) + End(block2)
assert isinstance(close[0], ReasoningMessageEndEvent)
assert close[0].message_id == "block2"
async def test_session_id_matches_thread_id():
"""Session created by run_agent_stream uses the client thread_id as session_id."""
from conftest import StubAgent
from agent_framework_ag_ui import AgentFrameworkAgent
stub = StubAgent()
agent = AgentFrameworkAgent(agent=stub)
payload = {
"thread_id": "my-thread-123",
"run_id": "run-1",
"messages": [{"role": "user", "content": "Hello"}],
}
_ = [event async for event in agent.run(payload)]
assert stub.last_session is not None
assert stub.last_session.session_id == "my-thread-123"
async def test_session_id_matches_camel_case_thread_id():
"""Session uses threadId (camelCase) as session_id when snake_case is absent."""
from conftest import StubAgent
from agent_framework_ag_ui import AgentFrameworkAgent
stub = StubAgent()
agent = AgentFrameworkAgent(agent=stub)
payload = {
"threadId": "camel-thread-456",
"run_id": "run-2",
"messages": [{"role": "user", "content": "Hello"}],
}
_ = [event async for event in agent.run(payload)]
assert stub.last_session is not None
assert stub.last_session.session_id == "camel-thread-456"
async def test_session_id_matches_thread_id_with_service_session():
"""Session uses thread_id as session_id even when use_service_session is enabled."""
from conftest import StubAgent
from agent_framework_ag_ui import AgentFrameworkAgent
stub = StubAgent()
agent = AgentFrameworkAgent(agent=stub, use_service_session=True)
payload = {
"thread_id": "service-thread-789",
"run_id": "run-3",
"messages": [{"role": "user", "content": "Hello"}],
}
_ = [event async for event in agent.run(payload)]
assert stub.last_session is not None
assert stub.last_session.session_id == "service-thread-789"
assert stub.last_session.service_session_id == "service-thread-789"
async def test_session_id_generated_when_no_thread_id():
"""Session gets a generated UUID as session_id when no thread_id is provided."""
import uuid
from conftest import StubAgent
from agent_framework_ag_ui import AgentFrameworkAgent
stub = StubAgent()
agent = AgentFrameworkAgent(agent=stub)
payload = {
"run_id": "run-4",
"messages": [{"role": "user", "content": "Hello"}],
}
_ = [event async for event in agent.run(payload)]
assert stub.last_session is not None
# Should be a valid UUID (auto-generated)
uuid.UUID(stub.last_session.session_id)
async def test_service_session_no_thread_id_generates_uuid():
"""With use_service_session=True and no thread_id, session_id is a UUID and service_session_id is None."""
import uuid
from conftest import StubAgent
from agent_framework_ag_ui import AgentFrameworkAgent
stub = StubAgent()
agent = AgentFrameworkAgent(agent=stub, use_service_session=True)
payload = {
"run_id": "run-5",
"messages": [{"role": "user", "content": "Hello"}],
}
_ = [event async for event in agent.run(payload)]
assert stub.last_session is not None
# session_id should be a valid auto-generated UUID
uuid.UUID(stub.last_session.session_id)
# service_session_id should be None since no thread_id was supplied
assert stub.last_session.service_session_id is None
+2 -2
View File
@@ -4,7 +4,7 @@ description = "Anthropic integration for Microsoft Agent Framework."
authors = [{ name = "Microsoft", email = "af-support@microsoft.com"}]
readme = "README.md"
requires-python = ">=3.10"
version = "1.0.0b260423"
version = "1.0.0b260421"
license-files = ["LICENSE"]
urls.homepage = "https://aka.ms/agent-framework"
urls.source = "https://github.com/microsoft/agent-framework/tree/main/python"
@@ -23,7 +23,7 @@ classifiers = [
"Typing :: Typed",
]
dependencies = [
"agent-framework-core>=1.1.1,<2",
"agent-framework-core>=1.1.0,<2",
"anthropic>=0.80.0,<0.80.1",
]
@@ -4,7 +4,7 @@ description = "Azure AI Search integration for Microsoft Agent Framework."
authors = [{ name = "Microsoft", email = "af-support@microsoft.com"}]
readme = "README.md"
requires-python = ">=3.10"
version = "1.0.0b260423"
version = "1.0.0b260421"
license-files = ["LICENSE"]
urls.homepage = "https://aka.ms/agent-framework"
urls.source = "https://github.com/microsoft/agent-framework/tree/main/python"
@@ -23,7 +23,7 @@ classifiers = [
"Typing :: Typed",
]
dependencies = [
"agent-framework-core>=1.1.1,<2",
"agent-framework-core>=1.1.0,<2",
"azure-search-documents>=11.7.0b2,<11.7.0b3",
]
+2 -2
View File
@@ -4,7 +4,7 @@ description = "Azure Cosmos DB history provider integration for Microsoft Agent
authors = [{ name = "Microsoft", email = "af-support@microsoft.com"}]
readme = "README.md"
requires-python = ">=3.10"
version = "1.0.0b260423"
version = "1.0.0b260421"
license-files = ["LICENSE"]
urls.homepage = "https://aka.ms/agent-framework"
urls.source = "https://github.com/microsoft/agent-framework/tree/main/python"
@@ -23,7 +23,7 @@ classifiers = [
"Typing :: Typed",
]
dependencies = [
"agent-framework-core>=1.1.1,<2",
"agent-framework-core>=1.1.0,<2",
"azure-cosmos>=4.3.0,<5",
]
@@ -4,7 +4,7 @@ description = "Azure Functions integration for Microsoft Agent Framework."
authors = [{ name = "Microsoft", email = "af-support@microsoft.com"}]
readme = "README.md"
requires-python = ">=3.10"
version = "1.0.0b260423"
version = "1.0.0b260421"
license-files = ["LICENSE"]
urls.homepage = "https://aka.ms/agent-framework"
urls.source = "https://github.com/microsoft/agent-framework/tree/main/python"
@@ -22,7 +22,7 @@ classifiers = [
"Typing :: Typed",
]
dependencies = [
"agent-framework-core>=1.1.1,<2",
"agent-framework-core>=1.1.0,<2",
"agent-framework-durabletask",
"azure-functions>=1.24.0,<2",
"azure-functions-durable>=1.3.1,<2",
+2 -2
View File
@@ -4,7 +4,7 @@ description = "Amazon Bedrock integration for Microsoft Agent Framework."
authors = [{ name = "Microsoft", email = "af-support@microsoft.com"}]
readme = "README.md"
requires-python = ">=3.10"
version = "1.0.0b260423"
version = "1.0.0b260421"
license-files = ["LICENSE"]
urls.homepage = "https://aka.ms/agent-framework"
urls.source = "https://github.com/microsoft/agent-framework/tree/main/python"
@@ -23,7 +23,7 @@ classifiers = [
"Typing :: Typed",
]
dependencies = [
"agent-framework-core>=1.1.1,<2",
"agent-framework-core>=1.1.0,<2",
"boto3>=1.35.0,<2.0.0",
"botocore>=1.35.0,<2.0.0",
]
+2 -2
View File
@@ -4,7 +4,7 @@ description = "OpenAI ChatKit integration for Microsoft Agent Framework."
authors = [{ name = "Microsoft", email = "af-support@microsoft.com"}]
readme = "README.md"
requires-python = ">=3.10"
version = "1.0.0b260423"
version = "1.0.0b260421"
license-files = ["LICENSE"]
urls.homepage = "https://aka.ms/agent-framework"
urls.source = "https://github.com/microsoft/agent-framework/tree/main/python"
@@ -22,7 +22,7 @@ classifiers = [
"Typing :: Typed",
]
dependencies = [
"agent-framework-core>=1.1.1,<2",
"agent-framework-core>=1.1.0,<2",
"openai-chatkit>=1.4.1,<2.0.0",
]
+2 -2
View File
@@ -4,7 +4,7 @@ description = "Claude Agent SDK integration for Microsoft Agent Framework."
authors = [{ name = "Microsoft", email = "af-support@microsoft.com"}]
readme = "README.md"
requires-python = ">=3.10"
version = "1.0.0b260423"
version = "1.0.0b260421"
license-files = ["LICENSE"]
urls.homepage = "https://aka.ms/agent-framework"
urls.source = "https://github.com/microsoft/agent-framework/tree/main/python"
@@ -23,7 +23,7 @@ classifiers = [
"Typing :: Typed",
]
dependencies = [
"agent-framework-core>=1.1.1,<2",
"agent-framework-core>=1.1.0,<2",
"claude-agent-sdk>=0.1.36,<0.1.49",
]
+2 -2
View File
@@ -4,7 +4,7 @@ description = "Copilot Studio integration for Microsoft Agent Framework."
authors = [{ name = "Microsoft", email = "af-support@microsoft.com"}]
readme = "README.md"
requires-python = ">=3.10"
version = "1.0.0b260423"
version = "1.0.0b260421"
license-files = ["LICENSE"]
urls.homepage = "https://aka.ms/agent-framework"
urls.source = "https://github.com/microsoft/agent-framework/tree/main/python"
@@ -23,7 +23,7 @@ classifiers = [
"Typing :: Typed",
]
dependencies = [
"agent-framework-core>=1.1.1,<2",
"agent-framework-core>=1.1.0,<2",
"microsoft-agents-copilotstudio-client>=0.3.1,<0.3.2",
]
@@ -125,7 +125,6 @@ from ._telemetry import (
prepend_agent_framework_to_user_agent,
)
from ._tools import (
SKIP_PARSING,
FunctionInvocationConfiguration,
FunctionInvocationLayer,
FunctionTool,
@@ -259,7 +258,6 @@ __all__ = [
"GROUP_INDEX_KEY",
"GROUP_KIND_KEY",
"GROUP_TOKEN_COUNT_KEY",
"SKIP_PARSING",
"SUMMARIZED_BY_SUMMARY_ID_KEY",
"SUMMARY_OF_GROUP_IDS_KEY",
"SUMMARY_OF_MESSAGE_IDS_KEY",
+12 -88
View File
@@ -94,33 +94,6 @@ ApprovalMode: TypeAlias = Literal["always_require", "never_require"]
ChatClientT = TypeVar("ChatClientT", bound="SupportsChatGetResponse[Any]")
ResponseModelBoundT = TypeVar("ResponseModelBoundT", bound=BaseModel)
class _SkipParsingSentinel:
"""Sentinel signaling that :meth:`FunctionTool.invoke` should return the raw value.
When passed as ``result_parser`` to :class:`FunctionTool` (or the ``@tool`` decorator),
the default :meth:`FunctionTool.parse_result` is bypassed and the wrapped function's
return value is returned unchanged from :meth:`FunctionTool.invoke`. Callers may also
request the raw value on a per-call basis by passing ``skip_parsing=True`` to
:meth:`FunctionTool.invoke`.
Use the module-level ``SKIP_PARSING`` singleton — do not instantiate this class.
"""
_instance: ClassVar[_SkipParsingSentinel | None] = None
def __new__(cls) -> _SkipParsingSentinel:
if cls._instance is None:
cls._instance = super().__new__(cls)
return cls._instance
def __repr__(self) -> str:
return "SKIP_PARSING"
SKIP_PARSING: Final[_SkipParsingSentinel] = _SkipParsingSentinel()
"""Sentinel for ``FunctionTool(result_parser=...)`` meaning "do not parse the result"."""
# region Helpers
@@ -306,7 +279,7 @@ class FunctionTool(SerializationMixin):
additional_properties: dict[str, Any] | None = None,
func: Callable[..., Any] | None = None,
input_model: type[BaseModel] | Mapping[str, Any] | None = None,
result_parser: Callable[[Any], str | list[Content]] | _SkipParsingSentinel | None = None,
result_parser: Callable[[Any], str | list[Content]] | None = None,
**kwargs: Any,
) -> None:
"""Initialize the FunctionTool.
@@ -354,11 +327,9 @@ class FunctionTool(SerializationMixin):
result_parser: An optional callable with signature ``Callable[[Any], str]`` that
overrides the default result parsing behavior. When provided, this callable
is used to convert the raw function return value to a string instead of the
built-in :meth:`parse_result` logic. Pass the :data:`SKIP_PARSING` sentinel
instead of a callable to opt out of parsing entirely; in that case
:meth:`invoke` returns the wrapped function's raw return value. Depending
on your function, it may be easiest to just do the serialization directly
in the function body rather than providing a custom ``result_parser``.
built-in :meth:`parse_result` logic. Depending on your function, it may be
easiest to just do the serialization directly in the function body rather
than providing a custom ``result_parser``.
**kwargs: Additional keyword arguments.
"""
# Core attributes (formerly from BaseTool)
@@ -537,65 +508,31 @@ class FunctionTool(SerializationMixin):
self.invocation_exception_count += 1
raise
@overload
async def invoke(
self,
*,
arguments: BaseModel | Mapping[str, Any] | None = None,
context: FunctionInvocationContext | None = None,
tool_call_id: str | None = None,
skip_parsing: Literal[True],
**kwargs: Any,
) -> Any: ...
@overload
async def invoke(
self,
*,
arguments: BaseModel | Mapping[str, Any] | None = None,
context: FunctionInvocationContext | None = None,
tool_call_id: str | None = None,
skip_parsing: Literal[False] = False,
**kwargs: Any,
) -> list[Content]: ...
async def invoke(
self,
*,
arguments: BaseModel | Mapping[str, Any] | None = None,
context: FunctionInvocationContext | None = None,
tool_call_id: str | None = None,
skip_parsing: bool = False,
**kwargs: Any,
) -> list[Content] | Any:
) -> list[Content]:
"""Run the AI function with the provided arguments as a Pydantic model.
The raw return value of the wrapped function is automatically parsed into a
``list[Content]`` using :meth:`parse_result` or the custom ``result_parser``
configured on the tool. Every result — text, rich media, or serialized
objects — is represented uniformly as Content items.
Parsing can be skipped in two ways: configure the tool with
``result_parser=SKIP_PARSING`` to always skip parsing, or pass
``skip_parsing=True`` per call. Either way the wrapped function's raw value
is returned. This is intended for callers (e.g. sandboxed runtimes) that
consume the value from Python directly and would otherwise undo the
``Content`` wrapping.
if one was provided. Every result — text, rich media, or serialized objects —
is represented uniformly as Content items.
Keyword Args:
arguments: A mapping or model instance containing the arguments for the function.
context: Explicit function invocation context carrying runtime kwargs.
tool_call_id: Optional tool call identifier used for telemetry and tracing.
skip_parsing: When ``True``, bypass parsing and return the wrapped function's
raw value instead of a ``list[Content]``. Defaults to ``False``.
kwargs: Direct function argument values. When provided, every keyword
must match a declared tool parameter. Runtime data must be passed
via ``context``.
Returns:
``list[Content]`` by default. The raw function return value (``Any``) when
``skip_parsing=True`` (or the tool was constructed with
``result_parser=SKIP_PARSING``).
A list of Content items representing the tool output.
Raises:
TypeError: If arguments is not mapping-like or fails schema checks.
@@ -607,9 +544,7 @@ class FunctionTool(SerializationMixin):
from ._types import Content
from .observability import OBSERVABILITY_SETTINGS
configured_parser = self.result_parser
skip_parsing = skip_parsing or configured_parser is SKIP_PARSING
parser = configured_parser if callable(configured_parser) else FunctionTool.parse_result
parser = self.result_parser or FunctionTool.parse_result
parameter_names = set(self.parameters().get("properties", {}).keys())
direct_argument_kwargs = (
@@ -681,10 +616,6 @@ class FunctionTool(SerializationMixin):
logger.debug(f"Function arguments: {observable_kwargs}")
res = self.__call__(**call_kwargs)
result = await res if inspect.isawaitable(res) else res
if skip_parsing:
logger.info(f"Function {self.name} succeeded.")
logger.debug(f"Function result: {type(result).__name__}")
return result
try:
parsed = parser(result)
except Exception:
@@ -740,13 +671,6 @@ class FunctionTool(SerializationMixin):
logger.error(f"Function failed. Error: {exception}")
raise
else:
if skip_parsing:
logger.info(f"Function {self.name} succeeded.")
if OBSERVABILITY_SETTINGS.SENSITIVE_DATA_ENABLED: # type: ignore[name-defined]
result_str = str(result)
span.set_attribute(OtelAttr.TOOL_RESULT, result_str)
logger.debug(f"Function result: {result_str}")
return result
try:
parsed = parser(result)
except Exception:
@@ -1143,7 +1067,7 @@ def tool(
max_invocations: int | None = None,
max_invocation_exceptions: int | None = None,
additional_properties: dict[str, Any] | None = None,
result_parser: Callable[[Any], str | list[Content]] | _SkipParsingSentinel | None = None,
result_parser: Callable[[Any], str | list[Content]] | None = None,
) -> FunctionTool: ...
@@ -1159,7 +1083,7 @@ def tool(
max_invocations: int | None = None,
max_invocation_exceptions: int | None = None,
additional_properties: dict[str, Any] | None = None,
result_parser: Callable[[Any], str | list[Content]] | _SkipParsingSentinel | None = None,
result_parser: Callable[[Any], str | list[Content]] | None = None,
) -> Callable[[Callable[..., Any]], FunctionTool]: ...
@@ -1174,7 +1098,7 @@ def tool(
max_invocations: int | None = None,
max_invocation_exceptions: int | None = None,
additional_properties: dict[str, Any] | None = None,
result_parser: Callable[[Any], str | list[Content]] | _SkipParsingSentinel | None = None,
result_parser: Callable[[Any], str | list[Content]] | None = None,
) -> FunctionTool | Callable[[Callable[..., Any]], FunctionTool]:
"""Decorate a function to turn it into a FunctionTool that can be passed to models and executed automatically.
+1 -1
View File
@@ -4,7 +4,7 @@ description = "Microsoft Agent Framework for building AI Agents with Python. Thi
authors = [{ name = "Microsoft", email = "af-support@microsoft.com"}]
readme = "README.md"
requires-python = ">=3.10"
version = "1.1.1"
version = "1.1.0"
license-files = ["LICENSE"]
urls.homepage = "https://aka.ms/agent-framework"
urls.source = "https://github.com/microsoft/agent-framework/tree/main/python"
@@ -8,7 +8,6 @@ from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanE
from pydantic import BaseModel
from agent_framework import (
SKIP_PARSING,
Content,
FunctionTool,
tool,
@@ -1301,165 +1300,4 @@ def test_normalize_tools_flattens_mapping_like_toolbox_with_tools_attr() -> None
assert normalized[1] is standalone
# region SKIP_PARSING sentinel & skip_parsing
async def test_invoke_skip_parsing_returns_native_value() -> None:
"""invoke(skip_parsing=True) returns the wrapped function's raw value."""
@tool
def get_weather(city: str) -> dict[str, Any]:
"""Get the weather."""
return {"city": city, "temperature_c": 21.5, "conditions": "partly cloudy"}
raw = await get_weather.invoke(arguments={"city": "Seattle"}, skip_parsing=True)
assert isinstance(raw, dict)
assert raw == {"city": "Seattle", "temperature_c": 21.5, "conditions": "partly cloudy"}
async def test_invoke_skip_parsing_passes_through_custom_objects() -> None:
"""skip_parsing must not call str()/repr() on the result."""
class Custom: # noqa: B903
def __init__(self, value: int) -> None:
self.value = value
@tool
def make() -> Custom:
"""Make a custom object."""
return Custom(42)
raw = await make.invoke(skip_parsing=True)
assert isinstance(raw, Custom)
assert raw.value == 42
async def test_invoke_skip_parsing_awaits_async_functions() -> None:
@tool
async def slow(x: int) -> int:
"""Async tool."""
return x * 2
raw = await slow.invoke(arguments={"x": 21}, skip_parsing=True)
assert raw == 42
async def test_invoke_skip_parsing_bypasses_configured_result_parser() -> None:
"""The tool's own result_parser is bypassed when skip_parsing=True is requested."""
parser_calls: list[Any] = []
def parser(value: Any) -> str:
parser_calls.append(value)
return "PARSED"
@tool(result_parser=parser)
def make_dict() -> dict[str, int]:
"""Returns a dict."""
return {"a": 1}
raw = await make_dict.invoke(skip_parsing=True)
assert raw == {"a": 1}
assert parser_calls == []
# Sanity: omitting skip_parsing still applies the configured parser.
parsed = await make_dict.invoke()
assert parsed[0].type == "text"
assert parsed[0].text == "PARSED"
async def test_constructor_skip_parsing_sentinel_returns_raw_by_default() -> None:
"""Constructing a tool with result_parser=SKIP_PARSING makes invoke return the raw value."""
@tool(result_parser=SKIP_PARSING)
def make_dict() -> dict[str, int]:
"""Returns a dict."""
return {"a": 1}
raw = await make_dict.invoke()
assert raw == {"a": 1}
async def test_invoke_skip_parsing_validates_arguments() -> None:
"""Argument validation is shared with the default path."""
@tool
def adder(x: int, y: int) -> int:
"""Add."""
return x + y
with pytest.raises(TypeError):
await adder.invoke(arguments={"x": "not-an-int", "y": 1}, skip_parsing=True)
async def test_invoke_skip_parsing_rejects_unexpected_runtime_kwargs() -> None:
@tool
async def echo(message: str) -> str:
"""Echo."""
return message
with pytest.raises(TypeError, match="Unexpected keyword argument"):
await echo.invoke(arguments={"message": "hi"}, skip_parsing=True, api_token="secret")
async def test_invoke_skip_parsing_raises_for_declaration_only_tool() -> None:
declared = FunctionTool(name="dummy", description="declaration only")
from agent_framework.exceptions import ToolException
with pytest.raises(ToolException):
await declared.invoke(arguments={}, skip_parsing=True)
async def test_invoke_skip_parsing_records_telemetry(span_exporter: InMemorySpanExporter) -> None:
"""skip_parsing participates in OTEL spans and records str(raw) as TOOL_RESULT."""
@tool(name="raw_tool", description="raw tool")
def returns_dict(x: int) -> dict[str, int]:
"""Returns a dict."""
return {"value": x}
span_exporter.clear()
raw = await returns_dict.invoke(arguments={"x": 5}, tool_call_id="raw_call", skip_parsing=True)
assert raw == {"value": 5}
spans = span_exporter.get_finished_spans()
assert len(spans) == 1
span = spans[0]
assert span.attributes[OtelAttr.TOOL_NAME] == "raw_tool"
assert span.attributes[OtelAttr.TOOL_CALL_ID] == "raw_call"
assert span.attributes[OtelAttr.TOOL_RESULT] == "{'value': 5}"
async def test_invoke_default_path_records_parsed_telemetry(
span_exporter: InMemorySpanExporter,
) -> None:
"""Regression: omitting skip_parsing still records the parsed result in telemetry."""
def parser(value: Any) -> str:
return f"parsed:{value}"
@tool(name="parsed_tool", description="parsed", result_parser=parser)
def returns_int() -> int:
"""Returns an int."""
return 7
span_exporter.clear()
parsed = await returns_int.invoke(tool_call_id="parsed_call")
assert parsed[0].text == "parsed:7"
spans = span_exporter.get_finished_spans()
assert len(spans) == 1
assert spans[0].attributes[OtelAttr.TOOL_RESULT] == "parsed:7"
def test_skip_parsing_is_singleton() -> None:
"""SKIP_PARSING is a singleton; instantiation returns the same object."""
from agent_framework._tools import _SkipParsingSentinel
assert _SkipParsingSentinel() is SKIP_PARSING
assert repr(SKIP_PARSING) == "SKIP_PARSING"
# endregion
+2 -2
View File
@@ -4,7 +4,7 @@ description = "Declarative specification support for Microsoft Agent Framework."
authors = [{ name = "Microsoft", email = "af-support@microsoft.com"}]
readme = "README.md"
requires-python = ">=3.10"
version = "1.0.0b260423"
version = "1.0.0b260421"
license-files = ["LICENSE"]
urls.homepage = "https://aka.ms/agent-framework"
urls.source = "https://github.com/microsoft/agent-framework/tree/main/python"
@@ -22,7 +22,7 @@ classifiers = [
"Typing :: Typed",
]
dependencies = [
"agent-framework-core>=1.1.1,<2",
"agent-framework-core>=1.1.0,<2",
"powerfx>=0.0.32,<0.0.35; python_version < '3.14'",
"pyyaml>=6.0,<7.0",
]
+2 -2
View File
@@ -4,7 +4,7 @@ description = "Debug UI for Microsoft Agent Framework with OpenAI-compatible API
authors = [{ name = "Microsoft", email = "af-support@microsoft.com"}]
readme = "README.md"
requires-python = ">=3.10"
version = "1.0.0b260423"
version = "1.0.0b260421"
license-files = ["LICENSE"]
urls.homepage = "https://github.com/microsoft/agent-framework"
urls.source = "https://github.com/microsoft/agent-framework/tree/main/python"
@@ -23,7 +23,7 @@ classifiers = [
"Typing :: Typed",
]
dependencies = [
"agent-framework-core>=1.1.1,<2",
"agent-framework-core>=1.1.0,<2",
"openai>=1.99.0,<3",
"opentelemetry-sdk>=1.39.0,<2",
"fastapi>=0.115.0,<0.133.1",
+2 -2
View File
@@ -4,7 +4,7 @@ description = "Durable Task integration for Microsoft Agent Framework."
authors = [{ name = "Microsoft", email = "af-support@microsoft.com"}]
readme = "README.md"
requires-python = ">=3.10"
version = "1.0.0b260423"
version = "1.0.0b260421"
license-files = ["LICENSE"]
urls.homepage = "https://aka.ms/agent-framework"
urls.source = "https://github.com/microsoft/agent-framework/tree/main/python"
@@ -22,7 +22,7 @@ classifiers = [
"Typing :: Typed",
]
dependencies = [
"agent-framework-core>=1.1.1,<2",
"agent-framework-core>=1.1.0,<2",
"durabletask>=1.3.0,<2",
"durabletask-azuremanaged>=1.3.0,<2",
"python-dateutil>=2.8.0,<3",
@@ -455,18 +455,8 @@ class RawFoundryChatClient( # type: ignore[misc]
Returns:
An MCPTool configuration ready to pass to an Agent.
Raises:
ValueError: If neither ``url`` nor ``project_connection_id`` is supplied
— one is required by the Foundry Responses API.
"""
if not url and not project_connection_id:
raise ValueError("MCP tool requires either 'url' or 'project_connection_id' to be specified.")
mcp_kwargs: dict[str, Any] = {"server_label": name.replace(" ", "_"), **kwargs}
if url:
mcp_kwargs["server_url"] = url
mcp = FoundryMCPTool(**mcp_kwargs)
mcp = FoundryMCPTool(server_label=name.replace(" ", "_"), server_url=url or "", **kwargs)
if description:
mcp["server_description"] = description
@@ -133,55 +133,26 @@ def select_toolbox_tools(
return selected
def _validate_hosted_tool_payload(sanitized: Mapping[str, Any]) -> None:
"""Fail fast on hosted tool payloads that would always be rejected by the Responses API.
These mismatches are not injectable defaults — the caller must supply the
missing information — so surfacing a clear error here points at the toolbox
definition instead of letting the API return a generic 400.
"""
tool_type = sanitized.get("type")
if tool_type == "file_search" and not sanitized.get("vector_store_ids"):
raise ValueError(
"'file_search' tool is missing required 'vector_store_ids'. "
"If this came from a Foundry toolbox, update the toolbox definition "
"to include at least one vector store ID."
)
if tool_type == "mcp" and not sanitized.get("server_url") and not sanitized.get("project_connection_id"):
raise ValueError(
"'mcp' tool is missing both 'server_url' and 'project_connection_id'. "
"If this came from a Foundry toolbox, update the toolbox definition "
"to include one of these."
)
@experimental(feature_id=ExperimentalFeature.TOOLBOXES)
def sanitize_foundry_response_tool(tool_item: Any) -> Any:
"""Return a Responses-API-safe tool payload for Foundry hosted tools.
Reconciles known mismatches between toolbox reads and the Responses API:
Azure AI Projects toolbox reads can currently return hosted tool objects with
extra read-model decoration fields such as top-level ``name`` and
``description``. Azure AI Foundry rejects at least ``name`` on Responses API
requests with:
1. Toolbox reads can return hosted tool objects decorated with read-model
fields such as top-level ``name`` and ``description``. The Responses API
rejects at least ``name`` with ``Unknown parameter: 'tools[0].name'``.
These fields are stripped from non-function hosted tool payloads.
2. ``code_interpreter`` tools stored in a toolbox without a ``container``
field (the Azure SDK treats it as optional) are rejected by the Responses
API with ``Missing required parameter: 'tools[N].container'``. A default
``{"type": "auto"}`` container is injected when absent.
3. Hosted tools that are structurally incomplete in ways that cannot be
defaulted (``file_search`` without ``vector_store_ids``, ``mcp`` without
either ``server_url`` or ``project_connection_id``) raise ``ValueError``
with a message that points at the toolbox definition.
``Unknown parameter: 'tools[0].name'``.
These are workarounds until the toolbox/Responses proxy normalizes payloads
server-side.
We defensively strip these decoration fields for non-function hosted tools so
the round-trip
``toolbox.tools -> Agent(..., tools=...) -> run()`` works, while the Azure
SDK/service behavior is corrected upstream.
"""
if isinstance(tool_item, FoundryMCPTool):
sanitized: dict[str, Any] = dict(cast("Mapping[str, Any]", tool_item))
sanitized.pop("name", None)
sanitized.pop("description", None)
_validate_hosted_tool_payload(sanitized)
return sanitized
if isinstance(tool_item, Mapping):
@@ -190,9 +161,6 @@ def sanitize_foundry_response_tool(tool_item: Any) -> Any:
sanitized = dict(mapping)
sanitized.pop("name", None)
sanitized.pop("description", None)
if sanitized.get("type") == "code_interpreter" and "container" not in sanitized:
sanitized["container"] = {"type": "auto"}
_validate_hosted_tool_payload(sanitized)
return sanitized
return cast(Any, tool_item)
+2 -2
View File
@@ -4,7 +4,7 @@ description = "Microsoft Foundry integrations for Microsoft Agent Framework."
authors = [{ name = "Microsoft", email = "af-support@microsoft.com"}]
readme = "README.md"
requires-python = ">=3.10"
version = "1.1.1"
version = "1.1.0"
license-files = ["LICENSE"]
urls.homepage = "https://aka.ms/agent-framework"
urls.source = "https://github.com/microsoft/agent-framework/tree/main/python"
@@ -23,7 +23,7 @@ classifiers = [
"Typing :: Typed",
]
dependencies = [
"agent-framework-core>=1.1.1,<2",
"agent-framework-core>=1.1.0,<2",
"agent-framework-openai>=1.1.0,<2",
"azure-ai-inference>=1.0.0b9,<1.0.0b10",
"azure-ai-projects>=2.1.0,<3.0",
@@ -607,14 +607,6 @@ def test_get_mcp_tool_with_project_connection_id() -> None:
assert tool_config["project_connection_id"] == "conn-123"
assert tool_config["allowed_tools"] == ["search_docs"]
assert tool_config["server_label"] == "Docs_MCP"
# ``server_url`` should not be fabricated when only a project connection is supplied.
assert "server_url" not in tool_config
def test_get_mcp_tool_requires_url_or_project_connection_id() -> None:
"""Missing both ``url`` and ``project_connection_id`` is always invalid."""
with pytest.raises(ValueError, match="url.*project_connection_id"):
FoundryChatClient.get_mcp_tool(name="x")
def test_prepare_tools_for_openai_strips_extraneous_name_from_foundry_mcp_tool() -> None:
@@ -663,103 +655,6 @@ def test_prepare_tools_for_openai_strips_read_model_fields_from_toolbox_code_int
assert "description" not in prepared
def test_prepare_tools_for_openai_injects_default_container_for_code_interpreter_dict() -> None:
"""Toolbox-returned code_interpreter without a container must get a default injected.
The Azure SDK treats ``container`` as optional, but the Responses API rejects
``code_interpreter`` entries without one. The sanitizer backfills ``{"type": "auto"}``.
"""
project_client = MagicMock()
project_client.get_openai_client.return_value = _make_mock_openai_client()
client = FoundryChatClient(project_client=project_client, model="test-model")
tool = {
"type": "code_interpreter",
"name": "code_interpreter_t6bbtm",
}
response_tools = client._prepare_tools_for_openai([tool])
assert len(response_tools) == 1
prepared = response_tools[0]
assert prepared["type"] == "code_interpreter"
assert prepared["container"] == {"type": "auto"}
assert "name" not in prepared
def test_prepare_tools_for_openai_injects_default_container_for_code_interpreter_sdk_instance() -> None:
"""SDK ``CodeInterpreterTool`` instances without a container must also be backfilled.
Reproduces the toolbox creation path that calls
``CodeInterpreterTool(name="code_interpreter")`` without a container.
"""
from azure.ai.projects.models import CodeInterpreterTool
project_client = MagicMock()
project_client.get_openai_client.return_value = _make_mock_openai_client()
client = FoundryChatClient(project_client=project_client, model="test-model")
response_tools = client._prepare_tools_for_openai([CodeInterpreterTool(name="code_interpreter")])
assert len(response_tools) == 1
prepared = response_tools[0]
assert prepared["type"] == "code_interpreter"
assert prepared["container"] == {"type": "auto"}
assert "name" not in prepared
def test_prepare_tools_for_openai_preserves_existing_code_interpreter_container() -> None:
"""An already-populated container must not be overwritten by the sanitizer."""
project_client = MagicMock()
project_client.get_openai_client.return_value = _make_mock_openai_client()
client = FoundryChatClient(project_client=project_client, model="test-model")
explicit_container = {"file_ids": ["file_123"], "type": "auto"}
tool = {"type": "code_interpreter", "container": explicit_container}
response_tools = client._prepare_tools_for_openai([tool])
assert response_tools[0]["container"] == explicit_container
def test_prepare_tools_for_openai_rejects_file_search_without_vector_store_ids() -> None:
"""``file_search`` without ``vector_store_ids`` is always invalid — surface a clear error."""
project_client = MagicMock()
project_client.get_openai_client.return_value = _make_mock_openai_client()
client = FoundryChatClient(project_client=project_client, model="test-model")
with pytest.raises(ValueError, match="vector_store_ids"):
client._prepare_tools_for_openai([{"type": "file_search", "name": "fs"}])
def test_prepare_tools_for_openai_rejects_mcp_without_server_destination() -> None:
"""``mcp`` with neither ``server_url`` nor ``project_connection_id`` is always invalid."""
project_client = MagicMock()
project_client.get_openai_client.return_value = _make_mock_openai_client()
client = FoundryChatClient(project_client=project_client, model="test-model")
tool = FoundryMCPTool(server_label="orphan")
with pytest.raises(ValueError, match="server_url.*project_connection_id"):
client._prepare_tools_for_openai([tool])
def test_prepare_tools_for_openai_accepts_mcp_with_only_project_connection_id() -> None:
"""MCP tools backed by a Foundry connection (no ``server_url``) must still pass validation."""
project_client = MagicMock()
project_client.get_openai_client.return_value = _make_mock_openai_client()
client = FoundryChatClient(project_client=project_client, model="test-model")
tool = FoundryMCPTool(server_label="githubmcp")
tool["project_connection_id"] = "githubmcp"
response_tools = client._prepare_tools_for_openai([tool])
assert len(response_tools) == 1
assert response_tools[0]["project_connection_id"] == "githubmcp"
assert "server_url" not in response_tools[0]
def test_prepare_tools_for_openai_strips_name_from_non_function_hosted_tool_dicts() -> None:
"""All non-function hosted tool payloads should drop top-level read-model names."""
project_client = MagicMock()
@@ -4,7 +4,7 @@ description = "Foundry Hosting integration for Microsoft Agent Framework."
authors = [{ name = "Microsoft", email = "af-support@microsoft.com"}]
readme = "README.md"
requires-python = ">=3.10"
version = "1.0.0a260423"
version = "1.0.0a260421"
license-files = ["LICENSE"]
urls.homepage = "https://aka.ms/agent-framework"
urls.source = "https://github.com/microsoft/agent-framework/tree/main/python"
@@ -12,7 +12,7 @@ urls.release_notes = "https://github.com/microsoft/agent-framework/releases?q=ta
urls.issues = "https://github.com/microsoft/agent-framework/issues"
classifiers = [
"License :: OSI Approved :: MIT License",
"Development Status :: 3 - Alpha",
"Development Status :: 4 - Alpha",
"Intended Audience :: Developers",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.10",
@@ -23,7 +23,7 @@ classifiers = [
"Typing :: Typed",
]
dependencies = [
"agent-framework-core>=1.1.1,<2",
"agent-framework-core>=1.1.0,<2",
"azure-ai-agentserver-core==2.0.0b2",
"azure-ai-agentserver-responses==1.0.0b4",
"azure-ai-agentserver-invocations==1.0.0b2",
+2 -2
View File
@@ -4,7 +4,7 @@ description = "Foundry Local integration for Microsoft Agent Framework."
authors = [{ name = "Microsoft", email = "af-support@microsoft.com"}]
readme = "README.md"
requires-python = ">=3.10"
version = "1.0.0b260423"
version = "1.0.0b260421"
license-files = ["LICENSE"]
urls.homepage = "https://aka.ms/agent-framework"
urls.source = "https://github.com/microsoft/agent-framework/tree/main/python"
@@ -23,7 +23,7 @@ classifiers = [
"Typing :: Typed",
]
dependencies = [
"agent-framework-core>=1.1.1,<2",
"agent-framework-core>=1.1.0,<2",
"agent-framework-openai>=1.1.0,<2",
"foundry-local-sdk>=0.5.1,<0.5.2",
]
+2 -2
View File
@@ -4,7 +4,7 @@ description = "Google Gemini integration for Microsoft Agent Framework."
authors = [{ name = "Microsoft", email = "af-support@microsoft.com"}]
readme = "README.md"
requires-python = ">=3.10"
version = "1.0.0a260423"
version = "1.0.0a260421"
license-files = ["LICENSE"]
urls.homepage = "https://aka.ms/agent-framework"
urls.source = "https://github.com/microsoft/agent-framework/tree/main/python"
@@ -24,7 +24,7 @@ classifiers = [
"Typing :: Typed",
]
dependencies = [
"agent-framework-core>=1.1.1,<2.0",
"agent-framework-core>=1.1.0,<2.0",
"google-genai>=1.0.0,<2.0.0",
]
@@ -4,7 +4,7 @@ description = "GitHub Copilot integration for Microsoft Agent Framework."
authors = [{ name = "Microsoft", email = "af-support@microsoft.com"}]
readme = "README.md"
requires-python = ">=3.10"
version = "1.0.0b260423"
version = "1.0.0b260421"
license-files = ["LICENSE"]
urls.homepage = "https://aka.ms/agent-framework"
urls.source = "https://github.com/microsoft/agent-framework/tree/main/python"
@@ -23,7 +23,7 @@ classifiers = [
"Typing :: Typed",
]
dependencies = [
"agent-framework-core>=1.1.1,<2",
"agent-framework-core>=1.1.0,<2",
"github-copilot-sdk>=0.2.1,<=0.2.1; python_version >= '3.11'",
]
-6
View File
@@ -130,9 +130,3 @@ codeact = HyperlightCodeActProvider(
- `allowed_domains` accepts a single string target such as `"github.com"` to
allow all backend-supported methods, an explicit `(target, method_or_methods)`
tuple such as `("github.com", "GET")`, or an `AllowedDomain` named tuple.
- Tools registered with the sandbox return their native Python value
(`dict`, `list`, primitives, or custom objects) directly to the guest via the
Hyperlight FFI. Any `result_parser` configured on a `FunctionTool` is
intended for LLM-facing consumers and does not run on the sandbox path —
apply formatting inside the tool function itself if you need it for
in-sandbox consumers.
@@ -2,45 +2,42 @@
from __future__ import annotations
import ast
import asyncio
import copy
import mimetypes
import shutil
import threading
import time
from collections.abc import Callable, Sequence
from concurrent.futures import Future, ThreadPoolExecutor
from contextlib import suppress
from copy import copy
from dataclasses import dataclass, field
from dataclasses import dataclass
from pathlib import Path, PurePosixPath
from tempfile import TemporaryDirectory
from typing import Any, Protocol, TypeGuard, TypeVar, cast
from typing import Annotated, Any, Protocol, TypeGuard, cast
from urllib.parse import urlparse
from agent_framework import Content, FunctionTool
from agent_framework._tools import ApprovalMode, normalize_tools
from pydantic import BaseModel, Field
from ._instructions import build_codeact_instructions, build_execute_code_description
from ._types import AllowedDomain, AllowedDomainInput, FileMount, FileMountHostPath, FileMountInput
DEFAULT_HYPERLIGHT_BACKEND = "wasm"
DEFAULT_HYPERLIGHT_MODULE = "python_guest.path"
EXECUTE_CODE_TOOL_DESCRIPTION = "Execute Python in an isolated Hyperlight sandbox."
EXECUTE_CODE_INPUT_DESCRIPTION = "Python code to execute in an isolated Hyperlight sandbox."
OUTPUT_FILE_RETRY_ATTEMPTS = 10
OUTPUT_FILE_RETRY_DELAY_SECONDS = 0.1
EXECUTE_CODE_INPUT_SCHEMA: dict[str, Any] = {
"type": "object",
"title": "_ExecuteCodeInput",
"properties": {
"code": {
"type": "string",
"title": "Code",
"description": "Python code to execute in an isolated Hyperlight sandbox.",
},
},
"required": ["code"],
}
class _ExecuteCodeInput(BaseModel):
code: Annotated[str, Field(description=EXECUTE_CODE_INPUT_DESCRIPTION)]
@dataclass(frozen=True, slots=True)
class _StoredFileMount:
host_path: Path
mount_path: str
@dataclass(frozen=True, slots=True)
@@ -88,43 +85,13 @@ class SandboxRuntime(Protocol):
def execute(self, *, config: _RunConfig, code: str) -> list[Content]: ...
_T = TypeVar("_T")
class _SandboxWorker:
"""Single-threaded executor that confines all sandbox operations to one OS thread.
The Hyperlight ``WasmSandbox`` is declared ``unsendable`` in PyO3, meaning it can only be
accessed from the OS thread that created it; touching it from any other thread triggers a
Rust panic that cannot be caught from Python. Every cached :class:`_SandboxEntry` therefore
owns its own ``_SandboxWorker``, and *all* lifecycle and execution calls against the
underlying sandbox object must be routed through :meth:`submit`/:meth:`run`.
"""
__slots__ = ("_executor",)
def __init__(self, *, name: str = "hl-sandbox") -> None:
self._executor = ThreadPoolExecutor(max_workers=1, thread_name_prefix=name)
def submit(self, fn: Callable[..., _T], /, *args: Any, **kwargs: Any) -> Future[_T]:
return self._executor.submit(fn, *args, **kwargs)
def run(self, fn: Callable[..., _T], /, *args: Any, **kwargs: Any) -> _T:
return self._executor.submit(fn, *args, **kwargs).result()
def shutdown(self) -> None:
# Do not block on shutdown; stop accepting new tasks, but allow the currently running
# task and any already-queued tasks to finish before the worker thread exits.
self._executor.shutdown(wait=False, cancel_futures=False)
@dataclass
class _SandboxEntry:
sandbox: Any
snapshot: Any
input_dir: TemporaryDirectory[str] | None
output_dir: TemporaryDirectory[str] | None
worker: _SandboxWorker = field(default_factory=_SandboxWorker)
lock: threading.RLock
def _load_sandbox_class() -> type[Any]:
@@ -139,6 +106,10 @@ def _load_sandbox_class() -> type[Any]:
return Sandbox
def _passthrough_result_parser(result: Any) -> str:
return repr(result)
def _collect_tools(*tool_groups: Any) -> list[FunctionTool]:
tools_by_name: dict[str, FunctionTool] = {}
@@ -195,7 +166,7 @@ def _is_file_mount_pair(value: Any) -> TypeGuard[FileMount | tuple[FileMountHost
return isinstance(host_path, (str, Path)) and isinstance(mount_path, str)
def _normalize_file_mount_input(file_mount: FileMountInput) -> FileMount:
def _normalize_file_mount_input(file_mount: FileMountInput) -> _StoredFileMount:
host_path: FileMountHostPath
mount_path: str
if isinstance(file_mount, str):
@@ -205,7 +176,7 @@ def _normalize_file_mount_input(file_mount: FileMountInput) -> FileMount:
host_path = file_mount[0]
mount_path = file_mount[1]
return FileMount(
return _StoredFileMount(
host_path=_resolve_existing_path(host_path),
mount_path=_normalize_mount_path(mount_path),
)
@@ -474,13 +445,18 @@ def _build_execution_contents(
def _make_sandbox_callback(tool_obj: FunctionTool) -> Callable[..., Any]:
sandbox_tool = copy(tool_obj)
sandbox_tool = copy.copy(tool_obj)
# Auto-assign a passthrough parser so the raw return value round-trips through
# `ast.literal_eval` in the sandbox callback below. User-supplied parsers are
# left in place so callers can customize how results are exposed to the guest.
if sandbox_tool.result_parser is None:
sandbox_tool.result_parser = _passthrough_result_parser
def _callback(**kwargs: Any) -> Any:
async def _invoke() -> Any:
return await sandbox_tool.invoke(arguments=kwargs, skip_parsing=True)
async def _invoke() -> list[Content]:
return await sandbox_tool.invoke(arguments=kwargs)
# FunctionTool.invoke() is async. The real Hyperlight backend invokes
# FunctionTool.invoke() is always async. The real Hyperlight backend invokes
# registered callbacks synchronously via FFI, so this must be a sync function.
# We run the async call on a dedicated thread to avoid conflicts with any
# event loop that may be running on the current thread.
@@ -498,11 +474,22 @@ def _make_sandbox_callback(tool_obj: FunctionTool) -> Callable[..., Any]:
worker.join()
if error_box:
raise error_box[0]
# Return the raw value. The Hyperlight FFI marshals primitives (dict, list,
# str, int, float, bool, None) natively into the guest, and falls back to
# repr()/str() for unsupported types — so the guest receives real Python
# objects without a lossy host-side serialization round-trip.
return result_box[0]
contents: list[Content] = result_box[0]
values: list[Any] = []
for content in contents:
if content.type == "text" and content.text is not None:
try:
values.append(ast.literal_eval(content.text))
except (SyntaxError, ValueError):
values.append(content.text)
continue
values.append(content.to_dict())
if len(values) == 1:
return values[0]
return values
return _callback
@@ -522,7 +509,7 @@ def _clear_directory(output_dir: TemporaryDirectory[str] | None) -> None:
pass
class _SandboxRegistry(SandboxRuntime):
class _SandboxRegistry:
def __init__(self) -> None:
self._entries: dict[tuple[Any, ...], _SandboxEntry] = {}
self._entries_lock = threading.RLock()
@@ -530,54 +517,28 @@ class _SandboxRegistry(SandboxRuntime):
def execute(self, *, config: _RunConfig, code: str) -> list[Content]:
"""Execute code in a cached sandbox matching the given config.
Entries are keyed by ``config.cache_key()``. All operations against the underlying
sandbox object are routed through the entry's dedicated single-threaded worker, which
both serializes concurrent callers and satisfies the PyO3 ``unsendable`` invariant
that the sandbox can only be touched from the thread that created it.
Entries are keyed by ``config.cache_key()``. Concurrent calls with the same
key are serialized by the entry lock so they never race, but they share the
same sandbox instance. For true parallel execution, use distinct provider
instances or configs that produce different cache keys.
"""
entry = self._get_or_create_entry(config)
return entry.worker.run(self._run_on_worker, entry, code)
@staticmethod
def _run_on_worker(entry: _SandboxEntry, code: str) -> list[Content]:
entry.sandbox.restore(entry.snapshot)
_clear_directory(entry.output_dir)
result = entry.sandbox.run(code=code)
return _build_execution_contents(
result=result,
sandbox=entry.sandbox,
output_dir=entry.output_dir,
code=code,
)
def _get_or_create_entry(self, config: _RunConfig) -> _SandboxEntry:
cache_key = config.cache_key()
with self._entries_lock:
entry = self._entries.get(cache_key)
if entry is None:
entry = self._create_entry(config)
self._entries[cache_key] = entry
return entry
def close(self) -> None:
"""Shut down all per-entry worker threads and release per-entry resources.
Safe to call multiple times. Runs any sandbox close hook on the entry's
own worker thread to honor the PyO3 ``unsendable`` invariant.
"""
with self._entries_lock:
entries = list(self._entries.values())
self._entries.clear()
for entry in entries:
close_hook = getattr(entry.sandbox, "close", None) or getattr(entry.sandbox, "shutdown", None)
if callable(close_hook):
with suppress(Exception):
entry.worker.run(close_hook)
entry.worker.shutdown()
for tmp_dir in (entry.input_dir, entry.output_dir):
if tmp_dir is not None:
with suppress(Exception):
tmp_dir.cleanup()
with entry.lock:
entry.sandbox.restore(entry.snapshot)
_clear_directory(entry.output_dir)
result = entry.sandbox.run(code=code)
return _build_execution_contents(
result=result,
sandbox=entry.sandbox,
output_dir=entry.output_dir,
code=code,
)
def _create_entry(self, config: _RunConfig) -> _SandboxEntry:
input_dir_handle = TemporaryDirectory() if config.filesystem_enabled else None
@@ -617,37 +578,26 @@ class _SandboxRegistry(SandboxRuntime):
methods=list(allowed_domain.methods) if allowed_domain.methods is not None else None,
)
worker = _SandboxWorker()
def _build_sandbox() -> tuple[Any, Any]:
sandbox = _create_sandbox()
_configure_sandbox(sandbox=sandbox, expand_missing_scheme=False)
try:
sandbox.run("None")
except RuntimeError as exc:
if not _should_retry_allowed_domain_registration(error=exc, allowed_domains=config.allowed_domains):
raise
sandbox = _create_sandbox()
_configure_sandbox(sandbox=sandbox, expand_missing_scheme=True)
sandbox.run("None")
snapshot = sandbox.snapshot()
return sandbox, snapshot
sandbox = _create_sandbox()
_configure_sandbox(sandbox=sandbox, expand_missing_scheme=False)
try:
sandbox, snapshot = worker.run(_build_sandbox)
except BaseException:
worker.shutdown()
raise
sandbox.run("None")
except RuntimeError as exc:
if not _should_retry_allowed_domain_registration(error=exc, allowed_domains=config.allowed_domains):
raise
sandbox = _create_sandbox()
_configure_sandbox(sandbox=sandbox, expand_missing_scheme=True)
sandbox.run("None")
snapshot = sandbox.snapshot()
return _SandboxEntry(
sandbox=sandbox,
snapshot=snapshot,
input_dir=input_dir_handle,
output_dir=output_dir_handle,
worker=worker,
lock=threading.RLock(),
)
@@ -669,10 +619,10 @@ class HyperlightExecuteCodeTool(FunctionTool):
) -> None:
super().__init__(
name="execute_code",
description=EXECUTE_CODE_TOOL_DESCRIPTION,
description=EXECUTE_CODE_INPUT_DESCRIPTION,
approval_mode="never_require",
func=self._run_code,
input_model=EXECUTE_CODE_INPUT_SCHEMA,
input_model=_ExecuteCodeInput,
)
self._state_lock = threading.RLock()
self._registry = _registry or _SandboxRegistry()
@@ -682,7 +632,7 @@ class HyperlightExecuteCodeTool(FunctionTool):
self._module: str | None = module
self._module_path: str | None = module_path
self._managed_tools: list[FunctionTool] = []
self._file_mounts: dict[str, FileMount] = {}
self._file_mounts: dict[str, _StoredFileMount] = {}
self._allowed_domains: dict[str, AllowedDomain] = {}
if tools is not None:
@@ -698,7 +648,7 @@ class HyperlightExecuteCodeTool(FunctionTool):
def description(self) -> str:
state_lock = getattr(self, "_state_lock", None)
if state_lock is None:
return str(self.__dict__.get("description", EXECUTE_CODE_TOOL_DESCRIPTION))
return str(self.__dict__.get("description", EXECUTE_CODE_INPUT_DESCRIPTION))
with state_lock:
allowed_domains = sorted(self._allowed_domains.values(), key=lambda value: value.target)
@@ -891,9 +841,9 @@ class HyperlightExecuteCodeTool(FunctionTool):
workspace_signature = _path_tree_signature(workspace_root) if workspace_root is not None else ()
normalized_mounts = tuple(
_NormalizedFileMount(
host_path=Path(mount.host_path),
host_path=mount.host_path,
mount_path=mount.mount_path,
path_signature=_path_tree_signature(Path(mount.host_path)),
path_signature=_path_tree_signature(mount.host_path),
)
for mount in stored_mounts
)
+2 -2
View File
@@ -4,7 +4,7 @@ description = "Hyperlight CodeAct integrations for Microsoft Agent Framework."
authors = [{ name = "Microsoft", email = "af-support@microsoft.com"}]
readme = "README.md"
requires-python = ">=3.10"
version = "1.0.0a260423"
version = "1.0.0a260421"
license-files = ["LICENSE"]
urls.homepage = "https://aka.ms/agent-framework"
urls.source = "https://github.com/microsoft/agent-framework/tree/main/python"
@@ -22,7 +22,7 @@ classifiers = [
"Typing :: Typed",
]
dependencies = [
"agent-framework-core>=1.1.1,<2",
"agent-framework-core>=1.1.0,<2",
"hyperlight-sandbox>=0.3.0,<0.4",
"hyperlight-sandbox-backend-wasm>=0.3.0,<0.4 ; ((sys_platform == 'linux' and platform_machine == 'x86_64') or (sys_platform == 'win32' and platform_machine == 'AMD64')) and python_version < '3.14'",
"hyperlight-sandbox-python-guest>=0.3.0,<0.4",
@@ -937,191 +937,3 @@ async def test_run_code_does_not_block_event_loop() -> None:
assert concurrent_ran, "Event loop was blocked during sandbox execution"
assert result[0].type == "text"
class _ThreadAffinityFakeSandbox(_FakeSandbox):
"""Fake sandbox that records the OS thread of every method invocation.
Mirrors the PyO3 ``unsendable`` invariant of ``hyperlight_sandbox.WasmSandbox``:
if ``__init__``, ``register_tool``, ``allow_domain``, ``run``, ``snapshot`` or ``restore``
are ever called from more than one thread for a given instance, the test fails.
"""
affinity_failures: list[str] = []
def __init__(self, **kwargs: Any) -> None:
super().__init__(**kwargs)
self._owner_thread = threading.get_ident()
self.thread_ids: set[int] = {self._owner_thread}
def _record(self, method: str) -> None:
ident = threading.get_ident()
self.thread_ids.add(ident)
if ident != self._owner_thread:
_ThreadAffinityFakeSandbox.affinity_failures.append(
f"{method} called from thread {ident}, expected {self._owner_thread}"
)
def register_tool(self, name_or_tool: Any, callback: Any | None = None) -> None:
self._record("register_tool")
super().register_tool(name_or_tool, callback)
def allow_domain(self, target: str, methods: list[str] | None = None) -> None:
self._record("allow_domain")
super().allow_domain(target, methods)
def run(self, code: str) -> _FakeResult:
self._record("run")
return super().run(code)
def snapshot(self) -> str:
self._record("snapshot")
return super().snapshot()
def restore(self, snapshot: Any) -> None:
self._record("restore")
super().restore(snapshot)
async def test_sandbox_calls_are_pinned_to_owning_worker_thread(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Regression: WasmSandbox is unsendable; every sandbox call must run on its owner thread."""
_ThreadAffinityFakeSandbox.instances.clear()
_ThreadAffinityFakeSandbox.affinity_failures.clear()
monkeypatch.setattr(execute_code_module, "_load_sandbox_class", lambda: _ThreadAffinityFakeSandbox)
execute_code = HyperlightExecuteCodeTool()
# Invoke many times concurrently; asyncio.to_thread will spread these across the default
# executor's worker threads, which previously caused PyO3 to panic when a different thread
# touched the cached sandbox.
results = await asyncio.gather(*[execute_code.invoke(arguments={"code": "None"}) for _ in range(8)])
for result in results:
assert result[0].type == "text"
assert _ThreadAffinityFakeSandbox.affinity_failures == []
assert len(_ThreadAffinityFakeSandbox.instances) == 1
sandbox = _ThreadAffinityFakeSandbox.instances[0]
# All sandbox-touching calls must have stayed on a single owning thread, distinct from the
# caller thread that asyncio.to_thread used for dispatch.
assert sandbox.thread_ids == {sandbox._owner_thread}
assert sandbox._owner_thread != threading.get_ident()
async def test_sandbox_owner_thread_persists_across_dispatch_threads(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Sequential calls landing on different dispatch threads still share one sandbox thread."""
_ThreadAffinityFakeSandbox.instances.clear()
_ThreadAffinityFakeSandbox.affinity_failures.clear()
monkeypatch.setattr(execute_code_module, "_load_sandbox_class", lambda: _ThreadAffinityFakeSandbox)
execute_code = HyperlightExecuteCodeTool()
for _ in range(5):
result = await execute_code.invoke(arguments={"code": "None"})
assert result[0].type == "text"
assert _ThreadAffinityFakeSandbox.affinity_failures == []
assert len(_ThreadAffinityFakeSandbox.instances) == 1
def test_sandbox_registry_close_shuts_down_workers(monkeypatch: pytest.MonkeyPatch) -> None:
_FakeSandbox.instances.clear()
monkeypatch.setattr(execute_code_module, "_load_sandbox_class", lambda: _FakeSandbox)
registry = execute_code_module._SandboxRegistry()
execute_code = HyperlightExecuteCodeTool(_registry=registry)
asyncio.run(execute_code.invoke(arguments={"code": "None"}))
entries = list(registry._entries.values())
assert len(entries) == 1
worker = entries[0].worker
registry.close()
assert registry._entries == {}
# Submitting after shutdown must fail; this proves the executor was actually torn down.
with pytest.raises(RuntimeError):
worker.submit(lambda: None)
def test_sandbox_registry_close_releases_per_entry_resources(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> None:
"""close() must invoke any sandbox close hook and release temp directories."""
close_calls: list[int] = []
class _ClosableFakeSandbox(_FakeSandbox):
def close(self) -> None:
close_calls.append(1)
_FakeSandbox.instances.clear()
monkeypatch.setattr(execute_code_module, "_load_sandbox_class", lambda: _ClosableFakeSandbox)
workspace = tmp_path / "workspace"
workspace.mkdir()
registry = execute_code_module._SandboxRegistry()
execute_code = HyperlightExecuteCodeTool(workspace_root=workspace, _registry=registry)
asyncio.run(execute_code.invoke(arguments={"code": "None"}))
entries = list(registry._entries.values())
assert len(entries) == 1
entry = entries[0]
assert entry.input_dir is not None and entry.output_dir is not None
input_path = Path(entry.input_dir.name)
output_path = Path(entry.output_dir.name)
assert input_path.exists() and output_path.exists()
registry.close()
assert close_calls == [1]
assert not input_path.exists()
assert not output_path.exists()
async def test_make_sandbox_callback_returns_native_dict() -> None:
"""Host tool returning a dict must be forwarded as a native dict (no repr round-trip)."""
@tool
def get_weather(city: str) -> dict[str, Any]:
"""Get weather."""
return {"city": city, "temp_c": 21.5}
callback = execute_code_module._make_sandbox_callback(get_weather)
result = callback(city="Seattle")
assert isinstance(result, dict)
assert result == {"city": "Seattle", "temp_c": 21.5}
async def test_make_sandbox_callback_bypasses_user_result_parser() -> None:
"""Documented behavior change: result_parser is bypassed in the sandbox path."""
parser_calls: list[Any] = []
def parser(value: Any) -> str:
parser_calls.append(value)
return "PARSED"
@tool(result_parser=parser)
def make_payload() -> dict[str, int]:
"""Returns a dict."""
return {"a": 1, "b": 2}
callback = execute_code_module._make_sandbox_callback(make_payload)
result = callback()
assert result == {"a": 1, "b": 2}
assert parser_calls == [], "result_parser must not run on the sandbox path"
async def test_make_sandbox_callback_propagates_exceptions() -> None:
@tool
def boom(x: int) -> int:
"""Always fails."""
raise RuntimeError("nope")
callback = execute_code_module._make_sandbox_callback(boom)
with pytest.raises(RuntimeError, match="nope"):
callback(x=1)
+2 -2
View File
@@ -4,7 +4,7 @@ description = "Experimental modules for Microsoft Agent Framework"
authors = [{ name = "Microsoft", email = "af-support@microsoft.com"}]
readme = "README.md"
requires-python = ">=3.10"
version = "1.0.0b260423"
version = "1.0.0b260421"
license-files = ["LICENSE"]
urls.homepage = "https://aka.ms/agent-framework"
urls.source = "https://github.com/microsoft/agent-framework/tree/main/python"
@@ -22,7 +22,7 @@ classifiers = [
"Programming Language :: Python :: 3.14",
]
dependencies = [
"agent-framework-core>=1.1.1,<2",
"agent-framework-core>=1.1.0,<2",
]
[project.optional-dependencies]
+2 -2
View File
@@ -4,7 +4,7 @@ description = "Mem0 integration for Microsoft Agent Framework."
authors = [{ name = "Microsoft", email = "af-support@microsoft.com"}]
readme = "README.md"
requires-python = ">=3.10"
version = "1.0.0b260423"
version = "1.0.0b260421"
license-files = ["LICENSE"]
urls.homepage = "https://aka.ms/agent-framework"
urls.source = "https://github.com/microsoft/agent-framework/tree/main/python"
@@ -23,7 +23,7 @@ classifiers = [
"Typing :: Typed",
]
dependencies = [
"agent-framework-core>=1.1.1,<2",
"agent-framework-core>=1.1.0,<2",
"mem0ai>=1.0.0,<2",
]
+2 -2
View File
@@ -4,7 +4,7 @@ description = "Ollama integration for Microsoft Agent Framework."
authors = [{ name = "Microsoft", email = "af-support@microsoft.com"}]
readme = "README.md"
requires-python = ">=3.10"
version = "1.0.0b260423"
version = "1.0.0b260421"
license-files = ["LICENSE"]
urls.homepage = "https://learn.microsoft.com/en-us/agent-framework/"
urls.source = "https://github.com/microsoft/agent-framework/tree/main/python"
@@ -23,7 +23,7 @@ classifiers = [
"Typing :: Typed",
]
dependencies = [
"agent-framework-core>=1.1.1,<2",
"agent-framework-core>=1.1.0,<2",
"ollama>=0.5.3,<0.5.4",
]
+2 -2
View File
@@ -4,7 +4,7 @@ description = "OpenAI integrations for Microsoft Agent Framework."
authors = [{ name = "Microsoft", email = "af-support@microsoft.com"}]
readme = "README.md"
requires-python = ">=3.10"
version = "1.1.1"
version = "1.1.0"
license-files = ["LICENSE"]
urls.homepage = "https://aka.ms/agent-framework"
urls.source = "https://github.com/microsoft/agent-framework/tree/main/python"
@@ -23,7 +23,7 @@ classifiers = [
"Typing :: Typed",
]
dependencies = [
"agent-framework-core>=1.1.1,<2",
"agent-framework-core>=1.1.0,<2",
"openai>=1.99.0,<3",
]
@@ -4,7 +4,7 @@ description = "Orchestration patterns for Microsoft Agent Framework. Includes Se
authors = [{ name = "Microsoft", email = "af-support@microsoft.com"}]
readme = "README.md"
requires-python = ">=3.10"
version = "1.0.0b260423"
version = "1.0.0b260421"
license-files = ["LICENSE"]
urls.homepage = "https://aka.ms/agent-framework"
urls.source = "https://github.com/microsoft/agent-framework/tree/main/python"
@@ -23,7 +23,7 @@ classifiers = [
"Typing :: Typed",
]
dependencies = [
"agent-framework-core>=1.1.1,<2",
"agent-framework-core>=1.1.0,<2",
]
[tool.uv]
+2 -2
View File
@@ -4,7 +4,7 @@ description = "Microsoft Purview (Graph dataSecurityAndGovernance) integration f
authors = [{ name = "Microsoft", email = "af-support@microsoft.com"}]
readme = "README.md"
requires-python = ">=3.10"
version = "1.0.0b260423"
version = "1.0.0b260421"
license-files = ["LICENSE"]
urls.homepage = "https://github.com/microsoft/agent-framework"
urls.source = "https://github.com/microsoft/agent-framework/tree/main/python"
@@ -24,7 +24,7 @@ classifiers = [
"Typing :: Typed",
]
dependencies = [
"agent-framework-core>=1.1.1,<2",
"agent-framework-core>=1.1.0,<2",
"azure-core>=1.30.0,<2",
"httpx>=0.27.0,<0.29",
]
+2 -2
View File
@@ -4,7 +4,7 @@ description = "Redis integration for Microsoft Agent Framework."
authors = [{ name = "Microsoft", email = "af-support@microsoft.com"}]
readme = "README.md"
requires-python = ">=3.10"
version = "1.0.0b260423"
version = "1.0.0b260421"
license-files = ["LICENSE"]
urls.homepage = "https://aka.ms/agent-framework"
urls.source = "https://github.com/microsoft/agent-framework/tree/main/python"
@@ -23,7 +23,7 @@ classifiers = [
"Typing :: Typed",
]
dependencies = [
"agent-framework-core>=1.1.1,<2",
"agent-framework-core>=1.1.0,<2",
"redis>=6.4.0,<7.2.1",
"redisvl>=0.11.0,<0.16",
"numpy>=2.2.6,<3"
+2 -2
View File
@@ -4,7 +4,7 @@ description = "Microsoft Agent Framework for building AI Agents with Python. Thi
authors = [{ name = "Microsoft", email = "af-support@microsoft.com"}]
readme = "README.md"
requires-python = ">=3.10"
version = "1.1.1"
version = "1.1.0"
license-files = ["LICENSE"]
urls.homepage = "https://aka.ms/agent-framework"
urls.source = "https://github.com/microsoft/agent-framework/tree/main/python"
@@ -23,7 +23,7 @@ classifiers = [
"Typing :: Typed",
]
dependencies = [
"agent-framework-core[all]==1.1.1",
"agent-framework-core[all]==1.1.0",
]
[dependency-groups]
@@ -42,12 +42,11 @@ def create_sample_toolbox(name: str) -> str:
Toolboxes are normally configured in the Foundry portal or a deployment
script, not the application itself. This helper exists so the samples can
be run end-to-end without first setting a toolbox up by hand — delete any
existing toolbox under ``name``, then create a fresh version containing an
MCP tool, a web search tool, and a code interpreter tool. Returns the
created version identifier.
existing toolbox under ``name``, then create a fresh version containing a
single MCP tool. Returns the created version identifier.
"""
from azure.ai.projects import AIProjectClient
from azure.ai.projects.models import CodeInterpreterTool, MCPTool, Tool, WebSearchTool
from azure.ai.projects.models import MCPTool, Tool
from azure.core.exceptions import ResourceNotFoundError
with (
@@ -68,9 +67,6 @@ def create_sample_toolbox(name: str) -> str:
)
]
tools.append(WebSearchTool(name="web_search"))
tools.append(CodeInterpreterTool(name="code_interpreter"))
created = project_client.beta.toolboxes.create_version(
name=name,
description="Toolbox version with MCP require_approval set to 'never'.",
@@ -3,7 +3,6 @@
import os
import subprocess
from random import randint
from typing import Annotated
from agent_framework import Agent, tool
from agent_framework.foundry import FoundryChatClient
@@ -11,6 +10,7 @@ from agent_framework_foundry_hosting import ResponsesHostServer
from azure.identity import AzureCliCredential
from dotenv import load_dotenv
from pydantic import Field
from typing import Annotated
# Load environment variables from .env file
load_dotenv()
-11
View File
@@ -1,11 +0,0 @@
# Copyright (c) Microsoft. All rights reserved.
"""Flaky test report aggregation and trend generation.
Parses JUnit XML (``pytest.xml``) files produced by each CI job, merges
them with historical data, and generates a markdown trend report showing
per-test status across the last N runs.
Usage:
uv run python -m scripts.flaky_report <reports-dir> <history-file> <output-file>
"""
-20
View File
@@ -1,20 +0,0 @@
# Copyright (c) Microsoft. All rights reserved.
"""CLI entry point for the flaky test report tool.
Usage:
uv run python -m scripts.flaky_report <reports-dir> <history-file> <output-file>
Example (from python/ directory):
uv run python -m scripts.flaky_report \\
../flaky-reports/ \\
flaky-report-history.json \\
flaky-test-report.md
"""
import sys
from scripts.flaky_report.aggregate import main
if __name__ == "__main__":
sys.exit(main())
-396
View File
@@ -1,396 +0,0 @@
# Copyright (c) Microsoft. All rights reserved.
"""Aggregate per-provider JUnit XML test results and generate a trend report.
Parses ``pytest.xml`` (JUnit XML) files produced by each CI job, merges them
into a single run, combines with historical data, and generates a markdown
trend table — the same pattern used by ``scripts/sample_validation/aggregate.py``.
Usage (from CI):
python aggregate.py <reports-dir> <history-file> <output-file>
The reports directory is expected to contain subdirectories named
``test-results-<provider>/`` each containing a ``pytest.xml`` file
(created by ``actions/download-artifact``).
"""
from __future__ import annotations
import json
import sys
import xml.etree.ElementTree as ET
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
MAX_HISTORY = 5
STATUS_EMOJI = {
"passed": "âś…",
"failed": "❌",
"skipped": "⏭️",
"xfailed": "⚠️",
"error": "❌",
}
def _format_run_label(timestamp: str) -> str:
"""Format a timestamp as a compact column label (e.g. '04-16 00:57')."""
try:
dt = datetime.fromisoformat(timestamp)
return dt.strftime("%m-%d %H:%M")
except (ValueError, TypeError):
return timestamp[:16]
def _derive_provider(directory_name: str) -> str:
"""Derive a provider label from a report directory name.
``test-results-openai`` → ``OpenAI``
``test-results-azure-openai`` → ``Azure OpenAI``
"""
raw = directory_name.replace("test-results-", "")
known = {
"openai": "OpenAI",
"azure-openai": "Azure OpenAI",
"misc": "Misc (Anthropic, Ollama, MCP)",
"functions": "Functions",
"foundry": "Foundry",
"cosmos": "Cosmos",
"unit": "Unit",
}
if raw in known:
return known[raw]
parts = raw.split("-")
return " ".join(p.capitalize() for p in parts)
def _parse_junit_xml(xml_path: Path) -> list[dict[str, str]]:
"""Parse a JUnit XML file and return a list of test result dicts.
Each dict has keys: ``nodeid``, ``status``, ``duration``, ``message``.
"""
results: list[dict[str, str]] = []
try:
tree = ET.parse(xml_path) # noqa: S314
except ET.ParseError as exc:
print(f"Warning: failed to parse JUnit XML report '{xml_path}': {exc}", file=sys.stderr)
return results
root = tree.getroot()
# Handle both <testsuites><testsuite>... and <testsuite>... layouts
testcases: list[ET.Element] = []
if root.tag == "testsuites":
for suite in root.findall("testsuite"):
testcases.extend(suite.findall("testcase"))
elif root.tag == "testsuite":
testcases = list(root.findall("testcase"))
for tc in testcases:
classname = tc.get("classname", "")
name = tc.get("name", "")
duration = tc.get("time", "0")
# Use classname::name as a stable identifier.
# pytest writes classname as the dotted module path (possibly including
# a test class), e.g. "packages.openai.tests.openai.test_chat_client"
# or "packages.openai.tests.openai.test_chat_client.TestClass".
nodeid = f"{classname}::{name}" if classname else name
# Extract module/file name from classname for display context.
# pytest writes classname as a dotted path. For tests inside a class
# it appends the class name, e.g.:
# "packages.foundry.tests.foundry.test_foundry_embedding_client.TestFoundryEmbeddingIntegration"
# We want the file-level module: "test_foundry_embedding_client"
if classname:
parts = classname.rsplit(".", 2)
# If the last segment starts with uppercase it's a class name — take the one before it
if len(parts) >= 2 and parts[-1][0:1].isupper():
module = parts[-2]
else:
module = parts[-1]
else:
module = ""
# Determine status from child elements
failure = tc.find("failure")
error = tc.find("error")
skipped = tc.find("skipped")
if failure is not None:
status = "failed"
message = failure.get("message", "")
elif error is not None:
status = "error"
message = error.get("message", "")
elif skipped is not None:
# pytest marks xfail as <skipped type="pytest.xfail">
skip_type = skipped.get("type", "")
status = "xfailed" if "xfail" in skip_type else "skipped"
message = skipped.get("message", "")
else:
status = "passed"
message = ""
results.append({
"nodeid": nodeid,
"status": status,
"duration": duration,
"message": message,
"module": module,
})
return results
# ---------------------------------------------------------------------------
# Loading
# ---------------------------------------------------------------------------
def load_current_run(reports_dir: Path) -> dict[str, Any]:
"""Load per-provider JUnit XML reports from the current CI run and merge.
Args:
reports_dir: Directory containing ``test-results-<provider>/`` subdirs.
Returns:
Merged run dict with ``timestamp``, ``summary``, ``results``.
"""
combined_results: dict[str, dict[str, str]] = {} # nodeid → {status, provider}
# actions/download-artifact creates: reports_dir/test-results-openai/pytest.xml
xml_files: list[tuple[str, Path]] = []
if reports_dir.is_dir():
for subdir in sorted(reports_dir.iterdir()):
if subdir.is_dir():
xml_file = subdir / "pytest.xml"
if xml_file.exists():
xml_files.append((subdir.name, xml_file))
if not xml_files:
print(f"Warning: No pytest.xml files found in {reports_dir}")
return {
"timestamp": datetime.now(timezone.utc).isoformat(),
"summary": {
"total": 0,
"passed": 0,
"failed": 0,
"skipped": 0,
},
"results": {},
}
for dir_name, xml_file in xml_files:
print(f" Loading: {xml_file}")
provider = _derive_provider(dir_name)
tests = _parse_junit_xml(xml_file)
for test in tests:
combined_results[test["nodeid"]] = {
"status": test["status"],
"provider": provider,
"module": test.get("module", ""),
}
# Build summary counts using mutually exclusive status buckets.
# Errors are folded into the failed count for display purposes.
statuses = [r["status"] for r in combined_results.values()]
summary = {
"total": len(statuses),
"passed": statuses.count("passed"),
"failed": statuses.count("failed") + statuses.count("error"),
"skipped": statuses.count("skipped"),
}
return {
"timestamp": datetime.now(timezone.utc).isoformat(),
"summary": summary,
"results": combined_results,
}
def load_history(history_path: Path) -> list[dict[str, Any]]:
"""Load previous run history from a cache file."""
if history_path.exists():
with open(history_path, encoding="utf-8") as f:
data = json.load(f)
runs = data.get("runs", [])
print(f" Loaded {len(runs)} previous run(s) from history")
return runs
print(" No previous history found")
return []
def save_history(history_path: Path, runs: list[dict[str, Any]]) -> None:
"""Save run history, keeping only the last ``MAX_HISTORY`` entries."""
history_path.parent.mkdir(parents=True, exist_ok=True)
trimmed = runs[-MAX_HISTORY:]
with open(history_path, "w", encoding="utf-8") as f:
json.dump({"runs": trimmed}, f, indent=2)
print(f" Saved {len(trimmed)} run(s) to history")
# ---------------------------------------------------------------------------
# Report generation
# ---------------------------------------------------------------------------
def _short_name(nodeid: str) -> str:
"""Extract a short test name from a full nodeid.
``packages.openai.tests.openai.test_openai_chat_client::test_integration_options``
→ ``test_integration_options``
"""
return nodeid.split("::")[-1] if "::" in nodeid else nodeid
def generate_trend_report(runs: list[dict[str, Any]]) -> str:
"""Generate a markdown trend report from run history."""
lines = [
"# 🔬 Flaky Test Report",
"",
f"*Generated: {datetime.now(timezone.utc).strftime('%Y-%m-%d %H:%M UTC')}*",
"",
]
# --- Overall status table (most recent first) ---
lines.append("## Overall Status (Last 5 Runs)")
lines.append("")
lines.append("| Run | Total | ✅ Passed | ❌ Failed | ⏭️ Skipped |")
lines.append("|-----|-------|-----------|-----------|------------|")
for run in reversed(runs):
s = run.get("summary", {})
total = s.get("total", 0)
label = _format_run_label(run["timestamp"])
lines.append(
f"| {label} "
f"| {total} "
f"| {s.get('passed', 0)}/{total} "
f"| {s.get('failed', 0)}/{total} "
f"| {s.get('skipped', 0)}/{total} |"
)
for _ in range(MAX_HISTORY - len(runs)):
lines.append("| N/A | N/A | N/A | N/A | N/A |")
lines.append("")
# --- Per-test results table ---
lines.append("## Per-Test Results")
lines.append("")
# Collect all test nodeids, providers, and modules across all runs
all_tests: dict[str, str] = {} # nodeid → provider (from most recent run)
all_modules: dict[str, str] = {} # nodeid → module (from most recent run)
for run in runs:
for nodeid, info in run.get("results", {}).items():
provider = info.get("provider", "Unknown") if isinstance(info, dict) else "Unknown"
module = info.get("module", "") if isinstance(info, dict) else ""
all_tests[nodeid] = provider
all_modules[nodeid] = module
if not all_tests:
lines.append("*No test results available.*")
return "\n".join(lines)
# Build header (most recent run first)
header = "| Test | File | Provider |"
separator = "|------|------|----------|"
for run in reversed(runs):
label = _format_run_label(run["timestamp"])
header += f" {label} |"
separator += "------------|"
for _ in range(MAX_HISTORY - len(runs)):
header += " N/A |"
separator += "-----|"
lines.append(header)
lines.append(separator)
# Sort by provider then test name
for nodeid in sorted(all_tests, key=lambda n: (all_tests[n], n)):
provider = all_tests[nodeid]
module = all_modules.get(nodeid, "")
short = _short_name(nodeid)
row = f"| `{short}` | `{module}` | {provider} |"
for run in reversed(runs):
result = run.get("results", {}).get(nodeid)
if result is None:
emoji = "N/A"
else:
status = result.get("status", "N/A") if isinstance(result, dict) else result
emoji = STATUS_EMOJI.get(status, "âť“")
row += f" {emoji} |"
for _ in range(MAX_HISTORY - len(runs)):
row += " N/A |"
lines.append(row)
lines.append("")
lines.append("**Legend:** ✅ Passed · ❌ Failed · ⏭️ Skipped · ⚠️ Expected Failure (xfail) · N/A Not available")
lines.append("")
return "\n".join(lines)
# ---------------------------------------------------------------------------
# CLI entry point
# ---------------------------------------------------------------------------
def main() -> int:
if len(sys.argv) != 4:
print("Usage: python aggregate.py <reports-dir> <history-file> <output-file>")
return 1
reports_dir = Path(sys.argv[1])
history_path = Path(sys.argv[2])
output_path = Path(sys.argv[3])
print("Aggregating test results from JUnit XML...")
# Load current run's per-provider XML reports
print(f"\nLoading reports from {reports_dir}:")
current_run = load_current_run(reports_dir)
s = current_run.get("summary", {})
total = s.get("total", 0)
print(
f" Current run: {s.get('passed', 0)} passed, "
f"{s.get('failed', 0)} failed, "
f"{s.get('skipped', 0)} skipped "
f"(total: {total})"
)
# Load history and append current run (skip empty runs to avoid polluting trend)
print(f"\nLoading history from {history_path}:")
runs = load_history(history_path)
if total > 0:
runs.append(current_run)
runs = runs[-MAX_HISTORY:]
else:
print(" Skipping history append (no test results in current run)")
# Save updated history
print(f"\nSaving history to {history_path}:")
save_history(history_path, runs)
# Generate trend report
print("\nGenerating trend report...")
report = generate_trend_report(runs)
output_path.parent.mkdir(parents=True, exist_ok=True)
output_path.write_text(report, encoding="utf-8")
print(f"Trend report written to {output_path}")
# Print the report to stdout for CI visibility
print("\n" + "=" * 80)
print(report)
return 0
if __name__ == "__main__":
sys.exit(main())
+28 -28
View File
@@ -96,7 +96,7 @@ wheels = [
[[package]]
name = "agent-framework"
version = "1.1.1"
version = "1.1.0"
source = { virtual = "." }
dependencies = [
{ name = "agent-framework-core", extra = ["all"], marker = "sys_platform == 'darwin' or sys_platform == 'linux' or sys_platform == 'win32'" },
@@ -151,7 +151,7 @@ dev = [
[[package]]
name = "agent-framework-a2a"
version = "1.0.0b260423"
version = "1.0.0b260421"
source = { editable = "packages/a2a" }
dependencies = [
{ name = "a2a-sdk", marker = "sys_platform == 'darwin' or sys_platform == 'linux' or sys_platform == 'win32'" },
@@ -166,7 +166,7 @@ requires-dist = [
[[package]]
name = "agent-framework-ag-ui"
version = "1.0.0b260423"
version = "1.0.0b260421"
source = { editable = "packages/ag-ui" }
dependencies = [
{ name = "ag-ui-protocol", marker = "sys_platform == 'darwin' or sys_platform == 'linux' or sys_platform == 'win32'" },
@@ -194,7 +194,7 @@ provides-extras = ["dev"]
[[package]]
name = "agent-framework-anthropic"
version = "1.0.0b260423"
version = "1.0.0b260421"
source = { editable = "packages/anthropic" }
dependencies = [
{ name = "agent-framework-core", marker = "sys_platform == 'darwin' or sys_platform == 'linux' or sys_platform == 'win32'" },
@@ -209,7 +209,7 @@ requires-dist = [
[[package]]
name = "agent-framework-azure-ai-search"
version = "1.0.0b260423"
version = "1.0.0b260421"
source = { editable = "packages/azure-ai-search" }
dependencies = [
{ name = "agent-framework-core", marker = "sys_platform == 'darwin' or sys_platform == 'linux' or sys_platform == 'win32'" },
@@ -224,7 +224,7 @@ requires-dist = [
[[package]]
name = "agent-framework-azure-cosmos"
version = "1.0.0b260423"
version = "1.0.0b260421"
source = { editable = "packages/azure-cosmos" }
dependencies = [
{ name = "agent-framework-core", marker = "sys_platform == 'darwin' or sys_platform == 'linux' or sys_platform == 'win32'" },
@@ -239,7 +239,7 @@ requires-dist = [
[[package]]
name = "agent-framework-azurefunctions"
version = "1.0.0b260423"
version = "1.0.0b260421"
source = { editable = "packages/azurefunctions" }
dependencies = [
{ name = "agent-framework-core", marker = "sys_platform == 'darwin' or sys_platform == 'linux' or sys_platform == 'win32'" },
@@ -261,7 +261,7 @@ dev = []
[[package]]
name = "agent-framework-bedrock"
version = "1.0.0b260423"
version = "1.0.0b260421"
source = { editable = "packages/bedrock" }
dependencies = [
{ name = "agent-framework-core", marker = "sys_platform == 'darwin' or sys_platform == 'linux' or sys_platform == 'win32'" },
@@ -278,7 +278,7 @@ requires-dist = [
[[package]]
name = "agent-framework-chatkit"
version = "1.0.0b260423"
version = "1.0.0b260421"
source = { editable = "packages/chatkit" }
dependencies = [
{ name = "agent-framework-core", marker = "sys_platform == 'darwin' or sys_platform == 'linux' or sys_platform == 'win32'" },
@@ -293,7 +293,7 @@ requires-dist = [
[[package]]
name = "agent-framework-claude"
version = "1.0.0b260423"
version = "1.0.0b260421"
source = { editable = "packages/claude" }
dependencies = [
{ name = "agent-framework-core", marker = "sys_platform == 'darwin' or sys_platform == 'linux' or sys_platform == 'win32'" },
@@ -308,7 +308,7 @@ requires-dist = [
[[package]]
name = "agent-framework-copilotstudio"
version = "1.0.0b260423"
version = "1.0.0b260421"
source = { editable = "packages/copilotstudio" }
dependencies = [
{ name = "agent-framework-core", marker = "sys_platform == 'darwin' or sys_platform == 'linux' or sys_platform == 'win32'" },
@@ -323,7 +323,7 @@ requires-dist = [
[[package]]
name = "agent-framework-core"
version = "1.1.1"
version = "1.1.0"
source = { editable = "packages/core" }
dependencies = [
{ name = "opentelemetry-api", marker = "sys_platform == 'darwin' or sys_platform == 'linux' or sys_platform == 'win32'" },
@@ -395,7 +395,7 @@ provides-extras = ["all"]
[[package]]
name = "agent-framework-declarative"
version = "1.0.0b260423"
version = "1.0.0b260421"
source = { editable = "packages/declarative" }
dependencies = [
{ name = "agent-framework-core", marker = "sys_platform == 'darwin' or sys_platform == 'linux' or sys_platform == 'win32'" },
@@ -420,7 +420,7 @@ dev = [{ name = "types-pyyaml", specifier = "==6.0.12.20250915" }]
[[package]]
name = "agent-framework-devui"
version = "1.0.0b260423"
version = "1.0.0b260421"
source = { editable = "packages/devui" }
dependencies = [
{ name = "agent-framework-core", marker = "sys_platform == 'darwin' or sys_platform == 'linux' or sys_platform == 'win32'" },
@@ -458,7 +458,7 @@ provides-extras = ["dev", "all"]
[[package]]
name = "agent-framework-durabletask"
version = "1.0.0b260423"
version = "1.0.0b260421"
source = { editable = "packages/durabletask" }
dependencies = [
{ name = "agent-framework-core", marker = "sys_platform == 'darwin' or sys_platform == 'linux' or sys_platform == 'win32'" },
@@ -485,7 +485,7 @@ dev = [{ name = "types-python-dateutil", specifier = "==2.9.0.20260402" }]
[[package]]
name = "agent-framework-foundry"
version = "1.1.1"
version = "1.1.0"
source = { editable = "packages/foundry" }
dependencies = [
{ name = "agent-framework-core", marker = "sys_platform == 'darwin' or sys_platform == 'linux' or sys_platform == 'win32'" },
@@ -504,7 +504,7 @@ requires-dist = [
[[package]]
name = "agent-framework-foundry-hosting"
version = "1.0.0a260423"
version = "1.0.0a260421"
source = { editable = "packages/foundry_hosting" }
dependencies = [
{ name = "agent-framework-core", marker = "sys_platform == 'darwin' or sys_platform == 'linux' or sys_platform == 'win32'" },
@@ -523,7 +523,7 @@ requires-dist = [
[[package]]
name = "agent-framework-foundry-local"
version = "1.0.0b260423"
version = "1.0.0b260421"
source = { editable = "packages/foundry_local" }
dependencies = [
{ name = "agent-framework-core", marker = "sys_platform == 'darwin' or sys_platform == 'linux' or sys_platform == 'win32'" },
@@ -540,7 +540,7 @@ requires-dist = [
[[package]]
name = "agent-framework-gemini"
version = "1.0.0a260423"
version = "1.0.0a260421"
source = { editable = "packages/gemini" }
dependencies = [
{ name = "agent-framework-core", marker = "sys_platform == 'darwin' or sys_platform == 'linux' or sys_platform == 'win32'" },
@@ -555,7 +555,7 @@ requires-dist = [
[[package]]
name = "agent-framework-github-copilot"
version = "1.0.0b260423"
version = "1.0.0b260421"
source = { editable = "packages/github_copilot" }
dependencies = [
{ name = "agent-framework-core", marker = "sys_platform == 'darwin' or sys_platform == 'linux' or sys_platform == 'win32'" },
@@ -570,7 +570,7 @@ requires-dist = [
[[package]]
name = "agent-framework-hyperlight"
version = "1.0.0a260423"
version = "1.0.0a260421"
source = { editable = "packages/hyperlight" }
dependencies = [
{ name = "agent-framework-core", marker = "sys_platform == 'darwin' or sys_platform == 'linux' or sys_platform == 'win32'" },
@@ -589,7 +589,7 @@ requires-dist = [
[[package]]
name = "agent-framework-lab"
version = "1.0.0b260423"
version = "1.0.0b260421"
source = { editable = "packages/lab" }
dependencies = [
{ name = "agent-framework-core", marker = "sys_platform == 'darwin' or sys_platform == 'linux' or sys_platform == 'win32'" },
@@ -670,7 +670,7 @@ dev = [
[[package]]
name = "agent-framework-mem0"
version = "1.0.0b260423"
version = "1.0.0b260421"
source = { editable = "packages/mem0" }
dependencies = [
{ name = "agent-framework-core", marker = "sys_platform == 'darwin' or sys_platform == 'linux' or sys_platform == 'win32'" },
@@ -685,7 +685,7 @@ requires-dist = [
[[package]]
name = "agent-framework-ollama"
version = "1.0.0b260423"
version = "1.0.0b260421"
source = { editable = "packages/ollama" }
dependencies = [
{ name = "agent-framework-core", marker = "sys_platform == 'darwin' or sys_platform == 'linux' or sys_platform == 'win32'" },
@@ -700,7 +700,7 @@ requires-dist = [
[[package]]
name = "agent-framework-openai"
version = "1.1.1"
version = "1.1.0"
source = { editable = "packages/openai" }
dependencies = [
{ name = "agent-framework-core", marker = "sys_platform == 'darwin' or sys_platform == 'linux' or sys_platform == 'win32'" },
@@ -715,7 +715,7 @@ requires-dist = [
[[package]]
name = "agent-framework-orchestrations"
version = "1.0.0b260423"
version = "1.0.0b260421"
source = { editable = "packages/orchestrations" }
dependencies = [
{ name = "agent-framework-core", marker = "sys_platform == 'darwin' or sys_platform == 'linux' or sys_platform == 'win32'" },
@@ -726,7 +726,7 @@ requires-dist = [{ name = "agent-framework-core", editable = "packages/core" }]
[[package]]
name = "agent-framework-purview"
version = "1.0.0b260423"
version = "1.0.0b260421"
source = { editable = "packages/purview" }
dependencies = [
{ name = "agent-framework-core", marker = "sys_platform == 'darwin' or sys_platform == 'linux' or sys_platform == 'win32'" },
@@ -743,7 +743,7 @@ requires-dist = [
[[package]]
name = "agent-framework-redis"
version = "1.0.0b260423"
version = "1.0.0b260421"
source = { editable = "packages/redis" }
dependencies = [
{ name = "agent-framework-core", marker = "sys_platform == 'darwin' or sys_platform == 'linux' or sys_platform == 'win32'" },