Replaces every floating tag in our workflow and composite action files
with an immutable 40-character commit SHA, keeping the original `# vX`
comment so Dependabot can still propose version bumps. 186 occurrences
across 25 workflows and 2 composite actions.
Also widens the github-actions Dependabot entry to use the plural
`directories` key with `/.github/actions/*` so composite actions under
`.github/actions/<name>/action.yml` are kept up to date. Previously
Dependabot only scanned `.github/workflows` and the repo-root
`action.yml`, leaving our `python-setup` and `sample-validation-setup`
composite actions unmaintained.
* Split DurableTask/AzureFunctions integration tests into dedicated CI job
- Add -TestProjectNameExclude parameter to New-FilteredSolution.ps1
- Add 'functions' and 'core' path filters to paths-filter job
- Exclude DurableTask/AzureFunctions from main dotnet-test job
- Remove emulator setup from dotnet-test (no longer needed)
- Add new dotnet-test-functions job (ubuntu/net10.0 only, path-conditional)
- Update merge gate and report job to include dotnet-test-functions
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address PR feedback: add Workflows.Generators to core filter, drop dotnetChanges gate from functions job
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Re-enable Anthropic integration tests
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Upgrade Anthropic SDK 12.13.0 -> 12.20.0 to fix M.E.AI incompatibility
Fixes MissingMethodException on WebSearchToolResultContent.get_Results()
caused by Anthropic 12.13.0 being compiled against an older
Microsoft.Extensions.AI.Abstractions version.
Suppress RT0003 in AI.Abstractions.csproj as the transitive reference
from the upgraded Anthropic SDK conflicts with the explicit one.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix Anthropic unit test mocks for SDK 12.20.0 interface changes
Add missing interface members: IAnthropicClient.WebhookKey,
IBetaService.MemoryStores, IBetaService.Webhooks, IBetaService.UserProfiles
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Re-enable CheckSystem declarative integration tests
The CheckSystem.yaml tests were temporarily skipped in PR #4270 during
the Azure.AI.Projects 2.0.0-beta.1 SDK update. Since then, the system
variable plumbing (SystemScope, SetLastMessageAsync, conversation
initialization) has been significantly updated and stabilized. The
other tests in these same files pass reliably using the same
infrastructure.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix CheckSystem test case to expect 1 response
The CheckSystem workflow sends a 'PASSED!' SendActivity when all system
variables are populated, producing 1 AgentResponseEvent. The test case
had min_response_count: 0 with no max, so the assertion defaulted max
to 0 and failed with 'Response count greater than expected: 0 (Actual: 1)'.
Updated to expect exactly 1 response, matching the SendActivity pattern.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Re-enable Foundry OpenAPI server-side tool integration test
Remove Skip="For manual testing only" from
AsAIAgent_WithOpenAPITool_NativeSDKCreation_InvokesServerSideToolAsync.
The test already uses RetryFact(3 retries, 5s delay) to handle
transient failures from the external restcountries.com API.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Include workflow file in functions/core path filters
A PR editing only dotnet-build-and-test.yml would skip
dotnet-test-functions because the workflow path was missing
from both the functions and core path filter lists.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Rename filter parameters for consistency
TestProjectNameFilter -> TestProjectNameIncludeFilter
TestProjectNameExclude -> TestProjectNameExcludeFilter
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Remove unnecessary RT0003 warning suppression
The RT0003 suppression was added during the Anthropic SDK 12.20.0
upgrade but the warning no longer fires. Removing it to keep the
NoWarn list minimal.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Remove duplicate WebhookKey properties from merge
Both our branch and main added WebhookKey to the Anthropic test
mock classes, resulting in CS0102 duplicate definition errors.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* .NET: Hosted Agents - RAG Sample with Azure AI Search (#5693)
Adds a Hosted-AzureSearchRag sample plus a live Foundry.Hosting integration
test scenario backed by a real Azure AI Search index.
Sample (Hosted-AzureSearchRag): keyword-only Azure AI Search via
SearchClient adapter into TextSearchProvider, scope-aware
DevTemporaryTokenCredential consuming AZURE_BEARER_TOKEN_FOUNDRY +
AZURE_BEARER_TOKEN_SEARCH for local Docker, Dockerfile + contributor
Dockerfile mirroring Hosted-TextRag.
Integration test: AzureSearchRagHostedAgentFixture extends the PR #5598
HostedAgentFixture with the new azure-search-rag scenario branch in the
shared test container; AzureSearchRagHostedAgentTests asserts the model
returns canary tokens (TR-CANARY-7821, SHIP-CANARY-4493) that exist only
in the seeded documents - real proof the agent grounded its answer in
retrieved content rather than training data.
* Address PR 5701 Copilot review feedback
- Sample README: drop stale 'bootstraps the index on first run' line; index is pre-provisioned out of band
- Sample + TestContainer search adapters: propagate CancellationToken to await foreach via .WithCancellation()
* .NET: Foundry.Hosted IT - fix MSBuild parallel-output races
Two surgical changes inside the dotnet-foundry-hosted-it job:
1. Replace dotnet build <slnx> -f net10.0 with dotnet build <test.csproj>. The test csproj pins TargetFrameworks=net10.0 and its ProjectReference closure gives MSBuild a single-rooted graph, eliminating the duplicate inner-builds that race on bin/obj. Drops the two New-FilteredSolution.ps1 steps.
2. In it-build-image.ps1, drop the -UsePrebuiltProjectReferences switch and always pass --no-dependencies to dotnet publish. Publish now resolves TestContainer's framework refs by reading prebuilt DLLs and never re-touches them. Replaces the partial-mitigation in PR #5689 with a structural fix.
Local validation confirmed published Foundry.dll has identical mtime and bytes as the prebuild output.
* .NET: dotnet test - use --project flag for Microsoft Testing Platform
* Add dotnet integration test report to CI
- Add --report-junit flag to dotnet integration test step to generate
JUnit XML alongside TRX, with explicit --results-directory to
centralize output in IntegrationTestResults/
- Upload JUnit XML artifacts from each matrix leg (net10.0/ubuntu,
net472/windows) as dotnet-test-results-{framework}-{os}
- Add dotnet-integration-test-report job that downloads artifacts,
runs the existing aggregate.py script, posts markdown to Job Summary,
and saves trend history via actions/cache
- Refactor aggregate.py to discover JUnit XML files recursively,
supporting both pytest (pytest.xml) and xunit (*.junit.xml) layouts
- Handle provider name derivation for dotnet artifact naming convention
- Fix nodeid collision when same test runs under multiple frameworks
by qualifying keys with provider when collisions are detected
- Improve module extraction for dotnet C# classnames (recognizes
IntegrationTests/UnitTests namespace segments)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* chore: trigger dotnet CI for report validation
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: use .junit extension (not .junit.xml) for xunit v3 output
xUnit v3 generates files with .junit extension, not .junit.xml.
Update upload glob and aggregate.py discovery to match.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: use deterministic provider-qualified keys for dotnet tests
Always prefix dotnet test keys with provider (e.g. net10.0 (ubuntu)::TestName)
to ensure stable, comparable counts across runs regardless of file parse order.
Also show Executed (passed+failed) instead of Total in summary table.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: match Python report summary format (Total, passed/total, etc.)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat: split dotnet report into per-framework tables
Dotnet tests run on multiple frameworks (net10.0, net472). Instead of
one combined table with unstable totals, show separate sections per
framework — each with its own summary row and per-test table. Python
reports retain the original single-table format.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Re-enable 7 flaky dotnet integration tests with increased timeouts
Increase timeouts to reduce timing-related flakiness in LLM-backed
integration tests (issue #4971):
- ExternalClientTests: 60s -> 120s default timeout
- SamplesValidationBase: 60s -> 120s default timeout
- ConsoleAppSamplesValidation: 90s -> 150s for long-running tests
- AzureFunctions SamplesValidation: 2min -> 3min orchestration timeout,
60s -> 90s per-step WaitForConditionAsync timeouts
Remove all Skip=Flaky annotations and unused SkipFlakyTimingTest constants.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Re-skip LLM non-determinism flaky tests, keep timeout fixes
Re-skip SingleAgentOrchestrationHITLSampleValidationAsync and
LongRunningToolsSampleValidationAsync - these fail due to LLM producing
extra review notifications, not timeouts. Updated skip reasons to
accurately describe the root cause. Reverted unnecessary timeout change
on the skipped LongRunningTools test.
The remaining 5 re-enabled tests with timeout increases are stable.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Enable Anthropic integration tests in CI
Replace hardcoded skip with conditional skip pattern (matching
CopilotStudio approach): tests gracefully skip when ANTHROPIC_API_KEY
is missing, and run when present.
Changes:
- AnthropicChatCompletionFixture: try/catch in InitializeAsync with
Assert.Skip on missing config (replaces hardcoded SkipReason)
- AnthropicSkillsIntegrationTests: same pattern per test method
- dotnet-build-and-test.yml: wire up ANTHROPIC_API_KEY,
ANTHROPIC_CHAT_MODEL_NAME, and ANTHROPIC_REASONING_MODEL_NAME
env vars to the integration test step
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix missing System using in AnthropicSkillsIntegrationTests
Add 'using System;' for InvalidOperationException in try/catch blocks.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Skip flaky SingleAgentOrchestrationChainingSampleValidationAsync
LLM non-determinism causes Assert.NotNull failures on orchestration
results. Skip until test logic is hardened against non-deterministic
LLM responses.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Re-enable HITL and LongRunningTools tests with timeout and flexibility fixes
- Remove Skip attribute from SingleAgentOrchestrationHITLSampleValidationAsync
- Remove Skip attribute from LongRunningToolsSampleValidationAsync
- Increase timeout from 120s/90s to 180s to accommodate 2+ LLM round-trips
- Replace rigid 2-cycle assertion with flexible approval logic that handles
extra review cycles from LLM non-determinism
Fixes the two failure modes identified in #4971:
1. Timeout: 120s/90s was insufficient for multiple LLM calls under CI load
2. Extra notifications: Assert.Fail on 3rd+ review cycle was too rigid
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Increase AzureFunctions LongRunningTools test timeouts from 90s to 180s
The LongRunningToolsSampleValidationAsync test in the AzureFunctions integration
tests was failing in CI with TimeoutException at the 'Content published
notification is logged' step. The 90-second timeouts are too tight for CI
environments where LLM calls and orchestration overhead can be slow.
Increased all three WaitForConditionAsync timeouts from 90s to 180s:
- Waiting for human feedback notification
- Waiting for publish notification (the step that was failing)
- Waiting for orchestration completion
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Merge main and fix dotnet report path after flaky_report rename
Merge upstream/main which renamed scripts/flaky_report/ to
scripts/integration_test_report/ (from Python PR #5454). Update the
dotnet-build-and-test workflow to reference the new path.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add RetryFact to DurableTask and AzureFunctions integration tests
These tests interact with LLMs via stdin/stdout (DurableTask) or HTTP
(AzureFunctions) and are inherently non-deterministic. Unlike the Python
side which uses pytest-retry, the dotnet tests had no retry mechanism
and a single transient failure would fail the entire CI run.
Changes:
- Switch [Fact] to [RetryFact(2, 5000)] on all LLM-dependent tests
across ConsoleAppSamplesValidation, ExternalClientTests,
WorkflowConsoleAppSamplesValidation, and AzureFunctions SamplesValidation
- Add re-prompt mechanism to LongRunningToolsSampleValidationAsync:
if the LLM doesn't invoke the tool within 60s, re-send the prompt
(up to 2 retries) instead of burning the full timeout
- Reduce LongRunningTools timeout from 240s to 180s (re-prompt makes
the extra buffer unnecessary)
- Leave simple/deterministic tests as [Fact] (SingleAgent, unit tests)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add persist-credentials: false to Integration Test Report checkout step
Matches the convention used by other checkout steps in this workflow
to avoid leaving GITHUB_TOKEN credentials in the local git config.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* small fixes
* disable anthropic failing tests
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
CI publish step: gate the BuildProjectReferences=false fast-path on an explicit -UsePrebuiltProjectReferences switch (passed by the workflow) instead of marker detection. Adds a preflight error when stale obj/Release/net10.0 outputs would cause CS0579, with actionable recovery instructions.
Telemetry UT flake: AgentFrameworkResponseHandlerTelemetryTests was using a plain List<Activity> for OTel's InMemoryExporter. The exporter writes from background Activity completion callbacks while parallel tests on the same global ActivitySource feed every listener, racing against the assertion's enumeration and throwing 'Collection was modified'. Replaced with a small thread-safe ConcurrentActivityList that locks add/enumerate and returns a snapshot for assertions.
* Foundry.Hosting.IntegrationTests: scaffold project, fixtures, and 24 tests
Add a new integration test project for Foundry hosted agents alongside the existing Foundry.IntegrationTests project. The project provisions a real Foundry hosted agent per scenario via AgentAdministrationClient.CreateAgentVersionAsync, points it at a single test container image (built and pushed out of band by scripts/it-build-image.ps1 in a follow up commit), and exercises the agent through AIProjectClient.AsAIAgent.
Six scenario fixtures are introduced, each pointing at the same image but selecting behavior via the IT_SCENARIO environment variable on the HostedAgentDefinition:
- HappyPathHostedAgentFixture (round trip, multi turn, stored=false flag)
- ToolCallingHostedAgentFixture (server side AIFunctions)
- ToolCallingApprovalHostedAgentFixture (approval flow)
- ToolboxHostedAgentFixture (Foundry toolbox)
- McpToolboxHostedAgentFixture (MCP backed toolbox)
- CustomStorageHostedAgentFixture (custom storage provider)
24 tests across 6 test classes are scaffolded. All are tagged Skip pending the test container build and the end to end smoke iteration in follow up commits. Once the container is in place the Skip annotations can be removed scenario by scenario.
Adds an IT_HOSTED_AGENT_IMAGE constant to the shared TestSettings so every IT project agrees on the env var name the build script emits.
* Foundry.Hosting.IntegrationTests: add TestContainer, build script, slnx, README
Adds the rest of the integration test infrastructure on top of the previous scaffolding commit:
* Foundry.Hosting.IntegrationTests.TestContainer csproj and Program.cs implementing the multi scenario container (one image, IT_SCENARIO env var dispatches between happy-path, tool-calling, tool-calling-approval, toolbox, mcp-toolbox, and custom-storage). The toolbox, mcp-toolbox, and custom-storage branches are placeholders pending API surface stabilization.
* Dockerfile and dockerignore in the test container project, using the contributor pattern matching the investigation work (host side dotnet publish, container only does COPY out/).
* scripts/it-build-image.ps1 with mandatory Registry parameter (no hardcoded ACR), content hashed tags so unchanged source results in a no op push, and emits IT_HOSTED_AGENT_IMAGE for shells and CI to consume.
* slnx entry for both new projects.
* README in the IT project covering env vars, image build, scenario table, and current placeholder status.
Steps still pending: end to end smoke (step 5) and CI workflow integration (step 6) require a live Foundry deployment and ACR push, so they land in follow up commits.
* Foundry.Hosting.IntegrationTests: address PR 5598 review feedback
Fix issues raised by Copilot review:
* it-build-image.ps1: hash file contents, not the path list, so any source edit produces a fresh tag. Normalize Registry input by stripping scheme and trailing slash before deriving the ACR short name. Validate the short name is non empty.
* HostedAgentFixture: route GetAgentAsync through _adminClient (which has the FoundryFeaturesPolicy attached) instead of through _projectClient.AgentAdministrationClient (which does not).
* HostedAgentFixture FoundryFeaturesPolicy: replace Headers.Add with Remove plus Add so retries cannot accumulate duplicate headers.
* HappyPath, ToolCalling, ToolCallingApproval, CustomStorage tests: create the AgentSession before turn 1 and reuse it for both turns. The previous pattern created the session after turn 1 so turn 2 had no link to turn 1, defeating the multi turn assertion.
* .NET: Foundry.Hosting.IntegrationTests: constrain to net10.0 + dotnet format autofix
- Set <TargetFrameworks>net10.0</TargetFrameworks>: the project references both
Microsoft.Agents.AI.Foundry.Hosting (net8/9/10 only) and AgentConformance.IntegrationTests
(net10.0;net472 — inherits the tests-default TFM list). The intersection is net10.0;
the previous $(TargetFrameworksCore) triple caused NU1702 + System.Text.Json version
conflicts on the net8.0/net9.0 builds because AgentConformance had no matching asset.
- Apply `dotnet format` autofix on the test files (IDE0005, IDE0009, IDE0032, IMPORTS).
* .NET: Foundry.Hosting.IntegrationTests.TestContainer/Program.cs: add UTF-8 BOM
CI's check-format requires charset=utf-8-bom per .editorconfig.
* Foundry.Hosting IntegrationTests: wire end-to-end CI flow against hosted agents
Make the integration tests usable end-to-end against a live Foundry deployment, including
a per-run rebuild of the test container so framework code changes are exercised.
Fixture (HostedAgentFixture.cs)
* Switch from per-run unique agent names to stable scenario-keyed names (it-happy-path,
it-tool-calling, ...). The agent's managed identity carries the Azure AI User role on
the project scope, which is required for inbound inference; deleting the agent recycles
the MI and breaks that role assignment, so we keep the agent across runs and only churn
versions.
* Add IT_RUN_ID env var to defeat Foundry's content-addressed version dedup; otherwise a
rerun just receives the existing version and Dispose deletes it.
* PATCH the per-agent endpoint with AgentEndpointConfig (Responses protocol, version
selector at 100% to the new version). Without this, /agents/{name}/endpoint/protocols/
openai/responses returns HTTP 400.
* Build a per-agent ProjectOpenAIClient (not the cached projectClient.ProjectOpenAIClient,
which is bound to the project-level URL); set AgentName in options so the URL routes
through the agent endpoint, and add the Foundry-Features header to the inference
pipeline.
* Use Versions (which serializes to container_protocol_versions) instead of the
deprecated ProtocolVersions; the server now rejects the legacy field.
* On Dispose, delete only the version this fixture created. Never delete the agent.
Tests
* Tag every HostedAgentTests class with [Trait("Category", "FoundryHostedAgents")] so the
CI workflow can route them to a separate Foundry project than the rest of the
integration suite.
CI workflow (.github/workflows/dotnet-build-and-test.yml)
* Add a foundryHosting paths-filter covering Microsoft.Agents.AI.Foundry.Hosting and its
in-repo dependency chain (Foundry, Agents.AI, Agents.AI.Abstractions), the test
container, the test fixture, Directory.Packages.props, the build script, and this
workflow file. Skip the costly hosted-agent steps when none of those changed.
* Add "Build and push Foundry Hosted Agents test container" step that invokes
scripts/it-build-image.ps1 against vars.IT_HOSTED_AGENT_REGISTRY and pipes the resulting
IT_HOSTED_AGENT_IMAGE=<tag> into GITHUB_ENV.
* Add "Run Foundry Hosted Agents Integration Tests" step that filters in only the new
trait, with AZURE_AI_PROJECT_ENDPOINT/AZURE_AI_MODEL_DEPLOYMENT_NAME pointed at
IT_HOSTED_AGENT_PROJECT_ENDPOINT/IT_HOSTED_AGENT_MODEL_DEPLOYMENT_NAME (Tao project,
East US 2; the SK IT project's region does not yet support hosted agents preview).
* Exclude the new trait from the existing "Run Integration Tests" step.
* TEMP: drop the != 'pull_request' guard on the new steps and on Azure CLI Login when the
paths-filter triggers, so PR #5598 can validate the wiring before promoting to merge
queue only. Restore the original guard after one green PR run.
Build script (scripts/it-build-image.ps1)
* Hash now spans TestContainer source AND its referenced framework projects so any
framework code change forces a fresh tag and a real docker push; the previous
TestContainer-only hash silently reused stale images on framework edits.
Bootstrap script (dotnet/tests/Foundry.Hosting.IntegrationTests/scripts/it-bootstrap-agents.ps1)
* New idempotent script that creates the six stable scenario agents and grants Azure AI
User on the project scope to each agent's MI. Run once per Foundry project. Includes
AAD-graph propagation retries because newly created MIs take time to appear there.
README (dotnet/tests/Foundry.Hosting.IntegrationTests/README.md)
* Document the bootstrap prerequisite, the regional caveat (East US 2 is the only region
we have validated; East US returned "Unsupported region" at the time of writing), the
per-run image rebuild, and the CI wiring including the SP RBAC requirements.
SDK pin (TEMP)
* Bump Microsoft.Agents.AI.Foundry.Hosting's Azure.AI.Projects VersionOverride to
2.1.0-alpha.20260505.1 from the azure-sdk public daily feed (added to nuget.config).
This release is the first that builds the per-agent inference URL as
/agents/{name}/endpoint/protocols/openai (the 2.1.0-beta.1 release builds
.../openai/openai/v1, which the server rejects). Revert both the feed and the override
once the URL fix lands in a stable Azure.AI.Projects release.
* Foundry.Hosting IntegrationTests: revert alpha SDK pin; move endpoint PATCH to bootstrap
The alpha SDK pin (Azure.AI.Projects 2.1.0-alpha.20260505.1 from the azure-sdk public
daily feed) was needed only for the URL routing fix and the strongly-typed
AgentEndpointConfig/PatchAgentOptions wrapper. We do not need either right now: the
fixture stays compatible with the public 2.1.0-beta.1 by moving the one-time endpoint
PATCH to the bootstrap script (it sets version_selector to FixedRatio @latest, so each
new fixture run becomes the served version automatically without a per-run PATCH from
the test code). The hosted-agent invocation path will start working end-to-end once the
URL routing fix lands in a stable Azure.AI.Projects release; until then the tests stay
[Fact(Skip = ...)] as documented.
* Revert dotnet/nuget.config: drop the azure-sdk-for-net public feed.
* Revert Microsoft.Agents.AI.Foundry.Hosting.csproj VersionOverride to 2.1.0-beta.1.
* Revert Microsoft.Agents.AI.Foundry.UnitTests and Microsoft.Agents.AI.Foundry.Hosting.UnitTests
Azure.AI.Projects pin (they had been bumped to align Azure.Core 1.54 transitive).
* Drop the AgentEndpointConfig PATCH block from HostedAgentFixture.cs (the type is
alpha-only). Replace with a comment pointing at the bootstrap script.
* Bootstrap script (it-bootstrap-agents.ps1) now also PATCHes each agent's endpoint
with version_selector=@latest if not already set. Idempotent.
* Foundry.Hosting IntegrationTests: drop accidentally committed filtered.slnx
* Foundry.Hosting IntegrationTests: revert TEMP PR override on Azure CLI Login + IT steps
The previous attempt to validate the new hosted-agent IT wiring on PR #5598 failed
because the PR is from a fork (rogerbarreto/agent-framework-public). GitHub never passes
environment secrets to fork PRs regardless of event-name guards on individual steps,
so 'azure/login@v2' fails with 'client-id and tenant-id are not supplied'. Restore the
original github.event_name != 'pull_request' guard. The new steps will execute on
push to main and on merge_group runs.
* Foundry.Hosting IntegrationTests: invoke build-and-push script with absolute path
The pwsh shell on the GitHub Actions runner couldn't resolve ./scripts/it-build-image.ps1
when the step had no working-directory set; the step inherits the runner's PWD which is
not always the repo root after preceding steps. Use github.workspace explicitly to remove
the ambiguity.
* Foundry.Hosting IntegrationTests: move it-build-image.ps1 inside the IT project tree
The previous location at scripts/it-build-image.ps1 lived outside the sparse-checkout
paths the workflow uses (.github, dotnet, python, declarative-agents), so the runner
never had the file when the new step tried to invoke it. Move the script next to its
sibling it-bootstrap-agents.ps1 inside the IT project tree, and anchor its relative
paths to the repo root via so callers can invoke it from any PWD.
* Move scripts/it-build-image.ps1 -> dotnet/tests/Foundry.Hosting.IntegrationTests/scripts/it-build-image.ps1
* Add Push-Location to the resolved repo root inside the script (Pop-Location in finally)
so the existing relative paths (TestContainerProject, hashed src dirs) keep working
no matter where the script is invoked from.
* Update the workflow path filter and the step's invocation path to the new location.
* Foundry.Hosting IntegrationTests: enable 5 HappyPath tests on the live Foundry endpoint
The fixture already constructs ProjectOpenAIClient via the per-agent path that beta.1
supports (new ProjectOpenAIClient(uri, cred, opts { AgentName })), so no SDK pin bump
is required to run the smoke tests end-to-end. Un-skip the 5 tests that pass against
the live test container.
Tests un-skipped (verified passing locally against tao-foundry-prj):
* RunAsync_ReturnsNonEmptyTextAsync
* RunStreamingAsync_YieldsAtLeastOneUpdateAsync
* MultiTurn_WithPreviousResponseId_PreservesContextAsync
* StoredFalse_Baseline_DoesNotPersistResponseAsync
* Instructions_FromContainerDefinition_AreObeyedAsync
Tests still skipped with a more specific reason (4 of 9 in HappyPath plus all
ToolCalling*, McpToolbox, Toolbox, CustomStorage) because the test container does not
yet emit usable response_id / conversation_id chains, and the placeholder scenarios are
not implemented in the test container's Program.cs. These are test container limitations,
not infra bugs, and can be un-skipped as the container surfaces stabilize.
* Foundry.Hosting IntegrationTests: extract hosted IT into parallel job, add Workflows dep
Address Wesley's review feedback on PR #5598:
1. Pull Foundry hosted-agent IT into its own dotnet-foundry-hosted-it job that runs in parallel to dotnet-build and dotnet-test. Same path-filter gate keeps it skipped on unrelated edits. Builds only the filtered solution containing Foundry.Hosting.IntegrationTests and src deps. dotnet-build-and-test-check now waits on it too.
2. Add Microsoft.Agents.AI.Workflows to the foundryHosting paths-filter and to hashedDirs in it-build-image.ps1 since Foundry.Hosting transitively depends on it.
TFM constraint on the IT csproj stays at net10.0 because AgentConformance.IntegrationTests targets net10/net472 and is consumed by ~12 other IT projects on net472.
---------
Co-authored-by: Roger Barreto <rbarreto@microsoft.com>
* .NET: Upgrade to XUnit 3 and Microsoft Testing Platform (#4176)
* Fix copilot studio integration tests failure (#4209)
* Fix anthropic integration tests and skip reason (#4211)
* Remove accidental add of code coverage for integration tests (#4219)
* Add solution filtered parallel test run (#4226)
* Fix build paths (#4228)
* Fix coverage settings path and trait filter (#4229)
* Add project name filter to solution (#4231)
* Increase Integration Test Parallelism (#4241)
* Increase integration tests threads to 4x (#4242)
* Separate build and test into parallel jobs (#4243)
* Filter src by framework for tests build (#4244)
* Separate build and test into parallel jobs
* Filter source projects by framework for tests build
* Pre-build samples via tests to avoid timeouts (#4245)
* Separate build from run for console sample validation (#4251)
* Address PR comments (#4255)
* Merge and move scripts (#4308)
* .NET: Add Microsoft Fabric sample #3674 (#4230)
Co-authored-by: Chris <66376200+crickman@users.noreply.github.com>
* Python: Phase 2: Embedding clients for Ollama, Bedrock, and Azure AI Inference (#4207)
* Phase 2: Embedding clients for Ollama, Bedrock, and Azure AI Inference
Add embedding client implementations to existing provider packages:
- OllamaEmbeddingClient: Text embeddings via Ollama's embed API
- BedrockEmbeddingClient: Text embeddings via Amazon Titan on Bedrock
- AzureAIInferenceEmbeddingClient: Text and image embeddings via Azure AI
Inference, supporting Content | str input with separate model IDs for
text (AZURE_AI_INFERENCE_EMBEDDING_MODEL_ID) and image
(AZURE_AI_INFERENCE_IMAGE_EMBEDDING_MODEL_ID) endpoints
Additional changes:
- Rename EmbeddingCoT -> EmbeddingT, EmbeddingOptionsCoT -> EmbeddingOptionsT
- Add otel_provider_name passthrough to all embedding clients
- Register integration pytest marker in all packages
- Add lazy-loading namespace exports for Ollama and Bedrock embeddings
- Add image embedding sample using Cohere-embed-v3-english
- Add azure-ai-inference dependency to azure-ai package
Part of #1188
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix mypy duplicate name and ruff lint issues
- Rename second 'vector' variable to 'img_vector' in image embedding loop
- Combine nested with statements in tests
- Remove unused result assignments in tests
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* updates from feedback
* Fix CI failures in embedding usage handling
- Fix Azure AI embedding mypy issues by normalizing vectors to list[float],
safely accumulating optional usage token fields, and filtering None entries
before constructing GeneratedEmbeddings
- Avoid Bandit false positive by initializing usage details as an empty dict
- Update OpenAI embedding tests to assert canonical usage keys
(input_token_count/total_token_count)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* [Purview] Mark responses as responses and fix epoch bug for python long overflow (#4225)
* .NET: Support InvokeMcpTool for declarative workflows (#4204)
* Initial implementation of InvokeMcpTool in declarative workflow
* Cleaned up sample implementation
* Updated sample comments.
* Added missing executor routing attribute
* Fix PR comments.
* Updated based on PR comments.
* Updated based on PR comments.
* Removed unnecessary using statement.
* Update Python package versions to rc2 (#4258)
- Bump core and azure-ai to 1.0.0rc2
- Bump preview packages to 1.0.0b260225
- Update dependencies to >=1.0.0rc2
- Add CHANGELOG entries for changes since rc1
- Update uv.lock
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* .NET: Fixing issue where OpenTelemetry span is never exported in .NET in-process workflow execution (#4196)
* 1. Add reproduction test for issue #4155: workflow.run Activity never stopped in streaming OffThread path
The WorkflowRunActivity_IsStopped_Streaming_OffThread test demonstrates that
the workflow.run OpenTelemetry Activity created in StreamingRunEventStream.RunLoopAsync
is started but never stopped when using the OffThread/Default streaming execution.
The background run loop keeps running after event consumption completes, so the
using Activity? declaration never disposes until explicit StopAsync() is called.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2. Fix workflow.run Activity never stopped in streaming OffThread execution (#4155)
The workflow.run OpenTelemetry Activity in StreamingRunEventStream.RunLoopAsync
was scoped to the method lifetime via 'using'. Since the run loop only exits on
cancellation, the Activity was never stopped/exported until explicit disposal.
Fix: Remove 'using' and explicitly dispose the Activity when the workflow reaches
Idle status (all supersteps complete). A safety-net disposal in the finally block
handles cancellation and error paths.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add root-level workflow.session activity spanning run loop lifetime\n\nImplements two-level telemetry hierarchy per PR feedback from lokitoth:\n- workflow.session: spans the entire run loop / stream lifetime\n- workflow_invoke: per input-to-halt cycle, nested within the session\n\nThis ensures the session activity stays open across multiple turns,\nwhile individual run activities are created and disposed per cycle.\n\nAlso fixes linkedSource CancellationTokenSource disposal leak in\nStreamingRunEventStream (added using declaration)."
* Address Copilot review: fix Activity/CTS disposal, rename activity, add error tag\n\n1. LockstepRunEventStream: Remove 'using' from Activity in async iterator\n and manually dispose in finally block (fixes#4155 pattern). Also dispose\n linkedSource CTS in finally to prevent leak.\n2. Tags.cs: Add ErrorMessage (\"error.message\") tag for runtime errors,\n distinct from BuildErrorMessage (\"build.error.message\").\n3. ActivityNames: Rename WorkflowRun from \"workflow_invoke\" to \"workflow.run\"\n for cross-language consistency.\n4. WorkflowTelemetryContext: Fix XML doc to say \"outer/parent span\" instead\n of \"root-level span\".\n5. ObservabilityTests: Assert WorkflowSession absence when DisableWorkflowRun\n is true.\n6. WorkflowRunActivityStopTests: Fix streaming test race by disposing\n StreamingRun before asserting activities are stopped.\n7. StreamingRunEventStream/LockstepRunEventStream: Use Tags.ErrorMessage\n instead of Tags.BuildErrorMessage for runtime error events."
* Review fixes: revert workflow_invoke rename, use 'using' for linkedSource, move SessionStarted earlier\n\n- Revert ActivityNames.WorkflowRun back to \"workflow_invoke\" (OTEL semantic convention contract)\n- Use 'using' declaration for linkedSource CTS in LockstepRunEventStream (no timing sensitivity)\n- Move SessionStarted event before WaitForInputAsync in StreamingRunEventStream to match Lockstep behavior"
* Improve naming and comments in WorkflowRunActivityStopTests"
* Prevent session Activity.Current leak in lockstep mode, add nesting test
Save and restore Activity.Current in LockstepRunEventStream.Start() so the
session activity doesn't leak into caller code via AsyncLocal. Re-establish
Activity.Current = sessionActivity before creating the run activity in
TakeEventStreamAsync to preserve parent-child nesting.
Add test verifying app activities after RunAsync are not parented under the
session, and that the workflow_invoke activity nests under the session."
* Fix stale XML doc: WorkflowRun -> WorkflowInvoke in ObservabilityTests
---------
Co-authored-by: alliscode <bentho@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python / .NET Samples - Restructure and Improve Samples (Feature Branc… (#4092)
* Python: .NET Samples - Restructure and Improve Samples (Feature Branch) (#4091)
* Moved by agent (#4094)
* Fix readme links
* .NET Samples - Create `04-hosting` learning path step (#4098)
* Agent move
* Agent reorderd
* Remove A2A section from README
Removed A2A section from the Getting Started README.
* Agent fixed links
* Fix broken sample links in durable-agents README (#4101)
* Initial plan
* Fix broken internal links in documentation
Co-authored-by: crickman <66376200+crickman@users.noreply.github.com>
* Revert template link changes; keep only durable-agents README fix
Co-authored-by: crickman <66376200+crickman@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: crickman <66376200+crickman@users.noreply.github.com>
* .NET Samples - Create `03-workflows` learning path step (#4102)
* Fix solution project path
* Python: Fix broken markdown links to repo resources (outside /docs) (#4105)
* Initial plan
* Fix broken markdown links to repo resources
Co-authored-by: crickman <66376200+crickman@users.noreply.github.com>
* Update README to rename .NET Workflows Samples section
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: crickman <66376200+crickman@users.noreply.github.com>
* .NET Samples - Create `02-agents` learning path step (#4107)
* .NET: Fix broken relative link in GroupChatToolApproval README (#4108)
* Initial plan
* Fix broken link in GroupChatToolApproval README
Co-authored-by: crickman <66376200+crickman@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: crickman <66376200+crickman@users.noreply.github.com>
* Update labeler configuration for workflow samples
* .NET - Reorder Agents samples to start from Step01 instead of Step04 (#4110)
* Fix solution
* Resolve new sample paths
* Move new AgentSkills and AgentWithMemory_Step04 samples
* Fix link
* Fix readme path
* fix: update stale dotnet/samples/Durable path reference in AGENTS.md
Co-authored-by: crickman <66376200+crickman@users.noreply.github.com>
* Moved new sample
* Update solution
* Resolve merge (new sample)
* Sync to new sample - FoundryAgents_Step21_BingCustomSearch
* Updated README
* .NET Samples - Configuration Naming Update (#4149)
* .NET: Restore AzureFunctions index parity with ConsoleApps under DurableAgents samples (#4221)
* Clean-up `05_host_your_agent`
* Config setting consistency
* Refine samples
* AGENTS.md
* Move new samples
* Re-order samples
* Move new project and fixup solution
* Fixup model config
* Fix up new UT project
---------
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
* Python: Fix Bedrock embedding test stub missing meta attribute (#4287)
* Fix Bedrock embedding test stub missing meta attribute
* Increase test coverage so gate passes
* Python: (ag-ui): fix approval payloads being re-processed on subsequent conversation turns (#4232)
* Fix ag-ui tool call issue
* Safe json fix
* Python: Update workflow orchestration samples to use AzureOpenAIResponsesClient (#4285)
* Update workflow orchestration samples to use AzureOpenAIResponsesClient
* Fix broken link
* Move scripts to scripts folder
---------
Co-authored-by: Roger Barreto <19890735+rogerbarreto@users.noreply.github.com>
Co-authored-by: Chris <66376200+crickman@users.noreply.github.com>
Co-authored-by: Eduard van Valkenburg <eavanvalkenburg@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Rishabh Chawla <rishabhchawla1995@gmail.com>
Co-authored-by: Peter Ibekwe <109177538+peibekwe@users.noreply.github.com>
Co-authored-by: Dmytro Struk <13853051+dmytrostruk@users.noreply.github.com>
Co-authored-by: Ben Thomas <ben.thomas@microsoft.com>
Co-authored-by: alliscode <bentho@microsoft.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Evan Mattson <35585003+moonbox3@users.noreply.github.com>
* Fix encoding (#4309)
* Disable Parallelization for WorkflowRunActivityStopTests (#4313)
* Revert parallel disable (#4324)
* .NET: Disable flakey Workflow Observability tests (#4416)
* Disable flakey OffThread test
* Disable additional OffThread test
* Disable a further test
* Disable all observability tests
---------
Co-authored-by: Roger Barreto <19890735+rogerbarreto@users.noreply.github.com>
Co-authored-by: Chris <66376200+crickman@users.noreply.github.com>
Co-authored-by: Eduard van Valkenburg <eavanvalkenburg@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Rishabh Chawla <rishabhchawla1995@gmail.com>
Co-authored-by: Peter Ibekwe <109177538+peibekwe@users.noreply.github.com>
Co-authored-by: Dmytro Struk <13853051+dmytrostruk@users.noreply.github.com>
Co-authored-by: Ben Thomas <ben.thomas@microsoft.com>
Co-authored-by: alliscode <bentho@microsoft.com>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Evan Mattson <35585003+moonbox3@users.noreply.github.com>
* Updated merge test permissions
* Removed repo check
* Added fetch from main for comparison
* Updated path detection logic
* Small updates
* Reverted file rename
* Created dedicated workflows for integration tests
* Small fix for Python
* Small fixes
* Small update
* Small update
* Added tests check for Python
* Cosmos DB UT Fast Skip (Non-Configured Local envs) + Long running UT skip in pipeline when no CosmosDB changes happened
* Force a CosmosDB source code change to trigger the pipeline
* Address possible string boolean mismatch
* Add debug
* Enabling emulator always when running IT
* draft commit
* Added Cosmos agent thread and tests
* revert unnecessary changes and fix tests
* add multi-tenant support with hierarchical partition keys (and tests).
* enhance transactional batch
* address review comments
* Address PR review comments from @westey-m
* Merge upstream/main - resolve slnx conflicts
* use param validation helpers
* Replace useManagedIdentity boolean with TokenCredential parameter
* Remove redundant suppressions and fix tests
* Rename project from Microsoft.Agents.AI.Abstractions.CosmosNoSql to Microsoft.Agents.AI.CosmosNoSql
* Refactor constructors to use chaining pattern
* Reorder deserialization constructor parameters for consistency
* Remove database/container IDs from serialized state
* Remove auto-generation of MessageId
* Optimize AddMessagesAsync to avoid enumeration when possible
* Add MaxMessagesToRetrieve to limit context window
* Make Role nullable instead of defaulting
* Fix net472 build without rebasing 19 commits
* Add Cosmos DB emulator to CI workflow
* Fix Cosmos DB emulator tests: use Skip.If instead of Assert.Fail and start emulator before unit tests
* Replace Skip.If() with conditional return to fix compilation
* Use env var to skip Cosmos tests on non-Windows CI
* Add Xunit.SkippableFact package to properly skip Cosmos tests on Linux
* Change [Fact] to [SkippableFact] for proper test skipping behavior
* Remove stale Microsoft.Agents.AI.Abstractions.CosmosNoSql directory
* Fix code formatting: add braces, this. qualifications, and final newlines
* Fix file encoding to UTF-8 with BOM, fix import ordering, and remove unnecessary using directives
* Convert backing fields to auto-properties and remove Azure.Identity using directive
* Fix CosmosChatMessageStore.cs encoding back to UTF-8 with BOM
* Fix test file formatting: indentation, encoding, imports, this. qualifications, naming conventions, and simplify new expressions
* Fix const field naming violations: Remove s_ prefix from const fields and add this. qualification to Dispose call
* Add local .editorconfig for Cosmos DB tests to suppress IDE0005 false positives from multi-targeting
* Fix IDE1006 naming violations: Rename TestDatabaseId to s_testDatabaseId and add final newlines
* Address PR review comments
Address Wesley's review comments:
- Remove Cosmos DB package references from core projects
- Delete duplicate test files from old package structure
- Remove redundant parameter validation from extension methods
Address Kiran's review comments:
- Remove redundant 429 retry logic (SDK handles automatically)
- Add explicit RequestEntityTooLarge error handling
- Remove dead code in GetMessageCountAsync
- Add defensive partition key validation comments
* Fix IDE0001 formatting error in AgentProviderExtensions.cs. Use type alias to resolve namespace conflict between Azure.AI.Agents.Persistent.RunStatus and Microsoft.Agents.AI.Workflows.RunStatus. This eliminates the need for global:: qualifier which triggered the formatter warning.
* Update package versions for Aspire 13.0.0 compatibility
* Fix TargetFrameworks in Cosmos DB projects
- Replace with which is defined in Directory.Build.props
- Fix package reference from System.Linq.Async to System.Linq.AsyncEnumerable to match Directory.Packages.props
* Remove redundant counter, add partition key validation, use factory pattern for deserialization
* Upgrade to .NET 10
- Require .NET 10 SDK
- Include net10.0 assets in all assemblies
- Move net9.0-only targets to net10.0
- Update LangVersion to latest
- Remove complicated distinctions between debug target TFMs and release target TFMs
- Remove unnecessary package dependencies when built into netcoreapp
- Clean up some ifdefs
- Clean up some analyzer warnings
* Fix CI
* Improve conformance of OpenAI Responses API serving
* Update dotnet/src/Microsoft.Agents.AI.Hosting.OpenAI/Responses/AgentRunResponseExtensions.cs
Co-authored-by: Stephen Toub <stoub@microsoft.com>
* Update dotnet/src/Microsoft.Agents.AI.Hosting.OpenAI/Responses/AgentRunResponseExtensions.cs
Co-authored-by: Stephen Toub <stoub@microsoft.com>
* Sort packages
* Relax adherence where acceptable
* nit
* PromptCacheKey is not obsolete
* format
---------
Co-authored-by: Stephen Toub <stoub@microsoft.com>
* Rename AI Agent packages to use Microsoft.Agents.AI
* Fix for build
* Fix formatting
* Fix formatting
* Ignore in VSTHRD200 in migration samples
* Ignore in VSTHRD200 in migration samples
* Add some missing projects and run format
* Fix build errors
* Address code review feedback
* Fix merge issues
---------
Co-authored-by: Mark Wallace <markwallace@microsoft.com>
Roger Barreto ·
Giles Odigwe ·
Copilot ·
dependabot[bot] ·
westey ·
Chris ·
Dmytro Struk ·
Jacob Alber ·
Theo van Kraay ·
Stephen Toub ·
Reuben Bond ·
Mark Wallace ·