agent-framework/dotnet/tests/Foundry.Hosting.IntegrationTests

Foundry.Hosting.IntegrationTests

Integration tests for Microsoft.Agents.AI.Foundry.Hosting against real Foundry hosted agents.

How it works

Each test class is bound to a scenario fixture (e.g. HappyPathHostedAgentFixture, ToolCallingHostedAgentFixture). On InitializeAsync the fixture:

1. Reads AZURE_AI_PROJECT_ENDPOINT and IT_HOSTED_AGENT_IMAGE from the environment.
2. Targets a stable, scenario keyed agent name (e.g. it-happy-path). The agent is provisioned out of band by scripts/it-bootstrap-agents.ps1; tests only manage versions.
3. Calls AgentAdministrationClient.CreateAgentVersionAsync with a HostedAgentDefinition that points at the image, sets IT_SCENARIO=<scenario> in the container env vars, and adds a per-run IT_RUN_ID so each run gets a fresh content-addressed version (Foundry deduplicates versions by definition hash).
4. Polls until the agent reports AgentVersionStatus.Active (timeout: 5 minutes).
5. Patches the agent endpoint with AgentEndpointConfig (Responses protocol, version selector pointing 100% at the new version).
6. Builds a per-agent ProjectOpenAIClient with AgentName set on the options (this selects the /agents/{name}/endpoint/protocols/openai URL suffix; the cached projectClient.ProjectOpenAIClient cannot serve a hosted agent), wraps the ProjectResponsesClient as an AIAgent, and exposes it via Agent.

On DisposeAsync only the version created by this fixture is deleted. The agent itself is intentionally never deleted, because its managed identity must hold the pre-granted Azure AI User role on the project scope for inbound inference to succeed.

The container image is the same for every scenario. The IT_SCENARIO env var, set on the agent definition by each fixture, drives a switch in the test container's Program.cs to wire up the scenario specific behavior (tools, toolbox, custom storage, etc.).

Required environment variables

Variable	Source	Purpose
AZURE_AI_PROJECT_ENDPOINT	Foundry project	Where to provision the agent. Must be in a region that has the Hosted Agents preview enabled (e.g. East US 2).
AZURE_AI_MODEL_DEPLOYMENT_NAME	Foundry project	Model the agent uses. Defaults to gpt-4o inside the container.
IT_HOSTED_AGENT_IMAGE	scripts/it-build-image.ps1	ACR image reference the agent points at.

One-time bootstrap (per Foundry project)

Hosted agent invocation requires the agent's own managed identity to hold the Azure AI User role on the project scope. Because each agent's MI is created when the agent is first provisioned (and recycled on agent delete), the bootstrap creates the six stable scenario agents once and grants the role to each MI. The fixture then only manages versions under those existing agents, so the role grants survive across runs.

./scripts/it-bootstrap-agents.ps1 `
    -ProjectEndpoint "https://<account>.services.ai.azure.com/api/projects/<project>" `
    -Image "<acr>.azurecr.io/foundry-hosting-it:<tag>"

The script is idempotent. It requires Owner or User Access Administrator on the project scope (RBAC writes). Wait ~3 minutes after first-time grants for AAD propagation before running the tests.

Building and pushing the test container image

The test container source lives at dotnet/tests/Foundry.Hosting.IntegrationTests.TestContainer. Build and push it with:

$env:IT_REGISTRY = "<your-acr>.azurecr.io"
$env:IT_HOSTED_AGENT_IMAGE = (./scripts/it-build-image.ps1 -Registry $env:IT_REGISTRY | Select-String IT_HOSTED_AGENT_IMAGE).Line.Split('=', 2)[1]

The script tags the image by content hash of the test container source. If you didn't change anything since the last build, the push is a no op.

The Foundry project's account MI and project MI both need AcrPull on the registry.

Running the tests locally

$env:AZURE_AI_PROJECT_ENDPOINT = "https://<your-account>.services.ai.azure.com/api/projects/<your-project>"
$env:AZURE_AI_MODEL_DEPLOYMENT_NAME = "gpt-4o"
# IT_HOSTED_AGENT_IMAGE was set above.

dotnet test dotnet/tests/Foundry.Hosting.IntegrationTests/Foundry.Hosting.IntegrationTests.csproj

Note: all tests are currently tagged [Fact(Skip = ...)] until end to end smoke verification has run against a live Foundry deployment. Once a scenario has been exercised and the assertions stabilized, remove the Skip annotation on its tests.

All test classes carry [Trait("Category", "FoundryHostedAgents")] so the CI workflow can route them to a separate Foundry project than the rest of the integration tests (see .github/workflows/dotnet-build-and-test.yml).

CI wiring

The main "Run Integration Tests" step excludes this category. Two extra steps run only on ubuntu-latest for this category, gated on paths-filter.outputs.foundryHostingChanges so they execute only when the project under test, its dependency chain, the test container, the test fixture, or their tooling changed:

1. Build and push Foundry Hosted Agents test container invokes scripts/it-build-image.ps1 against vars.IT_HOSTED_AGENT_REGISTRY. The image is rebuilt every IT run; its tag is content-hashed across the test container source AND its referenced framework projects (Microsoft.Agents.AI.Foundry.Hosting, Microsoft.Agents.AI.Foundry, Microsoft.Agents.AI, Microsoft.Agents.AI.Abstractions), so unchanged content is a docker push no-op while any framework code change forces a fresh image. The script pipes its IT_HOSTED_AGENT_IMAGE=<tag> line into $GITHUB_ENV for the next step.

2. Run Foundry Hosted Agents Integration Tests executes only --filter-trait "Category=FoundryHostedAgents" with the env vars below mapped onto the names the fixture reads. IT_HOSTED_AGENT_IMAGE is the value just exported by step 1.

GitHub env var	Mapped to
IT_HOSTED_AGENT_PROJECT_ENDPOINT	AZURE_AI_PROJECT_ENDPOINT
IT_HOSTED_AGENT_MODEL_DEPLOYMENT_NAME	AZURE_AI_MODEL_DEPLOYMENT_NAME
IT_HOSTED_AGENT_REGISTRY	(consumed by it-build-image.ps1; not passed to tests)

Like all integration tests in this workflow, the steps run only on push and merge-queue events, never on plain pull_request. The path-filter list lives in the paths-filter job in .github/workflows/dotnet-build-and-test.yml under filters.foundryHosting and must stay in sync with $hashedDirs in scripts/it-build-image.ps1.

The CI service principal that backs secrets.AZURE_CLIENT_ID needs:

• Azure AI User on the hosted-agents Foundry project (to add/delete agent versions).
• AcrPush on the registry referenced by IT_HOSTED_AGENT_REGISTRY (to push the image).

The bootstrap script (and one-time AcrPull grants for the Foundry project's MIs) is a human-only operation; CI only adds and deletes versions under existing agents.

Scenarios

Fixture	IT_SCENARIO	Agent name	What it tests
HappyPathHostedAgentFixture	happy-path	it-happy-path	Round trip, streaming, multi turn (previous_response_id and conversation_id), stored=false flag in three combinations, instructions obeyed.
ToolCallingHostedAgentFixture	tool-calling	it-tool-calling	Server side AIFunction invocation; arguments; multi turn referencing prior tool result.
ToolCallingApprovalHostedAgentFixture	tool-calling-approval	it-tool-calling-approval	Approval requests raised, approved, denied.
ToolboxHostedAgentFixture	toolbox	it-toolbox	Server registered toolbox tool callable; client side additions visible (placeholder).
McpToolboxHostedAgentFixture	mcp-toolbox	it-mcp-toolbox	MCP backed tool invocation against https://learn.microsoft.com/api/mcp (placeholder).
CustomStorageHostedAgentFixture	custom-storage	it-custom-storage	Round trip with custom IResponsesStorageProvider; multi turn reads from the custom store (placeholder).

The placeholder scenarios will be wired up in the test container Program.cs once the relevant Microsoft.Agents.AI.Foundry.Hosting API surfaces stabilize.