* Add dotnet integration test report to CI
- Add --report-junit flag to dotnet integration test step to generate
JUnit XML alongside TRX, with explicit --results-directory to
centralize output in IntegrationTestResults/
- Upload JUnit XML artifacts from each matrix leg (net10.0/ubuntu,
net472/windows) as dotnet-test-results-{framework}-{os}
- Add dotnet-integration-test-report job that downloads artifacts,
runs the existing aggregate.py script, posts markdown to Job Summary,
and saves trend history via actions/cache
- Refactor aggregate.py to discover JUnit XML files recursively,
supporting both pytest (pytest.xml) and xunit (*.junit.xml) layouts
- Handle provider name derivation for dotnet artifact naming convention
- Fix nodeid collision when same test runs under multiple frameworks
by qualifying keys with provider when collisions are detected
- Improve module extraction for dotnet C# classnames (recognizes
IntegrationTests/UnitTests namespace segments)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* chore: trigger dotnet CI for report validation
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: use .junit extension (not .junit.xml) for xunit v3 output
xUnit v3 generates files with .junit extension, not .junit.xml.
Update upload glob and aggregate.py discovery to match.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: use deterministic provider-qualified keys for dotnet tests
Always prefix dotnet test keys with provider (e.g. net10.0 (ubuntu)::TestName)
to ensure stable, comparable counts across runs regardless of file parse order.
Also show Executed (passed+failed) instead of Total in summary table.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: match Python report summary format (Total, passed/total, etc.)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat: split dotnet report into per-framework tables
Dotnet tests run on multiple frameworks (net10.0, net472). Instead of
one combined table with unstable totals, show separate sections per
framework — each with its own summary row and per-test table. Python
reports retain the original single-table format.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Re-enable 7 flaky dotnet integration tests with increased timeouts
Increase timeouts to reduce timing-related flakiness in LLM-backed
integration tests (issue #4971):
- ExternalClientTests: 60s -> 120s default timeout
- SamplesValidationBase: 60s -> 120s default timeout
- ConsoleAppSamplesValidation: 90s -> 150s for long-running tests
- AzureFunctions SamplesValidation: 2min -> 3min orchestration timeout,
60s -> 90s per-step WaitForConditionAsync timeouts
Remove all Skip=Flaky annotations and unused SkipFlakyTimingTest constants.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Re-skip LLM non-determinism flaky tests, keep timeout fixes
Re-skip SingleAgentOrchestrationHITLSampleValidationAsync and
LongRunningToolsSampleValidationAsync - these fail due to LLM producing
extra review notifications, not timeouts. Updated skip reasons to
accurately describe the root cause. Reverted unnecessary timeout change
on the skipped LongRunningTools test.
The remaining 5 re-enabled tests with timeout increases are stable.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Enable Anthropic integration tests in CI
Replace hardcoded skip with conditional skip pattern (matching
CopilotStudio approach): tests gracefully skip when ANTHROPIC_API_KEY
is missing, and run when present.
Changes:
- AnthropicChatCompletionFixture: try/catch in InitializeAsync with
Assert.Skip on missing config (replaces hardcoded SkipReason)
- AnthropicSkillsIntegrationTests: same pattern per test method
- dotnet-build-and-test.yml: wire up ANTHROPIC_API_KEY,
ANTHROPIC_CHAT_MODEL_NAME, and ANTHROPIC_REASONING_MODEL_NAME
env vars to the integration test step
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix missing System using in AnthropicSkillsIntegrationTests
Add 'using System;' for InvalidOperationException in try/catch blocks.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Skip flaky SingleAgentOrchestrationChainingSampleValidationAsync
LLM non-determinism causes Assert.NotNull failures on orchestration
results. Skip until test logic is hardened against non-deterministic
LLM responses.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Re-enable HITL and LongRunningTools tests with timeout and flexibility fixes
- Remove Skip attribute from SingleAgentOrchestrationHITLSampleValidationAsync
- Remove Skip attribute from LongRunningToolsSampleValidationAsync
- Increase timeout from 120s/90s to 180s to accommodate 2+ LLM round-trips
- Replace rigid 2-cycle assertion with flexible approval logic that handles
extra review cycles from LLM non-determinism
Fixes the two failure modes identified in #4971:
1. Timeout: 120s/90s was insufficient for multiple LLM calls under CI load
2. Extra notifications: Assert.Fail on 3rd+ review cycle was too rigid
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Increase AzureFunctions LongRunningTools test timeouts from 90s to 180s
The LongRunningToolsSampleValidationAsync test in the AzureFunctions integration
tests was failing in CI with TimeoutException at the 'Content published
notification is logged' step. The 90-second timeouts are too tight for CI
environments where LLM calls and orchestration overhead can be slow.
Increased all three WaitForConditionAsync timeouts from 90s to 180s:
- Waiting for human feedback notification
- Waiting for publish notification (the step that was failing)
- Waiting for orchestration completion
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Merge main and fix dotnet report path after flaky_report rename
Merge upstream/main which renamed scripts/flaky_report/ to
scripts/integration_test_report/ (from Python PR #5454). Update the
dotnet-build-and-test workflow to reference the new path.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add RetryFact to DurableTask and AzureFunctions integration tests
These tests interact with LLMs via stdin/stdout (DurableTask) or HTTP
(AzureFunctions) and are inherently non-deterministic. Unlike the Python
side which uses pytest-retry, the dotnet tests had no retry mechanism
and a single transient failure would fail the entire CI run.
Changes:
- Switch [Fact] to [RetryFact(2, 5000)] on all LLM-dependent tests
across ConsoleAppSamplesValidation, ExternalClientTests,
WorkflowConsoleAppSamplesValidation, and AzureFunctions SamplesValidation
- Add re-prompt mechanism to LongRunningToolsSampleValidationAsync:
if the LLM doesn't invoke the tool within 60s, re-send the prompt
(up to 2 retries) instead of burning the full timeout
- Reduce LongRunningTools timeout from 240s to 180s (re-prompt makes
the extra buffer unnecessary)
- Leave simple/deterministic tests as [Fact] (SingleAgent, unit tests)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add persist-credentials: false to Integration Test Report checkout step
Matches the convention used by other checkout steps in this workflow
to avoid leaving GITHUB_TOKEN credentials in the local git config.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* small fixes
* disable anthropic failing tests
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Get Started with Microsoft Agent Framework for Python Developers
Quick Install
We recommend two common installation paths depending on your use case.
1. Development mode
If you are exploring or developing locally, install the entire framework with all sub-packages:
pip install agent-framework
This installs the core and every integration package, making sure that all features are available without additional steps. This is the simplest way to get started.
2. Selective install
If you only need specific integrations, you can install at a more granular level. This keeps dependencies lighter and focuses on what you actually plan to use. Some examples:
# Core only
# includes Azure OpenAI and OpenAI support by default
# also includes workflows and orchestrations
pip install agent-framework-core
# Core + Azure AI Foundry integration
pip install agent-framework-foundry
# Core + Microsoft Copilot Studio integration (preview package)
pip install agent-framework-copilotstudio --pre
# Core + both Microsoft Copilot Studio and Azure AI Foundry integration
pip install --pre agent-framework-copilotstudio agent-framework-foundry
This selective approach is useful when you know which integrations you need, and it is the recommended way to set up lightweight environments. Released packages such as agent-framework, agent-framework-core, and agent-framework-foundry no longer require --pre, while preview connectors such as agent-framework-copilotstudio still do.
Supported Platforms:
- Python: 3.10+
- OS: Windows, macOS, Linux
1. Setup API Keys
Set as environment variables, or create a .env file at your project root:
OPENAI_API_KEY=sk-...
OPENAI_MODEL=...
...
AZURE_OPENAI_API_KEY=...
AZURE_OPENAI_ENDPOINT=...
AZURE_OPENAI_MODEL=...
...
FOUNDRY_PROJECT_ENDPOINT=...
FOUNDRY_MODEL=...
For the generic OpenAI clients (OpenAIChatClient and OpenAIChatCompletionClient), configuration
resolves in this order:
- Explicit Azure inputs such as
credentialorazure_endpoint OPENAI_API_KEY/ explicit OpenAI API-key parameters- Azure environment fallback such as
AZURE_OPENAI_ENDPOINTandAZURE_OPENAI_API_KEY
This means mixed shells default to OpenAI when OPENAI_API_KEY is present. To force Azure routing,
pass an explicit Azure input such as credential=AzureCliCredential().
You can also override environment variables by explicitly passing configuration parameters to the chat client constructor:
from agent_framework.openai import OpenAIChatClient
client = OpenAIChatClient(
api_key='',
azure_endpoint='',
model='',
api_version='',
)
See the following setup guide for more information.
2. Create a Simple Agent
Create agents and invoke them directly:
import asyncio
from agent_framework import Agent
from agent_framework.openai import OpenAIChatClient
async def main():
agent = Agent(
client=OpenAIChatClient(),
instructions="""
1) A robot may not injure a human being...
2) A robot must obey orders given it by human beings...
3) A robot must protect its own existence...
Give me the TLDR in exactly 5 words.
"""
)
result = await agent.run("Summarize the Three Laws of Robotics")
print(result)
asyncio.run(main())
# Output: Protect humans, obey, self-preserve, prioritized.
3. Directly Use Chat Clients (No Agent Required)
You can use the chat client classes directly for advanced workflows:
import asyncio
from agent_framework import Message
from agent_framework.openai import OpenAIChatClient
async def main():
client = OpenAIChatClient()
messages = [
Message("system", ["You are a helpful assistant."]),
Message("user", ["Write a haiku about Agent Framework."])
]
response = await client.get_response(messages)
print(response.messages[0].text)
"""
Output:
Agents work in sync,
Framework threads through each task—
Code sparks collaboration.
"""
asyncio.run(main())
4. Build an Agent with Tools and Functions
Enhance your agent with custom tools and function calling:
import asyncio
from typing import Annotated
from random import randint
from pydantic import Field
from agent_framework import Agent
from agent_framework.openai import OpenAIChatClient
def get_weather(
location: Annotated[str, Field(description="The location to get the weather for.")],
) -> str:
"""Get the weather for a given location."""
conditions = ["sunny", "cloudy", "rainy", "stormy"]
return f"The weather in {location} is {conditions[randint(0, 3)]} with a high of {randint(10, 30)}°C."
def get_menu_specials() -> str:
"""Get today's menu specials."""
return """
Special Soup: Clam Chowder
Special Salad: Cobb Salad
Special Drink: Chai Tea
"""
async def main():
agent = Agent(
client=OpenAIChatClient(),
instructions="You are a helpful assistant that can provide weather and restaurant information.",
tools=[get_weather, get_menu_specials]
)
response = await agent.run("What's the weather in Amsterdam and what are today's specials?")
print(response)
"""
Output:
The weather in Amsterdam is sunny with a high of 22°C. Today's specials include
Clam Chowder soup, Cobb Salad, and Chai Tea as the special drink.
"""
if __name__ == "__main__":
asyncio.run(main())
You can explore additional agent samples here.
5. Multi-Agent Orchestration
Coordinate multiple agents to collaborate on complex tasks using orchestration patterns:
import asyncio
from agent_framework import Agent
from agent_framework.openai import OpenAIChatClient
async def main():
# Create specialized agents
writer = Agent(
client=OpenAIChatClient(),
name="Writer",
instructions="You are a creative content writer. Generate and refine slogans based on feedback."
)
reviewer = Agent(
client=OpenAIChatClient(),
name="Reviewer",
instructions="You are a critical reviewer. Provide detailed feedback on proposed slogans."
)
# Sequential workflow: Writer creates, Reviewer provides feedback
task = "Create a slogan for a new electric SUV that is affordable and fun to drive."
# Step 1: Writer creates initial slogan
initial_result = await writer.run(task)
print(f"Writer: {initial_result}")
# Step 2: Reviewer provides feedback
feedback_request = f"Please review this slogan: {initial_result}"
feedback = await reviewer.run(feedback_request)
print(f"Reviewer: {feedback}")
# Step 3: Writer refines based on feedback
refinement_request = f"Please refine this slogan based on the feedback: {initial_result}\nFeedback: {feedback}"
final_result = await writer.run(refinement_request)
print(f"Final Slogan: {final_result}")
# Example Output:
# Writer: "Charge Forward: Affordable Adventure Awaits!"
# Reviewer: "Good energy, but 'Charge Forward' is overused in EV marketing..."
# Final Slogan: "Power Up Your Adventure: Premium Feel, Smart Price!"
if __name__ == "__main__":
asyncio.run(main())
For more advanced orchestration patterns including Sequential, Concurrent, Group Chat, Handoff, and Magentic orchestrations, see the orchestration samples.
More Examples & Samples
- Getting Started with Agents: Basic agent creation and tool usage
- Chat Client Examples: Direct chat client usage patterns
- Foundry Integration: Microsoft Foundry integration
- Workflow Samples: Advanced multi-agent patterns
Agent Framework Documentation
- Agent Framework Repository
- Python Package Documentation
- .NET Package Documentation
- Design Documents
- Learn docs are coming soon.