agent/agent-framework
1
0
Fork 0
mirror of https://github.com/microsoft/agent-framework.git synced 2026-06-16 21:04:09 +08:00
Code Issues Packages Projects Releases Wiki Activity
Files
540193ccefc39dd96c91da65ec35be40afc1a7b3
agent-framework/python/samples
T
Giles Odigwe 540193ccef Python: Reduce flaky integration tests and improve CI signal quality (#5454) 
* Enable Ollama integration tests in CI and rename report to Integration Test Report

- Install Ollama, cache models (qwen2.5:0.5b + nomic-embed-text), and start
  server in the Misc integration job for both workflow files
- Set OLLAMA_MODEL and OLLAMA_EMBEDDING_MODEL env vars so the 5 Ollama tests
  are no longer skipped
- Rename Flaky Test Report to Integration Test Report throughout (job names,
  artifact names, cache keys, file names, script titles/docstrings)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Bump Ollama model to qwen2.5:1.5b for better instruction following

The 0.5b model was too small to reliably follow simple prompts like
'Say Hello World', causing test assertion failures. The 1.5b model
follows instructions more reliably while still being small enough
for fast CI pulls (~1GB).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Re-enable reliable streaming integration tests

Remove the hard skip on test_03_reliable_streaming tests that was
temporarily disabled for instability investigation. CI infrastructure
(Azurite, DTS emulator, Redis, func CLI) is already in place.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Re-enable skipped Functions/DurableTask tests and bump timeout to 480s

- Remove hard skips from 4 tests in test_11_workflow_parallel.py
- Remove hard skip from test_conditional_branching in test_06_dt_multi_agent_orchestration_conditionals.py
- Increase pytest --timeout from 360 to 480 for Functions+DurableTask CI job
- Updated in both python-merge-tests.yml and python-integration-tests.yml

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Re-skip failing Functions/DurableTask tests with specific root causes

- test_11_workflow_parallel (4 tests): xdist worker crashes during execution
- test_conditional_branching: orchestration fails with RuntimeError, not a timeout
- Keep 480s timeout bump for remaining Functions tests

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix auth routing in samples 06/11: api_key -> credential for Azure OpenAI

Both samples passed a bearer token provider via api_key= which caused the
client to route to api.openai.com instead of Azure OpenAI, resulting in
401 Unauthorized. Changed to credential= which correctly triggers Azure
routing and picks up AZURE_OPENAI_ENDPOINT from the environment.

- samples/azure_functions/11_workflow_parallel/function_app.py: 1 fix
- samples/durabletask/06_multi_agent_orchestration_conditionals/worker.py: 2 fixes
- Re-enable 4 parallel workflow tests and 1 conditional branching test

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Re-skip parallel workflow tests: xdist worker distribution issue

The 4 parallel workflow tests crash because xdist worksteal distributes
them across separate workers, each spawning its own func process against
shared emulators. Auth fix (api_key->credential) was valid and stays.
test_conditional_branching now passes with the auth fix.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix E501 line-too-long in azurefunctions parallel test skip reasons

Wrap skip reason strings to stay within 120 char line limit.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add retry logic and port-conflict fix for Ollama CI setup

- Kill any auto-started Ollama before launching serve (fixes port
  conflict: 'address already in use')
- Retry ollama pull up to 3 times with 15s backoff (fixes 429 rate
  limit failures)
- Applied to both python-merge-tests.yml and python-integration-tests.yml

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix flaky integration tests and re-enable skipped tests

- Foundry agent: add allow_preview=True to custom client test
- Foundry hosting: raise max_output_tokens 50->200, add temperature,
  relax assertion in test_temperature_and_max_tokens
- Foundry embedding: update skip reason with root cause (endpoint mismatch)
- OpenAI file search: fix vector store indexing race condition by polling
  file_counts before querying; fix get_streaming_response -> get_response(stream=True)
- Azure OpenAI file search: remove skip (transient 500 resolved)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Remove temperature from foundry hosting test (unsupported by CI model)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Stabilize Ollama tool call integration tests with no-arg function

Use a no-argument greet() function instead of hello_world(arg1) for
integration tests. The 1.5B model in CI is unreliable at generating
correct tool call arguments, causing 'Argument parsing failed' errors.
A no-arg function eliminates this flakiness entirely.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Increase reliable streaming test timeouts from 30s to 60s

The LLM call through Azure OpenAI + Redis streaming pipeline can exceed
30s in CI due to cold starts or throttling. Raise to 60s to reduce
flaky timeouts while still bounded by pytest's 120s per-test limit.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Re-enable workflow parallel tests with xdist_group marker

The tests were skipped because xdist distributes module tests across
workers, each spawning their own func process (port conflicts). Adding
xdist_group forces all tests in this module onto a single worker so
the module-scoped function_app_for_test fixture works correctly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Revert "Re-enable workflow parallel tests with xdist_group marker"

This reverts commit 455c28da62.

* Rename flaky_report to integration_test_report and add try/finally cleanup

- Rename scripts/flaky_report/ to scripts/integration_test_report/ to
  reflect expanded scope beyond flaky-test detection
- Update workflow references in both CI files
- Wrap file search integration tests in try/finally to ensure vector
  store cleanup runs even on test failure or timeout

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix Ollama pull failure propagation and Azure OpenAI vector store readiness

- Ollama CI: fail the step immediately if model pull fails after 3
  retries instead of silently proceeding to tests
- Azure OpenAI file search: add the same vector-store readiness polling
  that was applied to the non-Azure OpenAI tests, preventing eventual
  consistency race conditions

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* remove load_dotenv from test file

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
540193ccef · 2026-05-01 00:41:39 +00:00
History
..
01-get-started
Python: Add requirements.txt and .env.example to the a2a/ sample for pip-based setup (#5510)
2026-04-27 22:22:07 +00:00
02-agents
Python: update FoundryAgent for hosted agent sessions (#5447)
2026-04-24 09:25:03 +00:00
03-workflows
Python: [BREAKING] Standardize orchestration terminal outputs as AgentResponse (#5301)
2026-04-29 00:35:36 +00:00
04-hosting
Python: Reduce flaky integration tests and improve CI signal quality (#5454)
2026-05-01 00:41:39 +00:00
05-end-to-end
Python: Update package dependencies (#5555)
2026-04-29 06:18:03 +00:00
autogen-migration
Fix handoff workflow context management and improve AG-UI demo (#5136)
2026-04-08 04:08:24 +00:00
semantic-kernel-migration
Fix handoff workflow context management and improve AG-UI demo (#5136)
2026-04-08 04:08:24 +00:00
shared
Python: Phase 2: Embedding clients for Ollama, Bedrock, and Azure AI Inference (#4207)
2026-02-25 17:45:08 +00:00
__init__.py
Python telemetry (#223)
2025-07-28 07:33:42 +00:00
AGENTS.md
Python: [BREAKING] update to v1.0.0 (#5062)
2026-04-02 15:26:30 +00:00
README.md
Python: (core): Add functional workflow API (#4238)
2026-04-24 09:41:20 +00:00
SAMPLE_GUIDELINES.md
Python: [BREAKING] Python: Provider-leading client design & OpenAI package extraction (#4818)
2026-03-25 09:56:29 +00:00

README.md

Python Samples

This directory contains samples demonstrating the capabilities of Microsoft Agent Framework for Python.

Structure

Folder Description
01-get-started/ Progressive tutorial: hello agent → hosting
02-agents/ Deep-dive by concept: tools, middleware, providers, orchestrations
03-workflows/ Workflow patterns: sequential, concurrent, state, declarative
04-hosting/ Deployment: Azure Functions, Durable Tasks, A2A
05-end-to-end/ Full applications, evaluation, demos

Getting Started

Start with 01-get-started/ and work through the numbered files:

  1. 01_hello_agent.py — Create and run your first agent
  2. 02_add_tools.py — Add function tools with @tool
  3. 03_multi_turn.py — Multi-turn conversations with AgentSession
  4. 04_memory.py — Agent memory with ContextProvider
  5. 05_functional_workflow_with_agents.py — Call agents inside a functional workflow
  6. 06_functional_workflow_basics.py — Write a workflow as a plain async function
  7. 07_first_graph_workflow.py — Build a workflow with executors and edges
  8. 08_host_your_agent.py — Host your agent via Azure Functions

Prerequisites

pip install agent-framework

Environment Variables

Samples call load_dotenv() to automatically load environment variables from a .env file in the python/ directory. This is a convenience for local development and testing.

For local development, set up your environment using any of these methods:

Option 1: Using a .env file (recommended for local development):

  1. Copy .env.example to .env in the python/ directory: 
    cp .env.example .env
  2. Edit .env and set your values (API keys, endpoints, etc.)

Option 2: Export environment variables directly:

export FOUNDRY_PROJECT_ENDPOINT="your-foundry-project-endpoint"
export FOUNDRY_MODEL="gpt-4o"

Option 3: Using env_file_path parameter (for per-client configuration):

All client classes (e.g., OpenAIChatClient, OpenAIChatCompletionClient) support an env_file_path parameter to load environment variables from a specific file:

from agent_framework.openai import OpenAIChatClient

# Load from a custom .env file
client = OpenAIChatClient(env_file_path="path/to/custom.env")

This allows different clients to use different configuration files if needed.

For the generic OpenAI clients (OpenAIChatClient and OpenAIChatCompletionClient), routing precedence is:

  1. Explicit Azure inputs such as credential, azure_endpoint, or api_version
  2. OPENAI_API_KEY / explicit OpenAI API-key parameters
  3. Azure environment fallback such as AZURE_OPENAI_ENDPOINT and AZURE_OPENAI_API_KEY

If you keep both OpenAI and Azure variables in your shell, the generic clients stay on OpenAI until you pass an explicit Azure input.

For the getting-started samples, you'll need at minimum:

FOUNDRY_PROJECT_ENDPOINT="your-foundry-project-endpoint"
FOUNDRY_MODEL="gpt-4o"

Consolidated sample env inventory

This is the single source of truth for package-level environment variables read by packages included by agent-framework-core[all]. It intentionally excludes variables that are only read by standalone samples, package sample folders, or tests. When package code adds, removes, or renames an environment variable, update this table in the same change.

Example values below are illustrative. For entries not backed by a single public class, the class column names the closest public surface, helper, or package-level initialization point that reads the variable.

package class env var example value
agent-framework-anthropic AnthropicClient ANTHROPIC_API_KEY sk-ant-api03-...
agent-framework-anthropic AnthropicClient ANTHROPIC_CHAT_MODEL claude-sonnet-4-5-20250929
agent-framework-foundry FoundryEmbeddingClient FOUNDRY_MODELS_ENDPOINT https://my-endpoint.inference.ai.azure.com
agent-framework-foundry FoundryEmbeddingClient FOUNDRY_MODELS_API_KEY env-key
agent-framework-foundry FoundryEmbeddingClient FOUNDRY_EMBEDDING_MODEL text-embedding-3-small
agent-framework-foundry FoundryEmbeddingClient FOUNDRY_IMAGE_EMBEDDING_MODEL Cohere-embed-v3-english
agent-framework-azure-ai-search AzureAISearchContextProvider AZURE_SEARCH_ENDPOINT https://my-search.search.windows.net
agent-framework-azure-ai-search AzureAISearchContextProvider AZURE_SEARCH_API_KEY search-key
agent-framework-azure-ai-search AzureAISearchContextProvider AZURE_SEARCH_INDEX_NAME hotels-index
agent-framework-azure-ai-search AzureAISearchContextProvider AZURE_SEARCH_KNOWLEDGE_BASE_NAME hotels-kb
agent-framework-azure-cosmos CosmosHistoryProvider AZURE_COSMOS_ENDPOINT https://my-cosmos.documents.azure.com:443/
agent-framework-azure-cosmos CosmosHistoryProvider AZURE_COSMOS_DATABASE_NAME agent-history
agent-framework-azure-cosmos CosmosHistoryProvider AZURE_COSMOS_CONTAINER_NAME messages
agent-framework-azure-cosmos CosmosHistoryProvider AZURE_COSMOS_KEY C2F...==
agent-framework-bedrock BedrockChatClient BEDROCK_REGION us-east-1
agent-framework-bedrock BedrockChatClient BEDROCK_CHAT_MODEL anthropic.claude-3-5-sonnet-20241022-v2:0
agent-framework-bedrock BedrockEmbeddingClient BEDROCK_REGION us-east-1
agent-framework-bedrock BedrockEmbeddingClient BEDROCK_EMBEDDING_MODEL amazon.titan-embed-text-v2:0
agent-framework-bedrock BedrockChatClient / BedrockEmbeddingClient AWS_ACCESS_KEY_ID AKIAIOSFODNN7EXAMPLE
agent-framework-bedrock BedrockChatClient / BedrockEmbeddingClient AWS_SECRET_ACCESS_KEY wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
agent-framework-bedrock BedrockChatClient / BedrockEmbeddingClient AWS_SESSION_TOKEN IQoJb3JpZ2luX2VjEO7//////////wEaCXVzLXdlc3QtMiJHMEUCIQD...
agent-framework-copilotstudio CopilotStudioAgent COPILOTSTUDIOAGENT__ENVIRONMENTID 00000000-0000-0000-0000-000000000000
agent-framework-copilotstudio CopilotStudioAgent COPILOTSTUDIOAGENT__SCHEMANAME cr123_agentname
agent-framework-copilotstudio CopilotStudioAgent COPILOTSTUDIOAGENT__TENANTID 11111111-1111-1111-1111-111111111111
agent-framework-copilotstudio CopilotStudioAgent COPILOTSTUDIOAGENT__AGENTAPPID 22222222-2222-2222-2222-222222222222
agent-framework-core enable_instrumentation() ENABLE_INSTRUMENTATION true
agent-framework-core enable_instrumentation() ENABLE_SENSITIVE_DATA false
agent-framework-core enable_instrumentation() ENABLE_CONSOLE_EXPORTERS true
agent-framework-core enable_instrumentation() OTEL_EXPORTER_OTLP_ENDPOINT http://localhost:4317
agent-framework-core enable_instrumentation() OTEL_EXPORTER_OTLP_TRACES_ENDPOINT http://localhost:4318/v1/traces
agent-framework-core enable_instrumentation() OTEL_EXPORTER_OTLP_METRICS_ENDPOINT http://localhost:4318/v1/metrics
agent-framework-core enable_instrumentation() OTEL_EXPORTER_OTLP_LOGS_ENDPOINT http://localhost:4318/v1/logs
agent-framework-core enable_instrumentation() OTEL_EXPORTER_OTLP_PROTOCOL grpc
agent-framework-core enable_instrumentation() OTEL_EXPORTER_OTLP_HEADERS api-key=demo
agent-framework-core enable_instrumentation() OTEL_EXPORTER_OTLP_TRACES_HEADERS api-key=trace-demo
agent-framework-core enable_instrumentation() OTEL_EXPORTER_OTLP_METRICS_HEADERS api-key=metric-demo
agent-framework-core enable_instrumentation() OTEL_EXPORTER_OTLP_LOGS_HEADERS api-key=log-demo
agent-framework-core enable_instrumentation() OTEL_SERVICE_NAME sample-agent
agent-framework-core enable_instrumentation() OTEL_SERVICE_VERSION 1.0.0
agent-framework-core enable_instrumentation() OTEL_RESOURCE_ATTRIBUTES deployment.environment=dev,service.namespace=agent-framework
agent-framework-devui DevUI server DEVUI_AUTH_TOKEN my-devui-token
agent-framework-foundry FoundryChatClient FOUNDRY_PROJECT_ENDPOINT https://my-project.services.ai.azure.com/api/projects/my-project
agent-framework-foundry FoundryChatClient FOUNDRY_MODEL gpt-4o
agent-framework-foundry FoundryAgent FOUNDRY_AGENT_NAME travel-planner
agent-framework-foundry FoundryAgent FOUNDRY_AGENT_VERSION v1
agent-framework-github-copilot GitHubCopilotAgent GITHUB_COPILOT_CLI_PATH copilot
agent-framework-github-copilot GitHubCopilotAgent GITHUB_COPILOT_MODEL gpt-5
agent-framework-github-copilot GitHubCopilotAgent GITHUB_COPILOT_TIMEOUT 60
agent-framework-github-copilot GitHubCopilotAgent GITHUB_COPILOT_LOG_LEVEL info
agent-framework-mem0 agent_framework_mem0 package import MEM0_TELEMETRY false
agent-framework-ollama OllamaChatClient OLLAMA_HOST http://localhost:11434
agent-framework-ollama OllamaChatClient OLLAMA_MODEL llama3.1:8b
agent-framework-openai OpenAIChatClient / OpenAIChatCompletionClient / OpenAIEmbeddingClient OPENAI_API_KEY sk-proj-...
agent-framework-openai OpenAIChatClient / OpenAIChatCompletionClient / OpenAIEmbeddingClient OPENAI_MODEL gpt-4o-mini
agent-framework-openai OpenAIChatClient OPENAI_CHAT_MODEL gpt-4.1-mini
agent-framework-openai OpenAIChatCompletionClient OPENAI_CHAT_COMPLETION_MODEL gpt-4o
agent-framework-openai OpenAIEmbeddingClient OPENAI_EMBEDDING_MODEL text-embedding-3-small
agent-framework-openai OpenAIChatClient / OpenAIChatCompletionClient / OpenAIEmbeddingClient OPENAI_BASE_URL https://api.openai.com/v1/
agent-framework-openai OpenAIChatClient / OpenAIChatCompletionClient / OpenAIEmbeddingClient OPENAI_ORG_ID org_123456789
agent-framework-openai OpenAIChatClient / OpenAIChatCompletionClient / OpenAIEmbeddingClient AZURE_OPENAI_ENDPOINT https://my-resource.openai.azure.com/
agent-framework-openai OpenAIChatClient / OpenAIChatCompletionClient / OpenAIEmbeddingClient AZURE_OPENAI_API_KEY sk-azure-...
agent-framework-openai OpenAIChatClient / OpenAIChatCompletionClient / OpenAIEmbeddingClient AZURE_OPENAI_API_VERSION 2024-10-21
agent-framework-openai OpenAIChatClient / OpenAIChatCompletionClient / OpenAIEmbeddingClient AZURE_OPENAI_BASE_URL https://my-resource.openai.azure.com/openai/v1/
agent-framework-openai OpenAIChatClient / OpenAIChatCompletionClient / OpenAIEmbeddingClient AZURE_OPENAI_MODEL gpt-4o
agent-framework-openai OpenAIChatClient AZURE_OPENAI_CHAT_MODEL gpt-4.1
agent-framework-openai OpenAIChatCompletionClient AZURE_OPENAI_CHAT_COMPLETION_MODEL gpt-4o-mini
agent-framework-openai OpenAIEmbeddingClient AZURE_OPENAI_EMBEDDING_MODEL text-embedding-3-large
agent-framework-openai OpenAIChatClient / OpenAIChatCompletionClient / OpenAIEmbeddingClient AZURE_OPENAI_RESOURCE_URL https://cognitiveservices.azure.com/

agent-framework-openai supports the Azure OpenAI client-specific deployment aliases listed above; keep packages/openai/README.md as the authoritative reference for the exact fallback order and package-specific behavior.

Note for production: In production environments, set environment variables through your deployment platform (e.g., Azure App Settings, Kubernetes ConfigMaps/Secrets) rather than using .env files. The load_dotenv() call in samples will have no effect when a .env file is not present, allowing environment variables to be loaded from the system.

For Azure authentication, run az login before running samples.

Note on XML tags

Some sample files include XML-style snippet tags (for example <snippet_name> and </snippet_name>). These are used by our documentation tooling and can be ignored or removed when you use the samples outside this repository.

Additional Resources

Reference in New Issue View Git Blame Copy Permalink