* feat(python): Add embedding abstractions and OpenAI implementation (Phase 1) This PR contains two parts: 1. **Overall migration plan** for porting vector stores and embeddings from Semantic Kernel to Agent Framework (docs/features/vector-stores-and-embeddings/README.md) covering all 10 phases from core abstractions through connectors and TextSearch. 2. **Phase 1 implementation** — core embedding abstractions and OpenAI/Azure OpenAI embedding clients: Core types (_types.py): - EmbeddingGenerationOptions TypedDict (total=False) - Embedding[EmbeddingT] generic class with model_id, dimensions, created_at - GeneratedEmbeddings[EmbeddingT, EmbeddingOptionsT] list container with options, usage - EmbeddingInputT (default str) and EmbeddingT (default list[float]) TypeVars Protocol + base class (_clients.py): - SupportsGetEmbeddings protocol — Generic[EmbeddingInputT, EmbeddingT, OptionsContraT] - BaseEmbeddingClient ABC — Generic[EmbeddingInputT, EmbeddingT, OptionsCoT] Telemetry (observability.py): - EmbeddingTelemetryLayer with gen_ai.operation.name = "embeddings" OpenAI implementation (openai/_embedding_client.py): - RawOpenAIEmbeddingClient, OpenAIEmbeddingClient, OpenAIEmbeddingOptions - Uses _ensure_client() factory pattern Azure OpenAI implementation (azure/_embedding_client.py): - AzureOpenAIEmbeddingClient following AzureOpenAIChatClient pattern - Supports API key, Entra ID credentials, env var configuration Tests: - 47 unit tests for types, protocol, base class, OpenAI, and Azure clients - 6 integration tests (gated behind RUN_INTEGRATION_TESTS + credentials) Samples: - samples/02-agents/embeddings/openai_embeddings.py - samples/02-agents/embeddings/azure_openai_embeddings.py Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: Add AzureOpenAIEmbeddingClient to azure __init__.pyi stub Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * ci: Add embedding env vars to Python integration tests Map OPENAI_EMBEDDING_MODEL_ID and AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME from GitHub vars to the integration test environment. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: Handle base64 encoding_format in OpenAI embedding client When encoding_format='base64' is used, the OpenAI API returns base64-encoded floats instead of a JSON array. Decode these automatically to list[float] so the return type stays consistent regardless of encoding format. Also adds a unit test for base64 decoding and fixes minor docstring/import issues. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: Only record INPUT_TOKENS for embedding telemetry Embeddings have no output/completion tokens. Remove OUTPUT_TOKENS recording which was double-counting prompt_tokens via the total_tokens fallback. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: Resolve mypy variance error and lint warning Use contravariant/covariant TypeVars for SupportsGetEmbeddings Protocol. Combine nested if into single statement in telemetry layer. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: Make EmbeddingCoT invariant for mypy compatibility GeneratedEmbeddings is invariant in its type param, so the Protocol TypeVar cannot be covariant. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: Address PR review - empty values guard, service_url for telemetry - Add early return for empty values in get_embeddings to avoid unnecessary API calls - Add service_url() method to RawOpenAIEmbeddingClient for proper telemetry endpoint reporting - Add test for empty values behavior Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Python: Fix OpenAI chat client compatibility with third-party endpoints and OTel 0.4.14 (#4161) * Fix system message content sent as list instead of string Some OpenAI-compatible endpoints (e.g. NVIDIA NIM) reject system messages when content is a list of content parts. This change flattens system and developer message content to a plain string in the Chat Completions client. Fixes https://github.com/microsoft/agent-framework/issues/1407 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix compatibility with opentelemetry-semantic-conventions-ai 0.4.14 Version 0.4.14 removed several LLM_* attributes from SpanAttributes (LLM_SYSTEM, LLM_REQUEST_MODEL, LLM_RESPONSE_MODEL, LLM_REQUEST_MAX_TOKENS, LLM_REQUEST_TEMPERATURE, LLM_REQUEST_TOP_P, LLM_TOKEN_TYPE). Move these to the OtelAttr enum with their well-known gen_ai.* string values and update all references in observability.py and tests. Fixes https://github.com/microsoft/agent-framework/issues/4160 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Flatten text-only message content to string for all roles Extend the system/developer fix to all message roles. Text-only content lists are now post-processed into plain strings, while multimodal content (text + images/audio) remains as a list. This fixes compatibility with OpenAI-like endpoints that cannot deserialize list content (e.g. Foundry Local's Neutron backend). Partially fixes https://github.com/microsoft/agent-framework/issues/4084 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix streaming text lost when usage data in same chunk Some providers (e.g. Gemini) include both usage data and text content in the same streaming chunk. The early return on chunk.usage caused text and tool call parsing to be skipped entirely. Remove the early return and process usage alongside text/tool calls. Fixes https://github.com/microsoft/agent-framework/issues/3434 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix mypy errors in _chat_client.py Rename shadowed variable 'args' in system/developer branch to 'sys_args' and rename loop variable 'content' to 'msg_content' to avoid type conflict. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * reorder imports * fix: Use OtelAttr.REQUEST_MODEL instead of removed SpanAttributes.LLM_REQUEST_MODEL Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: Add score_threshold to vector store plan Reference SK .NET PR #13501 for score threshold filtering semantics. Include score_threshold in SearchOptions from Phase 3. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: Add reference to roji's SK .NET MEVD work for SQL connectors Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: Clear env vars in construction tests to avoid CI leakage Tests for missing API key / model ID now use monkeypatch.delenv to ensure env vars from the integration test environment don't prevent the expected ValueError from being raised. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Welcome to Microsoft Agent Framework!
Welcome to Microsoft's comprehensive multi-language framework for building, orchestrating, and deploying AI agents with support for both .NET and Python implementations. This framework provides everything from simple chat agents to complex multi-agent workflows with graph-based orchestration.
Watch the full Agent Framework introduction (30 min)
📋 Getting Started
📦 Installation
Python
pip install agent-framework --pre
# This will install all sub-packages, see `python/packages` for individual packages.
# It may take a minute on first install on Windows.
.NET
dotnet add package Microsoft.Agents.AI
📚 Documentation
- Overview - High level overview of the framework
- Quick Start - Get started with a simple agent
- Tutorials - Step by step tutorials
- User Guide - In-depth user guide for building agents and workflows
- Migration from Semantic Kernel - Guide to migrate from Semantic Kernel
- Migration from AutoGen - Guide to migrate from AutoGen
Still have questions? Join our weekly office hours or ask questions in our Discord channel to get help from the team and other users.
✨ Highlights
- Graph-based Workflows: Connect agents and deterministic functions using data flows with streaming, checkpointing, human-in-the-loop, and time-travel capabilities
- AF Labs: Experimental packages for cutting-edge features including benchmarking, reinforcement learning, and research initiatives
- DevUI: Interactive developer UI for agent development, testing, and debugging workflows
See the DevUI in action (1 min)
- Python and C#/.NET Support: Full framework support for both Python and C#/.NET implementations with consistent APIs
- Observability: Built-in OpenTelemetry integration for distributed tracing, monitoring, and debugging
- Multiple Agent Provider Support: Support for various LLM providers with more being added continuously
- Middleware: Flexible middleware system for request/response processing, exception handling, and custom pipelines
💬 We want your feedback!
- For bugs, please file a GitHub issue.
Quickstart
Basic Agent - Python
Create a simple Azure Responses Agent that writes a haiku about the Microsoft Agent Framework
# pip install agent-framework --pre
# Use `az login` to authenticate with Azure CLI
import os
import asyncio
from agent_framework.azure import AzureOpenAIResponsesClient
from azure.identity import AzureCliCredential
async def main():
# Initialize a chat agent with Azure OpenAI Responses
# the endpoint, deployment name, and api version can be set via environment variables
# or they can be passed in directly to the AzureOpenAIResponsesClient constructor
agent = AzureOpenAIResponsesClient(
# endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
# deployment_name=os.environ["AZURE_OPENAI_RESPONSES_DEPLOYMENT_NAME"],
# api_version=os.environ["AZURE_OPENAI_API_VERSION"],
# api_key=os.environ["AZURE_OPENAI_API_KEY"], # Optional if using AzureCliCredential
credential=AzureCliCredential(), # Optional, if using api_key
).as_agent(
name="HaikuBot",
instructions="You are an upbeat assistant that writes beautifully.",
)
print(await agent.run("Write a haiku about Microsoft Agent Framework."))
if __name__ == "__main__":
asyncio.run(main())
Basic Agent - .NET
Create a simple Agent, using OpenAI Responses, that writes a haiku about the Microsoft Agent Framework
// dotnet add package Microsoft.Agents.AI.OpenAI --prerelease
using Microsoft.Agents.AI;
using OpenAI;
using OpenAI.Responses;
// Replace the <apikey> with your OpenAI API key.
var agent = new OpenAIClient("<apikey>")
.GetResponsesClient("gpt-4o-mini")
.AsAIAgent(name: "HaikuBot", instructions: "You are an upbeat assistant that writes beautifully.");
Console.WriteLine(await agent.RunAsync("Write a haiku about Microsoft Agent Framework."));
Create a simple Agent, using Azure OpenAI Responses with token based auth, that writes a haiku about the Microsoft Agent Framework
// dotnet add package Microsoft.Agents.AI.OpenAI --prerelease
// dotnet add package Azure.Identity
// Use `az login` to authenticate with Azure CLI
using System.ClientModel.Primitives;
using Azure.Identity;
using Microsoft.Agents.AI;
using OpenAI;
using OpenAI.Responses;
// Replace <resource> and gpt-4o-mini with your Azure OpenAI resource name and deployment name.
var agent = new OpenAIClient(
new BearerTokenPolicy(new AzureCliCredential(), "https://ai.azure.com/.default"),
new OpenAIClientOptions() { Endpoint = new Uri("https://<resource>.openai.azure.com/openai/v1") })
.GetResponsesClient("gpt-4o-mini")
.AsAIAgent(name: "HaikuBot", instructions: "You are an upbeat assistant that writes beautifully.");
Console.WriteLine(await agent.RunAsync("Write a haiku about Microsoft Agent Framework."));
More Examples & Samples
Python
- Getting Started with Agents: progressive tutorial from hello-world to hosting
- Agent Concepts: deep-dive samples by topic (tools, middleware, providers, etc.)
- Getting Started with Workflows: workflow creation and integration with agents
.NET
- Getting Started with Agents: basic agent creation and tool usage
- Agent Provider Samples: samples showing different agent providers
- Workflow Samples: advanced multi-agent patterns and workflow orchestration
Contributor Resources
Important Notes
If you use the Microsoft Agent Framework to build applications that operate with third-party servers or agents, you do so at your own risk. We recommend reviewing all data being shared with third-party servers or agents and being cognizant of third-party practices for retention and location of data. It is your responsibility to manage whether your data will flow outside of your organization's Azure compliance and geographic boundaries and any related implications.
