agent/agent-framework
1
0
Fork 0
mirror of https://github.com/microsoft/agent-framework.git synced 2026-06-16 21:04:09 +08:00
Code Issues Packages Projects Releases Wiki Activity

Python: updated python design and dev setup (#171)

Browse Source 
* updated python design and dev setup

* updated dev setup

* updated dev setup
This commit is contained in:
Eduard van Valkenburg
2025-07-28 11:28:58 +02:00
committed by GitHub
Unverified
parent eafb333a3c
commit 08a1b714b4
2 changed files with 518 additions and 106 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/design/python-package-setup.md
+55 -106
View File
@@ -27,10 +27,10 @@
### Sample getting started code
```python
from typing import Annotated
from agent_framework import Agent, ai_tool
from agent_framework import Agent, ai_function
from agent_framework.openai import OpenAIChatClient
@ai_tool(description="Get the current weather in a given location")
@ai_function(description="Get the current weather in a given location")
async def get_weather(location: Annotated[str, "The location as a city name"]) -> str:
"""Get the current weather in a given location."""
# Implementation of the tool to get weather
@@ -50,12 +50,12 @@ print(response)
Overall the following structure is proposed:
* agent-framework
* tier 0; components, will be exposed directly from `agent_framework`:
* core components, will be exposed directly from `agent_framework`:
* (single) agents (includes threads)
* tools (includes MCP and OpenAPI)
* models/types (name tbd, will include the equivalent of MEAI for dotnet; content types and client abstractions)
* logging
* tier 1; components, will be exposed from `agent_framework.<component>`:
* advanced components, will be exposed from `agent_framework.<component>`:
* context_providers (tbd)
* guardrails / filters
* vector_data (vector stores and other MEVD pieces)
@@ -65,12 +65,12 @@ Overall the following structure is proposed:
* utils (optional)
* telemetry (could also be observability or monitoring)
* workflows (includes multi-agent orchestration)
* tier 2; extensions
* Extensions are any additional functionality that is useful for a user, to reduce friction they will imported in a similar way as tier 1, however the code for them will be in a separate package, so that they can be installed separately, they must have everything in a folder with the same name as the package, and without a `__init__.py` file in the root, so that they can be used as a namespace package.
* connectors; subpackages
* Subpackages are any additional functionality that is useful for a user, to reduce friction they will imported in a similar way as advanced components, however the code for them will be in a separate package, so that they can be installed separately, they must expose all public items, in their main `__init__.py` file, so that they can be imported from the main package without additional import levels.
In the main package a corresponding folder will be created, with a `__init__.py` file that lazy imports the public items from the subpackage, so that they can be exposed from the main package.
* Some examples are:
* openai
* azure
* will be exposed through i.e. `agent_framework.openai` and `agent_framework.azure`
* azure (non LLM integrations)
* will be exposed through i.e. `agent_framework.azure`
* anything other then a connector that we want to expose as a separate package, for instance:
* mem0 (memory management)
* would be exposed through i.e. `agent_framework.mem0`
@@ -79,7 +79,6 @@ Overall the following structure is proposed:
* tests
* samples
* extensions
* openai
* azure
* ...
@@ -90,25 +89,39 @@ Internal imports will be done using relative imports, so that the package can be
The resulting file structure will be as follows:
```plaintext
agent_framework/
__init__.py
__init__.pyi
_agents.py
_tools.py
_models.py
_logging.py
context_providers.py
guardrails.py
exceptions.py
evaluations.py
utils.py
telemetry.py
templates.py
text_search.py
vector_data.py
workflows.py
py.typed
extensions/
packages/
main/
agent_framework/
openai/
__init__.py
_chat_client.py
_shared.py
exceptions.py
__init__.py
__init__.pyi
_agents.py
_tools.py
_models.py
_logging.py
context_providers.py
guardrails.py
exceptions.py
evaluations.py
utils.py
telemetry.py
templates.py
text_search.py
vector_data.py
workflows.py
py.typed
tests/
unit/
test_types.py
integration/
test_chat_clients.py
pyproject.toml
README.md
...
mem0/
agent_framework/
mem0/
@@ -117,26 +130,7 @@ extensions/
...
redis/
...
openai/
agent_framework/
openai/
__init__.py
_chat.py
_embeddings.py
...
tests/
unit/
test_openai_client.py
test_openai_tools.py
...
integration/
test_openai_integration.py
samples/ (optional)
...
pyproject.toml
README.md
...
google/
google/
agent_framework/
google/
__init__.py
@@ -156,13 +150,6 @@ extensions/
README.md
...
...
tests/
__init__.py
unit/
conftest.py
test_agents.py
test_types.py
...
samples/
...
pyproject.toml
@@ -174,10 +161,10 @@ uv.lock
We might add a template subpackage as well, to make it easy to setup, this could be based on the first one that is added.
In the `DEV_SETUP.md` we will add instructions for how to deal with the path depth issues, especially on Windows, where the maximum path length can be a problem.
In the [`DEV_SETUP.md`](../../python/DEV_SETUP.md) we will add instructions for how to deal with the path depth issues, especially on Windows, where the maximum path length can be a problem.
#### Evolving the package structure
For each of the tier 1 components, we have two reason why we may split them into a folder, with an `__init__.py` and optionally a `_files.py`:
For each of the advanced components, we have two reason why we may split them into a folder, with an `__init__.py` and optionally a `_files.py`:
1. If the file becomes too large, we can split it into multiple `_files`, while still keeping the public interface in the `__init__.py` file, this is a non-breaking change
2. If we want to partially or fully move that code into a separate package.
In this case we do need to lazy load anything that was moved from the main package to the subpackage, so that existing code still works, and if the subpackage is not installed we can raise a meaningful error.
@@ -197,14 +184,7 @@ agent_framework/
## Coding standards
* We use google docstyles for docstrings.
* We use the following setup for ruff:
```toml
[tool.ruff]
line-length = 120
target-version = "py310"
include = ["*.py", "*.pyi", "**/pyproject.toml", "*.ipynb"]
preview = true
Coding standards will be maintained in the [`DEV_SETUP.md`](../../python/DEV_SETUP.md) file.
[tool.ruff.lint]
fixable = ["ALL"]
@@ -281,8 +261,8 @@ The logging will be simplified, there will be one logger in the base package:
* name: `agent_framework` - used for all logging in the abstractions and base components
Each of the other subpackages for connectors will have a similar single logger.
* name: `agent_framework.connectors.openai`
* name: `agent_framework.connectors.azure`
* name: `agent_framework.openai`
* name: `agent_framework.azure`
This means that when a logger is needed, it should be created like this:
```python
@@ -290,7 +270,7 @@ from agent_framework import get_logger
logger = get_logger()
#or in a subpackage:
logger = get_logger('agent_framework.connectors.openai')
logger = get_logger('agent_framework.openai')
```
The implementation should be something like this:
```python
@@ -300,10 +280,10 @@ import logging
def get_logger(name: str = "agent_framework") -> logging.Logger:
"""
Get a logger with the specified name, defaulting to 'agent_framework'.
Args:
name (str): The name of the logger. Defaults to 'agent_framework'.
Returns:
logging.Logger: The configured logger instance.
"""
@@ -311,7 +291,7 @@ def get_logger(name: str = "agent_framework") -> logging.Logger:
# create the specifics for the logger, such as setting the level, handlers, etc.
return logger
```
This will ensure that the logger is created with the correct name and configuration, and it will be consistent across the package.
This will ensure that the logger is created with the correct name and configuration, and it will be consistent across the package.
Further there should be a easy way to configure the log levels, either through a environment variable or with a similar function as the get_logger.
@@ -320,7 +300,7 @@ This will not be allowed:
import logging
logger = logging.getLogger(__name__)
```
```
This is allowed but discouraged, if the get_logger function has been called at least once then this will return the same logger as the get_logger function, however that might not have happened and then the logging experience (in terms of formats and handlers, etc) is not consistent across the package:
```python
@@ -332,41 +312,10 @@ logger = logging.getLogger("agent_framework")
#### Telemetry
Telemetry will be based on OpenTelemetry (OTel), and will be implemented in the `agent_framework.telemetry` package.
We will also add headers with user-agent strings where applicable, these will include `agent-framework-python` and the version.
We should consider auto-instrumentation and provide an implementation of it to the OTel community.
### Function definitions
To make the code easier to use, we will be very deliberate about the ordering and marking of function parameters.
This means that we will use the following conventions:
* Only parameters that are fully expected to be passed and only if there are a very limited number of them, let's say 3 or less, can they be supplied as positional parameters (still with a keyword, _almost_ never positional only).
* All other parameters should be supplied as keyword parameters, this is especially important to configure correctly when using Pydantic or dataclasses.
* If there are multiple required parameters, and they do not have a order that is common sense, then they will all use keyword parameters.
* If we use `kwargs` we will document how and what we use them for, this might be a reference to a outside package's documentation or an explanation of what the `kwargs` are used for.
* If we want to combine `kwargs` for multiple things, such as partly for a external client constructor, and partly for our own use, we will try to keep those separate, by adding a parameter, such as `client_kwargs` with type `dict[str, Any]`, and then use that to pass the kwargs to the client constructor (by using `Client(**client_kwargs)`), while using the `**kwargs` parameters for other uses, which are then also well documented.
### Attributes vs inheritance
When the parameters are the same except for one, we will use attributes, instead of inheritance, to minimize the conceptual overhead of understanding the code. Off course there are exceptions and these things will be decided on a case by case basis, but the general rule is that if the parameters are the same, we will use attributes.
```python
# ✅ preferred
from agent_framework import ChatMessage
user_msg = ChatMessage(
role="user",
content="Hello, world!"
)
asst_msg = ChatMessage(
role="assistant",
content="Hello, world!"
)
# ❌ not preferred
from agent_framework import UserMessage, AssistantMessage
user_msg = UserMessage(
content="Hello, world!"
)
asst_msg = AssistantMessage(
content="Hello, world!"
)
```
### Build and release
The build step will be done in GHA, adding the package to the release and then we call into Azure DevOps to use the ESRP pipeline to publish to pypi. This is how SK already works, we will just have to adapt it to the new package structure.
python/DEV_SETUP.md
+463
View File
@@ -0,0 +1,463 @@
# Dev Setup
This document describes how to setup your environment with Python and uv,
if you're working on new features or a bug fix for Agent Framework, or simply
want to run the tests included.
## System setup
## If you're on WSL
Check that you've cloned the repository to `~/workspace` or a similar folder.
Avoid `/mnt/c/` and prefer using your WSL user's home directory.
Ensure you have the WSL extension for VSCode installed.
## Using uv
uv allows us to use AF from the local files, without worrying about paths, as
if you had AF pip package installed.
To install AF and all the required tools in your system, first, navigate to the directory containing
this DEV_SETUP using your chosen shell.
### For windows (non-WSL)
Check the [uv documentation](https://docs.astral.sh/uv/getting-started/installation/) for the installation instructions. At the time of writing this is the command to install uv:
```powershell
powershell -c "irm https://astral.sh/uv/install.ps1 | iex"
```
### For WSL, Linux or MacOS
Check the [uv documentation](https://docs.astral.sh/uv/getting-started/installation/) for the installation instructions. At the time of writing this is the command to install uv:
```bash
curl -LsSf https://astral.sh/uv/install.sh | sh
```
### After installing uv
You can then run the following commands manually:
```bash
# Install Python 3.10, 3.11, 3.12, and 3.13
uv python install 3.10 3.11 3.12 3.13
# Create a virtual environment with Python 3.10 (you can change this to 3.11, 3.12 or 3.13)
$PYTHON_VERSION = "3.10"
uv venv --python $PYTHON_VERSION
# Install AF and all dependencies
uv sync --dev
# Install all the tools and dependencies
uv run poe install
# Install pre-commit hooks
uv run poe pre-commit-install
```
Alternatively, you can reinstall the venv, pacakges, dependencies and pre-commit hooks with a single command (but this requires poe in the current env), this is especially useful if you want to switch python versions:
```bash
uv run poe setup -p 3.13
```
You can then run different commands through Poe the Poet, use `uv run poe` to discover which ones.
## VSCode Setup
Install the [Python extension](https://marketplace.visualstudio.com/items?itemName=ms-python.python) for VSCode.
Open the `python` folder in [VSCode](https://code.visualstudio.com/docs/editor/workspaces).
> The workspace for python should be rooted in the `./python` folder.
Open any of the `.py` files in the project and run the `Python: Select Interpreter`
command from the command palette. Make sure the virtual env (default path is `.venv`) created by `uv` is selected.
## LLM setup
Make sure you have an
[OpenAI API Key](https://platform.openai.com) or
[Azure OpenAI service key](https://learn.microsoft.com/azure/cognitive-services/openai/quickstart?pivots=rest-api)
There are two methods to manage keys, secrets, and endpoints:
1. Store them in environment variables. AF Python leverages pydantic settings to load keys, secrets, and endpoints from the environment.
> When you are using VSCode and have the python extension setup, it automatically loads environment variables from a `.env` file, so you don't have to manually set them in the terminal.
> During runtime on different platforms, environment settings set as part of the deployments should be used.
2. Store them in a separate `.env` file, like `dev.env`, you can then pass that name into the constructor for most services, to the `env_file_path` parameter, see below.
> Make sure to add `*.env` to your `.gitignore` file.
There are a lot of settings, for a more extensive list of settings, see [ALL_SETTINGS.md](./samples/concepts/setup/ALL_SETTINGS.md).
### Example for file-based setup with OpenAI Chat Completions
To configure a `.env` file with just the keys needed for OpenAI Chat Completions, you can create a `openai.env` (this name is just as an example, a single `.env` with all required keys is more common) file in the root of the `python` folder with the following content:
Content of `.env` or `openai.env`:
```env
OPENAI_API_KEY=""
OPENAI_CHAT_MODEL_ID="gpt-4o-mini"
```
You will then configure the ChatClient class with the keyword argument `env_file_path`:
```python
from agent_framework.openai import OpenAIChatClient
chat_client = OpenAIChatClient(env_file_path="openai.env")
```
## Coding Standards
### Code Style and Formatting
We use [ruff](https://github.com/astral-sh/ruff) for both linting and formatting with the following configuration:
- **Line length**: 120 characters
- **Target Python version**: 3.10+
- **Google-style docstrings**: All public functions, classes, and modules should have docstrings following Google conventions
### Function Parameter Guidelines
To make the code easier to use and maintain:
- **Positional parameters**: Only use for up to 3 fully expected parameters
- **Keyword parameters**: Use for all other parameters, especially when there are multiple required parameters without obvious ordering
- **Avoid additional imports**: Do not require the user to import additional modules to use the function, so provide string based overrides when applicable, for instance:
```python
def create_agent(name: str, tool_mode: ChatToolMode) -> Agent:
# Implementation here
```
Should be:
```python
def create_agent(name: str, tool_mode: Literal['auto', 'required', 'none'] | ChatToolMode) -> Agent:
# Implementation here
if isinstance(tool_mode, str):
tool_mode = ChatToolMode(tool_mode)
```
- **Document kwargs**: Always document how `kwargs` are used, either by referencing external documentation or explaining their purpose
- **Separate kwargs**: When combining kwargs for multiple purposes, use specific parameters like `client_kwargs: dict[str, Any]` instead of mixing everything in `**kwargs`
Example:
```python
chat_completion = OpenAIChatClient(env_file_path="openai.env")
```
## Tests
All the tests are located in the `tests` folder of each package. There are tests that are marked with a `@skip_if_..._integration_tests_disabled` decorator, these are integration tests that require an external service to be running, like OpenAI or Azure OpenAI.
If you want to run these tests, you need to set the environment variable `RUN_INTEGRATION_TESTS` to `true` and have the appropriate key per services set in your environment or in a `.env` file.
Alternatively, you can run them using VSCode Tasks. Open the command palette
(`Ctrl+Shift+P`) and type `Tasks: Run Task`. Select `Test` from the list.
If you want to run the tests for a single package, you can use the `uv run poe test` command with the package name as an argument. For example, to run the tests for the `agent_framework` package, you can use:
```bash
uv run poe --directory packages/main test
```
These commands also output the coverage report.
## Implementation Decisions
### Asynchronous programming
It's important to note that most of this library is written with asynchronous in mind. The
developer should always assume everything is asynchronous. One can use the function signature
with either `async def` or `def` to understand if something is asynchronous or not.
### Documentation
Each file should have a single first line containing: # Copyright (c) Microsoft. All rights reserved.
We follow the [Google Docstring](https://github.com/google/styleguide/blob/gh-pages/pyguide.md#383-functions-and-methods) style guide for functions and methods.
They are currently not checked for private functions (functions starting with '_').
They should contain:
- Single line explaining what the function does, ending with a period.
- If necessary to further explain the logic a newline follows the first line and then the explanation is given.
- The following three sections are optional, and if used should be separated by a single empty line.
- Arguments are then specified after a header called `Args:`, with each argument being specified in the following format:
- `arg_name`: Explanation of the argument.
- if a longer explanation is needed for a argument, it should be placed on the next line, indented by 4 spaces.
- Type and default values do not have to be specified, they will be pulled from the definition.
- Returns are specified after a header called `Returns:` or `Yields:`, with the return type and explanation of the return value.
- Finally, a header for exceptions can be added, called `Raises:`, with each exception being specified in the following format:
- `ExceptionType`: Explanation of the exception.
- if a longer explanation is needed for a exception, it should be placed on the next line, indented by 4 spaces.
Putting them all together, gives you at minimum this:
```python
def equal(arg1: str, arg2: str) -> bool:
"""Compares two strings and returns True if they are the same."""
...
```
Or a complete version of this:
```python
def equal(arg1: str, arg2: str) -> bool:
"""Compares two strings and returns True if they are the same.
Here is extra explanation of the logic involved.
Args:
arg1: The first string to compare.
arg2: The second string to compare.
Returns:
True if the strings are the same, False otherwise.
"""
```
### Attributes vs Inheritance
Prefer attributes over inheritance when parameters are mostly the same:
```python
# ✅ Preferred - using attributes
from agent_framework import ChatMessage
user_msg = ChatMessage(role="user", content="Hello, world!")
asst_msg = ChatMessage(role="assistant", content="Hello, world!")
# ❌ Not preferred - unnecessary inheritance
from agent_framework import UserMessage, AssistantMessage
user_msg = UserMessage(content="Hello, world!")
asst_msg = AssistantMessage(content="Hello, world!")
```
### Logging
Use the centralized logging system:
```python
from agent_framework import get_logger
# For main package
logger = get_logger()
# For subpackages
logger = get_logger('agent_framework.azure')
```
**Do not use** direct logging module imports:
```python
# ❌ Avoid this
import logging
logger = logging.getLogger(__name__)
```
### Import Structure
The package follows a flat import structure:
- **Core**: Import directly from `agent_framework`
```python
from agent_framework import ChatClientAgent, ai_function
```
- **Components**: Import from `agent_framework.<component>`
```python
from agent_framework.vector_data import VectorStoreModel
from agent_framework.guardrails import ContentFilter
```
- **Connectors**: Import from `agent_framework.<vendor/platform>`
```python
from agent_framework.openai import OpenAIChatClient
from agent_framework.azure import AzureChatClient
```
## Testing
### Running Tests
```bash
# Run all tests with coverage
uv run poe test
# Run specific test file
uv run pytest tests/test_agents.py
# Run with verbose output
uv run pytest -v
```
### Test Coverage
- Target: Minimum 80% test coverage for all packages
- Coverage reports are generated automatically during test runs
- Tests should be in corresponding `test_*.py` files in the `tests/` directory
## Documentation
### Building Documentation
```bash
# Build documentation
uv run poe docs-build
# Serve documentation locally with auto-reload
uv run poe docs-serve
# Check documentation for warnings
uv run poe docs-check
```
### Docstring Style
Use Google-style docstrings for all public APIs:
```python
def create_agent(name: str, chat_client: ChatClient) -> Agent:
"""Create a new agent with the specified configuration.
Args:
name: The name of the agent.
chat_client: The chat client to use for communication.
Returns:
True if the strings are the same, False otherwise.
Raises:
ValueError: If one of the strings is empty.
"""
...
```
If in doubt, use the link above to read much more considerations of what to do and when, or use common sense.
## Coding standards
```plaintext
agent_framework/
├── __init__.py # Tier 0: Core components
├── _agents.py # Agent implementations
├── _tools.py # Tool definitions
├── _models.py # Type definitions
├── _logging.py # Logging utilities
├── context_providers.py # Tier 1: Context providers
├── guardrails.py # Tier 1: Guardrails and filters
├── vector_data.py # Tier 1: Vector stores
├── workflows.py # Tier 1: Multi-agent orchestration
└── azure/ # Tier 2: Azure connectors (lazy loaded)
└── __init__.py # Imports from agent-framework-azure
```
### Pydantic and Serialization
This section describes how one can enable serialization for their class using Pydantic.
For more info you can refer to the [Pydantic Documentation](https://docs.pydantic.dev/latest/).
#### Upgrading existing classes to use Pydantic
Let's take the following example:
```python
class A:
def __init__(self, a: int, b: float, c: List[float], d: dict[str, tuple[float, str]] = {}):
self.a = a
self.b = b
self.c = c
self.d = d
```
You would convert this to a Pydantic class by sub-classing from the `AFBaseModel` class.
```python
from pydantic import Field
from ._pydantic import AFBaseModel
class A(AFBaseModel):
# The notation for the fields is similar to dataclasses.
a: int
b: float
c: list[float]
# Only, instead of using dataclasses.field, you would use pydantic.Field
d: dict[str, tuple[float, str]] = Field(default_factory=dict)
```
#### Classes with data that need to be serialized, and some of them are Generic types
Let's take the following example:
```python
from typing import TypeVar
T1 = TypeVar("T1")
T2 = TypeVar("T2", bound=<some class>)
class A:
def __init__(a: int, b: T1, c: T2):
self.a = a
self.b = b
self.c = c
```
You can use the `AFBaseModel` to convert these to pydantic serializable classes.
```python
from typing import Generic, TypeVar
from ._pydantic import AFBaseModel
T1 = TypeVar("T1")
T2 = TypeVar("T2", bound=<some class>)
class A(AFBaseModel, Generic[T1, T2]):
# T1 and T2 must be specified in the Generic argument otherwise, pydantic will
# NOT be able to serialize this class
a: int
b: T1
c: T2
```
## Code quality checks
To run the same checks that run during a commit and the GitHub Action `Python Code Quality`, you can use this command, from the [python](../python) folder:
```bash
uv run poe check
```
Ideally you should run these checks before committing any changes, when you install using the instructions above the pre-commit hooks should be installed already.
## Code Coverage
We try to maintain a high code coverage for the project. To run the code coverage on the unit tests, you can use the following command:
```bash
uv run poe test
```
This will show you which files are not covered by the tests, including the specific lines not covered. Make sure to consider the untested lines from the code you are working on, but feel free to add other tests as well, that is always welcome!
## Catching up with the latest changes
There are many people committing to Semantic Kernel, so it is important to keep your local repository up to date. To do this, you can run the following commands:
```bash
git fetch upstream main
git rebase upstream/main
git push --force-with-lease
```
or:
```bash
git fetch upstream main
git merge upstream/main
git push
```
This is assuming the upstream branch refers to the main repository. If you have a different name for the upstream branch, you can replace `upstream` with the name of your upstream branch.
After running the rebase command, you may need to resolve any conflicts that arise. If you are unsure how to resolve a conflict, please refer to the [GitHub's documentation on resolving conflicts](https://docs.github.com/en/get-started/using-git/resolving-merge-conflicts-after-a-git-rebase), or for [VSCode](https://code.visualstudio.com/docs/sourcecontrol/overview#_merge-conflicts).