mirror of
https://github.com/microsoft/agent-framework.git
synced 2026-06-16 21:04:09 +08:00
Python: restructure: Python samples into progressive 01-05 layout (#3862)
* restructure: Python samples into progressive 01-05 layout - 01-get-started/: 6 numbered steps (hello agent → hosting) - 02-agents/: all agent concept samples (tools, middleware, providers, etc.) - 03-workflows/: ALL existing workflow samples preserved as-is - 04-hosting/: azure-functions, durabletask, a2a - 05-end-to-end/: demos, evaluation, hosted agents - Old files moved to _to_delete/ for review - Added AGENTS.md with structure documentation - autogen-migration/ and semantic-kernel-migration/ preserved at root * fix: switch to AzureOpenAI Foundry, fix CI failures - Switch all 01-get-started samples to AzureOpenAIResponsesClient with Azure AI Foundry project endpoint (AZURE_AI_PROJECT_ENDPOINT + AZURE_OPENAI_RESPONSES_DEPLOYMENT_NAME + AzureCliCredential) - Add _to_delete/ and 05-end-to-end/ to pyrightconfig.samples.json excludes - Fix test paths in packages/ that referenced old getting_started/ dirs: durabletask conftest + streaming test, azurefunctions conftest, devui conftest + capture_messages + openai_sdk_integration - Fix workflow_as_agent_human_in_the_loop.py import (sibling import) - Update hosting READMEs and tool comment paths - Replace root README.md with new structure overview - Update AGENTS.md to document Azure OpenAI Foundry as default provider * cleanup: remove _to_delete folder, copy resource files to active dirs All files in _to_delete/ were either: - Exact duplicates of files in the new structure (240 files) - Same file with only comment path updates (100 files) - One import-fix diff (workflow_as_agent_human_in_the_loop.py) - One superseded minimal_sample.py Resource files (sample.pdf, countries.json, employees.pdf, weather.json) copied to 02-agents/sample_assets/ and 02-agents/resources/ since active samples reference them. * fix: address PR review comments, centralize resources, remove root duplicates - Fix type annotation in 04_memory.py (string union -> proper types) - Fix old sample paths in observability files - Fix grammar/spelling in observability samples - Move sample_assets/ and resources/ to shared/ folder - Remove 8 duplicate observability files from 02-agents root - Update resource path references in multimodal_input and provider samples * fix: update broken links from old getting_started paths to new structure - Update relative paths in READMEs: getting_started/ → 01-get-started/, 02-agents/, 03-workflows/, 04-hosting/, 05-end-to-end/ - Fix absolute GitHub URLs in package READMEs - Fix broken link in ollama package README * fix: convert absolute GitHub URLs to relative paths for link checker Absolute URLs to python/samples/ on main branch 404 until PR merges. Converted to relative paths that linkspector can verify locally. * fix: update link for handoff sample moved to orchestrations/ * fix: update chatkit-integration README path from demos/ to 05-end-to-end/ * fix: update broken links in orchestrations README to match flat directory structure
This commit is contained in:
Eduard van Valkenburg
committed by
GitHub
Unverified
GitHub
parent
69dcfe31ee
commit
a2856d3b92
@@ -0,0 +1,49 @@
|
||||
# Observability Configuration
|
||||
# ===========================
|
||||
|
||||
# Standard OpenTelemetry environment variables
|
||||
# See https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/
|
||||
|
||||
# OTLP Endpoint (for Aspire Dashboard, Jaeger, etc.)
|
||||
# Default protocol is gRPC (port 4317), HTTP uses port 4318
|
||||
OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:4317"
|
||||
|
||||
# Optional: Override endpoint for specific signals
|
||||
# OTEL_EXPORTER_OTLP_TRACES_ENDPOINT="http://localhost:4317"
|
||||
# OTEL_EXPORTER_OTLP_METRICS_ENDPOINT="http://localhost:4317"
|
||||
# OTEL_EXPORTER_OTLP_LOGS_ENDPOINT="http://localhost:4317"
|
||||
|
||||
# Optional: Specify protocol (grpc or http)
|
||||
# OTEL_EXPORTER_OTLP_PROTOCOL="grpc"
|
||||
|
||||
# Optional: Add headers (e.g., for authentication)
|
||||
# OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer token,x-api-key=key"
|
||||
|
||||
# Optional: Service identification
|
||||
# OTEL_SERVICE_NAME="my-agent-app"
|
||||
# OTEL_SERVICE_VERSION="1.0.0"
|
||||
# OTEL_RESOURCE_ATTRIBUTES="deployment.environment=dev,host.name=localhost"
|
||||
|
||||
# Agent Framework specific settings
|
||||
# ==================================
|
||||
|
||||
# Enable sensitive data logging (prompts, responses, etc.)
|
||||
# WARNING: Only enable in dev/test environments
|
||||
ENABLE_SENSITIVE_DATA=true
|
||||
|
||||
# Optional: Enable console exporters for debugging
|
||||
# ENABLE_CONSOLE_EXPORTERS=true
|
||||
|
||||
# Optional: Enable observability (automatically enabled if env vars are set or configure_otel_providers() is called)
|
||||
# ENABLE_INSTRUMENTATION=true
|
||||
|
||||
# OpenAI specific variables
|
||||
# ==========================
|
||||
OPENAI_API_KEY="..."
|
||||
OPENAI_RESPONSES_MODEL_ID="gpt-4o-2024-08-06"
|
||||
OPENAI_CHAT_MODEL_ID="gpt-4o-2024-08-06"
|
||||
|
||||
# Azure AI Foundry specific variables
|
||||
# ====================================
|
||||
AZURE_AI_PROJECT_ENDPOINT="..."
|
||||
AZURE_AI_MODEL_DEPLOYMENT_NAME="gpt-4o-mini"
|
||||
@@ -0,0 +1,411 @@
|
||||
# Agent Framework Python Observability
|
||||
|
||||
This sample folder shows how a Python application can be configured to send Agent Framework observability data to the Application Performance Management (APM) vendor(s) of your choice based on the OpenTelemetry standard.
|
||||
|
||||
In this sample, we provide options to send telemetry to [Application Insights](https://learn.microsoft.com/en-us/azure/azure-monitor/app/app-insights-overview), [Aspire Dashboard](https://learn.microsoft.com/en-us/dotnet/aspire/fundamentals/dashboard/overview?tabs=bash) and the console.
|
||||
|
||||
> **Quick Start**: For local development without Azure setup, you can use the [Aspire Dashboard](https://learn.microsoft.com/en-us/dotnet/aspire/fundamentals/dashboard/standalone) which runs locally via Docker and provides an excellent telemetry viewing experience for OpenTelemetry data. Or you can use the built-in tracing module of the [AI Toolkit for VS Code](https://marketplace.visualstudio.com/items?itemName=ms-windows-ai-studio.windows-ai-studio).
|
||||
|
||||
> Note that it is also possible to use other Application Performance Management (APM) vendors. An example is [Prometheus](https://prometheus.io/docs/introduction/overview/). Please refer to this [page](https://opentelemetry.io/docs/languages/python/exporters/) to learn more about exporters.
|
||||
|
||||
For more information, please refer to the following resources:
|
||||
|
||||
1. [Azure Monitor OpenTelemetry Exporter](https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/monitor/azure-monitor-opentelemetry-exporter)
|
||||
2. [Aspire Dashboard for Python Apps](https://learn.microsoft.com/en-us/dotnet/aspire/fundamentals/dashboard/standalone-for-python?tabs=flask%2Cwindows)
|
||||
3. [AI Toolkit for VS Code](https://marketplace.visualstudio.com/items?itemName=ms-windows-ai-studio.windows-ai-studio)
|
||||
4. [Python Logging](https://docs.python.org/3/library/logging.html)
|
||||
5. [Observability in Python](https://www.cncf.io/blog/2022/04/22/opentelemetry-and-python-a-complete-instrumentation-guide/)
|
||||
|
||||
## What to expect
|
||||
|
||||
The Agent Framework Python SDK is designed to efficiently generate comprehensive logs, traces, and metrics throughout the flow of agent/model invocation and tool execution. This allows you to effectively monitor your AI application's performance and accurately track token consumption. It does so based on the Semantic Conventions for GenAI defined by OpenTelemetry, and the workflows emit their own spans to provide end-to-end visibility.
|
||||
|
||||
Next to what happens in the code when you run, we also make setting up observability as easy as possible. By calling a single function `configure_otel_providers()` from the `agent_framework.observability` module, you can enable telemetry for traces, logs, and metrics. The function automatically reads standard OpenTelemetry environment variables to configure exporters and providers, making it simple to get started.
|
||||
|
||||
### Five patterns for configuring observability
|
||||
|
||||
We've identified multiple ways to configure observability in your application, depending on your needs:
|
||||
|
||||
**1. Standard otel environment variables, configured for you**
|
||||
|
||||
The simplest approach - configure everything via environment variables:
|
||||
|
||||
```python
|
||||
from agent_framework.observability import configure_otel_providers
|
||||
|
||||
# Reads OTEL_EXPORTER_OTLP_* environment variables automatically
|
||||
configure_otel_providers()
|
||||
```
|
||||
Or if you just want console exporters:
|
||||
```python
|
||||
from agent_framework.observability import configure_otel_providers
|
||||
# Enable console exporters via environment variable
|
||||
|
||||
configure_otel_providers(enable_console_exporters=True)
|
||||
```
|
||||
This is the **recommended approach** for getting started.
|
||||
|
||||
**2. Custom Exporters**
|
||||
One level more control over the exporters that are created is to do that yourself, and then pass them to `configure_otel_providers()`. We will still create the providers for you, but you can customize the exporters as needed:
|
||||
|
||||
```python
|
||||
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
|
||||
from opentelemetry.exporter.otlp.proto.grpc._log_exporter import OTLPLogExporter
|
||||
from opentelemetry.exporter.otlp.proto.grpc.metric_exporter import OTLPMetricExporter
|
||||
from agent_framework.observability import configure_otel_providers
|
||||
|
||||
# Create custom exporters with specific configuration
|
||||
exporters = [
|
||||
OTLPSpanExporter(endpoint="http://localhost:4317", compression=Compression.Gzip),
|
||||
OTLPLogExporter(endpoint="http://localhost:4317"),
|
||||
OTLPMetricExporter(endpoint="http://localhost:4317"),
|
||||
]
|
||||
|
||||
# These will be added alongside any exporters from environment variables
|
||||
configure_otel_providers(exporters=exporters, enable_sensitive_data=True)
|
||||
```
|
||||
|
||||
**3. Third party setup**
|
||||
|
||||
A lot of third party specific otel package, have their own easy setup methods, for example Azure Monitor has `configure_azure_monitor()`. You can use those methods to setup the third party first, and then call `enable_instrumentation()` from the `agent_framework.observability` module to activate the Agent Framework telemetry code paths. In all these cases, if you already setup observability via environment variables, you don't need to call `enable_instrumentation()` as it will be enabled automatically.
|
||||
|
||||
```python
|
||||
from azure.monitor.opentelemetry import configure_azure_monitor
|
||||
from agent_framework.observability import create_resource, enable_instrumentation
|
||||
|
||||
# Configure Azure Monitor first
|
||||
configure_azure_monitor(
|
||||
connection_string="InstrumentationKey=...",
|
||||
resource=create_resource(), # Uses OTEL_SERVICE_NAME, etc.
|
||||
enable_live_metrics=True,
|
||||
)
|
||||
|
||||
# Then activate Agent Framework's telemetry code paths
|
||||
# This is optional if ENABLE_INSTRUMENTATION and or ENABLE_SENSITIVE_DATA are set in env vars
|
||||
enable_instrumentation(enable_sensitive_data=False)
|
||||
```
|
||||
For Azure AI projects, use the `client.configure_azure_monitor()` method which wraps the calls to `configure_azure_monitor()` and `enable_instrumentation()`:
|
||||
|
||||
```python
|
||||
from agent_framework.azure import AzureAIClient
|
||||
from azure.ai.projects.aio import AIProjectClient
|
||||
|
||||
async with (
|
||||
AIProjectClient(...) as project_client,
|
||||
AzureAIClient(project_client=project_client) as client,
|
||||
):
|
||||
# Automatically configures Azure Monitor with connection string from project
|
||||
await client.configure_azure_monitor(enable_live_metrics=True)
|
||||
```
|
||||
|
||||
Or with [Langfuse](https://langfuse.com/integrations/frameworks/microsoft-agent-framework):
|
||||
|
||||
```python
|
||||
# environment should be setup correctly, with langfuse urls and keys
|
||||
from agent_framework.observability import enable_instrumentation
|
||||
from langfuse import get_client
|
||||
|
||||
langfuse = get_client()
|
||||
|
||||
# Verify connection
|
||||
if langfuse.auth_check():
|
||||
print("Langfuse client is authenticated and ready!")
|
||||
else:
|
||||
print("Authentication failed. Please check your credentials and host.")
|
||||
|
||||
# Then activate Agent Framework's telemetry code paths
|
||||
# This is optional if ENABLE_INSTRUMENTATION and or ENABLE_SENSITIVE_DATA are set in env vars
|
||||
enable_instrumentation(enable_sensitive_data=False)
|
||||
```
|
||||
|
||||
**4. Manual setup**
|
||||
Of course you can also do a complete manual setup of exporters, providers, and instrumentation. Please refer to sample [advanced_manual_setup_console_output.py](./advanced_manual_setup_console_output.py) for a comprehensive example of how to manually setup exporters and providers for traces, logs, and metrics that will get sent to the console. This gives you full control over which exporters and providers to use. We do have a helper function `create_resource()` in the `agent_framework.observability` module that you can use to create a resource with the appropriate service name and version based on environment variables or standard defaults for Agent Framework, this is not used in the sample.
|
||||
|
||||
**5. Auto-instrumentation (zero-code)**
|
||||
You can also use the [OpenTelemetry CLI tool](https://opentelemetry.io/docs/instrumentation/python/getting-started/#automatic-instrumentation) to automatically instrument your application without changing any code. Please refer to sample [advanced_zero_code.py](./advanced_zero_code.py) for an example of how to use the CLI tool to enable instrumentation for Agent Framework applications.
|
||||
|
||||
## Configuration
|
||||
|
||||
### Dependencies
|
||||
|
||||
As part of Agent Framework we use the following OpenTelemetry packages:
|
||||
- `opentelemetry-api`
|
||||
- `opentelemetry-sdk`
|
||||
- `opentelemetry-semantic-conventions-ai`
|
||||
|
||||
We do not install exporters by default, so you will need to add those yourself, this prevents us from installing unnecessary dependencies. For Application Insights, you will need to install `azure-monitor-opentelemetry`. For Aspire Dashboard or other OTLP compatible backends, you will need to install `opentelemetry-exporter-otlp-proto-grpc`. For HTTP protocol support, you will also need to install `opentelemetry-exporter-otlp-proto-http`.
|
||||
|
||||
And for many others, different packages are used, so refer to the documentation of the specific exporter you want to use.
|
||||
|
||||
### Environment variables
|
||||
|
||||
The following environment variables are used to turn on/off observability of the Agent Framework:
|
||||
|
||||
- `ENABLE_INSTRUMENTATION`
|
||||
- `ENABLE_SENSITIVE_DATA`
|
||||
- `ENABLE_CONSOLE_EXPORTERS`
|
||||
|
||||
All of these are booleans and default to `false`.
|
||||
|
||||
Finally we have `VS_CODE_EXTENSION_PORT` which you can set to a port, which can be used to setup the AI Toolkit for VS Code tracing integration. See [here](https://marketplace.visualstudio.com/items?itemName=ms-windows-ai-studio.windows-ai-studio#tracing) for more details.
|
||||
|
||||
The framework will emit observability data when the `ENABLE_INSTRUMENTATION` environment variable is set to `true`. If both are `true` then it will also emit sensitive information. When these are not set, or set to false, you can use the `enable_instrumentation()` function from the `agent_framework.observability` module to turn on instrumentation programmatically. This is useful when you want to control this via code instead of environment variables.
|
||||
|
||||
> **Note**: Sensitive information includes prompts, responses, and more, and should only be enabled in a development or test environment. It is not recommended to enable this in production environments as it may expose sensitive data.
|
||||
|
||||
The two other variables, `ENABLE_CONSOLE_EXPORTERS` and `VS_CODE_EXTENSION_PORT`, are used to configure where the observability data is sent. Those are only activated when calling `configure_otel_providers()`.
|
||||
|
||||
#### Environment variables for `configure_otel_providers()`
|
||||
|
||||
The `configure_otel_providers()` function automatically reads **standard OpenTelemetry environment variables** to configure exporters:
|
||||
|
||||
**OTLP Configuration** (for Aspire Dashboard, Jaeger, etc.):
|
||||
- `OTEL_EXPORTER_OTLP_ENDPOINT` - Base endpoint for all signals (e.g., `http://localhost:4317`)
|
||||
- `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` - Traces-specific endpoint (overrides base)
|
||||
- `OTEL_EXPORTER_OTLP_METRICS_ENDPOINT` - Metrics-specific endpoint (overrides base)
|
||||
- `OTEL_EXPORTER_OTLP_LOGS_ENDPOINT` - Logs-specific endpoint (overrides base)
|
||||
- `OTEL_EXPORTER_OTLP_PROTOCOL` - Protocol to use (`grpc` or `http`, default: `grpc`)
|
||||
- `OTEL_EXPORTER_OTLP_HEADERS` - Headers for all signals (e.g., `key1=value1,key2=value2`)
|
||||
- `OTEL_EXPORTER_OTLP_TRACES_HEADERS` - Traces-specific headers (overrides base)
|
||||
- `OTEL_EXPORTER_OTLP_METRICS_HEADERS` - Metrics-specific headers (overrides base)
|
||||
- `OTEL_EXPORTER_OTLP_LOGS_HEADERS` - Logs-specific headers (overrides base)
|
||||
|
||||
**Service Identification**:
|
||||
- `OTEL_SERVICE_NAME` - Service name (default: `agent_framework`)
|
||||
- `OTEL_SERVICE_VERSION` - Service version (default: package version)
|
||||
- `OTEL_RESOURCE_ATTRIBUTES` - Additional resource attributes (e.g., `key1=value1,key2=value2`)
|
||||
|
||||
> **Note**: These are standard OpenTelemetry environment variables. See the [OpenTelemetry spec](https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/) for more details.
|
||||
|
||||
#### Logging
|
||||
Agent Framework has a built-in logging configuration that works well with telemetry. It sets the format to a standard format that includes timestamp, pathname, line number, and log level. You can use that by calling the `setup_logging()` function from the `agent_framework` module.
|
||||
|
||||
```python
|
||||
from agent_framework import setup_logging
|
||||
|
||||
setup_logging()
|
||||
```
|
||||
You can control at what level logging happens and thus what logs get exported, you can do this, by adding this:
|
||||
|
||||
```python
|
||||
import logging
|
||||
|
||||
logger = logging.getLogger()
|
||||
logger.setLevel(logging.NOTSET)
|
||||
```
|
||||
This gets the root logger and sets the level of that, automatically other loggers inherit from that one, and you will get detailed logs in your telemetry.
|
||||
|
||||
## Samples
|
||||
|
||||
This folder contains different samples demonstrating how to use telemetry in various scenarios.
|
||||
|
||||
| Sample | Description |
|
||||
|--------|-------------|
|
||||
| [configure_otel_providers_with_parameters.py](./configure_otel_providers_with_parameters.py) | **Recommended starting point**: Shows how to create custom exporters with specific configuration and pass them to `configure_otel_providers()`. Useful for advanced scenarios. |
|
||||
| [configure_otel_providers_with_env_var.py](./configure_otel_providers_with_env_var.py) | Shows how to setup telemetry using standard OpenTelemetry environment variables (`OTEL_EXPORTER_OTLP_*`). |
|
||||
| [agent_observability.py](./agent_observability.py) | Shows telemetry collection for an agentic application with tool calls using environment variables. |
|
||||
| [agent_with_foundry_tracing.py](./agent_with_foundry_tracing.py) | Shows Azure Monitor integration with Foundry for any chat client. |
|
||||
| [azure_ai_agent_observability.py](./azure_ai_agent_observability.py) | Shows Azure Monitor integration for a AzureAIClient. |
|
||||
| [advanced_manual_setup_console_output.py](./advanced_manual_setup_console_output.py) | Advanced: Shows manual setup of exporters and providers with console output. Useful for understanding how observability works under the hood. |
|
||||
| [advanced_zero_code.py](./advanced_zero_code.py) | Advanced: Shows zero-code telemetry setup using the `opentelemetry-enable_instrumentation` CLI tool. |
|
||||
| [workflow_observability.py](./workflow_observability.py) | Shows telemetry collection for a workflow with multiple executors and message passing. |
|
||||
|
||||
### Running the samples
|
||||
|
||||
1. Open a terminal and navigate to this folder: `python/samples/02-agents/observability/`. This is necessary for the `.env` file to be read correctly.
|
||||
2. Create a `.env` file if one doesn't already exist in this folder. Please refer to the [example file](./.env.example).
|
||||
> **Note**: You can start with just `ENABLE_INSTRUMENTATION=true` and add `OTEL_EXPORTER_OTLP_ENDPOINT` or other configuration as needed. If no exporters are configured, you can set `ENABLE_CONSOLE_EXPORTERS=true` for console output.
|
||||
3. Activate your python virtual environment, and then run `python configure_otel_providers_with_env_var.py` or others.
|
||||
|
||||
> Each sample will print the Operation/Trace ID, which can be used later for filtering logs and traces in Application Insights or Aspire Dashboard.
|
||||
|
||||
# Appendix
|
||||
|
||||
## Azure Monitor Queries
|
||||
|
||||
When you are in Azure Monitor and want to have a overall view of the span, use this query in the logs section:
|
||||
|
||||
```kusto
|
||||
dependencies
|
||||
| where operation_Id in (dependencies
|
||||
| project operation_Id, timestamp
|
||||
| order by timestamp desc
|
||||
| summarize operations = make_set(operation_Id), timestamp = max(timestamp) by operation_Id
|
||||
| order by timestamp desc
|
||||
| project operation_Id
|
||||
| take 2)
|
||||
| evaluate bag_unpack(customDimensions)
|
||||
| extend tool_call_id = tostring(["gen_ai.tool.call.id"])
|
||||
| join kind=leftouter (customMetrics
|
||||
| extend tool_call_id = tostring(customDimensions['gen_ai.tool.call.id'])
|
||||
| where isnotempty(tool_call_id)
|
||||
| project tool_call_duration = value, tool_call_id)
|
||||
on tool_call_id
|
||||
| project-keep timestamp, target, operation_Id, tool_call_duration, duration, gen_ai*
|
||||
| order by timestamp asc
|
||||
```
|
||||
|
||||
### Grafana dashboards with Application Insights data
|
||||
Besides the Application Insights native UI, you can also use Grafana to visualize the telemetry data in Application Insights. There are two tailored dashboards for you to get started quickly:
|
||||
|
||||
#### Agent Overview dashboard
|
||||
Open dashboard in Azure portal: <https://aka.ms/amg/dash/af-agent>
|
||||
|
||||
|
||||
#### Workflow Overview dashboard
|
||||
Open dashboard in Azure portal: <https://aka.ms/amg/dash/af-workflow>
|
||||
|
||||
|
||||
## Migration Guide
|
||||
|
||||
We've done a major update to the observability API in Agent Framework Python SDK. The new API simplifies configuration by relying more on standard OpenTelemetry environment variables and have split the instrumentation from the configuration.
|
||||
|
||||
If you're updating from a previous version of the Agent Framework, here are the key changes to the observability API:
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Old Variable | New Variable | Notes |
|
||||
|-------------|--------------|-------|
|
||||
| `OTLP_ENDPOINT` | `OTEL_EXPORTER_OTLP_ENDPOINT` | Standard OpenTelemetry env var |
|
||||
| `APPLICATIONINSIGHTS_CONNECTION_STRING` | N/A | Use `configure_azure_monitor()` |
|
||||
| N/A | `ENABLE_CONSOLE_EXPORTERS` | New opt-in flag for console output |
|
||||
|
||||
### OTLP Configuration
|
||||
|
||||
**Before (Deprecated):**
|
||||
```
|
||||
from agent_framework.observability import setup_observability
|
||||
# Via parameter
|
||||
setup_observability(otlp_endpoint="http://localhost:4317")
|
||||
|
||||
# Via environment variable
|
||||
# OTLP_ENDPOINT=http://localhost:4317
|
||||
setup_observability()
|
||||
```
|
||||
|
||||
**After (Current):**
|
||||
```python
|
||||
from agent_framework.observability import configure_otel_providers
|
||||
# Via standard OTEL environment variable (recommended)
|
||||
# OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
|
||||
configure_otel_providers()
|
||||
|
||||
# Or via custom exporters
|
||||
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
|
||||
from opentelemetry.exporter.otlp.proto.grpc._log_exporter import OTLPLogExporter
|
||||
from opentelemetry.exporter.otlp.proto.grpc.metric_exporter import OTLPMetricExporter
|
||||
|
||||
configure_otel_providers(exporters=[
|
||||
OTLPSpanExporter(endpoint="http://localhost:4317"),
|
||||
OTLPLogExporter(endpoint="http://localhost:4317"),
|
||||
OTLPMetricExporter(endpoint="http://localhost:4317"),
|
||||
])
|
||||
```
|
||||
|
||||
### Azure Monitor Configuration
|
||||
|
||||
**Before (Deprecated):**
|
||||
```
|
||||
from agent_framework.observability import setup_observability
|
||||
|
||||
setup_observability(
|
||||
applicationinsights_connection_string="InstrumentationKey=...",
|
||||
applicationinsights_live_metrics=True,
|
||||
)
|
||||
```
|
||||
|
||||
**After (Current):**
|
||||
```python
|
||||
# For Azure AI projects
|
||||
from agent_framework.azure import AzureAIClient
|
||||
from azure.ai.projects.aio import AIProjectClient
|
||||
|
||||
async with (
|
||||
AIProjectClient(...) as project_client,
|
||||
AzureAIClient(project_client=project_client) as client,
|
||||
):
|
||||
await client.configure_azure_monitor(enable_live_metrics=True)
|
||||
|
||||
# For non-Azure AI projects
|
||||
from azure.monitor.opentelemetry import configure_azure_monitor
|
||||
from agent_framework.observability import create_resource, enable_instrumentation
|
||||
|
||||
configure_azure_monitor(
|
||||
connection_string="InstrumentationKey=...",
|
||||
resource=create_resource(),
|
||||
enable_live_metrics=True,
|
||||
)
|
||||
enable_instrumentation()
|
||||
```
|
||||
|
||||
### Console Output
|
||||
|
||||
**Before (Deprecated):**
|
||||
```
|
||||
from agent_framework.observability import setup_observability
|
||||
|
||||
# Console was used as automatic fallback
|
||||
setup_observability() # Would output to console if no exporters configured
|
||||
```
|
||||
|
||||
**After (Current):**
|
||||
```python
|
||||
from agent_framework.observability import configure_otel_providers
|
||||
|
||||
# Console exporters are now opt-in
|
||||
# ENABLE_CONSOLE_EXPORTERS=true
|
||||
configure_otel_providers()
|
||||
|
||||
# Or programmatically
|
||||
configure_otel_providers(enable_console_exporters=True)
|
||||
```
|
||||
|
||||
### Benefits of New API
|
||||
|
||||
1. **Standards Compliant**: Uses standard OpenTelemetry environment variables
|
||||
2. **Simpler**: Less configuration needed, more relies on environment
|
||||
3. **Flexible**: Easy to add custom exporters alongside environment-based ones
|
||||
4. **Cleaner Separation**: Azure Monitor setup is in Azure-specific client
|
||||
5. **Better Compatibility**: Works with any OTEL-compatible tool (Jaeger, Zipkin, Prometheus, etc.)
|
||||
|
||||
## Aspire Dashboard
|
||||
|
||||
The [Aspire Dashboard](https://learn.microsoft.com/en-us/dotnet/aspire/fundamentals/dashboard/standalone) is a local telemetry viewing tool that provides an excellent experience for viewing OpenTelemetry data without requiring Azure setup.
|
||||
|
||||
### Setting up Aspire Dashboard with Docker
|
||||
|
||||
The easiest way to run the Aspire Dashboard locally is using Docker:
|
||||
|
||||
```bash
|
||||
# Pull and run the Aspire Dashboard container
|
||||
docker run --rm -it -d \
|
||||
-p 18888:18888 \
|
||||
-p 4317:18889 \
|
||||
--name aspire-dashboard \
|
||||
mcr.microsoft.com/dotnet/aspire-dashboard:latest
|
||||
```
|
||||
|
||||
This will start the dashboard with:
|
||||
|
||||
- **Web UI**: Available at <http://localhost:18888>
|
||||
- **OTLP endpoint**: Available at `http://localhost:4317` for your applications to send telemetry data
|
||||
|
||||
### Configuring your application
|
||||
|
||||
Make sure your `.env` file includes the OTLP endpoint:
|
||||
|
||||
```bash
|
||||
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
|
||||
```
|
||||
|
||||
Or set it as an environment variable when running your samples:
|
||||
|
||||
```bash
|
||||
ENABLE_INSTRUMENTATION=true OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 python configure_otel_providers_with_env_var.py
|
||||
```
|
||||
|
||||
### Viewing telemetry data
|
||||
|
||||
> Make sure you have the dashboard running to receive telemetry data.
|
||||
|
||||
Once your sample finishes running, navigate to <http://localhost:18888> in a web browser to see the telemetry data. Follow the [Aspire Dashboard exploration guide](https://learn.microsoft.com/en-us/dotnet/aspire/fundamentals/dashboard/explore) to authenticate to the dashboard and start exploring your traces, logs, and metrics!
|
||||
@@ -0,0 +1,127 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
from random import randint
|
||||
from typing import Annotated
|
||||
|
||||
from agent_framework import tool
|
||||
from agent_framework.observability import enable_instrumentation
|
||||
from agent_framework.openai import OpenAIChatClient
|
||||
from opentelemetry._logs import set_logger_provider
|
||||
from opentelemetry.metrics import set_meter_provider
|
||||
from opentelemetry.sdk._logs import LoggerProvider, LoggingHandler
|
||||
from opentelemetry.sdk._logs.export import BatchLogRecordProcessor, ConsoleLogExporter
|
||||
from opentelemetry.sdk.metrics import MeterProvider
|
||||
from opentelemetry.sdk.metrics.export import ConsoleMetricExporter, PeriodicExportingMetricReader
|
||||
from opentelemetry.sdk.resources import Resource
|
||||
from opentelemetry.sdk.trace import TracerProvider
|
||||
from opentelemetry.sdk.trace.export import BatchSpanProcessor, ConsoleSpanExporter
|
||||
from opentelemetry.semconv._incubating.attributes.service_attributes import SERVICE_NAME
|
||||
from opentelemetry.trace import set_tracer_provider
|
||||
from pydantic import Field
|
||||
|
||||
"""
|
||||
This sample shows how to manually configure to send traces, logs, and metrics to the console,
|
||||
without using the `configure_otel_providers` helper function.
|
||||
"""
|
||||
|
||||
resource = Resource.create({SERVICE_NAME: "ManualSetup"})
|
||||
|
||||
|
||||
def setup_logging():
|
||||
# Create and set a global logger provider for the application.
|
||||
logger_provider = LoggerProvider(resource=resource)
|
||||
# Log processors are initialized with an exporter which is responsible
|
||||
logger_provider.add_log_record_processor(BatchLogRecordProcessor(ConsoleLogExporter()))
|
||||
# Sets the global default logger provider
|
||||
set_logger_provider(logger_provider)
|
||||
# Create a logging handler to write logging records, in OTLP format, to the exporter.
|
||||
handler = LoggingHandler()
|
||||
# Attach the handler to the root logger. `getLogger()` with no arguments returns the root logger.
|
||||
# Events from all child loggers will be processed by this handler.
|
||||
logger = logging.getLogger()
|
||||
logger.addHandler(handler)
|
||||
# Set the logging level to NOTSET to allow all records to be processed by the handler.
|
||||
logger.setLevel(logging.NOTSET)
|
||||
|
||||
|
||||
def setup_tracing():
|
||||
# Initialize a trace provider for the application. This is a factory for creating tracers.
|
||||
tracer_provider = TracerProvider(resource=resource)
|
||||
# Span processors are initialized with an exporter which is responsible
|
||||
# for sending the telemetry data to a particular backend.
|
||||
tracer_provider.add_span_processor(BatchSpanProcessor(ConsoleSpanExporter()))
|
||||
# Sets the global default tracer provider
|
||||
set_tracer_provider(tracer_provider)
|
||||
|
||||
|
||||
def setup_metrics():
|
||||
# Initialize a metric provider for the application. This is a factory for creating meters.
|
||||
meter_provider = MeterProvider(
|
||||
metric_readers=[PeriodicExportingMetricReader(ConsoleMetricExporter(), export_interval_millis=5000)],
|
||||
resource=resource,
|
||||
)
|
||||
# Sets the global default meter provider
|
||||
set_meter_provider(meter_provider)
|
||||
|
||||
|
||||
# NOTE: approval_mode="never_require" is for sample brevity. Use "always_require" in production; see samples/02-agents/tools/function_tool_with_approval.py and samples/02-agents/tools/function_tool_with_approval_and_threads.py.
|
||||
@tool(approval_mode="never_require")
|
||||
async def get_weather(
|
||||
location: Annotated[str, Field(description="The location to get the weather for.")],
|
||||
) -> str:
|
||||
"""Get the weather for a given location."""
|
||||
await asyncio.sleep(randint(0, 10) / 10.0) # Simulate a network call
|
||||
conditions = ["sunny", "cloudy", "rainy", "stormy"]
|
||||
return f"The weather in {location} is {conditions[randint(0, 3)]} with a high of {randint(10, 30)}°C."
|
||||
|
||||
|
||||
async def run_chat_client() -> None:
|
||||
"""Run an AI service.
|
||||
|
||||
This function runs an AI service and prints the output.
|
||||
Telemetry will be collected for the service execution behind the scenes,
|
||||
and the traces will be sent to the configured telemetry backend.
|
||||
|
||||
The telemetry will include information about the AI service execution.
|
||||
|
||||
Args:
|
||||
stream: Whether to use streaming for the plugin
|
||||
|
||||
Remarks:
|
||||
When function calling is outside the open telemetry loop
|
||||
each of the call to the model is handled as a seperate span,
|
||||
while when the open telemetry is put last, a single span
|
||||
is shown, which might include one or more rounds of function calling.
|
||||
|
||||
So for the scenario below, you should see the following:
|
||||
|
||||
2 spans with gen_ai.operation.name=chat
|
||||
The first has finish_reason "tool_calls"
|
||||
The second has finish_reason "stop"
|
||||
2 spans with gen_ai.operation.name=execute_tool
|
||||
|
||||
"""
|
||||
client = OpenAIChatClient()
|
||||
message = "What's the weather in Amsterdam and in Paris?"
|
||||
print(f"User: {message}")
|
||||
print("Assistant: ", end="")
|
||||
async for chunk in client.get_response(message, tools=get_weather, stream=True):
|
||||
if str(chunk):
|
||||
print(str(chunk), end="")
|
||||
print("")
|
||||
|
||||
|
||||
async def main():
|
||||
"""Run the selected scenario(s)."""
|
||||
setup_logging()
|
||||
setup_tracing()
|
||||
setup_metrics()
|
||||
enable_instrumentation()
|
||||
|
||||
await run_chat_client()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,104 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
import asyncio
|
||||
from random import randint
|
||||
from typing import TYPE_CHECKING, Annotated
|
||||
|
||||
from agent_framework import tool
|
||||
from agent_framework.observability import get_tracer
|
||||
from agent_framework.openai import OpenAIResponsesClient
|
||||
from opentelemetry.trace import SpanKind
|
||||
from opentelemetry.trace.span import format_trace_id
|
||||
from pydantic import Field
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from agent_framework import SupportsChatGetResponse
|
||||
|
||||
|
||||
"""
|
||||
This sample shows how you can configure observability of an application with zero code changes.
|
||||
It relies on the OpenTelemetry auto-instrumentation capabilities, and the observability setup
|
||||
is done via environment variables.
|
||||
|
||||
Follow the install guidance from https://opentelemetry.io/docs/zero-code/python/ to install the OpenTelemetry CLI tool.
|
||||
|
||||
And setup a local OpenTelemetry Collector instance to receive the traces and metrics (and update the endpoint below).
|
||||
|
||||
Then you can run:
|
||||
```bash
|
||||
opentelemetry-enable_instrumentation \
|
||||
--traces_exporter otlp \
|
||||
--metrics_exporter otlp \
|
||||
--service_name agent_framework \
|
||||
--exporter_otlp_endpoint http://localhost:4317 \
|
||||
python python/samples/02-agents/observability/advanced_zero_code.py
|
||||
```
|
||||
(or use uv run in front when you've done the install within your uv virtual environment)
|
||||
|
||||
You can also set the environment variables instead of passing them as CLI arguments.
|
||||
|
||||
"""
|
||||
|
||||
|
||||
# NOTE: approval_mode="never_require" is for sample brevity. Use "always_require" in production; see samples/02-agents/tools/function_tool_with_approval.py and samples/02-agents/tools/function_tool_with_approval_and_threads.py.
|
||||
@tool(approval_mode="never_require")
|
||||
async def get_weather(
|
||||
location: Annotated[str, Field(description="The location to get the weather for.")],
|
||||
) -> str:
|
||||
"""Get the weather for a given location."""
|
||||
await asyncio.sleep(randint(0, 10) / 10.0) # Simulate a network call
|
||||
conditions = ["sunny", "cloudy", "rainy", "stormy"]
|
||||
return f"The weather in {location} is {conditions[randint(0, 3)]} with a high of {randint(10, 30)}°C."
|
||||
|
||||
|
||||
async def run_chat_client(client: "SupportsChatGetResponse", stream: bool = False) -> None:
|
||||
"""Run an AI service.
|
||||
|
||||
This function runs an AI service and prints the output.
|
||||
Telemetry will be collected for the service execution behind the scenes,
|
||||
and the traces will be sent to the configured telemetry backend.
|
||||
|
||||
The telemetry will include information about the AI service execution.
|
||||
|
||||
Args:
|
||||
stream: Whether to use streaming for the plugin
|
||||
|
||||
Remarks:
|
||||
When function calling is outside the open telemetry loop
|
||||
each of the call to the model is handled as a separate span,
|
||||
while when the open telemetry is put last, a single span
|
||||
is shown, which might include one or more rounds of function calling.
|
||||
|
||||
So for the scenario below, you should see the following:
|
||||
|
||||
2 spans with gen_ai.operation.name=chat
|
||||
The first has finish_reason "tool_calls"
|
||||
The second has finish_reason "stop"
|
||||
2 spans with gen_ai.operation.name=execute_tool
|
||||
|
||||
"""
|
||||
message = "What's the weather in Amsterdam and in Paris?"
|
||||
print(f"User: {message}")
|
||||
if stream:
|
||||
print("Assistant: ", end="")
|
||||
async for chunk in client.get_response(message, tools=get_weather, stream=True):
|
||||
if str(chunk):
|
||||
print(str(chunk), end="")
|
||||
print("")
|
||||
else:
|
||||
response = await client.get_response(message, tools=get_weather)
|
||||
print(f"Assistant: {response}")
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
with get_tracer().start_as_current_span("Zero Code", kind=SpanKind.CLIENT) as current_span:
|
||||
print(f"Trace ID: {format_trace_id(current_span.get_span_context().trace_id)}")
|
||||
|
||||
client = OpenAIResponsesClient()
|
||||
|
||||
await run_chat_client(client, stream=True)
|
||||
await run_chat_client(client, stream=False)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,63 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
import asyncio
|
||||
from random import randint
|
||||
from typing import Annotated
|
||||
|
||||
from agent_framework import Agent, tool
|
||||
from agent_framework.observability import configure_otel_providers, get_tracer
|
||||
from agent_framework.openai import OpenAIChatClient
|
||||
from opentelemetry.trace import SpanKind
|
||||
from opentelemetry.trace.span import format_trace_id
|
||||
from pydantic import Field
|
||||
|
||||
"""
|
||||
This sample shows how you can observe an agent in Agent Framework by using the
|
||||
same observability setup function.
|
||||
"""
|
||||
|
||||
|
||||
# NOTE: approval_mode="never_require" is for sample brevity. Use "always_require" in production; see samples/02-agents/tools/function_tool_with_approval.py and samples/02-agents/tools/function_tool_with_approval_and_threads.py.
|
||||
@tool(approval_mode="never_require")
|
||||
async def get_weather(
|
||||
location: Annotated[str, Field(description="The location to get the weather for.")],
|
||||
) -> str:
|
||||
"""Get the weather for a given location."""
|
||||
await asyncio.sleep(randint(0, 10) / 10.0) # Simulate a network call
|
||||
conditions = ["sunny", "cloudy", "rainy", "stormy"]
|
||||
return f"The weather in {location} is {conditions[randint(0, 3)]} with a high of {randint(10, 30)}°C."
|
||||
|
||||
|
||||
async def main():
|
||||
# calling `configure_otel_providers` will *enable* tracing and create the necessary tracing, logging
|
||||
# and metrics providers based on environment variables.
|
||||
# See the .env.example file for the available configuration options.
|
||||
configure_otel_providers()
|
||||
|
||||
questions = ["What's the weather in Amsterdam?", "and in Paris, and which is better?", "Why is the sky blue?"]
|
||||
|
||||
with get_tracer().start_as_current_span("Scenario: Agent Chat", kind=SpanKind.CLIENT) as current_span:
|
||||
print(f"Trace ID: {format_trace_id(current_span.get_span_context().trace_id)}")
|
||||
|
||||
agent = Agent(
|
||||
client=OpenAIChatClient(),
|
||||
tools=get_weather,
|
||||
name="WeatherAgent",
|
||||
instructions="You are a weather assistant.",
|
||||
id="weather-agent",
|
||||
)
|
||||
thread = agent.get_new_thread()
|
||||
for question in questions:
|
||||
print(f"\nUser: {question}")
|
||||
print(f"{agent.name}: ", end="")
|
||||
async for update in agent.run(
|
||||
question,
|
||||
thread=thread,
|
||||
stream=True,
|
||||
):
|
||||
if update.text:
|
||||
print(update.text, end="")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,105 @@
|
||||
# /// script
|
||||
# requires-python = ">=3.10"
|
||||
# dependencies = [
|
||||
# "azure-monitor-opentelemetry",
|
||||
# ]
|
||||
# ///
|
||||
# Run with any PEP 723 compatible runner, e.g.:
|
||||
# uv run python/samples/02-agents/observability/agent_with_foundry_tracing.py
|
||||
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import os
|
||||
from random import randint
|
||||
from typing import Annotated
|
||||
|
||||
import dotenv
|
||||
from agent_framework import Agent, tool
|
||||
from agent_framework.observability import create_resource, enable_instrumentation, get_tracer
|
||||
from agent_framework.openai import OpenAIResponsesClient
|
||||
from azure.ai.projects.aio import AIProjectClient
|
||||
from azure.identity.aio import AzureCliCredential
|
||||
from azure.monitor.opentelemetry import configure_azure_monitor
|
||||
from opentelemetry.trace import SpanKind
|
||||
from opentelemetry.trace.span import format_trace_id
|
||||
from pydantic import Field
|
||||
|
||||
"""
|
||||
This sample shows you can can setup telemetry in Microsoft Foundry for a custom agent.
|
||||
First ensure you have a Foundry workspace with Application Insights enabled.
|
||||
And use the Operate tab to Register an Agent.
|
||||
Set the OpenTelemetry agent ID to the value used below in the Agent creation: `weather-agent` (or change both).
|
||||
The sample uses the Azure Monitor OpenTelemetry exporter to send traces to Application Insights.
|
||||
So ensure you have the `azure-monitor-opentelemetry` package installed.
|
||||
"""
|
||||
|
||||
# For loading the `AZURE_AI_PROJECT_ENDPOINT` environment variable
|
||||
dotenv.load_dotenv()
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# NOTE: approval_mode="never_require" is for sample brevity. Use "always_require" in production; see samples/02-agents/tools/function_tool_with_approval.py and samples/02-agents/tools/function_tool_with_approval_and_threads.py.
|
||||
@tool(approval_mode="never_require")
|
||||
async def get_weather(
|
||||
location: Annotated[str, Field(description="The location to get the weather for.")],
|
||||
) -> str:
|
||||
"""Get the weather for a given location."""
|
||||
await asyncio.sleep(randint(0, 10) / 10.0) # Simulate a network call
|
||||
conditions = ["sunny", "cloudy", "rainy", "stormy"]
|
||||
return f"The weather in {location} is {conditions[randint(0, 3)]} with a high of {randint(10, 30)}°C."
|
||||
|
||||
|
||||
async def main():
|
||||
async with (
|
||||
AzureCliCredential() as credential,
|
||||
AIProjectClient(endpoint=os.environ["AZURE_AI_PROJECT_ENDPOINT"], credential=credential) as project_client,
|
||||
):
|
||||
# This will enable tracing and configure the application to send telemetry data to the
|
||||
# Application Insights instance attached to the Azure AI project.
|
||||
# This will override any existing configuration.
|
||||
try:
|
||||
conn_string = await project_client.telemetry.get_application_insights_connection_string()
|
||||
except Exception:
|
||||
logger.warning(
|
||||
"No Application Insights connection string found for the Azure AI Project. "
|
||||
"Please ensure Application Insights is configured in your Azure AI project, "
|
||||
"or call configure_otel_providers() manually with custom exporters."
|
||||
)
|
||||
return
|
||||
configure_azure_monitor(
|
||||
connection_string=conn_string,
|
||||
enable_live_metrics=True,
|
||||
resource=create_resource(),
|
||||
enable_performance_counters=False,
|
||||
)
|
||||
# This call is not necessary if you have the environment variable ENABLE_INSTRUMENTATION=true set
|
||||
# If not or set to false, or if you want to enable or disable sensitive data collection, call this function.
|
||||
enable_instrumentation(enable_sensitive_data=True)
|
||||
print("Observability is set up. Starting Weather Agent...")
|
||||
|
||||
questions = ["What's the weather in Amsterdam?", "and in Paris, and which is better?", "Why is the sky blue?"]
|
||||
|
||||
with get_tracer().start_as_current_span("Weather Agent Chat", kind=SpanKind.CLIENT) as current_span:
|
||||
print(f"Trace ID: {format_trace_id(current_span.get_span_context().trace_id)}")
|
||||
|
||||
agent = Agent(
|
||||
client=OpenAIResponsesClient(),
|
||||
tools=get_weather,
|
||||
name="WeatherAgent",
|
||||
instructions="You are a weather assistant.",
|
||||
id="weather-agent",
|
||||
)
|
||||
thread = agent.get_new_thread()
|
||||
for question in questions:
|
||||
print(f"\nUser: {question}")
|
||||
print(f"{agent.name}: ", end="")
|
||||
async for update in agent.run(question, thread=thread, stream=True):
|
||||
if update.text:
|
||||
print(update.text, end="")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,76 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
from random import randint
|
||||
from typing import Annotated
|
||||
|
||||
import dotenv
|
||||
from agent_framework import Agent, tool
|
||||
from agent_framework.azure import AzureAIClient
|
||||
from agent_framework.observability import get_tracer
|
||||
from azure.ai.projects.aio import AIProjectClient
|
||||
from azure.identity.aio import AzureCliCredential
|
||||
from opentelemetry.trace import SpanKind
|
||||
from opentelemetry.trace.span import format_trace_id
|
||||
from pydantic import Field
|
||||
|
||||
"""
|
||||
This sample shows you can setup telemetry for an Azure AI agent.
|
||||
It uses the Azure AI client to setup the telemetry, this calls out to
|
||||
Azure AI for the connection string of the attached Application Insights
|
||||
instance.
|
||||
|
||||
You must add an Application Insights instance to your Azure AI project
|
||||
for this sample to work.
|
||||
"""
|
||||
|
||||
# For loading the `AZURE_AI_PROJECT_ENDPOINT` environment variable
|
||||
dotenv.load_dotenv()
|
||||
|
||||
|
||||
# NOTE: approval_mode="never_require" is for sample brevity. Use "always_require" in production; see samples/02-agents/tools/function_tool_with_approval.py and samples/02-agents/tools/function_tool_with_approval_and_threads.py.
|
||||
@tool(approval_mode="never_require")
|
||||
async def get_weather(
|
||||
location: Annotated[str, Field(description="The location to get the weather for.")],
|
||||
) -> str:
|
||||
"""Get the weather for a given location."""
|
||||
await asyncio.sleep(randint(0, 10) / 10.0) # Simulate a network call
|
||||
conditions = ["sunny", "cloudy", "rainy", "stormy"]
|
||||
return f"The weather in {location} is {conditions[randint(0, 3)]} with a high of {randint(10, 30)}°C."
|
||||
|
||||
|
||||
async def main():
|
||||
async with (
|
||||
AzureCliCredential() as credential,
|
||||
AIProjectClient(endpoint=os.environ["AZURE_AI_PROJECT_ENDPOINT"], credential=credential) as project_client,
|
||||
AzureAIClient(project_client=project_client) as client,
|
||||
):
|
||||
# This will enable tracing and configure the application to send telemetry data to the
|
||||
# Application Insights instance attached to the Azure AI project.
|
||||
# This will override any existing configuration.
|
||||
await client.configure_azure_monitor(enable_live_metrics=True)
|
||||
|
||||
questions = ["What's the weather in Amsterdam?", "and in Paris, and which is better?", "Why is the sky blue?"]
|
||||
|
||||
with get_tracer().start_as_current_span("Single Agent Chat", kind=SpanKind.CLIENT) as current_span:
|
||||
print(f"Trace ID: {format_trace_id(current_span.get_span_context().trace_id)}")
|
||||
|
||||
agent = Agent(
|
||||
client=client,
|
||||
tools=get_weather,
|
||||
name="WeatherAgent",
|
||||
instructions="You are a weather assistant.",
|
||||
id="edvan-weather-agent",
|
||||
)
|
||||
thread = agent.get_new_thread()
|
||||
for question in questions:
|
||||
print(f"\nUser: {question}")
|
||||
print(f"{agent.name}: ", end="")
|
||||
async for update in agent.run(question, thread=thread, stream=True):
|
||||
if update.text:
|
||||
print(update.text, end="")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,136 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
import argparse
|
||||
import asyncio
|
||||
from contextlib import suppress
|
||||
from random import randint
|
||||
from typing import TYPE_CHECKING, Annotated, Literal
|
||||
|
||||
from agent_framework import tool
|
||||
from agent_framework.observability import configure_otel_providers, get_tracer
|
||||
from agent_framework.openai import OpenAIResponsesClient
|
||||
from opentelemetry import trace
|
||||
from opentelemetry.trace.span import format_trace_id
|
||||
from pydantic import Field
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from agent_framework import SupportsChatGetResponse
|
||||
|
||||
"""
|
||||
This sample shows how you can configure observability of an application via the
|
||||
`configure_otel_providers` function with environment variables.
|
||||
|
||||
When you run this sample with an OTLP endpoint or an Application Insights connection string,
|
||||
you should see traces, logs, and metrics in the configured backend.
|
||||
|
||||
If no OTLP endpoint or Application Insights connection string is configured, the sample will
|
||||
output traces, logs, and metrics to the console.
|
||||
"""
|
||||
|
||||
# Define the scenarios that can be run to show the telemetry data collected by the SDK
|
||||
SCENARIOS = ["client", "client_stream", "tool", "all"]
|
||||
|
||||
|
||||
# NOTE: approval_mode="never_require" is for sample brevity. Use "always_require" in production; see samples/02-agents/tools/function_tool_with_approval.py and samples/02-agents/tools/function_tool_with_approval_and_threads.py.
|
||||
@tool(approval_mode="never_require")
|
||||
async def get_weather(
|
||||
location: Annotated[str, Field(description="The location to get the weather for.")],
|
||||
) -> str:
|
||||
"""Get the weather for a given location."""
|
||||
await asyncio.sleep(randint(0, 10) / 10.0) # Simulate a network call
|
||||
conditions = ["sunny", "cloudy", "rainy", "stormy"]
|
||||
return f"The weather in {location} is {conditions[randint(0, 3)]} with a high of {randint(10, 30)}°C."
|
||||
|
||||
|
||||
async def run_chat_client(client: "SupportsChatGetResponse", stream: bool = False) -> None:
|
||||
"""Run an AI service.
|
||||
|
||||
This function runs an AI service and prints the output.
|
||||
Telemetry will be collected for the service execution behind the scenes,
|
||||
and the traces will be sent to the configured telemetry backend.
|
||||
|
||||
The telemetry will include information about the AI service execution.
|
||||
|
||||
Args:
|
||||
client: The chat client to use.
|
||||
stream: Whether to use streaming for the response
|
||||
|
||||
Remarks:
|
||||
For the scenario below, you should see the following:
|
||||
1 Client span, with 4 children:
|
||||
2 Internal span with gen_ai.operation.name=chat
|
||||
The first has finish_reason "tool_calls"
|
||||
The second has finish_reason "stop"
|
||||
2 Internal span with gen_ai.operation.name=execute_tool
|
||||
|
||||
"""
|
||||
scenario_name = "Chat Client Stream" if stream else "Chat Client"
|
||||
with get_tracer().start_as_current_span(name=f"Scenario: {scenario_name}", kind=trace.SpanKind.CLIENT):
|
||||
print("Running scenario:", scenario_name)
|
||||
message = "What's the weather in Amsterdam and in Paris?"
|
||||
print(f"User: {message}")
|
||||
if stream:
|
||||
print("Assistant: ", end="")
|
||||
async for chunk in client.get_response(message, tools=get_weather, stream=True):
|
||||
if str(chunk):
|
||||
print(str(chunk), end="")
|
||||
print("")
|
||||
else:
|
||||
response = await client.get_response(message, tools=get_weather)
|
||||
print(f"Assistant: {response}")
|
||||
|
||||
|
||||
async def run_tool() -> None:
|
||||
"""Run a AI function.
|
||||
|
||||
This function runs a AI function and prints the output.
|
||||
Telemetry will be collected for the function execution behind the scenes,
|
||||
and the traces will be sent to the configured telemetry backend.
|
||||
|
||||
The telemetry will include information about the AI function execution
|
||||
and the AI service execution.
|
||||
"""
|
||||
with get_tracer().start_as_current_span("Scenario: AI Function", kind=trace.SpanKind.CLIENT):
|
||||
print("Running scenario: AI Function")
|
||||
func = tool(get_weather)
|
||||
weather = await func.invoke(location="Amsterdam")
|
||||
print(f"Weather in Amsterdam:\n{weather}")
|
||||
|
||||
|
||||
async def main(scenario: Literal["client", "client_stream", "tool", "all"] = "all"):
|
||||
"""Run the selected scenario(s)."""
|
||||
|
||||
# This will enable tracing and create the necessary tracing, logging and metrics providers
|
||||
# based on environment variables. See the .env.example file for the available configuration options.
|
||||
configure_otel_providers()
|
||||
|
||||
with get_tracer().start_as_current_span("Sample Scenarios", kind=trace.SpanKind.CLIENT) as current_span:
|
||||
print(f"Trace ID: {format_trace_id(current_span.get_span_context().trace_id)}")
|
||||
|
||||
client = OpenAIResponsesClient()
|
||||
|
||||
# Scenarios where telemetry is collected in the SDK, from the most basic to the most complex.
|
||||
if scenario == "tool" or scenario == "all":
|
||||
with suppress(Exception):
|
||||
await run_tool()
|
||||
if scenario == "client_stream" or scenario == "all":
|
||||
with suppress(Exception):
|
||||
await run_chat_client(client, stream=True)
|
||||
if scenario == "client" or scenario == "all":
|
||||
with suppress(Exception):
|
||||
await run_chat_client(client, stream=False)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
arg_parser = argparse.ArgumentParser()
|
||||
|
||||
arg_parser.add_argument(
|
||||
"--scenario",
|
||||
type=str,
|
||||
choices=SCENARIOS,
|
||||
default="all",
|
||||
help="The scenario to run. Default is all.",
|
||||
)
|
||||
|
||||
args = arg_parser.parse_args()
|
||||
asyncio.run(main(args.scenario))
|
||||
@@ -0,0 +1,171 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
import argparse
|
||||
import asyncio
|
||||
from contextlib import suppress
|
||||
from random import randint
|
||||
from typing import TYPE_CHECKING, Annotated, Literal
|
||||
|
||||
from agent_framework import setup_logging, tool
|
||||
from agent_framework.observability import configure_otel_providers, get_tracer
|
||||
from agent_framework.openai import OpenAIResponsesClient
|
||||
from opentelemetry import trace
|
||||
from opentelemetry.trace.span import format_trace_id
|
||||
from pydantic import Field
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from agent_framework import SupportsChatGetResponse
|
||||
|
||||
"""
|
||||
This sample shows how you can configure observability with custom exporters passed directly
|
||||
to the `configure_otel_providers()` function.
|
||||
|
||||
This approach gives you full control over exporter configuration (endpoints, headers, compression, etc.)
|
||||
and allows you to add multiple exporters programmatically.
|
||||
|
||||
For standard OTLP setup, it's recommended to use environment variables (see configure_otel_providers_with_env_var.py).
|
||||
Use this approach when you need custom exporter configuration beyond what environment variables provide.
|
||||
"""
|
||||
|
||||
# Define the scenarios that can be run to show the telemetry data collected by the SDK
|
||||
SCENARIOS = ["client", "client_stream", "tool", "all"]
|
||||
|
||||
|
||||
# NOTE: approval_mode="never_require" is for sample brevity. Use "always_require" in production; see samples/02-agents/tools/function_tool_with_approval.py and samples/02-agents/tools/function_tool_with_approval_and_threads.py.
|
||||
@tool(approval_mode="never_require")
|
||||
async def get_weather(
|
||||
location: Annotated[str, Field(description="The location to get the weather for.")],
|
||||
) -> str:
|
||||
"""Get the weather for a given location."""
|
||||
await asyncio.sleep(randint(0, 10) / 10.0) # Simulate a network call
|
||||
conditions = ["sunny", "cloudy", "rainy", "stormy"]
|
||||
return f"The weather in {location} is {conditions[randint(0, 3)]} with a high of {randint(10, 30)}°C."
|
||||
|
||||
|
||||
async def run_chat_client(client: "SupportsChatGetResponse", stream: bool = False) -> None:
|
||||
"""Run an AI service.
|
||||
|
||||
This function runs an AI service and prints the output.
|
||||
Telemetry will be collected for the service execution behind the scenes,
|
||||
and the traces will be sent to the configured telemetry backend.
|
||||
|
||||
The telemetry will include information about the AI service execution.
|
||||
|
||||
Args:
|
||||
client: The chat client to use.
|
||||
stream: Whether to use streaming for the response
|
||||
|
||||
Remarks:
|
||||
For the scenario below, you should see the following:
|
||||
1 Client span, with 4 children:
|
||||
2 Internal span with gen_ai.operation.name=chat
|
||||
The first has finish_reason "tool_calls"
|
||||
The second has finish_reason "stop"
|
||||
2 Internal span with gen_ai.operation.name=execute_tool
|
||||
|
||||
"""
|
||||
scenario_name = "Chat Client Stream" if stream else "Chat Client"
|
||||
with get_tracer().start_as_current_span(name=f"Scenario: {scenario_name}", kind=trace.SpanKind.CLIENT):
|
||||
print("Running scenario:", scenario_name)
|
||||
message = "What's the weather in Amsterdam and in Paris?"
|
||||
print(f"User: {message}")
|
||||
if stream:
|
||||
print("Assistant: ", end="")
|
||||
async for chunk in client.get_response(message, stream=True, tools=get_weather):
|
||||
if str(chunk):
|
||||
print(str(chunk), end="")
|
||||
print("")
|
||||
else:
|
||||
response = await client.get_response(message, tools=get_weather)
|
||||
print(f"Assistant: {response}")
|
||||
|
||||
|
||||
async def run_tool() -> None:
|
||||
"""Run a AI function.
|
||||
|
||||
This function runs a AI function and prints the output.
|
||||
Telemetry will be collected for the function execution behind the scenes,
|
||||
and the traces will be sent to the configured telemetry backend.
|
||||
|
||||
The telemetry will include information about the AI function execution
|
||||
and the AI service execution.
|
||||
"""
|
||||
with get_tracer().start_as_current_span("Scenario: AI Function", kind=trace.SpanKind.CLIENT):
|
||||
print("Running scenario: AI Function")
|
||||
func = tool(get_weather)
|
||||
weather = await func.invoke(location="Amsterdam")
|
||||
print(f"Weather in Amsterdam:\n{weather}")
|
||||
|
||||
|
||||
async def main(scenario: Literal["client", "client_stream", "tool", "all"] = "all"):
|
||||
"""Run the selected scenario(s)."""
|
||||
|
||||
# Setup the logging with the more complete format
|
||||
setup_logging()
|
||||
|
||||
# Create custom OTLP exporters with specific configuration
|
||||
# Note: You need to install opentelemetry-exporter-otlp-proto-grpc or -http separately
|
||||
try:
|
||||
from opentelemetry.exporter.otlp.proto.grpc._log_exporter import ( # pyright: ignore[reportMissingImports]
|
||||
OTLPLogExporter,
|
||||
)
|
||||
from opentelemetry.exporter.otlp.proto.grpc.metric_exporter import ( # pyright: ignore[reportMissingImports]
|
||||
OTLPMetricExporter,
|
||||
)
|
||||
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import ( # pyright: ignore[reportMissingImports]
|
||||
OTLPSpanExporter,
|
||||
)
|
||||
|
||||
# Create exporters with custom configuration
|
||||
# These will be added to any exporters configured via environment variables
|
||||
custom_exporters = [
|
||||
OTLPSpanExporter(endpoint="http://localhost:4317"),
|
||||
OTLPMetricExporter(endpoint="http://localhost:4317"),
|
||||
OTLPLogExporter(endpoint="http://localhost:4317"),
|
||||
]
|
||||
except ImportError:
|
||||
print(
|
||||
"Warning: opentelemetry-exporter-otlp-proto-grpc not installed. "
|
||||
"Install with: pip install opentelemetry-exporter-otlp-proto-grpc"
|
||||
)
|
||||
print("Continuing without custom exporters...\n")
|
||||
custom_exporters = []
|
||||
|
||||
# Setup observability with custom exporters and sensitive data enabled
|
||||
# The exporters parameter allows you to add custom exporters alongside
|
||||
# those configured via environment variables (OTEL_EXPORTER_OTLP_*)
|
||||
configure_otel_providers(
|
||||
enable_sensitive_data=True,
|
||||
exporters=custom_exporters,
|
||||
)
|
||||
|
||||
with get_tracer().start_as_current_span("Sample Scenarios", kind=trace.SpanKind.CLIENT) as current_span:
|
||||
print(f"Trace ID: {format_trace_id(current_span.get_span_context().trace_id)}")
|
||||
|
||||
client = OpenAIResponsesClient()
|
||||
|
||||
# Scenarios where telemetry is collected in the SDK, from the most basic to the most complex.
|
||||
if scenario == "tool" or scenario == "all":
|
||||
with suppress(Exception):
|
||||
await run_tool()
|
||||
if scenario == "client_stream" or scenario == "all":
|
||||
with suppress(Exception):
|
||||
await run_chat_client(client, stream=True)
|
||||
if scenario == "client" or scenario == "all":
|
||||
with suppress(Exception):
|
||||
await run_chat_client(client, stream=False)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
arg_parser = argparse.ArgumentParser()
|
||||
|
||||
arg_parser.add_argument(
|
||||
"--scenario",
|
||||
type=str,
|
||||
choices=SCENARIOS,
|
||||
default="all",
|
||||
help="The scenario to run. Default is all.",
|
||||
)
|
||||
|
||||
args = arg_parser.parse_args()
|
||||
asyncio.run(main(args.scenario))
|
||||
@@ -0,0 +1,116 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
import asyncio
|
||||
|
||||
from agent_framework import (
|
||||
Executor,
|
||||
WorkflowBuilder,
|
||||
WorkflowContext,
|
||||
handler,
|
||||
)
|
||||
from agent_framework.observability import configure_otel_providers, get_tracer
|
||||
from opentelemetry.trace import SpanKind
|
||||
from opentelemetry.trace.span import format_trace_id
|
||||
from typing_extensions import Never
|
||||
|
||||
"""
|
||||
This sample shows the telemetry collected when running a Agent Framework workflow.
|
||||
|
||||
This simple workflow consists of two executors arranged sequentially:
|
||||
1. An executor that converts input text to uppercase.
|
||||
2. An executor that reverses the uppercase text.
|
||||
|
||||
The workflow receives an initial string message, processes it through the two executors,
|
||||
and yields the final result.
|
||||
|
||||
Telemetry data that the workflow system emits includes:
|
||||
- Overall workflow build & execution spans
|
||||
- workflow.build (events: build.started, build.validation_completed, build.completed, edge_group.process)
|
||||
- workflow.run (events: workflow.started, workflow.completed or workflow.error)
|
||||
- Individual executor processing spans
|
||||
- executor.process (for each executor invocation)
|
||||
- Message publishing between executors
|
||||
- message.send (for each outbound message)
|
||||
|
||||
Prerequisites:
|
||||
- Basic understanding of workflow executors, edges, and messages.
|
||||
- Basic understanding of OpenTelemetry concepts like spans and traces.
|
||||
"""
|
||||
|
||||
|
||||
# Executors for sequential workflow
|
||||
class UpperCaseExecutor(Executor):
|
||||
"""An executor that converts text to uppercase."""
|
||||
|
||||
@handler
|
||||
async def to_upper_case(self, text: str, ctx: WorkflowContext[str]) -> None:
|
||||
"""Execute the task by converting the input string to uppercase."""
|
||||
print(f"UpperCaseExecutor: Processing '{text}'")
|
||||
result = text.upper()
|
||||
print(f"UpperCaseExecutor: Result '{result}'")
|
||||
|
||||
# Send the result to the next executor in the workflow.
|
||||
await ctx.send_message(result)
|
||||
|
||||
|
||||
class ReverseTextExecutor(Executor):
|
||||
"""An executor that reverses text."""
|
||||
|
||||
@handler
|
||||
async def reverse_text(self, text: str, ctx: WorkflowContext[Never, str]) -> None:
|
||||
"""Execute the task by reversing the input string."""
|
||||
print(f"ReverseTextExecutor: Processing '{text}'")
|
||||
result = text[::-1]
|
||||
print(f"ReverseTextExecutor: Result '{result}'")
|
||||
|
||||
# Yield the output.
|
||||
await ctx.yield_output(result)
|
||||
|
||||
|
||||
async def run_sequential_workflow() -> None:
|
||||
"""Run a simple sequential workflow demonstrating telemetry collection.
|
||||
|
||||
This workflow processes a string through two executors in sequence:
|
||||
1. UpperCaseExecutor converts the input to uppercase
|
||||
2. ReverseTextExecutor reverses the string and completes the workflow
|
||||
"""
|
||||
# Step 1: Create the executors.
|
||||
upper_case_executor = UpperCaseExecutor(id="upper_case_executor")
|
||||
reverse_text_executor = ReverseTextExecutor(id="reverse_text_executor")
|
||||
|
||||
# Step 2: Build the workflow with the defined edges.
|
||||
workflow = (
|
||||
WorkflowBuilder(start_executor=upper_case_executor)
|
||||
.add_edge(upper_case_executor, reverse_text_executor)
|
||||
.build()
|
||||
)
|
||||
|
||||
# Step 3: Run the workflow with an initial message.
|
||||
input_text = "hello world"
|
||||
print(f"Starting workflow with input: '{input_text}'")
|
||||
|
||||
output_event = None
|
||||
async for event in workflow.run("Hello world", stream=True):
|
||||
if event.type == "output":
|
||||
# The WorkflowOutputEvent contains the final result.
|
||||
output_event = event
|
||||
|
||||
if output_event:
|
||||
print(f"Workflow completed with result: '{output_event.data}'")
|
||||
|
||||
|
||||
async def main():
|
||||
"""Run the telemetry sample with a simple sequential workflow."""
|
||||
# This will enable tracing and create the necessary tracing, logging and metrics providers
|
||||
# based on environment variables. See the .env.example file for the available configuration options.
|
||||
configure_otel_providers()
|
||||
|
||||
with get_tracer().start_as_current_span("Sequential Workflow Scenario", kind=SpanKind.CLIENT) as current_span:
|
||||
print(f"Trace ID: {format_trace_id(current_span.get_span_context().trace_id)}")
|
||||
|
||||
# Run the sequential workflow scenario
|
||||
await run_sequential_workflow()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
Reference in New Issue
Block a user