mirror of
https://github.com/microsoft/agent-framework.git
synced 2026-06-16 21:04:09 +08:00
[BREAKING] Python: Enable instrumentation by default (#5865)
* Enable instrumentation by default * Update samples * Optimization when span is not recording * Address Copilot comments * Revert uv.lock * Add warning * Formatting * Fix mypy * Add disable_instrumentation() with sticky user-intent semantics Add a public disable_instrumentation() entry point so users can explicitly opt out of Agent Framework telemetry, with a sticky-disable flag that makes the user's intent "leading" — no framework code path (foundry's configure_azure_monitor, configure_otel_providers, enable_instrumentation, enable_sensitive_telemetry, or direct OBSERVABILITY_SETTINGS.enable_* writes) can re-enable instrumentation until the user explicitly clears the disable with enable_instrumentation(force=True) / enable_sensitive_telemetry(force=True). Also addresses the two remaining unresolved review threads on the PR: 1. test_observability_settings_defaults_instrumentation_true pins the new "ENABLE_INSTRUMENTATION defaults to True when env unset" behavior. 2. test_enable_instrumentation_reads_env_sensitive_data restores coverage for the post-import load_dotenv() fallback path. Implementation: - ObservabilitySettings.enable_instrumentation / enable_sensitive_data become properties backed by _enable_*. While _user_disabled is True, the getters return False and the setters drop True writes (defense in depth so third- party writes can't subvert the disable). - Public is_user_disabled read-only property lets integrations (e.g. foundry's configure_azure_monitor) cheaply check the disable state without poking at privates. - enable_instrumentation() and enable_sensitive_telemetry() short-circuit with an info log when disabled; gain a force=True kwarg that clears the disable. - configure_otel_providers() still creates providers / exporters / views so a later force-enable can use them, but logs an info message when called while disabled. - Foundry's FoundryChatClient.configure_azure_monitor and FoundryAgent.configure_azure_monitor early-return when the user has disabled, so Azure Monitor's global providers aren't installed unnecessarily. Tests: 11 new tests covering default-on, env re-read at call time, sticky behavior against each re-enable surface (enable_instrumentation, enable_sensitive_telemetry, configure_otel_providers, direct attribute writes), force=True override, re-arming the disable, and the __all__ export. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: document disable_instrumentation() and force=True paths Add a "Disabling instrumentation" section to the observability sample README that walks through: - The distinction between the ENABLE_INSTRUMENTATION env var (initial, non-sticky) and disable_instrumentation() (process-wide, sticky). - Why the sticky semantics matter: framework integrations like FoundryChatClient.configure_azure_monitor() can call enable_instrumentation() as part of their setup, and the user's opt-out needs to win. - All five surfaces guarded by the sticky disable (property reads, public enable functions, configure_otel_providers, direct attribute writes, is_user_disabled-aware integrations). - The force=True escape hatch on both enable_instrumentation() and enable_sensitive_telemetry(). - How third-party integrations should consult OBSERVABILITY_SETTINGS.is_user_disabled. - The limits of the disable (does not tear down existing providers / in-flight spans / third-party instrumentation, does not persist across processes). Cross-links the new section from the ENABLE_INSTRUMENTATION row in the env vars table. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: soften disable_instrumentation() overclaim about telemetry guarantees Replace 'no telemetry will be emitted no matter what' (which is too strong, since callers can still pass force=True or mutate private attributes) with language framing the disable as a user-intent contract that library and framework code is expected to honor: the framework actively short-circuits the public enable paths, force=True and private-attribute writes are acknowledged as out-of-contract escape hatches that integrations should not use on the user's behalf. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: correct observability Dependencies section - opentelemetry-sdk is no longer a hard dependency; it is lazily imported by create_resource(), create_metric_views(), and configure_otel_providers() with a clear ImportError when missing. Day-to-day instrumentation works with opentelemetry-api alone provided some other component configures the global OpenTelemetry providers (Azure Monitor, an APM agent, application bootstrap, etc.). - opentelemetry-semantic-conventions-ai is no longer used anywhere in the source; remove it from the listed dependencies. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: replace stale observability migration guide with current PR's only relevant migration The old guide documented the move away from setup_observability(otlp_endpoint=...) which was an earlier-release API change unrelated to this PR and stale enough that it's more confusing than helpful at this point. Replace it with a short note on the single migration this PR introduces: callers of enable_instrumentation(enable_sensitive_data=True) should switch to enable_sensitive_telemetry(). Cross-link to the Disabling instrumentation section for the rare 'force on without enabling sensitive data' use case where enable_instrumentation() still applies. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Eduard van Valkenburg <eavanvalkenburg@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
committed by
GitHub
Unverified
parent
0ba552b84c
commit
72a6157c6a
@@ -27,6 +27,9 @@ OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:4317"
|
||||
# Agent Framework specific settings
|
||||
# ==================================
|
||||
|
||||
# Observability is enabled by default. Set to "false" to opt out.
|
||||
# ENABLE_INSTRUMENTATION=false
|
||||
|
||||
# Enable sensitive data logging (prompts, responses, etc.)
|
||||
# WARNING: Only enable in dev/test environments
|
||||
ENABLE_SENSITIVE_DATA=true
|
||||
@@ -34,9 +37,6 @@ ENABLE_SENSITIVE_DATA=true
|
||||
# Optional: Enable console exporters for debugging
|
||||
# ENABLE_CONSOLE_EXPORTERS=true
|
||||
|
||||
# Optional: Enable observability (automatically enabled if env vars are set or configure_otel_providers() is called)
|
||||
# ENABLE_INSTRUMENTATION=true
|
||||
|
||||
# OpenAI specific variables
|
||||
# ==========================
|
||||
OPENAI_API_KEY="..."
|
||||
|
||||
@@ -1,12 +1,12 @@
|
||||
# Agent Framework Observability
|
||||
|
||||
This sample folder shows how a Python application can be configured to send Agent Framework observability data to the Application Performance Management (APM) vendor(s) of your choice based on the OpenTelemetry standard.
|
||||
These samples show how to send Agent Framework observability data to the Application Performance Management (APM) backend of your choice, based on the OpenTelemetry standard.
|
||||
|
||||
In this sample, we provide options to send telemetry to [Application Insights](https://learn.microsoft.com/en-us/azure/azure-monitor/app/app-insights-overview), [Aspire Dashboard](https://learn.microsoft.com/en-us/dotnet/aspire/fundamentals/dashboard/overview?tabs=bash) and the console.
|
||||
The samples target [Application Insights](https://learn.microsoft.com/en-us/azure/azure-monitor/app/app-insights-overview), the [Aspire Dashboard](https://learn.microsoft.com/en-us/dotnet/aspire/fundamentals/dashboard/overview?tabs=bash), and the console, but any OTLP-compatible backend works.
|
||||
|
||||
> **Quick Start**: For local development without Azure setup, you can use the [Aspire Dashboard](https://learn.microsoft.com/en-us/dotnet/aspire/fundamentals/dashboard/standalone) which runs locally via Docker and provides an excellent telemetry viewing experience for OpenTelemetry data. Or you can use the built-in tracing module of the [AI Toolkit for VS Code](https://marketplace.visualstudio.com/items?itemName=ms-windows-ai-studio.windows-ai-studio).
|
||||
> **Quick Start**: For local development without Azure setup, use the [Aspire Dashboard](https://learn.microsoft.com/en-us/dotnet/aspire/fundamentals/dashboard/standalone) (runs locally via Docker), or the built-in tracing module of the [AI Toolkit for VS Code](https://marketplace.visualstudio.com/items?itemName=ms-windows-ai-studio.windows-ai-studio).
|
||||
|
||||
> Note that it is also possible to use other Application Performance Management (APM) vendors. An example is [Prometheus](https://prometheus.io/docs/introduction/overview/). Please refer to this [page](https://opentelemetry.io/docs/languages/python/exporters/) to learn more about exporters.
|
||||
> Other backends such as [Prometheus](https://prometheus.io/docs/introduction/overview/) are also supported. See the [OpenTelemetry Python exporters](https://opentelemetry.io/docs/languages/python/exporters/) page for the full list.
|
||||
|
||||
For more information, please refer to the following resources:
|
||||
|
||||
@@ -18,19 +18,15 @@ For more information, please refer to the following resources:
|
||||
|
||||
## What to expect
|
||||
|
||||
The Agent Framework Python SDK is designed to efficiently generate comprehensive logs, traces, and metrics throughout the flow of agent/model invocation and tool execution. This allows you to effectively monitor your AI application's performance and accurately track token consumption. It does so based on the Semantic Conventions for GenAI defined by OpenTelemetry, and the workflows emit their own spans to provide end-to-end visibility.
|
||||
The Agent Framework Python SDK is **natively instrumented** to emit logs, traces, and metrics throughout agent/model invocation and tool execution, so you can monitor your AI application's performance and track token consumption. Instrumentation follows the OpenTelemetry [Semantic Conventions for GenAI](https://opentelemetry.io/docs/specs/semconv/gen-ai/), and workflows emit their own spans for end-to-end visibility.
|
||||
|
||||
Next to what happens in the code when you run, we also make setting up observability as easy as possible. By calling a single function `configure_otel_providers()` from the `agent_framework.observability` module, you can enable telemetry for traces, logs, and metrics. The function automatically reads standard OpenTelemetry environment variables to configure exporters and providers, making it simple to get started.
|
||||
|
||||
### MCP trace propagation
|
||||
|
||||
Whenever there is an active OpenTelemetry span context, Agent Framework automatically propagates trace context to MCP servers via the `params._meta` field of `tools/call` requests. It uses the globally-configured OpenTelemetry propagator(s) (W3C Trace Context by default, producing `traceparent` and `tracestate`), so custom propagators (B3, Jaeger, etc.) are also supported. This enables distributed tracing across agent-to-MCP-server boundaries, compliant with the [MCP `_meta` specification](https://modelcontextprotocol.io/specification/2025-11-25/basic#_meta).
|
||||
|
||||
**Scope:** automatic `_meta` injection applies only to MCP sessions that the agent process itself opens — `MCPStreamableHTTPTool`, `MCPStdioTool`, and `MCPWebsocketTool` (or any other client-opened `MCPTool` subclass). It does **not** apply to hosted/provider-managed MCP tool configurations such as `FoundryChatClient.get_mcp_tool(...)`, `OpenAIChatClient.get_mcp_tool(...)`, `AnthropicClient.get_mcp_tool(...)`, `GeminiChatClient.get_mcp_tool(...)`, or toolbox-fetched tools (for example, `toolbox = await client.get_toolbox(...)`, then passing `toolbox.tools` into `Agent(tools=...)`), because in those cases the `tools/call` message is issued by the provider service runtime rather than by the agent process. As a result, the framework has no opportunity to inject trace context into those requests, and propagating `traceparent`/`tracestate` across that hosted-service boundary is the responsibility of the service runtime, not Agent Framework. If end-to-end distributed tracing to the downstream MCP server is required, use a client-opened MCP transport instead of a hosted connector.
|
||||
Setting up observability is also easy: a single call to `configure_otel_providers()` from the `agent_framework.observability` module wires up the trace, log, and metric providers. It reads the standard OpenTelemetry environment variables to configure exporters automatically.
|
||||
|
||||
### Five patterns for configuring observability
|
||||
|
||||
We've identified multiple ways to configure observability in your application, depending on your needs:
|
||||
> Setting up observability has two parts: (1) **instrumentation**, the code that generates telemetry, and (2) **exporter/provider configuration**, which decides where that telemetry is sent. Agent Framework is natively instrumented and **enabled by default**, so you only need to handle the second part.
|
||||
|
||||
There are five common ways to do that, depending on your needs:
|
||||
|
||||
**1. Standard otel environment variables, configured for you**
|
||||
|
||||
@@ -42,22 +38,29 @@ from agent_framework.observability import configure_otel_providers
|
||||
# Reads OTEL_EXPORTER_OTLP_* environment variables automatically
|
||||
configure_otel_providers()
|
||||
```
|
||||
|
||||
Or if you just want console exporters:
|
||||
|
||||
```python
|
||||
from agent_framework.observability import configure_otel_providers
|
||||
# Enable console exporters via environment variable
|
||||
|
||||
configure_otel_providers(enable_console_exporters=True)
|
||||
# It is also possible to set ENABLE_CONSOLE_EXPORTERS=true in environment
|
||||
# variables instead of calling `configure_otel_providers()` with the parameter.
|
||||
# The framework will automatically read that and set up console exporters.
|
||||
```
|
||||
|
||||
This is the **recommended approach** for getting started.
|
||||
|
||||
**2. Custom Exporters**
|
||||
One level more control over the exporters that are created is to do that yourself, and then pass them to `configure_otel_providers()`. We will still create the providers for you, but you can customize the exporters as needed:
|
||||
|
||||
For more control, construct exporters yourself and pass them to `configure_otel_providers()`. The framework still creates the providers for you:
|
||||
|
||||
```python
|
||||
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
|
||||
from opentelemetry.exporter.otlp.proto.grpc._log_exporter import OTLPLogExporter
|
||||
from opentelemetry.exporter.otlp.proto.grpc.metric_exporter import OTLPMetricExporter
|
||||
from opentelemetry.exporter.otlp.proto.grpc.exporter import Compression
|
||||
from agent_framework.observability import configure_otel_providers
|
||||
|
||||
# Create custom exporters with specific configuration
|
||||
@@ -67,17 +70,17 @@ exporters = [
|
||||
OTLPMetricExporter(endpoint="http://localhost:4317"),
|
||||
]
|
||||
|
||||
# These will be added alongside any exporters from environment variables
|
||||
configure_otel_providers(exporters=exporters, enable_sensitive_data=True)
|
||||
# These are added alongside any exporters configured from environment variables
|
||||
configure_otel_providers(exporters=exporters)
|
||||
```
|
||||
|
||||
**3. Third party setup**
|
||||
**3. Third-party setup**
|
||||
|
||||
A lot of third party specific otel package, have their own easy setup methods, for example Azure Monitor has `configure_azure_monitor()`. You can use those methods to setup the third party first, and then call `enable_instrumentation()` from the `agent_framework.observability` module to activate the Agent Framework telemetry code paths. In all these cases, if you already setup observability via environment variables, you don't need to call `enable_instrumentation()` as it will be enabled automatically.
|
||||
Many third-party OTel packages ship their own setup helpers (for example, Azure Monitor's `configure_azure_monitor()`). You can use those directly — Agent Framework instrumentation is on by default, so no extra wiring is needed. To also capture sensitive data, call `enable_sensitive_telemetry()` from `agent_framework.observability`.
|
||||
|
||||
```python
|
||||
from azure.monitor.opentelemetry import configure_azure_monitor
|
||||
from agent_framework.observability import create_resource, enable_instrumentation
|
||||
from agent_framework.observability import create_resource, enable_sensitive_telemetry
|
||||
|
||||
# Configure Azure Monitor first
|
||||
configure_azure_monitor(
|
||||
@@ -86,10 +89,10 @@ configure_azure_monitor(
|
||||
enable_live_metrics=True,
|
||||
)
|
||||
|
||||
# Then activate Agent Framework's telemetry code paths
|
||||
# This is optional if ENABLE_INSTRUMENTATION and or ENABLE_SENSITIVE_DATA are set in env vars
|
||||
enable_instrumentation(enable_sensitive_data=False)
|
||||
# Optional: opt in to capturing sensitive data
|
||||
enable_sensitive_telemetry()
|
||||
```
|
||||
|
||||
For Microsoft Foundry projects, use `client.configure_azure_monitor()` which retrieves the connection string from the project and configures everything:
|
||||
|
||||
```python
|
||||
@@ -110,7 +113,7 @@ Or with [Langfuse](https://langfuse.com/integrations/frameworks/microsoft-agent-
|
||||
|
||||
```python
|
||||
# environment should be setup correctly, with langfuse urls and keys
|
||||
from agent_framework.observability import enable_instrumentation
|
||||
from agent_framework.observability import enable_sensitive_telemetry
|
||||
from langfuse import get_client
|
||||
|
||||
langfuse = get_client()
|
||||
@@ -121,9 +124,9 @@ if langfuse.auth_check():
|
||||
else:
|
||||
print("Authentication failed. Please check your credentials and host.")
|
||||
|
||||
# Then activate Agent Framework's telemetry code paths
|
||||
# This is optional if ENABLE_INSTRUMENTATION and or ENABLE_SENSITIVE_DATA are set in env vars
|
||||
enable_instrumentation(enable_sensitive_data=False)
|
||||
# Agent Framework instrumentation is on by default.
|
||||
# Optional: opt in to capturing sensitive data
|
||||
enable_sensitive_telemetry()
|
||||
```
|
||||
|
||||
Or with [Comet Opik](https://www.comet.com/docs/opik/integrations/microsoft-agent-framework):
|
||||
@@ -131,53 +134,152 @@ Or with [Comet Opik](https://www.comet.com/docs/opik/integrations/microsoft-agen
|
||||
```python
|
||||
import os
|
||||
|
||||
from agent_framework.observability import enable_instrumentation
|
||||
from agent_framework.observability import enable_sensitive_telemetry
|
||||
|
||||
# Use Opik OTLP settings from your project settings
|
||||
os.environ["OTEL_EXPORTER_OTLP_ENDPOINT"] = "<opik_otlp_endpoint>"
|
||||
os.environ["OTEL_EXPORTER_OTLP_HEADERS"] = "<opik_otlp_headers>"
|
||||
|
||||
# Then activate Agent Framework's telemetry code paths
|
||||
# This is optional if ENABLE_INSTRUMENTATION and or ENABLE_SENSITIVE_DATA are set in env vars
|
||||
enable_instrumentation(enable_sensitive_data=False)
|
||||
# Agent Framework instrumentation is on by default.
|
||||
# Optional: opt in to capturing sensitive data
|
||||
enable_sensitive_telemetry()
|
||||
```
|
||||
|
||||
**4. Manual setup**
|
||||
Of course you can also do a complete manual setup of exporters, providers, and instrumentation. Please refer to sample [advanced_manual_setup_console_output.py](./advanced_manual_setup_console_output.py) for a comprehensive example of how to manually setup exporters and providers for traces, logs, and metrics that will get sent to the console. This gives you full control over which exporters and providers to use. We do have a helper function `create_resource()` in the `agent_framework.observability` module that you can use to create a resource with the appropriate service name and version based on environment variables or standard defaults for Agent Framework, this is not used in the sample.
|
||||
|
||||
**5. Auto-instrumentation (zero-code)**
|
||||
You can also use the [OpenTelemetry CLI tool](https://opentelemetry.io/docs/instrumentation/python/getting-started/#automatic-instrumentation) to automatically instrument your application without changing any code. Please refer to sample [advanced_zero_code.py](./advanced_zero_code.py) for an example of how to use the CLI tool to enable instrumentation for Agent Framework applications.
|
||||
For full control, set up providers and exporters yourself. See [advanced_manual_setup_console_output.py](./advanced_manual_setup_console_output.py) for a complete example that sends traces, logs, and metrics to the console. The `create_resource()` helper in `agent_framework.observability` can build a resource with the appropriate service name and version from environment variables (or sensible defaults), although the sample does not use it.
|
||||
|
||||
**5. Zero-code provider/exporter configuration**
|
||||
|
||||
Because Agent Framework is **natively instrumented** with OpenTelemetry, you do not need to auto-instrument the framework itself. You can, however, use the [`opentelemetry-instrument`](https://opentelemetry.io/docs/zero-code/python/) CLI wrapper to configure the global tracer/meter providers and exporters from environment variables (or CLI flags) at process startup. Your application code then does not need to call `configure_otel_providers()` — the native spans and metrics from Agent Framework are picked up by the globally configured pipeline. See [advanced_zero_code.py](./advanced_zero_code.py) for an example.
|
||||
|
||||
### MCP trace propagation
|
||||
|
||||
Whenever there is an active OpenTelemetry span context, Agent Framework automatically propagates trace context to MCP servers via the `params._meta` field of `tools/call` requests. It uses the globally configured OpenTelemetry propagator(s) — W3C Trace Context by default (producing `traceparent` and `tracestate`) — so custom propagators (B3, Jaeger, etc.) are also supported. This enables distributed tracing across agent-to-MCP-server boundaries, compliant with the [MCP `_meta` specification](https://modelcontextprotocol.io/specification/2025-11-25/basic#_meta).
|
||||
|
||||
**Scope:** automatic `_meta` injection applies only to MCP sessions that the agent process itself opens — `MCPStreamableHTTPTool`, `MCPStdioTool`, and `MCPWebsocketTool` (or any other client-opened `MCPTool` subclass). It does **not** apply to hosted or provider-managed MCP tool configurations such as `FoundryChatClient.get_mcp_tool(...)`, `OpenAIChatClient.get_mcp_tool(...)`, `AnthropicClient.get_mcp_tool(...)`, `GeminiChatClient.get_mcp_tool(...)`, or toolbox-fetched tools (e.g. `toolbox = await client.get_toolbox(...)` then `Agent(tools=toolbox.tools)`). In those cases the `tools/call` message is issued by the provider service runtime rather than by the agent process, so propagating `traceparent`/`tracestate` across that boundary is the service runtime's responsibility. If you need end-to-end distributed tracing to the downstream MCP server, use a client-opened MCP transport instead of a hosted connector.
|
||||
|
||||
## Configuration
|
||||
|
||||
### Dependencies
|
||||
|
||||
As part of Agent Framework we use the following OpenTelemetry packages:
|
||||
- `opentelemetry-api`
|
||||
- `opentelemetry-sdk`
|
||||
- `opentelemetry-semantic-conventions-ai`
|
||||
Agent Framework's core depends on **`opentelemetry-api`** only — the API package is enough for the instrumentation hooks (spans, meters, log records) to emit telemetry, and it has no runtime side effects when no provider is configured.
|
||||
|
||||
We do not install exporters by default, so you will need to add those yourself, this prevents us from installing unnecessary dependencies. For Application Insights, you will need to install `azure-monitor-opentelemetry`. For Aspire Dashboard or other OTLP compatible backends, you will need to install `opentelemetry-exporter-otlp-proto-grpc`. For HTTP protocol support, you will also need to install `opentelemetry-exporter-otlp-proto-http`.
|
||||
If you want the framework to set up providers / exporters for you via `configure_otel_providers()` (or to use the `create_resource()` / `create_metric_views()` helpers), you also need the OpenTelemetry SDK:
|
||||
|
||||
And for many others, different packages are used, so refer to the documentation of the specific exporter you want to use.
|
||||
```bash
|
||||
pip install opentelemetry-sdk
|
||||
```
|
||||
|
||||
If `opentelemetry-sdk` is missing, those helper functions raise a clear `ImportError` telling you to install it. Day-to-day instrumentation still works without the SDK as long as some other component (e.g. `azure-monitor-opentelemetry`, your application bootstrap, an APM agent) has configured the global OpenTelemetry providers.
|
||||
|
||||
Exporters are **not** installed by default — install only what you need:
|
||||
- **Application Insights**: `azure-monitor-opentelemetry`
|
||||
- **Aspire Dashboard or other OTLP/gRPC backends**: `opentelemetry-exporter-otlp-proto-grpc`
|
||||
- **OTLP over HTTP**: `opentelemetry-exporter-otlp-proto-http`
|
||||
|
||||
For other backends, refer to the documentation of the specific exporter.
|
||||
|
||||
### Environment variables
|
||||
|
||||
The following environment variables are used to turn on/off observability of the Agent Framework:
|
||||
Agent Framework reads the following environment variables:
|
||||
|
||||
- `ENABLE_INSTRUMENTATION`
|
||||
- `ENABLE_SENSITIVE_DATA`
|
||||
- `ENABLE_CONSOLE_EXPORTERS`
|
||||
| Variable | Default | Purpose |
|
||||
|----------|---------|---------|
|
||||
| `ENABLE_INSTRUMENTATION` | `true` | Set to `false` to disable native instrumentation. See [Disabling instrumentation](#disabling-instrumentation) for the programmatic alternative with sticky semantics. |
|
||||
| `ENABLE_SENSITIVE_DATA` | `false` | Set to `true` to emit sensitive data (prompts, responses, etc.). |
|
||||
| `ENABLE_CONSOLE_EXPORTERS` | `false` | Set to `true` to add console exporters. Only used by `configure_otel_providers()`. |
|
||||
| `VS_CODE_EXTENSION_PORT` | unset | Port used by the [AI Toolkit for VS Code](https://marketplace.visualstudio.com/items?itemName=ms-windows-ai-studio.windows-ai-studio#tracing) tracing integration. Only used by `configure_otel_providers()`. |
|
||||
|
||||
All of these are booleans and default to `false`.
|
||||
You can also call `enable_sensitive_telemetry()` from `agent_framework.observability` to opt in to sensitive-data capture programmatically.
|
||||
|
||||
Finally we have `VS_CODE_EXTENSION_PORT` which you can set to a port, which can be used to setup the AI Toolkit for VS Code tracing integration. See [here](https://marketplace.visualstudio.com/items?itemName=ms-windows-ai-studio.windows-ai-studio#tracing) for more details.
|
||||
> **Note**: Sensitive data includes prompts, responses, and tool arguments. Only enable it in development or test environments — it may expose user or system secrets in production.
|
||||
|
||||
The framework will emit observability data when the `ENABLE_INSTRUMENTATION` environment variable is set to `true`. If both are `true` then it will also emit sensitive information. When these are not set, or set to false, you can use the `enable_instrumentation()` function from the `agent_framework.observability` module to turn on instrumentation programmatically. This is useful when you want to control this via code instead of environment variables.
|
||||
### Disabling instrumentation
|
||||
|
||||
> **Note**: Sensitive information includes prompts, responses, and more, and should only be enabled in a development or test environment. It is not recommended to enable this in production environments as it may expose sensitive data.
|
||||
There are two ways to turn Agent Framework's native instrumentation off, and they have **different scopes**:
|
||||
|
||||
The two other variables, `ENABLE_CONSOLE_EXPORTERS` and `VS_CODE_EXTENSION_PORT`, are used to configure where the observability data is sent. Those are only activated when calling `configure_otel_providers()`.
|
||||
| Approach | Scope | Sticky? | When framework code calls `enable_instrumentation()` later, what happens? |
|
||||
|----------|-------|---------|---------------------------------------------------------------------------|
|
||||
| `ENABLE_INSTRUMENTATION=false` in the environment | Initial settings only | No | Instrumentation flips back **on**. |
|
||||
| `disable_instrumentation()` called from code | Process-wide, sticky | Yes | Instrumentation **stays off** — the user-disable intent wins. |
|
||||
|
||||
If you want telemetry off **and want it to stay off**, use `disable_instrumentation()`.
|
||||
|
||||
#### Sticky semantics — why this matters
|
||||
|
||||
Framework integrations and third-party libraries can call `enable_instrumentation()`, `enable_sensitive_telemetry()`, or `configure_otel_providers()` as part of their own setup. For example, `FoundryChatClient.configure_azure_monitor()` calls `enable_instrumentation()` after wiring up Azure Monitor. That's normally what you want — but if **you** have explicitly opted out, you don't want any of those calls to silently re-enable telemetry.
|
||||
|
||||
`disable_instrumentation()` solves this by setting a **sticky** flag on `OBSERVABILITY_SETTINGS` that remains in effect until you explicitly clear it. While the flag is set:
|
||||
|
||||
1. `OBSERVABILITY_SETTINGS.enable_instrumentation` and `enable_sensitive_data` **read as `False`** regardless of the stored value.
|
||||
2. `enable_instrumentation()` and `enable_sensitive_telemetry()` are **no-ops** and log an info-level message.
|
||||
3. `configure_otel_providers()` still configures providers / exporters / views (so a later force-enable can use them), but does not flip instrumentation on.
|
||||
4. Direct attribute writes like `OBSERVABILITY_SETTINGS.enable_instrumentation = True` from any code are **silently dropped** (defense in depth).
|
||||
5. Integrations that consult `OBSERVABILITY_SETTINGS.is_user_disabled` (e.g. `FoundryChatClient.configure_azure_monitor()`, `FoundryAgent.configure_azure_monitor()`) **skip their setup entirely**, so global Azure Monitor providers aren't installed unnecessarily.
|
||||
|
||||
```python
|
||||
from agent_framework.observability import disable_instrumentation
|
||||
|
||||
# After this call, Agent Framework expresses your intent to opt out of telemetry.
|
||||
# Library and framework code is expected to honor that intent and not flip
|
||||
# instrumentation back on (e.g. by calling `enable_instrumentation()`,
|
||||
# `enable_sensitive_telemetry()`, or writing to public attributes on
|
||||
# `OBSERVABILITY_SETTINGS`). The framework actively short-circuits the public
|
||||
# enable paths so the user's intent stays leading. A determined caller can still
|
||||
# pass `force=True` or mutate private (`_`-prefixed) attributes to bypass it,
|
||||
# but those are out-of-contract escape hatches that should not be used by
|
||||
# integrations on the user's behalf.
|
||||
disable_instrumentation()
|
||||
```
|
||||
|
||||
#### Forcing re-enablement after a disable
|
||||
|
||||
To intentionally re-enable telemetry after `disable_instrumentation()`, pass `force=True` to either of the two public enable helpers. This is the only way to clear the sticky disable, so the user's opt-out can only be reversed by a deliberate user opt-in:
|
||||
|
||||
```python
|
||||
from agent_framework.observability import (
|
||||
disable_instrumentation,
|
||||
enable_instrumentation,
|
||||
enable_sensitive_telemetry,
|
||||
)
|
||||
|
||||
disable_instrumentation()
|
||||
|
||||
# Without force=True, these are no-ops while the disable is sticky:
|
||||
enable_instrumentation() # logs info, does nothing
|
||||
enable_sensitive_telemetry() # logs info, does nothing
|
||||
|
||||
# With force=True, the sticky disable is cleared and the call proceeds:
|
||||
enable_instrumentation(force=True)
|
||||
# or
|
||||
enable_sensitive_telemetry(force=True)
|
||||
|
||||
# After a force-enable you can `disable_instrumentation()` again to re-arm
|
||||
# the sticky disable.
|
||||
```
|
||||
|
||||
#### Checking the disable state from integrations
|
||||
|
||||
If you're writing an integration that performs telemetry setup as a side effect (e.g. provisioning a third-party exporter), consult the public read-only `is_user_disabled` property and early-return when it's set:
|
||||
|
||||
```python
|
||||
from agent_framework.observability import OBSERVABILITY_SETTINGS
|
||||
|
||||
if OBSERVABILITY_SETTINGS.is_user_disabled:
|
||||
logger.info(
|
||||
"Skipping telemetry setup because the user called disable_instrumentation()."
|
||||
)
|
||||
return
|
||||
```
|
||||
|
||||
This is what the built-in `FoundryChatClient.configure_azure_monitor()` and `FoundryAgent.configure_azure_monitor()` do — so calling `disable_instrumentation()` reliably prevents Azure Monitor's global providers from being installed by those helpers.
|
||||
|
||||
#### What `disable_instrumentation()` does **not** do
|
||||
|
||||
- It does not tear down OpenTelemetry providers, exporters, or in-flight spans that were already set up before the disable call. It only gates **future** captures by Agent Framework code paths.
|
||||
- It does not stop telemetry from third-party instrumentations (e.g. `azure-monitor-opentelemetry`'s system metrics) that are wired up outside Agent Framework. Configure those separately if needed.
|
||||
- It does not persist across processes. Each Python process starts with the disable flag cleared; if you always want telemetry off in a given environment, set `ENABLE_INSTRUMENTATION=false` as an environment variable in addition to (or instead of) the programmatic call.
|
||||
|
||||
#### Environment variables for `configure_otel_providers()`
|
||||
|
||||
@@ -202,7 +304,8 @@ The `configure_otel_providers()` function automatically reads **standard OpenTel
|
||||
> **Note**: These are standard OpenTelemetry environment variables. See the [OpenTelemetry spec](https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/) for more details.
|
||||
|
||||
#### Logging
|
||||
Use standard Python logging configuration to align logs with telemetry output.
|
||||
|
||||
Use standard Python logging configuration to align logs with telemetry output:
|
||||
|
||||
```python
|
||||
import logging
|
||||
@@ -212,15 +315,14 @@ logging.basicConfig(
|
||||
datefmt="%Y-%m-%d %H:%M:%S",
|
||||
)
|
||||
```
|
||||
You can control at what level logging happens and thus what logs get exported, you can do this, by adding this:
|
||||
|
||||
To control which logs are exported, adjust the root logger level — other loggers inherit from it by default:
|
||||
|
||||
```python
|
||||
import logging
|
||||
|
||||
logger = logging.getLogger()
|
||||
logger.setLevel(logging.NOTSET)
|
||||
logging.getLogger().setLevel(logging.NOTSET)
|
||||
```
|
||||
This gets the root logger and sets the level of that, automatically other loggers inherit from that one, and you will get detailed logs in your telemetry.
|
||||
|
||||
## Samples
|
||||
|
||||
@@ -228,36 +330,35 @@ This folder contains different samples demonstrating how to use telemetry in var
|
||||
|
||||
| Sample | Description |
|
||||
|--------|-------------|
|
||||
| [configure_otel_providers_with_parameters.py](./configure_otel_providers_with_parameters.py) | **Recommended starting point**: Shows how to create custom exporters with specific configuration and pass them to `configure_otel_providers()`. Useful for advanced scenarios. |
|
||||
| [configure_otel_providers_with_env_var.py](./configure_otel_providers_with_env_var.py) | Shows how to setup telemetry using standard OpenTelemetry environment variables (`OTEL_EXPORTER_OTLP_*`). |
|
||||
| [agent_observability.py](./agent_observability.py) | Shows telemetry collection for an agentic application with tool calls using environment variables. |
|
||||
| [foundry_tracing.py](./foundry_tracing.py) | Shows Azure Monitor integration with Foundry for any chat client. |
|
||||
| [advanced_manual_setup_console_output.py](./advanced_manual_setup_console_output.py) | Advanced: Shows manual setup of exporters and providers with console output. Useful for understanding how observability works under the hood. |
|
||||
| [advanced_zero_code.py](./advanced_zero_code.py) | Advanced: Shows zero-code telemetry setup using the `opentelemetry-enable_instrumentation` CLI tool. |
|
||||
| [workflow_observability.py](./workflow_observability.py) | Shows telemetry collection for a workflow with multiple executors and message passing. |
|
||||
| [configure_otel_providers_with_env_var.py](./configure_otel_providers_with_env_var.py) | **Recommended starting point**: configure telemetry using standard OpenTelemetry environment variables (`OTEL_EXPORTER_OTLP_*`). |
|
||||
| [configure_otel_providers_with_parameters.py](./configure_otel_providers_with_parameters.py) | Create custom exporters with specific configuration and pass them to `configure_otel_providers()`. |
|
||||
| [agent_observability.py](./agent_observability.py) | Telemetry collection for an agentic application with tool calls. |
|
||||
| [foundry_tracing.py](./foundry_tracing.py) | Azure Monitor integration with Microsoft Foundry. |
|
||||
| [workflow_observability.py](./workflow_observability.py) | Telemetry collection for a workflow with multiple executors and message passing. |
|
||||
| [advanced_manual_setup_console_output.py](./advanced_manual_setup_console_output.py) | Advanced: manual setup of exporters and providers with console output — useful for understanding how observability works under the hood. |
|
||||
| [advanced_zero_code.py](./advanced_zero_code.py) | Advanced: zero-code provider/exporter setup using the `opentelemetry-instrument` CLI wrapper. |
|
||||
|
||||
### Running the samples
|
||||
|
||||
1. Open a terminal and navigate to this folder: `python/samples/02-agents/observability/`. This is necessary for the `.env` file to be read correctly.
|
||||
2. Create a `.env` file if one doesn't already exist in this folder. Please refer to the [example file](./.env.example).
|
||||
> **Note**: You can start with just `ENABLE_INSTRUMENTATION=true` and add `OTEL_EXPORTER_OTLP_ENDPOINT` or other configuration as needed. If no exporters are configured, you can set `ENABLE_CONSOLE_EXPORTERS=true` for console output.
|
||||
3. Choose one environment-loading approach:
|
||||
- **A. Sample-managed loading (current samples):** run from this folder so the sample's `load_dotenv()` call can find `.env`.
|
||||
- **B. Shell/IDE-managed environment:** set/export environment variables directly, or use an IDE run configuration that injects env vars / `.env`.
|
||||
- **C. Explicit env file in code:** pass `env_file_path` to APIs like `configure_otel_providers(env_file_path=".env")` (or your own settings loader path).
|
||||
- **D. CLI-managed env file:** run with `uv` and pass the file explicitly, for example:
|
||||
`uv run --env-file=.env python configure_otel_providers_with_env_var.py`
|
||||
4. Activate your python virtual environment, then run a sample (for example `python configure_otel_providers_with_env_var.py`).
|
||||
1. Open a terminal in this folder (`python/samples/02-agents/observability/`) so that `.env` is found.
|
||||
2. Create a `.env` file if you don't already have one. See [.env.example](./.env.example).
|
||||
> Instrumentation is on by default. Set `OTEL_EXPORTER_OTLP_ENDPOINT` (or other configuration) as needed. With no exporters configured, set `ENABLE_CONSOLE_EXPORTERS=true` for console output.
|
||||
3. Pick an environment-loading approach:
|
||||
- **A. Sample-managed:** run from this folder so the sample's `load_dotenv()` call can find `.env`.
|
||||
- **B. Shell/IDE-managed:** export environment variables, or use an IDE run configuration that injects them.
|
||||
- **C. Explicit env file in code:** pass `env_file_path` to APIs like `configure_otel_providers(env_file_path=".env")`.
|
||||
- **D. CLI-managed:** run with `uv` and pass the file explicitly, e.g. `uv run --env-file=.env python configure_otel_providers_with_env_var.py`.
|
||||
4. Activate your virtual environment, then run a sample (e.g. `python configure_otel_providers_with_env_var.py`).
|
||||
|
||||
> If you do manual provider setup (e.g., Azure Monitor), call `enable_instrumentation()` to turn on Agent Framework telemetry code paths; if you want Agent Framework to configure exporters/providers for you, call `configure_otel_providers(...)`.
|
||||
> If you set up providers manually (e.g. Azure Monitor), Agent Framework instrumentation is still on by default. Call `enable_sensitive_telemetry()` if you also want to capture sensitive data. To have Agent Framework configure exporters and providers for you, call `configure_otel_providers(...)`.
|
||||
|
||||
> Each sample will print the Operation/Trace ID, which can be used later for filtering logs and traces in Application Insights or Aspire Dashboard.
|
||||
> Each sample prints its Operation/Trace ID, which you can use to filter logs and traces in Application Insights or the Aspire Dashboard.
|
||||
|
||||
# Appendix
|
||||
|
||||
## Azure Monitor Queries
|
||||
|
||||
When you are in Azure Monitor and want to have a overall view of the span, use this query in the logs section:
|
||||
For an overall view of a span in Azure Monitor, run this query in the Logs section:
|
||||
|
||||
```kusto
|
||||
dependencies
|
||||
@@ -280,7 +381,8 @@ dependencies
|
||||
```
|
||||
|
||||
### Grafana dashboards with Application Insights data
|
||||
Besides the Application Insights native UI, you can also use Grafana to visualize the telemetry data in Application Insights. There are two tailored dashboards for you to get started quickly:
|
||||
|
||||
In addition to the native Application Insights UI, you can use Grafana to visualize the same telemetry data. Two tailored dashboards are available to get you started:
|
||||
|
||||
#### Agent Overview dashboard
|
||||
Open dashboard in Azure portal: <https://aka.ms/amg/dash/af-agent>
|
||||
@@ -292,117 +394,27 @@ Open dashboard in Azure portal: <https://aka.ms/amg/dash/af-workflow>
|
||||
|
||||
## Migration Guide
|
||||
|
||||
We've done a major update to the observability API in Agent Framework Python SDK. The new API simplifies configuration by relying more on standard OpenTelemetry environment variables and have split the instrumentation from the configuration.
|
||||
Instrumentation is now **enabled by default** (you no longer have to opt in by calling `enable_instrumentation()` at startup), and the way you opt in to capturing sensitive payloads has its own dedicated function.
|
||||
|
||||
If you're updating from a previous version of the Agent Framework, here are the key changes to the observability API:
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Old Variable | New Variable | Notes |
|
||||
|-------------|--------------|-------|
|
||||
| `OTLP_ENDPOINT` | `OTEL_EXPORTER_OTLP_ENDPOINT` | Standard OpenTelemetry env var |
|
||||
| `APPLICATIONINSIGHTS_CONNECTION_STRING` | N/A | Use `configure_azure_monitor()` |
|
||||
| N/A | `ENABLE_CONSOLE_EXPORTERS` | New opt-in flag for console output |
|
||||
|
||||
### OTLP Configuration
|
||||
|
||||
**Before (Deprecated):**
|
||||
```
|
||||
from agent_framework.observability import setup_observability
|
||||
# Via parameter
|
||||
setup_observability(otlp_endpoint="http://localhost:4317")
|
||||
|
||||
# Via environment variable
|
||||
# OTLP_ENDPOINT=http://localhost:4317
|
||||
setup_observability()
|
||||
```
|
||||
|
||||
**After (Current):**
|
||||
```python
|
||||
from agent_framework.observability import configure_otel_providers
|
||||
# Via standard OTEL environment variable (recommended)
|
||||
# OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
|
||||
configure_otel_providers()
|
||||
|
||||
# Or via custom exporters
|
||||
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
|
||||
from opentelemetry.exporter.otlp.proto.grpc._log_exporter import OTLPLogExporter
|
||||
from opentelemetry.exporter.otlp.proto.grpc.metric_exporter import OTLPMetricExporter
|
||||
|
||||
configure_otel_providers(exporters=[
|
||||
OTLPSpanExporter(endpoint="http://localhost:4317"),
|
||||
OTLPLogExporter(endpoint="http://localhost:4317"),
|
||||
OTLPMetricExporter(endpoint="http://localhost:4317"),
|
||||
])
|
||||
```
|
||||
|
||||
### Azure Monitor Configuration
|
||||
|
||||
**Before (Deprecated):**
|
||||
```
|
||||
from agent_framework.observability import setup_observability
|
||||
|
||||
setup_observability(
|
||||
applicationinsights_connection_string="InstrumentationKey=...",
|
||||
applicationinsights_live_metrics=True,
|
||||
)
|
||||
```
|
||||
|
||||
**After (Current):**
|
||||
If your code previously did:
|
||||
|
||||
```python
|
||||
from agent_framework.foundry import FoundryChatClient
|
||||
from agent_framework.observability import create_resource, enable_instrumentation
|
||||
from azure.identity import AzureCliCredential
|
||||
from azure.monitor.opentelemetry import configure_azure_monitor
|
||||
from agent_framework.observability import enable_instrumentation
|
||||
|
||||
async def main():
|
||||
# For Microsoft Foundry projects
|
||||
client = FoundryChatClient(
|
||||
project_endpoint="https://your-project.services.ai.azure.com",
|
||||
model="gpt-4o",
|
||||
credential=AzureCliCredential(),
|
||||
)
|
||||
await client.configure_azure_monitor(enable_live_metrics=True)
|
||||
|
||||
# For non-Azure AI projects
|
||||
configure_azure_monitor(
|
||||
connection_string="InstrumentationKey=...",
|
||||
resource=create_resource(),
|
||||
enable_live_metrics=True,
|
||||
)
|
||||
enable_instrumentation()
|
||||
enable_instrumentation(enable_sensitive_data=True)
|
||||
```
|
||||
|
||||
### Console Output
|
||||
replace it with:
|
||||
|
||||
**Before (Deprecated):**
|
||||
```
|
||||
from agent_framework.observability import setup_observability
|
||||
|
||||
# Console was used as automatic fallback
|
||||
setup_observability() # Would output to console if no exporters configured
|
||||
```
|
||||
|
||||
**After (Current):**
|
||||
```python
|
||||
from agent_framework.observability import configure_otel_providers
|
||||
from agent_framework.observability import enable_sensitive_telemetry
|
||||
|
||||
# Console exporters are now opt-in
|
||||
# ENABLE_CONSOLE_EXPORTERS=true
|
||||
configure_otel_providers()
|
||||
|
||||
# Or programmatically
|
||||
configure_otel_providers(enable_console_exporters=True)
|
||||
enable_sensitive_telemetry()
|
||||
```
|
||||
|
||||
### Benefits of New API
|
||||
`enable_sensitive_telemetry()` ensures that instrumentation is on and turns sensitive-event capture on in one call. `enable_instrumentation()` still exists for the rare case where you want to programmatically force instrumentation on without enabling sensitive data (e.g. to override `ENABLE_INSTRUMENTATION=false`), and it now also accepts `force=True` to clear a previous `disable_instrumentation()` — see [Disabling instrumentation](#disabling-instrumentation).
|
||||
|
||||
1. **Standards Compliant**: Uses standard OpenTelemetry environment variables
|
||||
2. **Simpler**: Less configuration needed, more relies on environment
|
||||
3. **Flexible**: Easy to add custom exporters alongside environment-based ones
|
||||
4. **Cleaner Separation**: Azure Monitor setup is in Azure-specific client
|
||||
5. **Better Compatibility**: Works with any OTEL-compatible tool (Jaeger, Zipkin, Prometheus, etc.)
|
||||
> **Note**: Sensitive data includes prompts, responses, and tool arguments. Only enable it in development or test environments — it may expose user or system secrets in production.
|
||||
|
||||
## Aspire Dashboard
|
||||
|
||||
@@ -437,7 +449,7 @@ OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
|
||||
Or set it as an environment variable when running your samples:
|
||||
|
||||
```bash
|
||||
ENABLE_INSTRUMENTATION=true OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 python configure_otel_providers_with_env_var.py
|
||||
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 python configure_otel_providers_with_env_var.py
|
||||
```
|
||||
|
||||
### Viewing telemetry data
|
||||
|
||||
@@ -7,7 +7,7 @@ from typing import Annotated
|
||||
|
||||
from agent_framework import Message, tool
|
||||
from agent_framework.foundry import FoundryChatClient
|
||||
from agent_framework.observability import enable_instrumentation
|
||||
from agent_framework.observability import enable_sensitive_telemetry
|
||||
from azure.identity import AzureCliCredential
|
||||
from dotenv import load_dotenv
|
||||
from opentelemetry._logs import set_logger_provider
|
||||
@@ -135,7 +135,8 @@ async def main():
|
||||
setup_logging()
|
||||
setup_tracing()
|
||||
setup_metrics()
|
||||
enable_instrumentation()
|
||||
# Instrumentation is enabled by default; call this to also capture sensitive data.
|
||||
enable_sensitive_telemetry()
|
||||
|
||||
await run_chat_client()
|
||||
|
||||
|
||||
@@ -19,13 +19,20 @@ if TYPE_CHECKING:
|
||||
|
||||
"""
|
||||
This sample shows how you can configure observability of an application with zero code changes.
|
||||
It relies on the OpenTelemetry auto-instrumentation capabilities, and the observability setup
|
||||
is done via environment variables.
|
||||
|
||||
Follow the install guidance from https://opentelemetry.io/docs/zero-code/python/ to install the OpenTelemetry CLI tool,
|
||||
when using `uv` there are some additional steps, so follow the instructions carefully.
|
||||
Agent Framework is natively instrumented with OpenTelemetry, so no auto-instrumentation of the
|
||||
framework itself is required. Running the `opentelemetry-instrument` CLI wrapper simply configures
|
||||
the global tracer/meter providers and exporters from environment variables (or CLI flags) at
|
||||
process startup, so the application code does not need to set them up explicitly. The native
|
||||
spans/metrics emitted by Agent Framework are then picked up by that globally configured pipeline.
|
||||
|
||||
And setup a local OpenTelemetry Collector instance to receive the traces and metrics (and update the endpoint below).
|
||||
See: https://opentelemetry.io/docs/zero-code/python/
|
||||
|
||||
Install the OpenTelemetry CLI tool following the guidance above (when using `uv` there are some
|
||||
additional steps, so follow the instructions carefully).
|
||||
|
||||
Then setup a local OpenTelemetry Collector instance to receive the traces and metrics (and update
|
||||
the endpoint below).
|
||||
|
||||
Then you can run:
|
||||
```bash
|
||||
|
||||
@@ -22,9 +22,6 @@ What this example shows:
|
||||
- executor_completed events (type='executor_completed') contain the messages sent via ctx.send_message() in event.data
|
||||
- How to generically observe all executor I/O through workflow streaming events
|
||||
|
||||
This approach allows you to enable_instrumentation any workflow for observability without
|
||||
changing the executor implementations.
|
||||
|
||||
Prerequisites:
|
||||
- No external services required.
|
||||
"""
|
||||
|
||||
-1
@@ -1,4 +1,3 @@
|
||||
FOUNDRY_PROJECT_ENDPOINT="..."
|
||||
AZURE_AI_MODEL_DEPLOYMENT_NAME="..."
|
||||
ENABLE_INSTRUMENTATION=true
|
||||
ENABLE_SENSITIVE_DATA=true
|
||||
+1
-1
@@ -16,7 +16,7 @@ The agent is hosted using the [Agent Framework](https://github.com/microsoft/age
|
||||
|
||||
### Instrumentation
|
||||
|
||||
Agent Framework is [**natively instrumented**](https://learn.microsoft.com/en-us/agent-framework/agents/observability?pivots=programming-language-python) to capture diagnostics and telemetry for agent execution, but it's turned off by default. This sample demonstrates how to enable instrumentation via environment variables in `agent.manifest.yaml` and `agent.yaml`. The relevant environment variables are `ENABLE_INSTRUMENTATION` and `ENABLE_SENSITIVE_DATA`, which can be set to `true` to enable diagnostics and capture sensitive events respectively.
|
||||
Agent Framework is [**natively instrumented**](https://learn.microsoft.com/en-us/agent-framework/agents/observability?pivots=programming-language-python) to capture diagnostics and telemetry for agent execution. Instrumentation is enabled by default. To also capture sensitive event payloads (prompts, tool arguments, etc.) set `ENABLE_SENSITIVE_DATA=true`. This sample demonstrates how to manage these settings via environment variables in `agent.manifest.yaml` and `agent.yaml`.
|
||||
|
||||
Foundry Hosted Agent has built-in observability thus you don't need to set up exporters manually to capture telemetry from your code. The traces, metrics, and logs generated by the agent are automatically collected and made available through Foundry's observability stack via Azure Monitor/Application Insights. The `APPLICATIONINSIGHTS_CONNECTION_STRING` environment variable is injected when the agent is deployed to Foundry, however it is still required to be set in your environment if you want to run the agent host locally and have telemetry sent to Application Insights from your local environment.
|
||||
|
||||
|
||||
-2
@@ -17,8 +17,6 @@ template:
|
||||
environment_variables:
|
||||
- name: AZURE_AI_MODEL_DEPLOYMENT_NAME
|
||||
value: "{{AZURE_AI_MODEL_DEPLOYMENT_NAME}}"
|
||||
- name: ENABLE_INSTRUMENTATION
|
||||
value: true
|
||||
- name: ENABLE_SENSITIVE_DATA
|
||||
value: true
|
||||
resources:
|
||||
|
||||
+3
-5
@@ -5,12 +5,10 @@ protocols:
|
||||
- protocol: responses
|
||||
version: 1.0.0
|
||||
resources:
|
||||
cpu: '0.25'
|
||||
memory: '0.5Gi'
|
||||
cpu: "0.25"
|
||||
memory: "0.5Gi"
|
||||
environment_variables:
|
||||
- name: AZURE_AI_MODEL_DEPLOYMENT_NAME
|
||||
value: ${AZURE_AI_MODEL_DEPLOYMENT_NAME}
|
||||
- name: ENABLE_INSTRUMENTATION
|
||||
value: true
|
||||
- name: ENABLE_SENSITIVE_DATA
|
||||
value: true
|
||||
value: true
|
||||
|
||||
+16
-16
@@ -90,7 +90,7 @@ Example values below are illustrative. For entries not backed by a single public
|
||||
column names the closest public surface, helper, or package-level initialization point that reads the
|
||||
variable.
|
||||
|
||||
| package | class | env var | example value |
|
||||
| package | class/module | env var | example value |
|
||||
| --- | --- | --- | --- |
|
||||
| `agent-framework-anthropic` | `AnthropicClient` | `ANTHROPIC_API_KEY` | `sk-ant-api03-...` |
|
||||
| `agent-framework-anthropic` | `AnthropicClient` | `ANTHROPIC_CHAT_MODEL` | `claude-sonnet-4-5-20250929` |
|
||||
@@ -117,21 +117,21 @@ variable.
|
||||
| `agent-framework-copilotstudio` | `CopilotStudioAgent` | `COPILOTSTUDIOAGENT__SCHEMANAME` | `cr123_agentname` |
|
||||
| `agent-framework-copilotstudio` | `CopilotStudioAgent` | `COPILOTSTUDIOAGENT__TENANTID` | `11111111-1111-1111-1111-111111111111` |
|
||||
| `agent-framework-copilotstudio` | `CopilotStudioAgent` | `COPILOTSTUDIOAGENT__AGENTAPPID` | `22222222-2222-2222-2222-222222222222` |
|
||||
| `agent-framework-core` | `enable_instrumentation()` | `ENABLE_INSTRUMENTATION` | `true` |
|
||||
| `agent-framework-core` | `enable_instrumentation()` | `ENABLE_SENSITIVE_DATA` | `false` |
|
||||
| `agent-framework-core` | `enable_instrumentation()` | `ENABLE_CONSOLE_EXPORTERS` | `true` |
|
||||
| `agent-framework-core` | `enable_instrumentation()` | `OTEL_EXPORTER_OTLP_ENDPOINT` | `http://localhost:4317` |
|
||||
| `agent-framework-core` | `enable_instrumentation()` | `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` | `http://localhost:4318/v1/traces` |
|
||||
| `agent-framework-core` | `enable_instrumentation()` | `OTEL_EXPORTER_OTLP_METRICS_ENDPOINT` | `http://localhost:4318/v1/metrics` |
|
||||
| `agent-framework-core` | `enable_instrumentation()` | `OTEL_EXPORTER_OTLP_LOGS_ENDPOINT` | `http://localhost:4318/v1/logs` |
|
||||
| `agent-framework-core` | `enable_instrumentation()` | `OTEL_EXPORTER_OTLP_PROTOCOL` | `grpc` |
|
||||
| `agent-framework-core` | `enable_instrumentation()` | `OTEL_EXPORTER_OTLP_HEADERS` | `api-key=demo` |
|
||||
| `agent-framework-core` | `enable_instrumentation()` | `OTEL_EXPORTER_OTLP_TRACES_HEADERS` | `api-key=trace-demo` |
|
||||
| `agent-framework-core` | `enable_instrumentation()` | `OTEL_EXPORTER_OTLP_METRICS_HEADERS` | `api-key=metric-demo` |
|
||||
| `agent-framework-core` | `enable_instrumentation()` | `OTEL_EXPORTER_OTLP_LOGS_HEADERS` | `api-key=log-demo` |
|
||||
| `agent-framework-core` | `enable_instrumentation()` | `OTEL_SERVICE_NAME` | `sample-agent` |
|
||||
| `agent-framework-core` | `enable_instrumentation()` | `OTEL_SERVICE_VERSION` | `1.0.0` |
|
||||
| `agent-framework-core` | `enable_instrumentation()` | `OTEL_RESOURCE_ATTRIBUTES` | `deployment.environment=dev,service.namespace=agent-framework` |
|
||||
| `agent-framework-core` | `observability` | `ENABLE_INSTRUMENTATION` | `true` |
|
||||
| `agent-framework-core` | `observability` | `ENABLE_SENSITIVE_DATA` | `false` |
|
||||
| `agent-framework-core` | `observability` | `ENABLE_CONSOLE_EXPORTERS` | `true` |
|
||||
| `agent-framework-core` | `observability` | `OTEL_EXPORTER_OTLP_ENDPOINT` | `http://localhost:4317` |
|
||||
| `agent-framework-core` | `observability` | `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` | `http://localhost:4318/v1/traces` |
|
||||
| `agent-framework-core` | `observability` | `OTEL_EXPORTER_OTLP_METRICS_ENDPOINT` | `http://localhost:4318/v1/metrics` |
|
||||
| `agent-framework-core` | `observability` | `OTEL_EXPORTER_OTLP_LOGS_ENDPOINT` | `http://localhost:4318/v1/logs` |
|
||||
| `agent-framework-core` | `observability` | `OTEL_EXPORTER_OTLP_PROTOCOL` | `grpc` |
|
||||
| `agent-framework-core` | `observability` | `OTEL_EXPORTER_OTLP_HEADERS` | `api-key=demo` |
|
||||
| `agent-framework-core` | `observability` | `OTEL_EXPORTER_OTLP_TRACES_HEADERS` | `api-key=trace-demo` |
|
||||
| `agent-framework-core` | `observability` | `OTEL_EXPORTER_OTLP_METRICS_HEADERS` | `api-key=metric-demo` |
|
||||
| `agent-framework-core` | `observability` | `OTEL_EXPORTER_OTLP_LOGS_HEADERS` | `api-key=log-demo` |
|
||||
| `agent-framework-core` | `observability` | `OTEL_SERVICE_NAME` | `sample-agent` |
|
||||
| `agent-framework-core` | `observability` | `OTEL_SERVICE_VERSION` | `1.0.0` |
|
||||
| `agent-framework-core` | `observability` | `OTEL_RESOURCE_ATTRIBUTES` | `deployment.environment=dev,service.namespace=agent-framework` |
|
||||
| `agent-framework-devui` | `DevUI server` | `DEVUI_AUTH_TOKEN` | `my-devui-token` |
|
||||
| `agent-framework-foundry` | `FoundryChatClient` | `FOUNDRY_PROJECT_ENDPOINT` | `https://my-project.services.ai.azure.com/api/projects/my-project` |
|
||||
| `agent-framework-foundry` | `FoundryChatClient` | `FOUNDRY_MODEL` | `gpt-4o` |
|
||||
|
||||
Reference in New Issue
Block a user