mirror of
https://github.com/microsoft/agent-framework.git
synced 2026-06-16 21:04:09 +08:00
[BREAKING] Python: Enable instrumentation by default (#5865)
* Enable instrumentation by default * Update samples * Optimization when span is not recording * Address Copilot comments * Revert uv.lock * Add warning * Formatting * Fix mypy * Add disable_instrumentation() with sticky user-intent semantics Add a public disable_instrumentation() entry point so users can explicitly opt out of Agent Framework telemetry, with a sticky-disable flag that makes the user's intent "leading" — no framework code path (foundry's configure_azure_monitor, configure_otel_providers, enable_instrumentation, enable_sensitive_telemetry, or direct OBSERVABILITY_SETTINGS.enable_* writes) can re-enable instrumentation until the user explicitly clears the disable with enable_instrumentation(force=True) / enable_sensitive_telemetry(force=True). Also addresses the two remaining unresolved review threads on the PR: 1. test_observability_settings_defaults_instrumentation_true pins the new "ENABLE_INSTRUMENTATION defaults to True when env unset" behavior. 2. test_enable_instrumentation_reads_env_sensitive_data restores coverage for the post-import load_dotenv() fallback path. Implementation: - ObservabilitySettings.enable_instrumentation / enable_sensitive_data become properties backed by _enable_*. While _user_disabled is True, the getters return False and the setters drop True writes (defense in depth so third- party writes can't subvert the disable). - Public is_user_disabled read-only property lets integrations (e.g. foundry's configure_azure_monitor) cheaply check the disable state without poking at privates. - enable_instrumentation() and enable_sensitive_telemetry() short-circuit with an info log when disabled; gain a force=True kwarg that clears the disable. - configure_otel_providers() still creates providers / exporters / views so a later force-enable can use them, but logs an info message when called while disabled. - Foundry's FoundryChatClient.configure_azure_monitor and FoundryAgent.configure_azure_monitor early-return when the user has disabled, so Azure Monitor's global providers aren't installed unnecessarily. Tests: 11 new tests covering default-on, env re-read at call time, sticky behavior against each re-enable surface (enable_instrumentation, enable_sensitive_telemetry, configure_otel_providers, direct attribute writes), force=True override, re-arming the disable, and the __all__ export. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: document disable_instrumentation() and force=True paths Add a "Disabling instrumentation" section to the observability sample README that walks through: - The distinction between the ENABLE_INSTRUMENTATION env var (initial, non-sticky) and disable_instrumentation() (process-wide, sticky). - Why the sticky semantics matter: framework integrations like FoundryChatClient.configure_azure_monitor() can call enable_instrumentation() as part of their setup, and the user's opt-out needs to win. - All five surfaces guarded by the sticky disable (property reads, public enable functions, configure_otel_providers, direct attribute writes, is_user_disabled-aware integrations). - The force=True escape hatch on both enable_instrumentation() and enable_sensitive_telemetry(). - How third-party integrations should consult OBSERVABILITY_SETTINGS.is_user_disabled. - The limits of the disable (does not tear down existing providers / in-flight spans / third-party instrumentation, does not persist across processes). Cross-links the new section from the ENABLE_INSTRUMENTATION row in the env vars table. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: soften disable_instrumentation() overclaim about telemetry guarantees Replace 'no telemetry will be emitted no matter what' (which is too strong, since callers can still pass force=True or mutate private attributes) with language framing the disable as a user-intent contract that library and framework code is expected to honor: the framework actively short-circuits the public enable paths, force=True and private-attribute writes are acknowledged as out-of-contract escape hatches that integrations should not use on the user's behalf. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: correct observability Dependencies section - opentelemetry-sdk is no longer a hard dependency; it is lazily imported by create_resource(), create_metric_views(), and configure_otel_providers() with a clear ImportError when missing. Day-to-day instrumentation works with opentelemetry-api alone provided some other component configures the global OpenTelemetry providers (Azure Monitor, an APM agent, application bootstrap, etc.). - opentelemetry-semantic-conventions-ai is no longer used anywhere in the source; remove it from the listed dependencies. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: replace stale observability migration guide with current PR's only relevant migration The old guide documented the move away from setup_observability(otlp_endpoint=...) which was an earlier-release API change unrelated to this PR and stale enough that it's more confusing than helpful at this point. Replace it with a short note on the single migration this PR introduces: callers of enable_instrumentation(enable_sensitive_data=True) should switch to enable_sensitive_telemetry(). Cross-link to the Disabling instrumentation section for the rare 'force on without enabling sensitive data' use case where enable_instrumentation() still applies. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Eduard van Valkenburg <eavanvalkenburg@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
Tao Chen
committed by
GitHub
Unverified
GitHub
parent
0ba552b84c
commit
72a6157c6a
@@ -4,6 +4,8 @@
|
||||
|
||||
Commonly used exports:
|
||||
- enable_instrumentation
|
||||
- disable_instrumentation
|
||||
- enable_sensitive_telemetry
|
||||
- configure_otel_providers
|
||||
- AgentTelemetryLayer
|
||||
- ChatTelemetryLayer
|
||||
@@ -80,7 +82,9 @@ __all__ = [
|
||||
"configure_otel_providers",
|
||||
"create_metric_views",
|
||||
"create_resource",
|
||||
"disable_instrumentation",
|
||||
"enable_instrumentation",
|
||||
"enable_sensitive_telemetry",
|
||||
"get_meter",
|
||||
"get_tracer",
|
||||
]
|
||||
@@ -643,8 +647,8 @@ class ObservabilitySettings:
|
||||
Sensitive events should only be enabled on test and development environments.
|
||||
|
||||
Keyword Args:
|
||||
enable_instrumentation: Enable OpenTelemetry diagnostics. Default is False.
|
||||
Can be set via environment variable ENABLE_INSTRUMENTATION.
|
||||
enable_instrumentation: Enable OpenTelemetry diagnostics. Default is True.
|
||||
Can be disabled by setting environment variable ENABLE_INSTRUMENTATION=false.
|
||||
enable_sensitive_data: Enable OpenTelemetry sensitive events. Default is False.
|
||||
Can be set via environment variable ENABLE_SENSITIVE_DATA.
|
||||
enable_console_exporters: Enable console exporters for traces, logs, and metrics.
|
||||
@@ -659,12 +663,12 @@ class ObservabilitySettings:
|
||||
from agent_framework import ObservabilitySettings
|
||||
|
||||
# Using environment variables
|
||||
# Set ENABLE_INSTRUMENTATION=true
|
||||
# Instrumentation is enabled by default; set ENABLE_INSTRUMENTATION=false to disable.
|
||||
# Set ENABLE_CONSOLE_EXPORTERS=true
|
||||
settings = ObservabilitySettings()
|
||||
|
||||
# Or passing parameters directly
|
||||
settings = ObservabilitySettings(enable_instrumentation=True, enable_console_exporters=True)
|
||||
settings = ObservabilitySettings(enable_console_exporters=True)
|
||||
"""
|
||||
|
||||
def __init__(self, **kwargs: Any) -> None:
|
||||
@@ -677,14 +681,74 @@ class ObservabilitySettings:
|
||||
env_file_encoding=env_file_encoding,
|
||||
**kwargs,
|
||||
)
|
||||
self.enable_instrumentation: bool = data.get("enable_instrumentation") or False
|
||||
self.enable_sensitive_data: bool = data.get("enable_sensitive_data") or False
|
||||
# Sticky-disable flag, set by `disable_instrumentation()`. When True, this
|
||||
# singleton refuses to be re-enabled by any subsequent assignment to the
|
||||
# `enable_instrumentation` / `enable_sensitive_data` properties (including
|
||||
# direct third-party writes). It can only be cleared by an explicit
|
||||
# `enable_instrumentation(force=True)` / `enable_sensitive_telemetry(force=True)`
|
||||
# call, which is the user re-stating their intent.
|
||||
self._user_disabled: bool = False
|
||||
# `enable_instrumentation` is defaulted to True if not set
|
||||
instrumentation_value = data.get("enable_instrumentation")
|
||||
self._enable_instrumentation: bool = True if instrumentation_value is None else instrumentation_value
|
||||
self._enable_sensitive_data: bool = data.get("enable_sensitive_data") or False
|
||||
if self._enable_sensitive_data and not self._enable_instrumentation:
|
||||
logger.warning(
|
||||
"Sensitive data capture is enabled but instrumentation is disabled. "
|
||||
"Sensitive data will not be captured. Please enable instrumentation to capture sensitive data."
|
||||
)
|
||||
|
||||
self.enable_console_exporters: bool = data.get("enable_console_exporters") or False
|
||||
self.vs_code_extension_port: int | None = data.get("vs_code_extension_port")
|
||||
self.env_file_path = env_file_path
|
||||
self.env_file_encoding = env_file_encoding
|
||||
self._executed_setup = False
|
||||
|
||||
@property
|
||||
def enable_instrumentation(self) -> bool:
|
||||
"""Whether instrumentation is enabled.
|
||||
|
||||
Always returns False once ``disable_instrumentation()`` has been called,
|
||||
regardless of the stored value, until ``enable_instrumentation(force=True)``
|
||||
clears the sticky disable.
|
||||
"""
|
||||
if self._user_disabled:
|
||||
return False
|
||||
return self._enable_instrumentation
|
||||
|
||||
@enable_instrumentation.setter
|
||||
def enable_instrumentation(self, value: bool) -> None:
|
||||
if self._user_disabled and value:
|
||||
# Defense in depth: a third-party (or internal) write of True is
|
||||
# silently dropped while the user-disabled flag is set, so the
|
||||
# sticky disable cannot be circumvented by direct attribute writes.
|
||||
logger.debug(
|
||||
"Ignoring enable_instrumentation=True assignment: instrumentation was explicitly disabled via "
|
||||
"disable_instrumentation(). Call enable_instrumentation(force=True) to clear the disable."
|
||||
)
|
||||
return
|
||||
self._enable_instrumentation = value
|
||||
|
||||
@property
|
||||
def enable_sensitive_data(self) -> bool:
|
||||
"""Whether sensitive-data capture is enabled.
|
||||
|
||||
Always returns False once ``disable_instrumentation()`` has been called.
|
||||
"""
|
||||
if self._user_disabled:
|
||||
return False
|
||||
return self._enable_sensitive_data
|
||||
|
||||
@enable_sensitive_data.setter
|
||||
def enable_sensitive_data(self, value: bool) -> None:
|
||||
if self._user_disabled and value:
|
||||
logger.debug(
|
||||
"Ignoring enable_sensitive_data=True assignment: instrumentation was explicitly disabled via "
|
||||
"disable_instrumentation(). Call enable_sensitive_telemetry(force=True) to clear the disable."
|
||||
)
|
||||
return
|
||||
self._enable_sensitive_data = value
|
||||
|
||||
@property
|
||||
def ENABLED(self) -> bool:
|
||||
"""Check if model diagnostics are enabled.
|
||||
@@ -706,6 +770,17 @@ class ObservabilitySettings:
|
||||
"""Check if the setup has been executed."""
|
||||
return self._executed_setup
|
||||
|
||||
@property
|
||||
def is_user_disabled(self) -> bool:
|
||||
"""Whether ``disable_instrumentation()`` has been called and the disable is still in effect.
|
||||
|
||||
Integrations that perform telemetry setup as a side-effect (e.g. provisioning Azure Monitor
|
||||
providers from a Foundry project's connection string) should consult this flag before doing
|
||||
their setup work, so the user's explicit opt-out is respected end-to-end and not just at the
|
||||
framework's span-emission boundary.
|
||||
"""
|
||||
return self._user_disabled
|
||||
|
||||
def _configure(
|
||||
self,
|
||||
*,
|
||||
@@ -951,24 +1026,91 @@ def _read_int_env(name: str, *, default: int | None = None) -> int | None:
|
||||
return default
|
||||
|
||||
|
||||
def enable_sensitive_telemetry(*, force: bool = False) -> None:
|
||||
"""Enable capture of sensitive data in telemetry for your application.
|
||||
|
||||
Instrumentation is enabled by default; this method exists to opt-in to capturing
|
||||
sensitive event payloads (e.g., chat messages, tool arguments).
|
||||
|
||||
This method does not configure exporters or providers. It also ensures that
|
||||
instrumentation is enabled (in case it was explicitly disabled via the
|
||||
ENABLE_INSTRUMENTATION environment variable).
|
||||
|
||||
Keyword Args:
|
||||
force: When True, clears any sticky disable previously set by
|
||||
``disable_instrumentation()`` before enabling. Without it, calls are
|
||||
no-ops if instrumentation has been explicitly disabled.
|
||||
|
||||
Warning:
|
||||
Sensitive events should only be enabled on test and development environments.
|
||||
"""
|
||||
global OBSERVABILITY_SETTINGS
|
||||
if OBSERVABILITY_SETTINGS._user_disabled and not force: # type: ignore[reportPrivateUsage]
|
||||
logger.info(
|
||||
"enable_sensitive_telemetry() ignored: instrumentation was explicitly disabled via "
|
||||
"disable_instrumentation(). Pass force=True to re-enable."
|
||||
)
|
||||
return
|
||||
if force:
|
||||
OBSERVABILITY_SETTINGS._user_disabled = False # type: ignore[reportPrivateUsage]
|
||||
OBSERVABILITY_SETTINGS.enable_instrumentation = True
|
||||
OBSERVABILITY_SETTINGS.enable_sensitive_data = True
|
||||
|
||||
|
||||
def disable_instrumentation() -> None:
|
||||
"""Explicitly disable Agent Framework instrumentation for this process.
|
||||
|
||||
The disable is **sticky**: subsequent attempts by framework auto-setup paths,
|
||||
library integrations, ``enable_instrumentation()``, ``enable_sensitive_telemetry()``,
|
||||
``configure_otel_providers()``, or direct writes to
|
||||
``OBSERVABILITY_SETTINGS.enable_instrumentation`` are ignored and no spans, metrics,
|
||||
or logs are emitted by Agent Framework code paths.
|
||||
|
||||
To override the disable later, call ``enable_instrumentation(force=True)`` or
|
||||
``enable_sensitive_telemetry(force=True)``. This makes the user's intent to opt out
|
||||
win against framework code that would otherwise re-enable instrumentation
|
||||
automatically.
|
||||
|
||||
Note:
|
||||
Disabling does not tear down already-configured OpenTelemetry providers,
|
||||
exporters, or in-flight spans; it gates future captures by Agent Framework
|
||||
instrumentation only. To stop emitting telemetry from third-party
|
||||
instrumentations as well, configure them separately.
|
||||
"""
|
||||
global OBSERVABILITY_SETTINGS
|
||||
OBSERVABILITY_SETTINGS._user_disabled = True # type: ignore[reportPrivateUsage]
|
||||
OBSERVABILITY_SETTINGS._enable_instrumentation = False # type: ignore[reportPrivateUsage]
|
||||
OBSERVABILITY_SETTINGS._enable_sensitive_data = False # type: ignore[reportPrivateUsage]
|
||||
|
||||
|
||||
def enable_instrumentation(
|
||||
*,
|
||||
enable_sensitive_data: bool | None = None,
|
||||
force: bool = False,
|
||||
) -> None:
|
||||
"""Enable instrumentation for your application.
|
||||
"""Enable instrumentation for Microsoft Agent Framework.
|
||||
|
||||
Calling this method implies you want to enable observability in your application.
|
||||
|
||||
This method does not configure exporters or providers.
|
||||
It only updates the global variables that trigger the instrumentation code.
|
||||
If you have already set the environment variable ENABLE_INSTRUMENTATION=true,
|
||||
calling this method has no effect, unless you want to enable or disable sensitive data events.
|
||||
Note that instrumentation is enabled by default, so this method is only necessary
|
||||
if you need a programmatic way to enable it (e.g., if you are not sure whether the
|
||||
environment variable ENABLE_INSTRUMENTATION is set to True or False and want to
|
||||
ensure it is enabled).
|
||||
|
||||
Keyword Args:
|
||||
enable_sensitive_data: Enable OpenTelemetry sensitive events. Overrides
|
||||
the environment variable ENABLE_SENSITIVE_DATA if set. Default is None.
|
||||
force: When True, clears any sticky disable previously set by
|
||||
``disable_instrumentation()`` before enabling. Without it, calls are
|
||||
no-ops if instrumentation has been explicitly disabled.
|
||||
"""
|
||||
global OBSERVABILITY_SETTINGS
|
||||
if OBSERVABILITY_SETTINGS._user_disabled and not force: # type: ignore[reportPrivateUsage]
|
||||
logger.info(
|
||||
"enable_instrumentation() ignored: instrumentation was explicitly disabled via "
|
||||
"disable_instrumentation(). Pass force=True to re-enable."
|
||||
)
|
||||
return
|
||||
if force:
|
||||
OBSERVABILITY_SETTINGS._user_disabled = False # type: ignore[reportPrivateUsage]
|
||||
OBSERVABILITY_SETTINGS.enable_instrumentation = True
|
||||
if enable_sensitive_data is not None:
|
||||
OBSERVABILITY_SETTINGS.enable_sensitive_data = enable_sensitive_data
|
||||
@@ -1008,7 +1150,7 @@ def configure_otel_providers(
|
||||
Since you can only setup one provider per signal type (logs, traces, metrics),
|
||||
you can choose to use this method and take the exporter and provider that we created.
|
||||
Alternatively, you can setup the providers yourself, or through another library
|
||||
(e.g., Azure Monitor) and just call `enable_instrumentation()` to enable instrumentation.
|
||||
(e.g., Azure Monitor) and just call `enable_sensitive_telemetry()` to opt-in to sensitive data capture.
|
||||
|
||||
Note:
|
||||
By default, the Agent Framework emits metrics with the prefixes `agent_framework`
|
||||
@@ -1042,7 +1184,6 @@ def configure_otel_providers(
|
||||
from agent_framework.observability import configure_otel_providers
|
||||
|
||||
# Using environment variables (recommended)
|
||||
# Set ENABLE_INSTRUMENTATION=true
|
||||
# Set OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
|
||||
configure_otel_providers()
|
||||
|
||||
@@ -1087,18 +1228,25 @@ def configure_otel_providers(
|
||||
.. code-block:: python
|
||||
|
||||
# when azure monitor is installed
|
||||
from agent_framework.observability import enable_instrumentation
|
||||
from agent_framework.observability import enable_sensitive_telemetry
|
||||
from azure.monitor.opentelemetry import configure_azure_monitor
|
||||
|
||||
connection_string = "InstrumentationKey=your_instrumentation_key_here;..."
|
||||
configure_azure_monitor(connection_string=connection_string)
|
||||
enable_instrumentation()
|
||||
# Optional: opt into capturing sensitive data
|
||||
enable_sensitive_telemetry()
|
||||
|
||||
References:
|
||||
- https://opentelemetry.io/docs/languages/sdk-configuration/general/
|
||||
- https://opentelemetry.io/docs/languages/sdk-configuration/otlp-exporter/
|
||||
"""
|
||||
global OBSERVABILITY_SETTINGS
|
||||
if OBSERVABILITY_SETTINGS._user_disabled: # type: ignore[reportPrivateUsage]
|
||||
logger.info(
|
||||
"configure_otel_providers(): instrumentation was explicitly disabled via "
|
||||
"disable_instrumentation(); providers and exporters will still be configured but "
|
||||
"Agent Framework will emit no telemetry until enable_instrumentation(force=True) is called."
|
||||
)
|
||||
if env_file_path:
|
||||
# Build kwargs, excluding None values
|
||||
settings_kwargs: dict[str, Any] = {
|
||||
@@ -1280,7 +1428,7 @@ class ChatTelemetryLayer(Generic[OptionsCoT]):
|
||||
if stream:
|
||||
span = _start_streaming_span(attributes, OtelAttr.REQUEST_MODEL)
|
||||
|
||||
if OBSERVABILITY_SETTINGS.SENSITIVE_DATA_ENABLED and messages:
|
||||
if OBSERVABILITY_SETTINGS.SENSITIVE_DATA_ENABLED and messages and span.is_recording():
|
||||
_capture_messages(
|
||||
span=span,
|
||||
provider_name=provider_name,
|
||||
@@ -1344,6 +1492,7 @@ class ChatTelemetryLayer(Generic[OptionsCoT]):
|
||||
OBSERVABILITY_SETTINGS.SENSITIVE_DATA_ENABLED
|
||||
and isinstance(response, ChatResponse)
|
||||
and response.messages
|
||||
and span.is_recording()
|
||||
):
|
||||
_capture_messages(
|
||||
span=span,
|
||||
@@ -1374,7 +1523,7 @@ class ChatTelemetryLayer(Generic[OptionsCoT]):
|
||||
|
||||
async def _get_response() -> ChatResponse:
|
||||
with _get_span(attributes=attributes, span_name_attribute=OtelAttr.REQUEST_MODEL) as span:
|
||||
if OBSERVABILITY_SETTINGS.SENSITIVE_DATA_ENABLED and messages:
|
||||
if OBSERVABILITY_SETTINGS.SENSITIVE_DATA_ENABLED and messages and span.is_recording():
|
||||
_capture_messages(
|
||||
span=span,
|
||||
provider_name=provider_name,
|
||||
@@ -1408,7 +1557,7 @@ class ChatTelemetryLayer(Generic[OptionsCoT]):
|
||||
duration=duration,
|
||||
)
|
||||
_mark_inner_response_telemetry_captured(response)
|
||||
if OBSERVABILITY_SETTINGS.SENSITIVE_DATA_ENABLED and response.messages:
|
||||
if OBSERVABILITY_SETTINGS.SENSITIVE_DATA_ENABLED and response.messages and span.is_recording():
|
||||
finish_reason = cast(
|
||||
"FinishReason | None",
|
||||
response.finish_reason if response.finish_reason in FINISH_REASON_MAP else None,
|
||||
@@ -1552,7 +1701,7 @@ class AgentTelemetryLayer:
|
||||
if stream:
|
||||
span = _start_streaming_span(attributes, OtelAttr.AGENT_NAME)
|
||||
|
||||
if OBSERVABILITY_SETTINGS.SENSITIVE_DATA_ENABLED and messages:
|
||||
if OBSERVABILITY_SETTINGS.SENSITIVE_DATA_ENABLED and messages and span.is_recording():
|
||||
_capture_messages(
|
||||
span=span,
|
||||
provider_name=provider_name,
|
||||
@@ -1613,6 +1762,7 @@ class AgentTelemetryLayer:
|
||||
OBSERVABILITY_SETTINGS.SENSITIVE_DATA_ENABLED
|
||||
and isinstance(response, AgentResponse)
|
||||
and response.messages
|
||||
and span.is_recording()
|
||||
):
|
||||
_capture_messages(
|
||||
span=span,
|
||||
@@ -1645,7 +1795,7 @@ class AgentTelemetryLayer:
|
||||
async def _run() -> AgentResponse[Any]:
|
||||
try:
|
||||
with _get_span(attributes=attributes, span_name_attribute=OtelAttr.AGENT_NAME) as span:
|
||||
if OBSERVABILITY_SETTINGS.SENSITIVE_DATA_ENABLED and messages:
|
||||
if OBSERVABILITY_SETTINGS.SENSITIVE_DATA_ENABLED and messages and span.is_recording():
|
||||
_capture_messages(
|
||||
span=span,
|
||||
provider_name=provider_name,
|
||||
@@ -1669,7 +1819,7 @@ class AgentTelemetryLayer:
|
||||
)
|
||||
_apply_accumulated_usage(response_attributes, inner_response_telemetry_captured_fields)
|
||||
_capture_response(span=span, attributes=response_attributes, duration=duration)
|
||||
if OBSERVABILITY_SETTINGS.SENSITIVE_DATA_ENABLED and response.messages:
|
||||
if OBSERVABILITY_SETTINGS.SENSITIVE_DATA_ENABLED and response.messages and span.is_recording():
|
||||
_capture_messages(
|
||||
span=span,
|
||||
provider_name=provider_name,
|
||||
|
||||
@@ -1015,11 +1015,25 @@ def test_observability_settings_is_setup_initial(monkeypatch):
|
||||
assert settings.is_setup is False
|
||||
|
||||
|
||||
# region Test enable_instrumentation function
|
||||
def test_enable_sensitive_telemetry_function(monkeypatch):
|
||||
"""Test enable_sensitive_telemetry function enables instrumentation."""
|
||||
import importlib
|
||||
|
||||
monkeypatch.setenv("ENABLE_INSTRUMENTATION", "false")
|
||||
monkeypatch.setenv("ENABLE_SENSITIVE_DATA", "false")
|
||||
|
||||
observability = importlib.import_module("agent_framework.observability")
|
||||
importlib.reload(observability)
|
||||
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_instrumentation is False
|
||||
|
||||
observability.enable_sensitive_telemetry()
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_instrumentation is True
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_sensitive_data is True
|
||||
|
||||
|
||||
def test_enable_instrumentation_function(monkeypatch):
|
||||
"""Test enable_instrumentation function enables instrumentation."""
|
||||
"""Test enable_instrumentation function enables instrumentation when disabled via env."""
|
||||
import importlib
|
||||
|
||||
monkeypatch.setenv("ENABLE_INSTRUMENTATION", "false")
|
||||
@@ -1032,10 +1046,12 @@ def test_enable_instrumentation_function(monkeypatch):
|
||||
|
||||
observability.enable_instrumentation()
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_instrumentation is True
|
||||
# Sensitive data should remain False when not explicitly enabled
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_sensitive_data is False
|
||||
|
||||
|
||||
def test_enable_instrumentation_with_sensitive_data(monkeypatch):
|
||||
"""Test enable_instrumentation function with sensitive_data parameter."""
|
||||
"""Test enable_instrumentation function with explicit sensitive_data parameter."""
|
||||
import importlib
|
||||
|
||||
monkeypatch.setenv("ENABLE_INSTRUMENTATION", "false")
|
||||
@@ -1049,111 +1065,6 @@ def test_enable_instrumentation_with_sensitive_data(monkeypatch):
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_sensitive_data is True
|
||||
|
||||
|
||||
def test_enable_instrumentation_reads_env_sensitive_data(monkeypatch):
|
||||
"""Test enable_instrumentation re-reads ENABLE_SENSITIVE_DATA from os.environ when not explicitly passed."""
|
||||
import importlib
|
||||
|
||||
monkeypatch.setenv("ENABLE_INSTRUMENTATION", "false")
|
||||
monkeypatch.setenv("ENABLE_SENSITIVE_DATA", "false")
|
||||
|
||||
observability = importlib.import_module("agent_framework.observability")
|
||||
importlib.reload(observability)
|
||||
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_sensitive_data is False
|
||||
|
||||
# Simulate load_dotenv() setting env var after import
|
||||
monkeypatch.setenv("ENABLE_SENSITIVE_DATA", "true")
|
||||
|
||||
observability.enable_instrumentation()
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_instrumentation is True
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_sensitive_data is True
|
||||
|
||||
|
||||
def test_configure_otel_providers_reads_env_sensitive_data(monkeypatch):
|
||||
"""Test configure_otel_providers re-reads ENABLE_SENSITIVE_DATA from os.environ when not explicitly passed."""
|
||||
import importlib
|
||||
|
||||
monkeypatch.setenv("ENABLE_INSTRUMENTATION", "false")
|
||||
monkeypatch.setenv("ENABLE_SENSITIVE_DATA", "false")
|
||||
monkeypatch.delenv("VS_CODE_EXTENSION_PORT", raising=False)
|
||||
monkeypatch.delenv("ENABLE_CONSOLE_EXPORTERS", raising=False)
|
||||
for key in [
|
||||
"OTEL_EXPORTER_OTLP_ENDPOINT",
|
||||
"OTEL_EXPORTER_OTLP_TRACES_ENDPOINT",
|
||||
"OTEL_EXPORTER_OTLP_METRICS_ENDPOINT",
|
||||
"OTEL_EXPORTER_OTLP_LOGS_ENDPOINT",
|
||||
]:
|
||||
monkeypatch.delenv(key, raising=False)
|
||||
|
||||
observability = importlib.import_module("agent_framework.observability")
|
||||
importlib.reload(observability)
|
||||
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_sensitive_data is False
|
||||
|
||||
# Simulate load_dotenv() setting env var after import
|
||||
monkeypatch.setenv("ENABLE_SENSITIVE_DATA", "true")
|
||||
|
||||
with patch.object(observability.OBSERVABILITY_SETTINGS, "_configure"):
|
||||
observability.configure_otel_providers()
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_instrumentation is True
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_sensitive_data is True
|
||||
|
||||
|
||||
def test_configure_otel_providers_reads_env_vs_code_port(monkeypatch):
|
||||
"""Test configure_otel_providers re-reads VS_CODE_EXTENSION_PORT from os.environ when not explicitly passed."""
|
||||
import importlib
|
||||
from unittest.mock import patch as mock_patch
|
||||
|
||||
monkeypatch.setenv("ENABLE_INSTRUMENTATION", "false")
|
||||
monkeypatch.delenv("VS_CODE_EXTENSION_PORT", raising=False)
|
||||
monkeypatch.delenv("ENABLE_CONSOLE_EXPORTERS", raising=False)
|
||||
for key in [
|
||||
"OTEL_EXPORTER_OTLP_ENDPOINT",
|
||||
"OTEL_EXPORTER_OTLP_TRACES_ENDPOINT",
|
||||
"OTEL_EXPORTER_OTLP_METRICS_ENDPOINT",
|
||||
"OTEL_EXPORTER_OTLP_LOGS_ENDPOINT",
|
||||
]:
|
||||
monkeypatch.delenv(key, raising=False)
|
||||
|
||||
observability = importlib.import_module("agent_framework.observability")
|
||||
importlib.reload(observability)
|
||||
|
||||
assert observability.OBSERVABILITY_SETTINGS.vs_code_extension_port is None
|
||||
|
||||
# Simulate load_dotenv() setting env var after import
|
||||
monkeypatch.setenv("VS_CODE_EXTENSION_PORT", "4317")
|
||||
|
||||
# Mock _configure to avoid needing optional OTLP gRPC exporter dependency
|
||||
with mock_patch.object(observability.OBSERVABILITY_SETTINGS, "_configure"):
|
||||
observability.configure_otel_providers()
|
||||
assert observability.OBSERVABILITY_SETTINGS.vs_code_extension_port == 4317
|
||||
|
||||
|
||||
def test_configure_otel_providers_explicit_param_overrides_env(monkeypatch):
|
||||
"""Test that explicit parameters to configure_otel_providers override env vars."""
|
||||
import importlib
|
||||
|
||||
monkeypatch.setenv("ENABLE_INSTRUMENTATION", "false")
|
||||
monkeypatch.setenv("ENABLE_SENSITIVE_DATA", "true")
|
||||
monkeypatch.delenv("VS_CODE_EXTENSION_PORT", raising=False)
|
||||
monkeypatch.delenv("ENABLE_CONSOLE_EXPORTERS", raising=False)
|
||||
for key in [
|
||||
"OTEL_EXPORTER_OTLP_ENDPOINT",
|
||||
"OTEL_EXPORTER_OTLP_TRACES_ENDPOINT",
|
||||
"OTEL_EXPORTER_OTLP_METRICS_ENDPOINT",
|
||||
"OTEL_EXPORTER_OTLP_LOGS_ENDPOINT",
|
||||
]:
|
||||
monkeypatch.delenv(key, raising=False)
|
||||
|
||||
observability = importlib.import_module("agent_framework.observability")
|
||||
importlib.reload(observability)
|
||||
|
||||
# Explicit False should override the env var True
|
||||
with patch.object(observability.OBSERVABILITY_SETTINGS, "_configure"):
|
||||
observability.configure_otel_providers(enable_sensitive_data=False)
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_sensitive_data is False
|
||||
|
||||
|
||||
def test_enable_instrumentation_explicit_param_overrides_env(monkeypatch):
|
||||
"""Test that explicit enable_sensitive_data parameter to enable_instrumentation overrides env var."""
|
||||
import importlib
|
||||
@@ -1269,6 +1180,161 @@ def test_enable_instrumentation_preserves_console_exporters_after_env_removed(mo
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_console_exporters is True
|
||||
|
||||
|
||||
def test_configure_otel_providers_reads_env_sensitive_data(monkeypatch):
|
||||
"""Test configure_otel_providers re-reads ENABLE_SENSITIVE_DATA from os.environ when not explicitly passed."""
|
||||
import importlib
|
||||
|
||||
monkeypatch.setenv("ENABLE_INSTRUMENTATION", "false")
|
||||
monkeypatch.setenv("ENABLE_SENSITIVE_DATA", "false")
|
||||
monkeypatch.delenv("VS_CODE_EXTENSION_PORT", raising=False)
|
||||
monkeypatch.delenv("ENABLE_CONSOLE_EXPORTERS", raising=False)
|
||||
for key in [
|
||||
"OTEL_EXPORTER_OTLP_ENDPOINT",
|
||||
"OTEL_EXPORTER_OTLP_TRACES_ENDPOINT",
|
||||
"OTEL_EXPORTER_OTLP_METRICS_ENDPOINT",
|
||||
"OTEL_EXPORTER_OTLP_LOGS_ENDPOINT",
|
||||
]:
|
||||
monkeypatch.delenv(key, raising=False)
|
||||
|
||||
observability = importlib.import_module("agent_framework.observability")
|
||||
importlib.reload(observability)
|
||||
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_sensitive_data is False
|
||||
|
||||
# Simulate load_dotenv() setting env var after import
|
||||
monkeypatch.setenv("ENABLE_SENSITIVE_DATA", "true")
|
||||
|
||||
with patch.object(observability.OBSERVABILITY_SETTINGS, "_configure"):
|
||||
observability.configure_otel_providers()
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_instrumentation is True
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_sensitive_data is True
|
||||
|
||||
|
||||
def test_configure_otel_providers_reads_env_vs_code_port(monkeypatch):
|
||||
"""Test configure_otel_providers re-reads VS_CODE_EXTENSION_PORT from os.environ when not explicitly passed."""
|
||||
import importlib
|
||||
from unittest.mock import patch as mock_patch
|
||||
|
||||
monkeypatch.setenv("ENABLE_INSTRUMENTATION", "false")
|
||||
monkeypatch.delenv("VS_CODE_EXTENSION_PORT", raising=False)
|
||||
monkeypatch.delenv("ENABLE_CONSOLE_EXPORTERS", raising=False)
|
||||
for key in [
|
||||
"OTEL_EXPORTER_OTLP_ENDPOINT",
|
||||
"OTEL_EXPORTER_OTLP_TRACES_ENDPOINT",
|
||||
"OTEL_EXPORTER_OTLP_METRICS_ENDPOINT",
|
||||
"OTEL_EXPORTER_OTLP_LOGS_ENDPOINT",
|
||||
]:
|
||||
monkeypatch.delenv(key, raising=False)
|
||||
|
||||
observability = importlib.import_module("agent_framework.observability")
|
||||
importlib.reload(observability)
|
||||
|
||||
assert observability.OBSERVABILITY_SETTINGS.vs_code_extension_port is None
|
||||
|
||||
# Simulate load_dotenv() setting env var after import
|
||||
monkeypatch.setenv("VS_CODE_EXTENSION_PORT", "4317")
|
||||
|
||||
# Mock _configure to avoid needing optional OTLP gRPC exporter dependency
|
||||
with mock_patch.object(observability.OBSERVABILITY_SETTINGS, "_configure"):
|
||||
observability.configure_otel_providers()
|
||||
assert observability.OBSERVABILITY_SETTINGS.vs_code_extension_port == 4317
|
||||
|
||||
|
||||
def test_configure_otel_providers_explicit_param_overrides_env(monkeypatch):
|
||||
"""Test that explicit parameters to configure_otel_providers override env vars."""
|
||||
import importlib
|
||||
|
||||
monkeypatch.setenv("ENABLE_INSTRUMENTATION", "false")
|
||||
monkeypatch.setenv("ENABLE_SENSITIVE_DATA", "true")
|
||||
monkeypatch.delenv("VS_CODE_EXTENSION_PORT", raising=False)
|
||||
monkeypatch.delenv("ENABLE_CONSOLE_EXPORTERS", raising=False)
|
||||
for key in [
|
||||
"OTEL_EXPORTER_OTLP_ENDPOINT",
|
||||
"OTEL_EXPORTER_OTLP_TRACES_ENDPOINT",
|
||||
"OTEL_EXPORTER_OTLP_METRICS_ENDPOINT",
|
||||
"OTEL_EXPORTER_OTLP_LOGS_ENDPOINT",
|
||||
]:
|
||||
monkeypatch.delenv(key, raising=False)
|
||||
|
||||
observability = importlib.import_module("agent_framework.observability")
|
||||
importlib.reload(observability)
|
||||
|
||||
# Explicit False should override the env var True
|
||||
with patch.object(observability.OBSERVABILITY_SETTINGS, "_configure"):
|
||||
observability.configure_otel_providers(enable_sensitive_data=False)
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_sensitive_data is False
|
||||
|
||||
|
||||
def test_enable_sensitive_telemetry_does_not_touch_console_exporters(monkeypatch):
|
||||
"""Test enable_sensitive_telemetry does not modify enable_console_exporters (it is an exporter concern)."""
|
||||
import importlib
|
||||
|
||||
monkeypatch.setenv("ENABLE_INSTRUMENTATION", "false")
|
||||
monkeypatch.delenv("ENABLE_CONSOLE_EXPORTERS", raising=False)
|
||||
|
||||
observability = importlib.import_module("agent_framework.observability")
|
||||
importlib.reload(observability)
|
||||
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_console_exporters is False
|
||||
|
||||
# Simulate load_dotenv() setting env var after import
|
||||
monkeypatch.setenv("ENABLE_CONSOLE_EXPORTERS", "true")
|
||||
|
||||
observability.enable_sensitive_telemetry()
|
||||
# enable_console_exporters is not managed by enable_sensitive_telemetry;
|
||||
# it is only read by configure_otel_providers.
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_console_exporters is False
|
||||
|
||||
|
||||
def test_enable_sensitive_telemetry_does_not_clobber_console_exporters(monkeypatch):
|
||||
"""Test enable_sensitive_telemetry does not reset enable_console_exporters set by prior configure call."""
|
||||
import importlib
|
||||
|
||||
monkeypatch.setenv("ENABLE_INSTRUMENTATION", "false")
|
||||
monkeypatch.delenv("ENABLE_CONSOLE_EXPORTERS", raising=False)
|
||||
monkeypatch.delenv("ENABLE_SENSITIVE_DATA", raising=False)
|
||||
monkeypatch.delenv("VS_CODE_EXTENSION_PORT", raising=False)
|
||||
for key in [
|
||||
"OTEL_EXPORTER_OTLP_ENDPOINT",
|
||||
"OTEL_EXPORTER_OTLP_TRACES_ENDPOINT",
|
||||
"OTEL_EXPORTER_OTLP_METRICS_ENDPOINT",
|
||||
"OTEL_EXPORTER_OTLP_LOGS_ENDPOINT",
|
||||
]:
|
||||
monkeypatch.delenv(key, raising=False)
|
||||
|
||||
observability = importlib.import_module("agent_framework.observability")
|
||||
importlib.reload(observability)
|
||||
|
||||
# Set console exporters via configure_otel_providers
|
||||
with patch.object(observability.OBSERVABILITY_SETTINGS, "_configure"):
|
||||
observability.configure_otel_providers(enable_console_exporters=True)
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_console_exporters is True
|
||||
|
||||
# Calling enable_sensitive_telemetry should not clobber the value
|
||||
observability.enable_sensitive_telemetry()
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_console_exporters is True
|
||||
|
||||
|
||||
def test_enable_sensitive_telemetry_preserves_console_exporters_after_env_removed(monkeypatch):
|
||||
"""Test enable_sensitive_telemetry preserves enable_console_exporters when env var is removed after reload."""
|
||||
import importlib
|
||||
|
||||
monkeypatch.setenv("ENABLE_INSTRUMENTATION", "false")
|
||||
monkeypatch.setenv("ENABLE_CONSOLE_EXPORTERS", "true")
|
||||
|
||||
observability = importlib.import_module("agent_framework.observability")
|
||||
importlib.reload(observability)
|
||||
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_console_exporters is True
|
||||
|
||||
# Remove the env var after reload
|
||||
monkeypatch.delenv("ENABLE_CONSOLE_EXPORTERS", raising=False)
|
||||
|
||||
# enable_sensitive_telemetry should not reset the value
|
||||
observability.enable_sensitive_telemetry()
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_console_exporters is True
|
||||
|
||||
|
||||
def test_configure_otel_providers_reads_env_console_exporters(monkeypatch):
|
||||
"""Test configure_otel_providers re-reads ENABLE_CONSOLE_EXPORTERS from os.environ when not explicitly passed."""
|
||||
import importlib
|
||||
@@ -1321,6 +1387,189 @@ def test_configure_otel_providers_explicit_console_exporters_overrides_env(monke
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_console_exporters is False
|
||||
|
||||
|
||||
# region Test default-on instrumentation
|
||||
|
||||
|
||||
def test_observability_settings_defaults_instrumentation_true(monkeypatch):
|
||||
"""ENABLE_INSTRUMENTATION unset → ObservabilitySettings defaults to True."""
|
||||
from agent_framework.observability import ObservabilitySettings
|
||||
|
||||
monkeypatch.delenv("ENABLE_INSTRUMENTATION", raising=False)
|
||||
settings = ObservabilitySettings()
|
||||
assert settings.enable_instrumentation is True
|
||||
|
||||
|
||||
def test_enable_instrumentation_reads_env_sensitive_data(monkeypatch):
|
||||
"""No-arg enable_instrumentation() re-reads ENABLE_SENSITIVE_DATA from env at call time.
|
||||
|
||||
Covers the fallback branch where the env var is set AFTER import (e.g. via load_dotenv()).
|
||||
"""
|
||||
import importlib
|
||||
|
||||
monkeypatch.setenv("ENABLE_INSTRUMENTATION", "false")
|
||||
monkeypatch.delenv("ENABLE_SENSITIVE_DATA", raising=False)
|
||||
|
||||
observability = importlib.import_module("agent_framework.observability")
|
||||
importlib.reload(observability)
|
||||
|
||||
# Simulate load_dotenv() setting the env var after import
|
||||
monkeypatch.setenv("ENABLE_SENSITIVE_DATA", "true")
|
||||
observability.enable_instrumentation()
|
||||
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_instrumentation is True
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_sensitive_data is True
|
||||
|
||||
|
||||
# region Test disable_instrumentation sticky behavior
|
||||
|
||||
|
||||
def test_disable_instrumentation_flips_settings_off(monkeypatch):
|
||||
"""disable_instrumentation() immediately turns instrumentation and sensitive data off."""
|
||||
import importlib
|
||||
|
||||
monkeypatch.delenv("ENABLE_INSTRUMENTATION", raising=False)
|
||||
monkeypatch.setenv("ENABLE_SENSITIVE_DATA", "true")
|
||||
|
||||
observability = importlib.import_module("agent_framework.observability")
|
||||
importlib.reload(observability)
|
||||
|
||||
observability.enable_sensitive_telemetry()
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_instrumentation is True
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_sensitive_data is True
|
||||
assert observability.OBSERVABILITY_SETTINGS.SENSITIVE_DATA_ENABLED is True
|
||||
|
||||
observability.disable_instrumentation()
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_instrumentation is False
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_sensitive_data is False
|
||||
assert observability.OBSERVABILITY_SETTINGS.SENSITIVE_DATA_ENABLED is False
|
||||
assert observability.OBSERVABILITY_SETTINGS.ENABLED is False
|
||||
|
||||
|
||||
def test_disable_instrumentation_is_sticky_against_enable_instrumentation(monkeypatch):
|
||||
"""Sticky disable: enable_instrumentation() without force is a no-op after disable."""
|
||||
import importlib
|
||||
|
||||
monkeypatch.delenv("ENABLE_INSTRUMENTATION", raising=False)
|
||||
monkeypatch.delenv("ENABLE_SENSITIVE_DATA", raising=False)
|
||||
|
||||
observability = importlib.import_module("agent_framework.observability")
|
||||
importlib.reload(observability)
|
||||
|
||||
observability.disable_instrumentation()
|
||||
observability.enable_instrumentation(enable_sensitive_data=True)
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_instrumentation is False
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_sensitive_data is False
|
||||
|
||||
|
||||
def test_disable_instrumentation_is_sticky_against_enable_sensitive_telemetry(monkeypatch):
|
||||
"""Sticky disable: enable_sensitive_telemetry() without force is a no-op after disable."""
|
||||
import importlib
|
||||
|
||||
monkeypatch.delenv("ENABLE_INSTRUMENTATION", raising=False)
|
||||
monkeypatch.delenv("ENABLE_SENSITIVE_DATA", raising=False)
|
||||
|
||||
observability = importlib.import_module("agent_framework.observability")
|
||||
importlib.reload(observability)
|
||||
|
||||
observability.disable_instrumentation()
|
||||
observability.enable_sensitive_telemetry()
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_instrumentation is False
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_sensitive_data is False
|
||||
|
||||
|
||||
def test_disable_instrumentation_is_sticky_against_configure_otel_providers(monkeypatch):
|
||||
"""Sticky disable: configure_otel_providers() does not flip instrumentation back on."""
|
||||
import importlib
|
||||
|
||||
monkeypatch.delenv("ENABLE_INSTRUMENTATION", raising=False)
|
||||
monkeypatch.delenv("ENABLE_SENSITIVE_DATA", raising=False)
|
||||
|
||||
observability = importlib.import_module("agent_framework.observability")
|
||||
importlib.reload(observability)
|
||||
|
||||
observability.disable_instrumentation()
|
||||
with patch.object(observability.OBSERVABILITY_SETTINGS, "_configure"):
|
||||
observability.configure_otel_providers(enable_sensitive_data=True)
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_instrumentation is False
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_sensitive_data is False
|
||||
|
||||
|
||||
def test_disable_instrumentation_intercepts_direct_attribute_writes(monkeypatch):
|
||||
"""Sticky disable: direct OBSERVABILITY_SETTINGS.enable_instrumentation = True is intercepted."""
|
||||
import importlib
|
||||
|
||||
monkeypatch.delenv("ENABLE_INSTRUMENTATION", raising=False)
|
||||
monkeypatch.delenv("ENABLE_SENSITIVE_DATA", raising=False)
|
||||
|
||||
observability = importlib.import_module("agent_framework.observability")
|
||||
importlib.reload(observability)
|
||||
|
||||
observability.disable_instrumentation()
|
||||
observability.OBSERVABILITY_SETTINGS.enable_instrumentation = True
|
||||
observability.OBSERVABILITY_SETTINGS.enable_sensitive_data = True
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_instrumentation is False
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_sensitive_data is False
|
||||
|
||||
|
||||
def test_enable_instrumentation_force_clears_disable(monkeypatch):
|
||||
"""enable_instrumentation(force=True) clears the sticky disable."""
|
||||
import importlib
|
||||
|
||||
monkeypatch.delenv("ENABLE_INSTRUMENTATION", raising=False)
|
||||
monkeypatch.delenv("ENABLE_SENSITIVE_DATA", raising=False)
|
||||
|
||||
observability = importlib.import_module("agent_framework.observability")
|
||||
importlib.reload(observability)
|
||||
|
||||
observability.disable_instrumentation()
|
||||
observability.enable_instrumentation(force=True, enable_sensitive_data=True)
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_instrumentation is True
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_sensitive_data is True
|
||||
|
||||
|
||||
def test_enable_sensitive_telemetry_force_clears_disable(monkeypatch):
|
||||
"""enable_sensitive_telemetry(force=True) clears the sticky disable."""
|
||||
import importlib
|
||||
|
||||
monkeypatch.delenv("ENABLE_INSTRUMENTATION", raising=False)
|
||||
monkeypatch.delenv("ENABLE_SENSITIVE_DATA", raising=False)
|
||||
|
||||
observability = importlib.import_module("agent_framework.observability")
|
||||
importlib.reload(observability)
|
||||
|
||||
observability.disable_instrumentation()
|
||||
observability.enable_sensitive_telemetry(force=True)
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_instrumentation is True
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_sensitive_data is True
|
||||
|
||||
|
||||
def test_disable_instrumentation_persists_after_force_until_redisabled(monkeypatch):
|
||||
"""After force-enable then disable again, the sticky disable is re-armed."""
|
||||
import importlib
|
||||
|
||||
monkeypatch.delenv("ENABLE_INSTRUMENTATION", raising=False)
|
||||
monkeypatch.delenv("ENABLE_SENSITIVE_DATA", raising=False)
|
||||
|
||||
observability = importlib.import_module("agent_framework.observability")
|
||||
importlib.reload(observability)
|
||||
|
||||
observability.disable_instrumentation()
|
||||
observability.enable_instrumentation(force=True)
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_instrumentation is True
|
||||
|
||||
observability.disable_instrumentation()
|
||||
observability.enable_instrumentation()
|
||||
assert observability.OBSERVABILITY_SETTINGS.enable_instrumentation is False
|
||||
|
||||
|
||||
def test_disable_instrumentation_in_all(monkeypatch):
|
||||
"""disable_instrumentation must be re-exported from the module's __all__."""
|
||||
import agent_framework.observability as observability
|
||||
|
||||
assert "disable_instrumentation" in observability.__all__
|
||||
assert callable(observability.disable_instrumentation)
|
||||
|
||||
|
||||
# region Test _to_otel_part content types
|
||||
|
||||
|
||||
@@ -3797,3 +4046,135 @@ async def test_agent_streaming_execute_failure_closes_span_and_resets_contextvar
|
||||
agent_spans = [s for s in spans if s.attributes.get(OtelAttr.OPERATION.value) == OtelAttr.AGENT_INVOKE_OPERATION]
|
||||
assert len(agent_spans) == 1
|
||||
assert agent_spans[0].status.status_code == StatusCode.ERROR
|
||||
|
||||
|
||||
# region Test heavy operations skipped when span is not recording
|
||||
#
|
||||
# When ``ENABLE_INSTRUMENTATION`` is on (the default) but no OpenTelemetry
|
||||
# tracer provider has been configured, the global provider is the
|
||||
# ``ProxyTracerProvider`` which returns non-recording spans. The telemetry
|
||||
# layers gate sensitive-data serialization (``_capture_messages``) on
|
||||
# ``span.is_recording()`` so that we don't pay the JSON-serialization cost
|
||||
# when the span is going to be dropped anyway. The tests below verify that
|
||||
# behavior by patching ``get_tracer`` to return a ``NoOpTracer``.
|
||||
|
||||
|
||||
@pytest.mark.parametrize("enable_sensitive_data", [True], indirect=True)
|
||||
async def test_chat_capture_messages_skipped_when_span_not_recording(
|
||||
mock_chat_client, span_exporter: InMemorySpanExporter, enable_sensitive_data
|
||||
):
|
||||
"""Heavy message serialization is skipped when no provider is configured (non-streaming)."""
|
||||
from opentelemetry.trace import NoOpTracer
|
||||
|
||||
client = mock_chat_client()
|
||||
messages = [Message(role="user", contents=["Test"])]
|
||||
span_exporter.clear()
|
||||
|
||||
with (
|
||||
patch("agent_framework.observability.get_tracer", return_value=NoOpTracer()),
|
||||
patch("agent_framework.observability._capture_messages") as mock_capture_messages,
|
||||
patch("agent_framework.observability._capture_response") as mock_capture_response,
|
||||
):
|
||||
response = await client.get_response(messages=messages, options={"model": "Test"})
|
||||
|
||||
assert response is not None
|
||||
# Sensitive-data serialization must be skipped because span.is_recording() is False.
|
||||
assert mock_capture_messages.call_count == 0
|
||||
# _capture_response still runs so that metric histograms continue to record.
|
||||
assert mock_capture_response.call_count == 1
|
||||
|
||||
|
||||
@pytest.mark.parametrize("enable_sensitive_data", [True], indirect=True)
|
||||
async def test_chat_streaming_capture_messages_skipped_when_span_not_recording(
|
||||
mock_chat_client, span_exporter: InMemorySpanExporter, enable_sensitive_data
|
||||
):
|
||||
"""Heavy message serialization is skipped when no provider is configured (streaming)."""
|
||||
from opentelemetry.trace import NoOpTracer
|
||||
|
||||
client = mock_chat_client()
|
||||
messages = [Message(role="user", contents=["Test"])]
|
||||
span_exporter.clear()
|
||||
|
||||
with (
|
||||
patch("agent_framework.observability.get_tracer", return_value=NoOpTracer()),
|
||||
patch("agent_framework.observability._capture_messages") as mock_capture_messages,
|
||||
patch("agent_framework.observability._capture_response") as mock_capture_response,
|
||||
):
|
||||
updates: list[ChatResponseUpdate] = []
|
||||
stream = client.get_response(messages=messages, stream=True, options={"model": "Test"})
|
||||
async for update in stream:
|
||||
updates.append(update)
|
||||
await stream.get_final_response()
|
||||
|
||||
assert len(updates) == 2
|
||||
assert mock_capture_messages.call_count == 0
|
||||
assert mock_capture_response.call_count == 1
|
||||
|
||||
|
||||
@pytest.mark.parametrize("enable_sensitive_data", [True], indirect=True)
|
||||
async def test_agent_capture_messages_skipped_when_span_not_recording(
|
||||
mock_chat_agent, span_exporter: InMemorySpanExporter, enable_sensitive_data
|
||||
):
|
||||
"""Agent heavy serialization is skipped when no provider is configured (non-streaming)."""
|
||||
from opentelemetry.trace import NoOpTracer
|
||||
|
||||
agent = mock_chat_agent()
|
||||
span_exporter.clear()
|
||||
|
||||
with (
|
||||
patch("agent_framework.observability.get_tracer", return_value=NoOpTracer()),
|
||||
patch("agent_framework.observability._capture_messages") as mock_capture_messages,
|
||||
patch("agent_framework.observability._capture_response") as mock_capture_response,
|
||||
):
|
||||
response = await agent.run("Test message")
|
||||
|
||||
assert response is not None
|
||||
assert mock_capture_messages.call_count == 0
|
||||
assert mock_capture_response.call_count == 1
|
||||
|
||||
|
||||
@pytest.mark.parametrize("enable_sensitive_data", [True], indirect=True)
|
||||
async def test_agent_streaming_capture_messages_skipped_when_span_not_recording(
|
||||
mock_chat_agent, span_exporter: InMemorySpanExporter, enable_sensitive_data
|
||||
):
|
||||
"""Agent heavy serialization is skipped when no provider is configured (streaming)."""
|
||||
from opentelemetry.trace import NoOpTracer
|
||||
|
||||
agent = mock_chat_agent()
|
||||
span_exporter.clear()
|
||||
|
||||
with (
|
||||
patch("agent_framework.observability.get_tracer", return_value=NoOpTracer()),
|
||||
patch("agent_framework.observability._capture_messages") as mock_capture_messages,
|
||||
patch("agent_framework.observability._capture_response") as mock_capture_response,
|
||||
):
|
||||
updates: list[Any] = []
|
||||
stream = agent.run("Test message", stream=True)
|
||||
async for update in stream:
|
||||
updates.append(update)
|
||||
await stream.get_final_response()
|
||||
|
||||
assert len(updates) == 2
|
||||
assert mock_capture_messages.call_count == 0
|
||||
assert mock_capture_response.call_count == 1
|
||||
|
||||
|
||||
@pytest.mark.parametrize("enable_sensitive_data", [True], indirect=True)
|
||||
async def test_chat_capture_messages_called_when_span_recording(
|
||||
mock_chat_client, span_exporter: InMemorySpanExporter, enable_sensitive_data
|
||||
):
|
||||
"""Sanity check: with a real recording provider, sensitive-data capture still runs."""
|
||||
client = mock_chat_client()
|
||||
messages = [Message(role="user", contents=["Test"])]
|
||||
span_exporter.clear()
|
||||
|
||||
with (
|
||||
patch("agent_framework.observability._capture_messages") as mock_capture_messages,
|
||||
patch("agent_framework.observability._capture_response") as mock_capture_response,
|
||||
):
|
||||
response = await client.get_response(messages=messages, options={"model": "Test"})
|
||||
|
||||
assert response is not None
|
||||
# Two _capture_messages calls: one for input, one for output messages.
|
||||
assert mock_capture_messages.call_count == 2
|
||||
assert mock_capture_response.call_count == 1
|
||||
|
||||
Reference in New Issue
Block a user