fix: resolve mypy redundant-cast errors while keeping pyright happy

Use cast(list[Any], x) with type: ignore[redundant-cast] comments to satisfy both mypy (which considers casting Any redundant) and pyright strict mode (which needs explicit casts to narrow Unknown types). Also fix evaluator decorator check_name type annotation to be explicitly str, resolving mypy str|Any|None mismatch. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Foundry Evals integration for Python
2026-06-16 21:04:09 +08:00 · 2026-03-20 15:25:07 -07:00 · 2026-03-20 14:24:21 -07:00 · 2026-03-19 19:18:46 +00:00 · 2026-03-19 10:57:43 +00:00 · 2026-03-19 10:45:42 +00:00
138 changed files with 11331 additions and 678 deletions
@@ -3,6 +3,7 @@
  "image": "mcr.microsoft.com/devcontainers/dotnet",
  "features": {
    "ghcr.io/devcontainers/features/azure-cli:1.2.9": {},
+    "ghcr.io/devcontainers/features/docker-in-docker:2": {},
    "ghcr.io/devcontainers/features/github-cli:1": {
      "version": "2"
    },
@@ -75,7 +75,7 @@ jobs:
          os: ${{ runner.os }}
        env:
          UV_CACHE_DIR: /tmp/.uv-cache
-      - name: Run fmt, lint, pyright in parallel across packages
+      - name: Run syntax and pyright across packages
        run: uv run poe check-packages

  samples-markdown:
@@ -104,10 +104,8 @@ jobs:
          os: ${{ runner.os }}
        env:
          UV_CACHE_DIR: /tmp/.uv-cache
-      - name: Run samples lint
-        run: uv run poe samples-lint
-      - name: Run samples syntax check
-        run: uv run poe samples-syntax
+      - name: Run samples checks
+        run: uv run poe check -S
      - name: Run markdown code lint
        run: uv run poe markdown-code-lint

@@ -140,4 +138,4 @@ jobs:
      - name: Run Mypy
        env:
          GITHUB_BASE_REF: ${{ github.event.pull_request.base.ref || github.base_ref || 'main' }}
-        run: uv run poe ci-mypy
+        run: uv run python scripts/workspace_poe_tasks.py ci-mypy
@@ -38,7 +38,7 @@ jobs:
        id: validate_ranges
        # Keep workflow running so we can still publish diagnostics from this run.
        continue-on-error: true
-        run: uv run poe validate-dependency-bounds-project --mode upper --project "*"
+        run: uv run poe validate-dependency-bounds-project --mode upper --package "*"
        working-directory: ./python

      - name: Upload dependency range report
@@ -203,7 +203,7 @@ jobs:
          cat > "${PR_BODY_FILE}" <<'EOF'
          This PR was generated by the dependency range validation workflow.

-          - Ran `uv run poe validate-dependency-bounds-project --mode upper --project "*"`
+          - Ran `uv run poe validate-dependency-bounds-project --mode upper --package "*"`
          - Updated package dependency bounds
          - Refreshed `python/uv.lock` with `uv lock --upgrade`
          EOF
@@ -48,9 +48,8 @@ jobs:
          os: ${{ runner.os }}
      - name: Test with pytest (unit tests only)
        run: >
-          uv run poe all-tests
+          uv run poe test -A
          -m "not integration"
-          -n logical --dist worksteal
          --timeout=120 --session-timeout=900 --timeout_method thread
          --retries 2 --retry-delay 5

@@ -100,9 +100,8 @@ jobs:
          os: ${{ runner.os }}
      - name: Test with pytest (unit tests only)
        run: >
-          uv run poe all-tests
+          uv run poe test -A
          -m "not integration"
-          -n logical --dist worksteal
          --timeout=120 --session-timeout=900 --timeout_method thread
          --retries 2 --retry-delay 5
        working-directory: ./python
@@ -32,13 +32,13 @@ jobs:
        id: python-setup
        uses: ./.github/actions/python-setup
        with:
-          python-version: ${{ matrix.python-version }}
+          python-version: ${{ env.UV_PYTHON }}
          os: ${{ runner.os }}
        env:
          # Configure a constant location for the uv cache
          UV_CACHE_DIR: /tmp/.uv-cache
      - name: Run all tests with coverage report
-        run: uv run poe all-tests-cov --cov-report=xml:python-coverage.xml -q --junitxml=pytest.xml
+        run: uv run poe test -A -C --cov-report=xml:python-coverage.xml -q --junitxml=pytest.xml
      - name: Check coverage threshold
        run: python ${{ github.workspace }}/.github/workflows/python-check-coverage.py python-coverage.xml ${{ env.COVERAGE_THRESHOLD }}
      - name: Upload coverage report
@@ -40,7 +40,7 @@ jobs:
          UV_CACHE_DIR: /tmp/.uv-cache
      # Unit tests
      - name: Run all tests
-        run: uv run poe all-tests ${{ matrix.python-version == '3.10' && '--ignore-glob=packages/github_copilot/**' || '' }}
+        run: uv run poe test -A
        working-directory: ./python

      # Surface failing tests
@@ -149,6 +149,7 @@
    <!-- Symbols -->
    <PackageVersion Include="Microsoft.SourceLink.GitHub" Version="8.0.0" />
    <!-- Toolset -->
+    <PackageVersion Include="ReferenceTrimmer" Version="3.4.5" />
    <PackageVersion Include="Microsoft.CodeAnalysis.Analyzers" Version="3.11.0" />
    <PackageVersion Include="Microsoft.CodeAnalysis.CSharp" Version="4.14.0" />
    <PackageVersion Include="Microsoft.CodeAnalysis.NetAnalyzers" Version="10.0.100" />
@@ -9,7 +9,7 @@
  </PropertyGroup>

  <ItemGroup>
-    <ProjectReference Include="..\..\..\..\src\Microsoft.Agents.AI\Microsoft.Agents.AI.csproj" />
+    <ProjectReference Include="..\..\..\..\src\Microsoft.Agents.AI.Abstractions\Microsoft.Agents.AI.Abstractions.csproj" />
  </ItemGroup>

 </Project>
@@ -12,13 +12,9 @@
    <PackageReference Include="Azure.AI.OpenAI" />
    <PackageReference Include="Azure.Identity" />
    <PackageReference Include="Microsoft.Extensions.AI.OpenAI" />
-    <PackageReference Include="Microsoft.Agents.ObjectModel" />
-    <PackageReference Include="Microsoft.Agents.ObjectModel.Json" />
-    <PackageReference Include="Microsoft.Agents.ObjectModel.PowerFx" />
  </ItemGroup>

  <ItemGroup>
-    <ProjectReference Include="..\..\..\..\src\Microsoft.Agents.AI.Declarative\Microsoft.Agents.AI.Declarative.csproj" />
    <ProjectReference Include="..\..\..\..\src\Microsoft.Agents.AI.OpenAI\Microsoft.Agents.AI.OpenAI.csproj" />
  </ItemGroup>

@@ -61,6 +61,12 @@ public static class Program
            {
                Console.WriteLine($"{outputEvent}");
            }
+
+            if (evt is WorkflowErrorEvent errorEvent)
+            {
+                Console.WriteLine($"Workflow error: {errorEvent.Exception?.Message}");
+                Console.WriteLine($"Details: {errorEvent.Exception}");
+            }
        }
    }
 }
@@ -175,7 +181,9 @@ internal sealed class FeedbackEvent(FeedbackResult feedbackResult) : WorkflowEve
 /// <summary>
 /// A custom executor that uses an AI agent to provide feedback on a slogan.
 /// </summary>
-internal sealed class FeedbackExecutor : Executor<SloganResult>
+[SendsMessage(typeof(FeedbackResult))]
+[YieldsOutput(typeof(string))]
+internal sealed partial class FeedbackExecutor : Executor<SloganResult>
 {
    private readonly AIAgent _agent;
    private AgentSession? _session;
@@ -14,7 +14,6 @@
    <PackageReference Include="Azure.Identity" />
    <PackageReference Include="Microsoft.Extensions.Hosting" />
    <PackageReference Include="System.Net.ServerSentEvents" />
-    <PackageReference Include="Microsoft.Bcl.AsyncInterfaces" />
  </ItemGroup>

  <ItemGroup>
@@ -15,7 +15,6 @@

  <ItemGroup>
    <ProjectReference Include="..\..\..\..\src\Microsoft.Agents.AI.Hosting.AGUI.AspNetCore\Microsoft.Agents.AI.Hosting.AGUI.AspNetCore.csproj" />
-    <ProjectReference Include="..\..\..\..\src\Microsoft.Agents.AI.AGUI\Microsoft.Agents.AI.AGUI.csproj" />
    <ProjectReference Include="..\..\..\..\src\Microsoft.Agents.AI.OpenAI\Microsoft.Agents.AI.OpenAI.csproj" />
  </ItemGroup>

@@ -9,10 +9,12 @@
  
  <ItemGroup>
    <ProjectReference Include="..\..\..\..\src\Microsoft.Agents.AI.Hosting.A2A\Microsoft.Agents.AI.Hosting.A2A.csproj" />
-	  <ProjectReference Include="..\..\..\..\src\Microsoft.Agents.AI.Hosting.OpenAI\Microsoft.Agents.AI.Hosting.OpenAI.csproj" />
-    <ProjectReference Include="..\..\..\..\src\Microsoft.Agents.AI.Hosting\Microsoft.Agents.AI.Hosting.csproj" />
    <ProjectReference Include="..\AgentWebChat.ServiceDefaults\AgentWebChat.ServiceDefaults.csproj" />
    <ProjectReference Include="..\..\..\..\src\Microsoft.Agents.AI.Abstractions\Microsoft.Agents.AI.Abstractions.csproj" />
  </ItemGroup>

+  <ItemGroup>
+    <PackageReference Include="OpenAI" />
+    <PackageReference Include="Microsoft.Extensions.AI.OpenAI" />
+  </ItemGroup>
 </Project>
@@ -0,0 +1,9 @@
+<Project>
+
+  <Import Project="../Directory.Build.props" />
+
+  <ItemGroup>
+    <PackageReference Include="ReferenceTrimmer" PrivateAssets="all" IncludeAssets="build;analyzers;buildTransitive" />
+  </ItemGroup>
+
+</Project>
@@ -16,12 +16,9 @@
    <Description>Provides Microsoft Agent Framework support for Agent-User Interaction (AG-UI) protocol client functionality.</Description>
  </PropertyGroup>

-  <ItemGroup>
-    <ProjectReference Include="..\Microsoft.Agents.AI.Abstractions\Microsoft.Agents.AI.Abstractions.csproj" />
-  </ItemGroup>
-
  <ItemGroup>
    <PackageReference Include="Microsoft.Extensions.AI" />
+    <PackageReference Include="Microsoft.Extensions.Logging.Abstractions" />
    <PackageReference Include="System.Net.ServerSentEvents" />
    <PackageReference Include="System.Net.Http.Json" />
    <PackageReference Include="System.Threading.Channels" />
@@ -104,17 +104,15 @@ public sealed class InMemoryChatHistoryProvider : ChatHistoryProvider
    {
        State state = this._sessionState.GetOrInitializeState(context.Session);

-        if (this.ReducerTriggerEvent is InMemoryChatHistoryProviderOptions.ChatReducerTriggerEvent.AfterMessageAdded && this.ChatReducer is not null)
-        {
-            // Reduce existing messages before adding new messages from the current turn.
-            // This ensures messages from the current turn (including function calls and tool results)
-            // are always preserved in full and are not immediately reduced.
-            await ReduceMessagesAsync(this.ChatReducer, state, cancellationToken).ConfigureAwait(false);
-        }
-
        // Add request and response messages to the provider
        var allNewMessages = context.RequestMessages.Concat(context.ResponseMessages ?? []);
        state.Messages.AddRange(allNewMessages);
+
+        if (this.ReducerTriggerEvent is InMemoryChatHistoryProviderOptions.ChatReducerTriggerEvent.AfterMessageAdded && this.ChatReducer is not null)
+        {
+            // Apply pre-write reduction strategy if configured
+            await ReduceMessagesAsync(this.ChatReducer, state, cancellationToken).ConfigureAwait(false);
+        }
    }

    private static async Task ReduceMessagesAsync(IChatReducer reducer, State state, CancellationToken cancellationToken = default)
@@ -28,7 +28,6 @@

  <ItemGroup>
    <PackageReference Include="Microsoft.Extensions.AI.Abstractions" />
-    <PackageReference Include="Microsoft.Extensions.Logging.Abstractions" />
  </ItemGroup>

  <ItemGroup>
@@ -18,7 +18,7 @@
  </PropertyGroup>

  <ItemGroup>
-    <ProjectReference Include="..\Microsoft.Agents.AI.Hosting\Microsoft.Agents.AI.Hosting.csproj" />
+    <ProjectReference Include="..\Microsoft.Agents.AI\Microsoft.Agents.AI.csproj" />
  </ItemGroup>

  <ItemGroup>
@@ -15,6 +15,10 @@
    <IsPackable>false</IsPackable>
  </PropertyGroup>

+  <ItemGroup>
+    <PackageReference Include="Microsoft.Extensions.Logging.Abstractions" />
+  </ItemGroup>
+
  <ItemGroup>
    <ProjectReference Include="..\Microsoft.Agents.AI.Abstractions\Microsoft.Agents.AI.Abstractions.csproj" />
  </ItemGroup>
@@ -26,7 +26,7 @@
    <PackageReference Include="Microsoft.PowerFx.Interpreter" />
    <PackageReference Include="Microsoft.Extensions.Configuration" />
    <PackageReference Include="Microsoft.Extensions.Logging" />
-    <PackageReference Include="System.CodeDom" />
+    <PackageReference Include="System.CodeDom" TreatAsUsed="true" />
    <PackageReference Include="System.Collections.Immutable" />
  </ItemGroup>

@@ -68,7 +68,7 @@ internal static class SemanticAnalyzer
        string classKey = GetClassKey(classSymbol);
        bool isPartialClass = IsPartialClass(classSymbol, cancellationToken);
        bool derivesFromExecutor = DerivesFromExecutor(classSymbol);
-        bool configureProtocol = HasConfigureProtocolDefined(classSymbol);
+        bool hasManualConfigureProtocol = HasConfigureProtocolDefined(classSymbol);

        // Extract class metadata
        string? @namespace = classSymbol.ContainingNamespace?.IsGlobalNamespace == true
@@ -97,7 +97,7 @@ internal static class SemanticAnalyzer
        return new MethodAnalysisResult(
            classKey, @namespace, className, genericParameters, isNested, containingTypeChain,
            baseHasConfigureProtocol, classSendTypes, classYieldTypes,
-            isPartialClass, derivesFromExecutor, configureProtocol,
+            isPartialClass, derivesFromExecutor, hasManualConfigureProtocol,
            classLocation,
            handler,
            Diagnostics: new ImmutableEquatableArray<DiagnosticInfo>(methodDiagnostics.ToImmutable()));
@@ -149,7 +149,7 @@ internal static class SemanticAnalyzer
            return AnalysisResult.WithDiagnostics(allDiagnostics.ToImmutable());
        }

-        if (first.HasManualConfigureRoutes)
+        if (first.HasManualConfigureProtocol)
        {
            allDiagnostics.Add(Diagnostic.Create(
                DiagnosticDescriptors.ConfigureProtocolAlreadyDefined,
@@ -212,6 +212,7 @@ internal static class SemanticAnalyzer
        bool isPartialClass = IsPartialClass(classSymbol, cancellationToken);
        bool derivesFromExecutor = DerivesFromExecutor(classSymbol);
        bool hasManualConfigureProtocol = HasConfigureProtocolDefined(classSymbol);
+        bool baseHasConfigureProtocol = BaseHasConfigureProtocol(classSymbol);

        string? @namespace = classSymbol.ContainingNamespace?.IsGlobalNamespace == true
            ? null
@@ -241,6 +242,7 @@ internal static class SemanticAnalyzer
                    isPartialClass,
                    derivesFromExecutor,
                    hasManualConfigureProtocol,
+                    baseHasConfigureProtocol,
                    classLocation,
                    typeName,
                    attributeKind));
@@ -321,7 +323,7 @@ internal static class SemanticAnalyzer
            first.GenericParameters,
            first.IsNested,
            first.ContainingTypeChain,
-            BaseHasConfigureProtocol: false, // Not relevant for protocol-only
+            first.BaseHasConfigureProtocol,
            Handlers: ImmutableEquatableArray<HandlerInfo>.Empty,
            ClassSendTypes: new ImmutableEquatableArray<string>(sendTypes.ToImmutable()),
            ClassYieldTypes: new ImmutableEquatableArray<string>(yieldTypes.ToImmutable()));
@@ -5,7 +5,7 @@ namespace Microsoft.Agents.AI.Workflows.Generators.Models;
 /// <summary>
 /// Represents protocol type information extracted from class-level [SendsMessage] or [YieldsOutput] attributes.
 /// Used by the incremental generator pipeline to capture classes that declare protocol types
-/// but may not have [MessageHandler] methods (e.g., when ConfigureRoutes is manually implemented).
+/// but may not have [MessageHandler] methods (e.g., when ConfigureProtocol is manually implemented).
 /// </summary>
 /// <param name="ClassKey">Unique identifier for the class (fully qualified name).</param>
 /// <param name="Namespace">The namespace of the class.</param>
@@ -15,7 +15,8 @@ namespace Microsoft.Agents.AI.Workflows.Generators.Models;
 /// <param name="ContainingTypeChain">The chain of containing types for nested classes. Empty if not nested.</param>
 /// <param name="IsPartialClass">Whether the class is declared as partial.</param>
 /// <param name="DerivesFromExecutor">Whether the class derives from Executor.</param>
-/// <param name="HasManualConfigureRoutes">Whether the class has a manually defined ConfigureRoutes method.</param>
+/// <param name="HasManualConfigureProtocol">Whether the class has a manually defined ConfigureProtocol method.</param>
+/// <param name="BaseHasConfigureProtocol">Whether a base class already overrides ConfigureProtocol.</param>
 /// <param name="ClassLocation">Location info for diagnostics.</param>
 /// <param name="TypeName">The fully qualified type name from the attribute.</param>
 /// <param name="AttributeKind">Whether this is from a SendsMessage or YieldsOutput attribute.</param>
@@ -28,7 +29,8 @@ internal sealed record ClassProtocolInfo(
    string ContainingTypeChain,
    bool IsPartialClass,
    bool DerivesFromExecutor,
-    bool HasManualConfigureRoutes,
+    bool HasManualConfigureProtocol,
+    bool BaseHasConfigureProtocol,
    DiagnosticLocationInfo? ClassLocation,
    string TypeName,
    ProtocolAttributeKind AttributeKind)
@@ -38,5 +40,5 @@ internal sealed record ClassProtocolInfo(
    /// </summary>
    public static ClassProtocolInfo Empty { get; } = new(
        string.Empty, null, string.Empty, null, false, string.Empty,
-        false, false, false, null, string.Empty, ProtocolAttributeKind.Send);
+        false, false, false, false, null, string.Empty, ProtocolAttributeKind.Send);
 }
@@ -8,7 +8,7 @@ namespace Microsoft.Agents.AI.Workflows.Generators.Models;
 /// Uses value-equatable types to support incremental generator caching.
 /// </summary>
 /// <remarks>
-/// Class-level validation (IsPartialClass, DerivesFromExecutor, HasManualConfigureRoutes)
+/// Class-level validation (IsPartialClass, DerivesFromExecutor, HasManualConfigureProtocol)
 /// is extracted here but validated once per class in CombineMethodResults to avoid
 /// redundant validation work when a class has multiple handlers.
 /// </remarks>
@@ -29,7 +29,7 @@ internal sealed record MethodAnalysisResult(
    // Class-level facts (used for validation in CombineMethodResults)
    bool IsPartialClass,
    bool DerivesFromExecutor,
-    bool HasManualConfigureRoutes,
+    bool HasManualConfigureProtocol,

    // Class location for diagnostics (value-equatable)
    DiagnosticLocationInfo? ClassLocation,
@@ -3,25 +3,25 @@
 using System.Collections.Generic;
 using System.Linq;
 using System.Text.Json.Serialization;
-using System.Threading;
 using Microsoft.Agents.AI.Workflows.Checkpointing;

 namespace Microsoft.Agents.AI.Workflows.Execution;

 internal sealed class FanInEdgeState
 {
-    private List<PortableMessageEnvelope> _pendingMessages;
+    private readonly object _syncLock = new();
+
    public FanInEdgeState(FanInEdgeData fanInEdge)
    {
        this.SourceIds = fanInEdge.SourceIds.ToArray();
        this.Unseen = [.. this.SourceIds];

-        this._pendingMessages = [];
+        this.PendingMessages = [];
    }

    public string[] SourceIds { get; }
    public HashSet<string> Unseen { get; private set; }
-    public List<PortableMessageEnvelope> PendingMessages => this._pendingMessages;
+    public List<PortableMessageEnvelope> PendingMessages { get; private set; }

    [JsonConstructor]
    public FanInEdgeState(string[] sourceIds, HashSet<string> unseen, List<PortableMessageEnvelope> pendingMessages)
@@ -29,28 +29,35 @@ internal sealed class FanInEdgeState
        this.SourceIds = sourceIds;
        this.Unseen = unseen;

-        this._pendingMessages = pendingMessages;
+        this.PendingMessages = pendingMessages;
    }

    public IEnumerable<IGrouping<ExecutorIdentity, MessageEnvelope>>? ProcessMessage(string sourceId, MessageEnvelope envelope)
    {
-        this.PendingMessages.Add(new(envelope));
-        this.Unseen.Remove(sourceId);
+        List<PortableMessageEnvelope>? takenMessages = null;

-        if (this.Unseen.Count == 0)
+        // Serialize concurrent calls from parallel executor tasks during superstep execution.
+        // NOTE - IMPORTANT: If this ProcessMessage method ever becomes async, replace this lock with an async friendly solution to avoid deadlocks.
+        lock (this._syncLock)
        {
-            List<PortableMessageEnvelope> takenMessages = Interlocked.Exchange(ref this._pendingMessages, []);
-            this.Unseen = [.. this.SourceIds];
+            this.PendingMessages.Add(new(envelope));
+            this.Unseen.Remove(sourceId);

-            if (takenMessages.Count == 0)
+            if (this.Unseen.Count == 0)
            {
-                return null;
+                takenMessages = this.PendingMessages;
+                this.PendingMessages = [];
+                this.Unseen = [.. this.SourceIds];
            }
-
-            return takenMessages.Select(portable => portable.ToMessageEnvelope())
-                                .GroupBy(keySelector: messageEnvelope => messageEnvelope.Source);
        }

-        return null;
+        if (takenMessages is null || takenMessages.Count == 0)
+        {
+            return null;
+        }
+
+        return takenMessages
+            .Select(portable => portable.ToMessageEnvelope())
+            .GroupBy(messageEnvelope => messageEnvelope.Source);
    }
 }
@@ -0,0 +1,59 @@
+// Copyright (c) Microsoft. All rights reserved.
+
+using System.Collections.Generic;
+using System.Diagnostics.CodeAnalysis;
+using System.Threading;
+using System.Threading.Tasks;
+using Microsoft.Extensions.AI;
+using Microsoft.Shared.DiagnosticIds;
+using Microsoft.Shared.Diagnostics;
+
+namespace Microsoft.Agents.AI.Compaction;
+
+/// <summary>
+/// Provides extension methods for <see cref="CompactionStrategy"/>.
+/// </summary>
+[Experimental(DiagnosticIds.Experiments.AgentsAIExperiments)]
+public static class ChatStrategyExtensions
+{
+    /// <summary>
+    /// Returns an <see cref="IChatReducer"/> that applies this <see cref="CompactionStrategy"/> to reduce a list of messages.
+    /// </summary>
+    /// <param name="strategy">The compaction strategy to wrap as an <see cref="IChatReducer"/>.</param>
+    /// <returns>
+    /// An <see cref="IChatReducer"/> that, on each call to <see cref="IChatReducer.ReduceAsync"/>, builds a
+    /// <see cref="CompactionMessageIndex"/> from the supplied messages and applies the strategy's compaction logic,
+    /// returning the resulting included messages.
+    /// </returns>
+    /// <remarks>
+    /// This allows any <see cref="CompactionStrategy"/> to be used wherever an <see cref="IChatReducer"/> is expected,
+    /// bridging the compaction pipeline into systems bound to the <c>Microsoft.Extensions.AI</c> <see cref="IChatReducer"/> contract.
+    /// </remarks>
+    public static IChatReducer AsChatReducer(this CompactionStrategy strategy)
+    {
+        Throw.IfNull(strategy);
+
+        return new CompactionStrategyChatReducer(strategy);
+    }
+
+    /// <summary>
+    /// An <see cref="IChatReducer"/> adapter that delegates to a <see cref="CompactionStrategy"/>.
+    /// </summary>
+    private sealed class CompactionStrategyChatReducer : IChatReducer
+    {
+        private readonly CompactionStrategy _strategy;
+
+        public CompactionStrategyChatReducer(CompactionStrategy strategy)
+        {
+            this._strategy = strategy;
+        }
+
+        /// <inheritdoc/>
+        public async Task<IEnumerable<ChatMessage>> ReduceAsync(IEnumerable<ChatMessage> messages, CancellationToken cancellationToken = default)
+        {
+            CompactionMessageIndex index = CompactionMessageIndex.Create([.. messages]);
+            await this._strategy.CompactAsync(index, cancellationToken: cancellationToken).ConfigureAwait(false);
+            return index.GetIncludedMessages();
+        }
+    }
+}
@@ -1,5 +1,6 @@
 // Copyright (c) Microsoft. All rights reserved.

+using System;
 using System.Collections.Generic;
 using System.Diagnostics.CodeAnalysis;
 using System.Threading;
@@ -30,6 +31,12 @@ namespace Microsoft.Agents.AI.Compaction;
 /// </code>
 /// </para>
 /// <para>
+/// A custom <see cref="ToolCallFormatter"/> can be supplied to override the default YAML-like
+/// summary format. The formatter receives the <see cref="CompactionMessageGroup"/> being collapsed
+/// and must return the replacement summary string. <see cref="DefaultToolCallFormatter"/> is the
+/// built-in default and can be reused inside a custom formatter when needed.
+/// </para>
+/// <para>
 /// <see cref="MinimumPreservedGroups"/> is a hard floor: even if the <see cref="CompactionStrategy.Target"/>
 /// has not been reached, compaction will not touch the last <see cref="MinimumPreservedGroups"/> non-system groups.
 /// </para>
@@ -62,7 +69,10 @@ public sealed class ToolResultCompactionStrategy : CompactionStrategy
    /// An optional target condition that controls when compaction stops. When <see langword="null"/>,
    /// defaults to the inverse of the <paramref name="trigger"/> — compaction stops as soon as the trigger would no longer fire.
    /// </param>
-    public ToolResultCompactionStrategy(CompactionTrigger trigger, int minimumPreservedGroups = DefaultMinimumPreserved, CompactionTrigger? target = null)
+    public ToolResultCompactionStrategy(
+        CompactionTrigger trigger,
+        int minimumPreservedGroups = DefaultMinimumPreserved,
+        CompactionTrigger? target = null)
        : base(trigger, target)
    {
        this.MinimumPreservedGroups = EnsureNonNegative(minimumPreservedGroups);
@@ -74,6 +84,13 @@ public sealed class ToolResultCompactionStrategy : CompactionStrategy
    /// </summary>
    public int MinimumPreservedGroups { get; }

+    /// <summary>
+    /// An optional custom formatter that converts a <see cref="CompactionMessageGroup"/> into a summary string.
+    /// When <see langword="null"/>, <see cref="DefaultToolCallFormatter"/> is used, which produces a YAML-like
+    /// block listing each tool name and its results.
+    /// </summary>
+    public Func<CompactionMessageGroup, string>? ToolCallFormatter { get; init; }
+
    /// <inheritdoc/>
    protected override ValueTask<bool> CompactCoreAsync(CompactionMessageIndex index, ILogger logger, CancellationToken cancellationToken)
    {
@@ -120,7 +137,7 @@ public sealed class ToolResultCompactionStrategy : CompactionStrategy
            int idx = eligibleIndices[e] + offset;
            CompactionMessageGroup group = index.Groups[idx];

-            string summary = BuildToolCallSummary(group);
+            string summary = (this.ToolCallFormatter ?? DefaultToolCallFormatter).Invoke(group);

            // Exclude the original group and insert a collapsed replacement
            group.IsExcluded = true;
@@ -145,14 +162,18 @@ public sealed class ToolResultCompactionStrategy : CompactionStrategy
    }

    /// <summary>
-    /// Builds a concise summary string for a tool call group, including tool names,
+    /// The default formatter that produces a YAML-like summary of tool call groups, including tool names,
    /// results, and deduplication counts for repeated tool names.
    /// </summary>
-    private static string BuildToolCallSummary(CompactionMessageGroup group)
+    /// <remarks>
+    /// This is the formatter used when no custom <see cref="ToolCallFormatter"/> is supplied.
+    /// It can be referenced directly in a custom formatter to augment or wrap the default output.
+    /// </remarks>
+    public static string DefaultToolCallFormatter(CompactionMessageGroup group)
    {
        // Collect function calls (callId, name) and results (callId → result text)
        List<(string CallId, string Name)> functionCalls = [];
-        Dictionary<string, string> resultsByCallId = new();
+        Dictionary<string, string> resultsByCallId = [];
        List<string> plainTextResults = [];

        foreach (ChatMessage message in group.Messages)
@@ -187,7 +208,7 @@ public sealed class ToolResultCompactionStrategy : CompactionStrategy
        // grouping by tool name while preserving first-seen order.
        int plainTextIdx = 0;
        List<string> orderedNames = [];
-        Dictionary<string, List<string>> groupedResults = new();
+        Dictionary<string, List<string>> groupedResults = [];

        foreach ((string callId, string name) in functionCalls)
        {
@@ -175,15 +175,23 @@ public sealed partial class FileAgentSkillsProvider : AIContextProvider
            try
            {
                _ = string.Format(optionsInstructions, string.Empty);
-                promptTemplate = optionsInstructions;
            }
            catch (FormatException ex)
            {
                throw new ArgumentException(
-                    "The provided SkillsInstructionPrompt is not a valid format string. It must contain a '{0}' placeholder and escape any literal '{' or '}' by doubling them ('{{' or '}}').",
+                    "The provided SkillsInstructionPrompt is not a valid format string.",
                    nameof(options),
                    ex);
            }
+
+            if (optionsInstructions.IndexOf("{0}", StringComparison.Ordinal) < 0)
+            {
+                throw new ArgumentException(
+                    "The provided SkillsInstructionPrompt must contain a '{0}' placeholder for the generated skills list.",
+                    nameof(options));
+            }
+
+            promptTemplate = optionsInstructions;
        }

        if (skills.Count == 0)
@@ -243,8 +243,7 @@ public class InMemoryChatHistoryProviderTests
        var session = CreateMockSession();

        // Arrange
-        // Existing messages in state from a previous turn.
-        var existingMessages = new List<ChatMessage>
+        var originalMessages = new List<ChatMessage>
        {
            new(ChatRole.User, "Hello"),
            new(ChatRole.Assistant, "Hi there!")
@@ -254,78 +253,22 @@ public class InMemoryChatHistoryProviderTests
            new(ChatRole.User, "Reduced")
        };

-        // New messages being added in the current turn.
-        var newRequestMessage = new ChatMessage(ChatRole.User, "New message");
-        var newResponseMessage = new ChatMessage(ChatRole.Assistant, "New response");
-
        var reducerMock = new Mock<IChatReducer>();
        reducerMock
-            .Setup(r => r.ReduceAsync(It.Is<List<ChatMessage>>(x => x.SequenceEqual(existingMessages)), It.IsAny<CancellationToken>()))
+            .Setup(r => r.ReduceAsync(It.Is<List<ChatMessage>>(x => x.SequenceEqual(originalMessages)), It.IsAny<CancellationToken>()))
            .ReturnsAsync(reducedMessages);

        var provider = new InMemoryChatHistoryProvider(new() { ChatReducer = reducerMock.Object, ReducerTriggerEvent = InMemoryChatHistoryProviderOptions.ChatReducerTriggerEvent.AfterMessageAdded });
-        provider.SetMessages(session, new List<ChatMessage>(existingMessages));

        // Act
-        var context = new ChatHistoryProvider.InvokedContext(s_mockAgent, session, [newRequestMessage], [newResponseMessage]);
+        var context = new ChatHistoryProvider.InvokedContext(s_mockAgent, session, originalMessages, []);
        await provider.InvokedAsync(context, CancellationToken.None);

        // Assert
-        // The reducer is called on existing messages before the new ones are added.
-        reducerMock.Verify(r => r.ReduceAsync(It.Is<List<ChatMessage>>(x => x.SequenceEqual(existingMessages)), It.IsAny<CancellationToken>()), Times.Once);
-
-        // Final state: reduced existing messages + new current-turn messages (preserved in full).
        var messages = provider.GetMessages(session);
-        Assert.Equal(3, messages.Count);
+        Assert.Single(messages);
        Assert.Equal("Reduced", messages[0].Text);
-        Assert.Equal("New message", messages[1].Text);
-        Assert.Equal("New response", messages[2].Text);
-    }
-
-    [Fact]
-    public async Task AddMessagesAsync_WithReducer_AfterMessageAdded_PreservesCurrentTurnFunctionCallsAsync()
-    {
-        var session = CreateMockSession();
-
-        // Arrange - verify that function call and tool result messages from the current turn are preserved
-        // even when a reducer is configured with AfterMessageAdded trigger. The reducer should only
-        // be applied to existing (previous-turn) messages, not to the new messages being added.
-        var existingMessages = new List<ChatMessage>
-        {
-            new(ChatRole.User, "Previous question"),
-            new(ChatRole.Assistant, "Previous answer")
-        };
-
-        var reducerMock = new Mock<IChatReducer>();
-        reducerMock
-            .Setup(r => r.ReduceAsync(It.IsAny<IEnumerable<ChatMessage>>(), It.IsAny<CancellationToken>()))
-            .ReturnsAsync([]);  // Simulates an aggressive reducer that clears all messages it receives
-
-        var provider = new InMemoryChatHistoryProvider(new() { ChatReducer = reducerMock.Object, ReducerTriggerEvent = InMemoryChatHistoryProviderOptions.ChatReducerTriggerEvent.AfterMessageAdded });
-        provider.SetMessages(session, new List<ChatMessage>(existingMessages));
-
-        var requestMessages = new List<ChatMessage>
-        {
-            new(ChatRole.User, "What is the weather in Taggia?")
-        };
-        var responseMessages = new List<ChatMessage>
-        {
-            new(ChatRole.Assistant, [new FunctionCallContent("call1", "GetWeather", new Dictionary<string, object?> { ["location"] = "Taggia" })]),
-            new(ChatRole.Tool, [new FunctionResultContent("call1", "Cloudy with a high of 15°C")]),
-            new(ChatRole.Assistant, "The weather in Taggia is cloudy with a high of 15°C.")
-        };
-
-        // Act
-        var context = new ChatHistoryProvider.InvokedContext(s_mockAgent, session, requestMessages, responseMessages);
-        await provider.InvokedAsync(context, CancellationToken.None);
-
-        // Assert - all current-turn messages (including function call and tool result) are preserved
-        var messages = provider.GetMessages(session);
-        Assert.Equal(4, messages.Count);
-        Assert.Equal("What is the weather in Taggia?", messages[0].Text);
-        Assert.True(messages[1].Contents.OfType<FunctionCallContent>().Any(), "Function call message should be preserved");
-        Assert.True(messages[2].Contents.OfType<FunctionResultContent>().Any(), "Tool result message should be preserved");
-        Assert.Equal("The weather in Taggia is cloudy with a high of 15°C.", messages[3].Text);
+        reducerMock.Verify(r => r.ReduceAsync(It.Is<List<ChatMessage>>(x => x.SequenceEqual(originalMessages)), It.IsAny<CancellationToken>()), Times.Once);
    }

    [Fact]
@@ -8,7 +8,6 @@

  <ItemGroup>
    <PackageReference Include="Microsoft.AspNetCore.TestHost" />
-    <PackageReference Include="OpenAI" />
  </ItemGroup>

  <ItemGroup>
@@ -10,7 +10,6 @@

  <ItemGroup Condition="!$([MSBuild]::IsTargetFrameworkCompatible($(TargetFramework), 'net10.0'))">
    <PackageReference Include="Microsoft.Bcl.AsyncInterfaces" />
-    <PackageReference Include="System.Linq.AsyncEnumerable" />
  </ItemGroup>
  
  <ItemGroup>
@@ -6,7 +6,6 @@

  <PropertyGroup>
    <InjectSharedIntegrationTestCode>true</InjectSharedIntegrationTestCode>
-    <InjectSharedBuildTestCode>true</InjectSharedBuildTestCode>
  </PropertyGroup>

  <ItemGroup>
@@ -127,6 +127,42 @@ public sealed class FileAgentSkillsProviderTests : IDisposable
        Assert.Equal("options", ex.ParamName);
    }

+    [Fact]
+    public void Constructor_PromptWithoutPlaceholder_ThrowsArgumentException()
+    {
+        // Arrange -- valid format string but missing the required placeholder
+        var options = new FileAgentSkillsProviderOptions
+        {
+            SkillsInstructionPrompt = "No placeholder here"
+        };
+
+        var ex = Assert.Throws<ArgumentException>(() => new FileAgentSkillsProvider(this._testRoot, options));
+        Assert.Contains("{0}", ex.Message);
+        Assert.Equal("options", ex.ParamName);
+    }
+
+    [Fact]
+    public async Task Constructor_PromptWithPlaceholder_AppliesCustomTemplateAsync()
+    {
+        // Arrange — valid custom template with {0} placeholder
+        this.CreateSkill("custom-tpl-skill", "Custom template skill", "Body.");
+        var options = new FileAgentSkillsProviderOptions
+        {
+            SkillsInstructionPrompt = "== Skills ==\n{0}\n== End =="
+        };
+        var provider = new FileAgentSkillsProvider(this._testRoot, options);
+        var invokingContext = new AIContextProvider.InvokingContext(this._agent, session: null, new AIContext());
+
+        // Act
+        var result = await provider.InvokingAsync(invokingContext, CancellationToken.None);
+
+        // Assert — the custom template wraps the skill list
+        Assert.NotNull(result.Instructions);
+        Assert.StartsWith("== Skills ==", result.Instructions);
+        Assert.Contains("custom-tpl-skill", result.Instructions);
+        Assert.Contains("== End ==", result.Instructions);
+    }
+
    [Fact]
    public async Task InvokingCoreAsync_SkillNamesAreXmlEscapedAsync()
    {
@@ -0,0 +1,159 @@
+// Copyright (c) Microsoft. All rights reserved.
+
+using System;
+using System.Collections.Generic;
+using System.Linq;
+using System.Threading;
+using System.Threading.Tasks;
+using Microsoft.Agents.AI.Compaction;
+using Microsoft.Extensions.AI;
+
+namespace Microsoft.Agents.AI.UnitTests.Compaction;
+
+/// <summary>
+/// Contains tests for the <see cref="ChatStrategyExtensions"/> class.
+/// </summary>
+public class ChatStrategyExtensionsTests
+{
+    [Fact]
+    public void AsChatReducerNullStrategyThrows()
+    {
+        // Act & Assert
+        Assert.Throws<ArgumentNullException>(() => ((CompactionStrategy)null!).AsChatReducer());
+    }
+
+    [Fact]
+    public void AsChatReducerReturnsIChatReducer()
+    {
+        // Arrange
+        ChatReducerCompactionStrategy strategy = new(new IdentityReducer(), CompactionTriggers.Always);
+
+        // Act
+        IChatReducer reducer = strategy.AsChatReducer();
+
+        // Assert
+        Assert.NotNull(reducer);
+    }
+
+    [Fact]
+    public async Task ReduceAsyncReturnsAllMessagesWhenStrategyDoesNotCompactAsync()
+    {
+        // Arrange — trigger never fires, so no compaction occurs
+        ChatReducerCompactionStrategy strategy = new(new IdentityReducer(), CompactionTriggers.Never);
+        IChatReducer reducer = strategy.AsChatReducer();
+
+        List<ChatMessage> messages =
+        [
+            new(ChatRole.User, "Hello"),
+            new(ChatRole.Assistant, "Hi!"),
+        ];
+
+        // Act
+        IEnumerable<ChatMessage> result = await reducer.ReduceAsync(messages, CancellationToken.None);
+
+        // Assert
+        Assert.Equal(messages, result);
+    }
+
+    [Fact]
+    public async Task ReduceAsyncCompactsMessagesWhenStrategyFiresAsync()
+    {
+        // Arrange — reducer keeps only the last message
+        ChatReducerCompactionStrategy strategy = new(
+            new TakeLastReducer(1),
+            CompactionTriggers.Always);
+        IChatReducer reducer = strategy.AsChatReducer();
+
+        List<ChatMessage> messages =
+        [
+            new(ChatRole.User, "First"),
+            new(ChatRole.Assistant, "Response 1"),
+            new(ChatRole.User, "Second"),
+        ];
+
+        // Act
+        IEnumerable<ChatMessage> result = await reducer.ReduceAsync(messages, CancellationToken.None);
+
+        // Assert
+        List<ChatMessage> resultList = [.. result];
+        Assert.Single(resultList);
+        Assert.Equal("Second", resultList[0].Text);
+    }
+
+    [Fact]
+    public async Task ReduceAsyncPassesCancellationTokenToStrategyAsync()
+    {
+        // Arrange
+        using CancellationTokenSource cts = new();
+        CancellationToken capturedToken = default;
+
+        CapturingReducer capturingReducer = new(token => capturedToken = token);
+        ChatReducerCompactionStrategy strategy = new(capturingReducer, CompactionTriggers.Always);
+        IChatReducer reducer = strategy.AsChatReducer();
+
+        List<ChatMessage> messages =
+        [
+            new(ChatRole.User, "Hello"),
+            new(ChatRole.User, "World"),
+        ];
+
+        // Act
+        await reducer.ReduceAsync(messages, cts.Token);
+
+        // Assert
+        Assert.Equal(cts.Token, capturedToken);
+    }
+
+    [Fact]
+    public async Task ReduceAsyncEmptyMessagesReturnsEmptyAsync()
+    {
+        // Arrange
+        ChatReducerCompactionStrategy strategy = new(new IdentityReducer(), CompactionTriggers.Always);
+        IChatReducer reducer = strategy.AsChatReducer();
+
+        // Act
+        IEnumerable<ChatMessage> result = await reducer.ReduceAsync([], CancellationToken.None);
+
+        // Assert
+        Assert.Empty(result);
+    }
+
+    /// <summary>
+    /// An <see cref="IChatReducer"/> that returns messages unchanged.
+    /// </summary>
+    private sealed class IdentityReducer : IChatReducer
+    {
+        public Task<IEnumerable<ChatMessage>> ReduceAsync(IEnumerable<ChatMessage> messages, CancellationToken cancellationToken = default)
+            => Task.FromResult(messages);
+    }
+
+    /// <summary>
+    /// An <see cref="IChatReducer"/> that keeps only the last <c>n</c> messages.
+    /// </summary>
+    private sealed class TakeLastReducer : IChatReducer
+    {
+        private readonly int _count;
+
+        public TakeLastReducer(int count) => this._count = count;
+
+        public Task<IEnumerable<ChatMessage>> ReduceAsync(IEnumerable<ChatMessage> messages, CancellationToken cancellationToken = default)
+            => Task.FromResult(messages.Reverse().Take(this._count));
+    }
+
+    /// <summary>
+    /// An <see cref="IChatReducer"/> that captures the <see cref="CancellationToken"/> passed to <see cref="ReduceAsync"/>.
+    /// </summary>
+    private sealed class CapturingReducer : IChatReducer
+    {
+        private readonly Action<CancellationToken> _capture;
+
+        public CapturingReducer(Action<CancellationToken> capture) => this._capture = capture;
+
+        public Task<IEnumerable<ChatMessage>> ReduceAsync(IEnumerable<ChatMessage> messages, CancellationToken cancellationToken = default)
+        {
+            this._capture(cancellationToken);
+            IEnumerable<ChatMessage> reducedMessages = [messages.Reverse().First()];
+            return Task.FromResult(reducedMessages);
+        }
+    }
+}
@@ -1,5 +1,6 @@
 // Copyright (c) Microsoft. All rights reserved.

+using System;
 using System.Collections.Generic;
 using System.Threading.Tasks;
 using Microsoft.Agents.AI.Compaction;
@@ -348,4 +349,90 @@ public class ToolResultCompactionStrategyTests
        List<ChatMessage> included = [.. groups.GetIncludedMessages()];
        Assert.Equal("[Tool Calls]\nget_weather:\n  - Sunny\n  - Rainy\nsearch_docs:\n  - Found 3 docs", included[1].Text);
    }
+
+    [Fact]
+    public async Task CompactAsyncUsesCustomFormatterAsync()
+    {
+        // Arrange — custom formatter that produces a collapsed message count
+        static string CustomFormatter(CompactionMessageGroup group) =>
+            $"[Collapsed: {group.Messages.Count} messages]";
+
+        ToolResultCompactionStrategy strategy = new(
+            trigger: _ => true,
+            minimumPreservedGroups: 1)
+        {
+            ToolCallFormatter = CustomFormatter,
+        };
+
+        CompactionMessageIndex groups = CompactionMessageIndex.Create(
+        [
+            new ChatMessage(ChatRole.User, "Q1"),
+            new ChatMessage(ChatRole.Assistant, [new FunctionCallContent("c1", "get_weather")]),
+            new ChatMessage(ChatRole.Tool, "Sunny"),
+            new ChatMessage(ChatRole.User, "Q2"),
+        ]);
+
+        // Act
+        bool result = await strategy.CompactAsync(groups);
+
+        // Assert — custom formatter output used instead of default YAML-like format
+        Assert.True(result);
+        List<ChatMessage> included = [.. groups.GetIncludedMessages()];
+        Assert.Equal("[Collapsed: 2 messages]", included[1].Text);
+    }
+
+    [Fact]
+    public void ToolCallFormatterPropertyIsNullWhenNoneProvided()
+    {
+        // Arrange
+        ToolResultCompactionStrategy strategy = new(CompactionTriggers.Always);
+
+        // Assert — ToolCallFormatter is null when no custom formatter is provided
+        Assert.Null(strategy.ToolCallFormatter);
+    }
+
+    [Fact]
+    public void ToolCallFormatterPropertyReturnsCustomFormatterWhenProvided()
+    {
+        // Arrange
+        Func<CompactionMessageGroup, string> customFormatter = static _ => "custom";
+        ToolResultCompactionStrategy strategy = new(
+            CompactionTriggers.Always)
+        {
+            ToolCallFormatter = customFormatter
+        };
+
+        // Assert — ToolCallFormatter is the injected custom function
+        Assert.Same(customFormatter, strategy.ToolCallFormatter);
+    }
+
+    [Fact]
+    public async Task CompactAsyncCustomFormatterCanDelegateToDefaultAsync()
+    {
+        // Arrange — custom formatter that wraps the default output
+        static string WrappingFormatter(CompactionMessageGroup group) =>
+            $"CUSTOM_PREFIX\n{ToolResultCompactionStrategy.DefaultToolCallFormatter(group)}";
+
+        ToolResultCompactionStrategy strategy = new(
+            trigger: _ => true,
+            minimumPreservedGroups: 1)
+        {
+            ToolCallFormatter = WrappingFormatter
+        };
+
+        CompactionMessageIndex groups = CompactionMessageIndex.Create(
+        [
+            new ChatMessage(ChatRole.User, "Q1"),
+            new ChatMessage(ChatRole.Assistant, [new FunctionCallContent("c1", "fn")]),
+            new ChatMessage(ChatRole.Tool, "result"),
+            new ChatMessage(ChatRole.User, "Q2"),
+        ]);
+
+        // Act
+        await strategy.CompactAsync(groups);
+
+        // Assert — wrapped default output
+        List<ChatMessage> included = [.. groups.GetIncludedMessages()];
+        Assert.Equal("CUSTOM_PREFIX\n[Tool Calls]\nfn:\n  - result", included[1].Text);
+    }
 }
@@ -16,7 +16,6 @@

  <ItemGroup>
    <PackageReference Include="Azure.Identity" />
-    <PackageReference Include="FluentAssertions" />
    <PackageReference Include="Microsoft.CodeAnalysis.CSharp" />
    <PackageReference Include="Microsoft.Extensions.Configuration.UserSecrets" />
    <PackageReference Include="Microsoft.Extensions.Configuration.EnvironmentVariables" />
@@ -651,7 +651,7 @@ public class ExecutorRouteGeneratorTests
    }

    [Fact]
-    public void PartialClass_SendsYieldsInBothFiles_GeneratesAlOverrides()
+    public void PartialClass_SendsYieldsInBothFiles_GeneratesAllOverrides()
    {
        // File 1: Partial with one handler
        var file1 = """
@@ -700,7 +700,7 @@ public class ExecutorRouteGeneratorTests
        generated.Should().RegisterSentMessageType("string")
                      .And.RegisterSentMessageType("int")
                      .And.RegisterYieldedOutputType("string")
-                      .And.RegisterYieldedOutputType("string");
+                      .And.RegisterYieldedOutputType("int");
    }

    #endregion
@@ -1046,6 +1046,85 @@ public class ExecutorRouteGeneratorTests
                      .And.RegisterSentMessageType("global::TestNamespace.BroadcastMessage");
    }

+    [Fact]
+    public void ProtocolOnly_DerivesFromExecutorOfT_GeneratesBaseCall()
+    {
+        // A protocol-only partial executor deriving from Executor<T>
+        // has a base class that already overrides ConfigureProtocol. The generator must emit
+        // "return base.ConfigureProtocol(protocolBuilder)" so inherited handler registrations
+        // are preserved — not "return protocolBuilder" which silently drops them.
+        var source = """
+            using System;
+            using System.Threading;
+            using System.Threading.Tasks;
+            using Microsoft.Agents.AI.Workflows;
+
+            namespace TestNamespace;
+
+            public class FeedbackResult { }
+
+            [SendsMessage(typeof(FeedbackResult))]
+            [YieldsOutput(typeof(string))]
+            public partial class FeedbackExecutor : Executor<string>
+            {
+                public FeedbackExecutor() : base("feedback") { }
+
+                public override System.Threading.Tasks.ValueTask HandleAsync(string message, IWorkflowContext context, System.Threading.CancellationToken cancellationToken = default)
+                    => default;
+            }
+            """;
+
+        var result = GeneratorTestHelper.RunGenerator(source);
+
+        result.RunResult.GeneratedTrees.Should().HaveCount(1);
+        result.RunResult.Diagnostics.Should().BeEmpty();
+
+        var generated = result.RunResult.GeneratedTrees[0].ToString();
+
+        // Base class Executor<T> overrides ConfigureProtocol, so the generated override
+        // must chain to base to preserve the inherited handler registration.
+        generated.Should().Contain("return base.ConfigureProtocol(protocolBuilder)",
+            because: "Executor<T> overrides ConfigureProtocol, so base must be called to preserve its handler registration");
+        generated.Should().Contain(".SendsMessage<global::TestNamespace.FeedbackResult>()");
+        generated.Should().Contain(".YieldsOutput<string>()");
+    }
+
+    [Fact]
+    public void ProtocolOnly_DerivesDirectlyFromExecutor_DoesNotGenerateBaseCall()
+    {
+        // A protocol-only partial executor deriving directly from Executor (abstract base
+        // with no non-abstract ConfigureProtocol override) should generate "return protocolBuilder"
+        // rather than "return base.ConfigureProtocol(protocolBuilder)".
+        var source = """
+            using System;
+            using System.Threading;
+            using System.Threading.Tasks;
+            using Microsoft.Agents.AI.Workflows;
+
+            namespace TestNamespace;
+
+            public class BroadcastMessage { }
+
+            [SendsMessage(typeof(BroadcastMessage))]
+            public partial class BroadcastExecutor : Executor
+            {
+                public BroadcastExecutor() : base("broadcast") { }
+            }
+            """;
+
+        var result = GeneratorTestHelper.RunGenerator(source);
+
+        result.RunResult.GeneratedTrees.Should().HaveCount(1);
+        result.RunResult.Diagnostics.Should().BeEmpty();
+
+        var generated = result.RunResult.GeneratedTrees[0].ToString();
+
+        // Executor's ConfigureProtocol is abstract — no base call needed.
+        generated.Should().Contain("return protocolBuilder",
+            because: "Executor base class has no non-abstract ConfigureProtocol, so no base call is needed");
+        generated.Should().NotContain("base.ConfigureProtocol");
+    }
+
    #endregion

    #region Generic Executor Tests
@@ -2,6 +2,7 @@

 using System;
 using System.Collections.Generic;
+using System.Linq;
 using System.Threading;
 using System.Threading.Tasks;
 using FluentAssertions;
@@ -199,4 +200,43 @@ public class EdgeRunnerTests
            mapping.CheckDeliveries(["executor3"], ["part1", "part2", "final part"]);
        }
    }
+
+    [Fact]
+    public async Task Test_FanInEdgeRunner_ConcurrentProcessingAsync()
+    {
+        // Arrange
+        const int SourceCount = 4;
+        const int Iterations = 50;
+
+        string[] sourceIds = Enumerable.Range(0, SourceCount).Select(i => $"source{i}").ToArray();
+        const string SinkId = "sink";
+
+        TestRunContext runContext = new();
+        List<Executor> executors = [.. sourceIds.Select(id => (Executor)new ForwardMessageExecutor<string>(id)), new ForwardMessageExecutor<string>(SinkId)];
+        runContext.ConfigureExecutors(executors);
+
+        FanInEdgeData edgeData = new(sourceIds.ToList(), SinkId, new EdgeId(0), null);
+        FanInEdgeRunner runner = new(runContext, edgeData);
+
+        for (int iteration = 0; iteration < Iterations; iteration++)
+        {
+            // Act: send messages from all sources concurrently
+            using Barrier barrier = new(SourceCount);
+            Task<DeliveryMapping?>[] tasks = sourceIds.Select(sourceId => Task.Run(async () =>
+            {
+                barrier.SignalAndWait();
+                return await runner.ChaseEdgeAsync(new($"msg-from-{sourceId}", sourceId), stepTracer: null, CancellationToken.None);
+            })).ToArray();
+
+            DeliveryMapping?[] results = await Task.WhenAll(tasks);
+
+            // Assert: exactly one task should return a non-null mapping with all messages
+            DeliveryMapping?[] nonNullResults = results.Where(r => r is not null).ToArray();
+            nonNullResults.Should().HaveCount(1, $"iteration {iteration}: exactly one thread should release the batch");
+
+            DeliveryMapping mapping = nonNullResults[0]!;
+            HashSet<object> expectedMessages = [.. sourceIds.Select(id => (object)$"msg-from-{id}")];
+            mapping.CheckDeliveries([SinkId], expectedMessages);
+        }
+    }
 }
@@ -13,26 +13,34 @@ description: >
 All commands run from the `python/` directory:

 ```bash
-# Format code (ruff format, parallel across packages)
-uv run poe fmt
-
-# Lint and auto-fix (ruff check, parallel across packages)
-uv run poe lint
+# Syntax formatting + checks (parallel across packages by default)
+uv run poe syntax
+uv run poe syntax -P core
+uv run poe syntax -F    # Format only
+uv run poe syntax -C    # Check only
+uv run poe syntax -S    # Samples only

 # Type checking
-uv run poe pyright       # Pyright (parallel across packages)
-uv run poe mypy          # MyPy (parallel across packages)
+uv run poe pyright       # Pyright fan-out across packages
+uv run poe pyright -P core
+uv run poe pyright -A
+uv run poe mypy          # MyPy fan-out across packages
+uv run poe mypy -P core
+uv run poe mypy -A
 uv run poe typing        # Both pyright and mypy
+uv run poe typing -P core
+uv run poe typing -A

-# All package-level checks in parallel (fmt + lint + pyright + mypy)
+# All package-level checks in parallel (syntax + pyright)
 uv run poe check-packages

 # Full check (packages + samples + tests + markdown)
 uv run poe check
+uv run poe check -P core

 # Samples only
-uv run poe samples-lint     # Ruff lint on samples/
-uv run poe samples-syntax   # Pyright syntax check on samples/
+uv run poe check -S
+uv run poe pyright -S

 # Markdown code blocks
 uv run poe markdown-code-lint
@@ -40,8 +48,8 @@ uv run poe markdown-code-lint

 ## Pre-commit Hooks (prek)

-Prek hooks run automatically on commit. They check only changed files and run
-package-level checks in parallel for affected packages only.
+Prek hooks run automatically on commit. They stay lightweight and only check
+changed files.

 ```bash
 # Install hooks
@@ -54,8 +62,10 @@ uv run prek run -a
 uv run prek run --last-commit
 ```

-When core package changes, type-checking (mypy, pyright) runs across all packages
-since type changes propagate. Format and lint only run in changed packages.
+They run changed-package syntax formatting/checking, markdown code lint only
+when markdown files change, and sample syntax lint/pyright only when files
+under `samples/` change.
+They intentionally do not run workspace `pyright` or `mypy` by default.

 ## Ruff Configuration

@@ -80,6 +90,6 @@ in-process with streaming output.

 CI splits into 4 parallel jobs:
 1. **Pre-commit hooks** — lightweight hooks (SKIP=poe-check)
-2. **Package checks** — fmt/lint/pyright via check-packages
-3. **Samples & markdown** — samples-lint, samples-syntax, markdown-code-lint
+2. **Package checks** — syntax/pyright via check-packages
+3. **Samples & markdown** — `check -S` plus `markdown-code-lint`
 4. **Mypy** — change-detected mypy checks
@@ -47,17 +47,17 @@ uv run poe upgrade-dev-dependencies

 # First, run workspace-wide lower/upper compatibility gates
 uv run poe validate-dependency-bounds-test
-# Defaults to --project "*"; pass a package to scope test mode
-uv run poe validate-dependency-bounds-test --project <workspace-package-name>
+# Defaults to --package "*"; pass a package to scope test mode
+uv run poe validate-dependency-bounds-test --package core

 # Then expand bounds for one dependency in the target package
-uv run poe validate-dependency-bounds-project --mode both --project <workspace-package-name> --dependency "<dependency-name>"
+uv run poe validate-dependency-bounds-project --mode both --package core --dependency "<dependency-name>"

 # Repo-wide automation can reuse the same task
-uv run poe validate-dependency-bounds-project --mode upper --project "*"
+uv run poe validate-dependency-bounds-project --mode upper --package "*"

 # Add a dependency to one project and run both validators for that project/dependency
-uv run poe add-dependency-and-validate-bounds --project <workspace-package-name> --dependency "<dependency-spec>"
+uv run poe add-dependency-and-validate-bounds --package core --dependency "<dependency-spec>"
 ```

 ### Dependency Bound Notes
@@ -66,7 +66,7 @@ uv run poe add-dependency-and-validate-bounds --project <workspace-package-name>
 - Prerelease (`dev`/`a`/`b`/`rc`) and `<1.0` dependencies should use hard bounds with an explicit upper cap (avoid open-ended ranges).
 - For `<1.0` dependencies, prefer the broadest validated range the package can really support. That may be a patch line, a minor line, or multiple minor lines when checks/tests show the broader lane is compatible.
 - Prefer supporting multiple majors when practical; if APIs diverge across supported majors, use version-conditional imports/paths.
- For dependency changes, run workspace-wide bound gates first, then `validate-dependency-bounds-project --mode both` for the target package/dependency to keep minimum and maximum constraints current. The same task can also drive repo-wide upper-bound automation by using `--project "*"` and omitting `--dependency`.
+- For dependency changes, run workspace-wide bound gates first, then `validate-dependency-bounds-project --mode both` for the target package/dependency to keep minimum and maximum constraints current. The same task can also drive repo-wide upper-bound automation by using `--package "*"` and omitting `--dependency`.
 - Prefer targeted lock updates with `uv lock --upgrade-package <dependency-name>` to reduce `uv.lock` merge conflicts.
 - Use `add-dependency-and-validate-bounds` for package-scoped dependency additions plus bound validation in one command.
 - Use `upgrade-dev-dependencies` for repo-wide dev tooling refreshes; it repins dev dependencies, refreshes `uv.lock`, and reruns `check`, `typing`, and `test`.
@@ -108,12 +108,12 @@ def __getattr__(name: str) -> Any:
 Recommended dependency workflow during connector implementation:

 1. Add the dependency to the target package:
-   `uv run poe add-dependency-to-project --project <workspace-package-name> --dependency "<dependency-spec>"`
+   `uv run poe add-dependency-to-project --package core --dependency "<dependency-spec>"`
 2. Implement connector code and tests.
 3. Validate dependency bounds for that package/dependency:
-   `uv run poe validate-dependency-bounds-project --mode both --project <workspace-package-name> --dependency "<dependency-name>"`
+   `uv run poe validate-dependency-bounds-project --mode both --package core --dependency "<dependency-name>"`
 4. If the package has meaningful tests/checks that validate dependency compatibility, you can use the add + validation flow in one command:
-   `uv run poe add-dependency-and-validate-bounds --project <workspace-package-name> --dependency "<dependency-spec>"`
+   `uv run poe add-dependency-and-validate-bounds --package core --dependency "<dependency-spec>"`
   If compatibility checks are not in place yet, add the dependency first, then implement tests before running bound validation.

 ### Promotion to Stable
@@ -41,11 +41,14 @@ Do **not** add sample-only dependencies to the root `pyproject.toml` dev group.
 ## Syntax Checking

 ```bash
-# Check samples for syntax errors and missing imports
-uv run poe samples-syntax
+# Format + lint samples
+uv run poe syntax -S

-# Lint samples
-uv run poe samples-lint
+# Check samples for syntax errors and missing imports
+uv run poe pyright -S
+
+# Lint samples only
+uv run poe syntax -S -C
 ```

 ## Documentation
@@ -17,20 +17,27 @@ We run tests in two stages, for a PR each commit is tested with unit tests only
 # Run tests for all packages in parallel
 uv run poe test

-# Run tests for a specific package
-uv run --directory packages/core poe test
+# Run tests for a specific workspace package
+uv run poe test -P core

-# Run all tests in a single pytest invocation (faster, uses pytest-xdist)
-uv run poe all-tests
+# Run all selected tests in a single pytest invocation
+uv run poe test -A

 # With coverage
-uv run poe all-tests-cov
+uv run poe test -A -C
+uv run poe test -P core -C

 # Run only unit tests (exclude integration tests)
-uv run poe all-tests -m "not integration"
+uv run poe test -A -m "not integration"

 # Run only integration tests
-uv run poe all-tests -m integration
+uv run poe test -A -m integration
+```
+
+Direct package execution still works when you need it:
+
+```bash
+uv run --directory packages/core poe test
 ```

 ## Test Configuration
@@ -38,7 +45,7 @@ uv run poe all-tests -m integration
 - **Async mode**: `asyncio_mode = "auto"` is enabled — do NOT use `@pytest.mark.asyncio`, but do mark tests with `async def` and use `await` for async calls
 - **Timeout**: Default 60 seconds per test
 - **Import mode**: `importlib` for cross-package isolation
- **Parallelization**: Large packages (core, ag-ui, orchestrations, anthropic) use `pytest-xdist` (`-n auto --dist worksteal`) in their `poe test` task. The `all-tests` task also uses xdist across all packages.
+- **Parallelization**: Large packages (core, ag-ui, orchestrations, anthropic) use `pytest-xdist` (`-n auto --dist worksteal`) in their `poe test` task. The aggregate `uv run poe test -A` sweep also uses xdist across the selected packages.

 ## Test Directory Structure

@@ -52,10 +52,10 @@ repos:
    hooks:
      - id: poe-check
        name: Run checks through Poe
-        entry: uv run poe prek-check
+        entry: uv run python scripts/workspace_poe_tasks.py prek-check
        language: system
  - repo: https://github.com/PyCQA/bandit
-    rev: 1.9.3
+    rev: 1.9.4
    hooks:
      - id: bandit
        name: Bandit Security Checks
@@ -63,7 +63,7 @@ repos:
        additional_dependencies: ["bandit[toml]"]
  - repo: https://github.com/astral-sh/uv-pre-commit
    # uv version.
-    rev: 0.10.0
+    rev: 0.10.10
    hooks:
      # Update the uv lockfile
      - id: uv-lock
@@ -9,9 +9,8 @@
            "command": "uv",
            "args": [
                "run",
-                "prek",
-                "run",
-                "-a"
+                "poe",
+                "check"
            ],
            "problemMatcher": {
                "owner": "python",
@@ -32,13 +31,13 @@
            }
        },
        {
-            "label": "Format",
+            "label": "Syntax",
            "type": "shell",
            "command": "uv",
            "args": [
                "run",
                "poe",
-                "fmt",
+                "syntax",
            ],
            "problemMatcher": {
                "owner": "python",
@@ -59,13 +58,42 @@
            }
        },
        {
-            "label": "Lint",
+            "label": "Syntax (format only)",
            "type": "shell",
            "command": "uv",
            "args": [
                "run",
                "poe",
-                "lint",
+                "syntax",
+                "-F",
+            ],
+            "problemMatcher": {
+                "owner": "python",
+                "fileLocation": [
+                    "relative",
+                    "${workspaceFolder}"
+                ],
+                "pattern": {
+                    "regexp": "^(.*):(\\d+):(\\d+):\\s+(.*)$",
+                    "file": 1,
+                    "line": 2,
+                    "column": 3,
+                    "message": 4
+                }
+            },
+            "presentation": {
+                "panel": "shared"
+            }
+        },
+        {
+            "label": "Syntax (check only)",
+            "type": "shell",
+            "command": "uv",
+            "args": [
+                "run",
+                "poe",
+                "syntax",
+                "-C",
            ],
            "problemMatcher": {
                "owner": "python",
@@ -169,7 +197,14 @@
        {
            "label": "Create Venv",
            "type": "shell",
-            "command": "uv venv PYTHON=${input:py_version}",
+            "command": "uv",
+            "args": [
+                "run",
+                "poe",
+                "venv",
+                "-P",
+                "${input:py_version}"
+            ],
            "presentation": {
                "reveal": "always",
                "panel": "new"
@@ -184,7 +219,8 @@
                "run",
                "poe",
                "setup",
-                "--python=${input:py_version}"
+                "-P",
+                "${input:py_version}"
            ],
            "presentation": {
                "reveal": "always",
@@ -200,11 +236,12 @@
                "3.10",
                "3.11",
                "3.12",
-                "3.13"
+                "3.13",
+                "3.14"
            ],
            "id": "py_version",
            "description": "Python version",
-            "default": "3.10"
+            "default": "3.13"
        }
    ]
-}
+}
@@ -403,7 +403,7 @@ So we use bounded ranges for external package dependencies in `pyproject.toml`:
 - For `<1.0.0` dependencies, use a known-good bounded range with an explicit upper cap. Prefer the broadest validated range the package can actually support: that may be a patch line, a minor line, or multiple minor lines (for example: `a2a-sdk>=0.3.5,<0.4.0`, `fastapi>=0.115.0,<0.136.0`, `uvicorn>=0.30.0,<0.39.0`).
 - For prerelease (`dev`/`a`/`b`/`rc`) dependencies, use a known-good bounded range with a hard upper cap and keep the range only as broad as the package's validation coverage justifies.
 - Prefer keeping support for multiple major versions when practical. This may mean that the upper bound spans multiple major versions when the dependency maintains backward compatibility; if APIs differ between supported majors, version-conditional imports/branches are acceptable to preserve compatibility.
- When adding or changing an external dependency, first run `uv run poe validate-dependency-bounds-test` to validate workspace-wide lower/upper compatibility, then run `uv run poe validate-dependency-bounds-project --mode both --project <workspace-package-name> --dependency "<dependency-name>"` to expand package-scoped bounds.
+- When adding or changing an external dependency, first run `uv run poe validate-dependency-bounds-test` to validate workspace-wide lower/upper compatibility, then run `uv run poe validate-dependency-bounds-project --mode both --package <workspace-package-name> --dependency "<dependency-name>"` to expand package-scoped bounds.

 ### Installation Options

@@ -123,28 +123,39 @@ client = OpenAIChatClient(env_file_path="openai.env")

 All the tests are located in the `tests` folder of each package. Tests marked with `@pytest.mark.integration` and `@skip_if_..._integration_tests_disabled` are integration tests that require external services (e.g., OpenAI, Azure OpenAI). They are automatically skipped when the required API keys or service endpoints are not configured in your environment or `.env` file.

-You can select or exclude integration tests using pytest markers:
+The root `test` command now supports both project-scoped fan-out and a single aggregate sweep:

 ```bash
-# Run only unit tests (exclude integration tests)
-uv run poe all-tests -m "not integration"
+# Run package-local tests across all workspace packages
+uv run poe test

-# Run only integration tests
-uv run poe all-tests -m integration
+# Run tests for one workspace package
+uv run poe test -P core
+
+# Run an aggregate pytest sweep across the selected packages
+uv run poe test -A
+
+# Run only unit tests in aggregate mode
+uv run poe test -A -m "not integration"
+
+# Run only integration tests in aggregate mode
+uv run poe test -A -m integration
+
+# Run tests with coverage for one package or an aggregate sweep
+uv run poe test -P core -C
+uv run poe test -A -C
 ```

 Alternatively, you can run them using VSCode Tasks. Open the command palette
 (`Ctrl+Shift+P`) and type `Tasks: Run Task`. Select `Test` from the list.

-If you want to run the tests for a single package, you can use the `uv run poe test` command with the package name as an argument. For example, to run the tests for the `agent_framework` package, you can use:
+Direct package execution still works when you need it:

 ```bash
 uv run poe --directory packages/core test
 ```

-Large packages (core, ag-ui, orchestrations, anthropic) use `pytest-xdist` for parallel test execution within the package. The `all-tests` task also uses xdist across all packages.
-
-These commands also output the coverage report.
+Large packages (core, ag-ui, orchestrations, anthropic) use `pytest-xdist` for parallel test execution within the package. The aggregate `test -A` sweep also uses `pytest-xdist` across the selected packages.

 ## Code quality checks

@@ -158,10 +169,11 @@ Ideally you should run these checks before committing any changes, when you inst

 ## Code Coverage

-We try to maintain a high code coverage for the project. To run the code coverage on the unit tests, you can use the following command:
+We try to maintain a high code coverage for the project. To review coverage locally, use either a package-scoped run or the aggregate sweep:

 ```bash
-    uv run poe test
+uv run poe test -P core -C
+uv run poe test -A -C
 ```

 This will show you which files are not covered by the tests, including the specific lines not covered. Make sure to consider the untested lines from the code you are working on, but feel free to add other tests as well, that is always welcome!
@@ -213,7 +225,7 @@ Set up the development environment with a virtual environment, install dependenc
 ```bash
 uv run poe setup
 # or with specific Python version
-uv run poe setup --python 3.12
+uv run poe setup -P 3.12
 ```

 #### `install`
@@ -230,7 +242,7 @@ Create a virtual environment with specified Python version or switch python vers
 ```bash
 uv run poe venv
 # or with specific Python version
-uv run poe venv --python 3.12
+uv run poe venv -P 3.12
 ```

 #### `prek-install`
@@ -239,41 +251,89 @@ Install prek hooks:
 uv run poe prek-install
 ```

-### Code Quality and Formatting
+### Project-scoped command families

-Each of the following tasks run against both the main `agent-framework` package and the extension packages in parallel, ensuring consistent code quality across the project.
+These commands default to `--package "*"`, so they run across all workspace packages unless you narrow them with `-P/--package`:

-#### `fmt` (format)
-Format code using ruff (runs in parallel across all packages):
+#### `syntax`
+Run Ruff formatting plus Ruff lint checks by default:
 ```bash
-uv run poe fmt
+uv run poe syntax
+uv run poe syntax -P core
+uv run poe syntax -F        # format only
+uv run poe syntax -C        # lint/check only
 ```

-#### `lint`
-Run linting checks and fix issues (runs in parallel across all packages):
+#### `build`
+Build workspace packages and the root meta package:
 ```bash
-uv run poe lint
+uv run poe build
+uv run poe build -P core
+```
+
+#### `clean-dist`
+Clean generated dist artifacts:
+```bash
+uv run poe clean-dist
+uv run poe clean-dist -P core
+```
+
+### Dual-mode validation and test commands
+
+These command families share the same selector model:
+
+```bash
+uv run poe <command>              # project fan-out over --package "*"
+uv run poe <command> -P core      # one-project fan-out
+uv run poe <command> -A           # aggregate sweep where supported
 ```

 #### `pyright`
-Run Pyright type checking (runs in parallel across all packages):
+Run Pyright type checking:
 ```bash
 uv run poe pyright
+uv run poe pyright -P core
+uv run poe pyright -A
 ```

 #### `mypy`
-Run MyPy type checking (runs in parallel across all packages):
+Run MyPy type checking:
 ```bash
 uv run poe mypy
+uv run poe mypy -P core
+uv run poe mypy -A
 ```

 #### `typing`
-Run both Pyright and MyPy type checking:
+Run both Pyright and MyPy:
 ```bash
 uv run poe typing
+uv run poe typing -P core
+uv run poe typing -A
 ```

-### Code Validation
+#### `test`
+Run package-local tests in fan-out mode, or switch to one aggregate pytest sweep with `-A`:
+```bash
+uv run poe test
+uv run poe test -P core
+uv run poe test -P core -C
+uv run poe test -A
+uv run poe test -A -C
+```
+
+### Sample-target variants
+
+Use `-S/--samples` for sample-only validation instead of separate top-level commands:
+
+```bash
+uv run poe syntax -S
+uv run poe syntax -S -C
+uv run poe pyright -S
+uv run poe check -S
+```
+
+### Workspace validation and dependency commands

 #### `markdown-code-lint`
 Lint markdown code blocks:
@@ -281,26 +341,41 @@ Lint markdown code blocks:
 uv run poe markdown-code-lint
 ```

+#### `check-packages`
+Run the package-level syntax sweep (`syntax`) plus `pyright` across the selected projects:
+```bash
+uv run poe check-packages
+uv run poe check-packages -P core
+```
+
+#### `check`
+Run package syntax, pyright, and tests for the selected project set. Without `-P/--package`, it also includes sample checks and markdown lint:
+```bash
+uv run poe check
+uv run poe check -P core
+uv run poe check -S
+```
+
 #### `validate-dependency-bounds-test`
 Run workspace-wide dependency compatibility gates at lower and upper resolutions. This runs test + pyright across all packages and stops on first failure:
 ```bash
 uv run poe validate-dependency-bounds-test
-# Defaults to --project "*"; pass a package to scope test mode
-uv run poe validate-dependency-bounds-test --project <workspace-package-name>
+# Defaults to --package "*"; pass a package to scope test mode
+uv run poe validate-dependency-bounds-test -P core
 ```

 #### `validate-dependency-bounds-project`
 Validate and extend dependency bounds for a single dependency in a single package. Use `--mode lower`, `--mode upper`, or the default `--mode both`:
 ```bash
-uv run poe validate-dependency-bounds-project --mode both --project <workspace-package-name> --dependency "<dependency-name>"
+uv run poe validate-dependency-bounds-project -M both -P core -D "<dependency-name>"
 ```
-`--project` defaults to `*`, and `--dependency` is optional. Automation can use `--mode upper --project "*"` to run the upper-bound pass across the workspace.
+`--package` defaults to `*`, and `--dependency` is optional. Automation can use `--mode upper --package "*"` to run the upper-bound pass across the workspace.
 For `<1.0` dependencies, prefer the broadest validated range the package can really support. That may still be a single patch or minor line, but multi-minor ranges are fine when the package's checks/tests prove they work.

 #### `add-dependency-and-validate-bounds`
 Add an external dependency to a workspace project and run both validators for that same project/dependency:
 ```bash
-uv run poe add-dependency-and-validate-bounds --project <workspace-package-name> --dependency "<dependency-spec>"
+uv run poe add-dependency-and-validate-bounds -P core -D "<dependency-spec>"
 ```

 #### `upgrade-dev-dependencies`
@@ -310,72 +385,40 @@ uv run poe upgrade-dev-dependencies
 ```
 Use this for repo-wide dev tooling refreshes. For targeted runtime dependency upgrades, prefer `uv lock --upgrade-package <dependency-name>` plus the package-scoped bound validation tasks above.

-### Comprehensive Checks
-
-#### `check-packages`
-Run all package-level quality checks (format, lint, pyright, mypy) in parallel across all packages. This runs the full cross-product of (package × check) concurrently:
-```bash
-uv run poe check-packages
-```
-
-#### `check`
-Run all quality checks including package checks, samples, tests and markdown lint:
-```bash
-uv run poe check
-```
-
-### Testing
-
-#### `test`
-Run unit tests with coverage by invoking the `test` task in each package in parallel:
-```bash
-uv run poe test
-```
-
-To run tests for a specific package only, use the `--directory` flag:
-```bash
-# Run tests for the core package
-uv run --directory packages/core poe test
-
-# Run tests for the azure-ai package
-uv run --directory packages/azure-ai poe test
-```
-
-#### `all-tests`
-Run all tests in a single pytest invocation across all packages in parallel (excluding lab and devui). This is faster than `test` as it uses pytest's parallel execution:
-```bash
-uv run poe all-tests
-```
-
-#### `all-tests-cov`
-Same as `all-tests` but with coverage reporting enabled:
-```bash
-uv run poe all-tests-cov
-```
-
 ### Building and Publishing

-#### `build`
-Build all packages:
-```bash
-uv run poe build
-```
-
-#### `clean-dist`
-Clean the dist directories:
-```bash
-uv run poe clean-dist
-```
-
 #### `publish`
 Publish packages to PyPI:
 ```bash
 uv run poe publish
 ```

+### Compatibility aliases
+
+These legacy commands still work during the transition, but prefer the newer forms above:
+
+```bash
+uv run poe fmt             # prefer: uv run poe syntax -F
+uv run poe format          # prefer: uv run poe syntax -F
+uv run poe lint            # prefer: uv run poe syntax -C
+uv run poe all-tests       # prefer: uv run poe test -A
+uv run poe all-tests-cov   # prefer: uv run poe test -A -C
+uv run poe samples-lint    # prefer: uv run poe syntax -S -C
+uv run poe samples-syntax  # prefer: uv run poe pyright -S
+```
+
 ## Prek Hooks

-Prek hooks run automatically on commit and execute a subset of the checks on changed files only. Package-level checks (fmt, lint, pyright) run in parallel but only for packages with changed files. Markdown and sample checks are skipped when no relevant files were changed. If the `core` package is changed, all packages are checked. You can also run all checks using prek directly:
+Prek hooks run automatically on commit and stay intentionally lightweight:
+
+- changed-package syntax formatting
+- changed-package syntax lint/check
+- markdown code lint only when markdown files change
+- sample lint + sample pyright only when files under `samples/` change
+
+They do **not** run workspace `pyright` or `mypy` by default. Use `uv run poe pyright`, `uv run poe mypy`, `uv run poe typing`, `uv run poe check-packages`, or `uv run poe check` when you want deeper validation.
+
+You can run the installed hooks directly with:

 ```bash
 uv run prek run -a
@@ -35,10 +35,12 @@ from agent_framework import (
    AgentResponseUpdate,
    AgentSession,
    BaseAgent,
+    BaseHistoryProvider,
    Content,
    ContinuationToken,
    Message,
    ResponseStream,
+    SessionContext,
    normalize_messages,
    prepend_agent_framework_to_user_agent,
 )
@@ -284,17 +286,36 @@ class A2AAgent(AgentTelemetryLayer, BaseAgent):
            When stream=True: A ResponseStream of AgentResponseUpdate items.
        """
        del function_invocation_kwargs, client_kwargs, kwargs
+        normalized_messages = normalize_messages(messages)
+
        if continuation_token is not None:
            a2a_stream: AsyncIterable[A2AStreamItem] = self.client.resubscribe(
                TaskIdParams(id=continuation_token["task_id"])
            )
        else:
-            normalized_messages = normalize_messages(messages)
+            if not normalized_messages:
+                raise ValueError("At least one message is required when starting a new task (no continuation_token).")
            a2a_message = self._prepare_message_for_a2a(normalized_messages[-1])
            a2a_stream = self.client.send_message(a2a_message)

+        provider_session = session
+        if provider_session is None and self.context_providers:
+            provider_session = AgentSession()
+
+        session_context = SessionContext(
+            session_id=provider_session.session_id if provider_session else None,
+            service_session_id=provider_session.service_session_id if provider_session else None,
+            input_messages=normalized_messages or [],
+            options={},
+        )
+
        response = ResponseStream(
-            self._map_a2a_stream(a2a_stream, background=background),
+            self._map_a2a_stream(
+                a2a_stream,
+                background=background,
+                session=provider_session,
+                session_context=session_context,
+            ),
            finalizer=AgentResponse.from_updates,
        )
        if stream:
@@ -306,6 +327,8 @@ class A2AAgent(AgentTelemetryLayer, BaseAgent):
        a2a_stream: AsyncIterable[A2AStreamItem],
        *,
        background: bool = False,
+        session: AgentSession | None = None,
+        session_context: SessionContext | None = None,
    ) -> AsyncIterable[AgentResponseUpdate]:
        """Map raw A2A protocol items to AgentResponseUpdates.

@@ -316,24 +339,52 @@ class A2AAgent(AgentTelemetryLayer, BaseAgent):
            background: When False, in-progress task updates are silently
                consumed (the stream keeps iterating until a terminal state).
                When True, they are yielded with a continuation token.
+            session: The agent session for context providers.
+            session_context: The session context for context providers.
        """
+        if session_context is None:
+            session_context = SessionContext(input_messages=[], options={})
+
+        # Run before_run providers (forward order)
+        for provider in self.context_providers:
+            if isinstance(provider, BaseHistoryProvider) and not provider.load_messages:
+                continue
+            if session is None:
+                raise RuntimeError("Provider session must be available when context providers are configured.")
+            await provider.before_run(
+                agent=self,  # type: ignore[arg-type]
+                session=session,
+                context=session_context,
+                state=session.state.setdefault(provider.source_id, {}),
+            )
+
+        all_updates: list[AgentResponseUpdate] = []
        async for item in a2a_stream:
            if isinstance(item, A2AMessage):
                # Process A2A Message
                contents = self._parse_contents_from_a2a(item.parts)
-                yield AgentResponseUpdate(
+                update = AgentResponseUpdate(
                    contents=contents,
                    role="assistant" if item.role == A2ARole.agent else "user",
                    response_id=str(getattr(item, "message_id", uuid.uuid4())),
                    raw_representation=item,
                )
+                all_updates.append(update)
+                yield update
            elif isinstance(item, tuple) and len(item) == 2 and isinstance(item[0], Task):
                task, _update_event = item
                for update in self._updates_from_task(task, background=background):
+                    all_updates.append(update)
                    yield update
            else:
                raise NotImplementedError("Only Message and Task responses are supported")

+        # Set the response on the context for after_run providers
+        if all_updates:
+            session_context._response = AgentResponse.from_updates(all_updates)  # type: ignore[assignment]
+
+        await self._run_after_providers(session=session, context=session_context)
+
    # ------------------------------------------------------------------
    # Task helpers
    # ------------------------------------------------------------------
@@ -85,9 +85,13 @@ exclude_dirs = ["tests"]
 executor.type = "uv"
 include = "../../shared_tasks.toml"

-[tool.poe.tasks]
-mypy = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_a2a"
-test = 'pytest -m "not integration" --cov=agent_framework_a2a --cov-report=term-missing:skip-covered tests'
+[tool.poe.tasks.mypy]
+help = "Run MyPy for this package."
+cmd = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_a2a"
+
+[tool.poe.tasks.test]
+help = "Run the default unit test suite for this package."
+cmd = 'pytest -m "not integration" --cov=agent_framework_a2a --cov-report=term-missing:skip-covered tests'

 [build-system]
 requires = ["flit-core >= 3.11,<4.0"]
@@ -23,11 +23,14 @@ from a2a.types import Role as A2ARole
 from agent_framework import (
    AgentResponse,
    AgentResponseUpdate,
+    AgentSession,
+    BaseContextProvider,
    Content,
    Message,
+    SessionContext,
 )
 from agent_framework.a2a import A2AAgent
-from pytest import fixture, raises
+from pytest import fixture, mark, raises

 from agent_framework_a2a import A2AContinuationToken
 from agent_framework_a2a._agent import _get_uri_data  # type: ignore
@@ -851,3 +854,188 @@ async def test_poll_task_completed(a2a_agent: A2AAgent, mock_a2a_client: MockA2A


 # endregion
+
+
+# region Context Provider Tests
+
+
+class TrackingContextProvider(BaseContextProvider):
+    """A context provider that records when before_run and after_run are called."""
+
+    def __init__(self) -> None:
+        super().__init__(source_id="tracking-provider")
+        self.before_run_called = False
+        self.after_run_called = False
+        self.before_run_context: SessionContext | None = None
+        self.after_run_context: SessionContext | None = None
+
+    async def before_run(
+        self,
+        *,
+        agent: Any,
+        session: AgentSession,
+        context: SessionContext,
+        state: dict[str, Any],
+    ) -> None:
+        self.before_run_called = True
+        self.before_run_context = context
+
+    async def after_run(
+        self,
+        *,
+        agent: Any,
+        session: AgentSession,
+        context: SessionContext,
+        state: dict[str, Any],
+    ) -> None:
+        self.after_run_called = True
+        self.after_run_context = context
+
+
+async def test_run_invokes_context_providers(mock_a2a_client: MockA2AClient) -> None:
+    """Test that context providers are invoked during non-streaming run."""
+    provider = TrackingContextProvider()
+    agent = A2AAgent(
+        name="Test Agent",
+        client=mock_a2a_client,
+        context_providers=[provider],
+        http_client=None,
+    )
+    mock_a2a_client.add_message_response("msg-1", "Hello from A2A")
+    session = agent.create_session()
+
+    response = await agent.run("Hello", session=session)
+
+    assert provider.before_run_called
+    assert provider.after_run_called
+    assert response.text == "Hello from A2A"
+
+
+async def test_run_streaming_invokes_context_providers(mock_a2a_client: MockA2AClient) -> None:
+    """Test that context providers are invoked during streaming run."""
+    provider = TrackingContextProvider()
+    agent = A2AAgent(
+        name="Test Agent",
+        client=mock_a2a_client,
+        context_providers=[provider],
+        http_client=None,
+    )
+    mock_a2a_client.add_message_response("msg-1", "Streamed response")
+    session = agent.create_session()
+
+    stream = agent.run("Hello", stream=True, session=session)
+    updates = []
+    async for update in stream:
+        updates.append(update)
+
+    assert provider.before_run_called
+    assert provider.after_run_called
+    assert len(updates) == 1
+    assert updates[0].text == "Streamed response"
+
+
+async def test_context_providers_receive_response(mock_a2a_client: MockA2AClient) -> None:
+    """Test that after_run providers can access the response via session context."""
+    provider = TrackingContextProvider()
+    agent = A2AAgent(
+        name="Test Agent",
+        client=mock_a2a_client,
+        context_providers=[provider],
+        http_client=None,
+    )
+    mock_a2a_client.add_message_response("msg-1", "Response text")
+    session = agent.create_session()
+
+    await agent.run("Hello", session=session)
+
+    assert provider.after_run_context is not None
+    assert provider.after_run_context.response is not None
+    assert provider.after_run_context.response.text == "Response text"
+
+
+async def test_context_providers_receive_input_messages(mock_a2a_client: MockA2AClient) -> None:
+    """Test that before_run providers can access input messages via session context."""
+    provider = TrackingContextProvider()
+    agent = A2AAgent(
+        name="Test Agent",
+        client=mock_a2a_client,
+        context_providers=[provider],
+        http_client=None,
+    )
+    mock_a2a_client.add_message_response("msg-1", "Reply")
+    session = agent.create_session()
+
+    await agent.run("Hello world", session=session)
+
+    assert provider.before_run_context is not None
+    assert len(provider.before_run_context.input_messages) > 0
+    assert provider.before_run_context.input_messages[-1].text == "Hello world"
+
+
+async def test_run_without_context_providers(mock_a2a_client: MockA2AClient) -> None:
+    """Test that run works normally when no context providers are configured."""
+    agent = A2AAgent(
+        name="Test Agent",
+        client=mock_a2a_client,
+        http_client=None,
+    )
+    mock_a2a_client.add_message_response("msg-1", "Hello")
+
+    response = await agent.run("Hello")
+
+    assert response.text == "Hello"
+
+
+async def test_run_creates_session_for_providers_when_none_provided(mock_a2a_client: MockA2AClient) -> None:
+    """Test that a session is auto-created when context providers are configured but no session is passed."""
+    provider = TrackingContextProvider()
+    agent = A2AAgent(
+        name="Test Agent",
+        client=mock_a2a_client,
+        context_providers=[provider],
+        http_client=None,
+    )
+    mock_a2a_client.add_message_response("msg-1", "Hello")
+
+    await agent.run("Hello")
+
+    assert provider.before_run_called
+    assert provider.after_run_called
+
+
+@mark.parametrize("messages", [None, []])
+async def test_run_raises_when_no_messages_and_no_continuation_token(
+    mock_a2a_client: MockA2AClient, messages: list[str] | None
+) -> None:
+    """Test that run() raises ValueError when messages is None/empty and no continuation_token is provided."""
+    agent = A2AAgent(
+        name="Test Agent",
+        client=mock_a2a_client,
+        http_client=None,
+    )
+
+    with raises(ValueError, match="At least one message is required"):
+        await agent.run(messages)
+
+
+async def test_run_with_continuation_token_does_not_require_messages(mock_a2a_client: MockA2AClient) -> None:
+    """Test that run() does not raise when messages is None but a continuation_token is provided."""
+    task = Task(
+        id="task-cont",
+        context_id="ctx-cont",
+        status=TaskStatus(state=TaskState.completed, message=None),
+    )
+    mock_a2a_client.resubscribe_responses.append((task, None))
+
+    agent = A2AAgent(
+        name="Test Agent",
+        client=mock_a2a_client,
+        http_client=None,
+    )
+
+    token = A2AContinuationToken(task_id="task-cont", context_id="ctx-cont")
+    response = await agent.run(None, continuation_token=token)
+    assert response is not None
+
+
+# endregion
@@ -72,6 +72,10 @@ typeCheckingMode = "basic"
 executor.type = "uv"
 include = "../../shared_tasks.toml"

-[tool.poe.tasks]
-mypy = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_ag_ui"
-test = 'pytest -m "not integration" --cov=agent_framework_ag_ui --cov-report=term-missing:skip-covered -n auto --dist worksteal tests/ag_ui'
+[tool.poe.tasks.mypy]
+help = "Run MyPy for this package."
+cmd = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_ag_ui"
+
+[tool.poe.tasks.test]
+help = "Run the default unit test suite for this package."
+cmd = 'pytest -m "not integration" --cov=agent_framework_ag_ui --cov-report=term-missing:skip-covered -n auto --dist worksteal tests/ag_ui'
@@ -85,9 +85,13 @@ exclude_dirs = ["tests"]
 executor.type = "uv"
 include = "../../shared_tasks.toml"

-[tool.poe.tasks]
-mypy = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_anthropic"
-test = 'pytest -m "not integration" --cov=agent_framework_anthropic --cov-report=term-missing:skip-covered -n auto --dist worksteal tests'
+[tool.poe.tasks.mypy]
+help = "Run MyPy for this package."
+cmd = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_anthropic"
+
+[tool.poe.tasks.test]
+help = "Run the default unit test suite for this package."
+cmd = 'pytest -m "not integration" --cov=agent_framework_anthropic --cov-report=term-missing:skip-covered -n auto --dist worksteal tests'

 [build-system]
 requires = ["flit-core >= 3.11,<4.0"]
@@ -87,9 +87,13 @@ exclude_dirs = ["tests"]
 executor.type = "uv"
 include = "../../shared_tasks.toml"

-[tool.poe.tasks]
-mypy = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_azure_ai_search"
-test = 'pytest -m "not integration" --cov=agent_framework_azure_ai_search --cov-report=term-missing:skip-covered tests'
+[tool.poe.tasks.mypy]
+help = "Run MyPy for this package."
+cmd = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_azure_ai_search"
+
+[tool.poe.tasks.test]
+help = "Run the default unit test suite for this package."
+cmd = 'pytest -m "not integration" --cov=agent_framework_azure_ai_search --cov-report=term-missing:skip-covered tests'

 [build-system]
 requires = ["flit-core >= 3.11,<4.0"]
@@ -11,6 +11,11 @@ from ._embedding_client import (
    AzureAIInferenceEmbeddingSettings,
    RawAzureAIInferenceEmbeddingClient,
 )
+from ._foundry_evals import (
+    FoundryEvals,
+    evaluate_foundry_target,
+    evaluate_traces,
+)
 from ._foundry_memory_provider import FoundryMemoryProvider
 from ._project_provider import AzureAIProjectAgentProvider
 from ._shared import AzureAISettings
@@ -31,8 +36,11 @@ __all__ = [
    "AzureAIProjectAgentOptions",
    "AzureAIProjectAgentProvider",
    "AzureAISettings",
+    "FoundryEvals",
    "FoundryMemoryProvider",
    "RawAzureAIClient",
    "RawAzureAIInferenceEmbeddingClient",
    "__version__",
+    "evaluate_foundry_target",
+    "evaluate_traces",
 ]
@@ -0,0 +1,838 @@
+# Copyright (c) Microsoft. All rights reserved.
+
+"""Microsoft Foundry Evals integration for Microsoft Agent Framework.
+
+Provides ``FoundryEvals``, an ``Evaluator`` implementation backed by Azure AI
+Foundry's built-in evaluators. See docs/decisions/0018-foundry-evals-integration.md
+for the design rationale.
+
+Typical usage::
+
+    from agent_framework import evaluate_agent
+    from agent_framework_azure_ai import FoundryEvals
+
+    evals = FoundryEvals(project_client=project_client, model_deployment="gpt-4o")
+    results = await evaluate_agent(
+        agent=my_agent,
+        queries=["What's the weather in Seattle?"],
+        evaluators=evals,
+    )
+    assert results.all_passed
+    print(results.report_url)
+"""
+
+from __future__ import annotations
+
+import asyncio
+import logging
+from typing import TYPE_CHECKING, Any, Sequence, cast
+
+from agent_framework._evaluation import (
+    ConversationSplit,
+    ConversationSplitter,
+    EvalItem,
+    EvalItemResult,
+    EvalResults,
+    EvalScoreResult,
+)
+
+if TYPE_CHECKING:
+    from azure.ai.projects.aio import AIProjectClient
+    from openai import AsyncOpenAI
+
+logger = logging.getLogger(__name__)
+
+# Agent evaluators that accept query/response as conversation arrays.
+# Maintained manually — check https://learn.microsoft.com/en-us/azure/ai-studio/how-to/develop/evaluate-sdk
+# for the latest evaluator list. These are the evaluators that need conversation-format input.
+_AGENT_EVALUATORS: set[str] = {
+    "builtin.intent_resolution",
+    "builtin.task_adherence",
+    "builtin.task_completion",
+    "builtin.task_navigation_efficiency",
+    "builtin.tool_call_accuracy",
+    "builtin.tool_selection",
+    "builtin.tool_input_accuracy",
+    "builtin.tool_output_utilization",
+    "builtin.tool_call_success",
+}
+
+# Evaluators that additionally require tool_definitions.
+_TOOL_EVALUATORS: set[str] = {
+    "builtin.tool_call_accuracy",
+    "builtin.tool_selection",
+    "builtin.tool_input_accuracy",
+    "builtin.tool_output_utilization",
+    "builtin.tool_call_success",
+}
+
+_BUILTIN_EVALUATORS: dict[str, str] = {
+    # Agent behavior
+    "intent_resolution": "builtin.intent_resolution",
+    "task_adherence": "builtin.task_adherence",
+    "task_completion": "builtin.task_completion",
+    "task_navigation_efficiency": "builtin.task_navigation_efficiency",
+    # Tool usage
+    "tool_call_accuracy": "builtin.tool_call_accuracy",
+    "tool_selection": "builtin.tool_selection",
+    "tool_input_accuracy": "builtin.tool_input_accuracy",
+    "tool_output_utilization": "builtin.tool_output_utilization",
+    "tool_call_success": "builtin.tool_call_success",
+    # Quality
+    "coherence": "builtin.coherence",
+    "fluency": "builtin.fluency",
+    "relevance": "builtin.relevance",
+    "groundedness": "builtin.groundedness",
+    "response_completeness": "builtin.response_completeness",
+    "similarity": "builtin.similarity",
+    # Safety
+    "violence": "builtin.violence",
+    "sexual": "builtin.sexual",
+    "self_harm": "builtin.self_harm",
+    "hate_unfairness": "builtin.hate_unfairness",
+}
+
+# Default evaluator sets used when evaluators=None
+_DEFAULT_EVALUATORS: list[str] = [
+    "relevance",
+    "coherence",
+    "task_adherence",
+]
+
+_DEFAULT_TOOL_EVALUATORS: list[str] = [
+    "tool_call_accuracy",
+]
+
+
+def _resolve_evaluator(name: str) -> str:
+    """Resolve a short evaluator name to its fully-qualified ``builtin.*`` form.
+
+    Args:
+        name: Short name (e.g. ``"relevance"``) or fully-qualified name
+            (e.g. ``"builtin.relevance"``).
+
+    Returns:
+        The fully-qualified evaluator name.
+
+    Raises:
+        ValueError: If the name is not recognized.
+    """
+    if name.startswith("builtin."):
+        return name
+    resolved = _BUILTIN_EVALUATORS.get(name)
+    if resolved is None:
+        raise ValueError(f"Unknown evaluator '{name}'. Available: {sorted(_BUILTIN_EVALUATORS)}")
+    return resolved
+
+
+# ---------------------------------------------------------------------------
+# Internal helpers
+# ---------------------------------------------------------------------------
+
+
+def _build_testing_criteria(
+    evaluators: Sequence[str],
+    model_deployment: str,
+    *,
+    include_data_mapping: bool = False,
+) -> list[dict[str, Any]]:
+    """Build ``testing_criteria`` for ``evals.create()``.
+
+    Args:
+        evaluators: Evaluator names.
+        model_deployment: Model deployment for the LLM judge.
+        include_data_mapping: Whether to include field-level data mapping
+            (required for the JSONL data source, not needed for response-based).
+    """
+    criteria: list[dict[str, Any]] = []
+    for name in evaluators:
+        qualified = _resolve_evaluator(name)
+        short = name if not name.startswith("builtin.") else name.split(".")[-1]
+
+        entry: dict[str, Any] = {
+            "type": "azure_ai_evaluator",
+            "name": short,
+            "evaluator_name": qualified,
+            "initialization_parameters": {"deployment_name": model_deployment},
+        }
+
+        if include_data_mapping:
+            if qualified in _AGENT_EVALUATORS:
+                # Agent evaluators: query/response as conversation arrays
+                mapping: dict[str, str] = {
+                    "query": "{{item.query_messages}}",
+                    "response": "{{item.response_messages}}",
+                }
+            else:
+                # Quality evaluators: query/response as strings
+                mapping = {
+                    "query": "{{item.query}}",
+                    "response": "{{item.response}}",
+                }
+            if qualified == "builtin.groundedness":
+                mapping["context"] = "{{item.context}}"
+            if qualified in _TOOL_EVALUATORS:
+                mapping["tool_definitions"] = "{{item.tool_definitions}}"
+            entry["data_mapping"] = mapping
+
+        criteria.append(entry)
+    return criteria
+
+
+def _build_item_schema(*, has_context: bool = False, has_tools: bool = False) -> dict[str, Any]:
+    """Build the ``item_schema`` for custom JSONL eval definitions."""
+    properties: dict[str, Any] = {
+        "query": {"type": "string"},
+        "response": {"type": "string"},
+        "query_messages": {"type": "array"},
+        "response_messages": {"type": "array"},
+    }
+    if has_context:
+        properties["context"] = {"type": "string"}
+    if has_tools:
+        properties["tool_definitions"] = {"type": "array"}
+    return {
+        "type": "object",
+        "properties": properties,
+        "required": ["query", "response"],
+    }
+
+
+def _resolve_default_evaluators(
+    evaluators: Sequence[str] | None,
+    items: Sequence[EvalItem | dict[str, Any]] | None = None,
+) -> list[str]:
+    """Resolve evaluators, applying defaults when ``None``.
+
+    Defaults to relevance + coherence + task_adherence. Automatically adds
+    tool_call_accuracy when items contain tools.
+    """
+    if evaluators is not None:
+        return list(evaluators)
+
+    result = list(_DEFAULT_EVALUATORS)
+    if items is not None:
+        has_tools = any((item.tools if isinstance(item, EvalItem) else item.get("tool_definitions")) for item in items)
+        if has_tools:
+            result.extend(_DEFAULT_TOOL_EVALUATORS)
+    return result
+
+
+def _filter_tool_evaluators(
+    evaluators: list[str],
+    items: Sequence[EvalItem | dict[str, Any]],
+) -> list[str]:
+    """Remove tool evaluators if no items have tool definitions."""
+    has_tools = any((item.tools if isinstance(item, EvalItem) else item.get("tool_definitions")) for item in items)
+    if has_tools:
+        return evaluators
+    filtered = [e for e in evaluators if _resolve_evaluator(e) not in _TOOL_EVALUATORS]
+    return filtered if filtered else list(_DEFAULT_EVALUATORS)
+
+
+async def _ensure_async_result(func: Any, *args: Any, **kwargs: Any) -> Any:
+    """Invoke a sync or async client method transparently.
+
+    If ``func`` returns a coroutine (async client), awaits it directly.
+    Otherwise returns the already-resolved result.
+    """
+    import inspect
+
+    result = func(*args, **kwargs)
+    if inspect.isawaitable(result):
+        return await result
+    return result
+
+
+async def _poll_eval_run(
+    client: AsyncOpenAI,
+    eval_id: str,
+    run_id: str,
+    poll_interval: float = 5.0,
+    timeout: float = 600.0,
+    provider: str = "Microsoft Foundry",
+    *,
+    fetch_output_items: bool = True,
+) -> EvalResults:
+    """Poll an eval run until completion or timeout."""
+    loop = asyncio.get_event_loop()
+    deadline = loop.time() + timeout
+    while True:
+        run = await _ensure_async_result(client.evals.runs.retrieve, run_id=run_id, eval_id=eval_id)
+        if run.status in ("completed", "failed", "canceled"):
+            error_msg = None
+            if run.status == "failed":
+                error_msg = (
+                    getattr(run, "error", None)
+                    or getattr(run, "error_message", None)
+                    or getattr(run, "failure_reason", None)
+                )
+                if error_msg and not isinstance(error_msg, str):
+                    error_msg = str(error_msg)
+
+            items: list[EvalItemResult] = []
+            if fetch_output_items and run.status == "completed":
+                items = await _fetch_output_items(client, eval_id, run_id)
+
+            return EvalResults(
+                provider=provider,
+                eval_id=eval_id,
+                run_id=run_id,
+                status=run.status,
+                result_counts=_extract_result_counts(run),
+                report_url=getattr(run, "report_url", None),
+                error=error_msg,
+                per_evaluator=_extract_per_evaluator(run),
+                items=items,
+            )
+        remaining = deadline - loop.time()
+        if remaining <= 0:
+            return EvalResults(provider=provider, eval_id=eval_id, run_id=run_id, status="timeout")
+        logger.debug("Eval run %s status: %s (%.0fs remaining)", run_id, run.status, remaining)
+        await asyncio.sleep(min(poll_interval, remaining))
+
+
+def _extract_result_counts(run: Any) -> dict[str, int] | None:
+    """Safely extract result_counts from an eval run object."""
+    counts = getattr(run, "result_counts", None)
+    if counts is None:
+        return None
+    if isinstance(counts, dict):
+        return cast(dict[str, int], counts)
+    try:
+        attrs = cast(dict[str, Any], vars(counts))
+        return {str(k): v for k, v in attrs.items() if isinstance(v, int)}
+    except TypeError:
+        return None
+
+
+def _extract_per_evaluator(run: Any) -> dict[str, dict[str, int]]:
+    """Safely extract per-evaluator result breakdowns from an eval run."""
+    per_eval: dict[str, dict[str, int]] = {}
+    per_testing_criteria = getattr(run, "per_testing_criteria_results", None)
+    if per_testing_criteria is None:
+        return per_eval
+    try:
+        items = cast(list[Any], per_testing_criteria) if isinstance(per_testing_criteria, list) else []  # type: ignore[redundant-cast]
+        for item in items:
+            name: str = str(getattr(item, "name", None) or getattr(item, "testing_criteria", "unknown"))
+            counts = _extract_result_counts(item)
+            if name and counts:
+                per_eval[name] = counts
+    except (TypeError, AttributeError):
+        pass
+    return per_eval
+
+
+async def _fetch_output_items(
+    client: AsyncOpenAI,
+    eval_id: str,
+    run_id: str,
+) -> list[EvalItemResult]:
+    """Fetch per-item results from the output_items API.
+
+    Converts the provider-specific ``OutputItemListResponse`` objects into
+    provider-agnostic ``EvalItemResult`` instances with per-evaluator scores,
+    error categorization, and token usage.
+    """
+    items: list[EvalItemResult] = []
+    try:
+        output_items_page = await _ensure_async_result(
+            client.evals.runs.output_items.list,
+            run_id=run_id,
+            eval_id=eval_id,
+        )
+
+        for oi in output_items_page:
+            item_id = getattr(oi, "id", "") or ""
+            status = getattr(oi, "status", "unknown") or "unknown"
+
+            # Extract per-evaluator scores
+            scores: list[EvalScoreResult] = []
+            for r in getattr(oi, "results", []) or []:
+                scores.append(
+                    EvalScoreResult(
+                        name=getattr(r, "name", "unknown"),
+                        score=getattr(r, "score", 0.0),
+                        passed=getattr(r, "passed", None),
+                        sample=getattr(r, "sample", None),
+                    )
+                )
+
+            # Extract error info from sample
+            error_code: str | None = None
+            error_message: str | None = None
+            token_usage: dict[str, int] | None = None
+            input_text: str | None = None
+            output_text: str | None = None
+            response_id: str | None = None
+
+            sample = getattr(oi, "sample", None)
+            if sample is not None:
+                error = getattr(sample, "error", None)
+                if error is not None:
+                    code = getattr(error, "code", None)
+                    msg = getattr(error, "message", None)
+                    if code or msg:
+                        error_code = code or None
+                        error_message = msg or None
+
+                usage = getattr(sample, "usage", None)
+                if usage is not None:
+                    total = getattr(usage, "total_tokens", 0)
+                    if total:
+                        token_usage = {
+                            "prompt_tokens": getattr(usage, "prompt_tokens", 0),
+                            "completion_tokens": getattr(usage, "completion_tokens", 0),
+                            "total_tokens": total,
+                            "cached_tokens": getattr(usage, "cached_tokens", 0),
+                        }
+
+                # Extract input/output text
+                sample_input = getattr(sample, "input", None)
+                if sample_input:
+                    parts = [getattr(si, "content", "") for si in sample_input if getattr(si, "role", "") == "user"]
+                    if parts:
+                        input_text = " ".join(parts)
+
+                sample_output = getattr(sample, "output", None)
+                if sample_output:
+                    parts = [
+                        getattr(so, "content", "") or ""
+                        for so in sample_output
+                        if getattr(so, "role", "") == "assistant"
+                    ]
+                    if parts:
+                        output_text = " ".join(parts)
+
+            # Extract response_id from datasource_item
+            ds_item = getattr(oi, "datasource_item", None)
+            if ds_item and isinstance(ds_item, dict):
+                ds_dict = cast(dict[str, Any], ds_item)
+                resp_id_val = ds_dict.get("resp_id") or ds_dict.get("response_id")
+                response_id = str(resp_id_val) if resp_id_val else None
+
+            items.append(
+                EvalItemResult(
+                    item_id=item_id,
+                    status=status,
+                    scores=scores,
+                    error_code=error_code,
+                    error_message=error_message,
+                    response_id=response_id,
+                    input_text=input_text,
+                    output_text=output_text,
+                    token_usage=token_usage,
+                )
+            )
+    except Exception:
+        logger.debug("Could not fetch output_items for run %s", run_id, exc_info=True)
+
+    return items
+
+
+def _resolve_openai_client(
+    openai_client: AsyncOpenAI | None = None,
+    project_client: AIProjectClient | None = None,
+) -> AsyncOpenAI:
+    """Resolve an OpenAI client from explicit client or project_client."""
+    if openai_client is not None:
+        return openai_client
+    if project_client is not None:
+        return project_client.get_openai_client()
+    raise ValueError("Provide either 'openai_client' or 'project_client'.")
+
+
+# ---------------------------------------------------------------------------
+# FoundryEvals — Evaluator implementation for Microsoft Foundry
+# ---------------------------------------------------------------------------
+
+
+class FoundryEvals:
+    """Evaluation provider backed by Microsoft Foundry.
+
+    Implements the ``Evaluator`` protocol so it can be passed to the
+    provider-agnostic ``evaluate_agent()`` and
+    ``evaluate_workflow()`` functions from ``agent_framework``.
+
+    Also provides constants for built-in evaluator names for IDE
+    autocomplete and typo prevention::
+
+        from agent_framework_azure_ai import FoundryEvals
+
+        evaluators = [FoundryEvals.RELEVANCE, FoundryEvals.TOOL_CALL_ACCURACY]
+
+    The simplest usage::
+
+        from agent_framework import evaluate_agent
+        from agent_framework_azure_ai import FoundryEvals
+
+        evals = FoundryEvals(project_client=client, model_deployment="gpt-4o")
+        results = await evaluate_agent(agent=agent, queries=queries, evaluators=evals)
+
+    **Evaluator selection:**
+
+    By default, runs ``relevance``, ``coherence``, and ``task_adherence``.
+    Automatically adds ``tool_call_accuracy`` when items contain tool
+    definitions. Override with ``evaluators=``.
+
+    **Responses API optimization:**
+
+    When all items have a ``response_id`` and no tool evaluators are needed,
+    uses Foundry's server-side response retrieval path (no data upload).
+
+    Args:
+        project_client: An ``AIProjectClient`` instance (sync or async).
+            Provide this or *openai_client*.
+        openai_client: An ``AsyncOpenAI`` client with evals API.
+        model_deployment: Model deployment name for the evaluator LLM judge.
+        evaluators: Evaluator names (e.g. ``["relevance", "tool_call_accuracy"]``).
+            When ``None`` (default), uses smart defaults based on item data.
+        conversation_split: How to split multi-turn conversations into
+            query/response halves.  Defaults to ``LAST_TURN``.  Pass a
+            ``ConversationSplit`` enum value or a custom callable — see
+            ``ConversationSplitter``.
+        poll_interval: Seconds between status polls (default 5.0).
+        timeout: Maximum seconds to wait for completion (default 600.0).
+    """
+
+    # ---------------------------------------------------------------------------
+    # Built-in evaluator name constants
+    # ---------------------------------------------------------------------------
+
+    # Agent behavior
+    INTENT_RESOLUTION: str = "intent_resolution"
+    TASK_ADHERENCE: str = "task_adherence"
+    TASK_COMPLETION: str = "task_completion"
+    TASK_NAVIGATION_EFFICIENCY: str = "task_navigation_efficiency"
+
+    # Tool usage
+    TOOL_CALL_ACCURACY: str = "tool_call_accuracy"
+    TOOL_SELECTION: str = "tool_selection"
+    TOOL_INPUT_ACCURACY: str = "tool_input_accuracy"
+    TOOL_OUTPUT_UTILIZATION: str = "tool_output_utilization"
+    TOOL_CALL_SUCCESS: str = "tool_call_success"
+
+    # Quality
+    COHERENCE: str = "coherence"
+    FLUENCY: str = "fluency"
+    RELEVANCE: str = "relevance"
+    GROUNDEDNESS: str = "groundedness"
+    RESPONSE_COMPLETENESS: str = "response_completeness"
+    SIMILARITY: str = "similarity"
+
+    # Safety
+    VIOLENCE: str = "violence"
+    SEXUAL: str = "sexual"
+    SELF_HARM: str = "self_harm"
+    HATE_UNFAIRNESS: str = "hate_unfairness"
+
+    def __init__(
+        self,
+        *,
+        project_client: AIProjectClient | None = None,
+        openai_client: AsyncOpenAI | None = None,
+        model_deployment: str,
+        evaluators: Sequence[str] | None = None,
+        conversation_split: ConversationSplitter = ConversationSplit.LAST_TURN,
+        poll_interval: float = 5.0,
+        timeout: float = 600.0,
+    ):
+        self.name = "Microsoft Foundry"
+        self._client = _resolve_openai_client(openai_client, project_client)
+        self._model_deployment = model_deployment
+        self._evaluators = list(evaluators) if evaluators is not None else None
+        self._conversation_split = conversation_split
+        self._poll_interval = poll_interval
+        self._timeout = timeout
+
+    async def evaluate(
+        self,
+        items: Sequence[EvalItem],
+        *,
+        eval_name: str = "Agent Framework Eval",
+    ) -> EvalResults:
+        """Evaluate items using Foundry evaluators.
+
+        Implements the ``Evaluator`` protocol. Automatically selects the
+        optimal data path (Responses API vs JSONL dataset) and filters
+        tool evaluators for items without tool definitions.
+
+        Args:
+            items: Eval data items from ``AgentEvalConverter.to_eval_item()``.
+            eval_name: Display name for the evaluation run.
+
+        Returns:
+            ``EvalResults`` with status, counts, and portal link.
+        """
+        # Resolve evaluators with auto-detection
+        resolved = _resolve_default_evaluators(self._evaluators, items=items)
+        # Filter tool evaluators if items don't have tools
+        resolved = _filter_tool_evaluators(resolved, items)
+
+        # Standard JSONL dataset path
+        return await self._evaluate_via_dataset(items, resolved, eval_name)
+
+    # -- Internal evaluation paths --
+
+    async def _evaluate_via_responses(
+        self,
+        response_ids: Sequence[str],
+        evaluators: list[str],
+        eval_name: str,
+    ) -> EvalResults:
+        """Evaluate using Foundry's Responses API retrieval path."""
+        eval_obj = await _ensure_async_result(
+            self._client.evals.create,
+            name=eval_name,
+            data_source_config={"type": "azure_ai_source", "scenario": "responses"},
+            testing_criteria=_build_testing_criteria(evaluators, self._model_deployment),
+        )
+
+        data_source = {
+            "type": "azure_ai_responses",
+            "item_generation_params": {
+                "type": "response_retrieval",
+                "data_mapping": {"response_id": "{{item.resp_id}}"},
+                "source": {
+                    "type": "file_content",
+                    "content": [{"item": {"resp_id": rid}} for rid in response_ids],
+                },
+            },
+        }
+
+        run = await _ensure_async_result(
+            self._client.evals.runs.create,
+            eval_id=eval_obj.id,
+            name=f"{eval_name} Run",
+            data_source=data_source,
+        )
+
+        return await _poll_eval_run(
+            self._client,
+            eval_obj.id,
+            run.id,
+            self._poll_interval,
+            self._timeout,
+            provider=self.name,
+        )
+
+    async def _evaluate_via_dataset(
+        self,
+        items: Sequence[EvalItem],
+        evaluators: list[str],
+        eval_name: str,
+    ) -> EvalResults:
+        """Evaluate using JSONL dataset upload path."""
+        dicts = [item.to_eval_data(split=item.split_strategy or self._conversation_split) for item in items]
+        has_context = any("context" in d for d in dicts)
+        has_tools = any("tool_definitions" in d for d in dicts)
+
+        eval_obj = await _ensure_async_result(
+            self._client.evals.create,
+            name=eval_name,
+            data_source_config={
+                "type": "custom",
+                "item_schema": _build_item_schema(has_context=has_context, has_tools=has_tools),
+                "include_sample_schema": True,
+            },
+            testing_criteria=_build_testing_criteria(
+                evaluators,
+                self._model_deployment,
+                include_data_mapping=True,
+            ),
+        )
+
+        data_source = {
+            "type": "jsonl",
+            "source": {
+                "type": "file_content",
+                "content": [{"item": d} for d in dicts],
+            },
+        }
+
+        run = await _ensure_async_result(
+            self._client.evals.runs.create,
+            eval_id=eval_obj.id,
+            name=f"{eval_name} Run",
+            data_source=data_source,
+        )
+
+        return await _poll_eval_run(
+            self._client,
+            eval_obj.id,
+            run.id,
+            self._poll_interval,
+            self._timeout,
+            provider=self.name,
+        )
+
+
+# ---------------------------------------------------------------------------
+# Foundry-specific functions (not part of the Evaluator protocol)
+# ---------------------------------------------------------------------------
+
+
+async def evaluate_traces(
+    *,
+    evaluators: Sequence[str] | None = None,
+    openai_client: AsyncOpenAI | None = None,
+    project_client: AIProjectClient | None = None,
+    model_deployment: str,
+    response_ids: Sequence[str] | None = None,
+    trace_ids: Sequence[str] | None = None,
+    agent_id: str | None = None,
+    lookback_hours: int = 24,
+    eval_name: str = "Agent Framework Trace Eval",
+    poll_interval: float = 5.0,
+    timeout: float = 600.0,
+) -> EvalResults:
+    """Evaluate agent behavior from OTel traces or response IDs.
+
+    Foundry-specific function — works with any agent that emits OTel traces
+    to App Insights. Provide *response_ids* for specific responses,
+    *trace_ids* for specific traces, or *agent_id* with *lookback_hours*
+    to evaluate recent activity.
+
+    Args:
+        evaluators: Evaluator names (e.g. ``[FoundryEvals.RELEVANCE]``).
+            Defaults to relevance, coherence, and task_adherence.
+        openai_client: ``AsyncOpenAI`` client. Provide this or *project_client*.
+        project_client: An ``AIProjectClient`` instance.
+        model_deployment: Model deployment name for the evaluator LLM judge.
+        response_ids: Evaluate specific Responses API responses.
+        trace_ids: Evaluate specific OTel trace IDs from App Insights.
+        agent_id: Filter traces by agent ID (used with *lookback_hours*).
+        lookback_hours: Hours of trace history to evaluate (default 24).
+        eval_name: Display name for the evaluation.
+        poll_interval: Seconds between status polls.
+        timeout: Maximum seconds to wait for completion.
+
+    Returns:
+        ``EvalResults`` with status, result counts, and portal link.
+
+    Example::
+
+        results = await evaluate_traces(
+            response_ids=[response.response_id],
+            evaluators=[FoundryEvals.RELEVANCE],
+            project_client=project_client,
+            model_deployment="gpt-4o",
+        )
+    """
+    client = _resolve_openai_client(openai_client, project_client)
+    resolved_evaluators = _resolve_default_evaluators(evaluators)
+
+    if response_ids:
+        foundry = FoundryEvals(
+            openai_client=client,
+            model_deployment=model_deployment,
+            evaluators=resolved_evaluators,
+            poll_interval=poll_interval,
+            timeout=timeout,
+        )
+        return await foundry._evaluate_via_responses(  # pyright: ignore[reportPrivateUsage]
+            response_ids,
+            resolved_evaluators,
+            eval_name,
+        )
+
+    if not trace_ids and not agent_id:
+        raise ValueError("Provide at least one of: response_ids, trace_ids, or agent_id")
+
+    trace_source: dict[str, Any] = {
+        "type": "azure_ai_traces",
+        "lookback_hours": lookback_hours,
+    }
+    if trace_ids:
+        trace_source["trace_ids"] = list(trace_ids)
+    if agent_id:
+        trace_source["agent_id"] = agent_id
+
+    eval_obj = await _ensure_async_result(
+        client.evals.create,
+        name=eval_name,
+        data_source_config={"type": "azure_ai_source", "scenario": "traces"},
+        testing_criteria=_build_testing_criteria(resolved_evaluators, model_deployment),
+    )
+
+    run = await _ensure_async_result(
+        client.evals.runs.create,
+        eval_id=eval_obj.id,
+        name=f"{eval_name} Run",
+        data_source=trace_source,
+    )
+
+    return await _poll_eval_run(client, eval_obj.id, run.id, poll_interval, timeout)
+
+
+async def evaluate_foundry_target(
+    *,
+    target: dict[str, Any],
+    test_queries: Sequence[str],
+    evaluators: Sequence[str] | None = None,
+    openai_client: AsyncOpenAI | None = None,
+    project_client: AIProjectClient | None = None,
+    model_deployment: str,
+    eval_name: str = "Agent Framework Target Eval",
+    poll_interval: float = 5.0,
+    timeout: float = 600.0,
+) -> EvalResults:
+    """Evaluate a Foundry-registered agent or model deployment.
+
+    Foundry invokes the target, captures the output, and evaluates it. Use
+    this for scheduled evals, red teaming, and CI/CD quality gates.
+
+    Args:
+        target: Target configuration dict.
+        test_queries: Queries for Foundry to send to the target.
+        evaluators: Evaluator names.
+        openai_client: ``AsyncOpenAI`` client. Provide this or *project_client*.
+        project_client: An ``AIProjectClient`` instance.
+        model_deployment: Model deployment name for the evaluator LLM judge.
+        eval_name: Display name for the evaluation.
+        poll_interval: Seconds between status polls.
+        timeout: Maximum seconds to wait for completion.
+
+    Returns:
+        ``EvalResults`` with status, result counts, and portal link.
+
+    Example::
+
+        results = await evaluate_foundry_target(
+            target={"type": "azure_ai_agent", "name": "my-agent"},
+            test_queries=["Book a flight to Paris"],
+            project_client=project_client,
+            model_deployment="gpt-4o",
+        )
+    """
+    client = _resolve_openai_client(openai_client, project_client)
+    resolved_evaluators = _resolve_default_evaluators(evaluators)
+
+    eval_obj = await _ensure_async_result(
+        client.evals.create,
+        name=eval_name,
+        data_source_config={
+            "type": "azure_ai_source",
+            "scenario": "target_completions",
+        },
+        testing_criteria=_build_testing_criteria(resolved_evaluators, model_deployment),
+    )
+
+    data_source: dict[str, Any] = {
+        "type": "azure_ai_target_completions",
+        "target": target,
+        "source": {
+            "type": "file_content",
+            "content": [{"item": {"query": q}} for q in test_queries],
+        },
+    }
+
+    run = await _ensure_async_result(
+        client.evals.runs.create,
+        eval_id=eval_obj.id,
+        name=f"{eval_name} Run",
+        data_source=data_source,
+    )
+
+    return await _poll_eval_run(client, eval_obj.id, run.id, poll_interval, timeout)
@@ -85,11 +85,16 @@ exclude_dirs = ["tests"]
 executor.type = "uv"
 include = "../../shared_tasks.toml"

-[tool.poe.tasks]
-mypy = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_azure_ai"
-test = 'pytest -m "not integration" --cov=agent_framework_azure_ai --cov-report=term-missing:skip-covered tests'
+[tool.poe.tasks.mypy]
+help = "Run MyPy for this package."
+cmd = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_azure_ai"
+
+[tool.poe.tasks.test]
+help = "Run the default unit test suite for this package."
+cmd = 'pytest -m "not integration" --cov=agent_framework_azure_ai --cov-report=term-missing:skip-covered tests'

 [tool.poe.tasks.integration-tests]
+help = "Run the package integration test suite."
 cmd = """
 pytest --import-mode=importlib
 -n logical --dist worksteal
@@ -84,10 +84,17 @@ exclude_dirs = ["tests"]
 [tool.poe]
 executor.type = "uv"
 include = "../../shared_tasks.toml"
-[tool.poe.tasks]
-mypy = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_azure_cosmos"
-test = "pytest -m \"not integration\" --cov=agent_framework_azure_cosmos --cov-report=term-missing:skip-covered tests"
-integration-tests = "pytest tests/test_cosmos_history_provider.py -m integration"
+[tool.poe.tasks.mypy]
+help = "Run MyPy for this package."
+cmd = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_azure_cosmos"
+
+[tool.poe.tasks.test]
+help = "Run the default unit test suite for this package."
+cmd = "pytest -m \"not integration\" --cov=agent_framework_azure_cosmos --cov-report=term-missing:skip-covered tests"
+
+[tool.poe.tasks.integration-tests]
+help = "Run the package integration test suite."
+cmd = "pytest tests/test_cosmos_history_provider.py -m integration"

 [build-system]
 requires = ["flit-core >= 3.11,<4.0"]
@@ -91,9 +91,13 @@ exclude_dirs = ["tests"]
 executor.type = "uv"
 include = "../../shared_tasks.toml"

-[tool.poe.tasks]
-mypy = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_azurefunctions"
-test = 'pytest -m "not integration" --cov=agent_framework_azurefunctions --cov-report=term-missing:skip-covered tests'
+[tool.poe.tasks.mypy]
+help = "Run MyPy for this package."
+cmd = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_azurefunctions"
+
+[tool.poe.tasks.test]
+help = "Run the default unit test suite for this package."
+cmd = 'pytest -m "not integration" --cov=agent_framework_azurefunctions --cov-report=term-missing:skip-covered tests'

 [build-system]
 requires = ["flit-core >= 3.11,<4.0"]
@@ -84,9 +84,13 @@ exclude_dirs = ["tests"]
 executor.type = "uv"
 include = "../../shared_tasks.toml"

-[tool.poe.tasks]
-mypy = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_bedrock"
-test = 'pytest -m "not integration" --cov=agent_framework_bedrock --cov-report=term-missing:skip-covered tests'
+[tool.poe.tasks.mypy]
+help = "Run MyPy for this package."
+cmd = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_bedrock"
+
+[tool.poe.tasks.test]
+help = "Run the default unit test suite for this package."
+cmd = 'pytest -m "not integration" --cov=agent_framework_bedrock --cov-report=term-missing:skip-covered tests'

 [build-system]
 requires = ["hatchling"]
@@ -86,9 +86,13 @@ exclude_dirs = ["tests"]
 executor.type = "uv"
 include = "../../shared_tasks.toml"

-[tool.poe.tasks]
-mypy = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_chatkit"
-test = 'pytest -m "not integration" --cov=agent_framework_chatkit --cov-report=term-missing:skip-covered tests'
+[tool.poe.tasks.mypy]
+help = "Run MyPy for this package."
+cmd = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_chatkit"
+
+[tool.poe.tasks.test]
+help = "Run the default unit test suite for this package."
+cmd = 'pytest -m "not integration" --cov=agent_framework_chatkit --cov-report=term-missing:skip-covered tests'

 [build-system]
 requires = ["flit-core >= 3.11,<4.0"]
@@ -86,9 +86,13 @@ exclude_dirs = ["tests"]
 executor.type = "uv"
 include = "../../shared_tasks.toml"

-[tool.poe.tasks]
-mypy = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_claude"
-test = 'pytest -m "not integration" --cov=agent_framework_claude --cov-report=term-missing:skip-covered tests'
+[tool.poe.tasks.mypy]
+help = "Run MyPy for this package."
+cmd = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_claude"
+
+[tool.poe.tasks.test]
+help = "Run the default unit test suite for this package."
+cmd = 'pytest -m "not integration" --cov=agent_framework_claude --cov-report=term-missing:skip-covered tests'

 [build-system]
 requires = ["flit-core >= 3.11,<4.0"]
@@ -85,9 +85,13 @@ exclude_dirs = ["tests"]
 executor.type = "uv"
 include = "../../shared_tasks.toml"

-[tool.poe.tasks]
-mypy = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_copilotstudio"
-test = 'pytest -m "not integration" --cov=agent_framework_copilotstudio --cov-report=term-missing:skip-covered tests'
+[tool.poe.tasks.mypy]
+help = "Run MyPy for this package."
+cmd = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_copilotstudio"
+
+[tool.poe.tasks.test]
+help = "Run the default unit test suite for this package."
+cmd = 'pytest -m "not integration" --cov=agent_framework_copilotstudio --cov-report=term-missing:skip-covered tests'

 [build-system]
 requires = ["flit-core >= 3.11,<4.0"]
@@ -57,6 +57,27 @@ from ._compaction import (
    included_messages,
    included_token_count,
 )
+from ._evaluation import (
+    AgentEvalConverter,
+    CheckResult,
+    ConversationSplit,
+    ConversationSplitter,
+    EvalItem,
+    EvalItemResult,
+    EvalResults,
+    EvalScoreResult,
+    Evaluator,
+    ExpectedToolCall,
+    LocalEvaluator,
+    evaluate_agent,
+    evaluate_response,
+    evaluate_workflow,
+    evaluator,
+    keyword_check,
+    tool_call_args_match,
+    tool_called_check,
+    tool_calls_present,
+)
 from ._mcp import MCPStdioTool, MCPStreamableHTTPTool, MCPWebsocketTool
 from ._middleware import (
    AgentContext,
@@ -242,6 +263,7 @@ __all__ = [
    "USER_AGENT_TELEMETRY_DISABLED_ENV_VAR",
    "Agent",
    "AgentContext",
+    "AgentEvalConverter",
    "AgentExecutor",
    "AgentExecutorRequest",
    "AgentExecutorResponse",
@@ -268,11 +290,14 @@ __all__ = [
    "ChatOptions",
    "ChatResponse",
    "ChatResponseUpdate",
+    "CheckResult",
    "CheckpointStorage",
    "CompactionProvider",
    "CompactionStrategy",
    "Content",
    "ContinuationToken",
+    "ConversationSplit",
+    "ConversationSplitter",
    "Default",
    "Edge",
    "EdgeCondition",
@@ -281,7 +306,13 @@ __all__ = [
    "EmbeddingGenerationOptions",
    "EmbeddingInputT",
    "EmbeddingT",
+    "EvalItem",
+    "EvalItemResult",
+    "EvalResults",
+    "EvalScoreResult",
+    "Evaluator",
    "Executor",
+    "ExpectedToolCall",
    "FanInEdgeGroup",
    "FanOutEdgeGroup",
    "FileCheckpointStorage",
@@ -300,6 +331,7 @@ __all__ = [
    "InMemoryCheckpointStorage",
    "InMemoryHistoryProvider",
    "InProcRunnerContext",
+    "LocalEvaluator",
    "MCPStdioTool",
    "MCPStreamableHTTPTool",
    "MCPWebsocketTool",
@@ -379,11 +411,16 @@ __all__ = [
    "chat_middleware",
    "create_edge_runner",
    "detect_media_type_from_base64",
+    "evaluate_agent",
+    "evaluate_response",
+    "evaluate_workflow",
+    "evaluator",
    "executor",
    "function_middleware",
    "handler",
    "included_messages",
    "included_token_count",
+    "keyword_check",
    "load_settings",
    "map_chat_to_agent_update",
    "merge_chat_options",
@@ -396,6 +433,9 @@ __all__ = [
    "resolve_agent_id",
    "response_handler",
    "tool",
+    "tool_call_args_match",
+    "tool_called_check",
+    "tool_calls_present",
    "validate_chat_options",
    "validate_tool_mode",
    "validate_tools",
@@ -639,7 +639,7 @@ class RawAgent(BaseAgent, Generic[OptionsCoT]):  # type: ignore[misc]
                client=client,
                name="reasoning-agent",
                instructions="You are a reasoning assistant.",
-                options={
+                default_options={
                    "temperature": 0.7,
                    "max_tokens": 500,
                    "reasoning_effort": "high",  # OpenAI-specific, IDE will autocomplete!
@@ -697,6 +697,12 @@ class RawAgent(BaseAgent, Generic[OptionsCoT]):  # type: ignore[misc]
                If both this and a tokenizer on the underlying client are set, this one is used.
            kwargs: Any additional keyword arguments. Will be stored as ``additional_properties``.
        """
+        # Accept 'options' as an alias for 'default_options' so that
+        # Agent(options={"store": False}) works as expected instead of
+        # silently dropping the options into additional_properties.
+        if "options" in kwargs and default_options is None:
+            default_options = kwargs.pop("options")
+
        opts = dict(default_options) if default_options else {}

        if not isinstance(client, FunctionInvocationLayer) and isinstance(client, BaseChatClient):
@@ -72,6 +72,7 @@ if TYPE_CHECKING:
        Content,
        Message,
        ResponseStream,
+        UsageDetails,
    )

 else:
@@ -2095,6 +2096,7 @@ class FunctionInvocationLayer(Generic[OptionsCoT]):
            ChatResponse,
            ChatResponseUpdate,
            ResponseStream,
+            add_usage_details,
        )

        super_get_response = super().get_response  # type: ignore[misc]
@@ -2160,6 +2162,7 @@ class FunctionInvocationLayer(Generic[OptionsCoT]):
                prepped_messages = list(messages)
                fcc_messages: list[Message] = []
                response: ChatResponse[Any] | None = None
+                aggregated_usage: UsageDetails | None = None

                loop_enabled = self.function_invocation_configuration.get("enabled", True)
                max_iterations = self.function_invocation_configuration.get("max_iterations", DEFAULT_MAX_ITERATIONS)
@@ -2191,6 +2194,7 @@ class FunctionInvocationLayer(Generic[OptionsCoT]):
                            client_kwargs=filtered_kwargs,
                        ),
                    )
+                    aggregated_usage = add_usage_details(aggregated_usage, response.usage_details)

                    if response.conversation_id is not None:
                        _update_conversation_id(kwargs, response.conversation_id, mutable_options)
@@ -2207,6 +2211,7 @@ class FunctionInvocationLayer(Generic[OptionsCoT]):
                        execute_function_calls=execute_function_calls,
                    )
                    if result.get("action") == "return":
+                        response.usage_details = aggregated_usage
                        return response
                    total_function_calls += result.get("function_call_count", 0)
                    if result.get("action") == "stop":
@@ -2262,6 +2267,8 @@ class FunctionInvocationLayer(Generic[OptionsCoT]):
                        client_kwargs=filtered_kwargs,
                    ),
                )
+                aggregated_usage = add_usage_details(aggregated_usage, response.usage_details)
+                response.usage_details = aggregated_usage
                if fcc_messages:
                    for msg in reversed(fcc_messages):
                        response.messages.insert(0, msg)
@@ -287,9 +287,12 @@ class AgentExecutor(Executor):
            self._pending_responses_to_agent = pending_responses_payload

    def reset(self) -> None:
-        """Reset the internal cache of the executor."""
-        logger.debug("AgentExecutor %s: Resetting cache", self.id)
+        """Reset the internal cache and service session state of the executor for a new run."""
+        logger.debug("AgentExecutor %s: Resetting cache and service session", self.id)
        self._cache.clear()
+        # Clear service_session_id to prevent stale previous_response_id
+        # from leaking between workflow runs (e.g. in evaluate_workflow loops).
+        self._session.service_session_id = None

    async def _run_agent_and_emit(
        self,
@@ -345,6 +345,10 @@ class Workflow(DictConvertible):
                    self._runner.reset_iteration_count()
                    self._runner.context.reset_for_new_run()
                    self._state.clear()
+                    # Reset all executors (clears cached messages, sessions, etc.)
+                    for executor in self.executors.values():
+                        if hasattr(executor, "reset"):
+                            executor.reset()

                # Store run kwargs in State so executors can access them.
                # Only overwrite when new kwargs are explicitly provided or state was
@@ -8,9 +8,6 @@ import sys
 from collections.abc import Mapping, Sequence
 from typing import TYPE_CHECKING, Any, Generic, cast

-from openai.lib.azure import AsyncAzureOpenAI
-from openai.types.chat.chat_completion import Choice
-from openai.types.chat.chat_completion_chunk import Choice as ChunkChoice
 from pydantic import BaseModel

 from agent_framework import (
@@ -23,8 +20,7 @@ from agent_framework import (
    FunctionInvocationLayer,
 )
 from agent_framework.observability import ChatTelemetryLayer
-from agent_framework.openai import OpenAIChatOptions
-from agent_framework.openai._chat_client import RawOpenAIChatClient
+from agent_framework.openai._chat_client import OpenAIChatOptions, RawOpenAIChatClient

 from .._settings import load_settings
 from ._entra_id_authentication import AzureCredentialTypes, AzureTokenProvider
@@ -48,6 +44,10 @@ else:
    from typing_extensions import TypedDict  # type: ignore # pragma: no cover

 if TYPE_CHECKING:
+    from openai.lib.azure import AsyncAzureOpenAI
+    from openai.types.chat.chat_completion import Choice
+    from openai.types.chat.chat_completion_chunk import Choice as ChunkChoice
+
    from agent_framework._middleware import MiddlewareTypes

 logger: logging.Logger = logging.getLogger(__name__)
@@ -297,7 +297,9 @@ class AzureOpenAIChatClient(  # type: ignore[misc]
        For docs see:
        https://learn.microsoft.com/en-us/azure/ai-foundry/openai/references/on-your-data?tabs=python#context
        """
-        message = choice.message if isinstance(choice, Choice) else choice.delta
+        message = getattr(choice, "message", None)
+        if message is None:
+            message = getattr(choice, "delta", None)
        # When you enable asynchronous content filtering in Azure OpenAI, you may receive empty deltas
        if message is None:  # type: ignore
            return None
@@ -899,6 +899,25 @@ def get_meter(
 OBSERVABILITY_SETTINGS: ObservabilitySettings = ObservabilitySettings()


+def _read_bool_env(name: str, *, default: bool = False) -> bool:
+    """Read a boolean from an environment variable."""
+    value = os.getenv(name)
+    if value is None:
+        return default
+    return value.lower() in ("true", "1", "yes", "on")
+
+
+def _read_int_env(name: str, *, default: int | None = None) -> int | None:
+    """Read an optional integer from an environment variable."""
+    value = os.getenv(name)
+    if value is None:
+        return default
+    try:
+        return int(value)
+    except ValueError:
+        return default
+
+
 def enable_instrumentation(
    *,
    enable_sensitive_data: bool | None = None,
@@ -920,11 +939,15 @@ def enable_instrumentation(
    OBSERVABILITY_SETTINGS.enable_instrumentation = True
    if enable_sensitive_data is not None:
        OBSERVABILITY_SETTINGS.enable_sensitive_data = enable_sensitive_data
+    else:
+        # Re-read from current environment in case env vars were set after import (e.g. load_dotenv())
+        OBSERVABILITY_SETTINGS.enable_sensitive_data = _read_bool_env("ENABLE_SENSITIVE_DATA")


 def configure_otel_providers(
    *,
    enable_sensitive_data: bool | None = None,
+    enable_console_exporters: bool | None = None,
    exporters: list[LogRecordExporter | SpanExporter | MetricExporter] | None = None,
    views: list[View] | None = None,
    vs_code_extension_port: int | None = None,
@@ -963,6 +986,8 @@ def configure_otel_providers(
    Keyword Args:
        enable_sensitive_data: Enable OpenTelemetry sensitive events. Overrides
            the environment variable ENABLE_SENSITIVE_DATA if set. Default is None.
+        enable_console_exporters: Enable console exporters for traces, logs, and metrics.
+            Overrides the environment variable ENABLE_CONSOLE_EXPORTERS if set. Default is None.
        exporters: A list of custom exporters for logs, metrics or spans, or any combination.
            These will be added in addition to exporters configured via environment variables.
            Default is None.
@@ -1051,6 +1076,8 @@ def configure_otel_providers(
            settings_kwargs["env_file_encoding"] = env_file_encoding
        if enable_sensitive_data is not None:
            settings_kwargs["enable_sensitive_data"] = enable_sensitive_data
+        if enable_console_exporters is not None:
+            settings_kwargs["enable_console_exporters"] = enable_console_exporters
        if vs_code_extension_port is not None:
            settings_kwargs["vs_code_extension_port"] = vs_code_extension_port

@@ -1064,12 +1091,22 @@ def configure_otel_providers(
        OBSERVABILITY_SETTINGS._resource = updated_settings._resource  # type: ignore[reportPrivateUsage]
        OBSERVABILITY_SETTINGS._executed_setup = False  # type: ignore[reportPrivateUsage]
    else:
-        # Update the observability settings with the provided values
+        # Re-read settings from current environment in case env vars were set
+        # after import (e.g. via load_dotenv()). Explicit parameters take precedence.
        OBSERVABILITY_SETTINGS.enable_instrumentation = True
-        if enable_sensitive_data is not None:
-            OBSERVABILITY_SETTINGS.enable_sensitive_data = enable_sensitive_data
-        if vs_code_extension_port is not None:
-            OBSERVABILITY_SETTINGS.vs_code_extension_port = vs_code_extension_port
+        OBSERVABILITY_SETTINGS.enable_sensitive_data = (
+            enable_sensitive_data if enable_sensitive_data is not None else _read_bool_env("ENABLE_SENSITIVE_DATA")
+        )
+        OBSERVABILITY_SETTINGS.enable_console_exporters = (
+            enable_console_exporters
+            if enable_console_exporters is not None
+            else _read_bool_env("ENABLE_CONSOLE_EXPORTERS")
+        )
+        OBSERVABILITY_SETTINGS.vs_code_extension_port = (
+            vs_code_extension_port if vs_code_extension_port is not None else _read_int_env("VS_CODE_EXTENSION_PORT")
+        )
+        OBSERVABILITY_SETTINGS._resource = create_resource()  # type: ignore[reportPrivateUsage]
+        OBSERVABILITY_SETTINGS._executed_setup = False  # type: ignore[reportPrivateUsage]

    OBSERVABILITY_SETTINGS._configure(  # type: ignore[reportPrivateUsage]
        additional_exporters=exporters,
@@ -121,9 +121,13 @@ exclude_dirs = ["tests"]
 executor.type = "uv"
 include = "../../shared_tasks.toml"

-[tool.poe.tasks]
-mypy = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework"
-test = 'pytest -m "not integration" --cov=agent_framework --cov-report=term-missing:skip-covered -n auto --dist worksteal tests'
+[tool.poe.tasks.mypy]
+help = "Run MyPy for this package."
+cmd = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework"
+
+[tool.poe.tasks.test]
+help = "Run the default unit test suite for this package."
+cmd = 'pytest -m "not integration" --cov=agent_framework --cov-report=term-missing:skip-covered -n auto --dist worksteal tests'

 [tool.flit.module]
 name = "agent_framework"
@@ -652,6 +652,88 @@ async def test_streaming_with_none_delta(
    assert any(msg.contents for msg in results)


+# region _parse_text_from_openai direct unit tests
+
+
+def test_parse_text_from_openai_with_choice_message(azure_openai_unit_test_env: dict[str, str]) -> None:
+    """Test _parse_text_from_openai correctly reads message from a Choice."""
+    client = AzureOpenAIChatClient()
+    choice = Choice(
+        index=0,
+        message=ChatCompletionMessage(content="hello", role="assistant"),
+        finish_reason="stop",
+    )
+    result = client._parse_text_from_openai(choice)
+    assert result is not None
+    assert result.type == "text"
+    assert result.text == "hello"
+
+
+def test_parse_text_from_openai_with_chunk_choice_delta(azure_openai_unit_test_env: dict[str, str]) -> None:
+    """Test _parse_text_from_openai correctly reads delta from a ChunkChoice."""
+    client = AzureOpenAIChatClient()
+    choice = ChunkChoice(
+        index=0,
+        delta=ChunkChoiceDelta(content="streamed", role="assistant"),
+        finish_reason=None,
+    )
+    result = client._parse_text_from_openai(choice)
+    assert result is not None
+    assert result.type == "text"
+    assert result.text == "streamed"
+
+
+def test_parse_text_from_openai_refusal_choice(azure_openai_unit_test_env: dict[str, str]) -> None:
+    """Test _parse_text_from_openai returns refusal text from a Choice."""
+    client = AzureOpenAIChatClient()
+    choice = Choice(
+        index=0,
+        message=ChatCompletionMessage(content=None, role="assistant", refusal="I cannot help with that"),
+        finish_reason="stop",
+    )
+    result = client._parse_text_from_openai(choice)
+    assert result is not None
+    assert result.type == "text"
+    assert result.text == "I cannot help with that"
+
+
+def test_parse_text_from_openai_refusal_chunk_choice(azure_openai_unit_test_env: dict[str, str]) -> None:
+    """Test _parse_text_from_openai returns refusal text from a ChunkChoice."""
+    client = AzureOpenAIChatClient()
+    choice = ChunkChoice(
+        index=0,
+        delta=ChunkChoiceDelta(content=None, role="assistant", refusal="I cannot help with that"),
+        finish_reason=None,
+    )
+    result = client._parse_text_from_openai(choice)
+    assert result is not None
+    assert result.type == "text"
+    assert result.text == "I cannot help with that"
+
+
+def test_parse_text_from_openai_no_content_no_refusal(azure_openai_unit_test_env: dict[str, str]) -> None:
+    """Test _parse_text_from_openai returns None when no content or refusal."""
+    client = AzureOpenAIChatClient()
+    choice = Choice(
+        index=0,
+        message=ChatCompletionMessage(content=None, role="assistant"),
+        finish_reason="stop",
+    )
+    result = client._parse_text_from_openai(choice)
+    assert result is None
+
+
+def test_parse_text_from_openai_none_delta(azure_openai_unit_test_env: dict[str, str]) -> None:
+    """Test _parse_text_from_openai returns None when delta is None (async content filtering)."""
+    client = AzureOpenAIChatClient()
+    choice = ChunkChoice.model_construct(index=0, delta=None, finish_reason=None)
+    result = client._parse_text_from_openai(choice)
+    assert result is None
+
+
+# endregion
+
+
@patch.object(AsyncChatCompletions, "create", new_callable=AsyncMock)
 async def test_cmc_with_conversation_id(
    mock_create: AsyncMock,
@@ -0,0 +1,749 @@
+# Copyright (c) Microsoft. All rights reserved.
+
+"""Tests for evaluator checks and LocalEvaluator."""
+
+from __future__ import annotations
+
+import inspect
+
+import pytest
+
+from agent_framework._evaluation import (
+    CheckResult,
+    EvalItem,
+    ExpectedToolCall,
+    LocalEvaluator,
+    evaluator,
+    keyword_check,
+    tool_call_args_match,
+    tool_calls_present,
+)
+from agent_framework._types import Content, Message
+
+# ---------------------------------------------------------------------------
+# Helpers
+# ---------------------------------------------------------------------------
+
+
+def _make_item(
+    query: str = "What's the weather in Paris?",
+    response: str = "It's sunny and 75°F",
+    expected_output: str | None = None,
+    conversation: list | None = None,
+    tools: list | None = None,
+    context: str | None = None,
+) -> EvalItem:
+    if conversation is None:
+        conversation = [Message("user", [query]), Message("assistant", [response])]
+    return EvalItem(
+        conversation=conversation,
+        expected_output=expected_output,
+        tools=tools,
+        context=context,
+    )
+
+
+# ---------------------------------------------------------------------------
+# Tier 1: (query, response) -> result
+# ---------------------------------------------------------------------------
+
+
+class TestTier1SimpleChecks:
+    @pytest.mark.asyncio
+    async def test_bool_return_true(self):
+        @evaluator
+        def has_temperature(query: str, response: str) -> bool:
+            return "°F" in response
+
+        result = await has_temperature(_make_item())
+        assert result.passed is True
+        assert result.check_name == "has_temperature"
+
+    @pytest.mark.asyncio
+    async def test_bool_return_false(self):
+        @evaluator
+        def has_celsius(query: str, response: str) -> bool:
+            return "°C" in response
+
+        result = await has_celsius(_make_item())
+        assert result.passed is False
+
+    @pytest.mark.asyncio
+    async def test_float_return_passing(self):
+        @evaluator
+        def length_score(response: str) -> float:
+            return min(len(response) / 10, 1.0)
+
+        result = await length_score(_make_item())
+        assert result.passed is True
+        assert "score=" in result.reason
+
+    @pytest.mark.asyncio
+    async def test_float_return_failing(self):
+        @evaluator
+        def always_low(response: str) -> float:
+            return 0.1
+
+        result = await always_low(_make_item())
+        assert result.passed is False
+
+    @pytest.mark.asyncio
+    async def test_response_only(self):
+        """Function with only 'response' param should work."""
+
+        @evaluator
+        def is_short(response: str) -> bool:
+            return len(response) < 1000
+
+        result = await is_short(_make_item())
+        assert result.passed is True
+
+    @pytest.mark.asyncio
+    async def test_query_only(self):
+        """Function with only 'query' param should work."""
+
+        @evaluator
+        def is_question(query: str) -> bool:
+            return "?" in query
+
+        result = await is_question(_make_item())
+        assert result.passed is True
+
+
+# ---------------------------------------------------------------------------
+# Tier 2: (query, response, expected_output) -> result
+# ---------------------------------------------------------------------------
+
+
+class TestTier2GroundTruth:
+    @pytest.mark.asyncio
+    async def test_exact_match(self):
+        @evaluator
+        def exact_match(response: str, expected_output: str) -> bool:
+            return response.strip() == expected_output.strip()
+
+        item = _make_item(response="42", expected_output="42")
+        assert (await exact_match(item)).passed is True
+
+        item2 = _make_item(response="43", expected_output="42")
+        assert (await exact_match(item2)).passed is False
+
+    @pytest.mark.asyncio
+    async def test_expected_output_defaults_to_empty(self):
+        """When expected_output is None on the item, it should be passed as ''."""
+
+        @evaluator
+        def check_expected(expected_output: str) -> bool:
+            return expected_output == ""
+
+        result = await check_expected(_make_item(expected_output=None))
+        assert result.passed is True
+
+    @pytest.mark.asyncio
+    async def test_similarity_score(self):
+        @evaluator
+        def word_overlap(response: str, expected_output: str) -> float:
+            r_words = set(response.lower().split())
+            e_words = set(expected_output.lower().split())
+            if not e_words:
+                return 1.0
+            return len(r_words & e_words) / len(e_words)
+
+        item = _make_item(response="sunny warm day", expected_output="warm sunny afternoon")
+        result = await word_overlap(item)
+        assert result.passed is True  # 2/3 overlap ≥ 0.5
+
+
+# ---------------------------------------------------------------------------
+# Tier 3: full context (conversation, tools, context)
+# ---------------------------------------------------------------------------
+
+
+class TestTier3FullContext:
+    @pytest.mark.asyncio
+    async def test_conversation_access(self):
+        @evaluator
+        def multi_turn(query: str, response: str, *, conversation: list) -> bool:
+            return len(conversation) >= 2
+
+        item = _make_item(conversation=[Message("user", []), Message("assistant", [])])
+        assert (await multi_turn(item)).passed is True
+
+        item2 = _make_item(conversation=[Message("user", [])])
+        assert (await multi_turn(item2)).passed is False
+
+    @pytest.mark.asyncio
+    async def test_tools_access(self):
+        @evaluator
+        def has_tools(tools: list) -> bool:
+            return len(tools) > 0
+
+        mock_tool = type(
+            "MockTool",
+            (),
+            {"name": "get_weather", "description": "Get weather", "parameters": lambda self: {}},
+        )()
+        item = _make_item(tools=[mock_tool])
+        assert (await has_tools(item)).passed is True
+
+    @pytest.mark.asyncio
+    async def test_context_access(self):
+        @evaluator
+        def grounded(response: str, context: str) -> bool:
+            if not context:
+                return True
+            return any(word in response.lower() for word in context.lower().split())
+
+        item = _make_item(response="It's sunny", context="sunny warm")
+        assert (await grounded(item)).passed is True
+
+    @pytest.mark.asyncio
+    async def test_all_params(self):
+        @evaluator
+        def full_check(
+            query: str,
+            response: str,
+            expected_output: str,
+            conversation: list,
+            tools: list,
+            context: str,
+        ) -> bool:
+            return all([query, response, expected_output is not None, isinstance(conversation, list)])
+
+        item = _make_item(expected_output="foo", context="bar")
+        assert (await full_check(item)).passed is True
+
+
+# ---------------------------------------------------------------------------
+# Return type coercion
+# ---------------------------------------------------------------------------
+
+
+class TestReturnTypeCoercion:
+    @pytest.mark.asyncio
+    async def test_dict_with_score(self):
+        @evaluator
+        def scored(response: str) -> dict:
+            return {"score": 0.9, "reason": "good answer"}
+
+        result = await scored(_make_item())
+        assert result.passed is True
+        assert result.reason == "good answer"
+
+    @pytest.mark.asyncio
+    async def test_dict_with_score_below_threshold(self):
+        @evaluator
+        def low_scored(response: str) -> dict:
+            return {"score": 0.3}
+
+        result = await low_scored(_make_item())
+        assert result.passed is False
+
+    @pytest.mark.asyncio
+    async def test_dict_with_custom_threshold(self):
+        @evaluator
+        def custom_threshold(response: str) -> dict:
+            return {"score": 0.3, "threshold": 0.2}
+
+        result = await custom_threshold(_make_item())
+        assert result.passed is True
+
+    @pytest.mark.asyncio
+    async def test_dict_with_passed(self):
+        @evaluator
+        def explicit_pass(response: str) -> dict:
+            return {"passed": True, "reason": "all good"}
+
+        result = await explicit_pass(_make_item())
+        assert result.passed is True
+        assert result.reason == "all good"
+
+    @pytest.mark.asyncio
+    async def test_check_result_passthrough(self):
+        @evaluator
+        def returns_check_result(response: str) -> CheckResult:
+            return CheckResult(True, "direct result", "custom")
+
+        result = await returns_check_result(_make_item())
+        assert result.passed is True
+        assert result.reason == "direct result"
+        assert result.check_name == "custom"
+
+    @pytest.mark.asyncio
+    async def test_unsupported_return_type(self):
+        @evaluator
+        def bad_return(response: str) -> str:
+            return "oops"
+
+        with pytest.raises(TypeError, match="unsupported type"):
+            await bad_return(_make_item())
+
+    @pytest.mark.asyncio
+    async def test_int_return(self):
+        @evaluator
+        def int_score(response: str) -> int:
+            return 1
+
+        result = await int_score(_make_item())
+        assert result.passed is True
+
+
+# ---------------------------------------------------------------------------
+# Decorator variants
+# ---------------------------------------------------------------------------
+
+
+class TestDecoratorVariants:
+    @pytest.mark.asyncio
+    async def test_decorator_no_parens(self):
+        @evaluator
+        def my_check(response: str) -> bool:
+            return True
+
+        assert (await my_check(_make_item())).passed is True
+
+    @pytest.mark.asyncio
+    async def test_decorator_with_name(self):
+        @evaluator(name="custom_name")
+        def my_check(response: str) -> bool:
+            return True
+
+        assert my_check.__name__ == "custom_name"
+        result = await my_check(_make_item())
+        assert result.check_name == "custom_name"
+
+    @pytest.mark.asyncio
+    async def test_direct_call(self):
+        def raw_fn(query: str, response: str) -> bool:
+            return len(response) > 0
+
+        check = evaluator(raw_fn, name="direct")
+        result = await check(_make_item())
+        assert result.passed is True
+        assert result.check_name == "direct"
+
+
+# ---------------------------------------------------------------------------
+# Error handling
+# ---------------------------------------------------------------------------
+
+
+class TestErrorHandling:
+    @pytest.mark.asyncio
+    async def test_unknown_required_param_raises(self):
+        @evaluator
+        def bad_params(query: str, unknown_param: str) -> bool:
+            return True
+
+        with pytest.raises(TypeError, match="unknown required parameter"):
+            await bad_params(_make_item())
+
+    @pytest.mark.asyncio
+    async def test_unknown_optional_param_ok(self):
+        @evaluator
+        def optional_unknown(query: str, foo: str = "default") -> bool:
+            return foo == "default"
+
+        result = await optional_unknown(_make_item())
+        assert result.passed is True
+
+    @pytest.mark.asyncio
+    async def test_async_function_works_with_evaluator(self):
+        """Using an async function with @evaluator should work."""
+
+        @evaluator
+        async def async_fn(response: str) -> bool:
+            return True
+
+        result = async_fn(_make_item())
+        # Should return an awaitable
+        assert inspect.isawaitable(result)
+        check_result = await result
+        assert check_result.passed is True
+
+
+# ---------------------------------------------------------------------------
+# Integration with LocalEvaluator
+# ---------------------------------------------------------------------------
+
+
+class TestLocalEvaluatorIntegration:
+    @pytest.mark.asyncio
+    async def test_mixed_checks(self):
+        """Function evaluators mix with built-in checks in LocalEvaluator."""
+
+        @evaluator
+        def length_ok(response: str) -> bool:
+            return len(response) > 5
+
+        local = LocalEvaluator(
+            keyword_check("sunny"),
+            length_ok,
+        )
+        items = [_make_item()]
+        results = await local.evaluate(items, eval_name="mixed test")
+
+        assert results.status == "completed"
+        assert results.result_counts["passed"] == 1
+        assert results.result_counts["failed"] == 0
+
+    @pytest.mark.asyncio
+    async def test_evaluator_failure_counted(self):
+        @evaluator
+        def always_fail(response: str) -> bool:
+            return False
+
+        local = LocalEvaluator(always_fail)
+        results = await local.evaluate([_make_item()])
+
+        assert results.result_counts["failed"] == 1
+
+    @pytest.mark.asyncio
+    async def test_multiple_evaluators(self):
+        @evaluator
+        def check_a(response: str) -> float:
+            return 0.9
+
+        @evaluator
+        def check_b(query: str, response: str, expected_output: str) -> bool:
+            return True
+
+        @evaluator(name="check_c")
+        def check_c(response: str, conversation: list) -> dict:
+            return {"score": 0.8, "reason": "looks good"}
+
+        local = LocalEvaluator(check_a, check_b, check_c)
+        results = await local.evaluate([_make_item(expected_output="test")])
+
+        assert results.result_counts["passed"] == 1
+        assert "check_a" in results.per_evaluator
+        assert "check_b" in results.per_evaluator
+        assert "check_c" in results.per_evaluator
+
+
+# ---------------------------------------------------------------------------
+# Async evaluator (via @evaluator which handles async automatically)
+# ---------------------------------------------------------------------------
+
+
+class TestAsyncFunctionEvaluator:
+    @pytest.mark.asyncio
+    async def test_async_evaluator_in_local(self):
+        @evaluator
+        async def async_check(query: str, response: str) -> bool:
+            return len(response) > 0
+
+        local = LocalEvaluator(async_check)
+        results = await local.evaluate([_make_item()])
+        assert results.result_counts["passed"] == 1
+
+    @pytest.mark.asyncio
+    async def test_async_with_name(self):
+        @evaluator(name="named_async")
+        async def my_async(response: str) -> float:
+            return 0.75
+
+        result = await my_async(_make_item())
+        assert result.passed is True
+        assert result.check_name == "named_async"
+
+
+# ---------------------------------------------------------------------------
+# Auto-wrapping bare checks in evaluate_agent
+# ---------------------------------------------------------------------------
+
+
+class TestAutoWrapEvalChecks:
+    @pytest.mark.asyncio
+    async def test_bare_check_in_evaluators_list(self):
+        """Bare EvalCheck callables are auto-wrapped in LocalEvaluator."""
+        from agent_framework._evaluation import _run_evaluators
+
+        @evaluator
+        def is_long(response: str) -> bool:
+            return len(response.split()) > 2
+
+        items = [_make_item(response="It is sunny and warm today")]
+        results = await _run_evaluators(is_long, items, eval_name="test")
+        assert len(results) == 1
+        assert results[0].result_counts["passed"] == 1
+
+    @pytest.mark.asyncio
+    async def test_mixed_evaluators_and_checks(self):
+        """Mix of Evaluator instances and bare checks works."""
+        from agent_framework._evaluation import _run_evaluators
+
+        @evaluator
+        def has_words(response: str) -> bool:
+            return len(response.split()) > 0
+
+        local = LocalEvaluator(keyword_check("sunny"))
+
+        items = [_make_item(response="It is sunny")]
+        results = await _run_evaluators([local, has_words], items, eval_name="test")
+        assert len(results) == 2
+        assert all(r.result_counts["passed"] == 1 for r in results)
+
+    @pytest.mark.asyncio
+    async def test_adjacent_checks_grouped(self):
+        """Adjacent bare checks are grouped into a single LocalEvaluator."""
+        from agent_framework._evaluation import _run_evaluators
+
+        @evaluator
+        def check_a(response: str) -> bool:
+            return True
+
+        @evaluator
+        def check_b(response: str) -> bool:
+            return True
+
+        items = [_make_item()]
+        results = await _run_evaluators([check_a, check_b], items, eval_name="test")
+        # Two adjacent checks → one LocalEvaluator → one result
+        assert len(results) == 1
+        assert results[0].result_counts["passed"] == 1
+
+
+# ---------------------------------------------------------------------------
+# Expected Tool Calls
+# ---------------------------------------------------------------------------
+
+
+def _make_tool_call_item(
+    calls: list[tuple[str, dict | None]],
+    expected: list[ExpectedToolCall] | None = None,
+) -> EvalItem:
+    """Build an EvalItem with tool calls in the conversation."""
+    msgs: list[Message] = [Message("user", ["Do something"])]
+    for name, args in calls:
+        msgs.append(Message("assistant", [Content.from_function_call("call_" + name, name, arguments=args)]))
+    msgs.append(Message("assistant", ["Done"]))
+    return EvalItem(conversation=msgs, expected_tool_calls=expected)
+
+
+class TestExpectedToolCallType:
+    def test_name_only(self):
+        tc = ExpectedToolCall("get_weather")
+        assert tc.name == "get_weather"
+        assert tc.arguments is None
+
+    def test_name_and_args(self):
+        tc = ExpectedToolCall("get_weather", {"location": "NYC"})
+        assert tc.name == "get_weather"
+        assert tc.arguments == {"location": "NYC"}
+
+
+class TestToolCallsPresent:
+    def test_all_present(self):
+        item = _make_tool_call_item(
+            calls=[("get_weather", None), ("get_news", None)],
+            expected=[ExpectedToolCall("get_weather"), ExpectedToolCall("get_news")],
+        )
+        result = tool_calls_present(item)
+        assert result.passed is True
+        assert result.check_name == "tool_calls_present"
+
+    def test_missing_tool(self):
+        item = _make_tool_call_item(
+            calls=[("get_weather", None)],
+            expected=[ExpectedToolCall("get_weather"), ExpectedToolCall("get_news")],
+        )
+        result = tool_calls_present(item)
+        assert result.passed is False
+        assert "get_news" in result.reason
+
+    def test_extras_ok(self):
+        item = _make_tool_call_item(
+            calls=[("get_weather", None), ("get_news", None), ("get_stock", None)],
+            expected=[ExpectedToolCall("get_weather")],
+        )
+        result = tool_calls_present(item)
+        assert result.passed is True
+
+    def test_no_expected(self):
+        item = _make_tool_call_item(calls=[("get_weather", None)])
+        result = tool_calls_present(item)
+        assert result.passed is True
+        assert "No expected" in result.reason
+
+
+class TestToolCallArgsMatch:
+    def test_name_only_match(self):
+        item = _make_tool_call_item(
+            calls=[("get_weather", {"location": "NYC"})],
+            expected=[ExpectedToolCall("get_weather")],
+        )
+        result = tool_call_args_match(item)
+        assert result.passed is True
+
+    def test_args_exact_match(self):
+        item = _make_tool_call_item(
+            calls=[("get_weather", {"location": "NYC", "units": "fahrenheit"})],
+            expected=[ExpectedToolCall("get_weather", {"location": "NYC"})],
+        )
+        # Subset match — extra "units" key is OK
+        result = tool_call_args_match(item)
+        assert result.passed is True
+
+    def test_args_mismatch(self):
+        item = _make_tool_call_item(
+            calls=[("get_weather", {"location": "LA"})],
+            expected=[ExpectedToolCall("get_weather", {"location": "NYC"})],
+        )
+        result = tool_call_args_match(item)
+        assert result.passed is False
+        assert "args mismatch" in result.reason
+
+    def test_tool_not_called(self):
+        item = _make_tool_call_item(
+            calls=[("get_news", None)],
+            expected=[ExpectedToolCall("get_weather", {"location": "NYC"})],
+        )
+        result = tool_call_args_match(item)
+        assert result.passed is False
+        assert "not called" in result.reason
+
+    def test_multiple_expected(self):
+        item = _make_tool_call_item(
+            calls=[
+                ("get_weather", {"location": "NYC"}),
+                ("book_flight", {"destination": "LA", "date": "tomorrow"}),
+            ],
+            expected=[
+                ExpectedToolCall("get_weather", {"location": "NYC"}),
+                ExpectedToolCall("book_flight", {"destination": "LA"}),
+            ],
+        )
+        result = tool_call_args_match(item)
+        assert result.passed is True
+
+    def test_no_expected(self):
+        item = _make_tool_call_item(calls=[("get_weather", None)])
+        result = tool_call_args_match(item)
+        assert result.passed is True
+
+
+class TestExpectedToolCallsFieldInjection:
+    """Test that @evaluator can receive expected_tool_calls via parameter injection."""
+
+    @pytest.mark.asyncio
+    async def test_injection(self):
+        @evaluator
+        def check_tools(expected_tool_calls: list) -> bool:
+            return len(expected_tool_calls) == 2
+
+        item = _make_tool_call_item(
+            calls=[],
+            expected=[ExpectedToolCall("a"), ExpectedToolCall("b")],
+        )
+        result = await check_tools(item)
+        assert result.passed is True
+
+    @pytest.mark.asyncio
+    async def test_injection_empty_default(self):
+        @evaluator
+        def check_tools(expected_tool_calls: list) -> bool:
+            return len(expected_tool_calls) == 0
+
+        item = _make_tool_call_item(calls=[])
+        result = await check_tools(item)
+        assert result.passed is True
+
+
+# ---------------------------------------------------------------------------
+# Per-item results (auditing)
+# ---------------------------------------------------------------------------
+
+
+class TestPerItemResults:
+    """LocalEvaluator should produce per-item EvalItemResult with query/response."""
+
+    @pytest.mark.asyncio
+    async def test_items_populated_with_query_and_response(self):
+        @evaluator
+        def is_sunny(response: str) -> bool:
+            return "sunny" in response.lower()
+
+        item = _make_item(query="Weather?", response="It's sunny!")
+        local = LocalEvaluator(is_sunny)
+        results = await local.evaluate([item])
+
+        assert len(results.items) == 1
+        ri = results.items[0]
+        assert ri.item_id == "0"
+        assert ri.status == "pass"
+        assert ri.input_text == "Weather?"
+        assert ri.output_text == "It's sunny!"
+        assert len(ri.scores) == 1
+        assert ri.scores[0].name == "is_sunny"
+        assert ri.scores[0].passed is True
+
+    @pytest.mark.asyncio
+    async def test_items_populated_on_failure(self):
+        @evaluator
+        def always_fail(response: str) -> bool:
+            return False
+
+        item = _make_item(query="Hello", response="World")
+        local = LocalEvaluator(always_fail)
+        results = await local.evaluate([item])
+
+        assert len(results.items) == 1
+        ri = results.items[0]
+        assert ri.status == "fail"
+        assert ri.input_text == "Hello"
+        assert ri.output_text == "World"
+        assert ri.scores[0].passed is False
+        assert ri.scores[0].score == 0.0
+
+    @pytest.mark.asyncio
+    async def test_multiple_items_indexed(self):
+        @evaluator
+        def pass_all(response: str) -> bool:
+            return True
+
+        items = [
+            _make_item(query="Q1", response="R1"),
+            _make_item(query="Q2", response="R2"),
+        ]
+        local = LocalEvaluator(pass_all)
+        results = await local.evaluate(items)
+
+        assert len(results.items) == 2
+        assert results.items[0].item_id == "0"
+        assert results.items[0].input_text == "Q1"
+        assert results.items[0].output_text == "R1"
+        assert results.items[1].item_id == "1"
+        assert results.items[1].input_text == "Q2"
+        assert results.items[1].output_text == "R2"
+
+
+# ---------------------------------------------------------------------------
+# num_repetitions validation
+# ---------------------------------------------------------------------------
+
+
+class TestNumRepetitions:
+    """Tests for the num_repetitions parameter on evaluate_agent."""
+
+    @pytest.mark.asyncio
+    async def test_num_repetitions_validation_rejects_zero(self):
+        from agent_framework._evaluation import evaluate_agent
+
+        with pytest.raises(ValueError, match="num_repetitions must be >= 1"):
+            await evaluate_agent(
+                queries=["Hello"],
+                evaluators=LocalEvaluator(keyword_check("hello")),
+                num_repetitions=0,
+            )
+
+    @pytest.mark.asyncio
+    async def test_num_repetitions_validation_rejects_negative(self):
+        from agent_framework._evaluation import evaluate_agent
+
+        with pytest.raises(ValueError, match="num_repetitions must be >= 1"):
+            await evaluate_agent(
+                queries=["Hello"],
+                evaluators=LocalEvaluator(keyword_check("hello")),
+                num_repetitions=-1,
+            )
@@ -17,6 +17,7 @@ from agent_framework import (
    ChatResponseUpdate,
    Content,
    Message,
+    RawAgent,
    ResponseStream,
    SupportsAgentRun,
    UsageDetails,
@@ -1033,6 +1034,272 @@ def test_enable_instrumentation_with_sensitive_data(monkeypatch):
    assert observability.OBSERVABILITY_SETTINGS.enable_sensitive_data is True


+def test_enable_instrumentation_reads_env_sensitive_data(monkeypatch):
+    """Test enable_instrumentation re-reads ENABLE_SENSITIVE_DATA from os.environ when not explicitly passed."""
+    import importlib
+
+    monkeypatch.setenv("ENABLE_INSTRUMENTATION", "false")
+    monkeypatch.setenv("ENABLE_SENSITIVE_DATA", "false")
+
+    observability = importlib.import_module("agent_framework.observability")
+    importlib.reload(observability)
+
+    assert observability.OBSERVABILITY_SETTINGS.enable_sensitive_data is False
+
+    # Simulate load_dotenv() setting env var after import
+    monkeypatch.setenv("ENABLE_SENSITIVE_DATA", "true")
+
+    observability.enable_instrumentation()
+    assert observability.OBSERVABILITY_SETTINGS.enable_instrumentation is True
+    assert observability.OBSERVABILITY_SETTINGS.enable_sensitive_data is True
+
+
+def test_configure_otel_providers_reads_env_sensitive_data(monkeypatch):
+    """Test configure_otel_providers re-reads ENABLE_SENSITIVE_DATA from os.environ when not explicitly passed."""
+    import importlib
+
+    monkeypatch.setenv("ENABLE_INSTRUMENTATION", "false")
+    monkeypatch.setenv("ENABLE_SENSITIVE_DATA", "false")
+    monkeypatch.delenv("VS_CODE_EXTENSION_PORT", raising=False)
+    monkeypatch.delenv("ENABLE_CONSOLE_EXPORTERS", raising=False)
+    for key in [
+        "OTEL_EXPORTER_OTLP_ENDPOINT",
+        "OTEL_EXPORTER_OTLP_TRACES_ENDPOINT",
+        "OTEL_EXPORTER_OTLP_METRICS_ENDPOINT",
+        "OTEL_EXPORTER_OTLP_LOGS_ENDPOINT",
+    ]:
+        monkeypatch.delenv(key, raising=False)
+
+    observability = importlib.import_module("agent_framework.observability")
+    importlib.reload(observability)
+
+    assert observability.OBSERVABILITY_SETTINGS.enable_sensitive_data is False
+
+    # Simulate load_dotenv() setting env var after import
+    monkeypatch.setenv("ENABLE_SENSITIVE_DATA", "true")
+
+    observability.configure_otel_providers()
+    assert observability.OBSERVABILITY_SETTINGS.enable_instrumentation is True
+    assert observability.OBSERVABILITY_SETTINGS.enable_sensitive_data is True
+
+
+def test_configure_otel_providers_reads_env_vs_code_port(monkeypatch):
+    """Test configure_otel_providers re-reads VS_CODE_EXTENSION_PORT from os.environ when not explicitly passed."""
+    import importlib
+    from unittest.mock import patch as mock_patch
+
+    monkeypatch.setenv("ENABLE_INSTRUMENTATION", "false")
+    monkeypatch.delenv("VS_CODE_EXTENSION_PORT", raising=False)
+    monkeypatch.delenv("ENABLE_CONSOLE_EXPORTERS", raising=False)
+    for key in [
+        "OTEL_EXPORTER_OTLP_ENDPOINT",
+        "OTEL_EXPORTER_OTLP_TRACES_ENDPOINT",
+        "OTEL_EXPORTER_OTLP_METRICS_ENDPOINT",
+        "OTEL_EXPORTER_OTLP_LOGS_ENDPOINT",
+    ]:
+        monkeypatch.delenv(key, raising=False)
+
+    observability = importlib.import_module("agent_framework.observability")
+    importlib.reload(observability)
+
+    assert observability.OBSERVABILITY_SETTINGS.vs_code_extension_port is None
+
+    # Simulate load_dotenv() setting env var after import
+    monkeypatch.setenv("VS_CODE_EXTENSION_PORT", "4317")
+
+    # Mock _configure to avoid needing optional OTLP gRPC exporter dependency
+    with mock_patch.object(observability.OBSERVABILITY_SETTINGS, "_configure"):
+        observability.configure_otel_providers()
+    assert observability.OBSERVABILITY_SETTINGS.vs_code_extension_port == 4317
+
+
+def test_configure_otel_providers_explicit_param_overrides_env(monkeypatch):
+    """Test that explicit parameters to configure_otel_providers override env vars."""
+    import importlib
+
+    monkeypatch.setenv("ENABLE_INSTRUMENTATION", "false")
+    monkeypatch.setenv("ENABLE_SENSITIVE_DATA", "true")
+    monkeypatch.delenv("VS_CODE_EXTENSION_PORT", raising=False)
+    monkeypatch.delenv("ENABLE_CONSOLE_EXPORTERS", raising=False)
+    for key in [
+        "OTEL_EXPORTER_OTLP_ENDPOINT",
+        "OTEL_EXPORTER_OTLP_TRACES_ENDPOINT",
+        "OTEL_EXPORTER_OTLP_METRICS_ENDPOINT",
+        "OTEL_EXPORTER_OTLP_LOGS_ENDPOINT",
+    ]:
+        monkeypatch.delenv(key, raising=False)
+
+    observability = importlib.import_module("agent_framework.observability")
+    importlib.reload(observability)
+
+    # Explicit False should override the env var True
+    observability.configure_otel_providers(enable_sensitive_data=False)
+    assert observability.OBSERVABILITY_SETTINGS.enable_sensitive_data is False
+
+
+def test_enable_instrumentation_explicit_param_overrides_env(monkeypatch):
+    """Test that explicit enable_sensitive_data parameter to enable_instrumentation overrides env var."""
+    import importlib
+
+    monkeypatch.setenv("ENABLE_INSTRUMENTATION", "false")
+    monkeypatch.setenv("ENABLE_SENSITIVE_DATA", "true")
+
+    observability = importlib.import_module("agent_framework.observability")
+    importlib.reload(observability)
+
+    # Explicit False should override the env var True
+    observability.enable_instrumentation(enable_sensitive_data=False)
+    assert observability.OBSERVABILITY_SETTINGS.enable_instrumentation is True
+    assert observability.OBSERVABILITY_SETTINGS.enable_sensitive_data is False
+
+
+def test_enable_instrumentation_does_not_touch_console_exporters(monkeypatch):
+    """Test enable_instrumentation does not modify enable_console_exporters (it is an exporter concern)."""
+    import importlib
+
+    monkeypatch.setenv("ENABLE_INSTRUMENTATION", "false")
+    monkeypatch.delenv("ENABLE_CONSOLE_EXPORTERS", raising=False)
+
+    observability = importlib.import_module("agent_framework.observability")
+    importlib.reload(observability)
+
+    assert observability.OBSERVABILITY_SETTINGS.enable_console_exporters is False
+
+    # Simulate load_dotenv() setting env var after import
+    monkeypatch.setenv("ENABLE_CONSOLE_EXPORTERS", "true")
+
+    observability.enable_instrumentation()
+    # enable_console_exporters is not managed by enable_instrumentation;
+    # it is only read by configure_otel_providers.
+    assert observability.OBSERVABILITY_SETTINGS.enable_console_exporters is False
+
+
+def test_enable_instrumentation_does_not_clobber_console_exporters(monkeypatch):
+    """Test enable_instrumentation does not reset enable_console_exporters set by prior configure call."""
+    import importlib
+
+    monkeypatch.setenv("ENABLE_INSTRUMENTATION", "false")
+    monkeypatch.delenv("ENABLE_CONSOLE_EXPORTERS", raising=False)
+    monkeypatch.delenv("ENABLE_SENSITIVE_DATA", raising=False)
+    monkeypatch.delenv("VS_CODE_EXTENSION_PORT", raising=False)
+    for key in [
+        "OTEL_EXPORTER_OTLP_ENDPOINT",
+        "OTEL_EXPORTER_OTLP_TRACES_ENDPOINT",
+        "OTEL_EXPORTER_OTLP_METRICS_ENDPOINT",
+        "OTEL_EXPORTER_OTLP_LOGS_ENDPOINT",
+    ]:
+        monkeypatch.delenv(key, raising=False)
+
+    observability = importlib.import_module("agent_framework.observability")
+    importlib.reload(observability)
+
+    # Set console exporters via configure_otel_providers
+    observability.configure_otel_providers(enable_console_exporters=True)
+    assert observability.OBSERVABILITY_SETTINGS.enable_console_exporters is True
+
+    # Calling enable_instrumentation should not clobber the value
+    observability.enable_instrumentation()
+    assert observability.OBSERVABILITY_SETTINGS.enable_console_exporters is True
+
+
+def test_enable_instrumentation_with_sensitive_data_does_not_touch_console_exporters(monkeypatch):
+    """Test enable_console_exporters is untouched even when enable_sensitive_data is explicitly passed."""
+    import importlib
+
+    monkeypatch.setenv("ENABLE_INSTRUMENTATION", "false")
+    monkeypatch.delenv("ENABLE_CONSOLE_EXPORTERS", raising=False)
+    monkeypatch.delenv("ENABLE_SENSITIVE_DATA", raising=False)
+    monkeypatch.delenv("VS_CODE_EXTENSION_PORT", raising=False)
+    for key in [
+        "OTEL_EXPORTER_OTLP_ENDPOINT",
+        "OTEL_EXPORTER_OTLP_TRACES_ENDPOINT",
+        "OTEL_EXPORTER_OTLP_METRICS_ENDPOINT",
+        "OTEL_EXPORTER_OTLP_LOGS_ENDPOINT",
+    ]:
+        monkeypatch.delenv(key, raising=False)
+
+    observability = importlib.import_module("agent_framework.observability")
+    importlib.reload(observability)
+
+    # Set console exporters via configure_otel_providers
+    observability.configure_otel_providers(enable_console_exporters=True)
+    assert observability.OBSERVABILITY_SETTINGS.enable_console_exporters is True
+
+    # Calling enable_instrumentation with explicit sensitive_data should not clobber console exporters
+    observability.enable_instrumentation(enable_sensitive_data=True)
+    assert observability.OBSERVABILITY_SETTINGS.enable_console_exporters is True
+
+
+def test_enable_instrumentation_preserves_console_exporters_after_env_removed(monkeypatch):
+    """Test enable_instrumentation preserves enable_console_exporters when env var is removed after reload."""
+    import importlib
+
+    monkeypatch.setenv("ENABLE_INSTRUMENTATION", "false")
+    monkeypatch.setenv("ENABLE_CONSOLE_EXPORTERS", "true")
+
+    observability = importlib.import_module("agent_framework.observability")
+    importlib.reload(observability)
+
+    assert observability.OBSERVABILITY_SETTINGS.enable_console_exporters is True
+
+    # Remove the env var after reload
+    monkeypatch.delenv("ENABLE_CONSOLE_EXPORTERS", raising=False)
+
+    # enable_instrumentation should not reset the value
+    observability.enable_instrumentation()
+    assert observability.OBSERVABILITY_SETTINGS.enable_console_exporters is True
+
+
+def test_configure_otel_providers_reads_env_console_exporters(monkeypatch):
+    """Test configure_otel_providers re-reads ENABLE_CONSOLE_EXPORTERS from os.environ when not explicitly passed."""
+    import importlib
+
+    monkeypatch.setenv("ENABLE_INSTRUMENTATION", "false")
+    monkeypatch.delenv("VS_CODE_EXTENSION_PORT", raising=False)
+    monkeypatch.delenv("ENABLE_CONSOLE_EXPORTERS", raising=False)
+    for key in [
+        "OTEL_EXPORTER_OTLP_ENDPOINT",
+        "OTEL_EXPORTER_OTLP_TRACES_ENDPOINT",
+        "OTEL_EXPORTER_OTLP_METRICS_ENDPOINT",
+        "OTEL_EXPORTER_OTLP_LOGS_ENDPOINT",
+    ]:
+        monkeypatch.delenv(key, raising=False)
+
+    observability = importlib.import_module("agent_framework.observability")
+    importlib.reload(observability)
+
+    assert observability.OBSERVABILITY_SETTINGS.enable_console_exporters is False
+
+    # Simulate load_dotenv() setting env var after import
+    monkeypatch.setenv("ENABLE_CONSOLE_EXPORTERS", "true")
+
+    observability.configure_otel_providers()
+    assert observability.OBSERVABILITY_SETTINGS.enable_console_exporters is True
+
+
+def test_configure_otel_providers_explicit_console_exporters_overrides_env(monkeypatch):
+    """Test that explicit enable_console_exporters parameter overrides the environment variable."""
+    import importlib
+
+    monkeypatch.setenv("ENABLE_INSTRUMENTATION", "false")
+    monkeypatch.setenv("ENABLE_CONSOLE_EXPORTERS", "true")
+    monkeypatch.delenv("VS_CODE_EXTENSION_PORT", raising=False)
+    for key in [
+        "OTEL_EXPORTER_OTLP_ENDPOINT",
+        "OTEL_EXPORTER_OTLP_TRACES_ENDPOINT",
+        "OTEL_EXPORTER_OTLP_METRICS_ENDPOINT",
+        "OTEL_EXPORTER_OTLP_LOGS_ENDPOINT",
+    ]:
+        monkeypatch.delenv(key, raising=False)
+
+    observability = importlib.import_module("agent_framework.observability")
+    importlib.reload(observability)
+
+    # Explicit False should override the env var True
+    observability.configure_otel_providers(enable_console_exporters=False)
+    assert observability.OBSERVABILITY_SETTINGS.enable_console_exporters is False
+
+
 # region Test _to_otel_part content types


@@ -2781,3 +3048,143 @@ def test_get_meter_typeerror_fallback():
        meter = get_meter(name="test", attributes={"key": "val"})
        assert meter is not None
        assert call_count == 2
+
+
+# region Agent token usage aggregation
+
+
+@tool(name="get_weather", description="Get weather for a city", approval_mode="never_require")
+def _get_weather(city: str) -> str:
+    """Get weather for a city."""
+    return "Sunny, 72°F"
+
+
+@pytest.mark.parametrize("enable_sensitive_data", [False], indirect=True)
+async def test_agent_invoke_span_aggregates_usage_across_tool_calls(span_exporter: InMemorySpanExporter):
+    """The invoke_agent span should sum token usage from all chat completions in the function invocation loop."""
+    from tests.core.conftest import MockBaseChatClient
+
+    class _InstrumentedAgent(AgentTelemetryLayer, RawAgent):
+        pass
+
+    client = MockBaseChatClient()
+    client.run_responses = [
+        ChatResponse(
+            messages=Message(
+                role="assistant",
+                contents=[
+                    Content.from_function_call(call_id="call_1", name="get_weather", arguments='{"city": "Seattle"}')
+                ],
+            ),
+            usage_details=UsageDetails(input_token_count=2239, output_token_count=192),
+        ),
+        ChatResponse(
+            messages=Message(role="assistant", text="The weather in Seattle is sunny."),
+            usage_details=UsageDetails(input_token_count=2569, output_token_count=99),
+        ),
+    ]
+
+    agent = _InstrumentedAgent(client=client, name="test_agent", id="test_agent_id")
+
+    span_exporter.clear()
+    await agent.run(
+        messages="What is the weather in Seattle?",
+        options={"tools": [_get_weather], "tool_choice": "auto"},
+    )
+
+    spans = span_exporter.get_finished_spans()
+
+    invoke_spans = [s for s in spans if s.attributes.get(OtelAttr.OPERATION.value) == OtelAttr.AGENT_INVOKE_OPERATION]
+    assert len(invoke_spans) == 1
+    agent_span = invoke_spans[0]
+
+    chat_spans = [s for s in spans if s.attributes.get(OtelAttr.OPERATION.value) == OtelAttr.CHAT_COMPLETION_OPERATION]
+    assert len(chat_spans) == 2
+
+    # Individual chat spans retain their own usage
+    assert chat_spans[0].attributes.get(OtelAttr.INPUT_TOKENS) == 2239
+    assert chat_spans[0].attributes.get(OtelAttr.OUTPUT_TOKENS) == 192
+    assert chat_spans[1].attributes.get(OtelAttr.INPUT_TOKENS) == 2569
+    assert chat_spans[1].attributes.get(OtelAttr.OUTPUT_TOKENS) == 99
+
+    # The invoke_agent span must report the aggregate across all LLM round-trips
+    assert agent_span.attributes.get(OtelAttr.INPUT_TOKENS) == 2239 + 2569
+    assert agent_span.attributes.get(OtelAttr.OUTPUT_TOKENS) == 192 + 99
+
+
+@pytest.mark.parametrize("enable_sensitive_data", [False], indirect=True)
+async def test_agent_invoke_span_usage_single_call(span_exporter: InMemorySpanExporter):
+    """When only one chat completion occurs, the invoke_agent span usage equals that single call."""
+    from tests.core.conftest import MockBaseChatClient
+
+    class _InstrumentedAgent(AgentTelemetryLayer, RawAgent):
+        pass
+
+    client = MockBaseChatClient()
+    client.run_responses = [
+        ChatResponse(
+            messages=Message(role="assistant", text="Hello!"),
+            usage_details=UsageDetails(input_token_count=100, output_token_count=50),
+        ),
+    ]
+
+    agent = _InstrumentedAgent(client=client, name="test_agent", id="test_agent_id")
+
+    span_exporter.clear()
+    await agent.run(messages="Hi")
+
+    spans = span_exporter.get_finished_spans()
+    invoke_spans = [s for s in spans if s.attributes.get(OtelAttr.OPERATION.value) == OtelAttr.AGENT_INVOKE_OPERATION]
+    assert len(invoke_spans) == 1
+
+    assert invoke_spans[0].attributes.get(OtelAttr.INPUT_TOKENS) == 100
+    assert invoke_spans[0].attributes.get(OtelAttr.OUTPUT_TOKENS) == 50
+
+
+@pytest.mark.parametrize("enable_sensitive_data", [False], indirect=True)
+async def test_agent_invoke_span_aggregates_usage_on_max_iterations_exhaustion(span_exporter: InMemorySpanExporter):
+    """When the function invocation loop exhausts max_iterations, the final response aggregates usage
+    from all rounds."""
+    from tests.core.conftest import MockBaseChatClient
+
+    class _InstrumentedAgent(AgentTelemetryLayer, RawAgent):
+        pass
+
+    client = MockBaseChatClient(
+        function_invocation_configuration={"max_iterations": 1},
+    )
+    client.run_responses = [
+        # Iteration 0: model returns a tool call
+        ChatResponse(
+            messages=Message(
+                role="assistant",
+                contents=[
+                    Content.from_function_call(call_id="call_1", name="get_weather", arguments='{"city": "Seattle"}')
+                ],
+            ),
+            usage_details=UsageDetails(input_token_count=500, output_token_count=100),
+        ),
+        # Exhaustion path: consumed by tool_choice="none" final call (mock ignores usage)
+        ChatResponse(
+            messages=Message(role="assistant", text="placeholder"),
+            usage_details=UsageDetails(input_token_count=300, output_token_count=60),
+        ),
+    ]
+
+    agent = _InstrumentedAgent(client=client, name="test_agent", id="test_agent_id")
+
+    span_exporter.clear()
+    await agent.run(
+        messages="What is the weather in Seattle?",
+        options={"tools": [_get_weather], "tool_choice": "auto"},
+    )
+
+    spans = span_exporter.get_finished_spans()
+
+    invoke_spans = [s for s in spans if s.attributes.get(OtelAttr.OPERATION.value) == OtelAttr.AGENT_INVOKE_OPERATION]
+    assert len(invoke_spans) == 1
+    agent_span = invoke_spans[0]
+
+    # The invoke_agent span must aggregate usage from the in-loop call and the final exhaustion call
+    assert agent_span.attributes.get(OtelAttr.INPUT_TOKENS) == 500
+    assert agent_span.attributes.get(OtelAttr.OUTPUT_TOKENS) == 100
@@ -1,6 +1,7 @@
 # Copyright (c) Microsoft. All rights reserved.

 import base64
+import json
 from collections.abc import AsyncIterable, Sequence
 from dataclasses import dataclass
 from datetime import datetime, timezone
@@ -1710,6 +1711,47 @@ def test_content_roundtrip_preserves_compaction_annotation_dict() -> None:
    assert annotation[GROUP_TOKEN_COUNT_KEY] is None


+def test_content_from_dict_via_json() -> None:
+    """Test Content.from_dict with data parsed from a JSON string."""
+    data = json.loads(json.dumps({"type": "text", "text": "Hello world"}))
+    content = Content.from_dict(data)
+    assert content.type == "text"
+    assert content.text == "Hello world"
+
+
+def test_content_from_dict_roundtrip_via_json() -> None:
+    """Test Content.from_dict roundtrip via to_dict and json.dumps."""
+    original = Content.from_function_call(call_id="call1", name="my_func", arguments={"key": "value"})
+    data = json.loads(json.dumps(original.to_dict()))
+    restored = Content.from_dict(data)
+    assert restored.type == "function_call"
+    assert restored.call_id == "call1"
+    assert restored.name == "my_func"
+    assert restored.arguments == {"key": "value"}
+
+
+def test_content_to_dict_exclude_none() -> None:
+    """Test Content.to_dict excludes None fields by default."""
+    content = Content.from_text("Hello")
+    d = content.to_dict()
+    parsed = json.loads(json.dumps(d))
+    assert "uri" not in parsed
+
+    d_with_none = content.to_dict(exclude_none=False)
+    parsed_with_none = json.loads(json.dumps(d_with_none))
+    assert "uri" in parsed_with_none
+    assert parsed_with_none["uri"] is None
+
+
+def test_content_to_dict_exclude_fields() -> None:
+    """Test Content.to_dict with explicit field exclusion."""
+    content = Content.from_text("Hello")
+    d = content.to_dict(exclude={"text"})
+    parsed = json.loads(json.dumps(d))
+    assert "text" not in parsed
+    assert parsed["type"] == "text"
+
+
 def test_chat_response_roundtrip_preserves_compaction_annotation_dict() -> None:
    response = ChatResponse(
        messages=[
@@ -460,10 +460,10 @@ async def test_run_request_with_full_history_clears_service_session_id() -> None
    assert spy_agent._captured_service_session_id is None  # pyright: ignore[reportPrivateUsage]


-async def test_from_response_preserves_service_session_id() -> None:
-    """from_response hands off a prior agent's full conversation to the next executor.
-    The receiving executor's service_session_id is preserved so the API can continue
-    the conversation using previous_response_id."""
+async def test_from_response_clears_service_session_id_on_new_run() -> None:
+    """service_session_id set before a workflow run is cleared by the executor reset
+    that happens at the start of each run, preventing stale previous_response_id
+    from leaking between runs."""
    tool_agent = _ToolHistoryAgent(id="tool_agent2", name="ToolAgent", summary_text="Done.")
    tool_exec = AgentExecutor(tool_agent, id="tool_agent2")

@@ -477,4 +477,6 @@ async def test_from_response_preserves_service_session_id() -> None:
    result = await wf.run("start")
    assert result.get_outputs() is not None

-    assert spy_agent._captured_service_session_id == "resp_PREVIOUS_RUN"  # pyright: ignore[reportPrivateUsage]
+    # service_session_id is cleared at the start of run() to prevent stale
+    # previous_response_id from causing "No tool output found" errors on re-runs.
+    assert spy_agent._captured_service_session_id is None  # pyright: ignore[reportPrivateUsage]
@@ -92,9 +92,13 @@ exclude_dirs = ["tests"]
 executor.type = "uv"
 include = "../../shared_tasks.toml"

-[tool.poe.tasks]
-mypy = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_declarative"
-test = 'pytest -m "not integration" --cov=agent_framework_declarative --cov-report=term-missing:skip-covered tests'
+[tool.poe.tasks.mypy]
+help = "Run MyPy for this package."
+cmd = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_declarative"
+
+[tool.poe.tasks.test]
+help = "Run the default unit test suite for this package."
+cmd = 'pytest -m "not integration" --cov=agent_framework_declarative --cov-report=term-missing:skip-covered tests'

 [build-system]
 requires = ["flit-core >= 3.11,<4.0"]
@@ -98,9 +98,13 @@ exclude_dirs = ["tests"]
 executor.type = "uv"
 include = "../../shared_tasks.toml"

-[tool.poe.tasks]
-mypy = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_devui"
-test = 'pytest -m "not integration" --cov=agent_framework_devui --cov-report=term-missing:skip-covered tests'
+[tool.poe.tasks.mypy]
+help = "Run MyPy for this package."
+cmd = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_devui"
+
+[tool.poe.tasks.test]
+help = "Run the default unit test suite for this package."
+cmd = 'pytest -m "not integration" --cov=agent_framework_devui --cov-report=term-missing:skip-covered tests'

 [build-system]
 requires = ["flit-core >= 3.11,<4.0"]
@@ -1318,9 +1318,17 @@ class DurableAgentStateUnknownContent(DurableAgentStateContent):

    @staticmethod
    def from_unknown_content(content: Any) -> DurableAgentStateUnknownContent:
+        if isinstance(content, Content):
+            return DurableAgentStateUnknownContent(content=content.to_dict())
        return DurableAgentStateUnknownContent(content=content)

    def to_ai_content(self) -> Content:
        if not self.content:
            raise Exception("The content is missing and cannot be converted to valid AI content.")
+        content_value: Any = self.content
+        if isinstance(content_value, dict) and "type" in content_value:
+            try:
+                return Content.from_dict(cast(dict[str, Any], content_value))
+            except (ValueError, TypeError):
+                pass
        return Content(type=self.type, additional_properties={"content": self.content})  # type: ignore
@@ -97,9 +97,13 @@ exclude_dirs = ["tests"]
 executor.type = "uv"
 include = "../../shared_tasks.toml"

-[tool.poe.tasks]
-mypy = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_durabletask"
-test = 'pytest -m "not integration" --cov=agent_framework_durabletask --cov-report=term-missing:skip-covered tests'
+[tool.poe.tasks.mypy]
+help = "Run MyPy for this package."
+cmd = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_durabletask"
+
+[tool.poe.tasks.test]
+help = "Run the default unit test suite for this package."
+cmd = 'pytest -m "not integration" --cov=agent_framework_durabletask --cov-report=term-missing:skip-covered tests'

 [build-system]
 requires = ["flit-core >= 3.11,<4.0"]
@@ -2,16 +2,19 @@

 """Unit tests for DurableAgentState and related classes."""

+import json
 from datetime import datetime

 import pytest
-from agent_framework import UsageDetails
+from agent_framework import Content, Message, UsageDetails

 from agent_framework_durabletask._durable_agent_state import (
    DurableAgentState,
+    DurableAgentStateContent,
    DurableAgentStateMessage,
    DurableAgentStateRequest,
    DurableAgentStateTextContent,
+    DurableAgentStateUnknownContent,
    DurableAgentStateUsage,
 )
 from agent_framework_durabletask._models import RunRequest
@@ -373,5 +376,117 @@ class TestDurableAgentStateUsage:
        assert restored.get("total_token_count") == original.get("total_token_count")


+class TestDurableAgentStateUnknownContent:
+    """Test suite for DurableAgentStateUnknownContent serialization."""
+
+    def test_unknown_content_from_content_object_produces_serializable_dict(self) -> None:
+        """Test that from_unknown_content serializes Content objects to dicts."""
+        content = Content.from_mcp_server_tool_call(
+            call_id="call-1",
+            tool_name="search",
+            server_name="learn-mcp",
+            arguments={"query": "azure functions"},
+        )
+
+        unknown = DurableAgentStateUnknownContent.from_unknown_content(content)
+        result = unknown.to_dict()
+
+        # The content field should be a dict, not a Content object
+        assert isinstance(result["content"], dict)
+        assert result["content"]["type"] == "mcp_server_tool_call"
+
+    def test_unknown_content_to_dict_is_json_serializable(self) -> None:
+        """Test that to_dict output can be passed to json.dumps without error."""
+        content = Content.from_mcp_server_tool_result(
+            call_id="call-1",
+            output="Azure Functions documentation...",
+        )
+
+        unknown = DurableAgentStateUnknownContent.from_unknown_content(content)
+        result = unknown.to_dict()
+
+        # This must not raise TypeError
+        serialized = json.dumps(result)
+        assert serialized is not None
+
+    def test_unknown_content_round_trip_preserves_content(self) -> None:
+        """Test that Content objects survive serialization and deserialization."""
+        original = Content.from_mcp_server_tool_call(
+            call_id="call-1",
+            tool_name="fetch",
+            server_name="learn-mcp",
+            arguments={"url": "https://example.com"},
+        )
+
+        unknown = DurableAgentStateUnknownContent.from_unknown_content(original)
+        restored = unknown.to_ai_content()
+
+        assert restored.type == "mcp_server_tool_call"
+        assert restored.tool_name == "fetch"
+        assert restored.server_name == "learn-mcp"
+
+    def test_unknown_content_from_plain_dict_unchanged(self) -> None:
+        """Test that non-Content values are stored as-is."""
+        plain = {"some": "data"}
+
+        unknown = DurableAgentStateUnknownContent.from_unknown_content(plain)
+
+        assert unknown.content == {"some": "data"}
+
+    def test_unknown_content_to_ai_content_fallback_on_invalid_type_dict(self) -> None:
+        """Test that to_ai_content falls back when dict has 'type' but is not valid Content."""
+        invalid = {"type": "bogus_not_a_real_content_type", "extra": "stuff"}
+        unknown = DurableAgentStateUnknownContent(content=invalid)
+
+        result = unknown.to_ai_content()
+
+        assert result.type == "unknown"
+        assert result.additional_properties == {"content": invalid}
+
+    def test_from_ai_content_unknown_type_produces_serializable_state(self) -> None:
+        """Test that unknown content types in message conversion produce JSON-serializable state."""
+        content = Content.from_mcp_server_tool_call(
+            call_id="call-1",
+            tool_name="search",
+            server_name="learn-mcp",
+            arguments={"query": "create function app"},
+        )
+
+        durable_content = DurableAgentStateContent.from_ai_content(content)
+        data = durable_content.to_dict()
+
+        # Must be fully JSON-serializable
+        serialized = json.dumps(data)
+        assert serialized is not None
+
+    def test_state_with_mcp_content_is_json_serializable(self) -> None:
+        """Test that full DurableAgentState with MCP content can be serialized to JSON.
+
+        This reproduces the scenario from issue #4719 where agent state containing
+        MCP tool content could not be serialized by Azure Durable Functions.
+        """
+        state = DurableAgentState()
+        mcp_content = Content.from_mcp_server_tool_call(
+            call_id="call-1",
+            tool_name="search",
+            server_name="learn-mcp",
+            arguments={"query": "azure functions"},
+        )
+        message = DurableAgentStateMessage.from_chat_message(Message(role="assistant", contents=[mcp_content]))
+        state.data.conversation_history.append(
+            DurableAgentStateRequest(
+                correlation_id="test-mcp",
+                created_at=datetime.now(),
+                messages=[message],
+            )
+        )
+
+        state_dict = state.to_dict()
+
+        # This simulates what Azure Durable Functions does with entity state
+        serialized = json.dumps(state_dict)
+        assert serialized is not None
+
+
 if __name__ == "__main__":
    pytest.main([__file__, "-v", "--tb=short"])
@@ -84,9 +84,13 @@ exclude_dirs = ["tests"]
 executor.type = "uv"
 include = "../../shared_tasks.toml"

-[tool.poe.tasks]
-mypy = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_foundry_local"
-test = 'pytest -m "not integration" --cov=agent_framework_foundry_local --cov-report=term-missing:skip-covered tests'
+[tool.poe.tasks.mypy]
+help = "Run MyPy for this package."
+cmd = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_foundry_local"
+
+[tool.poe.tasks.test]
+help = "Run the default unit test suite for this package."
+cmd = 'pytest -m "not integration" --cov=agent_framework_foundry_local --cov-report=term-missing:skip-covered tests'

 [build-system]
 requires = ["flit-core >= 3.11,<4.0"]
@@ -85,14 +85,17 @@ exclude_dirs = ["tests"]
 executor.type = "uv"
 include = "../../shared_tasks.toml"

-[tool.poe.tasks]
-test = "pytest -m \"not integration\" --cov=agent_framework_github_copilot --cov-report=term-missing:skip-covered tests"
+[tool.poe.tasks.test]
+help = "Run the default unit test suite for this package."
+cmd = "pytest -m \"not integration\" --cov=agent_framework_github_copilot --cov-report=term-missing:skip-covered tests"

 [tool.poe.tasks.pyright]
+help = "Run Pyright for this package, skipping automatically on unsupported Python versions."
 shell = "python -c \"import sys; exit(0 if sys.version_info < (3,11) else 1)\" || pyright"
 interpreter = "posix"

 [tool.poe.tasks.mypy]
+help = "Run MyPy for this package, skipping automatically on unsupported Python versions."
 shell = "python -c \"import sys; exit(0 if sys.version_info < (3,11) else 1)\" || mypy --config-file $POE_ROOT/pyproject.toml agent_framework_github_copilot"
 interpreter = "posix"

@@ -71,10 +71,10 @@ uv run --directory packages/lab poe test
 uv run --directory packages/lab pytest -q -m "not integration"
 ```

-When you need to run package tasks from the repository root, use sequential mode to avoid launching all package tests in parallel:
+When you need to run lab tests from the repository root, scope the root task to the lab package:

 ```bash
-uv run poe test --seq
+uv run poe test -P lab
 ```

 Lightning observability tests intentionally exercise heavier tracing paths and are marked as `resource_intensive`:
@@ -146,17 +146,45 @@ exclude_dirs = ["gaia/tests", "lightning/tests", "tau2/tests"]
 [tool.poe]
 include = "../../shared_tasks.toml"

-[tool.poe.tasks]
-mypy-gaia = "mypy --config-file $POE_ROOT/pyproject.toml gaia/agent_framework_lab_gaia"
-mypy-lightning = "mypy --config-file $POE_ROOT/pyproject.toml lightning/agent_framework_lab_lightning"
-mypy-tau2 = "mypy --config-file $POE_ROOT/pyproject.toml tau2/agent_framework_lab_tau2"
-mypy = ["mypy-gaia", "mypy-lightning", "mypy-tau2"]
-test = 'pytest -m "not integration and not resource_intensive" --cov-report=term-missing:skip-covered --junitxml=test-results.xml'
-test-gaia = "pytest gaia/tests --cov=agent_framework_lab_gaia --cov-report=term-missing:skip-covered"
-test-lightning = "pytest lightning/tests --cov=agent_framework_lab_lightning --cov-report=term-missing:skip-covered"
-test-tau2 = "pytest tau2/tests --cov=agent_framework_lab_tau2 --cov-report=term-missing:skip-covered"
-build = "echo 'Skipping build'"
-publish = "echo 'Skipping publish'"
+[tool.poe.tasks.mypy-gaia]
+help = "Run MyPy for the lab GAIA package."
+cmd = "mypy --config-file $POE_ROOT/pyproject.toml gaia/agent_framework_lab_gaia"
+
+[tool.poe.tasks.mypy-lightning]
+help = "Run MyPy for the lab Lightning package."
+cmd = "mypy --config-file $POE_ROOT/pyproject.toml lightning/agent_framework_lab_lightning"
+
+[tool.poe.tasks.mypy-tau2]
+help = "Run MyPy for the lab Tau2 package."
+cmd = "mypy --config-file $POE_ROOT/pyproject.toml tau2/agent_framework_lab_tau2"
+
+[tool.poe.tasks.mypy]
+help = "Run MyPy across all lab subpackages."
+sequence = ["mypy-gaia", "mypy-lightning", "mypy-tau2"]
+
+[tool.poe.tasks.test]
+help = "Run the default lab unit test suite."
+cmd = 'pytest -m "not integration and not resource_intensive" --cov-report=term-missing:skip-covered --junitxml=test-results.xml'
+
+[tool.poe.tasks.test-gaia]
+help = "Run the GAIA lab test suite."
+cmd = "pytest gaia/tests --cov=agent_framework_lab_gaia --cov-report=term-missing:skip-covered"
+
+[tool.poe.tasks.test-lightning]
+help = "Run the Lightning lab test suite."
+cmd = "pytest lightning/tests --cov=agent_framework_lab_lightning --cov-report=term-missing:skip-covered"
+
+[tool.poe.tasks.test-tau2]
+help = "Run the Tau2 lab test suite."
+cmd = "pytest tau2/tests --cov=agent_framework_lab_tau2 --cov-report=term-missing:skip-covered"
+
+[tool.poe.tasks.build]
+help = "Skip build for the lab package."
+cmd = "echo 'Skipping build'"
+
+[tool.poe.tasks.publish]
+help = "Skip publish for the lab package."
+cmd = "echo 'Skipping publish'"

 [tool.pytest.ini_options]
 pythonpath = ["."]
@@ -85,9 +85,13 @@ exclude_dirs = ["tests"]
 executor.type = "uv"
 include = "../../shared_tasks.toml"

-[tool.poe.tasks]
-mypy = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_mem0"
-test = 'pytest -m "not integration" --cov=agent_framework_mem0 --cov-report=term-missing:skip-covered tests'
+[tool.poe.tasks.mypy]
+help = "Run MyPy for this package."
+cmd = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_mem0"
+
+[tool.poe.tasks.test]
+help = "Run the default unit test suite for this package."
+cmd = 'pytest -m "not integration" --cov=agent_framework_mem0 --cov-report=term-missing:skip-covered tests'

 [build-system]
 requires = ["flit-core >= 3.11,<4.0"]
@@ -88,9 +88,13 @@ exclude_dirs = ["tests"]
 executor.type = "uv"
 include = "../../shared_tasks.toml"

-[tool.poe.tasks]
-mypy = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_ollama"
-test = 'pytest -m "not integration" --cov=agent_framework_ollama --cov-report=term-missing:skip-covered tests'
+[tool.poe.tasks.mypy]
+help = "Run MyPy for this package."
+cmd = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_ollama"
+
+[tool.poe.tasks.test]
+help = "Run the default unit test suite for this package."
+cmd = 'pytest -m "not integration" --cov=agent_framework_ollama --cov-report=term-missing:skip-covered tests'

 [tool.uv.build-backend]
 module-name = "agent_framework_ollama"
@@ -83,9 +83,13 @@ exclude_dirs = ["tests"]
 executor.type = "uv"
 include = "../../shared_tasks.toml"

-[tool.poe.tasks]
-mypy = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_orchestrations"
-test = 'pytest -m "not integration" --cov=agent_framework_orchestrations --cov-report=term-missing:skip-covered -n auto --dist worksteal tests'
+[tool.poe.tasks.mypy]
+help = "Run MyPy for this package."
+cmd = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_orchestrations"
+
+[tool.poe.tasks.test]
+help = "Run the default unit test suite for this package."
+cmd = 'pytest -m "not integration" --cov=agent_framework_orchestrations --cov-report=term-missing:skip-covered -n auto --dist worksteal tests'

 [build-system]
 requires = ["flit-core >= 3.11,<4.0"]
@@ -84,9 +84,13 @@ exclude_dirs = ["tests"]
 executor.type = "uv"
 include = "../../shared_tasks.toml"

-[tool.poe.tasks]
-mypy = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_purview"
-test = 'pytest -m "not integration" --cov=agent_framework_purview --cov-report=term-missing:skip-covered tests'
+[tool.poe.tasks.mypy]
+help = "Run MyPy for this package."
+cmd = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_purview"
+
+[tool.poe.tasks.test]
+help = "Run the default unit test suite for this package."
+cmd = 'pytest -m "not integration" --cov=agent_framework_purview --cov-report=term-missing:skip-covered tests'

 [build-system]
 requires = ["flit-core >= 3.9,<4.0"]
@@ -87,9 +87,13 @@ exclude_dirs = ["tests"]
 executor.type = "uv"
 include = "../../shared_tasks.toml"

-[tool.poe.tasks]
-mypy = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_redis"
-test = 'pytest -m "not integration" --cov=agent_framework_redis --cov-report=term-missing:skip-covered tests'
+[tool.poe.tasks.mypy]
+help = "Run MyPy for this package."
+cmd = "mypy --config-file $POE_ROOT/pyproject.toml agent_framework_redis"
+
+[tool.poe.tasks.test]
+help = "Run the default unit test suite for this package."
+cmd = 'pytest -m "not integration" --cov=agent_framework_redis --cov-report=term-missing:skip-covered tests'

 [build-system]
 requires = ["flit-core >= 3.11,<4.0"]
@@ -225,94 +225,137 @@ exclude_dirs = ["tests", "scripts", "samples"]
 [tool.poe]
 executor.type = "uv"

-[tool.poe.tasks]
-markdown-code-lint = "uv run python scripts/check_md_code_blocks.py 'README.md' './packages/**/README.md' './samples/**/*.md' --exclude cookiecutter-agent-framework-lab --exclude tau2 --exclude 'packages/devui/frontend' --exclude context_providers/azure_ai_search"
-prek-install = "prek install --overwrite"
-install = "uv sync --all-packages --all-extras --dev --frozen --prerelease=if-necessary-or-explicit"
-test = "python scripts/run_tasks_in_packages_if_exists.py test"
-fmt = "python scripts/run_tasks_in_packages_if_exists.py fmt"
-format.ref = "fmt"
-lint = "python scripts/run_tasks_in_packages_if_exists.py lint"
-samples-lint = "ruff check samples --fix --exclude samples/autogen-migration,samples/semantic-kernel-migration --ignore E501,ASYNC,B901,TD002"
-pyright = "python scripts/run_tasks_in_packages_if_exists.py pyright"
-mypy = "python scripts/run_tasks_in_packages_if_exists.py mypy"
-typing = "python scripts/run_tasks_in_packages_if_exists.py mypy pyright"
-samples-syntax.shell = "pyright -p $(python -c \"import sys; print('pyrightconfig.samples.py310.json' if sys.version_info < (3,11) else 'pyrightconfig.samples.json')\") --warnings"
-samples-syntax.interpreter = "posix"
-# cleaning
-clean-dist-packages = "python scripts/run_tasks_in_packages_if_exists.py clean-dist"
-clean-dist-meta = "rm -rf dist"
-clean-dist = ["clean-dist-packages", "clean-dist-meta"]
-# build and publish
-build-packages = "python scripts/run_tasks_in_packages_if_exists.py build"
-build-meta = "python -m flit build"
-build = ["build-packages", "build-meta"]
-publish = "uv publish"
-# combined checks
-check-packages = "python scripts/run_tasks_in_packages_if_exists.py fmt lint pyright"
-check = ["check-packages", "samples-lint", "samples-syntax", "test", "markdown-code-lint"]
-
-[tool.poe.tasks.all-tests-cov]
-cmd = """
-pytest --import-mode=importlib
-m "not integration"
--cov=agent_framework
--cov=agent_framework_core
--cov=agent_framework_a2a
--cov=agent_framework_ag_ui
--cov=agent_framework_anthropic
--cov=agent_framework_azure_ai
--cov=agent_framework_azure_ai_search
--cov=agent_framework_azurefunctions
--cov=agent_framework_chatkit
--cov=agent_framework_copilotstudio
--cov=agent_framework_mem0
--cov=agent_framework_purview
--cov=agent_framework_redis
--cov=agent_framework_orchestrations
--cov=agent_framework_declarative
--cov-config=pyproject.toml
--cov-report=term-missing:skip-covered
--ignore-glob=packages/lab/**
--ignore-glob=packages/devui/**
-rs
-n logical --dist worksteal
-    packages/**/tests
-"""
-
-[tool.poe.tasks.all-tests]
-cmd = """
-pytest --import-mode=importlib
-m "not integration"
--ignore-glob=packages/lab/**
--ignore-glob=packages/devui/**
-rs
-n logical --dist worksteal
-    packages/**/tests
-"""
-
-[tool.poe.tasks.venv]
-cmd = "uv venv --clear --python $python"
-args = [{ name = "python", default = "3.13", options = ['-p', '--python'] }]
+# Workspace setup
+[tool.poe.tasks.install]
+help = "Install all workspace packages, extras, and dev dependencies from the lockfile."
+cmd = "uv sync --all-packages --all-extras --dev --frozen --prerelease=if-necessary-or-explicit"

 [tool.poe.tasks.setup]
+help = "Create the workspace virtual environment for -P/--python, install dependencies, and install prek hooks."
 sequence = [
    { ref = "venv --python $python"},
    { ref = "install" },
    { ref = "prek-install" }
 ]
-args = [{ name = "python", default = "3.13", options = ['-p', '--python'] }]
+args = [{ name = "python", default = "3.13", options = ['-P', '-p', '--python'] }]

+[tool.poe.tasks.venv]
+help = "Create or recreate the workspace virtual environment for -P/--python."
+cmd = "uv venv --clear --python $python"
+args = [{ name = "python", default = "3.13", options = ['-P', '-p', '--python'] }]
+
+[tool.poe.tasks.prek-install]
+help = "Install or refresh the prek git hooks."
+cmd = "prek install --overwrite"
+
+# Syntax, typing, and validation
+[tool.poe.tasks.syntax]
+help = "Run Ruff formatting and Ruff checks for -P/--package packages, or use -S/--samples; add -F/--format or -C/--check to narrow the mode."
+cmd = "python scripts/workspace_poe_tasks.py syntax"
+
+[tool.poe.tasks.fmt]
+help = "DEPRECATED: Use `syntax --format` instead."
+cmd = "python scripts/workspace_poe_tasks.py syntax --format"
+
+[tool.poe.tasks.format]
+help = "DEPRECATED: Use `syntax --format` instead."
+cmd = "python scripts/workspace_poe_tasks.py syntax --format"
+
+[tool.poe.tasks.lint]
+help = "DEPRECATED: Use `syntax --check` instead."
+cmd = "python scripts/workspace_poe_tasks.py syntax --check"
+
+[tool.poe.tasks.samples-lint]
+help = "DEPRECATED: Use `syntax --samples --check` instead."
+cmd = "python scripts/workspace_poe_tasks.py syntax --samples --check"
+
+[tool.poe.tasks.pyright]
+help = "Run Pyright for -P/--package packages, use -A/--all for one aggregate sweep, or use -S/--samples for sample checks."
+cmd = "python scripts/workspace_poe_tasks.py pyright"
+
+[tool.poe.tasks.mypy]
+help = "Run MyPy for -P/--package packages, or use -A/--all for one aggregate sweep."
+cmd = "python scripts/workspace_poe_tasks.py mypy"
+
+[tool.poe.tasks.typing]
+help = "Run both MyPy and Pyright for -P/--package packages, or use -A/--all for aggregate mode."
+cmd = "python scripts/workspace_poe_tasks.py typing"
+
+[tool.poe.tasks.samples-syntax]
+help = "DEPRECATED: Use `pyright --samples` instead."
+cmd = "python scripts/workspace_poe_tasks.py pyright --samples"
+
+[tool.poe.tasks.check-packages]
+help = "Run `syntax` and `pyright` for -P/--package packages."
+cmd = "python scripts/workspace_poe_tasks.py check-packages"
+
+[tool.poe.tasks.check]
+help = "Run package syntax, pyright, and tests for -P/--package packages; without -P also include sample checks and markdown code lint, or use -S/--samples for sample-only checks."
+cmd = "python scripts/workspace_poe_tasks.py check"
+
+[tool.poe.tasks.markdown-code-lint]
+help = "Lint Python code blocks embedded in README and sample markdown files."
+cmd = "uv run python scripts/check_md_code_blocks.py 'README.md' './packages/**/README.md' './samples/**/*.md' --exclude cookiecutter-agent-framework-lab --exclude tau2 --exclude 'packages/devui/frontend' --exclude context_providers/azure_ai_search"
+
+# Testing
+[tool.poe.tasks.test]
+help = "Run tests for -P/--package packages, or use -A/--all for one aggregate sweep; add -C/--cov for coverage."
+cmd = "python scripts/workspace_poe_tasks.py test"
+
+[tool.poe.tasks.all-tests]
+help = "DEPRECATED: Use `test --all` instead."
+cmd = "python scripts/workspace_poe_tasks.py test --all"
+
+[tool.poe.tasks.all-tests-cov]
+help = "DEPRECATED: Use `test --all --cov` instead."
+cmd = "python scripts/workspace_poe_tasks.py test --all --cov"
+
+# Build and publishing
+[tool.poe.tasks._clean-dist-packages]
+cmd = "python scripts/workspace_poe_tasks.py clean-dist"
+
+[tool.poe.tasks._clean-dist-meta]
+cmd = "rm -rf dist"
+
+[tool.poe.tasks.clean-dist]
+help = "Remove generated dist artifacts for -P/--package packages and the root meta package."
+sequence = [
+    { ref = "_clean-dist-packages --package ${project}" },
+    { ref = "_clean-dist-meta" },
+]
+args = [{ name = "project", default = "*", options = ["-P", "--package"] }]
+
+[tool.poe.tasks._build-packages]
+cmd = "python scripts/workspace_poe_tasks.py build"
+
+[tool.poe.tasks._build-meta]
+cmd = "python -m flit build"
+
+[tool.poe.tasks.build]
+help = "Build -P/--package packages and the root meta package."
+sequence = [
+    { ref = "_build-packages --package ${project}" },
+    { ref = "_build-meta" },
+]
+args = [{ name = "project", default = "*", options = ["-P", "--package"] }]
+
+[tool.poe.tasks.publish]
+help = "Publish built distributions with uv."
+cmd = "uv publish"
+
+# Dependency maintenance
 [tool.poe.tasks.upgrade-dev-dependency-pins]
+help = "Repin the workspace dev dependency versions used in pyproject.toml."
 cmd = "python -m scripts.dependencies.upgrade_dev_dependencies"

-[tool.poe.tasks.upgrade-lockfile]
+[tool.poe.tasks._upgrade-lockfile]
 cmd = "uv lock --upgrade"

 [tool.poe.tasks.upgrade-dev-dependencies]
+help = "Repin dev dependencies, refresh uv.lock, reinstall, and rerun validation commands."
 sequence = [
    { ref = "upgrade-dev-dependency-pins" },
-    { ref = "upgrade-lockfile" },
+    { ref = "_upgrade-lockfile" },
    { ref = "install" },
    { ref = "check" },
    { ref = "typing" },
@@ -320,17 +363,20 @@ sequence = [
 ]

 [tool.poe.tasks.add-dependency-to-project]
-cmd = "uv add --package ${project} ${dependency}"
+help = "Add a dependency to a -P/--package workspace package selected by short name such as `core`."
+cmd = "python -m scripts.dependencies.add_dependency_to_project --package ${project} --dependency ${dependency}"
 args = [
-    { name = "project", options = ["-p", "--project"] },
-    { name = "dependency", options = ["-d", "--dependency"] },
+    { name = "project", options = ["-P", "--package"] },
+    { name = "dependency", options = ["-D", "-d", "--dependency"] },
 ]

 [tool.poe.tasks.validate-dependency-bounds-test]
+help = "Run workspace dependency-bound validation in test mode, optionally scoped with -P/--package short names such as `core`."
 shell = "python -m scripts.dependencies.validate_dependency_bounds --mode test --package \"$project\""
-args = [{ name = "project", default = "*", options = ["-p", "--project"] }]
+args = [{ name = "project", default = "*", options = ["-P", "--package"] }]

 [tool.poe.tasks.validate-dependency-bounds-project]
+help = "Validate lower and upper dependency bounds for a -P/--package workspace package, optionally narrowed with -M/--mode and -D/--dependency."
 shell = """
 command=(python -m scripts.dependencies.validate_dependency_bounds --mode "${mode}" --package "${project}")
 if [ -n "${dependency}" ]; then
@@ -340,85 +386,22 @@ fi
 """
 interpreter = "bash"
 args = [
-    { name = "mode", default = "both", options = ["-m", "--mode"] },
-    { name = "project", default = "*", options = ["-p", "--project"] },
-    { name = "dependency", default = "", options = ["-d", "--dependency"] },
+    { name = "mode", default = "both", options = ["-M", "-m", "--mode"] },
+    { name = "project", default = "*", options = ["-P", "--package"] },
+    { name = "dependency", default = "", options = ["-D", "-d", "--dependency"] },
 ]

 [tool.poe.tasks.add-dependency-and-validate-bounds]
+help = "Add a dependency to a -P/--package workspace package selected by short name such as `core`, then validate its dependency bounds with -D/--dependency."
 sequence = [
-    { ref = "add-dependency-to-project --project ${project} --dependency ${dependency}" },
-    { ref = "validate-dependency-bounds-project --mode both --project ${project} --dependency ${dependency}" },
+    { ref = "add-dependency-to-project --package ${project} --dependency ${dependency}" },
+    { ref = "validate-dependency-bounds-project --mode both --package ${project} --dependency ${dependency}" },
 ]
 args = [
-    { name = "project", options = ["-p", "--project"] },
-    { name = "dependency", options = ["-d", "--dependency"] },
+    { name = "project", options = ["-P", "--package"] },
+    { name = "dependency", options = ["-D", "-d", "--dependency"] },
 ]

-[tool.poe.tasks.prek-pyright]
-cmd = "uv run python scripts/run_tasks_in_changed_packages.py pyright --files ${files}"
-args = [{ name = "files", default = ".", positional = true, multiple = true }]
-
-[tool.poe.tasks.prek-check-packages]
-cmd = "uv run python scripts/run_tasks_in_changed_packages.py fmt lint pyright --files ${files}"
-args = [{ name = "files", default = ".", positional = true, multiple = true }]
-
-[tool.poe.tasks.prek-markdown-code-lint]
-cmd = """uv run python scripts/check_md_code_blocks.py ${files} --no-glob
-    --exclude cookiecutter-agent-framework-lab --exclude tau2
-    --exclude packages/devui/frontend --exclude context_providers/azure_ai_search"""
-args = [{ name = "files", default = ".", positional = true, multiple = true }]
-
-[tool.poe.tasks.prek-samples-check]
-shell = """
-HAS_SAMPLES=false
-for f in ${files}; do
-    case "$f" in
-        samples/*) HAS_SAMPLES=true; break ;;
-    esac
-done
-if [ "$HAS_SAMPLES" = true ]; then
-    echo "Sample files changed, running samples checks..."
-    uv run ruff check samples --fix --exclude samples/autogen-migration,samples/semantic-kernel-migration --ignore E501,ASYNC,B901,TD002
-    uv run pyright -p pyrightconfig.samples.json --warnings
-else
-    echo "No sample files changed, skipping samples checks"
-fi
-"""
-interpreter = "bash"
-args = [{ name = "files", default = ".", positional = true, multiple = true }]
-
-
-[tool.poe.tasks.ci-mypy]
-shell = """
-# Try multiple strategies to get changed files
-if [ -n "$GITHUB_BASE_REF" ]; then
-    # In GitHub Actions PR context
-    git fetch origin $GITHUB_BASE_REF --depth=1 2>/dev/null || true
-    CHANGED_FILES=$(git diff --name-only origin/$GITHUB_BASE_REF...HEAD -- . 2>/dev/null || \
-                    git diff --name-only FETCH_HEAD...HEAD -- . 2>/dev/null || \
-                    git diff --name-only HEAD^...HEAD -- . 2>/dev/null || \
-                    echo ".")
-else
-    # Local development
-    CHANGED_FILES=$(git diff --name-only origin/main...HEAD -- . 2>/dev/null || \
-                    git diff --name-only main...HEAD -- . 2>/dev/null || \
-                    git diff --name-only HEAD~1 -- . 2>/dev/null || \
-                    echo ".")
-fi
-echo "Changed files: $CHANGED_FILES"
-uv run python scripts/run_tasks_in_changed_packages.py mypy --files $CHANGED_FILES
-"""
-interpreter = "bash"
-
-[tool.poe.tasks.prek-check]
-sequence = [
-    { ref = "prek-check-packages ${files}" },
-    { ref = "prek-markdown-code-lint ${files}" },
-    { ref = "prek-samples-check ${files}" }
-]
-args = [{ name = "files", default = ".", positional = true, multiple = true }]
-
 [tool.setuptools.packages.find]
 where = ["packages"]
 include = ["agent_framework**"]
@@ -29,7 +29,9 @@ async def main() -> None:
    client = OpenAIChatClient()

    try:
-        task = asyncio.create_task(client.get_response(messages=[Message(role="user", text="Tell me a fantasy story.")]))
+        task = asyncio.create_task(
+            client.get_response(messages=[Message(role="user", text="Tell me a fantasy story.")])
+        )
        await asyncio.sleep(1)
        task.cancel()
        await task
@@ -94,9 +94,7 @@ class EchoingChatClient(BaseChatClient[OptionsT]):
            response_text = f"{response_text} {suffix}"
        stream_delay_seconds = float(options.get("stream_delay_seconds", 0.05))

-        response_message = Message(
-            role="assistant", contents=[Content.from_text(response_text)]
-        )
+        response_message = Message(role="assistant", text=response_text)

        response = ChatResponse(
            messages=[response_message],
@@ -27,15 +27,9 @@ class KeepLastUserTurnStrategy:
            group_annotation = message.additional_properties.get(GROUP_ANNOTATION_KEY)
            group_id = group_annotation.get("id") if isinstance(group_annotation, dict) else None
            kind = group_annotation.get("kind") if isinstance(group_annotation, dict) else None
-            if (
-                isinstance(group_id, str)
-                and isinstance(kind, str)
-                and group_id not in group_kinds
-            ):
+            if isinstance(group_id, str) and isinstance(kind, str) and group_id not in group_kinds:
                group_kinds[group_id] = kind
-        user_group_ids = [
-            group_id for group_id in group_ids if group_kinds.get(group_id) == "user"
-        ]
+        user_group_ids = [group_id for group_id in group_ids if group_kinds.get(group_id) == "user"]
        if not user_group_ids:
            return False
        keep_user_group_id = user_group_ids[-1]
@@ -33,9 +33,7 @@ Key components:
 class TiktokenTokenizer(TokenizerProtocol):
    """TokenizerProtocol implementation backed by tiktoken's o200k_base (gpt-4.1 and up default) encoding."""

-    def __init__(
-        self, *, encoding_name: str = "o200k_base", model_name: str | None = None
-    ) -> None:
+    def __init__(self, *, encoding_name: str = "o200k_base", model_name: str | None = None) -> None:
        if model_name is not None:
            self._encoding = tiktoken.encoding_for_model(model_name)
        else:
@@ -62,10 +60,7 @@ def _build_messages() -> list[Message]:
        ),
        Message(
            role="user",
-            text=(
-                "Now provide a detailed checklist with owners, rollback "
-                "gates, and validation criteria."
-            ),
+            text=("Now provide a detailed checklist with owners, rollback gates, and validation criteria."),
        ),
        Message(
            role="assistant",
@@ -37,9 +37,7 @@ def get_weather(
    """Get the weather for a given location."""
    conditions = ["sunny", "cloudy", "rainy", "stormy"]
    temperature = 53
-    return (
-        f"The weather in {location} is {conditions[0]} with a high of {temperature}°C."
-    )
+    return f"The weather in {location} is {conditions[0]} with a high of {temperature}°C."


@tool(approval_mode="never_require")
@@ -68,9 +66,7 @@ class AddExclamation(Executor):
    """Add exclamation mark to text."""

    @handler
-    async def add_exclamation(
-        self, text: str, ctx: WorkflowContext[Never, str]
-    ) -> None:
+    async def add_exclamation(self, text: str, ctx: WorkflowContext[Never, str]) -> None:
        """Add exclamation and yield as workflow output."""
        result = f"{text}!"
        await ctx.yield_output(result)
@@ -0,0 +1,68 @@
+# Copyright (c) Microsoft. All rights reserved.
+
+"""Evaluate an agent with local checks — no API keys needed.
+
+Demonstrates the simplest evaluation workflow:
+1. Define checks using the @evaluator decorator
+2. Run evaluate_agent() which calls agent.run() under the covers
+3. Assert results in CI or inspect interactively
+
+Usage:
+    uv run python samples/02-agents/evaluation/evaluate_agent.py
+"""
+
+import asyncio
+
+from agent_framework import (
+    Agent,
+    LocalEvaluator,
+    evaluate_agent,
+    evaluator,
+    keyword_check,
+)
+
+
+# A custom check — parameter names determine what data you receive
+@evaluator
+def is_helpful(response: str) -> bool:
+    """Check the response isn't empty or a refusal."""
+    refusals = ["i can't", "i'm not able", "i don't know"]
+    return len(response) > 10 and not any(r in response.lower() for r in refusals)
+
+
+async def main():
+    agent = Agent(
+        model="gpt-4o-mini",
+        instructions="You are a helpful weather assistant.",
+    )
+
+    # Combine built-in and custom checks
+    local = LocalEvaluator(
+        keyword_check("weather"),  # response must mention "weather"
+        is_helpful,  # custom check
+    )
+
+    # evaluate_agent() calls agent.run() for each query, then evaluates
+    results = await evaluate_agent(
+        agent=agent,
+        queries=[
+            "What's the weather like in Seattle?",
+            "Will it rain in London tomorrow?",
+            "What should I wear for 30°C weather?",
+        ],
+        evaluators=local,
+    )
+
+    for r in results:
+        print(f"{r.provider}: {r.passed}/{r.total} passed")
+        for item in r.items:
+            print(f"  [{item.status}] Q: {item.input_text[:50]}  A: {item.output_text[:50]}...")
+            for score in item.scores:
+                print(f"    {score.name}: {'✓' if score.passed else '✗'}")
+
+    # Use in CI: will raise AssertionError if any check fails
+    # results[0].assert_passed()
+
+
+if __name__ == "__main__":
+    asyncio.run(main())
@@ -0,0 +1,64 @@
+# Copyright (c) Microsoft. All rights reserved.
+
+"""Evaluate an agent with expected outputs and tool call checks.
+
+Demonstrates ground-truth comparison and tool usage evaluation:
+1. Provide expected outputs alongside queries
+2. Use built-in tool_calls_present for tool verification
+3. Combine multiple evaluation criteria
+
+Usage:
+    uv run python samples/02-agents/evaluation/evaluate_with_expected.py
+"""
+
+import asyncio
+
+from agent_framework import (
+    Agent,
+    LocalEvaluator,
+    evaluate_agent,
+    evaluator,
+    tool_calls_present,
+)
+
+
+@evaluator
+def response_matches_expected(response: str, expected_output: str) -> float:
+    """Score based on word overlap with expected output."""
+    if not expected_output:
+        return 1.0
+    response_words = set(response.lower().split())
+    expected_words = set(expected_output.lower().split())
+    return len(response_words & expected_words) / max(len(expected_words), 1)
+
+
+async def main():
+    agent = Agent(
+        model="gpt-4o-mini",
+        instructions="You are a math tutor. Answer concisely.",
+    )
+
+    local = LocalEvaluator(
+        response_matches_expected,
+        tool_calls_present,  # verifies expected tools were called
+    )
+
+    results = await evaluate_agent(
+        agent=agent,
+        queries=["What is 2 + 2?", "What is the square root of 144?"],
+        expected_output=["4", "12"],
+        expected_tool_calls=[
+            [],  # no tools expected for simple math
+            [],
+        ],
+        evaluators=local,
+    )
+
+    for r in results:
+        print(f"{r.provider}: {r.passed}/{r.total} passed")
+        for item in r.items:
+            print(f"  [{item.status}] {item.input_text} → {item.output_text[:80]}")
+
+
+if __name__ == "__main__":
+    asyncio.run(main())
@@ -19,9 +19,7 @@ from copilot.generated.session_events import PermissionRequest
 from copilot.types import PermissionRequestResult


-def prompt_permission(
-    request: PermissionRequest, context: dict[str, str]
-) -> PermissionRequestResult:
+def prompt_permission(request: PermissionRequest, context: dict[str, str]) -> PermissionRequestResult:
    """Permission handler that prompts the user for approval."""
    print(f"\n[Permission Request: {request.kind}]")

@@ -75,7 +75,9 @@ unit_converter_skill = Skill(
 # ---------------------------------------------------------------------------
 # 2. Dynamic Resources — callable function via @skill.resource
 # ---------------------------------------------------------------------------
-@unit_converter_skill.resource(name="conversion-policy", description="Current conversion formatting and rounding policy")
+@unit_converter_skill.resource(
+    name="conversion-policy", description="Current conversion formatting and rounding policy"
+)
 def conversion_policy(**kwargs: Any) -> Any:
    """Return the current conversion policy.

@@ -148,8 +150,7 @@ async def main() -> None:
        print("Converting units")
        print("-" * 60)
        response = await agent.run(
-            "How many kilometers is a marathon (26.2 miles)? "
-            "And how many pounds is 75 kilograms?",
+            "How many kilometers is a marathon (26.2 miles)? And how many pounds is 75 kilograms?",
            precision=2,
        )
        print(f"Agent: {response}\n")
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
alliscodeandCopilot	b0a15914bf	fix: resolve mypy redundant-cast errors while keeping pyright happy Use cast(list[Any], x) with type: ignore[redundant-cast] comments to satisfy both mypy (which considers casting Any redundant) and pyright strict mode (which needs explicit casts to narrow Unknown types). Also fix evaluator decorator check_name type annotation to be explicitly str, resolving mypy str\|Any\|None mismatch. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>	2026-03-20 15:25:07 -07:00
alliscodeandCopilot	45527eed29	Foundry Evals integration for Python Merged and refactored eval module per Eduard's PR review: - Merge _eval.py + _local_eval.py into single _evaluation.py - Convert EvalItem from dataclass to regular class - Rename to_dict() to to_eval_data() - Convert _AgentEvalData to TypedDict - Simplify check system: unified async pattern with isawaitable - Parallelize checks and evaluators with asyncio.gather - Add all/any mode to tool_called_check - Fix bool(passed) truthy bug in _coerce_result - Remove deprecated function_evaluator/async_function_evaluator aliases - Remove _MinimalAgent, tighten evaluate_agent signature - Set self.name in __init__ (LocalEvaluator, FoundryEvals) - Limit FoundryEvals to AsyncOpenAI only - Type project_client as AIProjectClient - Remove NotImplementedError continuous eval code - Add evaluation samples in 02-agents/ and 03-workflows/ - Update all imports and tests (167 passing) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>	2026-03-20 14:24:21 -07:00
westeyandGitHub	100086a276	Add docker-in-docker feature to dev container (#4794 )	2026-03-19 19:18:46 +00:00
westey GitHub Copilot Autofix powered by AI	fc6721ca8e	.NET: Trim src references and add utility to enforce (#4693 ) * Trim src references and add utility to enforce * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>	2026-03-19 10:57:43 +00:00
Eduard van Valkenburg GitHub Copilot Copilot	4b21f38650	Python: Fix A2AAgent to invoke context providers before and after run (#4757 ) * Fix A2AAgent to invoke context providers before and after run A2AAgent.run() bypassed the context provider lifecycle (before_run/after_run) that BaseAgent defines as a contract for all agents. This caused A2AAgent to violate the semantic definition of BaseAgent, resulting in inconsistency with other agent implementations. The fix follows the same pattern used by WorkflowAgent: - Create SessionContext and run before_run on all context providers before processing the A2A stream - Collect response updates and run after_run on all context providers after the stream is fully consumed - Auto-create a session when context providers are configured but no session is explicitly passed Fixes #4754 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Apply pre-commit auto-fixes * Remove reproduction report from repository Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Address PR review feedback for #4754 - Validate messages when no continuation_token: raise ValueError if normalized_messages is empty, preventing IndexError on messages[-1] - Import BaseContextProvider/SessionContext from public agent_framework package instead of internal agent_framework._sessions module - Add test for ValueError on run(None) without continuation_token Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Improve test coverage for empty-messages guard in A2AAgent.run (#4754) - Parameterize test to cover both messages=None and messages=[] inputs - Add test verifying run(None, continuation_token=...) does not raise Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <copilot@github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>	2026-03-19 10:45:42 +00:00
Eduard van Valkenburg GitHub Copilot Copilot	bf8d9672e1	Python: Aggregate token usage across tool-call loop iterations in invoke_agent span (#4739 ) * Fix invoke_agent span to aggregate token usage across LLM calls (#4062) The FunctionInvocationLayer._get_response() loop was overwriting the response on each iteration, so usage_details only reflected the last chat completion call. Now tracks aggregated_usage across all iterations using add_usage_details() and sets it on the returned response. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Apply pre-commit auto-fixes * Remove reproduction report artifact Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Apply pre-commit auto-fixes * Apply pre-commit auto-fixes --------- Co-authored-by: Copilot <copilot@github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>	2026-03-19 06:41:33 +00:00
Peter IbekweandGitHub	5374dd47c5	.NET: Fix source generator bug that silently drops base class handler registrations for protocol-only partial executors (#4751 ) * Fix source generator bug that silently drops base class handler registrations for protocol-only partial executors * Fixed xml comments and variable naming.	2026-03-19 02:19:42 +00:00
Evan Mattson GitHub Copilot Copilot	29dfcbb584	.NET: Validate SkillsInstructionPrompt contains {0} placeholder in FileAgentSkillsProvider (#4642 ) * Fix FileAgentSkillsProvider accepting SkillsInstructionPrompt without {0} placeholder (#4638) BuildSkillsInstructionPrompt validated only format-string syntax via string.Format(template, ""), which silently accepted templates without a {0} placeholder. The generated skills list was then dropped from the final instructions. Tighten validation to format with a sentinel string and verify it appears in the output, rejecting templates that do not reference argument 0 with an ArgumentException. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix netstandard2.0 compat and simplify prompt template validation (#4638) - Replace string.Contains(string, StringComparison) with IndexOf for netstandard2.0/net472 compatibility - Remove sentinel round-trip check; validate {0} directly on the raw template string using IndexOf - Add positive test verifying custom SkillsInstructionPrompt with {0} is accepted and applied to output Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <copilot@github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>	2026-03-19 00:18:39 +00:00
Copilot GitHub crickman Copilot Autofix powered by AIcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>Chris Rickman	c9321b9028	.NET Compaction - Allow developer to specify a custom formatter for ToolResultCompactionStrategy (#4667 ) * Initial plan * Allow developer to specify custom formatter for ToolResultCompactionStrategy Co-authored-by: crickman <66376200+crickman@users.noreply.github.com> * Refine shape * Fix test expectation * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: crickman <66376200+crickman@users.noreply.github.com> Co-authored-by: Chris Rickman <crickman@microsoft.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>	2026-03-18 19:24:43 +00:00
Eduard van Valkenburg GitHub Copilot	f48c4512d3	Python: Simplify Python Poe tasks and unify package selectors (#4722 ) * updated automation tasks and commands, with alias for the time being * Restore aggregate test exclusions Preserve the legacy all-tests scope for test --all by excluding lab and devui from the default aggregate sweep, while still allowing explicit package selection. Also ignore hidden/generated test directories such as .mypy_cache during aggregate discovery. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * updated versions in pre-commit --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>	2026-03-18 18:39:11 +00:00
Copilot GitHub crickman Copilot Autofix powered by AIcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>Chris Rickman	d3d0100822	.NET Compaction - Add `AsChatReducer()` extension to expose `CompactionStrategy` as `IChatReducer` (#4664 ) * Initial plan * Add ChatStrategyExtensions.cs with AsChatReducer() extension method and tests Co-authored-by: crickman <66376200+crickman@users.noreply.github.com> * Refactor message list creation in ReduceAsync method * Remove unnecessary blank line in AsChatReducer method * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Fix test --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: crickman <66376200+crickman@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Co-authored-by: Chris Rickman <crickman@microsoft.com>	2026-03-18 17:25:54 +00:00
Eduard van Valkenburg GitHub Copilot Copilot	acaf6b7054	Python: Fix ENABLE_SENSITIVE_DATA env var ignored when set after module import (#4743 ) * Python: Re-read env vars in configure_otel_providers and enable_instrumentation (#4119) Fix ENABLE_SENSITIVE_DATA and VS_CODE_EXTENSION_PORT env vars being ignored when load_dotenv() runs after module import. The module-level OBSERVABILITY_SETTINGS singleton cached env state at import time, and configure_otel_providers() / enable_instrumentation() never re-read from os.environ when parameters were None. Both functions now construct a fresh ObservabilitySettings() to pick up current env vars when explicit parameters are not provided, matching the existing behavior of the env_file_path branch. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Address PR review feedback for #4119: avoid throwaway ObservabilitySettings - Add _read_bool_env/_read_int_env helpers to read env vars without constructing a full ObservabilitySettings (which calls create_resource()) - Replace ObservabilitySettings() in enable_instrumentation() and configure_otel_providers() else-branch with direct env reads - Add enable_console_exporters parameter to configure_otel_providers() for override parity with enable_sensitive_data and vs_code_extension_port - Propagate _resource and _executed_setup in the non-env_file_path branch - Make existing tests hermetic (clear VS_CODE_EXTENSION_PORT and ENABLE_CONSOLE_EXPORTERS env vars) - Add tests: enable_console_exporters env refresh, explicit param overrides for both enable_instrumentation() and configure_otel_providers() Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Address remaining review feedback for #4119 - Refresh enable_console_exporters in enable_instrumentation() for consistency with configure_otel_providers(), so env var changes after import are picked up by both public API functions - Make test_configure_otel_providers_reads_env_vs_code_port hermetic by clearing ENABLE_CONSOLE_EXPORTERS from the environment - Add test_enable_instrumentation_reads_env_console_exporters to cover the new refresh behavior Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Remove unconditional enable_console_exporters overwrite from enable_instrumentation() (#4119) enable_instrumentation() is documented as not configuring exporters, so managing enable_console_exporters there was a leaky abstraction. The unconditional _read_bool_env call silently reset the value to False when ENABLE_CONSOLE_EXPORTERS was absent from env, clobbering any value previously set by configure_otel_providers(enable_console_exporters=True). - Remove the unconditional overwrite line from enable_instrumentation() - Replace test_enable_instrumentation_reads_env_console_exporters with test_enable_instrumentation_does_not_touch_console_exporters - Add regression test: enable_instrumentation() does not clobber a previously configured enable_console_exporters value - Add test: explicit enable_sensitive_data param still leaves enable_console_exporters untouched Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <copilot@github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>	2026-03-18 15:58:22 +00:00
Hui MiaoandGitHub	c2fec6b51c	Python: Add foundry hosted agents samples for python (#4648 ) * Add two hosted agent samples using the foundry agent * Refactor formatting and improve readability in main.py * Add agent-framework dependency to requirements and update copyright notice in main.py files * Refactor agent imports and update credential handling in hosted agent samples * Update agent framework dependency in requirements for hosted agents * chore: update Python version to 3.14 and improve Dockerfile for hosted agents * feat: add hosted agent samples for Azure AI with local tools and multi-agent workflows * fix: update Azure AI client import and refactor agent initialization in hotel agent sample * feat: add hosted agent samples for Seattle hotel search and writer-reviewer workflow * fix: correct agent name in YAML configuration for local tools agent	2026-03-18 08:39:08 +00:00
Eduard van Valkenburg GitHub Copilot Copilot	705ed47a0b	Python: Fix missing methods on the `Content` class in durable tasks (#4738 ) * Fix Content serialization in DurableAgentStateUnknownContent (#4719) DurableAgentStateUnknownContent.from_unknown_content() stored raw Content objects without converting them to dicts, causing json.dumps to fail in Azure Durable Functions' entity state serialization. This affected content types not explicitly handled (e.g., mcp_server_tool_call/result). The fix converts Content objects to dicts via to_dict() when storing in DurableAgentStateUnknownContent, and restores them via Content.from_dict() in to_ai_content(). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add to_json and from_json methods to Content class (#4719) Add to_json() and from_json() methods to the Content class to match the serialization interface provided by SerializationMixin on other model classes. Also fix pre-existing pyright type errors in durabletask's DurableAgentStateUnknownContent.to_ai_content(). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Address PR review: add type guard, remove to_json, add fallback, and tests - Remove Content.to_json() per reviewer request (comment 3) - Add type guard in Content.from_json() for non-dict JSON (comments 1, 4) - Wrap json.JSONDecodeError as ValueError for consistent exception contract - Add try/except fallback in to_ai_content() for invalid Content dicts (comment 5) - Add test_content_to_dict_exclude_none and test_content_to_dict_exclude_fields (comment 2) - Add test_unknown_content_to_ai_content_fallback_on_invalid_type_dict (comment 5) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Apply pre-commit auto-fixes * Address review feedback for #4719: review comment fixes * Remove Content.from_json, move logic to consuming code (#4719) Remove the from_json convenience method from Content class per review feedback. This is the same trivial json.loads + from_dict wrapper as to_json which was already removed. Consumers should call json.loads and Content.from_dict directly. Update tests to use Content.from_dict(json.loads(...)) pattern and remove from_json-specific error handling tests (those errors are already covered by json.loads and Content.from_dict). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <copilot@github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>	2026-03-18 08:08:44 +00:00
Eduard van Valkenburg GitHub Copilot Copilot	192a283c9a	Python: Reduce Azure chat client import overhead (#4744 ) * Reduce Azure chat client import overhead Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix Azure chat client type annotations and add _parse_text_from_openai tests - Move Choice and ChunkChoice imports under TYPE_CHECKING to avoid runtime import cost (from __future__ annotations is already present) - Restore proper typed signature (Choice \| ChunkChoice) instead of Any - Add direct unit tests for _parse_text_from_openai covering: - Choice with message content - ChunkChoice with delta content - Refusal branch for both Choice and ChunkChoice - No content/no refusal returning None - None delta (async content filtering) returning None Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Copilot <copilot@github.com>	2026-03-18 08:05:42 +00:00
Peter IbekweandGitHub	c74b1b08eb	.NET: Fix race condition issue in FanInEdge while processing messages. (#4662 ) * Fix race condition issue in FanInEdge while processing messages. * refactored to limit the code segment under lock. * Remove extra materialization of the result. * Added comment to clarify future changes if process message is made async.	2026-03-18 00:36:10 +00:00